Masterarbeit
Comparing the Impact of UX Design Elements and LLMS on User-Chatbot Interactions
Completion
2025/08
Research Area
Students
Braian Alexis Salas Arriaga
Advisers
Verena Traubinger M.Sc.
Dr.-Ing. Sebastian Heil
Description
The development of large language models (LLMs) has increased the interest in conversational user interfaces (CUIs) like chatbots. Their capabilities have significantly expanded, leading to a step further into the adoption cycle in the software industry. Chatbots combine a CUI with a graphical interface and as such both interface types influence the user-chatbot interactions. While both graphical interfaces and the dialogue design of CUIs are known for influencing the user experience, they were not yet regarded both in combination for chatbots. To address this, a web-based chatbot will be developed to test their effects on users.
A chatbot for a meaningful use case will be implemented with several combinations of LLMs leveraging a RAG approach, and different interfaces, thus generating a series of model-interface combinations for an evaluation. Each chatbot user interface will contain specific graphical elements based on different designs: A menu-based approach for interactions, focusing on textual interactions, including an interactive avatar, and a combination of these elements. To understand the impact of the conversational and graphical components on the user, the LLM-interface combinations will be evaluated with user tests. The evaluation will be based on the same use case and compare aspects such as user satisfaction, efficiency, stimulation and ease of use with meaningful methods.
The objective of this thesis is the creation of a solution or the combination of existing approaches to solve the problem described above of comparing the effects of graphical UX design elements and LLMs on User-Chatbot Interactions. This comprises the following parts. An analysis of the state of the art on LLMs, current chatbot UX design, conversational design in CUIs, and other relevant work. The prototypical implementation should include combinations of different LLMs and UX elements. A suitable evaluation should be conducted, where these combinations are tested with qualitative or quantitative methods regarding their effects on the users and their compliance with elicited requirements based on the literature research.
