Web Engineering

Chatbots are widely used in everyday digital services, but their interfaces present patterns such as Chat Messages, Quick Replies, or Persistent Menus in many different ways. For the analysis of chatbot interfaces this means that time and effort are needed to manually code and label these patterns in screenshots or videos. Object Detection methods can be used for automatically finding and categorizing these interface patterns. A tool which uses such a method could help researchers in their analyses, although the variety of the design implementations can pose a challenge for the implementation.

This thesis aims to create and train such a tool. After researching, comparing, and choosing a meaningful machine learning approach, it has to be trained. For this, available datasets have to be labelled according to previously published patterns, including all of their variations. The dataset also has to be enhanced and split for training and testing purposes. The goal is to reliably find a selection of chatbot interface patterns throughout different implementations. A User Interface should help users in providing datasets to the tool and receiving easily understandable results providing information on which patterns were found in which dataset samples.

The objective of this thesis is the creation of a solution or the combination of existing approaches to solve the above described problem of automatically detecting and categorizing chatbot interface patterns. This includes the analysis of the state of the art of object detection, specific tools and frameworks related to this topic, the chatbot interface, and other relevant literature. From this analysis, a possible solution should be conceptualized and demonstrated by creating a prototypical implementation of an automated pattern detection tool, as well as a suitable evaluation based on its compliance with requirements which were extracted through the literature research.