{"id":9777,"date":"2021-01-01T00:00:00","date_gmt":"2021-01-01T00:00:00","guid":{"rendered":"https:\/\/vsr.informatik.tu-chemnitz.de\/student-projects\/2021\/9777\/"},"modified":"2026-09-16T14:42:16","modified_gmt":"2026-09-16T14:42:16","slug":"ui-metrics-the-impact-of-ui-object-detection-quality","status":"publish","type":"student_project","link":"https:\/\/vsr.informatik.tu-chemnitz.de\/edu\/studentprojects\/2021\/009\/","title":{"rendered":"UI Metrics: The Impact of UI Object Detection Quality"},"content":{"rendered":"\n<div class=\"wp-block-vsr-blocks-header\">\n\t<h3 class=\"grey linie\">\n\t\tDescription\t<\/h3>\n\t<\/div>\n\n\n\n<p class=\"wp-block-paragraph\">User Interface (UI) metrics are an important building block for engineering good frontends. It enables a systematic approach of iteratively building, evaluating, and improving user interfaces to optimize them towards desired target characteristics in terms of aesthetics, complexity and usability. Automatic computation of these metrics, reducing the need for expensive and time-consuming empirical evaluations with human users, is the key enabler for a high results quality and efficiency of this process, as it allows for high evaluation frequency and therefore for smaller increments being created more often. A significant share of UI metrics relies on knowledge of the composition of the UI such as simple count measures (number of objects, number of different sizes, number of images, type distributions), alignment and spatial distributions, Ngo&#8217;s balance, symmetry, proportion, and density metrics, or Salimun&#8217;s cohesion, economy, sequence, unity and regularity. Automatic computation of object-dependent UI metrics therefore relies on object detection techniques.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Through recent advances in the field of Deep Neural Networks, state-of-the-art UI Object detectors reach an object prediction quality of about .6 F1 score. While a significant improvement over previous works, this number is still relatively low. This thesis investigates the influence of UI Object Detection quality on object-dependent UI metrics. It will implement a representative set of object-dependent metrics and analyze the impact of using a state-of-the-art detector in comparison to a perfect ground truth. As UI metrics are often used as input features for further machine learning (ML) models, the impact will be analyzed on two levels: directly on the UI metrics and indirectly on the performance of existing ML models for UI evaluation. Based on these findings, the thesis aims at providing simple heuristic corrections that can be used to improve the overall performance when using UI Object detectors as input for automatic UI evaluation. The experiments and evaluation of the proposed correction solutions will be performed on existing datasets of the professorship (NSTU500, 6Domains100) and external (Rico).<\/p>\n\n\n\n<div class=\"wp-block-vsr-blocks-header\">\n\t<h3 class=\"grey linie\">\n\t\tDescription (German)\t<\/h3>\n\t<\/div>\n\n\n\n<p class=\"wp-block-paragraph\">User Interface (UI) Metriken sind ein wichtiger Baustein f\u00fcr die ingenieursm\u00e4\u00dfige Konstruktion von guten Frontends. Sie erm\u00f6glichen einen systematischen Ansatz des iterativen Erstellens, Evaluierens und Verbesserns von Nutzerschnittstellen um diese hinsichtlich angestrebter Ziel-Charakteristika in \u00c4sthetik, Komplexit\u00e4t und Nutzbarkeit zu optimieren. Die automatische Berechnung dieser Metriken reduziert die Notwendigkeit f\u00fcr kosten- und zeit-intensive empirische Evaluationen mit menschlichen Nutzern und ist ein Schl\u00fcssel f\u00fcr hohe Qualit\u00e4t und Effizienz des Entwicklungs-Prozesses, da dies eine hohe Evaluationsfrequenz und mithin kleinere und h\u00e4ufigere Iterationen erm\u00f6glicht. Ein signifikanter Anteil der UI Metriken setzt Kenntnis der Zusammensetzung der UI voraus, wie zum beispiel einfache Z\u00e4hlungs-basierte Ma\u00dfe (Anzahl der Objekte, Anzahl der verschiedenen Gr\u00f6\u00dfen, Anzahl der Bilder, Typ-Verteilungen), Ausrichtung und R\u00e4umliche Verteilungen, Ngo&#8217;s Balance, Symmetrie, Proportion und Dichte Metriken, oder Salimun&#8217;s Koh\u00e4sion, \u00d6konomie, Sequenz, Einheitlichkeit und Regelm\u00e4\u00dfigkeit. Die automatische Berechnung von objekt-abh\u00e4ngigen UI Metriken ist daher auf Techniken der Objecterkennung angewiesen.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Durch aktuelle Fortschritte im Gebiet der Deep Neural Networks erreichen neuste UI Objekt Detektoren eine Vorhersagequalit\u00e4t f\u00fcr UI Objekte von ca. .6 F1 Score. Obwohl dies eine signifikante Verbesserung gegen\u00fcber fr\u00fcheren Verfahren darstellt ist dieser Wert immer noch relativ niedrig. Diese Arbeit untersucht den Einfluss der Vorhersagequalit\u00e4t f\u00fcr UI Objekte auf objekt-abh\u00e4ngige UI Metriken. Es werden repr\u00e4sentative objekt-abh\u00e4ngige UI Metriken implementiert und der Einfluss durch den Einsatz eines aktuellen UI Objekt Detektors im Vergleich zu einer perfekten ground truth analysiert. Da UI Metriken oft als Eingabe-Features f\u00fcr weitere Modelle des maschinellen Lernens (ML) verwendet werden, wird der Einfluss auf zwei Ebenen untersucht: direkt an den UI Metriken und indirekt an der Leistung von existierenden ML Modellen f\u00fcr die Evaluation von UIs. Basierend auf den Ergebnissen zielt diese Arbeit darauf, einfache heuristische Korrekturma\u00dfnahmen anzubieten, welche die Gesamtqualit\u00e4t beim Einsatz von UI Objekt Detektoren als Datenquelle f\u00fcr die automatische UI Evaluation steigern. Die Experimente und Evaluation der vorgeschlagenen Korrektur-L\u00f6sungen wird mit existierenden Datens\u00e4tzen der Professur (NSTU500, 6Domains100) sowie externen (Rico) durchgef\u00fchrt.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>User Interface (UI) metrics are an important building block for engineering good frontends. It enables a systematic approach of iteratively building, evaluating, and improving user interfaces to optimize them towards desired target characteristics in terms of aesthetics, complexity and usability. Automatic computation of these metrics, reducing the need for expensive and time-consuming empirical evaluations with [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":11377,"comment_status":"open","ping_status":"closed","template":"","meta":{"student_project_code_name":"","student_project_completion":"2021-12-31","student_project_deadline":"2021-12-31","student_project_id_legacy":"2021-009","student_project_archive_id_legacy":"","student_project_thesis_link_legacy":"","student_project_description_link_legacy":"","student_project_medium_legacy":"","student_project_pubdate_legacy":"2021-12-31","student_project_schemadata_legacy":"","student_project_second_version_title_legacy":"","migration_allowed":true,"breadcrumb_parent":413,"footnotes":"","vsr_identifier_year":2021,"vsr_sequence_id":9,"vsr_primary_id":558},"student_project_type":[],"student_project_status":[],"student_project_group":[],"student_project_research_area":[],"student_project_tag":[278,320,317,321,319],"class_list":["post-9777","student_project","type-student_project","status-publish","has-post-thumbnail","hentry","student_project_tag-deep-learning","student_project_tag-hci","student_project_tag-object-detection","student_project_tag-ui-uix","student_project_tag-visual-analysis"],"_links":{"self":[{"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/student_project\/9777","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/student_project"}],"about":[{"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/types\/student_project"}],"author":[{"embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=9777"}],"version-history":[{"count":20,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/student_project\/9777\/revisions"}],"predecessor-version":[{"id":29922,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/student_project\/9777\/revisions\/29922"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/media\/11377"}],"wp:attachment":[{"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=9777"}],"wp:term":[{"taxonomy":"student_project_type","embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fstudent_project_type&post=9777"},{"taxonomy":"student_project_status","embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fstudent_project_status&post=9777"},{"taxonomy":"student_project_group","embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fstudent_project_group&post=9777"},{"taxonomy":"student_project_research_area","embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fstudent_project_research_area&post=9777"},{"taxonomy":"student_project_tag","embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fstudent_project_tag&post=9777"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}