{"id":17849,"date":"2018-01-01T00:00:00","date_gmt":"2018-01-01T00:00:00","guid":{"rendered":"https:\/\/vsr.informatik.tu-chemnitz.de\/publications\/hci-vision-for-automated-analysis-and-mining-of-web-user-interfaces\/"},"modified":"2026-10-29T14:13:03","modified_gmt":"2026-10-29T14:13:03","slug":"hci-vision-for-automated-analysis-and-mining-of-web-user-interfaces","status":"publish","type":"publication","link":"https:\/\/vsr.informatik.tu-chemnitz.de\/research\/publications\/2018\/005\/","title":{"rendered":"HCI Vision for Automated Analysis and Mining of Web User Interfaces"},"content":{"rendered":"\n<div class=\"wp-block-vsr-blocks-header\">\n\t<h3 class=\"grey linie\">\n\t\tAbstract\t<\/h3>\n\t<\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Most techniques for webpage structure and design mining are based on code analysis and are detached from a human user\u2019s perception of the web user interface (WUI). Our paper is dedicated to approaches that instead focus on analysis of webpage\u2019s visual representation \u2013 the way it is rendered in different browsers and environments and delivered to the end user. Specifically, we describe the software tool that we built, which takes a WUI screenshot and produces structured and machine-readable representation (JSON) of interface elements as made out by a human user. The implementation is based on OpenCV (image recognition functions), dlib (trained detector for the elements\u2019 classification), and Tesseract (label and content text recognition). To demonstrate feasibility of the approach, we describe application of our analyzer tool to auto-calculate certain measures for a WUI and to predict users\u2019 subjective impressions. Particularly, we assess UI visual complexity, which is known to significantly influence both cognitive and affective aspects of interaction. The results suggest the analyzer\u2019s output is mostly characteristic of the users\u2019 visual perception and can be useful for auto-assessing and comparing WUIs.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Most techniques for webpage structure and design mining are based on code analysis and are detached from a human user\u2019s perception of the web user interface (WUI). Our paper is dedicated to approaches that instead focus on analysis of webpage\u2019s visual representation \u2013 the way it is rendered in different browsers and environments and delivered [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"template":"","meta":{"publication_year":2018,"publication_location":"Proceedings of 18th International Conference on Web Engineering (ICWE2018)","publication_pages":"","publication_city":"C\u00e1ceres","publication_country":"Spain","publication_event_name":"18th International Conference on Web Engineering","publication_event_url":"http:\/\/icwe2018.webengineering.org\/","publication_event_start":"2018-06-05","publication_event_end":"2018-06-08","publication_doi":"","publication_isbn_print":"978-3-319-91661-3","publication_isbn_online":"","publication_issn":"","publication_pdf_url":"https:\/\/vsr.informatik.tu-chemnitz.de\/people\/heseba\/publications\/2018-005.pdf","publication_biblatex":"@inproceedings{hcivisionforautomate2018,\n  author = {Bakaev, Maxim and Heil, Sebastian and Khvorostov, Vladimir and Gaedke, Martin},\n  title = {{HCI Vision for Automated Analysis and Mining of Web User Interfaces}},\n  year = {2018},\n  booktitle = {Proceedings of 18th International Conference on Web Engineering (ICWE2018)},\n  isbn = {978-3-319-91661-3},\n  eventtitle = {18th International Conference on Web Engineering},\n  eventdate = {2018-06-05\/2018-06-08},\n  location = {C\u00e1ceres, Spain},\n  url = {https:\/\/vsr.informatik.tu-chemnitz.de\/people\/heseba\/publications\/2018-005.pdf},\n  keywords = {Web Engineering, vsr},\n}\n","publication_csl":"{\n    \"id\": \"17849\",\n    \"type\": \"paper-conference\",\n    \"title\": \"HCI Vision for Automated Analysis and Mining of Web User Interfaces\",\n    \"author\": [\n        {\n            \"given\": \"Maxim\",\n            \"family\": \"Bakaev\"\n        },\n        {\n            \"given\": \"Sebastian\",\n            \"family\": \"Heil\"\n        },\n        {\n            \"given\": \"Vladimir\",\n            \"family\": \"Khvorostov\"\n        },\n        {\n            \"given\": \"Martin\",\n            \"family\": \"Gaedke\"\n        }\n    ],\n    \"issued\": {\n        \"date-parts\": [\n            [\n                2018\n            ]\n        ]\n    },\n    \"container-title\": \"Proceedings of 18th International Conference on Web Engineering (ICWE2018)\",\n    \"event-place\": \"C\u00e1ceres, Spain\",\n    \"event-title\": \"18th International Conference on Web Engineering\",\n    \"URL\": \"https:\/\/vsr.informatik.tu-chemnitz.de\/people\/heseba\/publications\/2018-005.pdf\"\n}","publication_id_legacy":"2018-005","publication_isbn_legacy":"978-3-319-91661-3","publication_bibtex_url_legacy":"","publication_doi_url_legacy":"","migration_allowed":true,"breadcrumb_parent":348,"footnotes":"","vsr_identifier_year":2018,"vsr_sequence_id":5,"vsr_primary_id":282},"publication_type":[564],"publication_research_area":[571],"publication_tag":[572],"publication_status":[569],"class_list":["post-17849","publication","type-publication","status-publish","hentry","publication_type-conference-paper","publication_research_area-web-engineering","publication_tag-vsr","publication_status-accepted"],"_links":{"self":[{"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/publication\/17849","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/publication"}],"about":[{"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/types\/publication"}],"author":[{"embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/users\/1"}],"version-history":[{"count":8,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/publication\/17849\/revisions"}],"predecessor-version":[{"id":31215,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=\/wp\/v2\/publication\/17849\/revisions\/31215"}],"wp:attachment":[{"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=17849"}],"wp:term":[{"taxonomy":"publication_type","embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fpublication_type&post=17849"},{"taxonomy":"publication_research_area","embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fpublication_research_area&post=17849"},{"taxonomy":"publication_tag","embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fpublication_tag&post=17849"},{"taxonomy":"publication_status","embeddable":true,"href":"https:\/\/vsr.informatik.tu-chemnitz.de\/index.php?rest_route=%2Fwp%2Fv2%2Fpublication_status&post=17849"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}