PUBLICATION
How Many Data Does Machine Learning in Human–Computer Interaction Need?: Re-Estimating the Dataset Size for Convolutional Neural Network-Based Models of Visual Perception
Type
Journal Article
Year
2023
Authors
Dr.-Ing. Sebastian Heil
Vladimir Khvorostov
Research Area
Event
IT Professional vol. 25, no. 2
Published in
IT Professional
Download
Abstract
Artificial intelligence (AI)-based user-interface (UI) design and evaluation are currently constrained by the scarcity of human-generated training data. Correspondingly, choosing appropriate neural network (NN) architecture and carefully planning the sample size is essential for building accurate machine learning models. Previously, we have estimated that for a convolutional NN to produce better mean square errors (MSEs) than feature-based models, the required training dataset size should be approximately 3000. Our current validation study with roughly 4000 web UIs and 233 subjects suggests that the estimation should be closer to 17,000. We propose corrected regression models suggesting that the dataset size effect is better described using a logarithmic function. We also report significant differences in MSEs between the employed perception dimensions, with Aesthetics models having an MSE 21.5% worse than Complexity and 12.1% worse than Orderliness.
Reference
Bakaev, Maxim; Heil, Sebastian; Khvorostov, Vladimir; Gaedke, Martin: How Many Data Does Machine Learning in Human–Computer Interaction Need?: Re-Estimating the Dataset Size for Convolutional Neural Network-Based Models of Visual Perception. IT Professional, pp. 23-29, 2023.
