PUBLICATION
Crowdsourced Reverse Engineering: Experiences in Applying Crowdsourcing to Concept Assignment
Type
Book Section
Year
2019
Authors
Dr.-Ing. Sebastian Heil
Research Area
Event
Evaluation of Novel Approaches to Software Engineering 2018
Published in
Evaluation of Novel Approaches to Software Engineering, Revised Selected Papers, Communications in Computer and Information Science
ISBN/ISSN
978-3-030-22558-2
Download
Abstract
This article details the idea of Crowdsourced Reverse Engineering (CSRE) by analyzing three major challenges: 1) automatic task extraction, 2) source code anonymization and 3) results aggregation and quality control. We re-formulate the Reverse Engineering activity of concept assignment as a crowdsourced classification task to exemplify these challenges and describe suitable methods to address them. Our overview on existing research of crowdsourcing showcases examples of successful application in the field of Software Engineering and argues that Reverse Engineering activities like Concept Assignment are likely to also benefit from crowdsourcing by determining a high similarity in eight crowdsourcing dimensions to the microtasking model. Our experiments on the crowdsourcing platform microworkers.com support this, producing 187 results by 34 crowd workers which classified 10 code fragments with decent quality. We provide an extended analysis of the observed crowd workers’ behavior and report evidence of surprisingly high levels of engagement and efforts undertaken by the crowd. Concluding our experiences, this article indicates three open research challenges for future work.
Reference
Heil, Sebastian; Siegert, Valentin; Gaedke, Martin: Crowdsourced Reverse Engineering: Experiences in Applying Crowdsourcing to Concept Assignment. Evaluation of Novel Approaches to Software Engineering, Revised Selected Papers, Communications in Computer and Information Science, pp. 215-239, 2019.
