TY - GEN
T1 - Machine-crowd-expert model for increasing user engagement and annotation quality
AU - Méndez, Ana Elisa Méndez
AU - Cartwright, Mark
AU - Bello, Juan Pablo
N1 - Publisher Copyright:
© 2019 Copyright held by the owner/author(s).
PY - 2019/5/2
Y1 - 2019/5/2
N2 - Crowdsourcing and active learning have been combined in the past with the goal of reducing annotation costs. However, two issues arise with using AL and crowdsourcing: quality of the labels and user engagement. In this work, we propose a novel machine ⇔ crowd ⇔ expert loop model where the forward connections of the loop aim to improve the quality of the labels and the backward connections aim to increase user engagement. In addition, we propose a research agenda for evaluating the model.
AB - Crowdsourcing and active learning have been combined in the past with the goal of reducing annotation costs. However, two issues arise with using AL and crowdsourcing: quality of the labels and user engagement. In this work, we propose a novel machine ⇔ crowd ⇔ expert loop model where the forward connections of the loop aim to improve the quality of the labels and the backward connections aim to increase user engagement. In addition, we propose a research agenda for evaluating the model.
KW - Active learning
KW - Crowdsourcing
KW - Sound classification
UR - http://www.scopus.com/inward/record.url?scp=85067307264&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85067307264&partnerID=8YFLogxK
U2 - 10.1145/3290607.3313054
DO - 10.1145/3290607.3313054
M3 - Conference contribution
AN - SCOPUS:85067307264
T3 - Conference on Human Factors in Computing Systems - Proceedings
BT - CHI EA 2019 - Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems
PB - Association for Computing Machinery
T2 - 2019 CHI Conference on Human Factors in Computing Systems, CHI EA 2019
Y2 - 4 May 2019 through 9 May 2019
ER -