ThePlantGame: Actively training human annotators for domain-specific crowdsourcing

Maximilien Servajean; Alexis Joly; Dennis Shasha; Julien Champ; Esther Pacitti

doi:10.1145/2964284.2973820

ThePlantGame: Actively training human annotators for domain-specific crowdsourcing

Maximilien Servajean, Alexis Joly, Dennis Shasha, Julien Champ, Esther Pacitti

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In a typical citizen science/crowdsourcing environment, the contributors label items. When there are few labels, it is straightforward to train contributors and judge the quality of their labels by giving a few examples with known answers. Neither is true when there are thousands of domain-specific labels and annotators with heterogeneous skills. This demo paper presents an Active User Training framework implemented as a serious game called ThePlantGame. It is based on a set of data-driven algorithms allowing to (i) actively train annotators, and (ii) evaluate the quality of contributors' answers on new test items to optimize predictions.

Original language	English (US)
Title of host publication	MM 2016 - Proceedings of the 2016 ACM Multimedia Conference
Publisher	Association for Computing Machinery, Inc
Pages	720-721
Number of pages	2
ISBN (Electronic)	9781450336031
DOIs	https://doi.org/10.1145/2964284.2973820
State	Published - Oct 1 2016
Event	24th ACM Multimedia Conference, MM 2016 - Amsterdam, United Kingdom Duration: Oct 15 2016 → Oct 19 2016

Publication series

Name	MM 2016 - Proceedings of the 2016 ACM Multimedia Conference

Other

Other	24th ACM Multimedia Conference, MM 2016
Country/Territory	United Kingdom
City	Amsterdam
Period	10/15/16 → 10/19/16

ASJC Scopus subject areas

Computer Graphics and Computer-Aided Design
Human-Computer Interaction
Computer Vision and Pattern Recognition
Software

Access to Document

10.1145/2964284.2973820

Cite this

Servajean, M., Joly, A., Shasha, D., Champ, J., & Pacitti, E. (2016). ThePlantGame: Actively training human annotators for domain-specific crowdsourcing. In MM 2016 - Proceedings of the 2016 ACM Multimedia Conference (pp. 720-721). (MM 2016 - Proceedings of the 2016 ACM Multimedia Conference). Association for Computing Machinery, Inc. https://doi.org/10.1145/2964284.2973820

ThePlantGame: Actively training human annotators for domain-specific crowdsourcing. / Servajean, Maximilien; Joly, Alexis; Shasha, Dennis et al.
MM 2016 - Proceedings of the 2016 ACM Multimedia Conference. Association for Computing Machinery, Inc, 2016. p. 720-721 (MM 2016 - Proceedings of the 2016 ACM Multimedia Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Servajean, M, Joly, A, Shasha, D, Champ, J & Pacitti, E 2016, ThePlantGame: Actively training human annotators for domain-specific crowdsourcing. in MM 2016 - Proceedings of the 2016 ACM Multimedia Conference. MM 2016 - Proceedings of the 2016 ACM Multimedia Conference, Association for Computing Machinery, Inc, pp. 720-721, 24th ACM Multimedia Conference, MM 2016, Amsterdam, United Kingdom, 10/15/16. https://doi.org/10.1145/2964284.2973820

@inproceedings{59d308d722214fcb9c043f0a9e08402e,

title = "ThePlantGame: Actively training human annotators for domain-specific crowdsourcing",

abstract = "In a typical citizen science/crowdsourcing environment, the contributors label items. When there are few labels, it is straightforward to train contributors and judge the quality of their labels by giving a few examples with known answers. Neither is true when there are thousands of domain-specific labels and annotators with heterogeneous skills. This demo paper presents an Active User Training framework implemented as a serious game called ThePlantGame. It is based on a set of data-driven algorithms allowing to (i) actively train annotators, and (ii) evaluate the quality of contributors' answers on new test items to optimize predictions.",

author = "Maximilien Servajean and Alexis Joly and Dennis Shasha and Julien Champ and Esther Pacitti",

note = "Publisher Copyright: {\textcopyright} 2016 Copyright held by the owner/author(s).; 24th ACM Multimedia Conference, MM 2016 ; Conference date: 15-10-2016 Through 19-10-2016",

year = "2016",

month = oct,

day = "1",

doi = "10.1145/2964284.2973820",

language = "English (US)",

series = "MM 2016 - Proceedings of the 2016 ACM Multimedia Conference",

publisher = "Association for Computing Machinery, Inc",

pages = "720--721",

booktitle = "MM 2016 - Proceedings of the 2016 ACM Multimedia Conference",

}

TY - GEN

T1 - ThePlantGame

T2 - 24th ACM Multimedia Conference, MM 2016

AU - Servajean, Maximilien

AU - Joly, Alexis

AU - Shasha, Dennis

AU - Champ, Julien

AU - Pacitti, Esther

PY - 2016/10/1

Y1 - 2016/10/1

N2 - In a typical citizen science/crowdsourcing environment, the contributors label items. When there are few labels, it is straightforward to train contributors and judge the quality of their labels by giving a few examples with known answers. Neither is true when there are thousands of domain-specific labels and annotators with heterogeneous skills. This demo paper presents an Active User Training framework implemented as a serious game called ThePlantGame. It is based on a set of data-driven algorithms allowing to (i) actively train annotators, and (ii) evaluate the quality of contributors' answers on new test items to optimize predictions.

AB - In a typical citizen science/crowdsourcing environment, the contributors label items. When there are few labels, it is straightforward to train contributors and judge the quality of their labels by giving a few examples with known answers. Neither is true when there are thousands of domain-specific labels and annotators with heterogeneous skills. This demo paper presents an Active User Training framework implemented as a serious game called ThePlantGame. It is based on a set of data-driven algorithms allowing to (i) actively train annotators, and (ii) evaluate the quality of contributors' answers on new test items to optimize predictions.

UR - http://www.scopus.com/inward/record.url?scp=84994608331&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84994608331&partnerID=8YFLogxK

U2 - 10.1145/2964284.2973820

DO - 10.1145/2964284.2973820

M3 - Conference contribution

AN - SCOPUS:84994608331

T3 - MM 2016 - Proceedings of the 2016 ACM Multimedia Conference

SP - 720

EP - 721

BT - MM 2016 - Proceedings of the 2016 ACM Multimedia Conference

PB - Association for Computing Machinery, Inc

Y2 - 15 October 2016 through 19 October 2016

ER -

ThePlantGame: Actively training human annotators for domain-specific crowdsourcing

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this