The dynamics of micro-Task crowdsourcing: The case of amazon MTurk

Djellel Eddine Difallah, Michele Catasta, Gianluca Demartini, Panagiotis G. Ipeirotis, Philippe Cudré-Mauroux

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Micro-Task crowdsourcing is rapidly gaining popularity among research communities and businesses as a means to leverage Human Computation in their daily operations. Unlike any other service, a crowdsourcing platform is in fact a marketplace subject to human factors that affect its performance, both in terms of speed and quality. Indeed, such factors shape the dynamics of the crowdsourcing market. For example, a known behavior of such markets is that increasing the reward of a set of tasks would lead to faster results. However, it is still unclear how different dimensions interact with each other: reward, task type, market competition, requester reputation, etc. In this paper, we adopt a data-driven approach to (A) perform a long-Term analysis of a popular micro-Task crowd-sourcing platform and understand the evolution of its main actors (workers, requesters, tasks, and platform). (B) We leverage the main findings of our five year log analysis to propose features used in a predictive model aiming at deter-mining the expected performance of any batch at a specific point in time. We show that the number of tasks left in a batch and how recent the batch is are two key features of the prediction. (C) Finally, we conduct an analysis of the demand (new tasks posted by the requesters) and sup-ply (number of tasks completed by the workforce) and show how they affect task prices on the marketplace.

Original languageEnglish (US)
Title of host publicationWWW 2015 - Proceedings of the 24th International Conference on World Wide Web
PublisherAssociation for Computing Machinery, Inc
Pages238-247
Number of pages10
ISBN (Electronic)9781450334693
DOIs
StatePublished - May 18 2015
Event24th International Conference on World Wide Web, WWW 2015 - Florence, Italy
Duration: May 18 2015May 22 2015

Publication series

NameWWW 2015 - Proceedings of the 24th International Conference on World Wide Web

Other

Other24th International Conference on World Wide Web, WWW 2015
CountryItaly
CityFlorence
Period5/18/155/22/15

Keywords

  • Crowdsourcing
  • Forecasting
  • Tracking
  • Trend identification

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software

Fingerprint Dive into the research topics of 'The dynamics of micro-Task crowdsourcing: The case of amazon MTurk'. Together they form a unique fingerprint.

Cite this