Trust-aware optimal crowdsourcing with budget constraint

Xiangyang Liu, He He, John S. Baras

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Crowdsourcing has been extensively used for aggregating data from a large pool of workers. In a real crowdsourcing market, each answer obtained from a worker incurs cost. The cost is associated with both the level of trustworthiness of workers and the difficulty of tasks. Typically, access to expert-level (more trustworthy) workers is more expensive than to average crowd and completion of a challenging task is more costly than a click-away question. In this paper, we address the problem of optimal assignment of heterogeneous tasks to workers of varying trust levels with budget constraint. Specifically, we design a trust-aware task allocation algorithm that takes as inputs the estimated trust of workers and pre-set budget, and outputs the optimal assignment of tasks to workers. We derive the bound of total error probability that relates to budget, trustworthiness of crowds, and costs of obtaining labels from crowds naturally. Higher budget, more trustworthy crowds, and less costly jobs result in lower theoretical bound. Our allocation scheme does not depend on the specific design of the trust evaluation component. Therefore, it can be combined with generic trust evaluation algorithms. Our algorithm outperforms state-of-the-art by up to 30% on real data.

Original languageEnglish (US)
Title of host publication2015 IEEE International Conference on Communications, ICC 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781467364324
StatePublished - Sep 9 2015
EventIEEE International Conference on Communications, ICC 2015 - London, United Kingdom
Duration: Jun 8 2015Jun 12 2015

Publication series

NameIEEE International Conference on Communications
ISSN (Print)1550-3607


ConferenceIEEE International Conference on Communications, ICC 2015
Country/TerritoryUnited Kingdom

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Trust-aware optimal crowdsourcing with budget constraint'. Together they form a unique fingerprint.

Cite this