Microtask Design: Value, Engagement, Context, and Complexity

Abstract

Crowdsourcing and microtasks are a relatively new way to issue units of work to a large group of potential workers. This form of outsourcing to a vast on-demand workforce offers the potential to significantly change the way we work. But how can design impact how both the requester and the workforce interact and benefit from these tasks? This dissertation considers four aspects of microtask design: value, engagement, context, and complexity.

Through four distinct, but highly related, investigations these four facets are ex- plored, analysed and synthesised into a considered review of microtask design. First we build a picture of the demographic and financial status of these crowdworkers by surveying the US-based crowdworker labour-force on the Amazon Mechanical Turk platform. This improved understanding of the value of crowd work, not just to requesters but to workers as well, is crucial to appropriately listing tasks in a commoditised labour market. Second, worker engagement is also a significant factor, not just in quality and cost, but also in uptake and effective completion speed. By introducing a new metric, conversion rate, and contrasting a variety of differing presentational and conceptual features across two demographics, we demonstrate an improved understanding of how tasks engage workers.

The increasing use of mobile devices, including among crowdworkers, offers new opportunities to collect additional context about worker behaviour. Enhancing the data gathered by requesters can be used, not only to improve quality, but also to expand the types of tasks which can be effectively crowdsourced. This third contribution highlights enthusiasm by some workers for mobile tasks, and demon- strates how previously small-scale sensor-based data collection can increasingly be carried out by the crowd. Finally, the boundary between microtasks and macrotasks is investigated. Exploring how complex tasks, such as software development, can be successfully crowdsourced offers insight into how task design can influence suitability of these larger tasks on microtask markets.