A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk Best of CHI 2018
CHI 2018
April 21, 2018

A growing number of people are working as part of on-line crowd work, which has been characterized by its low wages; yet, we know little about wage distribution and causes of low/high earnings. We recorded 2,676 workers performing 3.8 million tasks on Amazon Mechanical Turk. Our task-level analysis revealed that workers earned a median hourly wage of only ~2/h,andonly4earnedmorethan7.25/h. The average requester pays more than $11/h, although lower-paying requesters post much more work. Our wage calculations are influenced by how unpaid work is included in our wage calculations, e.g., time spent searching for tasks, working on tasks that are rejected, and working on tasks that are ultimately not submitted. We further explore the characteristics of tasks and working patterns that yield higher hourly wages. Our analysis informs future platform design and worker tools to create a more positive future for crowd work.

The Unsupported Crowd: Exclusion of Indian Workers in Amazon Mechanical Turk Communities
Global Labour Research Centre International Graduate Student Symposium 2017
October 26, 2017

Amazon Mechanical Turk ("mTurk") has become popular among researchers in recent years, but few have considered how mTurk affects its most socioeconomically vulnerable users. New workers often find themselves lost on mTurk, which is a labour platform like no job they have experienced before. It is fiercely competitive but unregulated. Without help from offsite communities and tools, many prospective workers quit mTurk before their first task is approved. Indian “Turkers” are at an additional disadvantage because language differences and stereotypes prevent them from accessing community resources they need to succeed. This paper presents quantitative data about participation of Indian Turkers in offsite worker communities and interviews with one Indian Turker who has overcome these obstacles to become successful. The paper adds to the growing body of research on the lived experiences of crowd workers. Specifically, it contributes an exploration of the dynamics and consequences of exclusion in online worker communities.

What the “Crowd-Work” Economy Taught Me About Community
Zocalo Public Square
August 4, 2017

One second I would be frantically completing a batch, the next looking desperately for something to do. I might be designing a logo, then watching a video of a man immolated by ISIS, then translating business documents into French.
Shorter version

Slave to the keyboard: The broken promises of the gig economy
Transfer: European Review of Labour and Research

What the gig economy promises to full-time workers cut out of the traditional economy does not outweigh the consequences of being forced into this field of labour.

Crowd Guilds: Worker-led Reputation and Feedback on Crowdsourcing Platforms
CSCW 2017

Crowd workers are distributed and decentralized. While decentralization is designed to utilize independent judgment to promote high-quality results, it paradoxically undercuts behaviors and institutions that are critical to high-quality work. Reputation is one central example: crowdsourcing systems depend on reputation scores from decentralized workers and requesters, but these scores are notoriously inflated and uninformative. In this paper, we draw inspiration from historical worker guilds (e.g., in the silk trade) to design and implement crowd guilds: centralized groups of crowd workers who collectively certify each other's quality through double-blind peer assessment. A two-week field experiment compared crowd guilds to a traditional decentralized crowd work model. Crowd guilds produced reputation signals more strongly correlated with ground-truth worker quality than signals available on current crowd working platforms, and more accurate than in the traditional model.

A Mechanical Turk Worker’s Perspective
Journal of Media Ethics

How forcing Requesters into specific pay per piece rates will not increase worker income, but worker run platforms just might.

The Crowd: Naturally cooperative, unnaturally silenced
Ours to Hack and Own

Why platform cooperativism isn't just favourable to crowd workers, it's necessary for survival.

Book review by ZDNet
Chosen as one of Wired's Best Tech Books of 2017

Boomerang: Rebounding the Consequences of Reputation Feedback on Crowdsourcing Platforms
UIST 2016

Paid crowdsourcing platforms suffer from low-quality work and unfair rejections, but paradoxically, most workers and requesters have high reputation scores. These inflated scores, which make high-quality work and workers difficult to find, stem from social pressure to avoid giving negative feedback. We introduce Boomerang, a reputation system for crowdsourcing that elicits more accurate feedback by rebounding the consequences of feedback directly back onto the person who gave it. With Boomerang, requesters find that their highly-rated workers gain earliest access to their future tasks, and workers find tasks from their highly-rated requesters at the top of their task feed. Field experiments verify that Boomerang causes both workers and requesters to provide feedback that is more closely aligned with their private opinions. Inspired by a game-theoretic notion of incentive-compatibility, Boomerang opens opportunities for interaction design to incentivize honest reporting over strategic dishonesty.

Crowd Work: The Fury and the Fear (Chapter)
July 26, 2016

Abstract: In the ten years I've worked on Amazon Mechanical Turk, things have changed, but not nearly enough to protect us from being forced to engage in tedious, underpaid crowd work. We have to begin to investigate how these platforms have become what they are, and what we can do to create more ethical work spaces for ourselves to ensure we aren't exploited. The future of work depends on it.

Italian translation

Crowd Work: Shame, Secrets, and an Imminent Threat to Employment
Global Labour Column
June 16, 2016

Abstract: Crowd work: what it is, how it is impacting labour and economies, and how we can resist it.

Where doctors compete for patients and are paid in pennies: The future of work
Social Value and Intangibles Review
April, 2016

Imagine a future where doctors log into an online platform in order to find patients to treat, and their pay is per patient and so low they can hardly sustain their families. The way things are going, this future isn't too far off. Welcome to crowd work, a neoliberal paradise.

Daemo: a Self-Governed Crowdsourcing Marketplace (Poster)
UIST 2015
November 8, 2015

Abstract: Crowdsourcing marketplaces provide opportunities for autonomous and collaborative professional work as well as social engagement. However, in these marketplaces, workers feel disrespected due to unreasonable rejections and low payments, whereas requesters do not trust the results they receive. The lack of trust and uneven distribution of power among workers and requesters have raised serious concerns about sustainability of these marketplaces. To address the challenges of trust and power, this paper introduces Daemo, a self-governed crowdsourcing marketplace. We propose a prototype task to improve the work quality and open-governance model to achieve equitable representation. We envisage Daemo will enable workers to build sustainable careers and provide requesters with timely, quality labor for their businesses.

What you write doesn't represent me as an mTurk worker, so don't pretend it does
Turker Nation
February 20, 2015

mTurk is a great tapestry of people, each section totally different than the next. ... There is nothing you can say, no truth you can present, no set in stone rules about what all of these people want that can truly represent the needs and wants of all of them.

We Are Dynamo: Overcoming Stalling and Friction in Collective Action for Crowd Workers Best of CHI 2015
CHI 2015
February 10, 2015

Abstract: By lowering the costs of communication, the web promises to enable distributed collectives to act around shared issues. However, many collective action efforts never succeed: while the web’s affordances make it easy to gather, these same decentralizing characteristics impede any focus towards action. In this paper, we study challenges to collective action efforts through the lens of online labor by engaging with Amazon Mechanical Turk workers. Through a year of ethnographic fieldwork, we sought to understand online workers’ unique barriers to collective action. We then created Dynamo, a platform to support the Mechanical Turk community in forming publics around issues and then mobilizing. We found that collective action publics tread a precariously narrow path between the twin perils of stalling and friction, balancing with each step between losing momentum and flaring into acrimony. However, specially structured labor to maintain efforts' forward motion can help such publics take action.

Life as a mature student on campus
The Ryersonian
April 9, 2014

When returning to university as a mature student, things never go as planned. While you think you can seamlessly slip into the social life of young adults, the fact is that you will never feel as old as you do on campus.

The Myth of Low Cost, High Quality on Amazon's Mechanical Turk
Turker Nation
January 30, 2014

If you dig down into these studies and articles, you'll find the secret: sure, if you pay $1 the work will get done, but it will be completed by scammers using "bots" (automated answering systems), people who don't necessarily understand the instructions (as English is not their strongest language), or those who don't care about the quality of the work they complete. That's fine for simple work that doesn't require mastery of the English language, but every HIT is posted with the requirement that the answers provided actually offer what is being asked.