Academic Mentions & Citations:

Personalized and Diverse Task Composition in Crowdsourcing
Alsayasneh, M., Amer-Yahia, S., Gaussier, E., Leroy, V., Pilourdault, J., Borromeo, R.M., Toyama, M., Renders, J.M.

"Existing literature [45] and a thorough examination of crowdsourcing forums such as TurkerNation, reveal that workers spend non-negligible time discussing how to best select tasks depending on one’s goals, which requesters to ban, and which qualification levels are required for the latest tasks on AMT."

Cultural Heritage Infrastructures in Digital Humanities
Blanke, T., Kristel, C., Romary, L.

"A hall of shame of worst jobs offered to Turkers can be found on"

Chasing Pennies – My time as an Amazon Mechanical Turk Worker
May 1, 2017

"The difference between the two articles is stark. The Tech Republic article tells the story with ore impartiality than either The Kernal or Vocativ stories do. It also does something the others don’t: it reaches out to the foremost authority on being a Turker today: Kristy Milland."

Introduction: The Policy Implications of Virtual Work
Meil, P., Kirov, V.

There have been pockets of organizing and collective action to improve bargaining positions on platforms (Milland 2016... but there are real risks to attepts at organizing or speaking out for better working conditions since it is easy to monitor workers on platforms and also because reputation and ratings are the source of obtaining jobs.

In search of an adequate European policy response to the platform economy
Fabo, B., Karanovic, J., Dukova, K.

Finally, the workers must be given a real opportunity collectively to defend their interests. As a result of the lack of traditional trade union representation, platform workers are spontaneously establishing Internet tools such as, creating grass-roots, democratic, worker-driven forums in which platform workers meet virtually, exchange information and network across national borders...

Fairness and Transparency in Crowdsourcing
Borromeo, R.M., Laurent, T., Toyama, M., Amer-Yahia, S.

Transparency for workers comes from worker initiatives and communities mainly through forums such as Turker Nation... where workers share information about tasks, requesters and tools to enhance their experience. These tools are often worker-made scripts that disclose information hidden by the platform such as the time until automatic approval of a submission on AMT.

Understanding Workers, Developing Effective Tasks, and Enhancing Marketplace Dynamics: A Study of a Large Crowdsourcing Marketplace
A. Jain, A. Das Sarma, A. Parameswaran, J. Widom

Others attempt to understand worker motivations and behavior using (Turker Nation) forum data, and workers’ on- and off-network interactions.

Motivation-Aware Task Assignment in Crowdsourcing
J. Pilourdault. S. Amer-Yahia, D. Lee, S.B. Roy

A recent ethnomethodological study on Turker Nation [22] argued that in order to maintain the attractiveness of crowdsourcing platforms, it is critical to enable worker-centric optimization.

Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets
J.C. Chang, S. Amershi, E. Kamar

To prevent learning effects, we do not reuse crowdworkers across conditions for the same dataset, and randomize posting order of condition and dataset combinations so that crowdworkers subscribed to postings from our requester account using third party services ( were distributed across conditions

Understanding Workers, Developing Effective Tasks, and Enhancing Marketplace Dynamics: A Study of a Large Crowdsourcing Marketplace
Jain, A., Das Sarma, A., Parameswaran, A., Widom, J
Jan. 27, 2017

Others attempt to understand worker motivations and behavior using (Turker Nation) forum data [35], and workers’ on- and off-network interactions [21].

Subcontracting Microwork
Morris, M.R., Bigham, J.P., Brewer, R., Bragg, J., Kulkarni, A., Li, J., Savage, S.
May 6, 2017

The worker community strongly advocates for formal reputation models for requesters, and as many platforms do not yet support requester rating, this is often done unofficially on third-party sites like Turker Nation...

Uberworked and Underpaid: How Workers Are Disrupting the Digital Economy
Sholz, T.
Dec. 27, 2016

Milland reports that she made "double the poverty line" by turking full-time, which was enough to support her family and pay medical bills for two years after her husband lost his job.

Crowdsumers Take Over: Towards Valid Crowdsourcing of Consumer Research
Goodman, J.K., Paolacci, G.

To get a perspective on MTurk workers, we suggest requesters monitor the forums used by workers (e.g., ... Turkernation, ...), which can contain information about requesters’ reputations and ultimately inform researchers about the paradigms that might be less promising to conduct on MTurk.

Making Better Use of the Crowd
Vaughan, J.W.
December 5, 2016

If you’re interested in getting a workers’ perspective, long-time crowdworker and crowd advocate Kristy Milland has several related articles and some useful slides from her guest lecture at Penn.

Task Composition in Crowdsourcing
Amer-Yahia, S., Gaussier, E., Leroy, V., Pilourdault, J., Borromeo, R., Toyama, M.

Indeed, a thorough examination of crowdsourcing forums such as TurkerNation, reveals that workers spend non-negligible amounts of time discussing how to best select tasks depending on one’s goals, which requesters to ban, and which skills are required for the latest tasks on AMT.

Exploring Microtask Crowdsourcing as a Means of Fault Localization
Adriano, C.M., van der Hoek, A.

In order to mitigate distortions in the profiles of workers and the speed of recruitment, we did not post the study anywhere else but on MTurk. Hence, workers had to be logged into MTurk to discover our HITs. However, MTurk is known by popular and very active forums (e.g., Turker Nation [114], Turkopticon [115]), on which workers quickly share recommendations about available HITs.

Effort, Wage Premia, and Reservation Wages: A Field Test of the Shirking and Sorting Efficiency Wage Models
Hassairi, N.
Nov. 16, 2016

Workers communicate on 3rd-party web forums, share tips, and discuss jobs and employers (see, for example,

Big Data’s Little Brother: Enhancing Big Data in the Social Sciences with Micro-Task Marketplaces
Porter, N.D., Verdery, A.M., Gaddis, S.M.

We relied primarily on TurkerNation (, one of the largest and most active online communities that includes both workers and requesters.

Understanding the economy of the crowd
Soffer, P.
September 4, 2016

Increasingly, crowdworkers are organizing online to highlight unfair practices and direct their peers toward crowdwork platforms that treat them better. Because word travels fast, greater transparency in incentives and fairness in work practices also tend to attract motivated workers. Germany’s largest union, IG Metall, runs a site where crowdworkers rate platforms on their pay and fairness, and crowdworker forums are growing.

Running user studies with crowd workers
McInnis, B., Leshed, G.
September 2016

Based on advice from Silberman, we also decided to consult directly with veteran Turkers through the TurkerNation Internet-relay chat...

Crowdsourced Data Preprocessing with R and Amazon Mechanical Turk
Leeper, T.J.
Aug. 2016

Workers also communicate about the quality of HITs and requesters on fora such as ... Turker Nation (

A Comparison of Three Online Recruitment Strategies for Engaging Parents
Dworkin, J., Hessel, H., Gliske, K., Rudi, J.H.
August 26, 2016

MTurk workers use discussion boards such as ... to communicate about topics such as how reliably requesters pay and how well they estimate the time needed to complete their work...

Understanding crowdworkers’ learning practices
Margaryan, A.
August 10, 2016

Also we know that crowdworkers sometimes independently self-organise into physical communities centred around universities or internet cafes or into online communities using social media and internet-based tools (e.g. knowledge sharing tools such as Turkopticon, Dynamo, Turker Nation forum and various LinkedIn and Facebook groups for AMT workers) to help each other learn the system, share experience with specific tasks, identify reliable and unreliable employers or to exchange concerns about unfair payment practices or other issues they encounter in the course of their work on the crowdwork platforms (Kuek et al, 2015; Irani and Silberman, 2013; Salehi et al, 2015). However, whether and how crowdworkers may be using these or other fora and tools to self-organise specifically for the purpose of learning or professional development is not understood.

Understanding Discourse on Work and Job-Related Well-Being in Public Social Media
Liu, T., Homan, C.M., Alm, C.O., White, A.M., Lytle, M.C., Kautz, H.A.
August 7, 2016

We consulted with Turker Nation ( to ensure that the workers were treated and compensated fairly for their tasks.

Returning to Presence: The Effects of Mindfulness on Emotion Regulation Following Worry among Individuals with Analogue Generalized Anxiety Disorder
Goodnight, J.R.M.
August 9, 2016

Additionally, participants on MTurk may participate in communities online where they discuss their experiences with studies on the website (e.g., ...

Are all ‘research fields’ equal? Rethinking practice for the use of data from crowd-sourcing market places
Gleibs, I.H.
July 2016

To illustrate, there are many webpages and online discussion groups (e.g., ... Turker Nation) that discuss and explain how to make an income from MTurk.

Amazon Mechanical Turk in Organizational Psychology: An Evaluation and Practical Recommendations
Cheung, J.H., Burns, D.K., Sinclair, R.R., Sliter, M.
June 22, 2016

As Schmidt (2015) stated, there is a vibrant online community of MTurk Workers (a unique MTurk feature that is uncommon to other sample sources). Multiple websites have been developed for Workers to rate Requesters, comment on them and their HITs, and communicate with Requesters and other Workers about individual HITs. Some examples of MTurk online communities include ... Turker Nation ( A versatile crowdsourcing data acquisition platform for the behavioral sciences
Litman, L., Robinson, J., Abberbock, T.
April 12, 2016

A worker’s approval rating is updated every time the worker finishes a study and is either rejected or approved by a requester. The reciprocal relationship, whereby workers would rate requesters, is not provided by MTurk. However, various websites on which workers leave comments about specific HITs and requesters, such as Turkopticon (https://​turkopticon.​ucsd.​edu/​) and TurkerNation (http://​turkernation.​com/​), have been created, in part to provide workers with information about the quality of requesters and HITs.

Using Crowdsourced Online Experiments to Study Context-dependency of Behavior
Keuschnigg, M., Bader, F., Bracher, J.
April 12, 2016

Moreover, worker forums such as Turkopticon and Turker Nation, which rate requesters and HITs, may violate treatment integrity through cross-talk (Chandler et al., 2014). Projects discussed on forums, on the other hand, benefit from increased attention (Mason and Suri, 2011).

Consumer envy during service encounters
Gerardo, J. A., Mao, L., Matilla, A. S., Almanza, B.

The MTurk task was also advertised through two popular online MTurk forums: ...

Operating an employer reputation system: lessons from Turkopticon, 2008–2015
Silberman, M. S., Irani, L.

There are several worker forums, including, in no particular order, Turker Nation, MTurk Crowd, MTurk Grind, the MTurk and HITsWorthTurkingFor "subreddits" (i.e., sections on the content aggregator, and MTurk Forum.53 Each forum has different policies, politics, histories, and personalities—and differences between them sometimes erupt into interforum "drama." But many experienced Turkers acknowledge the centrality of forums to earning money on AMT, especially for those "Turking for a living." Discovering a forum appears to be a crucial turning point in the "careers" of financially successful Turkers; it is typically only by connecting to a community of more experienced workers that one can navigate AMT well enough to earn significant income.

Stories We Tell About Labor: Turkopticon and the Trouble with “Design”
L. C. Irani, M. S. Silberman
May 7, 2016

Kristy Milland, the operator of TurkerNation, also pointed out to us that these articles sometimes had perverse consequences. Journalists intended to draw attention to problematic work conditions and inspire political will to improve them. Yet Milland observed that these articles often publicized AMT as a place where people could get data processing work done cheaply.

The Communication Network Within the Crowd
M. Yin, M. L. Gray, S. Suri, J. W. Vaughan
Apr. 11, 2016

A quote from Zyskowksi and Milland indicates that some workers value online forums for both of these reasons: “If I had not found TurkerNation, I would not have made as much money for sure. And the fun we have when things are slow: priceless.”

The Crowd is a Collaborative Network
M. Gray, S. Suri, S. Ali, D. Kulkarni
Feb. 27, 2016

More recently, a 2014 study of TurkerNation, a popular forum dominated by U.S.-based MTurk users, found, through thematic coding of discussion threads and 29 open-ended, semi-structured interviews with forum participants, that U.S. MTurk workers also widely practice the forms of collaboration that we saw among India based workers (Zyskowski and Milland, not published).

Turking in a Global Labour Market
D. Martin, J. O’Neill, N. Gupta, B. V. Hanrahan
Jan. 13, 2016

In this paper we draw on qualitative data from two in-depth ethnomethodological studies: (a) a study of content and interaction on Turker Nation, a forum featuring primarily US Turkers, and (b) an ethnography of Indian Turkers.

Algorithms That Divide and Unite: Delocalization, Identity, and Collective Action in ‘Microwork’
V. Lehdonvirta

A handful of microworkers and supporters are continuing attempts to mobilize and organize digital precariots. For example, in 2014 Canadian MTurk worker Kristy Milland organized a campaign for workers to send demand letters to Amazon CEO Jeff Bezos (Harris 2014). Somewhat ironically, one of the ways in which Milland sought to mobilize workers was by hiring them via MTurk. In the absence of effective organization, becoming an employer is the most effective means that organizers like Milland have at their disposal.

Personalized Task Recommendation in Crowdsourcing Systems
D. Geiger

In order to obtain qualitative user input on metacrowd’s design, early versions of the service were put up for review in two of the leading forums ( for Mechanical Turk workers.

Crowds for Clouds: Recent Trends in Humanities Research Infrastructures
T. Blanke, C. Kristel, L. Romary
Dec 27, 2015

Workers' rights do not seem to follow the Turkers. There is, for instance, no guarantee of payment after the job and there are no benefits. has a hall of shame of worst jobs for Turkers.

Working on Low-Paid Micro-Task Crowdsourcing Platforms: An Existence, Relatedness and Growth View
S. Moussawi & M. Koufaris
Dec. 13, 2015

[There] is a forum called turker nation where others workers share their story so [I] feel part of the mturk community.

Job-related discourse on social media
T. Liu, C. M. Homan, C. O. Alm, A. M. White, M. C. Lytle-Flint, H. A. Kautz
Nov. 16, 2015

Before publishing the HITs, we also consulted with Turker Nation to ensure that the workers were treated and compensated fairly for their tasks.

Re-assembling the Assembly Line: Digital Labor Economies and Demands for an Ambient Workforce (video)
M.L. Gray
Nov. 3, 2015

The presentation draws on findings from a two-year collaborative study of crowdwork--“the process of taking tasks that would normally be delegated to an employee and distributing them to a large pool of online workers, the ‘crowd,’ in the form of an open call" (Felstiner, 2010).

Crowdsourced Data Management: Industry and Academic Perspectives
A. Marcus & A. Parameswaran
Oct. 30, 2015

Either through the platform or through forums that workers use (eg, TurkerNation), seek out worker feedback and respond to it quickly.

Lessons Learned from Crowdsourcing Complex Engineering Tasks
M. Staffelbach, P. Sempolinski, T. Kijewski-Correa, D. Thain, D. Wei, A. Kareem, G. Madey
Sep. 18, 2015

We also set up accounts on Turkernation and CloudMeBaby, which are forum websites designed for communication among crowd workers and employers. We posted our HIT’s information and introduced ourselves on these sites. Many crowd workers try very hard to keep their approval ratings high for qualifications, so working for a new requester can be a risk, because some new requesters reject a lot of HITs not understanding and/or caring about the damage it can do to crowd workers' ratings. Therefore, introducing ourselves and welcoming open communication made more crowd workers comfortable with considering and working on our HITs.

Conducting perception research over the internet: a tutorial review
A.T. Woods, C. Velasco, C.A. Levitan, X. Wan, C. Spence
July 27, 2015

In an excellent blog article, Spamgirl, aka Kristy Milland (2014a), highlights how low wage tasks will tend to be avoided by all, except by, for example, those using ‘bots’ to help automate their answers, and those in a desperate situation – which very likely impacts on data quality (see also Silberman, Milland, LaPlante, Ross, & Irani, 2015).

Commoditized Workers
A. Aloisi
July 8, 2015

“I am a human being, not an algorithm, and yet [employers] seem to think I am there just to serve their bidding”, quoting from the letter that Kristy Milland, a Canadian Turker, sent to Bezos ( last Christmas.

Worker-Centric Design for Software Crowdsourcing: Towards Cloud Careers
D. Murray-Rust, O. Scekic, D. Lin
May 29, 2015

A worker identifying people in CCTV photos or writing reviews of restaurants they have never visited might have a feeling that they are complicit in something ethically questionable: “Do not do any HITs that involve: filling in CAPTCHAs; secret shopping; test our web page; . . . If you feel in your gut it’s not on the level, IT’S NOT. Why? Because they are scams. . . ” spamgirl on TurkerNation [26]

On Wayfaring in Social Machines
D. Murray-Rust, S. Tarte, M. Hartswood, O. Green
May 18, 2015

At the other end of the scale, Mechanical Turk (AMT) explicitly prevents entanglement, with workers being kept as separate as possible. The desire to build a social fabric was strong enough that sites such as Turker Nation sprang up, in order to provide people with the possibilities of encounter which they desired. ... Alternatively, the designer can impose lines, restrictions and grids in an attempt to achieve a desired structure or purpose. In the latter case, just as Mechanical Turk workers founded Turker Nation, the agency of participants will be turned towards resisting or desisting the imposed order, rather than being incorporated into a resource that the social machine may use to renew and deepen itself.

J. Kleske
re:publica 2015, Berlin
May 7, 2015

Segment begins at 33:29.

TurkBench: Rendering the Market for Turkers
B.V. Hanrahan, J.K. Willamowski, S. Swaminathan, D.B. Martin
April 18, 2015

In addition to the tools developed by researchers, Turkers have created several scripts and plugins to make turking easier. For example, ... TurkAlert which notifies its users when certain HITs become available. ... Also like these tools we must operate within the constraints of the AMT infrastructure, which provides limited support for Turkers. Tools have dealt with this problem in a variety of ways, ... Still others (e.g. ... TurkAlert) crawl the available HITs to provide ongoing information about the status of the market.

The Controlled, Large Online Social Experimentation Platform (CLOSE)
K. Lakkaraju, B. Medina, A.N. Rogers, D.M. Trumbo, A. Speed, J.T. McClain
March 17, 2015

This mechanism has two benefits: 1. It prevents collusion: Even if subjects discuss the task on online forums (such as the popular ... when subjects...

The Value of Employer Reputation in the Absence of Contract Enforcement: A Randomized Experiment
A. Benson, A. Sojourner, A. Umyarov
January 13, 2015

Popular resources include ..., This is also the basis of resources such as, which alerts workers when a flagged employer posts a new task. ... The high incidence of repeat submissions may be for a number of factors, including: ... automated alerts (e.g. TurkAlert)...

Attentive Turkers: MTurk participants perform better on online attention checks than subject pool participants
D.J. Hauser, N. Schwarz
January, 2015

Forums searched included ..., and others. Google searching your requester name and HIT title often yields any available posts regarding your HIT.

The Digital Division of Labor: Socially Constructed Design Patterns of Amazon Mechanical Turk and the Governing of Human Computation Labor
M. Ellmer

Crowdmoderated tools as well as information and discussion websites such as TurkerNation, ... TurkAlert or channels on social media platforms (e.g., reddit), reduce information asymmetries in online labor markets, empower crowds to lobby for their needs and interests in digital working environments, and increase consciousness of the position of digital workers in the field of digital economy.

Chapter 4: Privacy in Crowdsourced Platforms
T. Kandappu, A. Friedman, V. Sivaraman, R. Boreli

In fact, a thread on Turker Nation (a forum dedicated to AMT discussions), predating Lease et al.'s work, reflects the surprise of workers who learned about this relation.

Cloud-Sourcing: Using an Online Labor Force to Detect Clouds and Cloud Shadows in Landsat Images
L. Yu, S.B. Ball, C.E. Blinn, K. Moeltner, S. Peery, V.A. Thomas, R.H. Wynne

The project was announced a few days prior to each phase at two popular blog sites for MT participants, Turker Nation...

Online crowdsourcing for efficient rating of speech: A validation study
T. McAllister Byun, P.F. Halpin, D. Szeredi
December 15, 2014

It is good practice for researchers to sign up for; perusing the message boards will give the requester a sense of accepted practices and will also make it possible to monitor one’s own reputation in the community.

Optimizing Human Computation to Save Time and Money
B. Livshits, G. Kastrinis
November 14, 2014

One way to measure worker sentiment is via sites such as various discussion fora such as ... sites like, which aim to highlight easy and wellpaying HITs (human interest tasks) and warn others about hits that, for instance, do not pay as well as advertised or require more time than expected to complete.

Crowd-Workers: Aggregating Information Across Turkers To Help Them Find Higher Paying Work
C. Callison-Burch
November 3, 2014

I would like to thank... Turker Nation's spamgirl for providing thoughtful feedback...

The politics of digital crowds
C. Aradau, T. Blanke
August, 2014

Turker Nation, an online forum for Turkers, ( has a hall of shame of the worst jobs.

Trick or Treat: Putting Peer Prediction to the Test
Z.A. Gao, A. Mao, Y. Chen, R.P. Adams
June 12, 2014

Workers constantly communicate through various forums, review requesters on sites such as TurkOpticon, and use third party services like TurkAlert to monitor requester activities. ... Popular sites include

Designs for a future microtask market (dissertation proposal)
M.S. Silberman
May 9, 2014

One reasonable view was expressed by spamgirl, well-known crowd worker and organizer... in Amazon's Mechanical Turk system (AMT), in 2010...

Being a Turker
D. Martin, B.V. Hanrahan, J. O'Neill, N. Gupta
February 19, 2014

Example 7: An Engaging Requester

Great requester, honest, good communicator, cares about what is best for us as much as is best for him. Highly recommended. You’ve proven to me that you are a good requester because you’ve asked here for help – you have my respect.

Understanding Indian Crowdworkers
N. Gupta, A. Crabtree, T. Rodden, D. Martin

We have also been studying Turkers by analyzing publicly displayed posts on forums such as Turker Nation, Turk Alert... to understand the Turker perspective and reasoning about their work, Turker-Requester relationships and other practical and ethical matters.

From critical design to critical infrastructure: lessons from turkopticon
L. Irani, M.S. Silberman

Turkopticon, alongside crucial worker communities such as Turker Nation...

Bucking the trend: improved evaluation and annotation practices for ESL error detection systems
J. Tetreault, M. Chodorow, N. Madnani

It is important to look at the reviews that Turkers provide outside of the HIT in venues such as Turkopticon and Turker Nation.

Turk-Life in India
N. Gupta, D. Martin, B. Hanrahan, J. O'Neill

Martin et. al analysed the publicly displayed posts of Turkers (primarily from the US) on the Turker Nation forum to understand their reasoning about work, community, and Turker- requester relationships.

The Chameleon-like Nature of Evaluative Adjectives
L. Karttunen, S. Peters, A. Zaenen, C. Condoravdi

We nevertheless paid them the same fee as the others: $1 for the completed task, more than the prevailing rate at the time, to maintain a good reputation as an employer in the Turker Nation community (

Beyond AMT: An Analysis of Crowd Work Platforms
D. Vakharia, M. Lease
October 7, 2013

Turker Nation Forum. Discussion Thread: *** IMPORTANT READ: HITs You Should NOT Do! ***

Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers
J. Chandler, P. Mueller, G. Paolacci
July 9, 2013

Moreover, plug-ins have been created that allow workers to complete the tasks of favored requesters (e.g.,, As a result, some workers may know more about the HITs available to them—and about the requesters who posted them—than is commonly assumed.

Mechanical Turk is Not Anonymous
M. Lease, J. Hullman, J.P. Bigham, M.S. Bernstein, J. Kim, W. Lasecki, S. Bakhshi, T. Mitra, R.C. Miller
March 6, 2013

While Amazon's Mechanical Turk (AMT) online workforce has been characterized by many people as being anonymous, we expose an aspect of AMT's system design that can be exploited to reveal a surprising amount of information about many AMT Workers, which may include personally identifying information (PII). This risk of PII exposure may surprise many Workers and Requesters today, as well as impact current institutional review board (IRB) oversight of human subjects research involving AMT Workers as participants.

Crowdsourcing for Speech: Economic, Legal and Ethical Analysis
G. Adda, J.J. Mariani, L. Besacier, H. Gelas
January 18, 2013

Below are some statements, from different sources... Turker Nation, illustrating the economic situation of some MTurk workers in the US:

Risks and rewards of crowdsourcing marketplaces
J. Chandler, G. Paolacci, P. Mueller

First, workers tend to follow favorite requesters by subscribing to websites that alert them whenever favored requesters make work available for completion (e.g.,

Experiences surveying the crowd: Reflections on methods, participation, and reliability
C.C. Marshall, F.M. Shipman

Posts in forums such as Turker Nation ( suggest that some respondents participate in surveys to offer their opinions about hot- button issues like privacy, permission, pseudonymity, and anonymity...

Saving, reusing, and remixing web video: using attitudes and practices to reveal social norms
C.C. Marshall, F.M. Shipman

The point system gave us a means to retain data from participants who might have found one or two questions confusing, or who completed HITs prior to accepting the work (a common practice reported in online forums like Turker Nation).

The Benefits of a Model of Annotation
R.J. Passonneau, B. Carpenter

To insure a high proportion of instances with high quality inferred labels, we piloted the HIT design and payment regimen with two trials of two and three words each, and discussed both with turkers on the Turker Nation message board.

Crowdsourcing for Speech Processing
M. Eskenazi, G.A. Levow, H. Meng, G. Parent, D. Suendermann (eds.)

Some regulation between requesters and workers exists through workers’ Blogs or Forums, such as the Mechanical Turk Blog ( or Mechanical Turk forum (

Webbasierte linguistische Forschung: Möglichkeiten und Begrenzungen beim Umgang mit Massendaten
B. Juska-Bacher, C. Biemann, U. Quasthoff

Aus diesem Grund wird auf Crowdarbeiterforen wie dem Mechanical Turk Forum ( oder Turker Nation ( vor Auftraggebern mit diesem Mechanismus gewarnt.

Accessible Crowdwork? Understanding the Value in and Challenge of Microtask Employment for People with Disabilities
K. Zyskowski, M.R. Morris, J.P. Bigham, M.L. Gray, S.K. Kane

Martin et al. analyzed posts on Turker Nation, one of the online web forums for mTurk workers.

Economic and Ethical Background of Crowdsourcing for Speech
G. Adda, J.J. Mariani, L. Besacier, H. Gelas

There is no social or health security, no guarantee of payment and no support coming from unions (which may however be replaced on the Internet by blogs and forums such as Turker Nation...

Collaboratively crowdsourcing workflows with turkomatic
A. Kulkarni, M. Can, B. Hartmann

We noticed that workers on Turker Nation, a discussion forum, posted messages about tasks created by Turkomatic, complaining about poor wording or excessive scope.

Optimization techniques for human computation-enabled data processing systems
A. Marcus

Turkers keep track of good and bad requesters, and share this information on message boards such as Turker Nation.

Conducting behavioral research on Amazon's Mechanical Turk
W. Mason, S. Suri

Turker Nation is an online bulletin board where workers routinely comment on requesters and communicate about individual HITs.

It's all about the Benjamins: An empirical study on incentivizing users to ignore security advice
N. Christin, S. Egelman, T. Vidas, J. Grossklags

We periodically browsed Mechanical Turk user forums such as Turker Nation... and noticed with interest that our task was ranked as legitimate because we were paying users on time and anti-virus software was not complaining about our program.

Crowdsourcing for Language Resource Development: Critical Analysis of Amazon Mechanical Turk Overpowering Use
G. Adda, B. Sagot, K. Fort, J. Mariani
Nov. 2011

Some regulation between Requesters and Turkers exists through Turkers' Blogs or Forums (for instance...

Human-powered sorts and joins
A. Marcus, E. Wu, D. Karger, S. Madden, R. Miller
Sep. 30, 2011

Turkers keep track of good and bad requesters, and share this information on message boards such as Turker Nation.

Web workers unite! addressing challenges of online laborers
B.B. Bederson, A.J. Quinn

This concern about wages is reflected in comments posted at Turker Nation [17], a web-based forum where AMT workers (often called “Turkers”) are free to discuss their situations openly.

Approaching the Symbol Grounding Problem with Probabilistic Graphical Models
S. Tellex, T. Kollar, S. Dickerson, M.R. Walter, A.G. Banerjee, S. Teller, N. Roy

We would like to thank ... the members of the Turker Nation forum.

Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation
S. Tellex, T. Kollar, S. Dickerson, M.R. Walter, A.G. Banerjee, S. Teller, N. Roy

We would like to thank ... the members of the Turker Nation forum.

Labeling Parts of Speech Using Untrained Annotators on Mechanical Turk
J.E. Mainzer

Since Mechanical Turk has no method of rating Requesters, unaffiliated sites such as Turker Nation... have been developed to discuss and review Requesters.

Working the crowd: employment and labor law in the crowdsourcing industry
A. Felstiner

Turkers have already created various online forums to share information and build relationships, (See, e.g., Turker Nation, http://turkers.proboards.eom/indcx.cgi) as have participants in other crowdsourcing platforms.

Putting Out a HIT: Crowdsourcing Malware Installs
C. Kanich, S. Checkoway, K. Mowery

One way to reduce the completed task would be to simply not allow (some percentage of) workers to submit or to refuse to pay them afterward. This is unlikely to be a viable solution in the long term as the workesr will complain to Amazon and warn other workers on Mechanical Turk-oriented forums such as Turker Nation.

CrowdDB: Answering Queries with Crowdsourcing
M.J. Franklin, D. Kossmann, T. Kraska, S. Ramesh, R. Xin

In another case, a bug in CrowdDB triggered false alarms in security warnings for browsers on the workers’ computers, and within hours concerns about our jobs appeared on Turker Nation.

Crowdsourcing Human-Based Computation
D. Wightman
October 16, 2010

Online message boards have been created to provide experienced Mechanical Turk users, often called turkers, with venues for rating companies that post HITs (Turker Nation Message Board System).

Sellers’ problems in human computation markets
M.S. Silberman, J. Ross, L. Irani, B. Tomlinson
July 25, 2010

"spamgirl's warnings pertain to HITs where (a) there is uncertainty about whether the requester intends to pay workers, (b) the requester intends to spam or defraud workers,and/or (c) the task is against AMT’s terms of use."

Automatic Task Design on Amazon Mechanical Turk (thesis)
E.H.C. Huang
April 6, 2010

For example, Turker Nation is an online forum where people discuss anything related to AMT, from efficient working strategies to requester reviews.

Ethics and tactics of professional crowdwork
M.S. Silberman, L. Irani, J. Ross

Administrator spamgirl on Turker Nation, a forum for workers, outlines tasks to avoid:...

WePS-3 Evaluation Campaign: Overview of the Web People Search Clustering and Attribute Extraction Tasks
J. Artiles, A. Borthwick, J. Gonzalo, S. Sekine, E. Amigo

The more interesting initiative, though, was the establishment of a two-way dialog with the Turkers by posting on the Turker Nation bulletin board 9. We understand Turker Nation to be the most popular venue for this kind of discussion...

Crowdsourcing, collaboration and creativity
A. Kittur

Unification sites like Turkernation let workers sanction employers who consistently do not approve legitimate work.

Mechanical Turk
W.L. Forssen, A. Renberg

This leads to fake HITs by non-serious requesters which in turn has lead to a separate part in the community forum Turker Nation, called Hall of Shame [7].

Annotating Large Email Datasets for Named Entity Recognition with Mechanical Turk
N. Lawson, K. Eustice, M. Perkowitz, M. Yetisgen-Yildiz

Out of 169,156 total HITs, we received 702 comments from 140 different workers, as well as over 50 email responses and a dedicated thread at

Crowdflow: Integrating machine learning with mechanical turk for speed-cost-quality flexibility
A.J. Quinn, B.B. Bederson, T. Yeh, J. Lin

The web site Turker Nation serves as a de facto blacklist of Requesters alleged to have used tactics that Turkers consider unfair.

Crowdsourcing for affective annotation of video: Development of a viewer-reported boredom corpus
M. Soleymani, M. Larson

Effort invested in establishing trust accumulates since users exchange information on requesters on Turker Nation... concerning the HITs and bonuses rewarded.

Diversifying Demining: An Experimental Crowdsourcing Method for Optical Mine Detection
D. Andersson

Further, there is an unofficial worker forum called Turker Nation, an active community in which the members discuss and comment on HITs, requesters related topics. Turker Nation was used for worker feedback in preparation of and during the time of the experiment.