Turkopticon: interrupting worker invisibility in amazon mechanical turk

Amazon Mechanical Turk

Amazon Mechanical Turk (MTurk) is a crowdsourcing marketplace that makes it easier for individuals and businesses to outsource their processes and jobs to a distributed workforce who can perform these tasks virtually. This could include anything from conducting simple data validation and research to more subjective tasks like survey participation, content moderation, and more. MTurk enables companies to harness the collective intelligence, skills, and insights from a global workforce to streamline business processes, augment data collection and analysis, and accelerate machine learning development.

Amazon will stop accepting new customers for Mechanical Turk | TechCrunch

These may be the last days of Amazon’s Mechanical Turk.

Using the Amazon Mechanical Turk Workforce - Amazon SageMaker AI

Use SageMaker AI to build, train, and host machine learning models in AWS.

Google Cabs And Uber Bots Will Challenge Jobs 'Below The API'

Working "below the API" is a dead end. Uber drivers, Amazon Mechanical Turk workers, 99design contestants, TaskRabbit taskers and HomeJoy cleaners are all targets for further automation.

Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks

Large language models (LLMs) are remarkable data annotators. They can be used to generate high-fidelity supervised training data, as well as survey and experimental data. With the widespread adoption of LLMs, human gold--standard annotations are key to understanding the capabilities of LLMs and the validity of their results. However, crowdsourcing, an important, inexpensive way to obtain human annotations, may itself be impacted by LLMs, as crowd workers have financial incentives to use LLMs to increase their productivity and income. To investigate this concern, we conducted a case study on the prevalence of LLM usage by crowd workers. We reran an abstract summarization task from the literature on Amazon Mechanical Turk and, through a combination of keystroke detection and synthetic text classification, estimate that 33-46% of crowd workers used LLMs when completing the task. Although generalization to other, less LLM-friendly tasks is unclear, our results call for platforms, researchers, and crowd workers to find new ways to ensure that human data remain human, perhaps using the methodology proposed here as a stepping stone. Code/data: https://github.com/epfl-dlab/GPTurk

The Consensus Trap: Dissecting Subjectivity and the “Ground Truth” Illusion in Data Annotation

As part of the Digital Library's transition to Open Access, new features for researchers are available in the Premium Edition. Click here to learn more.

Rethinking the human-readability infrastructure

As part of the Digital Library's transition to Open Access, new features for researchers are available in the Premium Edition. Click here to learn more.

Our Approach to Artificial Intelligence

We are experimenting with using AI tools to extend our work as a small nonprofit, so that we can focus our time on reinforcing human connections, conversations, and communities that have eroded.

What We're No Longer Seeing: AI and the Invisible Newcomer in Open Source

How AI is absorbing the visible friction that open-source communities have always relied on to see—and welcome—newcomers.

Socialized Proof of Work — Open Indie

There’s an increasing obsession with “humanness” these days: Humanness in the age of AI Worldcoin: a solution in search of its problem (...

Anthropic Economic Index report: Uneven geographic and enterprise AI adoption

In this report, we document patterns of Claude usage over time, in 150+ countries, across US states, and among businesses deploying Claude through the API. Based on a privacy-preserving analysis of 1 million conversations on Claude.ai and 1 million API transcripts, we have four key findings: (1) Users increasingly entrust Claude with more autonomy, with directive task delegation rising from 27% to 39% in the past eight months. (2) Claude usage is geographically concentrated with high income countries overrepresented in global usage relative to their working age population. (3) Local economic considerations shape patterns of use both in terms of topic and in mode of collaboration with Claude. (4) API customers use Claude to automate tasks with greater specialization among use cases most amenable to programmatic access. To enable researchers and policymakers to further study the impact of AI on the economy, we additionally open-source the underlying data for this report.

Revisiting creative behaviour as an epistemic process: lessons from 12 computational artists and designers | Proceedings of the 35th Australian Computer-Human Interaction Conference

As part of the Digital Library's transition to Open Access, new features for researchers are available in the Premium Edition. Click here to learn more.

Anthropic users face a new choice – opt out or share your chats for AI training | TechCrunch

Anthropic is making some major changes to how it handles user data. Users have until September 28 to take action.