In Pursuit of Ideal Data: Epistemic Enchantment and the Unintended Consequences of Datafying for Artificial Intelligence

Loss of Oversight: How AI systems may become harder to audit, monitor, and investigate

The safety of advanced AI systems increasingly depends on the ability to oversee them: to audit models for concerning behaviours before deployment, monitor their activity during operation, and investigate incidents after they occur. This report maps the landscape of AI oversight and assesses how it is likely to change. Drawing on 25 expert interviews across frontier AI developers, government, NGOs, and academia, together with a literature review and internal analysis, we examine five sources of oversight signal: model behaviour, chain-of-thought reasoning, internals activations and circuits, memory architectures, and honesty training. For each source, we identify the properties that current oversight relies on, the pathways by which these properties could degrade, and the technical levers available to preserve them. Our central finding is that literature and expert opinion support the conclusion that current oversight rests on foundations that are likely to erode, absent effective intervention. We recommend that developers track and report shifts in oversight-relevant properties, preserve oversight affordances by design, and invest in emerging oversight techniques as fallbacks against continued degradation of current methods.

The AI "Evaluation Crisis" Is an Opportunity to Get Data Flow Right

Why the AI evaluation crisis could force a reckoning on dataset provenance, attribution, and consent.

Qualitative Research in an Era of Artificial Intelligence: A Pragmatic Approach to Data Analysis, Workflow, and Computation

Computational developments—particularly artificial intelligence—are reshaping social scientific research and raising new questions for in-depth methods such as ethnography and qualitative interviewing. Building on classic debates about computers in qualitative data analysis, we revisit possibilities and dangers in an era of automation, large language model chatbots, and big data. We introduce a typology of contemporary approaches to using computers in qualitative research: streamlining workflows, scaling up projects, hybrid analytical methods, the sociology of computation, and technological rejection. Drawing from scaled team ethnographies and solo research integrating computational social science alongside in-depth observation, we describe methodological choices across study life cycles, from literature reviews through data collection, coding, text retrieval, and representation. We argue that new technologies hold potential to address long-standing methodological challenges when deployed with knowledge, purpose, and ethical commitment. Yet, a pragmatic approach—moving beyond technological optimism and dismissal—is essential given rapidly changing tools that are both generative and dangerous. Computation now saturates research infrastructure, from algorithmic literature searches to scholarly metrics, making computational literacy a core methodological competence in and beyond sociology. We conclude that when used carefully and transparently, contemporary computational tools can meaningfully expand, rather than displace, the irreducible insights of qualitative research.

WAO - The Workers' Algorithm Observatory

The WAO researches and audits work algorithms to empower workers. Companies increasingly rely on opaque, black-box algorithms to determine pay, scheduling, and access to work, while the data needed to understand and challenge these systems remains out of reach for workers and organizers. To address this gap, the WAO develops technology-enabled research tools, data infrastructure, and academic–organizer partnerships that support existing labor and worker organizations. By building meaningful relationships with worker organizers, crowdsourcing data, conducting transparent and policy-relevant research, and thinking deeply about the data workers need both now and in the next 5-20 years, the WAO aims to increase worker power, inform policymaking, and improve conditions for workers today and in the future.

Artificial intelligence and illusions of understanding in scientific research

Scientists are enthusiastically imagining ways in which artificial intelligence (AI) tools might improve research. Why are AI tools so attractive and what are the risks of implementing them across the research pipeline? Here we develop a taxonomy of scientists’ visions for AI, observing that their appeal comes from promises to improve productivity and objectivity by overcoming human shortcomings. But proposed AI solutions can also exploit our cognitive limitations, making us vulnerable to illusions of understanding in which we believe we understand more about the world than we actually do. Such illusions obscure the scientific community’s ability to see the formation of scientific monocultures, in which some types of methods, questions and viewpoints come to dominate alternative approaches, making science less innovative and more vulnerable to errors. The proliferation of AI tools in science risks introducing a phase of scientific enquiry in which we produce more but understand less. By analysing the appeal of these tools, we provide a framework for advancing discussions of responsible knowledge production in the age of AI.

The Limits of Data

Policymakers want to make decisions based on clear data, but important factors are lost when we rely solely on data. A philosopher writes:

Doctored data sets could trick AI agents

Researchers warn that ‘poisoned’ information can undermine the reliability of tools that analyse data autonomously.

Can “Conscious Data Contribution” Help Users to Exert “Data Leverage” Against Technology Companies?

Tech users currently have limited ability to act on concerns regarding the negative societal impacts of large tech companies. However, recent work suggests that users can exert leverage using their role in the generation of valuable data, for instance by withholding their data contributions to intelligent technologies. We propose and evaluate a new means to exert this type of leverage against tech companies: "conscious data contribution" (CDC).

Data Leverage: A Framework for Empowering the Public in its Relationship with Technology Companies

Many powerful computing technologies rely on implicit and explicit data contributions from the public. In this paper, we synthesize emerging research that seeks to better understand and help people action this data leverage. Drawing on prior work in areas including machine learning, human-computer interaction, and fairness and accountability in computing, we present a framework for understanding data leverage that highlights new opportunities to change technology company behavior related to privacy, economic inequality, content moderation and other areas of societal concern.

AI Scientists as Engines of Discovery: A Case for Development within Reformed Institutions

Agentic artificial intelligence (AI) systems are beginning to assist, accelerate, and partially automate scientific discovery, performing tasks that span literature synthesis, code generation, data analysis, hypothesis proposal, and model criticism. We argue that this transition is qualitative rather than incremental, and that suitably designed multi-agent systems may evolve from passive computational tools into ``AI scientists'' that can expand the hypothesis-generating and verification capacity of science. Such systems must be developed and deployed within a scientific ecosystem fit for purpose: institutions must be redesigned for verification, accountability, interpretability, and dual-use safety. We sketch how multi-agent architectures, illustrated by the prototype framework \textit{Denario}, accelerate the discovery cycle and traverse model spaces beyond human reach; examine what this implies for authorship, peer review, and the enduring role of human scientists; and close with recommendations for governing AI as an epistemic actor rather than a mere instrument.

Our Spreadsheet Overlords

Weatherby argues that the current discourse around AI, especially the buzz around “artificial general intelligence,” is a distraction from its actual impact: the expansion of bureaucracy through massive data systems…

Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions

AI assistants can impart value judgments that shape people's decisions and worldviews, yet little is known empirically about what values these systems rely on in practice. To address this, we develop a bottom-up, privacy-preserving method to extract the values (normative considerations stated or demonstrated in model responses) that Claude 3 and 3.5 models exhibit in hundreds of thousands of real-world interactions. We empirically discover and taxonomize 3,307 AI values and study how they vary by context. We find that Claude expresses many practical and epistemic values, and typically supports prosocial human values while resisting values like "moral nihilism". While some values appear consistently across contexts (e.g. "transparency"), many are more specialized and context-dependent, reflecting the diversity of human interlocutors and their varied contexts. For example, "harm prevention" emerges when Claude resists users, "historical accuracy" when responding to queries about controversial events, "healthy boundaries" when asked for relationship advice, and "human agency" in technology ethics discussions. By providing the first large-scale empirical mapping of AI values in deployment, our work creates a foundation for more grounded evaluation and design of values in AI systems.

Dario Amodei — The Urgency of Interpretability

In the decade that I have been working on AI, I’ve watched it grow from a tiny academic field to arguably the most important economic and geopolitical issue in the world.  In all that time, perhaps the most important lesson I’ve learned is this: the progress of the underlying technology is inexorable, driven by forces too powerful to stop, but the way in which it happens—the order in which things are built, the applications we choose, and the details of how it is rolled out to society—are eminently possible to change, and it’s possible to have great positive impact by doing so.  We can’t stop the bus, but we can steer it.  In the past I’ve written about the importance of deploying AI in a way that is positive for the world, and of ensuring that democracies build and wield the technology before autocracies do.  Over the last few months, I have become increasingly focused on an additional opportunity for steering the bus: the tantalizing possibility, opened up by some recent advances, that we could succeed at interpretability—that is, in understanding the inner workings of AI systems—before models reach an overwhelming level of power.

The Unaccountability Machine: Why Big Systems Make Terrible Decisions—and How the World Lost Its Mind

Longlisted for the 2024 Financial Times Book of the Year. How life and the economy became a black box—a collection of systems no one understands, producing outcomes no one likes.   Passengers get bumped from flights. Phone menus disconnect. Automated financial trades produce market collapse. Of all the challenges in modern life, some of the most vexing come from our relationships with automation: a large system does us wrong, and there’s nothing we can do about it. The problem, economist Dan Davies shows, is accountability sinks: systems in which decisions are delegated to a complex rule book or set of standard procedures, making it impossible to identify the source of mistakes when they happen. In our increasingly unhuman world—lives dominated by algorithms, artificial intelligence, and large organizations—these accountability sinks produce more than just aggravation. They make life and economy unknowable—a black box for no reason. In The Unaccountability Machine, Davies lays bare how markets, institutions, and even governments systematically generate outcomes that no one—not even those involved in making them—seems to want. Since the earliest days of the computer age, theorists have foreseen the dangers of complex systems without personal accountability. In response, British business scholar Stafford Beer developed an accountability-first approach to management called “cybernetics,” which might have taken off had his biggest client (the Chilean government) not fallen to a bloody coup in 1973. With his signature blend of economic and journalistic rigor, Davies examines what’s gone wrong since Beer, including what might have been had the world embraced cybernetics when it had the chance. The Unaccountability Machine is a revelatory and resonant account of how modern life became predisposed to dysfunction.

From the Platform Society to the AI Society: Towards Critical Studies of Generative AI

The era of AI has begun. Generative AI is rapidly reshaping knowledge production, culture, and political authority, giving rise to an emerging AI society. Yet this transformation did not emerge ex nihilo. This paper argues that the AI society can only be understood in relation to the platform society from which it arises. Tracing the transition from platforms to AI, we identify interlinked economic, epistemic, and political shifts. Economically, AI emerges within platform-based rentier capitalism but reconfigures the monopoly mechanisms on which its accumulation depends. Epistemically, LLMs mark a shift from predictive to generative epistemics, entangling theory formation and knowledge production with private research-as-a-service infrastructures. Politically, governance shifts from data politics to alignment politics: from shaping visibility to shaping what can be said, thought, and imagined. Together, these transformations signal a qualitative shift in mediation—from governing interaction to governing cognition itself—and call for a Critical AI Studies.

Architecting Trust in Artificial Epistemic Agents

Large language models increasingly function as epistemic agents -- entities that can 1) autonomously pursue epistemic goals and 2) actively shape our shared knowledge environment. They curate the information we receive, often supplanting traditional search-based methods, and are frequently used to generate both personal and deeply specialized advice. How they perform these functions, including whether they are reliable and properly calibrated to both individual and collective epistemic norms, is therefore highly consequential for the choices we make. We argue that the potential impact of epistemic AI agents on practices of knowledge creation, curation and synthesis, particularly in the context of complex multi-agent interactions, creates new informational interdependencies that necessitate a fundamental shift in evaluation and governance of AI. While a well-calibrated ecosystem could augment human judgment and collective decision-making, poorly aligned agents risk causing cognitive deskilling and epistemic drift, making the calibration of these models to human norms a high-stakes necessity. To ensure a beneficial human-AI knowledge ecosystem, we propose a framework centered on building and cultivating the trustworthiness of epistemic AI agents; aligning AI these agents with human epistemic goals; and reinforcing the surrounding socio-epistemic infrastructure. In this context, trustworthy AI agents must demonstrate epistemic competence, robust falsifiability, and epistemically virtuous behaviors, supported by technical provenance systems and "knowledge sanctuaries" designed to protect human resilience. This normative roadmap provides a path toward ensuring that future AI systems act as reliable partners in a robust and inclusive knowledge ecosystem.