A model of human behavior, and new ways to measure how faithfully AI can simulate people.
AI Behavioral Science
We outline a foundation for a new field of ``AI Behavioral Science,'' covering three perspectives. First, as AI becomes ubiquitous and is increasingly proprietary and opaque, it becomes vital to develop techniques for assessing AI behavior. We outline how tools developed to assess people's behaviors by social scientists can be used to assess and infer AI's behaviors biases, tendencies, and heuristics. Second, we also discuss how AI can change the ways in which we learn about human behavior. Beyond its computational power, AI offers new techniques for simulating, inferring, and predicting human behaviors that we outline and discuss. Third, as humans and AI are interacting in increasingly complex and intertwined systems, we need to understand the implications for the resulting economic and political outcomes. We outline issues that are increasingly pressing concerning the future of human-AI interactions and potential changes and disruptions that can ensue.
[Keynote 04] AgentSociety: Exploring Large Language Model Agents for Piloting Social Experiments
AI Simulation Platform for Human Behavior | Simile
Simulate how real customers respond to a launch, price change, or campaign — before you ship. Built by the Stanford researchers behind generative agents.
AI Behavioral Science: A Framework and Agenda
We discuss the challenges and opportunities present in the rapidly emerging area of “AI Behavioral Science.” We frame it via three subfields. First, as AI becomes ubiquitous and is increasingly proprietary and opaque, it becomes vital to develop models of AI and methods for assessing AI behavior. We outline how tools developed to assess people’s behaviors by social scientists can be used to model, assess and infer AI’s behaviors biases, tendencies, and heuristics. Second, we also discuss how AI can change the ways in which we learn about human behavior. Beyond its computational power, AI offers new techniques for simulating, inferring, predicting, and analyzing human behaviors. Third, as humans and AI are interacting in increasingly complex and intertwined systems, we need to analyze and model human-AI interactions including how human and AI behaviors depend on interactions at the individual level, how interacting systems of humans and AI behave, and ultimately how AI’s integration into society affects economic and political outcomes. We discuss current research, questions, agendas, and goals in each of these three subfields and how they depend upon each other.
AI: A Guide for Thinking Humans | Melanie Mitchell | Substack
I write about interesting new developments in AI. Click to read AI: A Guide for Thinking Humans, by Melanie Mitchell, a Substack publication with tens of thousands of subscribers.
Startups Target the Tricky Task of Making AI Seem More Human
Simulated human behavior companies raise millions
Gallup Begins Research on Simulated Responses
Gallup is exploring whether AI-generated agents perform well in predicting people's responses and where they fall short.
AI agent runs amok in Fedora and elsewhere
Agentic AI systems can be used to do a variety of things autonomously on behalf of a human user [...]
[Keynote 01] A Theory of Appropriateness: Social Norms for Humans and AIs
Take caution in using LLMs as human surrogates
Recent studies suggest large language models (LLMs) can generate human-like responses, aligning with human behavior in economic experiments, surveys, and political discourse. This has led many to propose that LLMs can be used as surrogates or simulations for humans in social science research. However, LLMs differ fundamentally from humans, relying on probabilistic patterns, absent the embodied experiences or survival objectives that shape human cognition. We assess the reasoning depth of LLMs using the 11-20 money request game. Nearly all advanced approaches fail to replicate human behavior distributions across many models. The causes of failure are diverse and unpredictable, relating to input language, roles, safeguarding, and more. These results warrant caution in using LLMs as surrogates or for simulating human behavior in research.
Why I don’t trust most human-AI interaction experimental research – Jason Collins blog
Behavioural economics, data science and artificial intelligence.
AIs are not people
Humans and other fleshy life-forms are valuable in ways that artificial intelligence never can be.
Our minds did not evolve for AI
AI is mental junk food: tasty and tempting, but ultimately unhealthy — a cognitive hot dog
