Abstract The growing availability of large datasets in a variety of domains presents an opportunity for researchers to use field data to better understand psychological concepts. I discuss, from an empirical economics point of view, steps for how to study cognition in large datasets. I use two recent papers that explore psychology in the used‐car market as motivating examples. These examples help illustrate the potential importance of big data as a way to explore human psychology and cognition. , The growing availability of large datasets in a variety of domains presents an opportunity for researchers to use field data to better understand psychological concepts. I discuss from an empirical economics point of view, steps for how to study cognition in large datasets and illustrate these steps with recent empirical papers.
Addressing the Precision-Breadth-Simplicity Impossible Trinity in Psychological Research: A Comprehensive Exploration Approach
Psychological research faces a fundamental challenge—the Precision-Breadth-Simplicity (PBS) impossible trinity. While experimental findings are often precise and simple, they tend to be narrow in scope. Conversely, broad-and-simple concepts frequently lack precision. Developing theories that are both precise and broad is scientifically valuable but inevitably introduces complexity, which conflicts with humans’ cognitive limitations in processing complexity. To address this impossible trinity, I propose a comprehensive exploration (CE) approach—a data-guided theory-building framework that involves: (1) designing experimental conditions in a stimulus-driven way, with minimal upfront theoretical specification; (2) conducting experiments with tens of millions of observations (e.g., 40 million responses in Huang, 2025a); (3) modeling the results through iterative improvements; and (4) producing the outcome: a moderately complex quantitative information-processing model to integrate diverse empirical findings. Inspired by similar strategies that drove breakthroughs in artificial intelligence (e.g., ImageNet’s role in advancing object recognition), the CE approach offers a promising path toward more integrative psychological theories. Initial implementations in visual working memory research demonstrate both its practicality and potential to transform how we study mental processes.
Living in Data: A Citizen's Guide to a Better Information Future (Paperback)
Jer Thorp’s analysis of the word “data” in 10,325 New York Times stories written between 1984 and 2018 shows a distinct trend: among the words most closely associated with “data,” we find not only its classic companions “information” and “digital,” but also a variety of new neighbors—from “scandal” and “misinformation” to “ethics,” “friends,” and “play.”To live in data in the twenty-first century
Consumers’ Mental Representation of Expenditures: Implications for Spending and Savings Decisions
Abstract People’s mental representation of expenditures is crucial to their budgeting. This article proposes that much like how they represent natural kinds (e.g., animals and plants), people represent expenditures in a hierarchical taxonomy. Seven studies, supported by six norming studies and three pilots, revealed that expenditures are represented hierarchically. We first recover people’s mental representations using a successive pile-sort method that asks people to form hierarchies of categories with common expenditures (e.g., rent, dining out, etc.). The pile-sort reveals consensus in people’s representations of expenditures and that these representations are relatively stable over time. Further, people’s adjustment in their spending behavior can be predicted by the distance between items in their representation. Specifically, when people overspent on an item, they spontaneously adjust spending more on taxonomically closer items. We examine this spontaneous adjustment behavior using both laboratory studies and field data with 6.5 million grocery shopping trips over 12 years. The findings highlight the connection between mental representation and consumer behavior, and they emphasize the importance of studying concepts and categories in the context of consumption.
Algorithm appreciation: People prefer algorithmic to human judgment
Even though computational algorithms often outperform human judgment, received wisdom suggests that people may be skeptical of relying on them (Dawes, 1979). Counter to this notion, results from six experiments show that lay people adhere more to advice when they think it comes from an algorithm than from a person. People showed this effect, what we call algorithm appreciation, when making numeric estimates about a visual stimulus (Experiment 1A) and forecasts about the popularity of songs and romantic attraction (Experiments 1B and 1C). Yet, researchers predicted the opposite result (Experiment 1D). Algorithm appreciation persisted when advice appeared jointly or separately (Experiment 2). However, algorithm appreciation waned when: people chose between an algorithm’s estimate and their own (versus an external advisor’s; Experiment 3) and they had expertise in forecasting (Experiment 4). Paradoxically, experienced professionals, who make forecasts on a regular basis, relied less on algorithmic advice than lay people did, which hurt their accuracy. These results shed light on the important question of when people rely on algorithmic advice over advice from people and have implications for the use of “big data” and algorithmic advice it generates.
Promoting User Data Autonomy During the Dissolution of a Monopolistic Firm
The deployment of AI in consumer products is currently focused on the use of so-called foundation models, large neural networks pre-trained on massive corpora of digital records. This emphasis on scaling up datasets and pre-training computation raises the risk of further consolidating the industry, and enabling monopolistic (or oligopolistic) behavior. Judges and regulators seeking to improve market competition may employ various remedies. This paper explores dissolution -- the breaking up of a monopolistic entity into smaller firms -- as one such remedy, focusing in particular on the technical challenges and opportunities involved in the breaking up of large models and datasets. We show how the framework of Conscious Data Contribution can enable user autonomy during under dissolution. Through a simulation study, we explore how fine-tuning and the phenomenon of "catastrophic forgetting" could actually prove beneficial as a type of machine unlearning that allows users to specify which data they want used for what purposes.
The Wisdom of Individuals: Exploring People's Knowledge About Everyday Events Using Iterated Learning
Abstract Determining the knowledge that guides human judgments is fundamental to understanding how people reason, make decisions, and form predictions. We use an experimental procedure called ‘‘iterated learning,’’ in which the responses that people give on one trial are used to generate the data they see on the next, to pinpoint the knowledge that informs people's predictions about everyday events (e.g., predicting the total box office gross of a movie from its current take). In particular, we use this method to discriminate between two models of human judgments: a simple Bayesian model ( Griffiths & Tenenbaum, 2006 ) and a recently proposed alternative model that assumes people store only a few instances of each type of event in memory (Min K ; Mozer, Pashler, & Homaei, 2008 ). Although testing these models using standard experimental procedures is difficult due to differences in the number of free parameters and the need to make assumptions about the knowledge of individual learners, we show that the two models make very different predictions about the outcome of iterated learning. The results of an experiment using this methodology provide a rich picture of how much people know about the distributions of everyday quantities, and they are inconsistent with the predictions of the Min K model. The results suggest that accurate predictions about everyday events reflect relatively sophisticated knowledge on the part of individuals.
The Inversion Problem: Why Algorithms Should Infer Mental State and Not Just Predict Behavior
More and more machine learning is applied to human behavior. Increasingly these algorithms suffer from a hidden—but serious—problem. It arises because they often predict one thing while hoping for another. Take a recommender system: It predicts clicks but hopes to identify preferences. Or take an algorithm that automates a radiologist: It predicts in-the-moment diagnoses while hoping to identify their reflective judgments. Psychology shows us the gaps between the objectives of such prediction tasks and the goals we hope to achieve: People can click mindlessly; experts can get tired and make systematic errors. We argue such situations are ubiquitous and call them “inversion problems”: The real goal requires understanding a mental state that is not directly measured in behavioral data but must instead be inverted from the behavior. Identifying and solving these problems require new tools that draw on both behavioral and computational science.
What makes something data?
This is a question I posted on BlueSky on Friday 11/21/25, inspired by a talk I recently attended about evaluation of “AI” systems. I think…
How Field Experiments in Economics Can Complement Psychological Research on Judgment Biases
This review summarizes results of field experiments examining individual behaviors across several market settings—from open-air markets to rideshare markets to tax-compliance markets—where people sort themselves into market roles wherein they make consequential decisions. Using three distinct examples from my own research on the endowment effect, left-digit bias, and omission bias, I showcase how field experiments can help researchers understand mediators, heterogeneity, and causal moderation involved in judgment biases in the field. In this manner, the review highlights that economic field experiments can serve an invaluable intellectual role alongside traditional laboratory research.
Beware of samples! A cognitive-ecological sampling approach to judgment biases.
Data Everyday: Data Literacy Practices in a Division I College Sports Context
The knowledge layer for collective intelligence
A portable data substrate for humans and their tools to think together.
The knowledge layer for collective intelligence
A portable data substrate for humans and their tools to think together.
The knowledge layer for collective intelligence
A portable data substrate for humans and their tools to think together.
The knowledge layer for collective intelligence
A portable data substrate for humans and their tools to think together.
