







Information provision experiments allow researchers to test economic theories and answer policy-relevant questions by varying the information set available to respondents. We survey the emerging literature using information provision experiments in economics and discuss applications in macroeconomics, finance, political economy, public economics, labor economics, and health economics. We also discuss design considerations and provide best-practice recommendations on how to (i) measure beliefs; (ii) design the information intervention; (iii) measure belief updating; (iv) deal with potential confounds, such as experimenter demand effects; and (v) recruit respondents using online panels. We finally discuss typical effect sizes and provide sample size recommendations.
Mechanism Experiments and Policy Evaluations
Randomized controlled trials are increasingly used to evaluate policies. How can we make these experiments as useful as possible for policy purposes? We argue greater use should be made of experiments that identify the behavioral mechanisms that are central to clearly specified policy questions, what we call "mechanism experiments." These types of experiments can be of great policy value even if the intervention that is tested (or its setting) does not correspond exactly to any realistic policy option.
Behavioral Intervention Construal: A Framework for Understanding Inferences from Behavioral Interventions
Organizations frequently use behavioral interventions—including incentives, messaging campaigns, and modifications to choice architecture—to influence people’s behavior. However, recent evidence suggests that such interventions often have inconsistent effects across implementations. To understand this variability, we build on the idea that interventions are not simply levers for changing behavior, they also signal useful information to decision makers. For example, when an option is set as the default, people may infer that it is recommended by the organization—an inference that can increase the likelihood that people stick with that option. However, when people perceive that defaults benefit the organization, those defaults may be rejected. Here, we propose a framework to better understand the effects of behavioral interventions by identifying the inferences they prompt people to draw. In developing this framework, we first characterize the defining features of behavioral interventions that determine how they are psychologically experienced. These features implicate five fundamental needs, which guide decision makers’ information-seeking processes. From these needs, we derive a principled typology of inferences drawn by decision makers in response to behavioral interventions, bringing together scattered evidence for these inferences from across the behavioral sciences. The resulting framework entails predictions about when specific inferences will arise and how these inferences influence the overall effects of behavioral interventions. Utilizing the framework, we provide a tool for enhancing intervention design. Taken together, the current research offers an integrative framework for advancing theory on behavioral interventions while also outlining actionable insights for organizations hoping to promote behavior change at scale.

Underspecified Human Decision Experiments Considered Harmful
Decision-making with information displays is a key focus of research in areas like human-AI collaboration and data visualization. However, what constitutes a decision problem, and what is required for an experiment to conclude that decisions are flawed, remain imprecise. We present a widely applicable definition of a decision problem synthesized from statistical decision theory and information economics. We claim that to attribute loss in human performance to bias, an experiment must provide the information that a rational agent would need to identify the normative decision. We evaluate whether recent empirical research on AI-assisted decisions achieves this standard. We find that only 10 (26%) of 39 studies that claim to identify biased behavior presented participants with sufficient information to make this claim in at least one treatment condition. We motivate the value of studying well-defined decision problems by describing a characterization of performance losses they allow to be conceived.

Unwillingness to pay for privacy: A field experiment
We measure willingness to pay for privacy in a field experiment. Participants bought at most one DVD from one of two competing online stores. One store consistently required more sensitive personal data than the other, but otherwise the stores were identical. In one treatment, DVDs were one Euro cheaper at the store requesting more personal information, and almost all buyers chose the cheaper store. Surprisingly, in the second treatment when prices were identical, participants bought from both shops equally often.
Rational Inattention: A Review
We review the recent literature on rational inattention, identify the main theoretical mechanisms, and explain how it helps us understand a variety of phenomena across fields of economics. The theory of rational inattention assumes that agents cannot process all available information, but they can choose which exact pieces of information to attend to. Several important results in economics have been built around imperfect information. Nowadays, many more forms of information than ever before are available due to new technologies, and yet we are able to digest little of it. Which form of imperfect information we possess and act upon is thus largely determined by which information we choose to pay attention to. These choices are driven by current economic conditions and imply behavior that features numerous empirically supported departures from standard models. Combining these insights about human limitations with the optimizing approach of neoclassical economics yields a new, generally applicable model.
Information Disclosure, Cognitive Biases, and Payday Borrowing
ABSTRACT Can psychology‐guided information disclosure induce borrowers to lower their use of high‐cost debt? In a field experiment at payday stores, we find that information that makes people think less narrowly (over time) about finance costs results in less borrowing. In particular, reinforcing the adding‐up dollar fees incurred when rolling over loans reduces the take‐up of future payday loans by 11% in the subsequent 4 months. Although we remain agnostic as to the overall sufficiency of better disclosure policy to “remedy” payday borrowing, we cast the 11% reduction in borrowing in light of the relative low cost of this policy.

How Field Experiments in Economics Can Complement Psychological Research on Judgment Biases
This review summarizes results of field experiments examining individual behaviors across several market settings—from open-air markets to rideshare markets to tax-compliance markets—where people sort themselves into market roles wherein they make consequential decisions. Using three distinct examples from my own research on the endowment effect, left-digit bias, and omission bias, I showcase how field experiments can help researchers understand mediators, heterogeneity, and causal moderation involved in judgment biases in the field. In this manner, the review highlights that economic field experiments can serve an invaluable intellectual role alongside traditional laboratory research.

Disclosure: Psychology Changes Everything
We review literature examining the effects of laws and regulations that require public disclosure of information. These requirements are most sensibly imposed in situations characterized by misaligned incentives and asymmetric information between, for example, a buyer and seller or an advisor and advisee. We review the economic literature relevant to such disclosure and then discuss how different psychological factors complicate, and in some cases radically change, the economic predictions. For example, limited attention, motivated attention, and biased assessments of probability on the part of information recipients can significantly diminish, or even reverse, the intended effects of disclosure requirements. In many cases, disclosure does not much affect the recipients of the information but does significantly affect the behavior of the providers, sometimes for the better and sometimes for the worse. We review research suggesting that simplified disclosure, standardized disclosure, vivid disclosure, and social comparison information can all be used to enhance the effectiveness of disclosure policies.

The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework
We review 74 experiments with no, low, or high performance-based financial incentives. The modal result has no effect on mean performance (though variance is usually reduced by higher payment). Higher incentive does improve performance often, typically judgment tasks that are responsive to better effort. Incentives also reduce “presentation” effects (e.g., generosity and risk-seeking). Incentive effects are comparable to effects of other variables, particularly “cognitive capital” and task “production” demands, and interact with those variables, so a narrow-minded focus on incentives alone is misguided. We also note that no replicated study has made rationality violations disappear purely by raising incentives.
Can Consumers Make Affordable Care Affordable? The Value of Choice Architecture
Tens of millions of people are currently choosing health coverage on a state or federal health insurance exchange as part of the Patient Protection and Affordable Care Act. We examine how well people make these choices, how well they think they do, and what can be done to improve these choices. We conducted 6 experiments asking people to choose the most cost-effective policy using websites modeled on current exchanges. Our results suggest there is significant room for improvement. Without interventions, respondents perform at near chance levels and show a significant bias, overweighting out-of-pocket expenses and deductibles. Financial incentives do not improve performance, and decision-makers do not realize that they are performing poorly. However, performance can be improved quite markedly by providing calculation aids, and by choosing a “smart” default. Implementing these psychologically based principles could save purchasers of policies and taxpayers approximately 10 billion dollars every year.
Are Information Disclosures Effective? Evidence from the Credit Card Market
Consumer protection in financial markets in the form of information disclosure is high on government agendas, even though there is little evidence of its effectiveness. We implement a randomized control trial in the credit card market for a large population of indebted cardholders and measure the impact of Truth-in-Lending-Act-type disclosures, de-biasing warning messages and social comparison information on default, indebtedness, account closings, and credit scores. We conduct extensive external validity exercises in several banks, with different disclosures, and with actual policy mandates. We find that providing salient interest rate disclosures had no effects, while comparisons and de-biasing messages had only modest effects at best.
Persuasion: Empirical Evidence
We provide a selective survey of empirical evidence on the effects as well as the drivers of persuasive communication. We consider persuasion directed at consumers, voters, donors, and investors. We organize our review around four questions. First, to what extent does persuasion affect the behavior of each of these groups? Second, what models best capture the response to persuasive communication? Third, what are persuaders' incentives, and what limits their ability to distort communications? Finally, what evidence exists on the way persuasion affects equilibrium outcomes in economics and politics?

Institutions and Behavior: Experimental Evidence on the Effects of Democracy
A novel experiment is used to show that the effect of a policy on the level of cooperation is greater when it is chosen democratically by the subjects than when it is exogenously imposed. In contrast to the previous literature, our experimental design allows us to control for selection effects (e.g., those who choose the policy may be affected differently by it). Our finding implies that democratic institutions may affect behavior directly in addition to having effects through the choice of policies. Our findings have implications for the generalizability of the results of randomized policy interventions. (JEL C91, D02, D12, D72)
How to Run Surveys: A Guide to Creating Your Own Identifying Variation and Revealing the Invisible
Surveys are an essential approach for eliciting otherwise invisible factors such as perceptions, knowledge and beliefs, attitudes, and reasoning. These factors are critical determinants of social, economic, and political outcomes. Surveys are not merely a research tool. They are also not only a way of collecting data. Instead, they involve creating the process that will generate the data. This allows the researcher to create their own identifying and controlled variation. Thanks to the rise of mobile technologies and platforms, surveys offer valuable opportunities either to study broadly representative samples or to focus on specific groups. This article offers guidance on the complete survey process, from the design of the questions and experiments to the recruitment of respondents and the collection of data to the analysis of survey responses. It covers issues related to the sampling process, selection and attrition, attention and carelessness, survey question design and measurement, response biases, and survey experiments.

Integrative experiments identify how punishment affects welfare in public goods games
Despite decades of research, the conditions under which punishment promotes cooperation remain unclear. Through an integrative experiment varying 14 design parameters of public goods games across 360 experimental conditions (147,618 decisions from 7100 participants), we reveal substantial heterogeneity in punishment effectiveness: Its impact on welfare ranges from 43% improvement to 44% reduction depending on the game parameters. To characterize these patterns, we developed models that outperformed human forecasters in predicting punishment effectiveness in new experiments. Communication emerges as the most important factor, followed by contribution framing (opt out versus opt in), contribution type (variable versus all-or-nothing), game length, and outcome visibility, though these factors often interact. The results reframe the debate from whether punishment works to when it does, demonstrating how integrative experiments enable discovery of generalizable patterns in social phenomena. , Editor’s summary People face conflicts between maximizing personal gain versus supporting collective interests. If we cooperatively recycle or donate to charities, it benefits society, but it also costs us time and resources that could be selfishly preserved for ourselves. We impose penalties to deter those undesirable or selfish behaviors, but under what conditions do punishments or penalties effectively modify behavior to benefit group welfare? Alsobay et al . systematically and simultaneously varied 14 factors together instead of in isolation. Punishment was unequivocally most effective when paired with consistent communication, particularly over time. Another effective factor was “opting out” or withdrawing some, but not all, endowments already in the public fund. These methodological advances revealed when, rather than whether, punishment works. —Ekeoma Uzogara , INTRODUCTION Human societies face many situations where individual and collective interests conflict, often referred to as social dilemmas. Costly peer punishment has been studied for more than 25 years in public goods games (stylized behavioral experiments in which individuals decide how much to contribute to a shared pool that benefits everyone) as a mechanism to promote cooperation. Prior research has identified many contextual factors that moderate punishment’s effectiveness, including game length, communication, group size, punishment cost, and so on. However, the specific conditions under which punishment improves group welfare remain unclear. RATIONALE We argue that this lack of clarity derives from the dominant experimental paradigm, in which any given study manipulates only one or a few theoretically informed factors. Because such studies differ in many ways (different experimental procedures, populations), their results are often difficult to compare or integrate. Consequently, one can list many factors that have some effect, but cannot say how much each matters relative to the others, or how they work together, and as a result, cannot predict when punishment will help or harm welfare in new settings. To address this fundamental knowledge gap, we use an integrative experimental design and systematically vary 14 parameters across 360 conditions (147,618 decisions from 7100 participants) to elucidate when punishment improves versus undermines welfare in public goods games, which factors matter most, and how they interact. RESULTS The effect of punishment on welfare ranged from 43% improvement to 44% reduction depending on the specific combination of game parameters. To characterize this heterogeneity, we trained a model that outperformed all 553 human forecasters (laypeople and experts) in predicting whether punishment would help or harm welfare in new experiments. Communication emerged as roughly three times more important than any other factor, followed by contribution framing (opt in versus opt out), contribution type (variable versus all-or-nothing), game length, and peer outcome visibility (whether participants can see others’ earnings). These factors often interact. For example, longer games enhance punishment’s effectiveness only when communication is available, and contribution framing effects depend on both contribution type and outcome visibility. CONCLUSION Many phenomena in social science are shaped by many factors whose interactions are consequential, yet the dominant experimental paradigm often limits its inquiry to “does a given effect exist?” and examines hypothesized factors in isolation. As a result, research programs can accumulate many partial explanations without a clear picture of how they combine to determine outcomes across settings. Knowing that factors matter individually is fundamentally different from knowing how much each matters and how they interact. The integrative approach implemented here offers one way forward. It varies many factors simultaneously within a shared design space, evaluates models by their predictive accuracy on new experiments, and probes those models to constrain and develop theory. Our hope is that integrative experiment designs, combined with models that integrate prediction and explanation, represent a path toward more cumulative social science. Integrative experiment reveals when punishment helps versus harms. We systematically varied 14 design parameters across 360 experimental conditions. The effect of punishment on cooperation efficiency ranged from −44% to +43% depending on the specific game parameters. Communication emerged as three times more important than any other factor, followed by contribution framing, contribution type, and game length.

Against theory-motivated experimentation: Can random experimental choice lead to better theories?
Scientists must choose which among many experiments to perform. We study the epistemic success of experimental choice strategies proposed by philosophers of science or executed by scientists themselves. We develop a multi-agent model of the scientific process that jointly formalizes its core aspects: active experimentation, theorizing, and social learning. We find that agents who choose new experiments at random develop the most informative and predictive theories of the world. The agents aiming to confirm, falsify theories, or resolve theoretical disagreements end up with an illusion of epistemic success: they develop promising accounts for the data they collected, while misrepresenting the ground truth that they intended to learn about. Agents experimenting in these theory-motivated ways acquire less diverse or less representative samples from the ground truth that also turn out to be easier to account for. Random data collection, on the other hand, combines virtues of diverse and representative sampling from a target scientific domain which enables cumulative development of the successful theoretical accounts of it. We suggest that randomization, already a gold standard within experiments, is also beneficial at the level of experiments themselves.
