Eliciting Beliefs with Random Generation Tasks
Elicitation methods, such as asking people to produce the deciles of a distribution, are standard practices in policy or applied statistics. Similarly, much of cognitive science and psychology focuses on determining people's people's beliefs or latent traits through questionnaires or judgment tasks. However, these approaches often only capture a rough outline of what people know and are usually limited to point estimates of people's beliefs. Here, we present a novel experimental paradigm that allows us to access people's beliefs and how variable these beliefs are. Our task is based on an established random generation paradigm in which participants produce quantities from a particular domain as randomly as possible. We hypothesize that due to the minds' general-purpose mechanisms for probabilistic inferences, these random sequences represent the participants' underlying prior beliefs. We show that our method can infer participants' beliefs for a wide range of numeric quantities at comparable accuracy as an established elicitation method. Moreover, these inferred beliefs are consistent with individual participants' generalization and inference patterns in a subsequent conditional prediction task. We then extend our approach to non-numeric belief elicitation, highlighting how our method can go beyond numeric elicitation and provide insight into complex beliefs that are challenging to assess experimentally. Empirically, our results highlight that people know the rough shapes of environmental distributions, and these beliefs guide inference and generalization. Moreover, using our novel approach, we also show that people know the fine details of environmental distributions. Finally, our experimental results show that random generation paradigms can be a useful tool for cognitive scientists, psychologists, and applied statisticians.
AI existential risk probabilities are (still) too unreliable to inform policy
How speculation gets laundered through pseudo-quantification
AI existential risk probabilities are too unreliable to inform policy
How speculation gets laundered through pseudo-quantification
Unreasonable Effectiveness of Mathematics - E. Wigner
Reprinted from Communications in Pure and Applied Mathematics, Vol. 13, No. I (February 1960). New York: John Wiley & Sons, Inc. Copyright © 1960 by John Wiley & Sons, Inc.
Causality
Written by one of the preeminent researchers in the field, this book provides a comprehensive exposition of modern analysis of causation. It shows how causality has grown from a nebulous concept into a mathematical theory with significant applications in the fields of statistics, artificial intelligence, economics, philosophy, cognitive science, and the health and social sciences. Judea Pearl presents and unifies the probabilistic, manipulative, counterfactual, and structural approaches to causation and devises simple mathematical tools for studying the relationships between causal connections and statistical associations. Cited in more than 2,100 scientific publications, it continues to liberate scientists from the traditional molds of statistical thinking. In this revised edition, Judea Pearl elucidates thorny issues, answers readers' questions, and offers a panoramic view of recent advances in this field of research. Causality will be of interest to students and professionals in a wide variety of fields. Dr Judea Pearl has received the 2011 Rumelhart Prize for his leading research in Artificial Intelligence (AI) and systems from The Cognitive Science Society.
Resource-rational belief revision can mitigate as well as amplify polarization
People's beliefs sometimes diverge after observing the same information, which has been interpreted as evidence of irrationality. This behaviour has been proposed to result from people's limited cognitive resources and motivated reasoning, but how belief revision differs across these explanations has not been formalized or compared to a rational norm. Further, while people may be biased relative to a normative ideal, they may still make optimal choices given their limited cognitive resources, or rationally balance the utility of holding accurate beliefs with the belief's intrinsic utility. Across two studies, we develop and test a unified computational account of belief polarization under these proposed mechanisms, showing that people's performance on a belief updating task best fits a limited-resource Bayesian model; external motivations may contribute to divergence (or convergence) by determining what pre-existing information people consider relevant to a situation, rather than by changing how people evaluate new information in isolation.
QBism: Quantum Theory as a Hero's Handbook
This paper represents an elaboration of the lectures delivered by one of us (CAF) during "Course 197 -- Foundations of Quantum Physics" at the International School of Physics "Enrico Fermi" in Varenna, Italy, July 2016. Much of the material for it is drawn from arXiv:1003.5209, arXiv:1401.7254, and arXiv:1405.2390. However there are substantial additions of original material in Sections 4, 7, 8 and 9, along with clarifications and expansions of the older content throughout. Topics include the meaning of subjective probability; no-cloning, teleportation, and quantum tomography from the subjectivist Bayesian perspective; the message QBism receives from Bell inequality violations (namely, that nature is creative); the import of symmetric informationally complete (SIC) quantum measurements for the technical side of QBism; quantum cosmology QBist-style; and a potential meaning for the holographic principle within QBism.
A Rational Analysis of the Effects of Sycophantic AI
People increasingly use large language models (LLMs) to explore ideas, gather information, and make sense of the world. In these interactions, they encounter agents that are overly agreeable. We argue that this sycophancy poses a unique epistemic risk to how individuals come to see the world: unlike hallucinations that introduce falsehoods, sycophancy distorts reality by returning responses that are biased to reinforce existing beliefs. We provide a rational analysis of this phenomenon, showing that when a Bayesian agent is provided with data that are sampled based on a current hypothesis the agent becomes increasingly confident about that hypothesis but does not make any progress towards the truth. We test this prediction using a modified Wason 2-4-6 rule discovery task where participants (N=557) interacted with AI agents providing different types of feedback. Unmodified LLM behavior suppressed discovery and inflated confidence comparably to explicitly sycophantic prompting. By contrast, unbiased sampling from the true distribution yielded discovery rates five times higher. These results reveal how sycophantic AI distorts belief, manufacturing certainty where there should be doubt.
The Consistent Histories Approach to Quantum Mechanics
The consistent histories, also known as decoherent histories, approachto quantum interpretation is broadly compatible with standard quantummechanics as found in textbooks. However, the concept ofmeasurement by which probabilities are introduced in standardquantum theory no longer plays a fundamental role. Instead,all quantum time dependence is probabilistic (stochastic),with probabilities given by the Born rule or its extensions. Byrequiring that the description of a quantum system be carried outusing a well-defined probabilistic sample space (called a“framework”) this approach resolves many well-knownquantum paradoxes of quantum foundations. In particular, quantummechanics is local and consistent with special relativity. Classicalmechanics emerges as a useful approximation to the more fundamentalquantum mechanics under suitable conditions. The price to be paid forthis is a set of rules for reasoning resembling, but very much simplerthan, those of quantum logic. An important philosophical implicationis the absence of a single universally-true state of affairs at eachinstant of time. However, there is a correspondence limit in which thenew quantum logic becomes standard logic in the macroscopic world ofeveryday experience, and the laws of classical mechanics emerge as agood approximation to an underlying, and in principle more exact,quantum description.
AI Epistemic Risks: Emerging Mechanisms & Evidence
<p>Advances in artificial intelligence pose risks to humanity's collective capacity to form accurate beliefs, reason well, and maintain a healthy information en
Bayesian Thinking in Everyday Life
More than 200 years ago, Thomas Bayes came up with a brilliant idea that has helped shape the world today, called Bayes Theorem. This…
Against theory-motivated experimentation: Can random experimental choice lead to better theories?
Scientists must choose which among many experiments to perform. We study the epistemic success of experimental choice strategies proposed by philosophers of science or executed by scientists themselves. We develop a multi-agent model of the scientific process that jointly formalizes its core aspects: active experimentation, theorizing, and social learning. We find that agents who choose new experiments at random develop the most informative and predictive theories of the world. The agents aiming to confirm, falsify theories, or resolve theoretical disagreements end up with an illusion of epistemic success: they develop promising accounts for the data they collected, while misrepresenting the ground truth that they intended to learn about. Agents experimenting in these theory-motivated ways acquire less diverse or less representative samples from the ground truth that also turn out to be easier to account for. Random data collection, on the other hand, combines virtues of diverse and representative sampling from a target scientific domain which enables cumulative development of the successful theoretical accounts of it. We suggest that randomization, already a gold standard within experiments, is also beneficial at the level of experiments themselves.
Prospect Theory, Mental Accounting, and Differences in Aggregated and Segregated Evaluation of Lottery Portfolios
If individuals have to evaluate a sequence of lotteries, their judgment is influenced by the presentation mode. Experimental studies have found significantly higher acceptance rates for a sequence of lotteries if the overall distribution was displayed instead of the set of lotteries itself. Mental accounting and loss aversion provide an easy and intuitive explanation for this phenomenon. In this paper we offer an explanation that incorporates further evaluation concepts of Prospect Theory. Our formal analysis of the difference in aggregated and segregated portfolio evaluation demonstrates that the higher attractiveness of the aggregated presentation mode is not a general phenomenon (as suggested in the literature) but depends on specific parameters of the lotteries. The theoretical findings are supported by an experimental study. In contrast to the existing evidence and in line with our theoretical results, we find for specific types of lotteries an even lower acceptance rate if the overall distribution is displayed.
From Predictive Algorithms to Automatic Generation of Anomalies
Machine learning algorithms can find predictive signals that researchers fail to notice; yet they are notoriously hard-to-interpret. How can we extract theoretical insights from these black boxes? History provides a clue. Facing a similar problem – how to extract theoretical insights from their intuitions – researchers often turned to “anomalies:” constructed examples that highlight flaws in an existing theory and spur the development of new ones. Canonical examples include the Allais paradox and the Kahneman-Tversky choice experiments for expected utility theory. We suggest anomalies can extract theoretical insights from black box predictive algorithms. We develop procedures to automatically generate anomalies for an existing theory when given a predictive algorithm. We cast anomaly generation as an adversarial game between a theory and a falsifier, the solutions to which are anomalies: instances where the black box algorithm predicts - were we to collect data - we would likely observe violations of the theory. As an illustration, we generate anomalies for expected utility theory using a large, publicly available dataset on real lottery choices. Based on an estimated neural network that predicts lottery choices, our procedures recover known anomalies and discover new ones for expected utility theory. In incentivized experiments, subjects violate expected utility theory on these algorithmically generated anomalies; moreover, the violation rates are similar to observed rates for the Allais paradox and Common ratio effect.
