The misunderstood limits of folk science: an illusion of explanatory depth

Illusions of Understanding in the Sciences

Scientists seek to understand the causes of observed phenomena. Beliefs that they have succeeded are based on understanding that is rarely or possibly never complete, and varies in depth and quality. Most often scientists believe they understand more than they do, making their belief an illusion. This illusion then persists in explanations scientists provide in print, in talks, or in discussions. The illusion that a scientist has a valid and complete explanation tends to be magnified when the data are well described by mathematical and computer simulation models due to the precision of such models and their ability to predict well; prediction does not imply causality, but gives the illusion that it does. The first part of this essay supports the case for the universality of partial and incomplete levels of understanding by showing the difficulty of reaching a deep level of understanding for even a simple analysis and model that most scientists use and believe they understand: linear regression. The second part highlights some implications of the existence of many levels of understanding and explanation, and their use by scientists for design, testing, analysis, and theory development. It discusses the way that deduction and induction depend on the levels of understanding and the implications of the illusion that a scientist’s understanding is deep. It makes a case that the many incomplete levels of understanding affect, often unwittingly, the ways scientists design experiments, test theories, comprehend, communicate, and teach.

Social learning preserves both useful and useless theories by canalizing learners’ exploration

In many domains, learning from others is crucial for leveraging cumulative cultural knowledge, which encapsulates the efforts of successive generations of innovators. However, anecdotal and experimental evidence suggests that reliance on social information can reduce the exploration of the problem space. Here, we experimentally investigate the extent to which cultural transmission fosters the persistence of arbitrary solutions in a context where participants are incentivized to improve a physical system across multiple trials. Participants were exposed to various theories about the system, ranging from accurate to misleading. Our findings indicate that even under conditions conducive to exploration, the transmission of cultural knowledge canalizes learners’ focus, limiting their consideration of alternative solutions. This effect was observed in both the theories produced and the solutions attempted by participants, irrespective of the accuracy of the provided theories. These results challenge the notion that arbitrary solutions persist only when they are efficient or intuitive and underscore the significant role of cultural transmission in shaping human knowledge and technologies.

Guessing reveals internal models of perceptual precision

When observers lack sufficient information to support a confident response, they often guess. Guessing plays a pervasive role in visual cognition and working memory, yet the mechanisms that govern how observers generate guesses remain poorly understood. Standard models traditionally assume that responses produced in the absence of information are either uniformly distributed over feature space or are perhaps weighted towards prevailing environmental statistics. In contrast, here we consider an intriguing alternative: that guesses incorporate observers’ knowledge of their own perceptual capacities. We empirically measured guessing by eliciting responses under extreme target uncertainty (Experiment 1) as well as a novel “0ms presentation” approach in which no stimulus appeared but subjects believed one had (Experiment 2). We evaluated three accounts of guesses under these conditions: unsystematic (lapse) responding, biases toward environmental statistics, and a self-representational account in which guesses reflect observers’ knowledge of their own feature-dependent precision (e.g., preferring to guess feature values they believe they would be likely to miss). Guess responses were non-uniform and systematically biased toward feature values typically encoded with the least precision (e.g., oblique orientations) — a counterintuitive bias away from high-frequency, high-fidelity feature values (e.g., cardinal orientations). This complementary relationship between guessing and perceptual fidelity held within individuals and across paradigms, and was recoverable via an empirical-guess mixture model that replaced the standard uniform assumption with empirically measured guess distributions. Our findings challenge prevailing views that guesses reflect random noise, and suggest instead that guessing behavior reflects metacognitive knowledge of internal precision. Rather than defaulting to environmental priors, observers appear to model their own sensory limitations and leverage these representations to inform decisions in the absence of evidence. These results reframe guessing as a theoretically informative behavior that expresses observers’ own beliefs about their perceptual capacities. Significance Guessing is commonly treated as random noise in models of perception and memory, assumed to reflect lapses or uninformed responses. Instead, we show that human guesses are systematically structured across feature space: observers preferentially guess values they typically encode with the least precision, revealing a consistent, strategic bias away from high-fidelity representations. By directly measuring guess behavior on stimulus-absent trials and integrating these empirical distributions into a mixture model, we find that guesses on stimulus-present trials can be systematically recovered, and that they too form the complement of perceptual precision. These findings challenge foundational psychophysical modeling assumptions and position guessing as a strategic, informative behavior that engages self-representation.

Believing what we do not believe: Acquiescence to superstitious beliefs and other powerful intuitions.

Food, foragers, and folklore: the role of narrative in human subsistence

Narrative is a species-typical, reliably developing, complex cognitive process whose design is unlikely to have emerged by chance. Moreover, the folklore record indicates that narrative content is consistent across widely divergent cultures. I have argued elsewhere that a storyteller may use narrative to manipulate an audience's representations of the social and/or physical environment to serve his or her own fitness ends. However, my subsequent research suggests that such manipulation results from a broader selection pressure which narrative effectively alleviates: information acquisition. By substituting verbal representations for potentially costly first-hand experience, narrative enables an individual to safely and efficiently acquire information pertinent to the pursuit of fitness in local habitats. If this hypothesis is true, narrative should be rich with information useful to the pursuit of fitness. One class of information integral to the accomplishment of this task is foraging knowledge. In this paper, then, I present evidence that foraging peoples use narrative to transmit subsistence information: specifically, I demonstrate how various narrative devices (e.g., setting, description, mimicry, anthropomorphism) are used to communicate foraging knowledge.

Sciences perceived as precise and consensual are more trusted

Research focused on the United States shows that people’s trust in science varies considerably between disciplines. Existing explanations of these trust gaps stress the role of ideology: when people perceive scientists of a particular discipline to be ideologically like-minded, they tend to trust them more. Here, we report two findings: first, trust gaps between disciplines also exist in France—a representative sample of the French population (N = 1012) trusted researchers in biology and physics more than researchers studying climate science, economics, or sociology. Second, the more precise and consensual participants perceive scientific findings to be, the more they tend to trust the scientists (across and within disciplines). While these findings are correlational, they align with a non-ideological explanation of trust in science: the rational impression account. This account proposes that people can come to trust scientists by relying on basic cognitive inference processes, which tend to be generally rational.

The Overhang

Using your deep knowledge, wide knowledge, taste, and agency

The Overhang

Using your deep knowledge, wide knowledge, taste, and agency

Metacognitive Myopia: A Major Obstacle on the Way to Rationality

The notion of metacognitive myopia refers to a conspicuous weakness of the quality control of memory and reasoning processes. Although people are often remarkably sensitive even to complex samples of information when making evaluative judgments and decisions, their uncritical and naive tendency to take the validity of sampled information for granted constitutes a major obstacle to rational behavior. After illustrating this phenomenon with reference to prominent biases (base-rate neglect, misattribution, perseverance), we decompose metacognitive myopia into two distinct but intertwined functions, monitoring and control. We offer explanations for why effectively monitoring the biases resulting from information sampling in an uncertain world is so difficult and why the control function is severely restricted by the lack of volitional control over mental actions. Because of these and other difficulties, metacognitive myopia constitutes a major obstacle to rational judgment and decision making.

How To Think Real Good | Meta-rationality

Practical epistemology: heuristics for how to think about difficult things, particularly using formal methods.

Debunking: A Meta-Analysis of the Psychological Efficacy of Messages Countering Misinformation

This meta-analysis investigated the factors underlying effective messages to counter attitudes and beliefs based on misinformation. Because misinformation can lead to poor decisions about consequential matters and is persistent and difficult to correct, debunking it is an important scientific and public-policy goal. This meta-analysis ( k = 52, N = 6,878) revealed large effects for presenting misinformation ( ds = 2.41–3.08), debunking ( ds = 1.14–1.33), and the persistence of misinformation in the face of debunking ( ds = 0.75–1.06). Persistence was stronger and the debunking effect was weaker when audiences generated reasons in support of the initial misinformation. A detailed debunking message correlated positively with the debunking effect. Surprisingly, however, a detailed debunking message also correlated positively with the misinformation-persistence effect.

Social-information seeking in development: The child as experimental psychologist

Research has established that children are “naive psychologists”, adept at understanding and navigating the social world from an early age. However, most of this work has focused on how children process information that they acquire incidentally, for example by passively observing others’ actions. Here, we draw on literature framing children as intuitive scientists, who actively seek information and test hypotheses, to propose a view of children as naive experimental psychologists. From this perspective, children play an active role in selecting and pursuing relevant social information (e.g., about agents’ goals, traits, or relationships), whereby their search strategies are influenced both by context and task demands, as well as their prior beliefs, concepts, and domain-specific naive theories. We argue that the particular challenges associated with learning and reasoning about other minds may necessitate that children leverage their active learning competences, and we outline how the social domain uniquely constrains and shapes the learning process. We review existing research on social-information seeking in children and adults, and identify directions for future research, emphasizing that children’s developing social cognition should be understood in terms of the active, exploratory role they take in learning about and participating in the social world.

The Illusion of Clarity • Ness Labs

The illusion of clarity is the confident feeling that you understand something, when in reality your grasp is full of gaps you’ve never noticed.

A Rational Analysis of the Effects of Sycophantic AI

People increasingly use large language models (LLMs) to explore ideas, gather information, and make sense of the world. In these interactions, they encounter agents that are overly agreeable. We argue that this sycophancy poses a unique epistemic risk to how individuals come to see the world: unlike hallucinations that introduce falsehoods, sycophancy distorts reality by returning responses that are biased to reinforce existing beliefs. We provide a rational analysis of this phenomenon, showing that when a Bayesian agent is provided with data that are sampled based on a current hypothesis the agent becomes increasingly confident about that hypothesis but does not make any progress towards the truth. We test this prediction using a modified Wason 2-4-6 rule discovery task where participants (N=557) interacted with AI agents providing different types of feedback. Unmodified LLM behavior suppressed discovery and inflated confidence comparably to explicitly sycophantic prompting. By contrast, unbiased sampling from the true distribution yielded discovery rates five times higher. These results reveal how sycophantic AI distorts belief, manufacturing certainty where there should be doubt.

Social-Information Seeking in Development: The Child as Experimental Psychologist

Research has established that children are “naive psychologists,” adept at understanding and navigating the social world from an early age. However, most of this work has focused on how children process information that they acquire incidentally, for example, by passively observing others’ actions. Here, we draw on literature framing children as intuitive scientists who actively seek information and test hypotheses to propose a view of children as naive experimental psychologists. From this perspective, children play an active role in selecting and pursuing relevant social information (e.g., about agents’ goals, traits, or relationships), whereby their search strategies are influenced both by context and task demands, as well as their prior beliefs, concepts, and domain-specific naive theories. We argue that the particular challenges associated with learning and reasoning about other minds may necessitate that children leverage their active learning competences, and we outline how the social domain uniquely constrains and shapes the learning process. We review existing research on social-information seeking in children and adults and identify directions for future research, emphasizing that children’s developing social cognition should be understood in terms of the active, exploratory role they take in learning about and participating in the social world.

Searching for explanations: How the Internet inflates estimates of internal knowledge.