Illusions of Understanding in the Sciences

Scientists seek to understand the causes of observed phenomena. Beliefs that they have succeeded are based on understanding that is rarely or possibly never complete, and varies in depth and quality. Most often scientists believe they understand more than they do, making their belief an illusion. This illusion then persists in explanations scientists provide in print, in talks, or in discussions. The illusion that a scientist has a valid and complete explanation tends to be magnified when the data are well described by mathematical and computer simulation models due to the precision of such models and their ability to predict well; prediction does not imply causality, but gives the illusion that it does. The first part of this essay supports the case for the universality of partial and incomplete levels of understanding by showing the difficulty of reaching a deep level of understanding for even a simple analysis and model that most scientists use and believe they understand: linear regression. The second part highlights some implications of the existence of many levels of understanding and explanation, and their use by scientists for design, testing, analysis, and theory development. It discusses the way that deduction and induction depend on the levels of understanding and the implications of the illusion that a scientist’s understanding is deep. It makes a case that the many incomplete levels of understanding affect, often unwittingly, the ways scientists design experiments, test theories, comprehend, communicate, and teach.

The End of Mathematics — Daniel Litt

I'm currently returning to Toronto from a summit on the future of mathematics, at OpenAI. Sebastian Bubeck asked me to talk a bit about the future we'd all like to avoid, where humans are mathematically disempowered. Jacob Tsimerman advised us to try to prioritize detail over correctness, and I have no doubt that I succeeded in deprioritizing correctness.

Mathematical Thinking Isn’t What You Think It Is | Quanta Magazine

The mathematician David Bessis claims that everyone is capable of, and can benefit greatly from, mathematical thinking.

Pluralistic: Goodhart’s Law (of AI) (11 Aug 2025) – Pluralistic: Daily links from Cory Doctorow

One way to think about AI's unwelcome intrusion into our lives can be summed up with Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure":

Mathematical Beauty, Truth and Proof in the Age of AI | Quanta Magazine

Mathematicians have started to prepare for a profound shift in what it means to do mathematics.

Qualitative research: standards, challenges, and guidelines

Qualitative research methods could help us to improve our understanding of medicine. Rather than thinking of qualitative and quantitative strategies as incompatible, they should be seen as complementary. Although procedures for textual interpretation differ from those of statistical analysis, because of the different type of data used and questions to be answered, the underlying principles are much the same. In this article I propose relevance, validity, and reflexivity as overall standards for qualitative inquiry.

CausalSmith · AI Causal Scientist

Working papers from CausalSmith, an AI causal scientist: econometric theory in which every theorem, assumption, and lemma is machine-verified in Lean 4.

Putting Paradoxes to Work: Contextuality in Measurement-Based Quantum Computation

We describe a joint cohomological framework for measurement-based quantum computation (MBQC) and the corresponding contextuality proofs. The central object in this framework is an element $$[\beta _\Psi ]$$[βΨ]in the second cohomology group of the chain complex describing a given MBQC. $$[\beta _\Psi ]$$[βΨ]contains the function computed therein up to gauge equivalence, and at the same time is a contextuality witness. The present cohomological description only applies to temporally flat MBQCs, and we outline an approach for extending it to the temporally ordered case.

Navier-Stokes and Lean

I was working on this week's post on Lean after reading Kevin Hartnett's book  The Proof in the Code: How a Truth Machine Is Transforming Ma...

Sample Size in Qualitative Interview Studies: Guided by Information Power

Sample sizes must be ascertained in qualitative studies like in quantitative studies but not by the same means. The prevailing concept for sample size in qualitative studies is “saturation.” Saturation is closely tied to a specific methodology, and the term is inconsistently applied. We propose the concept “information power” to guide adequate sample size for qualitative studies. Information power indicates that the more information the sample holds, relevant for the actual study, the lower amount of participants is needed. We suggest that the size of a sample with sufficient information power depends on (a) the aim of the study, (b) sample specificity, (c) use of established theory, (d) quality of dialogue, and (e) analysis strategy. We present a model where these elements of information and their relevant dimensions are related to information power. Application of this model in the planning and during data collection of a qualitative study is discussed.

‘Shut up and calculate’ does a disservice to quantum mechanics | Aeon Essays

The cliché has it that the Copenhagen interpretation demands adherence without deep enquiry. That does physics a disservice

Operadic consistency: a label-free signal for compositional...

Detecting LLM reasoning failures at inference time without ground-truth labels has motivated a wide range of confidence baselines, including self-consistency, semantic entropy, and P(True), built...

Back-to-basics: unobservable constructs and their ‘surplus meaning’ - (Un)rigorous AI

TL;DR — Because unobservable constructs hold ‘surplus meaning,’ your metric is not your construct.

The Kochen-Specker Theorem

The Kochen-Specker theorem is an important and subtle topic in the foundations of quantum mechanics (QM). The theorem demonstrates the impossibility of a certain type of interpretation of QM in terms of hidden variables (HV) that naturally suggests itself when one begins to consider the project of interpretating QM.We here present the theorem/argument and the foundational discussion surrounding it at different levels. The reader looking for a quick overview should read the following sections and subsections: 1, 2, 3.1, 3.2, 4, and 6. Those who read the whole entry will find proofs of some non-trivial claims in supplementary documents.

Should All Graphs Start at 0? – MeasuringU

And if people think you can show anything you want with statistics, then this cynicism certainly applies to statistics graphs, too.

Reversing Assumptions Technique — Think Jar Collective

Think Jar Collective contributor and creativity expert Michael Michalko shares a technique to challenge our own assumptions and in the process spark new thinking.