







Science needs more than answers
A Data Utopia for Science-of-Science
Here I want to briefly sketch out a vision for how to solve a key set of problems facing science-of-science researchers, using the relatively new idea of a ‘data trust.’ In my ideal wor…

Mathematician catches AI-generated research error by OpenAI president | Benjamin Golub posted on the topic | LinkedIn
In case you think the problem of science slop is hypothetical, here's something that happened on X yesterday: The president of OpenAI sharing wrong science slop as proud evidence of the power of mathematical AI. Within a day, the mathematician Tony Feng found a gap that undercut the paper's central argument; the author conceded, and (to his credit!) Brockman deleted the tweet as soon as he became aware. Feng was a hero to catch this. The author of the refuted claim is a mathematician but a self-described outsider to this area, with a prior big claim posted and quickly withdrawn. Greg Brockman is not an expert, either. Neither could check what they were amplifying. Heroes catching errors one at a time won't hold against the volume of AI-generated research now coming. We need an automated, independent verification layer for knowledge work, which is what we're building at Refine (refine.ink). Passing such verification should become a necessary step before big claims, especially outside one's own expertise. A link to a bigger-picture post on this is in the comments. | 23 comments on LinkedIn

Establishing trust in automated reasoning - MetaROR
Since its beginnings in the 1940s, automated reasoning by computers has become a tool of ever growing importance in scientific research. So far, the rules underlying automated reasoning have mainly been formulated by humans, in the form of program source code. Rules derived from large amounts of data, via machine learning techniques, are a complementary approach currently under intense development. The question of why we should trust these systems, and the results obtained with their help, has been discussed by early practitioners of computational science, but was later forgotten. The present work focuses on independent reviewing, an important source of trust in science, and identifies the characteristics of automated reasoning systems that affect their reviewability. It also discusses possible steps towards increasing reviewability and trustworthiness via a combination of technical and social measures.

Verification in the Age of AI Scientists
AI Scientists now operate at a scale that outpaces human capacity for manual review. Systems such as Sakana’s AI Scientist write entire workshop papers end-to-end. Lila Sciences runs autonomous ``AI Science Factories’’ that hypothesize, experiment, and iterate without human guidance. FutureHouse’s Kosmos and Robin generate thousands of candidate hypotheses in a single run, and Google’s Co-Scientist proposes testable experiments at a rate no laboratory can fully evaluate. Each of these systems emphasizes verified results, yet that standard ranges from near-perfect formal proof in mathematics to decade-long clinical trials in medicine, with no shared framework for judging sufficiency across domains. As outputs scale beyond what humans can manually inspect, the question of which results to trust becomes as hard as generating them. The bottleneck for AI for Science is no longer hypothesis generation, it is verification.

Why we don’t really know what the public thinks about science
Measuring trust isn’t enough. Furthering knowledge about the institutions and norms of science is the best way to build credibility.

Science Integrity Digest
A blog about science integrity, by Elisabeth Bik, for Harbers-Bik LLC. Support my work at Patreon.com/elisabethbik

Don't Just Fact-Check Misinformation. First, Understand The Value It Holds. A Conversation w Researchers Michael Simeone & Kristy Roschke
Podcast Episode · Why Should I Trust You? · June 25 · 1h 7m
Guest Post — When Errors Become Consensus: Science’s Self-Correction Can No Longer Keep Up - The Scholarly Kitchen
From 19th-century beef tea to paper mills and generative AI, the challenge is no longer detection alone; it is preventing bad claims from becoming embedded knowledge.

Half of social-science studies fail replication test in years-long project
Results from massive, ‘eagerly awaited’ initiative reinforce concerns about the credibility of science — but raise hope for solutions.

Verification · mu
The Trusted Verifier Program on mu: a democratic, plural approach to verification. Verification tells you an account is who they claim to be.
Verification · mu
The Trusted Verifier Program on mu: a democratic, plural approach to verification. Verification tells you an account is who they claim to be.
The Case Against Formal Verification, 50 Years Later - Ivan Gavran
Writings on software correctness, AI, formal verification, and other technical topics.

Age verification needs better privacy protections, report says
New America’s Open Technology Institute called for the use of zero-knowledge proofs, which verify a user’s age without disclosing any personal information.

Language agents achieve superhuman synthesis of scientific knowledge
High accuracy RAG for answering questions from scientific documents with citations