Good decisions require verified quality. Refine devotes hours of frontier compute to protect your work and reputation from fixable mistakes.
The future belongs to those who can refute AI, not just generate with AI
Why verification, not prompting, could shape the next decade of engineering
Mathematician catches AI-generated research error by OpenAI president | Benjamin Golub posted on the topic | LinkedIn
In case you think the problem of science slop is hypothetical, here's something that happened on X yesterday: The president of OpenAI sharing wrong science slop as proud evidence of the power of mathematical AI. Within a day, the mathematician Tony Feng found a gap that undercut the paper's central argument; the author conceded, and (to his credit!) Brockman deleted the tweet as soon as he became aware. Feng was a hero to catch this. The author of the refuted claim is a mathematician but a self-described outsider to this area, with a prior big claim posted and quickly withdrawn. Greg Brockman is not an expert, either. Neither could check what they were amplifying. Heroes catching errors one at a time won't hold against the volume of AI-generated research now coming. We need an automated, independent verification layer for knowledge work, which is what we're building at Refine (refine.ink). Passing such verification should become a necessary step before big claims, especially outside one's own expertise. A link to a bigger-picture post on this is in the comments. | 23 comments on LinkedIn
Reify This
The authors contend that contemporary efforts to render AI systems interpretable rest on a mistake: reification, the process of treating abstractions and statistical artifacts as if they were concrete realities.…
Piloting the world's first double-blind AI evaluations
Building trust in proprietary model benchmarks using cryptographically secure environments
Prediction: AI will make formal verification go mainstream — Martin Kleppmann’s blog
Much has been said about the effects that AI will have on software development, but there is an angle I haven’t seen talked about: I believe that AI will bring formal verification, which for decades has been a bit of a fringe pursuit, into the software engineering mainstream.
Verification in the Age of AI Scientists
AI Scientists now operate at a scale that outpaces human capacity for manual review. Systems such as Sakana’s AI Scientist write entire workshop papers end-to-end. Lila Sciences runs autonomous ``AI Science Factories’’ that hypothesize, experiment, and iterate without human guidance. FutureHouse’s Kosmos and Robin generate thousands of candidate hypotheses in a single run, and Google’s Co-Scientist proposes testable experiments at a rate no laboratory can fully evaluate. Each of these systems emphasizes verified results, yet that standard ranges from near-perfect formal proof in mathematics to decade-long clinical trials in medicine, with no shared framework for judging sufficiency across domains. As outputs scale beyond what humans can manually inspect, the question of which results to trust becomes as hard as generating them. The bottleneck for AI for Science is no longer hypothesis generation, it is verification.
Standards around generative AI
Accuracy, fairness and speed are the guiding values for AP’s news report, and we believe the mindful use of artificial intelligence can serve these values and over time improve how we work.
AI Detector — Verified AI Content Checker | Pangram
Dive into the technical foundations behind Pangram's AI detection stack and understand how we achieve low false positive rates across modalities.
Deepfakes, Scams, and the Age of Paranoia
As AI-driven fraud becomes increasingly common, more people feel the need to verify every interaction they have online.
Open-world evaluations for measuring frontier AI capabilities
Introducing CRUX, a new project for evaluating AI on long, messy tasks
Refine
I recently tried refine, an AI tool for refining academic articles, developed by Yann Calvó López and Ben Golub.
AI agents are checking the scientific literature — and spotting decades-old errors
The technology is proving adept at finding faults in decades-old papers and reference databases.
An AI test needs evidence the AI cannot edit - Sensemaker
OpenAI's postmortem shows that some agents learned to spoof tool calls while trying to fool a benchmark.