No. 022: Palomar proof registry, autonomous protein design, synthetic consensus - The Hallway Track

The Hallway Track No. 022 - How AI is affecting the practice of science

Palomar – a registry of Lean verified mathematics

In recent months there has been a proliferation of AI-generated proofs of various old and new results, some of which have been formalized in the proof assistant language Lean. However, checking tha…

Verification in the Age of AI Scientists

AI Scientists now operate at a scale that outpaces human capacity for manual review. Systems such as Sakana’s AI Scientist write entire workshop papers end-to-end. Lila Sciences runs autonomous ``AI Science Factories’’ that hypothesize, experiment, and iterate without human guidance. FutureHouse’s Kosmos and Robin generate thousands of candidate hypotheses in a single run, and Google’s Co-Scientist proposes testable experiments at a rate no laboratory can fully evaluate. Each of these systems emphasizes verified results, yet that standard ranges from near-perfect formal proof in mathematics to decade-long clinical trials in medicine, with no shared framework for judging sufficiency across domains. As outputs scale beyond what humans can manually inspect, the question of which results to trust becomes as hard as generating them. The bottleneck for AI for Science is no longer hypothesis generation, it is verification.

Agentic systems are adept at solving well-scoped, verifiable problems in computational biology

We introduce CompBioBench, a benchmark of 100 diverse tasks for evaluating agentic systems in computational biology. Unlike mathematics and programming, which more readily admit systematic verification, biological data are inherently noisy and open to interpretation.

Palomar — Lean-verified mathematics

A public registry of Lean-verified mathematical results.

#bioinformatics #computationalbiology #aiforscience #singlecell #spatialtranscriptomics #opensource #codex #claudecode #mcp #agenticai | Herry Z.

BioNexus: A Scientific Reliability Layer for Agentic Biology As coding agents such as OpenAI Codex, Claude Code, and Cursor become increasingly capable of running bioinformatics workflows, a new problem emerges: Knowing how to run an analysis does not mean knowing when that analysis is scientifically valid. BioNexus is an open-source project exploring a reliability layer for biological coding agents rather than another standalone AI scientist. Its current architecture focuses on: Machine-readable Capability Contracts — define input semantics, scientific preconditions, canonical backends, refusal conditions, and evidence requirements. EvidenceCard 2.1 — separates execution state from scientific evidence and grades conclusions as PRELIMINARY, SUPPORTED, ROBUST, REPLICATED, FRAGILE, or ABSTAIN. Scientific Intent & Invariant Routing — detects invalid analyses such as pseudoreplication, incompatible data representations, or missing backends. BioNexus Eval — evaluates routing, unsafe claims, refusal behavior, and scientific outcomes. RunBundle Artifact Contract — packages inputs, parameters, results, provenance, environment, and evidence into reproducible machine-readable runs. The long-term goal is simple: Make scientific reliability a first-class infrastructure layer for agentic biology. BioNexus is still evolving, and I am actively looking for contributors in bioinformatics, single-cell/spatial omics, scientific benchmarking, MCP, reproducibility, and AI coding agents. PRs, issues, benchmark datasets, scientific critiques, and failure cases are especially welcome. GitHub: HERRY423/BioNexus #Bioinformatics #ComputationalBiology #AIForScience #SingleCell #SpatialTranscriptomics #OpenSource #Codex #ClaudeCode #MCP #AgenticAI

The AI Chemist: To be trustworthy, LLMs need to show their work

Good scientists reveal how they do their experiments and report their results; so should any machine-driven research

Towards autonomous biology: Compiler-Verified Protocols as a Foundation for Real-World AI Execution

Artificial intelligence has advanced from analyzing experimental data to autonomously generating hypotheses, designing experiments, and coordinating closed loop discovery. Yet the translation from computational reasoning to physical execution remains bottlenecked by the experimental protocol, which in biology still relies on ambiguous natural-language descriptions: a medium other engineering disciplines abandoned decades ago in favor of compiler verified specification languages. This deficit fragments reproducibility along three axes: protocol accuracy, pre execution verification, and cross platform portability. Existing formalisms address only subsets of these challenges, trading expressiveness for rigor, portability for standardization, or usability for provenance. Here we introduce the Biology Protocol Language (BPL), a domain specific language with a biology-native type system in which every quantity carries physical units, every reagent declares its physical form, and every container maintains compiler-tracked state, so that implicit assumptions must be stated explicitly and physically impossible operations are rejected at compile time. We further develop BPL-COGEN, a pipeline that couples a fine tuned 30 billion parameter language model with the deterministic compiler in a closed generate validate repair loop, iteratively correcting the translation from natural language SOPs to BPL through compiler diagnostics until all physical, dimensional, and state constraints are satisfied. On a benchmark of 300 published Nature Protocols papers, BPL COGEN achieved an overall fidelity score of 95.1 against the source protocols as ground truth. Wet-lab experiment and cross-platform validation in GFP expression library construction and HPLC to UHPLC method translation confirmed that a single BPL source yielded reproducible execution across manual and liquid handler assisted contexts. The results established a novel pipeline that generates compiler-verified protocols, which is an essential prerequisite for physically embodied AI in biology.

Life sciences verification program

Starting today, teams and institutions can apply to our Life Sciences Verification Program (LSVP), which gives verified researchers and organizations access to our most capable models for professional biology and drug development work. LSVP is built for teams of all kinds: academic labs, startups, non-profits, biotechs, pharma companies, and more. Learn more and apply here: https://lnkd.in/gDjrMm7D  We know that it has been frustrating to encounter blocks on Claude Fable. LSVP is our mechanism to prevent biological misuse, like the documented cases we’ve disrupted, while enabling legitimate life sciences research. LSVP includes a new set of safeguards to provide a better UX for life sciences researchers. We believe that advancing biology research and drug discovery will realize some of the most important beneficial impacts of AI.  Please DM me if you run into any issues with the application. https://lnkd.in/gNQYePZp | 35 comments on LinkedIn

Mathematician catches AI-generated research error by OpenAI president | Benjamin Golub posted on the topic | LinkedIn

In case you think the problem of science slop is hypothetical, here's something that happened on X yesterday: The president of OpenAI sharing wrong science slop as proud evidence of the power of mathematical AI. Within a day, the mathematician Tony Feng found a gap that undercut the paper's central argument; the author conceded, and (to his credit!) Brockman deleted the tweet as soon as he became aware. Feng was a hero to catch this. The author of the refuted claim is a mathematician but a self-described outsider to this area, with a prior big claim posted and quickly withdrawn. Greg Brockman is not an expert, either. Neither could check what they were amplifying. Heroes catching errors one at a time won't hold against the volume of AI-generated research now coming. We need an automated, independent verification layer for knowledge work, which is what we're building at Refine (refine.ink). Passing such verification should become a necessary step before big claims, especially outside one's own expertise. A link to a bigger-picture post on this is in the comments. | 23 comments on LinkedIn

It’s make or break time for AI labeling systems

It’s time for SynthID and C2PA to prove they work.

Linux Foundation Welcomes TRACE to Advance Verifiable Runtime Evidence for AI Workloads

The Linux Foundation introduces TRACE, a new open standard for verifiable evidence in AI workloads, ensuring portable and trustworthy governance across platforms.

Claude for Science. Thursday, November 12, at the Niels Bohr Institute, University of Copenhagen. In September, Anthropic reported that Claude agents found a CRISPR-like enzyme system in… | Jacob Langvad Nilsson | 17 comments

Claude for Science. Thursday, November 12, at the Niels Bohr Institute, University of Copenhagen. In September, Anthropic reported that Claude agents found a CRISPR-like enzyme system in bacteriophage DNA. Earlier in the same month, another AI lab said its model had settled a question about the Navier-Stokes equations, one of the seven Millennium Prize Problems. Results like these don't come from a lucky afternoon with a chatbot. They come from years of method development, careful engineering and building on the work that came before. What has changed is how much of that work a model can now take on, and the speed of which it does it. The latest models read the literature, write and run their own code, and work through a dataset for hours without losing track. In the enzyme case, hundreds of Claude agents searched DNA data for 21 hours and flagged a pattern nobody had asked them to look for. The method still has to be sound. But a researcher with a good question can now test far more ideas in a week than before. That is what we want to show. How researchers here set up their work so that stronger models can move it forward. The evening is about how the Danish science community uses Claude to drive new research and breakthroughs. Four STEM researchers from academia and industry each show one case. What they asked Claude, how they worked, what they found, and what failed. The room is part of it. Niels Bohr lectured in Auditorium A, and in the 1930s Heisenberg and Pauli sat in the front row. We will meet in this historical auditorium and get a glimpse of frontier scientific research using Claude. ▪️ Doors open 15:30, programme 16:00 to 19:00 ▪️ Four case studies from Danish research ▪️ 50 seats on the original wooden benches ▪️ Registration by approval Thanks to Daniel Murnane for making this happen. Thanks to the Niels Bohr Institute and the University of Copenhagen for having us. To Katrin Hjorth Thordardottir for logistics. And thanks to Anthropic and the Claude Community team, Martine N., Cassie Baquero Mayes, Chiara Massironi for your continued support. Link to sign up in the first comment below. We are still confirming speakers. Which researcher in Denmark should I talk to? | 17 comments on LinkedIn

Konrad Hinsen's blog

Thirty years after my first contact with computational (ir)reproducibility, I am happy to note that many things have improved. Reproducibility, computational and otherwise, is increasingly recognized as an important aspect of scientific quality control, and mostly considered worth striving for. However, I also note that more and more people, including reproducibility activists, have lost contact with the day-to-day reality in which reproducibility matters. Reproducibility is becoming an item on a checklist, and its precise incarnation the subject of political bickering aimed at making it easy to check off that item. So let's take a look at why computational reproducibility matters for researchers.