Simon Michaux: "The Arcadians" - The Great Simplification

Episode 31: Science Is a Human Endeavor (feat. Molly Crockett and Lisa Messeri), April 15 2024 - Mystery AI Hype Theater 3000

Will AI someday do all our scientific research for us? Not likely. Drs. Molly Crockett and Lisa Messeri join for a takedown of the hype of

No. 018: Genesis Mission $5B, Challengescape problem directory, team science vs. solo AI agents - The Hallway Track

The Hallway Track No. 018 - How AI is affecting the practice of science

PerThirtySix

Thoughtful, data-driven explorations

Matt Rickard | Substack

Thoughts on engineering, startups, and AI. Click to read Matt Rickard, a Substack publication with thousands of subscribers.

Obsidian's CEO on why productivity tools need community more than AI - Boris Mann Tech Blog

The latest episode of Decoder features a deep dive about Obsidian with @stephango.com and why we're not rushing to add sparkly AI buttons everywhere www.theverge.com/decoder-podc...

Nature is Laughing at the AI Build Out -

I was catching up on Acquired episodes this week and listened to the spectacular “Google: The AI Company” episode that Ben and David put together. What the two of them have created with their podcast fascinates me in the inevitability of its success. I think Acquired proves that if you are smart and work your […]

NOVA | The Wisdom of the Crowds | Season 45 | Episode 6

Crowds hold a predictive power that can have startling accuracy.

AGENTWORLD | *

An exploration of planetary intelligence, tracing how computation and cosmology—from ancient mechanisms to AI—reshape Earth and its possible futures.

EurekaBench: Measuring Agentic Ability to Discover New Scientific Insights

When Isaac Newton discovered the law of gravitation, he did so through an iterative process of analyzing observed data such as planetary patterns, finding the underlying mechanisms by describing patterns in mathematical equations, and refining his theory against the Moon's orbit, revealing the startling insight that the same force governs both falling apples and orbiting planets. Would it be possible for AI agents to make similar discoveries? To measure this ability, we introduce EurekaBench, a cross-domain benchmark that tests AI agents' ability to conduct long-horizon experiments and discover mechanisms that explain observations. We evaluate these mechanisms by the scientific insights that can be derived from them. EurekaBench contains an expert-verified set of 26 long-horizon tasks across neuroscience, computer science, chemistry, astrophysics, geophysics, and plasma physics, with a total of 306 scientific insights that the discovered mechanisms are expected to support. Our evaluation framework tests three axes of scientific discovery: agents' ability to follow known scientific constraints, the predictive accuracy of the discovered mechanisms, and whether these mechanisms yield scientific insights or inform future research. Our results show that current AI agents often overly fixate on predictive accuracy optimization, surpassing human scientists, while falling substantially short in deriving scientific insights.