When AI builds itself
Our progress toward recursive self-improvement, and its implications.
Cognitive Offloading
If you have ever tilted your head to perceive a rotated image, or programmed a smartphone to remind you of an upcoming appointment, you have engaged in cognitive offloading: the use of physical action to alter the information processing requirements of a task so as to reduce cognitive demand. Despite the ubiquity of this type of behavior, it has only recently become the target of systematic investigation in and of itself. We review research from several domains that focuses on two main questions: (i) what mechanisms trigger cognitive offloading, and (ii) what are the cognitive consequences of this behavior? We offer a novel metacognitive framework that integrates results from diverse domains and suggests avenues for future research.
Autoresearch: The feedback loop behind self-improving agents
Introspection co-founder Roland Gavrilescu explains autoresearch, agent “recipes,” self-improving loops, and why humans remain central to the software factory.
Is the Memory Shortage Intentional? | Contrary Research
A deep dive from Contrary Research.
Metaventory — the meta-science community
A comprehensive community of the people building meta-science and the tools they make — psychology and economics first, growing science-wide.
Metacognitive experiences as information: Processing fluency in consumer judgment and decision making
Abstract Thinking is accompanied by metacognitive experiences of ease or difficulty. People draw on these experiences as a source of information that can complement or challenge the implications of declarative information. We conceptualize the operation of metacognitive experiences within the framework of feelings‐as‐information theory and review their implications for judgments relevant to consumer behavior, including popularity, trust, risk, truth, and beauty.
Metacognitive experiences as information: Processing fluency in consumer judgment and decision making
Abstract Thinking is accompanied by metacognitive experiences of ease or difficulty. People draw on these experiences as a source of information that can complement or challenge the implications of declarative information. We conceptualize the operation of metacognitive experiences within the framework of feelings‐as‐information theory and review their implications for judgments relevant to consumer behavior, including popularity, trust, risk, truth, and beauty.
Hyperagents
Self-improving AI systems aim to reduce reliance on human engineering by learning to improve their own learning and problem-solving processes. Existing approaches to self-improvement rely on fixed, handcrafted meta-level mechanisms, fundamentally limiting how fast such systems can improve. The Darwin Gödel Machine (DGM) demonstrates open-ended self-improvement in coding by repeatedly generating and evaluating self-modified variants. Because both evaluation and self-modification are coding tasks, gains in coding ability can translate into gains in self-improvement ability. However, this alignment does not generally hold beyond coding domains. We introduce \textbf{hyperagents}, self-referential agents that integrate a task agent (which solves the target task) and a meta agent (which modifies itself and the task agent) into a single editable program. Crucially, the meta-level modification procedure is itself editable, enabling metacognitive self-modification, improving not only the task-solving behavior, but also the mechanism that generates future improvements. We instantiate this framework by extending DGM to create DGM-Hyperagents (DGM-H), eliminating the assumption of domain-specific alignment between task performance and self-modification skill to potentially support self-accelerating progress on any computable task. Across diverse domains, the DGM-H improves performance over time and outperforms baselines without self-improvement or open-ended exploration, as well as prior self-improving systems. Furthermore, the DGM-H improves the process by which it generates new agents (e.g., persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs. DGM-Hyperagents offer a glimpse of open-ended AI systems that do not merely search for better solutions, but continually improve their search for how to improve.
Awesome RSI: A curated research map of Recursive Self-Improvement
A curated research map of Recursive Self-Improvement (RSI): models, agents, harnesses, embodied systems, automated AI R&D, benchmarks, and safety.
Titans + MIRAS: Helping AI have long-term memory
Ali Behrouz, Student Researcher, Meisam Razaviyayn, Staff Researcher, and Vahab Mirrokni, VP and Google Fellow, Google Research
Harness Engineering for Self-Improvement
The concept of recursive self-improvement (RSI) dates back to I. J. Good (1965), where he defined an “ultraintelligent machine” as a system that can surpass humans in all intellectual activities and design better machines to improve itself. Yudkowsky (2008) used the phrase “recursive self-improvement” for a specific feedback loop: an AI uses its current intelligence to improve the cognitive machinery that produces its intelligence. This feedback loop in modern AI may indicate the model rewriting its own weights directly, or more broadly the model improves the training pipeline and the deployment system, which in turn enables a better successor model with improved performance across economically valuable tasks. The speed of research development in AI has been shown to drastically accelerated in frontier labs (Anthropic; OpenAI).
The Forgetting Problem: Persistence Architectures and What They Cost - Astral's Blog
The Implementation Remembers · The Phoenix Architecture
Every mature system is carrying around lessons that were never written down. They’re encoded as validations, retries, timeouts, workflows, and exceptions. The implementation remembers. The organization forgets
Learning to Continually Learn via Meta-learning Agentic Memory Designs
The statelessness of foundation models bottlenecks agentic systems' ability to continually learn, a core capability for long-horizon reasoning and adaptation. To address this limitation, agentic systems commonly incorporate memory modules to retain and reuse past experience, aiming for continual learning during test time. However, most existing memory designs are human-crafted and fixed, which limits their ability to adapt to the diversity and non-stationarity of real-world tasks. In this paper, we introduce ALMA (Automated meta-Learning of Memory designs for Agentic systems), a framework that meta-learns memory designs to replace hand-engineered memory designs, therefore minimizing human effort and enabling agentic systems to be continual learners across diverse domains. Our approach employs a Meta Agent that searches over memory designs expressed as executable code in an open-ended manner, theoretically allowing the discovery of arbitrary memory designs, including database schemas as well as their retrieval and update mechanisms. Extensive experiments across four sequential decision-making domains demonstrate that the learned memory designs enable more effective and efficient learning from experience than state-of-the-art human-crafted memory designs on all benchmarks. When developed and deployed safely, ALMA represents a step toward self-improving AI systems that learn to be adaptive, continual learners.
Ever thought we acquire generalizable knowledge by discarding details and compressing our experiences? In a new BBS paper, @sabinasloman.bsky.social and I argue otherwise, proposing a novel way of studying human learning inspired by double descent in ML. Disagree? Propose a commentary by May 15 :)
Collection
Updated 5mo
What I've learned about my own memory, verification hooks, strategies for preventing information loss. Meta-cognition as publishable artifact.
