A working hypothesis about neurons, generalization, and the ecology of circuits.
History of Neuroscience and AI: From neurons to the first artificial neural networks
NeuroAI
Neuroscience, cognitive science, and AI are all questing for principles that help generalization. Learn more through a live, synchronous program designed for focused, hands-on learning.
Neural Computer: A New Machine Form Is Emerging
A research essay on Neural Computer: how it differs from agents, world models, and conventional computers; what runtime and CNC would mean; what current prototypes already show; and how software and hardware might change.
Understanding Artificial Neural Networks: Mysterianism about Known Mechanism is Mysticism
Mysterianism is the idea that human cognition, mind, cannot be understood. Taking this concept and applying it to known mechanism — such that claims are made that we do not know how engineered systems, such as artificial neural networks (ANNs), work, or that they constitute black boxes that we can only open with difficulty — is inappropriate at best and malicious at worst. We do know the mechanistic structure of such models because we designed and built them. We also do know their functional role (what they are for) as well as the mathematical function they are asked to approximate (map inputs to target outputs). Because mysterianist beliefs about known systems, such as ANNs, are often expressed, scientists need to sit up and take notice. We provide an error theory as to what is going on to help unpick this metatheoretical blunder. Ultimately, the problem is that 'understanding' is not a technical term in these cases: the word is co-opted for a specific narrative to sell 'artificial intelligence' through mystification. All computational systems, from pendulums to databases, will behave in ways we cannot predict or control — this is not a unique property of ANNs — and experts do indeed grasp the computational properties of these systems nonetheless.
Understanding Artificial Neural Networks: Mysterianism about Known Mechanism is Mysticism
Mysterianism is the idea that human cognition, mind, cannot be understood. Taking this concept and applying it to known mechanism — such that claims are made that we do not know how engineered systems, such as artificial neural networks (ANNs), work, or that they constitute black boxes that we can only open with difficulty — is inappropriate at best and malicious at worst. We do know the mechanistic structure of such models because we designed and built them. We also do know their functional role (what they are for) as well as the mathematical function they are asked to approximate (map inputs to target outputs). Because mysterianist beliefs about known systems, such as ANNs, are often expressed, scientists need to sit up and take notice. We provide an error theory as to what is going on to help unpick this metatheoretical blunder. Ultimately, the problem is that 'understanding' is not a technical term in these cases: the word is co-opted for a specific narrative to sell 'artificial intelligence' through mystification. All computational systems, from pendulums to databases, will behave in ways we cannot predict or control — this is not a unique property of ANNs — and experts do indeed grasp the computational properties of these systems nonetheless.
Symbols, neural networks, and mathematical intelligence
Some reflections on connectionism and the basis of higher-level cognition
Reverse-Engineering the Wetware: Spiking Networks, TD Errors, and the End of Matrix Math
An engineer-friendly deep dive into how the human brain truly processes information, learns without backpropagation, and uses spiking neural networks, predictive coding, and dopamine-driven TD errors—challenging our notions of AI and matrix math.
Are biological systems poised at criticality?
Many of life's most fascinating phenomena emerge from interactions among many elements--many amino acids determine the structure of a single protein, many genes determine the fate of a cell, many neurons are involved in shaping our thoughts and memories. Physicists have long hoped that these collective behaviors could be described using the ideas and methods of statistical mechanics. In the past few years, new, larger scale experiments have made it possible to construct statistical mechanics models of biological systems directly from real data. We review the surprising successes of this "inverse" approach, using examples form families of proteins, networks of neurons, and flocks of birds. Remarkably, in all these cases the models that emerge from the data are poised at a very special point in their parameter space--a critical point. This suggests there may be some deeper theoretical principle behind the behavior of these diverse systems.
The Global Brain Argument: Nodes, Computroniums and the Ai Megasystem (Target Paper for Special Issue)
The Global Brain Argument contends that many of us are, or will be, part of a global brain network that includes both biological and artificial intelligences (AIs), such as generative AIs ...
The Umwelt Representation Hypothesis: rethinking Universality
Recent studies reveal striking representational alignment between artificial neural networks (ANNs) and biological brains, leading to proposals that all sufficiently capable systems converge on universal representations of reality. We argue that this claim of Universality is premature. We introduce the Umwelt Representation Hypothesis, which proposes that alignment arises not from convergence toward a single global optimum but from overlap in the ecological constraints under which systems develop. We review empirical evidence showing that representational differences between species, individuals, and ANNs are systematic and adaptive, which is difficult to reconcile with Universality. Finally, we reframe ANN model comparison as a method for mapping clusters of alignment in the ecological constraint space rather than as a search for a single optimal world model.
Understanding neural networks through sparse circuits
OpenAI is exploring mechanistic interpretability to understand how neural networks reason. Our new sparse model approach could make AI systems more transparent and support safer, more reliable behavior.
We Don't Understand Neural Networks At The Algorithmic Level
The largest ongoing debate about AI is “Are Large Language Models (LLMs) intelligent?” That makes sense, at least: the evidence is ambiguous and the stakes a...
Is AI Hiding Its Full Power? With Geoffrey Hinton
AI Data Centers Will Be Obsolete (Geometric Reasoning Explained)
Why are neural networks and cryptographic ciphers so similar?
Several neural network design principles are almost identical to cipher design principles.
