Mysterianism is the idea that human cognition, mind, cannot be understood. Taking this concept and applying it to known mechanism — such that claims are made that we do not know how engineered systems, such as artificial neural networks (ANNs), work, or that they constitute black boxes that we can only open with difficulty — is inappropriate at best and malicious at worst. We do know the mechanistic structure of such models because we designed and built them. We also do know their functional role (what they are for) as well as the mathematical function they are asked to approximate (map inputs to target outputs). Because mysterianist beliefs about known systems, such as ANNs, are often expressed, scientists need to sit up and take notice. We provide an error theory as to what is going on to help unpick this metatheoretical blunder. Ultimately, the problem is that 'understanding' is not a technical term in these cases: the word is co-opted for a specific narrative to sell 'artificial intelligence' through mystification. All computational systems, from pendulums to databases, will behave in ways we cannot predict or control — this is not a unique property of ANNs — and experts do indeed grasp the computational properties of these systems nonetheless.
Understanding Artificial Neural Networks: Mysterianism about Known Mechanism is Mysticism
Mysterianism is the idea that human cognition, mind, cannot be understood. Taking this concept and applying it to known mechanism — such that claims are made that we do not know how engineered systems, such as artificial neural networks (ANNs), work, or that they constitute black boxes that we can only open with difficulty — is inappropriate at best and malicious at worst. We do know the mechanistic structure of such models because we designed and built them. We also do know their functional role (what they are for) as well as the mathematical function they are asked to approximate (map inputs to target outputs). Because mysterianist beliefs about known systems, such as ANNs, are often expressed, scientists need to sit up and take notice. We provide an error theory as to what is going on to help unpick this metatheoretical blunder. Ultimately, the problem is that 'understanding' is not a technical term in these cases: the word is co-opted for a specific narrative to sell 'artificial intelligence' through mystification. All computational systems, from pendulums to databases, will behave in ways we cannot predict or control — this is not a unique property of ANNs — and experts do indeed grasp the computational properties of these systems nonetheless.
Machine understanding
What do artificial intelligence (AI) systems “understand”? This question arises not only in assessing a system’s intelligence but also in evaluation practices to ensure the safe and responsible deployment of AI. Drawing on scholarship from philosophy and cognitive science, and informed by current practices in AI, we develop a framework for asking more precise questions and making more precise claims about machine understanding. We conceptualize understanding as a relation between a system (S) and a target of understanding (T), and we discuss how to specify the relation, the system, and the target, offering a landscape of options in each case. Our goal is not to defend a particular account of understanding, but to provide conceptual tools for those working to assess or advance machine understanding.
AI That Evolves in the Wild | Edge.org
I’m interested not in domesticated AI—the stuff that people are trying to sell. I'm interested in wild AI—AI that evolves in the wild. I’m a naturalist, so that’s the interesting thing to me. Thirty-four years ago there was a meeting just like this in which Stanislaw Ulam said to everybody in the room—they’re all mathematicians—"What makes you so sure that mathematical logic corresponds to the way we think?" It’s a higher-level symptom. It’s not how the brain works. All those guys knew fully well that the brain was not fundamentally logical.
AI is An Ideology, Not A Technology
At its core, "artificial intelligence" is a perilous belief that fails to recognize the agency of humans.
What Is Intelligence? Lessons from AI About Evolution, Computing, and Minds | Blaise Agüera y Arcas
Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics
What would it mean to have a scientific understanding of AI? Models are not static objects: they are snapshots of time-evolving processes shaped by data, objectives, architectures, and optimization dynamics. Yet much of AI research treats models as fixed artifacts, analyzing behaviors after training rather than asking why they emerge. This position paper argues that a science of AI must move beyond post-hoc fixes and study the training dynamics that produce model behavior. Such a science should support progressively stronger forms of understanding: predicting outcomes from early training signals, intervening when trajectories go wrong, and ultimately designing training procedures that more reliably produce desired properties. Scaling laws have made prediction routine for loss; the challenge is extending this success to capabilities, biases, robustness, and safety-relevant behaviors. We articulate requirements for such theories grounded in the history and philosophy of science, examine progress in mechanistic interpretability, fairness, memorization, and simplicity bias, and identify concrete open problems.
Why I believe current AI models can really think and understand - Part 2 of 3
Why computers can in principle produce thinking and understanding
Knowledge Collapse
AI companies are racing to mechanize mathematics. Where does that leave human understanding?
Why I believe current AI models can really think and understand - Part 1 of 2
How the human mind is different from our conception of it
Introducing TRIBE v2: A Predictive Foundation Model Trained to Understand How the Human Brain Processes Complex Stimuli | Keith Doelling
This is some very cool work by some awesome colleagues Jean-Rémi King, and Teon Brooks! Seriously not enough good things can be said about how cool it is. You should enjoy it and play with it. And kudos to Meta for open sourcing it. At the same time, I'm already seeing posts about how the model will replace fMRI experiments as researchers will simulate how the brain "really works" instead of running costly experiments. I think this goes WELL beyond what its creators intend. We are already seeing that use of AI in science allows you to explore charted ideas more thoroughly and much more rapidly but slows us down in finding novel ideas (https://lnkd.in/eMR2akqt). At the same time, there is growing concern that LLM performance will collapse as they are increasingly trained on their own output (https://lnkd.in/eavgfyuY). Leaving neuroscience to AI simulations risks following the same fate, where we generate seemingly new findings without gaining new meaning. A mechanistic understanding of how the brain works (if that is still your goal) will be found at the margins, in errors and idiosyncrasies of neural function. What TRIBE provides is a super useful and cool instantiation of our current understanding on how and where neural activity is instantiated in the brain. But it won't help us make groundbreaking new findings of how neural circuits lead to cognition and behavior. Experiments on real human brains, may be costly, but they will always be necessary!
Introducing TRIBE v2: A Predictive Foundation Model Trained to Understand How the Human Brain Processes Complex Stimuli | Keith Doelling
This is some very cool work by some awesome colleagues Jean-Rémi King, and Teon Brooks! Seriously not enough good things can be said about how cool it is. You should enjoy it and play with it. And kudos to Meta for open sourcing it. At the same time, I'm already seeing posts about how the model will replace fMRI experiments as researchers will simulate how the brain "really works" instead of running costly experiments. I think this goes WELL beyond what its creators intend. We are already seeing that use of AI in science allows you to explore charted ideas more thoroughly and much more rapidly but slows us down in finding novel ideas (https://lnkd.in/eMR2akqt). At the same time, there is growing concern that LLM performance will collapse as they are increasingly trained on their own output (https://lnkd.in/eavgfyuY). Leaving neuroscience to AI simulations risks following the same fate, where we generate seemingly new findings without gaining new meaning. A mechanistic understanding of how the brain works (if that is still your goal) will be found at the margins, in errors and idiosyncrasies of neural function. What TRIBE provides is a super useful and cool instantiation of our current understanding on how and where neural activity is instantiated in the brain. But it won't help us make groundbreaking new findings of how neural circuits lead to cognition and behavior. Experiments on real human brains, may be costly, but they will always be necessary!
We Don't Understand Neural Networks At The Algorithmic Level
The largest ongoing debate about AI is “Are Large Language Models (LLMs) intelligent?” That makes sense, at least: the evidence is ambiguous and the stakes a...
Is AI Hiding Its Full Power? With Geoffrey Hinton
Machine god metaphors eat your brain
There are other ways of thinking about AI
Model Collapse Ends AI Hype
History of Neuroscience and AI: From neurons to the first artificial neural networks