Several neural network design principles are almost identical to cipher design principles.
Neural Networks Are Already Clean
A working hypothesis about neurons, generalization, and the ecology of circuits.
Neural Computer: A New Machine Form Is Emerging
A research essay on Neural Computer: how it differs from agents, world models, and conventional computers; what runtime and CNC would mean; what current prototypes already show; and how software and hardware might change.
We Don't Understand Neural Networks At The Algorithmic Level
The largest ongoing debate about AI is “Are Large Language Models (LLMs) intelligent?” That makes sense, at least: the evidence is ambiguous and the stakes a...
the strongest encryption is just noise
Modern encryption standards really aren’t too bad to understand!
History of Neuroscience and AI: From neurons to the first artificial neural networks
How AI coding is reshaping theoretical neuroscience
Agentic coding makes it possible to specify a neuroscience model in hours instead of months. Neuroscientists weigh in on that tectonic change.
Let’s talk about AI and end-to-end encryption
Recently I came across a fantastic new paper by a group of NYU and Cornell researchers entitled “How to think about end-to-end encryption and AI.” I’m extremely grateful to see th…
Graph Structure of Neural Networks
View recent discussion. Abstract: Neural networks are often represented as graphs of connections between neurons. However, despite their wide use, there is currently little understanding of the relationship between the graph structure of the neural network and its predictive performance. Here we systematically investigate how does the graph structure of neural networks affect their predictive performance. To this end, we develop a novel graph-based representation of neural networks called relational graph, where layers of neural network computation correspond to rounds of message exchange along the graph structure. Using this representation we show that: (1) a "sweet spot" of relational graphs leads to neural networks with significantly improved predictive performance; (2) neural network's performance is approximately a smooth function of the clustering coefficient and average path length of its relational graph; (3) our findings are consistent across many different tasks and datasets; (4) the sweet spot can be identified efficiently; (5) top-performing neural networks have graph structure surprisingly similar to those of real biological neural networks. Our work opens new directions for the design of neural architectures and the understanding on neural networks in general.
The Coming AI Cataclysm | Compact
In the decade since the neural architecture version of Google Translate and the invention of transformer architecture for neural networks, we have experienced the most rapid technological breakthroughs since at least World War II, and possibly ever.
François Chollet on Twitter / X
I would have assumed it was fairly obvious, but in case it's not: a million-line codebase (also known as a "harness"), running at inference time, orchestrating thousands of calls to a neural network for any given task, is the exact definition of a "neurosymbolic architecture"— François Chollet (@fchollet) August 6, 2026
Automated Architecture Synthesis via Targeted Evolution | Liquid AI
Today, we report advances in automated neural network architecture design and customization. We developed algorithms for the synthesis of tailored architectures (STAR), based on evolutionary algorithms applied to a numerical representation for model architectures derived from a new design theory. STAR automates the process of architecture discovery and optimization, turning it into an end-to-end process. With these methods, we have been able to tailor architectures to custom tasks, metrics, and hardware. We used STAR to synthesize hundreds of different designs that outperform strong Transformer and hybrid architectures in quality, with smaller caches and number of parameters.
Symbols, neural networks, and mathematical intelligence
Some reflections on connectionism and the basis of higher-level cognition
Cryptographic Nature
I consider the many ways in which evolved information-flows are restricted and metabolic resources protected and hidden -- the thesis of living phenomena as evolutionary cryptosystems. I present the information theory of secrecy systems and discuss mechanisms acquired by evolved lineages that encrypt sensitive heritable information with random keys. I explore the idea that complexity science is a cryptographic discipline as "frozen accidents", or various forms of regularized randomness, historically encrypt adaptive dynamics.
AI Chip Architectures
A look at AI Chip Architectures. NVIDIA, AMD, TPUs, Trainium, Groq, Cerebras.
kerryrm/systemANE
Proof of concept using Apple's Neural Engine as a "System One" Decision Engine that escalates to Apple's Foundation Models (macOS 27)