New research reveals why even state-of-the-art large language models stumble on seemingly easy tasks—and what it takes to fix it
Why Can’t Powerful LLMs Learn Multiplication?
These days, large language models (LLMs) can handle increasingly complex tasks, writing complex code and engaging in sophisticated reasoning. But when it comes to 4-digit multiplication, a task taught in elementary school, even state-of-the-art systems fail. Why? A new paper by Computer Science PhD student Xiaoyan Bai and Faculty Co-Director of the Data Science Institute’s …
Why Can't Transformers Learn Multiplication?...
Language models are increasingly capable, yet still fail at a seemingly simple task of multi-digit multiplication. In this work, we study why, by reverse-engineering a model that successfully...
Why Can't Transformers Learn Multiplication?...
Language models are increasingly capable, yet still fail at a seemingly simple task of multi-digit multiplication. In this work, we study why, by reverse-engineering a model that successfully...
AI’s Memorization Crisis
Large language models don’t “learn”—they copy. And that could change everything for the tech industry.
We Don't Understand Neural Networks At The Algorithmic Level
The largest ongoing debate about AI is “Are Large Language Models (LLMs) intelligent?” That makes sense, at least: the evidence is ambiguous and the stakes a...
Large language models are not the problem
If a Large Language Model (LLM) can replicate your scientific contribution, the problem is not the LLM. What does it say about our field that so much of the anxiety about AI comes down to the fear that a machine could do what we do? Perhaps it says we should be doing something better.
Small Language Models are the Future of Agentic AI
Large language models (LLMs) are often praised for exhibiting near-human performance on a wide range of tasks and valued for their ability to hold a general conversation. The rise of agentic AI systems is, however, ushering in a mass of applications in which language models perform a small number of specialized tasks repetitively and with little variation. Here we lay out the position that small language models (SLMs) are sufficiently powerful, inherently more suitable, and necessarily more economical for many invocations in agentic systems, and are therefore the future of agentic AI. Our argumentation is grounded in the current level of capabilities exhibited by SLMs, the common architectures of agentic systems, and the economy of LM deployment. We further argue that in situations where general-purpose conversational abilities are essential, heterogeneous agentic systems (i.e., agents invoking multiple different models) are the natural choice. We discuss the potential barriers for the adoption of SLMs in agentic systems and outline a general LLM-to-SLM agent conversion algorithm. Our position, formulated as a value statement, highlights the significance of the operational and economic impact even a partial shift from LLMs to SLMs is to have on the AI agent industry. We aim to stimulate the discussion on the effective use of AI resources and hope to advance the efforts to lower the costs of AI of the present day. Calling for both contributions to and critique of our position, we commit to publishing all such correspondence at https://research.nvidia.com/labs/lpr/slm-agents.
AI learns language from skewed sources. That could change how we humans speak – and think | Bruce Schneier
Large language models aren’t trained on real-life conversations. As we encounter their language, it could affect our own
Knowledge Collapse
AI companies are racing to mechanize mathematics. Where does that leave human understanding?
The AI Revolution in Math Has Arrived | Quanta Magazine
AI is being used to prove new results at a rapid pace. Mathematicians think this is just the beginning.
machines will never understand language
I thought language was too complicated for machines to understand, until I got sick. a lifelong, meandering journey of parsing, learning, and fixing my broken body.
Mathematicians are grappling with the possibility that AI might eclipse them
I talked to 20 mathematicians about rapid AI progress in their field.
Eliciting Reasoning in Language Models with Cognitive Tools
The recent advent of reasoning models like OpenAI's o1 was met with excited speculation by the AI community about the mechanisms underlying these capabilities in closed models, followed by a rush of replication efforts, particularly from the open source community. These speculations were largely settled by the demonstration from DeepSeek-R1 that chains-of-thought and reinforcement learning (RL) can effectively replicate reasoning on top of base LLMs. However, it remains valuable to explore alternative methods for theoretically eliciting reasoning that could help elucidate the underlying mechanisms, as well as providing additional methods that may offer complementary benefits. Here, we build on the long-standing literature in cognitive psychology and cognitive architectures, which postulates that reasoning arises from the orchestrated, sequential execution of a set of modular, predetermined cognitive operations. Crucially, we implement this key idea within a modern agentic tool-calling framework. In particular, we endow an LLM with a small set of "cognitive tools" encapsulating specific reasoning operations, each executed by the LLM itself. Surprisingly, this simple strategy results in considerable gains in performance on standard mathematical reasoning benchmarks compared to base LLMs, for both closed and open-weight models. For instance, providing our "cognitive tools" to GPT-4.1 increases its pass@1 performance on AIME2024 from 32% to 53%, even surpassing the performance of o1-preview. In addition to its practical implications, this demonstration contributes to the debate regarding the role of post-training methods in eliciting reasoning in LLMs versus the role of inherent capabilities acquired during pre-training, and whether post-training merely uncovers these latent abilities.
Artificial intelligence is a familiar-looking monster, say Henry Farrell and Cosma Shalizi
The academics argue that large language models have much older cousins in markets and bureaucracies
