







The ability for language models to write coherent text can be explained as each token develops some affinity for other tokens.During next token prediction each of the previous tokens "vote" for the tokens they have high affinity for, which results in coherent continuation. https://t.co/APkIpDoi4N— Chomba Bupe (@ChombaBupe) April 21, 2024
Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential
Autoregressive language models are constrained by their inherently sequential nature, generating one token at a time. This paradigm limits inference speed and parallelism, especially during later stages of generation when the direction and semantics of text are relatively certain. In this work, we propose a novel framework that leverages the inherent knowledge of vanilla autoregressive language models about future tokens, combining techniques to realize this potential and enable simultaneous prediction of multiple subsequent tokens. Our approach introduces several key innovations: (1) a masked-input formulation where multiple future tokens are jointly predicted from a common prefix; (2) a gated LoRA formulation that preserves the original LLM's functionality, while equipping it for multi-token prediction; (3) a lightweight, learnable sampler module that generates coherent sequences from the predicted future tokens; (4) a set of auxiliary training losses, including a consistency loss, to enhance the coherence and accuracy of jointly generated tokens; and (5) a speculative generation strategy that expands tokens quadratically in the future while maintaining high fidelity. Our method achieves significant speedups through supervised fine-tuning on pretrained models. For example, it generates code and math nearly 5x faster, and improves general chat and knowledge tasks by almost 2.5x. These gains come without any loss in quality.

Understanding GPT tokenizers
Large language models such as GPT-3/4, LLaMA and PaLM work in terms of tokens. They take text, convert it into tokens (integers), then predict which tokens should come next. Playing …

Tokenization: A Survey for Modern NLP
While modern language models take raw text as their input and produce raw text as output, they do not operate over text directly. Hidden in the very first step of language model pipelines is...

Recursive Language Models: the paradigm of 2026
How we plan to manage extremely long contexts
.png?v=33da7708a390)
Mastering Token Limits and Memory in ChatGPT and other Large Language Models
A Guide for the Everyday User

Large language models, explained with a minimum of math and jargon
Want to really understand how large language models work? Here’s a gentle primer.

Words that Work: Using Large Language Models to Generate and Refine Hypotheses from Text
In this paper, we introduce a data-driven framework for generating and refining hypotheses from text. Our three-step approach–Hypothesize, Intervene, and Predic
How Large Language Models Actually Work
Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!
Intermediate token generation (ITG), where a model produces output before the solution, has become a standard method to improve the performance of language models on reasoning tasks. These intermediate tokens have been called \say{reasoning traces} or even \say{thinking traces} -- implicitly anthropomorphizing the traces, and implying that these traces resemble steps a human might take when solving a challenging problem, and as such can provide an interpretable window into the operation of the model's thinking process to the end user. In this position paper, we present evidence that this anthropomorphization isn't a harmless metaphor, and instead is quite dangerous -- it confuses the nature of these models and how to use them effectively, and leads to questionable research. We call on the community to avoid such anthropomorphization of intermediate tokens.

Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!
Intermediate token generation (ITG), where a model produces output before the solution, has become a standard method to improve the performance of language models on reasoning tasks. These intermediate tokens have been called \say{reasoning traces} or even \say{thinking traces} -- implicitly anthropomorphizing the traces, and implying that these traces resemble steps a human might take when solving a challenging problem, and as such can provide an interpretable window into the operation of the model's thinking process to the end user. In this position paper, we present evidence that this anthropomorphization isn't a harmless metaphor, and instead is quite dangerous -- it confuses the nature of these models and how to use them effectively, and leads to questionable research. We call on the community to avoid such anthropomorphization of intermediate tokens.

Language Machines
How generative AI systems capture a core function of language Looking at the emergence of generative AI, Language Machines presents a new theory of meaning i...

Hypothesis generation and updating in large language models
Large language models (LLMs) increasingly help people solve problems, from debugging code to repairing machinery. This process requires generating plausible hypotheses from partial descriptions,...

PoliSim@CHI 2026
Large Language Models are rapidly evolving from text generators into reasoning systems that can act as autonomous agents. When placed in social contexts, these agents display emergent behaviors such as forming coalitions, spreading information, and making collective decisions.
I finally understand how large language models actually work After reading the 2025 textbook “Foundations of LLMs” It blew my mind and cleared up years of confusion Here’s everything i learned (in plain english):
I finally understand how large language models actually workAfter reading the 2025 textbook “Foundations of LLMs”It blew my mind and cleared up years of confusionHere’s everything i learned (in plain english): pic.twitter.com/Db7NHTks55— Louis Gleeson (@aigleeson) July 31, 2025
Andrej Karpathy on Twitter / X
We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for… pic.twitter.com/uMFWTcBCsl— Andrej Karpathy (@karpathy) October 2, 2026
I'm creating a series of short form videos about how language models work technically. The goal is to be something in between "you know it's next token prediction" and "now you've taken a machine learning class." I'd love your thoughts so here are the first few! 🧵 youtube.com/shorts/VZB8XCcyllE
How does ChatGPT work? Or rather, language models in general- Part 1 attempting a lay explanation.
www.youtube.com