An Intuitive Introduction to Flow-Based Language Generation — Floor Eijkelboom

Inference-Time Scaling for Generative Pre-Training | Jiaming Song

A short note on the false dichotomy between autoregression and diffusion, why flow maps help, and what this might mean for language models.

Continuous diffusion language models

Fully discrete methods dominated for a few years, but language models based on continuous diffusion are making a comeback.

Continuous diffusion language models

Fully discrete methods dominated for a few years, but language models based on continuous diffusion are making a comeback.

LLMs and World Models, Part 1

How do Large Language Models Make Sense of Their “Worlds”?

Large language model

A large language model (LLM) is a neural network trained on a vast amount of text for natural language processing tasks, especially language generation. LLMs can typically generate, summarize, translate, and analyze text in many contexts, and are a foundational technology behind modern chatbots.[1] Biased or inaccurate training data can make an LLM's output less reliable.[2]

Understanding Understanding: A Pragmatic Framework Motivated by...

Motivated by the rapid ascent of Large Language Models (LLMs) and debates about the extent to which they possess human-level qualities, we propose a framework for testing whether any agent (be it...

Beyond Standard LLMs

Linear Attention Hybrids, Text Diffusion, Code World Models, and Small Recursive Transformers

Recursive Language Models: the paradigm of 2026

How we plan to manage extremely long contexts

PoliSim@CHI 2026

Large Language Models are rapidly evolving from text generators into reasoning systems that can act as autonomous agents. When placed in social contexts, these agents display emergent behaviors such as forming coalitions, spreading information, and making collective decisions.

Language Machines

How generative AI systems capture a core function of language Looking at the emergence of generative AI, Language Machines presents a new theory of meaning i...

Forcing Generative Models to Degenerate Ones: The Power of Data...

Growing applications of large language models (LLMs) trained by a third party raise serious concerns on the security vulnerability of LLMs.It has been demonstrated that malicious actors can...

Differences in Text Generated by Diffusion and Autoregressive Language Models

Diffusion language models (DLMs) are promising alternatives to autoregressive language models (ARMs), yet the intrinsic differences in their generated text remain underexplored. We first find empirically that off-the-shelf DLMs exhibit lower $n$-gram entropy, higher semantic coherence, and higher semantic diversity. To understand the cause, we conduct controlled experiments that decouple the effects of training objectives and decoding algorithms. Results suggest that the DLM training objective contributes to the increases in semantic coherence and semantic diversity, but has a minor influence on entropy. These differences are primarily driven by the bidirectional context; other components in the training objective, such as input masking, label masking, and the weighting function, have a much weaker influence. Further, our experiments demonstrate that the reduction in entropy stems from DLMs' decoding algorithms, particularly confidence-based remasking strategies. We provide a theoretical understanding for this entropy reduction phenomenon. Together, our work uncovers key mechanisms underlying the differences between DLMs and ARMs in text generation, and informs future design of training objectives and decoding algorithms in DLMs.

Language models are multiverse generators

"Actualities seem to float in a wider sea of possibilities from out of which they were chosen; and somewhere, indeterminism says, such possibilities exist, and form part of the truth."

MemGPT (MemGPT)

Large Language Models (LLMs)