







Can LLMs track state changes over long texts—like an object changing hands or a variable reassignment? On our new StateBench eval, frontier models including GPT-6 Sol & Claude Opus 5.5 fell to 13% accuracy or lower on the hardest, longest problems. 🧵 buff.ly/NF3iDYc
Nov 3, 2026 at 5:32 PM
Context Rot: How Increasing Input Tokens Impacts LLM Performance
Large Language Models (LLMs) are typically presumed to process context uniformly—that is, the model should handle the 10,000th token just as reliably as the 100th. However, in practice, this assumption does not hold. We observe that model performance varies significantly as input length changes, even on simple tasks. In this report, we evaluate 18 LLMs, including the state-of-the-art GPT-4.1, Claude 4, Gemini 2.5, and Qwen3 models. Our results reveal that models do not use their context uniformly; instead, their performance grows increasingly unreliable as input length grows.

Things we learned about LLMs in 2024
A lot has happened in the world of Large Language Models over the course of 2024. Here’s a review of things we figured out about the field in the past …

The shelf life of LLM findings
We revisited 50 published findings about language models across 20 models, tracing which fade, persist, or grow stronger.

Instant LLM Updates with Doc-to-LoRA and Text-to-LoRA
Recent LLM agents have shown impressive capabilities on complex computer use and long-horizon tasks. Yet, they still struggle with long-term memory and adaptation--two of the most important cognitive capabilities that still limit LLMs today. Without long-term memory, users have to provide LLMs with relevant content at the start of every new session, creating friction, discontinuity, and longer time-to-response. Additionally, due to the lack of adaptation, they do not learn from mistakes or user preferences from previous sessions, making each interaction as cumbersome as the first. Traditionally, these two problems are tackled by "updating" the model.
Improving Cross-Lingual Factual Recall via Consistency-Driven Reinforcement Learning
Large language models (LLMs) trained predominantly on English data encode substantial world knowledge, yet often fail to express it reliably in other languages, a phenomenon known as cross-lingual...
Replication Data for "State Media Control Influences Large Language Models"
Replication dataset for "State Media Control Influences Large Language Models," forthcoming in Nature (https://doi.org/10.1038/s41586-026-10506-7). We show through six studies that government control of the media across the world influences the output of large language models (LLMs) via their training data.
Replication Data for "State Media Control Influences Large Language Models"
Replication dataset for "State Media Control Influences Large Language Models," forthcoming in Nature (https://doi.org/10.1038/s41586-026-10506-7). We show through six studies that government control of the media across the world influences the output of large language models (LLMs) via their training data.
what are large language models actually doing? i read the 2025 textbook "Foundations of Large Language Models" by tong xiao and jingbo zhu and for the first time, i truly understood how they work. here’s everything you need to know about llms in 3 minutes↓
what are large language models actually doing?i read the 2025 textbook "Foundations of Large Language Models" by tong xiao and jingbo zhu and for the first time, i truly understood how they work.here’s everything you need to know about llms in 3 minutes↓ pic.twitter.com/OeYLuU3cWh— Alex Veremeyenko (@alex_verem) July 26, 2025
Wikipedia:LLM-assisted translation
This guideline applies to machine translation tools that include a large language model ("LLM"). Assume that it applies to any online translation tool unless you have confirmed there is no LLM element.
Hypothesis generation and updating in large language models
Large language models (LLMs) increasingly help people solve problems, from debugging code to repairing machinery. This process requires generating plausible hypotheses from partial descriptions,...

Alignment Is Proven To Be Solvable
That LLMs understand natural language as well as they do should dramatically change our understanding of the problem.

ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities
Recent large language models (LLMs) advancements sparked a growing research interest in tool assisted LLMs solving real-world challenges…

Embers of autoregression show how large language models are shaped by the problem they are trained to solve
The widespread adoption of large language models (LLMs) makes it important to recognize their strengths and limitations. We argue that to develop a holistic understanding of these systems, we must consider the problem that they were trained to solve: next-word prediction over Internet text. By recognizing the pressures that this task exerts, we can make predictions about the strategies that LLMs will adopt, allowing us to reason about when they will succeed or fail. Using this approach—which we call the teleological approach—we identify three factors that we hypothesize will influence LLM accuracy: the probability of the task to be performed, the probability of the target output, and the probability of the provided input. To test our predictions, we evaluate five LLMs (GPT-3.5, GPT-4, Claude 3, Llama 3, and Gemini 1.0) on 11 tasks, and we find robust evidence that LLMs are influenced by probability in the hypothesized ways. Many of the experiments reveal surprising failure modes. For instance, GPT-4’s accuracy at decoding a simple cipher is 51% when the output is a high-probability sentence but only 13% when it is low-probability, even though this task is a deterministic one for which probability should not matter. These results show that AI practitioners should be careful about using LLMs in low-probability situations. More broadly, we conclude that we should not evaluate LLMs as if they are humans but should instead treat them as a distinct type of system—one that has been shaped by its own particular set of pressures.
