Jiaxin Wen on Twitter / X

Concept Attractors in LLMs and their Applications

Empirically, we see that for prompts pip_{i}, pjp_{j} in each concept 𝒞\mathcal{C}, there exists a layer ll where:

Understanding LSTM Networks -- colah's blog

Humans don’t start their thinking from scratch every second. As you read this essay, you understand each word based on your understanding of previous words. You don’t throw everything away and start thinking from scratch again. Your thoughts have persistence.

LLM Daydreaming

Proposal & discussion of how default mode networks for LLMs are an example of missing capabilities for search and novelty in contemporary AI systems.

Here’s what’s really going on inside an LLM’s neural network

Anthropic's conceptual mapping helps explain why LLMs behave the way they do.

Brandon Stewart on Twitter / X

1/ New @Nature! We study how powerful institutions shape the information environment for LLMs. Commercial LLM training is opaque, so we trace a path from state-coordinated media -> training data -> model responses. pic.twitter.com/5LdFvzbFaf— Brandon Stewart (@b_m_stewart) May 13, 2026

Language model harnesses are compositional generalizers

Harnesses can lead to compositional generalization: we observe a property in training RLMs, in which similarly structured tasks are viewed as isomorphic and all individual LM calls in the harness become in-distribution.

Why Are LLMs Smart?

A popular way to explain how current LLMs work is to say that “all” they do is predict the next most likely word in a sentence.

elie on Twitter / X

nice pre training work by nous claiming ~2.5x efficiency gains, building on previous research like MTP/SuperBPE. overall intuition is that at each step you want the model to process and predict more tokens https://t.co/K33QtJiF5C pic.twitter.com/hDlSbv1Tg9— elie (@eliebakouch) May 13, 2026

What is LLM Wiki Pattern? Persistent Knowledge with LLM Wikis

FULL CREDIT :Andrej Karpathy

How LLMs Actually Work

A from-the-ground-up walkthrough of how modern LLMs work, from tokens to transformer blocks to the next-token loop

An Empirical Study of Example Forgetting during Deep Neural Network Learning

Inspired by the phenomenon of catastrophic forgetting, we investigate the learning dynamics of neural networks as they train on single classification tasks. Our goal is to understand whether a related phenomenon occurs when data does not undergo a clear distributional shift. We define a “forgetting event” to have occurred when an individual training example transitions […]

Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers

Clément Dumas4, Kit Fraser-Taliente6, Subhash Kantamneni6, Julian Minder3, Euan Ong6, Arnab Sen Sharma5, Daniel Wen1

Training great LLMs entirely from ground up in the wilderness as a startup — Yi Tay

Chronicles of training strong LLMs from scratch in the wild