







The Methods Behind How LLMs Represent Token Positions
How LLMs Actually Work
A from-the-ground-up walkthrough of how modern LLMs work, from tokens to transformer blocks to the next-token loop
I Built an LLM From Scratch
Rotary Positional Embeddings: A Detailed Look and Comprehensive Understanding
Since the “Attention Is All You Need” paper in 2017, the Transformer architecture has been a cornerstone in the realm of Natural Language…

Things we learned about LLMs in 2024
A lot has happened in the world of Large Language Models over the course of 2024. Here’s a review of things we figured out about the field in the past …

Alex Veremeyenko on Twitter / X
what are large language models actually doing?i read the 2025 textbook "Foundations of Large Language Models" by tong xiao and jingbo zhu and for the first time, i truly understood how they work.here’s everything you need to know about llms in 3 minutes↓ pic.twitter.com/OeYLuU3cWh— Alex Veremeyenko (@alex_verem) July 26, 2025
Embeddings: What they are and why they matter
Meandering on Manifolds: The Neural Geometry of Stories Over Time
To fully understand LLM representations, we must understand how they change dynamically, over the course of a prompt or conversation. We investigate these temporal dynamics with a simple case study: how do LLMs represent human emotions while reading short stories, both geometrically (in activation space) and temporally (changing from sentence to sentence)?

Meandering on Manifolds: The Neural Geometry of Stories Over Time
To fully understand LLM representations, we must understand how they change dynamically, over the course of a prompt or conversation. We investigate these temporal dynamics with a simple case study: how do LLMs represent human emotions while reading short stories, both geometrically (in activation space) and temporally (changing from sentence to sentence)?


what are large language models actually doing? i read the 2025 textbook "Foundations of Large Language Models" by tong xiao and jingbo zhu and for the first time, i truly understood how they work. here’s everything you need to know about llms in 3 minutes↓
what are large language models actually doing?i read the 2025 textbook "Foundations of Large Language Models" by tong xiao and jingbo zhu and for the first time, i truly understood how they work.here’s everything you need to know about llms in 3 minutes↓ pic.twitter.com/OeYLuU3cWh— Alex Veremeyenko (@alex_verem) July 26, 2025
Understanding Multimodal LLMs
An introduction to the main techniques and latest models

A Visual Guide to Quantization
Exploring memory-efficient techniques for LLMs

Alignment Is Proven To Be Solvable
That LLMs understand natural language as well as they do should dramatically change our understanding of the problem.

Embeddings: What they are and why they matter
Embeddings are a really neat trick that often come wrapped in a pile of intimidating jargon. If you can make it through that jargon, they unlock powerful and exciting techniques …

1/4 Do LLMs understand? "They understand in a way that’s very different from how humans understand," Dileep George, @dileeplearning.bsky.social, of Google DeepMind at the Simons Institute workshop on The Future of Language Models and Transformers. Video: simons.berkeley.edu/talks/dileep-george-google-de…
LLM is a learned distribution p on sequences of tokens. If you just sample a bunch of text from p willy nilly and put it in the training data and train you just get back p. But if you sample a bunch of text from p and throw out whatever's bad and train on what's left then you learn p(x|x is not bad)