Andrej Karpathy on Twitter / X

Stanford CS336 | Language Modeling from Scratch (Spring 2025 Archive)

Archived course website for Stanford CS336: Language Modeling from Scratch (Spring 2025), including schedule, assignments, logistics, and materials.

First Languages AI Reality | Mila

The FLAIR initiative aims to develop a method for the rapid creation of customized ASR models for Indigenous languages.

An Intuitive Introduction to Flow-Based Language Generation — Floor Eijkelboom

Why autoregressive models have a structural ceiling, why discrete-diffusion LLMs are still autoregression in disguise, and what flow-based language models actually give us.

The Kaitchup – AI on a Budget | Benjamin Marie | Substack

Weekly tutorials and news on adapting large language models (LLMs) to your tasks and hardware using the most recent techniques and models. The Kaitchup proposes a collection of 180+ AI notebooks regularly updated. Click to read The Kaitchup – AI on a Budget, by Benjamin Marie, a Substack publication with tens of thousands of subscribers.

Build A Large Language Model (From Scratch), Published by Manning, ISBN 978-1633437166

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Transformer Explainer: LLM Transformer Model Visually Explained

An interactive visualization tool showing you how transformer models work in large language models (LLM) like GPT.

Beyond Model Collapse: Scaling Up with Synthesized Data Requires...

Large Language Models (LLM) are increasingly trained on data generated by other LLM, either because generated text and images become part of the pre-training corpus, or because synthetized data is...

Understanding Understanding: A Pragmatic Framework Motivated by...

Motivated by the rapid ascent of Large Language Models (LLMs) and debates about the extent to which they possess human-level qualities, we propose a framework for testing whether any agent (be it...

Circuit Tracing: Revealing Computational Graphs in Language Models

We describe an approach to tracing the “step-by-step” computation involved when a model responds to a single prompt.

Olmo from Ai2

Our fully open language model and complete model flow.

Hypothesis generation and updating in large language models

Large language models (LLMs) increasingly help people solve problems, from debugging code to repairing machinery. This process requires generating plausible hypotheses from partial descriptions,...

Introducing Mu language model and how it enabled the agent in Windows Settings

We are excited to introduce our newest on-device small language model, Mu. This model addresses scenarios that require inferring complex input-output relationships and has been designed to operate efficiently, delivering high performance while runnin