Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers

Solving a Million-Step LLM Task with Zero Errors

LLMs have achieved remarkable breakthroughs in reasoning, insights, and tool use, but chaining these abilities into extended processes at the scale of those routinely executed by humans,...

Understanding Reasoning LLMs

Methods and Strategies for Building and Refining Reasoning Models

LLM-generated skills work, if you generate them afterwards

LLM “skills” are a short explanatory prompt for a particular task, typically bundled with helper scripts. A recent paper showed that while skills are useful to LLMs, LLM-authored skills are not. From the abstract:

The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities (Version 1.0)

Take caution in using LLMs as human surrogates | PNAS

Recent studies suggest large language models (LLMs) can generate human-like responses, aligning with human behavior in economic experiments, survey...

Why do LLMs make stuff up? New research peers under the hood.

Claude's faulty "known entity" neurons sometimes override its "don't answer" circuitry.

Stevens: a hackable AI assistant using a single SQLite table and a handful of cron jobs

There’s a lot of hype these days around patterns for building with AI. Agents, memory, RAG, assistants—so many buzzwords! But the reality is, you don’t need fancy techniques or libraries to build useful personal tools with LLMs.

Nebula AI - Memory Layer for AI Agents

Nebula is the AI memory layer turning every interaction into conceptual knowledge that continuously improves LLM applications.

Meet the Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases

Meet the Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases

LLM Knowledge Bases

A visual breakdown of Andrej Karpathy's approach to building personal knowledge bases powered by LLMs. Learn the 4-phase pipeline: ingest, compile, query, and maintain - with an interactive architecture diagram.

Build a Reasoning Model (From Scratch)

"An exceptional deep dive into the next frontier of AI.” —Aman Chadha, Google Build a Reasoning Model (From Scratch) is a practical guide to understanding how modern reasoning-oriented LLMs work by building their core methods step by step. The book tells a clear engineering story: start with a conventional pre-trained LLM, learn how text generation works, build reliable evaluation tools, improve reasoning through inference-time methods, then move into training-based approaches such as reinforcement learning and distillation. The progression is deliberate. Early chapters establish the baseline model and explain text generation, KV caching, and evaluation with math verifiers. The middle chapters show how reasoning can be improved without changing model weights, using chain-of-thought prompting, sampling, self-consistency, response scoring, and self-refinement. Later chapters move to changing the model itself through reinforcement learning with verifiable rewards, GRPO improvements, format rewards, and finally distillation from stronger reasoning models into smaller ones. The book is especially useful because it implements the core methods from scratch rather than treating them as black-box library calls. Readers see how self-consistency, self-refinement, Best-of-N, and training-based methods actually work, including their cost and latency trade-offs. It also discusses common failure modes, including cases where refinement can make answers worse. Difficult concepts such as softmax, temperature, and top-p sampling are clarified with code-linked explanations and diagrams, and visual workflows make pipelines and scoring methods easier to follow. Reading the book feels like following a guided technical build rather than a loose survey of AI topics. Each concept is introduced because the project now needs it. Diagrams, roadmaps, code listings, exercises, and repeated workflow summaries help readers stay oriented through advanced material. This structure reflects Sebastian Raschka’s professional strength: explaining complex machine learning topics by making every detail concrete and showing exactly where each section fits in the larger story. He does not treat mechanisms like evaluation, log-probabilities, KL regularization, or distillation as isolated abstractions; he connects them to the goal of making reasoning models understandable and implementable. Physically and organizationally, the book has eight chapters and seven substantial appendixes. That design keeps the main narrative focused while moving supporting material like references, exercise solutions, model source code, larger models, batching, evaluation alternatives, and chat interfaces into ordered appendixes. The result is a logically flowing book that remains hands-on, navigable, and technically deep without constantly interrupting the central build.

Here’s what’s really going on inside an LLM’s neural network

Anthropic's conceptual mapping helps explain why LLMs behave the way they do.

The rebel alliance

This blog is co-authored with Zoe Weinberg and Matt Hawes at ex/ante, and is a follow-up to our first blog post on the topic, 'You don't own your memory.' We need an open architecture that puts us in control of our memories while making their exploitation technically impossible. But how will this shift happen? In order to discover possible implementations, we must understand how our data informs LLMs. The three predominant context engineering techniques are prompt design, retrieval-augmented ...

Why Are LLMs Smart?

A popular way to explain how current LLMs work is to say that “all” they do is predict the next most likely word in a sentence.

Why Can’t Powerful LLMs Learn Multiplication?

These days, large language models (LLMs) can handle increasingly complex tasks, writing complex code and engaging in sophisticated reasoning. But when it comes to 4-digit multiplication, a task taught in elementary school, even state-of-the-art systems fail. Why? A new paper by Computer Science PhD student Xiaoyan Bai and Faculty Co-Director of the Data Science Institute’s …