Compression and LLMs are trying to solve the exact same problem: predicting what comes next. Learn the fundamentals of compression and how better prediction leads to better shrinkage.
Serving models at scale with vLLM | ngrok blog
Serving LLMs at scale is hard. Learn how inference works and how vLLM uses continuous batching, PagedAttention, prefix caching, and more to keep GPUs busy.
Beyond Model Collapse: Scaling Up with Synthesized Data Requires...
Large Language Models (LLM) are increasingly trained on data generated by other LLM, either because generated text and images become part of the pre-training corpus, or because synthetized data is...
Semantic Compression
An introduction to the idea that code should be approached with a mindset towards compressing it semantically, rather than orienting it around objects.
Prompt caching: 10x cheaper LLM tokens, but how? | ngrok blog
A far more detailed explanation of prompt caching than anyone asked for: how tokens, embeddings, and attention make cached LLM tokens 10x cheaper and faster.
Prompt caching: 10x cheaper LLM tokens, but how? | ngrok blog
A far more detailed explanation of prompt caching than anyone asked for: how tokens, embeddings, and attention make cached LLM tokens 10x cheaper and faster.
Compression Dictionary Transport - HTTP | MDN
Compression Dictionary Transport is a way of using a shared compression dictionary to dramatically reduce the transport size of HTTP responses.
Compression: Crash Course Computer Science #21
Best LLM for Coding 2026 | AI Coding Model Rankings & Benchmarks
Which AI model writes the best code? We rank every major LLM — open and closed source — across SWE-bench, HumanEval, LiveCodeBench, and Terminal-Bench coding benchmarks. Compare the best LLMs for coding, software engineering, and programming.
Large language model
A large language model (LLM) is a neural network trained on a vast amount of text for natural language processing tasks, especially language generation. LLMs can typically generate, summarize, translate, and analyze text in many contexts, and are a foundational technology behind modern chatbots.[1] Biased or inaccurate training data can make an LLM's output less reliable.[2]
The most important thing when working with LLMs
Blog post: The most important thing when working with LLMs by Steve Klabnik
Alex Cheema on Twitter / X
.@karpathy shouted out my work on @exolabs at @ycombinator AI SUS!“we use LLMs similarly to mainframes in the ‘70s - compute is timeshared by having a slice in the batch dimension. models will compress over time, and with this we’ll be able to run more on-device” pic.twitter.com/UKSrquQGAL— Alex Cheema (@alexocheema) June 18, 2025
Announcing Burn-LM (alpha): LLM Inference Engine
We're happy to announce the Burn-LM, an LLM inference engine based on Burn! The goal is to support any large model, LLM, VLM, and others, for inference but also for training (pre-training, post-training, and fine-tuning).
something that has come up fairly recently with LLMs - for coding, specifically - is that it’s become a lot easier to burn stupefying amounts of tokens on stuff very fast, with agents running 24/7 or managing more agents (see: Yegge’s Gas Town) even with low inference costs that adds up in a hurry
Jesse Felder
‘While some cling to the promise of an AI “revolution,” the cost of adoption is proving a stubborn bottleneck. These developments also suggest that the economics of replacing human labor with AI may be more complicated than some early forecasts originally implied.’ fortune.com/2026/05/22/microsoft-ai-cost-…
LLM is a learned distribution p on sequences of tokens. If you just sample a bunch of text from p willy nilly and put it in the training data and train you just get back p. But if you sample a bunch of text from p and throw out whatever's bad and train on what's left then you learn p(x|x is not bad)
