Prompt caching: 10x cheaper LLM tokens, but how? | ngrok blog

A far more detailed explanation of prompt caching than anyone asked for: how tokens, embeddings, and attention make cached LLM tokens 10x cheaper and faster.

I Think I Have LLM Burnout

I didn't expect to get so tired of reading LLM output.

The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities (Version 1.0)

Prompt caching: 10x cheaper LLM tokens, but how? | ngrok blog

A far more detailed explanation of prompt caching than anyone asked for.

LLMs and performative productivity

It's worth asking whether LLMs are actually making us more productive at all—and if so, what we might be sacrificing in return.

Alex Cheema on Twitter / X

.@karpathy shouted out my work on @exolabs at @ycombinator AI SUS!“we use LLMs similarly to mainframes in the ‘70s - compute is timeshared by having a slice in the batch dimension. models will compress over time, and with this we’ll be able to run more on-device” pic.twitter.com/UKSrquQGAL— Alex Cheema (@alexocheema) June 18, 2025

Fast and Hard Code

People pick languages differently with LLMs.

The Bitter Lesson of LLM Extensions

From ChatGPT Plugins to Agent Skills, a look at how we've been trying (and failing) to extend LLMs for the last three years.

Dan Shipper 📧 on Twitter / X

this is true and is a big reason why you don’t need to be a highly technical researcher to use LLMs in surprising and novel ways https://t.co/TuxNzXzToU— Dan Shipper 📧 (@danshipper) July 27, 2025

Announcing Burn-LM (alpha): LLM Inference Engine

We're happy to announce the Burn-LM, an LLM inference engine based on Burn! The goal is to support any large model, LLM, VLM, and others, for inference but also for training (pre-training, post-training, and fine-tuning).

The scientific case for being nice to your chatbot

New research confirms that LLMs often perform better when you encourage them. But why?

Xuhui Zhou on Twitter / X

Creating user simulators is a key to evaluating and training models for user-facing agentic applications. But are stronger LLMs better user simulators?TL;DR: not really.We ran the largest sim2real study for AI agents to date: 31 LLM simulators vs. 451 real humans across 165… pic.twitter.com/SkswSzeBrz— Xuhui Zhou (@nlpxuhui) March 19, 2026

LLMs Can Now Write GPU Kernels That Beat torch.compile - Break AI Scaling Limits in 7 Days

We're now seeing multi-agent systems that take your PyTorch code and produce CUDA or Triton kernels with 2x to 14x speedups over torch.compile(mode='max-autotune-no-cudagraphs'). Not on toy benchmarks. On real models like Llama-3.1-8B, Whisper, and Stable Diffusion. Learn proven techniques to shift the scaling law intercept and achieve 10-50% performance gains.

How LLMs Actually Work

A from-the-ground-up walkthrough of how modern LLMs work, from tokens to transformer blocks to the next-token loop