something that has come up fairly recently with LLMs - for coding, specifically - is that it’s become a lot easier to burn stupefying amounts of tokens on stuff very fast, with agents running 24/7 or managing more agents (see: Yegge’s Gas Town) even with low inference costs that adds up in a hurry
Jesse Felder
‘While some cling to the promise of an AI “revolution,” the cost of adoption is proving a stubborn bottleneck. These developments also suggest that the economics of replacing human labor with AI may be more complicated than some early forecasts originally implied.’ fortune.com/2026/05/22/microsoft-ai-cost-…
May 22, 2026 at 9:07 PM
Prompt caching: 10x cheaper LLM tokens, but how? | ngrok blog
A far more detailed explanation of prompt caching than anyone asked for: how tokens, embeddings, and attention make cached LLM tokens 10x cheaper and faster.
Lincoln Michel on Twitter / X
This was my read on AI from the beginning and it proved very true. Whatever hypothetical benefits LLMs have (and they have some), their main use is to waste everyone's time wading through slop. Longer colleague powerpoints. More spam emails. More slop clogging up all the systems. https://t.co/2gtGfqf7KF— Lincoln Michel (@TheLincoln) September 21, 2026
Andrej Karpathy on Twitter / X
LLM Knowledge BasesSomething I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating…— Andrej Karpathy (@karpathy) April 2, 2026
Prompt caching: 10x cheaper LLM tokens, but how? | ngrok blog
A far more detailed explanation of prompt caching than anyone asked for.
The Bitter Lesson of LLM Extensions
From ChatGPT Plugins to Agent Skills, a look at how we've been trying (and failing) to extend LLMs for the last three years.
Tokenization Tax 2026: LLM Tokenizer Comparison | ellamind
A measurement study of 20 LLM tokenizers across 12 languages and 21 kinds of text. On German, Claude 4.7+ needs 2.01x the tokens of the OpenAI reference, and Cohere Command A+ the fewest.
truefoundry/trueforge
The open-source agent harness - the runtime layer that turns an LLM into a working agent.
Alex Cheema on Twitter / X
.@karpathy shouted out my work on @exolabs at @ycombinator AI SUS!“we use LLMs similarly to mainframes in the ‘70s - compute is timeshared by having a slice in the batch dimension. models will compress over time, and with this we’ll be able to run more on-device” pic.twitter.com/UKSrquQGAL— Alex Cheema (@alexocheema) June 18, 2025
State of AI 2025: 100T Token LLM Usage Study | OpenRouter
Read OpenRouter's 2025 State of AI report — an empirical 100 trillion token study of real LLM usage, model trends, and developer insights.
Stevens: a hackable AI assistant using a single SQLite table and a handful of cron jobs
There’s a lot of hype these days around patterns for building with AI. Agents, memory, RAG, assistants—so many buzzwords! But the reality is, you don’t need fancy techniques or libraries to build useful personal tools with LLMs.
How LLMs Actually Work
A from-the-ground-up walkthrough of how modern LLMs work, from tokens to transformer blocks to the next-token loop
Agent Recipes
Explore common agent recipes with ready to copy code to improve your LLM applications.
i can’t help but think we’re far from nailing memory systems this one here is extremely interesting. two LLMs at once, one just managing and surfacing memory for the other
Asa ⭔
I'm not a fan of the decoupled 'memory retrieval → task execution' loop, so my agent has a subconscious background thread that looks for relevant, unique memory context in its experiential database while it runs and injects it on top of the live context window.
I feel like using LLMs to flag intent/semantics mismatches (eg do var names seem to match what you actually do) or indirect violations of API contracts or whatever as a supplement to static analysis to produce *better* code would be at least as high impact, and it's like 1% of the discourse
