Prompt Caching In Agents | EARENDIL

How prompt caching shapes the cost, latency, tools, and architecture of coding agents, and what Pi does to keep cache behavior visible.

Prompt caching: 10x cheaper LLM tokens, but how? | ngrok blog

A far more detailed explanation of prompt caching than anyone asked for: how tokens, embeddings, and attention make cached LLM tokens 10x cheaper and faster.

The Shape of Memory Benchmarks

Why the familiar memory benchmarks are outdated, how the agent-native work looks today and why design your own.

An Internet for the KV Cache: Rethinking Classical Infrastructure Boundaries in the LLM Inference Age

LLM inference has become a global-scale, heterogeneous workload spanning agents, retrieval, tool-use, code execution and multi-modal reasoning. These workloads naturally enable context reuse from overlapping inputs, creating a major opportunity to store and reuse the contexts' KV Caches instead of recomputing them. However, model-side advances that shrink the KV Cache and system-side advances that reduce compute, storage, and transfer costs are evolve independently within legacy cloud boundaries. We argue that future inference infrastructure should allow decoupling of compute and KV Cache storage across cloud and datacenters. The network becomes an active distribution channel; bandwidth, latency and pricing directly determines how the KV Cache should be managed. We propose a vision for an Internet for the KV Cache, with KV Cache management working as a content-distribution system. In this view, KV Cache storage and recompute decisions are driven by model, infrastructure, and application metrics, to enable adaptive, content-driven decisions for minimizing latency and cost.

shepherd-agents/shepherd

A runtime substrate that turns an agent's execution into a reversible, Git-like trace, so meta-agents can observe, fork, replay, and revert any run. Couples agent and environments in a copy-on-write fork ~5x faster than docker commit, with ~95% KV-cache reuse on replay. Framework built for meta-agents to supervise, optimize, and train other agents

Improving LM Studio's MLX Engine for Agentic Workflows

mlx-engine v1.8.5 dramatically improves performance for repeated, long-context agentic workflows by checkpointing your KV cache.

Michael Livs on Twitter / X

introducing pi-psst 🤫, your agent uses secrets without ever seeing them. no more scrubbing for @badlogicgames!secrets are injected as env vars to bash tool, scrubbed from all tool output, stored in local encrypted vault. the agent knows what's available but never sees a value.… pic.twitter.com/LB4RA5p4wF— Michael Livs (@micLivs) April 6, 2026

Prompt caching: 10x cheaper LLM tokens, but how? | ngrok blog

A far more detailed explanation of prompt caching than anyone asked for.

Agentation

The visual feedback tool for agents.

Agentation

The visual feedback tool for agents.

Skills, forks, and self-surgery: how agent harnesses grow

Claude Code, NanoClaw, and Pi take radically different approaches to harness extensibility. The tradeoff is always safety vs. agent agency.

Track, control, and optimize for AI agents and bots | Known Agents (formerly Dark Visitors)

Use Known Agents to turn the rising wave of AI agents, LLM assistants, and other bots crawling your website into a new growth channel for your business

The irrational effectiveness of the Pi harness

A month of coding with Pi, its small agent loop, and the skills and extensions I've added to it.

Prince Canuma on Twitter / X

Prompt caching with SSD offloading now active and works with all your favourite harnesses: Pi, Hermes, OpenCode, Claude code and more.Here is the performance of Qwen3-VL-4B-Instruct when using our new prompt caching. pic.twitter.com/ikMktUgpqB— Prince Canuma (@Prince_Canuma) May 6, 2026