Vary support is now available in Cache Rules on every plan. You can normalize known negotiation headers, pass exact values through to the origin when those small differences matter, or bypass cache when the variation is too unpredictable.
DiffsHub, from Pierre
View code changes from any public GitHub diff or patch URL with a super-freaking-fast, beautiful, and virtualized interface.
An Internet for the KV Cache: Rethinking Classical Infrastructure Boundaries in the LLM Inference Age
LLM inference has become a global-scale, heterogeneous workload spanning agents, retrieval, tool-use, code execution and multi-modal reasoning. These workloads naturally enable context reuse from overlapping inputs, creating a major opportunity to store and reuse the contexts' KV Caches instead of recomputing them. However, model-side advances that shrink the KV Cache and system-side advances that reduce compute, storage, and transfer costs are evolve independently within legacy cloud boundaries. We argue that future inference infrastructure should allow decoupling of compute and KV Cache storage across cloud and datacenters. The network becomes an active distribution channel; bandwidth, latency and pricing directly determines how the KV Cache should be managed. We propose a vision for an Internet for the KV Cache, with KV Cache management working as a content-distribution system. In this view, KV Cache storage and recompute decisions are driven by model, infrastructure, and application metrics, to enable adaptive, content-driven decisions for minimizing latency and cost.
Reorganizing Website Architecture for HTTP/2 and Beyond
This document discusses reorganizing website architecture for HTTP/2 and beyond. It summarizes some issues with HTTP/2 including errors in prioritization where some browsers fail to specify resource priority properly. It also discusses the problem of TCP head-of-line blocking where pending data in TCP buffers can delay higher priority resources. The document proposes solutions to these issues such as prioritizing resources on the server-side and writing only what can be sent immediately to avoid buffer blocking. It also examines the mixed success of HTTP/2 push and argues the server should not push already cached resources. - Download as a PDF or view online for free
Differ - Adaptive Software
Build software that adapts to each user — without forks or feature flags. One codebase, infinite variants.
Differ - Adaptive Software
Build software that adapts to each user — without forks or feature flags. One codebase, infinite variants.
Provider Variance: Introducing Exacto — OpenRouter Blog
Does the same model perform differently across different providers?
Prompt caching: 10x cheaper LLM tokens, but how? | ngrok blog
A far more detailed explanation of prompt caching than anyone asked for: how tokens, embeddings, and attention make cached LLM tokens 10x cheaper and faster.
Prompt caching: 10x cheaper LLM tokens, but how? | ngrok blog
A far more detailed explanation of prompt caching than anyone asked for.
On forking the Web
This document contains a set of informal notes on how to build an alternative specification to the Web, in such a way that hopefully prevents many of its drawbacks while still preserves many of the good points. This document is not an specification, and is therefore subject to change over time.
schalkneethling/css-benchpress
css-benchpress aims to help developers and tooling authors discover where CSS patterns become performance problems by growing realistic web-platform test cases until measurable regressions appear.
Web Tiles
Trust has been the defining constraint on the Web's evolution towards more powerful, more applicative capabilities. In a Web context, the user must be able to safely load any arbitrary URL, to safely click on any arbitrary link. The way in which this is achieved is that the runtime places strict limits on what a Web page can do, which in turn necessarily limits powerful capabilities. Could we get more power using a primitive that places more stringent constraints in what pages can do?
Web Tiles
Trust has been the defining constraint on the Web's evolution towards more powerful, more applicative capabilities. In a Web context, the user must be able to safely load any arbitrary URL, to safely click on any arbitrary link. The way in which this is achieved is that the runtime places strict limits on what a Web page can do, which in turn necessarily limits powerful capabilities. Could we get more power using a primitive that places more stringent constraints in what pages can do?
User-Agent header - HTTP | MDN
The HTTP User-Agent request header is a characteristic string that lets servers and network peers identify the application, operating system, vendor, and/or version of the requesting user agent.
I feel like using LLMs to flag intent/semantics mismatches (eg do var names seem to match what you actually do) or indirect violations of API contracts or whatever as a supplement to static analysis to produce *better* code would be at least as high impact, and it's like 1% of the discourse
@dholms.at can we fix that? iirc the rate limit headers RFC allows them on all responses? (Though obviously caching GET at CDN might skew things)
alice
i love services that are rate limited but also do not publish their rate limits in headers i'm looking at you plc.directory
