evilsocket/cake

Planetary-Scale Inference: Building a Distributed Inference Engine for the Public Internet

We are excited to share a preview of our distributed inference stack — engineered for consumer GPUs and the 100ms latencies of the public internet—plus a research roadmap that scales it into a planetary-scale inference engine.

Haggle: Clean-slate networking for mobile devices

Jing Su, James Scott, Pan Hui, Eben Upton, Meng How Lim, Christophe Diot, Jon Crowcroft, Ashvin Goel, Eyal de Lara

Msty Nexus - Inference Gateway

Put one governed inference layer behind your AI tools for model routing, local runtimes, provider keys, usage visibility, and guardrails.

ATOM: AT Protocol Over MoQ Transport

This document specifies how the Authenticated Transfer (AT) Protocol can leverage Media over QUIC Transport (MOQT) for efficient data synchronization across decentralized social networks. The AT Protocol's firehose event stream and repository synchronization mechanisms map naturally to MOQT's publish/subscribe model, enabling scalable relay infrastructure, priority-based delivery, and improved resilience for large-scale social data distribution. This specification addresses the challenges of the current WebSocket- based transport and demonstrates how MOQT's relay architecture, group-based caching, and multiplexed streams provide significant benefits for AT Protocol deployments at scale.

ericcurtin/inferrs

A TurboQuant inference server

An Internet for the KV Cache: Rethinking Classical Infrastructure Boundaries in the LLM Inference Age

LLM inference has become a global-scale, heterogeneous workload spanning agents, retrieval, tool-use, code execution and multi-modal reasoning. These workloads naturally enable context reuse from overlapping inputs, creating a major opportunity to store and reuse the contexts' KV Caches instead of recomputing them. However, model-side advances that shrink the KV Cache and system-side advances that reduce compute, storage, and transfer costs are evolve independently within legacy cloud boundaries. We argue that future inference infrastructure should allow decoupling of compute and KV Cache storage across cloud and datacenters. The network becomes an active distribution channel; bandwidth, latency and pricing directly determines how the KV Cache should be managed. We propose a vision for an Internet for the KV Cache, with KV Cache management working as a content-distribution system. In this view, KV Cache storage and recompute decisions are driven by model, infrastructure, and application metrics, to enable adaptive, content-driven decisions for minimizing latency and cost.

Argmax - Foundation Models On Device

Run private, real-time, and predictable inference workloads directly on users' devices.

Self-hosting a Matrix server for 5 years

Experiences with the Matrix protocol, Matrix Synapse server, bridges, and Element mobile apps.

Gajesh on Twitter / X

TL;DRapple has turn on this switch for everyone to participate in decentralized inferenceppl can rent out their unused compute space and anyone can use this with privacy guarantees https://t.co/LTP4zyjsdt pic.twitter.com/8Dvo7XK8jJ— Gajesh (@gajesh) February 18, 2026

AI proxy: fostering a more open ecosystem - Blog - Braintrust

Introducing Braintrust's latest feature: an AI proxy that lets you use open source models like LLaMa 2 and Mistral, as well as all of OpenAI's and Anthropic's models, behind a single interface with caching, security, and API key management built in.

Mesh-LLM/mesh-llm

Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.

Client-Server Game Architecture - Gabriel Gambetta

Client-Server Game Architecture | Client-Side Prediction and Server Reconciliation | Entity Interpolation | Lag Compensation | Live Demo

NEAR Intents | Simplifying Web3 Interactions

NEAR Intents let users and AI agents define what they want to do—NEAR handles the rest. A new framework for seamless, multichain execution without complexity, powered by solvers and Chain Signatures.

Open-Source Agentic Inference Benchmark | InferenceX

Compare AgentX, InferenceX's long-context, multi-turn coding scenario, with fixed-sequence AI inference across chips and frameworks. Public NVIDIA and AMD runs update when configurations change.