Pie is a programmable serving system for LLM inference.
Artificial
An LLM is a computer program. We should talk about it like a computer program.
Artificial
An LLM is a computer program. We should talk about it like a computer program.
Announcing Burn-LM (alpha): LLM Inference Engine
We're happy to announce the Burn-LM, an LLM inference engine based on Burn! The goal is to support any large model, LLM, VLM, and others, for inference but also for training (pre-training, post-training, and fine-tuning).
Home - NobodyWho
NobodyWho is an inference engine that lets you run LLMs locally on any device
Large language model
A large language model (LLM) is a neural network trained on a vast amount of text for natural language processing tasks, especially language generation. LLMs can typically generate, summarize, translate, and analyze text in many contexts, and are a foundational technology behind modern chatbots.[1] Biased or inaccurate training data can make an LLM's output less reliable.[2]
Overview - GroqDocs
Fast LLM inference, OpenAI-compatible. Simple to integrate, easy to scale. Start building in minutes.
llama.cpp/tools/server at master · ggml-org/llama.cpp
LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an account on GitHub.
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
jundot/omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
Demystifying llm-d and vLLM: The race to production
Learn how vLLM and llm-d work together for efficient and scalable large language model (LLM) inference. Discover the benefits of disaggregated scaling, expert-parallel scheduling, and KV cache-aware routing.
Solving a Million-Step LLM Task with Zero Errors
LLMs have achieved remarkable breakthroughs in reasoning, insights, and tool use, but chaining these abilities into extended processes at the scale of those routinely executed by humans,...
Andrej Karpathy on Twitter / X
The race for LLM "cognitive core" - a few billion param model that maximally sacrifices encyclopedic knowledge for capability. It lives always-on and by default on every computer as the kernel of LLM personal computing.Its features are slowly crystalizing:- Natively multimodal… https://t.co/2jsVevkTSJ— Andrej Karpathy (@karpathy) June 27, 2025
LLMs CAN'T DO ANYTHING
Can't count the r's in strawberry. Can't tell time. Can't taste a croissant. An exhaustive catalog of machine incapability.
Can LLMs Be Computers? | Percepta
We build a computer inside a transformer — executing arbitrary C programs for millions of steps with exponentially faster inference via 2D attention heads.
