Announcing Burn-LM (alpha): LLM Inference Engine

EricLBuehler/mistral.rs

Fast, flexible LLM inference

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Dria on Twitter / X

Introducing Inference Arena v2.0.An agentic experience that searches, analyzes, and delivers insights about LLM inference.When we first launched, our goal was simple: make it easier for developers to compare models, engines, and hardware without digging through scattered… pic.twitter.com/fgWgos48lW— Dria (@driaforall) September 30, 2025

leobeeson/llm_benchmarks

A collection of benchmarks and datasets for evaluating LLM.

I Think I Have LLM Burnout

I didn't expect to get so tired of reading LLM output.

Overview - GroqDocs

Fast LLM inference, OpenAI-compatible. Simple to integrate, easy to scale. Start building in minutes.

LLM in a Flash: Efficient Large Language Model Inference with Limited Memory

Large language models (LLMs) are central to modern natural language processing, delivering exceptional performance in various tasks…

Demystifying llm-d and vLLM: The race to production

Learn how vLLM and llm-d work together for efficient and scalable large language model (LLM) inference. Discover the benefits of disaggregated scaling, expert-parallel scheduling, and KV cache-aware routing.

llm-datasets

Curated list of datasets and tools for post-training.

Alex Cheema on Twitter / X

.@karpathy shouted out my work on @exolabs at @ycombinator AI SUS!“we use LLMs similarly to mainframes in the ‘70s - compute is timeshared by having a slice in the batch dimension. models will compress over time, and with this we’ll be able to run more on-device” pic.twitter.com/UKSrquQGAL— Alex Cheema (@alexocheema) June 18, 2025

OpenAI and Broadcom unveil LLM-optimized inference chip

OpenAI and Broadcom introduce Jalapeño, a custom AI chip built for LLM inference to improve performance, efficiency, and scale across AI systems.

Best LLM for Coding 2026 | AI Coding Model Rankings & Benchmarks

Which AI model writes the best code? We rank every major LLM — open and closed source — across SWE-bench, HumanEval, LiveCodeBench, and Terminal-Bench coding benchmarks. Compare the best LLMs for coding, software engineering, and programming.

predibase/lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

The State of On-Device LLMs

Xuan-Son Nguyen, an engineer at Hugging Face, specializes in on-device large language models (LLMs) and runtime optimization, working extensively with llam...

The State of On-Device LLMs

Xuan-Son Nguyen, an engineer at Hugging Face, specializes in on-device large language models (LLMs) and runtime optimization, working extensively with llam...

Patrick Collison on Twitter / X

I want some kind of LLM workflow tool.• Ability to manage a set of input files (Markdown or similar), plus other general-purpose context.• With real-time collaboration. (And maybe some concept of snapshots or VCS integration.)• And the ability to create/manage a inference…— Patrick Collison (@patrickc) June 6, 2026