LM Studio 0.3.10: 🔮 Speculative Decoding

tanishqkumar/ssd

A lightweight inference engine supporting speculative speculative decoding (SSD).

MTPLX

2.24x decode TPS increase On Qwen 3.6 27B @ temp 0.6 | Native MTP Speculative Decoding On Apple Silicon With No External Drafter.

Luce-Org/lucebox-hub

Fast LLM speculative inference server for consumer hardware.

ggml-org/llama.cpp

LLM inference in C/C++

ggml-org/llama.cpp

LLM inference in C/C++

Julie Kallini ✈️ ICML✨ on Twitter / X

Fast Byte Latent Transformer is accepted to ICML 2026! ⚡🥪Byte-level LMs promise to free us from subword tokenizers, but decoding one byte at a time is super slow.We make BLT generation more efficient with BLT-D: text diffusion for parallel byte decoding. 1/ pic.twitter.com/ZIvUgavXvt— Julie Kallini ✈️ ICML✨ (@JulieKallini) May 11, 2026

EricLBuehler/mistral.rs

Fast, flexible LLM inference

LLM in a Flash: Efficient Large Language Model Inference with Limited Memory

Large language models (LLMs) are central to modern natural language processing, delivering exceptional performance in various tasks…

ml-explore/mlx-lm

Run LLMs with MLX

Zhijian Liu on Twitter / X

🔥 DFlash x MLX is happening!Shoutout to @aryagm01 for the early work on this. We're building on the momentum. Native MLX support, more models (Qwen3.5), up to 4x faster. Lossless!👉 https://t.co/9CtLKDptNI pic.twitter.com/pMdaCH4oKi— Zhijian Liu (@zhijianliu_) April 15, 2026

Cerebras Launches World Fastest DeepSeek R1 Llama-70B Inference - Cerebras

Cerebras launches fastest DeepSeek

MemGPT (MemGPT)

Large Language Models (LLMs)

Defeating Nondeterminism in LLM Inference

Reproducibility is a bedrock of scientific progress. However, it’s remarkably difficult to get reproducible results out of large language models. For example, you might observe that asking ChatGPT the same question multiple times provides different results. This by itself is not surprising, since getting a result from a language model involves “sampling”, a process that converts the language model’s output into a probability distribution and probabilistically selects a token. What might be more surprising is that even when we adjust the temperature down to 0This means that the LLM always chooses the highest probability token, which is called greedy sampling. (thus making the sampling theoretically deterministic), LLM APIs are still not deterministic in practice (see past discussions here, here, or here). Even when running inference on your own hardware with an OSS inference library like vLLM or SGLang, sampling still isn’t deterministic (see here or here).

Demystifying llm-d and vLLM: The race to production

Learn how vLLM and llm-d work together for efficient and scalable large language model (LLM) inference. Discover the benefits of disaggregated scaling, expert-parallel scheduling, and KV cache-aware routing.

llama.cpp/docs/build.md at master · ggml-org/llama.cpp

LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an account on GitHub.

youngharold/tightwad

Mixed-vendor GPU inference cluster manager with speculative decoding