Disaggregating Prefill and Decode: Faster First Tokens, Faster Streams
Combining NVIDIA DGX Spark + Apple Mac Studio for 4x Faster LLM Inference with EXO 1.0
Disaggregating Prefill and Decode: Faster First Tokens, Faster Streams
Alex Cheema on Twitter / X
Excited to collab with @ashxhart on this!Why does prefill/decode disaggregation work on DGX Spark + Mac ?DGX Spark: ~350 TFLOPS@fp4, 128GB@273GB/sM5 Max MacBook Pro: ~70 TFLOPS@fp4, 128GB@614GB/sPrefill is compute-bound. Prefill on Spark runs ~5x faster than M5 Max.Decode… https://t.co/xl9l0ArRhs— Alex Cheema (@alexocheema) August 19, 2026
oMLX — LLM inference, optimized for your Mac
Native macOS inference server built on MLX with paged SSD KV caching for Apple Silicon.
NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
Thanks to NVIDIA’s early access program, we are thrilled to get our hands on the NVIDIA DGX™ Spark. It’s quite an unconventional system, as NVIDIA rarely releases compact, all-in-one machines that bri...
omlx/docs/experimental/dflash_mlx_integration.md at main · jundot/omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar - jundot/omlx
InferenceMAX™: Open Source Inference Benchmarking
NVIDIA GB200 NVL72, AMD MI355X, Throughput Token per GPU, Latency Tok/s/user, Perf per Dollar, Tokens per Provisioned Megawatt, DeepSeek R1 670B, GPTOSS 120B, Llama3 70B
XiongjieDai/GPU-Benchmarks-on-LLM-Inference
Multiple NVIDIA GPUs or Apple Silicon for Large Language Model Inference?
Optimizing On-Device Inference for Apple Silicon
A custom local engine that improves prefill and decode throughput
NVIDIA GPUs Work on macOS Again. The Driver Is a Miracle. The Inference Is Not.
We benchmarked an RTX 3090 over USB4 and profiled every kernel. GPUs use 1.2-1.6% of their memory bandwidth. The bottleneck is the compiler, not the cable.
Open Source AI Inference Benchmark | InferenceX
Compare AI inference performance across GPUs and frameworks. Real benchmarks on NVIDIA GB200, B200, AMD MI355X, and more. Free, open-source, continuously updated.
Exploring LLMs with MLX and the Neural Accelerators in the M5 GPU
Mac with Apple silicon is increasingly popular among AI developers and researchers interested in using their Mac to experiment with the…
For dual DGX spark users; GLM 5.3 flash got a 50%+ performance boost
For the last few months, I ran DeepSeek v4.0 flash (NVFP4). First 0731, then visionexp because it was a free improvement. I got around 65 tps decode…
jundot/omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
NVIDIA DGX Spark· Ollama Blog
The latest NVIDIA DGX Spark is here! Ollama has partnered with NVIDIA to ensure it runs fast and efficiently out-of-the-box.
Building with Open Models