Disaggregating Prefill and Decode: Faster First Tokens, Faster Streams
Combining NVIDIA DGX Spark + Apple Mac Studio for 4x Faster LLM Inference with EXO 1.0
Disaggregating Prefill and Decode: Faster First Tokens, Faster Streams
Alex Cheema on Twitter / X
Excited to collab with @ashxhart on this!Why does prefill/decode disaggregation work on DGX Spark + Mac ?DGX Spark: ~350 TFLOPS@fp4, 128GB@273GB/sM5 Max MacBook Pro: ~70 TFLOPS@fp4, 128GB@614GB/sPrefill is compute-bound. Prefill on Spark runs ~5x faster than M5 Max.Decode… https://t.co/xl9l0ArRhs— Alex Cheema (@alexocheema) August 19, 2026
NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
Thanks to NVIDIA’s early access program, we are thrilled to get our hands on the NVIDIA DGX™ Spark. It’s quite an unconventional system, as NVIDIA rarely releases compact, all-in-one machines that bri...
omlx/docs/experimental/dflash_mlx_integration.md at main · jundot/omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar - jundot/omlx
InferenceMAX™: Open Source Inference Benchmarking
NVIDIA GB200 NVL72, AMD MI355X, Throughput Token per GPU, Latency Tok/s/user, Perf per Dollar, Tokens per Provisioned Megawatt, DeepSeek R1 670B, GPTOSS 120B, Llama3 70B
XiongjieDai/GPU-Benchmarks-on-LLM-Inference
Multiple NVIDIA GPUs or Apple Silicon for Large Language Model Inference?
NVIDIA GPUs Work on macOS Again. The Driver Is a Miracle. The Inference Is Not.
We benchmarked an RTX 3090 over USB4 and profiled every kernel. GPUs use 1.2-1.6% of their memory bandwidth. The bottleneck is the compiler, not the cable.
Open Source AI Inference Benchmark | InferenceX
Compare AI inference performance across GPUs and frameworks. Real benchmarks on NVIDIA GB200, B200, AMD MI355X, and more. Free, open-source, continuously updated.
Exploring LLMs with MLX and the Neural Accelerators in the M5 GPU
Mac with Apple silicon is increasingly popular among AI developers and researchers interested in using their Mac to experiment with the…
jundot/omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
NVIDIA DGX Spark· Ollama Blog
The latest NVIDIA DGX Spark is here! Ollama has partnered with NVIDIA to ensure it runs fast and efficiently out-of-the-box.
Building with Open Models
NVIDIA and Microsoft Reinvent Windows PCs for the Age of Personal AI
RTX Spark — a 1-Petaflop Superchip, the Full CUDA and RTX Ecosystem, and Windows-Native Agents — a New Beginning for Personal Computers News Summary: NVIDIA RTX Spark powers the world’s first Windows PCs purpose-built for personal agents, featuring 1 petaflop of AI performance, industry-leading power efficiency, full-stack NVIDIA AI and graphics technology, and up to 128GB of unified memory. NVIDIA and Microsoft collaborate to deliver a native Windows experience for personal agents, including new security primitives and NVIDIA OpenShell to run agents securely on primary devices. RTX Spark lets creators, AI developers and gamers render ultralarge 90GB+ 3D scenes, edit 12K 4:2:2 video, generate 4K AI videos, run 120B-parameter LLMs with up to 1 million tokens context using agents locally, and play AAA games at 1440p and over 100 frames per second. Adobe is rearchitecting Photoshop and Premiere from the ground up for RTX Spark to deliver 2x faster AI and graphics performance. RTX
videlalvaro/ane-book
Production LLM inference on the Apple Neural Engine — a practitioner's guide, complete with converters, Swift runtimes, and validated model manifests
NVIDIA Levels Up Local AI Agents Across RTX PCs and DGX Spark
Announced at GTC Taipei at COMPUTEX, NVIDIA OpenShell brings secure agents to Windows with 2x inference performance on llama.cpp — plus, Adobe rebuilds its apps with performance and memory enhancements, and Blender adds NVIDIA DLSS 4.5 Ray Reconstruction for NVIDIA RTX Spark.
