A 4-Bit Model and a 1-Bit Index

Running NVFP4 Nemotron on a CPU, then mapping every embedding-compression method at matched byte budgets. The two quantizations compose.

Benjamin Marie on Twitter / X

Quantization and Qwen3-VL✔️ AutoRound W4A16 (INT4)✔️ AutoRound NVFP4 (llm compressor format)❌ support by vLLM (tried stable and dev releases)VLMs are now very easy to quantize. And then we can’t use the models with fast inference frameworks like vLLM and SGLang.It…— Benjamin Marie (@bnjmn_marie) November 5, 2025

Benchmarking Tesla GPUs - esologic

Decommissioned enterprise tesla GPUs are widely available and extremely cheap. In this post, I benchmark tesla GPUs to see if they are useful

shady-gang

Home to the Shady IR and Vcc compiler. shady-gang has 17 repositories available. Follow their code on GitHub.

Thireus/GGUF-Tool-Suite

Produce your own Dynamic 3.0 Quants and achieve optimum accuracy & SOTA quantization performance! Input a target size and the toolchain will create a GGUF recipe tuned to your hardware within seconds — flexible model sizing and lowest achievable perplexity/kld for GGUF enthusiasts seeking precise and automated dynamic quant production.

Vaibhav (VB) Srivastav on Twitter / X

🚨 Apple just released FastVLM on Hugging Face - 0.5, 1.5 and 7B real-time VLMs with WebGPU support 🤯> 85x faster and 3.4x smaller than comparable sized VLMs> 7.9x faster TTFT for larger models> designed to output fewer output tokens and reduce encoding time for high… pic.twitter.com/7cPBWdTQw3— Vaibhav (VB) Srivastav (@reach_vb) August 29, 2025

FOSDEM 2022 - LibVF.IO: vGPU & SR-IOV on Consumer GPUs using Nim

I'd like to showcase LibVF.IO's new LIME Runtime feature (Lime Is Mediated Emulation) and do a deep dive on open source vGPU technology in general.

NVlabs/cuda-oxide

cuda-oxide is an experimental Rust-to-CUDA compiler that lets you write (SIMT) GPU kernels in safe(ish), idiomatic Rust. It compiles standard Rust code directly to PTX — no DSLs, no foreign language bindings, just Rust.

ericcurtin/inferrs

A TurboQuant inference server

Demystifying llm-d and vLLM: The race to production

Learn how vLLM and llm-d work together for efficient and scalable large language model (LLM) inference. Discover the benefits of disaggregated scaling, expert-parallel scheduling, and KV cache-aware routing.

Unreal Scoop: Data-Driven CVars

If you use UE5, you might have noticed there’s a new-ish feature called Data-Driven Console Variables, and you might also have noticed that nothing comes up when you Google this. There’s not much to it, but in the interests of having something come up when you Google this, here’s what I’ve figured out about data-driven cvars.

Visualizing GL_NV_shader_sm_builtins

Using GL_NV_shader_sm_builtins to visualize Streaming Multiprocessors and Warps

Anemll on Twitter / X

Up to 3.5x faster LLM inference on Apple Neural Engine:ANE is a Tensor Processing Unit, unlike GPU, it requires fixed-shape tensors. The KV cache size is set at compile time. A fixed 4096 context always runs at the slowest speed, even for short replies.Variable Context: start… pic.twitter.com/a1mLlmMYq7— Anemll (@anemll) February 16, 2026

turbopuffer: fast search on object storage

Inaugural blog post about the development of turbopuffer, a search engine that uses object storage and SSD caching for cost-effective, low latency search. This post describes into the motivation behind its creation, its unique architecture, and how it significantly reduces costs for large-scale vector searches. Discover how turbopuffer is transforming search infrastructure for companies like Cursor and Suno, offering a scalable and reliable solution.