Running NVFP4 Nemotron on a CPU, then mapping every embedding-compression method at matched byte budgets. The two quantizations compose.
Quantization from the ground up | ngrok blog
A complete guide to what quantization is, how it works, and how it's used to compress large language models
Benjamin Marie on Twitter / X
Quantization and Qwen3-VL✔️ AutoRound W4A16 (INT4)✔️ AutoRound NVFP4 (llm compressor format)❌ support by vLLM (tried stable and dev releases)VLMs are now very easy to quantize. And then we can’t use the models with fast inference frameworks like vLLM and SGLang.It…— Benjamin Marie (@bnjmn_marie) November 5, 2025
Quantizer support in Mediabunny v1.52.0 | Mediabunny
Mediabunny v1.52.0 adds quantizer-based video encoding for AVC, HEVC, VP9, and AV1, enabling constant-quality video encoding.
Better bitmap performance with Roaring bitmaps
Bitmap indexes are commonly used in databases and search engines. By exploiting bit-level parallelism, they can significantly accelerate queries. However, they can use much memory, and thus we might prefer compressed bitmap indexes. Following Oracle's lead, bitmaps are often compressed using run-length encoding (RLE). Building on prior work, we introduce the Roaring compressed bitmap format: it uses packed arrays for compression instead of RLE. We compare it to two high-performance RLE-based bitmap encoding techniques: WAH (Word Aligned Hybrid compression scheme) and Concise (Compressed `n' Composable Integer Set). On synthetic and real data, we find that Roaring bitmaps (1) often compress significantly better (e.g., 2 times) and (2) are faster than the compressed alternatives (up to 900 times faster for intersections). Our results challenge the view that RLE-based bitmap compression is best.
Reverse-engineering GGUF | Post-Training Quantization
I built a 7-node ESP32-S3 cluster to run a ~0.4B LLM using 1.58-Bit (BitNet) ternary quantization...
1.1K votes, 64 comments. I build a 7-node ESP32-S3 cluster to run a ~0.4B LLM using 1.58-bit quantization. This builds on the idea of distributing…
BytePlus Free Trial: 4K AI Image/Video Models + LLM Tokens | 200 Free Images
BytePlus free trial, Seedream 4.0, Seedance 1.0, DeepSeek V3.1, GPT-OSS-120B, free 4K AI images, AI video generation, BytePlus LLM tokens
The Kaitchup Index: A Leaderboard for LLMs and Their Quantized Versions
Comparing formats like GGUF, GPTQ, and AWQ, with different bitwidths
TurboQuant: Redefining AI efficiency with extreme compression
Amir Zandieh, Research Scientist, and Vahab Mirrokni, VP and Google Fellow, Google Research
What’s MXFP4? The 4-Bit Secret Powering OpenAI’s GPT‑OSS Models on Modest Hardware
A Blog post by Rakshit Aralimatti on Hugging Face
Compression Dictionary Transport - HTTP | MDN
Compression Dictionary Transport is a way of using a shared compression dictionary to dramatically reduce the transport size of HTTP responses.
I built ChatGPT with Minecraft redstone!
Semantic Compression
An introduction to the idea that code should be approached with a mindset towards compressing it semantically, rather than orienting it around objects.
A Visual Guide to Quantization
Exploring memory-efficient techniques for LLMs
Google's TurboQuant landed for vLLM.cpp's KV cache – but it's not just useful for KV! A possibly more impactful implementation is for quantizing embedding vectors for RAG. github.com/oaustegard/remex is a Python and Mojo for just that purpose.
GitHub - oaustegard/remex: Retrieval-validated embedding compression. 4-8x smaller vectors, proven recall. Based on TurboQuant (ICLR 2026).
github.com