A complete guide to what quantization is, how it works, and how it's used to compress large language models
Reverse-engineering GGUF | Post-Training Quantization
AI-Efficiency/Awesome-Model-Quantization
A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.
Benjamin Marie on Twitter / X
Quantization and Qwen3-VL✔️ AutoRound W4A16 (INT4)✔️ AutoRound NVFP4 (llm compressor format)❌ support by vLLM (tried stable and dev releases)VLMs are now very easy to quantize. And then we can’t use the models with fast inference frameworks like vLLM and SGLang.It…— Benjamin Marie (@bnjmn_marie) November 5, 2025
A 4-Bit Model and a 1-Bit Index
Running NVFP4 Nemotron on a CPU, then mapping every embedding-compression method at matched byte budgets. The two quantizations compose.
How Large Language Models Actually Work
Semantic Compression
An introduction to the idea that code should be approached with a mindset towards compressing it semantically, rather than orienting it around objects.
Beyond Model Collapse: Scaling Up with Synthesized Data Requires...
Large Language Models (LLM) are increasingly trained on data generated by other LLM, either because generated text and images become part of the pre-training corpus, or because synthetized data is...
An Observation on Generalization
Tokenization: A Survey for Modern NLP
While modern language models take raw text as their input and produce raw text as output, they do not operate over text directly. Hidden in the very first step of language model pipelines is...
Caveman | Lithic Token Compression
A semantic constraint engine for Claude Code & Codex. Forces agentic communication into minimal-token lithic structures. Retain 100% technical accuracy while destroying up to 87% of output latency.
Generalization Dynamics of LM Pre-training
Generalization Dynamics of LM Pre-Training shows that language models can switch abruptly between shallow pattern-following and more transferable computation during pre-training, even when loss and...
Computation and its Connotations
A Review of Language Machines by Leif Weatherby
Jailbreaking Large Language Models: If You Torture the Model Long Enough, It Will Confess!
A Cautionary Tale…
A Visual Guide to Quantization
Exploring memory-efficient techniques for LLMs
