When your database CPU is at 100%, what is it actually doing? Memory stalls, CPU caches, and how radix partitioning makes GROUP BY faster.
Slow Software
What it means for software to be fast, and why most software is not.
Data-Oriented Design (Or Why You Might Be Shooting Yourself in The Foot With OOP)
Picture this: Toward the end of the development cycle, your game crawls, but you don’t see any obvious hotspots in the profiler. The culprit? Random memory access patterns and constant cache misses. In an attempt to improve performance, you try to parallelize parts of the code, but it takes heroic efforts, and, in the end, you barely get much of a speed-up due to all the synchronization you had to add. To top it off, the code is so complex that fixing bugs creates more problems, and the thought of adding new features is discarded right away. Sound familiar?
Atomic Commit In SQLite
An important feature of transactional databases like SQLite is "atomic commit". Atomic commit means that either all database changes within a single transaction occur or none of them occur. With atomic commit, it is as if many different writes to different sections of the database file occur instantaneously and simultaneously. Real hardware serializes writes to mass storage, and writing a single sector takes a finite amount of time. So it is impossible to truly write many different sectors of a database file simultaneously and/or instantaneously. But the atomic commit logic within SQLite makes it appear as if the changes for a transaction are all written instantaneously and simultaneously.
quick_cache - Rust
Lightweight, high performance concurrent cache. It allows very fast access to the cached items with little overhead compared to a plain concurrent hash table. No allocations are ever performed unless the cache internal state table needs growing (which will eventually stabilize).
turbopuffer: fast search on object storage
Inaugural blog post about the development of turbopuffer, a search engine that uses object storage and SSD caching for cost-effective, low latency search. This post describes into the motivation behind its creation, its unique architecture, and how it significantly reduces costs for large-scale vector searches. Discover how turbopuffer is transforming search infrastructure for companies like Cursor and Suno, offering a scalable and reliable solution.
Ubershaders: A Ridiculous Solution to an Impossible Problem
When you're playing your favorite game on Dolphin with a powerful computer, things should run fairly well. The game is running full speed, there are no graphical glitches, and you can use your favorite controller if you want. Yet, every time you go to a new area, or load a new effect, there's a very slight but noticeable "stutter." You turn off the framelimiter to check and your computer can run the game at well over full speed. What's going on? The slowdown when loading new areas, effects, models, and more is commonly referred to as "Shader Compilation Stuttering," by users and developers alike. This problem has been a part of Dolphin since the very beginning, but has only recently become more of a focus.
There continue to be reasons for software to be slow
A discussion on the various reasons why performance is slow, and which ones are unlikely to be affected by LLMs. Originally written as a counter-point to Dan Luu
Wirth's law
Wirth's law is Niklaus Wirth's variant of the Jevons paradox: Software is getting slower more rapidly than hardware is getting faster.
Software Acceleration and Desynchronization
A look at the ever-present drive to make software delivery faster and how it might break down various activity loops in organizations.
Slow developer experience will bottleneck fast models
Right now developer experience is measured in seconds. If your tests take a second to run, that’s good; if they take thirty seconds, that’s bad. Any faster than…
Towards Inserting One Billion Rows in SQLite Under A Minute - blag
This is a chronicle of my experiment where I set out to insert 1B rows in SQLite
Agner`s CPU blog
Software optimization resources | E-mail subscription to this blog | www.agner.org
Why Futhark?
A high-performance and high-level purely functional data-parallel array programming language that can execute on the GPU and CPU.
Making SQLite faster in Go
Make sqlite faster with a connection pool and prepared statements.
We're making Tailscale faster
Tailscale performance updates reduce memory use, increase throughput, and speed startup with multi-queue, writev, and netmap caching.
