







A list of key concepts for building and testing reliable distributed systems, with basic definitions and deep references.
Distributed Systems lecture series
Why Distributed Systems Are Hard to Develop — and How to Fix It
Here’s what’s missing when developing distributed systems today: Structure. Consistent, codified structure.

Home | cse223b
CSE 223B is a 4-unit graduate subject with lectures, paper discussions, a midterm, and technical report exercise, and a major term research project culminating in writing a research paper of your own. It will present abstractions and implementation techniques that facilitate the design of distributed systems–including both operating systems and sophisticated Internet servers–that can deal with the demands of real-world workloads. Topics will include efficient operating system primitives, high-performance network servers, load shedding, storage systems, security, and fault tolerance.

Software Should Work
An independent conference on software reliability in Columbia, MO on July 16-17, 2026

Atproto for distributed systems engineers - AT Protocol
In this article we're going to explore atproto from the perspective of distributed backend engineering.

SRE book second edition | Google Cloud Blog
The second edition of the Site Reliability Engineering book reflects the shifts in infrastructure management from AI, microservices, serverless, etc.

binhnguyennus/awesome-scalability
The Patterns of Scalable, Reliable, and Performant Large-Scale Systems
A Critique of the CAP Theorem
The CAP Theorem is a frequently cited impossibility result in distributed systems, especially among NoSQL distributed databases. In this paper we survey some of the confusion about the meaning of CAP, including inconsistencies and ambiguities in its definitions, and we highlight some problems in its formalization. CAP is often interpreted as proof that eventually consistent databases have better availability properties than strongly consistent databases; although there is some truth in this, we show that more careful reasoning is required. These problems cast doubt on the utility of CAP as a tool for reasoning about trade-offs in practical systems. As alternative to CAP, we propose a "delay-sensitivity" framework, which analyzes the sensitivity of operation latency to network delay, and which may help practitioners reason about the trade-offs between consistency guarantees and tolerance of network faults.

Cloud native Go: building reliable services in unreliable environments
What do Docker, Kubernetes, and Prometheus have in comm…

Notes on software quality
“The absence of problems” is the best definition I can come up with for quality.
Dynamical Systems Group | Complex Systems, Engineered
Systems engineering firm that operationalizes emerging technology for high-reliability organizations. Scientific Computing, MBSE, Systems Integration.

How we use HashiCorp Nomad
A walkthrough on how we are improving the reliability of our management services running in each data center by using Nomad for dynamic task scheduling.

Declarative Server with NixOS: A Reproducible Infrastructure | Mustafa Erbay
A technical guide exploring how to create a declarative server configuration with NixOS, covering reproducibility and rollback processes.

Fragile narrow laggy asynchronous mismatched pipes kill productivity - Tristan Hume
Something I’ve been thinking about recently is how when I’ve worked on any kind of distributed system, including systems as simple as a web app with frontend and backend code, probably upwards of 80% of my time is spent on things I wouldn’t need to do if it weren’t distributed. I came up with the following description of why I think this kind of programming requires so much effort: Everything is fragile narrow laggy asynchronous mismatched untrusted pipes. I think every programmer who’s worked on a networked system has encountered each of these issues, this is just my effort to coherently describe all of them in one place. I hope to prompt you to consider all the different hassles at once and think about how much harder/easier your job would be if you did/didn’t have to deal with these things. I think this is part of why web companies like Twitter seem to have so much lower impressiveness per engineer productivity than other places like game companies or SpaceX, although there’s other pieces to that puzzle. While part of the difficulty of distributed systems is inherent in physics, I think there’s lots of ideas for making each part of the problem easier, many already in common use, and I’ll try to mention lots of them. I hope that we as programmers continually develop more of these techniques and especially general implementations that simplify a problem. Like serialization libraries reducing the need for hand-written parsers/writers, I think there’s a lot of developer time out there to save by implementing generalized solutions where we currently painstakingly reimplement common patterns. I also think all these costs mean you should try really hard to avoid making your system distributed if you don’t have to.
I often get emails asking for ideas (or materials) to learn about distributed systems. I always recommend getting started with Gossip Glomers. Only six challenges, some easy and some difficult, but an absolute top tier fun and learning experience fly.io/dist-sys