This is a bit of an introduction to a long series of posts I’ve been writing about what, fundamentally, it is that makes databases scale. The whole series is going to take me a long time, but hopefully there’s something here folks will enjoy.
Designing Data-Intensive Applications (DDIA) — an O’Reilly book by Martin Kleppmann (The Wild Boar Book)
NoSQL… Big Data… Scalability… CAP Theorem… Eventual Consistency… Sharding…
AWS re:Invent 2018: Amazon DynamoDB Under the Hood: How We Built a Hyper-Scale Database (DAT321)
The Marketing Behind MongoDB
Countless NoSQL databases competed to be the database of choice. MongoDB's marketing strategy helped it become the winner.
Episode 1 - Mongo DB Is Web Scale
neodb-social/neodb
🧩 a self-hosted server tracking what you read/watch/listen/play, powering a global distributed community federating via ActivityPub and ATProto.
The Magic of Small Databases
Notes on personal libraries, collections and small indexes on the web
WillSearch – GuardianDB. A peer-to-peer database for the Decentralized Web.
A High-performance, local-first decentralized database built on Rust and Iroh
DuckLake Architecture Deep Dive
DuckLake is a next generation data lakehouse and open table format. It is significantly simpler and faster than Apache Iceberg or Delta Lake because it uses a SQL database for storing the catalog and metadata. This post covers an architecture deep dive with a detailed Q&A section. DuckLake consists of 3 components: storage, catalog, and compute. Each can be scaled independently, hosted locally with ease, and deployed to the cloud for production. MotherDuck offers managed DuckLake lakehouses for additional simplicity and performance.
Egres: A Postgres-compatible database for heavy, multi-tenant workloads
I love Postgres. It is an incredible database. But, its design is also 30 years old and most of what people fight in production comes from decisions made long time ago: VACUUM, transaction id wraparound, bloat, a process per connection…
Dremel: interactive analysis of web-scale datasets: Proceedings of the VLDB Endowment: Vol 3, No 1-2
Dremel is a scalable, interactive ad-hoc query system for analysis of read-only nested data. By combining multi-level execution trees and columnar data layout, it is capable of running aggregation queries over trillion-row tables in seconds. The system ...
A Web of Data
Thoughts on making a "Web of Data" instead of a "Web of Pages", and how that might let us take a step away from the dominance of large, complicated browsers.
surrealdb/surrealdb
A scalable, distributed, collaborative, document-graph database, for the realtime web
I've been thinking about the beauty of open data and how the AT Protocol and wider Atmosphere ecosystem has been recapturing that open data environment that once defined the web. It really does feel special. vale.rocks/posts/atproto-ethos #atproto #ATProtocol
The AT Protocol Harkens An Old Web Ethos
vale.rocks