🏔️ Sync protocol for hash-linked data
Willow - Willow’25
Peer-to-peer protocols which scale up, down, and sideways. Sychronisable data storage, fine-grained access control, private and efficient synchronisation, and secure data delivery by any means possible.
Conflict-Free Replicated Data Types
Replicating data under Eventual Consistency (EC) allows any replica to accept updates without remote synchronisation. This ensures performance and scalability in large-scale distributed systems (e.g., clouds). However, published EC approaches are ad-hoc and error-prone. Under a formal Strong Eventual Consistency (SEC) model, we study sufficient conditions for convergence. A data type that satisfies these conditions is called a Conflict-free Replicated Data Type (CRDT). Replicas of any CRDT are guaranteed to converge in a self-stabilising manner, despite any number of failures. This paper formalises two popular approaches (state- and operation-based) and their relevant sufficient conditions. We study a number of useful CRDTs, such as sets with clean semantics, supporting both add and remove operations, and consider in depth the more complex Graph data type. CRDT types can be composed to develop large-scale distributed applications, and have interesting theoretical properties.
A Conflict-Free Replicated JSON Datatype
Many applications model their data in a general-purpose storage format such as JSON. This data structure is modified by the application as a result of user input. Such modifications are well understood if performed sequentially on a single copy of the data, but if the data is replicated and modified concurrently on multiple devices, it is unclear what the semantics should be. In this paper we present an algorithm and formal semantics for a JSON data structure that automatically resolves concurrent modifications such that no updates are lost, and such that all replicas converge towards the same state (a conflict-free replicated datatype or CRDT). It supports arbitrarily nested list and map types, which can be modified by insertion, deletion and assignment. The algorithm performs all merging client-side and does not depend on ordering guarantees from the network, making it suitable for deployment on mobile devices with poor network connectivity, in peer-to-peer networks, and in messaging systems with end-to-end encryption.
atproto-experiments - Ha Ja Ba Ra La
A foundation for building tools on the AT Protocol using Unison Programming Language.
Streaming Data - AT Protocol Docs - AT Protocol
Synchronizing and streaming data from the AT Protocol.
n0-computer/iroh-docs
Multi-dimensional key-value documents with an efficient synchronization protocol.
snarfed/arroba
Python implementation of Bluesky PDS and AT Protocol, including repo, MST, and sync XRPC methods
Willow - Home
Peer-to-peer protocols which scale up, down, and sideways. Sychronisable data storage, fine-grained access control, private and efficient synchronisation, and secure data delivery by any means possible.
Protocol Buffers
Protocol Buffers are language-neutral, platform-neutral extensible mechanisms for serializing structured data.
ATOM: AT Protocol Over MoQ Transport
This document specifies how the Authenticated Transfer (AT) Protocol can leverage Media over QUIC Transport (MOQT) for efficient data synchronization across decentralized social networks. The AT Protocol's firehose event stream and repository synchronization mechanisms map naturally to MOQT's publish/subscribe model, enabling scalable relay infrastructure, priority-based delivery, and improved resilience for large-scale social data distribution. This specification addresses the challenges of the current WebSocket- based transport and demonstrates how MOQT's relay architecture, group-based caching, and multiplexed streams provide significant benefits for AT Protocol deployments at scale.
Peter van Hardenberg - Ink and Switch, Automerge
ewancroft.uk/tangled-sync
[READ-ONLY] Mirror of https://github.com/ewanc26/tangled-sync. Move from GitHub to Tangled.org
new data diary! this is the most in the weeds yet, all revolving around our proposed URI structure for permissioned data. turns out you can figure out a lot about a protocol from the uri! as always, let me know what you think :)
Permissioned Data Diary 5: What’s in a Name?
dholms.leaflet.pub