Bitesize Proto: Upserting ATProto Records - Hitchhiker's Guide to the Atmosphere
Tired of creating duplicate records in a repo from your atproto app?
Atproto spaces · aturi.to
Browse your permissioned atproto data: records kept outside your public repo.
STAR: ordered key-value archive format #21
Hello! I'd like to propose new work for DASL: the [STAR](https://tangled.org/microcosm.blue/star/tree/main/v1-star) (/STAR-lite) archive format. I originally made STAR as a stricter, simpler, safer, more efficient alternative to CARs for atproto public repository exports. Its v0 now has several library implementations and has at least two production producers and half a dozen consumers in the Atmosphere, but is very atproto-coupled. STAR is [being considered](https://github.com/bluesky-social/proposals/pull/114) as the standard format for [atproto spaces](https://github.com/bluesky-social/proposals/tree/main/0016-permissioned-data) repo exports. After iteration with @dholms.at and @bnewbold.net, STAR's spaces-compatible v1 is becoming a more general-purpose format for ordered key-value data, and I think, a potentially good fit with DASL. Following the DASL governance [steps for proposing new work](https://dasl.ing/governance.html#:~:text=Proposing%20New%20Work): ## What will be implemented A specification ([in progress](https://tangled.org/microcosm.blue/star/tree/main/v1-star)), reference implementations, test vectors. ## Who wants it Data exports in atproto is one established use-case. People have been motivated to request public archives in STAR-lite format from [hubble](https://web.hubble.microcosm.blue/): 260M STAR-lite archives served vs 6M CAR. Repositories compress to a bit under half the size compared to CARs, which translates directly into faster network backfill rate in many cases (this is not exactly apples-to-apples: STAR-lite omits the MST entirely since it's deterministic from the data and cheap to reconstruct). More on the pain it addresses [here](https://pckt.blog/b/data-service-diaries/war-on-cars-5xa5v73). The atproto spaces protocol proposes an LtHash set as its canonical data integrity structure instead of a MST: flat key-value with no tree or DAG structure at all to benefit from CAR's CID indexing, fitting more naturally (and without the incompressible CID overhead) into something like STAR. ## Fit with existing primitives STAR-lite builds on DRISL/DASL-CBOR, and uses `varint`-prefixed framing like DASL-CAR. Where DASL-CAR fits DAG-shaped data well, STAR-lite is a more natural fit for flat key-value data. ## Confirm willingness to serve as an editor I confirm! ## Other criteria not covered above Implementation: there is currently a v0 reference [rust](https://crates.io/crates/star-lite) + a third-party Objective-C implementation. I will be adding v1 support to the rust library soon. I think there will be more implementations, especially if it gets adopted by atproto spaces. Community Support: Existing growing uptake of the v0 format, possible standardized use in atproto-spaces, [active evolution](https://bsky.app/profile/bad-example.com/post/3mwc5n4f4gk24) based on feedback. --- Atmosphere post: https://bsky.app/profile/did:plc:lulmyldiq4sb2ikags5sfb25/post/3mwohvqxhpc2h Upcoming CID Congress presentation at https://luma.com/8w8swmx2
API documentation | Hubble
Export full repositories or check atmosphere-wide repo metadata: standard atproto endpoints and Hubble-specific queries.
Atproto spaces alpha updates
As promised, we just released another update to the spaces alpha today! This is a smaller release than last week. We have a few new features & protocol mechanisms (including account migration!). The meat of it is a refactor of simplespaces. This is backwards-incompatible and will require updating any of your apps that use simplespaces! Like last time, we’ve put together a prompt that you can give your coding agent to handle the migration. To summarize the changes: Space policies: we totally...
Atproto spaces alpha updates
As promised, we just released another update to the spaces alpha today! This is a smaller release than last week. We have a few new features & protocol mechanisms (including account migration!). The meat of it is a refactor of simplespaces. This is backwards-incompatible and will require updating any of your apps that use simplespaces! Like last time, we’ve put together a prompt that you can give your coding agent to handle the migration. To summarize the changes: Space policies: we totally...
Reintroducing Spaces - Daniel's Leaflets
Zooming out and re-motivating the design of spaces, the new primitive for non-public data on atproto.
Reintroducing Spaces - Daniel's Leaflets
Zooming out and re-motivating the design of spaces, the new primitive for non-public data on atproto.
#atproto space question: Is it fine to create space for each records to support per-record access control? Say all Tangled repository issues are spaces. My biggest concern is sync, because every issues are now standalone permissioned repos. But maybe that doesn't matter much?
@dholms.at and I pushed up IETF draft text for atproto repos and sync: ietf.org/archive/id/draft-holmgren-at-… ietf.org/archive/id/draft-holmgren-at-… these are versioned in github (per norm) over here: github.com/ietf-wg-atp/drafts
PART TWO: All the other reasons i don't like CARs for atproto repo exports! In this one: ambiguities and inefficiencies. (the biggest problem is already hopefully getting fixed in the IETF draft though, yay!)
War on CARs: the rest of it
data-service-diaries.pckt.blogCurated Lists
Useful lists for atproto
vāyumaṇḍala
atproto.camp — A directory of ATProtocol user groups
OpenPDS - AT Protocol PDS Directory
List: Independent PDSes (by Victoria) — Semble
Apps
trueberryless/awesome-atproto
