Auto-indexing service and GraphQL API for AT Protocol Records
Your Data Fits in Memory (GraphD Part 1)
We need a fast way to query multiple potentially large sets of data on-demand at interactive speeds. Sometimes the easiest solution to a hard problem is to build the right tool for the job.
Designing Data-Intensive Applications (DDIA) — an O’Reilly book by Martin Kleppmann (The Wild Boar Book)
NoSQL… Big Data… Scalability… CAP Theorem… Eventual Consistency… Sharding…
GraphQL
GraphQL is a query language designed to build client applications by providing an intuitive and flexible syntax and system for describing their data requirements and interactions.
Streaming Data - AT Protocol Docs - AT Protocol
Synchronizing and streaming data from the AT Protocol.
AT Explore
Explore and discover AT Protocol data with our fast, user-friendly browser. Search posts, profiles, and feeds across the decentralized social web — including apps like Bluesky.
Using GraphQL in Rust | Shuttle
This article details how to build a GraphQL server in Rust. We will explore using queries, mutations as well as subscriptions to make a fully working endpoint.
CompactLTJ: Space & Time Efficient Leapfrog Triejoin on Graph Databases
Leapfrog Triejoin (LTJ) is arguably the most practical and popular worst-case-optimal (wco) algorithm for solving basic graph patterns in graph databases. Its main drawback is that it needs the database triples (subject, predicate, object) represented as paths in a trie, for each of the six orders of subject, predicate, and object. The resulting blowup in space makes most systems disregard LTJ or implement it only partially, which makes their corresponding algorithms non-wco. In this paper we show that, by using compact data structures, it is possible to build an index that at the same time matches the query time performance of the fastest classic wco index, and uses a fraction of the space of non-wco indices (which are much slower). Concretely, we make use of compact tree representations to store functional tries using one bit per trie edge, instead of one pointer, and further reduce the space by storing partial tries. Our most compact variant uses 5–6 times less space than classic wco implementations and 2–3 times less than classic non-wco systems. At solving queries, it is on par with the fastest classic wco system, and 30–40 times faster than non-wco systems. We further incorporate improved query resolution strategies into CompactLTJ variants, which makes it considerably faster than classic wco systems as well, on queries that do not output too many results. Finally, we show how CompactLTJ can incorporate dynamism without altering its performance, even under very demanding update regimes. We leave a public fully-functional implementation of CompactLTJ that can be directly used by practitioners.
Indexing Standard Site - AT Protocol
This guest post from Steve Simkins, creator of Sequoia and docs.surf, outlines the strategy he used to index standard.site records.
QuickDID - AT Protocol Identity Resolution Service
High-performance handle-to-DID resolution service for the AT Protocol ecosystem. Resolve Bluesky and AT Protocol handles instantly.
Semantic Data Modeling, Graph Query, and SQL, Together at Last?
Our teams advance the state of the art through research, systems engineering, and collaboration across Google.
Launching QuickDID - Fast, Open Handle Resolution for the AT Protocol - Nick's Blog
QuickDID is a high-performance, open source handle resolution service for the ATmosphere that serves as both public infrastructure at https://quickdid.smokesignal.tools and deployable software under MIT license. It offers flexible caching strategies (memory/Redis/SQLite), scales from single-instance to distributed deployments, and includes production features like rate limiting and proactive cache refresh. Currently a release candidate, it provides a drop-in alternative to Bluesky's resolver while giving developers full control over their handle resolution infrastructure.
New demo version of atsearch.network is here 💃 quick recap: any AT Proto app can invent its own record type: scrobbles, books, photos, RPGs, etc... nothing searches across them so i built a search engine that reads your lexicon and figures out how to index you
