An entire Social Network in 1.6GB (GraphD Part 2)

Social Network User Data Storage

Social Network User Data Storage

Social Network User Data Storage

Your Data Fits in Memory (GraphD Part 1)

We need a fast way to query multiple potentially large sets of data on-demand at interactive speeds. Sometimes the easiest solution to a hard problem is to build the right tool for the job.

Consistently faster and smaller compressed bitmaps with Roaring | Daniel Lemire, Computer Science Professor

Computer Science Professor

A primer on Roaring bitmaps: what they are and how they work

A detailed primer on Roaring bitmaps explaining what they are, how they're different from traditional bitmaps, and how they work internally.

Cloud Atlas

Interactive map of the Bluesky social graph, initially compiled by https://theo.io, turbocharged with goose powers.

surrealdb/surrealdb

A scalable, distributed, collaborative, document-graph database, for the realtime web

Bluesky now has more than 10M users | TechCrunch

Social network Bluesky said Monday that it now has more than 10 million users. This is largely thanks to the rapid growth of the network in the past few

A survey of community search over big graphs

With the rapid development of information technologies, various big graphs are prevalent in many real applications (e.g., social media and knowledge bases). An important component of these graphs is the network community. Essentially, a community is a group of vertices which are densely connected internally. Community retrieval can be used in many real applications, such as event organization, friend recommendation, and so on. Consequently, how to efficiently find high-quality communities from big graphs is an important research topic in the era of big data. Recently, a large group of research works, called community search, have been proposed. They aim to provide efficient solutions for searching high-quality communities from large networks in real time. Nevertheless, these works focus on different types of graphs and formulate communities in different manners, and thus, it is desirable to have a comprehensive review of these works. In this survey, we conduct a thorough review of existing community search works. Moreover, we analyze and compare the quality of communities under their models, and the performance of different solutions. Furthermore, we point out new research directions. This survey does not only help researchers to have better understanding of existing community search solutions, but also provides practitioners a better judgment on choosing the proper solutions.

AT Explore

Explore and discover AT Protocol data with our fast, user-friendly browser. Search posts, profiles, and feeds across the decentralized social web — including apps like Bluesky.

Better bitmap performance with Roaring bitmaps

Bitmap indexes are commonly used in databases and search engines. By exploiting bit-level parallelism, they can significantly accelerate queries. However, they can use much memory, and thus we might prefer compressed bitmap indexes. Following Oracle's lead, bitmaps are often compressed using run-length encoding (RLE). Building on prior work, we introduce the Roaring compressed bitmap format: it uses packed arrays for compression instead of RLE. We compare it to two high-performance RLE-based bitmap encoding techniques: WAH (Word Aligned Hybrid compression scheme) and Concise (Compressed `n' Composable Integer Set). On synthetic and real data, we find that Roaring bitmaps (1) often compress significantly better (e.g., 2 times) and (2) are faster than the compressed alternatives (up to 900 times faster for intersections). Our results challenge the view that RLE-based bitmap compression is best.

Tim Berners-Lee, Giant Global Graph (2007)

A new vision for the web where the users master their data and link them to everybody's benefit in a giant global graph.