JuiceFS is a cloud-based high-performance distributed file system, based on object storage and database to achieve data and metadata separation architecture, easily carrying tens of billions of file storage, to meet the random read performance requirements of big data, AI, autonomous driving, gene sequencing and other massive data.
TigerFS
A filesystem backed by PostgreSQL, and a filesystem interface to PostgreSQL. TigerFS mounts a database as a directory. Every file is a real row. Writes are transactions. Multiple agents and humans can read and write concurrently with full ACID guarantees, locally or across machines. Any tool that works with files works out of the box.
Filecoin | World's Largest Decentralized Storage Network for Humanity's Data
Preserve humanity's most important information on Filecoin, the decentralized storage network with verifiable integrity, global redundancy, and no centralized control. Build apps, store data securely, or become a storage provider today.
Parquet
Apache Parquet Documentation Releases Apache Parquet is an open source, column-oriented data file format designed for efficient data storage and retrieval. It provides high performance compression and encoding schemes to handle complex data in bulk and is supported in many programming languages and analytics tools.
omnifs: access the world through your filesystem
omnifs turns APIs and data sources into files and directories your shell, scripts, CI jobs, notebooks, and agents can read.
Column Storage for the AI Era
In the past few years, we’ve seen a cambrian explosion of new columnar formats, challenging the hegemony of Parquet: Lance, Fastlanes, Nimble, Vortex, AnyBlox, F3 (File Format for the Future). The thinking is that the context has changed so much that the design of yore (the previous decade) is not going to cut it moving forward. This seemed a bit intriguing to me, especially since the main contribution of Parquet has been to provide a standard for columnar storage. Parquet is not simply a file format. As an open source project hosted by the ASF, it acts as a consensus building machine for the industry. Creating six new formats is not going to help with interroperability. I spent some time to understand a bit better how things actually changed and how Parquet needs to adapt to meet the demands of this new era. In this post I’ll discuss my findings.
Understanding new Cloud Storage hierarchical namespace | Google Cloud Blog
The new hierarchical namespace capabilities bring file system optimizations to Cloud Storage buckets.
Cross-Platform File System Operations Based on libuv
A cross-platform interface to file system operations, built on top of the libuv C library.
Launching S3 Files, making S3 buckets accessible as file systems | Amazon Web Services
Amazon S3 Files makes S3 buckets accessible as high-performance file systems on AWS compute resources, eliminating the tradeoff between object storage benefits and interactive file capabilities while enabling seamless data sharing with ~1ms latencies.
Fil One Object Storage
S3-compatible object storage on Filecoin with no egress fees and verifiable data integrity, proven daily.
jdefrancesco/dskDitto
Ultra fast and easy duplicate file finder. Awesome TUI/GUI to manage results.
Storj - Smarter cloud storage for your business.
Make the switch to Storj and get better global performance, unparalleled security, and save 80% on your cloud costs while cutting your carbon footprint.
