Alluxio and Solidigm Collaborate to Create an Advanced Cache Solution for AI Workloads

At a glance

Alluxio, a global leader in AI caching solutions, delivers the fastest cache for GPU-based AI workloads. Its scalable architecture supports tens of thousands of nodes, significantly reducing storage bandwidth consumption. The success of large language models (LLMs) around the world is largely due to Alluxio’s cutting-edge approach to solving AI storage challenges.

"Solidigm partnered with Alluxio to deliver a superior distributed AI caching solution. Using Solidigm's D5-P5336 as a read cache and D7-PS1010 for checkpoint writes with Alluxio's minimal overhead solution offers our customers the best combination of cost and performance for large-scale AI workloads. We optimized this solution to leverage full write bandwidth of a Solidigm D7-PS1010 Gen5 TLC SSD and the read bandwidth of Solidigm D5-P5336 Gen4 QLC while maintaining a write-amplification of 1.02 on TLC and QLC SSDs. Together, we hope to continue to deliver cost- and performance-optimized, low overhead solutions for our customers' AI needs,” says Greg Matson, Senior Vice President, Strategic Planning and Marketing at Solidigm.

Key Benefits of the Alluxio DORA Cache Architecture

DORA, short for Decentralized Object Repository Architecture, is the next-generation architecture of Alluxio.  As a distributed caching storage system, DORA offers low latency, high throughput, and cost savings while aiming to provide a high-performance data access layer for AI workloads. DORA leverages decentralized storage and metadata management to provide higher performance and availability, as well as pluggable data security and governance, enabling more scalability and efficient management of large-scale data access.

DORA's architectural goal

DORA's architectural goal

1. Scalability: Scalability is a top priority for DORA, which needs to support billions of files to meet the demands of data-intensive applications, such as AI training.
2. High Availability: DORA’s architecture is designed with high availability in mind, with 99.99% uptime and protection against single points of failure at the master level.
3. Performance: Performance is also a key goal for DORA, which prioritizes the speed of model training, model serving, as well as GPU utilization for AI workloads.

The architecture consists of four key components: the service registry, scheduler, client, and worker. Together, these components manage tasks such as service discovery, distributed load scheduling, and data storage, all while maintaining optimal performance across the system.

NVMe storage-related technical highlights

1. Page Data Store can leverage SSD max performance and maintain best WAF and endurance

DORA utilizes a battle-tested page store module for cache storage, enabling more granular caching of small to medium read requests on large files. This reliable page store technology has been proven in applications like Presto at Meta, Uber, and TikTok. DORA’s fine-grained caching has reduced read amplification by 150 times and increased file position read performance by up to 15X. 

The Page Data Store leverages a journal file system and organizes data into two levels of directories with fixed, large-sized chunk files. All writes are appended to these chunk files, and when objects within them are marked for deletion, the files themselves are only removed when no longer needed. This design ensures optimal performance, even with PCIe 5.0 TLC SSDs, while maintaining an SSD Write Amplification Factor (WAF) of 1, which maximizes SSD endurance.

For example, when using Alluxio’s read cache with QLC SSDs, the storage engine fully utilizes the endurance of QLC without causing any internal or garbage collection WAF, ensuring efficient operation on QLC NVMe SSDs.

2. Decentralized metadata store

DORA spreads metadata to every worker to ensure that metadata is always accessible and available. To optimize metadata access, DORA utilizes a two-level caching system for metadata entries. The first level of caching is the in-memory cache, which stores metadata entries in memory. This cache has a configurable maximum capacity and time-to-live (TTL) setting to set an expiration duration. The second level of caching is the persistent cache, which stores metadata entries on disk using RocksDB. The persistent cache has unbounded capacity, depending on available disk space, and also uses TTL-based cache eviction, avoiding any active sync or invalidation. The stored metadata is hashed by the full UFS path like the Page Store.

The combination of in-memory and persistent caching helps ensure that metadata is readily available and accessible, while also allowing for efficient use of system resources. The decentralization of metadata avoids the bottleneck in the architecture where metadata is primarily managed by the master nodes. With the ability to store up to 30 to 50 million files per DORA worker, the system can support large-scale data-intensive applications with billions of files.

Solidigm has verified that the combination of in-memory metadata design and RocksDB offers the optimal metadata storage solution. This design fully utilizes the read and write speeds of PCIe 4.0 QLC SSDs (7GB/s read, 3GB/s write) and PCIe 5.0 TLC SSDs (14.5GB/s read and write). Additionally, RocksDB consolidates many small writes into larger, sequential 2MB writes using a skiplist-based write buffer, which is highly efficient and helps minimize SSD WAF, further enhancing SSD endurance.

Performance testing: Alluxio with the Solidigm™
D7-PS1010 and D5-P5336

Test configuration

In a recent experiment using Intel's Gen5 BNC storage server with Solidigm D7-PS1010 and D5-P5336 SSDs, Alluxio demonstrated its capabilities for fast data ingestion and read performance, especially with GPU scaling. Notable results include:

• Cache Load Efficiency: The Alluxio cache engine effectively saturated the maximum read bandwidth of UFS QLC storage while writing to PCIe 5.0 TLC SSDs.
• Fuse Read Test: Alluxio’s Filesystem in Userspace (FUSE) framework nearly saturated a single PCIe 5.0 SSD’s read bandwidth, demonstrating minimal overhead when handling cache hits on NVMe SSDs.

We set up a single-node test for quick deployment to showcase the power of the Alluxio storage engine. It's important to note that Alluxio's greatest strength lies in its ability to leverage a host-side distributed replicated cache, which scales with GPUs and significantly reduces north-south storage bandwidth overhead. Even in single-node setups, Alluxio demonstrates exceptional efficiency, especially when paired with high-performance NVMe SSDs. For this test, we configured the cache with either PCIe 5.0 TLC or PCIe 4.0 QLC, while the underlying file store (UFS) used PCIe 4.0 QLC.

Key takeaways

1. The Alluxio cache load engine is highly efficient, capable of saturating the maximum read bandwidth of UFS QLC and ingesting data into PCIe 5.0 TLC cache SSDs. Even if the UFS supports a read bandwidth of 10GB/s, Alluxio can easily saturate the 9.3GB/s write bandwidth of the Solidigm D7-PS1010.

2. Alluxio’s page cache storage engine, built on the XFS journal file system, has been rigorously tested using the Solidigm Alluxio FIO emulator for longevity. The results show that, regardless of whether TLC or QLC SSDs are used, Alluxio achieves a WAF of 1.02. This near-ideal WAF of 1 maximizes SSD performance and endurance, delivering optimal results to end users.

The FUSE read overhead is minimal. When performing a FUSE read with a 100% cache hit on the SSD cache and bypassing the DRAM page cache, the FUSE framework can nearly saturate the read bandwidth of a single PCIe 5.0 SSD, reaching 14.8GB/s.

For customers seeking high-density cache solutions, Solidigm’s 61.44TB QLC SSD is an ideal option. Alluxio’s storage engine is highly WAF-friendly, excelling as a read-heavy cache with minimal writes, making QLC a perfect match for its read cache path. Thanks to Alluxio's efficient design, we can estimate the endurance of a QLC cache device based on a sequential write PBW of 213PB, which means Solidigm supports writing up to 213 petabytes of data. With 50% utilization over a 5-year period, which is a high threshold, QLC still supports a write bandwidth of 2900 MB/s, nearly reaching its maximum. Additionally, QLC provides up to 6GB/s random read bandwidth per SSD, enabling the creation of a highly cost-efficient Alluxio AI cache system.

How Solidigm SSDs provide the right storage for Alluxio

For read cache, the Solidigm D5-P5336 61.44TB QLC SSD offers exceptional performance and scalability. For checkpoint writes, the PCIe 5.0 Solidigm D7-PS1010 delivers world-class write performance.

Xuan Du, VP of Engineering at Alluxio says, “We collaborated closely with the Solidigm team to validate the performance benefits of running Alluxio’s distributed caching technology with Solidigm SSD and NVMe drives for AI model training workloads. Through our collaboration, we were able to further optimize Alluxio to maximize I/O throughput for large-scale AI workloads leveraging Solidigm drives.” 

   Xuan Du
   Vice President of Engineering at Alluxio

The Alluxio and Solidigm collaboration has yielded results showing that both Solidigm TLC and QLC SSDs significantly enhance Alluxio’s services while reducing operational costs. Solidigm also raises the bar in quality and reliability, supported by a dedicated customer care team that has provided excellent support for Alluxio.