The Growing Importance of Object Storage in AI Workloads

Solidigm™ S3 Fuse Investigation and Proof-of-Concept Design for AI Workloads

Object storage has become so popular that it is now considered the first-tier storage solution for AI workloads, encompassing not only inference but also AI training and VectorDB use cases. There is an increasing demand to proxy S3 with a file system that can seamlessly integrate with AI training and inference frameworks like PyTorch and CUDA. To address this, the industry offers the open-source project s3fs-fuse, enabling users to bridge S3 storage with file systems.

Key problems with the open-source s3fs-fuse

• High memory consumption: The open-source solution uses at least four C++ STL maps to cache results for file system operations like GetAttr and ReadDir. Each map requires 32GB of DRAM to handle 100 million objects, resulting in a roughly total of 128GB of DRAM to manage 100 million objects.
• Scalability issues: AI storage systems need to handle billions of small objects, making the current s3fs-fuse implementation unsuitable for scalable, future-proof, cloud-native AI deployments.
• Performance trade-offs: While the C++ STL map provides good performance for find and list operations, the associated memory consumption limits its scalability.

Key benefits of Solidigm S3 Fuse design architecture

Solidigm’s Cloud Storage Acceleration Layer (CSAL) group proposes an innovative architecture designed to provide customers with greater flexibility in defining performance SLAs and managing DRAM usage. By leveraging cutting-edge technologies like the SK hynix CXL memory module, Solidigm D7-P5810 SLC drive, Solidigm D7-PS1010 Gen5 TLC drive, and Solidigm D5-P5336 high-density QLC drive, this design ensures:

• Optimized resource usage for scalable AI storage systems.
• A balance between performance and memory consumption to meet diverse workload demands.
• Future-ready scalability to handle the growing needs of cloud-native AI deployments.

This architecture offers a forward-looking solution to address the limitations of existing open-source options and enable efficient AI training and inference workflows.

Solidigm S3 Fuse architecture overview

1. User file system operations

File system operations initiated by the user are first handled by the kernel’s FUSE driver. The FUSE driver redirects these operations to a user-mode service. In this architecture, the proof-of-concept S3 Fuse service developed by Solidigm manages core file system operations such as GetAttr, ReadDir, Read, and Write.

2. S3 FUSE cache

In the cache module design of Solidigm S3 FUSE cache, we chose the FIFO (First-In, First-Out) eviction algorithm. This is because FIFO not only achieves a higher hit rate (as demonstrated in the comparison tests with the LRU eviction algorithm in [3]) but also exhibits greater compatibility with the underlying NAND SSD. By working in conjunction with Solidigm's CSAL software, we have significantly reduced the write amplification on the NAND SSD. Through real-world testing, we've observed that the WAF (Write Amplification Factor) value is essentially reduced to 1.

3. Handling GetAttr and ReadDir requests

These requests are processed by Metadata Core, which utilizes the proof-of-concept S3-FIFO Key-Value store developed by Solidigm. For detailed insights into Solidigm's CSAL append cache and S3-FIFO mechanisms, please see links 1-3 in the reference section.

4. Metadata core and proof-of-concept key-value store

The metadata core is built upon the proof-of-concept KV cache store developed by Solidigm, which employs the proof-of-concept S3-FIFO cache algorithm. The FIFO and CSAL append cache mechanisms are optimized for SSDs, making them highly efficient. The Solidigm Gen5 NVMe SSD, D7-PS1010 delivers 9.4 GB/s write bandwidth and 14 GB/s read bandwidth. By designing the software to utilize FIFO writes, the SSD's Write Amplification Factor (WAF) is reduced to 1, ensuring that the entire bandwidth is available for user data. Consequently, the KV store’s primary queue can operate at 10 GB/s bandwidth.

5. KV store extension options

The small queue of the KV store can be extended using additional DRAM or SK hynix CXL memory modules, which offer up to 96GB per unit with latencies comparable to DRAM. 

6. Handling read and write requests

These requests rely on the Metadata Core to retrieve the file system’s chunk mapping. Data is then written to or read from large FIFO-based chunks, allowing users to choose QLC storage based on capacity requirements.

7. Local host optimization

If file information and data are available locally, the system directly returns them to the user. Otherwise, it fetches the source data from S3.

Performance testing

Performance benchmarking was conducted using microbenchmark tests [5] with C++ map and RocksDB configurations with Solidigm D7-PS1010 Gen5 TLC SSD. The results demonstrate significant efficiency and scalability advantages.

This architecture emphasizes optimized SSD utilization, scalability for AI workloads, and flexible integration options, making it a robust solution for modern AI storage challenges.

Insights from the Test_KV results

The test results for Test_KV provide the following key trends and observations:

Configuration context

RocksDB was tested with minimal tuning for this analysis. We utilized a memory write buffer and disabled compression and block cache to better control memory usage. Further fine-tuning could yield additional performance improvements.

Key takeaways

• C++ map performance: C++ maps delivered the best performance. Even when RocksDB was allocated 3GB of DRAM, utilizing almost all the skiplist in the memory table, its performance was 4x worse than the C++ map. This is expected since skiplist traversal involves navigating multiple linked lists, unlike the direct access nature of a C++ map.
• Memory savings with RocksDB: By reducing RocksDB's memory usage while maintaining similar performance to the 3GB DRAM configuration, we observed significant DRAM savings of up to 2GB without compromising the performance SLA.
• Extreme DRAM efficiency: Even with only 100MB of DRAM allocated, RocksDB maintained an average latency of 15 microseconds. This flexibility allows system designers to make resource and performance SLA trade-offs based on specific use cases.
• Efficiency in proof-of-concept S3 Fuse: When leveraging RocksDB, our proof-of-concept S3 Fuse design demonstrated significantly higher efficiency than S3 List for example Minio List operations when handling 10 million objects.
• These results underscore the potential for RocksDB to balance resource efficiency with performance, offering scalability and configurability for demanding storage systems.

How Solidigm SSDs provide the right storage for Solidigm’s proof-of-concept FUSE design

For S3 Fuse cache, the Solidigm D5-P5336 61.44TB QLC drive offers exceptional performance and scalability. For checkpoint writes, the PCIe 5.0 Solidigm D7-PS1010 delivers world-class write performance.