GPUDirect Storage (GDS) vs POSIX Bounce Buffers
Traditional file I/O follows the POSIX standard: data is read from NVMe flash storage into host kernel memory (page cache), copied into host user-space memory, and finally transferred across PCIe into GPU memory. GPUDirect Storage (GDS) establishes a direct DMA path from NVMe-oF controllers directly into GPU High Bandwidth Memory.
Elimination of CPU Bounce Buffers
Traditional I/O requires 3 memory copies and 2 CPU context switches. GDS uses peer-to-peer DMA over PCIe, eliminating intermediate CPU memory staging and freeing CPU cores for data preprocessing.
CPU Utilization Reduction
Streaming multi-gigabyte dataset batches at 50 GB/s using POSIX I/O consumes 60%–80% of host CPU cycles in memory copy and kernel interrupt handling. GDS drops host CPU utilization to under 2%.
NVMe-oF Remote Flash Storage
GDS works seamlessly over remote NVMe-over-Fabrics (RoCE/InfiniBand), allowing multi-node clusters to stream training datasets from high-throughput distributed storage arrays directly into GPU memory.
GDS eliminates both the kernel-to-user copy (T_copy) and the host-to-device transfer (T_PCIe, H2D), reducing end-to-end data ingest latency by up to 70% and increasing sustained sequential read bandwidth to 95% of line rate.
- Verify GDS driver status using gdscheck -p; ensure cuFile library is loaded and direct I/O mode is active.
- Format local scratch NVMe arrays with XFS and direct block alignment (4KB / 64KB) to maximize GDS DMA efficiency.
- For training datasets, store data in uncompressed binary chunks (e.g., NumPy memory-maps or Arrow files) compatible with GDS cuFileRead.