HGX Server Baseboard Anatomy
A modern enterprise AI server (such as an 8x H100/B200 HGX system) is a complex electro-mechanical computing engine designed for 10.2 kW of power delivery, balanced PCIe Gen5 switching trees, and dedicated high-speed scale-out networking for each GPU.
Dual-Socket Host CPU & PCIe Switch Tree
Two host CPUs (e.g., dual Intel Xeon or AMD EPYC) connect to 4 PCIe Gen5 switch fabrics (e.g., Broadcom PEX). Each switch pairs 2 GPUs with 2 dedicated 400Gbps network interfaces (NICs) to maintain direct peer-to-peer data paths.
1:1 GPU-to-NIC Ratio
In production distributed training, every GPU requires its own dedicated 400Gbps ConnectX-7 NIC. Sharing one NIC among multiple GPUs creates immediate inter-node network bottlenecks during AllGather and ReduceScatter phases.
Dedicated DPU Management Fabric
BlueField-3 Data Processing Units (DPUs) offload storage virtualization, NVMe-oF target controllers, and cluster health telemetry, isolating host OS kernels from network interruptions.
With 900 GB/s NVLink per GPU and a 400 Gbps (50 GB/s) NIC per GPU, intra-node bandwidth is 18x higher than inter-node fabric bandwidth. This physical asymmetry requires 3D parallelism to strictly isolate high-frequency tensor parallel collectives within the node.
- Ensure OSFP/QSFP optical transceiver cables are seated with proper bend radius; excessive bend creates optical signal attenuation and high packet error rates.
- Verify that PCIe switches operate in Gen5 speed (32 GT/s) with 16 lanes per device; link training failures can silently drop speeds to Gen4 or Gen3.
- Configure nvidia-peermem kernel modules to enable direct RDMA transfers between CX-7 NICs and GPU HBM over the PCIe switch tree.