I'm trying to make sense of the claim that the HPE NS9 X5 supports 270 ports via dual-fabric HDR200 InfiniBand.
- One ConnectX-7 typically supports 16 ports max, so you'd need 17 NICs. But how would that fit into a standard server chassis?
- Even assuming PCIe 4.0 x16 slots, the available 32GB/s falls short of the ~50GB/s needed for full-bandwidth on 270 ports.
Could this be:
- A switch-based topology (e.g., NS9 X5 as an endpoint connected to an external SN3700)?
- Port virtualization using SR-IOV or NPAR?
- Or is there any secret sauce like PCIe 5.0/CXL magic going on?
Has anyone benchmarked this with ib_write_bw or seen performance data?
Appreciate any thoughts or references to official docs. I'm wondering if this is just port virtualization dressed up in marketing terms.
- One ConnectX-7 typically supports 16 ports max, so you'd need 17 NICs. But how would that fit into a standard server chassis?
- Even assuming PCIe 4.0 x16 slots, the available 32GB/s falls short of the ~50GB/s needed for full-bandwidth on 270 ports.
Could this be:
- A switch-based topology (e.g., NS9 X5 as an endpoint connected to an external SN3700)?
- Port virtualization using SR-IOV or NPAR?
- Or is there any secret sauce like PCIe 5.0/CXL magic going on?
Has anyone benchmarked this with ib_write_bw or seen performance data?
Appreciate any thoughts or references to official docs. I'm wondering if this is just port virtualization dressed up in marketing terms.