Broadcom PEX88000 PCIE4 & PEX9700 PCIE3

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Ethan Waldo

New Member
Dec 9, 2018
10
2
3
The Broadcom PEX88000 PCIE 4 and Broadcom PEX9700 PCIE 3 family of chips have some very intriguing features, most interestingly Tunneled Windows Connection (TWC) that allows true multi-host to host communication via DMA. The previous generation PEX8700 only supported Non-Transparent Bridge (NTB) for more of a host failover cluster scenario. These chips allow SR-IOV Virtual Functions (VFs) from one host to be utilized in a completely different host. To top it off, they support a Virtual Ethernet NIC DMA for host-to-host communication instead of directly using TWC. At PCIE 3 x16 performance would theoretically exceed Infiniband 100Gb EDR by 25Gb/s and PCIE 4 x16 would have x2.5 EDR bandwidth at around 250Gb/s. Latency would be on par or slightly exceed RDMA without the application customization RDMA requires.

All that being said hardware is scarce or unobtanium. I did stumble upon the Broadcom P411W-32P that has the PEX88048 which pc-pitstop and provantage seem to sell new at prices somewhat on par with eBay EDR HBAs. Although it's targeted for NVMe attachment without requiring bi-furcation, I'm wondering if it could be purposed towards multi-host NIC DMA without impossible firmware modification.
 
  • Like
Reactions: Cixelyn