Infiniband 40Gbps question

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Psynapsx

New Member
Feb 21, 2021
29
4
3
hello,

I need a cost effective solution for a 40Gbps connection.
I have an all-flash NVMe storage server that is being heavily used 0-24 by 4 other servers (each has Mellanox CX312A 10GBe Dual Port ConnectX-3 NIC)

I was thinking about getting a Mikrotik CRS326-24S+2Q+RM. It has two 40G QSFP+ ports. I would get a HP Infiniband 2-Port QSFP 10GB/40GB 764284-B21 for my storage server. It seems it's just a rebranded Mellanox CX354A.
Then I would buy this Transceiver and this MTP Fibre cable.

Would all of this work to have a 40Gbps connection to my NVMe storage and theoretically saturate the 10Gbps connections from each of my 4 other servers to the storage?
thank you
 

Jab.R

New Member
Apr 16, 2016
16
1
3
59
Cincinnati OH
The reason the HP Infiniband adapter might be so cheap is that you might not be able to burn the Mellanox firmware which allows it to do Ethernet.
 

klui

Well-Known Member
Feb 3, 2019
824
453
63
They're cheap because they're EoL. 764284-B21 is a CX3 Pro, which is great because it supports RoCE v2, and it's an FDR/Ethernet part. You can use mlxconfig to change the ports to ethernet.
 
  • Like
Reactions: Psynapsx