40GbE - will that hardware combination work?

[DeepB]

Hello,

I am upgrading my homeserver to a 3-node CEPH+Proxmox HA-VM Cluster.

As 1GbE will probably be too slow for CEPH I am considering upgrading. 10GbE and 40GbE is not that big a price difference so I am thinking of going directly to 40GbE.
I am situated in Europe.

As my netorking experience stops at 10GbE-Fiber I have a few questions if my setup will work:

Central Switch: Mellanox MSX6025T-1SFS InfiniBand Switch 36x 56 Gbps FDR IB Ports 712495-B21 (~220€)
DAC Cables: Brocade 40G-QSFP-QSFP-C-0101 1.0m QSFP-QSFP Kabel (SAS) (~11€)
PCIe-QSFP-Cards: MELLANOX - MHZH29-XTR (~36€)
MTP-LC Breakout cable: LWL Breakoutkabel MTP auf LC Kabel, 1m, MTP Buchse auf 4 LC, OM4 BIF, Magenta
QSFP-Transceiver: Mellenox QSFP+ Modul | 40G QSFP+ MC2210411-SR4 40GBASE-SR4 Transceiver

My questions:
1) Does this setup work? Especially the combination DAC/Switch/PCIe-Card?
2) Can I use this transceiver with the Switch and a Breakout cable? So I can connect one qsfp 40GbE poprt to 4 SFP+ 10GbE ports?
3) Are there any alternative better/cheaper suggestions than the one above?

thanks
Daniel

 

[Emek]

MSX6025T-1SFS is unmanaged Infiniband only.
You need 6012, 6018 or 6036 + gateway license if you want ethernet.

The cards are connectx-2. 3 is Better and not that
expensive.

The cables should be ok, but DAC would be cheaper.

I have some switches and cards if you're interested.

 

[DeepB]

I definately am interested, please let me know how much you want for them (by private message?)

 

[Emek]

PM sent

 

[DeepB]

Anybody knows if the Mellanox IS5030 can do ethernet?

 

[Emek]

Infiniband only.

 

[DeepB]

ok, thanks for the answer.

is there a list of all mellanox ethernet-capable swtiches somewehre?