Proxmox, ConnectX-3 QSFP+ to SFP+

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

superempie

Member
Sep 25, 2015
82
10
8
The Netherlands
My goal:
I'm currently in the process of migrating from VMware to Proxmox. Bought two almost new SFF boxes (Lenovo ThinkCentre M70S SFF) for my new base infra and want to connect them to my QNAP QSW-M1208-8C at 10GBit through SFP+.

Research done:
Next to some searching on the internet I was looking for Intel X520-DA2's with SFP+ before I found out you could find adapters from QSFP+ to SFP+ (source: https://www.servethehome.com/using-a-40gbe-qsfp-nic-with-a-10gbe-switch-sfp/ , https://forums.servethehome.com/index.php?threads/mellanox-qsfp-to-sfp-adapter-10.26005/). Also looked at Intel X710's, but they are more expensive. I prefer Intel or Mellanox NICs.

Situation:
I still have some Mellanox ConnectX-3 MCX354A-QCBT flashed to MCX354A-FCBT with QSFP+ on hand with low profile brackets. They aren't supported in newer VMware releases anymore, but still seem to be in Linux. Have done OFED installations before. Looks like the QSFP+ to SFP+ adapters are an option to keep the cost down.
For cables I was looking at: Cisco SFP-H10GB-CU3M Compatible 10GBASE-CU SFP+ DAC Twinax Cable - FS.com Europe

Disclaimer:
Could have posted this in the Linux section, but seems to me more like a networking thing. Mod can move it as needed.

Question:
Before I go buying other cables or even other NICs, any advice/experiences/...? Am I missing something? Am I on the right track? Or am I cheaping out?
 

DavidWJohnston

Active Member
Sep 30, 2020
242
188
43
Your plan should work, though I have not tried this exact combination. Your NICs appear to specify 10G/40G capability so they should link at 10G just fine. The DAC looks like a good choice.

Personally, I have done the opposite - Using a QSFP+ to SFP+ adapter in a switch in 4x10G breakout mode, and linking the 1st channel only at 10G. I do have to hard set the speed to 10G for it to work.

For the adapters, they can be annoyingly expensive, but if you're only buying 2 it's not so bad. Sticking with fs.com should be a safe option. I've bought plenty of sketchy ones on eBay and they've always worked, but YMMV.

I would like to add - Are those NICs dual port? If so you have the option of running another 40G DAC directly between your boxes - If you have some workload that could benefit from that or you just want to experiment, buying it in the same shipment may make sense.
 

superempie

Member
Sep 25, 2015
82
10
8
The Netherlands
The NICs are dual port, so it could be an option to run the second port on 40G in a card to card configuration. I already have a QSFP+ cable, so it might be useful for VM migrations. Will have a look at it. Thanks for the insights.