ConnectX 3 vs 4 vs 5 and semi-related optics question

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
jolness

jolness

Member
May 25, 2020
35
9
8
Hey all,
I am going to grab a a 10Gbe NIC for my servers and I think Mellanox seems like where I will go (relatively cheap, even for the SFP28 models if I ever want/need to use 25Gbe and they are transceiver/optic agnostic) but honestly I am hitting decision paralysis.
Prices are
ConnectX-3 $20ish
ConnectX-4 (lx) - $30ish
ConnectX-5 - $60ish
I know the former 2 are EoL (and maybe the connectX-5, seems like they do not support NICs as long as Intel who I am more familliar with) but I am using them with TrueNAS scale and Proxmox so my hunch is it really doesn't matter.
At the moment I am leaning toward the 4. More modern, SFP28 and an extra $10 is not meaningful. An extra $30 isn't much of a jump but if it's a "doesn't matter for my usecase, even in 5 years time" situation then might as well not. If there are benefits to the 5 I am missing, please let me know.
Just want to make sure I am understanding properly.
My big concerns are
1) it being viable in linux based OSes for the next few years at least
2) a 120mm noctua fan at 100% blowing over the PCIe slots being enough airflow since I do not have rack mount chassis (can get a little fan to mount to the heatsink if needed but currently have the fan cooling my HBA and thermals are fine on the card, even under heavy I/O)

Optics:
I have a Sodola 8 Rj45+ 1 SFP+ port unmanaged switch (new style with the two rows of 4 RJ45), as far as I can tell, they do not seem to be the least bit picky about optics but want to make sure before I order the transceivers with the NICs and 1 for the switch.

Thanks in advance!
 
N

nexox

Well-Known Member
May 3, 2023
1,340
613
113
The driver for the 4 and later cards is unified and quite a lot of people run ConnectX 4 in production still, so that should be supported for a really long time. I'd just get the 4 Lx, by the time you have a good reason to use a 5 they will probably be less than $30.
 
  • Like
Reactions: jolness
jolness

jolness

Member
May 25, 2020
35
9
8
@nexox that was what I had gathered but wanted to make certain. I didn't know that about the drivers. Most of my focus was if the large PC OEM branded ones could be flashed to run mellanox firmware (yes without much work it seems). I couldn't find a reason to get the X5 but figured there might be some issue on the 4 or feature on the 5 worth worrying about.
They're agnostic on optics, correct? I haven't found any "mellanox compatible" like with brands who do try to restrict things.

Appreciate the response!
 
N

nexox

Well-Known Member
May 3, 2023
1,340
613
113
The features added to the ConnectX 5 are mostly rather niche and often hard to even use with the publicly released drivers, the transceivers aren't brand locked and I haven't found anything that doesn't work with my SFP28 cards. Do make sure to buy one with the correct bracket for your system, brackets on their own are like 40% of the price of a whole NIC.
 
  • Like
Reactions: jolness
jolness

jolness

Member
May 25, 2020
35
9
8
@nexox Perfect. My needs are basic. I just need an SFP+ compatible card that runs under Linux and doesn't have a ton of bugs.
I found a super cheap low profile bracket clad CX5 but couldn't even find a full width bracket. The ones for the CX4 ended up making the "cheap" one I found no cheaper once I paid for shipping on both items lol. I made that mistake on an HBA many years ago and I lucked out, the seller was super nice and tossed a full width one in a bubble mailer for free. That was 6-7 years ago and I still check that seller first whenever I need something.

Anyways, I appreciate the help. I was 90% sure this was the direction I would go but I've made a few 90% purchases and end up finding I should have checked for that last 10%. o_O
 
  • Like
Reactions: nexox
I

i386

Well-Known Member
Mar 18, 2016
4,448
1,663
113
35
Germany
Cx3 roce v1 only
Cx4 First Gen with 28gbit/s serdes (25 & 100gbe), roce v1 & v2
Cx5 second Gen with 28gbit/s serdes, First Gen with pcie 4.0 Support (for dual Port skus)
 
  • Like
Reactions: jolness
You must log in or register to reply here.
Top Bottom