Looking for a SFP+ card for my FreeNAS box

May 1, 2020
37
8
8
INFO EDIT:
I'm looking at getting a Brocade ICX6450-48P and upgrading my server and desktop to 10GbE. My server is a FreeNAS 11.3 box with PCIe2.0 x8 ports, and my desktop is a modern Windows 10 build.

I'm trying to find a SFP+ NIC for my FreeNAS box but I'm a little overwhelmed with the options. I've seen people recommending all sorts of cards from Intel to Mellanox to SolarFlare. Narrowing them down to a reasonable price and trying to confirm compatibility has gotten my head spinning a bit.

It's running on an older box that has PCIe Gen 2 x8 ports available on the riser. My fallback would be this card, if has good driver support, because it should be the mezzanine card for my server box. However, at $70 I think I can go cheaper as I've seen cards like this SolarFlare for $20 or less, like this HP NC550SFP.

I'd happily snap up one of those cheaper cards but I saw some posts elsewhere that suggested that drivers could be an issue for some cards (not these specifically) and that got me worried. I'm having trouble verifying if these are 100% supported or not.

I am also looking for a SFP+ NIC for my desktop PC running Win10. I'm planning on getting a 10GBase-T adapter/transceiver for the desktop and the switch I'm planning on getting so that I can get 10Gbit access to my FreeNAS server.

I'd love any help or advice you guys can give because I've officially managed to hit information overload/analysis paralysis.

EDIT: I'd also appreciate suggestions for what transceiver to use for connecting the server to the switch. I don't know if it's better to get normal transceivers and then a separate cable or to get one of those direct attach cables that just plugs into both.
 
Last edited:

zack$

Active Member
Aug 16, 2018
584
222
43
Get a Chelsio card for the FreeNAS box and a Mellanox Card for the Win10 box. You can get some cheap DACs that will work as well:



 
May 1, 2020
37
8
8
Get a Chelsio card for the FreeNAS box and a Mellanox Card for the Win10 box. You can get some cheap DACs that will work as well:

I forgot to mention that the server should be full height. Is there a listing where I can get a full height shield(?) separately or should I just look for another listing of the same card that comes with a full height shield(?).

Not that it's a big deal, but what's the reason behind doing 2 different cards for the server and PC. Is it drivers? 10Gbase-T compatibility? or something else?

Then I'll just look around for something if it's not a big deal. I can either get the $10 ones from FS.com or I'll see if I can get a single off ebay somewhere.
 

neb50

Member
Aug 28, 2018
57
15
8
The Mellanox Connect 2 and 3 cards should work out of the box in FreeNAS 11.2 and above. I have used both in the past with no issues at all with both single and dual port models. They will also work with Windows, so you can use the same cards and buy a spare that will work for both if wanted/needed.
 

zack$

Active Member
Aug 16, 2018
584
222
43
Drivers, yeh. The Chelsio are rock solid on FreeNAS. The Mellanox cards work on anything and is likely your best investment for driver support for Windows for a while.

While the X2 still work, IMHO the X3 is a better fit as its pcie 3.0 and will likely have better driver support on many different platforms in the future.

Apart from the above, you can get an Intel card based on the 82599EN chip and that will work fine on everything. However, it would probably cost more and for FreeNAS you may need an Intel based DAC or transceiver.

Those DACs are Arista branded and work well. For the price (brand new), I doubt you will get a better deal anywhere else.

Edit: The solarflare cards are good but driver support is gonna be an issue. They do work on FreeNAS with the sfxge driver and the newer sfn7xxxx cards should still work on Win10.
 
Last edited:

RTM

Active Member
Jan 26, 2014
665
240
43
It's running on an older box that has PCIe Gen 2 x8 ports available on the riser. My fallback would be this card, if has good driver support, because it should be the mezzanine card for my server box. However, at $70 I think I can go cheaper as I've seen cards like this SolarFlare for $20 or less, like this HP NC550SFP.
I am not quite sure I follow everything you are wanting to do, but the first mezz card is obviously Dell (probably proprietary, and also does not appear to be useful for Ethernet, unless I am missing something), the two others are standard form factor as far as I can tell, something does not quite add up. Are you saying that you ideally want to use the mezzanine slot (and that you have a Dell server) in your server, but you might consider using a different regular slot?

If all you have is a mezzanine slot, you should definitely verify what it is you have, as many of those are proprietary and will not take "standard" sized cards. You may want to give us some information on your server. If you have both available as options but you prefer mezz slots, then you may want to look at the options provided by the server manufacturer with that specific server, you may be able to get a good deal (though I doubt better than 20$).
 
May 1, 2020
37
8
8
I am not quite sure I follow everything you are wanting to do, but the first mezz card is obviously Dell (probably proprietary, and also does not appear to be useful for Ethernet, unless I am missing something), the two others are standard form factor as far as I can tell, something does not quite add up. Are you saying that you ideally want to use the mezzanine slot (and that you have a Dell server) in your server, but you might consider using a different regular slot?

If all you have is a mezzanine slot, you should definitely verify what it is you have, as many of those are proprietary and will not take "standard" sized cards. You may want to give us some information on your server. If you have both available as options but you prefer mezz slots, then you may want to look at the options provided by the server manufacturer with that specific server, you may be able to get a good deal (though I doubt better than 20$).
I am rather certain that the mezzanine card is the proper one for my Dell C2100. I liked the idea of it because I could be certain that it wouldn't mind PCIe Gen2, since it's made for this machine, and also the idea of it saving my proper slots for anything I might grab in the future (like a small gpu for NVENC).

However, I am more interested in getting a cheap (or reasonably cheap) card that I won't have to worry about driver issues or configuration problems.

Drivers, yeh. The Chelsio are rock solid on FreeNAS. The Mellanox cards work on anything and is likely your best investment for driver support for Windows for a while.
Yeah, I'm looking at the FreeBSD release page to try and make sure things line up driver-wise and it looks like you are right about the Chelsio (which based on some searching I guess is the T420-CR but not labeled normally? and is $50 cheaper than normal) and Mellanox.

I also saw in another thread (that I think you starred in) that the older Chelsio cards indeed don't work well on modern Windows and that makes the Mellanox a good choice.

Edit: The solarflare cards are good but driver support is gonna be an issue. They do work on FreeNAS with the sfxge driver and the newer sfn7xxxx cards should still work on Win10.
Yeah, looking at the FreeBSD docs I'm not sure what SolarFlare cards are supported. They only list SFC9000 family controllers as supported with the sfxge driver and I'm not certain which cards that covers (their website isn't the best). So I can't tell for certain if they would be properly supported there.

Looking at the driver portion of their site, suggests that they do have drivers for the SFN7xxxx series for Windows Server 2016 and I'd expect those drivers to work on Win10. Though I don't know if it'd actually be worth trying to save the $20 over the Mellanox card here.

Those DACs are Arista branded and work well. For the price (brand new), I doubt you will get a better deal anywhere else.
My only hold-up on getting those specific cables is just that I don't think I need 5 of them. Only my server would be direct attach, so that would be at max 2 cables.

You can find HP branded CX3 dual port 10/40/56G cards for less than $30, and they can be flashed to Mellanox firmware easily :D

That looks cool, but I'm not certain that I need/want a QSFP card. The switch I'm looking at doesn't have QSFP ports, so I'd need either a breakout cable or converter and that would add to the price. Even if I did have a QSFP port on the switch, 40GbE would be far past what I'd need for it.

Though if you can find a 2x10GbE version (I have no idea the product numbers) and a tutorial for flashing them over I'd be very interested. It'd drop the costs a little bit and I could reasonably get matching cards for everyone (just feels cleaner that way).

P.S. sorry for the text wall everyone, I wanted to reply to everyone and comments kept coming in as I was composing :p
 
May 1, 2020
37
8
8
@zack$ do you know if there are any better options for a full height bracket for the Chelsio? I've only found ones from China that'll take about a month to arrive. I think that your picks are actually really good options (after I did all that further looking up and seeing how much cheaper these listings are than most others) but I didn't want to jump on the Chelsio until I could find a better bracket.

If not then I can always grab it and just wait for the replacement to arrive or 3D print one if I'm feeling daring.
 

SDelong

New Member
Jul 30, 2018
4
1
3
You can find HP branded CX3 dual port 10/40/56G cards for less than $30, and they can be flashed to Mellanox firmware easily :D


@BenjaminCalibration Bought 5 from this seller with an offer of $15 each as an FYI. You can pick up QSFP to SFP adapters as well. Might be a better choice if you plan to upgrade in the future.
 
May 1, 2020
37
8
8
@BenjaminCalibration Bought 5 from this seller with an offer of $15 each as an FYI. You can pick up QSFP to SFP adapters as well. Might be a better choice if you plan to upgrade in the future.
I'm looking for a QSFP to SFP adapter that would make the total price comparable but having trouble. Also I'll need to get full height brackets for them.

Lastly, can you attach only 1 or 2 connectors on a QSFP to SFP DAC? Or would I have to use a QSFP to SFP adapter with a SFP-SFP DAC?
 

SDelong

New Member
Jul 30, 2018
4
1
3
I'm sure ebay has some full height brackets floating around.

Not sure I'm following your question. If your going to get a 6450 you wouldn't need a QSFP DAC. It would be switch SFP port > SFP DAC > an adapter(MAM1Q00A-QSA for example) > NIC
 
May 1, 2020
37
8
8
Not sure I'm following your question. If your going to get a 6450 you wouldn't need a QSFP DAC. It would be switch SFP port > SFP DAC > an adapter(MAM1Q00A-QSA for example) > NIC
That was option 2. I could've seen it being easier in some situations to do a qsfp to 4xsfp dac and leave some disconnected (if that even works) but the other option was exactly what you posted.

Also do you know any good listings for the adapters?
 

SDelong

New Member
Jul 30, 2018
4
1
3
Ah so you were thinking a QSFP breakout cable, QSFP side to the nic? No clue if that would work, someone smarter than me would have to answer.

No idea, I'd target $15-20 each shipped though.