FreeNAS 9.3 & 10Gbe

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

DBordello

Member
Jan 11, 2013
43
0
6
I am looking to add 10GBe between two Hyper-V hosts and my FreeNAS setup.

What cards do we like on the FreeNAS side? I have heard some iffy information on the Intel cards.

I would prefer 10GBASE-T, since that seems the simplest.

What is working well for everybody?
 

DBordello

Member
Jan 11, 2013
43
0
6
I am thinking I should relax the 10 GBASE-T requirement. It appears SFP+ is much more common. If I can get 2 port cards, I can avoid a switch anyways.

However, SFP is new and scary :)

As always, budget is a consideration. eBay would be ideal.
 

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
Chelsio is well supported on BSD variants, else Intel I would recommend. I have MLX 40G eth but that's at the vSphere layer and using vmxnet3 10G vnics on my FreeNAS.

YMMV :-D
 

K D

Well-Known Member
Dec 24, 2016
1,439
320
83
30041
I have used Mellanox CX2 and CX3 along with Intel X552. But in an esxi/freenas AIO config like @whitey .
 

CJRoss

Member
May 31, 2017
91
6
8
I did 10GBASE-T with my FreeNAS but that was using the built in Intel NIC of my Xeon D. You need to be running at least 9.10 in order to get the drivers.

The Chelsio NICs are the recommended NICs for FreeNAS. SFP+ is cheaper and easier to find than 10GBASE-T.
 

zhoulander

Active Member
Feb 1, 2016
181
46
28
Do you mean Freenas 9.10.3 and not 9.3?

As of 9.10 Mellanox driver support is included. ConnectX2/3 cards are plug and play and really cheap. I've used ConnectX2 and 3 (SFP+) and also Chelsio T420-BT (Base-T) cards in my Freenas boxes.
 

DBordello

Member
Jan 11, 2013
43
0
6
Do you mean Freenas 9.10.3 and not 9.3?

As of 9.10 Mellanox driver support is included. ConnectX2/3 cards are plug and play and really cheap. I've used ConnectX2 and 3 (SFP+) and also Chelsio T420-BT (Base-T) cards in my Freenas boxes.
The price on the ConnectX2 & 3 are fantastic. If these work well, this might be the route I end up going.
 

i386

Well-Known Member
Mar 18, 2016
4,221
1,540
113
34
Germany
The bracket looks weird. Make sure that it's a normal pcie card and not some proprietary/custom card that can fry your server(*)/hosts.

* There are some mellanox flexlom cards that look like pcie cards but have a different pin layouts. A user on this board fried his server when he put such a card in a normal pcie slot.
 

PigLover

Moderator
Jan 26, 2011
3,184
1,545
113
Thoughts on this card? MHQH29B-XTR MELLANOX CONNECTX-2 VPI 10Gbe DUAL PORT ADAPTER CARD | eBay

My goal would be to pickup 3 of them, and connect my FreeNAS to each of my Hyper-V servers. I assume these will work on Hyper-V or ESXi.

What cables should I look for? 3ft should be plenty.
Two issues with this one:
- Its a 40Gbe card using QSFP connections (not SFP+)
- The bracket is not a normal "PCIe" bracket - you'll need to get a stardard full-height bracket for it.

You can use the card for SFP+/10Gbe but you'll need to get adapters for it. That and the need to replace the bracket (probably at stupid prices) would tend to make me avoid this one.
 

herby

Active Member
Aug 18, 2013
187
53
28
Two issues with this one:
- Its a 40Gbe card using QSFP connections (not SFP+)
- The bracket is not a normal "PCIe" bracket - you'll need to get a stardard full-height bracket for it.

You can use the card for SFP+/10Gbe but you'll need to get adapters for it. That and the need to replace the bracket (probably at stupid prices) would tend to make me avoid this one.
You can actually get brackets from Mellanox directly pretty cheap: Mellanox Infiniband and Ethernet Adapter / Network Interface Card Brackets

Looks like $4.30 + shipping for that card's bracket. Could be a pretty good deal if you actually want 40Gbe/don't mind using the QSFP adapter; which cost another $10-$15 per port.
 

herby

Active Member
Aug 18, 2013
187
53
28
Oh wait this card isn't 40 GbE at all. I just pulled up the manual it's 10 GbE or 40 Gb InfiniBand.

I don't think this card is what you want at all.
 

herby

Active Member
Aug 18, 2013
187
53
28
I added 10 GbE to my two Xenservers and Freenas earlier this year. The cheapest way I could find was two single port Dell branded Connectx-2 cards with (I think Cisco) DACs for the Xenservers, and a dual port HP (516937-B21) Connectx-2 for the Freenas.
 

DBordello

Member
Jan 11, 2013
43
0
6
@DBordello
If you want to use 40gbe look on ebay for "HP 649281-B21", that's a dual qsfp port 40gbe vpi card.
649281-B21 656089-001 661685-001 HP IB FDR/EN 10/40GB 2P 544QSFP ADAPTER | eBay
After some decision making on other fronts, I am finally ready to pull the trigger. If I can go 40gbe, why not :)

I am looking at something like this:
649281-B21 656089-001 661685-001 HP IB FDR/EN 10/40GB 2P 544QSFP ADAPTER | eBay

In addition to the card, what other cables, etc. do I need? The servers are within 1m of each other.
 
Last edited: