10Gb/40Gb upgrade considerations - advice wanted

[iZohanX]

Hi folks,

I'm about to have my incoming WAN connection from my ISP upgraded to a 3Gb plan.
The connection will be provided via SFP+ module into a router/switch of my choice

My current plan is a 1Gb plan, coming in via SFP into a Ruckus ICX 7150 -> ESXi host -> pfSense
The ESXi host specs are as follows:
i5-6600K
16GB RAM
120 GB SSD
Intel i350-T4 (4ports@1Gb)

The upgrade to 3Gb WAN would require me to move to 10 gig cable equipment in my ESXi host and my PC in order to make the most out of it.

My main considerations are that I also own 2 Brocade 6610-48P
Which have 40-Gig ports - I never quite understood if I'm able to use them as 40-Gig uplinks to an ESXi host/NAS
I'm thinking, if I'm already upgrading, why not "go big"?!

• If I am able to use them as uplinks, what would be the best NIC option to purchase that's also economical?
  I've seen the connectx-2/3 cards going fairly cheap but I'm not sure if they would work nicely with the Brocade switch or if getting the drivers to behave with ESXi/pfSense (in the future, I plan on moving pfSense to run on bare metal on a seperate box rather then as a VM).
  
  I was also considering the Chelsio T580 cards, as I understand from the STH article on the matter they are fully supported in pfSense/FreeBSD but as they are on the pricier side as compared to the Connectx variety I was having second thoughts.
  
  If going the 40gig route, one thing that really trumps me is what cables do the 40gig modules use, I understand they are called MTP/MPO (one of which is the same as the other but rather a name for a patended plug or something?), but I also understand they have a polatiry?
  Do correct me if I'm wrong about it.
  
  
• If I'm unable to use them as 40Gb uplinks, I would just use the 10gig SFP+ ports wich brings me to my other consideration - Which 10 gig SFP+ cards can I find that are fairly cheap but would also be fully capable

My end goal is to be able to fully utilize my 3Gb WAN connection, but also be able to upgrade the capability of file transfers between my TrueNAS and my main PC
All the machines including my PC will be upgraded with the same NIC once I've come up to the conclusion of the best path.

To be clear, currently I don't have any workloads that really require ultra high speed between my NAS and my PC, but that's because most of my storage drives are in my PC, which is something I plan on changing as I build out my home lab.

I would appreciate any input you guys could provide me with and I thank you for taking the time of reading through this =)

 

[LodeRunner]

40G QSFPs: fs.com has BiDi models that can do 40G over a single OM3/4 pair. If pfSense is a VM on you ESXi host, then all that matters is that ESXi supports the NIC; pfSense should only see a VMXNET3 device and have no awareness of what the actual hardware is, unless you're doing SR-IOV, in which case, I have no experience and thus no knowledge to pass along.

Connectx-3 cards come in both 10G SFP+ and 40/56G QSFP flavors. Just be sure it's a card that has Ethernet mode as there are some IB only models. Install card, install drive, configure as Ethernet mode, and off you go. I think for the IB/Eth cards it needs to be a VPI model to have Eth mode?

Here's the single port CX3 card in my workstation; it's a 10G Ethernet only card:

 

[iZohanX]

I went ahead and got myself a couple of Connectx-3 FDR cards (should be capable of 40gig Ethernet from what I read).
Hoping that installing and configurations go smoothly

Thanks for the advice