10-Gigabit Ethernet (10GbE) Networking - NICs, Switches, etc.

[Patrick]

Specs say up to 5m DAC cable should work at 10gbps.

 

[Patrick]

Bought all of the cards with bulbs, there were only 3 of em to begin with
But cards without bulbs @ 45USD
Brocade 1020 Dual Port 10GB PCI E Fcoe CNA HBA Adapter | eBay

Just snagged 3x for $99.99 total. Not bad for 10GbE.

 

[azev]

Just snagged 3x for $99.99 total. Not bad for 10GbE.

I read somewhere online that those brocade card requires special cables/Dac to work, is this true?
How bout SPF compatibility?

 

[rnavarro]

Hey All,

I'm doing a fair amount of research on the 10G thing. I went ahead and pulled the trigger on 4x of the brocade 1020 cards:

http://forums.servethehome.com/networking/2955-brocade-1020-cna-10gbe-pcie-cards.html

Ended up getting them for $30ea best offer.

I'm going to drop these into my C6100, from what I understand direct connecting only works between two hosts, so I can't setup a ring between the 4 nodes (is that right?)

So with that it looks like I'll need a switch, cost is a big concern for me.

I've been looking at these:

Dell PowerConnect 6248 48 Port Gigabit 10 100 1000 Layer 3 L3 Managed Switch | eBay

or

Dell Force 10 SA 01 GE 48T Switch | eBay

I like the price of these, but there's only two 10G ports:

Quanta LB4M HP Brocade Open Source 48 Port Gigabit Switch w 2X SFP Tested | eBay

I planned to connect all of this up with DAC cables like this:
Brocade 10g SFP Fcoe 3M Fcoe Active Cable 58 1000027 01 | eBay

Am I asking for trouble with these switch options? Should I be looking at something else?

 

[PigLover]

The Dell 6248 and Force 10 switches you listed do have SFP ports...but they appear to be 1Gbe SFP and not SFP+. You need SFP+ to get use 10Gbe.

The Quanta switch does indeed support 2x SFP+ but - as you point out - only supports 2 links.

I use a Cisco SG500X-48, but these are ~$1300-$1800 on ebay right now. I know there are some similar Dell switches available but I don't recall the part #s.

 

[NetWise]

The Dell pc6200's do have a 10gbe 2 port module for the rear, that supports SFP+, but I don't have them to comment if they'll work in this scenario. The other Dell option would be a PC8024F.

 

[nry]

I have a Dell 5524 which has 2x SFP+ ports, seems to work pretty well but wasn't cheap, think it was around £400 here in the UK!

Interested in these Brocade NICs, would they work with the SFP-H10GB-CUxM leads?

 

[rnavarro]

Thanks for all the replies, the infiniband route is starting to look a little more reasonable. But I'll keep looking around for some good deals.

 

[s0lid]

Oh right i've had this lying around for a while in my rack cabin:

Fulcrum Monaco, doesn't work though. Kinda suspecting the capacitors. Most of the surface mounts are normal aluminium caps and the memory backup capacitor is bulged. Will look into it when I get parts from digikey.

 

[jingjing]

I think there is a reason Infiniband has gone from the brink of becoming irrelevant to growing at a fast pace. 10GbE over twisted pair is a 6-7 year old standard at this point.

 

[33_viper_33]

I think there is a reason Infiniband has gone from the brink of becoming irrelevant to growing at a fast pace. 10GbE over twisted pair is a 6-7 year old standard at this point.

And still hasn't gone main stream. One would think it would be hard to find a server board with 1GBE now, not the other way around.

 

[dukzcry1]

Obviously, i'm not requiring a quick drag-n-drop way as usual way. Just an achievable way to transfer files between 2 computers linked w/ my right new IB LAN !!!

 

[soapbox]

Just snagged 3x for $99.99 total. Not bad for 10GbE.

If I put one of these Brocade 1010/1020 cards in each of my two Win7/8 computers with SSD drives in them, and connect the two SPF+ cards directly with Brocade active twinax cable, will I be able to transfer files between the two computers at 10Gbit (or as much as SSDs performance will be)?

Does this kind of connection require the same kind of setup as with ethernet (IPs, workgroups), or is it different?

 

[jingjing]

So many different things at the same time : Infiniband, 10GbE RJ45, Thunderbolt ...all cards seem to be dual-port : can it be aggregated (20Gbs between hosts ?) or it has another goal (daisy-chain ?) ?

 

[whitey]

Any chance on the several pages we built upon this topic/thread being restored since 'the great crash/outage'? I don't really feel like re-populating all the info I posted up and was hoping others could benefit from the good knowledge that was recently shared in this thread. We were up to page 16 or 17 so we're missing 3-4 pages :-(

Any chance of a semi-recent backup being around that we could inject the missing data back into this post?

 

[Mastaba]

I'm planning to go to the 10Gb way for three servers (two ZFS boxes and one windows machine) backupping stuff between them and serving several clients, what do you think of the Netgear GS728TXS? It's way cheaper than the Cisco SG500X-24, is it missing something important or we pay the name?

Since 10GBase-T need power hungry switches (=noise and power consumption i would like to avoid as much as possible) and i as wanted more than 8 ports (=XS708E seems the only decent alternative, the XS721T consuming much more power and being much more noisy) for more clients i though of chosing SFP+ instead.
I think of getting three Supermicro AOC-STGN-I2S and three Supermicro SFP+ CBL-0348L 3m, is there some compatibility issues or problems i didn't take into account between all this hardware?

I will use OmniOS and nappit to run the ZFS servers, it seems Illumos is compatible with "Intel Corporation 82599EB 10-Gigabit SFP+ Network Connection", does that mean the 82599 based AOC-STGN-I2S is also compatible?
Because it's cheaper than the Intel X520 and i read there's some compatibility issues with non-intel gear on the newegg review page.

 

[whitey]

Seems to be a Korean/Chinese gentlemen that has hacked OmniOS to support the Brocade 1020's. Heck if I could translate it even w/ sw translation tools/browser language conversion support.

I have the Intel X520DA2's in my vSphere/ESXi 5.5 boxes as well as my all-in-one hypervisor/SAN build running OmniOS and they purr (Near theoretical 10G speeds w/ iperf VM to VM hypervisor host to hypervisor host through fully-tuned end-to end jumbo frames enabled network). YMMV w/ other 10G NICS/CNA's.

Intel is WELL supported generally is all I know but I understand the pricepoint hurts...hell I paid for 3 of the 520's before I tried a 1020 just to see if it will link up and perform as well as the 520's connected to my HP Procurve 2910al 10G SFP+ ports. On a side note, I just popped the 1020 into a spare OmniOS build and no detectey :-(

Hey anyone know on a side note why my Brocade 1020 adapter I got the other day shows BIOS not installed on boot but when I network install CentOS 7 onto it it detects them fine as interfaces (installed over a Broadcom 1G conn) as I don't have my Brocade active Twinax/DAC yet though to fully confirm if the Bios not installed looking error is catastrophic and I need to go run a Brocade/Q-Logic tools/driver live cd to update/flash it if so. Any comments would be greatly appreciated?

 

[firenity]

Hey anyone know on a side note why my Brocade 1020 adapter I got the other day shows BIOS not installed on boot but when I network install CentOS 7 onto it it detects them fine as interfaces (installed over a Broadcom 1G conn) as I don't have my Brocade active Twinax/DAC yet though to fully confirm if the Bios not installed looking error is catastrophic and I need to go run a Brocade/Q-Logic tools/driver live cd to update/flash it if so. Any comments would be greatly appreciated?

Mine said the same when I first got them (old boot image).

I updated the boot image (BIOS) with the Brocade Command Line Utility (BCU):
bcu boot –update <boot-firmware> -a

 

[Quorthon]

Hi all !

In the mind to spending my core network / server 10gb : I have a DLINK DGS 3120 48pc ports switch, with two stacks ports (cx4) that can be configured as a single port 10gb.

According to you, I can use a CX4 cable type to QSFP to connect the switch to a map Mellanox connect x3?

 

[whitey]

OK, so I updated the BIOS/boot code via their live linux CD to 3.2.4.0 and loaded the ESXi drivers, the adapters show up but wont link. I see this error on my HP Procurve 2910al?

Anyone have any idea why I cannot get my 1020 w/ active twinax to play nice w/ my HP Procurve 2910al?

I ordered this cable:

Brocade 10G 1M FCoE Active Cable 58-1000026-01 30AWG SFP TO SFP