10-Gigabit Ethernet (10GbE) Networking - NICs, Switches, etc.

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
They won't be listed as fiber switches. Look for SFP+ switches. The fiber is connected using an SFP+ module.

google xsm7224s completed auctions on ebay, find dude that is selling them for the right price, email him with offer - done..
 

Scout255

Member
Feb 12, 2013
58
0
6
They were selling for around ~1200ish in like new condition. Apparantly that seller still has a few, they are just not on ebay you need to email him.
 

RADCOM

New Member
Aug 27, 2012
11
4
3
Great thread, I have been thinking about this for quite a while, thanks for all your efforts. Noob networker here with a few questions. So I can connect two 10Gbe network cards via GBIC>optical fibre cable>GBIC or use DAC? Are these direct attach cables and GBICs vendor specific i.e. do you have to use HP with HP? Will having a dedicated PC software switch really slow things down? I have no goal as such (fast link between two boxes running VM's would be nice) I just like playing around and trying to learn.
I have EMULEX LIGHTPULSE LPE1150 PCI-E FIBRE HBA CARD and two HP NC523SFP
Note I spell fibre differently I'm a Brit :)
 

33_viper_33

Member
Aug 3, 2013
204
3
18
Are these direct attach cables and GBICs vendor specific i.e. do you have to use HP with HP? Will having a dedicated PC software switch really slow things down?
I have the same question. I played with a qlogic card (i think) connected to my dell powerconnect 5224. I never go it working. I used some finisar SFP modules for the switch. I could never get the switch to even acknowledge there was something plugged into the SFP Ports. My theory is that the switch used a proprietary standard for DAC stacking purposes.

My goal is to do the same thing, software 10GB switch. I know it won't be the fastest switch in the world. For home use, I think it will be ok. I'm using ESXi with pfsense in a VM. I'm using openindiana with napp-it for a storage VM and windows 2012 essentials for my DC and PC backup. Openindiana and Windows communicate via iSCSI. I have 10GbE virtual NICs in all VMs to include the pfsense vm, and a Intel x540 NIC for 10 GbE physical connections to my backup server and main machine. My VMs are saturating my dell c6100's SATA controller over the Vswithch from the openindiana to the windows 2012 VMs. Openindiana definitely loves the 32gb of RAM in the node and doesn't give it back very willingly. I need to limit its shared memory and see if there is a performance hit. The dual L5520 processors do not appear very busy which is promising. When I get back to the states, I'll do some testing with the physical network cards with some faster SATA controllers and multiple SSDs. I won't be back home to try this for a few months though.

I would like to try this with a miniITX e3-1200v3 setup or hear of someone else's experiences. It would be awesome to have a NAS, Switch, and DC all in one package utilizing less than 30w idle. My fear is that this setup will crumble under the pressure of fast file transfers from openindiana to windows to backup server over 10GbE, especially with only 32gb of RAM max for ESXi. Anyone with input, please let me know.

As you can probably tell, my ultimate goal 10Gb/s with minimal power draw for very large file backups. My powerconnect 5224 is a power hog and I don't NEED that many ports. Therefore, I’m utilizing Intel pro/1000 NICs in my pfsense box to serve my main connections; firing up the switch only when extra ports are needed. This is another reason I'm looking into fiber since it is suppose to be lower power draw.
 

NetWise

Active Member
Jun 29, 2012
596
133
43
Edmonton, AB, Canada
33_viper_33: Are you trying to use 10GbE SFP+ cards connected to the 1GbE SFP+ ports on a > 10 year old Layer 2 1GbE switch? If I'm reading that right, that's likely the source of your issue.

When I do configs with Force10 S4810, Cisco Nexus 5548P, or PowerConnect 8024F, almost always I get away with TwinAx copper. Far cheaper and easier to work with in general. Except for Cisco, who are a bunch of lousy robbers :(
 

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
lol I get 7-8gbit with e1000e driver :) in a vm!

SFP is gigabit, sfp+ is 10gbe, It doesn't make sense to go 10gbe to 1gbe :) that is the job of a switch to pc not switch to switch!!!!
 

33_viper_33

Member
Aug 3, 2013
204
3
18
Are you trying to use 10GbE SFP+ cards connected to the 1GbE SFP+ ports on a > 10 year old Layer 2 1GbE switch? If I'm reading that right, that's likely the source of your issue.
:(
No, I was playing with 1Gb/s equipment just to learn about it. My intent was to connect two dissimilar switches. Link aggregation works just fine for what I do.

Recently, I've been getting greedy with my need for speed. I'm also about to replace my main storage server which will require the movement of over 12TB. I'm also backing up other servers off site at random intervals that are many TB and change drastically between backups. Bottom line is, I have a want for speed with out the budget to support a $4,000 switch. I can get by with my 2 Intel X520s and an Intel x540 cards for now. But as with any geek, I want more...

How do you like your Force 10? I've been eyeing them. Any idea on its power consumption? I like the looks of the netgear xs708. The price is almost reasonable...
 

mini-me01

New Member
Aug 6, 2013
25
0
1
I purchased a xsm7224s switch for my home lab which I have not installed yet. I am a bit concerned about the max 200w power consumption but will see how this goes with 4-5 ports active. I have been thinking about installing 2-3 Emulex OCE10102 cards in my ESXi box/VSwitch to have a workable 10Gbe switch that meets my needs with much lower power consumption. Has anyone tried this? Pros/Cons to this vs. xsm7224s for a lab environment?
 

33_viper_33

Member
Aug 3, 2013
204
3
18
Mini-Me01: I would be very interested to see your xsm7224's power usage, especially at idle. If you get an opertunity, please post.
 

PigLover

Moderator
Jan 26, 2011
3,186
1,546
113
Based on other switches of the same age and capability I think you can expect about 70-90w nominal load with just a few ports plugged in. Would be very interesting if it is much less than that.

I am interested in noise levels. I have a Juniper EX2500, which is similar but without the 10gbase-t ports and it is fairly loud (~55-60 Dba nominal). Other switches in this class are too.
 

nry

Active Member
Feb 22, 2013
312
61
28
My Dell 8024F switch consumes around 90w, the GUI says 78w idle with one SFP+ DAC connected though....
I think mkrad said this was the same as the Netgear xsm7224 in my thread.

On another issus of mine, I have posted this in my build thread but figure I'd post it in here too as it will probably get a little more exposure.

I have a couple of HP X240 1.2m DAC SFP+ cables. I am attempting to connect the following with these:
Intel X520-DA2 to 8024F
Dell 5524 switch to 8024F

But I just get the error on the 8024F switch
Code:
	AUG 15 03:19:19	DRIVER	Invalid Transceiver present in the 0/11 (slot/port)
I have updated to the latest firmware on the 8024F switch with no change.

If I connect one of the Intel X520-DA2 cards to my Dell 5524 switch it works perfectly so the cable and NIC are ok.
If I connect one of the Intel X520-DA2 cards to the Dell 8024F via a Dell (Cisco I believe) 5m DAC SFP+ cable it works perfectly, so the 8024F switch works.

It seems that the HP X240 cable may be incompatible with the 8024F switch, anyone got any suggestions for solutions to fixing this problem?
If not I will have to bite the bullet and buy a bunch of 5m cables and attempt to manage the excess cable
 

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
Run the diagnostics on the dac and see what brand it appears to be on working and non-working condition (switch side)
 

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
I purchased a xsm7224s switch for my home lab which I have not installed yet. I am a bit concerned about the max 200w power consumption but will see how this goes with 4-5 ports active. I have been thinking about installing 2-3 Emulex OCE10102 cards in my ESXi box/VSwitch to have a workable 10Gbe switch that meets my needs with much lower power consumption. Has anyone tried this? Pros/Cons to this vs. xsm7224s for a lab environment?
they all use the same delta power supply and fans.. did you disable the unused ports ?? Don't use the base-T ports at all those are huge power hogs.

if you get bored of it I'll buy it off you :)
 

nry

Active Member
Feb 22, 2013
312
61
28
Run the diagnostics on the dac and see what brand it appears to be on working and non-working condition (switch side)
Any idea on how to do this?

Can find some diagnostic tools for PHY on the 10GbaseT ports, and fiber ports to show optical transceiver diagnostics. But nothing else really.
Looking around page 1556 of this
ftp://ftp.dell.com/Manuals/Common/powerconnect-8024_Reference Guide_en-us.pdf
All I can find is
It will display an error if executed against a copper port
or passive or active direct attach cables
 

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
The nic on a windows box has amazing amounts of diagnostics. What I was looking for was the eeprom dump. Intel cards do look for something, and say if the eeprom was blank or defective, that might cause a failure.

X240 is a H3C cable, so I don't think those have the crypto chips in them like the X244(provision latest generation).

I bet you could google-fu a way to read the eeprom content and then compare/modify to determine if the bad cable is just a bad-eeprom.

Intel nic's have alwasy had issues with 10GBASE-T energy efficiency but that's not really a problem with DAC cables - but if a dac cable was throwing FF for a field that might need to be something else, it might think it is talking to the wrong cable.

AGAIN - CHECK the boards on used cards you buy. I found using a scanner is a great way, then overlapping two identical cards. I found about 2 out of a dozen cards had caps blown off or chips on edge knocked off. Oddly they still worked! One was a voltage reading system. I guess with DAC it didn't matter but you can imagine it couldn't be cool to have a few chips missing from a nic or blown off because someone plugged it into the wrong thing.

Remember DAC cables are not ethernet! infiniband and SAS used the exact same cable. You can imagine if someone plugged a cable into a different technology and blew the eeprom it could still function as a cable but be dead as far as "meeting spec".

But don't these cables carry lifetime warranty? I might ask hp! Worst case you get a no, best case you get a brand new cable.
 

mini-me01

New Member
Aug 6, 2013
25
0
1
they all use the same delta power supply and fans.. did you disable the unused ports ?? Don't use the base-T ports at all those are huge power hogs.

if you get bored of it I'll buy it off you :)
Straight out of the box, I am using 102W with two 10Gb DAC Cables and one 1Gb SR fiber connection. Where can I disable unused ports in the GUI/CLI?
 

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
Last edited:

mini-me01

New Member
Aug 6, 2013
25
0
1
http://www.dell.com/downloads/global/products/pwcnt/en/powerconnect-8024f-test-and-review.pdf

one of the most efficient switches of that era!

did you know you can turn on cut-through switching on these guys ;) you should try it . there's also rumours of being able to overclock them (DAC only) - there's a leeway on speed and some folks cheat a little by running a little faster.

overclocking ethernet or infiniband -> now that is something that sounds FUN!
I only have one PS so 102W was the total. I would guess that without anything plugged in, you are looking at 98-100W.

how do you turn on cut-through switching? I am used to procurve switches and there are so many buried config options on netgear that a few pointers on cut-through and disabling unused ports would be a big help!