Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[anemoiac]

@EngChiSTH

Thanks for the feedback! I hesitated when I asked about the Brocade being the "best cheap used enterprise 10G switch in 2021" because I know that such a thing doesn't actually exist. Obviously, my specific use-case will dictate what's "best" for me, and what's best for me might not be a good choice for someone else.

The main problem I'm facing is my lack of knowledge/experience, as I'm basically diving headfirst into this world starting from scratch. I have virtually zero networking knowledge. I'm in the process of setting up a homelab to learn more about a variety of IT-related topics and, hopefully, launch an IT career.

As of now, my network consists of a dumb switch plugged into an HP T730 Plus running pfSense. I've recently acquired a half rack and a couple of Dell PowerEdge servers, one of which will operate primarily as a NAS and the other a virtualization server. I have a Windows/Linux PC and a Mac mini that will regularly interact with these servers over the network. My rack sits right next to the desk that houses the Mac and PC, so I won't need to run wire more than a few feet.

I don't have a specific requirement for my network to be 10G, but it seems like it would improve my experience while also compelling me to learn a bit more about the associated tech. I don't know enough at this point to care about the difference between fiber and copper, but my budget is fairly limited - I'd like to keep my spending to a few hundred dollars to begin. I already have 10G SFP+ NICs in the Dell servers, but that budget will need to include a switch + cables + transceivers + a NIC for my Windows/Linux desktop (I'm not worried about the Thunderbolt 3 - SFP+ adapter that the Mac mini will require yet, as that will probably cost half as much as everything else combined). Based on that description, the MikroTik switches seem like a good fit, but I'd prefer to use an enterprise switch with full L3 capabilities.

From what I've read so far, I see no reason why a Brocade ICX switch won't suit my needs; I'm just trying to be (at least a little bit more) thorough before I pull the trigger. Thanks again!

 

[EngChiSTH]

@EngChiSTH

Thanks for the feedback! I hesitated when I asked about the Brocade being the "best cheap used enterprise 10G switch in 2021" because I know that such a thing doesn't actually exist. Obviously, my specific use-case will dictate what's "best" for me, and what's best for me might not be a good choice for someone else.

The main problem I'm facing is my lack of knowledge/experience, as I'm basically diving headfirst into this world starting from scratch. I have virtually zero networking knowledge. I'm in the process of setting up a homelab to learn more about a variety of IT-related topics and, hopefully, launch an IT career.

As of now, my network consists of a dumb switch plugged into an HP T730 Plus running pfSense. I've recently acquired a half rack and a couple of Dell PowerEdge servers, one of which will operate primarily as a NAS and the other a virtualization server. I have a Windows/Linux PC and a Mac mini that will regularly interact with these servers over the network. My rack sits right next to the desk that houses the Mac and PC, so I won't need to run wire more than a few feet.

I don't have a specific requirement for my network to be 10G, but it seems like it would improve my experience while also compelling me to learn a bit more about the associated tech. I don't know enough at this point to care about the difference between fiber and copper, but my budget is fairly limited - I'd like to keep my spending to a few hundred dollars to begin. I already have 10G SFP+ NICs in the Dell servers, but that budget will need to include a switch + cables + transceivers + a NIC for my Windows/Linux desktop (I'm not worried about the Thunderbolt 3 - SFP+ adapter that the Mac mini will require yet, as that will probably cost half as much as everything else combined). Based on that description, the MikroTik switches seem like a good fit, but I'd prefer to use an enterprise switch with full L3 capabilities.

From what I've read so far, I see no reason why a Brocade ICX switch won't suit my needs; I'm just trying to be (at least a little bit more) thorough before I pull the trigger. Thanks again!

Leaving the topic of starting IT career out of this discussion (for that homelab would help only in few very specific sub-specialties dealing with actual network/hardware), yes if what you want is to play with 10Gb speeds (and you already have the SFP+ cards), then either MikroTik or Brocade would work. If you want to play with VLANS and "L3 stuff" then Brocade is probably better if you go. I looked at e-bay and cheapest is $125 with shipping include in USA . this is more than many of us paid historically and could be just normal flow of prices on e-bay


if you have specific questions, feel free to ask

 

[ArmedAviator]

@anemoiac

You'll be well served by an ICX that suits your needs. Good prices on the used market, free firmware direct from the manufacturer (albeit it the 6xxx series are all EoL), and good performance.

If you can, go with the 7xxx series. If price is a major concern, the 6xxx series is just fine for your needs.

Since you're trying to learn networking, I recommend getting two or more managed switches to practice setting up and using various protocols between them (VLAN, LAG, RSTP, CDP, LLDP, stacking, etc.).

I have an ICX 6610 and an ICX 6450 and use them both on my home network. They've been great for learning and for home use. If I find another dirt cheap ICX6610-48P, I'm going to get a second and stack them i my rack for additional redundancy.

Oh and 10 gig has made a tremendous improvement in my use cases at home. Proxmox cluster migration between 3 R710s occurs significantly faster - really makes a difference when one of my VMs has a total of 6TB of virtual drives. Trust transferred it betwen hosts last night as I upgraded HDDs and re-created my RAID array on one host. Sustained speed was limited by only the HDDs/RAID. Also using a Gluster cluster is great for my NAS needs with 10Gig or better as a client writing to Gluster sends the data simultaneously to all Gluster server nodes - so when I see my PC writing files to the cluster at 400MB/sec, the network throughput is actually 800MB/sec since it's sending identical data to two servers simultaneously. That'd be awful on Gigabit.

 

[anemoiac]

@EngChiSTH @ArmedAviator

Thanks a bunch for the advice! I think I am going to go ahead and pull the trigger on an ICX switch. It seems like it will be a good learning experience, and it should be more than enough for my needs. The 7250 looks ideal, as the 8 SFP+ ports will allow me to expand beyond the 4 devices I currently plan to use with 10G connections. I like the idea of getting a second switch to practice with, but I think that will have to be on the back-burner until I finish my current setup.

I'm glad to hear that 10G made a noticeable difference on your home network. I forgot to mention that my VM server will be using all flash storage, and I intend to set up SSD caching on my NAS server, so I suspect that I'll be able to take advantage of some of that speed as well.

Thanks again!

 

[RoachedCoach]

How do you mount the sunon fan on top of the ASIC heat sink? A picture would be nice.

Just find two screws long enough to pass through the fan and grip to the heat sink. It sounds clunky AF, but it actually works and is very clean.

No pics, I'm not pulling that sucker out of the rack, sorry.

 

[RedX1]

this is the closest you'll get but it hasn't been updated in a while Ruckus ICX Target Path Selection Guide | Technical Documents | Ruckus Wireless Support

8090 is the "recommended" train, 8092 is slightly newer but still stable in my experience, 8095 is brand new and still has issues

Hi



Just some recent experience with ICX7250) and 08.0.95ba Firmware.



Following the interest here I have now several Brocade ICX switches, the latest a ICX7250P came with Ruckus ICX FastIron 08.0.95ba (GA) Firmware.

I obtained this switch at very low cost, as the POE board reports a failure to initialise. The switch works normally.

Some reading of the Firmware upgrade guide - RUCKUS FastIron Software Upgrade Guide, 08.0.95 shows that the UFI firmware train is considerably different to pre 08.0.87 versions.

https://support.alcadis.nl/Support_...s FastIron 08.0.95 Software Upgrade Guide.pdf

The POE board shows no obvious signs of physical damage.

It may be that a reloading of the POE firmware will resolve the issue, but I am reluctant to attempt that without more knowledge.

Downgrading to a previous firmware release appears somewhat more complex and I will use the switch without the POE feature until I have more information.

Thanks again to @fohdeesha and all the other contributors to this great body of work, any guidance would be greatly appreciated.



Thanks again.





RedX1

 

[tommybackeast]

Best Practices Upgrade instructions : 7250-48p and 7150-c12p are currently both on 8080e.

Recently I believe fohdeesha made the comment that 8090 is the recommended version; but also that 8092 is stable.

1) as a noob, I lack the technical skill to decide if 8090 or 8092 should be used, can anyone comment / suggest ?

2) as I am coming from 8080e, my readings show there are a few different methods to upgrade; can anyone suggest which upgrade method is "best" . I am fine with CLI if shown the steps.

Thank you all very much

 

[tubs-ffm]

Almost the same result: 7250-24p, same Mechatronics fans in the back (2), same Sunon 6x6x1 fan on top of the ASIC. I wouldn't call it silent, but a very large improvement. Oh, I also cut out the fan grill (or what ever you call the holes over the fan). ASIC 57C, PSU 36C (without much room ventilation).

Will the 7250-24p also run with one of his two fans disconnected? Somewhere I was reading about it for advice of the 6xxxx series. But I cannot find it any more.

I just got mine from eBay. Reset and upgrading sw from 08030 to 08080e and finally to 08092e acc. the manual linked in this thread worked perfectly. Now I can start playing. But the fan noise is much worse than I expected. And as I plan only to use 3 PoE devices and 1/3 of the installed ports I guess that installed fan capacity is much over my needs.

Edit:
OK. I could not wait any longer and tried out. It runs with one fan only. But not really an improvement.
I now will look where to order Sunon MagLev KDE1204PKV3-MS from a reliable source.

 

[tubs-ffm]

2) as I am coming from 8080e, my readings show there are a few different methods to upgrade; can anyone suggest which upgrade method is "best" . I am fine with CLI if shown the steps.

Today when my 7250-24P arrived, for me it was the first time to have a Brocade / Ruckus in my hands. I did upgrade to 08080e acc. instruction from fohdeesha and afterwards upgrade to 08092e acc. manual (fastIron-08092-upgradeguide.pdf). I used the USB method. Nothing complicate. Copy the files to the USB flash drive and foll the instruction.

Loading the Flash Code Using USB

1. Copy the application image and the boot image to the flash from the USB.
ICX7450-48P Router# copy disk0 flash SPR08080e.bin primary
ICX7450-48P Router# copy disk0 flash spz10114.bin bootrom

2.
a. Reload the device and immediately perform a UFI upgrade.
ICX7450-48P Router# boot system flash primary
ICX7450-48P Router# copy disk0 flash SPR08092eufi.bin primary

b. Re-copy the 08.0.92 UFI to the primary and the secondary flash partition to avoid boot image mismatch.
ICX7450-48P Router# copy disk0 flash SPR08092eufi.bin secondary

c. Enter the show version command to verify that the UFI image has loaded successfully.

 

[adman_c]

Will the 7250-24p also run with one of his two fans disconnected? Somewhere I was reading about it for advice of the 6xxxx series. But I cannot find it any more.

I just got mine from eBay. Reset and upgrading sw from 08030 to 08080e and finally to 08092e acc. the manual linked in this thread worked perfectly. Now I can start playing. But the fan noise is much worse than I expected. And as I plan only to use 3 PoE devices and 1/3 of the installed ports I guess that installed fan capacity is much over my needs.

Edit:
OK. I could not wait any longer and tried out. It runs with one fan only. But not really an improvement.
I now will look where to order Sunon MagLev KDE1204PKV3-MS from a reliable source.

I would hesitate to use those MagLev fans. While they are quieter at full blast than the stock fans, I found that they had a deeply unpleasant resonance/buzzing when running at the lower fan speed voltage in my 6450. Check out @rootwyrm's fan post of doom for some other recommendations. I'm currently running a pair of Delta FFB0412HN-5T20 that are notably quieter than my stock fans were. I also bought a couple of the Gryphon GDA4028-12BB to try out, but the Deltas have been inoffensive enough that I haven't been inclined to take down my whole network to see if the Gryphons are better.

 

[tommybackeast]

Today when my 7250-24P arrived, for me it was the first time to have a Brocade / Ruckus in my hands. I did upgrade to 08080e acc. instruction from fohdeesha and afterwards upgrade to 08092e acc. manual (fastIron-08092-upgradeguide.pdf). I used the USB method. Nothing complicate. Copy the files to the USB flash drive and foll the instruction.

Loading the Flash Code Using USB

1. Copy the application image and the boot image to the flash from the USB.
ICX7450-48P Router# copy disk0 flash SPR08080e.bin primary
ICX7450-48P Router# copy disk0 flash spz10114.bin bootrom

2.
a. Reload the device and immediately perform a UFI upgrade.
ICX7450-48P Router# boot system flash primary
ICX7450-48P Router# copy disk0 flash SPR08092eufi.bin primary

b. Re-copy the 08.0.92 UFI to the primary and the secondary flash partition to avoid boot image mismatch.
ICX7450-48P Router# copy disk0 flash SPR08092eufi.bin secondary

c. Enter the show version command to verify that the UFI image has loaded successfully.

Thank you for the note

Shall do more reading tonight when have time; but remain confused which train to use, 8090 or 8092

 

[Vesalius]

Thank you for the note

Shall do more reading tonight when have time; but remain confused which train to use, 8090 or 8092

Either will work fine for you. 8092 has been out for a long time and has no show stoppers, as per fodeesha and others. Ruckus blanket enterprise recommended version will always lag behind what is stable, just the nature of the beast with corporate recommendations. I would go with 8092 and not have to worry about another upgrade later, but that is just my opinion.

Ruckus software engineers have already moved on to 8095 and that is where some refinement still needs to happen.

 

[infoMatt]

I would hesitate to use those MagLev fans. While they are quieter at full blast than the stock fans, I found that they had a deeply unpleasant resonance/buzzing when running at the lower fan speed voltage in my 6450. Check out @rootwyrm's fan post of doom for some other recommendations. I'm currently running a pair of Delta FFB0412HN-5T20 that are notably quieter than my stock fans were. I also bought a couple of the Gryphon GDA4028-12BB to try out, but the Deltas have been inoffensive enough that I haven't been inclined to take down my whole network to see if the Gryphons are better.

I can confirm in a sense... I've just bought two KDE1204PKVX, and they have a low-ish frequency hum... Way less obtrusive than the stock Foxconn my unit had. But I've put a 10µF cap between positive and negative pins on the sockets, it helps a lot to "smooth out" the hard PWM of the unit.

 

[tubs-ffm]

Either will work fine for you. 8092 has been out for a long time and has no show stoppers, as per fodeesha and others. Ruckus blanket enterprise recommended version will always lag behind what is stable, just the nature of the beast with corporate recommendations. I would go with 8092 and not have to worry about another upgrade later, but that is just my opinion.

This exactly what the base for my decision. My thoughts I included in my decision. Maybe it can help

• "If I do not know the difference, it might be not relevant for me."
• It is not a decision for live. There is the possibility to downgrade again.
• I am home user. The recommendation for enterprise is for a rock-solid system and to minimize support effort.
• There already is a newer version 08095 out and 08092 is out since a while. My smartphone might have more bugs.

 

[noduck]

Will the 7250-24p also run with one of his two fans disconnected? Somewhere I was reading about it for advice of the 6xxxx series. But I cannot find it any more.

My 7250-24p shut down when I disconnect one of the fans (this was with 08.0.80e).

 

[alex.forencich]

Another data point: I just got an ICX7150-24-4X1G as I needed a couple more 10G interfaces. I was fearing an annoyingly loud switch that I would have to go retrofit with quieter fans, but it turns out (to my surprise and delight) that the ICX7150-24-4X1G is actually fanless. So, if you find one at a low cost, that's definitely a good option if you're concerned about acoustics.

 

[jjacobs]

I've seen one or two of these (ICX7150) pop up at a very good price. The fanless models and the ones with a quiet mode are appealing. What should I be aware of? Fakes? Licensing issues? Are the licenses already applied acording to the model number, for example a ICX7150-48P-4X10GR-RMT3 being sold as new for $750 or will I have potential issues licensing that?

 

[infoMatt]

I've seen one or two of these (ICX7150) pop up at a very good price. The fanless models and the ones with a quiet mode are appealing. What should I be aware of? Fakes? Licensing issues? Are the licenses already applied acording to the model number, for example a ICX7150-48P-4X10GR-RMT3 being sold as new for $750 or will I have potential issues licensing that?

Licenses are "honor based" on the 7xxx series... basically you promise that you'll buy them and it let you use the extended functions.

 

[mwarps]

So I got my hands on an ICX7250 24-port unit. So far, no response from serial console, and the fans stay at max on power.

First power up, it took about five minutes to "boot" (the fans got quiet, and an ethernet link established), but nothing ever showed on serial.

Do I have a brick?

 

[LodeRunner]

The behavior sounds like a normal boot, I'd verify your serial settings and make sure you have the right COM port number by checking Device Manager. I have a USB serial device that can randomly change what COM number gets assigned when it's plugged in.

Settings for ICX are 9600/8/N/1 and no flow control. PuTTY defaults to XON/XOFF.

Oh and you connected that USB port to a serial port, not to a USB port on your system, right? The mini-USB on the 7250 is wired for serial; there's no USB device inside, so connecting to a USB port on your host will burn it out.