Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[noduck]

Wanted to post an update on my infinite quest to silence an ICX 7250-48P (I run this thing in a fairly warm closet with bad circulation). Call me obsessive, but I wasn't a fan (no pun intended) of my temps creeping into the low 70s after my previous fan swap. I believe I have come up with the perfect silent solution for this rig.

Replace all 3 rear fans with the Mechatronics MR4020X12B1-RSR fan. They will read just fine in CLI. Then add a Sunon MF60101V3-1000U-A99 fan to the ASIC heat sink (you'll still have around 1/3" to spare in the case, plenty). Wire it in with Fan 3. All the fans are reduced load vs stock so they won't draw too much and the Sunon only has power and ground, so doesn't affect the read on Fan 3.

The switch now runs next-to-dead silent; temperatures on the ASIC dropped ~20 degrees over my previous configuration into the low to mid 50s degrees and the PSU is in the upper 20s which is great. Been running around a week, everything has been perfect. I'm in love.

Almost the same result: 7250-24p, same Mechatronics fans in the back (2), same Sunon 6x6x1 fan on top of the ASIC. I wouldn't call it silent, but a very large improvement. Oh, I also cut out the fan grill (or what ever you call the holes over the fan). ASIC 57C, PSU 36C (without much room ventilation).

I tried Sunon in the back before (MB40201VX-000U), and maybe the same absolute noise level as the Mechatronics, but the Sunon had a more annoying sound.

I mounted the ASIC fan with two screws into the ASIC fins (I went through my collection of screws until I found something that worked).

 

[anemoiac]

Is this (Brocade ICX series) still a logical route to cheap 10Gb home networking in 2020? I know the cheaper 10Gb MikroTik switches are pretty popular these days, but I think I'd like to have L3 switching capabilities. Thanks in advance to anyone who can advise!

*Edit: I meant 2021...

 

[EngChiSTH]

Is this (Brocade ICX series) still a logical route to cheap 10Gb home networking in 2020? I know the cheaper 10Gb MikroTik switches are pretty popular these days, but I think I'd like to have L3 switching capabilities. Thanks in advance to anyone who can advise!

This was answered by @fohdeesha earlier - you are comparing different functionality sets of enterprise hardware vs more consumer oriented set. if you want basics of 10 Gb transfers, either should work for you , I think cheapest Miktorik are in <$150 range for basic switches with SFP+. Brocade basic is slightly cheaper however it used (e-bay) and more powerful in terms of learning. I dont know if Mikrotik has a line speed switching on their basic sets (i think i read somewhere it is done in CPU and it much slower for L3).

I use and like Mikrotik but not for their 10G. I use and like Brocade ICX (found them due to this switch).

 

[anemoiac]

Thanks for the input! I've looked through a few dozen pages of this thread, but wasn't able to find a previous instance of my question. I guess that's one downside to ~ 250 page threads.

Anyway, from reading about both Brocade and MikroTik 10G switches, I've gotten a similar impression to you. It seems like the cheap MikroTiks get bogged down when doing L3 switching. While I'm sure the MikroTik's performance would be adequate for my actual use-case (a relatively basic home network), I'm interested in learning a bit about networking and enterprise networking hardware.

Perhaps a more precise phrasing of my question would be: Does the Brocade ICX series still represent the best cheap used enterprise 10G switch in 2021, or have other switches entered the discussion over the past few years?

 

[EngChiSTH]

Thanks for the input! I've looked through a few dozen pages of this thread, but wasn't able to find a previous instance of my question. I guess that's one downside to ~ 250 page threads.

Anyway, from reading about both Brocade and MikroTik 10G switches, I've gotten a similar impression to you. It seems like the cheap MikroTiks get bogged down when doing L3 switching. While I'm sure the MikroTik's performance would be adequate for my actual use-case (a relatively basic home network), I'm interested in learning a bit about networking and enterprise networking hardware.

Perhaps a more precise phrasing of my question would be: Does the Brocade ICX series still represent the best cheap used enterprise 10G switch in 2021, or have other switches entered the discussion over the past few years?

10G in general? fiber options only? what existing infrastructure (if any) you have (i.e. you already have your residence wired for cat6)?

I would start with what you are solving for and what is your budget vs 'is this a best tool' approach - best tool for what?
i.e. I want to transfer data to my central storage (off the shelf NAS , my ZFS storage server, etc) as fast as possible and willing to spend X on it
or I want to shape and shift my network to test out functionality X, Y, Z (VLANs, etc) and my budget is X


start with goals and go from there.
if you have nothing in terms of existing infrastructure, dont care about copper or fiber, and want it as cheap as possible then yes Brocade may fit the bill depending on what you see on the E-bay (which ebbs and flows depending on demand). i.e. i got mine for <$90 and am happy with it.

if you have specific budget allocated , may be cheap is not your primary goal . or if you have specific speeds, then may be 10G is not the right tech to play with.

 

[gb00s]

...
I mounted the ASIC fan with two screws into the ASIC fins (I went through my collection of screws until I found something that worked).

Can you share a photo please. Thank you in advance.

 

[anemoiac]

@EngChiSTH

Thanks for the feedback! I hesitated when I asked about the Brocade being the "best cheap used enterprise 10G switch in 2021" because I know that such a thing doesn't actually exist. Obviously, my specific use-case will dictate what's "best" for me, and what's best for me might not be a good choice for someone else.

The main problem I'm facing is my lack of knowledge/experience, as I'm basically diving headfirst into this world starting from scratch. I have virtually zero networking knowledge. I'm in the process of setting up a homelab to learn more about a variety of IT-related topics and, hopefully, launch an IT career.

As of now, my network consists of a dumb switch plugged into an HP T730 Plus running pfSense. I've recently acquired a half rack and a couple of Dell PowerEdge servers, one of which will operate primarily as a NAS and the other a virtualization server. I have a Windows/Linux PC and a Mac mini that will regularly interact with these servers over the network. My rack sits right next to the desk that houses the Mac and PC, so I won't need to run wire more than a few feet.

I don't have a specific requirement for my network to be 10G, but it seems like it would improve my experience while also compelling me to learn a bit more about the associated tech. I don't know enough at this point to care about the difference between fiber and copper, but my budget is fairly limited - I'd like to keep my spending to a few hundred dollars to begin. I already have 10G SFP+ NICs in the Dell servers, but that budget will need to include a switch + cables + transceivers + a NIC for my Windows/Linux desktop (I'm not worried about the Thunderbolt 3 - SFP+ adapter that the Mac mini will require yet, as that will probably cost half as much as everything else combined). Based on that description, the MikroTik switches seem like a good fit, but I'd prefer to use an enterprise switch with full L3 capabilities.

From what I've read so far, I see no reason why a Brocade ICX switch won't suit my needs; I'm just trying to be (at least a little bit more) thorough before I pull the trigger. Thanks again!

 

[EngChiSTH]

@EngChiSTH

Thanks for the feedback! I hesitated when I asked about the Brocade being the "best cheap used enterprise 10G switch in 2021" because I know that such a thing doesn't actually exist. Obviously, my specific use-case will dictate what's "best" for me, and what's best for me might not be a good choice for someone else.

The main problem I'm facing is my lack of knowledge/experience, as I'm basically diving headfirst into this world starting from scratch. I have virtually zero networking knowledge. I'm in the process of setting up a homelab to learn more about a variety of IT-related topics and, hopefully, launch an IT career.

As of now, my network consists of a dumb switch plugged into an HP T730 Plus running pfSense. I've recently acquired a half rack and a couple of Dell PowerEdge servers, one of which will operate primarily as a NAS and the other a virtualization server. I have a Windows/Linux PC and a Mac mini that will regularly interact with these servers over the network. My rack sits right next to the desk that houses the Mac and PC, so I won't need to run wire more than a few feet.

I don't have a specific requirement for my network to be 10G, but it seems like it would improve my experience while also compelling me to learn a bit more about the associated tech. I don't know enough at this point to care about the difference between fiber and copper, but my budget is fairly limited - I'd like to keep my spending to a few hundred dollars to begin. I already have 10G SFP+ NICs in the Dell servers, but that budget will need to include a switch + cables + transceivers + a NIC for my Windows/Linux desktop (I'm not worried about the Thunderbolt 3 - SFP+ adapter that the Mac mini will require yet, as that will probably cost half as much as everything else combined). Based on that description, the MikroTik switches seem like a good fit, but I'd prefer to use an enterprise switch with full L3 capabilities.

From what I've read so far, I see no reason why a Brocade ICX switch won't suit my needs; I'm just trying to be (at least a little bit more) thorough before I pull the trigger. Thanks again!

Leaving the topic of starting IT career out of this discussion (for that homelab would help only in few very specific sub-specialties dealing with actual network/hardware), yes if what you want is to play with 10Gb speeds (and you already have the SFP+ cards), then either MikroTik or Brocade would work. If you want to play with VLANS and "L3 stuff" then Brocade is probably better if you go. I looked at e-bay and cheapest is $125 with shipping include in USA . this is more than many of us paid historically and could be just normal flow of prices on e-bay


if you have specific questions, feel free to ask

 

[ArmedAviator]

@anemoiac

You'll be well served by an ICX that suits your needs. Good prices on the used market, free firmware direct from the manufacturer (albeit it the 6xxx series are all EoL), and good performance.

If you can, go with the 7xxx series. If price is a major concern, the 6xxx series is just fine for your needs.

Since you're trying to learn networking, I recommend getting two or more managed switches to practice setting up and using various protocols between them (VLAN, LAG, RSTP, CDP, LLDP, stacking, etc.).

I have an ICX 6610 and an ICX 6450 and use them both on my home network. They've been great for learning and for home use. If I find another dirt cheap ICX6610-48P, I'm going to get a second and stack them i my rack for additional redundancy.

Oh and 10 gig has made a tremendous improvement in my use cases at home. Proxmox cluster migration between 3 R710s occurs significantly faster - really makes a difference when one of my VMs has a total of 6TB of virtual drives. Trust transferred it betwen hosts last night as I upgraded HDDs and re-created my RAID array on one host. Sustained speed was limited by only the HDDs/RAID. Also using a Gluster cluster is great for my NAS needs with 10Gig or better as a client writing to Gluster sends the data simultaneously to all Gluster server nodes - so when I see my PC writing files to the cluster at 400MB/sec, the network throughput is actually 800MB/sec since it's sending identical data to two servers simultaneously. That'd be awful on Gigabit.

 

[anemoiac]

@EngChiSTH @ArmedAviator

Thanks a bunch for the advice! I think I am going to go ahead and pull the trigger on an ICX switch. It seems like it will be a good learning experience, and it should be more than enough for my needs. The 7250 looks ideal, as the 8 SFP+ ports will allow me to expand beyond the 4 devices I currently plan to use with 10G connections. I like the idea of getting a second switch to practice with, but I think that will have to be on the back-burner until I finish my current setup.

I'm glad to hear that 10G made a noticeable difference on your home network. I forgot to mention that my VM server will be using all flash storage, and I intend to set up SSD caching on my NAS server, so I suspect that I'll be able to take advantage of some of that speed as well.

Thanks again!

 

[RoachedCoach]

How do you mount the sunon fan on top of the ASIC heat sink? A picture would be nice.

Just find two screws long enough to pass through the fan and grip to the heat sink. It sounds clunky AF, but it actually works and is very clean.

No pics, I'm not pulling that sucker out of the rack, sorry.

 

[RedX1]

this is the closest you'll get but it hasn't been updated in a while Ruckus ICX Target Path Selection Guide | Technical Documents | Ruckus Wireless Support

8090 is the "recommended" train, 8092 is slightly newer but still stable in my experience, 8095 is brand new and still has issues

Hi



Just some recent experience with ICX7250) and 08.0.95ba Firmware.



Following the interest here I have now several Brocade ICX switches, the latest a ICX7250P came with Ruckus ICX FastIron 08.0.95ba (GA) Firmware.

I obtained this switch at very low cost, as the POE board reports a failure to initialise. The switch works normally.

Some reading of the Firmware upgrade guide - RUCKUS FastIron Software Upgrade Guide, 08.0.95 shows that the UFI firmware train is considerably different to pre 08.0.87 versions.

https://support.alcadis.nl/Support_...s FastIron 08.0.95 Software Upgrade Guide.pdf

The POE board shows no obvious signs of physical damage.

It may be that a reloading of the POE firmware will resolve the issue, but I am reluctant to attempt that without more knowledge.

Downgrading to a previous firmware release appears somewhat more complex and I will use the switch without the POE feature until I have more information.

Thanks again to @fohdeesha and all the other contributors to this great body of work, any guidance would be greatly appreciated.



Thanks again.





RedX1

 

[tommybackeast]

Best Practices Upgrade instructions : 7250-48p and 7150-c12p are currently both on 8080e.

Recently I believe fohdeesha made the comment that 8090 is the recommended version; but also that 8092 is stable.

1) as a noob, I lack the technical skill to decide if 8090 or 8092 should be used, can anyone comment / suggest ?

2) as I am coming from 8080e, my readings show there are a few different methods to upgrade; can anyone suggest which upgrade method is "best" . I am fine with CLI if shown the steps.

Thank you all very much

 

[tubs-ffm]

Almost the same result: 7250-24p, same Mechatronics fans in the back (2), same Sunon 6x6x1 fan on top of the ASIC. I wouldn't call it silent, but a very large improvement. Oh, I also cut out the fan grill (or what ever you call the holes over the fan). ASIC 57C, PSU 36C (without much room ventilation).

Will the 7250-24p also run with one of his two fans disconnected? Somewhere I was reading about it for advice of the 6xxxx series. But I cannot find it any more.

I just got mine from eBay. Reset and upgrading sw from 08030 to 08080e and finally to 08092e acc. the manual linked in this thread worked perfectly. Now I can start playing. But the fan noise is much worse than I expected. And as I plan only to use 3 PoE devices and 1/3 of the installed ports I guess that installed fan capacity is much over my needs.

Edit:
OK. I could not wait any longer and tried out. It runs with one fan only. But not really an improvement.
I now will look where to order Sunon MagLev KDE1204PKV3-MS from a reliable source.

 

[tubs-ffm]

2) as I am coming from 8080e, my readings show there are a few different methods to upgrade; can anyone suggest which upgrade method is "best" . I am fine with CLI if shown the steps.

Today when my 7250-24P arrived, for me it was the first time to have a Brocade / Ruckus in my hands. I did upgrade to 08080e acc. instruction from fohdeesha and afterwards upgrade to 08092e acc. manual (fastIron-08092-upgradeguide.pdf). I used the USB method. Nothing complicate. Copy the files to the USB flash drive and foll the instruction.

Loading the Flash Code Using USB

1. Copy the application image and the boot image to the flash from the USB.
ICX7450-48P Router# copy disk0 flash SPR08080e.bin primary
ICX7450-48P Router# copy disk0 flash spz10114.bin bootrom

2.
a. Reload the device and immediately perform a UFI upgrade.
ICX7450-48P Router# boot system flash primary
ICX7450-48P Router# copy disk0 flash SPR08092eufi.bin primary

b. Re-copy the 08.0.92 UFI to the primary and the secondary flash partition to avoid boot image mismatch.
ICX7450-48P Router# copy disk0 flash SPR08092eufi.bin secondary

c. Enter the show version command to verify that the UFI image has loaded successfully.

 

[adman_c]

Will the 7250-24p also run with one of his two fans disconnected? Somewhere I was reading about it for advice of the 6xxxx series. But I cannot find it any more.

I just got mine from eBay. Reset and upgrading sw from 08030 to 08080e and finally to 08092e acc. the manual linked in this thread worked perfectly. Now I can start playing. But the fan noise is much worse than I expected. And as I plan only to use 3 PoE devices and 1/3 of the installed ports I guess that installed fan capacity is much over my needs.

Edit:
OK. I could not wait any longer and tried out. It runs with one fan only. But not really an improvement.
I now will look where to order Sunon MagLev KDE1204PKV3-MS from a reliable source.

I would hesitate to use those MagLev fans. While they are quieter at full blast than the stock fans, I found that they had a deeply unpleasant resonance/buzzing when running at the lower fan speed voltage in my 6450. Check out @rootwyrm's fan post of doom for some other recommendations. I'm currently running a pair of Delta FFB0412HN-5T20 that are notably quieter than my stock fans were. I also bought a couple of the Gryphon GDA4028-12BB to try out, but the Deltas have been inoffensive enough that I haven't been inclined to take down my whole network to see if the Gryphons are better.

 

[tommybackeast]

Today when my 7250-24P arrived, for me it was the first time to have a Brocade / Ruckus in my hands. I did upgrade to 08080e acc. instruction from fohdeesha and afterwards upgrade to 08092e acc. manual (fastIron-08092-upgradeguide.pdf). I used the USB method. Nothing complicate. Copy the files to the USB flash drive and foll the instruction.

Loading the Flash Code Using USB

1. Copy the application image and the boot image to the flash from the USB.
ICX7450-48P Router# copy disk0 flash SPR08080e.bin primary
ICX7450-48P Router# copy disk0 flash spz10114.bin bootrom

2.
a. Reload the device and immediately perform a UFI upgrade.
ICX7450-48P Router# boot system flash primary
ICX7450-48P Router# copy disk0 flash SPR08092eufi.bin primary

b. Re-copy the 08.0.92 UFI to the primary and the secondary flash partition to avoid boot image mismatch.
ICX7450-48P Router# copy disk0 flash SPR08092eufi.bin secondary

c. Enter the show version command to verify that the UFI image has loaded successfully.

Thank you for the note

Shall do more reading tonight when have time; but remain confused which train to use, 8090 or 8092

 

[Vesalius]

Thank you for the note

Shall do more reading tonight when have time; but remain confused which train to use, 8090 or 8092

Either will work fine for you. 8092 has been out for a long time and has no show stoppers, as per fodeesha and others. Ruckus blanket enterprise recommended version will always lag behind what is stable, just the nature of the beast with corporate recommendations. I would go with 8092 and not have to worry about another upgrade later, but that is just my opinion.

Ruckus software engineers have already moved on to 8095 and that is where some refinement still needs to happen.

 

[infoMatt]

I would hesitate to use those MagLev fans. While they are quieter at full blast than the stock fans, I found that they had a deeply unpleasant resonance/buzzing when running at the lower fan speed voltage in my 6450. Check out @rootwyrm's fan post of doom for some other recommendations. I'm currently running a pair of Delta FFB0412HN-5T20 that are notably quieter than my stock fans were. I also bought a couple of the Gryphon GDA4028-12BB to try out, but the Deltas have been inoffensive enough that I haven't been inclined to take down my whole network to see if the Gryphons are better.

I can confirm in a sense... I've just bought two KDE1204PKVX, and they have a low-ish frequency hum... Way less obtrusive than the stock Foxconn my unit had. But I've put a 10µF cap between positive and negative pins on the sockets, it helps a lot to "smooth out" the hard PWM of the unit.

 

[tubs-ffm]

Either will work fine for you. 8092 has been out for a long time and has no show stoppers, as per fodeesha and others. Ruckus blanket enterprise recommended version will always lag behind what is stable, just the nature of the beast with corporate recommendations. I would go with 8092 and not have to worry about another upgrade later, but that is just my opinion.

This exactly what the base for my decision. My thoughts I included in my decision. Maybe it can help

• "If I do not know the difference, it might be not relevant for me."
• It is not a decision for live. There is the possibility to downgrade again.
• I am home user. The recommendation for enterprise is for a rock-solid system and to minimize support effort.
• There already is a newer version 08095 out and 08092 is out since a while. My smartphone might have more bugs.