Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

kevindd992002

Member
Oct 4, 2021
125
6
18
Ok, I think I have a good idea of what you guys are saying. I have just a couple of devices in my home rack. A Synology NAS, NAS extension bay, and pfsense. I think I'm set with the ICX6610-48p-E (exhaust airflow) then so all my devices have front-to-back airflow.
 

juju

Member
Sep 29, 2021
38
1
8
I have an IoT device on my network which is not using the dns server on the ICX7250 and always defaults to 0.0.0.0. ( so cant resolve dns names). All other devices on my network correctly use the dns server ip I have setup on my 7250. Other devices on the same vlan as this IoT device have no such issue with dns resolution. Is there a way to force a specific device to use the dns server of its vlan ?
 

Rttg

Member
May 21, 2020
74
49
18
I have an IoT device on my network which is not using the dns server on the ICX7250 and always defaults to 0.0.0.0. ( so cant resolve dns names).
Are you sure the IoT device isn’t trying to use a hardcoded DNS server?

Some will to ensure they can ‘phone home’ (and avoid ad blockers). If that’s the case, you may need to use a DNAT rule to rewrite DNS traffic to your preferred server(s).
 

juju

Member
Sep 29, 2021
38
1
8
Are you sure the IoT device isn’t trying to use a hardcoded DNS server?
It is entirely possible it is - I suspect that is the issue. In this case, it has a default dns server of 0.0.0.0 ( if you query the device for dns server info). Not sure if its returning 0.0.0.0 because its not getting the dns info from the ICX switch or its hardcoded. Can you give some pointers how to do the DNAT rule ? Where ? On my pfsense switch or on the ICX7250?
 

Rttg

Member
May 21, 2020
74
49
18
Can you give some pointers how to do the DNAT rule ? Where ? On my pfsense switch or on the ICX7250?
The ICX can’t do NAT - it’d have to run on your firewall/internet gateway.

Can’t say how to do it on pfsense - I’m running VyOS so don’t know the syntax/config there.
 

seatrope

Member
Oct 5, 2018
35
12
8
Maine
www.ychng.com
Hi all, thanks for the accumulated wealth of information. Have had a ICX6610 and a couple of ICX6450s for a while now but just started to use the L3 features.

I've gotten VLANs and inter-VLAN routing on the ICX6610 up and running, with pfSense only serving firewall duties. DHCP/DNS is via a piHole with dnsmasq serving all the subnets.

I have been searching and can't figure out how to do this. @fohdeesha any help would be appreciated - thanks again for your help with the switches a few years back!!
I either need to:
1) Stack the ICX6610 and one ICX6450 (i know only the front ports can be used as a "peri-trunk") but still preserve all the back 40Gbe ports for data use (I have a couple of servers connected via 40Gbe 1/2/1 and 1/2/6). It won't let me enable stacking without pulling the back ports into it too. I saw where you have used one 40Gbe and one breakout port for stacking btw two ICX6610 but this is somewhat different, i guess.
OR
2) if the above is impossible, what's the best way to pass all the VLANs defined on the 6610 to the 6450 and distribute them as untagged ports? I tried making the link between the 6610-6450 a dual-mode port on both sides with the same VLAN numbers - that did not work and I could not ping the 6610 interface IP after I did that.

Apologize in advance for the n00b questions but am slowly learning L3 networking.

Thanks so much!
 

seatrope

Member
Oct 5, 2018
35
12
8
Maine
www.ychng.com
Well, I figured out 2) myself. Stupid mistake of course - should just have tagged all the vlans and not used dual-mode for the "uplink".
Also, forgot to assign IPs on the vifs.

But still have no idea how to do 1). Would be nice to connect 2 10G ports in a "ring" topology stack btw 6610 - 6450.

Thanks!
 

sfrode

New Member
I blew the power supply on my ICX7250-48 a while back. Don't know why it happened, but It might have been temp related since the switch was pretty toasty when it happened.

In case anyone else ends up in the same situation; FSP200-P35-A12 can be used as a replacement with a couple of mods:
  • need to swap 12V/GND on the internal power plug between PSU and EPS board
  • need to extend the internal power cable between the PSU and EPS board or just move the EPS board
 
Last edited:

kfriis

Member
Apr 8, 2015
54
7
8
49
I have noticed that "sometimes" after a complete power outage, the ICX-6450 will not restart/reboot. The only way to turn the switch ON again is to unplug the power cord and plug it right back in. After this, the switch boots and functions normally. But it will not "take power" or reboot without pulling the power cord.

Has anybody experienced this? Is there some kind of configuration I should be checking to ensure that the switch will reboot/restart after a power outage?

Thank you!
 

heromode

Well-Known Member
May 25, 2020
436
260
63
I have noticed that "sometimes" after a complete power outage, the ICX-6450 will not restart/reboot. The only way to turn the switch ON again is to unplug the power cord and plug it right back in. After this, the switch boots and functions normally. But it will not "take power" or reboot without pulling the power cord.

Has anybody experienced this? Is there some kind of configuration I should be checking to ensure that the switch will reboot/restart after a power outage?

Thank you!
only thing i can think of is it's trying to power on at the exact same time as all your other gear. I remember @fohdeesha mentioning that these things are designed for 'pulling the plug' power recycling, and no need to be afraid of that. Maybe the PSU just doesn't get enough juice the exact second all your other gear powers on?
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,877
3,377
113
34
fohdeesha.com
I have noticed that "sometimes" after a complete power outage, the ICX-6450 will not restart/reboot. The only way to turn the switch ON again is to unplug the power cord and plug it right back in. After this, the switch boots and functions normally. But it will not "take power" or reboot without pulling the power cord.

Has anybody experienced this? Is there some kind of configuration I should be checking to ensure that the switch will reboot/restart after a power outage?

Thank you!
there's no "power-on" for these things, if power is applied to the board, they boot. Are these really brief power flickers, like a second or so? All I can think of is it's losing power just shortly enough that the voltage supplied to the CPU board doesn't drop enough to completely (the PSU is going to have ~400ms of hold-up time) to reset the CPU and it gets stuck in a half-reset state. have ran into this on a lot of electronics with very brief power flickers that don't quite drop voltage completely to micros/CPUs

It's also possible when your power is returning it's high voltage (or sagging) to the point the PSU is going into protection, and the PSU is not going to output power until it's reset (power removed and applied again)
 

jbrukardt

Member
Feb 4, 2016
91
41
18
90
So in order...
- the best fans for this are unobtanium due to COVID-19, but don't use Noctua. Noctua is overpriced junk - it has the lowest flow rate and lowest static pressure. Anything from Sunon or NMB or Nidec or Delta is fine. They also cost far less.
- 08.0.30t for the 6450 is the maximum possible
- see first post for that, but if the switch was only configuration reset and not factory reset it may have them installed. Do a show licenses first always.

Edit: to expound upon my 'unobtanium' fan choice, bear in mind, I have been doing systems integration for 30 years and have VERY extensive rackmount equipment experience. I know why Brocade chose the fans they did, I have worked with those fans, and they are the same fans I would have chosen. (BTW, at speed 2 the 6450-40P is attempting to move 90.9CFM. Yes, each of those 40mm fans is rated for 30.3CFM free air.)
If you want to quiet down these units, the BIGGEST change you can make is to remove the stamped grill with a Dremel. It's all sharp edges, and significantly blocks the hub as well. Turbulence from things like grills are what generate serious noise. If you still need protection, tack on a wire grill on the OUTSIDE of the chassis. HOWEVER, this will result in slightly reduced cooling to the front right corner. I won't bother going into my unobtanium choice, because unless you're ready to order at least 250 of them, you can't buy them.

So here's some fans you can buy that I'd recommend.
  • Delta EFB0412VHD-F00 - $12.40/ea @ Digikey
    There's a REASON Delta's a top pick for ODM and OEM. 40x40x20mm, 10.1CFM, 0.416in H2O (which is insane,) 32.5dBA @ 1m, rated for 70k hours at 50C. Make sure it's THAT part number and not the Rev C, which is a vastly inferior part.
  • Comair Rotron "Gryphon" GDA4028-12BB - discontinued, alas
    If you can find these? BUY THESE. 40x40x28mm, but 11CFM, 0.34in H2O, 8800RPM, but just 31.4dBA! I am still mad they discontinued them. No, I am not selling any of my spares.
  • Mechatronics MR4020E12B1-RSR - $7.50 @ Digikey (when in stock)
    READ THE PAGE. These are non-stocked currently! However these offer excellent balance between flow and noise. 40x40x20mm, 15.8CFM, 0.45in H2O, and 39dBA @ 1m at 11000RPM. (Yes, it's about the quietest 11k fan ever made.) You need the E12B1 though - NOT the B2. The B1 has a tach, the B2 is a rotor lock wire.
  • Mechatronics MR4020H12B1-RSR - about $8
    The 'slightly slower' version pushing 13.6CFM at just 35.1dBA. Note that you cannot use the B1+6 that DigiKey stocks. The B1+6 is a 4-wire PWM, and the ICX's 12V control will burn up the motor.
  • Mechatronics MH4028L12B1-RSR - PLEASE TELL US ALL IF YOU FIND A SOURCE!
    These. Are. GLORIOUS. 40x40x28mm, 12.81CFM, 0.37in H2O, 39dBA, I just LOVE these fans. But nobody stocks the low speed version. And the next step up (the M) is already over 45dBA. But they have amazing harmonics due to the 4028L being less than half the speed the frame was designed for (Max 16,500RPM!)
  • NMB-MAT 1608VL-04W-B69-B00 - $10.47 @ Digikey
    These are an excellent 'middle of the middle' choice. 40x40x20mm, 11.3CFM, 0.399in H2O, 34.6dBA @ 1m at 9500RPM, and 40k hour lifetime. They're also generally stocked by multiple vendors. Really great fans for general use, but believe me, that 40k hour lifetime is near spot on - expect to replace every 3 years or so.
  • Sunon PSD1204PHB1-A(2).Z.F.PWM.GN "Tiny Terror" - also currently unobtainable
    These are new design MagLev parts, and really impressive. 40x40x15mm (so the thinnest here,) but 14CFM, 0.63in H2O, and 44.2dBA @ 1m at 12,000RPM. They're also a LOT easier than the Mechatronics; Sunon MOQ is just 30 for a non-custom part, expect around $10-12/ea. These are awesome fans if you can get your hands on them and can stand the noise.
And while we're here, let's talk about the Sunon KDE MagLevs. Which will be a very short talk: KDE MagLevs are terrible for this application. They're cheap, plentiful, and useless. They are NOT designed for applications like this. The KDE1204PKVX that you can find anywhere and everywhere? It is objectively the worst. 10.8CFM, 27.5dBA, sounds great, right? It also has a static pressure of 0.27in H2O. These were designed for very free flow applications. The ICX chassis grills are the opposite.
I also don't recommend SanAce because the PSUs you all complain about on the 6610's? Those are SanAce fans. SanAces have a very distinct harmonic that everyone hates. That harmonic is also why SanAces perform the way they do. You can't have one without the other. If you think those 6610's are bad, at least there is much, much worse.

As far as selecting fans? The closer to stock flow, the better, however you achieve it. Installing 10CFM peak fans in an ICX6450-48 or 48P is a net cooling reduction of over 70%! So, you know, don't do that unless you want to run very hot or are not using POE! This is why I make angry noises at people who think they can just swap fans around in equipment like this. For the love of god people, the ICX6610 fan modules? 2x Delta FFB0412UHN's per module, that's over 50CFM per module, over 100CFM excluding the power supplies! Nobody puts that kind of brute force in unless they NEED it. Especially as those FFBs are close to $20/each at quantity.
But as I said: how you get there doesn't matter as long as you get there or accept the trade-off. Ducting a pair of 45CFM 120mm fans on the side of your 6450-48P? That'd work too. No, seriously! (Won't work on the exhaust side because you just don't have enough opening and it's too much pressure loss.) Add more, quieter fans? Also valid. Strap tiny quiet fans to the heatsinks? Absolutely will help. Only care about performance at all costs? Throw in some contra-rotating fans! Want to get stock cooling performance out of a 6450-48? Cut out 2 more fan holes, drop in 3 x 9CFM, dead silent and ready to rock.
FYSA, the Comair's are available at

12V .20A 2.4W Gryphon Series DC Fan w/Tachometer Comair Rotron GDA4028-12BB

my order just shipped.
 
  • Like
Reactions: Lone Wolf

jbrukardt

Member
Feb 4, 2016
91
41
18
90
Is there something weird about the POE on Brocade switches? Grabbed a 6450-48P and it wont seem to deliver power to any of my POE splitters that I used to power things throughout the house. The straight POE items work fine (unifi APs, Amcrest Cameras), but the switch wont seem to negotiate with the splitter.

This is the splitter in question: https://www.amazon.com/gp/product/B07CNKX14C/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

Used it for years with my Avaya 4548gt-PWR to power low-power USB devices (way way sub 2.4A stuff). Never had an issue

The brocade wont recognize it at all, wont even negotiate with it. Im wondering if its because i dont use the data link portion?

heres my "show inline power"


Should be showing up on port 41 (although ive tried it on many different ports, same result), but just no registration at all.

Also... you'll note i turned POE off on port 24.... because it was detected the standard ethernet port of a motherboard as a 802.11af device and sending 400mw to it... not cool.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,877
3,377
113
34
fohdeesha.com
Is there something weird about the POE on Brocade switches? Grabbed a 6450-48P and it wont seem to deliver power to any of my POE splitters that I used to power things throughout the house. The straight POE items work fine (unifi APs, Amcrest Cameras), but the switch wont seem to negotiate with the splitter.

This is the splitter in question: https://www.amazon.com/gp/product/B07CNKX14C/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

Used it for years with my Avaya 4548gt-PWR to power low-power USB devices (way way sub 2.4A stuff). Never had an issue

The brocade wont recognize it at all, wont even negotiate with it. Im wondering if its because i dont use the data link portion?

heres my "show inline power"


Should be showing up on port 41 (although ive tried it on many different ports, same result), but just no registration at all.

Also... you'll note i turned POE off on port 24.... because it was detected the standard ethernet port of a motherboard as a 802.11af device and sending 400mw to it... not cool.
if it negotiated 11af ethernet on a non-PoE device, among other things, the poe board might be bad
 

daleq

New Member
Mar 11, 2015
16
5
3
60
Many thanks @fohdeesha for all that you've done to help others configure/maintain these switches. This is an amazing gift!

Like @sdyoung, I have a switch (ICX 6450-48P) with an old bootloader. I followed Fohdeesha's advice to work-around this initial hiccup.

Then, using pointers from @sdyoung and @Lone Wolf, I was able to get access to the web GUI using the following;
Code:
>enable
#configure terminal
(config)#ip address 10.1.1.3/24
(config)#username root password MyRootPassword
(config)#username dale privilege 0 password MyPassword
(config)#aaa authentication login default enable local
(config)#aaa authentication web-server default local
(config)#write memory
(I created the non-root user because I could not get 'root' to login on the Web GUI)
Then, rebooting the switch by unplugging and plugging back in.

My understanding from Lone Wolf's post is that I need to update the primary image via the Web GUI.
I successfully
- logged into the Web GUI as the 'dale' user
- scrolled down the left-menu to TFTP configuration
- set TFTP Server: 10.1.1.2 (my computer with TFTPd64 running)
- set Configuration File Name: ICX64xx/ICX64R08030u.bin (from Fohdeesha's instructions)

I have also temporarily disabled my Win10 firewall.

When I click 'Copy from Server to Flash' button in Web GUI, I'm seeing
- the progress bar start on the web gui
- tftpd64 shows bits being transferred for about 5 seconds (500,000-ish bytes)

Then I get an error on the page
Status : TFTP out of buffer space error.

This error shows in the console too
Code:
ICX6450-48P Switch#!!! Flash Task Got unblocked without cause, fd 13, ffp
stack: 015aacc0 015a626c 01777e34

TFTP out of buffer space error.
I don't know if the problem is with TFTPD64 on my windows box or with the config on the switch. Or, maybe I need to figure out why I can't login as root on Web Gui. Any suggestions to troubleshoot and resolve?

Many many thanks to Fohdeesha and all who have contributed on this thread to get me this far. I greatly appreciate it.