Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

neb50

Member
Aug 28, 2018
73
23
18
Configuration changes won't stick via GUI...

I set up an ICX 6430 using the most recent files found here: Brocade Overview - Fohdeesha Docs

I was using the web GUI to configure inline power settings. After I made changes (and Saved), the desired settings took effect. But the next time I booted up the switch, the inline power settings went back to their previous state.

I went into the CLI and made the desired inline power changes and after the next rebooting those changes held.

Is there something I'm missing about saving changes from the GUI? Or some way to resolve this issue? My skill level says the web GUI is more my speed than the CLI.
I think you can save it by going to the very bottom on the left pane and select Command->Save to Flash
 

deff_janiels

New Member
Mar 28, 2021
2
4
3
One question about fan modding on the ICX6450:

Ive read something about newer firmware which doesnt support alternative fans anymore here:


So, is this true or is there a workaround for it? There are those Sunon silent fans ive seen in another video about this switch.

Maybe someone can tell me something :)

Edit: NVM this is for the 7250, but maybe the same problem exists on the 6450?
I recently picked up a couple 7250s and transplanted my aftermarket fans (I haven't been able to find a record of which specific model, but I believe it's one of the Sunon maglev models discussed early in the thread) from my 6450s. My 6450s and 7250s are running the latest firmware on @fohdeesha 's site and don't throw any errors.

To my ear, the 6450s were still borderline too loud with the new fans. When I moved the fans into my 7250s, the temps are quite high and send the fans into the "speed 2" regime every 5 minutes or so. This is with ~20W of POE and 6 10G devices in each switch (half DAC, half RJ45 transceivers).

While I still have both my 6450s and 7250s I may make a separate thread about fans with noise samples since it seems like a frequent topic.
 
  • Like
Reactions: cap

jasonwc

Member
Dec 31, 2018
49
18
8
Regarding fan noise, I noticed that my ICX6610-48p was running at 55dB with either a single or dual PSU (2 x Rev C) and dual fans. I couldn't figure out why the fan noise didn't drop with two power supplies, as that was widely reported by others. After restarting the switch for an unrelated reason, the two fans significantly reduced RPM, and my sound meter now measures 50dB or slightly below. The only difference was that I had both power supplies connected to power when I reloaded the switch.
 

Propaganda

Active Member
Dec 6, 2017
154
62
28
43
I am stacking 2x 7250s, do I need to dedicate 4x 10G ports (2x per switch) to the stack or can I just use 2x (1x per switch)?
 
Last edited:

Propaganda

Active Member
Dec 6, 2017
154
62
28
43
You can use either configuration. 2 ports is a 'linear stack', 4 ports is a 'ring stack'.
I ran "stack interactive-setup" and it seems like it automatically made a ring stack. Do I need to reset things or is there a way to transform it into a linear stack?
 

kpfleming

Active Member
Dec 28, 2021
383
205
43
Pelham NY USA
Do you have two cables connecting the switches (using four ports)? If so, that's why it built a ring stack, because that's the preferred configuration (since it provides redundancy).
 

Scarlet

Member
Jul 29, 2019
86
38
18
You should be able to change the stacking configuration. This is how it looks on my 6450:
Code:
stack unit 1

  module 1 icx6450-24p-poe-port-management-module

  module 2 icx6450-sfp-plus-4port-40g-module

  priority 255

  stack-port 1/2/1 1/2/3

stack unit 2

  module 1 icx6450-24p-poe-port-management-module

  module 2 icx6450-sfp-plus-4port-40g-module

  priority 240

  stack-port 2/2/1

stack enable

On unit 2 I set the stack ports using
Code:
SSH@ICX6450-24P Router switch-01(config)#stack unit 1
SSH@ICX6450-24P Router switch-01(config-unit-1)#stack-port 2/2/1
I would expect that something similar is available on the 7250 no matter how the initial configuration was done.
 

gertsch

New Member
Dec 15, 2021
1
0
1
yours came with the layer2 only firmware - scroll down to the note where it starts with "Note: If during the router-interface ve 1 command earlier " - read that blurb on how to get over to the l3 image
So I'm very new to networking and build up my homelab as I go at the moment, so happy to learn :)
I just got the 6430-C12 for very cheap and tried to follow the guide. I guess this model does not run the Layer3 fw right?
Is this also the reason, that the " ip route 0.0.0.0/0 192.168.0.1" command is throwing an invalid input error?
 

Propaganda

Active Member
Dec 6, 2017
154
62
28
43
Do you have two cables connecting the switches (using four ports)? If so, that's why it built a ring stack, because that's the preferred configuration (since it provides redundancy).
You should be able to change the stacking configuration. This is how it looks on my 6450:
Code:
stack unit 1

  module 1 icx6450-24p-poe-port-management-module

  module 2 icx6450-sfp-plus-4port-40g-module

  priority 255

  stack-port 1/2/1 1/2/3

stack unit 2

  module 1 icx6450-24p-poe-port-management-module

  module 2 icx6450-sfp-plus-4port-40g-module

  priority 240

  stack-port 2/2/1

stack enable

On unit 2 I set the stack ports using
Code:
SSH@ICX6450-24P Router switch-01(config)#stack unit 1
SSH@ICX6450-24P Router switch-01(config-unit-1)#stack-port 2/2/1
I would expect that something similar is available on the 7250 no matter how the initial configuration was done.
I am pretty sure I originally setup the stack using only a single cable (1/2/1 <->2/2/1) but it seemed to automatically consume 4 ports. I don't really know if the redundancy is worth the loss of 2 ports for my setup at the moment. Nearly all the devices connected to the 10G ports are already doubled up failsafe between the switch's and I am not sure how often a 3M DAC cable breaks but I'll have spares around if it does. I think I figured out a way to remove the ports with the following command, not really setup to test it at the moment since I have things torn apart at the moment.

Code:
stack unit 1
no multi-stack-port 1/2/3 and 2/2/3
exit
 

jasonwc

Member
Dec 31, 2018
49
18
8
So I'm very new to networking and build up my homelab as I go at the moment, so happy to learn :)
I just got the 6430-C12 for very cheap and tried to follow the guide. I guess this model does not run the Layer3 fw right?
Is this also the reason, that the " ip route 0.0.0.0/0 192.168.0.1" command is throwing an invalid input error?
https://webresources.ruckuswireless.com/pdf/datasheets/ds-icx-6430-6450.pdf

See page 6. It appears that all of the 6430 series are Layer 2-only.
 

Wolfcastle

Member
Jan 3, 2022
50
23
8
I apologize if this has been answered already but this is a monster of a thread and I haven't read through it all.
1) Is there any reason the ICX 7450 isn't being considered in the top post? Are licenses also available for this model? Seems like a good option compared to the ICX 6610 for lower power/quieter operation unless I am missing something?
2) The top post lists the ICX 6610-24P port as ~80 watts, I am guessing that is no POE, no traffic, is that correct? The 7450-24 seems like a really good option at ~65W for 10% traffic and likely a newer feature set.

I am putting in a 6610 in my electrical room where noise doesn't matter but I am debating whether it's worthwhile to have a 7450 or 7250 switch in my office closet to service the second floor of my house vs having a Mikrotik 10G SFP switch. Cost is pretty similar between the two small/simple switches vs the Brocade prices on eBay but the power consumption and noise in a closet with little ventilation makes me hesitate. The 7450 would be pretty nice in that would have a 40G uplink to connect to my 6610 and obviously the Brocades are L3 switches but I don't think that makes any difference for the handful of devices on the "daughter" switch (all wireless will be run off the 6610).

Would love to have advice if anyone has done something similar.
 

jasonwc

Member
Dec 31, 2018
49
18
8
The ICX6610 offers 16 10G (8 SFP+ and 2 x QSFP+ breakout ports) + 2 x 40G QSFP+ ports built-in. The most common layout I've seen for the ICX7450 is 4 x 10G SFP + 2 x 40G QSFP+. You can get a max of 3 40G QSFP+ ports with no 10G ports or a max of 12 10G ports with no 40G ports. While the ICX6610 only offers PoE+ (30W), the 7450 also offers high-power (90W) PoH ports. The ICX6610 supports line-rate MACSEC encryption on the 8 SFP+ ports, which would be useful if you were running fiber outdoors to connect two buildings. So, I think it comes down to what features are important to you, how much noise you can tolerate, and how important power consumption is to you.

In terms of noise, I really don't think the ICX6610 is bad with two Rev B/C (front to back exhaust) power supplies and two fans. It's actually similar to what I measured for an ICX6450-48p with all 3 fans connected.

Regarding power consumption, the first post indicates that the power consumption isn't particularly linked to the amount of traffic. I think power consumption only increased a few watts going from 10% to 100%.
 
Last edited:

Wolfcastle

Member
Jan 3, 2022
50
23
8
The ICX6610 offers 16 10G (8 SFP+ and 2 x QSFP+ breakout ports) + 2 x 40G QSFP+ ports built-in. The most common layout I've seen for the ICX7450 is 4 x 10G SFP + 2 x 40G QSFP+. You can get a max of 3 40G QSFP+ ports with no 10G ports or a max of 12 10G ports with no 40G ports. While the ICX6610 only offers PoE+ (30W), the 7450 also offers high-power (90W) PoH ports. So, I think it comes down to how what features are important to you, how much noise you can tolerate, and how important power consumption is to you.

In terms of noise, I really don't think the ICX6610 is bad with two Rev B/C (front to back exhaust) power supplies and two fans. It's actually similar to what I measured for an ICX6450-48p with all 3 fans connected.
Yes that layout for the 7450 is what I am seeing on eBay as well. I just wanted to make sure there are no licensing limitations since there are workarounds for the models in the first post of the thread.

For my specific case, I am not using any POE on my office switch (looking at the 24 non P version), if I ever do need to run anything POE from the office switch I would add in a POE injector. The only real constraint is that my office has a couple of PCs that I want to connect at 10G to the servers on the 6610. 40G uplink is a nice to have but realistically 10G is more than enough for now.

In terms of noise, you're probably looking at them in a rackmount config with good ventilation not thrown into a closet with almost no ventilation. I don't think I am comfortable throwing 100+W 24/7 into that closet!
 

Mithril

Active Member
Sep 13, 2019
354
106
43
Has anyone used the external power option for the switches that support it like the 6450? I'd be keen on a pinout and if it's possible to wire up your own DC-DC external supply.
 

evanh

New Member
Oct 25, 2021
10
3
1
If instead you want working 40G, and your current card can't be flashed, many folks here can confirm that the HP 649281-B21 can be flashed easily using the above instructions. I've already done two cards myself, and they only cost $30. Unlike the CX311A NICs, these have working digital optical monitoring, and it seems to work with any manufacturer, since it works with the KAIAM transceivers with detailed info.
Just want to say thanks again for your guidance and precedence! My m.2 > pcie riser arrived today so I was able to run both cards on the linux box long enough to flash them including the PSID change. Put the card back in my OpnSense box and booted and... no connection... but then remembered I still had it in the 4x10 slot on the switch. One slot over into 1/2/1 and we're showing full 40G connection! Huzzah!
 

covfefe

New Member
Jan 9, 2022
4
0
1
  • Comair Rotron "Gryphon" GDA4028-12BB - discontinued, alas
    If you can find these? BUY THESE. 40x40x28mm, but 11CFM, 0.34in H2O, 8800RPM, but just 31.4dBA! I am still mad they discontinued them. No, I am not selling any of my spares.
I got 3 of them.. but they came with bare wires.. so I ordered ends to put on them..

I'm looking for the wire diagram for the original fans to know which wire is which wire.. the original fans three wires green yellow and black ( from left to right) but I dont know what green= or yellow= or black= , I assume positive, negative and tachometer.
Did you end up figuring out the wiring? Just about to order a Gryphon GDA4028 for my ICX7250-24.
My best guess is :
Code:
Fan Red-positive     <-> Brocade Yellow=12V
Fan Black-negative   <-> Brocade Black=Ground
Fan White-Tachometer <-> Brocade Red=Sensor
 

casualpie

New Member
May 20, 2021
2
0
1
Hi folks. Long time lurker, first time poster. I have a recently acquired Brocade 6610 that seems to get in a pissing match with my server every morning before it just stops talking to it like a jilted lover. Wondering if anyone else has had similar issues.

Server is running latest edition of Proxmox.
NIC is a Mellanox CX354-FCCT, latest firmware. Port being used is set to ethernet mode. Used from ebay, but shows no problems elsewhere.
Cable is a FS.com QSFP+ 40gbps passive DAC. Brand new from vendor.
Switch is a 6610-48p, updated and fully licensed thanks to a rockstar member here we all know and appreciate. Used from a local sale.

At some point overnight, the 40gb connection will start to flap, up and down within a second, every second until link error disable threshold is reached and the port goes into err-disable for 5 seconds. It'll repeat the cycle until the port is eventually released from err-disable, but the link doesn't come back up. Then I wake up and go replug the DAC and it works fine again all day. Looking up the statistics on the switch shows several hundred InErrors accumulate, but that's all. The server shows no signs anything is wrong (other than losing connection). If it's worth noting, during the day the interface will accumulate a few more InErrors (less than 10) but with no effect on performance or link. Heavy traffic doesn't seem to be the issue, as it successfully performed a VM backup over the connection about an hour before the incident.

I've tried switching NICs and DACs for identical models and configuration. Tried changing to the other QSFP port (yes the 40gb port not the breakouts). Flow control is enabled on the switch and NIC (ethtool says the interface supports pause frame use Symmetric Receive-only, but ethtool -a lists TX and RX = on). Link error disable is set on the interface with a 5 second recovery time. Errdisable recovery is set for all causes and 10 seconds recovery. Network is flat, no VLANs or special configurations apart from the error handling mentioned.

Unfortunately I don't have another switch to test with, but directly linking two servers together on the same port/DAC results in a stable connection with no hiccups.

Has anyone had a similar issue before, or am I chasing ghosts when they're in the house next door? Any troubleshooting tips would be appreciated as well.
 
Last edited: