Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

rootpeer

Member
Oct 19, 2019
71
16
8
It sounds like your ICX devices don't have an IP route to your VPN endpoints through the VPN server/gateway. Are you running OpenVPN on your regular (default) router on the network, or on another machine?
I am running OpenVPN on my pfSense firewall(s). Everything else is accessible just fine. This happens at 3 separate locations with 4 separate switches. Every location is running OpenVPN on pfSense.

Edit: I am not using the OOB management port, I am using a veth interface.


Code:
$ ssh -o KexAlgorithms=+diffie-hellman-group1-sha1 -o PreferredAuthentications=password -o PubkeyAuthentication=no root@10.0.10.2 -vvv
OpenSSH_8.9p1, OpenSSL 1.1.1m  14 Dec 2021
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: resolve_canonicalize: hostname 10.0.10.2 is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/user/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/user/.ssh/known_hosts2'
debug3: ssh_connect_direct: entering
debug1: Connecting to 10.0.10.2 [10.0.10.2] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug1: connect to address 10.0.10.2 port 22: Connection timed out
ssh: connect to host 10.0.10.2 port 22: Connection timed out
Edit2: OK I fixed it, @kpfleming you were right. I had skipped the Network Configuration Default Route & DNS setup from the guide: ICX6xxx Advanced - Fohdeesha Docs

I can now access the switch from my OpenVPN tunnel.
 
Last edited:
  • Like
Reactions: kpfleming

floydcohen

New Member
Jun 6, 2022
2
0
1
First off, let's just state I'm an idiot. OK, done. I've only made it sequentially to page 60 of this thread, but did a lot of targeted searches in the hopes of finding a solution.

Got an ICX6450-24p from ebay, everything was working great... except a slight fan whine. So, before ordering some of the recommended sunon or noctura fans, I figured I'd tried to see if the switch could run fanless or with the case off.
First, I disconnected both fans with cover off. Seems to run fine with ambient temps for 30 minutes, never even reaching the warning threshold.
So..... next, I covered it back up. Turned on and figured let it cook for a bit, assuming the critical shutdown would protect the switch. Well, I went about and did chores and like 15 minutes later came back to a dead switch.

Dead switch = no lights, no fans. Only a slight relay click/pop near power supply when applying power. Kill-a-watt shows 9 watts. Serial connection shows nothing.

Left it to cool down without power, nothing.
Left it to cool down with power, nothing. Google-fu seems to imply there's a 5 minute restart timer if/when temperature hits critical.

So, now trying to troubleshoot: If I disconnect the 14-pin power connector to intermediate/EPS board, no click when applying power. If I disconnect the 10-pin power from intermediate/EPS to mainboard, no click when applying power. Assuming the board is like a PC power supply, there's a "power button" switch that triggers the click when I apply power. Tried holding down the front reset button for 30+ seconds with and without applying power.

Most likely I fried the mainboard and/or the power supply. Tried removing the POE board/power cables too.

So my last hope for now, is that perhaps there's a jumper, fuse or something physical not letting the power supply (assuming it is working) apply power to the mainboard (led/fans/cpu/ram) because of an overheat event or perhaps broken temp sensor. Already ordered another icx6450-24p, so can at least check power supply swap and see what happens when that arrives.
Also if anyone has diagrams and suggestions on what to test on the plugs with a meter, that would be great! My googlefu hasn't been good at finding info on their proprietary stuff.

ebay for just icx6450-24p replacement power supply comes up empty, but the ICX6400 external power supplies seems like a way to "fix" this, although most items cost almost as much as another switch anyways lol. So if anyone has experience with the ICX6400 EPS being the main/only power supply with a presumably working mainboard, that'd be reassuring to hear.

Editted to add: no obvious burn marks, no burning/burnt smell.
 
Last edited:

c4th4r5y5

New Member
Jun 30, 2022
3
1
3
Hi all,

I am having trouble running the update_primary and update_uboot commands on my newly acquired ICX6450

(sorry if this error was troubleshooted in the past, I failed to see anyone mention this specifically)

Code:
ICX64XX-boot>> update_primary
Using egiga0 device
TFTP from server 10.0.0.192; our IP address is 10.0.0.202
Download Filename 'ICX64xx/ICX64R08030u.bin'.
Load address: 0x3000000
Download to address: 0x3000000
Loading: %#################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #######################
done
Bytes transferred = 9871112 (969f08 hex)
Error: The data payload crc 1bbc09c2 and the calculated  crc  8a109441 mismatch
I get the same error when I run the update_uboot command (except the payload/calculated crc varies). Its the ICX6450-48p

Code:
ICX64XX-boot>> version

10.1.01T310 (Apr  9 2014 - 18:01:02)
(Apr  9 2014 - 18:01:14)
CPLD version 1.1.0
PoE Software Version: N/A
I do not see any apparent problems with the switch other than this (which is step 1 lol), maybe someone experienced this or has an idea how to fix that?

Thanks for the thread/docs it is very nice!
 
Last edited:

Lone Wolf

Member
Apr 3, 2022
47
9
8
Hi all,

I am having trouble running the update_primary and update_uboot commands on my newly acquired ICX6450

(sorry if this error was troubleshooted in the past, I failed to see anyone mention this specifically)

I get the same error when I run the update_uboot command (except the payload/calculated crc varies). Its the ICX6450-48p


I do not see any apparent problems with the switch other than this (which is step 1 lol), maybe someone experienced this or has an idea how to fix that?

Thanks for the thread/docs it is very nice!
If you type in 'show ver' what does it show?
 

c4th4r5y5

New Member
Jun 30, 2022
3
1
3
If you type in 'show ver' what does it show?

Code:
 Copyright (c) 1996-2015 Brocade Communications Systems, Inc. All rights reserved.
    UNIT 1: compiled on Jan 26 2016 at 22:02:54 labeled as ICX64S08030f
                (8496808 bytes) from Primary ICX64S08030f.bin
        SW: Version 08.0.30fT311
  Boot-Monitor Image size = 786944, Version:10.1.01T310 (kxz10105)
  HW: Stackable ICX6450-48-HPOE
==========================================================================
UNIT 1: SL 1: ICX6450-48P POE 48-port Management Module
         Serial  #: BZT2546H7JE
         License: BASE_SOFT_PACKAGE   (LID: dbvHKJLjMlg)
         P-ENGINE  0: type DEF0, rev 01
         P-ENGINE  1: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450-SFP-Plus 4port 40G Module
==========================================================================
  800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
  512 MB DRAM
STACKID 1  system uptime is 1 minute(s) 18 second(s)
The system : started=cold start
I tried again today with some network changes and still the same error as my initial attempt

Looking at the values and what I am inputing for the file names, I could assume maybe my switch is already updated?
 

floydcohen

New Member
Jun 6, 2022
2
0
1
Code:
Error: The data payload crc 1bbc09c2 and the calculated  crc  8a109441 mismatch
Sounds like maybe the firmware files got corrupted. Try redownloading the original/extract again. On the really really really off chance, maybe try a different tftp server. Back in the old days, tftp servers would transfer 7-bit by default, not 8-bit.
 

c4th4r5y5

New Member
Jun 30, 2022
3
1
3
Code:
Error: The data payload crc 1bbc09c2 and the calculated  crc  8a109441 mismatch
Sounds like maybe the firmware files got corrupted. Try redownloading the original/extract again. On the really really really off chance, maybe try a different tftp server. Back in the old days, tftp servers would transfer 7-bit by default, not 8-bit.

I just tried again, this time I disabled 'Option Negociation' and made it 'Bind TFTP to address' in tftpd64 and it worked! Hopefully this clears it for future reference


(I downloaded it three time before playing with the options and also changed the way I was unzipping to no avail)
 
  • Like
Reactions: Lone Wolf

epicurean

Active Member
Sep 29, 2014
756
67
28
is it possible to do LACP on 2 of the 4 x10GB ports that were broken out from the QSFP ports of an ICX6610?
If so, how is that done?
 

kimbo

New Member
Jun 15, 2022
6
0
1
Hi again,

I've finally been able to set up my Brocade ICX 6450-24, and my recently arrived ICX 6450-48!

Two things...

On the ICX 6450-48 the fans never spin down from full speed, even though "show chassis" says it's at "speed 1". Is this a known issue?

And I'd like to make another request for any recommendations on a specific (quiet) model of Sunon, Delta or Mechatronics fan that I can get in the UK? They all seem to be out of stock or shipped from china (which I'll do if needs be, but I'm not sure how to know I'm not ordering a fake).

Thank you for the great set-up guide and all of the great information on here! :)
 

Lone Wolf

Member
Apr 3, 2022
47
9
8
On the ICX 6450-48 the fans never spin down from full speed, even though "show chassis" says it's at "speed 1". Is this a known issue?

And I'd like to make another request for any recommendations on a specific (quiet) model of Sunon, Delta or Mechatronics fan that I can get in the UK? They all seem to be out of stock or shipped from china (which I'll do if needs be, but I'm not sure how to know I'm not ordering a fake).
Is the 6450-48 updated to the latest firmware?

I went with Noctua fans on mine. They don't move as much air as the Sunon and Deltas but specs wise they are quieter so that's why I went with those. I have my switch in my entertainment center in the living room so being silent was important. I also don't have much plugged into it. If its a fully loaded switch it might need a beefier fan.
 
  • Like
Reactions: kimbo

anemoiac

New Member
Jan 7, 2021
24
11
3
Is there a Brocade command to revert to automatic fan control after manually setting the fan speed on a switch? Asking for a friend...

*I have a Quanta LB6M that's been flashed with Brocade TurboIron FW and my query wasn't getting any attention in the associated thread, so I thought I'd ask here.
 

VMman

Loading...
Jun 26, 2013
91
29
18
Hi All,

I've finally had time to setup my 6610s that have been gathering dust.

I'm new to the ICX ranges and L3 switches in general.

I have 3 6610's that have been upgraded and licenced ( many thanks to @fohdeesha) for all the great work and knowledge shared with the community!

I have connected the three 6610s units using a ring method as per the manual by using 6x QSFP DAC cables.
I wanted to know if this config is setup correctly, in summary please rate my stack :cool:

ring.jpg

6610 stack.png
 

kimbo

New Member
Jun 15, 2022
6
0
1
Is the 6450-48 updated to the latest firmware?
I think so. The update failed twice with CRC errors, but then seemed to work. Is there a way to check if it updated successfully?
Do you think the CRC errors might be connected (or it might just have been my bad wifi)?

I went with Noctua fans on mine. They don't move as much air as the Sunon and Deltas but specs wise they are quieter so that's why I went with those. I have my switch in my entertainment center in the living room so being silent was important. I also don't have much plugged into it. If its a fully loaded switch it might need a beefier fan.
Cool. Did you go for the NF-A4x10 or NF-A4x20?
 

Scarlet

Member
Jul 29, 2019
86
38
18
I went with Noctua fans on mine. They don't move as much air as the Sunon and Deltas but specs wise they are quieter so that's why I went with those. I have my switch in my entertainment center in the living room so being silent was important. I also don't have much plugged into it. If its a fully loaded switch it might need a beefier fan.
Check your temperatures with "show chassis" to make sure the noctuas acutally keep the switch cool enough. The ASIC is running at full speed all the time and gets quite hot.

I use Sunon Fans and they have to spin up on hot days to cool down the switch.

Jul 5 20:19:45:W:System:Stack unit 1 Fan speed changed automatically to 1
Jul 5 20:01:55:W:System:Stack unit 1 Fan speed changed automatically to 2
Jul 5 19:38:39:W:System:Stack unit 1 Fan speed changed automatically to 1
Jul 5 19:21:34:W:System:Stack unit 1 Fan speed changed automatically to 2
Jul 5 18:59:03:W:System:Stack unit 1 Fan speed changed automatically to 1
Jul 5 18:43:23:W:System:Stack unit 1 Fan speed changed automatically to 2
Jul 5 18:21:22:W:System:Stack unit 1 Fan speed changed automatically to 1
Jul 5 18:06:32:W:System:Stack unit 1 Fan speed changed automatically to 2
Jul 5 17:42:56:W:System:Stack unit 1 Fan speed changed automatically to 1
Jul 5 17:29:01:W:System:Stack unit 1 Fan speed changed automatically to 2
Jul 5 17:04:55:W:System:Stack unit 1 Fan speed changed automatically to 1
Jul 5 16:50:10:W:System:Stack unit 1 Fan speed changed automatically to 2
Jul 5 16:28:29:W:System:Stack unit 1 Fan speed changed automatically to 1
Jul 5 16:11:14:W:System:Stack unit 1 Fan speed changed automatically to 2
Jul 5 15:50:13:W:System:Stack unit 1 Fan speed changed automatically to 1
Jul 5 15:33:03:W:System:Stack unit 1 Fan speed changed automatically to 2
Jul 5 15:11:37:W:System:Stack unit 1 Fan speed changed automatically to 1
Jul 5 14:55:17:W:System:Stack unit 1 Fan speed changed automatically to 2
Jul 5 14:34:32:W:System:Stack unit 1 Fan speed changed automatically to 1
Jul 5 14:18:06:W:System:Stack unit 1 Fan speed changed automatically to 2
 

Lone Wolf

Member
Apr 3, 2022
47
9
8
I think so. The update failed twice with CRC errors, but then seemed to work. Is there a way to check if it updated successfully?
Do you think the CRC errors might be connected (or it might just have been my bad wifi)?
If you run 'show ver' it will tell you which firmware is running. On mine it shows SW: Version 08.0.30uT313, which is the latest for this switch. I did have some errors earlier on tho with the EEPROM and it had something to do with temp files in the file system. There is a note about that a month or two ago earlier in this thread from Fohdeesha. Although updating through wifi is never the best option if you can help it, expecially if you don't have rock solid signal strength. If you run 'show ver' and get the same version, then you know you are on the latest.


Cool. Did you go for the NF-A4x10 or NF-A4x20?
NF-A4x20 FLX. It has more airflow than the A4x10, less noise and more static pressure. Seems a no-brainer :D And they fit just fine (the built in fans are 20mm as well). If you plan on having several POE devices and lots of ethernet ports hooked up, you might want one of the Sunon or Deltas that move more air as more heat will be generated.

Whichever fan you get, you will likely need to swap two of the pins. I had to with the Nuctuas. I'd give you that info from the notes I took, but I can't seem to make sense of what I jotted down :D I'll be pulling the switch apart soon so I'm going to figure it out again.
 

Lone Wolf

Member
Apr 3, 2022
47
9
8
Check your temperatures with "show chassis" to make sure the noctuas acutally keep the switch cool enough. The ASIC is running at full speed all the time and gets quite hot.
The temps are quite fine. It's currently sitting at 57 degrees with two Noctuas. Two or three times a day on hot days it will hit fan speed 2 for a minute or two (which is also silent) and then its fine again. I don't like it thermal cycling like that tho, so I just picked up a third noctua fan and will install it soon. I'm figuring the extra air flow will keep it in fan speed 1 all the time. I monitor the switch with LibreNMS so I have nice graphs of the temperature to refer back to. It's never stayed in fan speed 2 for any length of time.

I also have very little plugged into my switch. Three 10Gig DACs, one POE Ethernet and one other Ethernet cable that rarely get used. I would've preferred the 6450-24 port instead of the 48, but all I could find was the 48. If I started adding more - especially POE devices - I would be expecting to get fans with better airflow to handle it.