Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Vesalius

Active Member
Nov 25, 2019
155
122
43
The thread there states one needs to flash to a non-UFI image then upgrade. Is this true or is that a typo and one needs to flash to UFI?
Must be a typo, you can upgrade from the UFI firmwares to 9, same as going to 8.0.95. I went directly from 8.0.95 to 9.

I think the issue for this poster was that you have to jump in with both feet on 9.0*, both primary and secondary boot need to be 9.0* can't keep 8.0.9* on the secondary.
 
Last edited:

LodeRunner

Active Member
Apr 27, 2019
221
90
28
Must be a typo, you can upgrade from the UFI firmwares to 9, same as going to 8.0.95. I went directly from 8.0.95 to 9.

I think the issue for this poster was that you have to jump in with both feet on 9.0*, both primary and secondary boot need to be 9.0* can't keep 8.0.9* on the secondary.
Looking at the show flash output they were trying to keep an 8.0.30 image which is not UFI and had boot loader 10.1.14. Not doing going to work at all. I suspect if they had updated to 8.0.92/95 and UFI on the primary, then loading 9.0 into secondary might work? Or copy pri to sec then put 9.0 on pri.

I think I'll test that on a spare 7150 this weekend; worst case I have to flash it from uBoot to an older loader and non UFI image to recover.

From the linked post:
Code:
SW_ICX7450#sh flash
Stack unit 1:
  Compressed Pri Code size = 28713156, Version:08.0.30hT211 (SPS08030h.bin)
  Compressed Sec Code size = 33554432, Version:09.0.00T213 (SPR09000.bin)
  Compressed Boot-Monitor Image size = 786944, Version:10.1.14T215
  Code Flash Free Space = 1650638848
 

Vesalius

Active Member
Nov 25, 2019
155
122
43
Looking at the show flash output they were trying to keep an 8.0.30 image which is not UFI and had boot loader 10.1.14. Not doing going to work at all. I suspect if they had updated to 8.0.92/95 and UFI on the primary, then loading 9.0 into secondary might work? Or copy pri to sec then put 9.0 on pri.

I think I'll test that on a spare 7150 this weekend; worst case I have to flash it from uBoot to an older loader and non UFI image to recover.

From the linked post:
Code:
SW_ICX7450#sh flash
Stack unit 1:
  Compressed Pri Code size = 28713156, Version:08.0.30hT211 (SPS08030h.bin)
  Compressed Sec Code size = 33554432, Version:09.0.00T213 (SPR09000.bin)
  Compressed Boot-Monitor Image size = 786944, Version:10.1.14T215
  Code Flash Free Space = 1650638848
You are likely right, I obviously did not parse the original question all that well. Although if the configs are not backward compatible between 8.0.9* and 9.*, do both the primary and and secondary keep separate startup configs to account for this? Could be an issue there, but i'm not sure.
 

LodeRunner

Active Member
Apr 27, 2019
221
90
28
Yeah if you wanted to roll back to v8 you might have to halt it in uBoot and do a factory reset, or just delete the start file from v9 before rebooting back to 8.
 

richtj99

Member
Jul 8, 2017
56
0
6
48
Sorry to ask a possible obvious question but... I upgraded my 6450 to a 7250. I have two esxi servers 720 & 730 -

730 has two SFP+ ports, plugged into the 7250 & the 7250 says both units are 10gb but ESXI 7.0 shows both links down. One is a DAC & one is the Brocade 57-0000075-01.

The 720 has the same setup but ESXI 6.7 - 1 dac, 1 Brocade 57-0000075-01 - Esxi 6.7 shows 1 10gb up & 1 10gb down.

Code:
1/2/5      Up      Forward Full 10G   None  Yes 1
1/2/6      Up      Forward Full 10G   None  Yes 1
1/2/7      Up      Forward Full 10G   None  Yes 1
1/2/8      Up      Forward Full 10G   None  Yes 1

The switch shows everything is up? Am I missing something or is this a ESXI "Feature"?

Sorry if I am late to the party.
 

theobassaw

New Member
Sep 30, 2021
6
1
3
I'm having a weird issue where the licenses of one of my icx-6450 became invalid. I decided to redo your licensing guide but the same issue persists. The switch is in the stack and is not the master. I wonder if refollowing the guide only applies to the master and doesn't affect the other member. Even though I apply the license to unit 2.
 

audiobahn

New Member
Sep 29, 2021
23
1
3
Just out of curiosity, do the ICX6450 show up in the Ruckus Unleashed interface or does it need to be a 7150/7250?

Also, interested to hear people's experience with the reliability of the ICX6450-48P? A ICX7150-C12P fits my requirements fine but really struggling to justify paying >500USD (400GBP) for one when used ICX6450-48P are 150GBP and used ICX7250-48P are around £300.

There is also the option of going for the vanilla ICX6540-48 with no PoE option and just get use injectors for my two APs... Cheap as chips option at 70GBP :)

Thoughts?
 
Last edited:

Vesalius

Active Member
Nov 25, 2019
155
122
43
Just out of curiosity, do the ICX6450 show up in the Ruckus Unleashed interface or does it need to be a 7150/7250?

Also, interested to hear people's experience with the reliability of the ICX6450-48P? A ICX7150-C12P fits my requirements fine but really struggling to justify paying >500USD (400GBP) for one when used ICX6450-48P are 150GBP and used ICX7250-48P are around £300.

There is also the option of going for the vanilla ICX6540-48 with no PoE option and just get use injectors for my two APs... Cheap as chips option at 70GBP :)

Thoughts?
Needs to be a icx7*** series and a recent FastIron firmware for unleashed integration, but the unleashed ICX integration really isn’t anything to pay extra for.

the icx6450 is great and reliable as most enterprise grade hardware especially for the price. If you have the space and don’t need fanless dead silent no reason to get the 7150 c12’s when 7250-48p and 6450-48p’s can be had so cheaply.
 

Spearfoot

Active Member
Apr 22, 2015
113
49
28
Sorry to ask a possible obvious question but... I upgraded my 6450 to a 7250. I have two esxi servers 720 & 730 -

730 has two SFP+ ports, plugged into the 7250 & the 7250 says both units are 10gb but ESXI 7.0 shows both links down. One is a DAC & one is the Brocade 57-0000075-01.

The 720 has the same setup but ESXI 6.7 - 1 dac, 1 Brocade 57-0000075-01 - Esxi 6.7 shows 1 10gb up & 1 10gb down.

Code:
1/2/5      Up      Forward Full 10G   None  Yes 1
1/2/6      Up      Forward Full 10G   None  Yes 1
1/2/7      Up      Forward Full 10G   None  Yes 1
1/2/8      Up      Forward Full 10G   None  Yes 1

The switch shows everything is up? Am I missing something or is this a ESXI "Feature"?

Sorry if I am late to the party.
When I used an Intel X520-DA2 NIC in one of my ESXi 6.7 servers I found that reloading my 6610 switch would take the NIC down -- no idea why. But I could get both ports back up with these commands:
Code:
esxcli network nic up -n vmnic4
esxcli network nic up -n vmnic5
Maybe something similar will work for you too?

Nowadays I use a SolarFlare SFN6122F, which isn't affected by reloading the switch.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,297
2,388
113
30
fohdeesha.com
I'm having a weird issue where the licenses of one of my icx-6450 became invalid. I decided to redo your licensing guide but the same issue persists. The switch is in the stack and is not the master. I wonder if refollowing the guide only applies to the master and doesn't affect the other member. Even though I apply the license to unit 2.
the "hw pid-prom" commands that set the correct license ID, if ran on a stack, will only apply to the master switch, there's no way to make them run on the secondary. it sounds like you ran it on the master, then imported the matching licenses on both switches? that would certainly make the second one go invalid as it no longer has a matching LID

I think you'd have to now unstack the second 6450 and wipe it, then follow the license guide on it to get the LID changed and licenses imported. then once its all valid, and your master is too, re-add it to the stack. this is why it's highly recommended to license all your stuff before stacking it :)

(theoretically, in a stack, you can jump to the console of secondary units by using the "rconsole 2" command, but I've tested this and it won't let you run the "hw pid" commands from that remote console)
---------------------
Also I suppose instead of unstacking it, you could also just use the other hidden method of changing the serial and LID, from the bootloader. console cable to your secondary 6450, reboot it and stop it in the u-boot bootloader, then run:

eeprom clear
eeprom sn 2ax5o2jk68e
reset

it'll reboot and let it boot fully and rejoin the stack like usual. I think you might have to reboot it twice with this method as the first boot after clearing the LID from u-boot has it re-generate one, but not take effect. second reboot has it come up fully changed. then ssh to the stack and import licenses to it, by specifying unit 2:


copy tftp license 192.168.1.8 ICX64xx/1-6450-prem.xml unit 2
copy tftp license 192.168.1.8 ICX64xx/2-6450-ports.xml unit 2



 

theobassaw

New Member
Sep 30, 2021
6
1
3
the "hw pid-prom" commands that set the correct license ID, if ran on a stack, will only apply to the master switch, there's no way to make them run on the secondary. it sounds like you ran it on the master, then imported the matching licenses on both switches? that would certainly make the second one go invalid as it no longer has a matching LID

I think you'd have to now unstack the second 6450 and wipe it, then follow the license guide on it to get the LID changed and licenses imported. then once its all valid, and your master is too, re-add it to the stack. this is why it's highly recommended to license all your stuff before stacking it :)

(theoretically, in a stack, you can jump to the console of secondary units by using the "rconsole 2" command, but I've tested this and it won't let you run the "hw pid" commands from that remote console)
---------------------
Also I suppose instead of unstacking it, you could also just use the other hidden method of changing the serial and LID, from the bootloader. console cable to your secondary 6450, reboot it and stop it in the u-boot bootloader, then run:

eeprom clear
eeprom sn 2ax5o2jk68e
reset

it'll reboot and let it boot fully and rejoin the stack like usual. I think you might have to reboot it twice with this method as the first boot after clearing the LID from u-boot has it re-generate one, but not take effect. second reboot has it come up fully changed. then ssh to the stack and import licenses to it, by specifying unit 2:


copy tftp license 192.168.1.8 ICX64xx/1-6450-prem.xml unit 2
copy tftp license 192.168.1.8 ICX64xx/2-6450-ports.xml unit 2



After unstacking it, setting the serial, and adding the licenses again. It's still displayed as invalid. Not sure why.
 

theobassaw

New Member
Sep 30, 2021
6
1
3
post the unedited output of both "show version" and "show license", i'm sure it's something simple
Code:
Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
    UNIT 1: compiled on Feb 13 2019 at 17:44:29 labeled as ICX64R08030t
                (9868556 bytes) from Primary ICX64R08030t.bin
        SW: Version 08.0.30tT313
  Boot-Monitor Image size = 786944, Version:10.1.05T310 (kxz10105)
  HW: Stackable ICX6450-24
==========================================================================
UNIT 1: SL 1: ICX6450-24 24-port Management Module
         Serial  #: 2 `5o2j
         License: ICX6450_BASE_ROUTER_SOFT_PACKAGE   (LID: H `KTH3    )
         P-ENGINE  0: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450-SFP-Plus 4port 40G Module
==========================================================================
  800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
  512 MB DRAM
STACKID 1  system uptime is 2 day(s) 3 hour(s) 39 minute(s) 51 second(s)
The system started at 19:00:31 Eastern Wed Dec 31 1969

 The system : started=warm start         reloaded=by "reload"
Code:
Index    Lic Mode        Lic Name               Lid/Serial No  Lic Type    Status     Lic Period    Lic Capacity
Stack unit 1:
1        Node Lock       ICX6450-PREM-LIC-SW    H4CKTH3PLN8    Normal      Invalid    Unlimited         1
2        Node Lock       ICX6450-10G-LIC-POD    H4CKTH3PLN8    Normal      Invalid    Unlimited         2
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,297
2,388
113
30
fohdeesha.com
Code:
UNIT 1: SL 1: ICX6450-24 24-port Management Module
         Serial  #: 2 `5o2j
well there's your problem, how did you set the serial to that? Be sure to run these exact commands, copy/paste them over a telnet/ssh/serial console connection. Are you on a non-US PC region settings/keyboard/etc? it's almost like certain characters got turned into something else:

Code:
enable
hw pid-prom serial 2ax5o2jk68e
hw pid-prom clear-sw-lid
reload
 

theobassaw

New Member
Sep 30, 2021
6
1
3
well there's your problem, how did you set the serial to that? Be sure to run these exact commands, copy/paste them over a telnet/ssh/serial console connection. Are you on a non-US PC region settings/keyboard/etc? it's almost like certain characters got turned into something else:

Code:
enable
hw pid-prom serial 2ax5o2jk68e
hw pid-prom clear-sw-lid
reload
I'm honestly confused as well. I'm doing this via ssh, with just a copy and paste. My keyboard is a Corsair K55, and I am in the US so I assume everything is US settings. I gonna try just manually typing the commands

Edit: Made no difference copy and paste, or manually typing it in. And it is the same keyboard I used with the other brocade icx6450 and it was fine
 
Last edited:

Astelith

New Member
Aug 4, 2021
2
0
1
Hi All!
I just got a couple of 6610 for my Proxmox cluster, anyone can tell me where to find the license to unlock the 10Gb ports? if I want the 8 in front and the breakout unlocked do I need 4 separate licenses?

A big thanks in advance,

Max
 

richtj99

Member
Jul 8, 2017
56
0
6
48
I currently have two 7250's and two 30gb lags - I am debating buying more fiber but am trying to figure if I need it. Is there a way to see the traffic statistics in a 'human' way? du -hs /traffic?

Im a little confused on how to see how much bandwidth my lag is using vs the lag capacity.

Freshly rebooted

Code:
Office7250#sh statistics lag 2

 Lag lg2 Counters:
         InOctets          55135230695           OutOctets            1903329184
           InPkts             42194771             OutPkts             20688153
  InBroadcastPkts                18043    OutBroadcastPkts                15654
  InMulticastPkts                 6399    OutMulticastPkts                11212
    InUnicastPkts             42169960      OutUnicastPkts             20661287
        InBadPkts                    0
      InFragments                    0
       InDiscards                    0           OutErrors                    0
              CRC                    0          Collisions                    0
         InErrors                    0      LateCollisions                    0
      InGiantPkts                    0
      InShortPkts                    0
         InJabber                    0
   InFlowCtrlPkts                    0     OutFlowCtrlPkts                    0
     InBitsPerSec            356719384       OutBitsPerSec            356719384
     InPktsPerSec                34072       OutPktsPerSec                16706
    InUtilization                0.89%      OutUtilization                0.03%

Office7250#sh statistics lag 5

 Lag lg5 Counters:
         InOctets           1770536724           OutOctets           60676083553
           InPkts             22583371             OutPkts             43451094
  InBroadcastPkts                 9978    OutBroadcastPkts                24850
  InMulticastPkts                  455    OutMulticastPkts                17323
    InUnicastPkts             22572829      OutUnicastPkts             43408921
        InBadPkts                    0
      InFragments                    0
       InDiscards                    0           OutErrors                    0
              CRC                    0          Collisions                    0
         InErrors                    0      LateCollisions                    0
      InGiantPkts                    0
      InShortPkts                    0
         InJabber                    0
   InFlowCtrlPkts                    0     OutFlowCtrlPkts                    0
     InBitsPerSec             11768232       OutBitsPerSec             11768232
     InPktsPerSec                18365       OutPktsPerSec                35346
    InUtilization                0.03%      OutUtilization                0.99%
 

LodeRunner

Active Member
Apr 27, 2019
221
90
28
Configure SNMP, get interface statistics that way. Use a package like LibreNMS to graph it or parse the SNMP data into Influx (using Telegraf maybe?) and display with Grafana or other equivalent tool. The on-switch statistics are either instantaneous or a short rolling average, I haven't really dug into them. So an actual monitoring solution so you can look at the averages is the best way to get what you need.
 
  • Like
Reactions: fohdeesha