Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

theobassaw

New Member
Sep 30, 2021
6
1
3
post the unedited output of both "show version" and "show license", i'm sure it's something simple
Code:
Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
    UNIT 1: compiled on Feb 13 2019 at 17:44:29 labeled as ICX64R08030t
                (9868556 bytes) from Primary ICX64R08030t.bin
        SW: Version 08.0.30tT313
  Boot-Monitor Image size = 786944, Version:10.1.05T310 (kxz10105)
  HW: Stackable ICX6450-24
==========================================================================
UNIT 1: SL 1: ICX6450-24 24-port Management Module
         Serial  #: 2 `5o2j
         License: ICX6450_BASE_ROUTER_SOFT_PACKAGE   (LID: H `KTH3    )
         P-ENGINE  0: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450-SFP-Plus 4port 40G Module
==========================================================================
  800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
  512 MB DRAM
STACKID 1  system uptime is 2 day(s) 3 hour(s) 39 minute(s) 51 second(s)
The system started at 19:00:31 Eastern Wed Dec 31 1969

 The system : started=warm start         reloaded=by "reload"
Code:
Index    Lic Mode        Lic Name               Lid/Serial No  Lic Type    Status     Lic Period    Lic Capacity
Stack unit 1:
1        Node Lock       ICX6450-PREM-LIC-SW    H4CKTH3PLN8    Normal      Invalid    Unlimited         1
2        Node Lock       ICX6450-10G-LIC-POD    H4CKTH3PLN8    Normal      Invalid    Unlimited         2
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,920
3,455
113
34
fohdeesha.com
Code:
UNIT 1: SL 1: ICX6450-24 24-port Management Module
         Serial  #: 2 `5o2j
well there's your problem, how did you set the serial to that? Be sure to run these exact commands, copy/paste them over a telnet/ssh/serial console connection. Are you on a non-US PC region settings/keyboard/etc? it's almost like certain characters got turned into something else:

Code:
enable
hw pid-prom serial 2ax5o2jk68e
hw pid-prom clear-sw-lid
reload
 

theobassaw

New Member
Sep 30, 2021
6
1
3
well there's your problem, how did you set the serial to that? Be sure to run these exact commands, copy/paste them over a telnet/ssh/serial console connection. Are you on a non-US PC region settings/keyboard/etc? it's almost like certain characters got turned into something else:

Code:
enable
hw pid-prom serial 2ax5o2jk68e
hw pid-prom clear-sw-lid
reload
I'm honestly confused as well. I'm doing this via ssh, with just a copy and paste. My keyboard is a Corsair K55, and I am in the US so I assume everything is US settings. I gonna try just manually typing the commands

Edit: Made no difference copy and paste, or manually typing it in. And it is the same keyboard I used with the other brocade icx6450 and it was fine
 
Last edited:

Astelith

New Member
Aug 4, 2021
2
0
1
Hi All!
I just got a couple of 6610 for my Proxmox cluster, anyone can tell me where to find the license to unlock the 10Gb ports? if I want the 8 in front and the breakout unlocked do I need 4 separate licenses?

A big thanks in advance,

Max
 

richtj99

Member
Jul 8, 2017
70
1
8
52
I currently have two 7250's and two 30gb lags - I am debating buying more fiber but am trying to figure if I need it. Is there a way to see the traffic statistics in a 'human' way? du -hs /traffic?

Im a little confused on how to see how much bandwidth my lag is using vs the lag capacity.

Freshly rebooted

Code:
Office7250#sh statistics lag 2

 Lag lg2 Counters:
         InOctets          55135230695           OutOctets            1903329184
           InPkts             42194771             OutPkts             20688153
  InBroadcastPkts                18043    OutBroadcastPkts                15654
  InMulticastPkts                 6399    OutMulticastPkts                11212
    InUnicastPkts             42169960      OutUnicastPkts             20661287
        InBadPkts                    0
      InFragments                    0
       InDiscards                    0           OutErrors                    0
              CRC                    0          Collisions                    0
         InErrors                    0      LateCollisions                    0
      InGiantPkts                    0
      InShortPkts                    0
         InJabber                    0
   InFlowCtrlPkts                    0     OutFlowCtrlPkts                    0
     InBitsPerSec            356719384       OutBitsPerSec            356719384
     InPktsPerSec                34072       OutPktsPerSec                16706
    InUtilization                0.89%      OutUtilization                0.03%

Office7250#sh statistics lag 5

 Lag lg5 Counters:
         InOctets           1770536724           OutOctets           60676083553
           InPkts             22583371             OutPkts             43451094
  InBroadcastPkts                 9978    OutBroadcastPkts                24850
  InMulticastPkts                  455    OutMulticastPkts                17323
    InUnicastPkts             22572829      OutUnicastPkts             43408921
        InBadPkts                    0
      InFragments                    0
       InDiscards                    0           OutErrors                    0
              CRC                    0          Collisions                    0
         InErrors                    0      LateCollisions                    0
      InGiantPkts                    0
      InShortPkts                    0
         InJabber                    0
   InFlowCtrlPkts                    0     OutFlowCtrlPkts                    0
     InBitsPerSec             11768232       OutBitsPerSec             11768232
     InPktsPerSec                18365       OutPktsPerSec                35346
    InUtilization                0.03%      OutUtilization                0.99%
 

LodeRunner

Active Member
Apr 27, 2019
557
237
43
Configure SNMP, get interface statistics that way. Use a package like LibreNMS to graph it or parse the SNMP data into Influx (using Telegraf maybe?) and display with Grafana or other equivalent tool. The on-switch statistics are either instantaneous or a short rolling average, I haven't really dug into them. So an actual monitoring solution so you can look at the averages is the best way to get what you need.
 
  • Like
Reactions: fohdeesha

theobassaw

New Member
Sep 30, 2021
6
1
3
I have managed to fix the serial problem. Yet, the licenses are still invalid. Any other suggestions on how to resolve this?

Code:
Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
    UNIT 1: compiled on Feb 13 2019 at 17:44:29 labeled as ICX64R08030t
                (9868556 bytes) from Primary ICX64R08030t.bin
        SW: Version 08.0.30tT313
  Boot-Monitor Image size = 786944, Version:10.1.05T310 (kxz10105)
  HW: Stackable ICX6450-24
==========================================================================
UNIT 1: SL 1: ICX6450-24 24-port Management Module
         Serial  #: 2ax5o2jk68e
         License: ICX6450_BASE_ROUTER_SOFT_PACKAGE   (LID: H4CKTH3PLN8)
         P-ENGINE  0: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450-SFP-Plus 4port 40G Module
==========================================================================
  800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
  512 MB DRAM
STACKID 1  system uptime is 10 minute(s) 33 second(s)
The system started at 19:00:30 Eastern Wed Dec 31 1969

 The system : started=warm start         reloaded=by "reload"

Index    Lic Mode        Lic Name               Lid/Serial No  Lic Type    Status     Lic Period    Lic Capacity
Stack unit 1:
1        Node Lock       ICX6450-PREM-LIC-SW    H4CKTH3PLN8    Normal      Invalid    Unlimited         1
2        Node Lock       ICX6450-10G-LIC-POD    H4CKTH3PLN8    Normal      Invalid    Unlimited         2
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,920
3,455
113
34
fohdeesha.com
I have managed to fix the serial problem. Yet, the licenses are still invalid. Any other suggestions on how to resolve this?

Code:
Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
    UNIT 1: compiled on Feb 13 2019 at 17:44:29 labeled as ICX64R08030t
                (9868556 bytes) from Primary ICX64R08030t.bin
        SW: Version 08.0.30tT313
  Boot-Monitor Image size = 786944, Version:10.1.05T310 (kxz10105)
  HW: Stackable ICX6450-24
==========================================================================
UNIT 1: SL 1: ICX6450-24 24-port Management Module
         Serial  #: 2ax5o2jk68e
         License: ICX6450_BASE_ROUTER_SOFT_PACKAGE   (LID: H4CKTH3PLN8)
         P-ENGINE  0: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450-SFP-Plus 4port 40G Module
==========================================================================
  800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
  512 MB DRAM
STACKID 1  system uptime is 10 minute(s) 33 second(s)
The system started at 19:00:30 Eastern Wed Dec 31 1969

The system : started=warm start         reloaded=by "reload"

Index    Lic Mode        Lic Name               Lid/Serial No  Lic Type    Status     Lic Period    Lic Capacity
Stack unit 1:
1        Node Lock       ICX6450-PREM-LIC-SW    H4CKTH3PLN8    Normal      Invalid    Unlimited         1
2        Node Lock       ICX6450-10G-LIC-POD    H4CKTH3PLN8    Normal      Invalid    Unlimited         2
delete all the lics, reboot the switch, then re-import the licenses

if that still doesn't work you might still have a weird space or invisible char in your serial string
 

cap

New Member
Sep 20, 2021
12
14
3
California
I thought I'd introduce myself. I got intoxicated by this thread, and after reading the first 50 pages of it, I drunk-purchased a new sealed ICX 7250-24 through ebay. I cannot express how much I do not need this switch! Nevertheless it is super cool and I could use some advice, request below.

It came with software 08.0.30eT211 and bootloader 10.1.05T215 (Mar 19 2015 - 16:39:20). There is a post a few pages back in which @fohdeesha explains that this bootloader is too old to flash the 8090 firmware recommended in the guide. That post contains instructions to flash a newer bootloader first. I believe there is a small error in those commands. It should be setenv uboot instead of setenv image_name, so in total:
Code:
setenv ipaddr 192.168.1.50
setenv netmask 255.255.255.0

#tell the switch the IP of your tftp server:
setenv serverip 192.168.1.8
setenv uboot ICX7xxx/spz10118.bin
update_uboot
reset
#smash b to get back into new bootloader
That got me going and I am now running 08.0.95dT213.

Between ordering and receiving the switch, I have read (but not memorized) the entire thread. After my item shipped (of course), I learned that it lacks two capabilities that I imagined it would have, and that made the idea of owning this switch so appealing to me:
  • I dreamt that this could be my sole piece of networking equipment aside from my cable modem and wireless access points. But I learned that it does not do NAT. I now understand that this is absent from most Layer-3 switches. My bad.
  • I believed I would be able to interface to my ISP's dreadful router on the router's 2.5gbe port, via the Mikrotik S+RJ10 SFP+ module. That question has been thoroughly explored in the thread and now I see that the whole idea is a no-go. (I purchased a couple of S+RJ10 modules and I can't get them to work at all on the ICX 7250. Through show media the vendor is shown, but they are recognized as SFP rather than SFP+. I could not get a link even when connected to a 1GB device on the other end, and having configured speed-duplex on the interface. Oh well, something for the parts bin. I might use them somewhere someday.)
This leaves me needing some method to connect to my ISP's router at 2.5gbe. I am open to using a suitable router for that task, but at the risk of having to hand over my nerd card, I confess that at the moment I am letting the ISP's router handle NAT (which it seems good at) and even DHCP and DNS (at which it is very poor in the feature and UI departments). I had been using a small Protectli device running pfSense until my WAN speed outstripped the ability of my pfSense box to keep up.

I basically need nothing more than a media converter to go between 10gbe SFP+ and 2.5gbe RJ45. This is where I would like your collective opinions. Some choices:
  • Netgear MS510TX switch for US$270. It has a long track record and good user reports for this exact purpose, passing traffic between 10gbe and 2.5gbe without getting all knotted up. It is rack-mountable, has an internal power supply, and is quiet enough. But it pains me to spend new money on a device that is four years old.
  • Qnap QSW-2104-2S-US switch for US$140. This is a consumer desktop switch with 2 SFP+ ports and 4 2.5G RJ45 ports. It is so new that I have not seen a single substantial review of it. No telling if it can reliably do the job I would ask of it. Other disadvantages for me are that it is not rack mountable, and that it is passively cooled. My equipment is in my garage, and it does get extremely hot in there in summer.
  • Mikrotik RB5009UG+S+IN for about US$200. Another brand new product. This is a router so I could use it in ways more flexible than the above switches, so it's not apples–apples comparison. Based on reading threads like this I'm also wary of introducing Mikrotik into my critical path.
I wonder what y'all think about interfacing 10gbe to 2.5gbe.
 
Last edited:

NateS

Active Member
Apr 19, 2021
158
91
28
Sacramento, CA, US
This leaves me needing some method to connect to my ISP's router at 2.5gbe. I am open to using a suitable router for that task, but at the risk of having to hand over my nerd card, I confess that at the moment I am letting the ISP's router handle NAT (which it seems good at) and even DHCP and DNS (at which it is very poor in the feature and UI departments). I had been using a small Protectli device running pfSense until my WAN speed outstripped the ability of my pfSense box to keep up.
Another option would be to get a newer pfSense (or opnSense) box, and put in two network cards, a 10g SFP+ and a multigig NbaseT. This would likely cost about the same or less than the switch options you listed, but it would also be a much more capable router/NAT/DNS/DHCP/etc. box than your ISP's one.
 
  • Like
Reactions: fohdeesha

cap

New Member
Sep 20, 2021
12
14
3
California
My internet connection peaks at just over 1400Mbps download speed with a direct connection between my desktop and the ISP router's 2.5gbe port, through a USB 2.5gbe dongle. Through my old pfSense box, I was seeing peaks barely over 800Mbps. So it was a 75% improvement I was leaving on the table if I changed nothing.

I'm the first to admit that I need not do anything. I have some regrets about choosing now as the time to upgrade my network infrastructure, when the available improvement isn't going to change my life in any quantifiable way. I guess I was bored and wanted to play with some new (to me) hardware. My core switch has been a Dell 2816 that I bought new 15 years ago.
 
  • Like
Reactions: noduck

fohdeesha

Kaini Industries
Nov 20, 2016
2,920
3,455
113
34
fohdeesha.com
I thought I'd introduce myself. I got intoxicated by this thread, and after reading the first 50 pages of it, I drunk-purchased a new sealed ICX 7250-24 through ebay. I cannot express how much I do not need this switch! Nevertheless it is super cool and I could use some advice, request below.

It came with software 08.0.30eT211 and bootloader 10.1.05T215 (Mar 19 2015 - 16:39:20). There is a post a few pages back in which @fohdeesha explains that this bootloader is too old to flash the 8090 firmware recommended in the guide. That post contains instructions to flash a newer bootloader first. I believe there is a small error in those commands. It should be setenv uboot instead of setenv image_name, so in total:
Code:
setenv ipaddr 192.168.1.50
setenv netmask 255.255.255.0

#tell the switch the IP of your tftp server:
setenv serverip 192.168.1.8
setenv uboot ICX7xxx/spz10118.bin
update_uboot
reset
#smash b to get back into new bootloader
That got me going and I am now running 08.0.95dT213.

Between ordering and receiving the switch, I have read (but not memorized) the entire thread. After my item shipped (of course), I learned that it lacks two capabilities that I imagined it would have, and that made the idea of owning this switch so appealing to me:
  • I dreamt that this could be my sole piece of networking equipment aside from my cable modem and wireless access points. But I learned that it does not do NAT. I now understand that this is absent from most Layer-3 switches. My bad.
  • I believed I would be able to interface to my ISP's dreadful router on the router's 2.5gbe port, via the Mikrotik S+RJ10 SFP+ module. That question has been thoroughly explored in the thread and now I see that the whole idea is a no-go. (I purchased a couple of S+RJ10 modules and I can't get them to work at all on the ICX 7250. Through show media the vendor is shown, but they are recognized as SFP rather than SFP+. I could not get a link even when connected to a 1GB device on the other end, and having configured speed-duplex on the interface. Oh well, something for the parts bin. I might use them somewhere someday.)
This leaves me needing some method to connect to my ISP's router at 2.5gbe. I am open to using a suitable router for that task, but at the risk of having to hand over my nerd card, I confess that at the moment I am letting the ISP's router handle NAT (which it seems good at) and even DHCP and DNS (at which it is very poor in the feature and UI departments). I had been using a small Protectli device running pfSense until my WAN speed outstripped the ability of my pfSense box to keep up.

I basically need nothing more than a media converter to go between 10gbe SFP+ and 2.5gbe RJ45. This is where I would like your collective opinions. Some choices:
  • Netgear MS510TX switch for US$270. It has a long track record and good user reports for this exact purpose, passing traffic between 10gbe and 2.5gbe without getting all knotted up. It is rack-mountable, has an internal power supply, and is quiet enough. But it pains me to spend new money on a device that is four years old.
  • Qnap QSW-2104-2S-US switch for US$140. This is a consumer desktop switch with 2 SFP+ ports and 4 2.5G RJ45 ports. It is so new that I have not seen a single substantial review of it. No telling if it can reliably do the job I would ask of it. Other disadvantages for me are that it is not rack mountable, and that it is passively cooled. My equipment is in my garage, and it does get extremely hot in there in summer.
  • Mikrotik RB5009UG+S+IN for about US$200. Another brand new product. This is a router so I could use it in ways more flexible than the above switches, so it's not apples–apples comparison. Based on reading threads like this I'm also wary of introducing Mikrotik into my critical path.
I wonder what y'all think about interfacing 10gbe to 2.5gbe.

woops! good catch, must have been half asleep when I wrote that. edited the post (and the instructions will be on the actual guide site soon)

as for a media converter, I had planned on using something like this to adapt from multigig to 10gbase so I can pass it into one of my switches on a vlan, then pass that WAN vlan off to my opnsense router like I am now https://www.planet.com.tw/en/product/xt-705a

they go for about $170. of course you'll still need a firewall as L3 witches don't do anything like NAT as you've found out
 

cap

New Member
Sep 20, 2021
12
14
3
California
as for a media converter, I had planned on using something like this to adapt from multigig to 10gbase so I can pass it into one of my switches on a vlan, then pass that WAN vlan off to my opnsense router like I am now https://www.planet.com.tw/en/product/xt-705a

they go for about $170.
Nice! I looked around a bit and discovered that Optcore has a version too. The photo shows the 1G version but the description describes it as 10G. It's $109 here. 10GBASE-T Copper to 10G SFP+ Fiber Media Converter | Optcore.net