Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Good news is that I have confirmed these fan modules are the same as the ICX-FAN10-E's.
PING.
Okay. Fan 1 is the one I bought from eBay for $15, and Fan 2 is one of the original ones that came with the switch.
I need to set the route 0.0.0.0/0 22.22.33.21 (isp assigned upstream gateway) on the /30 VLAN, with the VE having IP 22.22.33.22.
hm, I think I am missing a static route from the /27 vlan to the /30 to make the /27 be able to reach internet.
ip route 0.0.0.0/0 ve30 (where ve30 is the VE in vlan having /30 subnet)
Will this work, or is it better to go with the IP assigned to that ve30 (22.22.33.22)? I read somewhere this will cause very large ARP cache?
I will actually have routers connected to the /27 VLAN ports, to NAT the IPs I got from the ISP.
Last edited:
Actually, can I just use the VLAN in which I have the /30 from ISP assigned to VE30 (22.22.33.22) if I just create another VE27 in the same VLAN with one of the assigned IPs from the /27 (and set its default route to the VE30) and use this VE27 IP as default gateway in connected PFSense routers, removing the need to have two vlans?
I'm having a similar issue as @Paul Mew I believe.
Everything is working fine on the default vlan. I only have 1 upstream connection to the router (1/1/1) and seems when I try to add it to vlan 2, it takes out everything on vlan 1 :/
with default vlan:
I add a new vlan
and after that I lose connection to the switch as it seems 1/1/1 gets removed from vlan 1 and only gets added to vlan 2.
I'm probably an idiot, but IDK anything about networking
I was trying to follow this guide, which does something similar (i think)
Everything is working fine on the default vlan. I only have 1 upstream connection to the router (1/1/1) and seems when I try to add it to vlan 2, it takes out everything on vlan 1 :/
with default vlan:
Code:
#show vlan
Total PORT-VLAN entries: 1
Maximum PORT-VLAN entries: 64
Legend: [Stk=Stack-Id, S=Slot]
PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree Off
Untagged Ports: (U1/M1) 1 2 3 4 5 6 7 8 9 10 11 12
Untagged Ports: (U1/M1) 13 14 15 16 17 18 19 20 21 22 23 24
Untagged Ports: (U1/M1) 25 26 27 28 29 30 31 32 33 34 35 36
Untagged Ports: (U1/M1) 37 38 39 40 41 42 43 44 45 46 47 48
Untagged Ports: (U1/M2) 1 2 3 4
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None
Mac-Vlan Ports: None
Monitoring: Disabled
Code:
#vlan 2
#tagged ethernet 1/1/1I'm probably an idiot, but IDK anything about networking
I was trying to follow this guide, which does something similar (i think)
If you're running 8.0.30 you need to configure dual-mode on E1/1/1.I'm having a similar issue as @Paul Mew I believe.
Everything is working fine on the default vlan. I only have 1 upstream connection to the router (1/1/1) and seems when I try to add it to vlan 2, it takes out everything on vlan 1 :/
with default vlan:
I add a new vlanCode:#show vlan Total PORT-VLAN entries: 1 Maximum PORT-VLAN entries: 64 Legend: [Stk=Stack-Id, S=Slot] PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree Off Untagged Ports: (U1/M1) 1 2 3 4 5 6 7 8 9 10 11 12 Untagged Ports: (U1/M1) 13 14 15 16 17 18 19 20 21 22 23 24 Untagged Ports: (U1/M1) 25 26 27 28 29 30 31 32 33 34 35 36 Untagged Ports: (U1/M1) 37 38 39 40 41 42 43 44 45 46 47 48 Untagged Ports: (U1/M2) 1 2 3 4 Tagged Ports: None Uplink Ports: None DualMode Ports: None Mac-Vlan Ports: None Monitoring: Disabled
and after that I lose connection to the switch as it seems 1/1/1 gets removed from vlan 1 and only gets added to vlan 2.Code:#vlan 2 #tagged ethernet 1/1/1
I'm probably an idiot, but IDK anything about networking
I was trying to follow this guide, which does something similar (i think)
Dual-mode VLAN ports
I believe you are overthinking this a bit
Routes are not specific to VLANs or VEs (generally). They are global entries in the routing table on the device. When you create a VE in VLAN30, and a VE in VLAN27, and put IP addresses/subnets on those VEs, then the ICX has routes to those subnets and can route traffic between them. The only additional step for devices on those subnets to talk to each other is for *them* to be configured to know that they should use the ICX as their gateway (either default - for all traffic, or explicit for those subnets).
In addition to the internal routes between the VLANs, you'll need a default route on the ICX to tell it how to route traffic to external destinations. That should be a 0.0.0.0 route to the ISP's provided gateway address, which will be an address within the /30 subnet. The ICX will know that it needs to send traffic out through that VE/VLAN due to the gateway address.
That should be it: any device on the /27 subnet with a default gateway set to the ICX will then be able to send traffic to external destinations.
And no, you cannot create two VEs in the same VLAN. In a non-VLAN world, you'd be using two physical interfaces on a router, connected to separate physical subnets, so you'll be emulating that using VLANs.
Thank you, will read it a few times and try to implement it that way. Was watching this videoI believe you are overthinking this a bit
Hello all,
I bought an ebay ICX7250-48 running SW: Version 08.0.30hT213 and running into issues getting it to 8095
I'm following the guide fohdeesha posted on page 1
I run through and update update of uboot with no issues.
factory set-default completes quickly
update_primary returns the below:
As far as I can tell everything looks good
however boot_primary returns the below:
MD5 sum on the downloaded 08090mc.bin is ea452ea2a066c27051bcb342ef67b13c and matches what in on the TFTP server and my downloaded copy fresh as of 2 hours ago.
show version tells me my uboot is right
Boot Environment
I'm sorry if this has been answered up thread but I can't seem to find the answer in my searching.
I bought an ebay ICX7250-48 running SW: Version 08.0.30hT213 and running into issues getting it to 8095
I'm following the guide fohdeesha posted on page 1
I run through and update update of uboot with no issues.
factory set-default completes quickly
update_primary returns the below:
Code:
ICX7250-Boot> update_primary
Updating PRIMARY image over TFTP
Using bcmiproc_eth-0 device
TFTP from server 192.168.1.12; our IP address is 192.168.1.2
Filename 'ICX7xxx/SPR08090mc.bin'.
Load address: 0x1000000
Loading: #################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
###########
done
Bytes transferred = 33554432 (2000000 hex)
img_sz=0x2000000 block_count = 0x20
NAND erase.spread: device 0 offset 0x0, size 0x2000000
Erasing at 0x1f00000 -- 100% complete.
OK
NAND write: device 0 offset 0x0, size 0x2000000
................................ 33554432 bytes written: OK
TFTP to Flash Done.however boot_primary returns the below:
Code:
resetting ...
Brocade Bootloader: 10.1.06T215 (Dec 14 2015 - 13:10:57)
Validate Shmoo parameters stored in flash ..... OK
Restoring Shmoo parameters from flash .....
Running simple memory test ..... OK
ICX7250-48 Copper (Non-POE), PVT1
SYS CPLD VER: 0x6, Released Ver: 0
Enter 'b' to stop at boot monitor: 0
bootdelay: ===
Booting image from Primary
NAND read: device 0 offset 0x0, size 0x2000000
................................ 33554432 bytes read: OK
Wrong Image Format for bootm command
ERROR: can't get kernel image!
could not boot from primary, no valid image; trying to boot from secondary
BOOTING image from Secondary
NAND read: device 0 offset 0x4000000, size 0x2000000
................................................................ 33554432 bytes read: OK
## Booting kernel from Legacy Image at 70000200 ...
Image Name: Image
ARM Linux Kernel Image (uncompressed)
Data Size: 31678084 Bytes = Load Address: 61008000
Entry Point: 61008000
Verifying Checksum ... OK
Loading Kernel Image ... OK
OK
Starting kernel ...show version tells me my uboot is right
Code:
Compressed Boot-Monitor Image size = 786944, Version:10.1.06T215 (spz10118 )Boot Environment
Code:
ICX7250-Boot> show_image
===========================
Primary :
NAND read: device 0 offset 0x0, size 0x200
512 bytes read: OK
SPR08090mc(33554432)
Secondary:
NAND read: device 0 offset 0x4000000, size 0x200
512 bytes read: OK
SPR08030h(31678660)
===========================
Last edited:
you need to go to an intermediate version boot and run first as there is a filesystem change that occurs. then you can go to later versions.
my recollection is like 8080 and SPZ10114
Last edited:
Follow the link below to get to a version capable of upgrading to 8095
Commscope Technical Content Portal
SSH key access on the 6450 series - once more.
So i now have my ICX6450-48p fully flashed with latest, licenced, and set up. As usual i had issues getting SSH key access to work:
First i generated a 2048 bit RSA key with
But this method doesn't add the
To add that line, i had to add the -e (export) command to ssh-keygen. I ran ssh-keygen again:
this added the ---- BEGIN SSH2 PUBLIC KEY ---- and ---- END SSH2 PUBLIC KEY ---- to my public keyfile. After uploading that public key it finally worked.
Note i did try to manually add those lines to my public keyfile at first, which did NOT work. It just resulted in
So i wrote this for anyone that has the same issues. I didn't notice this being documented in @fohdeesha 's guide for the ICX6450 series.
So i now have my ICX6450-48p fully flashed with latest, licenced, and set up. As usual i had issues getting SSH key access to work:
First i generated a 2048 bit RSA key with
Code:
ssh-keygen -t rsa -b 2048---- BEGIN SSH2 PUBLIC KEY ---- line to the public keyfile. This produces an error when attempting to upload the public key via tftp:
Code:
icx6450(config)#ERROR: key# 1 must begin with ---- BEGIN SSH2 PUBLIC KEY ----
Code:
ssh-keygen -t rsa -b 2048 -e
Enter file in which the key is (/home/user/.ssh/id_rsa):Note i did try to manually add those lines to my public keyfile at first, which did NOT work. It just resulted in
Permission denied (publickey)So i wrote this for anyone that has the same issues. I didn't notice this being documented in @fohdeesha 's guide for the ICX6450 series.
Last edited:
it probably reverted to the secondary bootloader that was an older version during that first boot (when it failed) - this happened to one or two other people. spz10118.bin is the recommended bootloader for 8090mc and is what ruckus distributes for that version, so if that was the bootloader actually booting it should have loaded 8090mc without issues. These things like to fall back to the secondary bootloader sometimes when in the middle of a manual upgrade, need to add a couple commands to the guide to prevent it
I went and try to do it, but I found out that the ISP is giving me these with /30 /27 in a 301VLAN. 1/3/1 is my fiber from ISP. With the below VLAN setup I can use U1/M1 ports 11 and 12 to get my VOIP phone and TV channels running directly, then if I go to U1/M1 2,4,6,8,10 I can either get a DHCP IP from ISP and if I go with VLAN 301 on the same ports I can use the /30.I believe you are overthinking this a bit
.
.
.
That should be it: any device on the /27 subnet with a default gateway set to the ICX will then be able to send traffic to external destinations.
Now if I were to set VE on VLAN 301 with the ISP provided IP/gateway from /30, will I be able to get any device which will be set with /27 and default gateway to the ICX reach internet if connected to these ports? I just want to add the /30 in the 301 somehow to get rid of the pfsense doing the routing between /30 /27 I am using now.
Code:
PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree Off
Untagged Ports: (U1/M1) 11 12
Untagged Ports: (U1/M3) 2
Tagged Ports: None
Uplink Ports: None
DualMode Ports: (U1/M1) 2 4 6 8 10
DualMode Ports: (U1/M3) 1
Mac-Vlan Ports: None
Monitoring: Disabled
PORT-VLAN 400, Name CORE, Priority level0, Spanning tree Off
Untagged Ports: (U1/M3) 3 5 7
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None
Mac-Vlan Ports: None
Monitoring: Disabled
PORT-VLAN 301, Name ISP, Priority level0, Spanning tree Off
Untagged Ports: None
Tagged Ports: (U1/M1) 2 4 6 8 10
Tagged Ports: (U1/M3) 1
Uplink Ports: None
DualMode Ports: None
Mac-Vlan Ports: None
Monitoring: Disabled
PORT-VLAN 1000, Name Lan, Priority level0, Spanning tree Off
Untagged Ports: (U1/M1) 25 26 27 28 29 30 31 32 33 34 35 36
Untagged Ports: (U1/M1) 37 38 39 40 41 42 43 44 45 46 47 48
Untagged Ports: (U1/M3) 4 6 8
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None
Mac-Vlan Ports: None
Monitoring: DisabledThanks for the thumbs up @fohdeesha , if only you would recognize the significance of my tape mod Really, anyone running their switch with less than 3 fans should remember to block the empty fan cutouts with a piece of tape, the results are pretty dramatic.
Anyways, for my fan mod on the 6450, i'm planning on trying the Arctic S4028-6K
they have 2 models, 6K and 15K, and they seem to be designed exactly for our purpose. Plus they are very affordable.
Really, anyone running their switch with less than 3 fans should remember to block the empty fan cutouts with a piece of tape, the results are pretty dramatic.Anyways, for my fan mod on the 6450, i'm planning on trying the Arctic S4028-6K
they have 2 models, 6K and 15K, and they seem to be designed exactly for our purpose. Plus they are very affordable.
Hi folks, I'm the proud owner of a straight-from-ebay ICX6450 and R500, and I can't power one from the other. The R500 does power up from another PoE switch, but doesn't power up off any port on the 6450. This is a 6450-24P according to the labeling, and I've upgraded the PoE firmware as per @fohdeesha's fantastic guide, set inline power on all the ports, but there's no power coming out of any of them.
One possibly peculiar thing is that although show inline power detail works, unlike other people's results from upthread mine doesn't show any values under cumulative statuses:
So, erm, what is the extremely obvious thing that I've forgotten to do?
One possibly peculiar thing is that although show inline power detail works, unlike other people's results from upthread mine doesn't show any values under cumulative statuses:
SSH@core#show inline power detail
Power Supply Data On stack 1:
++++++++++++++++++
Power Supply Data:
++++++++++++++++++
Power Supply #1:
Max Curr: 6.8 Amps
Voltage: 54.0 Volts
Capacity: 370 Watts
power supply 2 is not present
POE Details Info. On Stack 1 :
General PoE Data:
+++++++++++++++++
Firmware
Version
----------------
02.1.0 Build 004
Cumulative Port State Data:
+++++++++++++++++++++++++++
#Ports #Ports #Ports #Ports #Ports #Ports #Ports
Admin-On Admin-Off Oper-On Oper-Off Off-Denied Off-No-PD Off-Fault
-------------------------------------------------------------------------
Cumulative Port Power Data:
+++++++++++++++++++++++++++
#Ports #Ports #Ports Power Power
Pri: 1 Pri: 2 Pri: 3 Consumption Allocation
-----------------------------------------------
SSH@core#
Power Supply Data On stack 1:
++++++++++++++++++
Power Supply Data:
++++++++++++++++++
Power Supply #1:
Max Curr: 6.8 Amps
Voltage: 54.0 Volts
Capacity: 370 Watts
power supply 2 is not present
POE Details Info. On Stack 1 :
General PoE Data:
+++++++++++++++++
Firmware
Version
----------------
02.1.0 Build 004
Cumulative Port State Data:
+++++++++++++++++++++++++++
#Ports #Ports #Ports #Ports #Ports #Ports #Ports
Admin-On Admin-Off Oper-On Oper-Off Off-Denied Off-No-PD Off-Fault
-------------------------------------------------------------------------
Cumulative Port Power Data:
+++++++++++++++++++++++++++
#Ports #Ports #Ports Power Power
Pri: 1 Pri: 2 Pri: 3 Consumption Allocation
-----------------------------------------------
SSH@core#
So, erm, what is the extremely obvious thing that I've forgotten to do?
SSH@core#show ver
Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
UNIT 1: compiled on Apr 23 2020 at 10:57:06 labeled as ICX64R08030u
(9871112 bytes) from Primary ICX64R08030u.bin
SW: Version 08.0.30uT313
Boot-Monitor Image size = 512, Version:10.1.05T310 (kxz10105)
HW: Stackable ICX6450-24-HPOE
==========================================================================
UNIT 1: SL 1: ICX6450-24P POE 24-port Management Module
Serial #: 2ax5o2jk68e
License: ICX6450_PREM_ROUTER_SOFT_PACKAGE (LID: H4CKTH3PLN8)
P-ENGINE 0: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450-SFP-Plus 4port 40G Module
==========================================================================
800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
512 MB DRAM
STACKID 1 system uptime is 32 minute(s) 47 second(s)
The system started at 22:01:02 Pacific Fri May 06 2022
The system : started=warm start reloaded=by "reload"
SSH@core#
Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
UNIT 1: compiled on Apr 23 2020 at 10:57:06 labeled as ICX64R08030u
(9871112 bytes) from Primary ICX64R08030u.bin
SW: Version 08.0.30uT313
Boot-Monitor Image size = 512, Version:10.1.05T310 (kxz10105)
HW: Stackable ICX6450-24-HPOE
==========================================================================
UNIT 1: SL 1: ICX6450-24P POE 24-port Management Module
Serial #: 2ax5o2jk68e
License: ICX6450_PREM_ROUTER_SOFT_PACKAGE (LID: H4CKTH3PLN8)
P-ENGINE 0: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450-SFP-Plus 4port 40G Module
==========================================================================
800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
512 MB DRAM
STACKID 1 system uptime is 32 minute(s) 47 second(s)
The system started at 22:01:02 Pacific Fri May 06 2022
The system : started=warm start reloaded=by "reload"
SSH@core#
SSH@core#show inline power
Power Capacity: Total is 370000 mWatts. Current Free is 370000 mWatts.
Power Allocations: Requests Honored 0 times
Port Admin Oper ---Power(mWatts)--- PD Type PD Class Pri Fault/
State State Consumed Allocated Error
--------------------------------------------------------------------------
1/1/1 On Off 0 0 n/a n/a 3 n/a
1/1/2 On Off 0 0 n/a n/a 3 n/a
1/1/3 On Off 0 0 n/a n/a 3 n/a
1/1/4 On Off 0 0 n/a n/a 3 n/a
1/1/5 On Off 0 0 n/a n/a 3 n/a
1/1/6 On Off 0 0 n/a n/a 3 n/a
1/1/7 On Off 0 0 n/a n/a 3 n/a
1/1/8 On Off 0 0 n/a n/a 3 n/a
1/1/9 On Off 0 0 n/a n/a 3 n/a
1/1/10 On Off 0 0 n/a n/a 3 n/a
1/1/11 On Off 0 0 n/a n/a 3 n/a
1/1/12 On Off 0 0 n/a n/a 3 n/a
1/1/13 On Off 0 0 n/a n/a 3 n/a
1/1/14 On Off 0 0 n/a n/a 3 n/a
1/1/15 On Off 0 0 n/a n/a 3 n/a
1/1/16 On Off 0 0 n/a n/a 3 n/a
1/1/17 On Off 0 0 n/a n/a 3 n/a
1/1/18 On Off 0 0 n/a n/a 3 n/a
1/1/19 On Off 0 0 n/a n/a 3 n/a
1/1/20 On Off 0 0 n/a n/a 3 n/a
1/1/21 On Off 0 0 n/a n/a 3 n/a
1/1/22 On Off 0 0 n/a n/a 3 n/a
1/1/23 On Off 0 0 n/a n/a 3 n/a
1/1/24 On Off 0 0 n/a n/a 3 n/a
--------------------------------------------------------------------------
Total 0 0
SSH@core#
Power Capacity: Total is 370000 mWatts. Current Free is 370000 mWatts.
Power Allocations: Requests Honored 0 times
Port Admin Oper ---Power(mWatts)--- PD Type PD Class Pri Fault/
State State Consumed Allocated Error
--------------------------------------------------------------------------
1/1/1 On Off 0 0 n/a n/a 3 n/a
1/1/2 On Off 0 0 n/a n/a 3 n/a
1/1/3 On Off 0 0 n/a n/a 3 n/a
1/1/4 On Off 0 0 n/a n/a 3 n/a
1/1/5 On Off 0 0 n/a n/a 3 n/a
1/1/6 On Off 0 0 n/a n/a 3 n/a
1/1/7 On Off 0 0 n/a n/a 3 n/a
1/1/8 On Off 0 0 n/a n/a 3 n/a
1/1/9 On Off 0 0 n/a n/a 3 n/a
1/1/10 On Off 0 0 n/a n/a 3 n/a
1/1/11 On Off 0 0 n/a n/a 3 n/a
1/1/12 On Off 0 0 n/a n/a 3 n/a
1/1/13 On Off 0 0 n/a n/a 3 n/a
1/1/14 On Off 0 0 n/a n/a 3 n/a
1/1/15 On Off 0 0 n/a n/a 3 n/a
1/1/16 On Off 0 0 n/a n/a 3 n/a
1/1/17 On Off 0 0 n/a n/a 3 n/a
1/1/18 On Off 0 0 n/a n/a 3 n/a
1/1/19 On Off 0 0 n/a n/a 3 n/a
1/1/20 On Off 0 0 n/a n/a 3 n/a
1/1/21 On Off 0 0 n/a n/a 3 n/a
1/1/22 On Off 0 0 n/a n/a 3 n/a
1/1/23 On Off 0 0 n/a n/a 3 n/a
1/1/24 On Off 0 0 n/a n/a 3 n/a
--------------------------------------------------------------------------
Total 0 0
SSH@core#
Last edited:
This thread is just bonkers, truly insane levels of deep diving on display here by all - bravo!
Somewhat predictably I've now got an ICX6450-24P on its way from USA to UK, to join a Lenovo m720q/pfSense combo that STH also convinced me I desperately needed. You guys have a lot to answer for... I'm excited to start playing
Somewhat predictably I've now got an ICX6450-24P on its way from USA to UK, to join a Lenovo m720q/pfSense combo that STH also convinced me I desperately needed. You guys have a lot to answer for... I'm excited to start playing