Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Craig Curtin

Member
Jun 18, 2017
94
19
8
58
dont have a console cable but forgot a the most basic check, to see if it was already running the latest version.

still cant connect to any NTP server, ping external IP (can do dns lookups), all devices can see the switch but any device connected to the router can not see anything that connected directly to the switch
Well you are going to need to post your config up here - we are not psychic - presuming you have your root level password on the switch to dump the config. If you do not then you are wasting your time and need to get a console cable.

Craig
 

Cobra0101

New Member
Nov 22, 2022
10
0
1
Well you are going to need to post your config up here - we are not psychic - presuming you have your root level password on the switch to dump the config. If you do not then you are wasting your time and need to get a console cable.

Craig
thanks solved the problem I forgot to set the default gateway.

Hopefully a quick question, if the switch is running in router mode can some of the interfaced still behave like a switch(layer 2)
from reading the manual don't think this is possible as it says that if using layer 3 modes you can only have 1 IP per subnet. Is there any way to get around this limitation? Want to use some of the ports for my main subnet which is also on my main router. So 4 SPF+ and 8 rj45s working on my main subnet and then 8 ports for 8 VLANs/subnets and the remaining 8 as spares.
 
Last edited:

Craig Curtin

Member
Jun 18, 2017
94
19
8
58
thanks solved the problem I forgot to set the default gateway.

Hopefully a quick question, if the switch is running in router mode can some of the interfaced still behave like a switch(layer 2)
Yes as long as you do not assign a routing interface to the VLAN in question then the ports will all act as Layer 2 and you will have to do offboard routing.

Craig
 

Damo

Member
Sep 7, 2022
49
5
8
Can someone link a compatible MM-SFP+ transceiver for these switches on Amazon.co.uk or Ebay.co.uk
 

rootpeer

Member
Oct 19, 2019
71
16
8
I have a weird issue and I need some help.

I have two ICX6450 switches connected to each other.
SW1 has VLAN1 on 10.1.0.0/24 and SW2 has VLAN1 on 10.2.0.0/24.
Both switches are connected via VLAN1012 on 10.1.2.0/24.
Both switches have ve's on VLAN1012 and VLAN1.
Both switches have static routes configured for each other's VLAN1 subnet via their "partner's" VLAN1012 ve. I will post the config below.

I am trying to route from SW1 VLAN1 to SW2 VLAN2. Using a host on SW1, I can ping some hosts on SW2 but not others. Traceroute does not help at all. The host is configured with a static IP and the SW1 VLAN1 ve IP as its gateway for this troubleshooting session.

SW1:
Code:
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
ve1        Up      N/A     N/A  N/A   None  N/A N/A  N/A 609c.9f79.4b20                 
ve1012     Up      N/A     N/A  N/A   None  N/A N/A  N/A 609c.9f79.4b20   

###################################################

interface ve 1                                                   
 ip address 10.1.0.2 255.255.255.0
!
interface ve 1012
 ip address 10.1.2.240 255.255.255.0

################################################

        Destination        Gateway         Port          Cost          Type Uptime
1       0.0.0.0/0          10.1.0.1        ve 1          1/1           S    6d11h
2       10.1.0.0/24        DIRECT          ve 1          0/0           D    6d11h
3       10.1.2.0/24        DIRECT          ve 1012       0/0           D    5d21h
4       10.2.0.0/24        10.1.2.241      ve 1012       1/1           S    5d21h

SW2:
Code:
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
ve1        Up      N/A     N/A  N/A   None  N/A N/A  N/A cc4e.2465.ea40                 
ve1012     Up      N/A     N/A  N/A   None  N/A N/A  N/A cc4e.2465.ea40

#################################################

interface ve 1
 ip address 10.2.0.2 255.255.255.0
!
interface ve 1012
 ip address 10.1.2.241 255.255.255.0

#################################################

        Destination        Gateway         Port          Cost          Type Uptime
1       0.0.0.0/0          10.2.0.1        ve 1          1/1           S    27m30s
2       10.1.0.0/24        10.1.2.240      ve 1012       1/1           S    5d21h
3       10.1.2.0/24        DIRECT          ve 1012       0/0           D    7d0h 
4       10.2.0.0/24        DIRECT          ve 1          0/0           D    25d4h


traceroute to reachable SW2 host from SW1 host:
Code:
$ traceroute 10.2.0.20
traceroute to 10.2.0.20 (10.2.0.20), 30 hops max, 60 byte packets
 1  _gateway (10.1.0.2)  1.420 ms  2.451 ms  1.737 ms
 2  10.1.2.241 (10.1.2.241)  219.946 ms  219.910 ms  219.886 ms
 3  10.2.0.20 (10.2.0.20)  219.929 ms  219.911 ms  219.877 ms
traceroute to an unreachable SW2 host from SW1 host:
Code:
$ traceroute 10.2.0.38
traceroute to 10.2.0.38 (10.2.0.38), 30 hops max, 60 byte packets
 1  _gateway (10.1.0.2)  1.229 ms  2.380 ms  1.677 ms
 2  10.1.2.241 (10.1.2.241)  2.658 ms  8.522 ms  8.492 ms
 3  * * *
 4  * * *
 5  * * *
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *
traceroute to a second unreachable SW2 host from SW1 host:
Code:
$ traceroute 10.2.0.4
traceroute to 10.2.0.4 (10.2.0.4), 30 hops max, 60 byte packets
 1  _gateway (10.1.0.2)  7.037 ms  7.712 ms  8.369 ms
 2  10.1.2.241 (10.1.2.241)  5.443 ms  5.417 ms  5.392 ms
 3  10.1.2.116 (10.1.2.116)  6.600 ms  6.576 ms  6.550 ms
This last one is particularly confusing to me because 10.1.2.116 is a pfSense interface on VLAN1012 at SW2 location. Why is the SW2 trying to forward the packet to the pfSense interface instead of the actual host at 10.2.0.4?

Please help!
 

Craig Curtin

Member
Jun 18, 2017
94
19
8
58
I have a weird issue and I need some help.

I have two ICX6450 switches connected to each other.
SW1 has VLAN1 on 10.1.0.0/24 and SW2 has VLAN1 on 10.2.0.0/24.
Both switches are connected via VLAN1012 on 10.1.2.0/24.
Both switches have ve's on VLAN1012 and VLAN1.
Both switches have static routes configured for each other's VLAN1 subnet via their "partner's" VLAN1012 ve. I will post the config below.

I am trying to route from SW1 VLAN1 to SW2 VLAN2. Using a host on SW1, I can ping some hosts on SW2 but not others. Traceroute does not help at all. The host is configured with a static IP and the SW1 VLAN1 ve IP as its gateway for this troubleshooting session.

SW1:
Code:
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
ve1        Up      N/A     N/A  N/A   None  N/A N/A  N/A 609c.9f79.4b20                
ve1012     Up      N/A     N/A  N/A   None  N/A N/A  N/A 609c.9f79.4b20  

###################################################

interface ve 1                                                  
ip address 10.1.0.2 255.255.255.0
!
interface ve 1012
ip address 10.1.2.240 255.255.255.0

################################################

        Destination        Gateway         Port          Cost          Type Uptime
1       0.0.0.0/0          10.1.0.1        ve 1          1/1           S    6d11h
2       10.1.0.0/24        DIRECT          ve 1          0/0           D    6d11h
3       10.1.2.0/24        DIRECT          ve 1012       0/0           D    5d21h
4       10.2.0.0/24        10.1.2.241      ve 1012       1/1           S    5d21h

SW2:
Code:
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
ve1        Up      N/A     N/A  N/A   None  N/A N/A  N/A cc4e.2465.ea40                
ve1012     Up      N/A     N/A  N/A   None  N/A N/A  N/A cc4e.2465.ea40

#################################################

interface ve 1
ip address 10.2.0.2 255.255.255.0
!
interface ve 1012
ip address 10.1.2.241 255.255.255.0

#################################################

        Destination        Gateway         Port          Cost          Type Uptime
1       0.0.0.0/0          10.2.0.1        ve 1          1/1           S    27m30s
2       10.1.0.0/24        10.1.2.240      ve 1012       1/1           S    5d21h
3       10.1.2.0/24        DIRECT          ve 1012       0/0           D    7d0h
4       10.2.0.0/24        DIRECT          ve 1          0/0           D    25d4h


traceroute to reachable SW2 host from SW1 host:
Code:
$ traceroute 10.2.0.20
traceroute to 10.2.0.20 (10.2.0.20), 30 hops max, 60 byte packets
1  _gateway (10.1.0.2)  1.420 ms  2.451 ms  1.737 ms
2  10.1.2.241 (10.1.2.241)  219.946 ms  219.910 ms  219.886 ms
3  10.2.0.20 (10.2.0.20)  219.929 ms  219.911 ms  219.877 ms
traceroute to an unreachable SW2 host from SW1 host:
Code:
$ traceroute 10.2.0.38
traceroute to 10.2.0.38 (10.2.0.38), 30 hops max, 60 byte packets
1  _gateway (10.1.0.2)  1.229 ms  2.380 ms  1.677 ms
2  10.1.2.241 (10.1.2.241)  2.658 ms  8.522 ms  8.492 ms
3  * * *
4  * * *
5  * * *
6  * * *
7  * * *
8  * * *
9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *
traceroute to a second unreachable SW2 host from SW1 host:
Code:
$ traceroute 10.2.0.4
traceroute to 10.2.0.4 (10.2.0.4), 30 hops max, 60 byte packets
1  _gateway (10.1.0.2)  7.037 ms  7.712 ms  8.369 ms
2  10.1.2.241 (10.1.2.241)  5.443 ms  5.417 ms  5.392 ms
3  10.1.2.116 (10.1.2.116)  6.600 ms  6.576 ms  6.550 ms
This last one is particularly confusing to me because 10.1.2.116 is a pfSense interface on VLAN1012 at SW2 location. Why is the SW2 trying to forward the packet to the pfSense interface instead of the actual host at 10.2.0.4?

Please help!
I am no expert on the Brocade - but having different IP subnets on the same VLAN is just asking for trouble in other implementations - not sure why you would try and do that as even if it does work it is just asking for confusion

Personally i never use the VLAN 1 (default VLAN) for any traffic and change it to a different number as part of my switch setup

Craig
 

rootpeer

Member
Oct 19, 2019
71
16
8
I am no expert on the Brocade - but having different IP subnets on the same VLAN is just asking for trouble in other implementations - not sure why you would try and do that as even if it does work it is just asking for confusion

Personally i never use the VLAN 1 (default VLAN) for any traffic and change it to a different number as part of my switch setup

Craig
The switches are at separate locations connected via a pair of Ubiquiti antennas. The two VLAN1s are not connected at L2, thus the need to route between them through VLAN1012 at L3. So, even if they both have the same VLAN tag, they are two separate networks. I don't see how them having the same tag is relevant here, they just need to forward the packets to the next hop that is statically assigned.
 

vangoose

Active Member
May 21, 2019
305
88
28
Canada
The switches are at separate locations connected via a pair of Ubiquiti antennas. The two VLAN1s are not connected at L2, thus the need to route between them through VLAN1012 at L3. So, even if they both have the same VLAN tag, they are two separate networks. I don't see how them having the same tag is relevant here, they just need to forward the packets to the next hop that is statically assigned.
You just need to configure route on each switch to so it knows where to forward traffics to. Your client only needs 1 default gateway, no need to insert custom routes.
 

ramicio

Member
Nov 30, 2022
32
4
8
Hello. I recently got an ICX-6610-24p switch, and I am having trouble with the 40g ports. I bought an Intel XL710-QDA1 NIC, and my QSFP+ DAC cable is one from FS, just "generic." I am using Ubuntu Server 20.04. I did the tutorial for licensing and whatnot and everything seems to work. 10g front ports work. On the back I have the QSFP+ cable plugged into the top left of the 4 ports. I cannot get an IP address via DHCP. It, however, does show up as a client on my router (where I have static DHCP stuff set). Pings to and fro just fine if I use an internal IP address. Go to ping a site like google, and it can't even get any DNS record for that. I updated the firmware of the NIC. No change. I can set it to a static IP address and no change. ifconfig shows almost as many RX errors as there are RX packets. None on the TX side of things. If I plug it into the breakout ports, nothing (even though I followed the tutorial and that's not in the config anymore). If I try the other proper 40g port (bottom left), I get no link, whatsoever. Should I be barking up the tree to the switch seller (fried ports), or is there more I should be trying?

Thank you.
 

dtremit

New Member
Aug 20, 2018
3
1
3
Quick question for those more familiar with these switches than I — if I need to change from the switch image to the router image, can I safely do so from the normal config prompt copy command, and then reboot (rather than going through the boot prompt)?

I configured my ICX6450 ages ago following @fohdeesha 's excellent guide, and I thought I had used the router version of the firmware — but it looks like at least my secondary image is the switch version:

Code:
SSH@icx6450#show flash
Stack unit 1:
  Compressed Pri Code size = 9871112, Version:08.0.30uT313 (primary)
  Compressed Sec Code size = 8526668, Version:08.0.30kT311 (ICX64S08030k.bin)
  Compressed Boot-Monitor Image size = 786944, Version:10.1.05T310
  Code Flash Free Space = 32514048
And indeed, I don't seem to be able to get some of the L3 features to work...
 

rootpeer

Member
Oct 19, 2019
71
16
8
Hello. I recently got an ICX-6610-24p switch, and I am having trouble with the 40g ports. I bought an Intel XL710-QDA1 NIC, and my QSFP+ DAC cable is one from FS, just "generic." I am using Ubuntu Server 20.04. I did the tutorial for licensing and whatnot and everything seems to work. 10g front ports work. On the back I have the QSFP+ cable plugged into the top left of the 4 ports. I cannot get an IP address via DHCP. It, however, does show up as a client on my router (where I have static DHCP stuff set). Pings to and fro just fine if I use an internal IP address. Go to ping a site like google, and it can't even get any DNS record for that. I updated the firmware of the NIC. No change. I can set it to a static IP address and no change. ifconfig shows almost as many RX errors as there are RX packets. None on the TX side of things. If I plug it into the breakout ports, nothing (even though I followed the tutorial and that's not in the config anymore). If I try the other proper 40g port (bottom left), I get no link, whatsoever. Should I be barking up the tree to the switch seller (fried ports), or is there more I should be trying?

Thank you.
It doesn't sound like this is your problem but might be worth investigating. The Intel NICs don't like modules that are not Intel branded. There is a kernel module option to disable this with SFP+ cards but not with the X710. Have a look at this before you start troubleshooting the switch: https://forum.vyos.io/t/unsupported-sfp-transceivers-on-intel-nic/6923/3
 

ramicio

Member
Nov 30, 2022
32
4
8
It doesn't sound like this is your problem but might be worth investigating. The Intel NICs don't like modules that are not Intel branded. There is a kernel module option to disable this with SFP+ cards but not with the X710. Have a look at this before you start troubleshooting the switch: Unsupported SFP+ transceivers on Intel NIC
The thing is there is packet flow when plugged into the "top left" port (on the back of the Brocade). The packet flow is nothing but errors. But if I plug the DAC into the port below (bottom left) it, no link is being established, whatsoever. No lights on the NIC, nothing. Also, removed the whole "breakout" thing from the config for the other 2 ports, and they act the same as the "bottom left" port. Nothing, whatsoever from them. It's a cable, not an optical module. Ethtool shows it as "40000baseCR4/Full." So, the top-left port is the only one that has signs of life, and it's just nothing but errors. No other ports show any signs of life.

Tried the link. Did make. No luck. Just spits out "ioctl: Invalid argument"
 
  • Like
Reactions: rootpeer

ramicio

Member
Nov 30, 2022
32
4
8
Had to go into the c file and change 1572 to 1584. No idea about any of the offsets. I see stuff about that all over, and I have no idea where they're getting the numbers, and I'm not a programmer, whatsoever, so I have no idea what any of that means. All I know is if I run that tool again it says it's unlocked. No change in results.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,616
2,810
113
32
fohdeesha.com
Quick question for those more familiar with these switches than I — if I need to change from the switch image to the router image, can I safely do so from the normal config prompt copy command, and then reboot (rather than going through the boot prompt)?

I configured my ICX6450 ages ago following @fohdeesha 's excellent guide, and I thought I had used the router version of the firmware — but it looks like at least my secondary image is the switch version:

Code:
SSH@icx6450#show flash
Stack unit 1:
  Compressed Pri Code size = 9871112, Version:08.0.30uT313 (primary)
  Compressed Sec Code size = 8526668, Version:08.0.30kT311 (ICX64S08030k.bin)
  Compressed Boot-Monitor Image size = 786944, Version:10.1.05T310
  Code Flash Free Space = 32514048
And indeed, I don't seem to be able to get some of the L3 features to work...
it doesn't matter what's in the secondary slot because by default it boots from the primary, it will only boot from the secondary if the primary is corrupted or you explicitly tell it to. What's your output of "show version" and what l3 features are you having issues with?
 

Mailkit

New Member
Nov 6, 2017
8
0
1
77
Hello,

Is it possible to change the welcome screen to my liking and remove brocade name?

Thanking you
 

dtremit

New Member
Aug 20, 2018
3
1
3
it doesn't matter what's in the secondary slot because by default it boots from the primary, it will only boot from the secondary if the primary is corrupted or you explicitly tell it to. What's your output of "show version" and what l3 features are you having issues with?
Thanks for the quick reply — and for your guide, which was really invaluable! (Very long-time lurker here, I'm afraid.)

EDIT — realized after I posted this that I did in fact miss something in my config — I set up the router-interface for all of my new VLANs, but not the existing one they'll use to get outbound. D'oh.
 
Last edited:
  • Like
Reactions: fohdeesha

Cobra0101

New Member
Nov 22, 2022
10
0
1
what wrong with my running config, please? Cant connect to my web management, or if connect to my switch my router management but can ping to external site from the router now. Running in router mode



!
aaa authentication snmp-server default local
aaa authentication web-server default local
aaa authentication login default local
ip default-network 10.0.0.0/23
ip dns server-address 10.0.0.1 10.0.0.5
ip route 0.0.0.0/0 10.0.0.1
!
username *** password .....
!
!
clock timezone us Alaska
!
!
ntp
server 134.0.16.1 minpoll 5
server 162.159.200.1
server 217.114.59.66
server 185.83.169.27
!
!
web-management enable ethe 1/1/1
!
interface ethernet 1/1/1
ip address 10.0.0.4 255.255.255.0
!
!
end
 
Last edited:

Craig Curtin

Member
Jun 18, 2017
94
19
8
58
Hello. I recently got an ICX-6610-24p switch, and I am having trouble with the 40g ports. I bought an Intel XL710-QDA1 NIC, and my QSFP+ DAC cable is one from FS, just "generic." I am using Ubuntu Server 20.04. I did the tutorial for licensing and whatnot and everything seems to work. 10g front ports work. On the back I have the QSFP+ cable plugged into the top left of the 4 ports. I cannot get an IP address via DHCP. It, however, does show up as a client on my router (where I have static DHCP stuff set). Pings to and fro just fine if I use an internal IP address. Go to ping a site like google, and it can't even get any DNS record for that. I updated the firmware of the NIC. No change. I can set it to a static IP address and no change. ifconfig shows almost as many RX errors as there are RX packets. None on the TX side of things. If I plug it into the breakout ports, nothing (even though I followed the tutorial and that's not in the config anymore). If I try the other proper 40g port (bottom left), I get no link, whatsoever. Should I be barking up the tree to the switch seller (fried ports), or is there more I should be trying?

Thank you.
You need to post up your config and also provide a little more detail - are you saying you are connecting the Intel card to the breakout cable - what is then mean to be providing DHCP to the Intel card and which switch port is that connected to ?

Craig