Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[ProxmoxProphet]

I bought an ICX7250-48P switch a few weeks ago and for the most part have been happy with it, but I have been experiencing some strange issues with ipv6 not working and I hope someone can shed some light on this and point me in the right direction of how to fix this. So there are really multiple IPv6 problems, let me explain:


First, before I go on I will explain a little bit about my network topology:

So I have a fiber WAN from my ISP, and they do not provide me with IPv6, but I have found a workaround from this a few years ago, so I rent some VPSes and on those I am given 2x IPv4 addresses and a routed /48 IPv6 subnet. So I just set up a wireguard tunnel between the VPS and a virtualized openwrt instance in my homelab and I route the /48 through the tunnel and assign it to my VMs at home, pointing the default gateway of my VMs to the link-local IPv6 of the virtual router. This has worked flawlessly for the past few years.


Before I got the ICX7250, I was using a Mikrotik CRS309-1G-8S+IN switch and that never gave me problems with IPv6. But now, ever since I got the ICX7250, I notice strange things with IPv6 communication not working. First of all, the hosts in certain VLANs (for example VLAN 100) cannot ping or reach ANY other IPv6 addresses (link local or GUA) for other devices on the same VLAN, BUT it can reach the ICX7250's ipv6 address on that interface. Meanwhile on other VLANs, such as VLAN 199, IPv6 communication works just fine, and VMs can reach other hosts on the same VLAN and can reach the virtual router to get IPv6 connectivity.


Keep in mind, in this scenario here, the IPv6 traffic should just be switched, not routed (the idea here on VLAN 100 is that the hosts point their default gateway for v4 and v6 to the virtual router's IP (on VLAN100), not the switch's IP, and have static routes to other internal subnets via the switch's IP), so I don't know why it's not working! Maybe the fact that the hosts can reach the ICX7250's IPv6 address but not other IPv6 addresses of hosts in the same VLAN gives a hint into the problem??

Could this be a problem with neighbor discovery?


Also I created another VLAN 676, to serve as a VPN VLAN (all traffic in this VLAN gets forwarded thru the wireguard tunnel not thru my ISP connection)
And I gave the switch an IPv6 address on that interface, and set the gateway to the virtual router's IPv6 on another interface, and on that interface VLAN 676, hosts can reach the IPv6 internet, but they cannot ping other hosts' on other interfaces of my switch's GUA ipv6 addresses, and strangely when I run a traceroute, either out from a host on VL676 or to a host on VL676 from the ipv6 internet, the traceroute doesnt complete because the switch sends some icmp6 unreachable message. Why are those messages being blocked and how can I let them pass?

(In this scenario the IPv6 traffic is routed not switched, the idea here on VLAN 676 is that the hosts point their v4 and v6 default gateway to the switch)

Spoiler: Traceroute from WAN showing this

traceroute to 2001:db8:69:446::3 (2001:db8:69:446::3), 30 hops max, 80 byte packets
1 2001:db8:3:fed5::1 (2001:db8:3:fed5::1) 2.583 ms 2.540 ms 2.537 ms
2 fd90:5366:7:420:69::1002 (fd90:5366:7:420:69::1002) 154.649 ms 154.647 ms 154.644 ms
3 2001:db8:10:97a::1 (2001:db8:10:97a::1) 79.472 ms 79.470 ms 79.468 ms
4 2001:db8:69:100::1 (2001:db8:69:100::1) 86.495 ms * *
5 2001:db8:69:999::2 (2001:db8:69:999::2) 86.482 ms !N * *

The traceroute doesnt compelete but if I ping 2001:db8:69:446::3 it works (by the way 2001:db8:69:999::2 is the switch's IPv6 on VLAN99)


I tried playing with the "ipv6 enable" directive on VLAN 100 to see if I put that there would it switch IPv6 traffic but it did not make a difference.


I would greatly appreciate if someone could let me know what is going on here!


I have pasted my configuration below (IP addresses have been modified for privacy)

Spoiler: ICX7250 Config

Current configuration:
!
ver 08.0.95mT213
!
stack unit 1
module 1 icx7250-48p-poe-port-management-module
module 2 icx7250-sfp-plus-8port-80g-module
!
!
global-stp
!
!
!
vlan 1 name DEFAULT-VLAN by port
!
vlan 50 name VL50-Users by port
tagged ethe 1/2/2
untagged ethe 1/1/1 ethe 1/1/25 to 1/1/36
router-interface ve 50
spanning-tree 802-1w
spanning-tree 802-1w priority 40000
!
vlan 77 name Transit by port
untagged ethe 1/2/1
router-interface ve 77
!
vlan 99 name Transit2 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
router-interface ve 99
!
vlan 100 name VL100 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
router-interface ve 100
!
vlan 199 name DMZ1-VL by port
tagged ethe 1/1/1 ethe 1/2/2
untagged ethe 1/1/39
router-interface ve 199
!
vlan 200 name VL200 by port
tagged ethe 1/1/1 ethe 1/2/2
untagged ethe 1/1/2 to 1/1/6
router-interface ve 200
!
vlan 399 name DMZ3 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
untagged ethe 1/1/37 to 1/1/38
router-interface ve 399
!
vlan 500 name Voice-VL by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
untagged ethe 1/1/13
router-interface ve 500
!
!
vlan 676 name VPN1 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/1/25 ethe 1/2/2
router-interface ve 676
!
!
!
vlan 2223 name MGMT-VL by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/1 to 1/2/2
router-interface ve 2223
!
!
!
!
vlan 1237 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
router-interface ve 1237
!
!
!
!
!
!
system-max ip-static-route 2048
system-max ip-route-default-vrf 9000
system-max ip6-route-default-vrf 256
system-max ip-route-vrf 500
!
vrf USER-VRF
rd 693:50
ip router-id 10.31.1.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.25.100.1
ip route 10.5.0.0/22 10.31.7.1
ip route 10.55.0.0/24 ve 1237 tag 50
ip route 10.64.0.0/24 10.31.3.1
ip route 10.11.0.0/23 next-hop-vrf SERVER-VRF 10.30.10.11
ip route 10.11.10.0/24 next-hop-vrf default-vrf 10.25.100.1
ip route 10.124.0.0/16 10.31.3.1
ip route 10.200.0.0/22 ve 199 tag 50
ip route 10.125.0.0/24 ve 2223 tag 50
ip route 10.220.0.0/22 10.31.0.2
ip route 10.221.0.0/22 10.31.7.1
ip route 10.223.0.0/22 next-hop-vrf SERVER-VRF 10.30.10.1
ip route 10.25.100.0/29 ve 77 tag 50
ip route 10.27.0.0/24 ve 99 tag 50
ip route 10.30.8.0/21 ve 100 tag 50
exit-address-family
exit-vrf
!
vrf SERVER-VRF
rd 693:100
ip router-id 10.30.9.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.25.100.1
ip route 10.5.0.0/22 10.30.10.1
ip route 10.36.8.0/22 ve 399 tag 100
ip route 10.64.0.0/24 10.30.13.100
ip route 10.11.10.0/24 next-hop-vrf default-vrf 10.25.100.1
ip route 10.124.0.0/16 10.30.13.100
ip route 10.125.0.0/24 ve 2223 tag 100
ip route 10.220.0.0/22 10.30.12.1
ip route 10.221.0.0/22 10.30.14.1
ip route 10.223.0.0/22 10.30.10.1
ip route 10.25.100.0/29 ve 77 tag 100
ip route 10.27.0.0/24 ve 99 tag 100
ip route 10.29.16.0/21 ve 200 tag 100
ip route 10.31.0.0/21 ve 50 tag 100
ip route 10.31.31.0/24 ve 676 tag 100
exit-address-family
exit-vrf
!
vrf DMZ-VRF
rd 693:999
ip router-id 10.99.0.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.27.0.40
ip route 10.27.0.0/24 ve 99 tag 199
ip route 10.30.0.0/21 ve 100 tag 199
ip route 10.31.0.0/21 ve 50 tag 199
exit-address-family
exit-vrf
!
vrf SECURE-VRF
rd 693:732
ip router-id 10.55.0.254
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.25.100.1
ip route 10.11.0.0/23 next-hop-vrf SERVER-VRF 10.30.10.11
ip route 10.220.0.0/22 next-hop-vrf SERVER-VRF 10.30.12.1
ip route 10.221.0.0/22 next-hop-vrf SERVER-VRF 10.30.14.1
ip route 10.223.0.0/22 next-hop-vrf SERVER-VRF 10.30.10.1
ip route 10.25.100.0/29 ve 77 tag 1237
ip route 10.27.0.0/24 ve 99 tag 1237
ip route 10.30.8.0/21 ve 100 tag 1237
ip route 10.31.6.200/29 ve 50 tag 1237
exit-address-family
exit-vrf
!
vrf SERVER2-VRF
rd 693:200
ip router-id 10.29.20.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.27.0.55
ip route 10.11.0.0/23 next-hop-vrf SERVER-VRF 10.30.10.11
ip route 10.11.10.0/24 next-hop-vrf default-vrf 10.25.100.1
ip route 10.220.0.0/22 next-hop-vrf SERVER-VRF 10.30.12.1
ip route 10.221.0.0/22 next-hop-vrf SERVER-VRF 10.30.14.1
ip route 10.223.0.0/22 10.29.23.1
ip route 10.30.8.0/21 ve 100 tag 200
exit-address-family
exit-vrf
!
vrf VPN-VRF
rd 693:676
ip router-id 10.31.31.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.27.0.4
ip route 10.27.0.0/24 ve 99 tag 676
ip route 10.30.8.0/21 ve 100 tag 676
exit-address-family
address-family ipv6
ipv6 route ::/0 ve 99 next-hop-vrf VPN-VRF fe80::be24:11ff:fefd:fdea
ipv6 route 2001:db8:69:999::/64 ve 99
exit-address-family
exit-vrf
!
!
!
optical-monitor
optical-monitor non-ruckus-optic-enable
aaa authentication web-server default local
aaa authentication login default local
aaa authentication login privilege-mode
enable telnet authentication
enable telnet password .....
enable aaa console
ip route 0.0.0.0/0 10.25.100.1
ip route 10.5.0.0/22 next-hop-vrf SERVER-VRF 10.30.10.1
ip route 10.36.8.0/22 ve 399
ip route 10.55.0.0/24 ve 1237
ip route 10.11.0.0/23 next-hop-vrf SERVER-VRF 10.30.10.11
ip route 10.11.10.0/24 10.25.100.1
ip route 10.200.0.0/22 ve 199
ip route 10.125.0.0/24 ve 2223
ip route 10.220.0.0/22 next-hop-vrf SERVER-VRF 10.30.12.1
ip route 10.221.0.0/22 next-hop-vrf SERVER-VRF 10.30.14.1
ip route 10.223.0.0/22 next-hop-vrf SERVER-VRF 10.30.10.1
ip route 10.29.16.0/21 ve 200
ip route 10.30.8.0/21 ve 100
ip route 10.31.0.0/21 ve 50
ip route 10.31.31.0/24 ve 676
!
ipv6 unicast-routing
ipv6 route ::/0 ve 99 fe80::be24:11ff:fefd:fdea
ipv6 route 2001:db8:69:440::/60 ve 676
!
username super password .....
username management password .....
!
!
!
!
clock summer-time
clock timezone gmt GMT-05
!
!
ntp
disable serve
server 162.159.200.1
server 162.159.200.123
!
!
web-management https
!
manager disable
!
!
manager port-list 987
!
!
!
!
!
!
!
!
!
interface management 1
no ip dhcp-client enable
ip address 10.69.0.1 255.255.255.0
!
interface ethernet 1/2/1
no optical-monitor
!
interface ethernet 1/2/2
no optical-monitor
!
interface ve 50
vrf forwarding USER-VRF
ip address 10.31.1.1 255.255.248.0
!
interface ve 77
ip address 10.25.100.2 255.255.255.248
!
interface ve 99
ip address 10.27.0.1 255.255.255.0
ipv6 address fe80::d0c0:7ff:fe2a:5b9f link-local
ipv6 address 2001:db8:69:999::2/64
ipv6 enable
!
interface ve 100
vrf forwarding SERVER-VRF
ip address 10.30.9.1 255.255.248.0
ipv6 address fe80::215:5dff:fe68:9a3f link-local
ipv6 enable
!
interface ve 199
vrf forwarding DMZ-VRF
ip address 10.200.1.1 255.255.252.0
!
interface ve 200
vrf forwarding SERVER2-VRF
ip address 10.29.20.1 255.255.248.0
ipv6 address fe80::21e:c9ff:fe48:6e8c link-local
!
interface ve 399
vrf forwarding DMZ-VRF
ip address 10.36.9.1 255.255.252.0
!
interface ve 500
ip address 10.80.0.1 255.255.255.0
ip helper-address 1 10.27.0.53
!
interface ve 676
vrf forwarding VPN-VRF
ip address 10.31.31.1 255.255.255.0
ipv6 address fe80::8ae3:7ff:fe94:1a2b link-local
ipv6 address 2001:db8:69:444::1/60
ipv6 enable
!
interface ve 2223
vrf forwarding SECURE-VRF
ip address 10.125.0.2 255.255.255.0
!
interface ve 1237
vrf forwarding SECURE-VRF
ip address 10.55.0.254 255.255.255.0
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
end

 

[kuleinc]

I have a 6610 switch and I didn't see anywhere if it was a bad idea to just enable POE on all the 1GB ports? for convenience... I think power draw only goes up if you are actually using the POE on a port right? Theres no chance the POE will fry any fairly recent hardware plugged into the switch that's non POE right?

 

[jei]

Tested the power consumption of a couple of 40Gb modules mentioned in this thread.

PMD-USB measures only the PCIE slot and does not account for PSU losses before it.

The modules were tested without active link / load but had LC UPC Duplex OM3 -cable physically connected.

Gear used:
- ElmorLabs PMD-USB Power Measurement Board
- ElmorLabs PMD-AD PCIE Power Measurement Board
- Supermicro X10SLL-F booted into Ubuntu 24.04 installation media desktop

NICs tested:
- Mellanox ConnectX-4 EN 50GbE single-port QSFP28 (MCX413A-GCAT)
- Intel X710-DA2 (mfg 10/2022)

Results:

	Voltage	Current	Power
MCX413A-GCAT only	11.94 V	0.5 A	5.97 W
MCX413A-GCAT with KAIAM XQX2502	11.94 V	0.6 A	7.164 W
MCX413A-GCAT with Juniper JNP-QSFP-40G-LX4	11.93 V	0.9 A	10.737 W
Intel X710-DA2 only	12 V	0.1 A	1.2 W
Chelsio T580-LP-CR only	11.95 V	0.8 A	9.56 W
Intel X710-T2L only	12 V	0.2 A	2.4 W
Intel X710-T2L with 1x 10Gb copper connected	11.98 V	0.3 A	3.594 W

edit: Added Intel X710-DA2
edit: Added Chelsio T580-LP-CR, Intel X710-T2L

 

[blunden]

Theres no chance the POE will fry any fairly recent hardware plugged into the switch that's non POE right?

The device you connect would have to negotiate PoE for any power to be provided to it so no, both new and old hardware should be safe. It's the non-standard passive PoE that you need to be careful with.

 

[86turbodsl]

Just got in a ICX6610 POE 48 port model from ebay. I started the update following fodeesha's doc.

I'm getting this in console endlessly:
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Resetting module in slot 1 again to recover from dev fault
PoE Info: Hard Resetting in slot 1....

From what i can tell, this means the POE board is dead right? Is there any fix or is this thing toast?
I can't seem to get to the right spot to flash the POE firmware. It never really finishes booting, i can get to monitor though.

Edit: Took POE board out of unit, don't see anything wrong with it, looks perfect infact. With board out, boots totally fine.

Any fix for this? a way to update poe firmware from monitor? I do need POE and i see it's a common issue with brocade switches.

 

[JD7]

the 6450 licenses show as invalid the only thing I can see is the LID is different on this one theres an F for H4CKFH3PLN8 and the others that work show H4CKTH3PLN8. with a T is that the reason and if so how do I change it?

Will I need to go into the linux shell and delete /fast_iron/cvpersistent, /fast_iron/meta_data.bin, or /fast_iron/cvpersistent?

 

[fohdeesha]

View attachment 36415
the 6450 licenses show as invalid the only thing I can see is the LID is different on this one theres an F for H4CKFH3PLN8 and the others that work show H4CKTH3PLN8. with a T is that the reason and if so how do I change it?

Will I need to go into the linux shell and delete /fast_iron/cvpersistent, /fast_iron/meta_data.bin, or /fast_iron/cvpersistent?

looks like your license imports or download got corrupted or something, just delete the licenses ("license delete" or something like that) and download and import the lics again

 

[fohdeesha]

Just got in a ICX6610 POE 48 port model from ebay. I started the update following fodeesha's doc.

I'm getting this in console endlessly:
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Resetting module in slot 1 again to recover from dev fault
PoE Info: Hard Resetting in slot 1....

From what i can tell, this means the POE board is dead right? Is there any fix or is this thing toast?
I can't seem to get to the right spot to flash the POE firmware. It never really finishes booting, i can get to monitor though.

Edit: Took POE board out of unit, don't see anything wrong with it, looks perfect infact. With board out, boots totally fine.

Any fix for this? a way to update poe firmware from monitor? I do need POE and i see it's a common issue with brocade switches.

poe board is bad, but it will still boot with it installed. the 6610s just take a while to boot

 

[fohdeesha]

I bought an ICX7250-48P switch a few weeks ago and for the most part have been happy with it, but I have been experiencing some strange issues with ipv6 not working and I hope someone can shed some light on this and point me in the right direction of how to fix this. So there are really multiple IPv6 problems, let me explain:


First, before I go on I will explain a little bit about my network topology:

So I have a fiber WAN from my ISP, and they do not provide me with IPv6, but I have found a workaround from this a few years ago, so I rent some VPSes and on those I am given 2x IPv4 addresses and a routed /48 IPv6 subnet. So I just set up a wireguard tunnel between the VPS and a virtualized openwrt instance in my homelab and I route the /48 through the tunnel and assign it to my VMs at home, pointing the default gateway of my VMs to the link-local IPv6 of the virtual router. This has worked flawlessly for the past few years.


Before I got the ICX7250, I was using a Mikrotik CRS309-1G-8S+IN switch and that never gave me problems with IPv6. But now, ever since I got the ICX7250, I notice strange things with IPv6 communication not working. First of all, the hosts in certain VLANs (for example VLAN 100) cannot ping or reach ANY other IPv6 addresses (link local or GUA) for other devices on the same VLAN, BUT it can reach the ICX7250's ipv6 address on that interface. Meanwhile on other VLANs, such as VLAN 199, IPv6 communication works just fine, and VMs can reach other hosts on the same VLAN and can reach the virtual router to get IPv6 connectivity.


Keep in mind, in this scenario here, the IPv6 traffic should just be switched, not routed (the idea here on VLAN 100 is that the hosts point their default gateway for v4 and v6 to the virtual router's IP (on VLAN100), not the switch's IP, and have static routes to other internal subnets via the switch's IP), so I don't know why it's not working! Maybe the fact that the hosts can reach the ICX7250's IPv6 address but not other IPv6 addresses of hosts in the same VLAN gives a hint into the problem??

Could this be a problem with neighbor discovery?


Also I created another VLAN 676, to serve as a VPN VLAN (all traffic in this VLAN gets forwarded thru the wireguard tunnel not thru my ISP connection)
And I gave the switch an IPv6 address on that interface, and set the gateway to the virtual router's IPv6 on another interface, and on that interface VLAN 676, hosts can reach the IPv6 internet, but they cannot ping other hosts' on other interfaces of my switch's GUA ipv6 addresses, and strangely when I run a traceroute, either out from a host on VL676 or to a host on VL676 from the ipv6 internet, the traceroute doesnt complete because the switch sends some icmp6 unreachable message. Why are those messages being blocked and how can I let them pass?

(In this scenario the IPv6 traffic is routed not switched, the idea here on VLAN 676 is that the hosts point their v4 and v6 default gateway to the switch)

Spoiler: Traceroute from WAN showing this

traceroute to 2001:db8:69:446::3 (2001:db8:69:446::3), 30 hops max, 80 byte packets
1 2001:db8:3:fed5::1 (2001:db8:3:fed5::1) 2.583 ms 2.540 ms 2.537 ms
2 fd90:5366:7:420:69::1002 (fd90:5366:7:420:69::1002) 154.649 ms 154.647 ms 154.644 ms
3 2001:db8:10:97a::1 (2001:db8:10:97a::1) 79.472 ms 79.470 ms 79.468 ms
4 2001:db8:69:100::1 (2001:db8:69:100::1) 86.495 ms * *
5 2001:db8:69:999::2 (2001:db8:69:999::2) 86.482 ms !N * *

The traceroute doesnt compelete but if I ping 2001:db8:69:446::3 it works (by the way 2001:db8:69:999::2 is the switch's IPv6 on VLAN99)


I tried playing with the "ipv6 enable" directive on VLAN 100 to see if I put that there would it switch IPv6 traffic but it did not make a difference.


I would greatly appreciate if someone could let me know what is going on here!


I have pasted my configuration below (IP addresses have been modified for privacy)

Spoiler: ICX7250 Config

Current configuration:
!
ver 08.0.95mT213
!
stack unit 1
module 1 icx7250-48p-poe-port-management-module
module 2 icx7250-sfp-plus-8port-80g-module
!
!
global-stp
!
!
!
vlan 1 name DEFAULT-VLAN by port
!
vlan 50 name VL50-Users by port
tagged ethe 1/2/2
untagged ethe 1/1/1 ethe 1/1/25 to 1/1/36
router-interface ve 50
spanning-tree 802-1w
spanning-tree 802-1w priority 40000
!
vlan 77 name Transit by port
untagged ethe 1/2/1
router-interface ve 77
!
vlan 99 name Transit2 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
router-interface ve 99
!
vlan 100 name VL100 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
router-interface ve 100
!
vlan 199 name DMZ1-VL by port
tagged ethe 1/1/1 ethe 1/2/2
untagged ethe 1/1/39
router-interface ve 199
!
vlan 200 name VL200 by port
tagged ethe 1/1/1 ethe 1/2/2
untagged ethe 1/1/2 to 1/1/6
router-interface ve 200
!
vlan 399 name DMZ3 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
untagged ethe 1/1/37 to 1/1/38
router-interface ve 399
!
vlan 500 name Voice-VL by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
untagged ethe 1/1/13
router-interface ve 500
!
!
vlan 676 name VPN1 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/1/25 ethe 1/2/2
router-interface ve 676
!
!
!
vlan 2223 name MGMT-VL by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/1 to 1/2/2
router-interface ve 2223
!
!
!
!
vlan 1237 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
router-interface ve 1237
!
!
!
!
!
!
system-max ip-static-route 2048
system-max ip-route-default-vrf 9000
system-max ip6-route-default-vrf 256
system-max ip-route-vrf 500
!
vrf USER-VRF
rd 693:50
ip router-id 10.31.1.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.25.100.1
ip route 10.5.0.0/22 10.31.7.1
ip route 10.55.0.0/24 ve 1237 tag 50
ip route 10.64.0.0/24 10.31.3.1
ip route 10.11.0.0/23 next-hop-vrf SERVER-VRF 10.30.10.11
ip route 10.11.10.0/24 next-hop-vrf default-vrf 10.25.100.1
ip route 10.124.0.0/16 10.31.3.1
ip route 10.200.0.0/22 ve 199 tag 50
ip route 10.125.0.0/24 ve 2223 tag 50
ip route 10.220.0.0/22 10.31.0.2
ip route 10.221.0.0/22 10.31.7.1
ip route 10.223.0.0/22 next-hop-vrf SERVER-VRF 10.30.10.1
ip route 10.25.100.0/29 ve 77 tag 50
ip route 10.27.0.0/24 ve 99 tag 50
ip route 10.30.8.0/21 ve 100 tag 50
exit-address-family
exit-vrf
!
vrf SERVER-VRF
rd 693:100
ip router-id 10.30.9.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.25.100.1
ip route 10.5.0.0/22 10.30.10.1
ip route 10.36.8.0/22 ve 399 tag 100
ip route 10.64.0.0/24 10.30.13.100
ip route 10.11.10.0/24 next-hop-vrf default-vrf 10.25.100.1
ip route 10.124.0.0/16 10.30.13.100
ip route 10.125.0.0/24 ve 2223 tag 100
ip route 10.220.0.0/22 10.30.12.1
ip route 10.221.0.0/22 10.30.14.1
ip route 10.223.0.0/22 10.30.10.1
ip route 10.25.100.0/29 ve 77 tag 100
ip route 10.27.0.0/24 ve 99 tag 100
ip route 10.29.16.0/21 ve 200 tag 100
ip route 10.31.0.0/21 ve 50 tag 100
ip route 10.31.31.0/24 ve 676 tag 100
exit-address-family
exit-vrf
!
vrf DMZ-VRF
rd 693:999
ip router-id 10.99.0.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.27.0.40
ip route 10.27.0.0/24 ve 99 tag 199
ip route 10.30.0.0/21 ve 100 tag 199
ip route 10.31.0.0/21 ve 50 tag 199
exit-address-family
exit-vrf
!
vrf SECURE-VRF
rd 693:732
ip router-id 10.55.0.254
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.25.100.1
ip route 10.11.0.0/23 next-hop-vrf SERVER-VRF 10.30.10.11
ip route 10.220.0.0/22 next-hop-vrf SERVER-VRF 10.30.12.1
ip route 10.221.0.0/22 next-hop-vrf SERVER-VRF 10.30.14.1
ip route 10.223.0.0/22 next-hop-vrf SERVER-VRF 10.30.10.1
ip route 10.25.100.0/29 ve 77 tag 1237
ip route 10.27.0.0/24 ve 99 tag 1237
ip route 10.30.8.0/21 ve 100 tag 1237
ip route 10.31.6.200/29 ve 50 tag 1237
exit-address-family
exit-vrf
!
vrf SERVER2-VRF
rd 693:200
ip router-id 10.29.20.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.27.0.55
ip route 10.11.0.0/23 next-hop-vrf SERVER-VRF 10.30.10.11
ip route 10.11.10.0/24 next-hop-vrf default-vrf 10.25.100.1
ip route 10.220.0.0/22 next-hop-vrf SERVER-VRF 10.30.12.1
ip route 10.221.0.0/22 next-hop-vrf SERVER-VRF 10.30.14.1
ip route 10.223.0.0/22 10.29.23.1
ip route 10.30.8.0/21 ve 100 tag 200
exit-address-family
exit-vrf
!
vrf VPN-VRF
rd 693:676
ip router-id 10.31.31.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 10.27.0.4
ip route 10.27.0.0/24 ve 99 tag 676
ip route 10.30.8.0/21 ve 100 tag 676
exit-address-family
address-family ipv6
ipv6 route ::/0 ve 99 next-hop-vrf VPN-VRF fe80::be24:11ff:fefd:fdea
ipv6 route 2001:db8:69:999::/64 ve 99
exit-address-family
exit-vrf
!
!
!
optical-monitor
optical-monitor non-ruckus-optic-enable
aaa authentication web-server default local
aaa authentication login default local
aaa authentication login privilege-mode
enable telnet authentication
enable telnet password .....
enable aaa console
ip route 0.0.0.0/0 10.25.100.1
ip route 10.5.0.0/22 next-hop-vrf SERVER-VRF 10.30.10.1
ip route 10.36.8.0/22 ve 399
ip route 10.55.0.0/24 ve 1237
ip route 10.11.0.0/23 next-hop-vrf SERVER-VRF 10.30.10.11
ip route 10.11.10.0/24 10.25.100.1
ip route 10.200.0.0/22 ve 199
ip route 10.125.0.0/24 ve 2223
ip route 10.220.0.0/22 next-hop-vrf SERVER-VRF 10.30.12.1
ip route 10.221.0.0/22 next-hop-vrf SERVER-VRF 10.30.14.1
ip route 10.223.0.0/22 next-hop-vrf SERVER-VRF 10.30.10.1
ip route 10.29.16.0/21 ve 200
ip route 10.30.8.0/21 ve 100
ip route 10.31.0.0/21 ve 50
ip route 10.31.31.0/24 ve 676
!
ipv6 unicast-routing
ipv6 route ::/0 ve 99 fe80::be24:11ff:fefd:fdea
ipv6 route 2001:db8:69:440::/60 ve 676
!
username super password .....
username management password .....
!
!
!
!
clock summer-time
clock timezone gmt GMT-05
!
!
ntp
disable serve
server 162.159.200.1
server 162.159.200.123
!
!
web-management https
!
manager disable
!
!
manager port-list 987
!
!
!
!
!
!
!
!
!
interface management 1
no ip dhcp-client enable
ip address 10.69.0.1 255.255.255.0
!
interface ethernet 1/2/1
no optical-monitor
!
interface ethernet 1/2/2
no optical-monitor
!
interface ve 50
vrf forwarding USER-VRF
ip address 10.31.1.1 255.255.248.0
!
interface ve 77
ip address 10.25.100.2 255.255.255.248
!
interface ve 99
ip address 10.27.0.1 255.255.255.0
ipv6 address fe80::d0c0:7ff:fe2a:5b9f link-local
ipv6 address 2001:db8:69:999::2/64
ipv6 enable
!
interface ve 100
vrf forwarding SERVER-VRF
ip address 10.30.9.1 255.255.248.0
ipv6 address fe80::215:5dff:fe68:9a3f link-local
ipv6 enable
!
interface ve 199
vrf forwarding DMZ-VRF
ip address 10.200.1.1 255.255.252.0
!
interface ve 200
vrf forwarding SERVER2-VRF
ip address 10.29.20.1 255.255.248.0
ipv6 address fe80::21e:c9ff:fe48:6e8c link-local
!
interface ve 399
vrf forwarding DMZ-VRF
ip address 10.36.9.1 255.255.252.0
!
interface ve 500
ip address 10.80.0.1 255.255.255.0
ip helper-address 1 10.27.0.53
!
interface ve 676
vrf forwarding VPN-VRF
ip address 10.31.31.1 255.255.255.0
ipv6 address fe80::8ae3:7ff:fe94:1a2b link-local
ipv6 address 2001:db8:69:444::1/60
ipv6 enable
!
interface ve 2223
vrf forwarding SECURE-VRF
ip address 10.125.0.2 255.255.255.0
!
interface ve 1237
vrf forwarding SECURE-VRF
ip address 10.55.0.254 255.255.255.0
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
end

why are you defining the next-hop in all your routes as an interface and not as the next-hop IP of whatever router you intend for the routes to hit? also make sure you have ipv6 enable on all your ve interfaces. and make sure you're running the latest 8095 codetrain at least

 

[86turbodsl]

poe board is bad, but it will still boot with it installed. the 6610s just take a while to boot

You mean the errors will eventually quit? I let it run over 20 mins making errors.

 

[JD7]

looks like your license imports or download got corrupted or something, just delete the licenses ("license delete" or something like that) and download and import the lics again

I deleted the licenses and then imported them again yesterday but still same thing I'll download it again when I get back home today, thank you for your help.

 

[fohdeesha]

You mean the errors will eventually quit? I let it run over 20 mins making errors.

itll put those errors over the serial console every so often but the switch it fine, just hit enter a bunch and youll get the cli prompt. you can also just remove the poe board like you already did

 

[thehedgefrog]

Hey everyone - anything known that would cause a 6610 to start randomly locking up?

No logs since when it locks up, console is unresponsive. All ports go down but SFP+ lights stay on. No error LEDs, they stay normal. Once every 5-6 lockups it reboots by itself, else it needs a hard reset.

Changed nothing at all, it just started happening about 2 weeks ago. Locks up 2-5 times a day. I wasn't home for 4 days and somehow it didn't happen, so I'm thinking it might be something on the network?

Anyways just wondered if it was fixable or time to buy a new switch.

 

[BossHoss]

icx 7250-48p
i cant seem to get anywhere with the serial/tftp. . i get a connection with putty but it seems like most of the commands are doing nothing or something other than expected and for the tftp i tried using the included preset version but im not seeing any activity at all on the "Tftpd64 by Ph.Jounin" window. admittedly im a total noob and have no prior experience with managed switches, serial connections, or tftp. i never get to the username and password prompt and when i reboot this is what it gives me. any help or tips would be appreciated.

 

[Ralph_IT]

Have you stopped boot by smashing letter 'b', as the docs say?

 

[fohdeesha]

Have you stopped boot by smashing letter 'b', as the docs say?

what he said. you're letting it boot into the full oS

 

[BossHoss]

what he said. you're letting it boot into the full oS

Have you stopped boot by smashing letter 'b', as the docs say?

Yes I followed the directions up to the point where it said a username/password prompt should come up but it never did so I then let it boot to see what it would do and posted the screenshot. I'll try again when I get home from work in the morning and show the bootloader screen

 

[JD7]

Yes I followed the directions up to the point where it said a username/password prompt should come up but it never did so I then let it boot to see what it would do and posted the screenshot. I'll try again when I get home from work in the morning and show the bootloader screen

did you set an IP address for the switch and also have the TFTP device your using in the same subnet as the switch? connected to the management port has to be the management port if I remember correctly. for the first part before booting into the OS.

 

[BossHoss]

did you set an IP address for the switch and also have the TFTP device your using in the same subnet as the switch? connected to the management port has to be the management port if I remember correctly. for the first part before booting into the OS.

Yes used management port. As for the tftp maybe that is the problem? I used the listed commands from the documents to set the IP to 10.0.2.2 . I use 10.0.0.xxx for all my homelab stuff but wasnt sure what ip were unused in that range. Maybe part of the issue? I didn't change anything inside the tftp program. I may have misunderstood the documentation but I was under the impression that the included tftp program was pre-configured and not to be messed with.

I'll try again in the morning and get screenshots of the bootloader window, tftp settings and whatever else may help.

I didn't get any notifications that Windows Defender was blocking anything but I'll try turning that off also. Whatever the issue is I can almost guarantee it is user error, like I said this is all new to me and I had to watch a YouTube tutorial just to establish a serial connection through putty.

One thing that was throwing me off was that most of the commands I would enter from the documentation didn't seem to get any reply from the system and usually anything that did get a reply was saying that it was an incomplete or invalid command or something like that. I'm not sure if that is normal or not but it seemed a whole lot like I was just typing to myself. I'm more familiar with using a terminal for 3D printers and they usually output some sort of reply.

 

[fohdeesha]

Yes used management port. As for the tftp maybe that is the problem? I used the listed commands from the documents to set the IP to 10.0.2.2 . I use 10.0.0.xxx for all my homelab stuff but wasnt sure what ip were unused in that range. Maybe part of the issue? I didn't change anything inside the tftp program. I may have misunderstood the documentation but I was under the impression that the included tftp program was pre-configured and not to be messed with.

I'll try again in the morning and get screenshots of the bootloader window, tftp settings and whatever else may help.

I didn't get any notifications that Windows Defender was blocking anything but I'll try turning that off also. Whatever the issue is I can almost guarantee it is user error, like I said this is all new to me and I had to watch a YouTube tutorial just to establish a serial connection through putty.

One thing that was throwing me off was that most of the commands I would enter from the documentation didn't seem to get any reply from the system and usually anything that did get a reply was saying that it was an incomplete or invalid command or something like that. I'm not sure if that is normal or not but it seemed a whole lot like I was just typing to myself. I'm more familiar with using a terminal for 3D printers and they usually output some sort of reply.

you need to get into the bootloader to begin, not the full OS. you're not interrupting it by hitting B like it says