Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[safado]

Need some help as I'm stuck and not sure where to go to next.

I have a 6610 that is licensed, has the stack removed per Fodeesha etc. I added a 6650 about a year ago and used a 40GB port to connect to the 6610 and it worked perfectly. No issues at all. I had a reboot a couple of days ago and since then I've been unable to get access to the 6650 over the network. I've re-ran setup and assigned the IP, single VLAN, checked licensing and everything else. Wiped back to the default config and started over. I've found that if I connect a 10gb port on the front to a 10gb port of the 6610 it works. The minute I try to use a 40gb qsfp port in the rear (and remove the 10gb port) it will fail. It won't fail by plugging in a qsfp. On the 6650 it shows the port active and state forwarding but on the 6610 it will show it connected and negotiated at 40gb but under state it will show "blocked". Any idea how to get the 6650 and the 6610 chained together (separate switches not a stack) leveraging the 40gb qsfp connection (1/21 or 1/2/6)? Thanks for any help!

Well I ended up resolving it. Somewhere between disabling STP and removing a LAG I know I didn’t create the port state changed and it’s now working. Of course after I ordered another 6610 to get around the issue. Ha!

 

[codyjd1]

Awesome thanks for the reply! I will check it out and hope I can get it all configured when they arrive! Thanks again!

I have received the switches and have them setup. I can confirm that you can stack them with ports 1 and 3 on each switch and still have ports 2 and 4 available as regular data ports.

 

[madmoot]

I should have the switch in my hands in about a week - will check fan voltage then. Just trying to get some replacement cooling lined up to have on hand asap. This thing will be living in my home office about 8 ft away, otherwise I wouldn't really care about the noise level

So just a quick update... Ended up swapping out the single stock 7250 Nidec fan for a Delta EFB0412VHD-F00 and put a Sunon MF60101V2-1000U-A99 on the ASIC (V3s were out of stock on Digikey), which has a slightly higher CFM. Sensor 1 temps settled out at ~65 degs and barely audible about a foot away from me. Very pleased to say the least. FWIW, temps were about 3-4 degs cooler with the ASIC fan blowing down, so i've left it like that for now. Thanks @RoachedCoach and @anomaly for all the guidance.

 

[bpye]

What's going on with FI 10 and the 7150s? Just noticed on the doc for "chassis fanless" - Commscope Technical Content Portal -

10.0.10d	This command was re-introduced for RUCKUS ICX 7150-ES devices.
10.0.10e	This command was re-introduced for ICX 7150 devices.

The release notes still show the ICX 7150 as unsupported - so one of them is wrong.

 

[anomaly]

What's going on with FI 10 and the 7150s? Just noticed on the doc for "chassis fanless" - Commscope Technical Content Portal -

10.0.10d	This command was re-introduced for RUCKUS ICX 7150-ES devices.
10.0.10e	This command was re-introduced for ICX 7150 devices.

The release notes still show the ICX 7150 as unsupported - so one of them is wrong.

Does this mean they re-introduced fanless support for some models? It might be interesting to run a BinDiff/binary diffing tool between those firmware versions, and figure out what they changed. That will give up where the fan logic is located.

 

[bpye]

Well it's less the fact that fanless is re-introduced, and more that it suggests 10.0.10e should run on ICX 7150 switches, and not just the ICX 7150-ES switches.

Of course it could just be a typo.

 

[anomaly]

If you have both firmware packages, I would love to be able to download them

 

[fohdeesha]

If you have both firmware packages, I would love to be able to download them

they're all freely available on ruckus' site, you just need to create a free account

 

[GOSHKU]

Hi,

i bought two used Brocade ICX 6450-48

I used erase command i dont have any license. (or its basic, i dont know, i am not so familliar)

What i need to do to be able to use the 2 or all 4 10G ports?

When i am booting - i have this message in the serial console -
"No license present for port 1/2/2
No license present for port 1/2/4"

Does that mean that i can use 2 or no one 10G SFP+ ports?

Thanks a lot,
Greetings

 

[mshook]

I used erase command i dont have any license. (or its basic, i dont know, i am not so familliar)

What i need to do to be able to use the 2 or all 4 10G ports?

Hi,

everything you need is in the first post of @fohdeesha , namely this link Setup / Config / Licensing Guide v2

 

[ProxmoxProphet]

I bought an ICX7250-48P a few months ago, and, other than some strange IPv6 issues that resolved once I upgraded to 09.0.10 it's been great, but recently I noticed another frustrating issue that I don't know how to fix and I hope someone who encountered this issue can reply with how they fixed it.

I noticed a problem recently with the switch not being able to handle wifi devices roaming between different APs after moving to a new house where I had to buy multiple APs to get connectivity in the whole house (my old house was smaller so I had only 1 AP and therefore never encountered this problem). So here is an explanation of how the problem presents itself: Let's say I am downstairs, connected to my home wifi via the downstairs AP on my phone, I can access the internet just fine, I can access locally-hosted services like nextcloud, jellyfin, etc just fine BUT when I go upstairs to my bedroom, and my phone roams from the downstairs AP to the AP in the upstairs hallway, now the phone can no longer reach anything that is more than one hop away. It can reach other devices in the same VLAN just fine, it can reach other wifi devices, the switch on that VLAN, etc, but it can no longer reach the internet or any locally-hosted services (because the locally-hosted servers are on a different VLAN, and the uplink from the switch to my firewall is also on its own VLAN, so its 2 hops away to those). And if I statically assign the phone a different IP address, then all of a sudden now it can reach the internet and locally-hosted services again (until it roams to a different AP once again and the problem happens again). And this issue is not just with my phone but with any Wi-Fi connected device

Also, another problem I noticed a while ago as well that frustrates me: Sometimes, when a device on the network is assigned an IP that was previously leased to some other device by the DHCP server, then that device cannot reach the internet or anything else thats more than one hop away unless I give it a different IP. I'm not sure but maybe these two issues are related, because the problematic behavior (not being able to reach anything more than 1 hop away) is present in both


Here is a brief description of my network topology in case it would help anyone understand something better:

ISP fiber cable comes into my home and connects to the ONT, then the ONT connects to my opnsense firewall with ethernet. From there, my opnsense firewall connects to a 2.5G/10G managed switch via SFP+ DAC. The 2.5G ports on the switch are used to connect downstairs AP and other things on the user VLAN, and the other SFP+ port on that switch is connected to a singlemode fiber cable which runs to the upstairs hallway closet where my ICX7250 is. This cable carries userVLAN traffic from downstairs to the ICX7250 and the traffic for the transit VLAN between the ICX7250 and Opnsense is also sent down this cable. Then, there are two more APs upstairs connected to the ICX7250 (on the same VLAN as the downstairs APs) as well as some other servers and PCs, a fiber cable that connects to a Mikrotik 10G switch on the third floor, and other things connected to the ICX7250 as well.

Hi, thanks for sharing your experience, it's good to hear that you don't have this problem, but also strange, I'm not sure what is causing this problem for me.

I walked with my phone to downstairs and back (confirming in the TP link app that the phone did actually roam to the downstairs AP and back to the office AP) and then I ran that command, but I didn't see any output about my phone, or any other devices, roaming between APs. The only output was stuff about failed and successful SSH logins, some stuff about STP, and some stuff about the switch disabling PoE on certain ports because a non-PoE devices was connected, but nothing about AP roaming.

I also thought that the 2.5G/10G switch may be responsible for the issues, but I checked the configuration on that and the appropriate VLANs were passed, the ethernet ports on that switch are all untagged for my User VLAN (VLAN 110) and the SFP+ ports are both tagged for the User VLAN and the Transit VLAN.

Also, I also notice this issue not just when roaming between downstairs and second floor APs, but also when roaming between the second floor APs and the AP on the third floor in my office, and the 2.5G/10G switch is on the first floor, so I don't think that 2.5/10G switch would be involved when roaming between second and third floor APs. Also I checked the configuration on the Mikrotik switch on the 3rd floor (which connects to the 3rd floor AP) and the VLANs were configured appropriately on there as well

It seems assigning a different rather than just a static address is what's fixing the issue.

When a client (e.g. 10.10.10.1) roams from an AP to a different AP, and those APs are attached to different ports on the Brocade switch (e.g. ports 1/1/4 and 1/1/5), you should see log messages akin to;

<timestamp>:next hop router 10.10.10.1 moved from port 1/1/4 to port 1/1/5

which is something to look for.

I'd run a traceroute from the client to the internet, and from the expected hop after where traffic is being blocked back to the client, or at least from further upstream if you can't run a traceroute on the hop immediately after where traffic is being blocked.

I have not been able to update on this as I have been away, but I have tried both running a traceroute from affected clients who cant reach the internet, to an IP on the internet, and also a traceroute from the opnsense firewall to the affected devices, and here are the results:

When I run a traceroute from an affected client (a phone) on the User VLAN to something on the internet (1.1.1.1) , the traceroute reaches the switch's IP on the same VLAN as the device, but then the traceroute dies there, it times out at the second hop (which should be the opnsense router's IP on the transit VLAN). When I run a traceroute from the opnsense firewall to the client device's IP, it reaches the switch's IP on the transit VLAN, but then the traffic dies there and does not go to the client device. So it seems the traffic is dying at the switch.

Also, when a device roams between APs, I can see that it's IP is now on a different port using the "show arp ve 150" command but there are no log messages generated when I run the log command.

Any advice on how to resolve this would be greatly appreciated!

 

[fohdeesha]

I have not been able to update on this as I have been away, but I have tried both running a traceroute from affected clients who cant reach the internet, to an IP on the internet, and also a traceroute from the opnsense firewall to the affected devices, and here are the results:

When I run a traceroute from an affected client (a phone) on the User VLAN to something on the internet (1.1.1.1) , the traceroute reaches the switch's IP on the same VLAN as the device, but then the traceroute dies there, it times out at the second hop (which should be the opnsense router's IP on the transit VLAN). When I run a traceroute from the opnsense firewall to the client device's IP, it reaches the switch's IP on the transit VLAN, but then the traffic dies there and does not go to the client device. So it seems the traffic is dying at the switch.

Also, when a device roams between APs, I can see that it's IP is now on a different port using the "show arp ve 150" command but there are no log messages generated when I run the log command.

Any advice on how to resolve this would be greatly appreciated!

Idk how to read so I can't read the history of this issue but have you added a static route on opnsense pointed at your lan with a nexthop of the switches transit vlan ip

 

[ProxmoxProphet]

Idk how to read so I can't read the history of this issue but have you added a static route on opnsense pointed at your lan with a nexthop of the switches transit vlan ip

Yes, I have static routes on my opnsense firewall for every one of my VLANs' subnets, with a nexthop of the switch's transit vlan IP.
And when a device is not experiencing this problem the opnsense can reach stuff in the VLANs/vice versa just fine and the connection works normally, its just intermittently this issue happens where certain devices can't reach the opnsense or anything on the internet (until a different local IP is assigned to the device, which temporarily relieves the problem until the next time it happens)

 

[fohdeesha]

so it's intermittent?? getting a new IP assigned fixing it sounds like something may have a dupe IP somewhere or someone's arp table is ****y. can you/have you posted your icx config

 

[ProxmoxProphet]

so it's intermittent?? getting a new IP assigned fixing it sounds like something may have a dupe IP somewhere or someone's arp table is ****y. can you/have you posted your icx config

Yes, it works fine most of the time, but randomly, a client device will lose internet connectivity, and can reach the switch and anything else on the same VLAN (50), and may or may not be able to reach devices on other VLANs (server VLAN, etc) but cannot reach the opnsense on the transit VLAN, and therefore can't reach the internet. And statically assigning the device a different IP, or using the DHCP server to assign it a different IP, temporarily alleviates the problem (until it occurs again). And I have checked the DHCP server lease assignments and it is not assigning duplicate IPs
By the way, the User VLAN and Transit VLAN are in two different VRFs (User VLAN is in User-VRF and Transit VLAN in default VRF) not sure if this is relevant. (but as you can see in my config I have the relevant inter-vrf routes set up)

Edit: I just did a test with my opnsense firewall and one of the devices that is currently being affected. I tried to ping the opnsense's IP on the transit VLAN from the affected device, and it timed out on there, but at the same time I was running tcpdump on the transit interface on the opnsense, and I saw that the traffic from the affected device was indeed reaching the opnsense firewall, and the opnsense was even replying to the pings. When I tried to ping something on the internet from that device, I could see the ICMP packets going out and even the replies from the remote IP I was pinging on the tcpdump, but on the device it was all timing out and it never gets those replies to that traffic. So this indicates to me there is no problems with the opnsense and that the traffic is dying at the switch for some reason.

Also, I noticed that in the event that some device is having this issue, and I alleviate it by assigning that device a different LAN IP on the DHCP server, and then if, some while later, some other device connecting to the network is leased out that same "bad IP" that that affected device had before, then internet access will also not work for that device either, unless I give it a different IP. Not sure what this means or if it's significant

I will paste my config below

Spoiler

SSH@ICX7250-48P Router>show run
Current configuration:
!
ver 09.0.10j_cd2T213
!
stack unit 1
module 1 icx7250-48p-poe-port-management-module
module 2 icx7250-sfp-plus-8port-80g-module
!
!
!
global-stp
!
!
vlan 1 name DEFAULT-VLAN by port
!
vlan 33 name PVEBYP by port
tagged ethe 1/2/1
untagged ethe 1/1/11
!
vlan 50 name VL50-Users by port
tagged ethe 1/2/1 to 1/2/2
untagged ethe 1/1/1 ethe 1/1/25 to 1/1/36
!
vlan 77 name Transit by port
untagged ethe 1/2/1
!
vlan 99 name Transit2 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2

!

vlan 100 name VL100 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2

untagged ethe 1/1/7
!
vlan 198 name DMZ5 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2

!
vlan 199 name DMZ1-VL by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2

untagged ethe 1/1/39
!
vlan 200 name VL200 by port
tagged ethe 1/1/1 ethe 1/2/2
untagged ethe 1/1/2 to 1/1/6
!
vlan 399 name DMZ3 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
untagged ethe 1/1/37 to 1/1/38
!
vlan 500 name Voice-VL by port

tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2
untagged ethe 1/1/13
!

vlan 876 name VPN1 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/1/25 ethe 1/2/2
!
vlan 1551 name MGMT-VL by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/1 to 1/2/2
!
vlan 3227 by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/2/2

!
!
!

!

system-max ip-static-route 2048
system-max ip-route-default-vrf 9000
system-max ip6-route-default-vrf 256
system-max ip-route-vrf 500
!
!
!

vrf USER-VRF

rd 693:50
ip router-id 172.31.1.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 172.25.100.1

ip route 10.5.0.0/23 172.31.0.1
ip route 10.5.2.0/24 next-hop-vrf SERVER-VRF 172.30.10.1
ip route 10.5.3.0/24 172.31.7.1
ip route 10.55.0.0/24 ve 3227 tag 50
ip route 10.64.0.0/24 172.31.3.1
ip route 10.82.0.0/23 next-hop-vrf SERVER-VRF 172.30.10.82

ip route 10.82.10.0/24 next-hop-vrf default-vrf 172.25.100.1
ip route 10.124.0.0/16 172.31.3.1

ip route 10.200.0.0/22 ve 199 tag 50
ip route 10.250.0.0/24 ve 1551 tag 50

ip route 172.17.0.0/16 172.31.0.2
--More--, next page: Space, next line: Return key, quit: Control-c
ip route 172.20.0.0/22 172.31.0.2

ip route 172.21.0.0/22 172.31.7.1
ip route 172.23.0.0/22 next-hop-vrf SERVER-VRF 172.30.10.1

ip route 172.25.100.0/29 ve 77 tag 50
ip route 172.27.0.0/24 ve 99 tag 50

ip route 172.30.8.0/21 ve 100 tag 50
!
vrf SERVER-VRF

rd 693:100
ip router-id 172.30.9.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 172.25.100.1

ip route 10.5.0.0/22 172.30.10.1
ip route 10.36.8.0/22 ve 399 tag 100
ip route 10.64.0.0/24 172.30.13.100

ip route 10.82.10.0/24 next-hop-vrf default-vrf 172.25.100.1
ip route 10.124.0.0/16 172.30.13.100

ip route 10.250.0.0/24 ve 1551 tag 100
ip route 11.200.32.0/21 ve 200

ip route 172.20.50.0/24 172.30.15.1
ip route 172.25.100.0/29 ve 77 tag 100
ip route 172.27.0.0/24 ve 99 tag 100

ip route 172.31.0.0/21 ve 50 tag 100
ip route 172.31.31.0/24 ve 876 tag 100
ip route 198.19.2.0/24 172.30.12.1
ip route 198.19.3.0/24 172.30.10.1

ip route 198.19.4.0/24 172.30.14.1
address-family ipv6
ipv6 route ::/0 ve 100 fe80::44:e6ff:fe77:76bb

!

vrf DMZ-VRF
rd 693:999
ip router-id 10.99.0.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 172.27.0.40

ip route 172.27.0.0/24 ve 99 tag 199
address-family ipv6


ipv6 route ::/0 ve 199 fe80::c5:34ff:fefd:5767
!

vrf SECURE-VRF
rd 693:732

ip router-id 10.55.0.254
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 172.25.100.1
ip route 10.82.0.0/23 next-hop-vrf SERVER-VRF 172.30.10.82
ip route 172.17.17.0/24 next-hop-vrf SERVER-VRF 172.30.10.82

ip route 172.17.50.0/23 next-hop-vrf SERVER-VRF 172.30.10.82
ip route 172.20.0.0/22 next-hop-vrf SERVER-VRF 172.30.12.1
ip route 172.21.0.0/22 next-hop-vrf SERVER-VRF 172.30.14.1

ip route 172.23.0.0/22 next-hop-vrf SERVER-VRF 172.30.10.1
--More--, next page: Space, next line: Return key, quit: Control-c
ip route 172.25.100.0/29 ve 77 tag 3227
ip route 172.27.0.0/24 ve 99 tag 3227
--More--, next page: Space, next line: Return key, quit: Control-c
ip route 172.30.8.0/21 ve 100 tag 3227
ip route 172.31.6.200/29 ve 50 tag 3227
!
vrf SERVER2-VRF

rd 693:200
ip router-id 172.29.20.1

address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 172.27.0.55

ip route 10.82.0.0/23 next-hop-vrf SERVER-VRF 172.30.10.82
ip route 10.82.10.0/24 next-hop-vrf default-vrf 172.25.100.1
ip route 172.30.8.0/21 ve 100 tag 200

!
vrf VPN-VRF
rd 693:876

ip router-id 172.31.31.1
address-family ipv4
ip route 0.0.0.0/0 next-hop-vrf default-vrf 172.27.0.4

ip route 172.27.0.0/24 ve 99 tag 876
ip route 172.30.8.0/21 ve 100 tag 876

address-family ipv6
ipv6 route 2001:db8:a:999::/64 ve 99
!
ip route 0.0.0.0/0 172.25.100.1


ip route 10.5.0.0/22 next-hop-vrf SERVER-VRF 172.30.10.1
ip route 10.36.8.0/22 ve 399

ip route 10.55.0.0/24 ve 3227
ip route 10.82.0.0/23 next-hop-vrf SERVER-VRF 172.30.10.82
ip route 10.82.10.0/24 172.25.100.1
ip route 10.200.0.0/22 ve 199

ip route 10.250.0.0/24 ve 1551
ip route 11.200.32.0/21 ve 200
ip route 172.30.8.0/21 ve 100

ip route 172.31.0.0/21 ve 50
ip route 172.31.31.0/24 ve 876

ip route 198.19.2.0/24 next-hop-vrf SERVER-VRF 172.30.12.1
ip route 198.19.3.0/24 next-hop-vrf SERVER-VRF 172.30.10.1
ip route 198.19.4.0/24 next-hop-vrf SERVER-VRF 172.30.14.1
optical-monitor

optical-monitor non-ruckus-optic-enable
ipv6 route ::/0 ve 99 fe80::be24:11ff:fefd:fdea
ipv6 route 2001:db8:a:1::/60 ve 876

!

!
!
clock summer-time
clock timezone gmt GMT-05
ip tftp blocksize 8192


ipv6 unicast-routing
!

!
ntp

disable serve
server 162.159.200.1
--More--, next page: Space, next line: Return key, quit: Control-c
server 162.159.200.123
!
!
!
!
!
--More--, next page: Space, next line: Return key, quit: Control-c
!

!

!
!
--More--, next page: Space, next line: Return key, quit: Control-c



interface management 1

ip address 10.166.72.50 255.255.255.0

no ip dhcp-client enable

!
interface ethernet 1/2/1

no optical-monitor
!

interface ethernet 1/2/2
no optical-monitor

!
interface ve 50

vrf forwarding USER-VRF

ip address 172.31.1.1 255.255.248.0
--More--, next page: Space, next line: Return key, quit: Control-c
ipv6 nd suppress-ra
!

interface ve 77
ip address 172.25.100.2 255.255.255.248

!

interface ve 99
ip address 172.27.0.1 255.255.255.0

ipv6 address fe80::d0c0:7ff:fe2a:5b9f link-local
ipv6 address 2001:db8:a:999::2/64
ipv6 enable

ipv6 nd suppress-ra
!

interface ve 100
--More--, next page: Space, next line: Return key, quit: Control-c
vrf forwarding SERVER-VRF

ip address 172.30.9.1 255.255.248.0

--More--, next page: Space, next line: Return key, quit: Control-c
ipv6 address fe80::215:5dff:fe68:9a3f link-local

ipv6 address 2001:db8:a:1ff::/56

ipv6 address 2001:db8:a:114::/59
ipv6 address 2001:db8:a:511::/59
ipv6 address f001:db8:a:ff::/56
ipv6 enable
--More--, next page: Space, next line: Return key, quit: Control-c
ipv6 nd suppress-ra

!

interface ve 198

vrf forwarding DMZ-VRF
ip address 100.101.98.1 255.255.254.0


ipv6 nd suppress-ra
!

interface ve 199

vrf forwarding DMZ-VRF

ip address 10.200.1.1 255.255.252.0

ipv6 address fe80::91a6:8150:2195:1912 link-local

ipv6 enable

ipv6 nd suppress-ra
!

--More--, next page: Space, next line: Return key, quit: Control-c
interface ve 200
vrf forwarding SERVER2-VRF

ip address 11.200.32.1 255.255.248.0

ipv6 address fe80::21e:c9ff:fe48:6e8c link-local

ipv6 nd suppress-ra

!

interface ve 399
vrf forwarding DMZ-VRF

ip address 10.36.9.1 255.255.252.0

ipv6 address fe80::3f10:6fca:aa3a:2a60 link-local

ipv6 enable
ipv6 nd suppress-ra

!

interface ve 500

ip address 10.80.0.1 255.255.255.0

ip helper-address 1 172.27.0.53
ipv6 nd suppress-ra

!
interface ve 876

vrf forwarding VPN-VRF
ip address 172.31.31.1 255.255.255.0

ipv6 address fe80::8ae3:7ff:fe94:1a2b link-local
ipv6 address 2001:db8:a:444::1/60
--More--, next page: Space, next line: Return key, quit: Control-c
ipv6 enable
!
interface ve 1551
vrf forwarding SECURE-VRF
ip address 10.250.0.2 255.255.255.0
--More--, next page: Space, next line: Return key, quit: Control-c
ipv6 nd suppress-ra
!
interface ve 3227

vrf forwarding SECURE-VRF
--More--, next page: Space, next line: Return key, quit: Control-c
ip address 10.55.0.254 255.255.255.0
ipv6 nd suppress-ra
!
!
--More--, next page: Space, next line: Return key, quit: Control-c



!
!






!

!
--More--, next page: Space, next line: Return key, quit: Control-c

!

!


username super password .....
username management password .....

!
aaa authentication login default local

!
aaa authentication web-server default local

!
aaa authentication snmp-server default local
!



!




!
--More--, next page: Space, next line: Return key, quit: Control-c



!
no telnet server

!
--More--, next page: Space, next line: Return key, quit: Control-c
manager disable
manager port-list 987

!
--More--, next page: Space, next line: Return key, quit: Control-c



!



!


end

 

[dbvader]

So this indicates to me there is no problems with the opnsense and that the traffic is dying at the switch for some reason.

Does the 'bad' ip ever become good (usable) again? I can imagine the bad ip becoming good when the switch is reset? Does it become good when the opnsense is reset?

Seems like figuring out what (which device reset) allows a bad ip to become good would be a lead to identifying the cause. Feels like it's going to be the switch but who knows. I guess nothing is showing up in the switch logs?

 

[adam.n]

I tray to run 40GbE port with tag but it looks like switch can transmit 802.1q packets on 40GbE link but can't receive them.

My config

ver 08.0.30uT7f3
!
stack unit 1
module 1 icx6610-48p-poe-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module

stack disable

vlan 100 name remote-access by port
tagged ethe 1/2/1 ethe 1/3/1 to 1/3/3
router-interface ve 100

interface ve 100
ip address 169.254.170.2 255.255.255.0

ethe 1/2/1 - 40 GibE link to Host1 with address 169.254.170.1
ethe 1/3/1 - 10 GibE link to Host2 with address 169.254.170.3

I can ping from Host2 169.254.170.3 to switch 169.254.170.2 via ethe 1/3/1

I can't ping:

1. from Host1 169.254.170.1 to switch 169.254.170.2 via ethe 1/2/1
2. from Host2 169.254.170.3 to Host1 169.254.170.1 via ethe 1/3/1 and next ethe 1/2/1

When I run tcpdump on Host1 40GbE there are 802.1q packet

15:31:19.780972 ae:71:ab:91:59:c9 > 3c:fd:fe:a4:d4:59, ethertype 802.1Q (0x8100), length 64: vlan 100, p 0, ethertype ARP (0x0806), Reply 169.254.170.3 is-at ae:71:ab:91:59:c9, length 46

15:31:19.780979 3c:fd:fe:a4:d4:59 > ae:71:ab:91:59:c9, ethertype 802.1Q (0x8100), length 102: vlan 100, p 0, ethertype IPv4 (0x0800), 169.254.170.1 > 169.254.170.3: ICMP echo reply, id 10, seq 1, length 64

Switch also has mac-address table with mac on vlan 100 and 40GibE link

#show mac-address vlan 100
Total active entries from VLAN 100 = 4
MAC-Address Port Type Index VLAN
ae71.ab91.59c9 1/3/1 Dynamic 3452 100
3cfd.fea4.d459 1/2/1 Dynamic 57524 100
a4bf.0114.49af 1/3/1 Dynamic 31376 100
0021.5676.d981 1/3/1 Dynamic 24464 100

I tried to reboot switch, didn't help.

Any one with here with working 40GibE link and 802.1q?

 

[hmw]

Has anyone had problems when using GBase-T transceivers on the ICX-7xxx series and running them at 100Mbps? Want to get an all fiber switch but still be able to use my iDRAC ports - which are 100 mbit copper. Worried that some of the ports wont support 100mbit even with a gbase-t transceiver

 

[jdmag00]

Has anyone else had issues with the ICX7250 and 10g BaseT? I am using Mikrotik 10g baseT SFP+ modules that worked fine in an ICX6610, but the 7250 doesn't seem to recognize them? I attempted to force the speed to 10G but that didn't seem to help.

 

[robsdesk]

Has anyone else had issues with the ICX7250 and 10g BaseT? I am using Mikrotik 10g baseT SFP+ modules that worked fine in an ICX6610, but the 7250 doesn't seem to recognize them? I attempted to force the speed to 10G but that didn't seem to help.

I've got a couple in my 7250, a Broadcom based 10gtek ASF-10G-T80 which is doing 10G over cat5e (very well indeed as it happens) & a noname AliExpress 30m one (which claims no 1G support but has negotiated just fine at that speed to the ONT from our ISP).

I didn't do anything special to make them work other than specify the link speed.