Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[BoGs]

> Postscript: All this after i got frustrated and ordered a 6610-24P and a couple of bidi optics to just stack everything. I'm probably going to be more happy with that anyway. It's only money right??

6610s eat a lot of power and are loud as long as that is ok you will love it. I am debating if I am going to replace my 7250 with 7150-48zp just too loud.

 

[86turbodsl]

The 6610 in the basement is livable, i don't spend much time down there. The other one is in a noisy workshop environment and i could care less. The power is something i will live with probably. Having both switches in a single management environment is very appealing. The backup for my system will be out there also, so having 40Gb connection is also appealing.

 

[scottshotgg]

Has anyone ever used a RADIUS server to do MAC auth on their Brocade switch? I have an ICX7250-48P and am trying to get MAC auth working, however for some reason I seem to have what I can only call "client VLAN flip-flop" where the switch continuously calls my RADIUS server and the client flip-flops back and forth from the VLAN the RADIUS server sent back to the default VLAN set. Its very odd and I can provide some more context if anyone has any ideas that would be great

 

[Cheburashka]

Hello everyone, thank you for this fantastic thread.

I've been exploring getting 10GB connectivity between my QNAP and desktop. I purchased the ICX7150-C12 and I have Mellanox ConnectX3 cards in my desktop and QNAP. I am unsure where in the Brocade/Ruckus is the ability to set a larger MTU than 1500. I have browsed through the web gui but I haven't been able to find a spot for it.

I've tried to find this answer in many posts here but haven't found much information on it.


Edit: I was checking the CLI mode and I entered interface 1/3/1 (10gb). I checked MTU and Jumbo and was able to enable Jumbo but it looks like it is switch wide?

Doing a show run, I see jumbo listed but it is not assigned to a port or vlan.

 

[kpfleming]

Edit: I was checking the CLI mode and I entered interface 1/3/1 (10gb). I checked MTU and Jumbo and was able to enable Jumbo but it looks like it is switch wide?

Yes, that is correct. It's a switch-level configuration setting. It has to be, because otherwise a port with 'jumbo' enabled would be unable to practically communicate with a port that had 'jumbo' disabled. In fact that's still an issue even with the switch-level setting, because if any node on the LAN isn't configured for the same MTU as the other nodes, there will be communication problems.

In other words: either all of the nodes must have the increased MTU, or none of them do. Making effective use of jumbo frames can be tricky.

 

[fohdeesha]

Hello everyone, thank you for this fantastic thread.

I've been exploring getting 10GB connectivity between my QNAP and desktop. I purchased the ICX7150-C12 and I have Mellanox ConnectX3 cards in my desktop and QNAP. I am unsure where in the Brocade/Ruckus is the ability to set a larger MTU than 1500. I have browsed through the web gui but I haven't been able to find a spot for it.

I've tried to find this answer in many posts here but haven't found much information on it.


Edit: I was checking the CLI mode and I entered interface 1/3/1 (10gb). I checked MTU and Jumbo and was able to enable Jumbo but it looks like it is switch wide?

Doing a show run, I see jumbo listed but it is not assigned to a port or vlan.

Why do you want to run jumbo frames in 2024? They're almost entirely unnecessary for 10gb, have you done any throughput testing etc showing that you need them for some reason?

 

[BBergle]

Hello! I just got a 7250-24P from ebay and after I connect to the console I only get these characters. ����
I have hit b and enter multiple times but nothing shows up but those question mark characters. I know my cable is good since I have another switch I just tested it on and it works fine. I reset the switch to factory settings and nothing has changed. Is the switch fried? Thank you

 

[fohdeesha]

Hello! I just got a 7250-24P from ebay and after I connect to the console I only get these characters. ����
I have hit b and enter multiple times but nothing shows up but those question mark characters. I know my cable is good since I have another switch I just tested it on and it works fine. I reset the switch to factory settings and nothing has changed. Is the switch fried? Thank you

are you using 9600 - 8 - n - 1 ? you could try a different baudrate on the offchance the last user changed it n the bootloader, 19200, 38400, 115200

 

[BBergle]

are you using 9600 - 8 - n - 1 ? you could try a different baudrate on the offchance the last user changed it n the bootloader, 19200, 38400, 115200

Yes I have tried 9600 and 11520

 

[fohdeesha]

Yes I have tried 9600 and 11520

did you try 115200 ?

 

[BBergle]

did you try 115200 ?

Sorry, yes that's what I meant

 

[BackupProphet]

Sometimes the issue is bad serial cable. Some of the Arista and Celestica switches I bought came with their own serial cables, and they work much better than old one I had.

 

[klui]

Then you have the situation with Mellanox. ESPCommunity

 

[ramicio]

Can anyone please help me with a new setup? I previously used a UDM-Pro and it did all of my routing. I am going with an OPNSense router. I have a bunch of VLANs on the switch, and I want the switch to do the routing. It could be a lot of traffic from any of those VLANs to my main one. I am testing VLAN 112, with an access point plugged into it. Connecting to it I am able to pull an IP address. I am unable to access the internet. I am unable to find any information on how to get.

Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
  module 1 icx6610-24p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
  stack-trunk 1/2/6 to 1/2/7
  stack-port 1/2/6
stack unit 2
  module 1 icx6610-24f-sf-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
  stack-trunk 2/2/1 to 2/2/2
  stack-trunk 2/2/6 to 2/2/7
  stack-port 2/2/1 2/2/6
stack enable
stack mac 748e.f8fe.93f6
!
global-stp
!
!
!
vlan 1 name DEFAULT-VLAN by port
 router-interface ve 1
!
vlan 101 name Staab by port
 tagged ethe 1/3/8 ethe 2/1/1
!
vlan 102 name Jones by port
 tagged ethe 1/3/8 ethe 2/1/2
!
vlan 103 name Wagner by port
 tagged ethe 1/3/8 ethe 2/1/3
!
vlan 104 name Beamesderfer by port
 tagged ethe 1/3/8 ethe 2/1/4
!
vlan 105 name Sowers by port
 tagged ethe 1/3/8 ethe 2/1/5
!
vlan 106 name Speece by port
 tagged ethe 1/3/8 ethe 2/1/6
!
vlan 107 name SoLeb-Broseph by port
 tagged ethe 1/3/8 ethe 2/1/7
!
vlan 108 name DuPont-Harding by port
 tagged ethe 1/3/8 ethe 2/1/8
!
vlan 109 name Gromis by port
 tagged ethe 1/3/8 ethe 2/1/9
!
vlan 110 name Swanger by port
 tagged ethe 1/3/8 ethe 2/1/10
!
vlan 111 name Hackman by port
 tagged ethe 1/3/8 ethe 2/1/11
!
vlan 112 name Miranda by port
 tagged ethe 1/3/8 ethe 2/1/12
 router-interface ve 112
!
!
!
!
!
aaa authentication web-server default local
aaa authentication login default local
hostname switcheroo
ip dhcp-client disable
ip dhcp-server enable
!
ip dhcp-server pool miranda
 dhcp-default-router 192.168.112.1
 dns-server 192.168.112.1
 excluded-address 192.168.112.1
 lease 1 0 0
 network 192.168.112.0 255.255.255.0
 deploy
!
!
no telnet server
username root password .....
!
!
hitless-failover enable
!
!
!
!
!
!
!
interface ethernet 1/1/3
 inline power
!
interface ethernet 1/1/6
 inline power
!
interface ethernet 1/1/7
 inline power
!
interface ethernet 1/1/13
 inline power
!
interface ethernet 1/3/1
 port-name Indoors
 speed-duplex 10G-full
!
interface ethernet 1/3/2
 speed-duplex 10G-full
!
interface ethernet 1/3/3
 speed-duplex 10G-full
!
interface ethernet 1/3/4
 speed-duplex 10G-full
!
interface ethernet 1/3/7
 speed-duplex 10G-full
!
interface ethernet 1/3/8
 dual-mode
 speed-duplex 10G-full
!
interface ethernet 2/1/1
 port-name Staab
 dual-mode
!
interface ethernet 2/1/2
 port-name Jones
 dual-mode
!
interface ethernet 2/1/3
 port-name Wagner
 dual-mode
!
interface ethernet 2/1/4
 port-name Beamesderfer
 dual-mode
!
interface ethernet 2/1/5
 port-name Sowers
 dual-mode
!
interface ethernet 2/1/6
 port-name Speece
 dual-mode
!
interface ethernet 2/1/7
 port-name SoLeb-Berger
 dual-mode
!
interface ethernet 2/1/8
 port-name DuPont-Harding
 dual-mode
!
interface ethernet 2/1/9
 port-name Gromis
 dual-mode
!
interface ethernet 2/1/10
 port-name Swanger
 dual-mode
!
interface ethernet 2/1/11
 port-name Hackman
 dual-mode
!
interface ethernet 2/1/12
 port-name Miranda
 dual-mode
!
interface ve 1
 ip address 192.168.1.5 255.255.255.0
!
interface ve 112
 ip address 192.168.112.1 255.255.255.0
!
!
!
!
!
lldp tagged-packets process
lldp run
!
!
!
!
end

 

[cyinite]

Can anyone please help me with a new setup? I previously used a UDM-Pro and it did all of my routing. I am going with an OPNSense router. I have a bunch of VLANs on the switch, and I want the switch to do the routing. It could be a lot of traffic from any of those VLANs to my main one. I am testing VLAN 112, with an access point plugged into it. Connecting to it I am able to pull an IP address. I am unable to access the internet. I am unable to find any information on how to get.

Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
  module 1 icx6610-24p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
  stack-trunk 1/2/6 to 1/2/7
  stack-port 1/2/6
stack unit 2
  module 1 icx6610-24f-sf-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
  stack-trunk 2/2/1 to 2/2/2
  stack-trunk 2/2/6 to 2/2/7
  stack-port 2/2/1 2/2/6
stack enable
stack mac 748e.f8fe.93f6
!
global-stp
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 101 name Staab by port
tagged ethe 1/3/8 ethe 2/1/1
!
vlan 102 name Jones by port
tagged ethe 1/3/8 ethe 2/1/2
!
vlan 103 name Wagner by port
tagged ethe 1/3/8 ethe 2/1/3
!
vlan 104 name Beamesderfer by port
tagged ethe 1/3/8 ethe 2/1/4
!
vlan 105 name Sowers by port
tagged ethe 1/3/8 ethe 2/1/5
!
vlan 106 name Speece by port
tagged ethe 1/3/8 ethe 2/1/6
!
vlan 107 name SoLeb-Broseph by port
tagged ethe 1/3/8 ethe 2/1/7
!
vlan 108 name DuPont-Harding by port
tagged ethe 1/3/8 ethe 2/1/8
!
vlan 109 name Gromis by port
tagged ethe 1/3/8 ethe 2/1/9
!
vlan 110 name Swanger by port
tagged ethe 1/3/8 ethe 2/1/10
!
vlan 111 name Hackman by port
tagged ethe 1/3/8 ethe 2/1/11
!
vlan 112 name Miranda by port
tagged ethe 1/3/8 ethe 2/1/12
router-interface ve 112
!
!
!
!
!
aaa authentication web-server default local
aaa authentication login default local
hostname switcheroo
ip dhcp-client disable
ip dhcp-server enable
!
ip dhcp-server pool miranda
dhcp-default-router 192.168.112.1
dns-server 192.168.112.1
excluded-address 192.168.112.1
lease 1 0 0
network 192.168.112.0 255.255.255.0
deploy
!
!
no telnet server
username root password .....
!
!
hitless-failover enable
!
!
!
!
!
!
!
interface ethernet 1/1/3
inline power
!
interface ethernet 1/1/6
inline power
!
interface ethernet 1/1/7
inline power
!
interface ethernet 1/1/13
inline power
!
interface ethernet 1/3/1
port-name Indoors
speed-duplex 10G-full
!
interface ethernet 1/3/2
speed-duplex 10G-full
!
interface ethernet 1/3/3
speed-duplex 10G-full
!
interface ethernet 1/3/4
speed-duplex 10G-full
!
interface ethernet 1/3/7
speed-duplex 10G-full
!
interface ethernet 1/3/8
dual-mode
speed-duplex 10G-full
!
interface ethernet 2/1/1
port-name Staab
dual-mode
!
interface ethernet 2/1/2
port-name Jones
dual-mode
!
interface ethernet 2/1/3
port-name Wagner
dual-mode
!
interface ethernet 2/1/4
port-name Beamesderfer
dual-mode
!
interface ethernet 2/1/5
port-name Sowers
dual-mode
!
interface ethernet 2/1/6
port-name Speece
dual-mode
!
interface ethernet 2/1/7
port-name SoLeb-Berger
dual-mode
!
interface ethernet 2/1/8
port-name DuPont-Harding
dual-mode
!
interface ethernet 2/1/9
port-name Gromis
dual-mode
!
interface ethernet 2/1/10
port-name Swanger
dual-mode
!
interface ethernet 2/1/11
port-name Hackman
dual-mode
!
interface ethernet 2/1/12
port-name Miranda
dual-mode
!
interface ve 1
ip address 192.168.1.5 255.255.255.0
!
interface ve 112
ip address 192.168.112.1 255.255.255.0
!
!
!
!
!
lldp tagged-packets process
lldp run
!
!
!
!
end

Welcome from Ubiquiti, their networking equipment definitely is as easy as it gets but congrats for making the leap. Looking around your running config, I noticed you might be missing everything you need. From my experience, I needed to run

1. ip route 0.0.0.0/0 [opnsense ip] on the switch
2. add the switches IP as a gateway in OPNsense under System > Gateways > Configuration
3. add the VLAN subnets to route through the switch under System > Routes > Configuration
4. finally add a outbound rule under Firewall > NAT > Outbound so that OPNsense NATs the VLANs

You are right about this being hard to find info for, I was only able to do this once I learned about it studying for my CCNA.

 

[ramicio]

Welcome from Ubiquiti, their networking equipment definitely is as easy as it gets but congrats for making the leap. Looking around your running config, I noticed you might be missing everything you need. From my experience, I needed to run

1. ip route 0.0.0.0/0 [opnsense ip] on the switch
2. add the switches IP as a gateway in OPNsense under System > Gateways > Configuration
3. add the VLAN subnets to route through the switch under System > Routes > Configuration
4. finally add a outbound rule under Firewall > NAT > Outbound so that OPNsense NATs the VLANs

You are right about this being hard to find info for, I was only able to do this once I learned about it studying for my CCNA.

Thank you. I will have to test it at a bit later point in time. Currently I have some pieces of hardware in use for keeping my UDM Pro connected to my modem because I am still using the Protect portion of it (pieces I was using before to test this Brocade thing). Until I get a UNVR here... That will be the only thing left here from Ubiquiti.

I'm not really understanding what any of that means after item #1.

Thank you.

 

[cyinite]

Thank you. I will have to test it at a bit later point in time. Currently I have some pieces of hardware in use for keeping my UDM Pro connected to my modem because I am still using the Protect portion of it (pieces I was using before to test this Brocade thing). Until I get a UNVR here... That will be the only thing left here from Ubiquiti.

I'm not really understanding what any of that means after item #1.

Thank you.

No worries. Looking back when I first started learning, fundamentals like routing were pretty intimating. Here's some resources to help with configuration:

• Gateways — OPNsense documentation
• Routes — OPNsense documentation
• Network Address Translation — OPNsense documentation

Best of luck to you!

 

[ramicio]

No worries. Looking back when I first started learning, fundamentals like routing were pretty intimating. Here's some resources to help with configuration:

• Gateways — OPNsense documentation
• Routes — OPNsense documentation
• Network Address Translation — OPNsense documentation

Best of luck to you!

I was watching those videos yesterday and the first VLAN one seems like what I already had been doing. The OPNSense stuff still makes zero sense to me. I am the type of person who needs to find someone who is doing the literal same thing as I am, and copying/pasting and inserting my own addresses. I am doing this for a few reasons. First, Ubiquiti and their insane security stuff...I don't need to reach this stuff outside of my network. So the whole using my UI account to get into my own local stuff, and needing to do 2FA for it is insane, along with it all needing to be encrypted. I can do plain HTTP with all of this new stuff so far, Omada being what I am now using for access points. My goal was to offload routing to the switch in case of local Plex traffic. I'd rather it not have to travel to the internet router and back into the switch, when the switch could do it. Literally the only traffic. My internet is 2.5 gigabits, so that leaves 7.5 gigabits for all that other traffic, so in reality, it would never get congested. With the Ubiquiti stuff, people were still seeing other's TVs broadcasting for being able to cast videos to them. This is at a campground where I live, where a few people get internet access from me, versus the sparse access points outdoors that the camp itself has set up. I'd preferably just pay someone to set this up for me, but have never gotten anyone interested in some side cash (I don't get it, seems like an easy gig, not setting up a whole company's infrastructure).

 

[kpfleming]

I'd preferably just pay someone to set this up for me, but have never gotten anyone interested in some side cash (I don't get it, seems like an easy gig, not setting up a whole company's infrastructure).

Speaking personally, I avoid doing this because it's not a one-time interaction; once it has been setup, any time there's an issue, or there's a need for a security patch to be applied, or something wonky happens with the upstream link(s), the person who did the 'side gig' is going to get a call.

 

[ramicio]

Speaking personally, I avoid doing this because it's not a one-time interaction; once it has been setup, any time there's an issue, or there's a need for a security patch to be applied, or something wonky happens with the upstream link(s), the person who did the 'side gig' is going to get a call.

That makes zero sense to me. Setting it up properly either works or it doesn't work. The person who did the side gig would be getting paid again if I needed more help anyway. Regardless, I guess I am going to give up. And I am probably going to just go back to the UDM Pro. I restarted this OPNSense box and one of the interfaces of the dual-port NIC just totally disappeared from existence. That, and I can't get any help with setting this up.