Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

BoGs

Active Member
Feb 18, 2019
165
39
28
If you are willing to give up that easily unfortunately this was not going to work out, there is no nice UI that you can use for these older switches. You will need to change some settings unplug the SFP plugs from your other switch and test it out. Unless you work in that field which most of us do not. It took me 2 months to switch from OPNSense to Mikrotik router as I wanted to route more then 2.5G and instead of building a new server I went for my largest link SFP+.

The reason why people are saying not to do side gig is so that you can learn, and ultimately success and being proud of what you achieve.

What you want to do out of the gate is more advanced then most, what you are looking for is to setup a transit VLAN between your switch and router, your switch becomes the router by setting up routable interfaces and you create a route on the switch for other vlan subnets to pass it to OPNSense and on the OPNSense to pass it to the switch. You will also have to pass the DHCP relay to wherever your server is.

I used this to read up on intervlan routing (InterVLAN Routing) or this (Inter-VLAN Routing Configuration - %%currentyear%% Step-by-Step Tutorial) and while I think its for Cisco you can lookup the keyword commands in the brocade command reference sheet. Keeping router on a stick is probably the easiest option as you can setup your OPNSense vlans on top of the interface and away you go with trunk ports. I would do that FIRST then potentially experimenting with _new_ vlans on different ports so you do not break the internet.

Good luck
 

ramicio

Member
Nov 30, 2022
78
14
8
If you are willing to give up that easily unfortunately this was not going to work out, there is no nice UI that you can use for these older switches. You will need to change some settings unplug the SFP plugs from your other switch and test it out. Unless you work in that field which most of us do not. It took me 2 months to switch from OPNSense to Mikrotik router as I wanted to route more then 2.5G and instead of building a new server I went for my largest link SFP+.

The reason why people are saying not to do side gig is so that you can learn, and ultimately success and being proud of what you achieve.

What you want to do out of the gate is more advanced then most, what you are looking for is to setup a transit VLAN between your switch and router, your switch becomes the router by setting up routable interfaces and you create a route on the switch for other vlan subnets to pass it to OPNSense and on the OPNSense to pass it to the switch. You will also have to pass the DHCP relay to wherever your server is.

I used this to read up on intervlan routing (InterVLAN Routing) or this (Inter-VLAN Routing Configuration - %%currentyear%% Step-by-Step Tutorial) and while I think its for Cisco you can lookup the keyword commands in the brocade command reference sheet. Keeping router on a stick is probably the easiest option as you can setup your OPNSense vlans on top of the interface and away you go with trunk ports. I would do that FIRST then potentially experimenting with _new_ vlans on different ports so you do not break the internet.

Good luck
I cannot learn how to do this unless someone literally shows me how to do this. Documents and videos are of no help to me. I don't understand the stuff. I only understand by seeing how it is done. I cannot experiment with more ports. My OPNSense router is 1u and has only a dual-port SFP+ NIC. Impossible.
 
Sep 22, 2015
68
33
18
I cannot learn how to do this unless someone literally shows me how to do this. Documents and videos are of no help to me. I don't understand the stuff. I only understand by seeing how it is done. I cannot experiment with more ports. My OPNSense router is 1u and has only a dual-port SFP+ NIC. Impossible.
That's why no one wants to configure your stuff as a "side gig." A) The knowledge is fairly esoteric and B) people who refuse to learn and just want someone else to do it all for them or feed them step by step instructions usually end being combative, demanding, and want far more support.

Sorry, my dude, but nothing in your tone sounds like you'd be an exception to this. My spider-sense from back when I did freelance IT is tingling. Good luck, I hope you can find someone to do it for you.
 

ramicio

Member
Nov 30, 2022
78
14
8
That's why no one wants to configure your stuff as a "side gig." A) The knowledge is fairly esoteric and B) people who refuse to learn and just want someone else to do it all for them or feed them step by step instructions usually end being combative, demanding, and want far more support.

Sorry, my dude, but nothing in your tone sounds like you'd be an exception to this. My spider-sense from back when I did freelance IT is tingling. Good luck, I hope you can find someone to do it for you.
Such a weird attitude, makes no logical sense. People are fine punching a clock to do that very same IT work, and no one there wants to know how the sausage is made. Meanwhile, someone who want the same help, want to know what they did, with a quick explainer, nope. Taboo. And it's not even esoteric. It's basic IT stuff. 1. What's combative? 2. What's demanding? 3. Where did I say I want support (and you're sounding like you're mocking me as if I'd want it for free, when I've said the opposite)? Not being combative and demanding. I don't understand how to do this, and posting generic articles for an idiot like me to read isn't helpful. The same effort spent to do that could just have been spent to show me literally how to do it. It's just industrial gatekeeping as far as I'm concerned.
 

bwahaha

Active Member
Jun 9, 2023
127
97
28
@ramicio you may not see the behavior they were referring to, but we can.
You didn't ask for a quick explanation, you asked for a hand holding.

1) what combative? well, that post, as example....
2) What's demanding? well that post, as example...
3) whats.... that? well, that post for example.

We get it, something seemingly simple, should be a simple answer. Unfortunately, you were given solid information and you said rejected it.

We aren't on a Professional Support forum, we're a hobbyist learning forum. Many of us are, were, or will be professionals, so when "outside of scope of homelab" pops in, and demands professional support, for free (yes, you offered to pay), and becomes combative, it irks us. We've dealt with demanding customers, clients, users, and we come here to get away from them.
 
Last edited:

Cheburashka

New Member
Aug 10, 2020
14
0
1
Why do you want to run jumbo frames in 2024? They're almost entirely unnecessary for 10gb, have you done any throughput testing etc showing that you need them for some reason?
I guess I was trying to figure out why between my QNAP, the ICX SW and my W11 desktop, am I getting 5.6gig with iperf. I was just going through threads and trying to identify ways to improve the performance of the transfer.

I have to do this test from a linux live boot ISO instead of Windows to see if the issue is within the OS or not.
 

hmw

Well-Known Member
Apr 29, 2019
649
269
63
I guess I was trying to figure out why between my QNAP, the ICX SW and my W11 desktop, am I getting 5.6gig with iperf. I was just going through threads and trying to identify ways to improve the performance of the transfer.

I have to do this test from a linux live boot ISO instead of Windows to see if the issue is within the OS or not.
jumbo frames might help by 10% but unless you're tunneling or encapsulating other protocols, they're almost never the issue.

try running iperf in parallel mode and see if you can saturate 10g by running 2-3 iperf streams in parallel ( for example iperf3 -P 3 ). Windows is notorious for not being able to saturate 10G with just one connection/thread. The other problem is something in between that might cause impedance mismatch like a SFP to 10GBase-T transceiver etc
 
  • Like
Reactions: fohdeesha

86turbodsl

Active Member
Feb 24, 2020
111
36
28
can anyone help me understand what a proper stack setup on a 6610 is supposed to look like? I have a 6610 48p in the house, and a 6610 24p in the shop. I ran stack secure-setup, it has a connection, it's complaining about a link. I thought one of the fibers might be mixed up because i've swapped sides a few times trying to get the link up. It's currently showing

One port, the left as green on each end. I am using 1/2/6 and 1/2/7 (bottom ports) on the big one and 3/2/1 and 3/2/2 (bottom) on the other end. 1/2/6 and 3/2/1 are showing as green currently. Should i assume i have a dead fiber if i tried switching sides on the one end of one of the fibers. Or swap optics out?

Stack Details
Unit IDTypeRoleMac AddressPriorityStateComment
1S ICX6610-48Pactive 748e.f8fb.f0a2128local Ready
3D ICX6610-24Pstandby cc4e.24c5.b6fc0remote Ready



alone: standalone, D: dynamic config, S: static config


Stack Port Status
Unit IDStack-port1Stack-port2
1dn (1/2/1)up (1/2/6)
3up (3/2/1-3/2/2)dn (3/2/6-3/2/7)







Stack Neighbors
Unit IDStack-port1Stack-port2
1none3
31none
 

DouglasteR

Active Member
Dec 19, 2015
142
38
28
Hi there guys,

The time has come and i´m on the verge to buy the 24p POE+ version and setup it as my main switch/dhcp in my home.

But i have one question first, can i connect my ConnectX3 - QSFP+ directly to the switch and connect at 40Gbps ?

My use case is learning switching, power some ip cameras and common soho networking.

Thanks.
 

BBergle

New Member
Nov 14, 2024
4
2
3
Sometimes the issue is bad serial cable. Some of the Arista and Celestica switches I bought came with their own serial cables, and they work much better than old one I had.
I am actually using a serial cable from an Arista switch I got on ebay. The seller sent me a new switch and I am having the same issue. It has to be one of the cables/adapters I am using. I feel like im going insane. I have tried three different computers. M2 Macbook pro, Intel Macbook Pro and a windows PC and they all have the same issue but the windows computer has horizontal lines instead of question marks. I have new cables coming tomorrow but my current ones work fine for my Arista switch so I really don't know what's going on. All I can seem to get on my terminal screen are these characters ����
 

Attachments

Last edited:

kpfleming

Active Member
Dec 28, 2021
456
236
43
Pelham NY USA
Hi there guys,

The time has come and i´m on the verge to buy the 24p POE+ version and setup it as my main switch/dhcp in my home.

But i have one question first, can i connect my ConnectX3 - QSFP+ directly to the switch and connect at 40Gbps ?

My use case is learning switching, power some ip cameras and common soho networking.

Thanks.
If by "the 24p POE+" version you mean the 7250-24P, then no, the high speed ports are 10Gbps maximum.
 

who

New Member
May 12, 2022
3
0
1
Hi everyone!
I believe this is my first time posting on this thread... ;)
Been following this thread for maybe 2 years or so.

First of all I'll like to thank fohdeesha for sharing his knowledge and expertise!

Initially I've gotten myself a ICX6430-12P, running since mid-2022 and recently ICX6450-12P (likely replace or keep as spare) as well as a 7250-24P.

I've encounter a small license issue on the 6450 last night as i was upgrading it.

Code:
ICX6450-C12PD Router#show license
Index    Lic Mode        Lic Name               Lid/Serial No  Lic Type    Status     Lic Period    Lic Capacity
Stack unit 1:
1        Node Lock       ICX6450-PREM-LIC-SW    H4C<snip>PLN8    Normal      Active     Unlimited         1
2        Node Lock       ICX6450-10G-LIC-POD    H4C<snip>PLN8    Normal      Invalid    Unlimited         2
The 10G license status is invalid.
Has anyone experienced this? I'm guessing a lot of you here are not using the 6450-12P so it might have slipped through the cracks.

I've removed and added it back, still shows the same invalid status.
SHA256 hash (bc1160442a077d0be720be4a20464f8d9ce837543e2ee94f262f9a38dc8c1bd2) of the license (2-6450-ports.xml) is the same from the latest brocade-08-03-2024.zip and brocade-09-16-2022.zip

Code:
ICX6450-C12PD Router#sh version
  Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
    UNIT 1: compiled on Apr 23 2020 at 10:57:06 labeled as ICX64R08030u
                (9871112 bytes) from Primary ICX64R08030u.bin
        SW: Version 08.0.30uT313
  Boot-Monitor Image size = 786944, Version:10.1.05T310 (kxz10105)
  HW: Stackable ICX6450-C12-PD
==========================================================================
UNIT 1: SL 1: ICX6450C 12-port-PD Management Module
         Serial  #: 2ax5o2jk68e
         License: ICX6450_PREM_ROUTER_SOFT_PACKAGE   (LID: H4CKTH3PLN8)
         P-ENGINE  0: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450C-Copper 2port 2G Module
==========================================================================
UNIT 1: SL 3: ICX6450C-Fiber 2port 2G Module
==========================================================================
  800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
  512 MB DRAM
STACKID 1  system uptime is 42 minute(s) 23 second(s)
The system started at 00:00:17 GMT+00 Thu Jan 01 1970

The system : started=cold start
Any inputs would be very much appreciated. I don't think I'll be using 10G soon but would like to keep my options open. :)

Does the 6450-C12-PD supports 10G in the first place? :p
If not, that might explain the invalid status. :D


UPDATE: Ops! My bad! Looks like there's no 10G ports on the 6450C.
 
Last edited:

molnart

Member
Feb 5, 2023
47
1
8
I just acquired an ICX7250-48p. I've been trying to get connected to the console but I've run into an issue. I've assembled a couple of the serial cables for these and using the settings that it should be (9600 8N1), all I get is gibberish. The way the output is produced seems like it is a boot up sequence and if I let it go, it will echo back what I type like it has gotten to the shell but the output is unreadable. Any ideas on how I can get it working?
have you been able to resolve this? i have tried multiple baud rates or cables, but still just getting garbage on the console (or no output even with one particular cable). i am not even able to type into the console, i get complete different characters than i am sending

also i have an USB FTDI board that I have tried with a spliced miniUSB cable. would it be safe to try the board on 5V voltage?
 
Last edited:

enanothebrave

New Member
Nov 30, 2024
2
0
1
I picked up an ICX7250 from ebay and wanted to check if anyone has come across this. I reached out to the seller as I'm assuming its borked. Booting into serial console presents the following. Keyboard input is not accepted so I cannot press b to interrupt the boot. I've tried two console cables, one made from a sacrificial mini usb, punched down to rj45 and connected to a cisco cable, and then one mini usb to db9 connected to a db9 to usb serial cable. Both cables present the same console log without the ability to type. The switch was advertised as coming from a working environment. Been searching the forums for some time looking for a similar situation, most folks seem to have one of the boot partitions working and I don't see much about not being able to type into the console.


EDIT: Ebay seller issued a refund and said don't ship it back. I may try and futz with the serial port and connect wires directly onto the motherboard just incase there is an issue there. Will let this post sit for a bit to see if anyone has any other guidance first though.


Brocade Bootloader: 10.1.05T215 (Mar 19 2015 - 16:39:20)

Validate Shmoo parameters stored in flash ..... OK
Restoring Shmoo parameters from flash .....
Running simple memory test ..... OK

ICX7250-48 Copper (POE), PVT1
SYS CPLD VER: 0x6, Released Ver: 0

Enter 'b' to stop at boot monitor: 0
bootdelay: ===
Booting image from Primary

NAND read: device 0 offset 0x0, size 0x2000000
................................ 33554432 bytes read: OK
Wrong Image Format for bootm command
ERROR: can't get kernel image!
could not boot from primary, no valid image; trying to boot from secondary
BOOTING image from Secondary

NAND read: device 0 offset 0x4000000, size 0x2000000
....................................................Skipping bad block 0x05a00000
Skipping bad block 0x05b00000
............ 33554432 bytes read: OK
Wrong Image Format for bootm command
ERROR: can't get kernel image!
ICX7250-Boot>
 
Last edited:

enanothebrave

New Member
Nov 30, 2024
2
0
1
have you been able to resolve this? i have tried multiple baud rates or cables, but still just getting garbage on the console (or no output even with one particular cable). i am not even able to type into the console, i get complete different characters than i am sending
If you are getting garbage/gibberish on a serial connection that usually has to do with an incorrect baud rate, should be 9600 for these. You should at least get clear text, I'm having the same issue with not being able to type though.
 

manki09

New Member
Sep 17, 2023
9
6
3
Hey Guys.
I've been running a Brocade ICX6610 for a while now with no issue. I recently upgraded my internet from 500mbps/20mbps to 2gb/300mbps. I have attached my current network diagram. What I changed since the upgrade is the modem from a TP-Link to Xfinity's XB8 and I changed from a HP T730 to a Lenovo m90q gen2.

I'm having a bit of an issue with traffic flowing through the ICX6610.

On 1GB enabled devices -
When I have both interfaces (LAN/WAN) on OPNSense to 2.5gbe or 10gbe my internet download rate is only 500-600mbps. If I lower either the LAN or WAN interface down to 1gbe I get about 940mbps just fine. Switch it back to 2.5gbe or 10gbe and the speed will drop back down to 500-600mbps.

On 40GB enabled devices (R730 to Windows 10 VM). It's a vary similar issue to 1GB devices, however it's about 1100mbps, which is better but not the 2gbe. If I switch the same VM to a 1GB port I again get the same 500-600gbps. I have also switched the VM over to the same vlan as the router interface to eliminate any intervlan issues, and the speeds are still the same.

If I hook a PC w/ 2.5gbe directly to my router I can get a consistent 2.1gbe download rate. So I know its not OPNSense directly causing the speed drops.

What I have noticed is dropped packets on interfaces when I do speed test.

Code:
Egress queues:
Queue counters    Queued packets    Dropped Packets
    0                 512                   0
    1                   0                   0
    2                   0                   0
    3                   0                   0
    4                   0                   0
    5                 281                   0
    6                   0                   0
    7             1941191                 919
The OPNSense LAN side is currently connected to the switch with a 10GB RJ45 SFP+, I have also testing with a cheap 2.5gb/10gb switch between the router and brocade with 2.5 to the router and twinax to the brocade

Anyone have any ideas what could cause this and a fix?


Edit:
I forgot to mention. I loaded OPNSense onto completely different hardware and got the same result.
I also originally used OPNSense in ESXi but moved it to bare metal to troubleshoot.
network diagram.png
 

Attachments

Last edited:

Haulien

New Member
Jun 3, 2024
1
0
1
Thanks for the guides! They've been really helpful on setting up.

I have one question however, I did the licensing for a ICX7250, but noticed it said it's a trial for 45 days. Is this something I should be worried about?