Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

acurax04

New Member
Feb 18, 2022
2
1
3
it sounds like you haven't actually plugged the switch into your network? The usb-c serial connection is not a network connection, as the guide states you need to connect the switches dedicated management ethernet port (NOT any of the regular ports) to your network
:oops: Thank you! I was able to complete the guide--appreciate this community.
 
  • Like
Reactions: fohdeesha

kilaketia

New Member
Mar 18, 2020
2
0
1
Hi, I bought a ICX6450-24P and I tried using the console port to configure it but it's dead silent. I bought this cable https://www.amazon.fr/gp/product/B01N1625DE/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1, I've tried another cable that comes with Zyxel router (simple RJ45 to DB9) with my server or another server to try to communicate with the switch. But no mater what I've tried it's stays silent. I can access it with telnet but no password is configured, http but no login combination works, same for ssh. I reset it multiple times...

Does anyone has an idea to help me ?
 

kilaketia

New Member
Mar 18, 2020
2
0
1
Alright I still don't have access via the console port, but I found a way to access privileged exec mode via telnet by using "enable system-monitoring all" instead of just enable...

It's on v8030t.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,874
3,374
113
34
fohdeesha.com
Sorry I probably didn't explain this the right way - I'm saying the breakout goes from card to switch so the QSFP side plugged into the Mellanox and 1 of the 10G plugged into the 4x10g module, but the more I type this out the more I realize the adapter is the way to go here. I completely believe that the breakout doesn't work from the switch side since it doesn't say anywhere in the software or on the hardware that breakout is doable.

I posted said Brocade optics because another thread on here (which I now for the life of me can't find) reporting them as functional with the 6610, but we have been blessed by now both fohdeesha himself & LodeRunner with better recommendations, so thanks for straightening me out! I barely skated by setting up my last 10g topology with a 7150, so my hilarious lack of fiber knowledge is really shining here.

mellanoxeseses (and most NICs) don't support qsfp breakout, it's a single connection of either 40gbe or 10gbe. If you want to drop it to a 10gbE SFP connection, search ebay for qsfp > sfp adapter and stick that in the NIC - Mellanox MAM1Q00A-QSA 655874-B21 40G QSFP+ To 10G SFP+ Network Cable Adapter | eBay
 

jcstill

New Member
Aug 10, 2021
4
0
1
Southern California
can you try to run that again (mainly the update u boot command), then reset the switch, (just send the command "reset" but be sure to stop it back into the bootloader again (if it boots all the way into the OS, it may downgrade uboot again). once you stop it back in uboot after the reboot, it should hopefully be in the new version of uboot, then try the update primary command again
I ran the u boot update again and reset the switch. The version is now 10.1.18. I ran into the same issue for the primary update where I'd get the "Primary image download failed". I think this had to do with my tftp server timing out. Rather than spending hours trying to diagnose, I just used tftp64.exe on a windows box and now am on v08.0.90 updating to 08.0.95.

Thanks for the help!
 

-MoNsTeRRR

New Member
May 24, 2020
5
1
3
Hey everyone,
Is anyone here running "Oxidized" on the ICX 6610 to backup the config ? I am have issues with getting it run on this switch and so I thought I would ask here on the thread where I have learned so much about this device.

I have "Oxidized" running on Librenms.

Thanks.
Hello i've the same issue as your :).

I'm using oxidized (not the librenms plugin) i'm running the latest release from oxidized (git install)
Code:
git show --summary
commit 4a4d0c4730700ff219f8af7710adac2973c827ef (HEAD -> master, origin/master, origin/HEAD)
I've added this command to bypass enable issue
Code:
aaa authentication enable implicit-user
I'm stuck at this point :
Code:
[...]
D, [2022-02-20T11:48:25.518942 #236703] DEBUG -- : lib/oxidized/input/cli.rb Running pre_logout commands at 192.168.0.1
D, [2022-02-20T11:48:25.519219 #236703] DEBUG -- : lib/oxidized/input/ssh.rb exit @ 192.168.0.1 with expect: nil
D, [2022-02-20T11:48:26.226598 #236703] DEBUG -- : lib/oxidized/worker.rb: 1 jobs running in parallel
D, [2022-02-20T11:48:27.227918 #236703] DEBUG -- : lib/oxidized/worker.rb: 1 jobs running in parallel
[...]
W, [2022-02-20T11:52:30.191710 #236703]  WARN -- : 192.168.0.1 raised Timeout::Error with msg "execution expired"
D, [2022-02-20T11:52:30.191783 #236703] DEBUG -- : lib/oxidized/node.rb: Oxidized::SSH failed for 192.168.0.1
D, [2022-02-20T11:52:30.191852 #236703] DEBUG -- : lib/oxidized/job.rb: Config fetched for 192.168.0.1 at 2022-02-20 10:52:30 UTC
W, [2022-02-20T11:52:30.508846 #236703]  WARN -- : /192.168.0.1 status no_connection, retry attempt 1
D, [2022-02-20T11:52:30.509142 #236703] DEBUG -- : lib/oxidized/worker.rb: Jobs running: 0 of 1 - ended: 1 of 2
D, [2022-02-20T11:52:30.509382 #236703] DEBUG -- : lib/oxidized/worker.rb: Added /192.168.0.1 to the job queue
As I see on github there as been an issue and you need to run two times the exit command as I see in lib/oxidized/model/fastiron.rb the pre_logout commands as two exit so I don't know what it's going on.

Does anyone know how to handle it ?
 

adman_c

Active Member
Feb 14, 2016
275
148
43
Chicago
Does anyone here have their WAN run through their switch (vlan isolated) to the router and then back? I’m trying out t-mobile 5g as a failover WAN, and my networking ”closet” is on the ground floor of my house. Unsurprisingly, cellular signals are much better on the top floor of my house. I have a network drop up there but no great way to connect directly from that to my pfsense router without going through the switch. Is it possible (and safe) to have a WAN_Transit VLAN that I can use to connect the t-mobile modem on the top floor through my ICX6450 to the second WAN port on my pfsense box? Thanks!
 

kpfleming

Active Member
Dec 28, 2021
432
222
43
Pelham NY USA
Does anyone here have their WAN run through their switch (vlan isolated) to the router and then back? I’m trying out t-mobile 5g as a failover WAN, and my networking ”closet” is on the ground floor of my house. Unsurprisingly, cellular signals are much better on the top floor of my house. I have a network drop up there but no great way to connect directly from that to my pfsense router without going through the switch. Is it possible (and safe) to have a WAN_Transit VLAN that I can use to connect the t-mobile modem on the top floor through my ICX6450 to the second WAN port on my pfsense box? Thanks!
Sure, both possible and 'safe'. It's just a layer 2 connection, should work fine. if you can put a regular unmanaged switch between the modem and the pfsense machine, you can put a fancy managed switch there too :)
 
  • Like
Reactions: adman_c

adman_c

Active Member
Feb 14, 2016
275
148
43
Chicago
Sure, both possible and 'safe'. It's just a layer 2 connection, should work fine. if you can put a regular unmanaged switch between the modem and the pfsense machine, you can put a fancy managed switch there too :)
I guess I’m wondering how worried I need be about a VLAN hopping attack by having a WAN connection pass through my primary switch before any firewalling happens.
 

Rttg

Member
May 21, 2020
74
49
18
I guess I’m wondering how worried I need be about a VLAN hopping attack
That be a major vulnerability in a fairly widely deployed switch - not impossible but highly unlikely. I only moonlight as a network engineer in my homelab, but I’d wager you’re pretty safe.
 
  • Like
Reactions: adman_c

ccie4526

Active Member
Jan 25, 2021
161
127
43
I guess I’m wondering how worried I need be about a VLAN hopping attack by having a WAN connection pass through my primary switch before any firewalling happens.
VLAN hopping attacks will normally only jump you off of whatever tagged VLAN you're on over to an untagged VLAN. If you don't have an untagged VLAN configured on your trunk links, then attack mitigated. Even better if you do NOT use VLAN 1 anywhere in your network.

I run internet outside on my VLAN 999, and my internal VLANs are using other tag numbers, and NOT VLAN 0001.

VLAN hopping attacks succeed because of either unmanaged (read: not VLAN-aware) switches, or use of VLAN 1. Don't do either of those.

And yeah, I moonlight as a network engineer at the world's largest beer brewery. And it's not the world's largest beer brewer. ;-)
 
  • Like
Reactions: adman_c

Dave Corder

Active Member
Dec 21, 2015
334
229
43
42
For a WAN transit VLAN, I'd be more concerned about the switch sending management/discovery frames and other "junk" to the T-Mobile router (and maybe having that device "lock onto" the switch's MAC address instead of your router's). I have a vague recollection of that sort of thing being discussed somewhere in this thread in the past...
 
  • Like
Reactions: adman_c

AgentXXL

New Member
Apr 23, 2020
23
5
3
will the ICX-6610 function/negotiate to 2.5Gbps speed with a sfp+ RJ45 module?
Only if you can find the specific SFP+ module that's known to work, specifically the Supermicro AOM-AQS-107-B0C2-CX. Be careful - there are many knock-offs that claim to be 100% compatible but all that I've found all use the Marvell chip vs the Supermicro which uses the Aquantia AQS-107. It appears to have a larger buffer which some report works properly with switches like the 6610 that don't specifically allow any SFP+ ports to link at 2.5 or 5Gbps. See this post for more details:

 
Last edited:
  • Like
Reactions: manutech

AgentXXL

New Member
Apr 23, 2020
23
5
3
I'm having no luck finding the Supermicro AOM-AQS-107-B0C2-CX in stock from a supplier that's based in or will ship to Canada. I'm looking at other options to get the onboard 5Gbps NIC on one of my systems working properly with my 6610. As the 6610 SFP+ ports only link at 1Gbp or 10Gbps, would I be able to use a Mikrotik CRS305 as a rate converter? I.E. attach my 5Gbps via my current Wiitek SFP+ module to one SFP+ port and then use a DAC cable to connect another SFP+ port to one on the 6610? I would be using the Mikrotik with SwitchOS instead of RouterOS.

The Mikrotik will actually work out cheaper than the Supermicro module, but I'm leary about wasting my time ordering one without knowing if it will actually work. Has anyone done this? TIA!
 

RobstarUSA

Active Member
Sep 15, 2016
235
104
43
I'm having no luck finding the Supermicro AOM-AQS-107-B0C2-CX in stock from a supplier that's based in or will ship to Canada. I'm looking at other options to get the onboard 5Gbps NIC on one of my systems working properly with my 6610. As the 6610 SFP+ ports only link at 1Gbp or 10Gbps, would I be able to use a Mikrotik CRS305 as a rate converter? I.E. attach my 5Gbps via my current Wiitek SFP+ module to one SFP+ port and then use a DAC cable to connect another SFP+ port to one on the 6610? I would be using the Mikrotik with SwitchOS instead of RouterOS.

The Mikrotik will actually work out cheaper than the Supermicro module, but I'm leary about wasting my time ordering one without knowing if it will actually work. Has anyone done this? TIA!
I'm actually doing this with a CRS317 (but 2.5Gbit/s nics, not 5). There is a LARGE learning curve for Mikrotik. I say this as someone who has 9 years of experience on probably 20-30 Cisco layer3 products -- including configuring & maintaining multicast and 4-5 different routing protocols. I did figue it out but RouterOS was the hardest thing I've had to learn in the networking world.

Also: Be away to ONLY GET THE MIKROTIK MGIG SFPS, or you will run into similar issues you are already having, because 3rd party mgig sfps don't report their speed propertly to the CRS317.
 

Rain

Active Member
May 13, 2013
279
125
43
As the 6610 SFP+ ports only link at 1Gbp or 10Gbps, would I be able to use a Mikrotik CRS305 as a rate converter? I.E. attach my 5Gbps via my current Wiitek SFP+ module to one SFP+ port and then use a DAC cable to connect another SFP+ port to one on the 6610? I would be using the Mikrotik with SwitchOS instead of RouterOS.
Using another switch in this fashion will definitely work, though you'll want to verify that the SFP+ ports support linking up & functioning at 2.5Gb/5Gb, otherwise you'll be in the same boat. Check the specifications before you buy!

It may be better & more future-proof to just purchase a switch that has 10Gb SFP+ uplinks and native RJ-45 NBASE-T ports if you need NBASE-T functionality and plan on using it for a few years.
 

AgentXXL

New Member
Apr 23, 2020
23
5
3
Also: Be away to ONLY GET THE MIKROTIK MGIG SFPS, or you will run into similar issues you are already having, because 3rd party mgig sfps don't report their speed propertly to the CRS317.
Is that the Mikrotik S+RJ10 module? Unfortunately that bumps the price of a solution back into the same range as one of the Supermicro SFP modules that would work with the 6610 @ 5Gbps. Of course the Mikrotik is a lot easier to find than the Supermicro module so perhaps that's what I'll have to do.

Using another switch in this fashion will definitely work, though you'll want to verify that the SFP+ ports support linking up & functioning at 2.5Gb/5Gb, otherwise you'll be in the same boat. Check the specifications before you buy!

It may be better & more future-proof to just purchase a switch that has 10Gb SFP+ uplinks and native RJ-45 NBASE-T ports if you need NBASE-T functionality and plan on using it for a few years.
Suggestions on a switch that does this without 'breaking the bank'? My understanding is that the CRS305 has 4 SFP+ ports that support 1, 2.5, 5 and 10 Gbps. My current Wiitek SFP module is seen by the 6610 as connecting at 10Gbps, but the system itself reports a 5Gbps link. I suspect I might run into the same issue using it on the CRS305. I purchased the Wiitek as it's affordable, readily available and according the this STH review, it supports 2.5Gbps and 5Gbps NBase-T. It just doesn't have enough buffer to accept the 10Gbps rate that the 6610 thinks it is, even if I rate limit the 6610 SFP port to less than 5Gbps.


I will be using this setup for a few years, but I'm also on a limited budget so I'd like to get the 5Gbps NIC working as affordably as possible. That's why I chose to purchase one of the 6610's, which is working fine for my other 10Gbps and 1Gbps devices.
 
Last edited:

noduck

Member
Sep 12, 2020
41
13
8
Suggestions on a switch that does this without 'breaking the bank'?
I am using a Netgear MS510TX for this purpose; it works really well for multi-Gig, several RJ45 with various combinations of speeds and one SFP+. It was $210 when I bought it 2 years ago (I cannot find it for sale now). I bought it before I got any of the Brocade switches.

Of course, it does not compare on features with either Mikrotik or Brocade, and only has web UI.
 
  • Like
Reactions: Rain and AgentXXL

blademan

New Member
Jan 7, 2022
7
2
3
Suggestions on a switch that does this without 'breaking the bank'? My understanding is that the CRS305 has 4 SFP+ ports that support 1, 2.5, 5 and 10 Gbps. My current Wiitek SFP module is seen by the 6610 as connecting at 10Gbps, but the system itself reports a 5Gbps link. I suspect I might run into the same issue using it on the CRS305. I purchased the Wiitek as it's affordable, readily available and according the this STH review, it supports 2.5Gbps and 5Gbps NBase-T. It just doesn't have enough buffer to accept the 10Gbps rate that the 6610 thinks it is, even if I rate limit the 6610 SFP port to less than 5Gbps.


I will be using this setup for a few years, but I'm also on a limited budget so I'd like to get the 5Gbps NIC working as affordably as possible. That's why I chose to purchase one of the 6610's, which is working fine for my other 10Gbps and 1Gbps devices.
I’m using a CRS305 and one of these “FLYPROFiber 10GBase-T SFP+ to RJ45 Transceiver, 10Gb SFP+ to RJ45 Ethernet Copper Module for MikroTik S+RJ10, CAT6A/CAT7, 100FT(30M)” based on STH YouTube and review. Negotiates at 2.5Gb, and in Amazon questions section Flypro says it will do NbaseT.
 
  • Like
Reactions: AgentXXL