Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Dave Corder

Active Member
Dec 21, 2015
353
238
43
42
I recently acquired one of these switches (ICX6610), but can only get two of the four QSFP+ ports on the back to work at all. They work at 40 gbits. I can get one of my QSFP+ cards to connect to one of the SFP+, but they only work at 1 gbit. What do I need to do to either make the the other two back ports to work or to make the front SFP ports work at 10 gbits?

I should note I updated the bootloader to grz10100

The command:
i2c write a 0 feedface00000200ffffffffffffffff 1
returned:
undefined device ID 10 i2c device write failed at 0, ERROR
Maybe I need to get an EEPROM for it?
Two of the QSFP+ ports on the back are 40 Gbps ports, the other two are 4x 10 Gbps breakout ports (that cannot be used as a single 40 Gbps port).
 

SuperMiguel

New Member
Jun 17, 2021
23
2
3
Which sfp+ Cable you recommended for the Brocade ICX6450? I need about 3ft of cabling also which nic?

Also is the OP still providing free licenses? I need to unlock the 10 g ports on the icx6450 if i end up getting one…

last question;) are there any switches with more than 4 10g ports that is not as loud as the icx6610?? I have one and it is pretty loud. Are the noise levels between regular 24 and 48poe the same?
 
Last edited:

epicurean

Active Member
Sep 29, 2014
787
81
28
The network in my home is suddenly very slow, especially when viewing the IP cameras. I used to use wireguard on my phone and can see all my ip cameras almost instantly, but now a few cameras come on and off and very sluggish. Plex also sluggish with nvidia shield. Intermittenly network connection also goes off. I have rebooted my switches but does not seem to make a difference.
I am not sure where to start troubleshooting. A dedicated PFsense router is connected to a ICX6450 48 POE switch, which is also connectedt o a unifi 16 POE switch. Half of my IP cams are in the unifi switch (which I intend to migrate to the ICX6450), and the other half in the ICX6450. 3 x R610 ruckus APs are connected to the ICX6450 , as is a single ruckus H510 as well.
The IP cameras in the ICX6450 are on vlan 80, whilst those in the unifi switch is on the main vlan 1 - in the process of being moved to vlan 80.

Any help appreciated. much thanks!
 

supacupa

New Member
Jun 2, 2021
15
2
3
Two of the QSFP+ ports on the back are 40 Gbps ports, the other two are 4x 10 Gbps breakout ports (that cannot be used as a single 40 Gbps port).
How does that work? I plug a 40 Gbps cable and it runs at 10g? or there are 4x10 Gbps connections?
 

DavidRa

Infrastructure Architect
Aug 3, 2015
337
157
43
Central Coast of NSW
www.pdconsec.net
How does that work? I plug a 40 Gbps cable and it runs at 10g? or there are 4x10 Gbps connections?
You would need a breakout cable - a DAC (which has the QSFP module on one end and it splits out to four 10Gbps SFP+ modules). One example is this one 40G QSFP+ Breakout Active Optical Cables (AOC) - not a recommendation, just first I found as an example.
 

supacupa

New Member
Jun 2, 2021
15
2
3
You would need a breakout cable - a DAC (which has the QSFP module on one end and it splits out to four 10Gbps SFP+ modules). One example is this one 40G QSFP+ Breakout Active Optical Cables (AOC) - not a recommendation, just first I found as an example.
OK, so it's just the reverse of what's up front. I think I understand. I have a breakout cable. Now I need to figure out how to unlock 10Gbit speeds.
 

rootpeer

Member
Oct 19, 2019
86
17
8
Does going into the port config and issuing a disable / enable reset it? It does that for PoE devices, but I haven't worked with one of those SFP ONU units, so I couldn't tell you if disable actually cuts power to the SFP itself.
No unfortunately it does not. It just disables the networking.
 

mmx

New Member
Dec 18, 2015
9
9
3
Hi all, longtime lurker, first time poster (in this thread, any way).

Thanks to @fohdeesha and the immense amount of knowledge found in this thread by numerous contributors, I bought 4x 6610s last year to replace the crappy Cisco SG200/SG220s we have at work. I've been taking my time with the migration as it's my first time working with such powerful devices and I want to make sure I've got everything covered. My goal is to offload inter-VLAN routing from pfSense onto the 6610s (via a transit VLAN).

I want to share a small discovery I made today in hopes of making these switches friendlier to newcomers like me.

One of my biggest challenges was managing ACLs through the CLI. I'm used to centrally managing firewall rules through a web panel, so having a non-visual interface takes some getting used to. Some of my concerns & questions were:
  1. How can I add/modify/remove a single ACL entry in an access-list?
  2. How can I reorder ACLs in an access-list?
  3. How can I write ACLs more efficiently? Is there an equivalent to pfSense's IP/Port aliases?
This is where Brocade Network Advisor comes in. Now I know it's EOL, but it's still quite a valuable tool for someone like me as it answers all the questions above. BNA is probably nothing new to the experienced out there, yet it's barely mentioned in this thread apart from @Jason Antes bringing it up in April 2021 and last week. I think it deserves some recognition even if it's mostly archaic and superseded by Ruckus.

One of the greatest features in BNA is the fact that you can create Networks, Network Groups, Services and Service Groups. These are basically an alternative to pfSense's aliases, and they're extremely useful when writing ACLs for several domain networks. You can even include Groups in Groups, equivalent to referencing an alias inside another alias in pfSense. Just this feature alone avoids having to repeat yourself, thus avoiding mistakes when writing ACLs for dozens of networks with similar rules. I've read through the documentation provided by Ruckus, and there's no way to replicate this functionality through the CLI (as far as I can tell).

For example, I want to create a single Service Group for all Active Directory Domain Controller ports. Here's a few screenshots to showcase the process. Service ports are protocol-specific, however you will still have to create separate ACLs for TCP and UDP. The final screenshot will show you how every individual entry gets created automagically with only 1-2 entries created in BNA.

For anyone interested, the version I have found online is 14.2.12 (IP only, no SAN support) and it doesn't require a paid licence. I've deployed this particular version without issue.

I hope this helps someone out there!
 
Last edited:

gregsachs

Active Member
Aug 14, 2018
605
212
43
Which sfp+ Cable you recommended for the Brocade ICX6450? I need about 3ft of cabling also which nic?

Also is the OP still providing free licenses? I need to unlock the 10 g ports on the icx6450 if i end up getting one…

last question;) are there any switches with more than 4 10g ports that is not as loud as the icx6610?? I have one and it is pretty loud. Are the noise levels between regular 24 and 48poe the same?
For a 3 ft run I'd use a dac, more robust. Stay at 5m or less, the 6450 doesn't like 10m dacs
 

Spearfoot

Active Member
Apr 22, 2015
111
51
28
Hi all, longtime lurker, first time poster (in this thread, any way).

Thanks to @fohdeesha and the immense amount of knowledge found in this thread by numerous contributors, I bought 4x 6610s last year to replace the crappy Cisco SG200/SG220s we have at work. I've been taking my time with the migration as it's my first time working with such powerful devices and I want to make sure I've got everything covered. My goal is to offload inter-VLAN routing from pfSense onto the 6610s (via a transit VLAN).

I want to share a small discovery I made today in hopes of making these switches friendlier to newcomers like me.

One of my biggest challenges was managing ACLs through the CLI. I'm used to centrally managing firewall rules through a web panel, so having a non-visual interface takes some getting used to. Some of my concerns & questions were:
  1. How can I add/modify/remove a single ACL entry in an access-list?
  2. How can I reorder ACLs in an access-list?
  3. How can I write ACLs more efficiently? Is there an equivalent to pfSense's IP/Port aliases?
This is where Brocade Network Advisor comes in. Now I know it's EOL, but it's still quite a valuable tool for someone like me as it answers all the questions above. BNA is probably nothing new to the experienced out there, yet it's barely mentioned in this thread apart from @Jason Antes bringing it up in April 2021 and last week. I think it deserves some recognition even if it's mostly archaic and superseded by Ruckus.

One of the greatest features in BNA is the fact that you can create Networks, Network Groups, Services and Service Groups. These are basically an alternative to pfSense's aliases, and they're extremely useful when writing ACLs for several domain networks. You can even include Groups in Groups, equivalent to referencing an alias inside another alias in pfSense. Just this feature alone avoids having to repeat yourself, thus avoiding mistakes when writing ACLs for dozens of networks with similar rules. I've read through the documentation provided by Ruckus, and there's no way to replicate this functionality through the CLI (as far as I can tell).

For example, I want to create a single Service Group for all Active Directory Domain Controller ports. Here's a few screenshots to showcase the process. Service ports are protocol-specific, however you will still have to create separate ACLs for TCP and UDP. The final screenshot will show you how every individual entry gets created automagically with only 1-2 entries created in BNA.

For anyone interested, the version I have found online is 14.2.12 (IP only, no SAN support) and it doesn't require a paid licence. I've deployed this particular version without issue.

I hope this helps someone out there!
Sounds interesting!

Do you have a Premium Support account at Ruckus? I get "That file is only available to Premium Support users." when I try downloading it from Ruckus. Also, the latest version I found there is 14.2.11:

 

mmx

New Member
Dec 18, 2015
9
9
3
Sounds interesting!

Do you have a Premium Support account at Ruckus? I get "That file is only available to Premium Support users." when I try downloading it from Ruckus. Also, the latest version I found there is 14.2.11:

Nope, I don't have any paid support plans. Check out the link I posted to the Reddit thread; one of the comments will give you a Google Drive link to download 14.2.12.
 
  • Like
Reactions: Jason Antes

Jason Antes

Active Member
Feb 28, 2020
230
78
28
Twin Cities
I have versions from 12.31 to 14.41. None of these versions like installing in Server 2019 so I went the route of installing on 2012R2 and then doing an in-place upgrade to 2019 after installation. The license I have required me to install the older 12.x series first and then upgrade to 14.x which is why I had to do the 2012R2 route. It is a great tool. PM me if you need help. ;););)
 
  • Like
Reactions: Spearfoot

Sundar

New Member
Oct 31, 2018
19
3
3
You can also buy Noctua NF-A4x20 FLX fans, i have modded 3 6450-24p with these ...
@juey : I just received my ICX 6450-24P. I do not use PoE and I have disabled PoE. The stock fans are still too loud for me. I am thinking of buying the Noctua NF-A4x20 FLX fans (I have been very happy with them in other switches like the Quanta LB9a).
I have a few questions:
- Will the standard 3-pin connector on the Noctua fans __JUST_WORK__?
- Or do I need to swap around the wires in any way?
- Did you replace BOTH the stock Sunon fans, or just one?

TIA
 

NateS

Active Member
Apr 19, 2021
158
91
28
Sacramento, CA, US
@juey : I just received my ICX 6450-24P. I do not use PoE and I have disabled PoE. The stock fans are still too loud for me. I am thinking of buying the Noctua NF-A4x20 FLX fans (I have been very happy with them in other switches like the Quanta LB9a).
I have a few questions:
- Will the standard 3-pin connector on the Noctua fans __JUST_WORK__?
- Or do I need to swap around the wires in any way?
- Did you replace BOTH the stock Sunon fans, or just one?

TIA
You need to swap the wires with any fan replacement -- the switch uses a non-standard pinout. It's very easy to do though. The center wire stays in place, and the left and right swap. Alternatively, you could also just break off the latching tab, then plug the whole thing in backwards.

On my switch, I used these Sunon fans for replacement. They're not perfectly silent, but very quiet, which was good enough for me. I'm sure the Noctuas would be even quieter. In my case, I swapped both, but since this switch doesn't care if they're present at all, it should be possible to just swap one and disconnect the other -- just watch you're temps if you're running a heavy load.
 

Jason Antes

Active Member
Feb 28, 2020
230
78
28
Twin Cities
I need to make a clarification, the last version I have from Brocade of BNA that supports IP is 14.3.0. 14.4.0 is SAN only.
I am uploading 14.3 to a google drive location and will share it out when I have it uploaded.

I looked at the 14.2.12 version compared from Rukus to the 14.3 from Brocade. If your goal is to only manage IP, go with 14.2.12. It has some CVE fixes and supports a newer version of JRE. I'm going to spin it up on a test VM and see if I can install it and use my licenses on it directly on 2019 Server.
 
  • Like
Reactions: mmx

Sundar

New Member
Oct 31, 2018
19
3
3
You need to swap the wires with any fan replacement -- the switch uses a non-standard pinout. It's very easy to do though. The center wire stays in place, and the left and right swap. Alternatively, you could also just break off the latching tab, then plug the whole thing in backwards.

On my switch, I used these Sunon fans for replacement. They're not perfectly silent, but very quiet, which was good enough for me. I'm sure the Noctuas would be even quieter. In my case, I swapped both, but since this switch doesn't care if they're present at all, it should be possible to just swap one and disconnect the other -- just watch you're temps if you're running a heavy load.
Thanks, @NateS
 

richtj99

Member
Jul 8, 2017
70
1
8
51
The network in my home is suddenly very slow, especially when viewing the IP cameras. I used to use wireguard on my phone and can see all my ip cameras almost instantly, but now a few cameras come on and off and very sluggish. Plex also sluggish with nvidia shield. Intermittenly network connection also goes off. I have rebooted my switches but does not seem to make a difference.
I am not sure where to start troubleshooting. A dedicated PFsense router is connected to a ICX6450 48 POE switch, which is also connectedt o a unifi 16 POE switch. Half of my IP cams are in the unifi switch (which I intend to migrate to the ICX6450), and the other half in the ICX6450. 3 x R610 ruckus APs are connected to the ICX6450 , as is a single ruckus H510 as well.
The IP cameras in the ICX6450 are on vlan 80, whilst those in the unifi switch is on the main vlan 1 - in the process of being moved to vlan 80.

Any help appreciated. much thanks!
Im wondering if this has to do with the pfsense firewall & the speed of Vlan 1 talking to vlan 80? I notice with my firewall, even with port rules enabled for inter-vlan traffic, there is a speed difference in transferring files from vlan 20 (nas no internet access) to my main vlan (25 with internet). Some VM's on Vlan 20 get better transfer speeds within vlan 20.

I switched a bunch of things to 10G & it runs better but I noticed that with a ton of IP cameras, two NVRS (dont ask), that there was so much traffic on my 1gb switches that my 1gb Verizon FIOS connection was only getting 400 speeds. When I turned off my two NVR's the speeds went to the mid 900 range which is closer to what I would expect. Once the NVR's went back on speeds slowed the whole network down.

When I moved from Cisco 1gb to these 6450/7250 units & made sure the only inter switch connectivity was through the 10gb connections it has been great.

I do have a lag setup so I am at 30gb between switches except:

Verizon Fios ONT to Firewall (1gb) to Unifi switch (1gb fiber) to icx6450.

just some thoughts - not sure if this is helpful.
 

richtj99

Member
Jul 8, 2017
70
1
8
51
Hi - I purchased a 7250-48p - first one came with no working serial port, seller sent a 'tested' replacement - serial does work, as soon as it boots up I get this error:


Code:
U1-MSG: PoE Severe Error: Power being injected on port 1/1/33. No new PDs can get powered on this unit.Configure "no inline power" on all Switch to Switch connected ports of this unit and peer unit(s) to resolve the issue.
I tried disabling POE on all ports, just port 33, etc. I am not sure what to do or if this is fixable.

I took a photo of the inside (attached).

Code:
ICX7250-48P Router# show inline power

Power Capacity:         Total is 740000 mWatts. Current Free is 740000 mWatts.

Power Allocations:      Requests Honored 48 times


 Port   Admin   Oper    ---Power(mWatts)---  PD Type  PD Class  Pri  Fault/
        State   State   Consumed  Allocated                          Error
--------------------------------------------------------------------------
  1/1/1 On      Off            0          0  n/a      n/a         3  n/a
  1/1/2 On      Off            0          0  n/a      n/a         3  n/a
  1/1/3 On      Off            0          0  n/a      n/a         3  n/a
  1/1/4 On      Off            0          0  n/a      n/a         3  n/a
  1/1/5 On      Off            0          0  n/a      n/a         3  n/a
  1/1/6 On      Off            0          0  n/a      n/a         3  n/a
  1/1/7 On      Off            0          0  n/a      n/a         3  n/a
  1/1/8 On      Off            0          0  n/a      n/a         3  n/a
  1/1/9 On      Off            0          0  n/a      n/a         3  n/a
 1/1/10 On      Off            0          0  n/a      n/a         3  n/a
 1/1/11 On      Off            0          0  n/a      n/a         3  n/a
 1/1/12 On      Off            0          0  n/a      n/a         3  n/a
 1/1/13 On      Off            0          0  n/a      n/a         3  n/a
 1/1/14 On      Off            0          0  n/a      n/a         3  n/a
 1/1/15 On      Off            0          0  n/a      n/a         3  n/a
 1/1/16 On      Off            0          0  n/a      n/a         3  n/a
 1/1/17 On      Off            0          0  n/a      n/a         3  n/a
 1/1/18 On      Off            0          0  n/a      n/a         3  n/a
 1/1/19 On      Off            0          0  n/a      n/a         3  n/a
 1/1/20 On      Off            0          0  n/a      n/a         3  n/a
 1/1/21 On      Off            0          0  n/a      n/a         3  n/a
 1/1/22 On      Off            0          0  n/a      n/a         3  n/a
 1/1/23 On      Off            0          0  n/a      n/a         3  n/a
 1/1/24 On      Off            0          0  n/a      n/a         3  n/a
 1/1/25 On      Off            0          0  n/a      n/a         3  n/a
 1/1/26 On      Off            0          0  n/a      n/a         3  n/a
 1/1/27 On      Off            0          0  n/a      n/a         3  n/a
 1/1/28 On      Off            0          0  n/a      n/a         3  n/a
 1/1/29 On      Off            0          0  n/a      n/a         3  n/a
 1/1/30 On      Off            0          0  n/a      n/a         3  n/a
 1/1/31 On      Off            0          0  n/a      n/a         3  n/a
 1/1/32 On      Off            0          0  n/a      n/a         3  n/a
 1/1/33 On      Off            0          0  n/a      n/a         3  voltage applied from ext src
 1/1/34 On      Off            0          0  n/a      n/a         3  n/a
 1/1/35 On      Off            0          0  n/a      n/a         3  n/a
 1/1/36 On      Off            0          0  n/a      n/a         3  n/a
 1/1/37 On      Off            0          0  n/a      n/a         3  n/a
 1/1/38 On      Off            0          0  n/a      n/a         3  n/a
 1/1/39 On      Off            0          0  n/a      n/a         3  n/a
 1/1/40 On      Off            0          0  n/a      n/a         3  n/a
 1/1/41 On      Off            0          0  n/a      n/a         3  n/a
 1/1/42 On      Off            0          0  n/a      n/a         3  n/a
 1/1/43 On      Off            0          0  n/a      n/a         3  n/a
 1/1/44 On      Off            0          0  n/a      n/a         3  n/a
 1/1/45 On      Off            0          0  n/a      n/a         3  n/a
 1/1/46 On      Off            0          0  n/a      n/a         3  n/a
 1/1/47 On      Off            0          0  n/a      n/a         3  n/a
 1/1/48 On      Off            0          0  n/a      n/a         3  n/a
--------------------------------------------------------------------------
 Total                         0          0
I disable port 33 poe then plug two poe devices into other random ports with POE enabled.

Code:
ICX7250-48P Router# U1-MSG: PoE Severe Error: PD on port 1/1/10 cannot be powered due to power being injected on another port of this unit.Configure "no inline power" on all Switch to Switch connected ports of this unit and peer unit(s) to resolve the issue.

U1-MSG: PoE Severe Error: PD on port 1/1/3 cannot be powered due to power being injected on another port of this unit.Configure "no inline power" on all Switch to Switch connected ports of this unit and peer unit(s) to resolve the issue.
Sort of at a loss on what to try?

Any suggestions would be great.
 

Attachments