Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[EvoDyn]

Is this what the 10G port adder license looks like? so 2 base + 2 from the license?

SSH@ICX6450-24P>sh lic
Index Lic Mode Lic Name Lid/Serial No Lic Type Status Lic Period Lic Capacity
Stack unit 1:
1 Node Lock ICX6450-10G-LIC-POD dbtF<snip> Normal Active Unlimited 2

I just picked up the same one as you with the same output. I believe you will need a second license based on this example as a fully licensed 6450.

Ignore what I said above. You are correct based on this and this post to only need a single license to have all four ports 10G on a ICX6450. To be fully licensed one would just need to add a ICX6450-PREM-LIC-SW license for advanced L3 features.

However, I am curious why there would be a second ICX6450-10G-LIC-POD license on that example.

 

[etherbadger]

Ignore what I said above. You are correct based on this and this post to only need a single license to have all four ports 10G on a ICX6450. To be fully licensed one would just need to add a ICX6450-PREM-LIC-SW license for advanced L3 features.

However, I am curious why there would be a second ICX6450-10G-LIC-POD license on that example.

My 3rd SFP+ port nic arrived today, so I can test it out and see if more than 2 come up at 10GB speeds.

I've got a small fleet of HP prodesk 600 SFF desktops for a proxmox cluster. been using 1gb speeds but want to try out ceph and gluster, so the 10gbe will be very nice.

 

[etherbadger]

My 3rd SFP+ port nic arrived today, so I can test it out and see if more than 2 come up at 10GB speeds.

seems to be working at 10G for 3 ports i have plugged in:

SSH@neuron>sh int br ethernet 1/2/1 to 1/2/4

Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/2/1 Up Forward Full 10G None Yes 1 0 cc4e.2451.d7c0
1/2/2 Up Forward Full 10G None Yes 1 0 cc4e.2451.d7c0
1/2/3 Up Forward Full 10G None Yes 1 0 cc4e.2451.d7c0
1/2/4 Down None None None None Yes 1 0 cc4e.2451.d7c0
SSH@neuron>sh lic
Index Lic Mode Lic Name Lid/Serial No Lic Type Status Lic Period Lic Capacity
Stack unit 1:
1 Node Lock ICX6450-10G-LIC-POD <> Normal Active Unlimited 2

 

[supacupa]

I recently acquired one of these switches (ICX6610), but can only get two of the four QSFP+ ports on the back to work at all. They work at 40 gbits. I can get one of my QSFP+ cards to connect to one of the SFP+, but they only work at 1 gbit. What do I need to do to either make the the other two back ports to work or to make the front SFP ports work at 10 gbits?

I should note I updated the bootloader to grz10100

The command:
i2c write a 0 feedface00000200ffffffffffffffff 1
returned:
undefined device ID 10 i2c device write failed at 0, ERROR
Maybe I need to get an EEPROM for it?

 

[LodeRunner]

Hello!

I have an ICX6450. I have a fiber SFP ONU installed in one of the 10Gb ports.

On switch reboots, sometimes the ONU hangs and I have to remove it from the port and re-insert it.

Is there a way to power-cycle the SFP port so I can automate this?

Does going into the port config and issuing a disable / enable reset it? It does that for PoE devices, but I haven't worked with one of those SFP ONU units, so I couldn't tell you if disable actually cuts power to the SFP itself.

 

[tozmo]

Well, I got my 7450-32zp finally up and running the latest firmware. The 2.5gbe ports are sadly nonfunctional, I can't figure out how I could fix them as the switch has physical damage to the exterior. I have a cobbled USBmini-serial cable, and updated bootrom/firmware from the regular USB port on the switch. copy flash bootrom is no longer a supported command after a certain version, apparently.

If anyone is wondering, it's my first 1u device, and it is LOUD imo. The startup is nutso loud, but even when it settles to regular speed, it's still audible due to the high pitch multiple rooms away. There are three fan settings: loud, jet engine, and then 85c is shutdown. I don't have the skill to do what that one guy did with rigging his own fans and looping wires into his PSU to make it more silent.

 

[supacupa]

If anyone is wondering, it's my first 1u device, and it is LOUD imo.

Can confirm. As a tinnitus sufferer this is God's gift from heaven.

 

[Dave Corder]

I recently acquired one of these switches (ICX6610), but can only get two of the four QSFP+ ports on the back to work at all. They work at 40 gbits. I can get one of my QSFP+ cards to connect to one of the SFP+, but they only work at 1 gbit. What do I need to do to either make the the other two back ports to work or to make the front SFP ports work at 10 gbits?

I should note I updated the bootloader to grz10100

The command:
i2c write a 0 feedface00000200ffffffffffffffff 1
returned:
undefined device ID 10 i2c device write failed at 0, ERROR
Maybe I need to get an EEPROM for it?

Two of the QSFP+ ports on the back are 40 Gbps ports, the other two are 4x 10 Gbps breakout ports (that cannot be used as a single 40 Gbps port).

 

[SuperMiguel]

Which sfp+ Cable you recommended for the Brocade ICX6450? I need about 3ft of cabling also which nic?

Also is the OP still providing free licenses? I need to unlock the 10 g ports on the icx6450 if i end up getting one…

last question are there any switches with more than 4 10g ports that is not as loud as the icx6610?? I have one and it is pretty loud. Are the noise levels between regular 24 and 48poe the same?

 

[epicurean]

The network in my home is suddenly very slow, especially when viewing the IP cameras. I used to use wireguard on my phone and can see all my ip cameras almost instantly, but now a few cameras come on and off and very sluggish. Plex also sluggish with nvidia shield. Intermittenly network connection also goes off. I have rebooted my switches but does not seem to make a difference.
I am not sure where to start troubleshooting. A dedicated PFsense router is connected to a ICX6450 48 POE switch, which is also connectedt o a unifi 16 POE switch. Half of my IP cams are in the unifi switch (which I intend to migrate to the ICX6450), and the other half in the ICX6450. 3 x R610 ruckus APs are connected to the ICX6450 , as is a single ruckus H510 as well.
The IP cameras in the ICX6450 are on vlan 80, whilst those in the unifi switch is on the main vlan 1 - in the process of being moved to vlan 80.

Any help appreciated. much thanks!

 

[supacupa]

Two of the QSFP+ ports on the back are 40 Gbps ports, the other two are 4x 10 Gbps breakout ports (that cannot be used as a single 40 Gbps port).

How does that work? I plug a 40 Gbps cable and it runs at 10g? or there are 4x10 Gbps connections?

 

[DavidRa]

How does that work? I plug a 40 Gbps cable and it runs at 10g? or there are 4x10 Gbps connections?

You would need a breakout cable - a DAC (which has the QSFP module on one end and it splits out to four 10Gbps SFP+ modules). One example is this one 40G QSFP+ Breakout Active Optical Cables (AOC) - not a recommendation, just first I found as an example.

 

[supacupa]

You would need a breakout cable - a DAC (which has the QSFP module on one end and it splits out to four 10Gbps SFP+ modules). One example is this one 40G QSFP+ Breakout Active Optical Cables (AOC) - not a recommendation, just first I found as an example.

OK, so it's just the reverse of what's up front. I think I understand. I have a breakout cable. Now I need to figure out how to unlock 10Gbit speeds.

 

[rootpeer]

Does going into the port config and issuing a disable / enable reset it? It does that for PoE devices, but I haven't worked with one of those SFP ONU units, so I couldn't tell you if disable actually cuts power to the SFP itself.

No unfortunately it does not. It just disables the networking.

 

[mmx]

Hi all, longtime lurker, first time poster (in this thread, any way).

Thanks to @fohdeesha and the immense amount of knowledge found in this thread by numerous contributors, I bought 4x 6610s last year to replace the crappy Cisco SG200/SG220s we have at work. I've been taking my time with the migration as it's my first time working with such powerful devices and I want to make sure I've got everything covered. My goal is to offload inter-VLAN routing from pfSense onto the 6610s (via a transit VLAN).

I want to share a small discovery I made today in hopes of making these switches friendlier to newcomers like me.

One of my biggest challenges was managing ACLs through the CLI. I'm used to centrally managing firewall rules through a web panel, so having a non-visual interface takes some getting used to. Some of my concerns & questions were:

1. How can I add/modify/remove a single ACL entry in an access-list?
2. How can I reorder ACLs in an access-list?
3. How can I write ACLs more efficiently? Is there an equivalent to pfSense's IP/Port aliases?

This is where Brocade Network Advisor comes in. Now I know it's EOL, but it's still quite a valuable tool for someone like me as it answers all the questions above. BNA is probably nothing new to the experienced out there, yet it's barely mentioned in this thread apart from @Jason Antes bringing it up in April 2021 and last week. I think it deserves some recognition even if it's mostly archaic and superseded by Ruckus.

One of the greatest features in BNA is the fact that you can create Networks, Network Groups, Services and Service Groups. These are basically an alternative to pfSense's aliases, and they're extremely useful when writing ACLs for several domain networks. You can even include Groups in Groups, equivalent to referencing an alias inside another alias in pfSense. Just this feature alone avoids having to repeat yourself, thus avoiding mistakes when writing ACLs for dozens of networks with similar rules. I've read through the documentation provided by Ruckus, and there's no way to replicate this functionality through the CLI (as far as I can tell).

For example, I want to create a single Service Group for all Active Directory Domain Controller ports. Here's a few screenshots to showcase the process. Service ports are protocol-specific, however you will still have to create separate ACLs for TCP and UDP. The final screenshot will show you how every individual entry gets created automagically with only 1-2 entries created in BNA.

For anyone interested, the version I have found online is 14.2.12 (IP only, no SAN support) and it doesn't require a paid licence. I've deployed this particular version without issue.

I hope this helps someone out there!

 

[gregsachs]

Which sfp+ Cable you recommended for the Brocade ICX6450? I need about 3ft of cabling also which nic?

Also is the OP still providing free licenses? I need to unlock the 10 g ports on the icx6450 if i end up getting one…

last question are there any switches with more than 4 10g ports that is not as loud as the icx6610?? I have one and it is pretty loud. Are the noise levels between regular 24 and 48poe the same?

For a 3 ft run I'd use a dac, more robust. Stay at 5m or less, the 6450 doesn't like 10m dacs

SFP-H10GB-CU1.5M Cisco Compatible 10G DAC Cable | eBay

For sale is aSFP-H10GB-CU1.5M Cisco Compatible. Local pick up is available and always free.

www.ebay.com

 

[Spearfoot]

Hi all, longtime lurker, first time poster (in this thread, any way).

Thanks to @fohdeesha and the immense amount of knowledge found in this thread by numerous contributors, I bought 4x 6610s last year to replace the crappy Cisco SG200/SG220s we have at work. I've been taking my time with the migration as it's my first time working with such powerful devices and I want to make sure I've got everything covered. My goal is to offload inter-VLAN routing from pfSense onto the 6610s (via a transit VLAN).

I want to share a small discovery I made today in hopes of making these switches friendlier to newcomers like me.

One of my biggest challenges was managing ACLs through the CLI. I'm used to centrally managing firewall rules through a web panel, so having a non-visual interface takes some getting used to. Some of my concerns & questions were:

1. How can I add/modify/remove a single ACL entry in an access-list?
2. How can I reorder ACLs in an access-list?
3. How can I write ACLs more efficiently? Is there an equivalent to pfSense's IP/Port aliases?

This is where Brocade Network Advisor comes in. Now I know it's EOL, but it's still quite a valuable tool for someone like me as it answers all the questions above. BNA is probably nothing new to the experienced out there, yet it's barely mentioned in this thread apart from @Jason Antes bringing it up in April 2021 and last week. I think it deserves some recognition even if it's mostly archaic and superseded by Ruckus.

One of the greatest features in BNA is the fact that you can create Networks, Network Groups, Services and Service Groups. These are basically an alternative to pfSense's aliases, and they're extremely useful when writing ACLs for several domain networks. You can even include Groups in Groups, equivalent to referencing an alias inside another alias in pfSense. Just this feature alone avoids having to repeat yourself, thus avoiding mistakes when writing ACLs for dozens of networks with similar rules. I've read through the documentation provided by Ruckus, and there's no way to replicate this functionality through the CLI (as far as I can tell).

For example, I want to create a single Service Group for all Active Directory Domain Controller ports. Here's a few screenshots to showcase the process. Service ports are protocol-specific, however you will still have to create separate ACLs for TCP and UDP. The final screenshot will show you how every individual entry gets created automagically with only 1-2 entries created in BNA.

For anyone interested, the version I have found online is 14.2.12 (IP only, no SAN support) and it doesn't require a paid licence. I've deployed this particular version without issue.

I hope this helps someone out there!

Sounds interesting!

Do you have a Premium Support account at Ruckus? I get "That file is only available to Premium Support users." when I try downloading it from Ruckus. Also, the latest version I found there is 14.2.11:

EOL RUCKUS Products | Product Families | Ruckus Wireless Support

support.ruckuswireless.com

 

[mmx]

Sounds interesting!

Do you have a Premium Support account at Ruckus? I get "That file is only available to Premium Support users." when I try downloading it from Ruckus. Also, the latest version I found there is 14.2.11:

https://support.ruckuswireless.com/...bna#sort=relevancy&f:mad:commonproducts=[BNA]

Nope, I don't have any paid support plans. Check out the link I posted to the Reddit thread; one of the comments will give you a Google Drive link to download 14.2.12.

 

[Jason Antes]

I have versions from 12.31 to 14.41. None of these versions like installing in Server 2019 so I went the route of installing on 2012R2 and then doing an in-place upgrade to 2019 after installation. The license I have required me to install the older 12.x series first and then upgrade to 14.x which is why I had to do the 2012R2 route. It is a great tool. PM me if you need help.

 

[Jason Antes]

I need to make a clarification, the last version I have from Brocade of BNA that supports IP is 14.3.0. 14.4.0 is SAN only.