Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

fohdeesha

Kaini Industries
Nov 20, 2016
2,268
2,332
113
30
fohdeesha.com
What are the specs of your PFsense box? How is the CPU / Mem usage?

Unifi XG (10gb?) - What VMS/NVR for the cameas? If it is Blue Iris - what are the stream megabits?

Is it 1 vlan for you 1 vlan for cameras? I am happy to help where I can but switching to 10gb was pretty immediate for me. Again my vlans are all set with all cameras on the same camera vlan.



Yeah I figured - bummer - can I safely use it as a non-poe unit? I can make an offer on it vs making him pay for shipping again. Then again I am not sure if a 7250-48p with POE disabled will use more electric than its worth. I want to make sure the rest of the switch isnt going to blow up. can I take the POE part of the board out or swap it?
PoE units don't really use anymore power than their non-PoE counterparts unless you're actively powering PoE devices. You can also just remove the entire PoE daughterboard, it's the one on top
 

richtj99

Member
Jul 8, 2017
54
0
6
48
That all sounds good to me. Is the SFP+ port plugged into the 6450? I dont think you need it but can you pop a 10gb sfp+ card into the PFSsense box going to the Unifi?

I have close to 50 cameras on my network all on 1 vlan & am not seeing a slowdown.

I dont use PFsense anymore but it did have good reporting broken out on lan/vlan (wan too) - what activity level is it showing?

Is the 6450 set for auto speed?

show int brief?

to confirm the slowdown is on the Lan vlan?

My pfsense is an i3 4xxxT CPU with 16GB, on an supermicro X10SLV motherboard
Yes, the Unifi 16XG has SFP+ ports and 4 10 GB RJ45 ports. I do have blue iris, but I yet to connec yet and trying to figure out the network bog down. The blue iris PC is a dedicated 1151 motherboard on a 1GB connection.

My cameras are on vlan 80. I have a different vlan for IOT devices. All other devices are on my main LAN
 

NablaSquaredG

Active Member
Aug 17, 2020
360
135
43
Anyone else got authentication issues via ssh on ICX6610 switches when key-authentication is enabled?

Debug session says:
Code:
....
debug1: Offering public key: /home/robin/.ssh/id_rsa.pub RSA SHA256:GKW7yzA1J1qkr1Cr9MhUwAbHbF2NrIPEgZXeOUOz3Us explicit
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: no key from blob. pkalg ssh-rsa: invalid format
ssh_dispatch_run_fatal: Connection to 10.10.2.50 port 22: invalid format
When I add ip ssh key-authentication no, it works fine...

SW version is 8030t, my SSH key is 4096 bits long

I have not setup the pub-key-file on the switch itself, however I wouldn't expect to see weird issues even if it should be set up
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,268
2,332
113
30
fohdeesha.com
Anyone else got authentication issues via ssh on ICX6610 switches when key-authentication is enabled?

Debug session says:
Code:
....
debug1: Offering public key: /home/robin/.ssh/id_rsa.pub RSA SHA256:GKW7yzA1J1qkr1Cr9MhUwAbHbF2NrIPEgZXeOUOz3Us explicit
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: no key from blob. pkalg ssh-rsa: invalid format
ssh_dispatch_run_fatal: Connection to 10.10.2.50 port 22: invalid format
When I add ip ssh key-authentication no, it works fine...

SW version is 8030t, my SSH key is 4096 bits long

I have not setup the pub-key-file on the switch itself, however I wouldn't expect to see weird issues even if it should be set up
the 6 series only supports 2048 length keys, you might be making it barf by offering too thicc of bits even when it doesn't have a pubkey
 

richtj99

Member
Jul 8, 2017
54
0
6
48
PoE units don't really use anymore power than their non-PoE counterparts unless you're actively powering PoE devices. You can also just remove the entire PoE daughterboard, it's the one on top
Is this the part (yellow?) Unplug, remove board? Can I take a daughter board from a 6450 to use in its place or is the issue something beyond the POE daughterboard?
 

Attachments

fohdeesha

Kaini Industries
Nov 20, 2016
2,268
2,332
113
30
fohdeesha.com
Is this the part (yellow?) Unplug, remove board? Can I take a daughter board from a 6450 to use in its place or is the issue something beyond the POE daughterboard?
That's the one, remove the cables with it too. Problem is almost definitely on the board somewhere, you could swap another one in if you had an exact replacement from the same model switch and same port count of that model. Something from a different model entirely like a 6450 will not work
 

NablaSquaredG

Active Member
Aug 17, 2020
360
135
43
the 6 series only supports 2048 length keys, you might be making it barf by offering too thicc of bits even when it doesn't have a pubkey
But it works via password based authentication... If the unit can't handle 4096 bits, password auth shouldn't work (except if Brocade messed up, which would, given my experiences with the switch so far, really surprise me)


On another note:
I just tried to configure the 10GBe Ports for stacking and... failed.

I've googled the error ("1/2/2 must be one of the stacking ports: 1/2/1 1/2/6") and literally the only result is this thread.

That's a bit disappointing... But I can't imagine that these ports do not support stacking.
The manual has an entire section dedicated to them with sentences like
"Periodic background diagnosis provides warning messages when any 4 x10-Gbps subport is down."
The most common connection error in forming an ICX 6610 stack is connecting a 40 Gbps port to a 4 x10-Gbps port, possibly because the two port types use the same type of cable. When this happens, the system may show one end is up, and one end is down. The stack cannot be formed, and the periodic background diagnosis does not run.
For example, the active controller console displays the following messages for an ICX 6610 stack if a 4 x10-Gbps sub-port is down. *** Warning! miss 4*10G link 5/2/8(down)to 1/2/8(down). Stack can still work. *** Warning! U1, dir=1, 4*10G ports: 1/2/8 are down. *** Warning! U5, dir=1, 4*10G ports: 5/2/8 are down. Please use "show stack conn" to view detailed connections You can suppress the error messages by configuring "stack suppress-warning"

Edit:
Uhh, I THINK I've got it!

Will add more infos later
 
Last edited:

NablaSquaredG

Active Member
Aug 17, 2020
360
135
43
So apparently it's not possible to ONLY use the 4x10G Port for stacking.

You can either have:
  • 4x10G and 40G as data port (no stacking)
  • 40G as stacking port and 4x10G as data ports
  • 40G and 4x10G as stacking port
4x10G as stacking port and 40G as data ports is NOT possible. Why?
Code:
SSH@tbm-switch-leaf-public-1(config-unit-1)#stack-port 1/2/2
Error! 1/2/2 must be one of the stacking ports: 1/2/1 1/2/6.
So that's where my issues started.
stack secure-setup does NOT use the 4x10G ports by default.

So what do you need to do? Well, trunk the 40G ports together with the 4x10G ports.

It's as easy as
Code:
stack-trunk 1/2/1 to 1/2/5
stack-trunk 1/2/6 to 1/2/10
First command trunks the upper two ports, second command trunks the lower two ports.

If you run show running-config you should see:
Code:
stack unit 1
  module 1 icx6610-48p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
  stack-trunk 1/2/1 to 1/2/2
  stack-trunk 1/2/6 to 1/2/7
  stack-port 1/2/1 1/2/6
(don't wonder that it's only to 1/2/2 and to 1/2/7 - It is not unusual that the first port serves as an identifier for the other ones)

You can then use stack secure-setup to set up everything. NOTE: I have NOT tested this in a three device setup (only two devices), so YMMV.

You will then see something like
Code:
   active                                                                     
     +---+        +---+                                                       
 =2/6| 1 |2/1=-2/6| 2 |2/1-                                                   
 |   +---+        +---+   |                                                   
 |                        |                                                   
 |------------------------|                                                   

*** Warning! detect 2 trunk-to-port links. The stack may not work.
  U1 stack-trunk (1/2/1-1/2/2) to U2 stack-port (2/2/6)
  U1 stack-trunk (1/2/6-1/2/7) to U2 stack-port (2/2/1)

U#  Stack-port1                                  Stack-port2           
1   up (1/2/1-1/2/2)                             up (1/2/6-1/2/7)       
    up ports: 1/2/1, 1/2/2, 1/2/3, 1/2/4, 1/2/5
    up ports: 1/2/6, 1/2/7, 1/2/8, 1/2/9, 1/2/10

2   up (2/2/1)                                   up (2/2/6)
Go ahead and trunk the other ports
Code:
(config-unit-2)#stack-trunk 2/2/1 to 2/2/5
(config-unit-2)#stack-trunk 2/2/6 to 2/2/10
and there you go:
Code:
SSH@tbm-switch-leaf-public-1#show stack stack-ports
    active       standby                                                       
     +---+        +---+                                                       
 =2/6| 1 |2/1==2/6| 2 |2/1=                                                   
 |   +---+        +---+   |                                                   
 |                        |                                                   
 |------------------------|                                                   

U#  Stack-port1                                  Stack-port2           
1   up (1/2/1-1/2/2)                             up (1/2/6-1/2/7)       
    up ports: 1/2/1, 1/2/2, 1/2/3, 1/2/4, 1/2/5
    up ports: 1/2/6, 1/2/7, 1/2/8, 1/2/9, 1/2/10

2   up (2/2/1-2/2/2)                             up (2/2/6-2/2/7)       
    up ports: 2/2/1, 2/2/2, 2/2/3, 2/2/4, 2/2/5
    up ports: 2/2/6, 2/2/7, 2/2/8, 2/2/9, 2/2/10

TL;DR: Ignore the Brötcom docs, they're not complete...
 

ClintE

Member
Feb 22, 2019
30
7
8
ICX6610-48P arrived today from eBay. Followed the docs and it worked perfectly. It's all up and updated and connected. Thank you @fohdeesha for the guide. No way I would have ever figured out how to get this thing going on my own.

So far the only problem is the fan in one of the two supplies is much louder than the other. They're both Rev. A but I think it's actually just the fan going out or a bad bearing or something. I guess I'll just have to learn to live with it as the replacement supplies are incredibly expensive compared to the whole switch with two supplies and fan modules.

I'm still waiting for a few cards to arrive to try out 10/40gbe but for now it's working nicely at 1g.
Not sure, but I don't think you need two PSU's in any of the 6610 models; they're for redundancy and don't split the load. If I'm wrong about this, someone please correct me.
 
  • Like
Reactions: Jason Antes

tozmo

Member
Feb 1, 2017
72
42
18
71
What do the CONN jumpers do? My 7450-32zp has busted 2.5 ports, but there is a CONN16 3-pin header next to it. There are a lot of headers on these switches, but what purpose do they serve?

The attached pic shows the two processors over the 2.5gbe ports (right) and the regular ones (left). The 8 Poe ports are off screen to the left and do not have the jumper
 

Attachments

Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,268
2,332
113
30
fohdeesha.com
What do the CONN jumpers do? My 7450-32zp has busted 2.5 ports, but there is a CONN16 3-pin header next to it. There are a lot of headers on these switches, but what purpose do they serve?
can you attach a picture? They're debug headers for factory use, stuff like CPLD JTAG, CPU JTAG, etc. 3-pin might be a debug jumper though, not sure. never seen a 3 pin in these
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,268
2,332
113
30
fohdeesha.com
The guide for the icx7250 provides some old firmware, is there any reason not to use the latest version on this site?
RUCKUS ICX 7250 Switch Hardware Installation Guide | Technical Documents | Ruckus Wireless Support
Haven't had a chance to re-write the guide with new instructions for getting to the recommended 8092 train - it's the new UFI images that require a bit more steps for switches with older firmware. Been working on it this week though. Note the very latest is the 8095 train, but it's highly unadvised to run it on anything resembling production, it's basically the beta branch. The recommended stable train is 8092, 8092e I believe being the latest
 
  • Like
Reactions: Jason Antes

Jason Antes

Active Member
Feb 28, 2020
163
45
28
Twin Cities
Here is a link to the BNA versions that I've uploaded. Haven't had a chance to test on 2019 yet for the latest version from Ruckus. Again, I'd use that version if you are wanting to manage IP as it's the most up to date with security fixes. If you need FC management then use the 14.3 version.

BNA
 

JMHershey125

New Member
May 13, 2021
2
0
1
Good afternoon,

I just picked up a ICX6610 switch, does this mean I have a fully licensed switch?

Code:
ICX6610-48P Router>show lic

Index    Lic Mode        Lic Name               Lid/Serial No  Lic Type    Status     Lic Period    Lic Capacity     

Stack unit 1:

1        Node Lock       ICX6610-PREM-LIC-SW    dzmHKHFlFIh    Normal      Invalid    Unlimited     0

2        Node Lock       ICX6610-10G-LIC-POD    dzrHKFHlHdN    Normal      Active     Unlimited     8

3        Node Lock       ICX6610-ADV-LIC-SW     dzrHKFHlHdN    Normal      Active     Unlimited     1

4        Node Lock       ICX-MACSEC-LIC         dzrHKFHlHdN    Normal      Active     Unlimited     1

ICX6610-48P Router>
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,268
2,332
113
30
fohdeesha.com
Good afternoon,

I just picked up a ICX6610 switch, does this mean I have a fully licensed switch?

Code:
ICX6610-48P Router>show lic

Index    Lic Mode        Lic Name               Lid/Serial No  Lic Type    Status     Lic Period    Lic Capacity    

Stack unit 1:

1        Node Lock       ICX6610-PREM-LIC-SW    dzmHKHFlFIh    Normal      Invalid    Unlimited     0

2        Node Lock       ICX6610-10G-LIC-POD    dzrHKFHlHdN    Normal      Active     Unlimited     8

3        Node Lock       ICX6610-ADV-LIC-SW     dzrHKFHlHdN    Normal      Active     Unlimited     1

4        Node Lock       ICX-MACSEC-LIC         dzrHKFHlHdN    Normal      Active     Unlimited     1

ICX6610-48P Router>
yes, with my licenses. Where did it come from?