Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

LodeRunner

Active Member
Apr 27, 2019
539
227
43
Does anyone have a link for a well-priced, Brocade actual (not "compatible"), 1 meter, SFP+, passive, DAC cable? I know the Brocade switches are agnostic, but I would like a real Brocade cable. I just want to be able to run "show optic."
Show optic does nothing for a DAC. There's no optical component, so there can't be optical monitoring, which is what the show optic command is for.

Edit: If you have non-Ruckus/Brocade coded optics, you can do "non-ruckus-optic-enable". Or just order them from FS.com coded for Brocade and stop worrying about it.
 
  • Like
Reactions: CIR-Engineering

CIR-Engineering

I am a functional adult?
Jan 14, 2021
85
30
18
47
Chicago USA
www.cir-engineering.com
Show optic does nothing for a DAC. There's no optical component, so there can't be optical monitoring, which is what the show optic command is for.

Edit: If you have non-Ruckus/Brocade coded optics, you can do "non-ruckus-optic-enable". Or just order them from FS.com coded for Brocade and stop worrying about it.
I wondered about that on a passive DAC... thanks for the clarification.

I have a 6450 and last I knew the "non-ruckus-optic-enable" command did not work on these switches? Am I remembering right/wrong? For fiber I have just been sticking to Brocade branded. They cost about the same anyway.
 

LodeRunner

Active Member
Apr 27, 2019
539
227
43
I wondered about that on a passive DAC... thanks for the clarification.

I have a 6450 and last I knew the "non-ruckus-optic-enable" command did not work on these switches? Am I remembering right/wrong? For fiber I have just been sticking to Brocade branded. They cost about the same anyway.
Oh, I only have 7k family units, so not sure what the process is for a 6k. But get it from FS.com and have it coded for Brocade and it won't matter. It's not like an encrypted signature or anything, just an ID burned into the SFP/DAC and the switch won't know the difference.

This is a FS.com DAC, with one end coded for Ruckus, on my 7450:
Code:
SSH@core#sh med e 1/3/1
Port   1/3/1: Type  : 40GBASE-Passive Copper  1m (QSFP+)
             Vendor: RUCKUS             Version: A
             Part# : 58-0000033-01      Serial#: S2109200480-1
SSH@core#sh med validation  e 1/3/1

Port       Supported Vendor               Type                                                                            
----------------------------------------------------------------------
1/3/1      Yes       RUCKUS                Type  : 40GBASE-Passive Copper  1m (QSFP+)
 

CIR-Engineering

I am a functional adult?
Jan 14, 2021
85
30
18
47
Chicago USA
www.cir-engineering.com
Code:
SSH@ICX6450-24P#sh med e 1/2/3
Port   1/2/3: Type  : 10GE SR 300m ((SFP+))
             Vendor: BROCADE            Version: A
             Part# : 57-0000075-01      Serial#: AAF210210000E3G
SSH@ICX6450-24P#sh med e 1/2/1
Port   1/2/1: Type  : 10GE SR 300m ((SFP+))
             Vendor: OEM                Version: 02
             Part# : SFP-10G-SR         Serial#: CSF101L34485
SSH@ICX6450-24P#sh med e 1/2/4
Port   1/2/4: Type  : 10GE SR 300m ((SFP+))
             Vendor: OEM                Version: 02
             Part# : SFP-10G-SR         Serial#: CSF101L34484
SSH@ICX6450-24P#show optic 1/2/3
 Port  Temperature   Tx Power     Rx Power       Tx Bias Current
+----+-----------+--------------+--------------+---------------+
1/2/3   49.0507 C  -002.6114 dBm -002.3837 dBm    8.366 mA
        Normal      Normal        Normal         Normal

SSH@ICX6450-24P#show optic 1/2/1
SSH@ICX6450-24P#show optic 1/2/4
1/2/3 is Brocade SFP+ fiber. 1/2/1 and 1/2/4 are ipolex 10GBase-T RJ45 SFP+ copper. One copper soon to be removed for a passive DAC since I moved my OPNsense box right next to the switch.
 

AndroidCat

Member
Mar 3, 2015
32
24
8
Still having weird problems in my home lab with 6610s

(....)

I am finding any changes on the switch ports that i make that are connected to one of the cards will invariably disable the port at the ESXi end and the only resolution appears to be a cold restart of the box - it does not matter which combination of ports/transceivers/breakouts - it is a fairly consistent problem

Something as simple as adding a port to a VLAN appears to be enough to trigger the problem

I have done an excerpt here of the config

Any Ideas ?

Craig
I had exactly the same problem as you.
Try to add the line to each physical interface:
Code:
no spanning-tree
e.g.
interface ethernet 1/2/1
 port-name xxxx
 no spanning-tree
Let me know if that helps.

The explanation is somewhere within this thread. I think it was about spanning tree being active per port even though it is globally disabled.
At least it helped in my case.
 

Rttg

Member
May 21, 2020
71
47
18
Does anyone have any pointers on debugging / profiling an ICX switch?

I’ve had an ICX6450-48P that’s been fairly bullet proof running 8.0.30u, but recently moved the core to an ICX7150-C12 on 8.0.95g to squeeze some longer runtime on my UPS. The 7150 had been working well for the first few days but locked up last night, with L2 traffic flowing through the switch but no L3 traffic passing across VEs and no response over the USB-C or the RJ45 serial consoles.

With no ability to access the switch via SSH or over serial, I pulled power and everything came back up just fine, but I’d like to figure out how to diagnose this if it happens again.

After reboot, I checked
Code:
sh log
but it looks as if the log was cleared on reboot.
 

AndroidCat

Member
Mar 3, 2015
32
24
8
Does anyone have any pointers on debugging / profiling an ICX switch?

I’ve had an ICX6450-48P that’s been fairly bullet proof running 8.0.30u, but recently moved the core to an ICX7150-C12 on 8.0.95g to squeeze some longer runtime on my UPS. The 7150 had been working well for the first few days but locked up last night, with L2 traffic flowing through the switch but no L3 traffic passing across VEs and no response over the USB-C or the RJ45 serial consoles.

With no ability to access the switch via SSH or over serial, I pulled power and everything came back up just fine, but I’d like to figure out how to diagnose this if it happens again.

After reboot, I checked
Code:
sh log
but it looks as if the log was cleared on reboot.
You need to stream your logs to external syslog server.
For instance:
Code:
logging host <IP>  udp-port 1514
logging enable rfc5424
 
  • Like
Reactions: Rttg

LodeRunner

Active Member
Apr 27, 2019
539
227
43
Code:
SSH@ICX6450-24P#sh med e 1/2/3
Port   1/2/3: Type  : 10GE SR 300m ((SFP+))
             Vendor: BROCADE            Version: A
             Part# : 57-0000075-01      Serial#: AAF210210000E3G
SSH@ICX6450-24P#sh med e 1/2/1
Port   1/2/1: Type  : 10GE SR 300m ((SFP+))
             Vendor: OEM                Version: 02
             Part# : SFP-10G-SR         Serial#: CSF101L34485
SSH@ICX6450-24P#sh med e 1/2/4
Port   1/2/4: Type  : 10GE SR 300m ((SFP+))
             Vendor: OEM                Version: 02
             Part# : SFP-10G-SR         Serial#: CSF101L34484
SSH@ICX6450-24P#show optic 1/2/3
Port  Temperature   Tx Power     Rx Power       Tx Bias Current
+----+-----------+--------------+--------------+---------------+
1/2/3   49.0507 C  -002.6114 dBm -002.3837 dBm    8.366 mA
        Normal      Normal        Normal         Normal

SSH@ICX6450-24P#show optic 1/2/1
SSH@ICX6450-24P#show optic 1/2/4
1/2/3 is Brocade SFP+ fiber. 1/2/1 and 1/2/4 are ipolex 10GBase-T RJ45 SFP+ copper. One copper soon to be removed for a passive DAC since I moved my OPNsense box right next to the switch.
Since 1/2/1 and 1/2/4 are RJ45 SFP+, then there will be no optic monitoring data for them either, Brocade or not. They're not optics. Also odd that 1/2/1 and 1/2/4 report themselves as 10G-SR; that's an optical designation, IIRC. And no 10G over CAT5/6 is going 300m, spec is 100m max.

I suspect if you go into conf t > int e 1/2/1 and issue whatever command enables optical monitoring, you'd get an error similar to my 7450:
Code:
SSH@core(config)#int e 1/3/1
SSH@core(config-lag-if-lg4)#optical-monitor 8
Port lg4 is 40G copper and cannot support Optical Monitoring feature.
 

grenskul

Active Member
Nov 8, 2020
131
60
28
Just got a 6450 . Trying to set up 2 lags of 2 10G ports each . Is there anyway I can see the speed of the lag (like in most other switches ? I do " show lag" and this is what I get I was expecting to see 20G somewhere.
Code:
Total number of LAGs:          3
Total number of deployed LAGs: 2
Total number of trunks created:2 (122 available)
LACP System Priority / ID:     1 / 748e.f8b8.86e0
LACP Long timeout:             120, default: 120
LACP Short timeout:            3, default: 3

=== LAG "desktop" ID 2 (dynamic Deployed) ===
LAG Configuration:
   Ports:         e 1/2/3 to 1/2/4
   Port Count:    2
   Primary Port:  1/2/4
   Trunk Type:    hash-based
   LACP Key:      20002
Deployment: HW Trunk ID 2
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/2/3      Down    None    None None  2     No  20   0   748e.f8b8.86e0
1/2/4      Down    None    None None  2     No  20   0   748e.f8b8.86e0

Port       [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/2/3           1        1   20002   Yes   S   Agg  Syn  No   No   Def  No   Dwn
1/2/4           1        1   20002   Yes   S   Agg  Syn  No   No   Def  No   Dwn


Partner Info and PDU Statistics
Port          Partner         Partner     LACP      LACP
             System ID         Key     Rx Count  Tx Count
1/2/3    1-0000.0000.0000       66       55      7079
1/2/4    1-0000.0000.0000       67       54      7078

=== LAG "unraid" ID 1 (dynamic Deployed) ===
LAG Configuration:
   Ports:         e 1/2/1 to 1/2/2
   Port Count:    2
   Primary Port:  1/2/1
   Trunk Type:    hash-based
   LACP Key:      20001
Deployment: HW Trunk ID 1
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/2/1      Up      Forward Full 10G   1     No  20   0   748e.f8b8.86e0
1/2/2      Up      Forward Full 10G   1     No  20   0   748e.f8b8.86e0

Port       [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/2/1           1        1   20001   Yes   L   Agg  Syn  Col  Dis  No   No   Ope
1/2/2           1        1   20001   Yes   L   Agg  Syn  Col  Dis  No   No   Ope


Partner Info and PDU Statistics
Port          Partner         Partner     LACP      LACP
             System ID         Key     Rx Count  Tx Count
1/2/1    65535-e435.c87e.3549       15      377      3417
1/2/2    65535-e435.c87e.3549       15      235      2910
 
Last edited:

klui

Well-Known Member
Feb 3, 2019
798
430
63
Newer versions show that, like 8.0.90. Maybe the feature you're looking for was added in 8.0.61 when they enhanced LAGs. See Terry Henry's YT channel for that.

The 6000 series have EOLed at 8.0.30.
 

LodeRunner

Active Member
Apr 27, 2019
539
227
43
Just got a 6450 . Trying to set up 2 lags of 2 10G ports each . Is there anyway I can see the speed of the lag (like in most other switches ? I do " show lag" and this is what I get I was expecting to see 20G somewhere.
[CODE
Total number of LAGs: 3
Total number of deployed LAGs: 2
Total number of trunks created:2 (122 available)
LACP System Priority / ID: 1 / 748e.f8b8.86e0
LACP Long timeout: 120, default: 120
LACP Short timeout: 3, default: 3

=== LAG "desktop" ID 2 (dynamic Deployed) ===
LAG Configuration:
Ports: e 1/2/3 to 1/2/4
Port Count: 2
Primary Port: 1/2/4
Trunk Type: hash-based
LACP Key: 20002
Deployment: HW Trunk ID 2
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/2/3 Down None None None 2 No 20 0 748e.f8b8.86e0
1/2/4 Down None None None 2 No 20 0 748e.f8b8.86e0

Port [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/2/3 1 1 20002 Yes S Agg Syn No No Def No Dwn
1/2/4 1 1 20002 Yes S Agg Syn No No Def No Dwn


Partner Info and PDU Statistics
Port Partner Partner LACP LACP
System ID Key Rx Count Tx Count
1/2/3 1-0000.0000.0000 66 55 7079
1/2/4 1-0000.0000.0000 67 54 7078

=== LAG "unraid" ID 1 (dynamic Deployed) ===
LAG Configuration:
Ports: e 1/2/1 to 1/2/2
Port Count: 2
Primary Port: 1/2/1
Trunk Type: hash-based
LACP Key: 20001
Deployment: HW Trunk ID 1
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/2/1 Up Forward Full 10G 1 No 20 0 748e.f8b8.86e0
1/2/2 Up Forward Full 10G 1 No 20 0 748e.f8b8.86e0

Port [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/2/1 1 1 20001 Yes L Agg Syn Col Dis No No Ope
1/2/2 1 1 20001 Yes L Agg Syn Col Dis No No Ope


Partner Info and PDU Statistics
Port Partner Partner LACP LACP
System ID Key Rx Count Tx Count
1/2/1 65535-e435.c87e.3549 15 377 3417
1/2/2 65535-e435.c87e.3549 15 235 2910




[/CODE]
“sh int br” shows link speeds I believe?
 

jayb998

New Member
Oct 31, 2022
9
5
3
Anyone running a 6450-24P with Noctuas? If so what kind of temps are you getting? Mine seems to constantly volley between ~58-64C... runs at fan speed 1, heats up, fan speed 2 cools it down, back to 1, repeat process. Like every 15 minutes or so. I only have 40W of POE draw which I consider to be a pretty light load considering this switch is rated for 370W.

I really like the noise level of the Noctuas (and was not thrilled with the Sunon KDEs I ordered previously) so I would like to keep the Noctuas if possible...

EDIT: Getting much better temps now with some network cabinet ventilation and rearranging devices in my rack. Still experimenting a bit.
 
Last edited:

LodeRunner

Active Member
Apr 27, 2019
539
227
43
Per port yes and I can see 10G on the ports that make up the lacp but nowhere have ai found the speed that actually makes up the lacp anywhere.
Maybe it's a newer thing then. When I do sh int br I get a list of all interfaces, including the LAGs:
Code:
lg4        Up      Forward Full 80G   4     Yes 1    0   cc4e.248b.3270
 

entertwined

New Member
Nov 12, 2022
1
1
1
I'm having issues updating my new 7250-24P to the latest 08095hufi.bin image in the download bundle. I followed the set up directions and got it running the non-UFI 08090mc.bin image, but when I try to run:

Code:
copy tftp flash <tftp server ip> ICX7xxx/SPR08095hufi.bin primary
I get the following output:

Code:
Load to buffer (8192 bytes per dot)
............................................
............................................
............................................
.............................................
<etc......>
...............................abort called
TFTP session timed out

Error in downloading bundle image

Error in processing bundle image
Oddly enough when I try to reflash the 08090mc.bin file via TFTP I get a slightly different error:

Code:
 !!! Downloading this application image can result in application-boot image mismatch. Please use UFI image.
Load to buffer (8192 bytes per dot)
.....................................
.....................................
......................................
etc......................abort called

TFTP session timed out

TFTP to Flash Error - code 5
My TFTP server logs show the correct file is being requested and at one point it worked to flash the original 08090mc.bin image, so I assume there is no configuration problem with the TFTP server or network. I've tried redownloading the firmware image a few times just in case it's some kind of file corruption. I've been running all this over serial, but I've also tried using SCP both to and from the switch per the instructions in the Ruckus 08.0.95 upgrade guide, but despite ironing out various issues with legacy SSH ciphers/options (I think) it still doesn't seem to work either. Very much appreciate any troubleshooting suggestions, I have a feeling I'm messing up something very simple but I can't figure out what it could be.


UPDATE: In case anyone else finds this, I just ended up flashing the firmware via USB. Still very confused why the TFTP flashing worked in the boot environment, but not when flashing a UFI image from within Fast Iron, I guess it must have been some network configuration change once the application was up and running.
 
Last edited:
  • Like
Reactions: Ibuytoomuchgadgets

Craig Curtin

Member
Jun 18, 2017
101
20
18
59
NOW... to build on this question a bit, two things:
  1. If I want to add a different VLAN for a different device port, do I essentially repeat the same steps? Do I need to re-add 1/2/1 to dual-mode after tagging it to a new VLAN, or is that part done?
  2. What if I want a hypervisor or a downstream switch or a WAP that passes traffic on multiple VLANs? (I have a Proxmox server and a couple of small Unifi switches.) Do I tag the device port with every VLAN it might conceivably use and put it in dual-mode/untagged on VLAN 1?
Thanks so much for the help!
1) Yes just repeat the steps i listed for additional ports and leave out the dual mode unless you need the VLAN 1 support
2) Yes if you have a "trunk" (Cisco namng) style device with multiple VLANs - just treat it like the Unifi as well - remembering that the opposite of cisco you add ports to VLANs not VLANs to ports.

Craig
 
  • Like
Reactions: jayb998

Craig Curtin

Member
Jun 18, 2017
101
20
18
59
I had exactly the same problem as you.
Try to add the line to each physical interface:
Code:
no spanning-tree
e.g.
interface ethernet 1/2/1
port-name xxxx
no spanning-tree
Let me know if that helps.

The explanation is somewhere within this thread. I think it was about spanning tree being active per port even though it is globally disabled.
At least it helped in my case.
Just trying this now - at first glance - just adding it to each of the 1/2/x ports does not appear to have made any difference - just migrating some VMs now so i can restart the Host

Craig
 

jayb998

New Member
Oct 31, 2022
9
5
3
Kudos for the assistance Craig. Everything you outlined above has worked like a charm. I was able to add 1/2/1 to the "new" VLANs and didn't need to re-add to dual-mode nor did I lose connection via SSH during the process. Setting up new downstream ports to my WAPs, switches and hypervisor was even easier.

Once I'm feeling brave enough I may dabble into L3 routing, but the UDM SE is working well enough right now as router-on-a-stick so no urgent need. Nice to know that the L3 is available if I grow into it in the future.