Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
B

ben255

New Member
Feb 24, 2021
1
1
3
Picked up a 7150-24p and got it updated and licensed using your guide! Thanks!

I also managed to get Brocade Network Advisor installed on linux (centos 7). Turns out that all the config files/scripts were written on a Windows system with DOS line endings, which linux does NOT like. Luckily there is a handy command to convert them.

Run install.bin, but when you reach the end, choose not to run the Configuration Wizard. Once that is done, install dos2unix from the standard repo (yum install dos2unix). Next, cd to /opt/Network_Advisor_IP14_2_12 and run "find . -type f -print0 | xargs -0 dos2unix" which will search through the entire directory and convert all text files to the correct Unix format.

Once that completes, run "bin/configwizard" and follow the instructions to configure BNA!

If you want to run it as a server, you'll need to modify the firewall to allow access.
 
  • Like
Reactions: Jason Antes
M

MassiveMeatMissile

New Member
Aug 22, 2020
3
1
3
When I got home today my entire network was down and my ICX 6610 had the fans at 100%, so I unplugged and replugged it into power; the fans still stuck at full bore, and the activity lights on the SFP ports are solid green. I cannot see it on the network anymore and I tried using a console cable but it's no longer working either.

Anyone have any troubleshooting suggestions? I feel like it may just be a noisy brick now.
 
L

LodeRunner

Active Member
Apr 27, 2019
554
236
43
MassiveMeatMissile said:
When I got home today my entire network was down and my ICX 6610 had the fans at 100%, so I unplugged and replugged it into power; the fans still stuck at full bore, and the activity lights on the SFP ports are solid green. I cannot see it on the network anymore and I tried using a console cable but it's no longer working either.

Anyone have any troubleshooting suggestions? I feel like it may just be a noisy brick now.
Click to expand...
If you power cycle it with console cable active, do you even see the boot loader?
 
D

deeceesth

New Member
Jul 30, 2021
17
4
3
I tried searching this thread for ideas on expanding sfp+ ports on these switches. Would a cheap L2 only SFP+ switch uplinked to a icx7250 do the trick or is there an affordable brocade option for getting more sfp+ ports?
 
N

neodied

New Member
Aug 1, 2021
2
3
3
I am having issues with the 40G breakout ports, where only the 4th port in each group (1/2/5 and 1/2/10) will link when using a QSFP-to-4xSFP+ breakout DAC (specific cable is a FS Brocade 40G-QSFP-4SFP-C-0201).

My Setup:
Switch - ICX 6610-48P with licenses, breakout QSFP plugged into XL2-5
Server 1 - Intel X520 PCIe NIC - ports 1 and 2 connected to breakout SFP+ transceivers 1 and 2 (1/2/2 and 1/2/3)
Server 2 - Intel 82599ES daughter card - ports 1 and 2 connected to breakout SFP+ transceivers 3 and 4 (1/2/4 and 1/2/5)

The successful link follows the switch port (changing the order of transceivers in a NIC, and trying different NICs results in 1/2/5 always linking). Plugging only one SFP+ breakout at a time into a NIC does not change the behavior, only the 1/2/5 or 1/2/10 ports will link up. Using a new cable (of the same model) does not resolve this, using XL7-10 (the second QSFP breakout port) also does not resolve (only 1/2/10 links up).

Any help figuring this out would be appreciated!

show interface brief ethernet 1/2/1 to 1/2/5
Code: 
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC         Name
1/2/1      Up      Forward Full 40G   13    Yes N/A  0   redacted    redacted
1/2/2      Down    None    None None  None  No  1    0   redacted    redacted
1/2/3      Down    None    None None  None  No  1    0   redacted    redacted
1/2/4      Down    None    None None  None  No  1    0   redacted    redacted
1/2/5      Up      Forward Full 10G   None  No  1    0   redacted
show stack
Code: 
***** Warning! stack is not enabled. *****

T=22d20h53m31.8: alone: standalone, D: dynamic cfg, S: static
ID   Type          Role    Mac Address    Pri State   Comment
1  S ICX6610-48P   alone   redacted         0 local   None:0


     +---+
  2/1| 1 |2/6
     +---+
Current stack management MAC is redacted
show run
Code: 
stack unit 1
  module 1 icx6610-48p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
ETA: I did a fair bit of googling and keyword searching this thread and found the combined stacking + breakout config instructions, but my setup is not stacking, just 100% data. The XL1 and XL6 ports are connected directly to a 40G NIC in another box.
 
Last edited:
J

Jason Antes

Active Member
Feb 28, 2020
230
78
28
Twin Cities
ben255 said:
Picked up a 7150-24p and got it updated and licensed using your guide! Thanks!

I also managed to get Brocade Network Advisor installed on linux (centos 7). Turns out that all the config files/scripts were written on a Windows system with DOS line endings, which linux does NOT like. Luckily there is a handy command to convert them.

Run install.bin, but when you reach the end, choose not to run the Configuration Wizard. Once that is done, install dos2unix from the standard repo (yum install dos2unix). Next, cd to /opt/Network_Advisor_IP14_2_12 and run "find . -type f -print0 | xargs -0 dos2unix" which will search through the entire directory and convert all text files to the correct Unix format.

Once that completes, run "bin/configwizard" and follow the instructions to configure BNA!

If you want to run it as a server, you'll need to modify the firewall to allow access.
Click to expand...
Should make that a VM template and get it uploaded. ;)
 
J

Jason Antes

Active Member
Feb 28, 2020
230
78
28
Twin Cities
deeceesth said:
I tried searching this thread for ideas on expanding sfp+ ports on these switches. Would a cheap L2 only SFP+ switch uplinked to a icx7250 do the trick or is there an affordable brocade option for getting more sfp+ ports?
Click to expand...
You could by any of several of the ICX line that are fairly cheap to do that.
 
A

anemoiac

New Member
Jan 7, 2021
25
13
3
FozzieBear said:
For my 7250-24P, I started with replacing the rear fans with 2x MF40201V3-1000U-G99. I found that the fans were near silent when running at speed 1, but it would intermittently kick up into speed 2 when the ASIC hit a high enough temp. This sits in a rack right next to me, so I added 1x MF60101V3-1000U-A99 that sits directly on top of the ASIC heatsink and is tapped into the constant 12v to the mainboard, and it never kicks into speed 2 now, even sitting in my rack that runs ~80F inside.
Click to expand...
Would you mind sharing how you secured the 3rd fan on top of the ASIC heatsink? I'm looking to do something similar, but am unsure of the best way to more forward.
 
P

Propaganda

Active Member
Dec 6, 2017
154
62
28
45
I am green to switch stacking and have some probably stupid questions here. I have been reading the stacking documentation for 08.0.90(k) firmware running on 2x 7250. I just want to make sure I am understanding correctly because I haven't seen definitive answers. I am using a single 10G port on each switch 1/2/1 and 2/2/1 for stacking, do I need to make a trunk or can I just use a single port on each like this? Can I reclaim the use of 1/2/3 and 2/2/3 as a data ports? Do I need to provide additional uplink ports between the switches or is all data transferred between using the stack ports?

Code: 
ICX7250-24P Router>show stack
T=19h39m5.5: alone: standalone, D: dynamic cfg, S: static
ID   Type          Role    Mac Address    Pri State   Comment
1  S ICX7250-24P   active  mmmm.aaaa.cccc   0 local   Ready
2  S ICX7250-24    standby mmmm.aaaa.cccc   0 remote  Ready

    active       standby
     +---+        +---+
  2/3| 1 |2/1--2/1| 2 |2/3
     +---+        +---+
Standby u2 - sent role but not acked
Role history: N: standalone, A: active, S: standby, M: member
U1: N->A, U2: M->S->N->M->S
 
Last edited:
Spearfoot

Spearfoot

Active Member
Apr 22, 2015
111
51
28
neodied said:
I am having issues with the 40G breakout ports, where only the 4th port in each group (1/2/5 and 1/2/10) will link when using a QSFP-to-4xSFP+ breakout DAC (specific cable is a FS Brocade 40G-QSFP-4SFP-C-0201).

My Setup:
Switch - ICX 6610-48P with licenses, breakout QSFP plugged into XL2-5
Server 1 - Intel X520 PCIe NIC - ports 1 and 2 connected to breakout SFP+ transceivers 1 and 2 (1/2/2 and 1/2/3)
Server 2 - Intel 82599ES daughter card - ports 1 and 2 connected to breakout SFP+ transceivers 3 and 4 (1/2/4 and 1/2/5)

The successful link follows the switch port (changing the order of transceivers in a NIC, and trying different NICs results in 1/2/5 always linking). Plugging only one SFP+ breakout at a time into a NIC does not change the behavior, only the 1/2/5 or 1/2/10 ports will link up. Using a new cable (of the same model) does not resolve this, using XL7-10 (the second QSFP breakout port) also does not resolve (only 1/2/10 links up).

Any help figuring this out would be appreciated!

show interface brief ethernet 1/2/1 to 1/2/5
Code: 
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC         Name
1/2/1      Up      Forward Full 40G   13    Yes N/A  0   redacted    redacted
1/2/2      Down    None    None None  None  No  1    0   redacted    redacted
1/2/3      Down    None    None None  None  No  1    0   redacted    redacted
1/2/4      Down    None    None None  None  No  1    0   redacted    redacted
1/2/5      Up      Forward Full 10G   None  No  1    0   redacted
show stack
Code: 
***** Warning! stack is not enabled. *****

T=22d20h53m31.8: alone: standalone, D: dynamic cfg, S: static
ID   Type          Role    Mac Address    Pri State   Comment
1  S ICX6610-48P   alone   redacted         0 local   None:0


     +---+
  2/1| 1 |2/6
     +---+
Current stack management MAC is redacted
show run
Code: 
stack unit 1
  module 1 icx6610-48p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
ETA: I did a fair bit of googling and keyword searching this thread and found the combined stacking + breakout config instructions, but my setup is not stacking, just 100% data. The XL1 and XL6 ports are connected directly to a 40G NIC in another box.
Click to expand...
I don't know if it makes a difference... but do you have 'stack disable' in your setup? Like this:
Code: 
stack unit 1
  module 1 icx6610-48p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
stack disable
 
  • Like
Reactions: neodied
fohdeesha

fohdeesha

Kaini Industries
Nov 20, 2016
2,919
3,444
113
34
fohdeesha.com
MassiveMeatMissile said:
When I got home today my entire network was down and my ICX 6610 had the fans at 100%, so I unplugged and replugged it into power; the fans still stuck at full bore, and the activity lights on the SFP ports are solid green. I cannot see it on the network anymore and I tried using a console cable but it's no longer working either.

Anyone have any troubleshooting suggestions? I feel like it may just be a noisy brick now.
Click to expand...
That's odd, sounds like it is indeed bricked, which in my experience is usually a bad/incomplete bootloader, but it typically doesn't happen for no reason - could be a bad bootloader flash/upgrade from a while back and it just hadn't been rebooted yet or similar. Depending on how invested you are and how close to indiana you are, I might be able to mail you my jtag unit if you want to flash a fresh bootloader via jtag and see if that's the issue
 
N

neodied

New Member
Aug 1, 2021
2
3
3
Spearfoot said:
I don't know if it makes a difference... but do you have 'stack disable' in your setup? Like this:
Code: 
stack unit 1
  module 1 icx6610-48p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
stack disable
Click to expand...
You are a lifesaver! Adding that, then a reboot solved the issue, now I'm getting link on all 4 QSFP breakout ports.
 
  • Like
Reactions: artbird309, Spearfoot and fohdeesha
M

MassiveMeatMissile

New Member
Aug 22, 2020
3
1
3
fohdeesha said:
That's odd, sounds like it is indeed bricked, which in my experience is usually a bad/incomplete bootloader, but it typically doesn't happen for no reason - could be a bad bootloader flash/upgrade from a while back and it just hadn't been rebooted yet or similar. Depending on how invested you are and how close to indiana you are, I might be able to mail you my jtag unit if you want to flash a fresh bootloader via jtag and see if that's the issue
Click to expand...
I appreciate the offer but I've decided to buy a new one already.
 
  • Like
Reactions: fohdeesha
O

OKGolombRuler

New Member
Mar 13, 2020
22
6
3
7150-c12p, MOCA, and Campus Fabric-- oh, my!

Hi folks--

First off, thanks to Fohdeesha and all the forum folks as a result of whose work and words I've learned so much (at least, failed so much less!). :)

Second, I've trawled this thread a fair bit but haven't seen much discussion of campus fabric, which is today's puzzle.

Requests:
  • Corrections for any of the below
  • Suggestions on smarter/alternative ways to do it? Stupid/crazy but practicable ideas welcomed.
  • Is this as simple as finding either a 7150 (to stack?) or a ... 7450? 7650? to serve as a CB?

Environment:
My house is rather hostile to 5ghz wifi and lacks ethernet and cell service but has cable TV jacks. Most of my network traffic is latency-sensitive, whether it's endless Zoom calls, VOIP/SIP traffic, SDR, etc. I do isolate IOT, guests, etc into VLANs.

I have a 6610 in the basement and a couple 7150-c12ps which I put in distant rooms, attached client devices and a POE AP, and home-run by way of individual pairs of MOCA bridges. And it's great, and mostly even works! but I've now got enough VLAN segmentation, firewall shenanigans, etc to want to manage the L2 switching fabric as one coherent unit. I'm tired of trunking vlans around on switch after switch each time I want to rewicker an experiment.

Obstacles:
  • The network topology is hub-and-spoke; there will be no redundant links here. (Which rules out Stacking but should work with Campus Fabric mode, I think?)
  • The 7150s will supposedly act as Port Extenders (which I read as akin to Cisco FEXes) if paired with a suitable (7150? 7x50?) Control Bridge, but my 6610 will not cut the mustard.
  • The 7150s do not support the 2.5gbps ethernet the MOCA bridges offer; so I'm leaving bandwidth on the table. (This is a modest annoyance, but I mention it in case my ignorance not silicon is the limiting factor).
  • It 'feels weird' to push every frame down a 1gbps link to the Control Bridge only to have it come right back up that same link. Maybe I misunderstand, and only frames destined for a different physical switch get pushed to the BC for processing? In which case they'd be traversing it anyway, no harm done

I'm still noodling on the 8.0.95 campus fabric guide, please feel free to point me to docs (or your own running examples!) if I missed anything.
 
L

LodeRunner

Active Member
Apr 27, 2019
554
236
43
I could be wrong, but I believe you're going to need a 7650, 7750, or 7850 to be a CB. My 7450, running 8.0.92e, certainly does not have the commands present to configure it as a CB.
 
  • Sad
Reactions: OKGolombRuler
You must log in or register to reply here.
Top Bottom