Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[Vesalius]

ok so i got through the updating and licensing,
at what point can i disconnect the serial cable and configure through web interface?

Web interface is not good (convoluted, poorly played out, has many options hidden or options not applicable to a specific switch) or reliable and really should not be counted on. Also little to no support can be found online from the company or here/reddit type users.

But you should be able to get to the point where you can ssh into the switch from a computer on the network and disconnect the serial cable.

 

[BoGs]

Web interface is not good (convoluted, poorly played out, has many options hidden or options not applicable to a specific switch) or reliable and really should not be counted on. Also little to no support can be found online from the company or here/reddit type users.

But you should be able to get to the point where you can ssh into the switch from a computer on the network and disconnect the serial cable.

This was mentioned, this is not an easyish switch. If all you want a L2 with SFP+ modules you are set and online. If you are looking for more advanced stuff you can read the documents in the zip or brocade has them online on what you want to do.

If you are looking for an easy UI a Unifi might be better but that is barely a L2+ switch, and I am trying to rip that crap out as the default support is "reset the switch" and coincidentally wipe out all vlan mapping and controller etc.

 

[86turbodsl]

poe board is bad, but it will still boot with it installed. the 6610s just take a while to boot

Got a replacement 6610 in and updated everything thru the guide. Works perfectly and got my first 10Gb connection on it. Thanks so much for this awesome resource fohdeesha!

 

[BossHoss]

Default Route & DNS
If you want your switch to be able to contact NTP servers for time synchronization, remote SNMP servers, etc, we need to give the switch a default route and a DNS server. Replace the IP with the IP of your gateway/router/etc. Assuming you are still at the configure terminal level:

ip dns server-address 192.168.1.1
ip route 0.0.0.0/0 192.168.1.1

at this point if my default gateway from isp is set to 10.0.0.1, the 7250 set to 10.0.0.202/24 my primary dns is set to 75.75.75.75
how would i want to enter that?

ip dns server-address 10.0.0.1
ip route 10.0.0.202/24 10.0.0.1

?

 

[frameshift18]

Replace the 192.168.1.1 IP with the IP of your gateway/router/etc.

 

[jei]

Tested the power consumption of a couple of 40Gb modules mentioned in this thread.

PMD-USB measures only the PCIE slot and does not account for PSU losses before it.

The modules were tested without active link / load but had LC UPC Duplex OM3 -cable physically connected.

Gear used:
- ElmorLabs PMD-USB Power Measurement Board
- ElmorLabs PMD-AD PCIE Power Measurement Board
- Supermicro X10SLL-F booted into Ubuntu 24.04 installation media desktop

NICs tested:
- Mellanox ConnectX-4 EN 50GbE single-port QSFP28 (MCX413A-GCAT)
- Intel X710-DA2 (mfg 10/2022)

Results:

	Voltage	Current	Power
MCX413A-GCAT only	11.94 V	0.5 A	5.97 W
MCX413A-GCAT with KAIAM XQX2502	11.94 V	0.6 A	7.164 W
MCX413A-GCAT with Juniper JNP-QSFP-40G-LX4	11.93 V	0.9 A	10.737 W
Intel X710-DA2 only	12 V	0.1 A	1.2 W
Chelsio T580-LP-CR only	11.95 V	0.8 A	9.56 W
Intel X710-T2L only	12 V	0.2 A	2.4 W
Intel X710-T2L with 1x 10Gb copper connected	11.98 V	0.3 A	3.594 W

edit: Added Intel X710-DA2
edit: Added Chelsio T580-LP-CR, Intel X710-T2L

Added Intel X710-DA2, Chelsio T580-LP-CR, Intel X710-T2L

 

[Craig Curtin]

Hey Guys,

Need to make some remote changes to a pair of 6450s that are joined by a single 10GB link.

They are remote to me and no on onsite has a lot of technical ability (maybe as far as pull a power cord etc) They are being used in a Ski lodge which is about 6 hours drive away from me.

They have essentially been setup as the default single vlan (1) with all ports in that VLAN (it was migrated from some older switches and they thought it was simplest to just duplicate the same setup)

We now wish to implement VLANs - the first part of which will be to put both the 10GB ports (at each end) into a new VLAN and have VLAN 1 as dual mode on this interface

Is there a batch way of sending commands to the switches - i have remote access to both - but the PC i am on is only connected to a single switch - so i need to be able to send commands in a batch to the "far" switch and have them execute and then configure the switch the PC is connected to to recognize the new dual mode port

I do have a PC on the 2nd switch - but it is only accessible by jumping across the 10GB link

SO need to attach to it first (windows) - then SSH into the switch, send the batch commands and have them complete and then do the same on the 1st switch from another PC

Ideas ?

Craig

 

[jode]

Ideas ?

If I understand you correctly, you have two 6450 switches in your LAN and they are connected via a 10gb link.

To simplify configuration and keep configurations in sync between these two switches the easiest way is to "stack" them. Once stacked, they work in unison and when connected to one switch you have control over the other as well (e.g. configure ports of the other switch).

Assuming you downloaded the package from the first post look for "fastiron-08030b-switchstackingguide.pdf" for the documentation of this feature.

 

[logixworx]

The manual for Brocade 6430/6450 has instructions for wall mounting the 6430, but says that the 6450p cannot be wall mounted. Is there a specific reason why this is so? Weight? convection? I need to vertically mount my 6450p using a common 4u wall mounted rack that can hold a 150 lbs load. Will i have any issues?

 

[NablaSquaredG]

The manual for Brocade 6430/6450 has instructions for wall mounting the 6430, but says that the 6450p cannot be wall mounted. Is there a specific reason why this is so? Weight? convection? I need to vertically mount my 6450p using a common 4u wall mounted rack that can hold a 150 lbs load. Will i have any issues?

I don't think you'll have any issues. The rack mount angles are pretty sturdy and you don't really have any leverage when mounting vertically

 

[macboy80]

I have just received my first ICX 6450-48P. I'm fairly savvy with tinkering, however I cannot get the serial connection to come up. My worry is that something more sinister is afoot. When powering on, the fans come up, very loud like full speed, and never go down, however the only lights that come on are the 4 X1-X4 lights. They stay solid amber for more than 5 minutes, and the PWR / DIAG lights never come on. I do get blinking green link lights on the management port as well.

Does anyone have an idea? Apologies in advance if this has been covered. I can't figure out how to search through a almost 500 page thread.

 

[NablaSquaredG]

When powering on, the fans come up, very loud like full speed, and never go down,

At some point, they should spin down. Is it really "never" or how long have you waited?

however I cannot get the serial connection to come up.

Just a quick sanity check:
You are connecting your Cisco-Style Console cable to the top port on the left, not the bottom one?

 

[macboy80]

At some point, they should spin down. Is it really "never" or how long have you waited?

I left it running as I researched, first that the usb-serial drivers were updated and then as I tried to find information on the power light not coming up. It was certainly more than 5 mins and probably more than 10 mins.

Just a quick sanity check:
You are connecting your Cisco-Style Console cable to the top port on the left, not the bottom one?

Yes. The cable is the one that came with the unit from the seller. A grey, Foxconn cable with RJ-45 on both ends and plugged in to a DB9 connector on one side. It is the one that is seen in many for-sale listings on the web. I am using this SABRENT USB 2.0 to Serial (9 Pin) DB 9 RS 232 Converter Cable, Prolific Chipset with Putty.

I was hoping some of you may have seen this behavior and could offer some assurance before went down the path of learning how to test this in Linux.

What I am most unsure of is:
1. The lack of PWR / DIAG lights, with it seemingly hung on the X1-4 lights. (Brocade docs seem to say call support if you don't have a PWR light.)
2. The amount of time it should take to show serial output / control the fan speed. (On a PC, full fans are usually NO POST.)
3. What kind of weird Win11 / driver options may be interfering with serial cable. (In the advanced properties, I'm seeing fifo and modem options.)

 

[JD7]

I left it running as I researched, first that the usb-serial drivers were updated and then as I tried to find information on the power light not coming up. It was certainly more than 5 mins and probably more than 10 mins.


Yes. The cable is the one that came with the unit from the seller. A grey, Foxconn cable with RJ-45 on both ends and plugged in to a DB9 connector on one side. It is the one that is seen in many for-sale listings on the web. I am using this SABRENT USB 2.0 to Serial (9 Pin) DB 9 RS 232 Converter Cable, Prolific Chipset with Putty.

I was hoping some of you may have seen this behavior and could offer some assurance before went down the path of learning how to test this in Linux.

What I am most unsure of is:
1. The lack of PWR / DIAG lights, with it seemingly hung on the X1-4 lights. (Brocade docs seem to say call support if you don't have a PWR light.)
2. The amount of time it should take to show serial output / control the fan speed. (On a PC, full fans are usually NO POST.)
3. What kind of weird Win11 / driver options may be interfering with serial cable. (In the advanced properties, I'm seeing fifo and modem options.)

I can test it tomorrow when I get home but 5mins at the most for it to show on the screen however if you have never used that adaptor or cable to get into any other switches before it might be your drivers and cable I use a win10 machine to get into my stuff. if its not configured your not gonna see a lot of lights until you enable some stuff like POE and such or give it a live connection to your network, it can still be running full fans and post but should slow down after its finished.

 

[fohdeesha]

I left it running as I researched, first that the usb-serial drivers were updated and then as I tried to find information on the power light not coming up. It was certainly more than 5 mins and probably more than 10 mins.


Yes. The cable is the one that came with the unit from the seller. A grey, Foxconn cable with RJ-45 on both ends and plugged in to a DB9 connector on one side. It is the one that is seen in many for-sale listings on the web. I am using this SABRENT USB 2.0 to Serial (9 Pin) DB 9 RS 232 Converter Cable, Prolific Chipset with Putty.

I was hoping some of you may have seen this behavior and could offer some assurance before went down the path of learning how to test this in Linux.

What I am most unsure of is:
1. The lack of PWR / DIAG lights, with it seemingly hung on the X1-4 lights. (Brocade docs seem to say call support if you don't have a PWR light.)
2. The amount of time it should take to show serial output / control the fan speed. (On a PC, full fans are usually NO POST.)
3. What kind of weird Win11 / driver options may be interfering with serial cable. (In the advanced properties, I'm seeing fifo and modem options.)

yeah that sounds dead

 

[macboy80]

Thank you for the replies. I have a different usb to serial cable coming and am going to try with Linux as well. I will advise when I know more. In the meantime, if anyone has any experience with the boot up indicator light pattern, I'd love to know what it should look like it.

Update / Edit: I've now used a FTDI and Prolific serial cable on both Windows and Ubuntu. No serial output, the continued odd boot indications, and your advice have led me to return the switch as faulty.

Update 2: I bought another one, and everything worked just as expected per @fohdeesha excellent guide. FYI. While in the bootloader, the fans do spin at 100%, and the lights are just like my broken switch with no PWR / DIAG and X1-4 lit amber. All that changes as expected as soon as the boot process starts.

 

[Matt0925]

Full disclaimer, I'm very new to home networking, managed switches and especially level 3 switches. So I have no idea if this is normal behavior, I have something configured wrong or if my switch has an issue.

I purchased a 7250 to replace an Aruba 2500. Followed the guide to get it up and running, place it into service. Runs for a while, then start having disconnects across the network. Oddly enough PC shows still connected but no Internet. Then it'll completely disconnect. Running OPNSense for a router, at the same time it started having errors. I disconnected the 7250, completely restarted the router and errors went away. They seemed to be an IP config issue?

Tonight I tried to reintroduced the 7250. I set it to run as level 2 since I don't know anything about level 3. Ran fine about 45 mins. Then disconnects started again. I bypassed the 7250, started looking into what it could be, checking the log I did get this-

May 13 23:08:15:I:System: Interface ethernet 1/2/8, state down
May 13 23:06:49:I:System: Interface ethernet 1/1/22, state down
May 13 23:06:40:I:System: Interface ethernet 1/1/29, state down
May 13 23:06:31:I:System: Interface ethernet 1/2/5, state down
May 13 23:06:27:I:System: Interface ethernet 1/2/7, state down
May 13 22:19:08:I:System: Interface ethernet 1/2/5, state up
May 13 22:18:59:I:System: Interface ethernet 1/2/5, state down
May 13 22:16:55:I:NTP: The system clock is not synchronized to any time source.
May 13 22:10:01:I:System: Interface ethernet 1/2/7, state up
May 13 22:09:57:I:System: Interface ethernet 1/2/7, state down
May 13 22:09:32:I:System: Interface ethernet 1/2/7, state up
May 13 22:09:27:I:System: Interface ethernet 1/2/7, state down
May 13 22:09:18:I:System: Interface ethernet 1/2/7, state up
May 13 22:09:01:I:System: Interface ethernet 1/2/7, state down
May 13 22:09:01:I:System: Interface ethernet 1/2/7, state up
May 13 22:08:49:I:System: Interface ethernet 1/2/7, state down

So ports appear to be up and down for some reason. I'm going to assume this is not typical behavior. Not sure if I've misconfigured something or if it's a hardware issue? Otherwise I have no idea why I'm getting constant disconnects

 

[TonyArrr]

Full disclaimer, I'm very new to home networking, managed switches and especially level 3 switches. So I have no idea if this is normal behavior, I have something configured wrong or if my switch has an issue.

I purchased a 7250 to replace an Aruba 2500. Followed the guide to get it up and running, place it into service. Runs for a while, then start having disconnects across the network. Oddly enough PC shows still connected but no Internet. Then it'll completely disconnect. Running OPNSense for a router, at the same time it started having errors. I disconnected the 7250, completely restarted the router and errors went away. They seemed to be an IP config issue?

Tonight I tried to reintroduced the 7250. I set it to run as level 2 since I don't know anything about level 3. Ran fine about 45 mins. Then disconnects started again. I bypassed the 7250, started looking into what it could be, checking the log I did get this-

May 13 23:08:15:I:System: Interface ethernet 1/2/8, state down
May 13 23:06:49:I:System: Interface ethernet 1/1/22, state down
May 13 23:06:40:I:System: Interface ethernet 1/1/29, state down
May 13 23:06:31:I:System: Interface ethernet 1/2/5, state down
May 13 23:06:27:I:System: Interface ethernet 1/2/7, state down
May 13 22:19:08:I:System: Interface ethernet 1/2/5, state up
May 13 22:18:59:I:System: Interface ethernet 1/2/5, state down
May 13 22:16:55:I:NTP: The system clock is not synchronized to any time source.
May 13 22:10:01:I:System: Interface ethernet 1/2/7, state up
May 13 22:09:57:I:System: Interface ethernet 1/2/7, state down
May 13 22:09:32:I:System: Interface ethernet 1/2/7, state up
May 13 22:09:27:I:System: Interface ethernet 1/2/7, state down
May 13 22:09:18:I:System: Interface ethernet 1/2/7, state up
May 13 22:09:01:I:System: Interface ethernet 1/2/7, state down
May 13 22:09:01:I:System: Interface ethernet 1/2/7, state up
May 13 22:08:49:I:System: Interface ethernet 1/2/7, state down

So ports appear to be up and down for some reason. I'm going to assume this is not typical behavior. Not sure if I've misconfigured something or if it's a hardware issue? Otherwise I have no idea why I'm getting constant disconnects

Could be helpful to share your config from the switch so folks can look over it and point out anything that might cause problems.
What is connected to what port?

 

[Matt0925]

Could be helpful to share your config from the switch so folks can look over it and point out anything that might cause problems.
What is connected to what port?

Sorry, probably should have thought of that. Config-

SSH@brocade(config)#show config!Startup-config data location is flash memory - Pastebin.com

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

pastebin.com

As for ports-

1/2/5- unRaid connected through Amazon DAC cable
1/2/7- OPNSense connected through ethernet using SPF+ to RJ45 transceiver
1/2/8- PC connected through ethernet using SPF+ to RJ45 transceiver

The one gig ports had a the backup unRaid connection, a UPS and a Nvidia Shield.

 

[fohdeesha]

Sorry, probably should have thought of that. Config-

SSH@brocade(config)#show config!Startup-config data location is flash memory - Pastebin.com

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

pastebin.com

As for ports-

1/2/5- unRaid connected through Amazon DAC cable
1/2/7- OPNSense connected through ethernet using SPF+ to RJ45 transceiver
1/2/8- PC connected through ethernet using SPF+ to RJ45 transceiver

The one gig ports had a the backup unRaid connection, a UPS and a Nvidia Shield.

looks fine to me, altho the "ip route 192.168.1.0/24 192.168.1.1" statement is redundant as you already have a default pointing to the same IP. that much flapping on every single port really looks like a hardware fault, can you unplug one connection at a time and see if it goes away? this includes the whole SFP+ copper module, maybe one of them is bad and friggin things up