Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[linuxsrc]

what's the output of "show pod"

Spoiler: Show Pod

show pod

Unit-Id: 1
PoD 10G license capacity: 24
PoD 10G license capacity used: 24
PoD 40G license capacity: 24
PoD 40G license capacity used: 6

PoD-ports Lic-Available Lic-Used
1/1/33 Yes Yes
1/1/34 Yes Yes
1/1/35 Yes Yes
1/1/36 Yes Yes
1/1/37 Yes Yes
1/1/38 Yes Yes
1/1/39 Yes Yes
1/1/40 Yes Yes
1/1/41 Yes Yes
1/1/42 Yes Yes
1/1/43 Yes Yes
1/1/44 Yes Yes
1/1/45 Yes Yes
1/1/46 Yes Yes
1/1/47 Yes Yes
1/1/48 Yes Yes
1/1/49 Yes Yes
1/1/50 Yes Yes
1/1/51 Yes Yes
1/1/52 Yes Yes
1/1/53 Yes Yes
1/1/54 Yes Yes
1/1/55 Yes Yes
1/1/56 Yes Yes
1/2/1 Yes Yes
1/2/2 Yes Yes
1/2/3 Yes Yes
1/2/4 Yes Yes
1/3/1 Yes Yes
1/3/2 Yes Yes
1/3/3 Yes Yes
1/3/4 Yes Yes
1/3/5 Yes Yes
1/3/6 Yes Yes
1/3/7 Yes Yes
1/3/8 Yes Yes

What is strange about this port, normally you see an insertion in the logs when a cable is inserted or removed. I have turned debug on via debug port. And usually, I will see

2023 Nov 21 14:05:19:I:System: Interface ethernet 1/1/3, state up
2023 Nov 21 14:05:18:I:System: Interface ethernet 1/1/3, state down

But these 2 ports on the ICX 1/2/3 and 1/2/4 don't do that

It acts like the are no changes at all.

I just put a breakout on the 1/3/1 and it links up fine and shows LLDP data on the port. So I don't think it is a backplane issue.
My next test will be to get a server connected to that port.

I have tested this now with a dual-port ConenctX-3 Pro. I plugged one into the Mellanox switch, works fine and passes data. The other to the ICX switch and nothing. It does the same, detects the media but does not start the interface.

 

[notdeadyet]

upgrading from 30t -> 30u solved the problem. Hat tip to a post I found on reddit here: https://www.reddit.com/r/homelab/comments/eyvny7 I sent the guy a pm and he let me know 30u solved the issue. Flashed that, issue fixed.

@Juggie I know that was a very old post of yours, but do you think you could share your switches config? I am really struggling to get multicast working on my network and I've tried everything.

 

[Andydude]

Wanted to plug my post on my fan changes to a 7250-48p.

https://forums.servethehome.com/index.php?threads/brocade-icx-series-cheap-powerful-10gbe-40gbe-switching.21107/page-159#post-256375

Hey RoachedCoach, big fan of your fan mod and the stock fans definitely are not passing my wife-test noise-wise ("is it always that loud??" after the fans spun down).

I can get a Sunon MF60101V3-1000U-A99 ok but I'm having no luck at all getting the Mechatronics MR4020X12B1-RSR in the UK and the digikey.com US site is also saying 0 in stock so maybe there is a stock issue or they're discontinuing them...?

Do you or anyone else know a replacement you can recommend that is similarly quiet and monitorable that I can source in the UK...?

Thanks!

 

[Ralph_IT]

Hey RoachedCoach, big fan of your fan mod and the stock fans definitely are not passing my wife-test noise-wise ("is it always that loud??" after the fans spun down).

I can get a Sunon MF60101V3-1000U-A99 ok but I'm having no luck at all getting the Mechatronics MR4020X12B1-RSR in the UK and the digikey.com US site is also saying 0 in stock so maybe there is a stock issue or they're discontinuing them...?

Do you or anyone else know a replacement you can recommend that is similarly quiet and monitorable that I can source in the UK...?

Thanks!

I, too, tried to follow @RoachedCoach mod but was not able to get the recommended Mechatronics.
At Mouser you cannot buy them if you are not a reseller and Digikey wanted 100€ + VAT + custom fees for 3 of them some weeks ago.
I bought the MF60101V3-1000U-A99 and 3 x MF40201VX-G99-A which are detected by the switch and keep it reasonably cool, but there are some drawbacks, at least for me.
The MF4020 "resonate" if there are 2 or more installed. I read @rootwyrm talking about this issue previously on this very thread, but don't know if he was talking about this model or just about all Sunon fans.
I wouldn't say the switch is quiet, but it's beareable.
If I can source some Mechatronics I'll try them to see if I can make it quieter.

 

[shpitz461]

I, too, tried to follow @RoachedCoach mod but was not able to get the recommended Mechatronics.
At Mouser you cannot buy them if you are not a reseller and Digikey wanted 100€ + VAT + custom fees for 3 of them some weeks ago.

If you are in the UK, I found this guy on eBay located in the UK:

FAN AXIAL 40X20MM 12VDC WIRE | eBay

Fan Type: Tubeaxial. Features: Auto Restart Speed Sensor (Tach). Weight: 0.053 lb (24.04 g). Approvals: CE cUL TUV UL. Ingress Protection: -. RPM: 12000 RPM.

www.ebay.com

 

[Andydude]

I, too, tried to follow @RoachedCoach mod but was not able to get the recommended Mechatronics.
At Mouser you cannot buy them if you are not a reseller and Digikey wanted 100€ + VAT + custom fees for 3 of them some weeks ago.
I bought the MF60101V3-1000U-A99 and 3 x MF40201VX-G99-A which are detected by the switch and keep it reasonably cool, but there are some drawbacks, at least for me.
The MF4020 "resonate" if there are 2 or more installed. I read @rootwyrm talking about this issue previously on this very thread, but don't know if he was talking about this model or just about all Sunon fans.
I wouldn't say the switch is quiet, but it's beareable.
If I can source some Mechatronics I'll try them to see if I can make it quieter.

180 of them just came in stock today at Digikey so I've ordered 4x which worked out to be €43 delivered with free shipping and Digikey are paying custom fees(!) so we'll see what happens.

Thanks for the recommendation!

 

[ivans89]

I ordered them too from digikey, but 5 to get free delivery with fedex.

 

[Andydude]

I ordered them too from digikey, but 5 to get free delivery with fedex.

Good luck, I'm putting 3x in a 7250-48 and 1x in a 6450-48.

Anyone know if just 1x MR4020X12B1-RSR in a 6450-48 will cool it ok? It's not PoE and I'm only probably plugging in 7-8 devices...

 

[Juggie]

@Juggie I know that was a very old post of yours, but do you think you could share your switches config? I am really struggling to get multicast working on my network and I've tried everything.

Multicast config is all default no changes. Once I upgraded the firmware it worked. What switch do you have? Does it have the latest firmware?

 

[Glynax]

For the Brocade ICX6610 is it possible to use all the rear ports as 40g? I'm guessing no but I thought I'd ask.

 

[Dave Corder]

For the Brocade ICX6610 is it possible to use all the rear ports as 40g? I'm guessing no but I thought I'd ask.

Nope

 

[bwahaha]

For the Brocade ICX6610 is it possible to use all the rear ports as 40g? I'm guessing no but I thought I'd ask.

no, two are 1x40g, two are 4x10g. You can LAG them together, but it'll still only be 10g.

 

[Gerhen]

I picked up a 7250-48 after finding this thread and reading through. Thanks to all who have contributed their time/insight here.

Curious to know how many are running 09.0.10x branch rather than the stable 08.0.95x branch ?

 

[diode]

Okay, finally got the switch in. I ended up getting an ICX7250.

So far I'm off to a rough start as I can't seem to get Windows to detect, and/or get the TFTP up and running.

This is the cable that I purchased; can anyone confirm if this is the correct cable and should work for connecting to the ICX7250?

RW RoutersWholesale USB 2.0 Console Cable Compatible/Replacement for Cisco A-Male to Mini-B Cord - 6 Feet (1.8 Meters)

I'm sure that this has been beaten to death throughout this thread......

I'm hoping that I didn't just fry the console port given what was mentioned in this thread post...

https://forums.servethehome.com/index.php?threads/brocade-icx-series-cheap-powerful-10gbe-40gbe-switching.21107/post-277688

Edit: I clearly see now that this is the WRONG cable. I guess we’ll see if I fried my port or not after making the new cable. Ugh.

I believe with all the information I conflated serial with console cable.

anyone know if the cable I linked is just a typical usb cable or could I have lucked out?

 

[Ralph_IT]

Okay, finally got the switch in. I ended up getting an ICX7250.

So far I'm off to a rough start as I can't seem to get Windows to detect, and/or get the TFTP up and running.

This is the cable that I purchased; can anyone confirm if this is the correct cable and should work for connecting to the ICX7250?

RW RoutersWholesale USB 2.0 Console Cable Compatible/Replacement for Cisco A-Male to Mini-B Cord - 6 Feet (1.8 Meters)

I'm sure that this has been beaten to death throughout this thread......

I'm hoping that I didn't just fry the console port given what was mentioned in this thread post...

https://forums.servethehome.com/index.php?threads/brocade-icx-series-cheap-powerful-10gbe-40gbe-switching.21107/post-277688

Edit: I clearly see now that this is the WRONG cable. I guess we’ll see if I fried my port or not after making the new cable. Ugh.

I believe with all the information I conflated serial with console cable.

anyone know if the cable I linked is just a typical usb cable or could I have lucked out?

The ICX7250 Mini-USB port is just a console port with Mini-USB form. It's not a built in console port as the ICX7150 series have.
As per frying the port, just a quote from @fohdeesha:

I would bet $50 the console port is fine, tons of people just in this thread have accidentally plugged regular USB ports into them, they can handle it fine. your issue is probably your usb to serial adapter, use it with another serial device to verify it actually works, or buy an ftdi model off amazon. also official console cables for these are only like 20 bucks to take the guesswork out of wiring Brocade ICX7250, ICX7450 ICX7750 DB9 Female to USB Mini Serial Console Cable 6FT | eBay

Just buy the Mini-USB adapter AND a USB to Serial Cable like this.
Both are dirty cheap.

Edit: Change the link to USB to Serial recommended.

 

[Andydude]

I picked up a 7250-48 after finding this thread and reading through. Thanks to all who have contributed their time/insight here.

Curious to know how many are running 09.0.10x branch rather than the stable 08.0.95x branch ?

I am and it seems fine to me, I just followed the guide to 8 and then sent the 9 file over afterwards. Just make sure you use the routing firmware file and not the switch one like I did at first...

 

[diode]

Okay, finally got the switch in. I ended up getting an ICX7250.

So far I'm off to a rough start as I can't seem to get Windows to detect, and/or get the TFTP up and running.

This is the cable that I purchased; can anyone confirm if this is the correct cable and should work for connecting to the ICX7250?

RW RoutersWholesale USB 2.0 Console Cable Compatible/Replacement for Cisco A-Male to Mini-B Cord - 6 Feet (1.8 Meters)

I'm sure that this has been beaten to death throughout this thread......

I'm hoping that I didn't just fry the console port given what was mentioned in this thread post...

https://forums.servethehome.com/index.php?threads/brocade-icx-series-cheap-powerful-10gbe-40gbe-switching.21107/post-277688

Thanks in advance!

The ICX7250 Mini-USB port is just a console port with Mini-USB form. It's not a built in console port as the ICX7150 series have.
As per frying the port, just a quote from @fohdeesha:


Just buy the Mini-USB adapter AND a USB to Serial Cable like this.
Both are dirty cheap.

Edit: Change the link to USB to Serial recommended.

You certainly made my morning! Thank you for the info and for not flaming me! This thread is so long a lot of the previously linked items would lead to dead ends. It's very reassuring to hear from the Godfather on the matter directly. My 15 minutes of digging last night didn't come up with that one. The cables should be here by Tuesday, so I'll know then what I'm looking at. I'll be sure to report back in the hopes that it might contribute a small fraction to the community.

Thanks again!



EDIT: For those that come across this in the future, connecting a standard USB cable to the console port did NOT fry the port. This is contrary to what some other posts mentioned WAAAYYY back in this thread. I'm hoping that maybe this will save some folks a sleepless night thinking they fried their switch while they wait for a new cable to arrive.

I did order the cables mentioned, but also made one of these which worked great. I guess I have several options now when the other cables arrive.

https://forums.servethehome.com/index.php?threads/diy-mini-usb-serial-dongle.24245/

 

[Gerhen]

I found this cable: Amazon.com: Aexus Cisco Console Cable 6 ft with USB Type A to Mini-B CAB-Console-USB= : Electronics

Reasonably priced and does away with needing an added converter at the computer end. Has anybody tried these?

 

[diode]

I found this cable: Amazon.com: Aexus Cisco Console Cable 6 ft with USB Type A to Mini-B CAB-Console-USB= : Electronics

Reasonably priced and does away with needing an added converter at the computer end. Has anybody tried these?

That's the cable that I had purchased and it does NOT seem to be working. After having a panic attack yesterday thinking that I just turned my new switch into e-waste, I am to understand that what you linked is a traditional USB cable and does not provide FTDI required to access the port.

See my previous comment where I finally understood that a console cable != a serial cable; these are two distinct characteristics.

Either that, or my switch is defective and I have no clue what I'm doing (very probable)

 

[jei]

I found this cable: Amazon.com: Aexus Cisco Console Cable 6 ft with USB Type A to Mini-B CAB-Console-USB= : Electronics

Reasonably priced and does away with needing an added converter at the computer end. Has anybody tried these?

Cisco Console Cable 6 ft with USB Type A and Mini-B CAB-Console-USB=

Connect your computer’s USB port to the console port on Cisco Routers and Switches USB Type A to USB Mini B Compatible with Cisco Routers, Switches and IOS 6 feet from end to end | Color: Blu…

www.aexus.ca

It's just a normal USB-A -> Mini-USB cable without FTDI.