Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[Zombielinux]

Ok, I'm at my wits end.

I've followed every post in this thread about QSFP breakout cables. I've got a Dell P8T4W, which appears to be just a longer 27GG5.

I've unstacked (no stack-trunk 1/2/1 to 1/2/2, no stack-trunk 1/2/6 to 1/2/7)

I've reseated connectors, but only one of 4 channels seem to come up.

They're in 1/2/2-1/2/5.

sh media says they're 40GE-Passive Copper

show interfaces brief e 1/2/1 to 1/2/10 says 1/2/2, 1/2/4, and 1/2/5 are down, despite being connected to servers.

1/2/2 and 1/2/3 are in a LAG group (that host is up) and 1/2/4 and 1/2/5 are in a LAG group (that host is not).

What am I missing here?

SOLVED: After unstacking, you need to reload the config, otherwise, only one of the ports comes up. Everything is 100% functional after that.

 

[ClintE]

I am running 3 of these: https://www.amazon.com/gp/product/B01M5LIUK5/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1 and they work fine, though they've gone up in price by a few bucks from the looks of it. Still cheaper than they used to be.

As Zombielinux says you are better off with the combo fiber transceivers/cards if your setup supports it (i.e. short runs to your equipment, you haven't bought the NICs yet, etc.). In addition to being more expensive the 10GBase-T also draw additional power and run pretty hot. If you are doing a long run, for example though a house, dealing with copper is most likely going to be easier than dealing with fiber but make sure you use high quality Cat6a cable.

This. We ended up running 1" Carlon ENT flexible conduit (smurf tube) and pulling multimode with cheap, quiet used Dell 5524's at the ends, with a 6610 in the office. Demands of 10G copper, such as near-perfect quality cabling, expensive transceivers and adapters, heat, etc. were deciding factors for fiber install. Have to quiet down the 6610, and that's not easy, but worth it in long run.

10G copper is great in business environment but doesn't compare with relatively inexpensive fiber install for the homelab.

Cheers!

 

[Mithril]

Dumb question: If I run a 6450 or 7250 on *just* 12v when it is a POE model, will that work just without POE, or will magic smoke come out?

 

[infoMatt]

Dumb question: If I run a 6450 or 7250 on *just* 12v when it is a POE model, will that work just without POE, or will magic smoke come out?

Well... how could you run those switches with an internal PSU with just 12V? Via the PSU#2 power socket at the back?

Oh, don't try to hook it up to an ATX PSU. The connector is the same but the voltages don't match, you'll end up damaging the board.

 

[Mithril]

Well... how could you run those switches with an internal PSU with just 12V? Via the PSU#2 power socket at the back?

Oh, don't try to hook it up to an ATX PSU. The connector is the same but the voltages don't match, you'll end up damaging the board.

Per the teardowns, the non POE units use simply 12v PSUs inside for those models. What I'm curious about is if the POE voltage drops, if the switch handles that ok (both firmware and physically).

 

[infoMatt]

Per the teardowns, the non POE units use simply 12v PSUs inside for those models. What I'm curious about is if the POE voltage drops, if the switch handles that ok (both firmware and physically).

As far as I've read in this topic, you can unplug the PoE board off of a -P model and it will boot as a non-PoE model, but I can't tell you unfortunately what will happen if the 50Vish rail dissapears. The PSU however has a single control board, so a fault on a rail results in a poweroff over a fault for all the voltages, and it won't come back up until you unplug the mains for a couple of seconds.

 

[tommybackeast]

[QUOTE="

1) Don't use the switch's DHCP server - it is non-authorative on the ICX6xxx series and this will not work with many devices. ISC's dhcpd server + BIND named for DNS is easy to configure together,, lightweight, and reliable.


[/QUOTE]

Can you explain what non-authorative DHCP means in this context ? google provided me with information that was too complex for me to understand.

Also, can DHCP and DNS server be used on a ICX7250 ?

 

[tommybackeast]

OK that definitely concretes the fact I need to get 8092 up on the guide instead of the old 8080

Thank you : naturally, you cannot have a guide for users who own more than one brocade / ruckus product; I am still on the older 8080 on 7250 and 7150-c12p as that was the version on your setup page. Since so many suggested it, also have a Ruckus r510AP :

My question (as I have no experience yet with firmware 8092 : is this the firmware that allows single pane of glass for 7250- 7150 - r510 AP ? if yes, can anyone comment on how well it has worked for them, thanks

 

[neb50]

Thank you : naturally, you cannot have a guide for users who own more than one brocade / ruckus product; I am still on the older 8080 on 7250 and 7150-c12p as that was the version on your setup page. Since so many suggested it, also have a Ruckus r510AP :

My question (as I have no experience yet with firmware 8092 : is this the firmware that allows single pane of glass for 7250- 7150 - r510 AP ? if yes, can anyone comment on how well it has worked for them, thanks

I switched to 8092 on my 7250 and use it with Ruckus Unleashed. I had to create a user/password that matched the Unleased, change some user permissions, and turn on LLDP.

You can see the port stat's in Unleashed for the switch and a few other things, but that is about it for me.

 

[klui]

1) Don't use the switch's DHCP server - it is non-authorative on the ICX6xxx series and this will not work with many devices. ISC's dhcpd server + BIND named for DNS is easy to configure together,, lightweight, and reliable.

Can you explain what non-authorative DHCP means in this context ? google provided me with information that was too complex for me to understand.

Also, can DHCP and DNS server be used on a ICX7250 ?

To be honest I wasn't aware of what the differences either. The DHCP entry in Wikipedia makes it more clear. Instead of the non-authoritative server rejecting a discovery request that is new to a network, it will ignore it waiting for an authoritative one to respond. If no other servers respond, it will wait until the client times out and then respond to a new discovery. The behavior of this request, timeout, retry would be implementation dependent on the client and that's where the compatibility comes from. Looks like Sonos doesn't have a robust implementation.

Dynamic Host Configuration Protocol - Wikipedia

en.wikipedia.org

 

[tommybackeast]

I switched to 8092 on my 7250 and use it with Ruckus Unleashed. I had to create a user/password that matched the Unleased, change some user permissions, and turn on LLDP.

You can see the port stat's in Unleashed for the switch and a few other things, but that is about it for me.

on the ICX7250 that was on 8080 ; that you upgraded to 8092 :: I had looked for the CLI to do this upgrade and was not able to find anything . May I ask where you found instructions on changing the firmware version from 8080 to 8092 ? thanks

 

[Vesalius]

on the ICX7250 that was on 8080 ; that you upgraded to 8092 :: I had looked for the CLI to do this upgrade and was not able to find anything . May I ask where you found instructions on changing the firmware version from 8080 to 8092 ? thanks

Try the link below for different options.

Commscope Technical Content Portal

docs.commscope.com

 

[neb50]

on the ICX7250 that was on 8080 ; that you upgraded to 8092 :: I had looked for the CLI to do this upgrade and was not able to find anything . May I ask where you found instructions on changing the firmware version from 8080 to 8092 ? thanks

Try the link below for different options.

Commscope Technical Content Portal

docs.commscope.com

I used the USB method listed in the link. I was already on 8080 so I just went to 8092 with the load Primary, reboot, reload Primary, load Secondary and then rebooted once more just to make sure everything took.

 

[Roelf Zomerman]

a weird one..

I have a Unifi CloudKey Gen2+ that over the weekend went into "Ethernet Disconnected" mode - but still receiving power from the Brocade. I thought about trashing it, but thought, let's test it simply on a spare switch first (Juniper 2200C) - and behold, the CK+ boots and works perfectly..

I tried setting the brocade port to 100-Full, 100-Half, 1Gb Master and 1Gb slave, but nothing..

any further ideas?

 

[rootpeer]

2 Questions:

1. Do the ICX6430 ears work on the ICX6450?
2. Does a break-out cable from a Mellanox SX6012 negotiate 1G?

My 6450 came without ears. I was thinking, instead of spending 50+ USD/EUR for ears or try to fabricate them, I can buy a 6430 at the same price, use its ears for my 6450 and then use the 6430 as a media converter for the basement where my SX6012 is.

 

[Roelf Zomerman]

1. Do the ICX6430 ears work on the ICX6450?

I guess so, the installation manual states the same "image" for the 6430 as the 6450 for mounting the ears, but there's probably someone in this group that has both switches and can confirm... like.. eeuh @fodeesha:
Brocade Communications Systems ICX 6430 Installation Manual (Page 31 of 110) | ManualsLib

 

[nerdalertdk]

This might not help

but ubiquiti edgeswitch ears fits on my 7150

 

[dv007]

Wanted to post an update on my infinite quest to silence an ICX 7250-48P (I run this thing in a fairly warm closet with bad circulation). Call me obsessive, but I wasn't a fan (no pun intended) of my temps creeping into the low 70s after my previous fan swap. I believe I have come up with the perfect silent solution for this rig.

Replace all 3 rear fans with the Mechatronics MR4020X12B1-RSR fan. They will read just fine in CLI. Then add a Sunon MF60101V3-1000U-A99 fan to the ASIC heat sink (you'll still have around 1/3" to spare in the case, plenty). Wire it in with Fan 3. All the fans are reduced load vs stock so they won't draw too much and the Sunon only has power and ground, so doesn't affect the read on Fan 3.

The switch now runs next-to-dead silent; temperatures on the ASIC dropped ~20 degrees over my previous configuration into the low to mid 50s degrees and the PSU is in the upper 20s which is great. Been running around a week, everything has been perfect. I'm in love.

How do you mount the sunon fan on top of the ASIC heat sink? A picture would be nice.

 

[gb00s]

I have a Ruckus 7250-48 and changed the fans to Noctua NF-A4x20 FLX 12V (3pin). Unfortunately, the switch doesn't fully boot with these fans. Just close before the boot sequence ends, the fans stop spinning and the switch reboots. This loop continues 3-5 times and then the switch switches off. I tried 6 of these fans. Same with all of them. Single and bundle of two.

Then I tried eg. Sunon MF40201VX-1000U-G99 and the switch boots perfectly fine. Ordered now some of these Sunon's for 3.70EUR per item as they are not really noticeable at 8k rpm and have double the airflow than the Noctuas.

Do these 7250-48 have some kind of fan rpm barrier to boot properly?

 

[UKEE]

I have a Ruckus 7250-48 and changed the fans to Noctua NF-A4x20 FLX 12V (3pin). Unfortunately, the switch doesn't fully boot with these fans. Just close before the boot sequence ends, the fans stop spinning and the switch reboots. This loop continues 3-5 times and then the switch switches off. I tried 6 of these fans. Same with all of them. Single and bundle of two.

Then I tried eg. Sunon MF40201VX-1000U-G99 and the switch boots perfectly fine. Ordered now some of these Sunon's for 3.70EUR per item as they are not really noticeable at 8k rpm and have double the airflow than the Noctuas.

Do these 7250-48 have some kind of fan rpm barrier to boot properly?

Yes, it does. Not sure what the minimum is though. I designed and tested a tach multiplier using an Arduino board but never got around to installing it.