Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[fohdeesha]

A list of all FW versions you can find on the support page.
Software Downloads | Ruckus Wireless Support

What I am asking myself is if there is somewhere a simple overview if the different "branches". At least to me it looks like branches. Please correct me if this is wrong. Old and newer branches are getting maintained in parallel: 08.0.80, 08.0.90, 08.0.92, 08.0.95

this is the closest you'll get but it hasn't been updated in a while Ruckus ICX Target Path Selection Guide | Technical Documents | Ruckus Wireless Support

8090 is the "recommended" train, 8092 is slightly newer but still stable in my experience, 8095 is brand new and still has issues

 

[fohdeesha]

Thanks for checking!

I'm running FW 8080e like in your guide. Is the 8092d a newer version I should try? I didn't see any reference to it.

My switch had been up for 180 days so I did reboot it. No change sadly. I really don't see how it's not the switch, considering I don't own anything else Brocade/Ruckus. I haven't done a WireShark or anything so perhaps I'll have to try it and see.

EDIT: I changed my switch DNS to only use the pi-hole and I can see the direct hits from my main switch to the ruckus domains. It is 100% doing the DNS hits directly even though all of the sz stuff is disabled.

Rebooting the switch did clear all the junk from the sz logs. I have this now with current data only, but it doesn't look very exciting.

>show sz logs
Start i/max/iter 0/5/1
Jan  7 23:48:52:ca_stop_timer>Cancelled Timer id 0, rc=0
Jan  7 23:48:52:ca_start_timer:95912>Started timer with l/p 30000/30000 for u17, rc 0, id 167172236
Jan  7 23:48:53:ca_stop_timer>Cancelled Timer id 0, rc=41
Jan  7 23:48:53:sz_execute_state_machine>Entering with state/event: INIT/0, DISABLE/2005
Jan  7 23:48:53:sz_execute_state_machine>Exit with state/event: DISABLED/1, NONE/2000 RC: 1
End i/max/iter 5/5/0

that's really strange, it shows as exiting the sz daemon entirely like mine does "sz_execute_state_machine>Exit with state/event" with no further logs. when it IS on, it logs each DNS request, but you don't have any such logs. it could be a bug in your version, 8092 is indeed newer but still stable, I would flash to that

 

[csementuh]

that's really strange, it shows as exiting the sz daemon entirely like mine does "sz_execute_state_machine>Exit with state/event" with no further logs. when it IS on, it logs each DNS request, but you don't have any such logs. it could be a bug in your version, 8092 is indeed newer but still stable, I would flash to that

Thanks I'll update the firmware and see if it helps!

 

[tubs-ffm]

I found a Brocade micro-USB cable [...]
The cable I received didn't exactly match the pictures I found of the official Brocade cable kit, so I mapped out the connections:

(ICX)  mUSB -> RJ45 --> DE9  (RS232)     [RJ45 wire colour]
       1       NC       NC
(RX)  2  -->  3  --->  3    (TX)        [red]
(TX)  3  -->  6  --->  2    (RX)        [white]
       4       NC       NC
(GND)  5  -->  5,4  ->  5,8  (GND, CTS)  [black, green]

Does this look right to you guys? Quickly going over the connections it seems RX/TX pins are correctly connected and the only difference to @fohdeesha's and @Skud's pinout is the CTS pin pulled to ground.
[...]

Is "CTS to ground" required?

I wanted to build an easy one out of existing parts by myself: standard RJ-45 to DB9 adaptor, mini-USB cable, scissors, RJ-45 crimp connector, RJ-45 crimping tool. But looks like this will not work.

As I cannot find the original cable or a 2nd supplier solution for a reasonable price, the solution from @Scud using mini-USB to RJ-45 keystone seems to be easier. RJ-45 to RS232 cables are easier to find.

Here is the pinout for the contraption I made. Seems to be working well.

 

[drtaru]

Is "CTS to ground" required?

I wanted to build an easy one out of existing parts by myself: standard RJ-45 to DB9 adaptor, mini-USB cable, scissors, RJ-45 crimp connector, RJ-45 crimping tool. But looks like this will not work.

As I cannot find the original cable or a 2nd supplier solution for a reasonable price, the solution from @Scud using mini-USB to RJ-45 keystone seems to be easier. RJ-45 to RS232 cables are easier to find.

Nope, I just built a cable using a Rs232 breakout board from Amazon and an old MiniUSB cable from a cell phone, Only connected TX RX and GND

 

[infoMatt]

Is "CTS to ground" required?

It isn't, as long as you turn off hardware and software flow control.

 

[csementuh]

that's really strange, it shows as exiting the sz daemon entirely like mine does "sz_execute_state_machine>Exit with state/event" with no further logs. when it IS on, it logs each DNS request, but you don't have any such logs. it could be a bug in your version, 8092 is indeed newer but still stable, I would flash to that

I was on 8080. I did a test and flashed to the newest 8090 firmware first. No change, right away the switch started hammering DNS requests.

I flashed the newest 8092 UFI image and POE firmware. Problem solved!

I guess it was a bug that was fixed somewhere along the line? Or something haha. Thanks for the help!

 

[drtaru]

Alright, just got an ICX7250-48 non POE and i need to quiet this thing down a bit. My rack is open and in the office with me so anything I can do to quiet down the fans would be great.
I would also love to not have to buy a bunch of fans to test as I'm trying to do this all on a budget and I've already spent way more than I was planning ha!

If anyone could just point me at a couple fans to buy and be done with it all I would be super grateful!

 

[fohdeesha]

I was on 8080. I did a test and flashed to the newest 8090 firmware first. No change, right away the switch started hammering DNS requests.

I flashed the newest 8092 UFI image and POE firmware. Problem solved!

I guess it was a bug that was fixed somewhere along the line? Or something haha. Thanks for the help!

OK that definitely concretes the fact I need to get 8092 up on the guide instead of the old 8080

 

[drtaru]

OK that definitely concretes the fact I need to get 8092 up on the guide instead of the old 8080

I did this today, used your guide to go to 8080e Licensed, and then used the ruckus manifest method to get to 8090 (will go to 8092 tomorrow probably)
Was super simple as I just dumped everything from the 8090 zip into the tftp server and the manifest file did the rest, it upgraded to ufw and everything.

 

[neb50]

OK that definitely concretes the fact I need to get 8092 up on the guide instead of the old 8080

I just switched my 7250 POE to 8092 about a month ago to get it to work with unleashed. I have been using a USB drive to load the new firmware as it is much easier than setting up the server.

I would have to go back and figure out how I got it to all work, but it is possible to add the 7250 to a Unleashed controller without having to clear the config on it.

 

[Zombielinux]

I've been following this post for a long while, but at 244+ pages long, it gets a bit unweildy to search through.

Is there a way to monitor power consumption (PoE or otherwise) and fan RPM from SNMP on an ICX6610? I know there are mystic commands that will get the fan rpm out, but I've not found an incantation that works for power usage (other than PoE usage).

 

[eduncan911]

Search function doesn't allow individual threads... Sorry if this has been asked before.

What inexpensive 10GBase-T do we recommend from eBay? Inexpensive in that, cheap.

Use cases:

* Xen/XenServer
* ESXi

Also, how would I guarantee original cards over counterfeits?

 

[Zombielinux]

@eduncan911 Search function does, when you type into the search box, change from "Everywhere" to "This thread" in the white box that drops down. This may not work on mobile.

You're better off running 10G fiber transceivers and cards. They're more plentiful and reliable overall.

 

[Ixian]

Search function doesn't allow individual threads... Sorry if this has been asked before.

What inexpensive 10GBase-T do we recommend from eBay? Inexpensive in that, cheap.

Use cases:

* Xen/XenServer
* ESXi

Also, how would I guarantee original cards over counterfeits?

I am running 3 of these: https://www.amazon.com/gp/product/B01M5LIUK5/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1 and they work fine, though they've gone up in price by a few bucks from the looks of it. Still cheaper than they used to be.

As Zombielinux says you are better off with the combo fiber transceivers/cards if your setup supports it (i.e. short runs to your equipment, you haven't bought the NICs yet, etc.). In addition to being more expensive the 10GBase-T also draw additional power and run pretty hot. If you are doing a long run, for example though a house, dealing with copper is most likely going to be easier than dealing with fiber but make sure you use high quality Cat6a cable.

 

[Ixian]

Can anyone assist with a simple config? Apologies if this is pretty simple:

My 6450-24p is working fine and I've swapped the fans for KDE1204PKVX's so it runs quieter. Now I want to get proper VLAN routing set up.

My setup is straightforward for a home: I currently run everything on one internal subnet, 192.168.86.0/24. Pfsense firewall, (2) Unifi POE WAPs, (4) Amcrest POE security cameras (2) NAS devices, and the usual assortment of laptops, phones, and streaming devices all hang off it.

Pfsense is on port 1
WAP on ports 13 and 14
Cameras on 15-18
NAS on (2) 10GBase-T transceivers in SFP+ ports
Remaining ports used for internal hard wiring to offices, bedrooms, etc.

What I want to do is isolate the POE cameras on their own VLAN/subnet and block them from accessing the internet. I only want the cameras to see my BlueIris server, which needs to stay on the primary VLAN.

I'm also interesting in having the switch handle all my internal L3 and just use the Pfsense box as an edge device, though I'm not sure what the best solution for DHCP would be in that case due to Pfsense being weird about networks directly attaching to it or not.

And finally, once I get the basic stuff done, I'll look in to isolating my IOT stuff as well (Konnected.io board, and so on) and perhaps even guest wireless.

Can anyone help with an example config of what this could look like? I need something to wrap my head around, being an amateur at this. Really appreciate any help.

 

[Zombielinux]

Ok, I'm at my wits end.

I've followed every post in this thread about QSFP breakout cables. I've got a Dell P8T4W, which appears to be just a longer 27GG5.

I've unstacked (no stack-trunk 1/2/1 to 1/2/2, no stack-trunk 1/2/6 to 1/2/7)

I've reseated connectors, but only one of 4 channels seem to come up.

They're in 1/2/2-1/2/5.

sh media says they're 40GE-Passive Copper

show interfaces brief e 1/2/1 to 1/2/10 says 1/2/2, 1/2/4, and 1/2/5 are down, despite being connected to servers.

1/2/2 and 1/2/3 are in a LAG group (that host is up) and 1/2/4 and 1/2/5 are in a LAG group (that host is not).

What am I missing here?

SOLVED: After unstacking, you need to reload the config, otherwise, only one of the ports comes up. Everything is 100% functional after that.

 

[ClintE]

I am running 3 of these: https://www.amazon.com/gp/product/B01M5LIUK5/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1 and they work fine, though they've gone up in price by a few bucks from the looks of it. Still cheaper than they used to be.

As Zombielinux says you are better off with the combo fiber transceivers/cards if your setup supports it (i.e. short runs to your equipment, you haven't bought the NICs yet, etc.). In addition to being more expensive the 10GBase-T also draw additional power and run pretty hot. If you are doing a long run, for example though a house, dealing with copper is most likely going to be easier than dealing with fiber but make sure you use high quality Cat6a cable.

This. We ended up running 1" Carlon ENT flexible conduit (smurf tube) and pulling multimode with cheap, quiet used Dell 5524's at the ends, with a 6610 in the office. Demands of 10G copper, such as near-perfect quality cabling, expensive transceivers and adapters, heat, etc. were deciding factors for fiber install. Have to quiet down the 6610, and that's not easy, but worth it in long run.

10G copper is great in business environment but doesn't compare with relatively inexpensive fiber install for the homelab.

Cheers!

 

[Mithril]

Dumb question: If I run a 6450 or 7250 on *just* 12v when it is a POE model, will that work just without POE, or will magic smoke come out?

 

[infoMatt]

Dumb question: If I run a 6450 or 7250 on *just* 12v when it is a POE model, will that work just without POE, or will magic smoke come out?

Well... how could you run those switches with an internal PSU with just 12V? Via the PSU#2 power socket at the back?

Oh, don't try to hook it up to an ATX PSU. The connector is the same but the voltages don't match, you'll end up damaging the board.