Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[happydaisy]

Unleashed is working for me on 9000a. Wasn't working on 9000.
Needed to add the community string for SNMP v1. It can be found in the unleashed config by typing show config from an ssh terminal on the unleashed ap.

I had also previously had unleashed integration on firmware 8090. 7250-24p.

Now i just need to figure out if aftermarket fans are safe for this thing. Been reading mixed reports in this thread

 

[Vesalius]

Unleashed is working for me on 9000a. Wasn't working on 9000.
Needed to add the community string for SNMP v1. It can be found in the unleashed config by typing show config from an ssh terminal on the unleashed ap.

I had also previously had unleashed integration on firmware 8090.

Ok, good to know. I guess the ruckus employee I quoted stating 9000 would not receive updates was wrong. Thanks, I was waiting for 9010.

 

[fohdeesha]

so, from what I remember digging through the scripts in the ruckus unleashed AP firmware, the ICX "integration" is just a bunch of "expect" bash scripts. I specifically recall they expect to be able to ssh to the switch IP with the credentials you provide, and then blindly "enable" and "conf t" from there. so if you have an "enable" authentication line in your switch config that makes users log in again when they issue "enable", it'll barf. check your icx config and make sure you don't have a:

aaa authentication enable default local

the v9000 thing is another issue, the initial GA release of 9000 didn't have support if I remember right, 9000a should though. And if your switch is succesfully added in the unleashed UI, it will automatically create the snmp entries and other required things in your switch config so those "shouldn't" need to be done manually. here's the master script from Unleashed 200.9, ICX stuff starts around line 5240

 

[Cncjerry]

Hey peoples, thanks for allowing me to join.

I'm a long-time networking person but might have made a mistake. I'm deploying 10GBe optical around my house for 6 servers. I bought a mikrotik 5 port forgetting I had 6 machines, duh. So I found an ICX6610-48p on ebay for a decent 'make an offer' price and then found that I had to license the 10Gb ports. What the heck? Talk about a ripoff. I took a scan of some of the text in this thread, will read more, but before I dig in too deeply, can I get around the licensing somehow?

I saw the note on the QSFP stacking ports, my engineer already told me that would probably work. Another question was one of the stacking ports was a 4x10 and I was wondering if a splitter cable would work on it? I thought we used splitters on the switches that are TOR on Exadata servers (I work for Oracle). This is a new area for me.

So is there an easy or not very expensive way to recover the 10Gb ports on the switch? My plan was to get rid of the 2 x 16 port GigE switches I am using and collect all that plus my small subnet of 6 x 10Gb servers into one switch.

Thanks again. great board.

Jerry

 

[fohdeesha]

Hey peoples, thanks for allowing me to join.

I'm a long-time networking person but might have made a mistake. I'm deploying 10GBe optical around my house for 6 servers. I bought a mikrotik 5 port forgetting I had 6 machines, duh. So I found an ICX6610-48p on ebay for a decent 'make an offer' price and then found that I had to license the 10Gb ports. What the heck? Talk about a ripoff. I took a scan of some of the text in this thread, will read more, but before I dig in too deeply, can I get around the licensing somehow?

I saw the note on the QSFP stacking ports, my engineer already told me that would probably work. Another question was one of the stacking ports was a 4x10 and I was wondering if a splitter cable would work on it? I thought we used splitters on the switches that are TOR on Exadata servers (I work for Oracle). This is a new area for me.

So is there an easy or not very expensive way to recover the 10Gb ports on the switch? My plan was to get rid of the 2 x 16 port GigE switches I am using and collect all that plus my small subnet of 6 x 10Gb servers into one switch.

Thanks again. great board.

Jerry

second sentence of the entire thread, first post

 

[tinfoil3d]

fohdeesha seriously much thanks for your work on icx and perc. Just super freaking amazing. I've nothing more to add. This has been super informative thread.

 

[EngChiSTH]

Any way to check the health of the management port on 7250 (or what I am doing wrong)

I am going from 6450 that is working fine to newly purchased from bay 7250 and following @fohdeesha instructions , I am getting nothing in Putty returned back to me on boot (fan spin up) or after boot.

Same workstation and Putty, connected to 6450 management port over same exact cable is getting me the prompt I would expect.
Anything I am missing? the Putty screen is just black, no data returned

Thank you

 

[itronin]

Any way to check the health of the management port on 7250 (or what I am doing wrong)

I am going from 6450 that is working fine to newly purchased from bay 7250 and following @fohdeesha instructions , I am getting nothing in Putty returned back to me on boot (fan spin up) or after boot.

Same workstation and Putty, connected to 6450 management port over same exact cable is getting me the prompt I would expect.
Anything I am missing? the Putty screen is just black, no data returned

Thank you

If you are using the exact same RJ45 serial cable you used on the 6450 on a port on the 7250 you will NOT get any serial output.

6450 has an RJ45 serial console connection.
7250 has a micro USB connection that is wired as a serial port. DO NOT plug in a standard "A to micro USB cable" betweeen the 7250 and a laptop or dekstop. If you do magic smoke may escape the 7250 and perhaps your computing device.

 

[EngChiSTH]

If you are using the exact same RJ45 serial cable you used on the 6450 on a port on the 7250 you will NOT get any serial output.

6450 has an RJ45 serial console connection.
7250 has a micro USB connection that is wired as a serial port. DO NOT plug in a standard "A to micro USB cable" betweeen the 7250 and a laptop or dekstop. If you do magic smoke may escape the 7250 and perhaps your computing device.

Got it , really good to know. Any recommendation on the cable(s) I need to get into console of 7250?
Any other way to get into the switch while I am waiting for the cable? it was 'reset' according to ebay seller


and for the record, I am trying to use the guide @fohdeesha provided below, connecting USB on my laptop through blue cable to management Ethernet port above the full size USB port. Should I be using something else instead? thank you

" Connect to the serial/console port using a program like Putty (9600 8N1), then connect the management ethernet port to your network (do NOT use any of the regular switch ports yet). The management port is the one above the full size USB port. "

 

[itronin]

Got it , really good to know. Any recommendation on the cable(s) I need to get into console of 7250?

here's a start.

US eBay search

I know its a long thread but you can find interesting results using STH search

In particular if you have in your spare parts bin a USB A Female cable that you can sacrifice and a keystone jack plus a 110 punch tool (or are careful with a makeshift 110 tool e.g. screwdriver) then you can fashion your own as documented.

there are some other posts in this mega thread from folks that have crafted cables as well.

 

[itronin]

and for the record, I am trying to use the guide @fohdeesha provided below, connecting USB on my laptop through blue cable to management Ethernet port above the full size USB port. Should I be using something else instead? thank you

" Connect to the serial/console port using a program like Putty (9600 8N1), then connect the management ethernet port to your network (do NOT use any of the regular switch ports yet). The management port is the one above the full size USB port. "

Right - the management ethernet port is ethernet and not serial and as the guide says "connect the management ethernet port to your network"

this part "Connect to the serial/console port using a program like Putty (9600 8N1)" is generic and the same text is used throughout the different guides because the Brocade switches have transitioned through more than a few different types of serial console connections. For example the 7150 has usb-c and a built-in serial to usb adapter. which is nice.

Look at the front of your switch. Go up from from the management ethernet port. See the Micro USB connection (?) and if its not worn off you should see a graphic "101010" (or something very similar looking) that's the Brocade icon for serial port and at least the icon does seem to be common across the ICX switches.

 

[EngChiSTH]

Right - the management ethernet port is ethernet and not serial and as the guide says "connect the management ethernet port to your network"

this part "Connect to the serial/console port using a program like Putty (9600 8N1)" is generic and the same text is used throughout the different guides because the Brocade switches have transitioned through more than a few different types of serial console connections. For example the 7150 has usb-c and a built-in serial to usb adapter. which is nice.

Look at the front of your switch. Go up from from the management ethernet port. See the Micro USB connection (?) and if its not worn off you should see a graphic "101010" (or something very similar looking) that's the Brocade icon for serial port and at least the icon does seem to be common across the ICX switches.

Thank you. I guess I can not set it up today since I don't have the specific cable 7250 wants. I will have to do some research to understand what options there are besides

a) DB9 to USB mini from ebay
b) Rolling my own (which I do not have a solder lamp for anymore)

also looking for something more modern than DB9 as current laptops hardly have that port ..

overall, I think this information on item specific to 7250 may be good in the page dedicated to 7250 ICX7250 / ICX7450 - Fohdeesha Docs so people dont have to repeat the searches..

any other suggestions on above would be welcomed. thank you

 

[MonumentNetworking]

Thank you. I guess I can not set it up today since I don't have the specific cable 7250 wants. I will have to do some research to understand what options there are besides

a) DB9 to USB mini from ebay
b) Rolling my own (which I do not have a solder lamp for anymore)

also looking for something more modern than DB9 as current laptops hardly have that port ..

overall, I think this information on item specific to 7250 may be good in the page dedicated to 7250 ICX7250 / ICX7450 - Fohdeesha Docs so people dont have to repeat the searches..

any other suggestions on above would be welcomed. thank you

You really don't have to solder this I did it with a keystone jack (home depot if necessary) and I didn't even punch the cables in. I literally just punched them in with a small piece of plastic. A junk USB mini cable to sacrifice and a keystone. Super simple you'll be amazed.

 

[EngChiSTH]

You really don't have to solder this I did it with a keystone jack (home depot if necessary) and I didn't even punch the cables in. I literally just punched them in with a small piece of plastic. A junk USB mini cable to sacrifice and a keystone. Super simple you'll be amazed.

thank you , may be PM for what you did? I have spare USB mini cables, have keystone , and punch tool (i did my own cat5/6 when did wiring for my house). I ordered the cable from ebay as backup option, however it would not be here for 4-6 days and am willing to try the homemade option

trying to understand end to now how it would work
a) usb mini cable, cut it to expose wires
b) punch it down to RJ45 keystone jack as described here ( DIY Mini-USB Serial Dongle | ServeTheHome Forums )
c) connect the usb mini into 7250 and connect what to do RJ45 jack just created? a standard cisco serial cable?

thank you!

 

[MonumentNetworking]

thank you , may be PM for what you did? I have spare USB mini cables, have keystone , and punch tool (i did my own cat5/6 when did wiring for my house). I ordered the cable from ebay as backup option, however it would not be here for 4-6 days and am willing to try the homemade option

trying to understand end to now how it would work
a) usb mini cable, cut it to expose wires
b) punch it down to RJ45 keystone jack as described here ( DIY Mini-USB Serial Dongle | ServeTheHome Forums )
c) connect the usb mini into 7250 and connect what to do RJ45 jack just created? a standard cisco serial cable?

thank you!

When I spliced my USB it had red green white and black.

Using t568b coloring on the jack


USB. Keystone

White is Green
Green is green/white
Black is blue
Red is ignored

Hop in putty, securecrt or whatever you prefer. Just remember 9600, N ,8, 1

 

[EngChiSTH]

When I spliced my USB it had red green white and black.

Using t568b coloring on the jack


USB. Keystone

White is Green
Green is green/white
Black is blue
Red is ignored

Hop in putty, securecrt or whatever you prefer. Just remember 9600, N ,8, 1

Thank you. Once created what cable are you connecting to the keystone?

 

[Cncjerry]

Another thing I didn't consider on this switch is power consumption. I thought I read it was 130w, I see you have it at 110w. My high tier (should be tears) rate here in Marin county, home of the hot tubbers, is $.36 kw/h. So at 110w it looks like it will cost me $29 per month, minus my two 16 port netgear Gige switches I'm replacing. They said this 6610 only has one fan and PS. Seems like a lot of power considering my server, with external storage cage and Tandberg LTO tape loader is only 360w.

Anyway, thanks for the pointer back to page 1, I read through it all, switch should be here next week.

This is a great site, wish I had found it a while back. I've been using an HP DL380 for my server, pooling all my disk on it using ZFS. My wife loves all the videos, movies, etc I put up using miniDLNA. The 10Gb connections really made the difference after I bought two dual socket dell servers for workstations. Those things heat the rooms and don't seem to have suspend mode so I have to shut them off at night. I bought one of the Sonnet thunderbolt 10Gb adapters for my mac mini M1. That is a really fast setup. I'll be sending the microtik back.

Now, on to 40Gb.

Jerry

 

[OKGolombRuler]

so, from what I remember digging through the scripts in the ruckus unleashed AP firmware, the ICX "integration" is just a bunch of "expect" bash scripts. I specifically recall they expect to be able to ssh to the switch IP with the credentials you provide, and then blindly "enable" and "conf t" from there. so if you have an "enable" authentication line in your switch config that makes users log in again when they issue "enable", it'll barf. check your icx config and make sure you don't have a:

aaa authentication enable default local

the v9000 thing is another issue, the initial GA release of 9000 didn't have support if I remember right, 9000a should though. And if your switch is succesfully added in the unleashed UI, it will automatically create the snmp entries and other required things in your switch config so those "shouldn't" need to be done manually. here's the master script from Unleashed 200.9, ICX stuff starts around line 5240

Looks like that did the trick, thanks!

Two important notes for folks looking to do this: it *does* silently change your config, specifically the SNMP community RO string and adding a UDP logging host:

Oct 16 17:23:21:I:COPY COMPLETED
Oct 16 17:23:21:I:Security: startup-config was changed by XXXX from ssh client XXXX
Oct 16 17:23:21:I:COPY_CONFIGURATION_TO_FLASH
Oct 16 17:23:14:I:SNMP: read-only community added by XXXX from ssh session.
Oct 16 17:23:14:I:SNMP: read-only community deleted by XXXX from ssh session.
Oct 16 17:23:02:I:System: Syslog server [ruckus_unleashed_ip] added by XXXX from ssh session.
Oct 16 17:22:52:I:Security: SSH login by XXXX from src IP XXXX from src MAC XXXX to PRIVILEGED EXEC mode using RSA as Server Host Key.
Oct 16 17:22:49:I:Security: SSH login by XXXX from src IP XXXX from src MAC XXXX to USER EXEC mode using RSA as Server Host Key.

Off to see what new trouble I can get into with this. Thanks again @fohdeesha!

 

[EngChiSTH]

When I spliced my USB it had red green white and black.

Using t568b coloring on the jack


USB. Keystone

White is Green
Green is green/white
Black is blue
Red is ignored

Hop in putty, securecrt or whatever you prefer. Just remember 9600, N ,8, 1

So I made the cable myself and was greeted with console from the switch - dead simple thank you to instructions above. Strip USB cable, connect 3 wires , use punch it tool , plug in console cable into the jack you created..

 

[Originalus]

i have upgraded my C12P to SPR09000ufi. It takes a while to startup after reload. i did check terminal and i get these errors:

Creating TUN device
Creating the first Tap interface
ERROR: Failure in gw add
Starting TPM Infra
Group tss and User tss created
hostname set to Ruckus-ICX
could not read config file /etc/radsecproxy.conf
No such file or directory
Starting TCSD Daemon in 'fi' namespace


Any ideas how to fix this?