Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Lambdariver

New Member
Mar 24, 2019
20
14
3
to make the sfp+ ports work at 10gb you have to pm some guy on here
Haha! Major thanks to said guy. I was able to unlock all features in the beef overload which will be fantastic for some hands-on learning in my lab. Don't want to look like a (complete) bozo when working with my network engineers on re-architecting the datacenters at work.

The switch does report the fans as failed tho.
Been meaning to whip up a circuit to fake the tach signal of the fan so it reports it's working correctly. If I get it working, I'll definitely post the schematic. The switch goes into a bootloop for me if it detects the "failed" fans. Have to install the originals long enough for the boot to complete, then can be hot-swapped back to the quiet ones.
 

Skippy96

New Member
May 14, 2019
19
6
3
I just purchased a 6610-24P that came with fans and no power supplies. Is it possible to use the RPS15-e power supplies on this (250w +POE) or do they only support the RPS16-e (1,000w + POE)? I know from the first post it says they only use 80W and it looks like the pinout is the same

Thanks
 

Skippy96

New Member
May 14, 2019
19
6
3
it looks like the -e or -i are to do with airflow direction (Intake, Exhaust)
https://fohdeesha.com/data/other/brocade/ICX6610 Datasheet.pdf

RPS15-E Brocade ICX 6610/6650 non-PoE 250 W PSU, power-supply-side exhaust airflow
RPS15-I Brocade ICX 6610/6650 non-PoE 250 W PSU, power-supply-side intake airflow
RPS16-E 1,000 W power supply for Brocade ICX 6610 PoE models, power-supply-side exhaust airflow
RPS16-I 1,000 W power supply for Brocade ICX 6610 PoE models, power-supply-side intake airflow
RPS16DC-E 510 W DC power supply for Brocade ICX 6610, power-supply-side exhaust airflow
RPS16DC-I 510 W DC power supply for Brocade ICX 6610, power-supply-side intake airflow

i Picked up some 15s rev C's but my switch is POE :( anyone want to trade? :)
 

bkvamme

New Member
Sep 23, 2018
7
1
3
Just picked up a ICX6610-48P-PE for 359USD including shipping to replace my now mixed assortment of switches. Currently have a ICX-6450-24, an HP 1810-24G and a Mikrotik CRS-309. Might actually end up with more or less the same power consumption...

 

fohdeesha

Kaini Industries
Nov 20, 2016
1,955
1,780
113
29
fohdeesha.com
I just purchased a 6610-24P that came with fans and no power supplies. Is it possible to use the RPS15-e power supplies on this (250w +POE) or do they only support the RPS16-e (1,000w + POE)? I know from the first post it says they only use 80W and it looks like the pinout is the same

Thanks
the RPS15 has no PoE rail whatsoever. You can use it in your PoE 6610 without issue, but you will have no functioning PoE. If you want PoE, you'll need to pick up the RPS16. the letter at the end is airflow direction and it needs to match the fan trays, E is normal front to back airflow
 

-Tilde

New Member
Apr 18, 2019
6
0
1
Picked up a new in box 7250 for about 160 usd (250nzd), but I don't believe it comes with ears? Has anyone got a source for a set or some compatible ones? My other option is a set of universal rails, but I don't really want it sliding around.

edit: some documentation says it comes with ears, others seem to say it is sold separately. I guess I'll see when I get it
 
Last edited:

-Tilde

New Member
Apr 18, 2019
6
0
1
So, what is the best way to connect the console to another system? I assume I have to use the included mini usb to rj45 cable, into the rj45 to DB-9/serial adapter, and then use an rs232 to USB adapter?

https://i.imgur.com/gd1J97W.jpg

Also, the ICX 7250 does come with rack mount hardware, as speculated by earlier post.

Edit: I'm an idiot and forgot my R710 has a serial port. I just used that.
 
Last edited:

Terry Wallace

PsyOps SysOp
Aug 13, 2018
138
79
28
52
Allen, TX
Yep thats exactly how I hooked mine up it worked fine. I actually pass the usb-serial adapter through my proxmox host directly to a vm box thats a minimal linux box with minicom running. So I always have a serial console ready to work with.

@-Tilde I'd recommend you pull the screws and pop the top off.. And swap the 3v coin battery (a standard CR-2032) for a new one. Then it should be good to go for another 5 years.
 

-Tilde

New Member
Apr 18, 2019
6
0
1
I'd recommend you pull the screws and pop the top off.. And swap the 3v coin battery (a standard CR-2032) for a new one. Then it should be good to go for another 5 years.
Mine was manufactured 08/2016, will it need a replacement?

Edit: What is the front USB A port for? The manual says it's for storage, storage of what?
 
Last edited:

Terry Wallace

PsyOps SysOp
Aug 13, 2018
138
79
28
52
Allen, TX
Well it was made 2 years ago. No telling how old battery was. Its more of a ‘do it now and never worry again’ deal. :)

Front usb is for copying off configuration, storing logs etc. hardly anyone uses it in a home lab.
 

TS440

New Member
May 6, 2019
3
1
3
After following this thread for a while I broke down and have a 7250-24P arriving shortly. I've built my dongle and am ready to update the firmware once it arrives. I see discussion on L2 vs. L3 firmware - I plan to use this as a L2 switch initially and when I get up to speed on the switch may move to L3 for inter-vlan routing. Can I just use L2 features if I have the L3 firmware installed, or do I need to specifically flash the L2 firmware? Thanks for the great thread and I appreciate any direction you can provide.
 
Last edited:

Terry Wallace

PsyOps SysOp
Aug 13, 2018
138
79
28
52
Allen, TX
In the downloaded software for 8080 image you will find 2 images. One with an S in its name for switch and one with an R in its name for Router.

Yes you have to flash one of them.
I’d install L3 router. That won’t hinder its use in l2
 

Hakujou

New Member
Apr 28, 2019
19
6
3
Hello,

Long time lurker here, I finally got an ICX6610 thanks to @fohdeesha !

I got it from g-electronics on ebay, like some people here, and I've issues with the 40Gbps breakout ports. It does seem to be the exact same issue as @N1h1l1ty on page 50 of this topic, as I have those messages on console at boot:
Code:
Parsing Config Data ...
------------------------------------------------------------------
M:9 L:0 - chow_qsfp_read, qsfp 3, error in seting up mux
------------------------------------------------------------------
M:9 L:0 - link_40G_4x10G_get_media: qsfp 3, port 1/2/7 error in reading qsfp
chow_40G_4x10G_get_media: error in reading qsfp 1/2/7
It was working for one of my servers (ESXi) but I could not make a second port work, even from the other QSFP port. Tested with two cables (some Cisco generic on eBay and a Brocade one from FiberStore), same result. All firmwares (bootloader, firmware and PoE firmware) are up-to-date.

Now with the exact same NIC (HP NC530SFP+) on an other server (FreeNAS), it doesn't work at all. It appears up on the interfaces brief, but goes down as soon as I tag it with a VLAN for some reason. It also goes up and down randomly without any vlan tagged (but traffic does not flows anyway, even when switch says it's up).

So I guess my QSFP module is broken ?
Honestly I don't think I'll ever need more than 8 SFP+ ports (all licensed, again thanks to @fohdeesha), but it's kind of a bummer to see it broken. I don't really want to send the switch back as everything is now cabled and I live in France, so shipping to USA would be really tedious.

Is the module replaceable easily ? Is there a way to purchase it alone for cheap ?
Or am I missing something obvious ?

Thanks a lot
Hello,

Sooooo... I received my new switch, from a different seller, reconfigured it and... Same behavior.
The error after Parsing Config Data is gone, but I tried:
  • Breakout to HP 530SFP+ on FreeNAS: Link goes up/down like crazy on the logs, no traffic flows
  • Breakout to HP 530SFP+ on ESXI 6.7: Link is up until I tag it with a VLAN, then it goes down and won't come up until I reboot the switch
  • Breakout to Chelsio T320 on FreeNAS: Seems to work fine with VLANs (nope, see below)
I'm confused. When I plug the HP NIC on front SFP+ ports, it works perfectly so it's definitely working fine, but for some reason seems to crash with the breakout ports.
I might end up using DAC on front ports to get those working, but with two QSFP breakout cables that's a shame, I have four servers and could have done 2x10Gbps on each...

EDIT: Tried a second port into the Chelsio card, same behaviour than with HP NIC. Looks like I can't connect more than one SFP+ port from the breakout...
If anyone has an idea, I'll take it.

EDIT2: Ok so I plugged every four of the SFP+ ports, and the error came back:

Code:
Parsing Config Data ...
------------------------------------------------------------------
M:9 L:0 - chow_qsfp_read, qsfp 3, error in seting up mux
------------------------------------------------------------------
M:9 L:0 - link_40G_4x10G_get_media: qsfp 3, port 1/2/2 error in reading qsfp
chow_40G_4x10G_get_media: error in reading qsfp 1/2/2
It's a different switch from a different seller, so either I'm a really unlucky guy, or there's something really wrong with my setup.

EDIT3: Just realized one of the port was still plugged on HP 530SFP+ when I did first edit. Unplugged it and magic, everything is working fine. So it seems the breakout ports really hates the HP NICs.
The error is still there at boot when e 1/2/2 is plugged in (even only with the Chelsio card) but it works. I disabled it and used an other port, the error is gone.

EDIT4: Ok so two SFP+ from breakout in a lag, I had to reboot the switch but it finally came up with VLAN. The others two SFP+ are unplugged as the switch doesn't seem to like my HP NICs.
I have other (weird) issues like one of my ESXi saw one NIC (gigabit) as down but the switch saw it up. Up/down on ESXi did nothing, but disable/enable on the switch made it come back up.
An other ESXi node (on a NUC, so Intel NIC this time) has gone crazy after rebooting the switch (up/down really fast). Rebooting NUC fixed the issue, but it seems to be recurring when I reboot the switch.
It also rebooted a few times on its own over the past few weeks, so I'll wait a little bit to see if those issues persists. If they do, I'll try to swap my current unit with the new one.
If the issues still persists, I don't know, maybe I'll buy some 7250 instead as I have some trust issues with the 6610 now...

EDIT5: It rebooted again for no reason and boom, rear SFP+ are broken again. I can't get them to go up again. Will try unit 2. I also still have the NIC going up and down in loop for the NUC. Rebooting the NUC fixed it.
 
Last edited:

Hakujou

New Member
Apr 28, 2019
19
6
3
I am searching for a 6610. But i am in europe. When buying a cheap one like this at ebay. What is wrong with it?
Are the port fully functional with full speed?
What about this unit?
You need a licence to turn the eight SFP+ front ports to 10Gbps, otherwise they'll work at 1Gbps.
PM @fohdeesha for that.
Otherwise assuming the switch is working fine everything will work at full speed, there's no license or limit on front GBase-T ports or back QSFP ports.
 

Hakujou

New Member
Apr 28, 2019
19
6
3
You need a licence to turn the eight SFP+ front ports to 10Gbps, otherwise they'll work at 1Gbps.
PM @fohdeesha for that.
Otherwise assuming the switch is working fine everything will work at full speed, there's no license or limit on front GBase-T ports or back QSFP ports.
Prices are usually insanely higher in Europe.
I usually buy from US and pay customs, it's still cheaper than paying those in EU. Your deal is not that bad, tho.