Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

infoMatt

Active Member
Apr 16, 2019
157
60
28
Do you have any examples of universal ears that have sufficient space to drill custom holes? I couldn't find anything that didn't have a bunch of holes that partially overlapped where I'd need them.
Sorry, I don't have such examples... my 6450 came with rack ears so I haven't searched for those.
 

Spearfoot

Member
Apr 22, 2015
65
25
18
62
If, like me, you're one of those unfortunate ICX7150-C12P owners whose power supply has died, take comfort. There are replacement units available. Digikey has at least two: items LB240S56K and GCS350PS56 at the top of the list linked below. These are both 56VDC supplies which will deliver adequate power even after derating for convection cooling. They're the same form factor and use the same connectors as the original Ruckus PSU.

Warning: they both cost over $100!

The LB240S56K is a 'big brother' of the 130W LB130S56 @fohdeesha mentioned in a Reddit post on this subject (see below), and which I've purchased on eBay and intend to try out. I'm only running a single PoE device off my switch (an AP) so the lower power rating shouldn't be a problem -- I hope. If it doesn't work out, I'm either going to spring for a new C12P with warranty, or one of the PSUs above. Haven't made up my mind about that yet.
https://www.reddit.com/r/homelab/comments/dz989v
 

EngineerNate

Member
Jun 3, 2017
61
10
8
31
Question for you guys, when I set up my current ToR switch, I had to disable some proprietary discovery protocols on my WAN VLAN to keep the switch from wigging out my cable modem. There were the standard protocols I could turn off in the web gui, then I had to go into the CLI to turn off the "Nortel Discovery Protocol" that was part of how Nortel switches of that type did auto stacking config if you stacked using the regular ports instead of the dedicated cables.

Do these (ICX7250) have anything like that I'll need to turn off? I don't have a cable modem right now but the config worked perfectly when I moved and switched to municipal fiber, so I'm inclined to set the new switch up identically before switching over (port mappings and all) and then work from there.
 

James Verbunk

New Member
Apr 13, 2018
6
0
1
15
Hey Everyone,

Does anyone have info on using the SDN capability in a 6610?

Apologies if I didn't read up to my answer yet (I'm 'only' on page 78).

-J
 

Spearfoot

Member
Apr 22, 2015
65
25
18
62
I had the same experience this morning! Bought an ICX7150-C12P 32 days ago on eBay... this morning it suddenly died. When I tested it I found that the front panel LED's briefly flash when it's plugged in, same as you. So now I'm off to find a new power supply. Thanks for the link.
This 54VDC may be a better choice. It's hard to tell what the original PSU voltage is; mine has no markings of any kind, and -- being dead -- I can't just meter it. I did see a post on the Reddit thread stating that the OEM PSU on these units is 54VDC.

 

fohdeesha

Kaini Industries
Nov 20, 2016
1,907
1,713
113
29
fohdeesha.com
Do these (ICX7250) have anything like that I'll need to turn off? I don't have a cable modem right now but the config worked perfectly when I moved and switched to municipal fiber, so I'm inclined to set the new switch up identically before switching over (port mappings and all) and then work from there.
Nope, nothing by default. It does support lldp and cdp for device discovery, and with those on your cable modem will see the mac address of your switch and get confused, which I'm sure is what happened with your nortel (cable modem should only be seeing 1 mac, the mac of your router). neither of these things are on by default however so after following my guide (and make a separate vlan for wan) it will work out of the box
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
1,907
1,713
113
29
fohdeesha.com
Hey Everyone,

Does anyone have info on using the SDN capability in a 6610?

Apologies if I didn't read up to my answer yet (I'm 'only' on page 78).

-J
download the firmware and documentation zip on my update guide, linked in the very first post of this thread. In that zip, you'll find fastiron-08030-sdnguide.pdf
 

EngineerNate

Member
Jun 3, 2017
61
10
8
31
Nope, nothing by default. It does support lldp and cdp for device discovery, and with those on your cable modem will see the mac address of your switch and get confused, which I'm sure is what happened with your nortel (cable modem should only be seeing 1 mac, the mac of your router). neither of these things are on by default however so after following my guide (and make a separate vlan for wan) it will work out of the box
Great! The Nortel gave me fits because I'd turned off all of discovery protocols I knew about (lldp etc) in the gui and it was still tripping up the modem. I had to mirror the modem's port and wireshark it to see what it was seeing to know that it was still sending out packets for it's own proprietary thing. It's been a great switch otherwise and at the time I purchased it it was a cheap into into 10G and PoE.
 

3nodeproblem

New Member
Jun 28, 2020
27
10
3
As my homelab is growing, it has become time for my first "serious" switch. After a couple of weeks of research, I am feeling ready to push the "buy" button on one of the Brocade models - the sweet spot for me seems to be the ICX6450-24P. My needs:
  • 8 PoE 1G ports (these are currently on an unmanaged switch that I could reuse, but would be nice to fit them all on the main switch. SBCs for Consul/Nomad/Vault servers on the same VLAN, so I think it could actually be OK with a shared 1G for them all. PoE budget is pretty low and even 80W total should be more than plenty)
  • 3 10G ports (glusterfs servers. I still haven't figured out if I really *need* 3 or if 2 would be sufficient as the third will be a glusterfs arbiter and I haven't figured out if full writes are sent from clients to arbiter if it's reduced to metadata on the client side)
  • Min 8 non-PoE 1G ports for other nodes
  • Low-power
  • Quiet
Now, it seems that either 7150-24P, 7250-24P, or 6450-24P would be reasonable choices here. (6610 disregarded for power and noise). I'm no stranger to fan modding. If I reuse my current PoE switch, I could also fit my current requirements in a C12P, at the cost of easy future scalability. However, having followed the domestic (Japan) and eBay secondary market for the past week or two, it looks like I have to do some tradeoffs. While eBay is an option, domestic would be nicer due to COVID and customs. My options seem to be:
  1. Domestic ICX6450-24 for ~100$. Reuse current switch for PoE.
  2. Domestic ICX6610-48P for ~100$. Severe overkill but I'll never (knock on wood) grow out of this.
  3. eBay International ICX6450-48P for ~180$.
  4. eBay International ICX6450-24P for ~460$.
  5. eBay International ICX7250-24P for ~540$.
  6. eBay International XX50-C12-PD for 200~250$, coupled with a ~160$ MikroTik CRS305-1G-4S+IN for SFP+ ports and current GB switch for SBC cluster.
  7. Hold my horses and closely watch the markets until something more suitable comes up.

So the main reason why I'm hesitant to go for either of the beefier but cheaper ones are power consumption and noise - the ratings are way higher for 48 port ones, and for 6610 compared to the others. If that is what I can expect under similar load, I'm thinking it *could* still be worth shell out 3x the price for the ICX6450-24P, or settle for the non-PoE version at the cost of needing two switches and not being able to add additional PoE devices (which I could make use of with new SBCs).

Now my main questions:
  1. Under similar load, is should I really expect power consumption to be significantly higher (around double) for the 48-port versions over the 24 port ones? Can something be done either in hardware or software to disable the higher 24 ports?
  2. Does the 6610 *have* to be such a noisy and hungry beast-hog compared to the 6450/7250? Can something be done to bring it down in line with the others considering I won't be utilizing near max capacity? The ICX6610-48P for 100$ looks really attractive but I'm afraid it'll end up a source of frustration.
 
Last edited:

EngineerNate

Member
Jun 3, 2017
61
10
8
31
The 6610 fans are non-moddable from what I've read in this thread.

The 6610-24p would also meet your requirements and be significantly better on power (80w vs 110w) than the 48p, as a middle ground.

Rev B and later PSUs are quieter. Quiet enough? That's a question only you can answer.

All that said, I didn't think the power draw between the two port counts of the 6450 was all that different. I'd probably go with the 6450-48p if I were you.
 
  • Like
Reactions: fohdeesha

3nodeproblem

New Member
Jun 28, 2020
27
10
3
All that said, I didn't think the power draw between the two port counts of the 6450 was all that different. I'd probably go with the 6450-48p if I were you.
Thanks for your input. The parent post states 25W for 24, 50W for 48 (double!), but my main question here is if that's considering all plugs utilized, or if power usage is more comparable with the same load on the two.
 

infoMatt

Active Member
Apr 16, 2019
157
60
28
Thanks for your input. The parent post states 25W for 24, 50W for 48 (double!), but my main question here is if that's considering all plugs utilized, or if power usage is more comparable with the same load on the two.
My 6450-24p idles at 33-34W, so take those figures with a grain of salt. The only times it draws 25W is during boot, when the main switching ASIC isn't yet managed. Disabling the unused ports doesn't change the power draw.
The -48/-48P version has two switching ASIC, so that explains the difference in power consumption... if I'd have to guess stated the difference I saw during boot, it would add some 15ish W at the plug (please keep in mind I'm in EU so 230V) for a grand total of ~50W.
 
Last edited:

bbqdt

Member
Sep 15, 2019
46
19
8
Does the 6610 *have* to be such a noisy and hungry beast-hog compared to the 6450/7250? Can something be done to bring it down in line with the others considering I won't be utilizing near max capacity? The ICX6610-48P for 100$ looks really attractive but I'm afraid it'll end up a source of frustration.

My ICX6610-24P idles around 75w.

48p will be around 110w. I think you can remove one of the poe boards though?
 
  • Like
Reactions: 3nodeproblem

infoMatt

Active Member
Apr 16, 2019
157
60
28
I think you can remove one of the poe boards though?
Maybe, but the PoE board by themselves doesn't draw much, maybe you can shave off a couple of watts, but is it worth the effort? I suspect that the major difference is due to the +54V supply rail idle draw than the boards. If there's no PD attached of course, in that case you have to add the power needed by the devices.
 
  • Like
Reactions: 3nodeproblem

3nodeproblem

New Member
Jun 28, 2020
27
10
3

My ICX6610-24P idles around 75w.

48p will be around 110w. I think you can remove one of the poe boards though?
Oh wow, that fan mod is impressive and admittedly a bit above my level

From the replies I gather that the surplus draw from the 6610 and 48 port models is high enough that I should disregard them for now and keep sending out offers for 6450/7150/7250-24p until something sticks (energy is not just about cost but also about ecology IMO!)
 
  • Like
Reactions: fohdeesha

pr09

New Member
Jun 21, 2020
6
0
1
Ended up mounting my ICX7250-24P with a CyberPower 4POSTRAIL kit. A bit overkill, but definitely no sag, and it's close enough to 1U to fit in right below a shelf. No width issues, which I was concerned about with some other shelves.

Now I'm onto fan woes. Tried the Noctua, but it couldn't keep it cool -- over 70C completely idle, even on full speed. Got the Sunon MF40201VX-1000U-G99, and it can keep it cool on full speed, but is very noticeably whiny. On low it's quiet, but goes over 90C. Time to keep trying more options, I suppose.

With all these fan changes, I realized that the unusual pinout is actually just the regular pinout backwards. So, rather than continuing to splice cables, I just bent the tab down and plugged it in backwards. Works perfectly, and doesn't make the fans unusable for other purposes.

 

pr09

New Member
Jun 21, 2020
6
0
1
With a lot of those plugs you can just use a pin to release the actual wire+contact from the plastic housing and swap things around.
Well I feel silly now. Thank you, a couple minutes after reading your comment I now have two pigtails that do exactly the swap needed.
 

Jason Antes

Member
Feb 28, 2020
76
20
8
Twin Cities
How big are those fans? I have some doubled up fans out of Proliant servers, they are really loud at full power but pretty quiet when spun down a bit. There was a lot of design research into those fans to move air efficiently.