Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

CIR-Engineering

I am a functional adult?
Jan 14, 2021
153
61
28
49
Chicago USA
www.cir-engineering.com
And another thing. Is this a CPU issue, a PoE issue, both? I'm trying to narrow down where to look on the board for problems. So often these are easy to fix. I have a few caps (but they are near the power supply) that are measuring as shorted so possibly there is one bad one somewhere (could also be normal). I may need to remove the heat sinks but then I'll probably also need to replace the thermal pads... Anyone else do SMD rework here? I'm not very familiar with the 7450 topography so I am looking for help on where to look on the PCB. The way I read the diagnostic error is that watchdog is restarting or shutting down one core of the CPU?
 

nutsnax

Active Member
Nov 6, 2014
347
151
43
114
I picked up a 7250 and can't use the 10gbe SFP+ ports as uplinks to other switches only as standard ports for servers. Is there some mode that I need to set on the switch to allow me to uplink to other switches?
 

jei

Active Member
Aug 8, 2021
193
117
43
Finland
I picked up a 7250 and can't use the 10gbe SFP+ ports as uplinks to other switches only as standard ports for servers. Is there some mode that I need to set on the switch to allow me to uplink to other switches?
Do you mean stacking? All 7250 models support SFP+ uplinks (data), all models except 7250-24G support stacking.
 

nutsnax

Active Member
Nov 6, 2014
347
151
43
114
Do you mean stacking? All 7250 models support SFP+ uplinks (data), all models except 7250-24G support stacking.
Ahh, so Uplinking is called Stacking in Brocade-land. Got it. Is that enabled on a per-port basis? If so can I plug a standard NIC into it afterward or would stacking need to be disabled?
 

jei

Active Member
Aug 8, 2021
193
117
43
Finland
Ahh, so Uplinking is called Stacking in Brocade-land. Got it. Is that enabled on a per-port basis? If so can I plug a standard NIC into it afterward or would stacking need to be disabled?
I'm by no means stacking expert but from reading the manuals and this thread, stacking needs to be disabled for "normal" switching on the port. (edit: It's port based, yes.)
 

CIR-Engineering

I am a functional adult?
Jan 14, 2021
153
61
28
49
Chicago USA
www.cir-engineering.com
Yeah this is the decision I most hate facing. Will this partially working through work well enough for me at least long enough that I can get another one to replace it, or do I yeet my cash back and do some more waiting.

As a in between measure, you might be able to talk the seller into a partial refund. If they were selling it as “tested and working”, but the built in self diagnostics say it’s faulty, then it’s not really as described.
If you can get a bit of your money back for the reduced functionality, it certainly makes it easier to keep in use knowing you need to be ready to replace it at potentially short notice..
Well for now partial refund it is! They gave me back $337 + tax (11% here) so now instead of about $640, I paid $246.

^Well, kinda like dating a dime if the mental stability isn't there. Do you REALLY want it, and if so how long can you last before you lose it...
For $246 I'll date the dime any time and see how crazy she is in the long run. It's from eBay Fastcrap whom I've bought from many times; they are awesome and have a hilarious name. It was advertised as fully functional and they immediately offered to take it back, but I asked about a partial refund, they offered $250. I sent a note and a counteroffer of $337 back, they said OK. I'll install the switch tonight in a working environment and make sure she's not completely dysfunctional. With eBay I can still get a full refund for 30 days if she's totally crazy.

I am still hoping to fix this switch. I don't have a thermal camera now and that would help, [EDIT: pulled camera out of packing for move to new house], but I am quite good at tracking down problems. It could be a problem with an important chip, but there is a very high chance it's just a shorted cap or fried resistor someplace. After I run it for a week and have some free time (like that ever happens) I'll try and find what's fried again.

Thanks to you all for your technical help, opinions, and encouragement.

Kind regards to fohdeesha and all of you who have been there for me. I truly appreciate all of your assistance!
 
Last edited:
  • Like
Reactions: TonyArrr and jei

TonyArrr

Active Member
Sep 22, 2021
169
85
28
Straylia
Ahh, so Uplinking is called Stacking in Brocade-land. Got it. Is that enabled on a per-port basis? If so can I plug a standard NIC into it afterward or would stacking need to be disabled?
Stacking logically combines the two or more switches so when you admin them, you just connect to one of them and can apply config to any port on any of the stacked switches. If you have more than two switches, you basically make a ring connection going through all of them.

You can also only stack compatible models. For example, I can’t stack my 7150-24 with my 7250-24.

Take a look at the stacking guide in the docs that come with @fohdeesha’s download
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,921
3,457
113
34
fohdeesha.com
Well for now partial refund it is! They gave me back $337 + tax (11% here) so now instead of about $640, I paid $246.



For $246 I'll date the dime any time and see how crazy she is in the long run. It's from eBay Fastcrap whom I've bought from many times; they are awesome and have a hilarious name. It was advertised as fully functional and they immediately offered to take it back, but I asked about a partial refund, they offered $250. I sent a note and a counteroffer of $337 back, they said OK. I'll install the switch tonight in a working environment and make sure she's not completely dysfunctional. With eBay I can still get a full refund for 30 days if she's totally crazy.

I am still hoping to fix this switch. I don't have a thermal camera now and that would help, [EDIT: pulled camera out of packing for move to new house], but I am quite good at tracking down problems. It could be a problem with an important chip, but there is a very high chance it's just a shorted cap or fried resistor someplace. After I run it for a week and have some free time (like that ever happens) I'll try and find what's fried again.

Thanks to you all for your technical help, opinions, and encouragement.

Kind regards to fohdeesha and all of you who have been there for me. I truly appreciate all of your assistance!
I would just forget all this happened and use it like it's brand new, because if I'm understanding you correctly, it operates perfectly. I wouldn't give too much credence to the output/results of dm commands, they weren't designed for end user use, a lot have to be ran in very specific environments to give reliable/believable output, and a ton of them just outright don't work on certain platforms. Regarding the PoE error that sometimes comes up, granted it's only sometimes, that leads me to believe even more what I said earlier, it's probably the PoE board trying to come up to soon before the PHY has or some other weird race condition. You can try updating to a newer codetrain like v9 to see if it goes away
 
  • Like
Reactions: CIR-Engineering

CIR-Engineering

I am a functional adult?
Jan 14, 2021
153
61
28
49
Chicago USA
www.cir-engineering.com
I would just forget all this happened and use it like it's brand new, because if I'm understanding you correctly, it operates perfectly. I wouldn't give too much credence to the output/results of dm commands, they weren't designed for end user use, a lot have to be ran in very specific environments to give reliable/believable output, and a ton of them just outright don't work on certain platforms. Regarding the PoE error that sometimes comes up, granted it's only sometimes, that leads me to believe even more what I said earlier, it's probably the PoE board trying to come up to soon before the PHY has or some other weird race condition. You can try updating to a newer codetrain like v9 to see if it goes away
Thanks for that.

EDIT 2: I don't know what I don't know or know. That AP has given trouble before and now everything is running fine all day. I'm just going to deploy the switch tonight while everyone is in bed and we'll see what happens. Then I'll know for sure.

I have discovered that there is something wrong with PoE on some of the ports. There are two PoE chips for the regular PoE (High PoE is separate on ports 1-8) that cover 16x of the 1 GB ports (17-24) and the 8x 2.5 GbE ports (25-32). The PoE seems to cut in and out on the right-side chip (closest to the outside of the board) and right side 1 GbE ports (theory now). I monitored my PoE 1 GbE WiFi access points on port 11 and 19 all night. Eleven was perfect, but port 19 keeps going up and down. I have a bad feeling the flaky chip covers the 2.5 GbE ports (but I have nothing here now to test PoE that is 2.5 GbE. I'm hoping it's not actually the PoE IC that's the problem, but instead its power supply chip. I'll know more when I dig.

But this could put a large obstacle in my deployment of 2.5 GbE WiFi access points.

EDIT: I broke my own rule and posted before coffee.

I also got an idea that I am trying now. I have PoE extractors that I am going to plug into the 2.5 GbE ports and power some things even if they aren't 2.5 GbE. I have one WiFi access point plugged into the 2.5 GbE ports now. It can't connect because it's only a 1 GbE AP, but it is drawing PoE. I'll monitor through serial and see if the PoE stays up.
 
Last edited:

Wolfcastle

Member
Jan 3, 2022
57
30
18
This has maybe been asked in this thread but has anyone tried to connect a 10Gbps Wifi 7 AP on the 10G ports for a 6610?

I am specifically looking at the TP-Link EAP773. I know it's not ideal in the sense that I would need to use a 10G port with a 10GBase-T SFP+ and a POE injector but I assume it should work right?
 

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
Having trouble with my 10g ports on a 6610. I had a dual lag connection with my storage server and it was working fine. Left the original 1g connection on it. Came back an hour later and the 10g ports were not connected anymore. Reverted back to the 1g connection and it works but nothing I do can make the 2 10g connections come back up. Moved the sfp adapters around, tried a 10g connection to another switch, works fine. A patch cable looped works fine. Tried clearing everything in cache, rebooting warm and cold. Any suggestions?

Edit. I am using an HP branded mellanox connectx3 dual card. Seems there might be issues with that card.
I had something very similar to this (search under my name in this thread) - 3 x ESXi hosts in a cluster - trying to use dual 10Gb ethernet ports (on a variety of different cards to the same switch - could not consistently get both connections to run - tried across various versions of ESXi (6.7, 7,8) and still the same - each time a restarted a host i was a fair chance of losing one of the 10GB connections and the only fix was a switch reset. Happened across multip 6610 as well.

In the end i gave up and use the 40GB ports for two of the hosts.

IN the other one i had to place two single port 10GB cards- dual port just would not do it

Craig
 

CIR-Engineering

I am a functional adult?
Jan 14, 2021
153
61
28
49
Chicago USA
www.cir-engineering.com
I would just forget all this happened and use it like it's brand new, because if I'm understanding you correctly, it operates perfectly. I wouldn't give too much credence to the output/results of dm commands, they weren't designed for end user use, a lot have to be ran in very specific environments to give reliable/believable output, and a ton of them just outright don't work on certain platforms. Regarding the PoE error that sometimes comes up, granted it's only sometimes, that leads me to believe even more what I said earlier, it's probably the PoE board trying to come up to soon before the PHY has or some other weird race condition. You can try updating to a newer codetrain like v9 to see if it goes away
Updated to 09010j and so far no PoE restarts... fingers crossed.

Web GUI is even more useless, but much prettier ;)
 

CIR-Engineering

I am a functional adult?
Jan 14, 2021
153
61
28
49
Chicago USA
www.cir-engineering.com
Snagged an R750 last night for $70 on eBay! Seller later wrote he mispriced but would ship anyway. I feel kinda bad for him, I think they left off a zero. I might write back and let him cancel? I also need four and don’t know if I’ll want to pay for more 750’s.
 
  • Wow
Reactions: fohdeesha

fohdeesha

Kaini Industries
Nov 20, 2016
2,921
3,457
113
34
fohdeesha.com
Snagged an R750 last night for $70 on eBay! Seller later wrote he mispriced but would ship anyway. I feel kinda bad for him, I think they left off a zero. I might write back and let him cancel? I also need four and don’t know if I’ll want to pay for more 750’s.
Damn, I thought the $200 employee pricing I got a couple years ago was good. They are incredible APs
 
  • Like
Reactions: CIR-Engineering

CIR-Engineering

I am a functional adult?
Jan 14, 2021
153
61
28
49
Chicago USA
www.cir-engineering.com
Damn, I thought the $200 employee pricing I got a couple years ago was good. They are incredible APs
Yeah, it has Unleashed already on it too... And it's new, not used. I'm wondering if just one could manage our new house instead of four 650's or something. I did have them run dual Ethernet right to the panty in the smack middle of the house and between the basement and second floor.

EDIT: They already shipped it!

Rebooted the switch and still no PoE errors. It's been running perfectly for twelve hours (still fingers crossed).

So hopefully (fingers crossed) I got a fully usable ICX7450-32ZP that may or may not have some problems that aren't relevant at $237, and stumbled on an R750 for $77. If this holds I'm going to Vegas!

I once bought two Lexicon LX7's off eBay that were sold as-is not working (audio dropping in and out on multiple channels) with grainy photos that didn't look great. Paid $1200 shipped for the pair. Got them here and repaired them in less than three hours, AND they cleaned up to 9.5/10. That was my best score on eBay ever and they are still in my system today. But if this 32ZP holds this will be a close second when combined with the R750.

Sometimes you win and sometimes you lose. This has been a good week (keeping fingers crossed!).
 
Last edited:

CircuitSlinger

New Member
May 30, 2024
5
3
3
www.youtube.com
I was one of the other winners on those ICX7450-32ZPs. Currently running the Diags just to make sure I don't run into any of the issues described by CIR-Engineering. I also ordered some extra SFP+ modules to replace the QSFP modules on the rear, hoping to replace my ICX6610-48P.
 
Last edited:
  • Like
Reactions: CIR-Engineering

CIR-Engineering

I am a functional adult?
Jan 14, 2021
153
61
28
49
Chicago USA
www.cir-engineering.com
I was one of the other winners on those ICX7450-32ZPs. Currently running the Diags just to make sure I don't run into any of the issues described by CIR-Engineering. I also ordered some extra SFP+ modules to replace the QSFP modules on the rear, hoping to replace my ICX6610-48P.
PLEASE! report back with your findings either way! I’m highly curious if mine is in fact “normal” or divergent.

I will probably also get one or two SFP+ modules at some point. I have zero need for 40 GbE but sometimes could benefit from a couple more 10 GbE ports.

How much did you pay for yours? Some went for $405, lucky dogs. I did focus on cleaner looking units with rack ears and two power supplies though. I bid on every unit, but after I scored I never bid higher than $390. I was hoping one would slip through cheaper.
 

86turbodsl

Active Member
Feb 24, 2020
111
36
28
I had something very similar to this (search under my name in this thread) - 3 x ESXi hosts in a cluster - trying to use dual 10Gb ethernet ports (on a variety of different cards to the same switch - could not consistently get both connections to run - tried across various versions of ESXi (6.7, 7,8) and still the same - each time a restarted a host i was a fair chance of losing one of the 10GB connections and the only fix was a switch reset. Happened across multip 6610 as well.

In the end i gave up and use the 40GB ports for two of the hosts.

IN the other one i had to place two single port 10GB cards- dual port just would not do it

Craig
Yeah I played around with it a good long while and think it's down to the mellanox card. The 10g ports on the other server are copper rj45 and work fine lacp. I have an Intel card on order to get rid of the mellanox. Not very impressed with that card or Nvidia support. Nothing I did was able to get that card to work with the provided tools. HP branded connectx3pro. I gave up.
 

CircuitSlinger

New Member
May 30, 2024
5
3
3
www.youtube.com
PLEASE! report back with your findings either way! I’m highly curious if mine is in fact “normal” or divergent.

I will probably also get one or two SFP+ modules at some point. I have zero need for 40 GbE but sometimes could benefit from a couple more 10 GbE ports.

How much did you pay for yours? Some went for $405, lucky dogs. I did focus on cleaner looking units with rack ears and two power supplies though. I bid on every unit, but after I scored I never bid higher than $390. I was hoping one would slip through cheaper.
I paid $488 after shipping/tax. So far so good, self tests all passed. Little dissapointed in the power draw, I'm seeing around 120w from the wall when just sitting idle and messing around in the console, was hoping it was do a little better than the 6610. I had spare rack ears from a 7250 that should work, and I didn't care about dual PSU for my lab.

Edit
I forgot to put my Kil-A-Watt into Watts instead of Volts. Rechecking idle power draw numbers.
Boot: 68w
Idle: 105w

Still not the improvement I was hoping for.

Edit 2
ICX7250 rack ears work perfectly.
 
Last edited: