Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

noduck

New Member
Sep 12, 2020
29
7
3
The issue you may have with that is the 7250 thinks it can send 10Gbps of traffic out that interface and the transceiver will drop up to 50% or 75% of the traffic since the link cannot send the traffic load. Normally flow control would catch that but in this case the remote end is not receiving more than it can handle since the packets will be dropped at the source interface, so it will never send flow control pause to slow it down. That means it's up to TCP to fix the problem with resends and the problem gets even worse.
Hmm... sounds like a valid concern. I haven't looked at all the configuration options yet, but would it be possible to address the issue with some kind of rate limiting?
 

infoMatt

Active Member
Apr 16, 2019
222
99
28
Is it normal to have the sensor 1 temperature this high?
No, it's definitely not normal unless you are in a 50+°C room...
This is the problem of the "quieter" fans, they can move a lot less air, or with a lot less pressure, both resulting in reduced airflow over the heatsink than the designed and higher temperatures/lower useful life expectancy for the component.
You have to decide if the downside balances with the lower noise or not.
 

dswartz

Active Member
Jul 14, 2011
590
74
28
Switches came in a couple of hours ago. Had to fumble through a bit, due to needing to upgrade OS from 2016. All good now - 2 node linear stack up and running. Off to the races!
 
  • Like
Reactions: ICXGURU

dodgy route

Member
Aug 12, 2020
42
57
18
Australia
So, I have successfully halved my temps on my ICX6610-48P.
About 10 or 11 pages back I identified the noise is too much and went on my way...

Had to wait for 2 Noctua 200mm black fans and a pulse generator to show up from Amazon US, everything else I had sitting around already.

Had some 3mm acrylic/perspex or whatever this is that came from an old telecom rack front door, the whole thing was destined to be recycled. A part of this door will be used to replace the lid of the ICX6610.
I didn't want to hack the case lid for my fan mods because... it is not necessary in any way, its pretty much fully reversible modification.
I cut out the shape according to the switch lid flipped and traced it out on the sheet, when it was cut I marked and drilled the holes precisely while the lid was on so the holes are exactly where they need to be. The holes were also countersunk for the original lid screws to be used, perfect.




Meantime my pulse generator arrived and I did 2 things.
1) Create an excel spreadsheet to record the Noctua 100mm fan RPM Hz, while testing the PWM output from a spare mobo/cpu setup.
I used my cheap digital multimeter that can measure temperature and frequency for this, you just need the positive probe on my model.
HzReported mobo RPMCalc RPM from Hz: (Hz * 60) / 2Duty %
29.1862873100
27.782583190
22.065066070
16.348548950
11.534334535

Based with this information and a similar fan model as the switch ones I added a formula into my spreadsheet to calculate RPM from Hz or Hz from RPM for playing with.
Here is the google sheet you can use to calculate whatever you need for fans
My pulse generator calculator and 200mm Noctua info Google Sheet

2) Used my multimeter to test out the pulse generator with my new found calculations from the excel spreadsheet information. I tested 750Hz and got 22500 RPM exactly on the motherboard, just as per my calculator :)




I tested a few different Hz on the mobo once I adjusted the pulse generator accordingly for same output, I used 666Hz for 20000 RPM but anything in this region appears to work fine for the switch, works perfectly, use the multimeter on the pulse generator to set the required Hz (ie RPM required for switch) - At this point I didn't actually measure the switch fans yet, but no matter... just before gutting the fans from the switch I actually measured the frequency and got 650Hz on all 3 fans, so 666 remained..

Next, with the pulse generator worked out, was to do the physical work for mounting the fans as well as wiring, piece of cake.
For the fans, I laid out the fans where I wanted them, covering PSU and the CPU's, they are ginormous so coverage was not difficult, nor placement, and used a scriber thing (anything thats sharp will work) to mark the holes to cut out.



I used a dremel 561 bit which is excellent for cutting this plastic sheet, made it very easy and controllable, even with my crappy cheap non dremel rip off.
The fan mounting holes were also scribed and drilled out to accept regular fan screws of 5mm, also countersunk from the bottom so they dont stick out from under the plastic sheet.


I had some anti noise strip things from years ago I used anywhere where fans contact the sheet and sheet contacts the case, its visible in one of the photos, its stuck not on the switch but on the sheet itself, use masking tape or something that draws on the plastic so you can see it from other side when putting the tape on.
Anyways fans mounted and that part was done, just before I attached the lid on with screws, I put some hot glue over all the fan screws so they are insulated from any possible contact with any switch electronics.


For the PSU to be cooled effectively, I removed the metal top lid and traced out the fan hole, I cut out this portion and kept it, its easily stuck back together when/if returning to stock, you will need it if you are putting on the metal lid on as things are going to short out likely without it..


Wiring was actually piece of cake but YMMW, but im no stranger to wiring and I have experience from more complex wiring jobs through my hobbies...
Anyways I found this instead of me describing it too much, from another thread on this forum, even tho i measured my own to confirm as I recommend anyways...


No stock switch wiring was harmed, I simply removed every part of it from the fan tray and the PSU, the whole rubbish thing is removed in one piece so if it ever gets passed on as stock, its just a simple return operation.

I had also found a very old and broken fan controller that used to be on a pci bracket, I managed to cut the PCB to just fit in the fan tray sideway to have adjustability from outside easily. The broken part was repaired, the regulator snapped off 2 or the 3 soldering pads, so emergency repairs to anchor it down and some wire to repair the electrical connection to the tracks seems to have done the job. Works great now.

Use the diagram above, but for a front to back flow switch you need the 3 grounds, pulse generator and fan controller connected together (and fans grounds if not using controller).
The 12V power is on the bottom 3rd pin from the left, connect that to the pulse generator and fan controller (and fan power if not using controller).
Next pin, bottom, 4th pin from the left and one above it in top row need to be connected to the tone generator output, as well as the power supply fan RPM input too.

My wiring is basically from old motherboard header pins, think reset and power switches, just remove the plastic bits and add a bit of heat shrink, they hold well, needed smaller wire pin for the psu fan header tho, so thats something to consider..


The single green wire going to the PSU, is for the PSU fan trickery


My fans are PWM but im not feeding them PWM, I used some scrap old 3pin pass through and normal connectors from other projects so I can remove lid easily with one connector easily accessed at the back of the switch, at the fan controller output.


All done, just put the lid on,


I forgot to take a pic of the lid screws in place, but they are and the thing is in the rack for testing over next week how it does, temps are 1/2'd since fan switch over.
I also no longer run the blanking plate on the second fan tray. May make a mesh cover to keep any possible larger creatures out of it before I wire it in to my network next week.


This thing is quiet as a mouse 1 meter away, so shouldnt disturb you if your head is next to it... let alone in a rack somewhere else :)
Speaking of racks, I hope no one has missed the fact my switch is now 2U, something to consider if you are vertically challenged...

The 200mm fans output a LOT of air, its good stuff, also my power draw at the wall is 112W once the switch is booted with only console cable.
Old fan config is 3x40mm 0.81A, new config is 2x200mm 0.08A and a lot more flow.

Here is the whole album link for teardown plus fan mods all pics I took. Includes videos of noise, tho was hard to capture noise after mods, my daughter was lightly playing in the background so its more background noise really then the switch itself, this was fans around 1/2 to 2/3rds speed on the controller..
No fan errors or anything weird going on, as the pulse generator is doing its thing, quite well too.

Here is a shot of temps after 3 hours of on time in the rack with much other heat/etc, keep in mind this is with now closer to 1/2 speed on the fan controller.
If I turn the fan controller up to 11, its still quiet - at 1m hardly audible, in a rack not a chance.
Code:
SSH@bigbeef>show chassis
The stack unit 1 chassis info:

Power supply 1 not present
Power supply 2 (AC - PoE) present, status ok
        Model Number:   23-0000142-02
        Serial Number:  WME
        Firmware Ver:    A
Power supply 2 Fan Air Flow Direction:  Front to Back

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 not present

Fan controlled temperature: 41.0 deg-C

Fan speed switching temperature thresholds:
                Speed 1: NM<----->84       deg-C
                Speed 2:       79<-----> 87 deg-C (shutdown)

Fan 1 Air Flow Direction:  Front to Back
MAC 1 Temperature Readings:
        Current temperature : 38.0 deg-C
MAC 2 Temperature Readings:
        Current temperature : 40.5 deg-C
CPU Temperature Readings:
        Current temperature : 39.5 deg-C
sensor A Temperature Readings:
        Current temperature : 27.5 deg-C
sensor B Temperature Readings:
        Current temperature : 29.5 deg-C
sensor C Temperature Readings:
        Current temperature : 29.5 deg-C
stacking card Temperature Readings:
        Current temperature : 41.0 deg-C
        Warning level.......: 84.0 deg-C
        Shutdown level......: 87.0 deg-C
Boot Prom MAC : cc4e.243b.a2bc
Management MAC: cc4e.243b.a2bc
Cheers and enjoy!
 
Last edited:

motoridersd

New Member
Sep 26, 2020
3
0
1
A friend pointed me to this thread after I was complaining about how shitty a Netgear business switch I just got was (buggy as hell and the web interface is awful), and wishing there was an affordable powerful CLI based switch. I had completely forgotten to search for Brocade used switches and had only tried Cisco, which were expensive and missing PoE. Used to have all Brocade at an old job and we really liked them.

Anyway, it looks like the 6430-24P would be a perfect solution for what I need. I just need at least 16 ports and to power a couple of PoE devices. Fanless should be plenty quiet for having inside the media cabinet.

Do these require a license? Anything I should be looking for when buying on eBay?

Thanks!
 

dswartz

Active Member
Jul 14, 2011
590
74
28
So I'm in the process of attaching the rackmount ears to the 2 7250-24 switches. Ran into an issue straight off. Although the printed labels on the 2 packages (one for each switch) containing the 8 screws used to attach the rackmount ears, the screws from one envelope just will not go in. I finally took one screw from each package and butted them head to head. I can see that the 'bad' screws are just a tiny bit wider - enough that they will not go into the screw holes on the side of the switch :( Does anyone know where I can obtain the right mounting hardware? Thanks!
 

dswartz

Active Member
Jul 14, 2011
590
74
28
I think I see the problem. Switch is supposed to include 8 6-32 sink head screws (for front mounting) and 8 8-32 sink-head screws for rear mounting. I seem to only have one bag of each. Time to search the boxes and see if I overlooked something...
 
  • Like
Reactions: tommybackeast

dswartz

Active Member
Jul 14, 2011
590
74
28
Well, that's what happens when you unpack too much at once :) Found the 2nd bag of 6-32 screws under something else.
 

infoMatt

Active Member
Apr 16, 2019
222
99
28
Do these require a license? Anything I should be looking for when buying on eBay?
No it should not need any license, otherwise the solution is written on the first few pages :)
Keep in mind that the 6430 is 1G only and only L2, it does not route packets between VLAN, should you need to do this you have to hop through an external router. Also, the PoE version is not fanless... only the "compact" 12 ports and the 6430 without PoE are fanless devices.
 
  • Like
Reactions: fohdeesha

motoridersd

New Member
Sep 26, 2020
3
0
1
Also, the PoE version is not fanless... only the "compact" 12 ports and the 6430 without PoE are fanless devices.
Ah! Thanks for this! Do you know what the noise level of these would be? Would the 6450 fan statements apply to the 6430?

Keep in mind that the 6430 is 1G only and only L2, it does not route packets between VLAN, should you need to do this you have to hop through an external router
And yes, that's fine. I use pfSense to handle all my VLAN routing, and 1G is perfectly fine for the basic setup I have at home. I don't think the Layer 3 capabilities of the 6450 would get much use in my case.
 

infoMatt

Active Member
Apr 16, 2019
222
99
28
Ah! Thanks for this! Do you know what the noise level of these would be? Would the 6450 fan statements apply to the 6430?
It depends on the "past life" of the unit and the fan chosen at the assembly, some fans are louder than other. But IIRC, the 6430 should have only one fan, instead of the two used on the 6450 (for 24 port version, the 6450-48p should have three fans). It's not a jet engine, but you don't want it near you for more than a couple of hours.
The fan models used should carry over from the 6450 to the 30, they are essentially "twins".

If the price difference is not that high, I'd opt for a 6450 instead of a 6430, it could still be used as an L2 only device, but if you want it to route in the future, it can. Ultimately, it's your choice ;) Both will serve you well.
 

motoridersd

New Member
Sep 26, 2020
3
0
1
Thanks a lot! So far it seems a 6430-24p is about $50 less than a 6450-24p, which at this price range is almost 50% more.

I assume the chassis are the same between the two 24p models, so I expect replacing the fan with a quieter one should be the same process on the 6430 as in the 6450. If the same modification options apply, I could probably leave only one fan on since my PoE load will be low.

I'll see what eBay has in the next few days
 

infoMatt

Active Member
Apr 16, 2019
222
99
28
I assume the chassis are the same between the two 24p models, so I expect replacing the fan with a quieter one should be the same process on the 6430 as in the 6450.
Yes they are basically the same... maybe one is a bit shorter than the other, but the chassis construction is the same.

at this price range is almost 50% more.
Yes, ok that is a pretty good decision factor. :)
 
  • Like
Reactions: motoridersd

fohdeesha

Kaini Industries
Nov 20, 2016
2,577
2,775
113
31
fohdeesha.com
So, I have successfully halved my temps on my ICX6610-48P.
About 10 or 11 pages back I identified the noise is too much and went on my way...

Had to wait for 2 Noctua 200mm black fans and a pulse generator to show up from Amazon US, everything else I had sitting around already.

Had some 3mm acrylic/perspex or whatever this is that came from an old telecom rack front door, the whole thing was destined to be recycled. A part of this door will be used to replace the lid of the ICX6610.
I didn't want to hack the case lid for my fan mods because... it is not necessary in any way, its pretty much fully reversible modification.
I cut out the shape according to the switch lid flipped and traced it out on the sheet, when it was cut I marked and drilled the holes precisely while the lid was on so the holes are exactly where they need to be. The holes were also countersunk for the original lid screws to be used, perfect.




Meantime my pulse generator arrived and I did 2 things.
1) Create an excel spreadsheet to record the Noctua 100mm fan RPM Hz, while testing the PWM output from a spare mobo/cpu setup.
I used my cheap digital multimeter that can measure temperature and frequency for this, you just need the positive probe on my model.
HzReported mobo RPMCalc RPM from Hz: (Hz * 60) / 2Duty %
29.1862873100
27.782583190
22.065066070
16.348548950
11.534334535

Based with this information and a similar fan model as the switch ones I added a formula into my spreadsheet to calculate RPM from Hz or Hz from RPM for playing with.
Here is the google sheet you can use to calculate whatever you need for fans
My pulse generator calculator and 200mm Noctua info Google Sheet

2) Used my multimeter to test out the pulse generator with my new found calculations from the excel spreadsheet information. I tested 750Hz and got 22500 RPM exactly on the motherboard, just as per my calculator :)




I tested a few different Hz on the mobo once I adjusted the pulse generator accordingly for same output, I used 666Hz for 20000 RPM but anything in this region appears to work fine for the switch, works perfectly, use the multimeter on the pulse generator to set the required Hz (ie RPM required for switch) - At this point I didn't actually measure the switch fans yet, but no matter... just before gutting the fans from the switch I actually measured the frequency and got 650Hz on all 3 fans, so 666 remained..

Next, with the pulse generator worked out, was to do the physical work for mounting the fans as well as wiring, piece of cake.
For the fans, I laid out the fans where I wanted them, covering PSU and the CPU's, they are ginormous so coverage was not difficult, nor placement, and used a scriber thing (anything thats sharp will work) to mark the holes to cut out.



I used a dremel 561 bit which is excellent for cutting this plastic sheet, made it very easy and controllable, even with my crappy cheap non dremel rip off.
The fan mounting holes were also scribed and drilled out to accept regular fan screws of 5mm, also countersunk from the bottom so they dont stick out from under the plastic sheet.


I had some anti noise strip things from years ago I used anywhere where fans contact the sheet and sheet contacts the case, its visible in one of the photos, its stuck not on the switch but on the sheet itself, use masking tape or something that draws on the plastic so you can see it from other side when putting the tape on.
Anyways fans mounted and that part was done, just before I attached the lid on with screws, I put some hot glue over all the fan screws so they are insulated from any possible contact with any switch electronics.


For the PSU to be cooled effectively, I removed the metal top lid and traced out the fan hole, I cut out this portion and kept it, its easily stuck back together when/if returning to stock, you will need it if you are putting on the metal lid on as things are going to short out likely without it..


Wiring was actually piece of cake but YMMW, but im no stranger to wiring and I have experience from more complex wiring jobs through my hobbies...
Anyways I found this instead of me describing it too much, from another thread on this forum, even tho i measured my own to confirm as I recommend anyways...


No stock switch wiring was harmed, I simply removed every part of it from the fan tray and the PSU, the whole rubbish thing is removed in one piece so if it ever gets passed on as stock, its just a simple return operation.

I had also found a very old and broken fan controller that used to be on a pci bracket, I managed to cut the PCB to just fit in the fan tray sideway to have adjustability from outside easily. The broken part was repaired, the regulator snapped off 2 or the 3 soldering pads, so emergency repairs to anchor it down and some wire to repair the electrical connection to the tracks seems to have done the job. Works great now.

Use the diagram above, but for a front to back flow switch you need the 3 grounds, pulse generator and fan controller connected together (and fans grounds if not using controller).
The 12V power is on the bottom 3rd pin from the left, connect that to the pulse generator and fan controller (and fan power if not using controller).
Next pin, bottom, 4th pin from the left and one above it in top row need to be connected to the tone generator output, as well as the power supply fan RPM input too.

My wiring is basically from old motherboard header pins, think reset and power switches, just remove the plastic bits and add a bit of heat shrink, they hold well, needed smaller wire pin for the psu fan header tho, so thats something to consider..


The single green wire going to the PSU, is for the PSU fan trickery


My fans are PWM but im not feeding them PWM, I used some scrap old 3pin pass through and normal connectors from other projects so I can remove lid easily with one connector easily accessed at the back of the switch, at the fan controller output.


All done, just put the lid on,


I forgot to take a pic of the lid screws in place, but they are and the thing is in the rack for testing over next week how it does, temps are 1/2'd since fan switch over.
I also no longer run the blanking plate on the second fan tray. May make a mesh cover to keep any possible larger creatures out of it before I wire it in to my network next week.


This thing is quiet as a mouse 1 meter away, so shouldnt disturb you if your head is next to it... let alone in a rack somewhere else :)
Speaking of racks, I hope no one has missed the fact my switch is now 2U, something to consider if you are vertically challenged...

The 200mm fans output a LOT of air, its good stuff, also my power draw at the wall is 112W once the switch is booted with only console cable.
Old fan config is 3x40mm 0.81A, new config is 2x200mm 0.08A and a lot more flow.

Here is the whole album link for teardown plus fan mods all pics I took. Includes videos of noise, tho was hard to capture noise after mods, my daughter was lightly playing in the background so its more background noise really then the switch itself, this was fans around 1/2 to 2/3rds speed on the controller..
No fan errors or anything weird going on, as the pulse generator is doing its thing, quite well too.

Here is a shot of temps after 3 hours of on time in the rack with much other heat/etc, keep in mind this is with now closer to 1/2 speed on the fan controller.
If I turn the fan controller up to 11, its still quiet - at 1m hardly audible, in a rack not a chance.
Code:
SSH@bigbeef>show chassis
The stack unit 1 chassis info:

Power supply 1 not present
Power supply 2 (AC - PoE) present, status ok
        Model Number:   23-0000142-02
        Serial Number:  WME
        Firmware Ver:    A
Power supply 2 Fan Air Flow Direction:  Front to Back

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 not present

Fan controlled temperature: 41.0 deg-C

Fan speed switching temperature thresholds:
                Speed 1: NM<----->84       deg-C
                Speed 2:       79<-----> 87 deg-C (shutdown)

Fan 1 Air Flow Direction:  Front to Back
MAC 1 Temperature Readings:
        Current temperature : 38.0 deg-C
MAC 2 Temperature Readings:
        Current temperature : 40.5 deg-C
CPU Temperature Readings:
        Current temperature : 39.5 deg-C
sensor A Temperature Readings:
        Current temperature : 27.5 deg-C
sensor B Temperature Readings:
        Current temperature : 29.5 deg-C
sensor C Temperature Readings:
        Current temperature : 29.5 deg-C
stacking card Temperature Readings:
        Current temperature : 41.0 deg-C
        Warning level.......: 84.0 deg-C
        Shutdown level......: 87.0 deg-C
Boot Prom MAC : cc4e.243b.a2bc
Management MAC: cc4e.243b.a2bc
Cheers and enjoy!
and now I can finally close this thread
 

tommybackeast

Active Member
Jun 10, 2018
290
111
43
if anyone wants a cheap lab box with 2x 10gbE, this guy is selling great condition FCX's with the 10gbE card for $29 plus shipping. These will do VRFs, BGP and OSPF for both IPv4 and IPv6, tunnels, IPv6 in IPv4 encap, etc Brocade FastIron FCX648S, GIGABIT 48 Port Switch w/ 2 Power Supplies 729198041304 | eBay
is this FCX648S in your list of "loudest switch I ever heard" category?


The 4 SFP Ports : are they 1GB SFP or 10GB SFP+ ? What does the two slots marked X1 and X2 do?
 

LodeRunner

Active Member
Apr 27, 2019
455
197
43
is this FCX648S in your list of "loudest switch I ever heard" category?


The 4 SFP Ports : are they 1GB SFP or 10GB SFP+ ? What does the two slots marked X1 and X2 do?
The X1 and X2 slots look to be XFP which can be 10Gb. I suspect the 4x SFP are regular SFP for 1Gb, not SFP+.

Edit: Yeah, that's a dual XFP card: Brocade FCX-2XG 2x 10G Ethernet Module for FCX Series Switches SKJ - Dedicated Networks, Inc.