Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

itronin

Well-Known Member
Nov 24, 2018
1,234
793
113
Denver, Colorado
What are the temps though?

I had heat issues and went to 2 fans, 2 psu, and heat issues went away. They're REv A.
In one I have a REv B but can't tell the difference... debating if I should swap that B to A for more air flow.
I have a 8-11*C difference between the 6610 on top vs on bottom too, unsure if it's location, A vs B, or due to the rack-ear not having the middle\vent hole.
My original switch purchase had 2 PSU and 2 FANS in each unit. I've only been running 1 switch due to noise and playing with the 40Gbe. If I can source 1 more rev B I will try and figure out if it is really possible to stack with a single 40Gbe and use the remaining 2 for a LAG.

I've marked a seller on the bay that has qty 4 rev B power supplies (confirmed via message), asking a lot for 'em but none of have moved since it was posted. May take a lower number though it will likely be about what I paid for these 2 combined. But my ears, oh my. huge huge difference. my CSE-417 is now the loudest thing in the rack and I removed some fans from it to quiet it down a bit (all flash, 2/3 full, temps are fine there).

I figured since I did not hear the fans ramp up (except at power on) that all must be okay and I was really just floored by the sound difference so I didn’t even check temps.

Original configs:
#1, ICX6610-48P-E, 1xRev A PSU-E , 1x Rev B PSU-E, 2x FAN-E
#2, ICX6610-48P-E, 2xRev A PSU-E, 2xFAN-E

New config ICX
#1 ICX6610-48P-E, 2x Rev B PSU-E, 2x FAN-E,
#2, ICX6610-48P-E, 1xRev A PSU-E , 1x Rev B PSU-E, 2x FAN-E

Code:
icx6610-stack#show chassis
The stack unit 1 chassis info:

Power supply 1 (AC - PoE) present, status ok
     Model Number:   23-0000142-02
   Serial Number:   B1S     
   Firmware Ver:     B
Power supply 1 Fan Air Flow Direction:  Front to Back
Power supply 2 (AC - PoE) present, status ok
     Model Number:   23-0000142-02
   Serial Number:   5WP     
   Firmware Ver:     B
Power supply 2 Fan Air Flow Direction:  Front to Back

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 ok, speed (auto): [[1]]<->2

Fan controlled temperature: 65.5 deg-C

Fan speed switching temperature thresholds:
       Speed 1: NM<----->76       deg-C
       Speed 2:       71<-----> 80 deg-C (shutdown)

Fan 1 Air Flow Direction:  Front to Back
Fan 2 Air Flow Direction:  Front to Back                         
MAC 1 Temperature Readings:
   Current temperature : 45.0 deg-C
MAC 2 Temperature Readings:
   Current temperature : 54.0 deg-C
CPU Temperature Readings:
   Current temperature : 65.5 deg-C
sensor A Temperature Readings:
   Current temperature : 52.0 deg-C
sensor B Temperature Readings:
   Current temperature : 50.5 deg-C
sensor C Temperature Readings:
   Current temperature : 37.0 deg-C
stacking card Temperature Readings:
   Current temperature : 64.0 deg-C
   Warning level.......: 77.0 deg-C
   Shutdown level......: 80.0 deg-C
 
  • Like
Reactions: T_Minus

kapone

Well-Known Member
May 23, 2015
1,095
642
113
While that switch above is not exactly cooking...it is certainly very very warm. I'm surprised the fans haven't ramped up.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
While that switch above is not exactly cooking...it is certainly very very warm. I'm surprised the fans haven't ramped up.

Which? These just posted?

MAC 1 Temperature Readings:
Current temperature : 45.0 deg-C
MAC 2 Temperature Readings:
Current temperature : 54.0 deg-C
CPU Temperature Readings:
Current temperature : 65.5 deg-C
sensor A Temperature Readings:
Current temperature : 52.0 deg-C
sensor B Temperature Readings:
Current temperature : 50.5 deg-C
sensor C Temperature Readings:
Current temperature : 37.0 deg-C


Mine that wasn't working was running at 79 and 80*C until we fixed the fan, and air flow situation, now both are in the 40s and 50s *C and work great :D
 
  • Like
Reactions: itronin

ekke

Member
Nov 16, 2015
166
8
18
45
Bought a 6450-24p poe .
Get these poe syslog messages.
upload_2019-12-21_23-56-59.png

I also got a Sunon MagLev KDE1204PKV3-MS.AF.GN 3pin 18dB-A Super Silence from ebay, gotta remove that high pitch noise of the current fan.
 

Attachments

fohdeesha

Kaini Industries
Nov 20, 2016
2,728
3,075
113
33
fohdeesha.com
My original switch purchase had 2 PSU and 2 FANS in each unit. I've only been running 1 switch due to noise and playing with the 40Gbe. If I can source 1 more rev B I will try and figure out if it is really possible to stack with a single 40Gbe and use the remaining 2 for a LAG.

I've marked a seller on the bay that has qty 4 rev B power supplies (confirmed via message), asking a lot for 'em but none of have moved since it was posted. May take a lower number though it will likely be about what I paid for these 2 combined. But my ears, oh my. huge huge difference. my CSE-417 is now the loudest thing in the rack and I removed some fans from it to quiet it down a bit (all flash, 2/3 full, temps are fine there).

I figured since I did not hear the fans ramp up (except at power on) that all must be okay and I was really just floored by the sound difference so I didn’t even check temps.

Original configs:
#1, ICX6610-48P-E, 1xRev A PSU-E , 1x Rev B PSU-E, 2x FAN-E
#2, ICX6610-48P-E, 2xRev A PSU-E, 2xFAN-E

New config ICX
#1 ICX6610-48P-E, 2x Rev B PSU-E, 2x FAN-E,
#2, ICX6610-48P-E, 1xRev A PSU-E , 1x Rev B PSU-E, 2x FAN-E

Code:
icx6610-stack#show chassis
The stack unit 1 chassis info:

Power supply 1 (AC - PoE) present, status ok
     Model Number:   23-0000142-02
   Serial Number:   B1S    
   Firmware Ver:     B
Power supply 1 Fan Air Flow Direction:  Front to Back
Power supply 2 (AC - PoE) present, status ok
     Model Number:   23-0000142-02
   Serial Number:   5WP    
   Firmware Ver:     B
Power supply 2 Fan Air Flow Direction:  Front to Back

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 ok, speed (auto): [[1]]<->2

Fan controlled temperature: 65.5 deg-C

Fan speed switching temperature thresholds:
       Speed 1: NM<----->76       deg-C
       Speed 2:       71<-----> 80 deg-C (shutdown)

Fan 1 Air Flow Direction:  Front to Back
Fan 2 Air Flow Direction:  Front to Back                        
MAC 1 Temperature Readings:
   Current temperature : 45.0 deg-C
MAC 2 Temperature Readings:
   Current temperature : 54.0 deg-C
CPU Temperature Readings:
   Current temperature : 65.5 deg-C
sensor A Temperature Readings:
   Current temperature : 52.0 deg-C
sensor B Temperature Readings:
   Current temperature : 50.5 deg-C
sensor C Temperature Readings:
   Current temperature : 37.0 deg-C
stacking card Temperature Readings:
   Current temperature : 64.0 deg-C
   Warning level.......: 77.0 deg-C
   Shutdown level......: 80.0 deg-C
it doesn't even bother to ramp up fans until 76c, 65 is more than fine. these things run hot (which is why I keep yelling at people to stop wasting their time trying to stick even slower fans in the 6610)
 
  • Like
Reactions: itronin

itronin

Well-Known Member
Nov 24, 2018
1,234
793
113
Denver, Colorado
it doesn't even bother to ramp up fans until 76c, 65 is more than fine. these things run hot (which is why I keep yelling at people to stop wasting their time trying to stick even slower fans in the 6610)
yes, folks please listen to Uncle Fohdeesha. Just find some deals and invest in rev B psu's . I am still non-sonically stunned by how quiet two 6610 POE rev B's are. Fohdeesha, you posted about the acoustics early on in the thread - just hadn't found a deal until last week and it was SO worth it!
 

dashpuppy

Member
Dec 16, 2018
48
21
8
Well I kinda had a decent weekend. First up was installing the new Fans into the 6450,, AHHH!! much silent ! And still passing air and keeping it cool.

Next was installing 10gig nic's into the Hp ML350 G8 server ( Fail ) they don't work :(

Ordered some PCI-e 10gig HP nic's instead of these Lenovo Emulex ones !!

Oh ! And i don't have an addiction !! So i bought 2 of these Brocade ICX 6430-C12's for 100$ One will be connected to my Brocade 6450 via Fiber then sitting behind my desk :) The other will be spare ? maybe...
 

Attachments

eduncan911

The New James Dean
Jul 27, 2015
648
506
93
eduncan911.com
EDIT: found the link to the published sources, though I haven't extracted it yet (on vacation/mobile). Reading the README for 6450 though indicates that there indeed binary blobs from Brocade/Arris to include. Humm... Makes sense not to include source to this libs as these most likely aren't under GPL. Will have to disassemble them and inspect further.

---

Has anyone successfully pulled down the source code and built the firmware? It was mentioned on the first page that under GPL licensing they are publishing, slowly.

For which device though? And would the process work for both the 6450 48p and 7250 48p ARM CPUs?

I'm on the fence for a 6450 or 7250 for their low noise and source code availability. I have four 10 Gbps 10GbaseT devices right now and technically could have a 5th, which would me swing towards the 7250 series.

However, building the firmware (with little to no proprietary binary blobs) is one thing that really interests me.

Maybe someone could point us to the source code links? I'll go back through the thread again and see if it was brought up...
 
Last edited:

CED6688

New Member
Dec 4, 2019
15
10
3
Ok. Now have 6610-24 (2x RevA PSUs), 6450-24, and 7250-24P that were purchased in that order.

Since there is a lot of discussion on noise:
- The 6450-24 with stock fans was so quiet (38-42db) that I was able to move it to a bedroom closet and nobody knows it is there w/ 2 10gb and a dozen 1gb connections. Everyone has different sound tolerance, but I had multiple people help listen for it and we literally could not hear it -- I suspect that people with louder fans may have bad fans. It nicely replaced both a Mikrotik CRS305 and an older Unifi switch and barely consumes any power. It's incredible for the price and I would highly recommend it for most people that only need a few 10gig ports for a NAS or such.

- The 7250-24P is louder than the 6450 (45-48db). I would say mid-way between my 6450 and 6610. Definitely louder than the 6450, but is at a wife-acceptable volume level with no modifications (stock fans/etc...) from the next room with the door open with 6 10gb and a dozen 1gb connections. Wouldn't put in the bedroom closet or want to sit next to it for extended periods, but would not classify it as loud. Love that the latest OS has more straight-forward handling of ports with both tagged/untagged VLANs, etc... but wouldn't base a buying decision based on that. This replaced a Mikrotik CRS309 and two Unifi switches.

- The 6610-24 with 2xRevA's was obviously louder (~53-58db depending on PSU fan speed) than both of the above with the same load as the 7250. Having just seen more of the RevB stories, I'm watching any keyword related to RPS-15-E/RSP-16-E PSUs on eBay. If the 7250 had BGP, I probably don't really need the QSFP ports and would just stick with the 7250. Would be curious if anyone has done any sound measurements of the actual drop from RevA to RevB's...also it seems that the drop comes with dual B's...would running the 24 with a single B PSU be materially quieter than one or two RevA's? I don't feel like I'm too far from acceptable sound levels and my other option is to start running fiber to the garage/attic or just give up. :)
 
  • Like
Reactions: dashpuppy

juey

Member
Oct 1, 2018
56
16
8
Germany
Ok. Now have 6610-24 (2x RevA PSUs), 6450-24, and 7250-24P that were purchased in that order.

Since there is a lot of discussion on noise:
- The 6450-24 with stock fans was so quiet (38-42db) that I was able to move it to a bedroom closet and nobody knows it is there w/ 2 10gb and a dozen 1gb connections. Everyone has different sound tolerance, but I had multiple people help listen for it and we literally could not hear it -- I suspect that people with louder fans may have bad fans. It nicely replaced both a Mikrotik CRS305 and an older Unifi switch and barely consumes any power. It's incredible for the price and I would highly recommend it for most people that only need a few 10gig ports for a NAS or such.

- The 7250-24P is louder than the 6450 (45-48db). I would say mid-way between my 6450 and 6610. Definitely louder than the 6450, but is at a wife-acceptable volume level with no modifications (stock fans/etc...) from the next room with the door open with 6 10gb and a dozen 1gb connections. Wouldn't put in the bedroom closet or want to sit next to it for extended periods, but would not classify it as loud. Love that the latest OS has more straight-forward handling of ports with both tagged/untagged VLANs, etc... but wouldn't base a buying decision based on that. This replaced a Mikrotik CRS309 and two Unifi switches.

- The 6610-24 with 2xRevA's was obviously louder (~53-58db depending on PSU fan speed) than both of the above with the same load as the 7250. Having just seen more of the RevB stories, I'm watching any keyword related to RPS-15-E/RSP-16-E PSUs on eBay. If the 7250 had BGP, I probably don't really need the QSFP ports and would just stick with the 7250. Would be curious if anyone has done any sound measurements of the actual drop from RevA to RevB's...also it seems that the drop comes with dual B's...would running the 24 with a single B PSU be materially quieter than one or two RevA's? I don't feel like I'm too far from acceptable sound levels and my other option is to start running fiber to the garage/attic or just give up. :)
You could create a mixed stack with the 6610 and the 6450, that further enables more features on the 6450 like openflow and possibly BGP ?
 

dashpuppy

Member
Dec 16, 2018
48
21
8
Ok. Now have 6610-24 (2x RevA PSUs), 6450-24, and 7250-24P that were purchased in that order.

Since there is a lot of discussion on noise:
- The 6450-24 with stock fans was so quiet (38-42db) that I was able to move it to a bedroom closet and nobody knows it is there w/ 2 10gb and a dozen 1gb connections. Everyone has different sound tolerance, but I had multiple people help listen for it and we literally could not hear it -- I suspect that people with louder fans may have bad fans. It nicely replaced both a Mikrotik CRS305 and an older Unifi switch and barely consumes any power. It's incredible for the price and I would highly recommend it for most people that only need a few 10gig ports for a NAS or such.

- The 7250-24P is louder than the 6450 (45-48db). I would say mid-way between my 6450 and 6610. Definitely louder than the 6450, but is at a wife-acceptable volume level with no modifications (stock fans/etc...) from the next room with the door open with 6 10gb and a dozen 1gb connections. Wouldn't put in the bedroom closet or want to sit next to it for extended periods, but would not classify it as loud. Love that the latest OS has more straight-forward handling of ports with both tagged/untagged VLANs, etc... but wouldn't base a buying decision based on that. This replaced a Mikrotik CRS309 and two Unifi switches.

- The 6610-24 with 2xRevA's was obviously louder (~53-58db depending on PSU fan speed) than both of the above with the same load as the 7250. Having just seen more of the RevB stories, I'm watching any keyword related to RPS-15-E/RSP-16-E PSUs on eBay. If the 7250 had BGP, I probably don't really need the QSFP ports and would just stick with the 7250. Would be curious if anyone has done any sound measurements of the actual drop from RevA to RevB's...also it seems that the drop comes with dual B's...would running the 24 with a single B PSU be materially quieter than one or two RevA's? I don't feel like I'm too far from acceptable sound levels and my other option is to start running fiber to the garage/attic or just give up. :)
The 6450-24p is not "NOT" quiet ! Maybe the non poe version. but the "P" version isn't. That's why i just changed my switch fans :)
 

sth

Active Member
Oct 29, 2015
379
91
28
I'm struggling to get a ICX7400-4 x 10GC module to connect up, I ran the dd diag command and got this output (trimmed for relevant bits). Is this likely a configuration issue, or busted hardware?

Code:
==========================================================================
UNIT 1: SL 2: ICX7400-4X10GF 4-port 40G Module
==========================================================================
UNIT 1: SL 3: ICX7400-4X10GC 4-port 40G Module
==========================================================================
UNIT 1: SL 4: ICX7400-1X40GQ 1-port 40G Module

<snip>

********************************************************
RUNNING PHY DIAGNOSTICS:
********************************************************

<snip>

Port 1/2/1 CPU Packet Diagnostics Running...
Port 1/2/1 CPU Packet Diagnostics - PASSED

Port 1/2/2 CPU Packet Diagnostics Running...
Port 1/2/2 CPU Packet Diagnostics - PASSED

Port 1/2/3 CPU Packet Diagnostics Running...
Port 1/2/3 CPU Packet Diagnostics - PASSED

Port 1/2/4 CPU Packet Diagnostics Running...
Port 1/2/4 CPU Packet Diagnostics - PASSED

Port 1/3/1 CPU Packet Diagnostics Running...
Port 1/3/1 CPU Packet Diagnostics - PASSED

Port 1/3/2 CPU Packet Diagnostics Running...
Port 1/3/2 CPU Packet Diagnostics - PASSED

Port 1/3/3 CPU Packet Diagnostics Running...
Port 1/3/3 CPU Packet Diagnostics - PASSED

Port 1/3/4 CPU Packet Diagnostics Running...
Port 1/3/4 CPU Packet Diagnostics - PASSED

Port 1/4/1 CPU Packet Diagnostics Running...
Port 1/4/1 CPU Packet Diagnostics - PASSED

<snip>
********************************************************
RUNNING PACKET LINE RATE DIAGNOSTICS:
********************************************************

Packet Line rate test is in progress......
port    sent            received        diff
--------------------------------------------------------------

<snip>

1/2/1   437101091       437101091       0               ##PASS
1/2/2   432729626       432729626       0               ##PASS
1/2/3   416756927       416756927       0               ##PASS
1/2/4   412123190       412123190       0               ##PASS
1/3/1   412579691       412579691       0               ##PASS
1/3/2   412056371       412056371       0               ##PASS
1/3/3   408056253 [  474.291120] [BrcdSoftlockup]: Tracker queued..
        408056253       0               ##PASS
1/3/4   402957714       402957714       0               ##PASS
1/4/1   456645136       456645136       0               ##PASS
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,728
3,075
113
33
fohdeesha.com
The 6450-24p is not "NOT" quiet ! Maybe the non poe version. but the "P" version isn't. That's why i just changed my switch fans :)
my stock 6450-24p is dead silent nearly. The main issue with these switches (and why everyone has vastly different noise reports), is due to the age, some fans have started to wear their bearings and dry up and they get much louder
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,728
3,075
113
33
fohdeesha.com
EDIT: found the link to the published sources, though I haven't extracted it yet (on vacation/mobile). Reading the README for 6450 though indicates that there indeed binary blobs from Brocade/Arris to include. Humm... Makes sense not to include source to this libs as these most likely aren't under GPL. Will have to disassemble them and inspect further.

---

Has anyone successfully pulled down the source code and built the firmware? It was mentioned on the first page that under GPL licensing they are publishing, slowly.

For which device though? And would the process work for both the 6450 48p and 7250 48p ARM CPUs?

I'm on the fence for a 6450 or 7250 for their low noise and source code availability. I have four 10 Gbps 10GbaseT devices right now and technically could have a 5th, which would me swing towards the 7250 series.

However, building the firmware (with little to no proprietary binary blobs) is one thing that really interests me.

Maybe someone could point us to the source code links? I'll go back through the thread again and see if it was brought up...
The only source code you are going to get for any of these switches is the u-boot bootloader (which is what you linked, I requested u-boot sources for all of their models some time ago and they are *slowly* complying). The actual FastIron software, which is what talks to the ASIC, manages chassis fans, gives you the actual switch management interface etc, is completely proprietary and one giant monolithic binary blob
 

ronclark

New Member
Dec 6, 2019
11
6
3
The 6450-24p is not "NOT" quiet ! Maybe the non poe version. but the "P" version isn't. That's why i just changed my switch fans :)
I have a 6650-48P Rev A power supplies. I can't hear it over my R710s and my MD1200.
my 6450 came with two power supplies and two fan chartrages. there rear exhaust. I am just running one power supply and one fan chartrage.
 

vangoose

Active Member
May 21, 2019
326
104
43
Canada
I have a 6650-48P Rev A power supplies. I can't hear it over my R710s and my MD1200.
my 6450 came with two power supplies and two fan chartrages. there rear exhaust. I am just running one power supply and one fan chartrage.
6450-48P?

6650 is very very loud even with rev B psus.