Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

juey

Member
Oct 1, 2018
56
15
8
Germany
going through some hidden commands, after 30 seconds of testing these seem to set the case fan speed on the 6450, but don't do anything on the 6610. haven't tested on the icx7xxx but I have a feeling they'll behave like the 6450

dd set_pwrfan_high
dd set_pwrfan_low
Been quite a while since you posted this, but, could you add these commands to your beautiful docs ?
 

itronin

Well-Known Member
Nov 24, 2018
978
624
93
Denver, Colorado
Noise, ICX 6610 and what an ENORMOUS difference two rev b PSU's make compared to 1 rev b and 1 rev a...

I was able to pick up 2 Rev B POE PSU's < 50 each off the bay (seller had 2 total). I installed them last night and I cannot believe how much of a difference it makes!

I need to find one more reasonably priced rev b for my other 6610.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,350
1,798
113
CA
Noise, ICX 6610 and what an ENORMOUS difference two rev b PSU's make compared to 1 rev b and 1 rev a...

I was able to pick up 2 Rev B POE PSU's < 50 each off the bay (seller had 2 total). I installed them last night and I cannot believe how much of a difference it makes!

I need to find one more reasonably priced rev b for my other 6610.
What are the temps though?

I had heat issues and went to 2 fans, 2 psu, and heat issues went away. They're REv A.
In one I have a REv B but can't tell the difference... debating if I should swap that B to A for more air flow.
I have a 8-11*C difference between the 6610 on top vs on bottom too, unsure if it's location, A vs B, or due to the rack-ear not having the middle\vent hole.
 

itronin

Well-Known Member
Nov 24, 2018
978
624
93
Denver, Colorado
What are the temps though?

I had heat issues and went to 2 fans, 2 psu, and heat issues went away. They're REv A.
In one I have a REv B but can't tell the difference... debating if I should swap that B to A for more air flow.
I have a 8-11*C difference between the 6610 on top vs on bottom too, unsure if it's location, A vs B, or due to the rack-ear not having the middle\vent hole.
My original switch purchase had 2 PSU and 2 FANS in each unit. I've only been running 1 switch due to noise and playing with the 40Gbe. If I can source 1 more rev B I will try and figure out if it is really possible to stack with a single 40Gbe and use the remaining 2 for a LAG.

I've marked a seller on the bay that has qty 4 rev B power supplies (confirmed via message), asking a lot for 'em but none of have moved since it was posted. May take a lower number though it will likely be about what I paid for these 2 combined. But my ears, oh my. huge huge difference. my CSE-417 is now the loudest thing in the rack and I removed some fans from it to quiet it down a bit (all flash, 2/3 full, temps are fine there).

I figured since I did not hear the fans ramp up (except at power on) that all must be okay and I was really just floored by the sound difference so I didn’t even check temps.

Original configs:
#1, ICX6610-48P-E, 1xRev A PSU-E , 1x Rev B PSU-E, 2x FAN-E
#2, ICX6610-48P-E, 2xRev A PSU-E, 2xFAN-E

New config ICX
#1 ICX6610-48P-E, 2x Rev B PSU-E, 2x FAN-E,
#2, ICX6610-48P-E, 1xRev A PSU-E , 1x Rev B PSU-E, 2x FAN-E

Code:
icx6610-stack#show chassis
The stack unit 1 chassis info:

Power supply 1 (AC - PoE) present, status ok
     Model Number:   23-0000142-02
   Serial Number:   B1S     
   Firmware Ver:     B
Power supply 1 Fan Air Flow Direction:  Front to Back
Power supply 2 (AC - PoE) present, status ok
     Model Number:   23-0000142-02
   Serial Number:   5WP     
   Firmware Ver:     B
Power supply 2 Fan Air Flow Direction:  Front to Back

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 ok, speed (auto): [[1]]<->2

Fan controlled temperature: 65.5 deg-C

Fan speed switching temperature thresholds:
       Speed 1: NM<----->76       deg-C
       Speed 2:       71<-----> 80 deg-C (shutdown)

Fan 1 Air Flow Direction:  Front to Back
Fan 2 Air Flow Direction:  Front to Back                         
MAC 1 Temperature Readings:
   Current temperature : 45.0 deg-C
MAC 2 Temperature Readings:
   Current temperature : 54.0 deg-C
CPU Temperature Readings:
   Current temperature : 65.5 deg-C
sensor A Temperature Readings:
   Current temperature : 52.0 deg-C
sensor B Temperature Readings:
   Current temperature : 50.5 deg-C
sensor C Temperature Readings:
   Current temperature : 37.0 deg-C
stacking card Temperature Readings:
   Current temperature : 64.0 deg-C
   Warning level.......: 77.0 deg-C
   Shutdown level......: 80.0 deg-C
 
  • Like
Reactions: T_Minus

kapone

Well-Known Member
May 23, 2015
1,059
635
113
While that switch above is not exactly cooking...it is certainly very very warm. I'm surprised the fans haven't ramped up.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,350
1,798
113
CA
While that switch above is not exactly cooking...it is certainly very very warm. I'm surprised the fans haven't ramped up.

Which? These just posted?

MAC 1 Temperature Readings:
Current temperature : 45.0 deg-C
MAC 2 Temperature Readings:
Current temperature : 54.0 deg-C
CPU Temperature Readings:
Current temperature : 65.5 deg-C
sensor A Temperature Readings:
Current temperature : 52.0 deg-C
sensor B Temperature Readings:
Current temperature : 50.5 deg-C
sensor C Temperature Readings:
Current temperature : 37.0 deg-C


Mine that wasn't working was running at 79 and 80*C until we fixed the fan, and air flow situation, now both are in the 40s and 50s *C and work great :D
 
  • Like
Reactions: itronin

ekke

Member
Nov 16, 2015
122
6
18
44
Bought a 6450-24p poe .
Get these poe syslog messages.
upload_2019-12-21_23-56-59.png

I also got a Sunon MagLev KDE1204PKV3-MS.AF.GN 3pin 18dB-A Super Silence from ebay, gotta remove that high pitch noise of the current fan.
 

Attachments

fohdeesha

Kaini Industries
Nov 20, 2016
2,587
2,776
113
31
fohdeesha.com
My original switch purchase had 2 PSU and 2 FANS in each unit. I've only been running 1 switch due to noise and playing with the 40Gbe. If I can source 1 more rev B I will try and figure out if it is really possible to stack with a single 40Gbe and use the remaining 2 for a LAG.

I've marked a seller on the bay that has qty 4 rev B power supplies (confirmed via message), asking a lot for 'em but none of have moved since it was posted. May take a lower number though it will likely be about what I paid for these 2 combined. But my ears, oh my. huge huge difference. my CSE-417 is now the loudest thing in the rack and I removed some fans from it to quiet it down a bit (all flash, 2/3 full, temps are fine there).

I figured since I did not hear the fans ramp up (except at power on) that all must be okay and I was really just floored by the sound difference so I didn’t even check temps.

Original configs:
#1, ICX6610-48P-E, 1xRev A PSU-E , 1x Rev B PSU-E, 2x FAN-E
#2, ICX6610-48P-E, 2xRev A PSU-E, 2xFAN-E

New config ICX
#1 ICX6610-48P-E, 2x Rev B PSU-E, 2x FAN-E,
#2, ICX6610-48P-E, 1xRev A PSU-E , 1x Rev B PSU-E, 2x FAN-E

Code:
icx6610-stack#show chassis
The stack unit 1 chassis info:

Power supply 1 (AC - PoE) present, status ok
     Model Number:   23-0000142-02
   Serial Number:   B1S    
   Firmware Ver:     B
Power supply 1 Fan Air Flow Direction:  Front to Back
Power supply 2 (AC - PoE) present, status ok
     Model Number:   23-0000142-02
   Serial Number:   5WP    
   Firmware Ver:     B
Power supply 2 Fan Air Flow Direction:  Front to Back

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 ok, speed (auto): [[1]]<->2

Fan controlled temperature: 65.5 deg-C

Fan speed switching temperature thresholds:
       Speed 1: NM<----->76       deg-C
       Speed 2:       71<-----> 80 deg-C (shutdown)

Fan 1 Air Flow Direction:  Front to Back
Fan 2 Air Flow Direction:  Front to Back                        
MAC 1 Temperature Readings:
   Current temperature : 45.0 deg-C
MAC 2 Temperature Readings:
   Current temperature : 54.0 deg-C
CPU Temperature Readings:
   Current temperature : 65.5 deg-C
sensor A Temperature Readings:
   Current temperature : 52.0 deg-C
sensor B Temperature Readings:
   Current temperature : 50.5 deg-C
sensor C Temperature Readings:
   Current temperature : 37.0 deg-C
stacking card Temperature Readings:
   Current temperature : 64.0 deg-C
   Warning level.......: 77.0 deg-C
   Shutdown level......: 80.0 deg-C
it doesn't even bother to ramp up fans until 76c, 65 is more than fine. these things run hot (which is why I keep yelling at people to stop wasting their time trying to stick even slower fans in the 6610)
 
  • Like
Reactions: itronin

itronin

Well-Known Member
Nov 24, 2018
978
624
93
Denver, Colorado
it doesn't even bother to ramp up fans until 76c, 65 is more than fine. these things run hot (which is why I keep yelling at people to stop wasting their time trying to stick even slower fans in the 6610)
yes, folks please listen to Uncle Fohdeesha. Just find some deals and invest in rev B psu's . I am still non-sonically stunned by how quiet two 6610 POE rev B's are. Fohdeesha, you posted about the acoustics early on in the thread - just hadn't found a deal until last week and it was SO worth it!
 

dashpuppy

Member
Dec 16, 2018
47
17
8
Well I kinda had a decent weekend. First up was installing the new Fans into the 6450,, AHHH!! much silent ! And still passing air and keeping it cool.

Next was installing 10gig nic's into the Hp ML350 G8 server ( Fail ) they don't work :(

Ordered some PCI-e 10gig HP nic's instead of these Lenovo Emulex ones !!

Oh ! And i don't have an addiction !! So i bought 2 of these Brocade ICX 6430-C12's for 100$ One will be connected to my Brocade 6450 via Fiber then sitting behind my desk :) The other will be spare ? maybe...
 

Attachments

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
EDIT: found the link to the published sources, though I haven't extracted it yet (on vacation/mobile). Reading the README for 6450 though indicates that there indeed binary blobs from Brocade/Arris to include. Humm... Makes sense not to include source to this libs as these most likely aren't under GPL. Will have to disassemble them and inspect further.

---

Has anyone successfully pulled down the source code and built the firmware? It was mentioned on the first page that under GPL licensing they are publishing, slowly.

For which device though? And would the process work for both the 6450 48p and 7250 48p ARM CPUs?

I'm on the fence for a 6450 or 7250 for their low noise and source code availability. I have four 10 Gbps 10GbaseT devices right now and technically could have a 5th, which would me swing towards the 7250 series.

However, building the firmware (with little to no proprietary binary blobs) is one thing that really interests me.

Maybe someone could point us to the source code links? I'll go back through the thread again and see if it was brought up...
 
Last edited:

CED6688

New Member
Dec 4, 2019
15
10
3
Ok. Now have 6610-24 (2x RevA PSUs), 6450-24, and 7250-24P that were purchased in that order.

Since there is a lot of discussion on noise:
- The 6450-24 with stock fans was so quiet (38-42db) that I was able to move it to a bedroom closet and nobody knows it is there w/ 2 10gb and a dozen 1gb connections. Everyone has different sound tolerance, but I had multiple people help listen for it and we literally could not hear it -- I suspect that people with louder fans may have bad fans. It nicely replaced both a Mikrotik CRS305 and an older Unifi switch and barely consumes any power. It's incredible for the price and I would highly recommend it for most people that only need a few 10gig ports for a NAS or such.

- The 7250-24P is louder than the 6450 (45-48db). I would say mid-way between my 6450 and 6610. Definitely louder than the 6450, but is at a wife-acceptable volume level with no modifications (stock fans/etc...) from the next room with the door open with 6 10gb and a dozen 1gb connections. Wouldn't put in the bedroom closet or want to sit next to it for extended periods, but would not classify it as loud. Love that the latest OS has more straight-forward handling of ports with both tagged/untagged VLANs, etc... but wouldn't base a buying decision based on that. This replaced a Mikrotik CRS309 and two Unifi switches.

- The 6610-24 with 2xRevA's was obviously louder (~53-58db depending on PSU fan speed) than both of the above with the same load as the 7250. Having just seen more of the RevB stories, I'm watching any keyword related to RPS-15-E/RSP-16-E PSUs on eBay. If the 7250 had BGP, I probably don't really need the QSFP ports and would just stick with the 7250. Would be curious if anyone has done any sound measurements of the actual drop from RevA to RevB's...also it seems that the drop comes with dual B's...would running the 24 with a single B PSU be materially quieter than one or two RevA's? I don't feel like I'm too far from acceptable sound levels and my other option is to start running fiber to the garage/attic or just give up. :)
 
  • Like
Reactions: dashpuppy

juey

Member
Oct 1, 2018
56
15
8
Germany
Ok. Now have 6610-24 (2x RevA PSUs), 6450-24, and 7250-24P that were purchased in that order.

Since there is a lot of discussion on noise:
- The 6450-24 with stock fans was so quiet (38-42db) that I was able to move it to a bedroom closet and nobody knows it is there w/ 2 10gb and a dozen 1gb connections. Everyone has different sound tolerance, but I had multiple people help listen for it and we literally could not hear it -- I suspect that people with louder fans may have bad fans. It nicely replaced both a Mikrotik CRS305 and an older Unifi switch and barely consumes any power. It's incredible for the price and I would highly recommend it for most people that only need a few 10gig ports for a NAS or such.

- The 7250-24P is louder than the 6450 (45-48db). I would say mid-way between my 6450 and 6610. Definitely louder than the 6450, but is at a wife-acceptable volume level with no modifications (stock fans/etc...) from the next room with the door open with 6 10gb and a dozen 1gb connections. Wouldn't put in the bedroom closet or want to sit next to it for extended periods, but would not classify it as loud. Love that the latest OS has more straight-forward handling of ports with both tagged/untagged VLANs, etc... but wouldn't base a buying decision based on that. This replaced a Mikrotik CRS309 and two Unifi switches.

- The 6610-24 with 2xRevA's was obviously louder (~53-58db depending on PSU fan speed) than both of the above with the same load as the 7250. Having just seen more of the RevB stories, I'm watching any keyword related to RPS-15-E/RSP-16-E PSUs on eBay. If the 7250 had BGP, I probably don't really need the QSFP ports and would just stick with the 7250. Would be curious if anyone has done any sound measurements of the actual drop from RevA to RevB's...also it seems that the drop comes with dual B's...would running the 24 with a single B PSU be materially quieter than one or two RevA's? I don't feel like I'm too far from acceptable sound levels and my other option is to start running fiber to the garage/attic or just give up. :)
You could create a mixed stack with the 6610 and the 6450, that further enables more features on the 6450 like openflow and possibly BGP ?
 

dashpuppy

Member
Dec 16, 2018
47
17
8
Ok. Now have 6610-24 (2x RevA PSUs), 6450-24, and 7250-24P that were purchased in that order.

Since there is a lot of discussion on noise:
- The 6450-24 with stock fans was so quiet (38-42db) that I was able to move it to a bedroom closet and nobody knows it is there w/ 2 10gb and a dozen 1gb connections. Everyone has different sound tolerance, but I had multiple people help listen for it and we literally could not hear it -- I suspect that people with louder fans may have bad fans. It nicely replaced both a Mikrotik CRS305 and an older Unifi switch and barely consumes any power. It's incredible for the price and I would highly recommend it for most people that only need a few 10gig ports for a NAS or such.

- The 7250-24P is louder than the 6450 (45-48db). I would say mid-way between my 6450 and 6610. Definitely louder than the 6450, but is at a wife-acceptable volume level with no modifications (stock fans/etc...) from the next room with the door open with 6 10gb and a dozen 1gb connections. Wouldn't put in the bedroom closet or want to sit next to it for extended periods, but would not classify it as loud. Love that the latest OS has more straight-forward handling of ports with both tagged/untagged VLANs, etc... but wouldn't base a buying decision based on that. This replaced a Mikrotik CRS309 and two Unifi switches.

- The 6610-24 with 2xRevA's was obviously louder (~53-58db depending on PSU fan speed) than both of the above with the same load as the 7250. Having just seen more of the RevB stories, I'm watching any keyword related to RPS-15-E/RSP-16-E PSUs on eBay. If the 7250 had BGP, I probably don't really need the QSFP ports and would just stick with the 7250. Would be curious if anyone has done any sound measurements of the actual drop from RevA to RevB's...also it seems that the drop comes with dual B's...would running the 24 with a single B PSU be materially quieter than one or two RevA's? I don't feel like I'm too far from acceptable sound levels and my other option is to start running fiber to the garage/attic or just give up. :)
The 6450-24p is not "NOT" quiet ! Maybe the non poe version. but the "P" version isn't. That's why i just changed my switch fans :)
 

sth

Active Member
Oct 29, 2015
353
80
28
I'm struggling to get a ICX7400-4 x 10GC module to connect up, I ran the dd diag command and got this output (trimmed for relevant bits). Is this likely a configuration issue, or busted hardware?

Code:
==========================================================================
UNIT 1: SL 2: ICX7400-4X10GF 4-port 40G Module
==========================================================================
UNIT 1: SL 3: ICX7400-4X10GC 4-port 40G Module
==========================================================================
UNIT 1: SL 4: ICX7400-1X40GQ 1-port 40G Module

<snip>

********************************************************
RUNNING PHY DIAGNOSTICS:
********************************************************

<snip>

Port 1/2/1 CPU Packet Diagnostics Running...
Port 1/2/1 CPU Packet Diagnostics - PASSED

Port 1/2/2 CPU Packet Diagnostics Running...
Port 1/2/2 CPU Packet Diagnostics - PASSED

Port 1/2/3 CPU Packet Diagnostics Running...
Port 1/2/3 CPU Packet Diagnostics - PASSED

Port 1/2/4 CPU Packet Diagnostics Running...
Port 1/2/4 CPU Packet Diagnostics - PASSED

Port 1/3/1 CPU Packet Diagnostics Running...
Port 1/3/1 CPU Packet Diagnostics - PASSED

Port 1/3/2 CPU Packet Diagnostics Running...
Port 1/3/2 CPU Packet Diagnostics - PASSED

Port 1/3/3 CPU Packet Diagnostics Running...
Port 1/3/3 CPU Packet Diagnostics - PASSED

Port 1/3/4 CPU Packet Diagnostics Running...
Port 1/3/4 CPU Packet Diagnostics - PASSED

Port 1/4/1 CPU Packet Diagnostics Running...
Port 1/4/1 CPU Packet Diagnostics - PASSED

<snip>
********************************************************
RUNNING PACKET LINE RATE DIAGNOSTICS:
********************************************************

Packet Line rate test is in progress......
port    sent            received        diff
--------------------------------------------------------------

<snip>

1/2/1   437101091       437101091       0               ##PASS
1/2/2   432729626       432729626       0               ##PASS
1/2/3   416756927       416756927       0               ##PASS
1/2/4   412123190       412123190       0               ##PASS
1/3/1   412579691       412579691       0               ##PASS
1/3/2   412056371       412056371       0               ##PASS
1/3/3   408056253 [  474.291120] [BrcdSoftlockup]: Tracker queued..
        408056253       0               ##PASS
1/3/4   402957714       402957714       0               ##PASS
1/4/1   456645136       456645136       0               ##PASS