Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

fohdeesha

Kaini Industries
Nov 20, 2016
2,741
3,108
113
33
fohdeesha.com
i have a ICX-6610 and for the life of me I can't get traffic on any other vlan out to the internet. for instance I have a vlan 5 which has the the router interface 10.5.0.1. any client on vlan 5 can ping the router interface and the management IP of the switch which is on the default vlan. but if I try and get a client to ping the upstream router at 192.168.1.1, that doesn't seem to work.

if I try the command "ping 192.168.1.1 source 10.5.0.1" on the switch I get no response. what am I missing here? I'm running router code associated with the downloads on page 1 of this thread. if can post my config if that would help
your 192.168.1.1 router needs a route telling it how to get back to 10.5.0.x (and any other vlan networks you have), otherwise its going to do a route lookup, match on the default route you have pointing to your ISP, and shove it out there
 
  • Like
Reactions: NachoCDN

TeleFragger

Active Member
Oct 26, 2016
263
55
28
51
I will be opening up a new can of fun worms....
I have posted about having an old style connectx-1 and 2 cards that are the old cx4 style plug... they work.. 10gb, fast transfers, etc..
wife wants utility room cleaned up so time to do so..
so out with the old fastiron 648s and I will be putting in an ICX7250-48P-2X10G...
I will be able to get rid of my 8 port poe switch that controls my reolink cameras, will be able to have 8 10gb sfp+ ports available to me, get rid of my hp procurve 6400cl and all the cables that go with it...
 
  • Like
Reactions: NachoCDN

NachoCDN

Active Member
Apr 18, 2016
111
91
28
53
your 192.168.1.1 router needs a route telling it how to get back to 10.5.0.x (and any other vlan networks you have), otherwise its going to do a route lookup, match on the default route you have pointing to your ISP, and shove it out there
the 192.168.1.1 has a route available as i have BGP routing setup between the pfsense box and the switch. as shown below:1709559105165.png

that's what is so perplexing about why this doesn't work. that being said, i do have a double nat situation going on as the pfsense router is behind another router.

i'd also like to add that the route back to say a client on the 10.5.0.0 seems to work as i'm able to ping 10.5.0.2 from the pfsense router at 192.168.1.1, so pinging in to the switch and the vlan 5 seems to work no problem.
 
Last edited:
  • Wow
Reactions: TeleFragger

TeleFragger

Active Member
Oct 26, 2016
263
55
28
51
I will be opening up a new can of fun worms....
I have posted about having an old style connectx-1 and 2 cards that are the old cx4 style plug... they work.. 10gb, fast transfers, etc..
wife wants utility room cleaned up so time to do so..
so out with the old fastiron 648s and I will be putting in an ICX7250-48P-2X10G...
I will be able to get rid of my 8 port poe switch that controls my reolink cameras, will be able to have 8 10gb sfp+ ports available to me, get rid of my hp procurve 6400cl and all the cables that go with it...
so i just realized on page 1 - what made me compare the two switches and decide on the 7250, i read:

  • 8x 10gbE SFP+
but google, everywhere I see says 6x 1gb and 2x 10gb....
am i missing something?

***********Edited
Found it.. now i understand..
Fixed ports: 1/10 Gbps SFP+ (10 GbE SPF+ optional upgrade license)
 
Last edited:
  • Like
Reactions: NachoCDN

TeleFragger

Active Member
Oct 26, 2016
263
55
28
51
so i just realized on page 1 - what made me compare the two switches and decide on the 7250, i read:

  • 8x 10gbE SFP+
but google, everywhere I see says 6x 1gb and 2x 10gb....
am i missing something?

***********Edited
Found it.. now i understand..
Fixed ports: 1/10 Gbps SFP+ (10 GbE SPF+ optional upgrade license)
here is what i will be doing...
giving others ideas/pricing ideas...


I just decided and went
$160 - Brocade 7250 POE
$100 - 5x intel x520-da2 nic each with 2x fiber modules so can move 1 into switch!!!!
$35 - 4x LC to LC fiber cables - need to order 1 more though
$15 - brocade to db9 cable - offer on ebay so waiting
__________________
$310 Total
 
  • Like
Reactions: itronin

Koop

Active Member
Jan 24, 2024
165
74
28
So I picked up an ICX7250 and went through all the initial Setup / Config / Licensing Guide - Thank you @fohdeesha for the excellent documentation to get things started, made things very easy. I did cisco courses about 17 years ago and man did it bring back memories. Plan to play with this switch and see what I can learn.

I wanted to ask first, what is best way to keep up on updates? I just happen to notice the webui posted by @Sealside and wow that actually looks nice. What is the best way to update while keeping my configuration? Sorry for such a noob question.

Next up- nearly silent - oh boy that is not my experience so far. This switch is easily the loudest component in my rack by 10 miles. Do you think there is an issue with the thermals in my unit that I can address? Or is it a matter of swapping the stock fans out? I did get the 48 port PoE version- it was a choice based on price (cheaper than any other option I could find). I found from a lot of googling @RoachedCoach 's post proclaiming success with the Sunon MF60101V3-1000U-A99 and Mechatronics MR4020X12B1-RSR fans. Where should I buy fans likes in in NA? Would others still recommend this path? Definitely want to do all I can to cool and quiet down my switch.

Happy I didn't go for the ICX6610 even though I did want the 40GbE. I opted for 7250 since I wanted things to stay quiet and figured let me start out at 10GbE since that's what I have on my TrueNAS box (only issue there is it's onboard rj45 but that's a problem to figure out later). If the 6610 is even louder than this then dayum.

Appreciate the thread and any suggestions anyone wishes to provide.
 
Last edited:

Gerhen

Member
Nov 29, 2023
43
4
8


I wanted to ask first, what is best way to keep up on updates?

Next up- nearly silent - oh boy that is not my experience so far. This switch is easily the loudest component in my rack by 10 miles.
Your config shouldn’t be impacted when you upgrade.

I normally run the copy tftp system-manifest command and direct it to copy into the primary flash memory to start. If the new firmware is runnning stable, you can then copy primary to secondary. If it isn’t stable…you can always fall back and reload from secondary.

Firmware can be obtained from the ruckus support site you just need to register an account/email. It’s a good idea to register and read the documentation before you update esp. if you intend to move from the 08 to 09 firmware (some ppl run that branch).

Did you let the switch run to steady state operation? Initially at startup, the fans are loud (like wear hearing protection loud), after a few minutes the fans reach steady state and become less loud though I wouldn’t it in the same room if I was WFH.
 

TeleFragger

Active Member
Oct 26, 2016
263
55
28
51
So I picked up an ICX7250 and went through all the initial Setup / Config / Licensing Guide - Thank you @fohdeesha for the excellent documentation to get things started, made things very easy.

Happy I didn't go for the ICX6610 even though I did want the 40GbE. I opted for 7250 since I wanted things to stay quiet and figured let me start out at 10GbE since that's what I have on my TrueNAS box (only issue there is it's onboard rj45 but that's a problem to figure out later). If the 6610 is even louder than this then dayum.

Appreciate the thread and any suggestions anyone wishes to provide.
I cant wait for my 7250 to show up. Did you go POE version?

for the rj45... 10gb... I am new too but from what my searching got me, you just need these to put into the sfp+ port


I too wanted the 40gb ports but have no 40gb cards... i might have to keep my old brocade FastIron 648S to "toy with" but yes once I saw the LOUDNESS of the other, I chose the 7250... can always upgrade again.. lol in the future.
 

NachoCDN

Active Member
Apr 18, 2016
111
91
28
53
for the rj45... 10gb... I am new too but from what my searching got me, you just need these to put into the sfp+ port
i got the same one for my 6610 and it worked out of the box no issue.
 
  • Love
Reactions: TeleFragger

Koop

Active Member
Jan 24, 2024
165
74
28
for the rj45... 10gb... I am new too but from what my searching got me, you just need these to put into the sfp+ port
I did go with the PoE version- because every other version was more expensive as far as I could find. Even those with less ports.

Word of caution from all I've ready these type of SFPs run super hot. Since my switch is already apparently too hot this may not be the best idea. Should work no problem but just something to keep in mind. No experience first hand with this- just that it seems to be pretty normal for these.

Firmware can be obtained from the ruckus support site you just need to register an account/email. It’s a good idea to register and read the documentation before you update esp. if you intend to move from the 08 to 09 firmware (some ppl run that branch).

Did you let the switch run to steady state operation? Initially at startup, the fans are loud (like wear hearing protection loud), after a few minutes the fans reach steady state and become less loud though I wouldn’t it in the same room if I was WFH.
Roger that on obtaining firmware and checking documentation first.

And yeah it was at steady operation- oh trust me there's a HUGE difference between start up and normal operation hah but even at the normal operation the sound level was pretty high. I have my rack nearby in my office and it's a bit much. Hoping there's more I can do to quiet it down.
 
  • Like
Reactions: TeleFragger

fohdeesha

Kaini Industries
Nov 20, 2016
2,741
3,108
113
33
fohdeesha.com
So I picked up an ICX7250 and went through all the initial Setup / Config / Licensing Guide - Thank you @fohdeesha for the excellent documentation to get things started, made things very easy. I did cisco courses about 17 years ago and man did it bring back memories. Plan to play with this switch and see what I can learn.

I wanted to ask first, what is best way to keep up on updates? I just happen to notice the webui posted by @Sealside and wow that actually looks nice. What is the best way to update while keeping my configuration? Sorry for such a noob question.

Next up- nearly silent - oh boy that is not my experience so far. This switch is easily the loudest component in my rack by 10 miles. Do you think there is an issue with the thermals in my unit that I can address? Or is it a matter of swapping the stock fans out? I did get the 48 port PoE version- it was a choice based on price (cheaper than any other option I could find). I found from a lot of googling @RoachedCoach 's post proclaiming success with the Sunon MF60101V3-1000U-A99 and Mechatronics MR4020X12B1-RSR fans. Where should I buy fans likes in in NA? Would others still recommend this path? Definitely want to do all I can to cool and quiet down my switch.

Happy I didn't go for the ICX6610 even though I did want the 40GbE. I opted for 7250 since I wanted things to stay quiet and figured let me start out at 10GbE since that's what I have on my TrueNAS box (only issue there is it's onboard rj45 but that's a problem to figure out later). If the 6610 is even louder than this then dayum.

Appreciate the thread and any suggestions anyone wishes to provide.
do the fans spin down and get significantly quieter after it finishes booting? Sounds like they may be stuck in full speed, it should not be that loud once it spins down
 

Koop

Active Member
Jan 24, 2024
165
74
28
do the fans spin down and get significantly quieter after it finishes booting? Sounds like they may be stuck in full speed, it should not be that loud once it spins down
Yeah absolutely, but they're still pretty loud even after they spun down from initial boot. Not what I would describe as quiet. I assumed it's the temp of the switch and the fans were still automatically turned up higher perhaps to compensate? Any way I can show what I am seeing?
 

Andydude

New Member
Oct 6, 2023
11
7
3
Yeah absolutely, but they're still pretty loud even after they spun down from initial boot. Not what I would describe as quiet. I assumed it's the temp of the switch and the fans were still automatically turned up higher perhaps to compensate? Any way I can show what I am seeing?
I used the RoachCoach mod and it worked for me: https://forums.servethehome.com/ind...be-40gbe-switching.21107/page-159#post-256375

@deff_janiels also followed the mod and took a good pic here: https://forums.servethehome.com/ind...erful-10gbe-40gbe-switching.21107/post-331607

Bought three Mechatronics MR4020X12B1-RSR from Digikey to replace the three rear fans and a Sunon MF60101V3-1000U-A99 to go directly on the IC heatsink and it's nice and quiet and sits in a rack at around 60 degrees.

Bash:
SSH@brocade-7250-core-switch#show chassis
The stack unit 1 chassis info:

Power supply 1 (AC - PoE) present, status ok
Power supply 2 not present
Power supply 3 not present

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 ok, speed (auto): [[1]]<->2
Fan 3 ok, speed (auto): [[1]]<->2

Fan controlled temperature:
        Rule 1/2 (MGMT THERMAL PLANE): 60.0 deg-C
        Rule 2/2 (AIR OUTLET NEAR PSU): 31.0 deg-C

Fan speed switching temperature thresholds:
        Rule 1/2 (MGMT THERMAL PLANE):
                Speed 1: NM<-----> 95       deg-C
                Speed 2:        85<----->105 deg-C (shutdown)
        Rule 2/2 (AIR OUTLET NEAR PSU):
                Speed 1: NM<-----> 41       deg-C
                Speed 2:        34<----->105 deg-C (shutdown)

Fan 1 Air Flow Direction:  Front to Back
Fan 2 Air Flow Direction:  Front to Back
Fan 3 Air Flow Direction:  Front to Back
Slot 1 Current Temperature: 60.0 deg-C (Sensor 1), 31.0 deg-C (Sensor 2)
        Warning level.......: 100.0 deg-C
        Shutdown level......: 105.0 deg-C
Boot Prom MAC : 609c.9f70.97a4
Management MAC: 609c.9f70.97a4
 

frogtech

Well-Known Member
Jan 4, 2016
1,482
272
83
35
So I picked up an ICX7250 and went through all the initial Setup / Config / Licensing Guide - Thank you @fohdeesha for the excellent documentation to get things started, made things very easy. I did cisco courses about 17 years ago and man did it bring back memories. Plan to play with this switch and see what I can learn.

I wanted to ask first, what is best way to keep up on updates? I just happen to notice the webui posted by @Sealside and wow that actually looks nice. What is the best way to update while keeping my configuration? Sorry for such a noob question.

Next up- nearly silent - oh boy that is not my experience so far. This switch is easily the loudest component in my rack by 10 miles. Do you think there is an issue with the thermals in my unit that I can address? Or is it a matter of swapping the stock fans out? I did get the 48 port PoE version- it was a choice based on price (cheaper than any other option I could find). I found from a lot of googling @RoachedCoach 's post proclaiming success with the Sunon MF60101V3-1000U-A99 and Mechatronics MR4020X12B1-RSR fans. Where should I buy fans likes in in NA? Would others still recommend this path? Definitely want to do all I can to cool and quiet down my switch.

Happy I didn't go for the ICX6610 even though I did want the 40GbE. I opted for 7250 since I wanted things to stay quiet and figured let me start out at 10GbE since that's what I have on my TrueNAS box (only issue there is it's onboard rj45 but that's a problem to figure out later). If the 6610 is even louder than this then dayum.

Appreciate the thread and any suggestions anyone wishes to provide.
I have not had much success in silencing ICX7250-24P switches, which are the 24 port PoE models that have 2 fans. I did some fan swaps, but I can't remember to which models, and it didn't really help much with noise and the temps are still kind of bad. One of my switches is 66c right now and the other is 59c. I added some shiddy lil 60mm slim fans on top of the main ASIC, but I don't think it contributes much besides being a nice cope.

I actually came to this thread seeing if anyone has finally yet settled on the "penultimate" fan for these switches that is a good compromise of both temps and noise.

I don't understand why it's so hard to find a good quality layer 3 switch that doesn't run at a million degrees with 747 fans. Those arista switches in the great deals forum seem tempting, but the unquestioningly better availability of documentation and software support for the ruckus products is just much nicer from QoL stand point, as opposed to having to "grift" around for firmware updates from other users or people who have support contracts

edit: i think i am actually going to try those arctic 15k fans and see how well they do. i'll report back. someone else in this thread seemed to indicate they were fine.
 
Last edited:

Sealside

Active Member
May 10, 2019
126
45
28
Stockholm/Sweden
I have not had much success in silencing ...

I don't understand why it's so hard to find a good quality layer 3 switch that doesn't run at a million degrees with 747 fans.
I have a juniper ex3300 24P running for several years with noctuas. It is dead silent appart from the regular white noise. You just pop them in and run.

What you can do on the 7250 is to run noctuas with an additional arduino/esp etc which will basically make the noctua to run full speed always. This is somewhat of a hack requiring additional hardware and software. (Same mod as for the 6610 more or less)
 

Koop

Active Member
Jan 24, 2024
165
74
28
You know, looking at the fans on Digikey I see you can order all manner of passive heatsinks and such too. Is there any value to potentially using a different heatsink on the broadcom chip? copper? Maybe a larger one if I can get the pushpins in the right place. Maybe it doesn't matter... hmm.. I wonder.
 
Last edited: