Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[hmw]

Ok, but @fohdeesha his photo looks like the Emerson's sticker. Delta stickers, at least mine here, looks much different.

Uploaded a short vid here >> VID_20230512_235938.mp4 with the fan noise. As all my other psu in the other switches sound the same, I guess the psu/fan is not broken.

Edit: sound is the same even with the switch fully booted.

Can you check the temps & the fan speed when corresponding the PSU is inside - are they different ?

 

[Nehalem501]

Hello,

I've recently acquired an ICX 6450-48 (non PoE version) and its (single) stock fan is very loud.
When reading this thread, I understand the fan should slow down after a while after the unit has finished booting, but mine seems stuck at full speed all the time without anything connected to the switch (I kept it on for over an hour once to check if the fan would slow down after some time).

I've already followed the guide to update the firmware to the latest version (08.0.30uT313).

The temperature displayed when running show chassis is always between 25° C and 40° C and for the fan it says Fan ok, speed (auto): [[1]]<->2

I don't know if it is supposed to work like that, but the fan blows air from inside the switch to outside the switch.

Is there something wrong with my unit, or should I just change the fan? (which one would be the quietest? I would prefer to keep some form of cooling rather than running it completely fanless, summers in France can get pretty hot and most homes don't have AC here)

Thanks for your help.

 

[NablaSquaredG]

Is there something wrong with my unit, or should I just change the fan?

Yes, Fan MOSFET fried. Seems to be a common issues with those switches.

 

[Nehalem501]

Yes, Fan MOSFET fried. Seems to be a common issues with those switches.

I'll try to replace it. Thanks.

 

[hmw]

Super interesting. I have a revision A power supply that is much more loud/annoying than my revision C Delta power supply. I do not have any revision C Emersons to test though.

I have a Rev B PSU that has newer firmware and is newer than the Rev C PSU. And I have Rev A PSUs that are the newer than B or C. No idea what Brocade does. OTOH, tested both B and C PSUs and it makes no difference to the noise

 

[hmw]

Appreciate the very thorough posting. I may have to pass on the 7650 since my daughter's room is near the utility room. It's a shame because it neatly meets all of the things I would want in a switch.

Decided to do some minimal sound proofing for the server rack. Started by adding some 1lb Mass Loaded Vinyl to the removable side and back panels



And followed it up with some cheap 1.5" acoustic foam



The front of the server rack is glass and has perforations for airflow - this wasn't covered completely, just enough so that air now flows through the lower part of the front door but the sound from the switch is blocked by a small MLV strip



All in all, took a few hours to cut and stick everything in place. The result is an appreciable reduction in noise - something like 5dB ~ 6dB. However - as noted previously, the switch has a really unpleasant acoustic profile and even when muted can be annoying. As you can see below - the noise reduction is great compared to my previous readings HERE



The problem is the harmonics at 1.6KHz - this is the unpleasant whine that I noted earlier. While muted it is still annoying. Luckily the rack is in the basement and I cannot hear it anywhere upstairs. If you have a utility closet, that might be enough to block the noise completely

 

[happydaisy]

I am trying to get PIM working for SSDP from VLAN50 to VLAN 1, without much success so far.
So I thought I would go back to basics and try to get SSDP working within VLAN50 with IGMP-snooping enabled, still not having much luck, if everything is patched through the ICX7250, multicast traffic just seems to get dropped.


The network consists of the main router (Mikrotik RB4011), the main switch (ICX7250-24p), a mini mikrotik serving out DHCP on VLAN50 only (RB750GL), it isn't doing any routing, just providing NTP and DHCP services to VLAN50 and finally 2 Ruckus R610 access points, one connected to the switch, one to the RB4011 router

With IGMP-Snooping disabled on all three devices (Mikrotik RB750GL, RB4011 and ICX7250-24p), everything works as expected. SSDP multicast queries flow through from the server to the client device on the wifi access point, no matter what the access point is connected to or how the devices are wired up.

With IGMP snooping enabled on the RB750GL and RB4011, SSDP queries can still be seen on the client laptop connected to the R610 via the main router, however, only if the ICX7250-24p is bypassed.

I am pretty sure I am missing something fundamental on the ICX7250.
So far I have tried the following commands:

enable
configure terminal
ip multicast

<test>

ip multicast active

<test>

ip multicast flood-unregistered

followed by

vlan 50
multicast

<test>

multicast active

<test>


Firmware on the ICX7250 is 09.0.10e
Mikrotik's both on latest stable 7.9 releases.

I will also say I have not dealt much with multicast/broadcast traffic before, still learning the ins and outs.

 

[akw2312]

Is it possible to force the fan speed on ICX7250?

I have changed the FAN on ICX7250, but the switch can not receive enough airflow on mode 1.

 

[itronin]

"Brocade E1MG-SX-OM 33210-100 Compare Brocade REAL & FAKE 1GbE SFP-SX"

This bay listing amused me!

 

[ManoftheSea]

A request for pointers to learn what I don't know: I see Foodesha has documented "Brocade does not restrict the use of optics or DACs by manufacturer, they'll take anything given it's the right protocol." But this implies that perhaps an Oracle-branded Intel X520-DA2 requires a properly-coded module. Where does one buy a Brocade-to-Intel (or Brocade-to-Oracle?, Oracle-to-Anything?) DAC?

 

[bitbckt]

FS.com

 

[ManoftheSea]

https://www.fs.com/products/69905.html

Okay, cool, I see the "dual-compatibility solutions" allows me to pick different ends. But I was also hoping for some advice as to whether the oracle-branded intel card takes an Intel module or something else, and how I might discover that (short of "plug it in, it doesn't work"). Or, if the Brocade isn't restricted, I just go with an Intel-to-Intel connection?
I guess I should pick up some fiber too, to share shipping costs. And a few other things that might come up as I test things out. Ah, discovery!

 

[kpfleming]

10Gtek also offers custom-coded DACs and AOCs.

 

[gb00s]

Is it possible to force the fan speed on ICX7250?

I have changed the FAN on ICX7250, but the switch can not receive enough airflow on mode 1.

Did you add a 60/80mm super slim (10mm) fan on the heatsink of the asic already?

 

[hmw]

Trying to setup OPNsense with the ICX7650 and VLANs, will have a separate Windows DNS/DHCP server for DHCP + make the OPNsense into a gateway with static routes back to the 7650.

I was reading about possible configurations and one is to use a transit VLAN between OPNsense and the L3 switch and another uses a routed port with an IP address set on that port ( so "route-only" )

Is there a reason to choose one method over the other?

 

[tegtke]

Does anyone have an updated link for the ICX 6650 mini usb console cable? I've programmed and deployed 4 ICX 6610s but can't seem to get traction with this 6650. Thanks in advance.

 

[gizzard]

I am getting "Failed to add ICX [192.168.x.xxx]. The ICX already exists. Please enter a different one." in my Unleashed (200.14.6.1.179) logs.

I recently upgraded my ICX7150-c12p to 09.0.10e from 08.0.95f and I may have inadvertently removed the switch in Unleashed then re-added it using the same IP. Anyone know how can I force Unleashed to stop trying to add the switch?

EDIT: problem solved by just restarting Unleashed...

 

[akw2312]

Did you add a 60/80mm super slim (10mm) fan on the heatsink of the asic already?

Yes, Sunon 8010 fan and Sunon 4020 fan

 

[u4096]

Trying to setup OPNsense with the ICX7650 and VLANs, will have a separate Windows DNS/DHCP server for DHCP + make the OPNsense into a gateway with static routes back to the 7650.

I was reading about possible configurations and one is to use a transit VLAN between OPNsense and the L3 switch and another uses a routed port with an IP address set on that port ( so "route-only" )

Is there a reason to choose one method over the other?

1. What is the transit VLAN method? Can you link to an example or elaborate? Sounds more complicated than the second method.

2. I have configured the "route-only" method in the past and it works very well. Especially because you now have a GUI for managing DHCP/DNS and can more easily create static DHCP mappings.

Like you described it's pretty simple. You just to have to setup the routes on OPNSense for all the VLANs you want to transit/egress to internet and configured a helper address for each corresponding VLAN in the 7650 config. Something like:

interface ve 3
 port-name LAN
 ip address 192.168.3.3 255.255.255.0
 ip helper-address 1 192.168.1.1

3. There is also a third method- you configure DHCP scopes on the ICX itself. However, I found that problematic. Too many lines to type lol ... and every time you make changes to each DHCP pool, you have to remember to type "deploy" otherwise changes won't get enabled. I forgot to do that so often that it became a nuissance and made me prefer option 2 .

 

[LodeRunner]

1. What is the transit VLAN method? Can you link to an example or elaborate? Sounds more complicated than the second method.

Transit VLAN is where you let the core switch handle all inter-VLAN routing and it has a default gateway on a separate VLAN that's a point to point to the firewall. The firewall has static routes for each subnet back via the switch's IP on the transit VLAN.

Here's a very barebones config I did for another post to demonstrate:

Spoiler

ICX7150-C12 Router#sh run
Current configuration:
!
ver 08.0.95eT213
!
stack unit 1
  module 1 icx7150-c12-poe-port-management-module
  module 2 icx7150-2-copper-port-2g-module
  module 3 icx7150-2-sfp-plus-port-20g-module
  stack-port 1/3/1
  stack-port 1/3/2
!
global-stp
vlan 1 name DEFAULT-VLAN by port
spanning-tree
!
vlan 11 by port
tagged ethe 1/1/1
untagged ethe 1/1/11
router-interface ve 11
!                                                            
vlan 12 by port
tagged ethe 1/1/1
untagged ethe 1/1/3
router-interface ve 12
!
vlan 20 by port
tagged ethe 1/1/1
router-interface ve 20
!
ip dhcp-client disable
ip route 0.0.0.0/0 172.16.21.2
!

!                                                            
interface ve 11
ip address 10.100.11.1 255.255.255.0
!
interface ve 12
ip address 10.100.12.1 255.255.255.0
ip helper-address 1 10.100.11.2
!
interface ve 20
ip address 172.16.21.1 255.255.255.0
end

Switch IP and route tables:

Spoiler

ICX7150-C12 Router#sh ip add
        IP Address       Type      Lease Time       Interface
       10.100.11.1       Static    N/A             ve11
       10.100.12.1       Static    N/A             ve12
       172.16.21.1       Static    N/A             ve20
ICX7150-C12 Router#sh ip rout
Total number of IP routes: 4
Type Codes - B:BGP D:Connected O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2
STATIC Codes - v:Inter-VRF
        Destination        Gateway         Port          Cost          Type Uptime
1       0.0.0.0/0          172.16.21.2     ve 20         1/1           S    18m57s
2       10.100.11.0/24     DIRECT          ve 11         0/0           D    26m26s
3       10.100.12.0/24     DIRECT          ve 12         0/0           D    28m5s
4       172.16.21.0/24     DIRECT          ve 20         0/0           D    19m28s
ICX7150-C12 Router#

pfSense gateway and routes:
View attachment 21045
View attachment 21046

Windows VM in VLAN 11, IP 10.100.11.2/24, running DHCP server, showing client with IP 10.100.12.2/24:
View attachment 21047
Works well if your inter-VLAN security needs can be handled with ACLs on the switch (or if you just want physical separation for your subnets.