Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

fohdeesha

Kaini Industries
Nov 20, 2016
2,875
3,374
113
34
fohdeesha.com
Hello all,

Plugged in a "new to me" 6450-24p, and the thing is hella loud. Ordered some Comair Rotron "Gryphon" GDA4028-12BB fans and it's... still hella loud. Pulled the fans out and running them outside the case, it looks like the most annoying part of the noise is the vibrations and the PWM buzz.

I ran across this post (https://forums.servethehome.com/ind...be-40gbe-switching.21107/page-187#post-269342) from earlier in this thred, where a guy used a 330µF to quiet it down. I ran the same test he did briefly, and it does indeed eliminate the PWM buzz.

I've spent the entirety of this evening trying to understand out how low-pass filters work, but I must be incredibly dense, because I'm still just as puzzled as when I started. Based on what I've read, it sounds like you need a resistor and a capacitor to make it work - yet it seems to work just fine with a capacitor by itself.

My question: Does a capacitor, when used in the manner described in that post, have a chance of damaging anything? And if so, and a resistor really should be used, how would one determine what resistor value to use? (and anyone know the PWM frequency on the 6450? I've got a pretty nice (Brymen) multimeter, but the DC frequency function is reading 0hz or 60hz depending when I measure it, and that guy's video show 3khz).

Thank you
When you cold boot it, does it start out really loud, then the fans spin way down about 2 or 3 minutes into boot? It sounds like it may be stuck in full fan speed, after it finishes booting the fans should spin down to like ~20% of their full boot speed
 

NateR

Member
Feb 10, 2022
33
23
8
When you cold boot it, does it start out really loud, then the fans spin way down about 2 or 3 minutes into boot? It sounds like it may be stuck in full fan speed, after it finishes booting the fans should spin down to like ~20% of their full boot speed
No, it definitely spins down. Just the PWM buzz is really annoying, and there's a bit of vibration I'll need to dampen (when I remove the fans from the case and add the 330µF capacitor like that guy did in his first video, it gets very tolerably quiet). PWM buzz is the most annoying part, really. The 330µF quieted it down nicely, but am currently looking for electronic guidance on whether just hooking up the capacitor like that is safe or not - I have a very basic understanding about how different components (capacitors, resistors, diodes, etc.) work, but anything beyond that I have very little understanding of.
 

kpfleming

Active Member
Dec 28, 2021
433
222
43
Pelham NY USA
I've spent the entirety of this evening trying to understand out how low-pass filters work, but I must be incredibly dense, because I'm still just as puzzled as when I started. Based on what I've read, it sounds like you need a resistor and a capacitor to make it work - yet it seems to work just fine with a capacitor by itself.
The fan is itself an inductor, and can be part of the filter.
 

juju

Member
Sep 29, 2021
38
1
8
I'm kind of wondering if the Brocade needed more commands to configure correctly & you missed something & the cisco needed less & was easier to make work. I'm going to try this with a brocade (switching away from a 3945e) and I'll post how it goes. My brocades are a stack of 6610s however, not the 64xx series.
@RobstarUSA - Did you ever get around to trying to setup mDNS/Bonjour ? I am interested in the outcome of this as well. It's quite surprising there isn't a good guide here on how to set this up properly on brocade switches. (working reliably, I mean ). Just like @nickf1227 , I can't seem to get it to work. I am however, much less knowledgeable in the space than he his. I have an ICX7250 which is working fine except this issue of mDNS/Bonjour traffic on my network. I won't mind switching to another brand just to get it to work .

From the Cisco DNA service for Bonjour documentation :

Cisco Digital Network Architecture (DNA) Service for Bonjour is a software-defined controller-based solution that enables devices to advertise and discover Bonjour services across Layer 3 network boundaries
Is there an equivalent for Brocade/Rukus switches?
 
Last edited:

paxswill

New Member
Jul 12, 2019
2
0
1
@RobstarUSA - Did you ever get around to trying to setup mDNS/Bonjour ? I am interested in the outcome of this as well. It's quite surprising there isn't a good guide here on how to set this up properly on brocade switches. (working reliably, I mean ). Just like @nickf1227 , I can't seem to get it to work. I am however, much less knowledgeable in the space than he his. I have an ICX7250 which is working fine except this issue of mDNS/Bonjour traffic on my network. I won't mind switching to another brand just to get it to work .

From the Cisco DNA service for Bonjour documentation :



Is there an equivalent for Brocade/Rukus switches?
AFAIK there isn't a built-in service to the switch for this. Avahi (common OSS mDNS implementation) has a reflector mode that will repeat mDNS broadcasts across different networks. I have this configured on a host with access to all tagged VLANs I want broadcasts repeated across.

As a side note, I'd recommend running Avahi like this on a dedicated host (or at least one that you don't want to actually use mDNS on) so that you can disable caching to work around a bug with continually changing hostnames on Apple devices
 

koifish59

Member
Sep 30, 2020
80
23
8
Any confirmed cheap SFP+ optics that work fine mellanox cards?

I have these Brocade 57-0000075-01 SFP+ modules and although they work well with the brocade switch, my mellanox connectx-4 card seems to not like them.

edit: nm, I just grabbed a handful of these Mellanox MFM1T02A-SR for $8 a pop
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,875
3,374
113
34
fohdeesha.com
Oh, interesting the 6610 does support a mixed stack. The 7k series docs are explicit about no mixed stacking.
to be fair the 6610 + 6450 mixed stacking is a bit of a misnomer, they stack in the sense of becoming one "unit" that you manage together, but under the hood the icx6450 is just acting as an L2 port extender - anything l3 and above it has to hairpin over to the 6610 master and back for the 6610 to handle, and also of course that means it's not redundant (if you lose the 6610, the remaining 6450 is just going to be able to handle local l2 traffic only and nothing else)
 

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,691
1,123
113
@fohdeesha Do you have any idea regarding my stacking issue? Is it possible to stack ICX6610s with the 4x10G QSFP breakout ports only and use the 40G non-breakout QSFP ports for uplink?

Is yes, how? I just can't get it to work.
 
Mar 9, 2022
3
0
1
Long time lurker, first I want to say a massive thanks to @fohdeesha for taking the time to post that huge thread. It is a great resource, and was the exact reason I chose a brocade for my next switch.

I bought a 7250 yesterday, and I already have some FS and other 10g optics, which from what I have read should work. From my experience with other switches though, work doesn't always mean full features, and I am hoping to monitor RX/TX optical power and log it/have alerts using SNMP for a fiber experiment. Do you know if I need brocade optics for that part of the SFPSHOW command (DOM/DDM monitoring) to work? If so, do brocade coded 3rd party work?
Also, if anyone has a part number for a bidi 10g optic they know works, please post it ;)
 

LodeRunner

Active Member
Apr 27, 2019
553
235
43
FS optics coded as Brocade work. I think there’s also a command to enable third party optic monitoring for no -Brocade coded optics.
 

liv3010m

New Member
Jul 22, 2018
22
2
3
Buenos Aires, Argentina
Hi guys!
To those who own a 6610 or had access to one, do you know if low profile fans (those that are lesser than 15mm in depth) will fit in its 1U chassis without having to implement those franken :p mods where the coolers are running from outside of the 1U space? Maybe not on the 48P / PoE variants, but on the simpler ones?

Thanks!
 

RobstarUSA

Active Member
Sep 15, 2016
235
104
43
@RobstarUSA - Did you ever get around to trying to setup mDNS/Bonjour ? I am interested in the outcome of this as well. It's quite surprising there isn't a good guide here on how to set this up properly on brocade switches. (working reliably, I mean ). Just like @nickf1227 , I can't seem to get it to work. I am however, much less knowledgeable in the space than he his. I have an ICX7250 which is working fine except this issue of mDNS/Bonjour traffic on my network. I won't mind switching to another brand just to get it to work .

From the Cisco DNA service for Bonjour documentation :



Is there an equivalent for Brocade/Rukus switches?
I have some backend work to do before I do this & I don't know when I'm going to get the time.

My test is going to be something like Lan -> Printers for printer discovery, however I need to move the default gateway off of my OPNSense box (opnsense doesn't support pim) to the brocade (and make an SVI) & put in a transit interface between the brocade & upstream OPNSense box. For routing to work I have to enable a routing protocol for v4/v6 between opnsense and the brocade (probably will do OSPF). This whole shebang is being held-up while I wait for opnsense to fix bugs in their unbound aliases which are not importing properlty for me, and the he.net tunnels to auto start. I've got 2 tickets open on opnsense forums/github and I don't want to complicate my opnsense box prior to those getting fixed.

So it may be a while....figure a few months before I get to try this.
 
Last edited:

clcorbin

Member
Feb 15, 2014
76
10
8
I'm at a lose. The front 1/3/x SFP+ ports work perfectly with fiber transceivers (Brocade and others) at 10Gb. I've tried three different 10Gb copper transceivers, including the one above and they never power up. Always says network cable unplugged.

Could I have something configured wrong that would disable copper transceivers? Or is it the Intel X540 on the other end of the cable? If I plug the X540 into one of the standard front ports (1/1/x), everything works perfectly fine. Just at 1Gb/s...

Push comes to shove, I could probably run another fiber run, but I know the copper cables work perfectly fine at 10GB from my old Aruba S3500s.

Of course, if I DID run another fiber cable, then I could use one of my 40Gb cards instead of the X540...

Either way, any ideas are appreciated!
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,875
3,374
113
34
fohdeesha.com
I'm at a lose. The front 1/3/x SFP+ ports work perfectly with fiber transceivers (Brocade and others) at 10Gb. I've tried three different 10Gb copper transceivers, including the one above and they never power up. Always says network cable unplugged.

Could I have something configured wrong that would disable copper transceivers? Or is it the Intel X540 on the other end of the cable? If I plug the X540 into one of the standard front ports (1/1/x), everything works perfectly fine. Just at 1Gb/s...

Push comes to shove, I could probably run another fiber run, but I know the copper cables work perfectly fine at 10GB from my old Aruba S3500s.

Of course, if I DID run another fiber cable, then I could use one of my 40Gb cards instead of the X540...

Either way, any ideas are appreciated!

Edit: just saw your added edit that plugging the link into a regular 1gbe port works, are you sure you're still using the 10gbe module on the Intel side when doing that, or switching to a different sfp module? Those mikrotik copper modules would never run at 1gbe for me, at least not the rev 1s. Sounds like your Intel card is only running the module at 1gbe if that's the case, which sounds like some of their vendor locking ****ery
 
Last edited:

clcorbin

Member
Feb 15, 2014
76
10
8
The x540 doesn't use SFP+ modules. The transceivers are built in. I've used it at 10Gb with my old Aruba S3500 with no issues. And using the exact same module from the Aruba in the ICX 6610 1/3/x slots (tried multiple slots), it won't connect.

I realized I hadn't tested the cable from end to end yet, so I was hoping that would be a fail, but the cable tester came back with a pass and verified I am using the correct cable at both ends.

So do they make any QSFP transceivers for copper 10Gb? I have three of the dual port Mellanox cards gathering dust I could use if I could find a transceiver for my computers end.
 

clcorbin

Member
Feb 15, 2014
76
10
8
I've been having a problem with one port (work laptop 1Gb LAN port) since I switched to the ICX 6610 stack. One 6610 is in the hobby room in the computer stack, the other is out in the garage (one switch would have been MORE THAN ENOUGH ports, but about half my network drops led to the hobby room (stuff I've added) and half to the garage (original drops)) linked by dual 40GB fiber on x/2/1 and x/2/6 stacking ports.

One port from the garage switch on vlan 50 comes back to my hobby room (again, existing cabling in the right place) and goes to a basic, unmanaged Netgear GS108 switch. My work laptop and my personal laptop are connected to it.

The personal laptop (Windows 10 Pro on my AD domain) has worked flawlessly. But the work laptop is being a bit of a pain. It will disconnect the LAN from the network at (what appears to be) random times and then reconnect like nothing happened a few HOURS later. I've seen it happen two or three times in a day. There IS a physical connection during this time, but it gets the default ip address and has no connectivity to anything. But it DOES know there is a cable connected to SOMETHING out there.

Every single time it has happened, I've checked my personal laptop and it is connected just fine. I've checked (and then swapped) the cable with no change. Once, I saw a message on the network adapter show it had failed authentication. Which is a bit strange, as I don't have 802.1x setup on my network.

Using the same network (other than the switches), I never saw this problem in the 1 1/2 years I've had this laptop at my house on my network.

Also, the WiFi (from an Aruba IAP-305 connected to the server rack switch) has remained flawless as well. Same computer, but it IS brought into a different VLAN while connected to WiFi.

Any suggestions on what to look at would be greatly appreciated!

Clint