Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

liv3010m

New Member
Jul 22, 2018
22
2
3
Buenos Aires, Argentina
Hi guys!
To those who own a 6610 or had access to one, do you know if low profile fans (those that are lesser than 15mm in depth) will fit in its 1U chassis without having to implement those franken :p mods where the coolers are running from outside of the 1U space? Maybe not on the 48P / PoE variants, but on the simpler ones?

Thanks!
 

RobstarUSA

Active Member
Sep 15, 2016
235
104
43
@RobstarUSA - Did you ever get around to trying to setup mDNS/Bonjour ? I am interested in the outcome of this as well. It's quite surprising there isn't a good guide here on how to set this up properly on brocade switches. (working reliably, I mean ). Just like @nickf1227 , I can't seem to get it to work. I am however, much less knowledgeable in the space than he his. I have an ICX7250 which is working fine except this issue of mDNS/Bonjour traffic on my network. I won't mind switching to another brand just to get it to work .

From the Cisco DNA service for Bonjour documentation :



Is there an equivalent for Brocade/Rukus switches?
I have some backend work to do before I do this & I don't know when I'm going to get the time.

My test is going to be something like Lan -> Printers for printer discovery, however I need to move the default gateway off of my OPNSense box (opnsense doesn't support pim) to the brocade (and make an SVI) & put in a transit interface between the brocade & upstream OPNSense box. For routing to work I have to enable a routing protocol for v4/v6 between opnsense and the brocade (probably will do OSPF). This whole shebang is being held-up while I wait for opnsense to fix bugs in their unbound aliases which are not importing properlty for me, and the he.net tunnels to auto start. I've got 2 tickets open on opnsense forums/github and I don't want to complicate my opnsense box prior to those getting fixed.

So it may be a while....figure a few months before I get to try this.
 
Last edited:

clcorbin

Member
Feb 15, 2014
74
10
8
I'm at a lose. The front 1/3/x SFP+ ports work perfectly with fiber transceivers (Brocade and others) at 10Gb. I've tried three different 10Gb copper transceivers, including the one above and they never power up. Always says network cable unplugged.

Could I have something configured wrong that would disable copper transceivers? Or is it the Intel X540 on the other end of the cable? If I plug the X540 into one of the standard front ports (1/1/x), everything works perfectly fine. Just at 1Gb/s...

Push comes to shove, I could probably run another fiber run, but I know the copper cables work perfectly fine at 10GB from my old Aruba S3500s.

Of course, if I DID run another fiber cable, then I could use one of my 40Gb cards instead of the X540...

Either way, any ideas are appreciated!
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,811
3,231
113
33
fohdeesha.com
I'm at a lose. The front 1/3/x SFP+ ports work perfectly with fiber transceivers (Brocade and others) at 10Gb. I've tried three different 10Gb copper transceivers, including the one above and they never power up. Always says network cable unplugged.

Could I have something configured wrong that would disable copper transceivers? Or is it the Intel X540 on the other end of the cable? If I plug the X540 into one of the standard front ports (1/1/x), everything works perfectly fine. Just at 1Gb/s...

Push comes to shove, I could probably run another fiber run, but I know the copper cables work perfectly fine at 10GB from my old Aruba S3500s.

Of course, if I DID run another fiber cable, then I could use one of my 40Gb cards instead of the X540...

Either way, any ideas are appreciated!

Edit: just saw your added edit that plugging the link into a regular 1gbe port works, are you sure you're still using the 10gbe module on the Intel side when doing that, or switching to a different sfp module? Those mikrotik copper modules would never run at 1gbe for me, at least not the rev 1s. Sounds like your Intel card is only running the module at 1gbe if that's the case, which sounds like some of their vendor locking ****ery
 
Last edited:

clcorbin

Member
Feb 15, 2014
74
10
8
The x540 doesn't use SFP+ modules. The transceivers are built in. I've used it at 10Gb with my old Aruba S3500 with no issues. And using the exact same module from the Aruba in the ICX 6610 1/3/x slots (tried multiple slots), it won't connect.

I realized I hadn't tested the cable from end to end yet, so I was hoping that would be a fail, but the cable tester came back with a pass and verified I am using the correct cable at both ends.

So do they make any QSFP transceivers for copper 10Gb? I have three of the dual port Mellanox cards gathering dust I could use if I could find a transceiver for my computers end.
 

clcorbin

Member
Feb 15, 2014
74
10
8
I've been having a problem with one port (work laptop 1Gb LAN port) since I switched to the ICX 6610 stack. One 6610 is in the hobby room in the computer stack, the other is out in the garage (one switch would have been MORE THAN ENOUGH ports, but about half my network drops led to the hobby room (stuff I've added) and half to the garage (original drops)) linked by dual 40GB fiber on x/2/1 and x/2/6 stacking ports.

One port from the garage switch on vlan 50 comes back to my hobby room (again, existing cabling in the right place) and goes to a basic, unmanaged Netgear GS108 switch. My work laptop and my personal laptop are connected to it.

The personal laptop (Windows 10 Pro on my AD domain) has worked flawlessly. But the work laptop is being a bit of a pain. It will disconnect the LAN from the network at (what appears to be) random times and then reconnect like nothing happened a few HOURS later. I've seen it happen two or three times in a day. There IS a physical connection during this time, but it gets the default ip address and has no connectivity to anything. But it DOES know there is a cable connected to SOMETHING out there.

Every single time it has happened, I've checked my personal laptop and it is connected just fine. I've checked (and then swapped) the cable with no change. Once, I saw a message on the network adapter show it had failed authentication. Which is a bit strange, as I don't have 802.1x setup on my network.

Using the same network (other than the switches), I never saw this problem in the 1 1/2 years I've had this laptop at my house on my network.

Also, the WiFi (from an Aruba IAP-305 connected to the server rack switch) has remained flawless as well. Same computer, but it IS brought into a different VLAN while connected to WiFi.

Any suggestions on what to look at would be greatly appreciated!

Clint
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,811
3,231
113
33
fohdeesha.com
The x540 doesn't use SFP+ modules. The transceivers are built in. I've used it at 10Gb with my old Aruba S3500 with no issues. And using the exact same module from the Aruba in the ICX 6610 1/3/x slots (tried multiple slots), it won't connect.

I realized I hadn't tested the cable from end to end yet, so I was hoping that would be a fail, but the cable tester came back with a pass and verified I am using the correct cable at both ends.

So do they make any QSFP transceivers for copper 10Gb? I have three of the dual port Mellanox cards gathering dust I could use if I could find a transceiver for my computers end.
Hmm, have you followed the setup and licensing guide for the 6610 to actually unlock the 10gbe ports? And followed the part of the guide that includes setting int 1/3/1 - 1/3/8 to speed 10g? I'd test a regular 10gb fiber link to something to rule something weird out first

Edit: just saw your first post you already tested the front ports at 10gbe with fiber already, I need to not browse while half asleep. Maybe see if you can find a short cable to test a link with, and/or reboot the 6610 once the copper module has been installed
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,811
3,231
113
33
fohdeesha.com
Last thought: has that x540 ever successfully linked up with 10gbe copper modules? (Eg is that what it was connected to on the Aruba previously?) I would probably test that if not, kind of sounds like the x540 might not like linking with these modules, could be the length of the run or something
 

_Adrian_

Member
Jun 25, 2012
48
5
8
Leduc, AB
It's literally at the top of the first post.

Sorry, noticed it last night when i was at home. was browsing on my cell and didn't notice the link.
Followed the setup down to a T and worked 90% of it.

Problem in the end was that i could not access the GUI with any credentials including the default ones
had to resort to using the no password option.

Any insight on this ?

Thanks for the info dump BTW
 

clcorbin

Member
Feb 15, 2014
74
10
8
Hmm, have you followed the setup and licensing guide for the 6610 to actually unlock the 10gbe ports? And followed the part of the guide that includes setting int 1/3/1 - 1/3/8 to speed 10g? I'd test a regular 10gb fiber link to something to rule something weird out first

Edit: just saw your first post you already tested the front ports at 10gbe with fiber already, I need to not browse while half asleep. Maybe see if you can find a short cable to test a link with, and/or reboot the 6610 once the copper module has been installed
You are the man! As I expected, something obvious and trivial. I had enabled 10g on the front ports. Of unit 2 (rack). But never unit 1!

A quick "speed-duplex 10g-full" and unplugging and replugging the cable had everything up and running perfectly.

Thank you Kind Sir!
 
  • Like
Reactions: fohdeesha

NateR

New Member
Feb 10, 2022
28
21
3
Hello all,

A question regarding LAG on an ICX6450:

I've enabled LAG on a couple of ports on the switch, and on both NICs within Proxmox (box has 2x 1gbe connections, remote device I'm testing with has 1x 10gbe connection). Outbound (from 2x 1gbe to 1x 10gbe device) iperf with 2 threads results in 1.8GB/s, whereas inbound (1x 10gbe to the 2x 1gbe device) maxes at about 1.2GB/s. The more threads, the higher the throughput, but once I get it up to 6 or more threads, it hits the 1.8GB/s point (5 threads or less are under 1.8GB/s).

I understand that with LAG, the different connections could be assigned to one NIC or the other, but with 2 threads it's definitely using both ports (1.2GB/s vs. the single-thread speed of 0.9GB/s), but still not maxing out both NICs. Is it routing more than half of the packets through the port that is set as the "primary?" That's what it looks like may be happening. If so, is there a way to more-equally weight the ports, or configure it to be more of a round-robin back-and-forth type of thing?

Thank you
 

NateR

New Member
Feb 10, 2022
28
21
3
I'm relatively happy with my ICX 6450-24P - it's certainly the loudest thing in the room, but FAR quieter than when it arrived. Mods:
-2x Gryphon GDA4028-12BB (ended up being about the same loudness as the stock fans, but the tone is less annoying).
-330μf electrolytic capacitor on one of the fans to smooth the annoying PWM buzzing.
-Custom-designed 3D printed TPU fan isolators (looks like crap because I've not yet tuned TPU, but it works):
1646984735324.png
1646984796213.png
 

kpfleming

Active Member
Dec 28, 2021
416
214
43
Pelham NY USA
Hello all,

A question regarding LAG on an ICX6450:

I've enabled LAG on a couple of ports on the switch, and on both NICs within Proxmox (box has 2x 1gbe connections, remote device I'm testing with has 1x 10gbe connection). Outbound (from 2x 1gbe to 1x 10gbe device) iperf with 2 threads results in 1.8GB/s, whereas inbound (1x 10gbe to the 2x 1gbe device) maxes at about 1.2GB/s. The more threads, the higher the throughput, but once I get it up to 6 or more threads, it hits the 1.8GB/s point (5 threads or less are under 1.8GB/s).

I understand that with LAG, the different connections could be assigned to one NIC or the other, but with 2 threads it's definitely using both ports (1.2GB/s vs. the single-thread speed of 0.9GB/s), but still not maxing out both NICs. Is it routing more than half of the packets through the port that is set as the "primary?" That's what it looks like may be happening. If so, is there a way to more-equally weight the ports, or configure it to be more of a round-robin back-and-forth type of thing?

Thank you
802.3ad/LACP (the underlying protocol used for LAGs) has a limited set of path-assignment algorithms; feel free to look them up on Wikipedia or you favorite other resource :)

Also keep in mind that each node decides how it will send traffic; the switch may be using a different algorithm than the 'remote device', there is no guarantee that you will see the same behavior in both directions unless you've explicitly configured them to use the same algorithms. It is not unusual to see traffic in one direction (of a bidrectional link) choose one path while traffic in the other direction chooses a different path.
 
  • Like
Reactions: CWin

geeky_rand

New Member
Sep 15, 2021
5
0
1
following the Setup/Configure/Licensing Guide V2 at the top of this post, I've noticed the brocade-12-20-2021.zip does not include a Switch firmware for the ICX6610, is that on purpose?
 

geeky_rand

New Member
Sep 15, 2021
5
0
1
No, only bin files in there are FCXR08030u.bin and grz10100.bin. Thus the reason I'm asking

~/tftp/brocade-12-20-2021$ find . -name '*.bin'
./TFTP Content/ICX7xxx/mnz10118.bin
./TFTP Content/ICX7xxx/SPR08095fufi.bin
./TFTP Content/ICX7xxx/spz10118.bin
./TFTP Content/ICX7xxx/SPR08090mc.bin
./TFTP Content/ICX6610-FCX/grz10100.bin
./TFTP Content/ICX6610-FCX/FCXR08030u.bin
./TFTP Content/ICX6650/ICXR08030u.bin
./TFTP Content/ICX6650/fxz10101.bin
./TFTP Content/ICX64xx/ICX64S08030u.bin
./TFTP Content/ICX64xx/ICX64R08030u.bin
./TFTP Content/ICX64xx/kxz10105.bin
 

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,564
1,008
113
I don't think there even is a L2 firmware for the ICX6610... And even if there was, there would be absolutely no point in using it, because the ICX6610 does routing at wirespeed.

If you don't need L3 features, don't enable / use them. Pretty simple. Don't make your life unnecessary hard