Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

fohdeesha

Kaini Industries
Nov 20, 2016
2,874
3,374
113
34
fohdeesha.com
The x540 doesn't use SFP+ modules. The transceivers are built in. I've used it at 10Gb with my old Aruba S3500 with no issues. And using the exact same module from the Aruba in the ICX 6610 1/3/x slots (tried multiple slots), it won't connect.

I realized I hadn't tested the cable from end to end yet, so I was hoping that would be a fail, but the cable tester came back with a pass and verified I am using the correct cable at both ends.

So do they make any QSFP transceivers for copper 10Gb? I have three of the dual port Mellanox cards gathering dust I could use if I could find a transceiver for my computers end.
Hmm, have you followed the setup and licensing guide for the 6610 to actually unlock the 10gbe ports? And followed the part of the guide that includes setting int 1/3/1 - 1/3/8 to speed 10g? I'd test a regular 10gb fiber link to something to rule something weird out first

Edit: just saw your first post you already tested the front ports at 10gbe with fiber already, I need to not browse while half asleep. Maybe see if you can find a short cable to test a link with, and/or reboot the 6610 once the copper module has been installed
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,874
3,374
113
34
fohdeesha.com
Last thought: has that x540 ever successfully linked up with 10gbe copper modules? (Eg is that what it was connected to on the Aruba previously?) I would probably test that if not, kind of sounds like the x540 might not like linking with these modules, could be the length of the run or something
 

_Adrian_

Member
Jun 25, 2012
48
5
8
Leduc, AB
It's literally at the top of the first post.

Sorry, noticed it last night when i was at home. was browsing on my cell and didn't notice the link.
Followed the setup down to a T and worked 90% of it.

Problem in the end was that i could not access the GUI with any credentials including the default ones
had to resort to using the no password option.

Any insight on this ?

Thanks for the info dump BTW
 

clcorbin

Member
Feb 15, 2014
76
10
8
Hmm, have you followed the setup and licensing guide for the 6610 to actually unlock the 10gbe ports? And followed the part of the guide that includes setting int 1/3/1 - 1/3/8 to speed 10g? I'd test a regular 10gb fiber link to something to rule something weird out first

Edit: just saw your first post you already tested the front ports at 10gbe with fiber already, I need to not browse while half asleep. Maybe see if you can find a short cable to test a link with, and/or reboot the 6610 once the copper module has been installed
You are the man! As I expected, something obvious and trivial. I had enabled 10g on the front ports. Of unit 2 (rack). But never unit 1!

A quick "speed-duplex 10g-full" and unplugging and replugging the cable had everything up and running perfectly.

Thank you Kind Sir!
 
  • Like
Reactions: fohdeesha

NateR

Member
Feb 10, 2022
33
23
8
Hello all,

A question regarding LAG on an ICX6450:

I've enabled LAG on a couple of ports on the switch, and on both NICs within Proxmox (box has 2x 1gbe connections, remote device I'm testing with has 1x 10gbe connection). Outbound (from 2x 1gbe to 1x 10gbe device) iperf with 2 threads results in 1.8GB/s, whereas inbound (1x 10gbe to the 2x 1gbe device) maxes at about 1.2GB/s. The more threads, the higher the throughput, but once I get it up to 6 or more threads, it hits the 1.8GB/s point (5 threads or less are under 1.8GB/s).

I understand that with LAG, the different connections could be assigned to one NIC or the other, but with 2 threads it's definitely using both ports (1.2GB/s vs. the single-thread speed of 0.9GB/s), but still not maxing out both NICs. Is it routing more than half of the packets through the port that is set as the "primary?" That's what it looks like may be happening. If so, is there a way to more-equally weight the ports, or configure it to be more of a round-robin back-and-forth type of thing?

Thank you
 

NateR

Member
Feb 10, 2022
33
23
8
I'm relatively happy with my ICX 6450-24P - it's certainly the loudest thing in the room, but FAR quieter than when it arrived. Mods:
-2x Gryphon GDA4028-12BB (ended up being about the same loudness as the stock fans, but the tone is less annoying).
-330μf electrolytic capacitor on one of the fans to smooth the annoying PWM buzzing.
-Custom-designed 3D printed TPU fan isolators (looks like crap because I've not yet tuned TPU, but it works):
1646984735324.png
1646984796213.png
 

kpfleming

Active Member
Dec 28, 2021
432
222
43
Pelham NY USA
Hello all,

A question regarding LAG on an ICX6450:

I've enabled LAG on a couple of ports on the switch, and on both NICs within Proxmox (box has 2x 1gbe connections, remote device I'm testing with has 1x 10gbe connection). Outbound (from 2x 1gbe to 1x 10gbe device) iperf with 2 threads results in 1.8GB/s, whereas inbound (1x 10gbe to the 2x 1gbe device) maxes at about 1.2GB/s. The more threads, the higher the throughput, but once I get it up to 6 or more threads, it hits the 1.8GB/s point (5 threads or less are under 1.8GB/s).

I understand that with LAG, the different connections could be assigned to one NIC or the other, but with 2 threads it's definitely using both ports (1.2GB/s vs. the single-thread speed of 0.9GB/s), but still not maxing out both NICs. Is it routing more than half of the packets through the port that is set as the "primary?" That's what it looks like may be happening. If so, is there a way to more-equally weight the ports, or configure it to be more of a round-robin back-and-forth type of thing?

Thank you
802.3ad/LACP (the underlying protocol used for LAGs) has a limited set of path-assignment algorithms; feel free to look them up on Wikipedia or you favorite other resource :)

Also keep in mind that each node decides how it will send traffic; the switch may be using a different algorithm than the 'remote device', there is no guarantee that you will see the same behavior in both directions unless you've explicitly configured them to use the same algorithms. It is not unusual to see traffic in one direction (of a bidrectional link) choose one path while traffic in the other direction chooses a different path.
 
  • Like
Reactions: CWin

geeky_rand

New Member
Sep 15, 2021
5
0
1
following the Setup/Configure/Licensing Guide V2 at the top of this post, I've noticed the brocade-12-20-2021.zip does not include a Switch firmware for the ICX6610, is that on purpose?
 

geeky_rand

New Member
Sep 15, 2021
5
0
1
No, only bin files in there are FCXR08030u.bin and grz10100.bin. Thus the reason I'm asking

~/tftp/brocade-12-20-2021$ find . -name '*.bin'
./TFTP Content/ICX7xxx/mnz10118.bin
./TFTP Content/ICX7xxx/SPR08095fufi.bin
./TFTP Content/ICX7xxx/spz10118.bin
./TFTP Content/ICX7xxx/SPR08090mc.bin
./TFTP Content/ICX6610-FCX/grz10100.bin
./TFTP Content/ICX6610-FCX/FCXR08030u.bin
./TFTP Content/ICX6650/ICXR08030u.bin
./TFTP Content/ICX6650/fxz10101.bin
./TFTP Content/ICX64xx/ICX64S08030u.bin
./TFTP Content/ICX64xx/ICX64R08030u.bin
./TFTP Content/ICX64xx/kxz10105.bin
 

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,686
1,117
113
I don't think there even is a L2 firmware for the ICX6610... And even if there was, there would be absolutely no point in using it, because the ICX6610 does routing at wirespeed.

If you don't need L3 features, don't enable / use them. Pretty simple. Don't make your life unnecessary hard
 

geeky_rand

New Member
Sep 15, 2021
5
0
1
I don't think there even is a L2 firmware for the ICX6610... And even if there was, there would be absolutely no point in using it, because the ICX6610 does routing at wirespeed.

If you don't need L3 features, don't enable / use them. Pretty simple. Don't make your life unnecessary hard
TIL. asked and answered. Thanks.
 

zunder1990

Active Member
Nov 15, 2012
230
81
28
I don't think there even is a L2 firmware for the ICX6610... And even if there was, there would be absolutely no point in using it, because the ICX6610 does routing at wirespeed.

If you don't need L3 features, don't enable / use them. Pretty simple. Don't make your life unnecessary hard
yes it does exists it is what i am using I got it from the ruckus website since I have an account. Here is the link if you have an account Log in to Ruckus Wireless
 

RobstarUSA

Active Member
Sep 15, 2016
235
104
43
Also: You can do switching with the firmware from the guide. I am. Lots of "access" single vlan ports with no ip address on them. Works fine.
 

geeky_rand

New Member
Sep 15, 2021
5
0
1
Also: You can do switching with the firmware from the guide. I am. Lots of "access" single vlan ports with no ip address on them. Works fine.
yes it does exists it is what i am using I got it from the ruckus website since I have an account. Here is the link if you have an account Log in to Ruckus Wireless
My issue was that dual-mode wasn't working, turns out it's a PROCESS. I figured it out now.

Thanks all.
 

lvx4

New Member
Jun 13, 2020
7
2
3
So I picked up a 7150-24 and a couple of ConnectX-3 cards and DACs. This is my first attempt at setting up 10g at home.

Reset the switch, licensed for 4x10gr, all seems good. RJ45 ports work and all that.

But I can't get the SFP+ ports to link. Tried with a pair of Mellanox CX311A cards as well as SFP+ DACs connected to a couple different machines. No lights, no link.

The switch sees the DACs, but they just won't link. When plugged into each other, CX311A cards will happily link but not when plugged into the switch.

What gives? What am I missing here? I'm sure it's something stupid, but this is all new hardware to me.

These are the DACs I'm using: https://www.amazon.com/dp/B09CDNL2Z8



edit:

Alright, I'm pretty dumb. speed-duplex 10g-full fixed it.
 
Last edited:
  • Like
Reactions: kpfleming

pinkypie

New Member
Dec 2, 2021
20
3
3
Got a 6450 up and running. I have VLANs set up but I am having trouble figuring out how to mange the switch from a VLAN.

If I plug into the switch, let's say port 1/1/1 with no VLAN assigned, I can manage the switch no problem. Problem is that I need to assign VLANs to all ports so I can use ACLs. How can I create a management VLAN to be able to manage the switch via ssh connection?

I've read the guides and reference material, watched videos and can't figure out how to create management VLAN.
 

LodeRunner

Active Member
Apr 27, 2019
553
235
43
Got a 6450 up and running. I have VLANs set up but I am having trouble figuring out how to mange the switch from a VLAN.

If I plug into the switch, let's say port 1/1/1 with no VLAN assigned, I can manage the switch no problem. Problem is that I need to assign VLANs to all ports so I can use ACLs. How can I create a management VLAN to be able to manage the switch via ssh connection?

I've read the guides and reference material, watched videos and can't figure out how to create management VLAN.
I believe that you just need to assign a router-interface to the VLAN that you want to have management access, then assign an IP to that VE.