Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

crackelf

Member
Apr 11, 2021
74
6
8
after reading through all of "that" - I find myself using the ignore member function for the first time. :eek: :(
XDDDDD LAWL ROFLCOPTER don't worry I'll let you get back to your beloved men's support forum.
Mod edited for crimes against vision.
 
Last edited by a moderator:
  • Sad
Reactions: AndroidCat

RobstarUSA

Active Member
Sep 15, 2016
235
104
43
Just had my first real weirdness with my 6610. I had a unifi access point go offline. Weird. I power cycled the POE port on my 6610, and while the port has 2 lights on, the cli says the port is down. I try the port next to it (19)...same issue. I grab a new spare out of the box, same issue. I test the RJ45 cable with a tester (all pairs good). I replace the cable anyway, same issue.

I reload the POE stack member these are attached into and everything is perfect again. ARGHH
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,875
3,374
113
34
fohdeesha.com
Just had my first real weirdness with my 6610. I had a unifi access point go offline. Weird. I power cycled the POE port on my 6610, and while the port has 2 lights on, the cli says the port is down. I try the port next to it (19)...same issue. I grab a new spare out of the box, same issue. I test the RJ45 cable with a tester (all pairs good). I replace the cable anyway, same issue.

I reload the POE stack member these are attached into and everything is perfect again. ARGHH
the spare you grabbed and tested, was it a spare AP, or a space ICX? if you had the problem even on a separate newly booted ICX, that def sounds like the unifi device was having issues. if it was a spare AP, and the same icx with the prob was still having the issue on the different AP, then yeah it sounds like a strange icx bug. what OS and PoE fw version are you on?
 
  • Like
Reactions: RobstarUSA

RobstarUSA

Active Member
Sep 15, 2016
235
104
43
the spare you grabbed and tested, was it a spare AP, or a space ICX? if you had the problem even on a separate newly booted ICX, that def sounds like the unifi device was having issues. if it was a spare AP, and the same icx with the prob was still having the issue on the different AP, then yeah it sounds like a strange icx bug. what OS and PoE fw version are you on?
Sorry for the confusion. I tried spare ap, cable & port in the same stack member. I have a 2 stack setup using the 40G QSFP connectors on back. I don't have another ICX6610 that has POE, so as a last resort I kicked just the POE stackmember which has my access points (only) in it.

I did everything according to the guide you setup (much thanks btw!).

SSH@BCORE#sh ver | i Version
SW: Version 08.0.30uT7f3
SW: Version 08.0.30uT7f3
Boot-Monitor Image size = 370695, Version:10.1.00T7f5 (grz10100)

Not sure offhand how to get the poe firmware version, but I remember upgrading it as per your guide.

I kinda wish I had waited for a deal, as I found the 48 port POE model with 2 PS for about $150 shipped on eBay which I thought was superb.
For my first one, non-poe, 48 port, It was $90 but only with one power supply.

To be fair I use Cisco all day at my day job and have had, at one time, 3 Cisco TAC Teams working for 24 hours straight on a multicast over GRE over ipsec issue that was not solvable by them. I found a workaround as I was on a tight time schedule & we eventually ended up removing that equipment as it was older (although still supported). I"m kinda famous @ work for finding issues :) There is an HP G6 or G7 BIOS out there that has a bios memory setting bug I found. HP Later released an updated bios and the only fix was the bug I found. So maybe it's just me & not the equpment :)
 
Last edited:

crackelf

Member
Apr 11, 2021
74
6
8
@fohdeesha The Infallible, confirmed fallible - ConnectX-4 Lx DO NOT work with KAIAM modules:
Another user is saying the fiber I have & transceivers won't work with the card...but I was convinced otherwise:
yes they're regular 40gbe ports. instead of expensive annoying mtp you can grab these BiDi optics and run 40gbE over cheap regular singlemode duplex LC fiber XQX2502 KAIAM QSFP+40G-LR4 Lite OPTICAL MODULE NEW PULLS | eBay
Code:
01:00.0 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] [15b3:1015]
    Kernel driver in use: mlx5_core
    Kernel modules: mlx5_core
Code:
show media ethernet 1/3/1
Port   1/3/1: Type  : 40GE-LR4   2km (QSFP+ LC)
         Vendor: KAIAM CORP           Version: 1A
         Part# : XQX2502              Serial#: KD60315209
Code:
Settings for enp1s0np0:
    Speed: Unknown!
    Duplex: Unknown! (255)
    Auto-negotiation: on
    Port: FIBRE
    PHYAD: 0
    Transceiver: internal
    Supports Wake-on: d
    Wake-on: d
        Current message level: 0x00000004 (4)
                               link
    Link detected: no
Proof that Mellanox programmed transceivers work:
Code:
show media ethernet 1/4/1
Port   1/4/1: Type  : 40GBASE-Passive Copper  2m (QSFP+)
         Vendor: Mellanox             Version: A3
         Part# : MC2207130-002        Serial#: MT1433VS02614
Code:
Settings for enp1s0np0:
    Speed: 40000Mb/s
    Duplex: Full
    Auto-negotiation: on
    Port: Direct Attach Copper
    PHYAD: 0
    Transceiver: internal
    Supports Wake-on: d
    Wake-on: d
        Current message level: 0x00000004 (4)
                               link
    Link detected: yes
Proof that KAIAM transceivers work properly with ConnectX-3 cards:
Code:
02:00.0 Network controller [0280]: Mellanox Technologies MT27500 Family [ConnectX-3] [15b3:1003]
    Kernel driver in use: mlx4_core
    Kernel modules: mlx4_core
Code:
show media ethernet 1/3/1
Port   1/3/1: Type  : 40GE-LR4   2km (QSFP+ LC)
         Vendor: KAIAM CORP           Version: 1A
         Part# : XQX2502              Serial#: KD60315209
Code:
Settings for enp2s0:
    Speed: 40000Mb/s
    Duplex: Full
    Auto-negotiation: off
    Port: FIBRE
    PHYAD: 0
    Transceiver: internal
    Supports Wake-on: d
    Wake-on: d
        Current message level: 0x00000014 (20)
                               link ifdown
    Link detected: yes
but as you've noticed, you're the only one in this thread having these issues. You could have bad fiber, you could have newer or older firmware, your motherboard might not like supplying extra power to the NIC, who knows.
I haven't noticed that. In fact two hours ago:
I have also had a fair amount of issues with SX6036 + LR4 Transceivers in the last couple of days (result: only Mellanox branded / programmed LR4 optics seem to be accepted by the SX6036)
...
I have tried Finisar FTL4C1QL2L LR4-Lite and FS.com modules and both only worked after I patched the vendor to "Mellanox"
Even worse ConnectX-5 and Infiniband, because the CX-5 seems to do a deeper inspection and still detects the modules as "FS" whereas the switch happily reports "Mellanox"
He's wrong like most people on the internet
So you're infallible? To me everyone here is just text on a screen. It's all equal in my eyes until I see something functional.
 

Zackey_TNT

New Member
Oct 31, 2020
3
2
3
yeah I think that last one might be the cringiest thing I've seen on here in years lmao
Could not agree more.... best part is, its showing on google results. I only just recently got here and this is what I found in relation to Brocade + Mellonox
That guy is crinnnnnnngggggeeee
 

crackelf

Member
Apr 11, 2021
74
6
8
yeah I think that last one might be the cringiest thing I've seen on here in years lmao
lol you guys should untuck your shirts every once in a while.

Still don't want to admit you were wrong though huh? It's not so hard, big man.
Could not agree more.... best part is, its showing on google results. I only just recently got here and this is what I found in relation to Brocade + Mellonox
That guy is crinnnnnnngggggeeee
This is the top SEO because there were no resources on this before me. The algorithm doesn't lie.

I'm simply out here contributing to the knowledge base through documentation. Keep up the "crinnnnnnngggggeeee" posts though buddy. Big lols from ur internet friends!

Nobody has brought any hard proof to this thread prior to my post. Just empty promises of "it works trust me!". Prior to my posts people would be lead to believe big @fohdeesha was right. I brought the receipts:
@fohdeesha The Infallible, confirmed fallible - ConnectX-4 Lx DO NOT work with KAIAM modules:
You got a nice new avatar for it!
My last artwork was removed for "crimes against vision" so I had to bring my A game :) I'm happy that it can be appreciated by our one true savior


edit: Don't worry I'm actually leaving this time. I'll miss you guys, and the fun we had here, but will keep people updated on any alternative modules and fiber in my other thread.
 
Last edited:

frogtech

Well-Known Member
Jan 4, 2016
1,500
279
83
36
Damn boys, ya'll popping off in here.

@fohdeesha

thanks for the setup guide. when I got my switches I tried following ruckus' update documentation which for the most part is probably fine but it seems to leave out some steps in their examples which caused me to somehow get a ruckus wireless bootloader flashed? don't ask how that's possible lol. very thankful that they have the preboot environment that lets you flash over TFTP
 

DaSnipe

New Member
Feb 27, 2022
3
2
3
Just wanted to thank you @fohdeesha for the massive guide, allowed me to purchase one of these ICX6450's on ebay for super cheap. Going to replace another tp-link POE switch and get my main sections of the network on 10gbpe. Got my TrueNAS and workstation running at 10gbpe. I got a FS.com RJ45 80m transceiver running 41m of Cat5e to the brocade and getting 10gbps connections and close to 6gbps on iperf/ 550 MB/s transfers.

I was going to ask a question but I figured it out before I finished this message haha. Now to see if this thing can power my Rb5009g via poe
 

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,691
1,123
113
Although this is not 100% correct - Mixed stacks of 6610 and 6450 are possible, for example.
 

NateR

Member
Feb 10, 2022
33
23
8
Hello all,

Plugged in a "new to me" 6450-24p, and the thing is hella loud. Ordered some Comair Rotron "Gryphon" GDA4028-12BB fans and it's... still hella loud. Pulled the fans out and running them outside the case, it looks like the most annoying part of the noise is the vibrations and the PWM buzz. Vibration isolation should be pretty easy, but the PWM buzz is the biggest issue at the moment.

I ran across this post (https://forums.servethehome.com/ind...be-40gbe-switching.21107/page-187#post-269342) from earlier in this thred, where a guy used a 330µF to quiet it down. I ran the same test he did briefly, and it does indeed eliminate the PWM buzz.

I've spent the entirety of this evening trying to understand out how low-pass filters work, but I must be incredibly dense, because I'm still just as puzzled as when I started. Based on what I've read, it sounds like you need a resistor and a capacitor to make it work - yet it seems to work just fine with a capacitor by itself.

My question: Does a capacitor, when used in the manner described in that post, have a chance of damaging anything? And if so, and a resistor really should be used, how would one determine what resistor value to use? (and anyone know the PWM frequency on the 6450? I've got a pretty nice (Brymen) multimeter, but the DC frequency function is reading 0hz or 60hz depending when I measure it, and that guy's video show 3khz).

Thank you
 
Last edited: