Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[Freebsd1976]

Thanks, that actually works pretty well! How did you connect the ASIC fan?

two way power asic fan ， use fan power（fan voltage will reduce to 4.5 or 5v when system fan on speed 1） or use 12v power from psu （always full speed for asic fan）
I use first method on icx7250-24 , temperature reduce 10C , but still a little bit high , now consider use second method

 

[mwarps]

After battling with it for over a week, I finally got SNMP working on my Brocade ICX 7250.

enable
conf t
snmp-server community private ro view admin
snmp-server view admin 1.3.6.1.4.1.1991 included
write mem


And I'm able to snmpwalk with:

snmpwalk -c private -v 2c $SWITCH_IP 1.3.6.1.4.1.1991

Now to get some SNMP to influxdb action going on!!!

 

[anemoiac]

two way power asic fan ， use fan power（fan voltage will reduce to 4.5 or 5v when system fan on speed 1） or use 12v power from psu （always full speed for asic fan）
I use first method on icx7250-24 , temperature reduce 10C , but still a little bit high , now consider use second method

Thanks! What's the best way to tap into the 12v power?

 

[OKGolombRuler]

After battling with it for over a week, I finally got SNMP working on my Brocade ICX 7250.

enable
conf t
snmp-server community private ro view admin
snmp-server view admin 1.3.6.1.4.1.1991 included
write mem


And I'm able to snmpwalk with:

snmpwalk -c private -v 2c $SWITCH_IP 1.3.6.1.4.1.1991

Now to get some SNMP to influxdb action going on!!!

I'd love to see a writeup of your InfluxDB setup when you get it working! I like pretty charts.

Also, for the good of the order-- I'm currently trying to make sense of Splunk (there's a low-volume free licence, maybe <5gb a day?)-- would folks be interested in a writeup on that, if anything useful shakes out of it?

 

[infoMatt]

there's a low-volume free licence, maybe <5gb a day?

500MB/day, actually.

 

[OKGolombRuler]

500MB/day, actually.

Ah, ok, I had put in for Develop | Splunkbase which is 10gb/day, 6 months renewable. Mea culpa! I think there's also a 50gb /6 month version, but frankly I'm not fussed about 500MB/day of logs as a limitation. I do wish they left the LDAP functionality in though; it's on my list to play with.

 

[up-n-atom]

Who wants to live dangerously?

I gave the FI reversing roughly a 2 year hiatus, but today my ADD got the better of me and FI nostalgia took precedence over work. 2.5Gbps was still on my agenda despite the interfering hardware. A big FU to high quality engineering and the DS100DF410 tucked inside!

I couldn't bother to dust off my Ghidra script FISeek - Pastebin.com to seek hidden commands in newer firmware, nor determine what disassembled firmware version was before me... It's dated and the hardcoded addresses within the script will not work on other versions but who cares (FYI It was 8080b and @fohdeesha has the list buried in their guide if you're interested).

5-10 minutes into it, I had what I needed, Broadcom Shell access! Don't know why it evaded me last time, since it is deadly obvious. I knew the command was tor but I was oblivious to enabling it. Stare at the disassembly for too long and you lose sight of things.

Now that I have the holy grail of switch privileged access, it's a no brainer to try out 2.5G. Guess what? It doesn't work outside of the switch ASIC Obvious, I know... Maybe the 7150 lineup isn't up to the engineering prowess of the 7250? I can only hope it's equal to that of Ubiquiti hardware Someone, please run dm i2c or take pictures of your 7150 PCB.

FYI You can replace dm with dd for the commands below.

Anyway, to enable access to Broadcom Shell, run dm device-command tor or the cheeky dm device-command 2762233 (It spells out BROCADE on a touch tone phone).

DANGER!!! YOU CAN ABSOLUTELY BRICK OR DAMAGE YOUR SWITCH! TREAD CAREFULLY!

Run tor help and get intimate with your switch.

For extra help you can view code at src/appl/diag · bcm-6.5.7_yocto-2.5.1 · Kosta Zertsekel / bcm-sdk

Eg. My 2.5G adventure (Yes, the WarpCore phys support 2500) [Internet] Bypassing the HH3K up to 2.5Gbps using a BCM57810S NIC - Bell Canada | DSLReports Forums

phy 1/2/1
Port xe0 (PHY addr 0xc1): WC-B1/13/0 (Warpcore PHY Driver)
        0x00: 0x0000    0x01: 0x0000    0x02: 0x0000    0x03: 0x0000
        0x04: 0x0000    0x05: 0x0000    0x06: 0x0000    0x07: 0x0000
...
tor phy info
  xe0( 50)   143  bff0    c1    c1              WC-B1/13/0     250000
  xe1( 51)   143  bff0    c1    c1              WC-B1/13/1     250000
  xe2( 52)   143  bff0    c1    c1              WC-B1/13/2     250000
...

enable
dm device-command tor
tor port xe2 speed=2500
tor port xe2
ping 192.168.1.10

:(

If someone discovers something of interest please do share because I've only played for a few hours.

If you updated to FastIron 09.0.0+ the tor enablement command changed to dm device-command tor enable.

 

[techknight1]

If you update ICX 7250 to 09.0.0 are the licensing requirements any different? I am toying with the idea of getting one and wanted to find out as much as I could about the licensing. If anyone could provide information it would greatly be appreciated.

 

[fohdeesha]

If you update ICX 7250 to 09.0.0 are the licensing requirements any different? I am toying with the idea of getting one and wanted to find out as much as I could about the licensing. If anyone could provide information it would greatly be appreciated.

Nope, same honor based lics. I'd like to emphasize to everyone that the v09000 release is a brand new codetrain entirely and a GA release, not even lettered yet. Expect many quirks and bugs, I've already ran into a handful poking around. The WEB UI is certainly....shinier than the old one, I personally don't care for any web UI but I have a feeling a lot of people here (casual users) would prefer it over the old web UI

 

[Ralph_IT]

New owner of a 6450-24 here.
Thanks to all for sharing your knowledge, specially @fohdeesha.
I had to enable L3 capabilities and upgrade the firmware, but the manual is very clear.

Unfortunately fan is loud (for me), even after booting and settle down the rpms a little.
I've seen that is a general consensus to use a Sunon MagLev 40x40x20, but I have one Noctua NF-A4 to spare.
Will it be enough or better use the Sunon?

Also, where do I have to register to access official docs for the switch? Don't know if its Broadcom (since they adquired Brocade) or Ruckus.

Thanks.

 

[fohdeesha]

New owner of a 6450-24 here.
Thanks to all for sharing your knowledge, specially @fohdeesha.
I had to enable L3 capabilities and upgrade the firmware, but the manual is very clear.

Unfortunately fan is loud (for me), even after booting and settle down the rpms a little.
I've seen that is a general consensus to use a Sunon MagLev 40x40x20, but I have one Noctua NF-A4 to spare.
Will it be enough or better use the Sunon?

Also, where do I have to register to access official docs for the switch? Don't know if its Broadcom (since they adquired Brocade) or Ruckus.

Thanks.

Did you read/use the upgrade and config guide linked in my OP? it contains a ZIP that has all of the official docs. Broadcom only acquired brocades fibrechannel stuff, ruckus acquired the switching line

 

[Ralph_IT]

Did you read/use the upgrade and config guide linked in my OP? it contains a ZIP that has all of the official docs. Broadcom only acquired brocades fibrechannel stuff, ruckus acquired the switching line

Yes, I did.
All the process went flawlessly. Just stopped before "Advanced Configuration" (not enough spare time).
As for the registration: didn't realized the ZIP had all the docs. Thanks again.

Just a little search with the right key words and I can reply to myself: Noctuas? Bad idea it seems

 

[techknight1]

Nope, same honor based lics. I'd like to emphasize to everyone that the v09000 release is a brand new codetrain entirely and a GA release, not even lettered yet. Expect many quirks and bugs, I've already ran into a handful poking around. The WEB UI is certainly....shinier than the old one, I personally don't care for any web UI but I have a feeling a lot of people here (casual users) would prefer it over the old web UI

I guess I didn't realize that the v09 was that new. I will more than likely stick with a version a bit older and tested.

I didn't know that Brocade had been split up like that. I thought Broadcom got it all. I have only ever used the Fibre Channel and FCoE equipment, which brings me to the next question.

Did Broadcom get the Brocade 8000 series? Or did Ruckus get it? It is a great 10Gb switch, albeit a little power hungry. It is also relatively quiet. Of course after having a Dell M1000e in a home lab everything but Force10 is quiet.

 

[klui]

Broadcom only acquired brocades fibrechannel stuff, ruckus acquired the switching line

Did it change hands again? I thought Extreme got the FC stuff?

 

[fohdeesha]

I guess I didn't realize that the v09 was that new. I will more than likely stick with a version a bit older and tested.

I didn't know that Brocade had been split up like that. I thought Broadcom got it all. I have only ever used the Fibre Channel and FCoE equipment, which brings me to the next question.

Did Broadcom get the Brocade 8000 series? Or did Ruckus get it? It is a great 10Gb switch, albeit a little power hungry. It is also relatively quiet. Of course after having a Dell M1000e in a home lab everything but Force10 is quiet.

Broadcom got that series of FC/fcoe switches including the 8000, yes

Brocade was split in a million pieces, the fibrechannel, fcoe and serveriron (adx) stuff all went to Broadcom who then EoLd almost all of it, the fastiron switching line went to ruckus, and the datacenter and routing line went to Extreme (ces/cer/mlx/vdx)

 

[fohdeesha]

cost of licenses has gone up: I require each requesting user to provide one URL to a youtube video they would categorize as "really good", the rest is up to you godspeed

 

[fohdeesha]

Two hours in and I have already been rickrolled. I've underestimated you guys

 

[techknight1]

Two hours in and I have already been rickrolled. I've underestimated you guys

What all do you provide licenses for?

 

[anemoiac]

For my 7250-24P, I started with replacing the rear fans with 2x MF40201V3-1000U-G99. I found that the fans were near silent when running at speed 1, but it would intermittently kick up into speed 2 when the ASIC hit a high enough temp. This sits in a rack right next to me, so I added 1x MF60101V3-1000U-A99 that sits directly on top of the ASIC heatsink and is tapped into the constant 12v to the mainboard, and it never kicks into speed 2 now, even sitting in my rack that runs ~80F inside.

@FozzieBear Do you mind sharing how you tapped into the 12v? I have the same setup, and am trying to get the fan I attached to the ASIC working happily with the others, but am not sure how to go about powering it.

 

[Vesalius]

What all do you provide licenses for?

Really? first of this very long thread has all the answers.