Turbocharge your Quanta LB6M - Flash To Brocade TurboIron - Big Update!

[fohdeesha]

Any idea if there's something similar that could be done for the lb4m? I have a few here that I can try things on and a jtag for if things go sideways

EDIT - found this, looks like somebody got u-boot to run on the lb4m

I don't know anything offhand that matches the lb4m exactly, it's a slightly strange architecture. You need something with an MPC8541 management CPU with a BCM56514 ASIC. On top of that, if you want the 10gig ports to work, it also needs to have a pair of BCM8706's converting the higig/XAUI interface on the main ASIC into the SFP+ interfaces

some potential brocade models that might match this topology are an FCX, a GS, and an FESX. I own all 3 and can take a look when I get some free time

EDIT: cross the FCX off the list, it uses a newer management CPU with support for 64 bit instructions, MPC8544E

EDIT2: the GS, FESX4 and newer FESX6 all use the way older MPC8245, so all of these are a no go. RIP

 

[fohdeesha]

That said, if one of them do match, and that method you quoted works to get rid of the vxworks bootloader and get uboot on there, it should be a simple process assuming everything else lines up. If I find something I'll pm you to test. Are you sure your JTAG unit supports the powerPC JTAG interface? It's quite a bit different and not many models do

 

[epers]

That said, if one of them do match, and that method you quoted works to get rid of the vxworks bootloader and get uboot on there, it should be a simple process assuming everything else lines up. If I find something I'll pm you to test. Are you sure your JTAG unit supports the powerPC JTAG interface? It's quite a bit different and not many models do

That method works, I've got uboot working on one of the 4m's here. Can't figure out how to boot anything with it, but I can tftp images into flash and/or ram via the outband mgmt port

U-Boot 1.3.0 (Apr  7 2011 - 23:27:58)

CPU:   8541, Version: 1.1, (0x80720011)
Core:  E500, Version: 2.0, (0x80200020)
Clock Configuration:
       CPU: 825 MHz, CCB: 330 MHz,
       DDR: 165 MHz, LBC:  41 MHz
L1:    D-cache 32 kB enabled
       I-cache 32 kB enabled
I2C:   ready
DRAM:  Initializing
    DDR: 512 MB
FLASH: 32 MB
L2 cache 256KB: enabled
Set ethaddr MAC address = c8:0a:a9:9e:5b:ca
Set eth1addr MAC address = c8:0a:a9:9e:5b:cb
In:    serial
Out:   serial
Err:   serial
Net:   TSEC0, TSEC1
Hit any key to stop autoboot:  0

The pronto/pica8 3290 might be a candidate, I'll do some more digging - http://www.pica8.com/documents/pica8-datasheet-48x1gbe-p3290-p3295.pdf

There's a 2x8 header on the board labeled "J2", I'm thinking that's the jtag interface. I'll check with the ICT engies today, but if we don't have a powerpc jtag I'm sure we can get one. Do you think something like this would work? - Wind River VisionICE II PowerPC MIPS RISC JTAG Emulator/Analyzer PCT-00146-002 | eBay

Worst case, the switches are only 60$ on ebay these days, so I'm not too worried about it - if nothing else I get a pair of psu's and a fan try to put on the spares shelf

 

[vrod]

How many have done this flash?

I did it on my quanta yesterday but I have started experiencing random freezes of the switch. Saw the interfaces flap a lot on one of my servers and the switch now doesn't seem to respond anymore, not even on ping. Looks like VLAN functions works as usual but the LACP pair I made does no longer function. Have anyone experienced this?

 

[fohdeesha]

How many have done this flash?

I did it on my quanta yesterday but I have started experiencing random freezes of the switch. Saw the interfaces flap a lot on one of my servers and the switch now doesn't seem to respond anymore, not even on ping. Looks like VLAN functions works as usual but the LACP pair I made does no longer function. Have anyone experienced this?

Between reports here and on reddit, I'd say a little over 50 chassis flashed, zero reported issues. Considering you were having some serious switch issues on quanta before flashing(one port/host not even being able to grab a dhcp address or pass traffic), I'd be willing to bet your problems are either cabling related, or your hardware was on the way out already (it happens, @verygeekys management port died shortly after he got it off ebay). Have you checked interface statistics for errors and other info?

When you say it doesn't respond and "freezes", are you trying to access it over a dedicated mgmt or serial port with known-good cabling, or over the ports that have a history of flapping and not being able to pass traffic?

 

[vrod]

I have only seen it on one server, there was a lot of link flapping. I went to the DC and rebooted the switch. saw more link flapping and it crashed again. I disconnected one of the ports from the server and it seems to run fine now.... I did not have time to investigate further but I'll be trying to swap the cables in a couple of days. Hopefully it will run smooth until then.

BTW. Serial also seems to be on crack when it happens.

 

[fohdeesha]

hrmm, one bad/flapping port should definitely not be interfering with serial output. It sounds like your hardware might be going bad if it's truly locking up. does "show log" show anything? how about "show chassis", temps look normal? "show cpu" as well

if you run "show tech-support" it'll spit out a ton of stuff including flash registers, if you can save that output by temporarily turning on logging in putty or something and send me the resulting file I can have a look through it to make sure the flash went ok and there's no sign of hardware issues

 

[vrod]

I looked at the logs:

show log

Spoiler

Dynamic Log Buffer (50 lines):
0d03h41m39s:I:Security: telnet login by un-authenticated telnet user from src IP 1.2.3.4, src MAC 0000.0000.0000 to USER EXEC mode
0d00h02m52s:I:System: Interface ethernet 26, state down
0d00h02m46s:I:Security: telnet logout by un-authenticated telnet user from src IP 1.2.3.4, src MAC 0000.0000.0000
0d00h01m36s:I:Security: telnet login by un-authenticated telnet user from src IP 1.2.3.4, src MAC 0000.0000.0000 to USER EXEC mode
0d00h00m34s:I:System: Logical link on interface ethernet 8 is up.
0d00h00m34s:I:System: Interface ethernet 8, state up
0d00h00m32s:I:System: Logical link on interface ethernet 8 is down.
0d00h00m31s:I:Trunk: Group (7, 8) created by 802.3ad link-aggregation module.
0d00h00m26s:W:System: Fan speed changed automatically to 2
0d00h00m25s:I:System: Interface ethernet 26, state up
0d00h00m25s:I:Trunk: Group (7, 8) removed by 802.3ad link-aggregation module.
0d00h00m24s:I:System: Interface ethernet 25, state up
0d00h00m24s:I:System: Interface ethernet 28, state up
0d00h00m24s:I:System: Interface ethernet 27, state up
0d00h00m24s:I:Trunk: Group (7, 8) created by 802.3ad link-aggregation module.
0d00h00m21s:I:System: Interface ethernet 6, state up
0d00h00m21s:I:System: Interface ethernet 12, state up
0d00h00m21s:I:System: Interface ethernet 11, state up
0d00h00m21s:I:System: Interface ethernet 10, state up
0d00h00m21s:I:System: Interface ethernet 9, state up
0d00h00m21s:I:System: Interface ethernet 5, state up
0d00h00m21s:I:System: Interface ethernet 2, state up
0d00h00m21s:I:System: Interface ethernet 1, state up
0d00h00m21s:I:System: Cold start
0d00h00m19s:W:System: Fan speed changed automatically to 1

show chassis

Spoiler

Power supply 1 (NA - NA - Regular) present, status ok
Power supply 2 not present

Fan 1 ok, speed (auto): 1<->[[2]]<->3
Fan 2 ok, speed (auto): 1<->[[2]]<->3
Fan 3 ok, speed (auto): 1<->[[2]]<->3

Fan controlled temperature: 53.0 deg-C

Fan speed switching temperature thresholds:
Speed 1: NM<----->30 deg-C
Speed 2: 25<----->65 deg-C
Speed 3: 60<----->90 deg-C (shutdown)


Exhaust Side Temperature Readings:
Current temperature : 52.0 deg-C
Warning level.......: 80.0 deg-C
Shutdown level......: 90.0 deg-C
Intake Side Temperature Readings:
Current temperature : 50.0 deg-C
Boot Prom MAC: 00e0.5200.0100

show cpu

Spoiler

2 percent busy, from 13471 sec ago
1 sec avg: 5 percent busy
5 sec avg: 2 percent busy
60 sec avg: 2 percent busy
300 sec avg: 2 percent busy

I will PM you the show tech-support file. thank you again.

 

[Blue)(Fusion]

Are you sure you have LACP set up correctly at both ends of the link? I had run into pfSense flapping with LACP LAG set up on both sides....or so I thought, but was accidentally in "static" LAG mode on the switch, which disabled LACP, creating confusion on the pfSense box.

 

[vrod]

I think that could be the case... with another server, after configuring lacp I see a lot of flapping there too. Isn’t bond-mode 4 LACP?

In the quanta switch, with the brocade firmware, I have set the lag to active.

 

[fohdeesha]

I need someone who has flashed to brocade and has been using the management 1 port succesfully to reboot into the bootloader, and pastebin me the output of "type boot-parameter"

 

[TheBloke]

Wow, you guys are amazing! Thank you so much for all the hard work in not only figuring this out and publishing it, but also in packaging everything up and creating clear and simple instructions. It's really appreciated.

Needless to say, I just did my LB6M and it worked perfectly. I got mine in the UK about nine months ago, but I've only just started using it properly. I've just got back into setting up my home Solaris NAS/VM server, and this week have been configuring and benchmarking my 2 x 10GB links between desktop and server. So naturally I had to jump on this switch upgrade the moment I realised it existed. I like having the latest/best possible stuff, even if I'll probably never use 99% of it

One tiny correction on the docs, for the sake of completeness: in the output paste following md 0xfff80000 20, you're missing a space at the end of line 2, ie after 06 (Apr 19 2011. It looks on screen like it's there, but it's not in the text. I only realised this because I copied your expected text block and the output from my router into separate files so I could diff them to be 100% safe. It showed 1 byte difference, but as the hex numbers on the left matched I realised what must have happened and it was no problem. Just thought I'd let you know.

Thanks again for this, it's really awesome. Now I need to go read some docs to learn what my new switch can do

 

[TheBloke]

I need someone who has flashed to brocade and has been using the management 1 port succesfully to reboot into the bootloader, and pastebin me the output of "type boot-parameter"

I literally just flashed mine and I know almost nothing about this, but I have confirmed that mgmt #1 is pingable and I can telnet to it. So I tried to do what you asked, but don't know if I did it right:

• Via serial, I entered: reload
• Then I hit b when it prompted: Enter 'b' to stop at boot monitor
• This got me the Monitor> prompt, at which I typed: type boot-parameter

But that only returned this error: Find not found 'boot-parameter'

I then checked what files I do have with dir, and I only have the following:

   6845067 [0000] primary
   6845067 [0000] secondary
       121 [9636] startup-config.txt
  13690255 bytes 3 File(s)

Quite likely I just don't understand what I'm meant to do - so probably someone else will get you what you need. But if I can try anything else, let me know.

PS. The file I do have, startup-config.txt, has the following contents:

Spoiler

ver 08.0.01eT203
!
!
!
!
!
!
!
!
!
interface management 1
 ip address 192.168.0.252 255.255.255.0
!
!
!
!
!
!
!
!
!
end

 

[fohdeesha]

Actually that's a good thing that it's not there, I was thinking it shouldn't be. I found out it's possible that the system does not bring the management port 1 up properly, which is what happened to @verygeeky's box when his mgmt port "died", and now a test box of mine. It can be brought up manually using some low level PHY test commands and then it works fine, so it's a software issue.


Can you do me one last favor, and completely pull power from your flashed quanta (so it's completely off), then re-power it? then check if mgmt 1 port links up and can pass traffic? that will 100% verify this for me. ( a warm reload is not enough to test this)

Will add the missing space to the guide, thanks!

 

[TheBloke]

OK good to know, thanks.

I just did the hard power cycle, and yes I was able to access the management port with ping, telnet and SSH. No problem at all.

 

[fohdeesha]

Thanks! That 100% confirms it's a software issue. It seems if you re-flash the bootloader from within the bootloader, it wipes some left over uboot environment variables the brocade depends on and doesn't rewrite them. Now to figure out where they're stored and get them back.

edit: rong, it happens to everyone

 

[fohdeesha]

Can you do me one last favor, reboot into the bootloader, and in there, do "reset 0" - this just reboots it. It should work as normal (just freezes on my incorrectly flashed test box)

 

[TheBloke]

Can you do me one last favor, reboot into the bootloader, and in there, do "reset 0" - this just reboots it. It should work as normal (just freezes on my incorrectly flashed test box)

No that doesn't seem to work. I get no output back, and hitting enter again gives me a different prompt and nothing seems to work from there on.

I actually did this by accident earlier, when I first dropped to bootloader to check for boot-parameter for you; I typed 'reset' instead of 'reload' and it did this same thing as it's doing now with 'reset 0'. In the end I hard cycled the power to get it going again.

***** Interrupted by entering 'b' *****
.
BOOT INFO: load monitor from boot flash, cksum = 7be5
2018/01/17 03:29:18: BOOT INFO: verify flash files.....
Monitor>
Monitor>reset 0
2018/01/17 03:29:27: reset 0
2018/01/17 03:29:45:
2018/01/17 03:30:42:
2018/01/17 03:30:45: ?
2018/01/17 03:30:49: reload
2018/01/17 03:30:52: asdgfdsksmlkigsd

 

[fohdeesha]

OK, good to know. seems like it's just a harmless bug in the brocade bootloader then. In future if you need to get out of the BL cleanly you can do "boot system flash primary"

and after pulling power after a frozen reset, your management port works still correct? Last question Thanks for all the help!

 

[TheBloke]

Yup, works fine

One (unimportant) thing I did notice: it looks like mgmt #1 is 100mbit now, where before it was gigabit? At least the (very cheap, dumb) gigabit Netgear switch I'm currently using for 1G is showing the yellow light for 100mbit now where it was definitely green for gigabit before I did the flash.

Or maybe that's configurable and I just haven't configured it? I don't see any likely options under ip ? though.

Doesn't matter at all, just a difference I noticed.