Turbocharge your Quanta LB6M - Flash To Brocade TurboIron - Big Update!

[fohdeesha]

so I completely wiped the flash entirely, reverted to quanta, management port works perfectly. flash to brocade, stops working immediately (can still bring it back with PHY test). I really have no clue what is going on, I'm worried it's an MII config register or clock issue

However that doesn't explain why only some of us are seeing this. Can anyone out there confirm after flashing, you can pull power from your unit, cold boot it, and mgmt comes up at 1gbe? What confuses me even more is how we're seeing different levels of failure (verygeekys failed completely, yours failed back to 1oombit, mine failed completely)

 

[AT S37=0]

My LB6M isn't powered at the moment, but I did take notes when it was up.

=> printenv
bootargs=root=/dev/ram rw mtdparts=physmap-flash.0:32256k(jffs2),512k(u-boot),128k@32512k(u-boot-env)
bootcmd=fsload 0x08000000 ${active} ; bootm 0x08000074 ; fsload 0x08000000 ${backup} ; bootm 0x08000074 ; fsload 0x08000000 image1 ; bootm 0x08000074 ; fsload 0x08000000 image2 ; bootm 0x08000074
bootdelay=3
loads_echo=1
eth1addr=00:E0:0C:00:01:FD
eth2addr=00:E0:0C:00:02:FD
loadaddr=0x08000000
mtdparts=mtdparts=physmap-flash.0:32256k(jffs2),512k(u-boot),128k@32512k(u-boot-env)
mtdids=nor0=physmap-flash.0
active=image1
backup=image2
ethact=TSEC0
baudrate=9600
ethaddr=54:AB:3A:42:0B:41
stdin=serial
stdout=serial
stderr=serial
partition=nor0,0
mtddevnum=0
mtddevname=jffs2

EEPROM Data :
0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 
0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 
0x4c 0x42 0x36 0x4d 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 
0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 
0x51 0x55 0x41 0x4e 0x54 0x41 0x20 0x4c 0x42 0x36 0x4d 0x00 0x00 0x00 0x00 0x00 
0x00 0x00 0x00 0x00 0x31 0x4c 0x42 0x36 0x42 0x5a 0x5a 0x30 0x53 0x54 0x4a 0x00 
0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x51 0x54 0x46 0x43 0x52 0x57 0x36 0x30 
0x33 0x30 0x31 0x31 0x38 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x31 0x00 0x00 0x00 
0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 
0x02 0x00 0x00 0x00 0xb1 0xa2 0xb3 0xe4 0x07 0xe0 0x01 0x09 0x54 0xab 0x3a 0x42 
0x0b 0x40 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 
0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01 0x32 0xde 0xc6 
0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 
0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 
0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 
0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 

U-Boot 2009.06 (Apr 19 2011 - 15:57:06)
CPU:   8541, Version: 1.1, (0x80720011)
Core:  E500, Version: 2.0, (0x80200020)
Clock Configuration:
       CPU0:825  MHz, 
       CCB:330  MHz,
       DDR:165  MHz (330 MT/s data rate), LBC:41.250 MHz
CPM:   330 MHz
L1:    D-cache 32 kB enabled
       I-cache 32 kB enabled
I2C:   ready
DRAM:  Initializing
Warning: some of the installed DIMMs can not operate this slowly.
    DDR: 512 MB
FLASH: 32 MB
L2:    256 KB enabled
Service port MAC address: 54:AB:3A:42:0B:41
In:    serial
Out:   serial
Err:   serial
Net:   TSEC0: PHY is Broadcom BCM5482 (143bcb1)
TSEC1: PHY is Broadcom BCM5482 (143bcb1)
TSEC0, TSEC1
Hit any key to stop autoboot:  0 
### JFFS2 loading 'image1' to 0x8000000
Scanning JFFS2 FS: . done.
### JFFS2 load complete: 9026139 bytes loaded to 0x8000000
## Booting kernel from Legacy Image at 08000074 ...
   Image Name:   FASTPATH System for lb4m
   Image Type:   PowerPC Linux Multi-File Image (gzip compressed)
   Data Size:    9025959 Bytes =  8.6 MB
   Load Address: 00000000
   Entry Point:  00000000
   Contents:
      Image 0: 1167342 Bytes =  1.1 MB
      Image 1: 1432683 Bytes =  1.4 MB
      Image 2: 474 Bytes =  0.5 kB
      Image 3: 6425435 Bytes =  6.1 MB
   Verifying Checksum ... OK
## Loading init Ramdisk from multi component Legacy Image at 08000074 ...
   Uncompressing Multi-File Image ... OK
   Loading Ramdisk to 1fa09000, end 1fb66c6b ... OK
   Loading Device Tree to 007fc000, end 007ffb63 ... OK
Model Name LB6M
Using Quanta LB Series machine description
Memory CAM mapping: CAM0=256Mb, CAM1=256Mb, CAM2=0Mb residual: 0Mb
Linux version 2.6.21.7 (gmirek@lc-rdub-26) (gcc version 4.1.2 (Wind River Linux Sourcery G++ 4.1-84)) #2 PREEMPT Thu Mar 31 09:38:27 EDT 2011
Found initrd at 0xdfa09000:0xdfb66c6b
Found MPC85xx PCI host bridge at 0x00000000e0008000. Firmware bus number: 0->0
Zone PFN ranges:
  DMA             0 ->   131072
  Normal     131072 ->   131072
early_node_map[1] active PFN ranges
    0:        0 ->   131072
Built 1 zonelists.  Total pages: 130048
Kernel command line: root=/dev/ram rw mtdparts=physmap-flash.0:32256k(jffs2),512k(u-boot),128k@32512k(u-boot-env)
mpic: Setting up MPIC " OpenPIC  " version 1.2 at e0040000, max 1 CPUs
mpic: ISU size: 4, shift: 2, mask: 3
mpic: Initializing for 60 sources
PID hash table entries: 2048 (order: 11, 8192 bytes)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 515328k/524288k available (2336k kernel code, 8468k reserved, 76k data, 86k bss, 132k init)
Mount-cache hash table entries: 512
NET: Registered protocol family 16
PCI: Probing PCI hardware
PCI: Cannot allocate resource region 0 of device 0000:00:12.0
Generic PHY: Registered new driver
NET: Registered protocol family 2
IP route cache hash table entries: 16384 (order: 4, 65536 bytes)
TCP established hash table entries: 65536 (order: 7, 524288 bytes)
TCP bind hash table entries: 65536 (order: 6, 262144 bytes)
TCP: Hash tables configured (established 65536 bind 65536)
TCP reno registered
checking if image is initramfs... it is
Freeing initrd memory: 1399k freed
JFFS2 version 2.2. (NAND) (C) 2001-2006 Red Hat, Inc.
io scheduler noop registered
io scheduler deadline registered (default)
Generic RTC Driver v1.07
Serial: 8250/16550 driver $Revision: 1.90 $ 2 ports, IRQ sharing enabled
serial8250.0: ttyS0 at MMIO map 0xe0004500 mem 0xe1000500 (irq = 26) is a 16550A
serial8250.0: ttyS1 at MMIO map 0xe0004600 mem 0xe1002600 (irq = 26) is a 16550A
RAMDISK driver initialized: 2 RAM disks of 32768K size 1024 blocksize
Gianfar MII Bus: probed
eth0: Gianfar Ethernet Controller Version 1.2, 54:ab:3a:42:0b:41 
eth0: Running with NAPI enabled
eth0: 256/256 RX/TX BD ring size
eth1: Gianfar Ethernet Controller Version 1.2, 00:e0:0c:00:01:fd 
eth1: Running with NAPI enabled
eth1: 256/256 RX/TX BD ring size
Broadcom BCM5411: Registered new driver
Broadcom BCM5421: Registered new driver
Broadcom BCM5461: Registered new driver
Broadcom BCM5464: Registered new driver
Broadcom BCM5481: Registered new driver
Broadcom BCM5482: Registered new driver
Broadcom BCM50610: Registered new driver
Broadcom BCM57780: Registered new driver
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
i2c /dev entries driver
physmap platform flash device: 02000000 at fe000000
physmap-flash.0: Found 1 x16 devices at 0x0 in 16-bit bank
 Amd/Fujitsu Extended Query Table at 0x0040
physmap-flash.0: CFI does not contain boot bank location. Assuming top.
number of CFI chips: 1
cfi_cmdset_0002: Disabling erase-suspend-program due to code brokenness.
3 cmdlinepart partitions found on MTD device physmap-flash.0
Creating 3 MTD partitions on "physmap-flash.0":
0x00000000-0x01f80000 : "jffs2"
0x01f80000-0x02000000 : "u-boot"
0x01fc0000-0x01fe0000 : "u-boot-env"
nf_conntrack version 0.5.0 (4096 buckets, 32768 max)
ip_tables: (C) 2000-2006 Netfilter Core Team
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
IPv6 over IPv4 tunneling driver
sit0: Disabled Privacy Extensions
NET: Registered protocol family 17
drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
Freeing unused kernel memory: 132k init
Starting pid 618, console /dev/ttyS0: '/etc/rc.d/rc.fastpath'
Mounting /dev/mtdblock0 at /mnt/fastpath.
Mounting tmpfs at /mnt/application...done.
FASTPATH Startup Rev: 6.2

 

[fohdeesha]

Perfect! thanks. Stock uboot envs were missing quite a lot of those, which would explain why it was such a PITA to get an image flashed back

 

[fohdeesha]

I Found the Issue with duplicate MAC addresses yesterday afternoon. I see that there is already a fix in the works. I have 2 flashed switches that i would like to try this on. They came from Unix+ and they have the MAC on the side of the case as well. If there is testing that i can do i will try and help. If i Could get the command to fix the MAC that would be great. Thanks

reboot into the brocade bootloader (hit b while booting when it says hit b to interrupt boot).
Once you're in the bootloader, just run "set ether-address 68a6.23fb.e3b4" - MAC Address Generator will generate proper random macs if you need them, just make sure to match the format in the example in quotes above.

The original MAC your chassis shipped with should be on a sticker on the side of the case, but it can be set to whatever as long as it's unique. Then just pull power to the switch and re-apply, should come up with a new set of macs as verified by "show int br" once the full OS has loaded

 

[fohdeesha]

So after spending several hours debugging the management port, I made zero progress, and I'm burnt out for now. I'm worried it's just plain old hardware incompatibility, most likely the MII configuration registers the e500 built-in MAC is sending on boot to the management port BCM PHY are not 100% correct, since the brocade code is designed to configure a single port BCM variant, but what's actually there on the LB6M is the two port BCM5482. This would also explain why the diag menu PHY test makes it work, that debug program was designed for an LB6M (even says LB6M TOPOLOGY when starting it), so it probably re-initializes the PHY with the proper MII registers meant for the dual port PHY we have.

What gets me is why it seems to only affect some people, and at different levels. no clue. I've bought a powerPC JTAG unit and in one last brute force effort I'm going to dump the entire actual Brocade TurboIron flash, and copy it directly to the quanta bit for bit, so there's *no* difference, and see if that changes anything. Worst case, those of you that really need your management port to work out of the box, and are having this symptom, you can at least now flash back to quanta.

In lighter news, the fan speed command made way more of a difference than I was expecting. here's a video:

 

[TheBloke]

Thanks for all your efforts, @fohdeesha . I'm perfectly happy with mine at 100mbit, and even if it died completely (which it still shows no sign of doing), I can live with in-band and serial only. Hopefully most others can survive too. But yeah it'd be interesting to get some wider confirmation - perhaps that 'reset' really did change something, though I guess that seems a lot less likely now that you've done the revert to Quanta and back to Brocade and it's still broken.

And yeah the fan speed difference is great, thanks for those details. I did mine last night. Although ultimately I decided not to lock it at speed 1, and instead just changed the fan-threshold levels. So now mine is on speed 1 until 55°, and speed 2 up to 70°. That should keep it at speed 1 for the vast majority of the time, only going to speed 2 if it's getting a bit toasty. Don't think I'll ever reach three (which sounds like jet engines etc!)

My chirpy fan appears to have died completely now, so I suppose at some point I'll investigate replacing that. Although maybe not - two fans at speed 1 is quieter still, and I'm currently holding at 50° with two fans on 1, so I'm probably OK for a while. Might have to revisit that when summer rolls around, but it's OK for now

 

[TheBloke]

By the way, how is it that the debug program we use to reset the management port can still be for an LB6M? I thought that would all be wiped when we do the upgrade. Is there some ROM chip that stores debug functions and can't be changed by the flashing procedure?

 

[fohdeesha]

Nah, it's not left over from anything quanta. It's part of the brocade software - when brocade wrote software for this reference design they clearly re-used some things from the LB6M broadcom references

 

[TheBloke]

Nah, it's not left over from anything quanta. It's part of the brocade software - when brocade wrote software for this reference design they clearly re-used some things from the LB6M broadcom references

Fair enough. Though FYI, I don't see LB6M mentioned anywhere when I run t2 and then any/all of the tests listed? I've run all the tests under t2, and I also tried t1, t3, t4 and t5 which are listed in the ? menu - but none of those others did anything, just went back to the prompt.

The only place I've seen LB6M mentioned is in the corrupted serial number.

Here's a pastebin of me running t2 then Ethernet Test.

 

[fohdeesha]

I believe it only shows up when you initiate it from the diag menu on boot. in bootloader it's "set diag-mode" or similar, then boot sys flash primary and it'll boot into it and you'll see LB6M TOPOLOGY scroll by

 

[TheBloke]

I believe it only shows up when you initiate it from the diag menu on boot. in bootloader it's "set diag-mode" or similar, then boot sys flash primary and it'll boot into it and you'll see LB6M TOPOLOGY scroll by

Ah thanks, yes I see it now - and also says it in the title of diag menu: Diagnostic Test Main Menu v1.0 ( LB6M )

 

[TheBloke]

Just opened mine (which is super quick and easy to do). My MAC is 08:9E:01:93:08:32 - a different prefix than the one I saw in the photos from an earlier poster, but also registered to Quanta.

..and then I realised that actually it was also on a sticker on the side of the case I looked at that earlier, but because it was a different prefix to the one I looked up, I thought it was something different. Oh well

 

[TheBloke]

Right I've set my MAC with set ether-address no problem.

I also changed my serial number and a few other details in the EEPROM, as I saw there was a way to do so in the diagnostic menus that set diag_mode provides.

You can also set the MAC address here at the same time, so you can set everything in one go without also needing ether-address.

Not sure if this was the method fohd was going to give for serial etc? But this worked for me:

1. Connect via serial
2. At the console prompt, hit Control-Y, release it, then press a single m followed by Enter.
3. You'll drop to an OS> prompt; type: set diag_mode
4. Type exit or press Control-Z to leave the OS> prompt, then reboot with reload or a power cycle
   1. Another way to achieve steps 1 - 4 is by:
      1. rebooting
      2. hitting 'b' to drop to bootloader
      3. typing set diag_mode, followed by boot sys flash primary
5. When it gets to the OS you won't get a normal prompt, but rather a special test/debug menu
6. Hit 2 for 'Board Information'
7. Press 2 again to show your current info
8. Then press 1 to change it
9. Data formats are:
   1. MAC address is formatted the same as for ether-address, dots between each 4 bytes, eg: 089E.0193.0832
   2. The part number has to be exactly 13 chars long, and the serial 11.
   3. Date formats are MMDDYY
10. When you're done, press Escape to go back to the top menu, then choose 4 (Manufacturing Test Mode Set)
11. Press 1 (Change to Normal Mode)
12. Hit Escape to go back to the top menu, then 7 to reset into the normal OS.

Here's what I set my info to:

        Manufacture EEPROM Information
  MAC address             : 089E.0193.0832
  Part Number             : LB6MSWITCH10G
  Version Number          : 01
  Serial Number           : 31LB6MB0070
  Bench Test Result       : Passed
  Burnin Test Result      : Passed
  Test Date               : 01/18/18
  Date of Mfg             : 01/01/12
  Date of Received        : 04/01/17
  Date of Burnin          : 01/18/18
  Mfg Test                : B6M
  RMA Date                : 36/12/25
  RMA Info                : BZZ0STK

Now show ver in the OS shows my correct serial number: Serial #: 31LB6MB0070

Not that I really care in a home setup, but may as well get everything neat and tidy

 

[BeardedKiltGuy]

reboot into the brocade bootloader (hit b while booting when it says hit b to interrupt boot).
Once you're in the bootloader, just run "set ether-address 68a6.23fb.e3b4" - MAC Address Generator will generate proper random macs if you need them, just make sure to match the format in the example in quotes above. Then just pull power to the switch and re-apply, should come up with a new set of macs as verified by "show int br" once the full OS has loaded

Hey ran that command this morning and it worked like a charm. no more collisions and the correct MAC is propagating to all of the interfaces.

 

[mixmansc]

Sweet! I need to do this to mine and set the correct mac and serial number.

 

[fohdeesha]

@TheBloke yup that's the method. Be aware that in the t2 test menu, the I2C/EEPROM test overwrites the EEPROM with garbage data, so everything you set with the diag menu will be lost, and you'll need to run it all over again. For obvious reasons "run all tests" will do the same, as it runs the i2c test. The ethernet test is still OK to run though, doesn't touch eeprom.

@BeardedKiltGuy @mixmansc can you both do me a huge favor and (assuming you're flashed to brocade), pull power to the unit, and re-apply to do a cold boot, and see if your management 1 port comes up properly at 1gbe and can pass traffic (use show int management 1 to show link speed). If it does come up properly on a cold boot, can you tell me what it's plugged into (type of switch on the other end) and roughly how long of cable? If anyone else running a flashed LB6M can do this as well, it would be a massive help - more data points the better. Need to figure out if TheBloke and I are taking crazy pills, or if this does indeed affect everyone and nobody has noticed

After the 12 total hours spent debugging that stupid port, from checking and writing PCI registers to completely removing the CPLD daughterboard to make sure it wasn't conflicting, I now feel like I know more about this stupid switch than the original engineers ever intended. I do have to thank them though for sneaking in all of these low level commands, this whole flashing business wouldn't be possible without them. I still find it hilarious that the debug menu for official Brocade TurboIrons says "LB6M Diag" in the title

 

[mixmansc]

Ok..... What I did was followed the original cross flash method (as you posted on brokeaid) exactly. Since I'm not putting this switch into use for a few more weeks after that I shut it down and it sat on a shelf since.

Connection is to an older Extreme Networks Summit 400-48T switch. Which is through about a 15 foot patch cable> to a cat 6 keystone jack > through probably 50 feet of cat 6 > to a cat 6 patch panel keystone > through a 6 foot cat 6 cable to the switch.

Just now brought it back up from a total cold boot and management 1 came up as 1 gig. But.......... it came up at half duplex. So just to be sure I just unplugged another computer on the network while watching the log on the Summit switch to be sure there is no cable issue or anything. The computer on that port was at 1gb FD. Using the same cable, just unplugged from that computer and plugged it into the LB6M it came up at 1gb HD. Have not configured any of the other RJ45 jacks but went ahead and plugged onto a couple of the others while watching the Summit log and each connects at 1gb half duplex. Management 2, 25, 27, did not matter.

Going to check some more and will update in a few.

 

[fohdeesha]

Oh god, wah? So even the copper switch ports came up at half duplex? I definitely haven't had or seen that issue anywhere, that's handled by a totally different PHY. Mine definitely come up at full duplex 1gbe, just the management port that's wonky. I feel like I'm on drugs

edit: WAIT, management 2 came up as half duplex? This sounds like your summit switch is bugged out or something, because that port doesn't come up *at all*. Like, the brocade OS doesn't even talk to that port because it shouldn't exist. Is there another switch you can plug it into?

 

[mixmansc]

LOL.... okay. At least its gigabit... albeit, at half duplex. Just dropped to OS and ran the ethernet test. This output that follows is from the Summit switch its connected to... Order is bottom up...

01/19/2018 08:14:10.65 Port 30 link active 1000Mbs FULL duplex
01/19/2018 08:14:08.02 Port 30 link down
01/19/2018 08:14:06.52 Port 30 link active 10Mbs HALF duplex
01/19/2018 08:14:04.16 Port 30 link down
01/19/2018 08:14:00.75 Port 30 link active 100Mbs HALF duplex
01/19/2018 08:13:48.58 Port 30 link down
01/19/2018 08:04:08.20 Port 30 link active 1000Mbs HALF duplex

So...... after running the test at the OS prompt it came back up at FD. That is the only time it has connected at FD that I can see. So, lets pull the cable and reconnect it. First to the mgmt 1 port again, then to mgmt 2, then to 25, then back to mgmt 1.

01/19/2018 08:18:11.61 Port 30 link active 1000Mbs FULL duplex
01/19/2018 08:18:05.49 Port 30 link down
01/19/2018 08:17:51.45 Port 30 link active 1000Mbs FULL duplex
01/19/2018 08:17:48.60 Port 30 link down
01/19/2018 08:17:40.74 Port 30 link active 1000Mbs HALF duplex
01/19/2018 08:17:35.49 Port 30 link down
01/19/2018 08:17:25.87 Port 30 link active 1000Mbs FULL duplex
01/19/2018 08:17:18.01 Port 30 link down

So.... that is also odd, mgmt 2 only comes up at hd. Just to be sure I just tried plugging and replugging on the other copper ports as well as mgmt 2 and mgmt 2 always comes up at half but everything else comes up at full. Another quirk, when I unplug mgmt 1 the top right led for that port lights up and stays lit up until I plug back into to that port.

So running the ethernet test from the OS prompt has the ports, except mgmt 2 now coming up at 1gb full duplex. Going to power cycle and verify if all of this repeats after a true cold boot.

 

[TheBloke]

I just plugged a cable into mgmt #2 and cold booted and it came up at 100mbit. Of course I can't see it in the OS or anything.

I'm about to get lunch but will test more on all copper ports a little later. I'm almost certain I've never seen a port at HD but will double check. My experience is 100mbit FD after cold boot, then 1GB FD after Ethernet Test.