Beware of EMC switches sold as Mellanox SX6XXX on eBay

[nbritton]

did i See the mellanox Secret in the command?
I hope Patrick/sth doesn't get a dmca takedown notification from Nvidia for that...

I see nothing...

[admin@switch-e1c06c ~]# strings /opt/tms/bin/genlicense | grep -i copyright
[admin@switch-e1c06c ~]#

Nvidia needs to learn how to share and stop punching down, I think Linus Torvalds said it best. I can just as easily implement this new shared memory instruction set architecture standard using any other vendor's equipment, Intel and AMD both would really love to get a stronger foothold in the HPC / Infiniband space. Mellanox EOL the SX platform long before the Nvidia acquisition, they have no revenue streams from this to my knowledge, so I see no harm there and making it more accessible to the technical community only benefits them in the long run.

My end game for these switches is for them to run an entirely open source operating system distribution, as for all intents and purposes everything we need is already include in the Mellanox OFED open source code repositories. I won't have any need for the MLNX-OS CLI or web GUI down the road, as the goal is to reimplement the software stack into a SDN controller and communicate with it using open source APIs and protocols. Conceptually I want to take VyOS, OVS DVR, ROCE, SR-IOV, ASAP2 and bolt them all together with a REST API for SDN management.

Ephesians 5:8-17
Walk as children of light (for the fruit of light is found in all that is good and right and true), and try to discern what is pleasing to the Lord. Take no part in the unfruitful works of darkness, but instead expose them. For it is shameful even to speak of the things that they do in secret. But when anything is exposed by the light, it becomes visible, for anything that becomes visible is light. Therefore it says, "Awake, O sleeper, and arise from the dead, and Christ will shine on you". Look carefully then how you walk, not as unwise but as wise, making the best use of the time, because the days are evil. Therefore do not be foolish, but understand what the will of the Lord is.

 

[Stephan]

"Nvidia needs to learn how to share"

Really not, they're a for-profit corporation. Just because they haven't turned full Nintendo or Oracle yet in the networking department they acquired by buying Mellanox, doesn't mean that they never will. I wager a guess some day they will, just google "nvidia virtio code 43". They fought people who paid them money for their graphics hardware for years from within their closed-source driver, if it detected it running within virtualization.

Anyhow. Good luck in your endeavours.

 

[neggles]

did i See the mellanox Secret in the command?
I hope Patrick/sth doesn't get a dmca takedown notification from Nvidia for that...

nVidia could not give a shit at this point, that secret only does something of any use on switches that are well past end of support. Technically Spectrum switches running Onyx also use the same secret, but the only license key you can generate with it is the one which unlocks _shell, and their support will quite happily tell you how to generate it.

Plus if you just run ltrace genlicense it shows up in cleartext in the trace output.

But, fine, edited the post.

just google "nvidia virtio code 43"

They stopped doing that almost two years ago now. The reason they did it in the first place was to try and sell vGPU licenses. They've since realized that J Random Citizen was never going to pay anyway, and nobody who's running vGPU in production is going to be suicidal enough to run without support.

This is something that other companies worked out many years ago - if you're running in production, you buy a license/support contract. If you're not, you were never going to pay - but if you can use it anyway, then you might turn your employer into a paying customer later on...

My end game for these switches is for them to run an entirely open source operating system distribution, as for all intents and purposes everything we need is already include in the Mellanox OFED open source code repositories. I won't have any need for the MLNX-OS CLI or web GUI down the road, as the goal is to reimplement the software stack into a SDN controller and communicate with it using open source APIs and protocols. Conceptually I want to take VyOS, OVS DVR, ROCE, SR-IOV, ASAP2 and bolt them all together with a REST API for SDN management.

I mean... these do support OpenFlow...

 

[nbritton]

nVidia could not give a shit at this point, that secret only does something of any use on switches that are well past end of support. Technically Spectrum switches running Onyx also use the same secret, but the only license key you can generate with it is the one which unlocks _shell, and their support will quite happily tell you how to generate it.

Plus if you just run ltrace genlicense it shows up in cleartext in the trace output.

But, fine, edited the post.



They stopped doing that almost two years ago now. The reason they did it in the first place was to try and sell vGPU licenses. They've since realized that J Random Citizen was never going to pay anyway, and nobody who's running vGPU in production is going to be suicidal enough to run without support.

This is something that other companies worked out many years ago - if you're running in production, you buy a license/support contract. If you're not, you were never going to pay - but if you can use it anyway, then you might turn your employer into a paying customer later on...



I mean... these do support OpenFlow...

As a retired principal computer systems engineer, the crap Nvidia tries to pull with their consumer products isn't going to fly in the hyperscaler space, we (professional open source practitioners) fought very hard to get Microsoft and Oracle out of our product stack because in hyperscale environments we don't even have a way of managing licensing crap at at scale, even RedHat's licensing tools were just to much to deal with so we dropped them entirely and switched to Debian / Ubuntu. The whole reason we stopped Nvidia from purchasing ARM is we knew what they would try to do with it, and we just will not allow another monopoly like Microsoft of the 1990s to happen again. Hard no, at best closed source is a security threat.

As neggles points out, the way we do things in this industry is through support contracts. The systems engineers who are making all of the magic happen don't have time to be nickel and dimed. However, once it is in production, management is very happy to pay for support contracts to ensure that the solutions we engineered continue working.

 

[nbritton]

You'll also find that mainline Linux has a switchdev driver for SwitchX-2, and the SoC used on the COM Express module is just a regular old NXP/FreeScale QoriQ PowerPC that has quite good mainline support as well thanks to OpenWrt.

I'm interested to know more about all this, especially the Mini COM Express SBC details, I was thinking about using OpenWrt as a starting point simply because they still actively target the PPC ISA, but if we can just attach a different SBC onto the Mini COM Express port on the SwitchX-2 ASIC board then the sky is the limit with respect to hot rodding the SX platform. I still can't believe my SX6012F is idling at just 27 watts, for a 56 Gb switch this is just incredible.

I've got the process figured out pretty well now and streamlined too just by using these two files that I dumped from my genuine MSX6012F-2BFS:

fw_SwitchX2-rel-9_4_5110-MT_1270110020.bin
fw_SwitchX2-fru_backplate-MT_1270110020.bin

Step 1: Flash SwitchX-2 ASIC using flint over Infiniband.

[admin@switch-e1c06c ~]# ibswitches
Switch    : 0xec0d9a030062a980 ports 12 "SwitchX -  Mellanox Technologies" base port 0 lid 2 lmc 0
Switch    : 0xf4521403005a7332 ports 13 "MF0;sx6012fx2-e1c06c:SX6012/U1" enhanced port 0 lid 1 lmc 0
[admin@switch-e1c06c ~]# flint -d lid-2 -image fw_SwitchX2-rel-9_4_5110-MT_1270110020.bin --allow_psid_change burn

Step 2: Flash FRU with a dump from a real MSX6012F-2BFS.

mellaggra _write_fru 1 0x51 1000 fw_SwitchX2-fru_backplate-MT_1270110020.bin

Step 3: TFTP netboot the manufacturing image using neggles method and flash 3.6.8012

run mfg_load;if test 0${filesize} -gt 0; then echo Booting mfg ; run mfg_args mfg_common_args;bootm ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r} ; else ; echo Failed mfg load ; fi
manufacture.sh -v -v -t -a -m ppc -k uni -u http://192.168.1.1/image-PPC_M460EX-3.6.8012.img

Step 4: Boot Into the new MLNX-OS installation using EMC U-Boot

MLNX-OS IMAGE 1:
setenv jffs2_args setenv bootargs root=/dev/mtdblock6 rootfstype=jffs2 rw reset_button=0 loglevel=6 loglevel=3
run jffs2_args boot_common_args;bootm ff000000 - ff1e0000

MLNX-OS IMAGE 2:
setenv jffs2_args setenv bootargs root=/dev/mtdblock7 rootfstype=jffs2 rw reset_button=0 loglevel=6 loglevel=3
run jffs2_args boot_common_args;bootm ff200000 - ff3e0000

Step 5: Configure MLNX-OS and reboot into vpi-single-switch profile

switch-e1c06c [standalone: master] > enable
switch-e1c06c [standalone: master] # configure terminal
switch-e1c06c [standalone: master] (config) # license install LK2-RESTRICTED_CMDS_GEN2-88A1-NEWD-BPNB-1
switch-e1c06c [standalone: master] (config) # license install LK2-EFM_SX-5G26-85J2-685K-115L-115M-115N-1A5P-2685-Q145-R125-T115-U118-8A07-UXQW-KH06
switch-e1c06c [standalone: master] (config) # write memory
switch-e1c06c [standalone: master] (config) # system profile vpi-single-switch

Step 6: Instal Mellanox's u-boot rom and bootmgr

I don't know yet, anyone?

So everything just works as expected, I can't even hear the fans because the system automatically idled them down to 8000 rpm after about 10 minutes, no tricks or firmware hacks. The only caveat is when I dumped the FRU backplate data from my genuine MSX6012F-2BFS and then cross flashed it onto the EMC switch it also copied over the serial number as well, but oh well I guess. I did see this error message in the logs but haven't investigated it yet:

Dec 8 04:02:09 switch-156f4e hwd[7053]: TID 1208143072: [hwd.NOTICE]: hwd_detect_fw_mismatches: Validating cache of chip fw mismatch

 

[neggles]

I'm interested to know more about all this, especially the Mini COM Express SBC details, I was thinking about using OpenWrt as a starting point simply because they still actively target the PPC ISA, but if we can just attach a different SBC onto the Mini COM Express port on the SwitchX-2 ASIC board then the sky is the limit with respect to hot rodding the SX platform. I still can't believe my SX6012F is idling at just 27 watts, for a 56 Gb switch this is just incredible.

Pics of the SoM (see attached for full size):


I can't remember which specific subtype it is, but I did work out who the ODM for the SoM was at one point. There aren't very many suppliers of MPC85xx SoMs...

As for actually making use of another SoM, that might be... difficult. They're using a standard connector, but IIRC a somewhat nonstandard pinout. You can see a full architectural diagram with i2c bus hookups, addresses, etc in the Open Compute Project specification and design collateral packages for the MSX1710-OCP, which is the SX6036 switch in OCP form. The SX6012, 6018, and 6036 (and their 10xx series InfiniBand-only brethren) are all the same architecture, just with different sizes of SwitchX-2 and different numbers of ports.

The power usage is pretty nuts, hey? I've gotten mine down to 22W with noctua fans in it (running flat out all the time)

I've got the process figured out pretty well now and streamlined too just by using these two files that I dumped from my genuine MSX6012F-2BFS:

Step 1: Flash SwitchX-2 ASIC using flint over Infiniband.

You can flash the SwitchX-2 from the main operating system over PCIe after firstboot (or from the manufacturing netboot OS image, even) - it doesn't actually need to be functional for MLNX-OS/Onyx to boot

I think there's an MLNX-OS command to flash the switch ASIC firmware, but there's also some info from @mpogr somewhere further up thread which details how to do it. Basically just have to start mst and run flint like usual.

Step 2: Flash FRU with a dump from a real MSX6012F-2BFS.

Here are some bash functions to binpatch a dump of the original backplate EEPROM to a 6012, starting with either an EMC or an SX1012 dump
(this was provided by... someone... earlier up the thread)

2nd file in that gist contains the relevant section of my notes - there are some bootstrap parameters which need changing as well as some instructions around the binpatching in the first bit of those

Step 3: TFTP netboot the manufacturing image using neggles method and flash 3.6.8012

run mfg_load;if test 0${filesize} -gt 0; then echo Booting mfg ; run mfg_args mfg_common_args;bootm ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r} ; else ; echo Failed mfg load ; fi
manufacture.sh -v -v -t -a -m ppc -k uni -u http://192.168.1.1/image-PPC_M460EX-3.6.8012.img

This'd be where you'd want to tell it to skip rebooting at the end, then apply the rest of the manufacturing DB changes listed in my notes

Step 6: Instal Mellanox's u-boot rom and bootmgr

I don't know yet, anyone?

That's weird. `manufacture.sh` should update u-boot automagically for you, but there's also an MLNX-OS command to apply a bootloader update. The part of my notes which deals with removing the u-boot password that firstboot will automatically apply (before rebooting out of the manufacturing OS) is related to that. Are you sure you let it do firstboot completely? it takes a while.

So everything just works as expected, I can't even hear the fans because the system automatically idled them down to 8000 rpm after about 10 minutes, no tricks or firmware hacks. The only caveat is when I dumped the FRU backplate data from my genuine MSX6012F-2BFS and then cross flashed it onto the EMC switch it also copied over the serial number as well, but oh well I guess. I did see this error message in the logs but haven't investigated it yet:

Dec 8 04:02:09 switch-156f4e hwd[7053]: TID 1208143072: [hwd.NOTICE]: hwd_detect_fw_mismatches: Validating cache of chip fw mismatch

That one's a red herring, it's just whining that the in-memory firmware info cache doesn't match. Which is because the cache is empty at the time.

 

[nbritton]

Pics of the SoM (see attached for full size):
View attachment 26022View attachment 26021

I can't remember which specific subtype it is, but I did work out who the ODM for the SoM was at one point. There aren't very many suppliers of MPC85xx SoMs...

As for actually making use of another SoM, that might be... difficult. They're using a standard connector, but IIRC a somewhat nonstandard pinout. You can see a full architectural diagram with i2c bus hookups, addresses, etc in the Open Compute Project specification and design collateral packages for the MSX1710-OCP, which is the SX6036 switch in OCP form. The SX6012, 6018, and 6036 (and their 10xx series InfiniBand-only brethren) are all the same architecture, just with different sizes of SwitchX-2 and different numbers of ports.

The power usage is pretty nuts, hey? I've gotten mine down to 22W with noctua fans in it (running flat out all the time)



You can flash the SwitchX-2 from the main operating system over PCIe after firstboot (or from the manufacturing netboot OS image, even) - it doesn't actually need to be functional for MLNX-OS/Onyx to boot

I think there's an MLNX-OS command to flash the switch ASIC firmware, but there's also some info from @mpogr somewhere further up thread which details how to do it. Basically just have to start mst and run flint like usual.



Here are some bash functions to binpatch a dump of the original backplate EEPROM to a 6012, starting with either an EMC or an SX1012 dump
(this was provided by... someone... earlier up the thread)

2nd file in that gist contains the relevant section of my notes - there are some bootstrap parameters which need changing as well as some instructions around the binpatching in the first bit of those



This'd be where you'd want to tell it to skip rebooting at the end, then apply the rest of the manufacturing DB changes listed in my notes



That's weird. `manufacture.sh` should update u-boot automagically for you, but there's also an MLNX-OS command to apply a bootloader update. The part of my notes which deals with removing the u-boot password that firstboot will automatically apply (before rebooting out of the manufacturing OS) is related to that. Are you sure you let it do firstboot completely? it takes a while.



That one's a red herring, it's just whining that the in-memory firmware info cache doesn't match. Which is because the cache is empty at the time.

So I totally deviated from the stock guide, I didn't run any of those firmware mod scripts and I just left the stock EMC u-boot as is. I found that the manufacturing image is more or less total garbage for getting the system ready for the backplate flash. What works best for me is to flash MLNX-OS version 3.4.1124, after first boot it's the only one that enumerates the switchx i2c stuff to flash the backplate. Any version number higher than this and if you attempt to login it never returns a prompt, but 3.4.1124 does if you give it about 10 minutes. Once it lets you get to a shell then you can run /opt/tms/bin/mellaggra _write_fru 1 0x51 1000 fw_SwitchX2-fru_backplate-MT_1270110020.bin, and then from here you can also install the 3.6.8012 image on to ROOT2 and reboot into that and everything just works.

I didn't need to add or modify anything in the /config/mfg/mfdb, it all seems to be there and everything just works like you would expect on a genuine SX6012F. No hacks at all required, other than flashing the backplate with the image I listed above, I don't think flashing the SwitchX ASIC was required as the first step as I noticed when in MLNX-OS 2.4.1124 that it downgraded the firmware to whatever version come this that release. This is probably the right place to flash the SwitchX ASCI firmware for the purpose of changing the psid, and then when you upgrade to MLNX-OS 3.6.8012 it will automatically update the SwitchX firmware to 9.4.5110 now that the psid matches.

I think because I just left the EMC u-boot install the Mellanox scripts didn't know what to do when checking version numbers et. al and just left it entirely untouched on both EMC switches I did this with. I see the u-boot rom in /opt/tms/lib/u-boot/m460ex/u-boot.bin, any know know the steps to manually flash it?

Thanks for the info about MSX1710-OCP, that should come in handy. I'm really starting to hate the software stack that we have access to for these switch, like I understand now why Mellanox abandoned this still perfectly functional hardware all because the entire software build stack for manufacturing, testing and functional QA is just garbage.

What I would really love to know is if their SX Director switches, the SX6506, SX6512, SX5618, SX5636 can also run in VPI and/or ETH fabric mode, I've never seen support for this officially listed anywhere but since they run the same MLNX-OS SX instance and are also SwitchX-2 ASCIs I've always wondered if they could secretly do this.

I pulled out a spare SX6036 to use as my self hosted development machine as it has a USB Type A connector. All I had to do was attach a SATA SSD to the system via a USB Attached SCSI Protocol adaptor.

mkfs.ext2 /dev/sda
mkdir /var/root/usbroot
mount /dev/sda /var/root/usbroot/

rsync -a --exclude /var/root/usbroot --exclude /dev --exclude /proc --exclude /sys / /var/root/usbroot/;
debootstrap xenial /var/root/usbroot/;
mkdir /var/root/usbroot/{dev,proc,sys};

mount --rbind /dev  /var/root/usbroot/dev;
mount --rbind /proc /var/root/usbroot/proc;
mount --rbind /sys  /var/root/usbroot/sys;
chroot /var/root/usbroot bash --login;
mount -o remount,rw /

# Cross Architecture Debootstrap Example
# debootstrap --foreign --arch powerpc xenial /mnt/usbroot
# debootstrap --second-stage;

[admin@sx6036fx1-628464 root]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda  276G  1.6G  260G   1% /
none           1014M  8.0K 1014M   1% /dev
tmpfs          1014M  6.6M 1008M   1% /dev/shm

cat /etc/apt/sources.list
deb [arch=powerpc] http://ports.ubuntu.com/ubuntu-ports xenial main restricted universe multiverse
deb-src [arch=powerpc] http://ports.ubuntu.com/ubuntu-ports xenial main restricted universe multiverse

deb [arch=all] http://ports.ubuntu.com/ubuntu-ports xenial-updates main restricted universe multiverse
deb-src [arch=all] http://ports.ubuntu.com/ubuntu-ports xenial-updates main restricted universe multiverse

deb [arch=all] http://ports.ubuntu.com/ubuntu-ports xenial-security main restricted universe multiverse
deb-src [arch=all] http://ports.ubuntu.com/ubuntu-ports xenial-security main restricted universe multiverse

deb [arch=all] http://ports.ubuntu.com/ubuntu-ports xenial-backports main restricted universe multiverse
deb-src [arch=all] http://ports.ubuntu.com/ubuntu-ports xenial-backports main restricted universe multiverse

deb [arch=all] http://ports.ubuntu.com/ubuntu-ports xenial-proposed main restricted universe multiverse
deb-src [arch=all] http://ports.ubuntu.com/ubuntu-ports xenial-proposed main restricted universe multiverse

 

[nbritton]

I thought I remembered someone had the u-boot source code with the patch additions for Mellanox AMCC M460EX already included?

Also please tell me, what are the functional differences if any between the sam460EX M460EX, 440EPx, and e500mc with respect to the u-boot loader. I believe the sam460EX u-boot include in the Ubuntu repo is a virtual qemu test target for that same original sam460ex reference board, but these seem to be roughly the same as the M460EX SoC, there is also a PPC 440EPx in the Dell m1000e that I have in mind for another project. Can I use the regular stock ubuntu powerpc-smp kernel, what about the powerpc-e500mc one? How does the Freescale e500mc relate in terms of the AMCC 460EX?

My main goal right now is to update u-boot to at least support GPT USB Flash Drive with Ext4 root filesystem, then install the PPC port of Ubuntu 16.04 on it via debootstrap and chroot, then get u-boot to boot directly from the USB flash that has Ubuntu on it, bypassing the need to rely on the SBC's NAND flash entirely. The 460EX flash on the SBC will be left as a stock install of MLNX-OS 3.6.8012, my wish is to use u-boot to directly boot up the ubuntu kernel using u-boot.

Also what is yboot, u-boot grub, u-boot-tools, mkvmlinuz? I'm still a little in the dark about how the boot process works on the PPC, and given there is no easy way to unbrick the bootloader I just want to make sure I have a reasonable exception that the updated u-boot rom image that I compiled has a reasonable expectation of working when I flash it to the real hardware. I should be able to QA test this custom u-boot rom from inside of a QEMU sam460ex virtual machine right?

Documentation/Platforms/PowerPC - QEMU
Amiga like OSes on QEMU
GitHub - qemu/u-boot-sam460ex

http://ports.ubuntu.com/ubuntu-ports/pool/main/u/u-boot/u-boot_2013.10-3_powerpc.deb
http://ports.ubuntu.com/ubuntu-ports/pool/main/u/u-boot/u-boot-tools_2013.10-3_powerpc.deb
http://ports.ubuntu.com/ubuntu-port...boot-tools_2016.01+dfsg1-2ubuntu5_powerpc.deb

 

[seanneko]

I saw earlier in the thread that someone was unable to get BiDi SFPs working in a SX6012. I may have missed something in this 63 page thread, but was a solution ever found?

I want to use Cisco QSFP-40G-SR-BD SFPs to have 40Gb links to servers. Is this possible with a SX60xx?

 

[RedX1]

I saw earlier in the thread that someone was unable to get BiDi SFPs working in a SX6012. I may have missed something in this 63 page thread, but was a solution ever found?

I want to use Cisco QSFP-40G-SR-BD SFPs to have 40Gb links to servers. Is this possible with a SX60xx?

Hello

I have no experience of Cisco LR modules, but the code below works for the Arista QSFP-40G-UNIV Transceiver and some other LR Modules.


Code:

configure terminal

fae cable-stamping-unlock 40g_lr4

exit

write memory




More info here.

Mellanox Interconnect Community


I hope this helps.


RedX1

 

[seanneko]

Hello

I have no experience of Cisco LR modules, but the code below works for the Arista QSFP-40G-UNIV Transceiver and some other LR Modules.


Code:

configure terminal

fae cable-stamping-unlock 40g_lr4

exit

write memory




More info here.

Mellanox Interconnect Community


I hope this helps.


RedX1

Thanks, unfortunately I already have a box of these Cisco SFPs though so they're the only ones I would be using. If I need to buy new SFPs then this isn't the right switch for me.

Does anyone happen to have a Cisco SR BiDi SFP and can try the above commands?

 

[nbritton]

Still waiting on advice from someone with first hand experience flashing u-boot to a newer version. I have already ported Debian Bullseye, Linux kernel version 6, and Mellanox OFED all recompiled for ppc32be, I have a fully operational developer build environment that I can chroot into. The trouble is I can't directly boot the kernel installed on the USB flash device as the USB mass storage implantation for u-boot 2009 is known defective, it doesn't work at all. I can figure out how to flash u-boot on my own through trial and error, but if I accidentally brick my development systems then I can't validate and publish the new system distribution...

 

[vishone]

Hello everyone
I also need to convert SX6012 now, who can give me conversion files and tutorials?

 

[cy384]

so I've managed to get my EMC sx6012 into a state where the uboot is password-locked AND mlnx-os errors out when I try to log in, leaving me with no way to do anything.

any tricks left, or is it totally bricked? Not afraid to mess around at the hardware level, but also don't want to start pulling off random eeproms and seeing what's on them.

 

[Stephan]

@cy384 Define "errors out when I try to log in". Lost password? Without U-Boot password (written in "CPU" EEPROM) you can only boot system 1 or 2 and do nothing more. Are both partitions defective? How did you manage that? If one system still starts somewhat, press RST button in front for 15 seconds to reset password to empty. See if you can login then. If both partitions are beyond repair, fastest way is to flash the EMC U-Boot which doesn't have a password: U-Boot 2009.01 SX_PPC_M460EX SX_3.2.0330-82-EMC ppc (Feb 27 2013 - 12:13:42) Then re-manufacture the switch. You will need a BDI 2000 and correct software on it. What country are you in?

 

[cy384]

@cy384 Define "errors out when I try to log in"...You will need a BDI 2000 and correct software on it. What country are you in?

hey, thanks for the quick response! I'm in the US.

basically what I did was:

• flash the switchx2 over infiniband, similarly to what nbritton is doing
• run manufacturing including the u-boot upgrade flag
• immediately reboot without clearing the u-boot password or flashing the FRU

now if I try to log in as admin (no password) on the serial console, it says initializing modules etc., and after ~15 minutes I get a message like "fatal internal error" which dumps me back at the login (same on both partitions)

I see some others have had this error, but unfortunately not at the same time as a locked u-boot!

p.s. I've designed a 3d-printable replacement top panel that holds 120mm fans, a few more tweaks and I'll upload the files

 

[Stephan]

@cy384 If you just try again to login, no dice? Like every 15 minutes. Looks like the switch wants to load the driver for wrong software running on the ASIC, and fails. Tried Ctrl-C like 10 times? Also during boot? Hit Ctrl-C like a mad man, maybe one of the scripts will fail and bootup ends you in an emergency shell.

If you can't get back to a shell, you need @fohdeesha 's help to Fedex you the BDI 2000, its power supply, a straight 16 pin JTAG cable and serial cable. Switch can be rescued. I have the EMC U-Boot (384 KiB) dump if you need it. I also fixed up the BDI config utility so it compiles cleanly on modern Linuxes.

If you want to help others later yourself, here is the best option right now: USED ABATRON BDI2000/C DEBUG INTERFACE GDB/WIND | eBay Bought mine from same seller. Even though I'm over in Europe. Ribbon cable displayed is wrong for our purposes, you need a female/female IDC 16-pin straight, no longer than 20 centimeters.

 

[cy384]

thanks for the suggestions, I'll sleep on it and see what looks like a good direction to go.

I'm wondering now, of the two firmwares (the backplate and the configuration(?)), are they on the CPU board or the switch board? Since I also have a working SX1012, I wonder if I could swap the CPU boards between them and reset something that way?

 

[BeTeP]

don't want to start pulling off random eeproms and seeing what's on them.

if you got a SOIC8 clip it should take you less than 10 minutes - to pull the dump from the FRU eeprom, blank the password hash and write it back. no need to wait for the BDI

 

[BeTeP]

are they on the CPU board or the switch board

it's a CPU FRU and it's on the CPU board