Flashing Delta 7024 to Dell 8024

Discussion in 'Networking' started by fohdeesha, May 17, 2018.

  1. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    932
    Likes Received:
    679
    Finally found someone willing to test the method I came up with, and it works great (nearly the same process as the LB6M flash, with some differences due to a larger bootloader from dell etc)

    Unlike the LB6M > Brocade flash where there were some slight hardware differences that caused things like the SFP+ lights to stop working, the Dell and Delta are absolutely identical. This means there's no changed behavior, all the lights work, etc.

    Unlike brocade, Dell does not ship the bootloader in a separate file, so the most difficult part was hunting it down in the 30MB stk image and ensuring the trimmed bootloader code had the correct geometry

    The guide is available here: (7024) Delta To Dell - Brokeaids

    For people who previously bought already-converted Delta 7024's off ebay (eg they came preloaded with dell firmware) - I personally confirmed you can run the cpld update command to flash the CPLD with the proper dell code. This will fix temp/fan status reporting, and should theoretically allow the fans to spin down a little quieter. Just update to the latest dell firmware, then update the bootcode from the boot menu. then when it's all booted and updated, just run "dev cpldUpdate" at the enable level. Check the CPLD section in the guide for the output you should expect. You'll need to unplug the switch fully for a few seconds and repower it to reset the cpld and get it running the newly flashed code
     
    #1
    Last edited: May 18, 2018
    mangodoc likes this.
  2. i6feetunder

    i6feetunder New Member

    Joined:
    Sep 20, 2017
    Messages:
    4
    Likes Received:
    0
    #wow, very impressive and nice documentation.
     
    #2
  3. javooooo

    javooooo New Member

    Joined:
    Jun 15, 2018
    Messages:
    11
    Likes Received:
    1
    Thanks for the tutorial @fohdeesha!

    Everything went well for me, up until the CPLD update. I get this error:

    Code:
    console#dev cpldUpdate
    
    Unrecognized device
    CPLD update exited with a return code of 0
    error_address 180595718, exit_code 6, format_version 2
    value = -1 = 0xffffffff
    Has anyone had this issue before? I've tried cold booting the switch, reinstalling the update again via TFTP from the Dell firmware. The rest of the process completed without any issues and it seems to be running the correct Dell bootloader and firmware.
     
    #3
  4. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    932
    Likes Received:
    679
    Hello! This actually seems to be a dell issue - if you search around, you'll find many legitimate Dell 8024 owners with this issue, and no fix from dell. (I'm guessing you know this, as I see now you've bumped the dell thread). First time I've personally seen this issue through, after 4 or 5 Delta crossflashes.

    If I had to take a wild guess, their CPLD flashing code is lazy, and when it sees a very old Altera CPLD revision it just gives up. It doesn't seem to be a huge issue, and all your CPLD related functions seems to be fine (it's reporting all the fans and psu's etc correctly)

    if you really want to mess with it, you can try flashing a really old FW image, which probably has a different updater that is maybe more forgiving of an altera device that out of date. That might be able to get it bumped up enough for the latest FW to bump it to the latest. (now that you have the proper Dell OS flashed, you can easily load in new firmware images using TFTP)

    like this one from 2014 for example: PowerConnect 8024/8024F, Firmware Version 5.1.3.7,A19 | Dell Canada

    (for all the old versions to try, go here - Support for PowerConnect 8024F | Drivers & downloads | Dell US then click the dropdown arrow by the download name, then click "older versions")

    Something else I've seen that might help is unplug it, plug it in to do a cold boot, wait at least 10 minutes for it to finish initializing vxworks, THEN try the update command. If you had any links or cables connected when you tried to update, UNPLUG them. Also only run the command over a serial connection
     
    #4
  5. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    932
    Likes Received:
    679
    In the secret boot menu mentioned in the guide, there's also a CPLD specific option that might have some low level commands or options to force a flash to the CPLD, but I don't have one to look at or test. If you go into the secret boot menu, hit the option for CPLD diagnostics, take a screenshot of all the options that come up and post it here

    there's also a raw VXworks shell in that menu that could also be promising, but once again I'd have to look at what all is available
     
    #5
  6. javooooo

    javooooo New Member

    Joined:
    Jun 15, 2018
    Messages:
    11
    Likes Received:
    1
    Thanks for the quick response! Yeah, it's an odd issue.

    Option 9 in the hidden menu just spits similar output to "dev cpldTest" in the OS:

    Code:
    Select option: 9
    Board Type                : Campbell 24F
    Board Version             : 2
    CPLD Revision             : 3 (0x3)
    RPS2 Present
    RPS2 Status               : Failed
    RPS2 Temperature          : Normal
    RPS2 Fan Status           : Working
    RPS1 Present
    RPS1 Status               : Working
    RPS1 Temperature          : Normal
    RPS1 Fan Status           : Working
    Port   SFP Status  Link Status  Tx Power
    ----   ----------  -----------  --------
       1  Not Present         Down  Disabled
       2  Not Present         Down  Disabled
       3  Not Present         Down  Disabled
       4  Not Present         Down  Disabled
       5  Not Present         Down  Disabled
       6  Not Present         Down  Disabled
       7  Not Present         Down  Disabled
       8  Not Present         Down  Disabled
       9  Not Present         Down  Disabled
      10  Not Present         Down  Disabled
      11  Not Present         Down  Disabled
      12  Not Present         Down  Disabled
      13  Not Present         Down  Disabled
      14  Not Present         Down  Disabled
      15  Not Present         Down  Disabled
      16  Not Present         Down  Disabled
      17  Not Present         Down  Disabled
      18  Not Present         Down  Disabled
      19  Not Present         Down  Disabled
      20  Not Present         Down  Disabled
      21  Not Present         Down  Disabled
      22  Not Present         Down  Disabled
      23  Not Present         Down  Disabled
      24  Not Present         Down  Disabled
    Fan1 Status               : Working
    Fan2 Status               : Working
    Fan3 Status               : Working
    As for the VXWorks option, I've attached a screenshot.

    I also tried the devshell option to execute the cpldUpdate command directly but that got a weird error too.

    Code:
    Boot code utilities menu
    Options are:
    1  - Read/Write memory
    2  - Display PCI bus
    3  - Display PCI bus details
    4  - Display core info and bus speeds
    5  - Display file system details
    6  - RAM test
    7  - File system directory listing
    8  - Generate and store VPD info
    9  - CPLD diagnostics
    10 - Switch diagnostics
    11 - Format file system
    12 - File system test
    13 - Comprehensive test (RAM, PCI, FLASH)
    14 - Start vxWorks shell
    15 - Devshell command interface
    16 - Upload a file through XMODEM
    17 - Download a file through XMODEM
    18 - Run flash diagnostics
    19 - Display Bootcode Build Information
    20 - List Main Menu Options
    0  - Return to main menu
    Select option: 15
    
    devshell> cpldUpdate
    
    executing 'cpldUpdate'
    
    No CPLD available!
    
    value = 19 = 0x13
    I tried an old FW version earlier and it didn't work either, but I'll go back to an even earlier version and try again.
     

    Attached Files:

    #6
  7. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    932
    Likes Received:
    679
    hmm. The only other thing I can think of is the scorched earth method: you can use a USB Blaster connected to the CPLD programming/JTAG header and just use Alteras devtool to dump the proper CPLD image right to the damn thing directly, but you'd need the CPLD image to dump.

    The chances of me succesfully digging that out of the Dell STK image are basically zero (I'm not a CPLD/FPGA guy so I have no idea what I'd even be looking for). So you'd need to obtain the proper image by connecting the USB Blaster tool to a dell 8024 that has had it's CPLD succesfully updated, and dumping it's contents with Alteras tool.

    All that said, I probably wouldn't worry about it unless you start getting odd behavior from your switch (which doesn't seem to be happening).
     
    #7
  8. javooooo

    javooooo New Member

    Joined:
    Jun 15, 2018
    Messages:
    11
    Likes Received:
    1
    Hmm... I've just tried versions from the 5.1 and 5.0 releases again and neither worked. I'm tempted to try 4.2 although I'm concerned it may install an older version than what's already on there.

    I do have another Dell 8024 that updated successfully so I will research the USB Blaster you mentioned further - although all of this is new to me so it sounds a bit too complex. I am just a bit concerned about the comments in the thread on Dell's site about the switches being unstable with the old CPLD version.
     
    #8
  9. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    932
    Likes Received:
    679
    Well since you already have a working cpld on hand, I don't think the USB blaster would be terribly difficult. I can help walk you through it, the altera program is a nice gui with dump and write type buttons, and a USB blaster clone is only about $15.

    As for the reports of odd behavior, I wouldn't give them any credence. I saw reports of links no longer working at 1gbe and odd multicast behavior, but the cpld has nothing to do with any of that. It's just a glorified low speed I/o multiplexer that runs the status lights, monitors fans and psus, etc. Strange cpld behavior would be psus not showing up, temps not being reported, link lights not working (ran into this personally with the quanta to brocade flash) etc

    First step to know if it's possible is take a high res (do not resize) picture of the inside of the delta (just take the top cover off). That'll tell me if it does indeed have a 10pin cpld header.

    If it does, it's theoretically as simple as connect the header on the working switch, run alteras devtool, click dump program and save it as a bin file somewhere.

    Then same process on broken switch, but click write program instead. If it works, I can host that bin file on the site/guide and anyone else with an un-updateable cpld and an extra $15 can fix theirs as well
     
    #9
    Last edited: Jun 16, 2018
  10. javooooo

    javooooo New Member

    Joined:
    Jun 15, 2018
    Messages:
    11
    Likes Received:
    1
    Ah, gotcha. I may wait a bit for my optics to arrive so I can check the link lights - if they work, there's probably no point in updating the CPLD. The fans are spinning down nicely and the ethernet lights work, so fingers crossed.

    The other unit I have is also a Delta that I converted following your guide, but the CPLD update did work on that one. I'll crack it open and take some photos next week once the optics arrive. I'll be converting a few more in the next few weeks so hopefully those have the newer revision chip.

    Cheers.
     
    #10
  11. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    932
    Likes Received:
    679
    So I ended up going down a CPLD hole and reading a lot of Altera whitepapers last night, and I have a rough idea of what's going on.

    The Altera CPLD's have two small flash areas:

    UFM = User Flash Memory - a small 8096 bit area for storing whatever you like, stuff like serials, metadata, vendor tags, CPLD software revision tags, etc. The CPLD itself doesn't actually use anything in here to run, but the host system can access it.

    CFM = Configuration Flash Memory - this is the flash that actually stores the compiled CPLD code that runs the thing.

    What seems to be going on, is the Dell CPLD flash routine first reads the UFM and looks for a known vendor string and version string, so it knows it has the correct device address and isn't about to flash some other part or unrelated CPLD. Some of the older Deltas and Dell chassis seem to have a different vendor and CPLD ident string written in here, and when the Dell update procedure sees them and doesn't recognize them, it quits the update process out of caution.

    Alteras free programming-only software (Quartus Prime Programmer) has nice simple "Read" and "Write" buttons that will read and write both flash areas to and from a file on your PC. So it should only be a 5 minute or less process to connect the USB Blaster to the board, click dump to a file on your desktop. That file will have a UFM with the correct vendor strings and tags that won't freak the dell out. Then just connect it to the "broken" switch and write.

    I ordered one to play with, I didn't realize how cheap the clones are - altera Mini Usb Blaster Cable For CPLD FPGA NIOS JTAG Altera Programmer | eBay

    I don't have any 8024's but I have a lot of switches with the same Altera MAX II CPLD (quanta lb6m, brocade icx6610, etc), so I'll play around with reading and writing.

    If you pick up a dongle and read the flash off the known-correct CPLD and send me the file it'd be a huge help. Can then write up a short little guide that anyone in that broken dell thread can use, just need to ebay or amazon a $7 dongle. I'll paypal you $7 if you need it for your troubles. More than anything I'm just curious what strings they actually have in the UFM
     
    #11
    Last edited: Jun 16, 2018
  12. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    932
    Likes Received:
    679
    So I ended up buying one of the $5 ebay USB Blaster clones off ebay, and downloading the free Quartus Prime programming software. When you plug in the clone dongle, it shows up as as a USB Blaster device with no found driver - so I manually installed the driver, choosing the driver inf files that ship with the Quartus programming software. This ended up causing BSOD's - which turned out to be a known issue with Altera and this dongle, even the genuine ones. They recommend an alternative driver which took me a while to find, but tracked down: http://fohdeesha.com/data/other/blaster-driver.zip

    Once I used that driver, everything else turned out to be incredibly easy - just connect the dongle to the 10 pin header, launch the software, and hit auto detect. It'll scan the JTAG chain and automatically figure out what CPLD model you have and apply the correct settings. Then you can just click "Examine" and dump the contents to a file. if you want to write that file back, you just click Program

    [​IMG]

    [​IMG]
     
    #12
    Last edited: Jul 3, 2018
  13. N1h1l1ty

    N1h1l1ty New Member

    Joined:
    Oct 21, 2018
    Messages:
    25
    Likes Received:
    1
    #13
Similar Threads: Flashing Delta
Forum Title Date
Networking Flashing stock Mellanox firmware to OEM (EMC) ConnectX-3 IB/Ethernet dual-port QSFP adapter Jun 12, 2018
Networking Anyone flashing Mellanox ConnectX-3 in qty? Aug 27, 2017
Networking Delta Fan Part Numbers Mar 18, 2018
Networking Need Delta 7024 owner willing to flash to Dell 8024 Jan 14, 2018
Networking Delta Networks ET-DT7024 24 Port 10GB SFP+ Sep 8, 2017

Share This Page