Fun with an MD1200/MD1220 & SC200/SC220

fohdeesha

Kaini Industries
Nov 20, 2016
2,186
2,097
113
30
fohdeesha.com
I use a lot of Dell MD1200 disk shelves, just got another one in and I needed to know if I need to update the firmware on it (because you can only do so with an H800 or H810 card HBA card, which I would have to dig out). didn't have a spare system with a SAS card to connect to it and poll the firmware version to see if it was outdated, so I got to thinking for a minute:

There's a mini-din "password reset" connector on the controllers, and to me that just screamed debug UART (I mean, the dumb expander MD1200 doesn't even have any functionality to password). Sure enough, after shoving some breadboard jumpers into it, I get a debug shell (RS232, 38400-8-n-1):

Code:
BlueDress.106.000 >?
unknown_cmd -> cmd:? args:1

BlueDress.106.000 >help
unknown_cmd -> cmd:help args:1

BlueDress.106.000 >


Pinout (facing the back of the controller):



However as you can see above, the shell isn't giving up any commands, and nothing I could guess would take. So I downloaded Dell's update package for the shelf, extracted out the firmware binary, and loaded it into Ghidra, then performed a string search until I found some:



those certainly look like commands, and after testing them they indeed were. Better yet, I eventually found the strings "devils" and "_devils", these make the shell spit out all possible commands, which you can find here: BlueDress.106.000 >_devils _boot Download the boot image (FW image 1 a - Pastebin.com

A few interesting things, such as temp reading:

Code:
BlueDress.106.000 >_temp_rd

  BP_1[2] = 20c
  BP_2[3] = 19c
  SIM0[0] = 25c
  SIM1[1] = 25c
  EXP0[4] = 41c
  EXP1[5] = 45c

  AVG = 29c
Arbitrarily control fan speed 0-100:
Code:
BlueDress.106.000 >set_speed 20
##if you go lower than 20%, it'll ramp back up
##if you really want lower than 20, you need to spoof the temp with set_temp
Detailed firmware version (it seems the MD firmware team is using github, or at least a build tool that defaults to owner: github):
Code:
BlueDress.106.000 >_ver
********************* Bench Build **********************
Build Owner   : GitHub
Build Number  : 000
Build Version : 106
Build Date    : Mon Aug 24 12:28:59 2015
FPGA Revision  : A9
Board Revision : 8
CPLD Revision  : 7
**************Firmware Image Information****************
          Image Region 0 (Always Boot)
             Revision         : 0.0.63.0
             Dell Version     : 1.01
             Total Image Size : 0003d89c [252060]
             Fast Boot        : Yes
             Image Address    : 0x14000000
             RegionOffset     : 0x00000000
             RegionSize       : 0x00080000
             RegionType       : 0x00000000
Active--->Image Region 1
             Revision         : 0.0.63.0
             Dell Version     : 1.06
             Total Image Size : 000414fc [267516]
             Fast Boot        : Yes
             Image Address    : 0x14080000
             RegionOffset     : 0x00080000
             RegionSize       : 0x00080000
             RegionType       : 0x00000001
          Image Region 2
             Revision         : 0.0.63.0
             Dell Version     : 1.06
             Total Image Size : 000414fc [267516]
             Fast Boot        : Yes
             Image Address    : 0x14100000
             RegionOffset     : 0x00100000
             RegionSize       : 0x00080000
             RegionType       : 0x00000002
********************************************************
The only way to update the firmware on these shelves is with a dell executable, and it will only do it if it's connected via an H800 or H810 HBA, so if you don't have one of those you were out of luck. However looking through the commands here, you can just drop it new firmware over serial via XMODEM.

I think my favorite find however is this:
Code:
  BlueDress.106.000 >ps_status
************************************************************************************
This command has not been implemented yet.
If you REALLY need this feature cont(r)act the Devil's firmware team for help.
CAUTION: It may not be wise to make cont(r)act with any of the Devil's team members.
************************************************************************************
This should be nearly the same if not identical on all their disk shelves with this connector, eg the md1000, md12000/md1220, md3000, etc. If you wanna dick around with it and don't wanna have to shove connectors into it, you can buy their "password reset" cable: New Fit for Dell Password Reset/Service Cable MN657 MD1200 MD3200 US Shipping | eBay - you could connect it to the serial port on your host, and tell your host to run on boot "_shutup 10" for arbitrary fan control for example
 
Last edited:

frameshift18

New Member
Jan 9, 2019
16
7
3
Awesome find! Works on my SC200's and SC220. Was literally just about to embark on a fan swapping endeavor on these P/S fans.. but 'set_speed' quiets them down, as long as it holds..
 
  • Like
Reactions: fohdeesha

fohdeesha

Kaini Industries
Nov 20, 2016
2,186
2,097
113
30
fohdeesha.com
Awesome find! Works on my SC200's and SC220. Was literally just about to embark on a fan swapping endeavor on these P/S fans.. but 'set_speed' quiets them down, as long as it holds..
I was finding with the set_speed or _shutup commands mine would ramp back up about 10 seconds later, it was the auto fan control profile taking over based on temp, so that's why in my example I used the temp spoof command to make it think it's much colder than it was. can you paste an example of the prompt you get on the SC series? For instance my MD1200 is codename bluedress, I did see references to reddress, I wonder if that's the SC200. There was even a command to switch it from a red dress model to a blue dress model, which makes me wonder if you can flash the SC and MD firmware on these controllers interchangeably. If you could paste the output of "_ver" that'd be awesome
 

frameshift18

New Member
Jan 9, 2019
16
7
3
I have not tried _shutup but set_speed 20 seems to be holding, at least for the last 15 mins or so. The available console commands are slightly different as well.

From one of my SC200's.. SC200 - Pastebin.com

The SC200 is "12Pack", the SC220 is "FullCase"

Below is from an SC200
Code:
12Pack.Slot1.1.03.000 >_ver
********************* Bench Build **********************
Build Owner   : GitHub
Build Number  : 000
Build Version : 1.03
Build Date    : Fri Jul 10 09:40:58 2015
FPGA Revision  : AA
Board Revision : 8
CPLD Revision  : 7
**************Firmware Image Information****************
          Image Region 0 (Always Boot)
             Revision         : 0.9.0.0
             Dell Version     : 1.00
             Total Image Size : 0005928c [365196]
             Fast Boot        : Yes
             Image Address    : 0x14000000
             RegionOffset     : 0x00000000
             RegionSize       : 0x00080000
             RegionType       : 0x00000000
Active--->Image Region 1
             Revision         : 0.9.0.0
             Dell Version     : 1.03
             Total Image Size : 00055a3c [350780]
             Fast Boot        : Yes
             Image Address    : 0x14080000
             RegionOffset     : 0x00080000
             RegionSize       : 0x00080000
             RegionType       : 0x00000001
          Image Region 2
             Revision         : 0.9.0.0
             Dell Version     : 1.01
             Total Image Size : 00059270 [365168]
             Fast Boot        : Yes
             Image Address    : 0x14100000
             RegionOffset     : 0x00100000
             RegionSize       : 0x00080000
             RegionType       : 0x00000002
********************************************************
 
  • Like
Reactions: fohdeesha

fohdeesha

Kaini Industries
Nov 20, 2016
2,186
2,097
113
30
fohdeesha.com
I have not tried _shutup but set_speed 20 seems to be holding, at least for the last 15 mins or so. The available console commands are slightly different as well.

From one of my SC200's.. SC200 - Pastebin.com

The SC200 is "12Pack", the SC220 is "FullCase"

Below is from an SC200
Code:
12Pack.Slot1.1.03.000 >_ver
********************* Bench Build **********************
Build Owner   : GitHub
Build Number  : 000
Build Version : 1.03
Build Date    : Fri Jul 10 09:40:58 2015
FPGA Revision  : AA
Board Revision : 8
CPLD Revision  : 7
**************Firmware Image Information****************
          Image Region 0 (Always Boot)
             Revision         : 0.9.0.0
             Dell Version     : 1.00
             Total Image Size : 0005928c [365196]
             Fast Boot        : Yes
             Image Address    : 0x14000000
             RegionOffset     : 0x00000000
             RegionSize       : 0x00080000
             RegionType       : 0x00000000
Active--->Image Region 1
             Revision         : 0.9.0.0
             Dell Version     : 1.03
             Total Image Size : 00055a3c [350780]
             Fast Boot        : Yes
             Image Address    : 0x14080000
             RegionOffset     : 0x00080000
             RegionSize       : 0x00080000
             RegionType       : 0x00000001
          Image Region 2
             Revision         : 0.9.0.0
             Dell Version     : 1.01
             Total Image Size : 00059270 [365168]
             Fast Boot        : Yes
             Image Address    : 0x14100000
             RegionOffset     : 0x00100000
             RegionSize       : 0x00080000
             RegionType       : 0x00000002
********************************************************
Very interesting, so it's indeed running a totally different firmware train. I also can't seem to find a firmware release for the SC series to download and analyze anywhere
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,186
2,097
113
30
fohdeesha.com
Was able to upgrade the firmware on another MD1200 via serial, so no more having to buy a Dell H810 just because their official updater tool refuses to work with any other HBA. do NOT use teraterm, it bricked one of my controllers. It currently has a bug with xmodem transfers: Ticket #37566: XMODEM not working correctly - Tera Term - OSDN

Ended up using the latest extraputty snapshot: Download in progress...

Just connect to the controller via serial and open extraputty, 38400-8-N-1. Hit enter a few times to be sure you get a prompt. Run the following command below, after doing so you'll have 60 seconds to initiate an xmodem transfer. Do so by going to the top of extraputty and clicking File Transfer > Xmodem > Send and selecting MD12_106.bin from the ZIP link below. Do NOT select "Xmodem 1K". You should get a transfer window that looks like this, takes about 1 minute:



Code:
_boot
#Click file transfer > xmodem > send > select the bin
When the transfer is done it will automatically reboot. Now when you run "_ver" you should see the "Active--->" arrow pointing to version 1.06:

Code:
**************Firmware Image Information****************
Active--->Image Region 0 (Always Boot)
             Revision         : 0.0.63.0
             Dell Version     : 1.06
             Total Image Size : 000414fc [267516]
             Fast Boot        : Yes
             Image Address    : 0x14000000
             RegionOffset     : 0x00000000
             RegionSize       : 0x00080000
             RegionType       : 0x00000000
          Image Region 1
             Revision         : 0.0.63.0
             Dell Version     : 1.06
             Total Image Size : 000414fc [267516]
             Fast Boot        : Yes
             Image Address    : 0x14080000
             RegionOffset     : 0x00080000
             RegionSize       : 0x00080000
             RegionType       : 0x00000001
          Image Region 2
             Revision         : 0.0.63.0
             Dell Version     : 1.06
             Total Image Size : 000414fc [267516]
             Fast Boot        : Yes
             Image Address    : 0x14100000
             RegionOffset     : 0x00100000
             RegionSize       : 0x00080000
             RegionType       : 0x00000002

That's it, you'll need to also do it for the other controller if the shelf has 2. The BIN is in this zip : https://fohdeesha.com/data/other/MD12_106.zip - which was extracted from dell's update package here - Dell MD12XX EMM Firmware | Driver Details | Dell Canada
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,186
2,097
113
30
fohdeesha.com
Since the controllers are only $14.99 shipped on ebay, ordered a replacement for the one I bricked thanks to teraterm's broken xmodem implementation, and then cracked it open and had a look around. Looks like they're basically just an LSI SAS2X36 expander in a fancy case: Broadcom Inc. | Connecting Everything

The management is ARM and it boots from NAND flash (M29DW323DB), I could connect to it via JTAG and write the firmware back to unbrick it, but the JTAG connector (top right) is not populated, and I'm not populating a ~20pin ARM JTAG header with a soldering iron when I can replace the whole thing for 15 bucks






full size pics in here: index - powered by h5ai v0.29.2 (https://larsjung.de/h5ai/)
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,186
2,097
113
30
fohdeesha.com
Upon first glance that board looks identical to my "Compellent SC2" board.
interesting, I had a suspicion they're just rebranded MD controllers running slightly different images. Wanna try flashing MD1200 firmware to one? :p (not sure what the new "md1200 controller" would think about being in a compellent backplane). I would say it would be easy to flash back, however I can't find SC200 firmware anywhere to flash. Looking through all the compellent software packages for an update binary buried somewhere but so far nothing. I thought I saw mention a long time ago that in the compellent OS crap you could initiate a firmware update of the shelves, so the BIN had to be buried in there somewhere at one point. You could theoretically flash back to stock FW by putting a stock module next to it and running "_flashpeer" but I have no guarantees
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,186
2,097
113
30
fohdeesha.com
found this report - Md1200 VD sc200 - Dell Community , apparently an md1200 controller in an SC200 chassis (which is what you'd be doing by flashing MD1200 firmware on your controller) acts strange in the sense it can only see half the shelf, and only the SAS out port works, I'm guessing it has to do with the 1 difference between the sc200 and the md1200 - the md1200 has the "split mode" switch which the controllers can deal with, while the sc200 is coded for unified mode only. If I had to guess, it's a difference in the backplane firmware.

If you get a chance, could you pastebin the output of "fru_read 2" and "fru_read 5" (FRU info of backplane #1 and #2)

here's the results of a standard md1200: BlueDress.106.000 >fru_read 2 00000000: 01 0a 19 01 00 00 20 bb 01 09 00 4d 1 - Pastebin.com
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,186
2,097
113
30
fohdeesha.com
activated big brain mode and went down an internet hole trying to find a single SC200/SC220 firmware bin, looked through all the public compellent tools and then realized I was looking in the wrong place - it would be in the compellent OS ISO/ZIP that runs on the compellent master hardware itself. They're locked down to compellent customers but through a bunch of googling I was able to find a zip from 2015 - Dell Storage Center Operating Systems 6.6.5 Release | Driver Details | Dell Canada

Searched the entire archive for the LSI expander firmware strings and I found it, SC200/SC220 v1.01 binary attached. Since it's from such an old compellent OS image, it's older firmware, but it goes to show it exists. If I can find someone with a compellent account or even a compellent OS (Dell Storage Center Operating System / SCOS) image from the last couple years, it should have the latest SC200/SC220 firmware, which I would imagine would be 1.06. Then everyone using these JBODs can at least get on recent firmware

So if anyone reading can get me a compellent SCOS image from the last couple years, pls message me!
 

Attachments

  • Like
Reactions: frameshift18

frameshift18

New Member
Jan 9, 2019
16
7
3
Dumped the flash from a v1.03 controller I have but when trying to upload it to another controller it all goes well and seems to work and the _ver output looks good but when you reboot it, _ver just looks corrupted and the region I flashed never goes active. not sure if there's a checksum somewhere or if the bin needs to be an exact size. I did truncate the bin to match the image size that _ver stated for 1.03. Uploading your 1.01 bin works as expected though. I was using _download to upload to region 1 or 2, to keep boot alone. I wasn't sure if you flash the boot region if it will boot if its corrupt or if if it will just move to another non corrupt region.

Also the help text for _download will say 1 and 2 are the active and nonactive regions respectively. But the active region can and will vary so disregard what it says and just use 0, 1, or 2 for what region you want to flash based on _ver.
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,186
2,097
113
30
fohdeesha.com
How'd you dump the flash, just used the _flashdump command to dump it to the serial terminal and then format it into a hex bin? If so and it sounds like it's not flashing right, I'd imagine something went wrong in the display > conversion process, it should indeed match the size "_ver" shows exactly

It turns out the "boot" section is a bit of a misnomer, it's just another flash slot like image 1 and 2, so it's better to think about it like image 0, image 1, image 2 - the really simple bootloader seemingly just chooses whichever has the largest version number and boots from that slot. So even if you flash my 1.01 to "boot" (slot 0), it'll just see the more recent version in image slot 1 and boot from that, so if you wanna downgrade you gotta flash the older image to all 3 slots

The bootloader is REALLY simple, seemingly only a command to display and modify raw memory (or maybe FPGA config registers, can't tell which):
Code:
help    d [loc]
        p  loc  value

        DesignWare (TM)
        DW_debugger
Instead of running at 38400-8-N-1, the bootloader also runs at 38400-7-N-1, which I've never seen before - that's why you get garbled/weird ASCII over serial on boot, for a few lines the card is sending 38400-7-N-1 bootloader output to your 38400-8-N-1 terminal session


Also got bored and flashed the SC200 image to my MD1200, and it indeed had no issue and became an SC200 controller:

Code:
**** Surly Startup Complete (Based on vendor Phase 9 drop 00.09.00.00) ****

12Pack.Slot0.101.0 >set_speed 10
12Pack.Slot0.101.1 >
Of course like I said earlier the backplanes are slightly different, so I'd imagine if I put actually drives in it, it would have odd behavior or not be able to see all the drives, being SC200 code talking to an MD1200 backplane. But I guess at least if you ever have an SC200 controller and need an MD1200 controller or vice versa, you can just flash them over to be whatever you need

If someone can get me a compellent / dell SCOS image from the last couple years it'll take like 20 seconds to pull out a clean SC200/SC220 v1.06 firmware image
 
Last edited:

frameshift18

New Member
Jan 9, 2019
16
7
3
How'd you dump the flash, just used the _flashdump command to dump it to the serial terminal and then format it into a hex bin? If so and it sounds like it's not flashing right, I'd imagine something went wrong in the display > conversion process, it should indeed match the size "_ver" shows exactly
Yeah I just used _flashdump and dumped the image size worth from the region with 1.03 in it. Then used sed and xxd to get it into a bin. The header and a lot of the strings are similar to your 1.01 so I'm not entirely sure why it doesn't work.

Also I might have a lead on a 7.01 or 7.02 scso release
 
Last edited:
  • Like
Reactions: fohdeesha

frameshift18

New Member
Jan 9, 2019
16
7
3
Got a hold of SCOS 7.2.1 which came out in Feb 2017.. it contains v1.03. Looks like the latest SCOS is around 7.4 from what I can gather.

Also comparing this to my dump it matches til about half way then my dump looks like its missing a chunk, so yeah, hiccups when it was dumping out to the console like you said.
 

Attachments

  • Like
Reactions: fohdeesha

fohdeesha

Kaini Industries
Nov 20, 2016
2,186
2,097
113
30
fohdeesha.com
Nice find, considering the latest v1.06 firmware for the md1200 came out in 2015, I'd wager the 1.03 image from that 2017 SCOS zip is the latest there is for it. I flashed it to my MD1200 out of curiosity and again everything seems to be working, and I can read/write to a sata SSD in it at full 6G sata speeds with no interposer. I'm going to run some more tests and move the drive around, but so far it seems to be working perfectly. I wonder if that poster who said when he mixed controllers he could only see half the drives, if perhaps he had the 24 slot SC220 chassis, but the MD1200 controller was in 12-drive mode.

Out of curiosity, could you flash one of your controllers with the MD1200 v1.06 firmware (and of course have it be the only controller in the chassis), and see if you can see all the slots/drives? Be sure it's set to the right mode, blue devil for the 12 drive chassis and red devil for the 24 slot chassis. can see what it is with the "chassistype" command:

chassistype Display and or Set Chassis Type: chassistype <0 = Blue Devil !0 = Red Devil>
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,186
2,097
113
30
fohdeesha.com
some info from my MD1200 with a controller flashed to SC200:

Code:
12Pack.Slot0.1.03.000 >sas_address

Expander SAS ADDRESS: 500c04f2d69a5d3f
ELI ADDRESS:          500c04f2d69a5d00
VPHY ADDRESS:         500c04f2d69a5d3d
ISIM ADDRESS:         500c04f2d69a5dbd

Slot:  0 Phy: 33 SAS ADDRESS: 500c04f2d69a5d21 SATA Drive 6G
Slot:  1 Phy: 32 SAS ADDRESS: 0000000000000000
Slot:  2 Phy: 24 SAS ADDRESS: 0000000000000000
Slot:  3 Phy: 27 SAS ADDRESS: 0000000000000000
Slot:  4 Phy: 26 SAS ADDRESS: 0000000000000000
Slot:  5 Phy: 25 SAS ADDRESS: 0000000000000000
Slot:  6 Phy: 30 SAS ADDRESS: 0000000000000000
Slot:  7 Phy: 29 SAS ADDRESS: 0000000000000000
Slot:  8 Phy: 28 SAS ADDRESS: 0000000000000000
Slot:  9 Phy: 31 SAS ADDRESS: 0000000000000000
Slot: 10 Phy: 34 SAS ADDRESS: 0000000000000000
Slot: 11 Phy: 35 SAS ADDRESS: 0000000000000000

12Pack.Slot0.1.03.000 >chassistype
Chassis Type: Blue Devil (12 drive)
Code:
root@r620tester:~# lsscsi
[0:0:0:0]    disk    ATA      INTEL SSDSC2BA40 2270  /dev/sda
[0:0:1:0]    disk    ATA      MK0400GCTZA      HPG0  /dev/sdb
[1:0:0:0]    disk    ATA      Samsung SSD 860  3B6Q  /dev/sdc
[1:0:1:0]    enclosu DELL     SC200            1.03  -
[6:0:0:0]    disk    ATA      KINGSTON SV300S3 BBF0  /dev/sde
[8:0:0:0]    cd/dvd  iDRAC    Virtual CD       0329  /dev/sr0
[8:0:0:1]    disk    iDRAC    Virtual Floppy   0329  /dev/sdd

Code:
root@r620tester:~# dd if=/dev/zero of=/dev/sdc bs=4M count=5000
5000+0 records in
5000+0 records out
20971520000 bytes (21 GB, 20 GiB) copied, 47.1326 s, 445 MB/s
root@r620tester:~# dd of=/dev/null if=/dev/sdc bs=4M count=5000
5000+0 records in
5000+0 records out
20971520000 bytes (21 GB, 20 GiB) copied, 39.3732 s, 533 MB/s
Code:
root@r620tester:~# smartctl -a /dev/sdc
smartctl 6.6 2017-11-05 r4594 [x86_64-linux-4.19.0-8-amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     Samsung SSD 860 EVO 1TB
Serial Number:    S3Z8NY0M714134W
LU WWN Device Id: 5 002538 e09705660
Firmware Version: RVT03B6Q
User Capacity:    1,000,204,886,016 bytes [1.00 TB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 T13/BSR INCITS 529 revision 5
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)