Fun with an MD1200/MD1220 & SC200/SC220

[Sellular]

Has anyone found a reliable way to spindown/spinup the harddrives, or the enclosure itself? I see there's a `_shutdown` command that can be used to power off the unit, but not sure if there's a good way to remotely turn it back up. Or if I should look into spinning down/up the hard drives themselves?

I'm asking because I recently (finally) got a UPS and this would be helpful in the case of a power outage to conserve internet up-time. I'm already turning off my servers automatically when power outage occurs so doing so on my MD1200 would be helpful too. Not sure if figuring out how to hack something together to turn off/on the connections to the power cables would work too

 

[PatchouliTC]

Has anyone found a reliable way to spindown/spinup the harddrives, or the enclosure itself? I see there's a `_shutdown` command that can be used to power off the unit, but not sure if there's a good way to remotely turn it back up. Or if I should look into spinning down/up the hard drives themselves?

I'm asking because I recently (finally) got a UPS and this would be helpful in the case of a power outage to conserve internet up-time. I'm already turning off my servers automatically when power outage occurs so doing so on my MD1200 would be helpful too. Not sure if figuring out how to hack something together to turn off/on the connections to the power cables would work too

i have read dell manual,it seems that `_shutdown` command may used when emm failed/temp to high/lost one power module too long,so i think this command used for shutdown if sys will be break in emm internal;

Also I find a post here [How to perform a graceful shutdown of MD3220 | DELL Technologies] this guy was look for soft shutdown because building power outage,it also show that DELL Modular Disk Storage Manager app has no option to "soft shutdown.",and dell Moderator says " Yes you are correct to power off the MD’s you will just flip the switch or pull the power cord. "
So i believe if power down,just make sure your hba card device success shutdown,then just wait ups shutdown[only sure when enclosure lost power no device use it] and just do nothing.(maybe when power back,your hbd card device should wait 1~2minute for MD1200 start then power on)
After all, looking at it from another perspective, the act of switching the power button on and off is similar to the act of keeping the power button on to turn the power on and off.

 

[mr44er]

Or if I should look into spinning down/up the hard drives themselves?

This! Let the disks do it on their own.

System hanging when trying to spin-up from HD idle

I've searched an found that sometimes LSI's SAS controllers will hang up when HDs are woken up from an idle state (idled using hd-idle). I'm experiencing this with my pve system. I've even swapped out controller cards using a different LSI controller and the problem persists. At this point, I...

forum.proxmox.com

 

[logixworx]

I have two SC220 stacked in the front of a closed 42u rack cabinet - I want to redirect 4u of SC220 exhaust into 2u of open space in the rear. I feel the SC220 exhaust is too far from the back of the cabinet anyway and may be blowing heat into the cab through the sides. Think of modifying a cantilever shelf into a baffle with a sloped top and sides. I could move my APC 7900B into that space. Any issue with this design?




illustration

 

[Flipt]

Thanks to everyone that contributed to this thread. I was about to do a fan swap before coming across this thread. Opted to try the serial cable method instead which is now working great. A cron job was scheduled to run the following script every minute:

#!/bin/bash
for i in {1..10}
do
stty -F /dev/ttyS0 speed 38400 cs8 -ixon raw
echo -ne "_shutup 20\n\r" > /dev/ttyS0
sleep 5
done
exit

 

[Andyman!]

Just made this account to say a huge thank you.


I'm completely new to the server world, and this is my first time diving into Dell gear. I made the rookie mistake of buying an SFF Dell server thinking I’d save myself £100… only to find out the largest spinning disk it supports is 2TB. That was a blow.


So I picked up a Dell MD1200 shelf and everything needed to expand — and suddenly that £100 saving turned into an extra £300 with cables, PCIe cards, and everything else.


When I powered it all on, the fans were insanely loud — my server lives right next to my desk (despite my wife’s best efforts to move it), and even she said she could live with the noise... but I couldn't.


After a full day of reading your posts and retrying things, I finally managed to get serial access working using a DIY cable setup. I went from 70dB down to 40dB just by running set_speed 20, and the difference is unreal — the room feels peaceful again.


I didn’t want to spend £30 on a prebuilt cable, so I DIY’d it for around £25 using different adapters and breakouts. It took a bit longer, but I learned so much along the way, and that knowledge is absolutely priceless.


Thank you again for sharing this info — you saved my ears, my sanity, and possibly my marriage.

 

[RCLapCar]

I thought I would post this to help out anyone still looking for a cable to connect to the MD1200. I was looking for some cables to quiet down multiple MD1200's and went down the many serial port rabbit hole. After some unrelated browsing I came across this item on Aliexpress.

https://www.aliexpress.com/item/1005005345205637.html

Basically this is a cable that has a USB connector on one end (with a serial to usb chip in it) and the 6 pin DIN connector on the other end. This lets you just plugin the USB into a server/PC and the other end directly into the MD1200. Works on Linux and Windows (with driver from Windows Update, so no sketchy files to install). No need for a serial port, serial to USB adapter or the password reset cable (or a hacked together one). This specific item is NOT for the MD1200 but if you contact the seller they will make one for the MD1200 instead. This store can make any sort of serial cables custom. They are in China so chatting with them happened in the evening for me but they were super responsible and the cables arrived in less than 2 weeks. Have been working without issue for a month now keeping the damn MD1200 fans quiet.

One more thing, when you go to that link make sure to select the second "color" option for the "USB-MD6 (FT232RLChip)" to get the version of the cable with the USB connector and serial to USB chip. The other option is basically a replica of the Dell password reset cable (just cheaper). You can also customize the length if you wish but I just went for the shortest one. In retrospect I probably could have asked them for a shorter cable as well but didn't think of that at the time.

Hope this helps someone.

 

[jabuzzard]

The SC200/SC220 are rebranded PowerVault MD1200/MD1220's which were a Dell designed/built product. The only other product which looks like it uses the same enclosure/backplane part number is some of the EqualLogic products like the PS4100. I've never tried this swap, and if it does work then you are limited by the licensing and software of the EQL product. This design very much follows the KISS principal, which I am a huge fan of for the simplicity and it being very reliable.

This is not true. The MS1200/MD1220 are most emphatically NOT designed and built by Dell. They are rebranded Engenio devices, originally from LSI, but the line was sold to NetApp in 2011. This holds for the entire PowerVault MD storage line from Dell.

The fact that you can turn an MD3200 into an MD1200, but the MD3200i and MD3600f can be managed by the same software that controls an IBM DS4200, DS4400, DS4500, DS4700, DS4800, and DS3400 something I have personally done. There are other IBM DS storage products that could be added to the list but those are the ones I have personally used. That should be a giant clue that they are not designed/built by Dell. It was also OEM'ed by Sun/Oracle in at least the StorageTek 2500 line (though they did their own management software from scratch, which is entirely different) and even by NetApp themselves in various FS arrays. There are probably others but these are the ones I have personally used.

Go Google some pictures of the Dell MD3060e and IBM DCS3700, and tell me that those are not the same OEM product with different branding.

I am pretty sure that the ME line is descended from Dot Hill rather than Xyratex, too. The storage array software on the ME line is definitely Dot Hill in origin.

 

[Deleted member 186362]

Just wanted to say thank you for the knowledge!! I just successfully flashed a MD3200 to a MD1200 for a home media server. Saved me a couple hundred dollars since the MD1200 usually sells for more.

 

[jabuzzard]

Ok, a weird one. After much hassle in getting a working serial port on a Windows machine, which ultimately involved me dragging the MD3200 out of the data centre to access a Windows machine with a serial port. I have flashed it to be an MD1200 via the serial port using TeraTerm, and everything is working as expected. Has anyone done this on Linux with screen and sx out of lrzsz, as it didn't work for me? As I type this, I wonder if the issue was with the USB 3 port on my laptop, as the adapter worked just fine on the server, but didn't work under either Windows 10 or Linux on my laptop, hum.

The two EMMs I had acquired were version 1.05, so I flashed them to 1.06 using sg_write_buffer. Everything is still good.

As it was initially an MD3200, the ready light meaning was inverted on the drives. No problem, I used sdparm to flip it on one drive at random, and it ended up being slot 2. That worked as expected, so I then ran the command to clear the RLM bit on all the drives.

Now, what is weird is that the ready light does not come on, regardless of the drive I put in slot 2, even though the drive itself is fine and believes the ready light should be on. The LED doesn't flicker either when doing IO on the drive. Power cycling both the attached server and the MD1220 makes no difference. It's like the ready light LED is dead. However, it seems unlikely to me that you could kill the ready light LED on a slot by running an "sdparm --clear=RLM --save /dev/sdxx" on a drive where the ready light meaning has already cleared.

I don't see anything on the serial console that allows me to manipulate the ready light. Does anyone have any ideas, or is the backplane knackered? I have yet to take the cover off and start probing with the oscilloscope.

 

[mophk]

Hey guys, Im kinda late to the party, but alas;

Ive bought a ps6210 4u with control modules 17 15. Is there any way to confirm that my case would accept the md12 control modules?
I cant get it to work at all, no serial, no management, no sfp+ activity and Im kinda bummed out.


Edit:
Because I was so bummed out, i went ahead and bought 2x sc2 module and the case woke up. im in the middle of process to update/flash etc everything.

 

[cam_e_ron]

Hey guys, Im kinda late to the party, but alas;

Ive bought a ps6210 4u with control modules 17 15. Is there any way to confirm that my case would accept the md12 control modules?
I cant get it to work at all, no serial, no management, no sfp+ activity and Im kinda bummed out.


Edit:
Because I was so bummed out, i went ahead and bought 2x sc2 module and the case woke up. im in the middle of process to update/flash etc everything.

Glad to see you got it working. I had the exact same thing happen to me. Been using this shelf for a few years now 24/7 and have had zero issues. I only run a single controller since the power draw is less but I have only 9 disks so far so not sure how well everything would work with a full shelf.

 

[mophk]

Glad to see you got it working. I had the exact same thing happen to me. Been using this shelf for a few years now 24/7 and have had zero issues. I only run a single controller since the power draw is less but I have only 9 disks so far so not sure how well everything would work with a full shelf.

i have 12 drives right now on 1 controler, 4 sas drives and rest sata and everything seems all right. ive plugged in the sas drives all over the unit, rest in the top part to minimise the risk of issues using only 1 controller.

 

[rome]

Like others, I came here to tell others "thank you!". I had originally used this thread in my decision in purchasing two MD1200 chassis as our storage solution here in the home lab. But an unusual situation has come up. The first shelf has been running wonderfully since purchase, but I made the mistake of not bringing the second shelf up with a full compliment of disks (which they came with, and I am still swimming in 1.5TB spinning rust) to really test before my 30-day warranty was up.

Fast forward many years pass to several weeks ago, I decided it was time to upgrade my storage game to using both shelves (run some more docker stuff, maybe split the shelf, snapshot backups, etc). When I did so, I found that the second shelf when fully loaded would turn on, operate and detect all disks, run for several minutes, then hard power off. Not great. A quick bit of scripting later, some trial and error, and I was looking at the voltage and current load for the shelf and found that the 5V and 12V rails had excessive load on idle with no drives loaded and the onboard thresholds set ridiculously low. A couple of new PSUs, and backplane later, and the load great on the 12V rail, but still seeing something on the 5V rail. If asked, 'loads of 5-10A on the 5V rail' at idle with no disks is what I would say. It's enough of an issue that I don't think leaving the unit on is a great idea.

Further tear down, I inspected the EMMs components and they visibly look fine. There was only one inductor (on what looked like the 5VDC supply rail of one EMM) that looked like it might have gotten spicy, but a conductivity tested passed fine. No direct shorts between 5VDC and ground plane. I'd be interested in anyone's opinions on what to test next, as at this point the only thing that I haven't replaced on the shelf is the EMMs and the chassis itself.

The two EMMs I had acquired were version 1.05, so I flashed them to 1.06 using sg_write_buffer. Everything is still good.

I would be very interested in what command options you used, as this seems very much to my interests.

In response to wavejumper digging into the firmware, I too spent a minute and dumped the 1.06 firmware into to Ghidra to see what there is to see, and was happy to see the list of various commands for the console. One specific thing of interest was at address 0x

0002B7DF

. That section of the firmware (0x0002b7d9[8]) was specific to the default inquiry response. What was specifically interesting was that the default bits at location 0x0002b7df (byte value 0x90) toggle on the reserved bit of byte 6b7 of the inquiry message. Further digging suggests that this value is never mutated by code, nor directly read by the MCU program, but is most likely access via memory of the expander processor itself. Because this bit is reserved and marked obsolete in SPC-3 and onward, this might be a bug in the firmware and result in unexpected results in interfacing.

If the value was set to 0x50 the EncServ and MultiP bits would be set correctly, and the reserved bit would not be flipped.

 

[jabuzzard]

I would be very interested in what command options you used, as this seems very much to my interests.

It is covered in page two of this topic.

sg_write_buffer -b 4k -m dmc_offs_save -I MD12_106.bin /dev/sgXX

where sgXX is the device name recovered from lsscsi -g You need to repeat for both ESM's even if only one is connected to the server (assuming you have two) and then hard power cycle the shelf. Presumably there is a way to reset the ESM via software over the SAS connection because when you do firmware updates from an MD32xx you don't need to hard power cycle the enclosure. However given there has been no firmware updates for around a decade now and thus there won't be anymore it is just a commissioning step and the need to hard power cycle the enclosure is not a big issue.

Historically there was huge warnings about running with less than two drives plugged in on Engenio stuff as there are minimum power draw requirements. These shelves are most definitely of Engenio heritage, OEM'ed by Dell, IBM, Sun/Oracle and NetApp before being purchased by NetApp from LSI. Probably why everyone moved to OEM'ing the Seagate stuff (which Seagate purchased, though I can't remember from who now).

 

[oldfett]

Probably why everyone moved to OEM'ing the Seagate stuff (which Seagate purchased, though I can't remember from who now).

Xyratex, good enclosures. More logs and diagnostics but then again also a little more complicated because of it.

 

[jabuzzard]

Xyratex, good enclosures. More logs and diagnostics but then again also a little more complicated because of it.

Maybe, but the actual RAID controllers are garbage compared to the Engenio arrays. HDD firmware updates require no I/O activity, you can't change RAID levels on existing arrays or add extra disk to them and thats just starters.

 

[oldfett]

Maybe, but the actual RAID controllers are garbage compared to the Engenio arrays. HDD firmware updates require no I/O activity, you can't change RAID levels on existing arrays or add extra disk to them and thats just starters.

Oh, interesting! I only ever used the JBOD's, glad I dodged that bullet!

 

[Cypher-]

First off, thank you to the legends in this thread who figured this stuff out. After some messing around I am able to modify the firmware on my SC220s to have the SES EncServ bit set and flash them with sg_write_buffer. Sorry for the long post but at least I'll have some new information.

Some backstory: A few years ago I had the opportunity to grab 10 SC220s full of SSDs. Because I didn't need all of the space at the time and because my power bill was high enough, I opted to use four of them and set them up with TrueNAS core. This worked great. I lost one drive over the years and was able to use `sesutil` to locate the drive. I'm currently doing a hardware refresh and figured I'd bring four more SC220s online so I have a duplicate of my data, and also bring up all my reinstalled hosts using the new storage. That unfortunately turned into a bit of a headache.

While the first four I configured have been running fine, these next four have been continuously dropping drives whenever they feel like it. My new TrueNAS install is using Scale so there is no more `sesutil`, so I tried what @Haim Gelfenbeyn suggested with sg_ses. This would not work for me. With the correct enclosure number and even though /sys/block/sdX/device/sas_address existed I would get "SAS address not found". I tried a bunch a different things but could not get any locate function to work. Having 96 drives to sort out without a location function is a nightmare. The best I could do was see which drives didn't have activity when doing a write.

At this point I decided to grab a magic cable from Amazon (the same one linked somewhere in this thread) and start messing with the enclosures. When the cable arrived I hooked it to my laptop and my trusty serial-to-USB adapter I've been using for I don't know how many years. Like others reported, when I rebooted the enclosure I received output but I could not type anything. Because I'm an idiot, I then ordered a cable from eBay rather than trying the serial port on my R740xd. As you can guess the next cable had the same problem. It turned out it was my serial-to-USB adapter. Both cables work fine when hooked to the serial port on my server.

So the next problem was I only have Proxmox to communicate with these enclosures. I use tio normally for serial communication and while it has xmodem capability, I could not get it to work. Using the _download function starts a timeout countdown for the transfer. I can then tell tio to send the file via xmodem but after doing that pressing any key will abort the transfer. At this point there was no more countdown but there was also not a progress bar. I am assuming the transfer was timing out in the background and that I actually needed to press a key to start it, but that keypress would abort it from the tio side. I tried a couple of things but was ultimately unsuccessful.

That brings me to trying to figure out the checksum issue. I could wipe firmware regions all day by letting the timeout finish with `_download (1/2) 0 y`, then reflash them with `sg_write_buffer -b 4k -m dmc_offs_save -I fw.bin /dev/sgXX` as long as it was the stock firmware. This was already discussed so that shouldn't be a surprise.

Before trying to figure this out myself I turned to our new lord and savior Claude. I uploaded the stock firmware image and basically asked if it was possible to figure out where the checksum was and how it was calculated. Not only did it respond with that, but without asking it wrote me a script to update the checksum for the modified version of the firmware. You can find the response with the script here: Great news — the analysis is conclusive. Here's a full breakdown of what was fou - Pastebin.com

Sure enough, this works perfectly. I started by wiping region 1 and 2 (mainly to make sure that I wasn't confused if it was running stock 1.03 or my custom 1.03 version), then using sg_write_buffer to flash the first region, then power-cycling the unit. When it came back it was running the custom firmware. I used `sg_write_buffer` to flash the other region which also worked fine.

Some other notes: I actually did this on the 1.03 firmware on the first page rather than the MD12_106.bin firmware since I have SC220s. I was hoping it would contain the same hex data (05 12 3D 00 90) which it did, so I modified the 90 to D0 as suggested by @wavejumper. encled works perfectly now. Later on I went one step further and modified the "1.03" that appears at the beginning of the file to "1.04" so I can easily tell which of my enclosures have the modified firmware. I don't know if that was a smart idea but it seems to be working fine.

I feel like a massive fraud using AI to solve the checksum issue after seeing all of the hard work you folks put in to figure this stuff out, but at least it gives another option to flash these units without needing ExtraPutty.

 

[jabuzzard]

You can always use sdparm to toggle the drive's ready light meaning. Either on or off, depending on what you have it by default. Usually it's on, meaning the drive has the presence LED on and blinks with activity, so

sdparm --set=RLM /dev/XXXX

Will turn it off, and the drive becomes readily apparent. Use --clear to turn on if you normally have it off (unusual, but some drives taken from enterprise arrays come like this). Note this does not survive a power cycle of the drive. If you want the status to persist, then add --save to the command line.

Saves spending a lot of time fiddling with the firmware on the enclosure to get the error LED working