How to reformat HDD & SSD to 512B Sector Size

[dtrv]

Hi!
I solved the problem with sedutil for TCG Enterprise - GitHub - scaledata/sedutil: Fork of DTA sedutil Self encrypting drive software
First I erased all bands (0-3) using the MSI password, then I was able to format the disk.

But what is the MSI password?

 

[dmitryvd]

But what is the MSI password?

I meant the MSID, of course.

 

[dtrv]

Thought there is a master password or something like that

Thanks alot for linking to sedutil!

 

[frogtech]

Are there any NetApp HGST SAS disks out there that simply are not compatible with any other system even if you format them to the 512 byte block? Maybe perhaps due to the firmware loaded on the disk itself?

Trying to avoid any potential gotchas with buying 25-30 disks at once and them not working even after re-formatting them.

 

[BLinux]

is it possible to reformat HGST SAS drives that are have native 512 blocks to 4k blocks?

 

[dtrv]

is it possible to reformat HGST SAS drives that are have native 512 blocks to 4k blocks?

I have seen HGST SSDs that firmware only supports 512, 520 or 528 byte blocksize.

 

[BLinux]

I have seen HGST SSDs that firmware only supports 512, 520 or 528 byte blocksize.

did you mean HDD or SSDs?

 

[Adeel Akram]

Are there any NetApp HGST SAS disks out there that simply are not compatible with any other system even if you format them to the 512 byte block? Maybe perhaps due to the firmware loaded on the disk itself?

Trying to avoid any potential gotchas with buying 25-30 disks at once and them not working even after re-formatting them.

Well we can't rule that out. I am currently stuck with 400 HGST 450GB 15K disks that are already formatted 512 bytes but refuse to work in any system that I have.
I have tried dozens of serves and expansions so far. Tried Every tool i can think of and so far nothing has worked.

I even have a 500$ reward for anyone who can break them. But so far no one has been able to crack them. :-(

 

[Oddworld]

Any progress on this? I have about 20x HGST drives that have been formatted from 520 to 512, but cannot write to them. I'm using 3x dell H310 cross-flashed to IT mode. I've tried:

Commands

/home/user/setblocksize -b512 /dev/sg11
sg_format --format --size=512 /dev/sg13 -v
sg_format --format --size=512 --six -v /dev/sg2
​

sginfo

Device Type 0
Vendor: SE_HGST
Product: DKR5D-J900SS
Revision level: B8B8​

sg_scan -i

dev/sg2: scsi0 channel=0 id=0 lun=0
SE_HGST DKR5D-J900SS B8B8 [rmb=0 cmdq=1 pqual=0 pdev=0x0]
/dev/sg3: scsi0 channel=0 id=1 lun=0
SE_HGST DKR5D-J900SS B8B8 [rmb=0 cmdq=1 pqual=0 pdev=0x0]
/dev/sg4: scsi0 channel=0 id=2 lun=0
SE_HGST DKR5D-J900SS B8B8 [rmb=0 cmdq=1 pqual=0 pdev=0x0]
/dev/sg5: scsi0 channel=0 id=3 lun=0
SE_HGST DKR5D-J900SS B8B8 [rmb=0 cmdq=1 pqual=0 pdev=0x0]
/dev/sg6: scsi0 channel=0 id=4 lun=0
SE_HGST DKR5D-J900SS B8B8 [rmb=0 cmdq=1 pqual=0 pdev=0x0]
/dev/sg7: scsi2 channel=0 id=0 lun=0
SE_HGST DKR5D-J900SS B8B8 [rmb=0 cmdq=1 pqual=0 pdev=0x0]
/dev/sg8: scsi2 channel=0 id=1 lun=0
SE_HGST DKR5D-J900SS B8B8 [rmb=0 cmdq=1 pqual=0 pdev=0x0]
/dev/sg9: scsi2 channel=0 id=2 lun=0
SE_HGST DKR5D-J900SS B8B8 [rmb=0 cmdq=1 pqual=0 pdev=0x0]
/dev/sg10: scsi2 channel=0 id=3 lun=0
SE_HGST DKR5D-J900SS B8B8 [rmb=0 cmdq=1 pqual=0 pdev=0x0]
/dev/sg11: scsi2 channel=0 id=4 lun=0
SE_HGST DKR5D-J900SS B8B8 [rmb=0 cmdq=1 pqual=0 pdev=0x0]
</code>​

 

[Terry Kennedy]

is it possible to reformat HGST SAS drives that are have native 512 blocks to 4k blocks?

I don't know of any HGST drive that comes in both 512n and 4Kn variants - it is usually 512e and 4Kn. These have different model numbers, so if it is possible to go from one to the other it would likely involve loading new firmware - and if people could get the firmware, you wouldn't see posts from people with 100's of unusable drives asking for firmware. The confusion probably stems from a typo on some HGST 4Kn drive labels where it says "This HDD has reduced LBA counts if formatted as 512 bytes + T10 PI (8 bytes)." That should be "...formatted as 4096 bytes + T10 PI..."

 

[Mirabis]

Months later finally trying again....

sg3_utils does not see the disks (sg_scan), but storcli does. However I can't format from there ; (

 

[ceorl yip]

Thank you Pete.
I bought the h310 controller... I'm in the process of formatting all my hus156045vls600 netapp to 512. So far, i was able to initialize one drive and tested moving and deleting some files... seems to work fine.

Has anyone run into reliability issues with these formatted netapp hdds?

I’ve read through these 4 pages about 20 times. Here’s my experience:
I purchased a few of the Hitachi HUS156060VLS600 formatted for NetApp with a 520 sector size. I had a Dell 2950 rack server in my lab with a Perc 6i Raid controller. While I could see the drives in Ctrl-R, I couldn’t do anything with them – no hot spare, no raid, nothing. I purchased a Dell H310 off ebay and had to flash it to be able to do anything with the drives. Before flashing, it would behave just like the embedded raid 6i raid controller. Here’s how I flashed the H310:

1) Create a USB boot disk (I used Rufus, Fat32, FreeDOS).
2) Download this zip file: www.hercnetworks.com/Misc/LSI-9211-8i.zip
3) Extract and place on boot disk (I created a folder called LSI for it on my boot disk)
4) Boot using boot disk, change dir to LSI and run:
megarec -writesbr 0 sbrempty.bin
megarec -cleanflash 0
5) Reboot. Enter boot disk again.
6) Change to LSI folder again and run:
Sas2flsh -o 6gbpsas.fw

That’s it! Reboot. I had windows installed on a disk connected directly to the motherboard. Download the SG3 utilities from http://sg.danny.cz/sg/p/sg3_utils-1.42_mw64exe.zip and extract to a folder on your C drive (I did SG3). Run an admin command prompt, change to SG3, then run:
sg_scan
You should see your drives listed as pd0, pd1, pd2, etc.
Be sure you know which drive number you want to format and run:
sg_format --format --size=512 -v pd1
if that spits out an error, try:
sg_format --format --size=512 –-six -v pd1

 

[mrcico]

Months later finally trying again....

sg3_utils does not see the disks (sg_scan), but storcli does. However I can't format from there ; (

Seems your raid card blocking your access. Use pass-through (like h200) raid card for format.

 

[minzak]

I have same problem with no Access when format my test ST3600957SS. (((

I meant the MSID, of course.

Can you tell detailed about it. how to find MSID for my HDD ST3600957SS, and how to erased all bands (0-3)?
Which one command i must run to erase that bands? Thanks.
Alexandr
styxx@i.ua

 

[LarsM]

Hi

I am having some issues with formatting drives from 512 to 520. At work I wipe storage systems with certified erasure software and the problem is that the software we use, have to format the drive from 520b (HP 3par storage, EMC etc) to 512b in order to erase the drive. Afterwards i have to manually format the drives back to 520b for the storage systems to recognize the drive again.

I have done it succesfully with various drives (300GB, 450GB, 600GB, 900GB, 400GB SSD 1,92TB SSD - ALL SAS) but now i have the issue below.
I am using the same syntax as with the other drives

Anyone know what the problem is?

sg_format --format --size=520 -6 -v -e /dev/sg1
HGST HMRP3000S5xnN7.2 3P00 peripheral_type: disk [0x0]
PROTECT=1
<< supports protection information>>
Unit serial number: P9GB09AW
LU name: 5000cca0581404d8
mode sense (6) cdb: 1a 00 01 00 fc 00
Mode Sense (block descriptor) data, prior to changes:
Mode sense number of blocks maxed out, set longlba
mode sense (10) cdb: 5a 10 01 00 00 00 00 00 fc 00
Mode Sense (block descriptor) data, prior to changes:
<<< longlba flag set (64 bit lba) >>>
Number of blocks=5860533168 [0x15d50a3b0]
Block size=512 [0x200]
mode select (10) cdb: 55 11 00 00 00 00 00 00 22 00
mode select (10):
Fixed format, current; Sense key: Illegal Request
Additional sense: Parameter list length error
Sense Key Specific: Error in Command: byte 7
MODE SELECT command: Illegal request sense key, apart from Invalid opcode

 

[Mirabis]

Months later finally trying again....

sg3_utils does not see the disks (sg_scan), but storcli does. However I can't format from there ; (

Ended up buying a PERC200 flashed to IT and managed to format the HDD's

 

[oxi]

Well we can't rule that out. I am currently stuck with 400 HGST 450GB 15K disks that are already formatted 512 bytes but refuse to work in any system that I have.
I have tried dozens of serves and expansions so far. Tried Every tool i can think of and so far nothing has worked.

I even have a 500$ reward for anyone who can break them. But so far no one has been able to crack them. :-(

Did you check the size from the firmware-chip? I saw that the HP HDD have 2MB and same model from HGST only 512KB.

 

[Allan Houston]

Hoping this will help someone as it took me ages to get a working solution...

I couldn't get sg_format to work with my EMC formatted HGST HUS72303CLAR3000 3TB drives, tried all the combinations of command line arguments mentioned here, but nada:

root@ubuntu:~# sg_format --format --size=512 -v /dev/sg2
HITACHI HUS72303CLAR3000 C442 peripheral_type: disk [0x0]
PROTECT=1
<< supports protection information>>
Unit serial number: YXGMGGTK
LU name: 5000cca046236594
mode sense (10) cdb: 5a 00 01 00 00 00 00 00 fc 00
Mode Sense (block descriptor) data, prior to changes:
Mode sense number of blocks maxed out, set longlba
mode sense (10) cdb: 5a 10 01 00 00 00 00 00 fc 00
Mode Sense (block descriptor) data, prior to changes:
<<< longlba flag set (64 bit lba) >>>
Number of blocks=5770371118 [0x157f0e02e]
Block size=520 [0x208]
mode select (10) cdb: 55 11 00 00 00 00 00 00 22 00
mode select (10):
Descriptor format, current; Sense key: Illegal Request
Additional sense: Parameter list length error
Descriptor type: Information: >> descriptor too short
00 00 00 00 00 00 00 00 00 00
Descriptor type: Sense key specific: Field pointer:
Error in Command: byte 7
Descriptor type: Field replaceable unit code: 0x0
Descriptor type: Block commands: Incorrect Length Indicator (ILI) clear
Descriptor type: Vendor specific [0x80]
f8 20
Descriptor type: Vendor specific [0x81]
00 0MODE SELECT command: Illegal request sense key, apart from Invalid opcode


...but I did have success with the setblocksize util!

I've put together a very slightly modded version of setblocksize here with higher possible and default timeouts (the default is way to low for bigger drives) as well as the instructions on how to compile it etc. etc.

ahouston/setblocksize

So far I've rescued 7 of my EMC 3TB HGST drives, waiting on the last batch to finish up

 

[Wingman5]

Hello all, I got the same...

I have 4 100GB Samsung SSD's from an EMC storage formatted at 520B/sector.

I tried all of the commands mentioned here at the forum, but I keep getting the following error:
A FORMAT UNIT will commence in 5 seconds
ALL data on /dev/sg4 will be DESTROYED
Press control-C to abort
format unit cdb: 04 18 00 00 00 00
format unit:
Fixed format, current; Sense key: Illegal Request
Additional sense: Invalid command operation code
Format unit command: Illegal request, invalid opcode sense key
FORMAT UNIT failed

I tried it as mentioned with the option --six but no luck.

does anyone have a good suggestion?

 

[neggles]

Try sg_format with the -S (secure erase mode) flag. May work better.

Can someone try to solve this riddle for me?

I have four HUSMR1610ASS204s that presumably came out of a 3PAR. They report themselves as being an HP-specific part number, HSCP0920S5xnNMRI, unlike the HUSMR1010ASS204 model units (same HP P/N) I have from the same seller. When I did my first sg_format -F -6 /dev/sgX to them, they mysteriously lost 70GB of capacity/blocks???

Screenshot below contains 3 drives, a 1010 and two 1610s. the 1010 (sg2) happily reformatted from 1015GB of 520b sectors to 1000GB of 512b, with the block count remaining unchanged. As you can see it has the same number of blocks as sg4, which is an un-sg_formatted 1610.

But sg3, the 1610 which I'd already formatted to 512b, has gone from 1,953,525,168 blocks to 1,788,974,512 blocks - 164,550,656 blocks have disappeared entirely! sg4 did this also after I formatted it;

What the heck is going on here? Smartmontools reports 0% write endurance used and <35TB lifetime writes - nothing for a drive of this size and rating. Even reformatting back to 520 bytes (which I did before the screenshot above) hasn't given me back the 70GB, nor has an sg_sanitize or running sg_format with the -S (secure erase) flag.

Anyone have a clue what's going on here?

Edit: quick update to this, took me less time to fix than to make the original post... oops. If I run an sg_format -r --count=1953525168 /dev/sgX (command the device to resize itself) after changing the block size, the capacity comes back and the drive seems to be totally OK with it;

I suspect this has to do with the HP firmware and the fact that these are labelled as "920GB" when they're actually 1000GB units internally (HUSMR1010ASS20 / HUSMR1610ASS20 parts are 1000GB parts) - the HP firmware appears to recommend a 930GB capacity instead of 1000, even though they were reporting 1000GB of capacity out of the box.

Weird.
Anyway, hopefully that helps someone who bought one of these weird 1610s.