How to reformat HDD & SSD to 512B Sector Size

[WiteWulf]

A little addendum, in case anyone runs into the same problem as me: I took the newly sg_format'd disks from the chassis I was using to format them in (and that they'd been confirmed as working when connected to the Cisco 12G Modular SAS RAID controller in JBOD mode) and popped them in my live server (another Cisco UCS chassis with the same model Cisco 12G Modular SAS RAID controller in it). The disks immediately went into error mode, "unconfigured bad", and the CIMC refused to do anything with them.

So I shut down the guests and the ESXi host on the machine, power cycled it and went into the RAID controller BIOS (pressing ctrl-r when prompted). The RAID controller was quite happy with the disks and let me configure them as RAID volume or as JBOD disks. When I saved the config and rebooted again the CIMC was happy with the disks and cleared all the errors.

 

[RulerOf]

I decided to get myself some 3PAR-branded DC HC530 drives that came formatted with 520-byte logical sectors. The datasheet says they can be reformatted between all of the various sector sizes, including "fast format," and I was able to convert them from 520/4160 to 512/4096. It's a two-step process, 520 -> 4160 via fast format, then 4160 -> 4096 via full format.

• Check the formatting, see it's 520 byte logical

[root@drew-metal ~]# sg_readcap /dev/sg48
READ CAPACITY (10) indicates device capacity too large
  now trying 16 byte cdb variant
Read Capacity results:
   Protection: prot_en=0, p_type=0, p_i_exponent=0
   Logical block provisioning: lbpme=0, lbprz=0
   Last LBA=26789019647 (0x63cbfffff), Number of logical blocks=26789019648
   Logical block length=520 bytes
   Logical blocks per physical block exponent=3 [so physical block length=4160 bytes]
   Lowest aligned LBA=0
Hence:
   Device size: 13930290216960 bytes, 13284960.0 MiB, 13930.29 GB, 13.93 TB

• Convert to 4160 using fast format

[root@drew-metal ~]# sg_format --format --size=4160 --ffmt=1 --six --quick -v /dev/sg48
    WDC       WLEB14T0S5xeN7.2  3P00   peripheral_type: disk [0x0]
      PROTECT=1
      << supports protection information>>
      Unit serial number:         9FAKEJ4L
      LU name: 5000cca2591ce380
    mode sense(6) cdb: 1a 00 01 00 fc 00
Mode Sense (block descriptor) data, prior to changes:
block count maxed out, set <<longlba>>
    mode sense(10) cdb: 5a 10 01 00 00 00 00 00 fc 00
  <<< longlba flag set (64 bit lba) >>>
  Number of blocks=26789019648 [0x63cc00000]
  Block size=520 [0x208]
    mode select(10) cdb: 55 11 00 00 00 00 00 00 24 00
    Format unit cdb: 04 10 00 00 01 00

Format unit has started
FORMAT UNIT Complete

• Validate 4160 conversion

[root@drew-metal ~]# sg_readcap /dev/sg48
Read Capacity results:
   Last LBA=3348627455 (0xc797ffff), Number of logical blocks=3348627456
   Logical block length=4160 bytes
Hence:
   Device size: 13930290216960 bytes, 13284960.0 MiB, 13930.29 GB, 13.93 TB

• Convert to 4096 (very slow, takes ~16 hours)

[root@drew-metal ~]# sg_format --format --size=4096 --ffmt=0 --six --quick -v /dev/sg48
    WDC       WLEB14T0S5xeN7.2  3P00   peripheral_type: disk [0x0]
      PROTECT=1
      << supports protection information>>
      Unit serial number:         9FAKEJ4L
      LU name: 5000cca2591ce380
    mode sense(6) cdb: 1a 00 01 00 fc 00
Mode Sense (block descriptor) data, prior to changes:
  Number of blocks=3348627456 [0xc7980000]
  Block size=4160 [0x1040]
    mode select(6) cdb: 15 11 00 00 18 00
    Format unit cdb: 04 18 00 00 00 00

Format unit has started
Format in progress, 0.00% done
Format in progress, 0.10% done
<snip>
Format in progress, 99.75% done
Format in progress, 99.85% done
Format in progress, 99.95% done
FORMAT UNIT Complete

• Validate the new block size

[root@drew-metal ~]# sg_readcap /dev/sg48
Read Capacity results:
   Last LBA=3418095615 (0xcbbbffff), Number of logical blocks=3418095616
   Logical block length=4096 bytes
Hence:
   Device size: 14000519643136 bytes, 13351936.0 MiB, 14000.52 GB, 14.00 TB

• Rescan disks to allow mounting, then enjoy your formatted disk

partprobe

 

[josep gatius]

Good evening,
I have a dubt. i bought several SSD. Toshiba PX04SUB040.
Server LENOVO -IBM
Controller of the server
RAID M5210 SAS/SATA

I can see the disks from AVAGO. But i can not interact with it. With the other ones i was able to.
When I turn on SG-SCAN it detects the first ones, but i can't detect the other 5 toshiba. Could it be possible that the problem is in the controller?
What do you think could be happening?
Thank you.

 

[MountainDew]

Has anyone come across WD Oracle branded hard drives with model WUH721414AL5200? I've used sg utils to remove the protection but I can't seem to have multiple systems with different controllers see the entire 14TB. It's always 13.7TB and shrinks to 12.5TB once formatted. A normal 14TB drive reports 12.7TB. The documentation for it is pretty lacking, if at all existent.

 

[CyklonDX]

Has anyone come across WD Oracle branded hard drives with model WUH721414AL5200? I've used sg utils to remove the protection but I can't seem to have multiple systems with different controllers see the entire 14TB. It's always 13.7TB and shrinks to 12.5TB once formatted. A normal 14TB drive reports 12.7TB. The documentation for it is pretty lacking, if at all existent.

i've only played with hgst oracle branded disks, they are formatted with non-standard sector size, but there was no issues formatting to 4kn or 512.
First run sg_format /dev/sdX (no changes will be made) - post that info here.

Once you are ready to format (and it will take a day or 2) you can trigger following (where size is 4096 = 4kn, and 512 is 512)
sg_format --format --size=4096 /dev/sdX

 

[MountainDew]

i've only played with hgst oracle branded disks, they are formatted with non-standard sector size, but there was no issues formatting to 4kn or 512.
First run sg_format /dev/sdX (no changes will be made) - post that info here.

Once you are ready to format (and it will take a day or 2) you can trigger following (where size is 4096 = 4kn, and 512 is 512)
sg_format --format --size=4096 /dev/sdX

I ran the first command twice before, which completed successfully. Prior to this post, I have already executed
sg_format --format --size=4096 -v -e /dev/sda
It came back with

Format unit command launched without error

Format unit has started
Format unit continuing,
    request sense or test unit ready can be used to monitor progress

I haven't been able to figure out how to monitor the progress as it just returned back to the prompt. The hard drive light continues to blink away and the hard drive itself isn't available in Proxmox

 

[amp88]

The hard drive light continues to blink away and the hard drive itself isn't available in Proxmox

I wonder if there's any output available through the journal. Can you run the following command in your Proxmox host shell, which will return the last 50 messages?

journalctl -n 50

 

[MountainDew]

I wonder if there's any output available through the journal. Can you run the following command in your Proxmox host shell, which will return the last 50 messages?

journalctl -n 50

just a bunch of entries complaining that it can't connect to my gmail, but nothing regarding that drive

 

[Dave Corder]

Maybe try this (substituting the value for --count that matches your other drives:

sg_format /dev/sda -F --size=4096 --count=2441609216

I had a similar issue with a drive showing more space than expected, and I fixed it by reducing the count.

 

[Dave Corder]

I ran the first command twice before, which completed successfully. Prior to this post, I have already executed
sg_format --format --size=4096 -v -e /dev/sda
It came back with

Format unit command launched without error

Format unit has started
Format unit continuing,
    request sense or test unit ready can be used to monitor progress

I haven't been able to figure out how to monitor the progress as it just returned back to the prompt. The hard drive light continues to blink away and the hard drive itself isn't available in Proxmox

From the man page:

-e, --earlythis option is active when --format is given. The default action of this utility is to poll the disk every 30 seconds to determine the progress of the format operation until it is finished. When this option is given this utility will exit "early" as soon as the format has commenced. Then the user can monitor the progress of the ongoing format operation with other utilities (e.g. sg_turs(8) or sg_requests(8)).

 

[MountainDew]

From the man page:

-e, --earlythis option is active when --format is given. The default action of this utility is to poll the disk every 30 seconds to determine the progress of the format operation until it is finished. When this option is given this utility will exit "early" as soon as the format has commenced. Then the user can monitor the progress of the ongoing format operation with other utilities (e.g. sg_turs(8) or sg_requests(8)).

Ya got me - my use of -e was simply because someone recommended it and I copy/pasted it. I appreciate you educating me on this when I should have done the same.

Update - the command

sg_format --format --size=4096 -v -e /dev/sda

appears to have done the trick.
smartctl data

smartctl 7.3 2022-02-28 r5338 [x86_64-linux-6.5.11-4-pve] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               WDC
Product:              W7214A520ORA014T
Revision:             A140
Compliance:           SPC-4
User Capacity:        14,000,519,643,136 bytes [14.0 TB]
Logical block size:   4096 bytes
LU is fully provisioned
Rotation Rate:        7200 rpm
Form Factor:          3.5 inches
Logical Unit id:      0x5000cca29bcb8180
Serial number:        -------:)
Device type:          disk
Transport protocol:   SAS (SPL-4)
Local Time is:        Thu Dec  7 11:54:36 2023 PST
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled

Thanks to all that helped!

 

[nosar77]

Hello, I purchased some HGST HUC101818CS4204 drives from ebay, and they were 520B but trying to format them back to 512 or 4kn the drive always throw an input/output error. There is no write protection on the drives. Do you think these are locked in drives with the VSP Hitachi firmware? should i return these?

 

[MountainDew]

What steps have you taken?

 

[nosar77]

What steps have you taken?

1. Tried to add them to a pool in truenas and kept getting an error which means I needed to format them from 520 to 512

2. Formatted them in truenas shell. Tried to add to pool/wipe got input/output error

3. Tried going into gparted live and didn't work. Tried formatting again using a different command with sg utility in gparted live. Still input output error
4. Tried using hugo to format the drive. Same result

5. Formatted the drive to 4kn using sg utility, same errors.

6. Right now I'm using cam control inside truenas to see if this formatting will work

Ideas: possibly the firmware is blocking the formating but I'm not getting any errors on format. If so how would I flash new firmware to these drives.

Any other suggestions would help.

 

[Fritz]

1. Tried to add them to a pool in truenas and kept getting an error which means I needed to format them from 520 to 512

2. Formatted them in truenas shell. Tried to add to pool/wipe got input/output error

3. Tried going into gparted live and didn't work. Tried formatting again using a different command with sg utility in gparted live. Still input output error
4. Tried using hugo to format the drive. Same result

5. Formatted the drive to 4kn using sg utility, same errors.

6. Right now I'm using cam control inside truenas to see if this formatting will work

Ideas: possibly the firmware is blocking the formating but I'm not getting any errors on format. If so how would I flash new firmware to these drives.

Any other suggestions would help.

You need SG_Utils to format them to 512 or 4k.

 

[nosar77]

You need SG_Utils to format them to 512 or 4k.

Sorry I should have been more specific I've formatted the drives 3 times with sg_utils, once with hgst hugo and right now I'm running a format with camcontrol. And the drive still send input output error if you try to do anything to it.

I did this inside truenas shell by accident, tried in Debian, and windows and still sends input output errors if you try to wipe the drive or add it to a pool

 

[Fritz]

Sounds like the drives are either locked or defective. I'd send them back.

 

[MountainDew]

Sorry I should have been more specific I've formatted the drives 3 times with sg_utils, once with hgst hugo and right now I'm running a format with camcontrol. And the drive still send input output error if you try to do anything to it.

I did this inside truenas shell by accident, tried in Debian, and windows and still sends input output errors if you try to wipe the drive or add it to a pool

If you've tried all the tricks in this thread I'd send them back.

 

[JHanko]

Am I correct in assuming that this will not work with NVME drives? I'm trying to access an Optane D4800X with no luck. I can't find anything in the /var/log/messages file regarding either drive connected to this machine. sg_scan-i returns nothing. When I was installing centOS linux, it showed both drives, but it showed the Optane had only 4096 bytes available.

 

[citizen]

I seem to be having this same problem when trying to convert the 520 to 512 (can do 4096 per drive sticker too)

Get a Protected Error message when attempt to format 520 to clear data

sg_format --format --size=520 -v -e /dev/sda
    NETAPP    X317_SMBPE06TA07  NA00   peripheral_type: disk [0x0]
      PROTECT=0
      Unit serial number: ZAD2KJTW
      LU name: 5000c50094bfb1db
    mode sense(10) cdb: [5a 00 01 00 00 00 00 00 fc 00]
Mode Sense (block descriptor) data, prior to changes:
block count maxed out, set <<longlba>>
    mode sense(10) cdb: [5a 10 01 00 00 00 00 00 fc 00]
  <<< longlba flag set (64 bit lba) >>>
  Number of blocks=11521787400 [0x2aec08608]
  Block size=520 [0x208]

A FORMAT UNIT will commence in 15 seconds
    ALL data on /dev/sda will be DESTROYED
        Press control-C to abort

A FORMAT UNIT will commence in 10 seconds
    ALL data on /dev/sda will be DESTROYED
        Press control-C to abort

A FORMAT UNIT will commence in 5 seconds
    ALL data on /dev/sda will be DESTROYED
        Press control-C to abort
    Format unit cdb: [04 18 00 00 00 00]
Format unit:
Descriptor format, current; Sense key: Data Protect
Additional sense: Access denied - no access rights
  Descriptor type: Information: 0x0000000000000000
  Descriptor type: Field replaceable unit code: 0x0
  Descriptor type: Vendor specific [0x80]
    00 00 00 00 00 20 00 00 00 00 00 00 00 00
Format unit command: Data protect, type: sense key; write protected media?
FORMAT UNIT failed

Does not even start when trying to format direct to 512 (same with 4096)

sg_format --format --size=512 -v -e /dev/sda
    NETAPP    X317_SMBPE06TA07  NA00   peripheral_type: disk [0x0]
      PROTECT=0
      Unit serial number: ZAD2KJTW
      LU name: 5000c50094bfb1db
    mode sense(10) cdb: [5a 00 01 00 00 00 00 00 fc 00]
Mode Sense (block descriptor) data, prior to changes:
block count maxed out, set <<longlba>>
    mode sense(10) cdb: [5a 10 01 00 00 00 00 00 fc 00]
  <<< longlba flag set (64 bit lba) >>>
  Number of blocks=11521787400 [0x2aec08608]
  Block size=520 [0x208]
    mode select(10) cdb: [55 11 00 00 00 00 00 00 24 00]
mode select(10):
Descriptor format, current; Sense key: Data Protect
Additional sense: Access denied - no access rights
  Descriptor type: Information: 0x0000000000000000
  Descriptor type: Field replaceable unit code: 0x0
  Descriptor type: Vendor specific [0x80]
    00 00 00 00 00 20 00 00 00 00 00 00 00 00
MODE SELECT command: Data protect, type: sense key; write protected media?

Results are same if use /dev/sda or /dev/sg0 for dive up until last drive on list.
sg_format -v --format --six --size=512 /dev/sda and sg_format -v --format --six --size=520 /dev/sda both give same results as above.

Looking over the posts a few have referred to setblocksize but I see no package in both debian or in rhel systems that will allow this install. setblocksize -b512 -t1200 /dev/sg0 if run does nothing as the package does not exist.




When sg_sanitize -B -v /dev/sg0 is run this is the output.

sg_sanitize -B -v /dev/sg0
    NETAPP    X317_SMBPE06TA07  NA00   peripheral_type: disk [0x0]
      PROTECT=0
      Unit serial number: ZAD2KJTW
      LU name: 5000c50094bfb1db

A SANITIZE will commence in 15 seconds
    ALL data on /dev/sg0 will be DESTROYED
        Press control-C to abort

A SANITIZE will commence in 10 seconds
    ALL data on /dev/sg0 will be DESTROYED
        Press control-C to abort

A SANITIZE will commence in 5 seconds
    ALL data on /dev/sg0 will be DESTROYED
        Press control-C to abort
Sanitize:
Descriptor format, current; Sense key: Illegal Request
Additional sense: Invalid field in cdb
  Descriptor type: Sense key specific: Field pointer:
        Error in Command: byte 1 bit 4
  Descriptor type: Field replaceable unit code: 0x1
  Descriptor type: Vendor specific [0x80]
    00 00 00 00 ff 20 00 00 00 00 00 00 00 00
Sanitize failed: Illegal request, type: sense key, apart from Invalid opcode

Their is also a reference to sedutil-cli but again not seeing the program on any repo.


As with everyone here no data on the drives just tying to remove the 520 byte sectors so that the drive can be made useful