hdparm power-up in standby vs LSI SAS 2308 controller

[madbrain]

I have an LSI SAS controller and 8 WD140EDFZ drives - WD 14TB, shucked. I use them in Raidz2 under Ubuntu 20.
Yesterday, I used hdparm to tune the power savings of the drives.
I achieved some great results, according to my kill-a-watt. Except ... so good that now the drives won't spin up at all.
After a reboot, they don't show up in the device list for my SAS controller at all !
I booted to Win10, same result - no disks. Getting into the Avago BIOS, no devices are found.

I pulled one disk out to try in a external USB SATA dock. I hear the drive spin up. It shows temporarily in device manager under Windows, for a few seconds.
Then, in very short order, disappears from that list, and I hear it spin down. I figured I need to send a command to disable the "power up standby" mode to the drive. But since the drive never shows up in the OS, I don't know how to send that command.

Ideally, in the old days, there would have been a jumper on the drive to reset this silliness, but I don't see any. There are just two pins with no label, which I'm guessing are an LED, but it could be anything. I tried shorting those pins (with the drive powered off) with a screwdriver, but unfortunately, that didn't help.

I was really afraid I had bricked all 8 drives with this feature.

My next attempt was to insert the drive in a SATA dock attached to the motherboard SATA controller.
Fortunately, the drive spun up, and showed up in the BIOS. And then, it was visible in both Linux and Windows 10. No data was lost. Phew !
The fix was simply :
hdparm -s 0 /dev/sda

FYI, what broke it was :
sudo hdparm --yes-i-know-what-i-am-doing -s 1 /dev/sda

And I learned about it from :

https://linuxconfig.org/change-hard-drive-s-sleep-standby-mode-timer-to-reduce-power-consumption

To turn standby mode on, we can use the -s 1 option. We must also supply the --yes-i-know-what-i-am-doing flag. As funny as that sounds, it truly is necessary to supply it in the hdparm command syntax, otherwise the command won’t work. This is done for extra verification that your BIOS and HDD firmware indeed support standy mode. These days, that should nearly always be the case.

So, clearly, neither my USB SATA dock nor the LSI SAS controller I have support this mode.
One is a 9207-8i and the other a 9207-4i4e .
I updated the BIOS with a 2015 BIOS version when I got them used a while back.
I guess it doesn't support this mode, surprisingly.
Is there an update ?

For now, I have to connect all drives to the motherboard SATA controller to revive them.

 

[Stephan]

You may be looking for the "-y" switch instead. Be aware, these modern disks support the old standby timer schemes little to not at all. You may find that hdparm cannot through the -S switch configure automatic drive standby entry/exit. These are rebadged 7200rpm enterprise drives which only support EPC and not the old APM. Read here: openSeaChest/openSeaChest_PowerControl.201.txt at develop · Seagate/openSeaChest

Also be aware, for ZFS use a very new kernel (up to what ZFS allows) because there might be a bug in connection with ZFS + SAS-Controller + old kernels where error messages appear in dmesg if you put drives into standby and they take a couple of seconds to wake back up again. Read ZFS io error when disks are in idle/standby/spindown mode · Issue #4713 · openzfs/zfs and 0.6.5.6 - I/O timeout during disk spin up · Issue #4638 · openzfs/zfs.

Personally I am using a patched hd-idle (GitHub - adelolmo/hd-idle: Hard Disk Idle Spin-Down Utility) on SATA links (no SAS) using this patch:

diff --git a/hdidle.go b/hdidle.go
index f7e7a33..6d1e9c8 100644
--- a/hdidle.go
+++ b/hdidle.go
@@ -24,12 +24,15 @@ import (
        "log"
        "math"
        "os"
+       "os/exec"
        "time"
 )
 
 const (
        SCSI       = "scsi"
        ATA        = "ata"
+       HDPARM     = "hdparm"
+       SEACHEST   = "seachest"
        dateFormat = "2006-01-02T15:04:05"
 )
 
@@ -219,6 +222,22 @@ func spindownDisk(device, command string) error {
                        return fmt.Errorf("cannot spindown ata disk %s:\n%s\n", device, err.Error())
                }
                return nil
+       case HDPARM:
+        cmd := exec.Command("hdparm", "-y", device)
+        stdoutStderr, err := cmd.CombinedOutput()
+        if err != nil {
+            fmt.Printf("cannot spindown disk %s with hdparm:\n%s\n", device, err.Error())
+            fmt.Printf("%s\n", stdoutStderr)
+        }
+        return nil
+       case SEACHEST:
+        cmd := exec.Command("openSeaChest_PowerControl", "-d", device, "--transitionPower", "standby_z", "-q")
+        stdoutStderr, err := cmd.CombinedOutput()
+        if err != nil {
+            fmt.Printf("cannot spindown disk %s with openSeaChest_PowerControl:\n%s\n", device, err.Error())
+            fmt.Printf("%s\n", stdoutStderr)
+        }
+        return nil
        }
        return nil
 }
diff --git a/main.go b/main.go
index 3838629..292516c 100644
--- a/main.go
+++ b/main.go
@@ -126,18 +126,18 @@ func main() {
                case "-c":
                        command, err := argument(index)
                        if err != nil {
-                               fmt.Println("Missing command_type after -c. Must be one of: scsi, ata.")
+                               fmt.Println("Missing command_type after -c. Must be one of: scsi, ata, hdparm, seachest.")
                                os.Exit(1)
                        }
                        switch command {
-                       case SCSI, ATA:
+                       case SCSI, ATA, HDPARM, SEACHEST:
                                if deviceConf == nil {
                                        config.Defaults.CommandType = command
                                        break
                                }
                                deviceConf.CommandType = command
                        default:
-                               fmt.Printf("Wrong command_type -c %s. Must be one of: scsi, ata.", command)
+                               fmt.Printf("Wrong command_type -c %s. Must be one of: scsi, ata, hdparm, seachest.", command)
                                os.Exit(1)
                        }

and the following config for /etc/conf.d/hd-idle:

HD_IDLE_OPTS="\
-i 0 -c seachest \
-a /dev/disk/by-id/ata-WDC_WD140EDFZ-11A0VA0_XXXXXXXX -i 900 \
-a /dev/disk/by-id/ata-WDC_WD140EDFZ-11A0VA0_XXXXXXXX -i 900 \
-a /dev/disk/by-id/ata-WDC_WD140EDFZ-11A0VA0_XXXXXXXX -i 900 \
-a /dev/disk/by-id/ata-WDC_WD140EDFZ-11A0VA0_XXXXXXXX -i 900 \
-a /dev/disk/by-id/ata-WDC_WD140EDFZ-11A0VA0_XXXXXXXX -i 900 \
-a /dev/disk/by-id/ata-WDC_WD140EDFZ-11A0VA0_XXXXXXXX -i 900 \
-a /dev/disk/by-id/ata-WDC_WD140EDFZ-11A0VA0_XXXXXXXX -i 900 \
-a /dev/disk/by-id/ata-WDC_WD140EDFZ-11A0VA0_XXXXXXXX -i 900"

I have built and installed GitHub - Seagate/openSeaChest: Cross platform utilities useful for performing various operations on SATA, SAS, NVMe, and USB storage devices. to enable hd-idle use the advanced EPC commands instead of disfunctional APM for my drives.

Recently there have been some fixes to hd-idle to better mimic what hdparm is doing, so if you are lucky, pure hd-idle in very latest git-version might just work using the standard -c ata or -c scsi, without any patch. If you find disks are not going into standby reliably(!), and there is really no disk I/O from Samba, NFS or whatever, try above patch and -c seachest.

 

[Stephan]

Addendum

Here is a script that will preset EPC in any and all "WDC WD140" drives present in the system:

#!/bin/sh

O="openSeaChest_PowerControl"
DRIVES=$(openSeaChest_PowerControl -s -q | awk '/^ATA.*WDC WD140/ { print $2 }')

for d in $DRIVES; do
    $O -d $d --setAPMLevel 1
    $O -d $d --EPCfeature enable

    $O -d $d --idle_b default
    $O -d $d --idle_b enable

    $O -d $d --standby_z 1800000
    $O -d $d --standby_z enable

    $O -d $d --showEPCSettings -q
done

Shouldn't be used as is, but you get the idea how to reset something if a drive is misbehaving.

Sample output from one of my WD140EDFZ drives:

# openSeaChest_PowerControl -d /dev/sg8 --showEPCSettings -q
Attempting to open handle "/dev/sg8"

===EPC Settings===
        * = timer is enabled
        C column = Changeable
        S column = Savable
        All times are in 100 milliseconds

Name       Current Timer Default Timer Saved Timer   Recovery Time C S
Idle A     *20           *20           *20           1             Y Y
Idle B     *6000         *6000         *6000         10            Y Y
Idle C      0             0             0            36            Y Y
Standby Y   0             0             0            36            Y Y
Standby Z  *18000         0            *18000        100           Y Y

PDF https://documents.westerndigital.co...t-manual-ultrastar-dc-hc530-sata-oem-spec.pdf page 109 is also recommended to see what is supported and what condition or timer means what.

 

[madbrain]

Thanks, I will take a look at that. For now, I got autosuspend to work for the whole NAS. I set it based on network bandwidth like this in /etc/autosuspend.conf :

[check.NetworkBandwidth]
enabled = true
interfaces = enp3s0
threshold_receive=600000
threshold_send=600000

This is enough of a threshold to suspend even if my VNC session from the LAN is left on.
It's enough for backups to the NAS to not be interrupted by suspend, also. Haven't really tried streaming from it yet, but I think it should be OK.

FYI, lowest I got the NAS at is 77W idle on the kill-a-watt, with all 8 drives manually suspended (hdparm -y). With all of them on, it's about 117W.

This is with the Z170-AR motherboard, i5-6600k CPU, 2 x 16GB RAM running at 1.2V, EPU mode (power saver) enabled in the BIOS. There are two LSI 9207 PCIe 3.0 8x cards, and one Aquantia AQC-107 NIC. PSU is a Raidmax RX-1200AE. Case is HAF-XM and has a number of large fans. Not sure if there is a way to get power consumption lower.