Power Efficient Enterprise SSDs (SATA or NVMe)

[nautilus7]

Hi, I am looking for info about which enterprise SSD models, either SATA or NVMe, support power efficiency features like ASPM (for the PCIe/NVMe drives) or DevSleep (for the SATA ones), when on idle. By enterprise, I mean drives with at least PLP protection and pottentially increased write endurance.

Do you know any such SSDs? Would you like to make a list of models that support these features? For SATA a screenshot of CrystalDiskInfo (or equivalent command in linux) would be nice and for PCIe/NVMe drives the output of the lspci (-vv) command to show the ASPM enabled/disabled status and L0, L1 exit latency would be needed.

SATA drives
============
Seagate Nytro 1351 series (to be confirmed)
Seagate Nytro 1551 series
Seagate Nytro 1361 series (to be confirmed)


NVMe Drives
============

 

[nautilus7]

I'll start with Seagate Nytro 1551 SSD (1.92TB)

 

[CyklonDX]

most enterprise grade hardware either goes either full support of power-saving nonsense or full powerformance demanded no powersaving/sleeping allowed. While having "support" for aspm it does not mean it will ever be used on enterprise hardware its likely fixed in place by firmware - the support is mainly there because of streamlining with consumer line.

Most enterprise nvme's (m.2/u.2/u.3) from hynix, micron, and intel will have disabled aspm.
Toshiba/Kioxia will have it enabled *(but real support for aspm will be dependent on backplane)
Samsung is a mixed bag.

All sas devices will have disabled power saving features. *(the manual commands likely will be working but firmware will be blocking disk to go there naturally without user input.)

 

[ca3y6]

Not answering your question but might still help. I compiled the specs of enterprise SSDs, including, when available, idle and full power consumption. So that should at least narrow your scope. Provided with no guarantee of correctness.

https://zbutil.blob.core.windows.net/public/ssd%20db%20v3.zip

 

[Vorwrath]

I'm not sure how relevant that is for SATA SSDs really. I've got some Micron 5300 Pro drives, which I think do not support sleep. But the idle power draw on them is only something like 1.5W. You'd need to have a pretty extreme amount of those SSDs before you'd find any noticable cost saving from putting them to sleep.

 

[FrankTL]

How do aspm and other deep sleep capabilities affect the various background processes that maintain ssd drive performance and health over time, such as garbage collection after trim during idle?

It was my understanding that the deepest power saving modes can prevent these from working properly on some (most? all?) drives.

 

[jode]

Not answering your question but might still help. I compiled the specs of enterprise SSDs, including, when available, idle and full power consumption. So that should at least narrow your scope. Provided with no guarantee of correctness.

https://zbutil.blob.core.windows.net/public/ssd%20db%20v3.zip

This is quite neat! I tried cross-referencing against a simple eBay search and found a bunch of models not yet in your list. Any way to help you build that out? Do you have another thread for this?

 

[ca3y6]

Sure, let me build a quick UI for it. I will get back to you in a few days

 

[CyklonDX]

How do aspm and other deep sleep capabilities affect the various background processes that maintain ssd drive performance and health over time, such as garbage collection after trim during idle?

It was my understanding that the deepest power saving modes can prevent these from working properly on some (most? all?) drives.

for nvme's just like any other pcie device depending power state it will slow down refreshes on pcie port so it will act as lower pcie lane GT/s.
i.e. nvme pcie x4 and during power savings modes it might go to pcie x2, and even pcie x1 before it enters full sleep state. Any requests over certain limit will cause it to 'wake up' and trim or any other function won't be stopped by lower power modes - they will be just slower if low power modes are enforced - else it will wake up to its full speed on request *you will only loose latency on access time due to waking up from lower power states.

on sata ssd's its typically sleep / awake only *(there are some more excotic sas ssd's that internally function as pcie device).

 

[FrankTL]

for nvme's just like any other pcie device depending power state it will slow down refreshes on pcie port so it will act as lower pcie lane GT/s.
i.e. nvme pcie x4 and during power savings modes it might go to pcie x2, and even pcie x1 before it enters full sleep state. Any requests over certain limit will cause it to 'wake up' and trim or any other function won't be stopped by lower power modes - they will be just slower if low power modes are enforced - else it will wake up to its full speed on request *you will only loose latency on access time due to waking up from lower power states.

on sata ssd's its typically sleep / awake only *(there are some more excotic sas ssd's that internally function as pcie device).

The point I was trying to make (probably should have been clearer):
It's essential that after a trim, the ssd drive can run various housekeeping processes that help to maintain the drive's long term health and performance - executed when the drive is idle in order not to negatively impact write/read performance.

When you switch the drive automatically to some deep sleep mode on idle, these housekeeping processes are prevented from running and can negatively impact an ssd drive's health and performance over time.

 

[CyklonDX]

the process is controller by disk own chip and firmware. I doubt it will allow itself p0 when its doing important things.

 

[FrankTL]

the process is controller by disk own chip and firmware. I doubt it will allow itself p0 when its doing important things.

You'd be surprised then.

Here's one manufacturer telling you not to let the drive enter deep sleep modes for this exact reason: My SSD Used To Be So Much Faster But Has Recently Slowed Down ... What Happened?

I bet other drives work similarly.

 

[CyklonDX]

This is not trim but another functionality *(i presume micron patented) that runs behind the scenes on the firmware. If you put your disk into sleep/power saving mode, and schedule TRIM to run, it will wake the disk and run it. If AGC or TRIM is already happening the disk won't go into power saving mode either.

Active Garbage Collection they are referencing here - redistributes used data blocks equally over least used blocks - by itself its not superior to TRIM in any way, and its really only useful for hw RAID systems where TRIM command will not work;
For normal user you will never need to have AGC running. As for it to mean anything you would need it if you are running hw raid solution, and/or are constantly writing during its operation then powering off, and/or again trim never runs for whatever the reason.

 

[FrankTL]

My understanding is the reverse - the garbage collection is only really effective with trim, as the drive doesn't know otherwise which blocks have become freed up by the operating system.

Once this is known (by issuing TRIM), the drive can run housekeeping tasks to move data to these freed up block in order for wear leveling, etc.
These housekeeping tasks need not necessarily run immediately after the TRIM - the drive may wanrt to wait till a bunch of blocks have become freed up, or run them only X times per day.
In deep sleep, the drive may not be able to run any of the housekeeping processes at all.

 

[ca3y6]

My understanding is the reverse - the garbage collection is only really effective with trim, as the drive doesn't know otherwise which blocks have become freed up by the operating system.

Enterprise drives are over provisioned to deal with this (a 4TB drive will be sold as a 3.84TB, or 3.2TB if it's a write intensive drive), the delta is spare capacity which enables you to do effective and performance consistent wear levelling even without trim.

 

[FrankTL]

Enterprise drives are over provisioned to deal with this (a 4TB drive will be sold as a 3.84TB, or 3.2TB if it's a write intensive drive), the delta is spare capacity which enables you to do effective and performance consistent wear levelling even without trim.

Trim or over provisioning - the need to run the various housekeeping tasks remains. The question is how the drive's deep sleep prevents/affects this exactly.

I'd love a ssd firmware engineer to chime in on this perceived issue, to confirm it or to put it to rest...

 

[ca3y6]

This is quite neat! I tried cross-referencing against a simple eBay search and found a bunch of models not yet in your list. Any way to help you build that out? Do you have another thread for this?

I created a public version. I will message you directly with some credentials to modify it. If anyone else is interested in helping, please send me a private conversation.

Enterprise SSD Specs Database

 

[nautilus7]

Awesome, thank you!

EDIT: Are these values copied from the drive's datasheet or are tested by you? Can you add some models like Intel s3700/s3710?

 

[ca3y6]

Copied from data sheets. If you would like to contribute you are more than welcome.

[edit] I made an editorial choice to focus on drives > 1TB, so skipped many older, smaller drives, including the S3700.

 

[nautilus7]

Oh, so only drives close to 1tb should be added? Smaller drives are sometimes used as boot drives, so might be useful.

Anyway, your choice and I respect it.