Half-decent M.2 in the midst of the AIpocalypse

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,490
579
113
To cut a long story short, it's looking like Supermicro are no longer willing to supply peons like me with anything so advanced as the H13SAE-MF, despite having an order in for nearly a year, so I'm being forced to re-evaluate my choice of motherboard for the next five seven years of the wug primary server.

I had an eye on the H13SAE-MF precisely because it'd let me use my healthy stash of Micron 7450 M.2 22110's as boot+scratch drives as there didn't seem to be anything else on the market that would let me use them at the time. But now it looks like I might be able to buy something that's 2xM.2 2280 at best.

The Micron 7450s are seemingly now as rare as rocking horse shit and essentially unobtanium here in the UK even if you're wishing to part with your first-born and a loved-one's favourite kidney. So, what are StHers experiences of other enterprise-esque M.2 drives that might still be available for purchase by mere mortals?

Or are people just generally getting on with using consumer M.2's for boot/scratch purposes? And if so are there any pitfalls I might need to be aware of?

I've been a fan of Micron for a long time, mainly for their excellent reliability, firmware support, and linux support; even if they weren't always the fastest chimps off the blocks (but as far as I'm concerned I've been happy with 500+ random IOPS for boot drives for the last 15 years). But that goodwill also appears to be a window that's rapidly closing. And similarly I think Supermicro are rapidly falling in to my "never again" bucket, but similarly I think that's largely their intention, because why deal with me when they can deal with someone who can afford to pay an order of magnitude more for the same chips?

I'd rather buy retail if possible, since grubbing through a hopeful firmware dump from some OEMs bitbucket and/or having to run a bunch of esoteric to commands to get an unsupported drive maybe kinda hopefully working isn't exactly my cuppa, but I'm willing to entertain any option at this point (if for no other reason it might be helpful to others reading this thread). And of course I'm most awfully aware of the fact I never really considered M.2 to be anywhere close to an enterprise format in the first place (no hot-swap? WTF?!) but we are where we are. I just want something reliable and low-power enough to boot and run an occasional heavy sequential read workload without having to change my entire hardware stack, or having to deal with stupid firmware quirks.

Apologies for using the horrible term "AIpocalypse" but it was either that or a TikTok video with a voluntary mandatory likesubscribe to a poorly-written folk song. I'll leave that one dangling as a vague threat.
 

nexox

Well-Known Member
May 3, 2023
2,087
1,061
113
Depending on the board you might be able to get an adapter that will turn or flip the m.2 socket to make 22110s viable, depends on what else is in the way. 2280s with PLP are just not that common, but there are some intermediate data center boot / industrial options that don't have capacitors but which do exchange some benchmark performance for properly implemented sync calls and "PLP for data at rest", which should get you most of the reliability - I believe Micron made a few but I don't remember the part numbers.
 

tinfoil3d

QSFP28
May 11, 2020
962
474
63
Japan
If the goal is to ust use your own 22110 you have wide selection of options of PCIe cards not limited to asus bifurcation ones, but also something like single pcie slot cards to M.2
Check this out https://forums.servethehome.com/index.php?threads/use-m-2-drive-bays-for-u-2-hot-swap.19818/
or this Does m2 NVMe to U.2 (SFF-8639) Adapter Card compatible with U.2 hot plug?
There is in fact a ton of options to use M.2 ports as well, like M.2 to 8643 cards: U.2 to M.2 Adapter for U.2 NVMe SSD
And far more if we touch aliexpress and grab some kapton tape.
 
  • Like
Reactions: nexox

TrevorH

Active Member
Oct 25, 2024
311
135
43
I found at least 2 UK sellers with H13SAE-MF boards in stock on the first page of my Google search - is there something wrong with those? Other than the price :eek:
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,490
579
113
TBH I'd rather just have two half-decent M.2 cards to use as boot drives rather than try jury-rigging an adapter for my 22110s, and similarly I don't really fancy the idea of having to switch to U.2|3 drives just yet nor waste any of my PCI slots.

I see Kingston have their own "DC" line with drives like the DC2000B, although it looks like it tops out at 960GB and has zero linux support.

Samsung (ugh) also have their PM9*, but when I last looked at them getting firmware updates even for windows was a stretch.

I see Phison appear to have also brought out their own line of enterprise M.2 but I've no idea if they're any good or not.

I found at least 2 UK sellers with H13SAE-MF boards in stock on the first page of my Google search - is there something wrong with those? Other than the price :eek:
For my retailer at least, SM are now refusing to ship them the board for my order unless I also buy a CPU bundled with it; the CPU I already own (bought at the same time as the motherboard) and I have to buy it through Supermicro (not the retailer) at a substantial markup from what it's available at even today. I suspect this is going to be the last Supermicro product I attempt to buy.
 

TrevorH

Active Member
Oct 25, 2024
311
135
43
I saw the H13SAE on lambda-tek and on a site I am not sure I've seen before called ServerO. They were both around £470+VAT and one said they had 15 in stock and the other 19.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,490
579
113
Lambda-Tek is my retailer in question; I've already pulled them up on not telling people about Supermicro's supposed onerous requirements on the product page. But that part hardly came as a surprise since getting hold of Supermicro kit at retail at all has gotten progressively more impossible over the last few years, with many companies giving up stocking their stuff entirely.

I've also not hard of Servero either, but they say they've been around since 2003.
 

louie1961

Well-Known Member
May 15, 2023
669
332
63
I have resorted to Micron 7300 Pro drives, as I don't mind Gen3 performance and they can be had at almost reasonable prices (by comparison) in the 2280 format


I also have had really good luck with SK Hynix P41 platinum drives. They are Gen 4, no PLP, but decent endurance (0.4 dwpd). I use them for a fast ZFS pool in proxmox. After a year running ZFS in proxmox they are at 0% wear out.

I wanted that AM5 supermicro board as well. but the cost of ECC DDR5 ram put me off of it. I ended up with a Asrock Rack X570D4U and a Ryzen 7 5800XT and couldn't really be happier.
 

nexox

Well-Known Member
May 3, 2023
2,087
1,061
113
although it looks like it tops out at 960GB and has zero linux support.
What does an NVMe drive need for Linux support? I've run all kinds and never had an issue because pretty much everything is standardized, even the firmware update process, though of course getting the firmware binary isn't always easy.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,490
579
113
I have resorted to Micron 7300 Pro drives, as I don't mind Gen3 performance and they can be had at almost reasonable prices (by comparison) in the 2280 format


I also have had really good luck with SK Hynix P41 platinum drives. They are Gen 4, no PLP, but decent endurance (0.4 dwpd). I use them for a fast ZFS pool in proxmox. After a year running ZFS in proxmox they are at 0% wear out.

I wanted that AM5 supermicro board as well. but the cost of ECC DDR5 ram put me off of it. I ended up with a Asrock Rack X570D4U and a Ryzen 7 5800XT and couldn't really be happier.
Yeah I have a couple of 7300s myself but should've bought more when I had the chance. Ditto my trusty Intel P4101s.

I've toyed with the idea of using consumer drives (I have a bunch of Crucial T500s doing decent duty as RAID1 pairs), but the SK hynix drives aren't very easy to get hold of here, but the idle power consumption on some review I read of the P41 and P51 drives was pretty awful.

I saw the hardware squeeze coming a way off and stocked up on a lot of stuff in advance (including the RAM) but my strategy failed to take in to account I might need to use 2280s rather than 22110's as I didn't think the motherboard would be the crux of the problem...

What does an NVMe drive need for Linux support? I've run all kinds and never had an issue because pretty much everything is standardized, even the firmware update process, though of course getting the firmware binary isn't always easy.
Aye, it's mostly the "getting hold of the firmware" part, which usually means running the windows application through a proxy or suchlike and trying to determine the firmware URLs; often doable (have done it myself with some crucial and WD consumer drives), but a complete PITA, and I'd much rather just deal with companies that realise linux exists.

There's also plenty of hearsay of companies refusing to honour support/warranty requests without going through their shitty windows application to get the SMART logs.
 

nexox

Well-Known Member
May 3, 2023
2,087
1,061
113
There's also plenty of hearsay of companies refusing to honour support/warranty requests without going through their shitty windows application to get the SMART logs.
I guess I didn't consider that, I have only ever had a handful of SSDs with an active warranty, sounds like a great reason for me to continue to avoid consumer NVMe.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,490
579
113
I don't think they have anything approaching PLP, but Sandisk's new "NAS"-branded NVME models (PDF) look like they might fit the bill; TLC flash and seemingly with decent durability, plus I'm reasonably certain I know how to obtain and flash the firmwares outside of their god-awful "we only support windows" dashboard application (but thankfully the XML repos are fairly easy to discover and parse).

7.68TB in an M.2 2280 is certainly an eye-opener, even if it's for ruinous money.

But having steered clear of prosumer SSDs for so long, is there any reason to stay away from the WD/Sandisk camp? I sure had a merry time helping a mate unfuck an SA510 many moons ago (IIRC they shipped with essentially a self-destructing firmware and needed a degree of cajoling to fix), but do they have a history of this? Any other StHers using relatively recent WD/Sandisk offerings?
 
  • Like
Reactions: nexox

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,490
579
113
Well in the end I bit the bullet and took a punt on one of the new Sandisk "NAS" 800 drives. Hardware-wise I suspect they're near-identical to the consumer-level Optimus GX Pro 8100 series, they certainly look it (lspci reports it as "Sandisk Corp WD PC SN8050S / WD_BLACK SN8100 NVMe SSD"). Bought direct from Sandisk as they seem to be the only people with them in stock here in rightpondia at the moment.

There don't seem to be any reviews out yet, so in case anyone's interested, here's what SMART and friends look like:
Code:
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-7.1.8+deb13-amd64] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       Sandisk NAS 800 NVMe 3840GB
Serial Number:                      
Firmware Version:                   64109100
PCI Vendor/Subsystem ID:            0x15b7
IEEE OUI Identifier:                0x001b44
Total NVM Capacity:                 3,840,755,982,336 [3.84 TB]
Unallocated NVM Capacity:           0
Controller ID:                      1
NVMe Version:                       2.0
Number of Namespaces:               1
Namespace 1 Size/Capacity:          3,840,755,982,336 [3.84 TB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            001b44 8b437d82b6
Local Time is:                      Thu Sep 17 19:39:12 2026 BST
Firmware Updates (0x14):            2 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x0055):     Comp DS_Mngmt Sav/Sel_Feat Timestmp
Log Page Attributes (0x3e):         Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg Pers_Ev_Lg Log0_FISE_MI
Maximum Data Transfer Size:         256 Pages
Warning  Comp. Temp. Threshold:     84 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x92):        NA_Fields NP_Fields *Other*

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     9.50W    7.66W       -    0  0  0  0        0       0
 1 +     5.50W    4.02W       -    1  1  1  1        0       0
 2 +     3.00W    2.79W       -    2  2  2  2        0       0
 3 -   0.0490W       -        -    3  3  3  3     1500    8500
 4 -   0.0030W       -        -    4  4  4  4     1400   48500

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0
 1 -    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        33 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    8 [4.09 MB]
Data Units Written:                 0
Host Read Commands:                 78
Host Write Commands:                0
Controller Busy Time:               0
Power Cycles:                       1
Power On Hours:                     0
Unsafe Shutdowns:                   0
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               45 Celsius
Temperature Sensor 2:               30 Celsius
Temperature Sensor 3:               37 Celsius

Error Information (NVMe Log 0x01, 16 of 256 entries)
No Errors Logged

Self-test Log (NVMe Log 0x06)
Self-test status: No self-test in progress
No Self-tests Logged
Power states:
Code:
root@testwug:~# nvme get-feature /dev/nvme0n1 -f 0x0c -H
get-feature:0x0c (Autonomous Power State Transition), Current value:0x00000001
       	Autonomous Power State Transition Enable (APSTE): Enabled
       	Auto PST Entries       	.................
       	Entry[ 0]
       	.................
       	Idle Time Prior to Transition (ITPT): 100 ms
       	Idle Transition Power State   (ITPS): 3
       	.................
       	Entry[ 1]
       	.................
       	Idle Time Prior to Transition (ITPT): 100 ms
       	Idle Transition Power State   (ITPS): 3
       	.................
       	Entry[ 2]
       	.................
       	Idle Time Prior to Transition (ITPT): 100 ms
       	Idle Transition Power State   (ITPS): 3
       	.................
       	Entry[ 3]
       	.................
       	Idle Time Prior to Transition (ITPT): 2000 ms
       	Idle Transition Power State   (ITPS): 4
       	.................
       	Entry[ 4]
       	.................
       	Idle Time Prior to Transition (ITPT): 0 ms
       	Idle Transition Power State   (ITPS): 0
       	.................
Other NVME features:
Code:
root@testwug:~# nvme get-feature /dev/nvme0n1 -H
get-feature:0x01 (Arbitration), Current value:0x00000006
       	High Priority Weight   (HPW): 1
       	Medium Priority Weight (MPW): 1
       	Low Priority Weight    (LPW): 1
       	Arbitration Burst       (AB): 64
get-feature:0x02 (Power Management), Current value:0x00000004
       	Workload Hint (WH): 0 - No Workload
       	Power State   (PS): 4
get-feature:0x03 (LBA Range Type), Current value:00000000
       	Number of LBA Ranges (NUM): 1
       	type       : 0 - Reserved
       	attributes : 0x1 - LBA range may be overwritten, LBA range should be visible from the OS/EFI/BIOS
       	slba       : 0
       	nlb        : 0x1bf1f72af
       	guid       : 00000000000000000000000000000000
get-feature:0x04 (Temperature Threshold), Current value:0x00000165
       	Temperature Threshold Hysteresis(TMPTHH): -273 °C (0 K)
       	Threshold Type Select         (THSEL): 0 - Over Temperature Threshold
       	Threshold Temperature Select (TMPSEL): 0 - Composite Temperature
       	Temperature Threshold         (TMPTH): 84 °C (357 K)
get-feature:0x05 (Error Recovery), Current value:00000000
       	Deallocated or Unwritten Logical Block Error Enable (DULBE): Disabled
       	Time Limited Error Recovery                          (TLER): 0 ms
get-feature:0x06 (Volatile Write Cache), Current value:0x00000001
       	Volatile Write Cache Enable (WCE): Enabled
get-feature:0x07 (Number of Queues), Current value:0x00fd00fd
       	Number of IO Completion Queues Allocated (NCQA): 254
       	Number of IO Submission Queues Allocated (NSQA): 254
get-feature:0x08 (Interrupt Coalescing), Current value:00000000
       	Aggregation Time     (TIME): 0 usec
       	Aggregation Threshold (THR): 1
get-feature:0x09 (Interrupt Vector Configuration), Current value:0x00010000
       	Coalescing Disable (CD): True
       	Interrupt Vector   (IV): 0
get-feature:0x0a (Write Atomicity Normal), Current value:00000000
       	Disable Normal (DN): False
get-feature:0x0b (Async Event Configuration), Current value:0x0000000f
       	Discovery Log Page Change Notices                         : Do not send async event
       	Host Discovery Log Page Change Notification               : Do not send async event
       	AVE Discovery Log Page Change Notification                : Do not send async event
       	Pull Model DDC Request Log Page Change Notification       : Do not send async event
       	Zone Descriptor Changed Notices                           : Do not send async event
       	Allocated Namespace Attribute Notices                     : Do not send async event
       	Reachability Group                                        : Do not send async event
       	Reachability Association                                  : Do not send async event
       	Temperature Threshold Hysteresis Recovery                 : Do not send async event
       	Normal NVM Subsystem Shutdown                             : Do not send async event
       	Endurance Group Event Aggregate Log Change Notices        : Do not send async event
       	LBA Status Information Notices                            : Do not send async event
       	Predictable Latency Event Aggregate Log Change Notices    : Do not send async event
       	Asymmetric Namespace Access Change Notices                : Do not send async event
       	Telemetry Log Notices                                     : Do not send async event
       	Firmware Activation Notices                               : Do not send async event
       	Namespace Attribute Notices                               : Do not send async event
       	SMART / Health Critical Warnings                          : Send async event
get-feature:0x0c (Autonomous Power State Transition), Current value:0x00000001
<snipped for brevity>
get-feature:0x0e (Timestamp), Current value:00000000
       	The timestamp is : 1,789,671,505,255 (Thu 17 Sep 2026 19:58:25 BST BST)
       	The Timestamp field was initialized with a Timestamp value using a Set Features command.
       	The controller counted time in milliseconds continuously since the Timestamp value was initialized.
get-feature:0x10 (Host Controlled Thermal Management), Current value:00000000
       	Thermal Management Temperature 1 (TMT1) : 0 K (-273 °C)
       	Thermal Management Temperature 2 (TMT2) : 0 K (-273 °C)
get-feature:0x11 (Non-Operational Power State Config), Current value:0x00000001
       	Non-Operational Power State Permissive Mode Enable (NOPPME): True
get-feature:0x16 (Host Behavior), Current value:00000000
       	Advanced Command Retry Enable (ACRE): True
       	Extended Telemetry Data Area 4 Supported (ETDAS): False
       	LBA Format Extension Enable (LBAFEE): False
       	Host Dispersed Namespace Support (HDISNS) : Disabled
       	Copy Descriptor Format 2h Enabled (CDF2E) : False
       	Copy Descriptor Format 3h Enabled (CDF3E) : False
       	Copy Descriptor Format 4h Enabled (CDF4E) : False
get-feature:0x17 (Sanitize), Current value:00000000
       	No-Deallocate Response Mode (NODRM) : 0
get-feature:0x1b (Power Loss Signaling Config), Current value:00000000
       	Power Loss Signaling Mode (PLSM): not enabled
get-feature:0x7d (Enhanced Controller Metadata), Current value:00000000
       	Num Metadata Element Descriptors: 0
get-feature:0x7e (Controller Metadata), Current value:00000000
       	Num Metadata Element Descriptors: 0
get-feature:0x7f (Namespace Metadata), Current value:00000000
       	Num Metadata Element Descriptors: 0
get-feature:0x85 (Boot Partition Write Protection Config), Current value:0x00000012
Whilst it looks like they can be easily converted to 4Kn, output from nvme suggests the performance is equal for both (most of my other NVME's list 4K as "best"):
Code:
root@testwug:~# nvme id-ns -H /dev/nvme0n1|grep 'LBA Format'
  [6:5] : 0    	Most significant 2 bits of Current LBA Format Selected
  [3:0] : 0    	Least significant 4 bits of Current LBA Format Selected
LBA Format  0 : Metadata Size: 0   bytes - Data Size: 512 bytes - Relative Performance: 0 Best (in use)
LBA Format  1 : Metadata Size: 0   bytes - Data Size: 4096 bytes - Relative Performance: 0 Best
From grepping through the binary of Sandisk's dashboard application, it looks like the firmware metadata and downloads are to be found at:

...although there doesn't seem to be anything published there yet for the SDSNP* model numbers (nor on the [older? Now defunct?] WD-branded firmware repo) so I guess we'll need to see what happens with future firmware updates.
 
Last edited:
  • Like
Reactions: nexox