Which Software RAID?

[TeeJayHoward]

OS: Windows (Server 2025 or 11 Workstation)
Mobo: WRX90E
Drives: 16x 4TB Crucial P3 Plus QLC Gen4 M.2
HBA: N/A, kinda. 4x ASUS Hyper M.2 Gen5 adapters

Goal: Maximum throughput with a minimum of ~55TB usable capacity (Basically all drives in one pool, no parity). No need for uptime, no need for redundancy, no need for anything like that. If the data gets lost when a drive dies, that's okay. I just need speed. ALL THE SPEED. I wanna make a 400GbE NIC cry.

Options:

1. Hyper-V w/ DDA via IOMMU passthrough of disks feeding a RHEL10 VM running ZFS
2. Just Stripe the disks through Disk Management
3. Storage Spaces
4. _____?

 

[gea]

Use Windows Server (needed for SMB Direct/RDMA), the key for ultra high performance
Just stripe disks or use Storage Spaces with ntfs/ReFS with striped (no parity, AUS >64K) Spaces over all disks.
With Storage Spaces you can add Spaces with redundancy when needed

Avoid a Storage VM, you need resources for two systems and must maintain both
Avoid ZFS for max performance (and when, consider OpenZFS on Windows, still beta but already quite good)

 

[BoredSysadmin]

I don't know what is your primary storage usage is, but due to no need in any redundancy, I suspect sustained sequential writes.
If so, after you won't saturate the 400gb nic with 16 of these or even 160 of them.
QLC has notoriously bad sustained write speeds - https://www.tomshardware.com/reviews/crucial-p3-plus-ssd-review-capacity-on-the-cheap/2
You'd be lucky to get even 200MBps from a single drive after cache exhaustion.

For smaller bursty writes, you'd probably be ok and see high speeds, maybe even saturate 400gbps nic

 

[TeeJayHoward]

You'd be lucky to get even 200MBps from a single drive after cache exhaustion.

Nope, nope, nope, that's not what I want to hear at all. You have my blessing to go ahead and change reality so that this statement is no longer true.

If I'm getting 3.2GB/s out of this pool, I'm considering it a failure. I was hoping for 10x that. And I really, REALLY don't wanna try to find the ~15K in my budget to swap these out with 9100 PRO or T710 8TB sticks.

 

[BoredSysadmin]

8x of 8TB WD BLACK SN8100 drives should do the trick. Only $8,000. Some other techs you'd have to use are RMDA, S2D, and a big CPU like TR Pro 7995WX .