Nonsensical diskspd results on tiered storage spaces volume

OnSeATer

New Member
Jul 31, 2016
17
0
1
42
I created a multi-resiliency volume in storage spaces using the commands quoted at the end of this post.

When I benchmark the resulting volume using Crystal Diskmark I get results that are more or less consistent with my expectations -- about 1,000 MB/s random 4K IO (250,000 IOPS) with 8 threads and a QD of 8. This is running on 2x intel DC P3520 (thank you Patrick for the quick shipping on my purchase!) drives, so that's actually slower than expected, but it's at least in the zone of reasonable.

However, when I benchmark it using diskspd, I get comically terrible results: 2MB/s (500 IOPS). This is basically what I would expect from the HDD tier, so it seems that (unlike Crystal Diskmark) diskspd is bypassing the SSD tier entirely.

The command I'm using is diskspd.exe -t8 -o8 -b4K -c8G -r -Sh w:\testfile.dat

In real world applications, the drives seem to be performing at speeds more consistent with the Crystal Diskmark results, so maybe this is just some quirk with diskspd? Am I just using it wrong? I get the same results even if I drop the -Sh flag from diskspd.

One odd thing is that diskspd gives correct results when I run it on a tiered storage spaces volume without mixed resiliency (e.g. where both the Performance and Capacity/HDD tiers are set to simple). Even odder is that I am certain this ran correctly when I tested it on the same exact hardware prior to reinstalling windows.

I am using Windows Server 2016. The HDD tier consists of a single SATA 8T WD Red drive attached via the onboard SAS controller of a X9DRD-7LN4F motherboard. The CPUs are 2x E5-2667. The SSD drives are, as noted above, 2x 2T Intel DC P3520 drives connected via startech u.2 to pcie adapters.

New-StoragePool -StoragePoolFriendlyName "mainpool" -StorageSubSystemFriendlyName (Get-StorageSubSystem).FriendlyName -PhysicalDisks (Get-PhysicalDisk -CanPool $true) -LogicalSectorSizeDefault 512 -FaultDomainAwarenessDefault PhysicalDisk

Get-Storagepool mainpool | Set-ResiliencySetting -Name Mirror -NumberOfColumnsDefault 1

Get-Storagepool mainpool | Set-ResiliencySetting -Name Simple -NumberOfColumnsDefault 1

New-StorageTier -StoragePoolFriendlyName mainpool -FriendlyName Capacity -MediaType HDD -ResiliencySettingName Simple -NumberOfColumns 1 -PhysicalDiskRedundancy 0 -FaultDomainAwareness PhysicalDisk

New-StorageTier -StoragePoolFriendlyName mainpool -FriendlyName Performance -MediaType SSD -ResiliencySettingName Mirror -NumberOfColumns 1 -PhysicalDiskRedundancy 1 -FaultDomainAwareness PhysicalDisk

New-Volume -StoragePoolFriendlyName mainpool -FriendlyName testthree -FileSystem ReFS -StorageTierFriendlyName Performance, HDD -StorageTierSizes 1TB, 4TB -AccessPath "W:"
 

i386

Well-Known Member
Mar 18, 2016
2,107
554
113
31
Germany
Crystal Diskmark is a gui for diskspd, just check the ressource folder ;D

Have you tested read only or also write tests (-w)?
 

OnSeATer

New Member
Jul 31, 2016
17
0
1
42
I know that's the craziest part!!

I took your suggestion and ran the write tests.

Crystal diskmark shows 500MB/s write speed, which makes sense subject to the same caveats above. Actually (unlike the read speed) it's better than I would have expected given that these are read optimized drives and are only supposed to be able to do 26k write IOPS. So maybe some caching taking place?

diskspd shows 281MB/s write speed, 72k iops. So a heck of a lot closer than on reads ... only a 2x factor different as opposed to 500x different! Exact command used was diskspd.exe -t8 -o8 -b4K -c8G -r -Sh -w100 w:\testfile.dat

Even weirder, after I did the write test, the read test results improved! Now the read test gives the same results as diskmark ... 1GB/s.

I literally ran the read test immediately before doing the write test, and got the same dismal 2MB/s

I'm happy it's working, but any idea what the heck could be going on here? I've been running these tests for literally days without success and now it suddenly starts working. I'm at a total loss.