I recently had the opportunity to test out a server with 24 10K drives for VM storage.
It was setup as 11vdevs Raid10 with 2 hot spares connected to esxi via dual 10gbit (4 paths vlan'd).
Single host was connected via 4x1gig connections, RoundRobin with iops=1.
I loaded it up with Core 13.0-u6.2 first then upgraded/switched to Scale (dragonfish).
Tests were run on a VM running 2022 with PV scsi drives.
It seems Core was more quite a bit more performant in the RND (Read/Write) metrics - which I think matters more for VM storage.
Core:
[Read]
SEQ 1MiB (Q= 8, T= 1): 468.461 MB/s [ 446.8 IOPS] < 17859.14 us>
SEQ 1MiB (Q= 1, T= 1): 206.553 MB/s [ 197.0 IOPS] < 5070.92 us>
RND 4KiB (Q= 32, T= 1): 408.790 MB/s [ 99802.2 IOPS] < 320.26 us>
RND 4KiB (Q= 1, T= 1): 25.302 MB/s [ 6177.2 IOPS] < 161.36 us>
[Write]
SEQ 1MiB (Q= 8, T= 1): 468.147 MB/s [ 446.5 IOPS] < 17855.56 us>
SEQ 1MiB (Q= 1, T= 1): 215.602 MB/s [ 205.6 IOPS] < 4858.70 us>
RND 4KiB (Q= 32, T= 1): 239.211 MB/s [ 58401.1 IOPS] < 547.17 us>
RND 4KiB (Q= 1, T= 1): 23.341 MB/s [ 5698.5 IOPS] < 174.77 us>
[Mix] Read 70%/Write 30%
SEQ 1MiB (Q= 8, T= 1): 523.640 MB/s [ 499.4 IOPS] < 15951.04 us>
SEQ 1MiB (Q= 1, T= 1): 210.139 MB/s [ 200.4 IOPS] < 4986.20 us>
RND 4KiB (Q= 32, T= 1): 289.934 MB/s [ 70784.7 IOPS] < 451.24 us>
RND 4KiB (Q= 1, T= 1): 24.156 MB/s [ 5897.5 IOPS] < 168.89 us>
SCALE:
[Read]
SEQ 1MiB (Q= 8, T= 1): 468.878 MB/s [ 447.2 IOPS] < 17851.04 us>
SEQ 1MiB (Q= 1, T= 1): 196.298 MB/s [ 187.2 IOPS] < 5336.07 us>
RND 4KiB (Q= 32, T= 1): 319.338 MB/s [ 77963.4 IOPS] < 408.65 us>
RND 4KiB (Q= 1, T= 1): 9.882 MB/s [ 2412.6 IOPS] < 412.82 us>
[Write]
SEQ 1MiB (Q= 8, T= 1): 465.517 MB/s [ 444.0 IOPS] < 17940.84 us>
SEQ 1MiB (Q= 1, T= 1): 210.540 MB/s [ 200.8 IOPS] < 4976.17 us>
RND 4KiB (Q= 32, T= 1): 193.882 MB/s [ 47334.5 IOPS] < 675.05 us>
RND 4KiB (Q= 1, T= 1): 9.571 MB/s [ 2336.7 IOPS] < 426.07 us>
[Mix] Read 70%/Write 30%
SEQ 1MiB (Q= 8, T= 1): 540.060 MB/s [ 515.0 IOPS] < 15473.02 us>
SEQ 1MiB (Q= 1, T= 1): 203.832 MB/s [ 194.4 IOPS] < 5137.22 us>
RND 4KiB (Q= 32, T= 1): 277.687 MB/s [ 67794.7 IOPS] < 471.18 us>
RND 4KiB (Q= 1, T= 1): 10.238 MB/s [ 2499.5 IOPS] < 398.73 us>
I'm not sure what this performance disparity is about. Technically everything should've been served up off Arc cache.
I didnt get a chance to test any further as I was running out of time.
Thoughts ?
It was setup as 11vdevs Raid10 with 2 hot spares connected to esxi via dual 10gbit (4 paths vlan'd).
Single host was connected via 4x1gig connections, RoundRobin with iops=1.
I loaded it up with Core 13.0-u6.2 first then upgraded/switched to Scale (dragonfish).
Tests were run on a VM running 2022 with PV scsi drives.
It seems Core was more quite a bit more performant in the RND (Read/Write) metrics - which I think matters more for VM storage.
Core:
[Read]
SEQ 1MiB (Q= 8, T= 1): 468.461 MB/s [ 446.8 IOPS] < 17859.14 us>
SEQ 1MiB (Q= 1, T= 1): 206.553 MB/s [ 197.0 IOPS] < 5070.92 us>
RND 4KiB (Q= 32, T= 1): 408.790 MB/s [ 99802.2 IOPS] < 320.26 us>
RND 4KiB (Q= 1, T= 1): 25.302 MB/s [ 6177.2 IOPS] < 161.36 us>
[Write]
SEQ 1MiB (Q= 8, T= 1): 468.147 MB/s [ 446.5 IOPS] < 17855.56 us>
SEQ 1MiB (Q= 1, T= 1): 215.602 MB/s [ 205.6 IOPS] < 4858.70 us>
RND 4KiB (Q= 32, T= 1): 239.211 MB/s [ 58401.1 IOPS] < 547.17 us>
RND 4KiB (Q= 1, T= 1): 23.341 MB/s [ 5698.5 IOPS] < 174.77 us>
[Mix] Read 70%/Write 30%
SEQ 1MiB (Q= 8, T= 1): 523.640 MB/s [ 499.4 IOPS] < 15951.04 us>
SEQ 1MiB (Q= 1, T= 1): 210.139 MB/s [ 200.4 IOPS] < 4986.20 us>
RND 4KiB (Q= 32, T= 1): 289.934 MB/s [ 70784.7 IOPS] < 451.24 us>
RND 4KiB (Q= 1, T= 1): 24.156 MB/s [ 5897.5 IOPS] < 168.89 us>
SCALE:
[Read]
SEQ 1MiB (Q= 8, T= 1): 468.878 MB/s [ 447.2 IOPS] < 17851.04 us>
SEQ 1MiB (Q= 1, T= 1): 196.298 MB/s [ 187.2 IOPS] < 5336.07 us>
RND 4KiB (Q= 32, T= 1): 319.338 MB/s [ 77963.4 IOPS] < 408.65 us>
RND 4KiB (Q= 1, T= 1): 9.882 MB/s [ 2412.6 IOPS] < 412.82 us>
[Write]
SEQ 1MiB (Q= 8, T= 1): 465.517 MB/s [ 444.0 IOPS] < 17940.84 us>
SEQ 1MiB (Q= 1, T= 1): 210.540 MB/s [ 200.8 IOPS] < 4976.17 us>
RND 4KiB (Q= 32, T= 1): 193.882 MB/s [ 47334.5 IOPS] < 675.05 us>
RND 4KiB (Q= 1, T= 1): 9.571 MB/s [ 2336.7 IOPS] < 426.07 us>
[Mix] Read 70%/Write 30%
SEQ 1MiB (Q= 8, T= 1): 540.060 MB/s [ 515.0 IOPS] < 15473.02 us>
SEQ 1MiB (Q= 1, T= 1): 203.832 MB/s [ 194.4 IOPS] < 5137.22 us>
RND 4KiB (Q= 32, T= 1): 277.687 MB/s [ 67794.7 IOPS] < 471.18 us>
RND 4KiB (Q= 1, T= 1): 10.238 MB/s [ 2499.5 IOPS] < 398.73 us>
I'm not sure what this performance disparity is about. Technically everything should've been served up off Arc cache.
I didnt get a chance to test any further as I was running out of time.
Thoughts ?