Hm thats kinda surprising.
Ran the same commands against my zeus backed pool of 8 spinners and get faster perf throughout?
Not looking to turn this thread off course but weird - especially after @gea's results.
Write
Ran the same commands against my zeus backed pool of 8 spinners and get faster perf throughout?
Not looking to turn this thread off course but weird - especially after @gea's results.
Code:
root@core ~/fio-2.2.10# ./fio --filename=test --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=4k --rwmixread=100 --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=4kreadtest --size=12000M
4kreadtest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
...
fio-2.2.10
Starting 16 processes
Jobs: 16 (f=16): [r(16)] [100.0% done] [60440KB/0KB/0KB /s] [15.2K/0/0 iops] [eta 00m:00s]
4kreadtest: (groupid=0, jobs=16): err= 0: pid=14144: Tue Jan 2 09:30:22 2018
read : io=3925.2MB, bw=66981KB/s, iops=16745, runt= 60008msec
slat (usec): min=1, max=113852, avg=484.11, stdev=2525.15
clat (usec): min=51, max=639863, avg=14801.33, stdev=31350.04
lat (usec): min=115, max=645943, avg=15285.61, stdev=31634.15
clat percentiles (usec):
| 1.00th=[ 502], 5.00th=[ 1336], 10.00th=[ 2512], 20.00th=[ 3728],
| 30.00th=[ 4640], 40.00th=[ 5472], 50.00th=[ 6432], 60.00th=[ 7456],
| 70.00th=[ 9280], 80.00th=[12352], 90.00th=[25472], 95.00th=[67072],
| 99.00th=[168960], 99.50th=[207872], 99.90th=[313344], 99.95th=[378880],
| 99.99th=[497664]
bw (KB /s): min= 152, max=12826, per=6.33%, avg=4239.99, stdev=1927.83
lat (usec) : 100=0.01%, 250=0.14%, 500=0.84%, 750=1.23%, 1000=1.24%
lat (msec) : 2=4.11%, 4=15.55%, 10=49.75%, 20=15.28%, 50=5.40%
lat (msec) : 100=3.66%, 250=2.54%, 500=0.24%, 750=0.01%
cpu : usr=0.14%, sys=0.65%, ctx=263530, majf=0, minf=168
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=1004845/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: io=3925.2MB, aggrb=66980KB/s, minb=66980KB/s, maxb=66980KB/s, mint=60008msec, maxt=60008msec
Disk stats (read/write):
dm-0: ios=1004064/72, merge=0/0, ticks=8743012/155236, in_queue=8919360, util=99.93%, aggrios=1004796/39, aggrmerge=2/33, aggrticks=8711148/132476, aggrin_queue=8844088, aggrutil=99.77%
sda: ios=1004796/39, merge=2/33, ticks=8711148/132476, in_queue=8844088, util=99.77%
Code:
./fio --filename=test --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=4k --rwmixread=0 --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=4kwritetest --size=12000M
4kwritetest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
...
fio-2.2.10
Starting 16 processes
Jobs: 16 (f=16): [w(16)] [100.0% done] [0KB/25324KB/0KB /s] [0/6331/0 iops] [eta 00m:00s]
4kwritetest: (groupid=0, jobs=16): err= 0: pid=14109: Tue Jan 2 09:27:57 2018
write: io=1218.8MB, bw=20703KB/s, iops=5175, runt= 60282msec
slat (usec): min=2, max=2230.3K, avg=1548.82, stdev=20996.31
clat (usec): min=310, max=2423.9K, avg=47875.43, stdev=127574.56
lat (usec): min=316, max=2423.9K, avg=49424.48, stdev=129380.33
clat percentiles (usec):
| 1.00th=[ 1592], 5.00th=[ 4960], 10.00th=[ 7520], 20.00th=[10176],
| 30.00th=[12352], 40.00th=[14656], 50.00th=[17792], 60.00th=[22400],
| 70.00th=[29056], 80.00th=[45312], 90.00th=[100864], 95.00th=[168960],
| 99.00th=[577536], 99.50th=[995328], 99.90th=[1777664], 99.95th=[2244608],
| 99.99th=[2342912]
bw (KB /s): min= 6, max= 4055, per=7.13%, avg=1475.84, stdev=765.72
lat (usec) : 500=0.01%, 750=0.07%, 1000=0.12%
lat (msec) : 2=1.61%, 4=2.26%, 10=15.13%, 20=35.94%, 50=26.39%
lat (msec) : 100=8.34%, 250=7.72%, 500=1.25%, 750=0.50%, 1000=0.16%
lat (msec) : 2000=0.42%, >=2000=0.08%
cpu : usr=0.09%, sys=0.24%, ctx=85428, majf=0, minf=183
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.9%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=312001/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: io=1218.8MB, aggrb=20702KB/s, minb=20702KB/s, maxb=20702KB/s, mint=60282msec, maxt=60282msec
Disk stats (read/write):
dm-0: ios=0/312027, merge=0/0, ticks=0/8702792, in_queue=8760896, util=99.92%, aggrios=0/312008, aggrmerge=0/12, aggrticks=0/8702352, aggrin_queue=8702292, aggrutil=99.86%
sda: ios=0/312008, merge=0/12, ticks=0/8702352, in_queue=8702292, util=99.86%