olav@sola:~/projects/github/fio$ ./fio --filename=/sqltank/kvm/ubuntu1604/xenial-server-cloudimg-amd64-disk1.img --readonly --direct=0 --sync=0 --rw=read --bs=128k --numjobs=18 --iodepth=1 --runtime=60 --numa_cpu_nodes=0-1 --numa_mem_policy=interleave:0-1 --time_based --group_reporting --name=journal-test
journal-test: (g=0): rw=read, bs=(R) 128KiB-128KiB, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=psync, iodepth=1
...
fio-3.5-56-g9634
Starting 18 processes
Jobs: 18 (f=18): [R(18)][100.0%][r=61.4GiB/s,w=0KiB/s][r=503k,w=0 IOPS][eta 00m:00s]
journal-test: (groupid=0, jobs=18): err= 0: pid=11869: Tue Mar 20 13:20:11 2018
read: IOPS=516k, BW=63.0GiB/s (67.7GB/s)(3782GiB/60002msec)
clat (usec): min=11, max=6583, avg=33.92, stdev=22.89
lat (usec): min=11, max=6583, avg=34.02, stdev=22.89
clat percentiles (usec):
| 1.00th=[ 21], 5.00th=[ 24], 10.00th=[ 26], 20.00th=[ 28],
| 30.00th=[ 29], 40.00th=[ 30], 50.00th=[ 32], 60.00th=[ 33],
| 70.00th=[ 35], 80.00th=[ 37], 90.00th=[ 41], 95.00th=[ 46],
| 99.00th=[ 92], 99.50th=[ 141], 99.90th=[ 326], 99.95th=[ 424],
| 99.99th=[ 816]
bw ( MiB/s): min= 1994, max= 4319, per=5.56%, avg=3589.47, stdev=426.53, samples=2143
iops : min=15954, max=34556, avg=28715.73, stdev=3412.27, samples=2143
lat (usec) : 20=0.98%, 50=96.20%, 100=1.96%, 250=0.63%, 500=0.18%
lat (usec) : 750=0.02%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%
cpu : usr=3.72%, sys=95.85%, ctx=1087349, majf=0, minf=686
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=30983214,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=63.0GiB/s (67.7GB/s), 63.0GiB/s-63.0GiB/s (67.7GB/s-67.7GB/s), io=3782GiB (4061GB), run=60002-60002msec