nutanix@NTNX-RJFNutanix-A-CVM:172.16.232.31:~/diagnostics$ ./diagnostics.py --display_latency_stats --run_iperf run
Running Iperf Test between CVMs
bandwidth between 172.16.232.31 and 172.16.232.32 is: 8.39 Gbits
bandwidth between 172.16.232.31 and 172.16.232.33 is: 8.23 Gbits
bandwidth between 172.16.232.31 and 172.16.232.34 is: 8.61 Gbits
Checking if an existing storage pool can be used ...
Using storage pool default-storage-pool-7712708351784 for the tests.
Checking if the diagnostics container/s exists ...
Container NTNX-NFS-DEFAULT-0 does not exist.
Creating a new container NTNX-NFS-DEFAULT-0 for the runs ... done.
Preparing 1 UVM/s on host 172.16.232.11 ...
Importing diagnostics image ... done.
Deploying the UVM on host 172.16.232.11 ... done.
Adding disks ... done.
Preparing 1 UVM/s on host 172.16.232.12 ...
Deploying the UVM on host 172.16.232.12 ... done.
Adding disks ... done.
Preparing 1 UVM/s on host 172.16.232.13 ...
Deploying the UVM on host 172.16.232.13 ... done.
Adding disks ... done.
Preparing 1 UVM/s on host 172.16.232.14 ...
Deploying the UVM on host 172.16.232.14 ... done.
Adding disks ... done.
Waiting for 1 VM on host 172.16.232.11 to bootup ... done. 3 remaining.
Waiting for 1 VM on host 172.16.232.12 to bootup ... done. 2 remaining.
Waiting for 1 VM on host 172.16.232.13 to bootup ... done. 1 remaining.
Waiting for 1 VM on host 172.16.232.14 to bootup ... done. 0 remaining.
Collect executable list.
Collect shared library list.
Start up persistent ssh connection to localhost:17000.
ERROR:root:Error connecting through control master
Connecting to localhost:17000 with password authentication.
Start up persistent ssh connection to localhost:17002.
ERROR:root:Error connecting through control master
Connecting to localhost:17002 with password authentication.
Start up persistent ssh connection to localhost:17004.
ERROR:root:Error connecting through control master
Connecting to localhost:17004 with password authentication.
Start up persistent ssh connection to localhost:17006.
ERROR:root:Error connecting through control master
Connecting to localhost:17006 with password authentication.
Prepare environment on SVMs.
Preserve original install in /home/nutanix/builds/stock.
Preserve original install in /home/nutanix/builds/stock.
Preserve original install in /home/nutanix/builds/stock.
Preserve original install in /home/nutanix/builds/stock.
Transfer shared libraries.
Transfer dynamic linker.
Transfer executables.
Run patchelf.
Transferring the build across the cluster.
Done!
Start running test prepare_disks
2020-05-21_19-02-02: Running setup "Prepare disks" ...
done.
Average SVM CPU: 172.16.232.31: 77% 172.16.232.32: 76% 172.16.232.33: 74% 172.16.232.34: 83%
SSD usage for disks:
Disk id: 51, SSD used : 1074 GB, SSD Capacity Bytes: 1523 GB, 70% used
Disk id: 53, SSD used : 1022 GB, SSD Capacity Bytes: 1463 GB, 69% used
Disk id: 57, SSD used : 901 GB, SSD Capacity Bytes: 1523 GB, 59% used
Disk id: 58, SSD used : 1031 GB, SSD Capacity Bytes: 1463 GB, 70% used
Disk id: 67, SSD used : 900 GB, SSD Capacity Bytes: 1523 GB, 59% used
Disk id: 63, SSD used : 646 GB, SSD Capacity Bytes: 1463 GB, 44% used
Disk id: 74, SSD used : 1171 GB, SSD Capacity Bytes: 1463 GB, 80% used
Disk id: 75, SSD used : 1217 GB, SSD Capacity Bytes: 1523 GB, 79% used
SSD usage for storage pool:
SSD tier usage 7957 GB, SSD total capacity 11949 GB, SSD Usage 66%
Duration prepare_disks : 68 secs
*******************************************************************************
Start running test fio_seq_write
Waiting for the hot cache to flush ........... done.
2020-05-21_19-05-02: Running test "Sequential write bandwidth" ...
963 MBps , latency(msec): min=31, max=2820, median=414
Average SVM CPU: 172.16.232.31: 71% 172.16.232.32: 63% 172.16.232.33: 64% 172.16.232.34: 68%
SSD usage for disks:
Disk id: 51, SSD used : 1074 GB, SSD Capacity Bytes: 1523 GB, 70% used
Disk id: 53, SSD used : 1023 GB, SSD Capacity Bytes: 1463 GB, 69% used
Disk id: 57, SSD used : 902 GB, SSD Capacity Bytes: 1523 GB, 59% used
Disk id: 58, SSD used : 1032 GB, SSD Capacity Bytes: 1463 GB, 70% used
Disk id: 67, SSD used : 903 GB, SSD Capacity Bytes: 1523 GB, 59% used
Disk id: 63, SSD used : 649 GB, SSD Capacity Bytes: 1463 GB, 44% used
Disk id: 74, SSD used : 1172 GB, SSD Capacity Bytes: 1463 GB, 80% used
Disk id: 75, SSD used : 1217 GB, SSD Capacity Bytes: 1523 GB, 79% used
SSD usage for storage pool:
SSD tier usage 7975 GB, SSD total capacity 11949 GB, SSD Usage 66%
Duration fio_seq_write : 65 secs
*******************************************************************************
Start running test fio_seq_read
Waiting for the hot cache to flush ........ done.
2020-05-21_19-07-33: Running test "Sequential read bandwidth" ...
3760 MBps , latency(msec): min=6, max=465, median=129
Average SVM CPU: 172.16.232.31: 50% 172.16.232.32: 44% 172.16.232.33: 43% 172.16.232.34: 37%
SSD usage for disks:
Disk id: 51, SSD used : 1074 GB, SSD Capacity Bytes: 1523 GB, 70% used
Disk id: 53, SSD used : 1023 GB, SSD Capacity Bytes: 1463 GB, 69% used
Disk id: 57, SSD used : 902 GB, SSD Capacity Bytes: 1523 GB, 59% used
Disk id: 58, SSD used : 1032 GB, SSD Capacity Bytes: 1463 GB, 70% used
Disk id: 67, SSD used : 903 GB, SSD Capacity Bytes: 1523 GB, 59% used
Disk id: 63, SSD used : 649 GB, SSD Capacity Bytes: 1463 GB, 44% used
Disk id: 74, SSD used : 1172 GB, SSD Capacity Bytes: 1463 GB, 80% used
Disk id: 75, SSD used : 1217 GB, SSD Capacity Bytes: 1523 GB, 79% used
SSD usage for storage pool:
SSD tier usage 7975 GB, SSD total capacity 11949 GB, SSD Usage 66%
Duration fio_seq_read : 18 secs
*******************************************************************************
Start running test fio_rand_read
Waiting for the hot cache to flush .......... done.
2020-05-21_19-09-34: Running test "Random read IOPS" ...
169427 IOPS , latency(msec): min=0, max=312, median=2
Average SVM CPU: 172.16.232.31: 87% 172.16.232.32: 86% 172.16.232.33: 85% 172.16.232.34: 85%
SSD usage for disks:
Disk id: 51, SSD used : 1074 GB, SSD Capacity Bytes: 1523 GB, 70% used
Disk id: 53, SSD used : 1023 GB, SSD Capacity Bytes: 1463 GB, 69% used
Disk id: 57, SSD used : 902 GB, SSD Capacity Bytes: 1523 GB, 59% used
Disk id: 58, SSD used : 1032 GB, SSD Capacity Bytes: 1463 GB, 70% used
Disk id: 67, SSD used : 903 GB, SSD Capacity Bytes: 1523 GB, 59% used
Disk id: 63, SSD used : 649 GB, SSD Capacity Bytes: 1463 GB, 44% used
Disk id: 74, SSD used : 1172 GB, SSD Capacity Bytes: 1463 GB, 80% used
Disk id: 75, SSD used : 1217 GB, SSD Capacity Bytes: 1523 GB, 79% used
SSD usage for storage pool:
SSD tier usage 7975 GB, SSD total capacity 11949 GB, SSD Usage 66%
Duration fio_rand_read : 106 secs
*******************************************************************************
Start running test fio_rand_write
Waiting for the hot cache to flush ......... done.
2020-05-21_19-12-55: Running test "Random write IOPS" ...
116727 IOPS , latency(msec): min=0, max=272, median=3
Average SVM CPU: 172.16.232.31: 88% 172.16.232.32: 82% 172.16.232.33: 81% 172.16.232.34: 82%
SSD usage for disks:
Disk id: 51, SSD used : 1074 GB, SSD Capacity Bytes: 1523 GB, 70% used
Disk id: 53, SSD used : 1023 GB, SSD Capacity Bytes: 1463 GB, 69% used
Disk id: 57, SSD used : 902 GB, SSD Capacity Bytes: 1523 GB, 59% used
Disk id: 58, SSD used : 1032 GB, SSD Capacity Bytes: 1463 GB, 70% used
Disk id: 67, SSD used : 903 GB, SSD Capacity Bytes: 1523 GB, 59% used
Disk id: 63, SSD used : 649 GB, SSD Capacity Bytes: 1463 GB, 44% used
Disk id: 74, SSD used : 1172 GB, SSD Capacity Bytes: 1463 GB, 80% used
Disk id: 75, SSD used : 1217 GB, SSD Capacity Bytes: 1523 GB, 79% used
SSD usage for storage pool:
SSD tier usage 7975 GB, SSD total capacity 11949 GB, SSD Usage 66%
Duration fio_rand_write : 106 secs
*******************************************************************************
Tests done.