So,
since I am unhappy with vsan atm I am playing around with the hardware to establish a baseline what it could do.
Currently I am running a few simple tests moving test VMs from one box to another via vMotion to test the potential speed.
I have various NVMe drives in both my boxes and a 56GbE link between them (switched).
1. If i move VMs from one drive to another on the same host I get speeds from 700MB/s to 1G/s (on both hosts)
2. Iperf between the two hosts is ok (4K MTU)
13.8 Gbits/sec (64K Window size), 21.6 Gbits/sec (1M Window size), single thread (A->B)
15.9 Gbits/sec (64K Window size), 19.8 Gbits/sec (1M Window size), single thread (B->A)
3. Transfer speeds between the two hosts is not so good - between 180MB/s and 260 MB/s - the same drives that push tripple that locally.
So -
is that to be expected?
Anything I could check?
I have the most recent nvme driver, most of the drives have the current fw.
Mellanox cards are using the ESX 6.5 driver o/c due to the lack of alternatives
CPU Utilization seems fine, have not seen a single core over 40% ...
Will test more with a third host, but am kind of lost (again)...
since I am unhappy with vsan atm I am playing around with the hardware to establish a baseline what it could do.
Currently I am running a few simple tests moving test VMs from one box to another via vMotion to test the potential speed.
I have various NVMe drives in both my boxes and a 56GbE link between them (switched).
1. If i move VMs from one drive to another on the same host I get speeds from 700MB/s to 1G/s (on both hosts)
2. Iperf between the two hosts is ok (4K MTU)
13.8 Gbits/sec (64K Window size), 21.6 Gbits/sec (1M Window size), single thread (A->B)
15.9 Gbits/sec (64K Window size), 19.8 Gbits/sec (1M Window size), single thread (B->A)
3. Transfer speeds between the two hosts is not so good - between 180MB/s and 260 MB/s - the same drives that push tripple that locally.
So -
is that to be expected?
Anything I could check?
I have the most recent nvme driver, most of the drives have the current fw.
Mellanox cards are using the ESX 6.5 driver o/c due to the lack of alternatives
CPU Utilization seems fine, have not seen a single core over 40% ...
Will test more with a third host, but am kind of lost (again)...