probably got the nic plugged into a gen 2 pcie slot
This is VMware ESXi, 7.0.2, 17630552 - latest build.Have the same on my CX3 Pro's. I think its a vSAN limitation
View attachment 18063
There are some updates in ESXi 7 including RDMA for vSan support that might alleviate this, but I have not updated yet (from 6.7)
Roce (v1 & 2) can work without any switch configuration, but in an environment where the switches buffer capacity is maxed out you want to take precautions that no frames (v1) or packets (v2) are dropped.Originally i thought rdma (roce) needed switch support, but ESXi 7 seems to run it without... so not sure
The os/filesystem can't optimze io if the queue is not deep enough.Why does it require that I increase the queue depth to get to the maximum speed?
Yes I was running vSAN, but i blew that away for now to run Windows servers on the nodes to cut through the confusion and experiment with configurations.It certainly would help to understand what you're testing here?
Initially you mentioned vSAN, now you're running a Ramdrive somewhere...
No explanation, no config details, nothing - makes it quite difficult to answer.