I've started testing iSER, and it does have lower latency, but it adds a bunch of new issues. Vsphere 6 cannot mount iscsi/iser LUNS that have a 4K block size. I've reformatted iodrive2s as 512 byte block size, and can mount them via iSER, but on the target side, I/O fails with "unaligned DMA" kernel messages. iSER exported zvols work, but has low performance compared to the NVMe or SSD raw device. So yea, NVMeOF seems the way forward, and I'm looking at avoiding ESXI, and using KVM instead to do so, since there are fewer limitations there.