Ok, I fixed the partition problem in ESXi and added the AiO benchmarks.
I deleted/ recreated a GPT partition with gparted and was now able to use the Optane under ESXi
Fazit for AiO (ESXi with a virtualized ZFS appliance)
Barebone Setup
Disk Pool 8k sync/unsync /s random sync/unsync/s seq sync/unsync/s dd sync /unsync /s
no slog 264K / 1.8M 1.8M / 72M 260M/ 957M 260M/ 957M
Optane Slog 1M / 1.8M 68M/ 73M 512M / 1023M 854M/957M
SSD Pool 8k sync/unsync /s random sync/unsync/s sequ sync /unsync /s dd sync/unsync/s
no slog 1M/ 1.9M 13.8M / 55.8M 341M/ 1023M 416M/ 896M
Optane Slog 1M / 1.8M 51M/53M 512M/ 1023M 775M/ 824M
Optane Pool 8k sync/unsync /s random sync/unsync/s sequ sync /unsync /s dd sync/unsync/s
one 900p 1.1M/ 1.9M 50M / 154M 511M/ 1023M 801M/ 1.4G
vs AiO setup with Slog or Optana as ESXi vdisk
Disk Pool 8k sync/unsync /s random sync/unsync/s seq sync/unsync/s dd sync /unsync /s
no slog 520K / 1.9M 1.6M / 65.8M 41,8M/ 1024M 283M/ 939M
Optane Slog 1.6M / 1.9M 39.4M/ 68.4M 512M / 1023M 849M/961M
SSD Pool 8k sync/unsync /s random sync/unsync/s sequ sync /unsync /s dd sync/unsync/s
no slog 1.5M/ 1.9M 16M / 50.2M 341M/ 1023M 423M/ 806M
Optane Slog 1.6M / 1.9M 38.2M/50.2M 512M/ 1023M 731M/ 806M
Optane Pool 8k sync/unsync /s random sync/unsync/s sequ sync /unsync /s dd sync/unsync/s
one 900p 1.6M/ 1.9M 32M / 75M 511M/ 1023M 711M/ 1.1G
Fazit Intel Optane and AiO
Despite the lower RAM and CPU performance of a virtualized SAN, the pool performance is nearly the same as the barebone values even when using an Slog as a vdisk. Even the last test with a basic ZFS pool on a vdisk shows that this pool is only minimal slower than on the barebone setup. Sync write performance with an Slog from a vdisk on Optane is > 700MB/s on the diskpool and the ssd pool. The optane only pool from a vdisk is in these benches quite similar to the disk or SSD pool and seesm limited by the server performance.
What are the consequenses for a „best of“ AiO setup?
Basically the most important thing is:
You want an Optane as Slog and propable L2ARC due the read ahead caching option on L2Arc.
For the pool itself, use disks or SSDs due their higher iops: Performance with an Optane Slog is superiour.
Pass-through of the Optane is not needed, not for performance reasons and as there is no cache involved
most propably not for security reasons. A pool from Optanes as datadisks would require a much faster machine.
Propably not a use case for AiO.
My suggested AiO setup
- use am USB stick ro boot ESXi
- create a local datastore on an Intel Optane 900P and place the napp-it storage VM onto
- Use an LSI HBA or Sata in pass-through mode for your datadisks
- add a 20 G vdisk on the Optane datastore to the napp-it storage VM and use as Slog for your datapool
- add a vdisk for on L2ARC (around 5 and no more than 10 x size of RAM)