Proxmox shared storage for VMs... Synology (btrfs) iSCSI LUNs vs. TrueNAS Scale ZFS over iSCSI?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Gio

Member
Apr 8, 2017
88
12
8
37
Has anyone done a benchmark / comparison / benefits analysis of these two options below? I'm looking to centralize all my VM storage to a NAS (currently all VMs are in each cluster node local-zfs storage)

Primarily looking for:
* Improved disk space savings via deduplication, since central NAS having duplicate/templates/same-in-kind VMs like Windows should save disk space due to deduplication.
* Improved performance (thanks to caching), Synology has write-back caching that my current cluster nodes do not.
* Bigger space availability (a big central place vs. small disks across many nodes), currently my cluster nodes have NVME disk space ranging from 512gb to 2tb - I am considering a NAS that's nvme only plus has backup hdd storage replication.


Some of my notes about the options I am considering and looking for folks with some experience in the below to correct me?

  • Synology NAS iSCSI - Proxmox LVM shared storage
    • LVM seems to allow any node in the cluster to be able to host any VMs and provide HA. But we lose proxmox snapshots...
    • Synology has write-back cache, which according to documentation by synology it should help with improving performance for frequently used blocks.
    • Synology uses btrfs under the hood, thick LUNs provisioned do not benefit from deduplication and Synology recommends against trying to enable it when LUNs are used for VM storage.
    • Very intuitive GUI and "just works" solution
  • TrueNAS Scale NAS ZFS over iSCSI - Proxmox ZFS
    • (unsure) seems like this would provide the "deduplication" benefits of multiple windows VMs that are related and save on storage space.
    • L2ARC caching should in theory be better and faster than synology btrfs + flashcache (I could be wrong)
    • Ability to take snapshots easily and closely integrated with PVE.
    • More complex setup than Synology, may require some elbow grease and some tweak in settings to make it perform its best.
 

TRACKER

Active Member
Jan 14, 2019
276
114
43
TCP based iSCSI will be a pain in the a... for a NVMe storage.


Build your storage RDMA based with iSER or NVMeOF
regular iSCSI (over TCP using vmware software iscsi adapter) worked pretty well on old hardware (sandy bridge/ivy bridge era).
I was able to achieve 1.5GB/s without any issue (that's on 25Gbps network).
After i switched to RDMA/iSCSI/iSER i am able to achieve 3-4GB/s with single session and same truenas scale hardware (on 100Gbps network already).