Sharing 'cluster shared volume' .VHDX hard drives between two VMs, should they be live updating?

[frogtech]

I have a .VHDX volume that's a cluster shared volume in a Server 2012 R2 failover cluster. I'm able to share the hard drive between two VMs. If I create a folder on VM2, it doesn't show in VM1 in the hard drive unless I reboot that machine or go into disk management > rescan disks > import disk. Is this normal behavior?

One thing I found of interest on TechNet's site was this:

When you use shared VHDX with local block storage, synchronization must occur for shared VHDX file access. If the virtual machines are running on different nodes, this involves network redirection for the synchronization activity. If you have fast block connectivity combined with slow intra-cluster network communication, this synchronization activity results in I/O variance (either redirected or direct) based on the node. This happens on a per file basis. (The entire CSV is not in redirected mode.) To increase performance, we recommend that you scale the intra-cluster network.

Be aware that this is not a consideration when you use file-based storage. When you use a compute layer of file-based storage over SMB together with a remote Scale-Out File Server (SOFS), this shared file access orchestration is performed by the SOFS. With a SOFS, SMB sessions are transitioned to optimize file access. The SMB sessions co-exist on the same node that performs the synchronization. As a result, there is no network communication.

I've used both the VHDX directly as just a VHD in the CSV and as a VHD in an SOFS share in the CSV. Doesn't seem to make any difference.

 

[DavidRa]

Can you clarify - I'll put down what I think you have, and you can tell me which bits are wrong.

• Two hosts, HV1 and HV2.
• There is a shared SAS? FC? iSCSI? volume configured as a CSV.
• The CSV hosts a VHDX file - Shared1.VHDX (in addition to dedicated VHDXs for each VM)
  
• One VM on each host:
  • VM1 on HV1
  • VM2 on HV2
• VM1 storage:
  • IDE:0:0 = C:\ = VM1OS.VHDX
  • SCSI:0:0 = D:\ = Shared1.VHDX
• VM2 storage:
  • IDE:0:0 = C:\ = VM2OS.VHDX
  • SCSI:0:0 = D:\ = Shared1.VHDX
• VM1 and VM2 are individual hosts (non-clustered).

If you can tell me which bits are wrong we can probably sort this out, but if it is configured as above, VM1 and VM2 must also be clustered (Shared1.VHDX will be seen as shared SAS storage). This is the only way each will properly manage data and metadata on the volume.

 

[DavidRa]

Also other important thing, you need to use cluster filetype, like ReFS not NTFS.

ReFS and NTFS are not clustered filesystems. Cluster awareness is implemented via a special "filter driver" that ensures a shared volume metadata is updated by the owning node.

Both NTFS and ReFS are suitable for clustered volumes.

 

[frogtech]

Hey David,

In ScaleIO the mapped block devices are seen to Windows as fibre channel SCSI block devices. Anyway, I have 5 hosts, pending 6 if I can troubleshoot its issue, in a failover cluster.

I have two CSVs, one is an SSD CSV and one is a HDD CSV. VM installations are on the SSD CSV, the shared .VHDX is on the HDD CSV, so.

C:\ClusterStorage\Virtualization\... - CSV for VMs
C:\ClusterStorage\Storage\... - CSV for Bulk storage, shared .VHDX, etc

Currently one node owns both the VMs I'm testing this with but of course I can migrate one to another node.

Regarding your designation of IDE and SCSI for drives attached to the VMs, I'm not sure how to determine. There's nothing special about them, like the VHDXs are just present in the CSVs that are made available by the ScaleIO block device driver.

VM1 and VM2 are individual hosts, you are right. Just individual hosts configured as highly available VMs in the 5 physical node hyperconverged failover cluster.

So do I have to install failover clustering feature on both the VMs? Is there a way this can be done on a Server 2012 R2 VM and a Windows 10 VM? Should I create a new failover cluster for just the VMs that need to share the disk or can they connect to my HV cluster?

One last thing, I take it that you have to add the shared VHDX as a shared disk in the VM failover cluster right?

Thanks,

Frogtech

 

[DavidRa]

In ScaleIO the mapped block devices are seen to Windows as fibre channel SCSI block devices. Anyway, I have 5 hosts, pending 6 if I can troubleshoot its issue, in a failover cluster.

OK, that sounds fine and should be no problem from the HV host perspective.

VM1 and VM2 are individual hosts, you are right. Just individual hosts configured as highly available VMs in the 5 physical node hyperconverged failover cluster.

So do I have to install failover clustering feature on both the VMs? Is there a way this can be done on a Server 2012 R2 VM and a Windows 10 VM? Should I create a new failover cluster for just the VMs that need to share the disk or can they connect to my HV cluster?

One last thing, I take it that you have to add the shared VHDX as a shared disk in the VM failover cluster right?

Ah. Yeah no that's not going to work.

The shared VHDX feature is there to support guest VM clustering. Think of it like a single physical disk attached to two physical hosts, rather than an SMB share. To get the two machines to share the metadata and not corrupt the disk, you need to have clustering enabled on all VMs accessing that disk, and have the disk configured as either a CSV or a resource dedicated to a particular cluster group. Only the Server SKUs support that, Windows 10 doesn't have the required components.

What are you trying to do with it? It sounds like you want to have data on that disk available to both VMs - in which case the right way to do it is with an SMB share, not VHDX sharing.

 

[KamiCrazy]

I might be rehashing information here.
The shared vhdx thing just mimics a SAS shared disk topology. It just presents the virtual disk to both virtual machines. It is what you do with the shared disk afterwards that matters.

The most common usage I can think of is to create failover clusters (SQL, file server, etc).
If you want to actually provide true concurrent access to the disk you'll need a clustered file system.