Vsphere with 157 snapshot Data recovery

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

thx0701tw

Member
Dec 2, 2016
33
12
8
46
www.linkedin.com
Sorry for my bad English .

This case are received from out customer . Raid 5 ,4 300G SAS hard drive.
Hypervisor vsphere 6 because the hard disk bad sector ,0 and 3 hard disk both been offline

The most important thing here is to mirror the hard disk first
SAS bad track hard disk to prepare SAS level data rescue equipment .DD3000 .DD3000 device can do SAS hard disk power control SAS control card to read. With SAS Retry


A good hard drive uses our full-mirror Server. 15 bay The empty bay stands ready to allow all client hard drives to reach our servers with the fastest 10GbE network.
To be the first time to rescue customer information out.


We already have DD (mirror) files for all the hard drives in four hours



Customer is very urgent for the aging so we dispatched three engineers in the holidays
So we try 2 plans .
Plan I. To the original hard disk structure back to the array Server


Bad track hard disk with a good hard drive are backed up to the new hard drive to play Mahjong 4 missing a game






The array card selected Foreign View Scan Foreign find a way to import the original array data

Note that this method is done with a new mirror of the hard disk. Please do not move the original hard disk and we have to keep the original hard disk DD mirror

Prepare USB Vsphere Boot to see if there is a chance to mount this Datastore ..
Did not see Datastore

To the interface of this instruction

Rescan Datastore (VMFS) Still no data

Plan II
Analysic vmfs file system to calculate the hard disk array order and Strip size.



Combined RAID generates a total RAID image




Find the VMFS partition.


The most important customer information is DB VM but actually empty inside


We analyzed the missing 2 hard disk under this directory vmdk. (Yes, you must use offline hard disk it will have the correct data)




But this time found that the client made a 15x snapshot XXXXXXX



This time in trouble ... Customers want the latest information
Analyze the VMDK snapshot, we try to fix VMDK inedex . Use SFTP send to vsphere.

Not even log in to the VM.



Manually register the VMs vim-cmd solo / registervm /vmfs/volumes/datastore_name/VM_directory/VM_name.vmx


VM import seemingly successful but there is error





The implementation of the wrong boot


At this time we must consider the continuity of these snapshot files ...

Convert and Repair Snapshots
Here to see if there is a way to convert the snapshot file into Flat file and do file inspection
http://www.vmware.com/pdf/VirtualDiskManager.pdf https://translate.googleusercontent...er.pdf&usg=ALkJrhhPemuhP2q_fVww2boCILePk0WtJA
vmware-vdiskmanager -R snapshot index file vmdk repair the snapshot file
vmware-vdiskmanager -r snapshot index file vmdk -t 0 target flat snapshot file
Use vmware tool vmware vdiskmanger convert snapshot file to flat first snapshot file success


Uncoveryable memory alloacation appears after conversion
After the change of different versions vmware vdiskmanger same error message with the computer. XXXXXXXXXXXXXXXXXXX



PANIC crash !!!

To change vmkfstools
vmkfstools -i XXXXX-000019.vmdk J880000019.vmdk -d thin

DiskLib_Check () failed for source disk 'XX-DB-000001.vmdk': The specified feature is not supported by this version (24).




A plan try to repair the VM
B plan as far as possible to get the latest version of the mysql fishing to each other ....
Mount VMDK first.


VMware official Mount program does not eat half loss VMDK ...>



Write a small program to extract incomplete VMDK data.



Mysql error, but still give it to the customer.


Repair the VM
Since the original tool does not support their own write a program to repair the snapshot

VMX also want to modify the device to turn to fix flat.vmdk




VM can be turned on


Perfect Boot DB are normal





About 99% of our VMs are restored by customers in about four business days
 

cheezehead

Active Member
Sep 23, 2012
730
176
43
Midwest, US
I've used the vmkfstools method for recovering from around 150 snapshots. In my case the backup software had a bug in it which caused it to fail quiescing the disk within it's timeout period and attempt to create the snapshot again..and again..and again.
 
  • Like
Reactions: thx0701tw