Dell R730 - WS 2019 - Hyper-V > Slow Guest VM BOOT

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

SIlviu

Member
May 27, 2016
83
8
8
Struggling with this problem for about a year but never had time to dig into it...

Dell R730, 2 x E5-2680 v4, 64GB RAM, SSD Enterprise Storage, VEEAM Backup & Replication

I have a VM with WS 2019 that it liked to boot in ~ 40 minutes 1 year ago and now it takes ~ 2 hours to boot. There are 2 cores at 50% and no disk or network activity on the physical server. The VM console shows Hyper-V logo and the loading dots.
VM Config is default and I changed: Disabled Secure Boot, 37GB RAM, 16 x CPU, on the network interface I disabled Virtual Machine Queue, Enabled Guest Services

What I tried:
- Updated BIOS, Firmware, Drivers
- New VM config and attached VHDX
- Its not related to the VEEAM problem with the reference points, I checked and its 0 and the .VMCX file is 96KB
- Moved VHDX to another RAID Storage
- Checked VEEAM Config Settings with other servers and are the same, all defaults.
- Checked Event Viewer and no related errors.
- Set BIOS to performance mode

Noticed that the VM starts in a few seconds if I restart the VM the same day it booted
I have 2 more VM's on this server and they work fine.
 
Last edited:

CyklonDX

Well-Known Member
Nov 8, 2022
835
272
63
what is your ssd 'enterprise' storage? endurance left?; also what are your settings in raid controller.
 

SIlviu

Member
May 27, 2016
83
8
8
what is your ssd 'enterprise' storage? endurance left?; also what are your settings in raid controller.
There is 0% activity on the raid volume where the VHDX and VM Config is located.
2 x Intel 960GB S4510 RAID1, 95% endurance, No Read Ahead , Write Through, Disk Cache Enabled
I also tried on 2 x 960GB Samsung PM893, same problem

I don't think its storage related as the I/O on the raid volume is 0%
 

CyklonDX

Well-Known Member
Nov 8, 2022
835
272
63
what about your os storage? is it same?
and does it have good ventilation?
 

SIlviu

Member
May 27, 2016
83
8
8
OS is 2 x SAS 600GB 10K RPM, RAID 1, No Read Ahead, Write Back, Disk Cache Enabled, No errors on disks 100% life and 0-5% activity when the VM is loading.
~ 30 degrease on the disks
 

SIlviu

Member
May 27, 2016
83
8
8
I want to mention that I have a Quadro P2000 Passthrough to this VM but I did create a fresh VM config without GPU Passthrough and it was the same.
EDIT: the VM just got online now after 2:30 hours did a reboot and the same day reboot in a few seconds is no longer valid...
 
Last edited:

oneplane

Well-Known Member
Jul 23, 2021
845
484
63
Depending on how far you are willing to go and what else is running, you could grab a random SSD, install proxmox on it, and add a windows vm in there. If that boots fast at least you'll know it's not a hardware issue, making it much easier to diagnose.
 

CyklonDX

Well-Known Member
Nov 8, 2022
835
272
63
so by default hyper-v creates files on c disk too. Ensure your whole vm is on ssd's;
also turn off disk cache on sas hdd's, see if you see any difference. (or try leaving it enabled on sas hdd's, and disable on sas ssd's) -- sometimes if you have slow disks it will fill cache, and just wait until they get offloaded.

next thing is page file... make sure its either disabled, or placed on your ssd's; and has static size (min,max of same size). Next thing is to disable numa spanning, and ensuring cpu arch is correct. Upgrade your raid controller too, potentially your ssd's may also have firmware upgrades on dell site.

Another potential is your quadro sits on same pcie lanes as your raid controller. (moving it to another slot may be better. the right pcie slot shares lanes with raid controller)
 

SIlviu

Member
May 27, 2016
83
8
8
I did use the MOVE setting from Hyper-V and it moves the config and storage, I played with disk cache, no change, raid controller and storage is already updated. Even if the quadro sits on the same pcie lane I don't think its a problem unless there is a hudge I/O and the only thing that the server is doing is to BOOT a single VM, I stopped the rest just for this.
Now I installed a fresh WS 2019 VM on the same SSD RAID and it boots without any problem in seconds
 

CyklonDX

Well-Known Member
Nov 8, 2022
835
272
63
"huge io" typical raid controller has 1-2GB of ram. Thus if its reading from to writing to a slow disk it will impact disk caching of all disks connected (and depending what type of disk caching is enabled it may as well as block r/w).
 

SIlviu

Member
May 27, 2016
83
8
8
Hi, forgot to mention that the problematic VM is a RDP server with ~20 users.
After I made a new VM the problem was fixed for a few months, now I have the same problem, for now it takes ~ 15min to boot but its increasing...
I think its smth related to VEEAM or RDP but I can't find a fix...
 

terainfo

New Member
Nov 14, 2023
2
0
1
Struggling with this problem for about a year but never had time to dig into it...

Dell R730, 2 x E5-2680 v4, 64GB RAM, SSD Enterprise Storage, VEEAM Backup & Replication

I have a VM with WS 2019 that it liked to boot in ~ 40 minutes 1 year ago and now it takes ~ 2 hours to boot. There are 2 cores at 50% and no disk or network activity on the physical server. The VM console shows Hyper-V logo and the loading dots.
VM Config is default and I changed: Disabled Secure Boot, 37GB RAM, 16 x CPU, on the network interface I disabled Virtual Machine Queue, Enabled Guest Services

What I tried:
- Updated BIOS, Firmware, Drivers
- New VM config and attached VHDX
- Its not related to the VEEAM problem with the reference points, I checked and its 0 and the .VMCX file is 96KB
- Moved VHDX to another RAID Storage
- Checked VEEAM Config Settings with other servers and are the same, all defaults.
- Checked Event Viewer and no related errors.
- Set BIOS to performance mode

Noticed that the VM starts in a few seconds if I restart the VM the same day it booted
I have 2 more VM's on this server and they work fine.

Did you fix it?
Hi, forgot to mention that the problematic VM is a RDP server with ~20 users.
After I made a new VM the problem was fixed for a few months, now I have the same problem, for now it takes ~ 15min to boot but its increasing...
I think its smth related to VEEAM or RDP but I can't find a fix...


same here! terminal service 60+ users. 40 min to boot :( ! we need fix this... im
I'm going crazy (i dont have veem)
 

SIlviu

Member
May 27, 2016
83
8
8
There are a few posts on the web with slow vm boot with terminal server but no real fix...