nested virtualization (how, and what do I need?)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

marcoi

Well-Known Member
Apr 6, 2013
1,532
288
83
Gotha Florida
It can be done. I got esxi running on bare metal and one VM running ws2012 with virtual box running. Also have win 8 VM with virtual box running.

What do you want to use as your base OS?
 

vl1969

Active Member
Feb 5, 2014
634
76
28
I know it's possible, but why do you need it?
there is only a few reasons I can see to do that.
I use it for testing on some configuration before real hardware setup.
but that is as far as I would go.
nested virtualization is a pain and overly complicates the setup, to be practical in real life.
exception is a powerful data center setup where you share the load amongst multiple tenants.
 

gigatexal

I'm here to learn
Nov 25, 2012
2,913
607
113
Portland, Oregon
alexandarnarayan.com
I know it's possible, but why do you need it?
there is only a few reasons I can see to do that.
I use it for testing on some configuration before real hardware setup.
but that is as far as I would go.
nested virtualization is a pain and overly complicates the setup, to be practical in real life.
exception is a powerful data center setup where you share the load amongst multiple tenants.
to mimic multi-hosts for learning without multiple physical hosts
 

marcoi

Well-Known Member
Apr 6, 2013
1,532
288
83
Gotha Florida
idk too much about fedora.
but in general you need to expose the hardware assisted virtualization to the guest, this allows it to see the cpu has vm capabilities. Also you may need to allow the host's network (vm switch) promiscuous mode set to accept/allow so your internal vm can bridge to nic.

some urls, again not fedora but the principles will most likely be the same
Nested Virtualization - VirtualBox inside ESXi - Stuff I've Figured Out
Ludovic Rivallain - Nested VMware ESXi with virtualbox

you obviously need bios settings to enable the vm options on the cpu.

just go experiment, thats part of the fun. :)
 
  • Like
Reactions: gigatexal

TLN

Active Member
Feb 26, 2016
523
84
28
34
I done that before. Probably still have nested VM somewhere.
I was running ESXi inside ESXi.

Remember that you have to manually edit vmx config, but that's it. Was working on my Xeon E3-1230v2/16Gb memory.
Will do that again with multiple nested VMs, when I'll build a new host. Just for testing(=fun)
 

TuxDude

Well-Known Member
Sep 17, 2011
616
338
63
Yes - you can do nested virtualization using Fedora25 / KVM as your host. By default you won't see the virt extensions passed into the VMs, there is a parameter you have to tweak for that. I've done it before, but can't remember the exact details right now. Googling things like 'nested virtualization libvirt' or 'nested virtualization virsh' should get you pointed in the right direction. I do seem to remember needing to edit the VMs XML to add the parameter - it wasn't exposed in the virt-manager GUI.
 
  • Like
Reactions: gigatexal

voxadam

Member
Apr 21, 2016
107
14
18
Portland, Oregon
I'm a little late to the party but here are a few notes I made for myself awhile back. I was researching nested virtualization for some cockamamie plan I had cooked up but never had the chance to implement. Hopefully one day I'll get around to it putting me one step closer to my ultimate goal of true world domination.

One day... one day.

Notes

• Enable Intel Virtualization Extensions (VT-x) in firmware
• Enable Intel Directed I/O (VT-d) in firmware
• Enable IOMMU in firmware
• Verify nested VMX is supported on host
cat /sys/module/kvm_intel/parameters/nested
◦ If nested parameter returns N or 0 add kvm-intel.nested=1 to kernel cmdline, reboot and check again​
• Enable huge page (hugetlbfs) support on host
• Verify Kernel Samepage Merging is enabled on host
◦ # cat /sys/kernel/mm/ksm/pages_shared
• Guest VMs should inherit the host CPU topology using the host-passthrough CPU model in libvirt/KVM

Links
HOWTO - KVM
HowTos/NestedVirt - CentOS Wiki
• RHEL7 Virtualization Tuning and Optimization Guide
Configure DevStack with KVM-based Nested Virtualization — DevStack 0.0.1.dev8190 documentation
How to assign devices with VT-d in KVM - KVM
Proxmox VE
Nested Virtualization - Proxmox VE
QEMU - ArchWiki
KVM - ArchWiki
Nested Virtualization in Xen - Xen
How to run Qemu & KVM on ESXi?
 
Last edited:
  • Like
Reactions: vl1969