1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

nested virtualization (how, and what do I need?)

Discussion in 'Linux Admins, Storage and Virtualization' started by gigatexal, Feb 1, 2017.

  1. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,030
    Likes Received:
    338
    How do I do it? What do I need for it?

    Would like to repurpose my dual 2670 system but doubt that I can unless I can passthrough the virt-features to the guest hypervisor from the host hypervisor
     
    #1
  2. marcoi

    marcoi Active Member

    Joined:
    Apr 6, 2013
    Messages:
    543
    Likes Received:
    66
    It can be done. I got esxi running on bare metal and one VM running ws2012 with virtual box running. Also have win 8 VM with virtual box running.

    What do you want to use as your base OS?
     
    #2
  3. vl1969

    vl1969 Active Member

    Joined:
    Feb 5, 2014
    Messages:
    396
    Likes Received:
    39
    I know it's possible, but why do you need it?
    there is only a few reasons I can see to do that.
    I use it for testing on some configuration before real hardware setup.
    but that is as far as I would go.
    nested virtualization is a pain and overly complicates the setup, to be practical in real life.
    exception is a powerful data center setup where you share the load amongst multiple tenants.
     
    #3
  4. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,030
    Likes Received:
    338
    preferably fedora 25

    was there much to it? i've a linode instance and they run kvm, and i don't see virtualization features enable, so I am guessing there's some config needed when/if i set this up
     
    #4
  5. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,030
    Likes Received:
    338
    to mimic multi-hosts for learning without multiple physical hosts
     
    #5
  6. marcoi

    marcoi Active Member

    Joined:
    Apr 6, 2013
    Messages:
    543
    Likes Received:
    66
    idk too much about fedora.
    but in general you need to expose the hardware assisted virtualization to the guest, this allows it to see the cpu has vm capabilities. Also you may need to allow the host's network (vm switch) promiscuous mode set to accept/allow so your internal vm can bridge to nic.

    some urls, again not fedora but the principles will most likely be the same
    Nested Virtualization - VirtualBox inside ESXi - Stuff I've Figured Out
    Ludovic Rivallain - Nested VMware ESXi with virtualbox

    you obviously need bios settings to enable the vm options on the cpu.

    just go experiment, thats part of the fun. :)
     
    #6
    gigatexal likes this.
  7. TLN

    TLN Member

    Joined:
    Feb 26, 2016
    Messages:
    224
    Likes Received:
    22
    I done that before. Probably still have nested VM somewhere.
    I was running ESXi inside ESXi.

    Remember that you have to manually edit vmx config, but that's it. Was working on my Xeon E3-1230v2/16Gb memory.
    Will do that again with multiple nested VMs, when I'll build a new host. Just for testing(=fun)
     
    #7
  8. Netwerkz101

    Netwerkz101 Active Member

    Joined:
    Dec 27, 2015
    Messages:
    165
    Likes Received:
    35
    If you go nested ESXI, one of the best resources out there was:
    VirtuallyGhetto.com - NESTED VIRTUALIZATION
     
    #8
  9. TuxDude

    TuxDude Well-Known Member

    Joined:
    Sep 17, 2011
    Messages:
    615
    Likes Received:
    333
    Yes - you can do nested virtualization using Fedora25 / KVM as your host. By default you won't see the virt extensions passed into the VMs, there is a parameter you have to tweak for that. I've done it before, but can't remember the exact details right now. Googling things like 'nested virtualization libvirt' or 'nested virtualization virsh' should get you pointed in the right direction. I do seem to remember needing to edit the VMs XML to add the parameter - it wasn't exposed in the virt-manager GUI.
     
    #9
    gigatexal likes this.
  10. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,030
    Likes Received:
    338
    thanks everyone!
     
    #10
  11. Marsh

    Marsh Moderator

    Joined:
    May 12, 2013
    Messages:
    1,315
    Likes Received:
    537
  12. Marsh

    Marsh Moderator

    Joined:
    May 12, 2013
    Messages:
    1,315
    Likes Received:
    537
    #12
    gigatexal likes this.
  13. voxadam

    voxadam Member

    Joined:
    Apr 21, 2016
    Messages:
    76
    Likes Received:
    6
    I'm a little late to the party but here are a few notes I made for myself awhile back. I was researching nested virtualization for some cockamamie plan I had cooked up but never had the chance to implement. Hopefully one day I'll get around to it putting me one step closer to my ultimate goal of true world domination.

    One day... one day.

    Notes

    • Enable Intel Virtualization Extensions (VT-x) in firmware
    • Enable Intel Directed I/O (VT-d) in firmware
    • Enable IOMMU in firmware
    • Verify nested VMX is supported on host
    cat /sys/module/kvm_intel/parameters/nested
    ◦ If nested parameter returns N or 0 add kvm-intel.nested=1 to kernel cmdline, reboot and check again​
    • Enable huge page (hugetlbfs) support on host
    • Verify Kernel Samepage Merging is enabled on host
    ◦ # cat /sys/kernel/mm/ksm/pages_shared
    • Guest VMs should inherit the host CPU topology using the host-passthrough CPU model in libvirt/KVM

    Links
    HOWTO - KVM
    HowTos/NestedVirt - CentOS Wiki
    • RHEL7 Virtualization Tuning and Optimization Guide
    Configure DevStack with KVM-based Nested Virtualization — DevStack 0.0.1.dev8190 documentation
    How to assign devices with VT-d in KVM - KVM
    Proxmox VE
    Nested Virtualization - Proxmox VE
    QEMU - ArchWiki
    KVM - ArchWiki
    Nested Virtualization in Xen - Xen
    How to run Qemu & KVM on ESXi?
     
    #13
    Last edited: Feb 9, 2017
    vl1969 likes this.
Similar Threads: nested virtualization
Forum Title Date
Linux Admins, Storage and Virtualization Recommended Virtualization Solution? Sep 11, 2017
Linux Admins, Storage and Virtualization Three virtualization setups: best performance? May 8, 2017
Linux Admins, Storage and Virtualization Mild brain picking (PXE boot multi OS from server, ethernet speed for virtualization, & more) Feb 1, 2017
Linux Admins, Storage and Virtualization Homeserver virtualization Mar 13, 2015

Share This Page