Skip to contentSkip to conversation

New: Nested Virt: VMX can't be initialized in L1 Xen ("Xen on KVM")

Bug 45931 [email protected] 1 participant 6 messages Aug 14, 2012 – Mar 17, 2015

Subsystems (3)
x86 KVM KVM x86
lore
Share & tools
lore

[email protected]

author
#0 lore raw

https://bugzilla.kernel.org/show_bug.cgi?id=45931

Summary: Nested Virt: VMX can't be initialized in L1 Xen ("Xen
on KVM")
Product: Virtualization
Version: unspecified
Kernel Version: 3.6-RC1
Platform: All
OS/Version: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: kvm
AssignedTo: [email protected]
ReportedBy: [email protected]
CC: [email protected]
Regression: No

Created an attachment (id=77661)
--> (https://bugzilla.kernel.org/attachment.cgi?id=77661)
xl-dmesg-in-L1-Xen.log

Hi folks,
I did some basic testing on nested virtualization on Intel x86-64 platform. Will KVM support Xen as L1 guest in nested virtualization ?

When I tried "Xen on KVM" mode, I found VMX can't be initialized in L1 Xen hypervisor.
I tried both "-cpu host" and "-cpu qemu64,+vmx" parameters in qemu-kvm command line.
-------some log in 'xl dmesg' command line in L1 Xen ---
(XEN) VMX: CPU0 has insufficent CPU-Based Exec Control (b299868c but requires min 2299968c)
(XEN) VMX: CPU0 has insufficent VMExit Control (00000200 but requires min 00008200)
(XEN) VMX: failed to initialise.
------more details in the attached "xl-dmesg-L1.log"----
In Xen source code, I found the following related to the mentioned error. "xen-unstable.hg/xen/include/asm-x86/hvm/vmx/vmcs.h"
#define CPU_BASED_RDTSC_EXITING 0x00001000
#define VM_EXIT_ACK_INTR_ON_EXIT 0x00008000

----
Avi confirmed there're two issues, and fixed 1st one (RDTSC) by the following patch.

All processors that support VMX have that feature, and guests (Xen) depend on it. As we already implement it, advertize it to the guest.

Signed-off-by Avi Kivity <[email protected]>
---
 arch/x86/kvm/vmx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/vmx.c +1 -1
index cc8ad98..8092f25 100644
@@ -1990,7 +1990,7 @@ static __init void nested_vmx_setup_ctls_msrs(void)
 #endif
         CPU_BASED_MOV_DR_EXITING | CPU_BASED_UNCOND_IO_EXITING |
         CPU_BASED_USE_IO_BITMAPS | CPU_BASED_MONITOR_EXITING |
-        CPU_BASED_RDPMC_EXITING |
+        CPU_BASED_RDPMC_EXITING | CPU_BASED_RDTSC_EXITING |
         CPU_BASED_ACTIVATE_SECONDARY_CONTROLS;
     /*
      * We can allow some features even when not supported by the
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
author
#1 lore raw

[Bug 45931] Nested Virt: VMX can't be initialized in L1 Xen ("Xen on KVM")

https://bugzilla.kernel.org/show_bug.cgi?id=45931

Nadav Har'El <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

             Blocks|                            |53601
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
author
#2 lore raw

[Bug 45931] Nested Virt: VMX can't be initialized in L1 Xen ("Xen on KVM")

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=45931

Nadav Har'El <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #1 from Nadav Har'El <[email protected]> 2013-02-11 15:10:48 --- After Avi fixed the missing support CPU_BASED_RDTSC_EXITING, what remains (at least) is missing support for VM_EXIT_ACK_INTR_ON_EXIT.

See the following thread on how this support might be added:
https://patchwork.kernel.org/patch/789632/

Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
author
#3 lore raw

[Bug 45931] Nested Virt: VMX can't be initialized in L1 Xen ("Xen on KVM")

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=45931

Zhou, Chao <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #2 from Zhou, Chao <[email protected]> ---
kvm.git + qemu.git：198c74f4_a41b2c99
host kernel :3.15.0-rc1
test on Romley_EP, crete L1(xen on kvm) guest with "-cpu host", then create L2 guest, L1 guest can boot up.

some log with command "xl dmesg" in L1 guest
(XEN) Platform timer is 100.000MHz HPET
(XEN) Allocated console ring of 32 KiB.
(XEN) VMX: Supported advanced features:

(XEN)  - APIC MMIO access virtualisation
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest

(XEN) HVM: ASIDs disabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected (XEN) HVM: HAP page sizes: 4kB, 2MB

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
author
#4 lore raw

[Bug 45931] Nested Virt: VMX can't be initialized in L1 Xen ("Xen on KVM")

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=45931

Paolo Bonzini <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|NEW                         |RESOLVED
                 CC|                            |[email protected]
         Resolution|---                         |CODE_FIX

--- Comment #3 from Paolo Bonzini <[email protected]> --- Patches were committed to kvm/next and will be included in 3.16.

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.