kvm/vmx: EPTP switching test
PATCH 1/1 kvm/vmx: EPTP switching test
This patch adds a new parameter: eptp_switching_test, which enables testing EPT switching on VMX if supported by hardware. All EPT entries are initialized to the same value so this adds no useful functionality by itself, but can be used to test VMFUNC performance, and serve as a basis for future features based on EPTP switching.
Support for nested virt is not enabled.
This was tested using the following code within guest:
#define VMX_VMFUNC ".byte 0x0f,0x01,0xd4"
static void vmfunc(unsigned int nr, unsigned int ept)
{
asm volatile(VMX_VMFUNC
:
: "a"(nr), "c"(ept)
: "memory");
}VMFUNC instruction cost was measured at ~122 cycles. (Note: recent versions of gnu toolchain support
the vmfunc instruction - removing the need for writing the bytecode manually).
---
I think I'd like to put this upstream so future eptp switching work can be implemented on top. Comments?
arch/x86/include/asm/vmx.h | 7 ++++ arch/x86/kvm/vmx.c | 84 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+)
arch/x86/include/asm/vmx.h +7 -0
@@ -69,10 +69,13 @@#define SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY 0x00000200#define SECONDARY_EXEC_PAUSE_LOOP_EXITING 0x00000400#define SECONDARY_EXEC_ENABLE_INVPCID 0x00001000 +#define SECONDARY_EXEC_ENABLE_VM_FUNCTIONS 0x00002000#define SECONDARY_EXEC_SHADOW_VMCS 0x00004000#define SECONDARY_EXEC_ENABLE_PML 0x00020000#define SECONDARY_EXEC_XSAVES 0x00100000 +/* Definitions for VM-function controls */ +#define VM_FUNCTION_EPTP_SWITCHING 0x00000001#define PIN_BASED_EXT_INTR_MASK 0x00000001#define PIN_BASED_NMI_EXITING 0x00000008
@@ -153,6 +156,8 @@ enum vmcs_field {APIC_ACCESS_ADDR_HIGH =0x00002015 ,POSTED_INTR_DESC_ADDR =0x00002016 ,POSTED_INTR_DESC_ADDR_HIGH =0x00002017 , +VM_FUNCTION_CTRL =0x00002018 , +VM_FUNCTION_CTRL_HIGH =0x00002019 ,EPT_POINTER =0x0000201a ,EPT_POINTER_HIGH =0x0000201b , EOI_EXIT_BITMAP0 = 0x0000201c,
@@ -163,6 +168,8 @@ enum vmcs_field {EOI_EXIT_BITMAP2_HIGH =0x00002021 ,EOI_EXIT_BITMAP3 =0x00002022 ,EOI_EXIT_BITMAP3_HIGH =0x00002023 , +EPTP_LIST_ADDRESS =0x00002024 , +EPTP_LIST_ADDRESS_HIGH =0x00002025 ,VMREAD_BITMAP =0x00002026 ,VMWRITE_BITMAP =0x00002028 , XSS_EXIT_BITMAP = 0x0000202C,
arch/x86/kvm/vmx.c +84 -0
152 lines not on this page · raw message
Signature · 1 line
MST
2015-11-15 18:00+0200, Michael S. Tsirkin:
Quoted 24 lines
This patch adds a new parameter:
eptp_switching_test, which enablestesting EPT switching on VMX if supported by hardware. All EPT entries are initialized to the same value so this adds no useful functionality by itself, but can be used to test VMFUNC performance, and serve as a basis for future features based on EPTP switching.
Support for nested virt is not enabled.
This was tested using the following code within guest:
#define VMX_VMFUNC ".byte 0x0f,0x01,0xd4" static void vmfunc(unsigned int nr, unsigned int ept) { asm volatile(VMX_VMFUNC : : "a"(nr), "c"(ept) : "memory"); }VMFUNC instruction cost was measured at ~122 cycles. (Note: recent versions of gnu toolchain support
the vmfunc instruction - removing the need for writing the bytecode manually).
Quoted 5 lines
Signed-off-by Michael S. Tsirkin <[email protected]>---
I think I'd like to put this upstream so future eptp switching work can be implemented on top. Comments?
I'd wait for the future. Patch is already on the list so people interested in benchmarking VMFUNC can quickly compile a kernel and developers will need to overwrite the code anyway.
(And I think that eptp switching is expected to be used in conjuction
with #VE, so it'd then make sense to implement a nop for it as well.)
arch/x86/include/asm/vmx.h static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf) · line 3038
Quoted 6 lines
arch/x86/include/asm/vmx.h +1
-0@@ -3011,6 +3035,7 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf)SECONDARY_EXEC_PAUSE_LOOP_EXITING |SECONDARY_EXEC_RDTSCP |SECONDARY_EXEC_ENABLE_INVPCID | + SECONDARY_EXEC_ENABLE_VM_FUNCTIONS |
The VMFUNC vmexit should be handled to prevent guests from triggering a WARN_ON on the host. (VMFUNC did just #UD before this patch.)
After that, it's ok for KVM.
Quoted 28 lines · On Mon, Nov 16, 2015 at 06:51:06PM +0100…
2015-11-15 18:00+0200, Michael S. Tsirkin:
Quoted 27 lines
This patch adds a new parameter:
eptp_switching_test, which enablestesting EPT switching on VMX if supported by hardware. All EPT entries are initialized to the same value so this adds no useful functionality by itself, but can be used to test VMFUNC performance, and serve as a basis for future features based on EPTP switching.
Support for nested virt is not enabled.
This was tested using the following code within guest:
#define VMX_VMFUNC ".byte 0x0f,0x01,0xd4" static void vmfunc(unsigned int nr, unsigned int ept) { asm volatile(VMX_VMFUNC : : "a"(nr), "c"(ept) : "memory"); }VMFUNC instruction cost was measured at ~122 cycles. (Note: recent versions of gnu toolchain support
the vmfunc instruction - removing the need for writing the bytecode manually).Signed-off-by Michael S. Tsirkin <[email protected]>---
Quoted 6 lines
Quoted 2 lines
I think I'd like to put this upstream so future eptp switching work can be implemented on top. Comments?
I'd wait for the future. Patch is already on the list so people interested in benchmarking VMFUNC can quickly compile a kernel and developers will need to overwrite the code anyway.
It'll bitrot though. But I'll let Paolo decide that.
Quoted 2 lines
(And I think that eptp switching is expected to be used in conjuction
with #VE, so it'd then make sense to implement a nop for it as well.)
No idea how would I even test it, so I'm not interested in #VE at this point. If you are - go ahead and post a patch for that on top though, why not.
Quoted 9 lines
arch/x86/include/asm/vmx.h static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf) · line 3038
Quoted 6 lines
arch/x86/include/asm/vmx.h +1
-0@@ -3011,6 +3035,7 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf)SECONDARY_EXEC_PAUSE_LOOP_EXITING |SECONDARY_EXEC_RDTSCP |SECONDARY_EXEC_ENABLE_INVPCID | + SECONDARY_EXEC_ENABLE_VM_FUNCTIONS |The VMFUNC vmexit should be handled to prevent guests from triggering a
WARN_ONon the host. (VMFUNC did just #UD before this patch.)
Do you mean VMFUNC other than EPTP switch 0? True, thanks!
Quoted 2 lines
After that, it's ok for KVM.
2015-11-16 19:59+0200, Michael S. Tsirkin:
Quoted 8 lines · On Mon, Nov 16, 2015 at 06:51:06PM +0100…
Quoted 3 lines
2015-11-15 18:00+0200, Michael S. Tsirkin:
(And I think that eptp switching is expected to be used in conjuction
with #VE, so it'd then make sense to implement a nop for it as well.)No idea how would I even test it, so I'm not interested in #VE at this point. If you are - go ahead and post a patch for that on top though, why not.
I thought that it's going to be simpler to provide functionality (that utilizes eptp switching) to the guest through #VE, which probably isn't true as I think more about it. (Not interested in implementing it :])
Quoted 6 lines · On Mon, Nov 16, 2015 at 10:18 AM, =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?=…
2015-11-16 19:59+0200, Michael S. Tsirkin:
Quoted 5 lines · On Mon, Nov 16, 2015 at 06:51:06PM +0100…
Quoted 3 lines
2015-11-15 18:00+0200, Michael S. Tsirkin:
(And I think that eptp switching is expected to be used in conjuction
with #VE, so it'd then make sense to implement a nop for it as well.)
Quoted 7 lines
Quoted 3 lines
No idea how would I even test it, so I'm not interested in #VE at this point. If you are - go ahead and post a patch for that on top though, why not.
I thought that it's going to be simpler to provide functionality (that utilizes eptp switching) to the guest through #VE, which probably isn't true as I think more about it. (Not interested in implementing it :])
I'm willing to review a #VE stub. I'm not likely to implement it.
--Andy
Paolo Bonzini
maintainerQuoted 6 lines · On 16/11/2015 19:18, =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= wrote:
Quoted 3 lines
Quoted 3 lines
No idea how would I even test it, so I'm not interested in #VE at this point. If you are - go ahead and post a patch for that on top though, why not.
I thought that it's going to be simpler to provide functionality (that utilizes eptp switching) to the guest through #VE, which probably isn't true as I think more about it. (Not interested in implementing it :])
#VE and EPTP switching are distinct features, one does not imply the other.
Unfortunately, EPTP switching is designed for a very specific use case where the hypervisor is effectively part of the kernel, and the kernel is trusted to some extent. Examples include antivirus software and virtual machines. Antiviruses do use VMFUNC, that's as far as I know the only current use case of it
(https://embedded.communities.intel.com/community/en/applications/blog/2013/06/13/roving-reporter-enhancing-retail-security-and-manageability-with-4th-generation-intel-core-processors).
So I'm against this patch, but only because I'm not sure why KVM would ever use EPTP switching in its current incarnation. The guest kernel is absolutely not trusted by KVM.
Paolo
Michael S. Tsirkin wrote on 2015-11-16:
Quoted 22 lines
This patch adds a new parameter:
eptp_switching_test, which enables testing EPT switching on VMX if supported by hardware. All EPT entries are initialized to the same value so this adds no useful functionality by itself, but can be used to test VMFUNC performance, and serve as a basis for future features based on EPTP switching.Support for nested virt is not enabled.
This was tested using the following code within guest:
#define VMX_VMFUNC ".byte 0x0f,0x01,0xd4" static void vmfunc(unsigned int nr, unsigned int ept) { asm volatile(VMX_VMFUNC : : "a"(nr), "c"(ept) : "memory"); }VMFUNC instruction cost was measured at ~122 cycles. (Note: recent versions of gnu toolchain support the vmfunc instruction - removing the need for writing the bytecode manually).
Quoted 5 lines
Signed-off-by Michael S. Tsirkin <[email protected]>---
I think I'd like to put this upstream so future eptp switching work can be implemented on top. Comments?
We have a different version in hand which is using separate EPTP. As you known, the patch will be more complex if using separate EPTP. And there are still lots of thing need to do in our version. We will send out for comments soon.
Best regards,
Yang
Paolo Bonzini
maintainerQuoted 1 line · Yang Z
We have a different version in hand which is using separate EPTP.
Can you say in advance what you are using EPTP switching for? Offlist if necessary.
Paolo
Quoted 5 lines · Paolo Bonzini
Quoted 1 line · Yang Z
We have a different version in hand which is using separate EPTP.
Can you say in advance what you are using EPTP switching for? Offlist if necessary.
Hi Paolo,
We are using EPTP switching for a protected inter-VM communication design, as shown in the slides (#23) here: http://events.linuxfoundation.org/sites/events/files/slides/Jun_Nakajima_NFV_KVM%202015_final.pdf
Best,
Wei
Paolo Bonzini
maintainerQuoted 5 lines · Wei W
Quoted 4 lines · Paolo Bonzini
Quoted 2 lines · Yang Z
We have a different version in hand which is using separate EPTP.
Quoted 8 lines
Quoted 2 lines
Can you say in advance what you are using EPTP switching for? Offlist if necessary.
Hi Paolo,
We are using EPTP switching for a protected inter-VM communication design, as shown in the slides (#23) here:
http://events.linuxfoundation.org/sites/events/files/slides/Jun_Nakajima_NFV_KVM%202015_final.pdf
[offlist, adding [email protected]]
If the EPTP switch is only adding extra data pages (e.g. mapping another guest's memory inside a PCI BAR), this can work.
However, slides 24 and 25 suggest that the executable pages change between the two EPTP views, similar to Jun's KVM Forum 2014 presentation. Michael and I explained in Seattle that this only works if the guest is trusted. I am a bit disappointed that Intel continued developing this feature without contacting us or without urging us to present our issues more formally.
I think I should make this very clear: I am not going to accept in KVM a feature that requires the guest to be trusted. A trusted guest kernel may make sense for other applications of VMFUNC (e.g. McAfee memory scan) but not for virtualization; if the guest is trusted, you don't have virtualization anymore.
Michael and I are going to present our findings to Intel soon. This will hopefully clarify why the guest has to be trusted. We will also present possible extensions to VMFUNC that enable its usage with untrusted guests, albeit only at CPL=0.
Asit Mallick is going to contact Jun about this so we can organize the meeting. Unfortunately it is going to be hard for everyone to attend since we have people in Europe, US and China, but we will share the slides.
Thanks,
Paolo