KVM: VMX: Intel PT configuration switch using XSAVES/XRSTORS on VM-Entry/Exit
cover KVM: VMX: Intel PT configuration switch using XSAVES/XRSTORS on VM-Entry/Exit
This patch set is mainly used for reduce the overhead of switch Intel PT configuation contex on VM-Entry/Exit by XSAVES/XRSTORS instructions.
I measured the cycles number of context witch on Manual and XSAVES/XRSTORES by rdtsc, and the data as below:
Manual save(rdmsr): ~334 cycles Manual restore(wrmsr): ~1668 cycles
XSAVES insturction: ~124 cycles XRSTORS instruction: ~378 cycles
Manual: Switch the configuration by rdmsr and wrmsr instruction,
and there have 8 registers need to be saved or restore. They are IA32_RTIT_OUTPUT_BASE, *_OUTPUT_MASK_PTRS, *_STATUS, *_CR3_MATCH, *_ADDR0_A, *_ADDR0_B, *_ADDR1_A, *_ADDR1_B.
XSAVES/XRSTORS: Switch the configuration context by XSAVES/XRSTORS
instructions. This patch set will allocate separate "struct fpu" structure to save host and guest PT state. Only a small portion of this structure will be used because we only save/restore PT state (not save AVX, AVX-512, MPX, PKRU and so on).
This patch set also do some code clean e.g. patch 2 will reuse the fpu pt_state to save the PT configuration contex and patch 3 will dymamic allocate Intel PT configuration state.
Luwei Kang (6):
x86/fpu: Introduce new fpu state for Intel processor trace
KVM: VMX: Reuse the pt_state structure for PT context
KVM: VMX: Dymamic allocate Intel PT configuration state
KVM: VMX: Allocate XSAVE area for Intel PT configuration
KVM: VMX: Intel PT configration context switch using XSAVES/XRSTORS
KVM: VMX: Get PT state from xsave area to variables
arch/x86/include/asm/fpu/types.h | 13 ++ arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/vmx/vmx.c | 338 ++++++++++++++++++++++++++------------- arch/x86/kvm/vmx/vmx.h | 21 +-- 4 files changed, 243 insertions(+), 131 deletions(-)
Signature · 1 line
1.8.3.1
PATCH 1/6 x86/fpu: Introduce new fpu state for Intel processor trace
Introduce new fpu state structure pt_state to save Intel processor trace configuration. The upcoming using XSAVES/XRSTORS to switch the Intel PT configuration on VM-Entry/Exit will use this structure.
--- arch/x86/include/asm/fpu/types.h | 13 +++++++++++++ 1 file changed, 13 insertions(+)
arch/x86/include/asm/fpu/types.h +13 -0
@@ -221,6 +221,19 @@ struct avx_512_hi16_state { } __packed;/* +* State component 8 is used for some 64-bit registers +* of Intel processor trace. +*/ +struct pt_state { +u64 rtit_ctl; +u64 rtit_output_base; +u64 rtit_output_mask; +u64 rtit_status; +u64 rtit_cr3_match; +u64 rtit_addrx_ab[0 ]; +} __packed; + +/* * State component 9: 32-bit PKRU register. The state is * 8 bytes long but only 4 bytes is used currently. */
Signature · 1 line
1.8.3.1
PATCH 2/6 KVM: VMX: Reuse the pt_state structure for PT context
In reply to Luwei KangRemove the previous pt_ctx structure and use pt_state to save the PT configuration because they are saved the same things.
Add *_ctx postfix to different with the upcoming host and guest fpu pointer for PT state.
--- arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/vmx/vmx.c | 96 +++++++++++++++++++++-------------------------- arch/x86/kvm/vmx/vmx.h | 16 +------- 3 files changed, 46 insertions(+), 68 deletions(-)
arch/x86/kvm/vmx/nested.c +1 -1
@@ -4201,7 +4201,7 @@ static int enter_vmx_operation(struct kvm_vcpu *vcpu) vmx->nested.vmxon = true;if (pt_mode ==PT_MODE_HOST_GUEST ) { - vmx->pt_desc.guest.ctl =0 ; + vmx->pt_desc.guest_ctx.rtit_ctl =0 ;pt_update_intercept_for_msr (vmx); }
arch/x86/kvm/vmx/vmx.c +43 -53
220 lines not on this page · raw message
arch/x86/kvm/vmx/vmx.h +2 -14
@@ -65,24 +65,12 @@ struct pi_desc {u32 rsvd[6 ]; }__aligned(64) ; -#define RTIT_ADDR_RANGE 4 - -struct pt_ctx { -u64 ctl; -u64 status; -u64 output_base; -u64 output_mask; -u64 cr3_match; -u64 addr_a[RTIT_ADDR_RANGE ]; -u64 addr_b[RTIT_ADDR_RANGE ]; -}; -struct pt_desc {u64 ctl_bitmask;u32 addr_range;u32 caps[PT_CPUID_REGS_NUM *PT_CPUID_LEAVES ]; -struct pt_ctx host; -struct pt_ctx guest; +struct pt_state host_ctx; +struct pt_state guest_ctx; };/*
Signature · 1 line
1.8.3.1
PATCH 3/6 KVM: VMX: Dymamic allocate Intel PT configuration state
In reply to Luwei KangThis patch change the Intel PT configuration state to structure pointer so that we only need to allocate the state buffer when Intel PT working in HOST_GUEST mode.
--- arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/vmx/vmx.c | 202 +++++++++++++++++++++++++++------------------- arch/x86/kvm/vmx/vmx.h | 6 +- 3 files changed, 121 insertions(+), 89 deletions(-)
arch/x86/kvm/vmx/nested.c +1 -1
@@ -4201,7 +4201,7 @@ static int enter_vmx_operation(struct kvm_vcpu *vcpu) vmx->nested.vmxon = true;if (pt_mode ==PT_MODE_HOST_GUEST ) { - vmx->pt_desc.guest_ctx.rtit_ctl =0 ; + vmx->pt_desc->guest_ctx->rtit_ctl =0 ;pt_update_intercept_for_msr (vmx); }
arch/x86/kvm/vmx/vmx.c +117 -85
414 lines not on this page · raw message
arch/x86/kvm/vmx/vmx.h +3 -3
@@ -69,8 +69,8 @@ struct pt_desc {u64 ctl_bitmask;u32 addr_range;u32 caps[PT_CPUID_REGS_NUM *PT_CPUID_LEAVES ]; -struct pt_state host_ctx; -struct pt_state guest_ctx; +struct pt_state *host_ctx; +struct pt_state *guest_ctx; };/*
@@ -259,7 +259,7 @@ struct vcpu_vmx {u64 msr_ia32_feature_control_valid_bits;u64 ept_pointer; -struct pt_desc pt_desc; +struct pt_desc *pt_desc; };enum ept_pointers_status {
Signature · 1 line
1.8.3.1
PATCH 4/6 KVM: VMX: Allocate XSAVE area for Intel PT configuration
In reply to Luwei KangAllocate XSAVE area for host and guest Intel PT configuration when Intel PT working in HOST_GUEST mode. Intel PT configuration state can be saved using XSAVES and restored by XRSTORS instruction.
--- arch/x86/kvm/vmx/vmx.c | 25 ++++++++++++++++++++++++- arch/x86/kvm/vmx/vmx.h | 3 +++ 2 files changed, 27 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +24 -1
@@ -1033,6 +1033,7 @@ static void pt_guest_exit(struct vcpu_vmx *vmx)static int pt_init (struct vcpu_vmx *vmx) { +unsigned int eax, ebx, ecx, edx;u32 pt_state_sz =sizeof (struct pt_state ) +sizeof (u64) *intel_pt_validate_hw_cap (PT_CAP_num_address_ranges) *2 ;
@@ -1044,13 +1045,35 @@ static int pt_init(struct vcpu_vmx *vmx) vmx->pt_desc->host_ctx = (struct pt_state *)(vmx->pt_desc +1 ); vmx->pt_desc->guest_ctx = (void *)vmx->pt_desc->host_ctx + pt_state_sz; +cpuid_count (XSTATE_CPUID ,1 , &eax, &ebx, &ecx, &edx); +if (ecx &XFEATURE_MASK_PT ) { + vmx->pt_desc->host_xs =kmem_cache_zalloc (x86_fpu_cache, +GFP_KERNEL_ACCOUNT ); + vmx->pt_desc->guest_xs =kmem_cache_zalloc (x86_fpu_cache, +GFP_KERNEL_ACCOUNT ); +if (!vmx->pt_desc->host_xs || !vmx->pt_desc->guest_xs) { +if (vmx->pt_desc->host_xs) +kmem_cache_free (x86_fpu_cache, + vmx->pt_desc->host_xs); +if (vmx->pt_desc->guest_xs) +kmem_cache_free (x86_fpu_cache, + vmx->pt_desc->guest_xs); + }else + vmx->pt_desc->pt_xsave = true; + } +return 0 ; }static void pt_uninit (struct vcpu_vmx *vmx) { -if (pt_mode ==PT_MODE_HOST_GUEST ) +if (pt_mode ==PT_MODE_HOST_GUEST ) {kfree (vmx->pt_desc); +if (vmx->pt_desc->pt_xsave) { +kmem_cache_free (x86_fpu_cache, vmx->pt_desc->host_xs); +kmem_cache_free (x86_fpu_cache, vmx->pt_desc->guest_xs); + } + } }void vmx_prepare_switch_to_guest (struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/vmx.h +3 -0
@@ -69,8 +69,11 @@ struct pt_desc {u64 ctl_bitmask;u32 addr_range;u32 caps[PT_CPUID_REGS_NUM *PT_CPUID_LEAVES ]; +bool pt_xsave;struct pt_state *host_ctx;struct pt_state *guest_ctx; +struct fpu *host_xs; +struct fpu *guest_xs; };/*
Signature · 1 line
1.8.3.1
PATCH 5/6 KVM: VMX: Intel PT configration context switch using XSAVES/XRSTORS
In reply to Luwei KangThis patch add the support of using XSAVES/XRSTORS to do the Intel processor trace context switch.
Because of native driver didn't set the XSS[bit8] to enabled the PT state in xsave area, so this patch only set this bit before XSAVE/XRSTORS intstuction executtion and restore the original value after.
The flag "initialized" need to be cleared when PT is change from enabled to disabled. Guest may modify PT MSRs when PT is disabled and they are only saved in variables.
We need to reload these value to HW manual when PT is enabled.
--- arch/x86/kvm/vmx/vmx.c | 80 ++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 65 insertions(+), 15 deletions(-)
arch/x86/kvm/vmx/vmx.c +65 -15
@@ -1002,33 +1002,83 @@ static inline void pt_save_msr(struct pt_state *ctx, u32 addr_range)static void pt_guest_enter (struct vcpu_vmx *vmx) { +struct pt_desc *desc; +int err; +if (pt_mode ==PT_MODE_SYSTEM )return ; -/* -* GUEST_IA32_RTIT_CTL is already set in the VMCS. -* Save host state before VM entry. -*/ -rdmsrl (MSR_IA32_RTIT_CTL , vmx->pt_desc->host_ctx->rtit_ctl); -if (vmx->pt_desc->guest_ctx->rtit_ctl &RTIT_CTL_TRACEEN ) { -wrmsrl (MSR_IA32_RTIT_CTL ,0 ); -pt_save_msr (vmx->pt_desc->host_ctx, vmx->pt_desc->addr_range); -pt_load_msr (vmx->pt_desc->guest_ctx, vmx->pt_desc->addr_range); + desc = vmx->pt_desc; + +rdmsrl (MSR_IA32_RTIT_CTL , desc->host_ctx->rtit_ctl); + +if (desc->guest_ctx->rtit_ctl &RTIT_CTL_TRACEEN ) { +if (likely (desc->pt_xsave)) { +wrmsrl (MSR_IA32_XSS , host_xss |XFEATURE_MASK_PT ); +/* +* XSAVES instruction will clears the TeaceEn after +* saving the value of RTIT_CTL and before saving any +* other PT state. +*/ +XSTATE_XSAVE (&desc->host_xs->state.xsave, +XFEATURE_MASK_PT ,0 , err); +/* +* Still need to load the guest PT state manual if +* PT stste not populated in xsave area. +*/ +if (desc->guest_xs->initialized) +XSTATE_XRESTORE (&desc->guest_xs->state.xsave, +XFEATURE_MASK_PT ,0 ); +else +pt_load_msr (desc->guest_ctx, desc->addr_range); + +wrmsrl (MSR_IA32_XSS , host_xss); + }else { +if (desc->host_ctx->rtit_ctl &RTIT_CTL_TRACEEN ) +wrmsrl (MSR_IA32_RTIT_CTL ,0 ); + +pt_save_msr (desc->host_ctx, desc->addr_range); +pt_load_msr (desc->guest_ctx, desc->addr_range); + } } }static void pt_guest_exit (struct vcpu_vmx *vmx) { +struct pt_desc *desc; +int err; +if (pt_mode ==PT_MODE_SYSTEM )return ; -if (vmx->pt_desc->guest_ctx->rtit_ctl &RTIT_CTL_TRACEEN ) { -pt_save_msr (vmx->pt_desc->guest_ctx, vmx->pt_desc->addr_range); -pt_load_msr (vmx->pt_desc->host_ctx, vmx->pt_desc->addr_range); - } + desc = vmx->pt_desc; + +if (desc->guest_ctx->rtit_ctl &RTIT_CTL_TRACEEN ) { +if (likely (desc->pt_xsave)) { +wrmsrl (MSR_IA32_XSS , host_xss |XFEATURE_MASK_PT ); +/* +* Save guest state. TraceEn is 0 before and after +* XSAVES instruction because RTIT_CTL will be cleared +* on VM-exit (VM Exit control bit25). +*/ +XSTATE_XSAVE (&desc->guest_xs->state.xsave, +XFEATURE_MASK_PT ,0 , err); + desc->guest_xs->initialized =1 ; +/* +* Resume host PT state and PT may enabled after this +* instruction if host PT is enabled before VM-entry. +*/ +XSTATE_XRESTORE (&desc->host_xs->state.xsave, +XFEATURE_MASK_PT ,0 ); +wrmsrl (MSR_IA32_XSS , host_xss); + }else { +pt_save_msr (desc->guest_ctx, desc->addr_range); +pt_load_msr (desc->host_ctx, desc->addr_range); -/* Reload host state (IA32_RTIT_CTL will be cleared on VM exit). */ -wrmsrl (MSR_IA32_RTIT_CTL , vmx->pt_desc->host_ctx->rtit_ctl); +wrmsrl (MSR_IA32_RTIT_CTL , desc->host_ctx->rtit_ctl); + } + }else if (desc->host_ctx->rtit_ctl &RTIT_CTL_TRACEEN ) +wrmsrl (MSR_IA32_RTIT_CTL , desc->host_ctx->rtit_ctl); }static int pt_init (struct vcpu_vmx *vmx)
Signature · 1 line
1.8.3.1
PATCH 6/6 KVM: VMX: Get PT state from xsave area to variables
In reply to Luwei KangThis patch get the Intel PT state from xsave area to variables when PT is change from enabled to disabled. Because PT state is saved/restored to/from xsave area by XSAVES/XRSTORES instructions when Intel PT is enabled. The KVM guest may read this MSRs when PT is disabled but the real value is saved in xsave area not variables.
--- arch/x86/kvm/vmx/vmx.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
arch/x86/kvm/vmx/vmx.c +13 -0
@@ -1000,6 +1000,16 @@ static inline void pt_save_msr(struct pt_state *ctx, u32 addr_range)rdmsrl (MSR_IA32_RTIT_ADDR0_A + i, ctx->rtit_addrx_ab[i]); } +static void pt_state_get (struct pt_state *ctx,struct fpu *fpu,u32 addr_range) +{ +char *buff = fpu->state.xsave.extended_state_area; + +/* skip riti_ctl register */ +memcpy (&ctx->rtit_output_base, buff +sizeof (u64), +sizeof (struct pt_state ) -sizeof (u64) + +sizeof (u64) * addr_range *2 ); +} +static void pt_guest_enter (struct vcpu_vmx *vmx) {struct pt_desc *desc;
@@ -1040,6 +1050,9 @@ static void pt_guest_enter(struct vcpu_vmx *vmx)pt_save_msr (desc->host_ctx, desc->addr_range);pt_load_msr (desc->guest_ctx, desc->addr_range); } + }else if (desc->pt_xsave && desc->guest_xs->initialized) { +pt_state_get (desc->guest_ctx, desc->guest_xs, desc->addr_range); + desc->guest_xs->initialized =0 ; } }
Signature · 1 line
1.8.3.1
Paolo Bonzini
maintainerQuoted 37 lines · Luwei Kang
This patch set is mainly used for reduce the overhead of switch Intel PT configuation contex on VM-Entry/Exit by XSAVES/XRSTORS instructions.
I measured the cycles number of context witch on Manual and XSAVES/XRSTORES by rdtsc, and the data as below:
Manual save(rdmsr): ~334 cycles Manual restore(wrmsr): ~1668 cyclesXSAVES insturction: ~124 cycles XRSTORS instruction: ~378 cyclesManual: Switch the configuration by rdmsr and wrmsr instruction,
and there have 8 registers need to be saved or restore. They areIA32_RTIT_OUTPUT_BASE,*_OUTPUT_MASK_PTRS,*_STATUS,*_CR3_MATCH,*_ADDR0_A,*_ADDR0_B,*_ADDR1_A,*_ADDR1_B.
XSAVES/XRSTORS: Switch the configuration context by XSAVES/XRSTORS
instructions. This patch set will allocate separate "struct fpu" structure to save host and guest PT state. Only a small portion of this structure will be used because we only save/restore PT state (not save AVX, AVX-512, MPX, PKRU and so on).This patch set also do some code clean e.g. patch 2 will reuse the fpu
pt_stateto save the PT configuration contex and patch 3 will dymamic allocate Intel PT configuration state.Luwei Kang (6):
x86/fpu: Introduce new fpu state for Intel processor trace
KVM: VMX: Reuse thept_statestructure for PT context
KVM: VMX: Dymamic allocate Intel PT configuration state
KVM: VMX: Allocate XSAVE area for Intel PT configuration
KVM: VMX: Intel PT configration context switch using XSAVES/XRSTORS
KVM: VMX: Get PT state from xsave area to variables
Quoted 6 lines
arch/x86/include/asm/fpu/types.h | 13 ++ arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/vmx/vmx.c | 338 ++++++++++++++++++++++++++------------- arch/x86/kvm/vmx/vmx.h | 21 +-- 4 files changed, 243 insertions(+), 131 deletions(-)
Luwei, I found I had missed this series. Can you check whether it needs a rebase, since I don't have hardware that supports it?
Paolo