Skip to contentSkip to conversation

KVM: VMX: Intel PT configuration switch using XSAVES/XRSTORS on VM-Entry/Exit

PATCH In review Luwei Kang 2 participants 8 messages May 16, 2019 – Nov 11, 2019

Subsystems (3)
lore
Share & tools
lore

Luwei Kang

author
#0 lore raw

cover KVM: VMX: Intel PT configuration switch using XSAVES/XRSTORS on VM-Entry/Exit

This patch set is mainly used for reduce the overhead of switch Intel PT configuation contex on VM-Entry/Exit by XSAVES/XRSTORS instructions.

I measured the cycles number of context witch on Manual and XSAVES/XRSTORES by rdtsc, and the data as below:

Manual save(rdmsr):     ~334  cycles
Manual restore(wrmsr):  ~1668 cycles
XSAVES insturction:     ~124  cycles
XRSTORS instruction:    ~378  cycles

Manual: Switch the configuration by rdmsr and wrmsr instruction,
and there have 8 registers need to be saved or restore. They are IA32_RTIT_OUTPUT_BASE, *_OUTPUT_MASK_PTRS, *_STATUS, *_CR3_MATCH, *_ADDR0_A, *_ADDR0_B, *_ADDR1_A, *_ADDR1_B.
XSAVES/XRSTORS: Switch the configuration context by XSAVES/XRSTORS
instructions. This patch set will allocate separate "struct fpu" structure to save host and guest PT state. Only a small portion of this structure will be used because we only save/restore PT state (not save AVX, AVX-512, MPX, PKRU and so on).

This patch set also do some code clean e.g. patch 2 will reuse the fpu pt_state to save the PT configuration contex and patch 3 will dymamic allocate Intel PT configuration state.

Luwei Kang (6):
x86/fpu: Introduce new fpu state for Intel processor trace
KVM: VMX: Reuse the pt_state structure for PT context
KVM: VMX: Dymamic allocate Intel PT configuration state
KVM: VMX: Allocate XSAVE area for Intel PT configuration
KVM: VMX: Intel PT configration context switch using XSAVES/XRSTORS
KVM: VMX: Get PT state from xsave area to variables

 arch/x86/include/asm/fpu/types.h |  13 ++
 arch/x86/kvm/vmx/nested.c        |   2 +-
 arch/x86/kvm/vmx/vmx.c           | 338 ++++++++++++++++++++++++++-------------
 arch/x86/kvm/vmx/vmx.h           |  21 +--
 4 files changed, 243 insertions(+), 131 deletions(-)
Signature · 1 line
1.8.3.1
author
#1 lore raw

PATCH 1/6 x86/fpu: Introduce new fpu state for Intel processor trace

Introduce new fpu state structure pt_state to save Intel processor trace configuration. The upcoming using XSAVES/XRSTORS to switch the Intel PT configuration on VM-Entry/Exit will use this structure.

---
 arch/x86/include/asm/fpu/types.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/x86/include/asm/fpu/types.h +13 -0
index 2e32e17..8cbb42e 100644
@@ -221,6 +221,19 @@ struct avx_512_hi16_state {
 } __packed;
 
 /*
+ * State component 8 is used for some 64-bit registers
+ * of Intel processor trace.
+ */
+struct pt_state {
+	u64 rtit_ctl;
+	u64 rtit_output_base;
+	u64 rtit_output_mask;
+	u64 rtit_status;
+	u64 rtit_cr3_match;
+	u64 rtit_addrx_ab[0];
+} __packed;
+
+/*
  * State component 9: 32-bit PKRU register.  The state is
  * 8 bytes long but only 4 bytes is used currently.
  */
Signature · 1 line
1.8.3.1
author
#2 lore raw

PATCH 2/6 KVM: VMX: Reuse the pt_state structure for PT context

In reply to Luwei Kang

Remove the previous pt_ctx structure and use pt_state to save the PT configuration because they are saved the same things.
Add *_ctx postfix to different with the upcoming host and guest fpu pointer for PT state.

---
 arch/x86/kvm/vmx/nested.c |  2 +-
 arch/x86/kvm/vmx/vmx.c    | 96 +++++++++++++++++++++--------------------------
 arch/x86/kvm/vmx/vmx.h    | 16 +-------
 3 files changed, 46 insertions(+), 68 deletions(-)
arch/x86/kvm/vmx/nested.c +1 -1
index f4b1ae4..e8d5c61 100644
@@ -4201,7 +4201,7 @@ static int enter_vmx_operation(struct kvm_vcpu *vcpu)
 	vmx->nested.vmxon = true;
 
 	if (pt_mode == PT_MODE_HOST_GUEST) {
-		vmx->pt_desc.guest.ctl = 0;
+		vmx->pt_desc.guest_ctx.rtit_ctl = 0;
 		pt_update_intercept_for_msr(vmx);
 	}
 
arch/x86/kvm/vmx/vmx.c +43 -53

220 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.h +2 -14
index 63d37cc..11ad856 100644
@@ -65,24 +65,12 @@ struct pi_desc {
 	u32 rsvd[6];
 } __aligned(64);
 
-#define RTIT_ADDR_RANGE		4
-
-struct pt_ctx {
-	u64 ctl;
-	u64 status;
-	u64 output_base;
-	u64 output_mask;
-	u64 cr3_match;
-	u64 addr_a[RTIT_ADDR_RANGE];
-	u64 addr_b[RTIT_ADDR_RANGE];
-};
-
 struct pt_desc {
 	u64 ctl_bitmask;
 	u32 addr_range;
 	u32 caps[PT_CPUID_REGS_NUM * PT_CPUID_LEAVES];
-	struct pt_ctx host;
-	struct pt_ctx guest;
+	struct pt_state host_ctx;
+	struct pt_state guest_ctx;
 };
 
 /*
Signature · 1 line
1.8.3.1
author
#3 lore raw

PATCH 3/6 KVM: VMX: Dymamic allocate Intel PT configuration state

In reply to Luwei Kang

This patch change the Intel PT configuration state to structure pointer so that we only need to allocate the state buffer when Intel PT working in HOST_GUEST mode.

---
 arch/x86/kvm/vmx/nested.c |   2 +-
 arch/x86/kvm/vmx/vmx.c    | 202 +++++++++++++++++++++++++++-------------------
 arch/x86/kvm/vmx/vmx.h    |   6 +-
 3 files changed, 121 insertions(+), 89 deletions(-)
arch/x86/kvm/vmx/nested.c +1 -1
index e8d5c61..349be88 100644
@@ -4201,7 +4201,7 @@ static int enter_vmx_operation(struct kvm_vcpu *vcpu)
 	vmx->nested.vmxon = true;
 
 	if (pt_mode == PT_MODE_HOST_GUEST) {
-		vmx->pt_desc.guest_ctx.rtit_ctl = 0;
+		vmx->pt_desc->guest_ctx->rtit_ctl = 0;
 		pt_update_intercept_for_msr(vmx);
 	}
 
arch/x86/kvm/vmx/vmx.c +117 -85

414 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.h +3 -3
index 11ad856..283f69d 100644
@@ -69,8 +69,8 @@ struct pt_desc {
 	u64 ctl_bitmask;
 	u32 addr_range;
 	u32 caps[PT_CPUID_REGS_NUM * PT_CPUID_LEAVES];
-	struct pt_state host_ctx;
-	struct pt_state guest_ctx;
+	struct pt_state *host_ctx;
+	struct pt_state *guest_ctx;
 };
 
 /*
@@ -259,7 +259,7 @@ struct vcpu_vmx {
 	u64 msr_ia32_feature_control_valid_bits;
 	u64 ept_pointer;
 
-	struct pt_desc pt_desc;
+	struct pt_desc *pt_desc;
 };
 
 enum ept_pointers_status {
Signature · 1 line
1.8.3.1
author
#4 lore raw

PATCH 4/6 KVM: VMX: Allocate XSAVE area for Intel PT configuration

In reply to Luwei Kang

Allocate XSAVE area for host and guest Intel PT configuration when Intel PT working in HOST_GUEST mode. Intel PT configuration state can be saved using XSAVES and restored by XRSTORS instruction.

---
 arch/x86/kvm/vmx/vmx.c | 25 ++++++++++++++++++++++++-
 arch/x86/kvm/vmx/vmx.h |  3 +++
 2 files changed, 27 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +24 -1
index 4595230..4691665 100644
@@ -1033,6 +1033,7 @@ static void pt_guest_exit(struct vcpu_vmx *vmx)
 
 static int pt_init(struct vcpu_vmx *vmx)
 {
+	unsigned int eax, ebx, ecx, edx;
 	u32 pt_state_sz = sizeof(struct pt_state) + sizeof(u64) *
 		intel_pt_validate_hw_cap(PT_CAP_num_address_ranges) * 2;
 
@@ -1044,13 +1045,35 @@ static int pt_init(struct vcpu_vmx *vmx)
 	vmx->pt_desc->host_ctx = (struct pt_state *)(vmx->pt_desc + 1);
 	vmx->pt_desc->guest_ctx = (void *)vmx->pt_desc->host_ctx + pt_state_sz;
 
+	cpuid_count(XSTATE_CPUID, 1, &eax, &ebx, &ecx, &edx);
+	if (ecx & XFEATURE_MASK_PT) {
+		vmx->pt_desc->host_xs = kmem_cache_zalloc(x86_fpu_cache,
+							GFP_KERNEL_ACCOUNT);
+		vmx->pt_desc->guest_xs = kmem_cache_zalloc(x86_fpu_cache,
+							GFP_KERNEL_ACCOUNT);
+		if (!vmx->pt_desc->host_xs || !vmx->pt_desc->guest_xs) {
+			if (vmx->pt_desc->host_xs)
+				kmem_cache_free(x86_fpu_cache,
+						vmx->pt_desc->host_xs);
+			if (vmx->pt_desc->guest_xs)
+				kmem_cache_free(x86_fpu_cache,
+						vmx->pt_desc->guest_xs);
+		} else
+			vmx->pt_desc->pt_xsave = true;
+	}
+
 	return 0;
 }
 
 static void pt_uninit(struct vcpu_vmx *vmx)
 {
-	if (pt_mode == PT_MODE_HOST_GUEST)
+	if (pt_mode == PT_MODE_HOST_GUEST) {
 		kfree(vmx->pt_desc);
+		if (vmx->pt_desc->pt_xsave) {
+			kmem_cache_free(x86_fpu_cache, vmx->pt_desc->host_xs);
+			kmem_cache_free(x86_fpu_cache, vmx->pt_desc->guest_xs);
+		}
+	}
 }
 
 void vmx_prepare_switch_to_guest(struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/vmx.h +3 -0
index 283f69d..e103991 100644
@@ -69,8 +69,11 @@ struct pt_desc {
 	u64 ctl_bitmask;
 	u32 addr_range;
 	u32 caps[PT_CPUID_REGS_NUM * PT_CPUID_LEAVES];
+	bool pt_xsave;
 	struct pt_state *host_ctx;
 	struct pt_state *guest_ctx;
+	struct fpu *host_xs;
+	struct fpu *guest_xs;
 };
 
 /*
Signature · 1 line
1.8.3.1
author
#5 lore raw

PATCH 5/6 KVM: VMX: Intel PT configration context switch using XSAVES/XRSTORS

In reply to Luwei Kang

This patch add the support of using XSAVES/XRSTORS to do the Intel processor trace context switch.

Because of native driver didn't set the XSS[bit8] to enabled the PT state in xsave area, so this patch only set this bit before XSAVE/XRSTORS intstuction executtion and restore the original value after.

The flag "initialized" need to be cleared when PT is change from enabled to disabled. Guest may modify PT MSRs when PT is disabled and they are only saved in variables.
We need to reload these value to HW manual when PT is enabled.

---
 arch/x86/kvm/vmx/vmx.c | 80 ++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 65 insertions(+), 15 deletions(-)
arch/x86/kvm/vmx/vmx.c +65 -15
index 4691665..d323e6b 100644
@@ -1002,33 +1002,83 @@ static inline void pt_save_msr(struct pt_state *ctx, u32 addr_range)
 
 static void pt_guest_enter(struct vcpu_vmx *vmx)
 {
+	struct pt_desc *desc;
+	int err;
+
 	if (pt_mode == PT_MODE_SYSTEM)
 		return;
 
-	/*
-	 * GUEST_IA32_RTIT_CTL is already set in the VMCS.
-	 * Save host state before VM entry.
-	 */
-	rdmsrl(MSR_IA32_RTIT_CTL, vmx->pt_desc->host_ctx->rtit_ctl);
-	if (vmx->pt_desc->guest_ctx->rtit_ctl & RTIT_CTL_TRACEEN) {
-		wrmsrl(MSR_IA32_RTIT_CTL, 0);
-		pt_save_msr(vmx->pt_desc->host_ctx, vmx->pt_desc->addr_range);
-		pt_load_msr(vmx->pt_desc->guest_ctx, vmx->pt_desc->addr_range);
+	desc = vmx->pt_desc;
+
+	rdmsrl(MSR_IA32_RTIT_CTL, desc->host_ctx->rtit_ctl);
+
+	if (desc->guest_ctx->rtit_ctl & RTIT_CTL_TRACEEN) {
+		if (likely(desc->pt_xsave)) {
+			wrmsrl(MSR_IA32_XSS, host_xss | XFEATURE_MASK_PT);
+			/*
+			 * XSAVES instruction will clears the TeaceEn after
+			 * saving the value of RTIT_CTL and before saving any
+			 * other PT state.
+			 */
+			XSTATE_XSAVE(&desc->host_xs->state.xsave,
+						XFEATURE_MASK_PT, 0, err);
+			/*
+			 * Still need to load the guest PT state manual if
+			 * PT stste not populated in xsave area.
+			 */
+			if (desc->guest_xs->initialized)
+				XSTATE_XRESTORE(&desc->guest_xs->state.xsave,
+						XFEATURE_MASK_PT, 0);
+			else
+				pt_load_msr(desc->guest_ctx, desc->addr_range);
+
+			wrmsrl(MSR_IA32_XSS, host_xss);
+		} else {
+			if (desc->host_ctx->rtit_ctl & RTIT_CTL_TRACEEN)
+				wrmsrl(MSR_IA32_RTIT_CTL, 0);
+
+			pt_save_msr(desc->host_ctx, desc->addr_range);
+			pt_load_msr(desc->guest_ctx, desc->addr_range);
+		}
 	}
 }
 
 static void pt_guest_exit(struct vcpu_vmx *vmx)
 {
+	struct pt_desc *desc;
+	int err;
+
 	if (pt_mode == PT_MODE_SYSTEM)
 		return;
 
-	if (vmx->pt_desc->guest_ctx->rtit_ctl & RTIT_CTL_TRACEEN) {
-		pt_save_msr(vmx->pt_desc->guest_ctx, vmx->pt_desc->addr_range);
-		pt_load_msr(vmx->pt_desc->host_ctx, vmx->pt_desc->addr_range);
-	}
+	desc = vmx->pt_desc;
+
+	if (desc->guest_ctx->rtit_ctl & RTIT_CTL_TRACEEN) {
+		if (likely(desc->pt_xsave)) {
+			wrmsrl(MSR_IA32_XSS, host_xss | XFEATURE_MASK_PT);
+			/*
+			 * Save guest state. TraceEn is 0 before and after
+			 * XSAVES instruction because RTIT_CTL will be cleared
+			 * on VM-exit (VM Exit control bit25).
+			 */
+			XSTATE_XSAVE(&desc->guest_xs->state.xsave,
+						XFEATURE_MASK_PT, 0, err);
+			desc->guest_xs->initialized = 1;
+			/*
+			 * Resume host PT state and PT may enabled after this
+			 * instruction if host PT is enabled before VM-entry.
+			 */
+			XSTATE_XRESTORE(&desc->host_xs->state.xsave,
+						XFEATURE_MASK_PT, 0);
+			wrmsrl(MSR_IA32_XSS, host_xss);
+		} else {
+			pt_save_msr(desc->guest_ctx, desc->addr_range);
+			pt_load_msr(desc->host_ctx, desc->addr_range);
 
-	/* Reload host state (IA32_RTIT_CTL will be cleared on VM exit). */
-	wrmsrl(MSR_IA32_RTIT_CTL, vmx->pt_desc->host_ctx->rtit_ctl);
+			wrmsrl(MSR_IA32_RTIT_CTL, desc->host_ctx->rtit_ctl);
+		}
+	} else if (desc->host_ctx->rtit_ctl & RTIT_CTL_TRACEEN)
+		wrmsrl(MSR_IA32_RTIT_CTL, desc->host_ctx->rtit_ctl);
 }
 
 static int pt_init(struct vcpu_vmx *vmx)
Signature · 1 line
1.8.3.1
author
#6 lore raw

PATCH 6/6 KVM: VMX: Get PT state from xsave area to variables

In reply to Luwei Kang

This patch get the Intel PT state from xsave area to variables when PT is change from enabled to disabled. Because PT state is saved/restored to/from xsave area by XSAVES/XRSTORES instructions when Intel PT is enabled. The KVM guest may read this MSRs when PT is disabled but the real value is saved in xsave area not variables.

---
 arch/x86/kvm/vmx/vmx.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/x86/kvm/vmx/vmx.c +13 -0
index d323e6b..d3e2569 100644
@@ -1000,6 +1000,16 @@ static inline void pt_save_msr(struct pt_state *ctx, u32 addr_range)
 		rdmsrl(MSR_IA32_RTIT_ADDR0_A + i, ctx->rtit_addrx_ab[i]);
 }
 
+static void pt_state_get(struct pt_state *ctx, struct fpu *fpu, u32 addr_range)
+{
+	char *buff = fpu->state.xsave.extended_state_area;
+
+	/* skip riti_ctl register */
+	memcpy(&ctx->rtit_output_base, buff + sizeof(u64),
+			sizeof(struct pt_state) - sizeof(u64) +
+			sizeof(u64) * addr_range * 2);
+}
+
 static void pt_guest_enter(struct vcpu_vmx *vmx)
 {
 	struct pt_desc *desc;
@@ -1040,6 +1050,9 @@ static void pt_guest_enter(struct vcpu_vmx *vmx)
 			pt_save_msr(desc->host_ctx, desc->addr_range);
 			pt_load_msr(desc->guest_ctx, desc->addr_range);
 		}
+	} else if (desc->pt_xsave && desc->guest_xs->initialized) {
+		pt_state_get(desc->guest_ctx, desc->guest_xs, desc->addr_range);
+		desc->guest_xs->initialized = 0;
 	}
 }
 
Signature · 1 line
1.8.3.1
maintainer
#7 lore raw
In reply to Luwei Kang
Quoted 37 lines · Luwei Kang

This patch set is mainly used for reduce the overhead of switch Intel PT configuation contex on VM-Entry/Exit by XSAVES/XRSTORS instructions.

I measured the cycles number of context witch on Manual and XSAVES/XRSTORES by rdtsc, and the data as below:

Manual save(rdmsr):     ~334  cycles
Manual restore(wrmsr):  ~1668 cycles
XSAVES insturction:     ~124  cycles
XRSTORS instruction:    ~378  cycles

Manual: Switch the configuration by rdmsr and wrmsr instruction,
and there have 8 registers need to be saved or restore. They are IA32_RTIT_OUTPUT_BASE, *_OUTPUT_MASK_PTRS, *_STATUS, *_CR3_MATCH, *_ADDR0_A, *_ADDR0_B, *_ADDR1_A, *_ADDR1_B.
XSAVES/XRSTORS: Switch the configuration context by XSAVES/XRSTORS
instructions. This patch set will allocate separate "struct fpu" structure to save host and guest PT state. Only a small portion of this structure will be used because we only save/restore PT state (not save AVX, AVX-512, MPX, PKRU and so on).

This patch set also do some code clean e.g. patch 2 will reuse the fpu pt_state to save the PT configuration contex and patch 3 will dymamic allocate Intel PT configuration state.

Luwei Kang (6):
x86/fpu: Introduce new fpu state for Intel processor trace
KVM: VMX: Reuse the pt_state structure for PT context
KVM: VMX: Dymamic allocate Intel PT configuration state
KVM: VMX: Allocate XSAVE area for Intel PT configuration
KVM: VMX: Intel PT configration context switch using XSAVES/XRSTORS
KVM: VMX: Get PT state from xsave area to variables

Quoted 6 lines
 arch/x86/include/asm/fpu/types.h |  13 ++
 arch/x86/kvm/vmx/nested.c        |   2 +-
 arch/x86/kvm/vmx/vmx.c           | 338 ++++++++++++++++++++++++++-------------
 arch/x86/kvm/vmx/vmx.h           |  21 +--
 4 files changed, 243 insertions(+), 131 deletions(-)

Luwei, I found I had missed this series. Can you check whether it needs a rebase, since I don't have hardware that supports it?

Paolo