Skip to contentSkip to conversation

Intel Processor Trace virtualization enabling

PATCH Superseded Luwei Kang 4 participants 32 messages May 22, 2018 – Jun 12, 2018

Subsystems (4)

v9 of 13 Superseded Latest is v13 →

lore
Share & tools
lore

Luwei Kang

author
#0 lore raw

v9 cover Intel Processor Trace virtualization enabling

Hi All,

Here is a patch-series which adding Processor Trace enabling in KVM guest. You can get It's software developer manuals from:
https://software.intel.com/sites/default/files/managed/c5/15/architecture-instruction-set-extensions-programming-reference.pdf
In Chapter 4 INTEL PROCESSOR TRACE: VMX IMPROVEMENTS.

Introduction:
Intel Processor Trace (Intel PT) is an extension of Intel Architecture that captures information about software execution using dedicated hardware facilities that cause only minimal performance perturbation to the software being traced. Details on the Intel PT infrastructure and trace capabilities can be found in the Intel 64 and IA-32 Architectures Software Developer’s Manual, Volume 3C.

The suite of architecture changes serve to simplify the process of virtualizing Intel PT for use by a guest software. There are two primary elements to this new architecture support for VMX support improvements made for Intel PT.
1. Addition of a new guest IA32_RTIT_CTL value field to the VMCS.
— This serves to speed and simplify the process of disabling trace on VM exit, and restoring it on VM entry.
2. Enabling use of EPT to redirect PT output.
— This enables the VMM to elect to virtualize the PT output buffer using EPT. In this mode, the CPU will treat PT output addresses as Guest Physical Addresses (GPAs) and translate them using EPT. This means that Intel PT output reads (of the ToPA table) and writes (of trace output) can cause EPT violations, and other output events.

Processor Trace virtualization can be work in one of 2 possible modes by set new option "pt_mode". Default value is system mode.
a. system-wide: trace both host/guest and output to host buffer;
b. host-guest: trace host/guest simultaneous and output to their respective buffer.

Quoted 1 line

From V8:

- move macro definition MSR_IA32_RTIT_ADDR_RANGE from msr-index.h to intel_pt.h;
- initialize the RTIT_CTL bitmask to ~0ULL.

Quoted 1 line

From V7:

- remove host only mode since it can be emulated by perf code;
- merge patch 8 and 9 to make code and data in the same patch;
- rename __pt_cap_get() to pt_cap_decode();
- other minor change.

Quoted 1 line

From V6:

- split pathes 1~2 to four separate patches (these patches do 2 things) and add more descriptions.

Quoted 1 line

From V5:

- rename the function from pt_cap_get_ex() to __pt_cap_get();
- replace the most of function from vmx_pt_supported() to "pt_mode == PT_MODE_HOST_GUEST"(or !=).

Quoted 1 line

From V4:

- add data check when setting the value of MSR_IA32_RTIT_CTL;
- Invoke new interface to set the intercept of MSRs read/write after "MSR bitmap per-vcpu" patches.

Quoted 1 line

From V3:

- change default mode to SYSTEM mode;
- add a new patch to move PT out of scattered features;
- add a new fucntion kvm_get_pt_addr_cnt() to get the number of address ranges;
- add a new function vmx_set_rtit_ctl() to set the value of guest RTIT_CTL, GUEST_IA32_RTIT_CTL and MSRs intercept.

Quoted 1 line

From v2:

- replace *_PT_SUPPRESS_PIP to *_PT_CONCEAL_PIP;
- clean SECONDARY_EXEC_PT_USE_GPA, VM_EXIT_CLEAR_IA32_RTIT_CTL and VM_ENTRY_LOAD_IA32_RTIT_CTL in SYSTEM mode. These bits must be all set or all clean;
- move processor tracing out of scattered features;
- add a new function to enable/disable intercept MSRs read/write;
- add all Intel PT MSRs read/write and disable intercept when PT is enabled in guest;
- disable Intel PT and enable intercept MSRs when L1 guest VMXON;
- performance optimization.
In Host only mode. we just need to save host RTIT_CTL before vm-entry and restore host RTIT_CTL after vm-exit;
In HOST_GUEST mode. we need to save and restore all MSRs only when PT has enabled in guest.
- use XSAVES/XRESTORES implement context switch.
Haven't implementation in this version and still in debuging. will make a separate patch work on this.

Quoted 1 line

From v1:

- remove guest-only mode because guest-only mode can be covered by host-guest mode;
- always set "use GPA for processor tracing" in secondary execution control if it can be;
- trap RTIT_CTL read/write. Forbid write this msr when VMXON in L1 hypervisor.

Chao Peng (7):
perf/x86/intel/pt: Move Intel-PT MSRs bit definitions to a public
header
perf/x86/intel/pt: Change pt_cap_get() to a public function
KVM: x86: Add Intel Processor Trace virtualization mode
KVM: x86: Add Intel Processor Trace cpuid emulation
KVM: x86: Add Intel Processor Trace context switch for each vcpu
KVM: x86: Implement Intel Processor Trace MSRs read/write emulation
KVM: x86: Set intercept for Intel PT MSRs read/write

Luwei Kang (5):
perf/x86/intel/pt: Add new bit definitions for Intel PT MSRs perf/x86/intel/pt: add new capability for Intel PT perf/x86/intel/pt: Introduce a new function to get capability of Intel
PT
KVM: x86: Introduce a function to initialize the PT configuration
KVM: x86: Disable Intel Processor Trace when VMXON in L1 guest

 arch/x86/events/intel/pt.c       |  12 +-
 arch/x86/events/intel/pt.h       |  58 ------
 arch/x86/include/asm/intel_pt.h  |  39 ++++
 arch/x86/include/asm/kvm_host.h  |   1 +
 arch/x86/include/asm/msr-index.h |  37 ++++
 arch/x86/include/asm/vmx.h       |   8 +
 arch/x86/kvm/cpuid.c             |  22 +-
 arch/x86/kvm/svm.c               |   6 +
 arch/x86/kvm/vmx.c               | 439 ++++++++++++++++++++++++++++++++++++++-
 arch/x86/kvm/x86.c               |  33 ++-
 10 files changed, 588 insertions(+), 67 deletions(-)
Signature · 1 line
1.8.3.1
author
#1 lore raw

PATCH v9 1/12 perf/x86/intel/pt: Move Intel-PT MSRs bit definitions to a public header

From: Chao Peng <[email protected]>

Intel Processor Trace virtualization enabling in KVM guest need to access these MSRs bit definitions, so move them to public header file msr-index.h.

---
 arch/x86/events/intel/pt.h       | 37 -------------------------------------
 arch/x86/include/asm/msr-index.h | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+), 37 deletions(-)
arch/x86/events/intel/pt.h +0 -37
index 0eb41d0..0050ca1 100644
@@ -20,43 +20,6 @@
 #define __INTEL_PT_H__
 
 /*
- * PT MSR bit definitions
- */
-#define RTIT_CTL_TRACEEN		BIT(0)
-#define RTIT_CTL_CYCLEACC		BIT(1)
-#define RTIT_CTL_OS			BIT(2)
-#define RTIT_CTL_USR			BIT(3)
-#define RTIT_CTL_PWR_EVT_EN		BIT(4)
-#define RTIT_CTL_FUP_ON_PTW		BIT(5)
-#define RTIT_CTL_CR3EN			BIT(7)
-#define RTIT_CTL_TOPA			BIT(8)
-#define RTIT_CTL_MTC_EN			BIT(9)
-#define RTIT_CTL_TSC_EN			BIT(10)
-#define RTIT_CTL_DISRETC		BIT(11)
-#define RTIT_CTL_PTW_EN			BIT(12)
-#define RTIT_CTL_BRANCH_EN		BIT(13)
-#define RTIT_CTL_MTC_RANGE_OFFSET	14
-#define RTIT_CTL_MTC_RANGE		(0x0full << RTIT_CTL_MTC_RANGE_OFFSET)
-#define RTIT_CTL_CYC_THRESH_OFFSET	19
-#define RTIT_CTL_CYC_THRESH		(0x0full << RTIT_CTL_CYC_THRESH_OFFSET)
-#define RTIT_CTL_PSB_FREQ_OFFSET	24
-#define RTIT_CTL_PSB_FREQ      		(0x0full << RTIT_CTL_PSB_FREQ_OFFSET)
-#define RTIT_CTL_ADDR0_OFFSET		32
-#define RTIT_CTL_ADDR0      		(0x0full << RTIT_CTL_ADDR0_OFFSET)
-#define RTIT_CTL_ADDR1_OFFSET		36
-#define RTIT_CTL_ADDR1      		(0x0full << RTIT_CTL_ADDR1_OFFSET)
-#define RTIT_CTL_ADDR2_OFFSET		40
-#define RTIT_CTL_ADDR2      		(0x0full << RTIT_CTL_ADDR2_OFFSET)
-#define RTIT_CTL_ADDR3_OFFSET		44
-#define RTIT_CTL_ADDR3      		(0x0full << RTIT_CTL_ADDR3_OFFSET)
-#define RTIT_STATUS_FILTEREN		BIT(0)
-#define RTIT_STATUS_CONTEXTEN		BIT(1)
-#define RTIT_STATUS_TRIGGEREN		BIT(2)
-#define RTIT_STATUS_BUFFOVF		BIT(3)
-#define RTIT_STATUS_ERROR		BIT(4)
-#define RTIT_STATUS_STOPPED		BIT(5)
-
-/*
  * Single-entry ToPA: when this close to region boundary, switch
  * buffers to avoid losing data.
  */
arch/x86/include/asm/msr-index.h +33 -0
index 53d5b1b..afe4e13 100644
@@ -106,7 +106,40 @@
 #define MSR_PEBS_LD_LAT_THRESHOLD	0x000003f6
 
 #define MSR_IA32_RTIT_CTL		0x00000570
+#define RTIT_CTL_TRACEEN		BIT(0)
+#define RTIT_CTL_CYCLEACC		BIT(1)
+#define RTIT_CTL_OS			BIT(2)
+#define RTIT_CTL_USR			BIT(3)
+#define RTIT_CTL_PWR_EVT_EN		BIT(4)
+#define RTIT_CTL_FUP_ON_PTW		BIT(5)
+#define RTIT_CTL_CR3EN			BIT(7)
+#define RTIT_CTL_TOPA			BIT(8)
+#define RTIT_CTL_MTC_EN			BIT(9)
+#define RTIT_CTL_TSC_EN			BIT(10)
+#define RTIT_CTL_DISRETC		BIT(11)
+#define RTIT_CTL_PTW_EN			BIT(12)
+#define RTIT_CTL_BRANCH_EN		BIT(13)
+#define RTIT_CTL_MTC_RANGE_OFFSET	14
+#define RTIT_CTL_MTC_RANGE		(0x0full << RTIT_CTL_MTC_RANGE_OFFSET)
+#define RTIT_CTL_CYC_THRESH_OFFSET	19
+#define RTIT_CTL_CYC_THRESH		(0x0full << RTIT_CTL_CYC_THRESH_OFFSET)
+#define RTIT_CTL_PSB_FREQ_OFFSET	24
+#define RTIT_CTL_PSB_FREQ		(0x0full << RTIT_CTL_PSB_FREQ_OFFSET)
+#define RTIT_CTL_ADDR0_OFFSET		32
+#define RTIT_CTL_ADDR0			(0x0full << RTIT_CTL_ADDR0_OFFSET)
+#define RTIT_CTL_ADDR1_OFFSET		36
+#define RTIT_CTL_ADDR1			(0x0full << RTIT_CTL_ADDR1_OFFSET)
+#define RTIT_CTL_ADDR2_OFFSET		40
+#define RTIT_CTL_ADDR2			(0x0full << RTIT_CTL_ADDR2_OFFSET)
+#define RTIT_CTL_ADDR3_OFFSET		44
+#define RTIT_CTL_ADDR3			(0x0full << RTIT_CTL_ADDR3_OFFSET)
 #define MSR_IA32_RTIT_STATUS		0x00000571
+#define RTIT_STATUS_FILTEREN		BIT(0)
+#define RTIT_STATUS_CONTEXTEN		BIT(1)
+#define RTIT_STATUS_TRIGGEREN		BIT(2)
+#define RTIT_STATUS_BUFFOVF		BIT(3)
+#define RTIT_STATUS_ERROR		BIT(4)
+#define RTIT_STATUS_STOPPED		BIT(5)
 #define MSR_IA32_RTIT_ADDR0_A		0x00000580
 #define MSR_IA32_RTIT_ADDR0_B		0x00000581
 #define MSR_IA32_RTIT_ADDR1_A		0x00000582
Signature · 1 line
1.8.3.1
#2 lore raw

RE: [PATCH v9 01/12] perf/x86/intel/pt: Move Intel-PT MSRs bit definitions to a public header

Quoted 106 lines

-----Original Message-----
From: Kang, Luwei
Sent: Tuesday, May 22, 2018 12:52 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Liang, Kan <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Kang, Luwei <[email protected]>
Subject: [PATCH v9 01/12] perf/x86/intel/pt: Move Intel-PT MSRs bit definitions to a public header

From: Chao Peng <[email protected]>

Intel Processor Trace virtualization enabling in KVM guest need to access these MSRs bit definitions, so move them to public header file msr-index.h.

---
 arch/x86/events/intel/pt.h       | 37 -------------------------------------
 arch/x86/include/asm/msr-index.h | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+), 37 deletions(-)
arch/x86/events/intel/pt.h +0 -37
@@ -20,43 +20,6 @@
 #define __INTEL_PT_H__
 
 /*
- * PT MSR bit definitions
- */
-#define RTIT_CTL_TRACEEN		BIT(0)
-#define RTIT_CTL_CYCLEACC		BIT(1)
-#define RTIT_CTL_OS			BIT(2)
-#define RTIT_CTL_USR			BIT(3)
-#define RTIT_CTL_PWR_EVT_EN		BIT(4)
-#define RTIT_CTL_FUP_ON_PTW		BIT(5)
-#define RTIT_CTL_CR3EN			BIT(7)
-#define RTIT_CTL_TOPA			BIT(8)
-#define RTIT_CTL_MTC_EN			BIT(9)
-#define RTIT_CTL_TSC_EN			BIT(10)
-#define RTIT_CTL_DISRETC		BIT(11)
-#define RTIT_CTL_PTW_EN			BIT(12)
-#define RTIT_CTL_BRANCH_EN		BIT(13)
-#define RTIT_CTL_MTC_RANGE_OFFSET	14
-#define RTIT_CTL_MTC_RANGE		(0x0full << RTIT_CTL_MTC_RANGE_OFFSET)
-#define RTIT_CTL_CYC_THRESH_OFFSET	19
-#define RTIT_CTL_CYC_THRESH		(0x0full << RTIT_CTL_CYC_THRESH_OFFSET)
-#define RTIT_CTL_PSB_FREQ_OFFSET	24
-#define RTIT_CTL_PSB_FREQ      		(0x0full << RTIT_CTL_PSB_FREQ_OFFSET)
-#define RTIT_CTL_ADDR0_OFFSET		32
-#define RTIT_CTL_ADDR0      		(0x0full << RTIT_CTL_ADDR0_OFFSET)
-#define RTIT_CTL_ADDR1_OFFSET		36
-#define RTIT_CTL_ADDR1      		(0x0full << RTIT_CTL_ADDR1_OFFSET)
-#define RTIT_CTL_ADDR2_OFFSET		40
-#define RTIT_CTL_ADDR2      		(0x0full << RTIT_CTL_ADDR2_OFFSET)
-#define RTIT_CTL_ADDR3_OFFSET		44
-#define RTIT_CTL_ADDR3      		(0x0full << RTIT_CTL_ADDR3_OFFSET)
-#define RTIT_STATUS_FILTEREN		BIT(0)
-#define RTIT_STATUS_CONTEXTEN		BIT(1)
-#define RTIT_STATUS_TRIGGEREN		BIT(2)
-#define RTIT_STATUS_BUFFOVF		BIT(3)
-#define RTIT_STATUS_ERROR		BIT(4)
-#define RTIT_STATUS_STOPPED		BIT(5)
-
-/*
  * Single-entry ToPA: when this close to region boundary, switch
  * buffers to avoid losing data.
  */
arch/x86/include/asm/msr-index.h +27 -0
index 53d5b1b..afe4e13 100644
@@ -106,7 +106,40 @@
 #define MSR_PEBS_LD_LAT_THRESHOLD	0x000003f6
 
 #define MSR_IA32_RTIT_CTL		0x00000570
+#define RTIT_CTL_TRACEEN		BIT(0)
+#define RTIT_CTL_CYCLEACC		BIT(1)
+#define RTIT_CTL_OS			BIT(2)
+#define RTIT_CTL_USR			BIT(3)
+#define RTIT_CTL_PWR_EVT_EN		BIT(4)
+#define RTIT_CTL_FUP_ON_PTW		BIT(5)
+#define RTIT_CTL_CR3EN			BIT(7)
+#define RTIT_CTL_TOPA			BIT(8)
+#define RTIT_CTL_MTC_EN			BIT(9)
+#define RTIT_CTL_TSC_EN			BIT(10)
+#define RTIT_CTL_DISRETC		BIT(11)
+#define RTIT_CTL_PTW_EN			BIT(12)
+#define RTIT_CTL_BRANCH_EN		BIT(13)
+#define RTIT_CTL_MTC_RANGE_OFFSET	14
+#define RTIT_CTL_MTC_RANGE		(0x0full << RTIT_CTL_MTC_RANGE_OFFSET)
+#define RTIT_CTL_CYC_THRESH_OFFSET	19
+#define RTIT_CTL_CYC_THRESH		(0x0full << RTIT_CTL_CYC_THRESH_OFFSET)
+#define RTIT_CTL_PSB_FREQ_OFFSET	24
+#define RTIT_CTL_PSB_FREQ		(0x0full << RTIT_CTL_PSB_FREQ_OFFSET)
+#define RTIT_CTL_ADDR0_OFFSET		32
+#define RTIT_CTL_ADDR0			(0x0full << RTIT_CTL_ADDR0_OFFSET)
+#define RTIT_CTL_ADDR1_OFFSET		36
+#define RTIT_CTL_ADDR1			(0x0full << RTIT_CTL_ADDR1_OFFSET)
+#define RTIT_CTL_ADDR2_OFFSET		40
+#define RTIT_CTL_ADDR2			(0x0full << RTIT_CTL_ADDR2_OFFSET)
+#define RTIT_CTL_ADDR3_OFFSET		44
+#define RTIT_CTL_ADDR3			(0x0full << RTIT_CTL_ADDR3_OFFSET)

arch/x86/include/asm/msr-index.h line 145

Quoted 10 lines
arch/x86/include/asm/msr-index.h +6 -0
 #define MSR_IA32_RTIT_STATUS		0x00000571
+#define RTIT_STATUS_FILTEREN		BIT(0)
+#define RTIT_STATUS_CONTEXTEN		BIT(1)
+#define RTIT_STATUS_TRIGGEREN		BIT(2)
+#define RTIT_STATUS_BUFFOVF		BIT(3)
+#define RTIT_STATUS_ERROR		BIT(4)
+#define RTIT_STATUS_STOPPED		BIT(5)
 #define MSR_IA32_RTIT_ADDR0_A		0x00000580
 #define MSR_IA32_RTIT_ADDR0_B		0x00000581
 #define MSR_IA32_RTIT_ADDR1_A		0x00000582

Hi,
Patch 1~5 have some code changes in x86 native for Intel Processor Trace virtualization enabling in KVM guest. I have sent patch set v9 which include some minor changes from old version.
Do you have any comments?

Thanks,
Luwei Kang

maintainer
#3 lore raw

Re: [PATCH v9 01/12] perf/x86/intel/pt: Move Intel-PT MSRs bit definitions to a public header

Quoted 3 lines · Luwei

Patch 1~5 have some code changes in x86 native for Intel Processor Trace virtualization enabling in KVM guest. I have sent patch set v9 which include some minor changes from old version.
Do you have any comments?

x86 folks, looks like this series is going to miss 4.18. Perf folks, can you review this and hopefully provide a topic branch asap after the end of the merge window?

Thanks,

Paolo

author
#4 lore raw

PATCH v9 2/12 perf/x86/intel/pt: Change pt_cap_get() to a public function

In reply to Luwei Kang

From: Chao Peng <[email protected]>

Change pt_cap_get() to a public function that KVM can access this function to check if specific feature is supported on hardware.

---
 arch/x86/events/intel/pt.c      |  3 ++-
 arch/x86/events/intel/pt.h      | 21 ---------------------
 arch/x86/include/asm/intel_pt.h | 23 +++++++++++++++++++++++
 3 files changed, 25 insertions(+), 22 deletions(-)
arch/x86/events/intel/pt.c +2 -1
index 3b99394..c80e2f5 100644
@@ -75,7 +75,7 @@
 	PT_CAP(psb_periods,		1, CPUID_EBX, 0xffff0000),
 };
 
-static u32 pt_cap_get(enum pt_capabilities cap)
+u32 pt_cap_get(enum pt_capabilities cap)
 {
 	struct pt_cap_desc *cd = &pt_caps[cap];
 	u32 c = pt_pmu.caps[cd->leaf * PT_CPUID_REGS_NUM + cd->reg];
@@ -83,6 +83,7 @@ static u32 pt_cap_get(enum pt_capabilities cap)
 
 	return (c & cd->mask) >> shift;
 }
+EXPORT_SYMBOL_GPL(pt_cap_get);
 
 static ssize_t pt_cap_show(struct device *cdev,
 			   struct device_attribute *attr,
arch/x86/events/intel/pt.h +0 -21
index 0050ca1..269e15a 100644
@@ -45,30 +45,9 @@ struct topa_entry {
 	u64	rsvd4	: 16;
 };
 
-#define PT_CPUID_LEAVES		2
-#define PT_CPUID_REGS_NUM	4 /* number of regsters (eax, ebx, ecx, edx) */
-
 /* TSC to Core Crystal Clock Ratio */
 #define CPUID_TSC_LEAF		0x15
 
-enum pt_capabilities {
-	PT_CAP_max_subleaf = 0,
-	PT_CAP_cr3_filtering,
-	PT_CAP_psb_cyc,
-	PT_CAP_ip_filtering,
-	PT_CAP_mtc,
-	PT_CAP_ptwrite,
-	PT_CAP_power_event_trace,
-	PT_CAP_topa_output,
-	PT_CAP_topa_multiple_entries,
-	PT_CAP_single_range_output,
-	PT_CAP_payloads_lip,
-	PT_CAP_num_address_ranges,
-	PT_CAP_mtc_periods,
-	PT_CAP_cycle_thresholds,
-	PT_CAP_psb_periods,
-};
-
 struct pt_pmu {
 	struct pmu		pmu;
 	u32			caps[PT_CPUID_REGS_NUM * PT_CPUID_LEAVES];
arch/x86/include/asm/intel_pt.h +23 -0
index b523f51..4270421 100644
@@ -2,10 +2,33 @@
 #ifndef _ASM_X86_INTEL_PT_H
 #define _ASM_X86_INTEL_PT_H
 
+#define PT_CPUID_LEAVES		2
+#define PT_CPUID_REGS_NUM	4 /* number of regsters (eax, ebx, ecx, edx) */
+
+enum pt_capabilities {
+	PT_CAP_max_subleaf = 0,
+	PT_CAP_cr3_filtering,
+	PT_CAP_psb_cyc,
+	PT_CAP_ip_filtering,
+	PT_CAP_mtc,
+	PT_CAP_ptwrite,
+	PT_CAP_power_event_trace,
+	PT_CAP_topa_output,
+	PT_CAP_topa_multiple_entries,
+	PT_CAP_single_range_output,
+	PT_CAP_payloads_lip,
+	PT_CAP_num_address_ranges,
+	PT_CAP_mtc_periods,
+	PT_CAP_cycle_thresholds,
+	PT_CAP_psb_periods,
+};
+
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
 void cpu_emergency_stop_pt(void);
+extern u32 pt_cap_get(enum pt_capabilities cap);
 #else
 static inline void cpu_emergency_stop_pt(void) {}
+static inline u32 pt_cap_get(enum pt_capabilities cap) { return 0; }
 #endif
 
 #endif /* _ASM_X86_INTEL_PT_H */
Signature · 1 line
1.8.3.1
author
#5 lore raw

PATCH v9 3/12 perf/x86/intel/pt: Add new bit definitions for Intel PT MSRs

In reply to Luwei Kang

These bit definitions are use for emulate MSRs read/write for KVM. For example, IA32_RTIT_CTL.FabricEn[bit 6] is available only when CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 1. If KVM guest try to set this bit with CPUID.(EAX=14H, ECX=0):ECX[bit3] = 0 a #GP would be injected to KVM guest.

---
 arch/x86/include/asm/msr-index.h | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/include/asm/msr-index.h +3 -0
index afe4e13..6ae2462 100644
@@ -112,6 +112,7 @@
 #define RTIT_CTL_USR			BIT(3)
 #define RTIT_CTL_PWR_EVT_EN		BIT(4)
 #define RTIT_CTL_FUP_ON_PTW		BIT(5)
+#define RTIT_CTL_FABRIC_EN		BIT(6)
 #define RTIT_CTL_CR3EN			BIT(7)
 #define RTIT_CTL_TOPA			BIT(8)
 #define RTIT_CTL_MTC_EN			BIT(9)
@@ -140,6 +141,8 @@
 #define RTIT_STATUS_BUFFOVF		BIT(3)
 #define RTIT_STATUS_ERROR		BIT(4)
 #define RTIT_STATUS_STOPPED		BIT(5)
+#define RTIT_STATUS_BYTECNT_OFFSET	32
+#define RTIT_STATUS_BYTECNT		(0x1ffffull << RTIT_STATUS_BYTECNT_OFFSET)
 #define MSR_IA32_RTIT_ADDR0_A		0x00000580
 #define MSR_IA32_RTIT_ADDR0_B		0x00000581
 #define MSR_IA32_RTIT_ADDR1_A		0x00000582
Signature · 1 line
1.8.3.1
#6 lore raw

Re: [PATCH v9 03/12] perf/x86/intel/pt: Add new bit definitions for Intel PT MSRs

Quoted 5 lines · Luwei Kang

These bit definitions are use for emulate MSRs read/write for KVM. For example, IA32_RTIT_CTL.FabricEn[bit 6] is available only when CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 1. If KVM guest try to set this bit with CPUID.(EAX=14H, ECX=0):ECX[bit3] = 0 a #GP would be injected to KVM guest.

Do we have anything in the guest that this feature will work with?

Regards,
--
Alex

#7 lore raw

RE: [PATCH v9 03/12] perf/x86/intel/pt: Add new bit definitions for Intel PT MSRs

Quoted 8 lines
Quoted 5 lines

These bit definitions are use for emulate MSRs read/write for KVM. For example, IA32_RTIT_CTL.FabricEn[bit 6] is available only when CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 1. If KVM guest try to set this bit with CPUID.(EAX=14H, ECX=0):ECX[bit3] = 0 a #GP would be injected to KVM guest.

Do we have anything in the guest that this feature will work with?

It depend on PT driver. KVM need to do some security check if kvm guest (maybe linux or other os) try to set any bits of these MSRs.

Thanks,
Luwei Kang

author
#8 lore raw

PATCH v9 4/12 perf/x86/intel/pt: add new capability for Intel PT

In reply to Luwei Kang

CPUID(EAX=14H,ECX=0):EBX[bit 3] = 1 indicates support of output to Trace Transport subsystem.
MSR IA32_RTIT_CTL.FabricEn[bit 6] is reserved if CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 0.
This is use for emulate IA32_RTIT_CTL MSR read/write in KVM. KVM guest write IA32_RTIT_CTL will trap to root mode and a #GP would be injected to guest if set IA32_RTIT_CTL.FabricEn with
CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 0.

---
 arch/x86/events/intel/pt.c      | 1 +
 arch/x86/include/asm/intel_pt.h | 1 +
 2 files changed, 2 insertions(+)
arch/x86/events/intel/pt.c +1 -0
index c80e2f5..f65f97a 100644
@@ -68,6 +68,7 @@
 	PT_CAP(topa_output,		0, CPUID_ECX, BIT(0)),
 	PT_CAP(topa_multiple_entries,	0, CPUID_ECX, BIT(1)),
 	PT_CAP(single_range_output,	0, CPUID_ECX, BIT(2)),
+	PT_CAP(output_subsys,		0, CPUID_ECX, BIT(3)),
 	PT_CAP(payloads_lip,		0, CPUID_ECX, BIT(31)),
 	PT_CAP(num_address_ranges,	1, CPUID_EAX, 0x3),
 	PT_CAP(mtc_periods,		1, CPUID_EAX, 0xffff0000),
arch/x86/include/asm/intel_pt.h +1 -0
index 4270421..2de4db0 100644
@@ -16,6 +16,7 @@ enum pt_capabilities {
 	PT_CAP_topa_output,
 	PT_CAP_topa_multiple_entries,
 	PT_CAP_single_range_output,
+	PT_CAP_output_subsys,
 	PT_CAP_payloads_lip,
 	PT_CAP_num_address_ranges,
 	PT_CAP_mtc_periods,
Signature · 1 line
1.8.3.1
#9 lore raw

Re: [PATCH v9 04/12] perf/x86/intel/pt: add new capability for Intel PT

Quoted 4 lines · Luwei Kang

CPUID(EAX=14H,ECX=0):EBX[bit 3] = 1 indicates support of output to Trace Transport subsystem.
MSR IA32_RTIT_CTL.FabricEn[bit 6] is reserved if CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 0.

This should instead say:

This adds support for "output to Trace Transport subsystem" capability of Intel PT, as documented in IA SDM Chapter 36.x.y.z. It means that PT can output its trace to an MMIO address range rather than system memory buffer.

Quoted 5 lines

This is use for emulate IA32_RTIT_CTL MSR read/write in KVM. KVM guest write IA32_RTIT_CTL will trap to root mode and a #GP would be injected to guest if set IA32_RTIT_CTL.FabricEn with
CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 0.

I'm not sure what this means, this patch has nothing to do with KVM as far as I can tell.

Aside from the commit message, this is a valid patch.

Thanks,
--
Alex

maintainer
#10 lore raw

Re: [PATCH v9 04/12] perf/x86/intel/pt: add new capability for Intel PT

Quoted 6 lines · Alexander Shishkin
Quoted 4 lines · Luwei Kang

CPUID(EAX=14H,ECX=0):EBX[bit 3] = 1 indicates support of output to Trace Transport subsystem.
MSR IA32_RTIT_CTL.FabricEn[bit 6] is reserved if CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 0.

Quoted 6 lines

This should instead say:

This adds support for "output to Trace Transport subsystem" capability of Intel PT, as documented in IA SDM Chapter 36.x.y.z. It means that PT can output its trace to an MMIO address range rather than system memory buffer.

The $subject also needs attention.

#11 lore raw

RE: [PATCH v9 04/12] perf/x86/intel/pt: add new capability for Intel PT

Quoted 6 lines
Quoted 6 lines · Luwei Kang
Quoted 4 lines

CPUID(EAX=14H,ECX=0):EBX[bit 3] = 1 indicates support of output to Trace Transport subsystem.
MSR IA32_RTIT_CTL.FabricEn[bit 6] is reserved if CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 0.

Quoted 8 lines
Quoted 6 lines

This should instead say:

This adds support for "output to Trace Transport subsystem" capability of Intel PT, as documented in IA SDM Chapter 36.x.y.z. It means that PT can output its trace to an MMIO address range rather than system memory buffer.

The $subject also needs attention.

A little don't understand. What is "$subject" mean?

Thanks,
Luwei Kang

#12 lore raw

RE: [PATCH v9 04/12] perf/x86/intel/pt: add new capability for Intel PT

In reply to Alexander Shishkin
Quoted 11 lines
Quoted 4 lines

CPUID(EAX=14H,ECX=0):EBX[bit 3] = 1 indicates support of output to Trace Transport subsystem.
MSR IA32_RTIT_CTL.FabricEn[bit 6] is reserved if CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 0.

This should instead say:

This adds support for "output to Trace Transport subsystem" capability of Intel PT, as documented in IA SDM Chapter 36.x.y.z. It means that PT can output its trace to an MMIO address range rather than system memory buffer.

Yes, you are right. In KVM point of view this bit use for MSR access security check.
KVM will track MSRs access in guest, an #GP will be injected to guest if guest try to write IA32_RTIT_CTL.FabricEn[bit 6] when "output_subsys" (CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 0) is not supported.

Thanks,
Luwei Kang

Quoted 14 lines
Quoted 5 lines

This is use for emulate IA32_RTIT_CTL MSR read/write in KVM. KVM guest write IA32_RTIT_CTL will trap to root mode and a #GP would be injected to guest if set IA32_RTIT_CTL.FabricEn with CPUID.(EAX=14H, ECX=0):ECX[bit 3] = 0.

I'm not sure what this means, this patch has nothing to do with KVM as far as I can tell.

Aside from the commit message, this is a valid patch.

Thanks,
--
Alex

author
#13 lore raw

PATCH v9 5/12 perf/x86/intel/pt: Introduce a new function to get capability of Intel PT

In reply to Luwei Kang

New function pt_cap_decode() will be invoked in KVM to check if a specific capability is available in KVM guest. Another function pt_cap_get() can only check the hardware capabilities but this may different with KVM guest because some features may not be exposed to guest.

---
 arch/x86/events/intel/pt.c      | 10 ++++++++--
 arch/x86/include/asm/intel_pt.h |  2 ++
 2 files changed, 10 insertions(+), 2 deletions(-)
arch/x86/events/intel/pt.c +8 -2
index f65f97a..18a2e80 100644
@@ -76,14 +76,20 @@
 	PT_CAP(psb_periods,		1, CPUID_EBX, 0xffff0000),
 };
 
-u32 pt_cap_get(enum pt_capabilities cap)
+u32 pt_cap_decode(u32 *caps, enum pt_capabilities cap)
 {
 	struct pt_cap_desc *cd = &pt_caps[cap];
-	u32 c = pt_pmu.caps[cd->leaf * PT_CPUID_REGS_NUM + cd->reg];
+	u32 c = caps[cd->leaf * PT_CPUID_REGS_NUM + cd->reg];
 	unsigned int shift = __ffs(cd->mask);
 
 	return (c & cd->mask) >> shift;
 }
+EXPORT_SYMBOL_GPL(pt_cap_decode);
+
+u32 pt_cap_get(enum pt_capabilities cap)
+{
+	return pt_cap_decode(pt_pmu.caps, cap);
+}
 EXPORT_SYMBOL_GPL(pt_cap_get);
 
 static ssize_t pt_cap_show(struct device *cdev,
arch/x86/include/asm/intel_pt.h +2 -0
index 2de4db0..9c71453 100644
@@ -27,9 +27,11 @@ enum pt_capabilities {
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
 void cpu_emergency_stop_pt(void);
 extern u32 pt_cap_get(enum pt_capabilities cap);
+extern u32 pt_cap_decode(u32 *caps, enum pt_capabilities cap);
 #else
 static inline void cpu_emergency_stop_pt(void) {}
 static inline u32 pt_cap_get(enum pt_capabilities cap) { return 0; }
+static u32 pt_cap_decode(u32 *caps, enum pt_capabilities cap) { return 0; }
 #endif
 
 #endif /* _ASM_X86_INTEL_PT_H */
Signature · 1 line
1.8.3.1
author
#14 lore raw

PATCH v9 6/12 KVM: x86: Add Intel Processor Trace virtualization mode

In reply to Luwei Kang

From: Chao Peng <[email protected]>

Intel PT virtualization can be work in one of 2 possible modes:
a. system-wide: trace both host and guest and output to host buffer; b. host-guest: trace host/guest simultaneous and output to their
respective buffer.

---
 arch/x86/include/asm/intel_pt.h  |  3 ++
 arch/x86/include/asm/msr-index.h |  1 +
 arch/x86/include/asm/vmx.h       |  8 +++++
 arch/x86/kvm/vmx.c               | 68 +++++++++++++++++++++++++++++++++++++---
 4 files changed, 76 insertions(+), 4 deletions(-)
arch/x86/include/asm/intel_pt.h +3 -0
index 9c71453..5748205 100644
@@ -5,6 +5,9 @@
 #define PT_CPUID_LEAVES		2
 #define PT_CPUID_REGS_NUM	4 /* number of regsters (eax, ebx, ecx, edx) */
 
+#define PT_MODE_SYSTEM		0
+#define PT_MODE_HOST_GUEST	1
+
 enum pt_capabilities {
 	PT_CAP_max_subleaf = 0,
 	PT_CAP_cr3_filtering,
arch/x86/include/asm/msr-index.h +1 -0
index 6ae2462..6b14325 100644
@@ -789,6 +789,7 @@
 #define VMX_BASIC_INOUT		0x0040000000000000LLU
 
 /* MSR_IA32_VMX_MISC bits */
+#define MSR_IA32_VMX_MISC_INTEL_PT                 (1ULL << 14)
 #define MSR_IA32_VMX_MISC_VMWRITE_SHADOW_RO_FIELDS (1ULL << 29)
 #define MSR_IA32_VMX_MISC_PREEMPTION_TIMER_SCALE   0x1F
 /* AMD-V MSRs */
arch/x86/include/asm/vmx.h +8 -0
index 5db8b0b..5936d72 100644
@@ -76,7 +76,9 @@
 #define SECONDARY_EXEC_SHADOW_VMCS              0x00004000
 #define SECONDARY_EXEC_RDSEED_EXITING		0x00010000
 #define SECONDARY_EXEC_ENABLE_PML               0x00020000
+#define SECONDARY_EXEC_PT_CONCEAL_VMX		0x00080000
 #define SECONDARY_EXEC_XSAVES			0x00100000
+#define SECONDARY_EXEC_PT_USE_GPA		0x01000000
 #define SECONDARY_EXEC_TSC_SCALING              0x02000000
 
 #define PIN_BASED_EXT_INTR_MASK                 0x00000001
@@ -97,6 +99,8 @@
 #define VM_EXIT_LOAD_IA32_EFER                  0x00200000
 #define VM_EXIT_SAVE_VMX_PREEMPTION_TIMER       0x00400000
 #define VM_EXIT_CLEAR_BNDCFGS                   0x00800000
+#define VM_EXIT_PT_CONCEAL_PIP			0x01000000
+#define VM_EXIT_CLEAR_IA32_RTIT_CTL		0x02000000
 
 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR	0x00036dff
 
@@ -108,6 +112,8 @@
 #define VM_ENTRY_LOAD_IA32_PAT			0x00004000
 #define VM_ENTRY_LOAD_IA32_EFER                 0x00008000
 #define VM_ENTRY_LOAD_BNDCFGS                   0x00010000
+#define VM_ENTRY_PT_CONCEAL_PIP			0x00020000
+#define VM_ENTRY_LOAD_IA32_RTIT_CTL		0x00040000
 
 #define VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR	0x000011ff
 
@@ -234,6 +240,8 @@ enum vmcs_field {
 	GUEST_PDPTR3_HIGH               = 0x00002811,
 	GUEST_BNDCFGS                   = 0x00002812,
 	GUEST_BNDCFGS_HIGH              = 0x00002813,
+	GUEST_IA32_RTIT_CTL		= 0x00002814,
+	GUEST_IA32_RTIT_CTL_HIGH	= 0x00002815,
 	HOST_IA32_PAT			= 0x00002c00,
 	HOST_IA32_PAT_HIGH		= 0x00002c01,
 	HOST_IA32_EFER			= 0x00002c02,
arch/x86/kvm/vmx.c +64 -4

134 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#15 lore raw

Re: [PATCH v9 06/12] KVM: x86: Add Intel Processor Trace virtualization mode

Quoted 6 lines · Luwei Kang

From: Chao Peng <[email protected]>

Intel PT virtualization can be work in one of 2 possible modes:
a. system-wide: trace both host and guest and output to host buffer; b. host-guest: trace host/guest simultaneous and output to their
respective buffer.

I think we discussed this before. That's not the choice that the user needs to make. The only choice that I see is what happens if the guest wants to use PT while it's already in use by the host. Otherwise, each of them gets to use PT as they would.

Regards,
--
Alex

#16 lore raw

RE: [PATCH v9 06/12] KVM: x86: Add Intel Processor Trace virtualization mode

Quoted 7 lines
Quoted 6 lines

From: Chao Peng <[email protected]>

Intel PT virtualization can be work in one of 2 possible modes:
a. system-wide: trace both host and guest and output to host buffer; b. host-guest: trace host/guest simultaneous and output to their
respective buffer.

Quoted 5 lines

I think we discussed this before. That's not the choice that the user needs to make. The only choice that I see is what happens if the guest wants to use PT while it's already in use by the host. Otherwise, each of them gets to use PT as they would.

Hi,
Thanks for your reply first. If guest want to use PT that we need to expose this feature to guest and guest will save these trace packets to guest buffer. Host will can't get the trace which are made by KVM guest. This is Host_Guest mode.
On the other hand, host may also want to trace hypervisor and KVM guest execution trace and output these trace in native. This is SYSTEM mode. PT can't be exposed to guest in this mode or KVM guest trace will be lost in native buffer. These are two different use case and can't coexist.

Thanks,
Luwei Kang

maintainer
#17 lore raw

Re: [PATCH v9 06/12] KVM: x86: Add Intel Processor Trace virtualization mode

In reply to Alexander Shishkin
Quoted 9 lines · Alexander Shishkin
Quoted 4 lines

Intel PT virtualization can be work in one of 2 possible modes:
a. system-wide: trace both host and guest and output to host buffer; b. host-guest: trace host/guest simultaneous and output to their
respective buffer.

I think we discussed this before. That's not the choice that the user needs to make. The only choice that I see is what happens if the guest wants to use PT while it's already in use by the host. Otherwise, each of them gets to use PT as they would.

That is a superset of this choice:

a. hide PT from guests completely (the "a" above)

b. host-guest with guest preference: same as "b" above

c. host-guest with host preference: trace host/guest simultaneously, but drop guest events if the host has tracing enabled.

so I don't think this choice is an issue.

Paolo

author
#18 lore raw

PATCH v9 7/12 KVM: x86: Add Intel Processor Trace cpuid emulation

In reply to Luwei Kang

From: Chao Peng <[email protected]>

Expose Intel Processor Trace to guest only when PT work in HOST_GUEST mode.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/cpuid.c            | 22 ++++++++++++++++++++--
 arch/x86/kvm/svm.c              |  6 ++++++
 arch/x86/kvm/vmx.c              |  6 ++++++
 4 files changed, 33 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index b27de80..8f3c7ea 100644
@@ -1026,6 +1026,7 @@ struct kvm_x86_ops {
 	bool (*mpx_supported)(void);
 	bool (*xsaves_supported)(void);
 	bool (*umip_emulated)(void);
+	bool (*pt_supported)(void);
 
 	int (*check_nested_events)(struct kvm_vcpu *vcpu, bool external_intr);
 
arch/x86/kvm/cpuid.c +20 -2
index 82055b9..e04bf67 100644
@@ -336,6 +336,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
 	unsigned f_mpx = kvm_mpx_supported() ? F(MPX) : 0;
 	unsigned f_xsaves = kvm_x86_ops->xsaves_supported() ? F(XSAVES) : 0;
 	unsigned f_umip = kvm_x86_ops->umip_emulated() ? F(UMIP) : 0;
+	unsigned f_intel_pt = kvm_x86_ops->pt_supported() ? F(INTEL_PT) : 0;
 
 	/* cpuid 1.edx */
 	const u32 kvm_cpuid_1_edx_x86_features =
@@ -393,7 +394,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
 		F(BMI2) | F(ERMS) | f_invpcid | F(RTM) | f_mpx | F(RDSEED) |
 		F(ADX) | F(SMAP) | F(AVX512IFMA) | F(AVX512F) | F(AVX512PF) |
 		F(AVX512ER) | F(AVX512CD) | F(CLFLUSHOPT) | F(CLWB) | F(AVX512DQ) |
-		F(SHA_NI) | F(AVX512BW) | F(AVX512VL);
+		F(SHA_NI) | F(AVX512BW) | F(AVX512VL) | f_intel_pt;
 
 	/* cpuid 0xD.1.eax */
 	const u32 kvm_cpuid_D_1_eax_x86_features =
@@ -423,7 +424,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
 
 	switch (function) {
 	case 0:
-		entry->eax = min(entry->eax, (u32)0xd);
+		entry->eax = min(entry->eax, (u32)(f_intel_pt ? 0x14 : 0xd));
 		break;
 	case 1:
 		entry->edx &= kvm_cpuid_1_edx_x86_features;
@@ -595,6 +596,23 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
 		}
 		break;
 	}
+	/* Intel PT */
+	case 0x14: {
+		int t, times = entry->eax;
+
+		if (!f_intel_pt)
+			break;
+
+		entry->flags |= KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
+		for (t = 1; t <= times; ++t) {
+			if (*nent >= maxnent)
+				goto out;
+			do_cpuid_1_ent(&entry[t], function, t);
+			entry[t].flags |= KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
+			++*nent;
+		}
+		break;
+	}
 	case KVM_CPUID_SIGNATURE: {
 		static const char signature[12] = "KVMKVMKVM\0\0";
 		const u32 *sigptr = (const u32 *)signature;
arch/x86/kvm/svm.c +6 -0
index 220e5a89..6df8075 100644
@@ -5876,6 +5876,11 @@ static bool svm_umip_emulated(void)
 	return false;
 }
 
+static bool svm_pt_supported(void)
+{
+	return false;
+}
+
 static bool svm_has_wbinvd_exit(void)
 {
 	return true;
@@ -7101,6 +7106,7 @@ static int svm_unregister_enc_region(struct kvm *kvm,
 	.mpx_supported = svm_mpx_supported,
 	.xsaves_supported = svm_xsaves_supported,
 	.umip_emulated = svm_umip_emulated,
+	.pt_supported = svm_pt_supported,
 
 	.set_supported_cpuid = svm_set_supported_cpuid,
 
arch/x86/kvm/vmx.c +6 -0
index bb96396..24aded4 100644
@@ -9588,6 +9588,11 @@ static bool vmx_xsaves_supported(void)
 		SECONDARY_EXEC_XSAVES;
 }
 
+static bool vmx_pt_supported(void)
+{
+	return (pt_mode == PT_MODE_HOST_GUEST);
+}
+
 static void vmx_recover_nmi_blocking(struct vcpu_vmx *vmx)
 {
 	u32 exit_intr_info;
@@ -12809,6 +12814,7 @@ static int enable_smi_window(struct kvm_vcpu *vcpu)
 	.mpx_supported = vmx_mpx_supported,
 	.xsaves_supported = vmx_xsaves_supported,
 	.umip_emulated = vmx_umip_emulated,
+	.pt_supported = vmx_pt_supported,
 
 	.check_nested_events = vmx_check_nested_events,
 
Signature · 1 line
1.8.3.1
author
#19 lore raw

PATCH v9 8/12 KVM: x86: Add Intel Processor Trace context switch for each vcpu

In reply to Luwei Kang

From: Chao Peng <[email protected]>

Load/Store Intel processor trace register in context switch. MSR IA32_RTIT_CTL is loaded/stored automatically from VMCS. In HOST_GUEST mode, we need load/resore PT MSRs only when PT is enabled in guest.

---
 arch/x86/include/asm/intel_pt.h |  2 +
 arch/x86/kvm/vmx.c              | 94 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+)
arch/x86/include/asm/intel_pt.h +2 -0
index 5748205..70f4139 100644
@@ -8,6 +8,8 @@
 #define PT_MODE_SYSTEM		0
 #define PT_MODE_HOST_GUEST	1
 
+#define RTIT_ADDR_RANGE		4
+
 enum pt_capabilities {
 	PT_CAP_max_subleaf = 0,
 	PT_CAP_cr3_filtering,
arch/x86/kvm/vmx.c +94 -0

130 lines not on this page · raw message

Signature · 1 line
1.8.3.1
author
#20 lore raw

PATCH v9 9/12 KVM: x86: Introduce a function to initialize the PT configuration

In reply to Luwei Kang

Initialize the Intel PT configuration when cpuid update. Include cpuid inforamtion, rtit_ctl bit mask and the number of address ranges.

---
 arch/x86/kvm/vmx.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)
arch/x86/kvm/vmx.c +70 -0
index 11fb90a..952ddf4 100644
@@ -10411,6 +10411,72 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)
 #undef cr4_fixed1_update
 }
 
+static void update_intel_pt_cfg(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	struct kvm_cpuid_entry2 *best = NULL;
+	int i;
+
+	for (i = 0; i < PT_CPUID_LEAVES; i++) {
+		best = kvm_find_cpuid_entry(vcpu, 0x14, i);
+		if (!best)
+			return;
+		vmx->pt_desc.caps[CPUID_EAX + i*PT_CPUID_REGS_NUM] = best->eax;
+		vmx->pt_desc.caps[CPUID_EBX + i*PT_CPUID_REGS_NUM] = best->ebx;
+		vmx->pt_desc.caps[CPUID_ECX + i*PT_CPUID_REGS_NUM] = best->ecx;
+		vmx->pt_desc.caps[CPUID_EDX + i*PT_CPUID_REGS_NUM] = best->edx;
+	}
+
+	/* Get the number of configurable Address Ranges for filtering */
+	vmx->pt_desc.addr_range = pt_cap_decode(vmx->pt_desc.caps,
+						PT_CAP_num_address_ranges);
+
+	/* Initialize and clear the no dependency bits */
+	vmx->pt_desc.ctl_bitmask = ~0ULL;
+	vmx->pt_desc.ctl_bitmask = ~(RTIT_CTL_TRACEEN | RTIT_CTL_OS |
+			RTIT_CTL_USR | RTIT_CTL_TSC_EN | RTIT_CTL_DISRETC);
+
+	/* If CPUID.(EAX=14H,ECX=0):EBX[0]=1 CR3Filter can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_cr3_filtering))
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_CR3EN;
+
+	/*
+	 * If CPUID.(EAX=14H,ECX=0):EBX[1]=1 CYCEn, CycThresh and
+	 * PSBFreq can be set
+	 */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_psb_cyc))
+		vmx->pt_desc.ctl_bitmask &= ~(RTIT_CTL_CYCLEACC |
+			RTIT_CTL_CYC_THRESH | RTIT_CTL_PSB_FREQ);
+
+	/*
+	 * If CPUID.(EAX=14H,ECX=0):EBX[3]=1 MTCEn BranchEn and
+	 * MTCFreq can be set
+	 */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_mtc))
+		vmx->pt_desc.ctl_bitmask &= ~(RTIT_CTL_MTC_EN |
+			RTIT_CTL_BRANCH_EN | RTIT_CTL_MTC_RANGE);
+
+	/* If CPUID.(EAX=14H,ECX=0):EBX[4]=1 FUPonPTW and PTWEn can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_ptwrite))
+		vmx->pt_desc.ctl_bitmask &= ~(RTIT_CTL_FUP_ON_PTW |
+						RTIT_CTL_PTW_EN);
+
+	/* If CPUID.(EAX=14H,ECX=0):EBX[5]=1 PwrEvEn can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_power_event_trace))
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_PWR_EVT_EN;
+
+	/* If CPUID.(EAX=14H,ECX=0):ECX[0]=1 ToPA can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_topa_output))
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_TOPA;
+	/* If CPUID.(EAX=14H,ECX=0):ECX[3]=1 FabircEn can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_output_subsys))
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_FABRIC_EN;
+
+	/* unmask address range configure area */
+	for (i = 0; i < vmx->pt_desc.addr_range; i++)
+		vmx->pt_desc.ctl_bitmask &= ~(0xf << (32 + i * 4));
+}
+
 static void vmx_cpuid_update(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
@@ -10429,6 +10495,10 @@ static void vmx_cpuid_update(struct kvm_vcpu *vcpu)
 
 	if (nested_vmx_allowed(vcpu))
 		nested_vmx_cr_fixed1_bits_update(vcpu);
+
+	if (boot_cpu_has(X86_FEATURE_INTEL_PT) &&
+		guest_cpuid_has(vcpu, X86_FEATURE_INTEL_PT))
+		update_intel_pt_cfg(vcpu);
 }
 
 static void vmx_set_supported_cpuid(u32 func, struct kvm_cpuid_entry2 *entry)
Signature · 1 line
1.8.3.1
#21 lore raw

Re: [PATCH v9 09/12] KVM: x86: Introduce a function to initialize the PT configuration

Quoted 1 line · Luwei Kang

Initialize the Intel PT configuration when cpuid update.

Is it the CPUID configuration? Is it the MSR configuration? Is it both? Kind of looks like both. Not sure what is the cpuid update, though.

Quoted 28 lines

Include cpuid inforamtion, rtit_ctl bit mask and the number of address ranges.

---
 arch/x86/kvm/vmx.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)
arch/x86/kvm/vmx.c +12 -0
index 11fb90a..952ddf4 100644
@@ -10411,6 +10411,72 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)
 #undef cr4_fixed1_update
 }
 
+static void update_intel_pt_cfg(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	struct kvm_cpuid_entry2 *best = NULL;
+	int i;
+
+	for (i = 0; i < PT_CPUID_LEAVES; i++) {
+		best = kvm_find_cpuid_entry(vcpu, 0x14, i);
+		if (!best)
+			return;
+		vmx->pt_desc.caps[CPUID_EAX + i*PT_CPUID_REGS_NUM] = best->eax;
+		vmx->pt_desc.caps[CPUID_EBX + i*PT_CPUID_REGS_NUM] = best->ebx;

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10435

Quoted 10 lines
arch/x86/kvm/vmx.c +10 -0
+		vmx->pt_desc.caps[CPUID_ECX + i*PT_CPUID_REGS_NUM] = best->ecx;
+		vmx->pt_desc.caps[CPUID_EDX + i*PT_CPUID_REGS_NUM] = best->edx;
+	}
+
+	/* Get the number of configurable Address Ranges for filtering */
+	vmx->pt_desc.addr_range = pt_cap_decode(vmx->pt_desc.caps,
+						PT_CAP_num_address_ranges);
+
+	/* Initialize and clear the no dependency bits */
+	vmx->pt_desc.ctl_bitmask = ~0ULL;

This looks redundant, doesn't it?

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10439

Quoted 4 lines
arch/x86/kvm/vmx.c +4 -0
+	vmx->pt_desc.ctl_bitmask = ~(RTIT_CTL_TRACEEN | RTIT_CTL_OS |
+			RTIT_CTL_USR | RTIT_CTL_TSC_EN | RTIT_CTL_DISRETC);
+
+	/* If CPUID.(EAX=14H,ECX=0):EBX[0]=1 CR3Filter can be set */

This comment makes it less clear than it would have been otherwise.

Quoted 21 lines
arch/x86/kvm/vmx.c +21 -0
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_cr3_filtering))
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_CR3EN;
+
+	/*
+	 * If CPUID.(EAX=14H,ECX=0):EBX[1]=1 CYCEn, CycThresh and
+	 * PSBFreq can be set
+	 */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_psb_cyc))
+		vmx->pt_desc.ctl_bitmask &= ~(RTIT_CTL_CYCLEACC |
+			RTIT_CTL_CYC_THRESH | RTIT_CTL_PSB_FREQ);
+
+	/*
+	 * If CPUID.(EAX=14H,ECX=0):EBX[3]=1 MTCEn BranchEn and
+	 * MTCFreq can be set
+	 */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_mtc))
+		vmx->pt_desc.ctl_bitmask &= ~(RTIT_CTL_MTC_EN |
+			RTIT_CTL_BRANCH_EN | RTIT_CTL_MTC_RANGE);
+
+	/* If CPUID.(EAX=14H,ECX=0):EBX[4]=1 FUPonPTW and PTWEn can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_ptwrite))

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10470

Quoted 10 lines
arch/x86/kvm/vmx.c +10 -0
+		vmx->pt_desc.ctl_bitmask &= ~(RTIT_CTL_FUP_ON_PTW |
+						RTIT_CTL_PTW_EN);
+
+	/* If CPUID.(EAX=14H,ECX=0):EBX[5]=1 PwrEvEn can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_power_event_trace))
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_PWR_EVT_EN;
+
+	/* If CPUID.(EAX=14H,ECX=0):ECX[0]=1 ToPA can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_topa_output))
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_TOPA;

If you want to be thorough, there's also PT_CAP_single_range_output, which tells us if RTIT_CTL_TOPA can be *unset*. Otherwise it's required.

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10473

Quoted 3 lines
arch/x86/kvm/vmx.c +3 -0
+	/* If CPUID.(EAX=14H,ECX=0):ECX[3]=1 FabircEn can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_output_subsys))
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_FABRIC_EN;

Are we sure we want to virtualize this and that it's safe?

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10477

Quoted 4 lines
arch/x86/kvm/vmx.c +4 -0
+
+	/* unmask address range configure area */
+	for (i = 0; i < vmx->pt_desc.addr_range; i++)
+		vmx->pt_desc.ctl_bitmask &= ~(0xf << (32 + i * 4));

So, the ctl_bitmask is all the bits that are not allowed?

Regards,
--
Alex

#22 lore raw

RE: [PATCH v9 09/12] KVM: x86: Introduce a function to initialize the PT configuration

Quoted 46 lines · Luwei Kang
Quoted 1 line

Initialize the Intel PT configuration when cpuid update.

Is it the CPUID configuration? Is it the MSR configuration? Is it both? Kind of looks like both. Not sure what is the cpuid update, though.

Quoted 17 lines

Include cpuid inforamtion, rtit_ctl bit mask and the number of address ranges.

---
 arch/x86/kvm/vmx.c | 70
++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)
arch/x86/kvm/vmx.c index renamed from arch/x86/kvm/vmx.c +0 -0

11fb90a..952ddf4 100644

arch/x86/kvm/vmx.c +0 -0
@@ -10411,6 +10411,72 @@ static void

nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) #undef cr4_fixed1_update }

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)

Quoted 10 lines
arch/x86/kvm/vmx.c +10 -0
+static void update_intel_pt_cfg(struct kvm_vcpu *vcpu) {
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	struct kvm_cpuid_entry2 *best = NULL;
+	int i;
+
+	for (i = 0; i < PT_CPUID_LEAVES; i++) {
+		best = kvm_find_cpuid_entry(vcpu, 0x14, i);
+		if (!best)
+			return;
+		vmx->pt_desc.caps[CPUID_EAX + i*PT_CPUID_REGS_NUM]

= best->eax;

arch/x86/kvm/vmx.c

Quoted 1 line
arch/x86/kvm/vmx.c +1 -0
+		vmx->pt_desc.caps[CPUID_EBX + i*PT_CPUID_REGS_NUM]

= best->ebx;

arch/x86/kvm/vmx.c

Quoted 1 line
arch/x86/kvm/vmx.c +1 -0
+		vmx->pt_desc.caps[CPUID_ECX + i*PT_CPUID_REGS_NUM]

= best->ecx;

arch/x86/kvm/vmx.c

Quoted 1 line
arch/x86/kvm/vmx.c +1 -0
+		vmx->pt_desc.caps[CPUID_EDX + i*PT_CPUID_REGS_NUM]

= best->edx;

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10433

Quoted 5 lines
arch/x86/kvm/vmx.c +5 -0
+	}
+
+	/* Get the number of configurable Address Ranges for filtering */
+	vmx->pt_desc.addr_range = pt_cap_decode(vmx->pt_desc.caps,
+
	PT_CAP_num_address_ranges);
Quoted 5 lines

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10435

Quoted 3 lines
arch/x86/kvm/vmx.c +3 -0
+
+	/* Initialize and clear the no dependency bits */
+	vmx->pt_desc.ctl_bitmask = ~0ULL;

This looks redundant, doesn't it?

This is a bit mask for RTIT_CTL MSR and it will make & with the value of RTIT_CLT from guest. The reserved bits will be 1 in this bit mask and the setable bits are 0.

Quoted 8 lines

arch/x86/kvm/vmx.c

Quoted 3 lines
arch/x86/kvm/vmx.c +2 -0
+	vmx->pt_desc.ctl_bitmask = ~(RTIT_CTL_TRACEEN | RTIT_CTL_OS |
+			RTIT_CTL_USR | RTIT_CTL_TSC_EN |

RTIT_CTL_DISRETC);

arch/x86/kvm/vmx.c

Quoted 2 lines
arch/x86/kvm/vmx.c +2 -0
+
+	/* If CPUID.(EAX=14H,ECX=0):EBX[0]=1 CR3Filter can be set */

This comment makes it less clear than it would have been otherwise.

What about like this?
/* If CPUID.(EAX=14H,ECX=0):EBX[0]=1 CR3Filter can be set otherwise will inject an #GP */

Quoted 29 lines
Quoted 11 lines
arch/x86/kvm/vmx.c +10 -0
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_cr3_filtering))
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_CR3EN;
+
+	/*
+	 * If CPUID.(EAX=14H,ECX=0):EBX[1]=1 CYCEn, CycThresh and
+	 * PSBFreq can be set
+	 */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_psb_cyc))
+		vmx->pt_desc.ctl_bitmask &= ~(RTIT_CTL_CYCLEACC |
+			RTIT_CTL_CYC_THRESH | RTIT_CTL_PSB_FREQ);

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)

Quoted 10 lines
arch/x86/kvm/vmx.c +10 -0
+
+	/*
+	 * If CPUID.(EAX=14H,ECX=0):EBX[3]=1 MTCEn BranchEn and
+	 * MTCFreq can be set
+	 */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_mtc))
+		vmx->pt_desc.ctl_bitmask &= ~(RTIT_CTL_MTC_EN |
+			RTIT_CTL_BRANCH_EN | RTIT_CTL_MTC_RANGE);
+
+	/* If CPUID.(EAX=14H,ECX=0):EBX[4]=1 FUPonPTW and PTWEn can

be set */

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)

Quoted 6 lines
arch/x86/kvm/vmx.c +6 -0
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_ptwrite))
+		vmx->pt_desc.ctl_bitmask &= ~(RTIT_CTL_FUP_ON_PTW |
+						RTIT_CTL_PTW_EN);
+
+	/* If CPUID.(EAX=14H,ECX=0):EBX[5]=1 PwrEvEn can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps,

PT_CAP_power_event_trace))

Quoted 8 lines

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10470

Quoted 5 lines
arch/x86/kvm/vmx.c +5 -0
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_PWR_EVT_EN;
+
+	/* If CPUID.(EAX=14H,ECX=0):ECX[0]=1 ToPA can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_topa_output))
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_TOPA;

If you want to be thorough, there's also PT_CAP_single_range_output, which tells us if RTIT_CTL_TOPA can be *unset*. Otherwise it's required.

Following the description in SDM, the default value of this bit (RTIT_CTL.ToPA) is 0. So I think we don't need to touch this bit when TOPA is not supported.

Quoted 6 lines

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10473

Quoted 4 lines
arch/x86/kvm/vmx.c +3 -0
+	/* If CPUID.(EAX=14H,ECX=0):ECX[3]=1 FabircEn can be set */
+	if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_output_subsys))
+		vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_FABRIC_EN;

Are we sure we want to virtualize this and that it's safe?

It depend on the CPUID information virtualization in guest. I think it can works (e.g. we can pass through the PCI card to guest and stream the trace to a MMIO address).

Quoted 7 lines

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10477

Quoted 5 lines
arch/x86/kvm/vmx.c +4 -0
+
+	/* unmask address range configure area */
+	for (i = 0; i < vmx->pt_desc.addr_range; i++)
+		vmx->pt_desc.ctl_bitmask &= ~(0xf << (32 + i * 4));

So, the ctl_bitmask is all the bits that are not allowed?

Yes, you are right.

Thanks,
Luwei Kang

maintainer
#23 lore raw

Re: [PATCH v9 09/12] KVM: x86: Introduce a function to initialize the PT configuration

Quoted 9 lines · Luwei
Quoted 3 lines

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10435

Quoted 2 lines
arch/x86/kvm/vmx.c +2 -0
+	/* Initialize and clear the no dependency bits */
+	vmx->pt_desc.ctl_bitmask = ~0ULL;

This looks redundant, doesn't it?

This is a bit mask for RTIT_CTL MSR and it will make & with the value of RTIT_CLT from guest. The reserved bits will be 1 in this bit mask and the setable bits are 0.

Quoted 3 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	vmx->pt_desc.ctl_bitmask = ~(RTIT_CTL_TRACEEN | RTIT_CTL_OS |
+			RTIT_CTL_USR | RTIT_CTL_TSC_EN |

RTIT_CTL_DISRETC);

But it's set twice, so the first assignment is superfluous.

Paolo

#24 lore raw

RE: [PATCH v9 09/12] KVM: x86: Introduce a function to initialize the PT configuration

Quoted 11 lines
Quoted 9 lines
Quoted 3 lines

arch/x86/kvm/vmx.c static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) · line 10435

Quoted 2 lines
arch/x86/kvm/vmx.c +2 -0
+	/* Initialize and clear the no dependency bits */
+	vmx->pt_desc.ctl_bitmask = ~0ULL;

This looks redundant, doesn't it?

This is a bit mask for RTIT_CTL MSR and it will make & with the value of RTIT_CLT from guest. The reserved bits will be 1 in this bit mask and the setable bits are 0.

Quoted 3 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	vmx->pt_desc.ctl_bitmask = ~(RTIT_CTL_TRACEEN | RTIT_CTL_OS |
+			RTIT_CTL_USR | RTIT_CTL_TSC_EN |

RTIT_CTL_DISRETC);

But it's set twice, so the first assignment is superfluous.

Oh, I see. Will remove it.

Thanks,
Luwei Kang

author
#25 lore raw

PATCH v9 10/12 KVM: x86: Implement Intel Processor Trace MSRs read/write emulation

In reply to Luwei Kang

From: Chao Peng <[email protected]>

This patch implement Intel Processor Trace MSRs read/write emulation.
Intel PT MSRs read/write need to be emulated when Intel PT MSRs is intercepted in guest and during live migration.

---
 arch/x86/include/asm/intel_pt.h |   8 ++
 arch/x86/kvm/vmx.c              | 172 ++++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/x86.c              |  33 +++++++-
 3 files changed, 212 insertions(+), 1 deletion(-)
arch/x86/include/asm/intel_pt.h +8 -0
index 70f4139..3a25dc1 100644
@@ -10,6 +10,14 @@
 
 #define RTIT_ADDR_RANGE		4
 
+#define MSR_IA32_RTIT_STATUS_MASK (~(RTIT_STATUS_FILTEREN | \
+		RTIT_STATUS_CONTEXTEN | RTIT_STATUS_TRIGGEREN | \
+		RTIT_STATUS_ERROR | RTIT_STATUS_STOPPED | \
+		RTIT_STATUS_BYTECNT))
+
+#define MSR_IA32_RTIT_OUTPUT_BASE_MASK \
+		(~((1UL << cpuid_query_maxphyaddr(vcpu)) - 1) | 0x7f)
+
 enum pt_capabilities {
 	PT_CAP_max_subleaf = 0,
 	PT_CAP_cr3_filtering,
arch/x86/kvm/vmx.c +172 -0

202 lines not on this page · raw message

arch/x86/kvm/x86.c +32 -1
index 182693f..8f3d465 100644
@@ -69,6 +69,7 @@
 #include <asm/irq_remapping.h>
 #include <asm/mshyperv.h>
 #include <asm/hypervisor.h>
+#include <asm/intel_pt.h>
 
 #define CREATE_TRACE_POINTS
 #include "trace.h"
@@ -1030,7 +1031,13 @@ bool kvm_rdpmc(struct kvm_vcpu *vcpu)
 #endif
 	MSR_IA32_TSC, MSR_IA32_CR_PAT, MSR_VM_HSAVE_PA,
 	MSR_IA32_FEATURE_CONTROL, MSR_IA32_BNDCFGS, MSR_TSC_AUX,
-	MSR_IA32_SPEC_CTRL, MSR_IA32_ARCH_CAPABILITIES
+	MSR_IA32_SPEC_CTRL, MSR_IA32_ARCH_CAPABILITIES,
+	MSR_IA32_RTIT_CTL, MSR_IA32_RTIT_STATUS, MSR_IA32_RTIT_CR3_MATCH,
+	MSR_IA32_RTIT_OUTPUT_BASE, MSR_IA32_RTIT_OUTPUT_MASK,
+	MSR_IA32_RTIT_ADDR0_A, MSR_IA32_RTIT_ADDR0_B,
+	MSR_IA32_RTIT_ADDR1_A, MSR_IA32_RTIT_ADDR1_B,
+	MSR_IA32_RTIT_ADDR2_A, MSR_IA32_RTIT_ADDR2_B,
+	MSR_IA32_RTIT_ADDR3_A, MSR_IA32_RTIT_ADDR3_B,
 };
 
 static unsigned num_msrs_to_save;
@@ -4599,6 +4606,30 @@ static void kvm_init_msr_list(void)
 			if (!kvm_x86_ops->rdtscp_supported())
 				continue;
 			break;
+		case MSR_IA32_RTIT_CTL:
+		case MSR_IA32_RTIT_STATUS:
+			if (!kvm_x86_ops->pt_supported())
+				continue;
+			break;
+		case MSR_IA32_RTIT_CR3_MATCH:
+			if (!kvm_x86_ops->pt_supported() ||
+					!pt_cap_get(PT_CAP_cr3_filtering))
+				continue;
+			break;
+		case MSR_IA32_RTIT_OUTPUT_BASE:
+		case MSR_IA32_RTIT_OUTPUT_MASK:
+			if (!kvm_x86_ops->pt_supported() ||
+				(!pt_cap_get(PT_CAP_topa_output) &&
+				 !pt_cap_get(PT_CAP_single_range_output)))
+				continue;
+			break;
+		case MSR_IA32_RTIT_ADDR0_A ... MSR_IA32_RTIT_ADDR3_B: {
+			if (!kvm_x86_ops->pt_supported() ||
+				msrs_to_save[i] - MSR_IA32_RTIT_ADDR0_A >=
+				2 * pt_cap_get(PT_CAP_num_address_ranges))
+				continue;
+			break;
+		}
 		default:
 			break;
 		}
Signature · 1 line
1.8.3.1
#26 lore raw

Re: [PATCH v9 10/12] KVM: x86: Implement Intel Processor Trace MSRs read/write emulation

Quoted 14 lines · Luwei Kang
arch/x86/kvm/vmx.c +14 -0
+	/*
+	 * Any MSR write that attempts to change bits marked reserved will
+	 * case a #GP fault.
+	 */
+	if (data & vmx->pt_desc.ctl_bitmask)
+		return 1;
+
+	/*
+	 * Any attempt to modify IA32_RTIT_CTL while TraceEn is set will
+	 * result in a #GP unless the same write also clears TraceEn.
+	 */
+	if ((vmx->pt_desc.guest.ctl & RTIT_CTL_TRACEEN) &&
+		((vmx->pt_desc.guest.ctl ^ data) & ~RTIT_CTL_TRACEEN))
+		return 1;

arch/x86/kvm/vmx.c static void vmx_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask) · line 2840

Quoted 10 lines
arch/x86/kvm/vmx.c +10 -0
+
+	/*
+	 * WRMSR to IA32_RTIT_CTL that sets TraceEn but clears this bit
+	 * and FabricEn would cause #GP, if
+	 * CPUID.(EAX=14H, ECX=0):ECX.SNGLRGNOUT[bit 2] = 0
+	 */
+	if ((data & RTIT_CTL_TRACEEN) && !(data & RTIT_CTL_TOPA) &&
+		!(data & RTIT_CTL_FABRIC_EN) &&
+		!pt_cap_decode(vmx->pt_desc.caps, PT_CAP_single_range_output))
+		return 1;

Ah, I see. But afaict this is still wrong: PT_CAP_single_range_output is only about allowing !RTIT_CTL_TOPA, FABRIC_EN should be checked separately against the new capability that you added in 4/12.

Regards,
--
Alex

#27 lore raw

RE: [PATCH v9 10/12] KVM: x86: Implement Intel Processor Trace MSRs read/write emulation

Quoted 24 lines
Quoted 13 lines
arch/x86/kvm/vmx.c +13 -0
+	/*
+	 * Any MSR write that attempts to change bits marked reserved will
+	 * case a #GP fault.
+	 */
+	if (data & vmx->pt_desc.ctl_bitmask)
+		return 1;
+
+	/*
+	 * Any attempt to modify IA32_RTIT_CTL while TraceEn is set will
+	 * result in a #GP unless the same write also clears TraceEn.
+	 */
+	if ((vmx->pt_desc.guest.ctl & RTIT_CTL_TRACEEN) &&
+		((vmx->pt_desc.guest.ctl ^ data) & ~RTIT_CTL_TRACEEN))

arch/x86/kvm/vmx.c static void vmx_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask)

Quoted 10 lines
arch/x86/kvm/vmx.c +10 -0
+		return 1;
+
+	/*
+	 * WRMSR to IA32_RTIT_CTL that sets TraceEn but clears this bit
+	 * and FabricEn would cause #GP, if
+	 * CPUID.(EAX=14H, ECX=0):ECX.SNGLRGNOUT[bit 2] = 0
+	 */
+	if ((data & RTIT_CTL_TRACEEN) && !(data & RTIT_CTL_TOPA) &&
+		!(data & RTIT_CTL_FABRIC_EN) &&
+		!pt_cap_decode(vmx->pt_desc.caps,

PT_CAP_single_range_output))

Quoted 3 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+		return 1;

Ah, I see. But afaict this is still wrong: PT_CAP_single_range_output is only about allowing !RTIT_CTL_TOPA,

This is follow the description in SDM (35.2.7.2 IA32_RTIT_CTL MSR -> ToPA -> Note:...)

Quoted 1 line

FABRIC_EN should be checked separately against the new capability that you added in 4/12.

Yes, it have has been handled in patch 9.
+ /* If CPUID.(EAX=14H,ECX=0):ECX[3]=1 FabircEn can be set */
+ if (pt_cap_decode(vmx->pt_desc.caps, PT_CAP_output_subsys))
+ vmx->pt_desc.ctl_bitmask &= ~RTIT_CTL_FABRIC_EN;

Thanks,
Luwei Kang

author
#28 lore raw

PATCH v9 11/12 KVM: x86: Set intercept for Intel PT MSRs read/write

In reply to Luwei Kang

From: Chao Peng <[email protected]>

Disable intercept Intel PT MSRs only when Intel PT is enabled in guest. But MSR_IA32_RTIT_CTL will alway be intercept.

---
 arch/x86/kvm/vmx.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
arch/x86/kvm/vmx.c +23 -0
index 770cb7c..a09157c 100644
@@ -948,6 +948,7 @@ static bool nested_vmx_is_page_fault_vmexit(struct vmcs12 *vmcs12,
 static void vmx_update_msr_bitmap(struct kvm_vcpu *vcpu);
 static void __always_inline vmx_disable_intercept_for_msr(unsigned long *msr_bitmap,
 							  u32 msr, int type);
+static void pt_set_intercept_for_msr(struct vcpu_vmx *vmx, bool flag);
 
 static DEFINE_PER_CPU(struct vmcs *, vmxarea);
 static DEFINE_PER_CPU(struct vmcs *, current_vmcs);
@@ -3999,6 +4000,7 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			vmx_rtit_ctl_check(vcpu, data))
 			return 1;
 		vmcs_write64(GUEST_IA32_RTIT_CTL, data);
+		pt_set_intercept_for_msr(vmx, !(data & RTIT_CTL_TRACEEN));
 		vmx->pt_desc.guest.ctl = data;
 		break;
 	case MSR_IA32_RTIT_STATUS:
@@ -5820,6 +5822,27 @@ static void vmx_update_msr_bitmap(struct kvm_vcpu *vcpu)
 	vmx->msr_bitmap_mode = mode;
 }
 
+static void pt_set_intercept_for_msr(struct vcpu_vmx *vmx, bool flag)
+{
+	unsigned long *msr_bitmap = vmx->vmcs01.msr_bitmap;
+	u32 i;
+
+	vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_RTIT_STATUS,
+							MSR_TYPE_RW, flag);
+	vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_RTIT_OUTPUT_BASE,
+							MSR_TYPE_RW, flag);
+	vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_RTIT_OUTPUT_MASK,
+							MSR_TYPE_RW, flag);
+	vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_RTIT_CR3_MATCH,
+							MSR_TYPE_RW, flag);
+	for (i = 0; i < vmx->pt_desc.addr_range; i++) {
+		vmx_set_intercept_for_msr(msr_bitmap,
+			MSR_IA32_RTIT_ADDR0_A + i * 2, MSR_TYPE_RW, flag);
+		vmx_set_intercept_for_msr(msr_bitmap,
+			MSR_IA32_RTIT_ADDR0_B + i * 2, MSR_TYPE_RW, flag);
+	}
+}
+
 static bool vmx_get_enable_apicv(struct kvm_vcpu *vcpu)
 {
 	return enable_apicv;
Signature · 1 line
1.8.3.1
#29 lore raw

Re: [PATCH v9 11/12] KVM: x86: Set intercept for Intel PT MSRs read/write

Quoted 5 lines · Luwei Kang

From: Chao Peng <[email protected]>

Disable intercept Intel PT MSRs only when Intel PT is enabled in guest. But MSR_IA32_RTIT_CTL will alway be intercept.

"I'd like to offer some suggestions as to how to make the commit message friendlier for reviewing.

Generally, for every patch, we want to explain the following: what we want, why we want it and how we want to go about getting it. We also would prefer to do it in english rather than in C, because for the latter we can just look at the code." [1]

I apologize for quoting myself or if I'm stating the obvious, but it looks appropriate here.

[1] https://marc.info/?l=linux-kernel&m=152233031020263

Regards,
--
Alex

#30 lore raw

RE: [PATCH v9 11/12] KVM: x86: Set intercept for Intel PT MSRs read/write

Quoted 13 lines
Quoted 4 lines

From: Chao Peng <[email protected]>

Disable intercept Intel PT MSRs only when Intel PT is enabled in guest. But MSR_IA32_RTIT_CTL will alway be intercept.

"I'd like to offer some suggestions as to how to make the commit message friendlier for reviewing.

Generally, for every patch, we want to explain the following: what we want, why we want it and how we want to go about getting it. We also would prefer to do it in english rather than in C, because for the latter we can just look at the code." [1]

Quoted 5 lines

I apologize for quoting myself or if I'm stating the obvious, but it looks appropriate here.

[1] https://marc.info/?l=linux-kernel&m=152233031020263

Thank you very much. My English is not good and lack some experience of how to make description more clearly. Will improve it.

Thanks,
Luwei Kang

author
#31 lore raw

PATCH v9 12/12 KVM: x86: Disable Intel Processor Trace when VMXON in L1 guest

In reply to Luwei Kang

Currently, Intel Processor Trace do not support tracing in L1 guest VMX operation(IA32_VMX_MISC[bit 14] is 0). As mentioned in SDM, on these type of processors, execution of the VMXON instruction will clears IA32_RTIT_CTL.TraceEn and any attempt to write IA32_RTIT_CTL causes a general-protection exception (#GP).

---
 arch/x86/kvm/vmx.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx.c +7 -1
index a09157c..093c1f7 100644
@@ -3997,7 +3997,8 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		break;
 	case MSR_IA32_RTIT_CTL:
 		if ((pt_mode != PT_MODE_HOST_GUEST) ||
-			vmx_rtit_ctl_check(vcpu, data))
+			vmx_rtit_ctl_check(vcpu, data) ||
+			vmx->nested.vmxon)
 			return 1;
 		vmcs_write64(GUEST_IA32_RTIT_CTL, data);
 		pt_set_intercept_for_msr(vmx, !(data & RTIT_CTL_TRACEEN));
@@ -8090,6 +8091,11 @@ static int handle_vmon(struct kvm_vcpu *vcpu)
 	if (ret)
 		return ret;
 
+	if (pt_mode == PT_MODE_HOST_GUEST) {
+		vmx->pt_desc.guest.ctl = 0;
+		pt_set_intercept_for_msr(vmx, 1);
+	}
+
 	nested_vmx_succeed(vcpu);
 	return kvm_skip_emulated_instruction(vcpu);
 }
Signature · 1 line
1.8.3.1