Skip to contentSkip to conversation

x86/pmu: PEBS fixes and new testcases

PATCH Applied Sean Christopherson 4 participants 15 messages Mar 6, 2024 – Jun 6, 2024 KVM

lore
Share & tools
lore

Sean Christopherson

author reviewer
#0 lore raw

cover x86/pmu: PEBS fixes and new testcases

One bug fix where pmu_pebs attempts to enable PEBS for fixed counter on CPUs without Extended PEBS, and two new testcases to verify adaptive PEBS functionality.

The new testcases are intended both to demonstrate that adaptive PEBS virtualization is currently broken, and to serve as a gatekeeper for re-enabling adapative PEBS in the future.

https://lore.kernel.org/all/[email protected]

Sean Christopherson (4):
x86/pmu: Enable PEBS on fixed counters iff baseline PEBS is support x86/pmu: Iterate over adaptive PEBS flag combinations x86/pmu: Test adaptive PEBS without any adaptive counters x86/pmu: Add a PEBS test to verify the host LBRs aren't leaked to the
guest

 lib/x86/pmu.h  |   6 ++-
 x86/pmu_pebs.c | 109 +++++++++++++++++++++++++++----------------------
 2 files changed, 66 insertions(+), 49 deletions(-)

base-commit: 55dd01b4f066577b49f03fb7146723c4a65822c4

Signature · 1 line
2.44.0.278.ge034bb2e1d-goog
author reviewer
#1 lore raw

PATCH 1/4 x86/pmu: Enable PEBS on fixed counters iff baseline PEBS is support

Enabling PEBS for fixed counters is only allowed if "Extended PEBS" is supported, and unfortunately "Extended PEBS" is bundled with a pile of other PEBS features under the "Baseline" umbrella. KVM emulates this correctly and disallows enabling PEBS on fixed counters if the vCPU doesn't have Baseline PEBS support.

---
 x86/pmu_pebs.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
x86/pmu_pebs.c +2 -2
index f7b52b90..050617cd 100644
@@ -356,13 +356,13 @@ static void check_pebs_counters(u64 pebs_data_cfg)
 	unsigned int idx;
 	u64 bitmask = 0;
 
-	for (idx = 0; idx < pmu.nr_fixed_counters; idx++)
+	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)
 		check_one_counter(FIXED, idx, pebs_data_cfg);
 
 	for (idx = 0; idx < max_nr_gp_events; idx++)
 		check_one_counter(GP, idx, pebs_data_cfg);
 
-	for (idx = 0; idx < pmu.nr_fixed_counters; idx++)
+	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)
 		bitmask |= BIT_ULL(FIXED_CNT_INDEX + idx);
 	for (idx = 0; idx < max_nr_gp_events; idx += 2)
 		bitmask |= BIT_ULL(idx);
Signature · 1 line
2.44.0.278.ge034bb2e1d-goog
Reviewed-by on v1 1/4
#2 lore raw

Re: [kvm-unit-tests PATCH 1/4] x86/pmu: Enable PEBS on fixed counters iff baseline PEBS is support

Quoted 21 lines · Sean Christopherson

Enabling PEBS for fixed counters is only allowed if "Extended PEBS" is supported, and unfortunately "Extended PEBS" is bundled with a pile of other PEBS features under the "Baseline" umbrella. KVM emulates this correctly and disallows enabling PEBS on fixed counters if the vCPU doesn't have Baseline PEBS support.

---
  x86/pmu_pebs.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
x86/pmu_pebs.c +1 -1
index f7b52b90..050617cd 100644
@@ -356,13 +356,13 @@ static void check_pebs_counters(u64 pebs_data_cfg)
  	unsigned int idx;
  	u64 bitmask = 0;
 
-	for (idx = 0; idx < pmu.nr_fixed_counters; idx++)
+	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)

x86/pmu_pebs.c static void check_pebs_counters(u64 pebs_data_cfg) · line 368

Quoted 10 lines
x86/pmu_pebs.c +1 -1
  		check_one_counter(FIXED, idx, pebs_data_cfg);
 
  	for (idx = 0; idx < max_nr_gp_events; idx++)
  		check_one_counter(GP, idx, pebs_data_cfg);
 
-	for (idx = 0; idx < pmu.nr_fixed_counters; idx++)
+	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)
  		bitmask |= BIT_ULL(FIXED_CNT_INDEX + idx);
  	for (idx = 0; idx < max_nr_gp_events; idx += 2)
  		bitmask |= BIT_ULL(idx);
author reviewer
#3 lore raw

PATCH 2/4 x86/pmu: Iterate over adaptive PEBS flag combinations

In reply to Sean Christopherson

Iterate over all possible combinations of adaptive PEBS flags, instead of simply testing each flag individually. There are currently only 16 possible combinations, i.e. there's no reason not to exhaustively test every one.

Opportunistically rename PEBS_DATACFG_GP to PEBS_DATACFG_GPRS to differentiate it from general purposes *counters*, which KVM also tends to abbreviate as "GP".

---
 lib/x86/pmu.h  |  6 +++++-
 x86/pmu_pebs.c | 20 +++++++++-----------
 2 files changed, 14 insertions(+), 12 deletions(-)
lib/x86/pmu.h +5 -1
index 8465e3c9..f07fbd93 100644
@@ -44,9 +44,13 @@
 #define GLOBAL_STATUS_BUFFER_OVF	BIT_ULL(GLOBAL_STATUS_BUFFER_OVF_BIT)
 
 #define PEBS_DATACFG_MEMINFO	BIT_ULL(0)
-#define PEBS_DATACFG_GP	BIT_ULL(1)
+#define PEBS_DATACFG_GPRS	BIT_ULL(1)
 #define PEBS_DATACFG_XMMS	BIT_ULL(2)
 #define PEBS_DATACFG_LBRS	BIT_ULL(3)
+#define PEBS_DATACFG_MASK	(PEBS_DATACFG_MEMINFO | \
+				 PEBS_DATACFG_GPRS | \
+				 PEBS_DATACFG_XMMS | \
+				 PEBS_DATACFG_LBRS)
 
 #define ICL_EVENTSEL_ADAPTIVE				(1ULL << 34)
 #define PEBS_DATACFG_LBR_SHIFT	24
x86/pmu_pebs.c +9 -11
index 050617cd..dff1ed26 100644
@@ -78,13 +78,6 @@ static uint32_t intel_arch_events[] = {
 	0x412e, /* PERF_COUNT_HW_CACHE_MISSES */
 };
 
-static u64 pebs_data_cfgs[] = {
-	PEBS_DATACFG_MEMINFO,
-	PEBS_DATACFG_GP,
-	PEBS_DATACFG_XMMS,
-	PEBS_DATACFG_LBRS | ((MAX_NUM_LBR_ENTRY -1) << PEBS_DATACFG_LBR_SHIFT),
-};
-
 /* Iterating each counter value is a waste of time, pick a few typical values. */
 static u64 counter_start_values[] = {
 	/* if PEBS counter doesn't overflow at all */
@@ -105,7 +98,7 @@ static unsigned int get_adaptive_pebs_record_size(u64 pebs_data_cfg)
 
 	if (pebs_data_cfg & PEBS_DATACFG_MEMINFO)
 		sz += sizeof(struct pebs_meminfo);
-	if (pebs_data_cfg & PEBS_DATACFG_GP)
+	if (pebs_data_cfg & PEBS_DATACFG_GPRS)
 		sz += sizeof(struct pebs_gprs);
 	if (pebs_data_cfg & PEBS_DATACFG_XMMS)
 		sz += sizeof(struct pebs_xmm);
@@ -419,9 +412,14 @@ int main(int ac, char **av)
 		if (!has_baseline)
 			continue;
 
-		for (j = 0; j < ARRAY_SIZE(pebs_data_cfgs); j++) {
-			report_prefix_pushf("Adaptive (0x%lx)", pebs_data_cfgs[j]);
-			check_pebs_counters(pebs_data_cfgs[j]);
+		for (j = 0; j <= PEBS_DATACFG_MASK; j++) {
+			u64 pebs_data_cfg = j;
+
+			if (pebs_data_cfg & PEBS_DATACFG_LBRS)
+				pebs_data_cfg |= ((MAX_NUM_LBR_ENTRY -1) << PEBS_DATACFG_LBR_SHIFT);
+
+			report_prefix_pushf("Adaptive (0x%lx)", pebs_data_cfg);
+			check_pebs_counters(pebs_data_cfg);
 			report_prefix_pop();
 		}
 	}
Signature · 1 line
2.44.0.278.ge034bb2e1d-goog
author reviewer
#4 lore raw

PATCH 3/4 x86/pmu: Test adaptive PEBS without any adaptive counters

In reply to Sean Christopherson

If adaptive PEBS is supported, verify that only basic PEBS records are generated for counters without their ADAPTIVE flag set. Per the SDM, adaptive records are generated if and only if both the per-counter flag *and* the global enable(s) in MSR_PEBS_DATA_CFG are set.

IA32_PERFEVTSELx.Adaptive_Record[34]: If this bit is set and IA32_PEBS_ENABLE.PEBS_EN_PMCx is set for the corresponding GP counter, an overflow of PMCx results in generation of an adaptive PEBS record with state information based on the selections made in MSR_PEBS_DATA_CFG. If this bit is not set, a basic record is generated.

and

IA32_FIXED_CTR_CTRL.FCx_Adaptive_Record: If this bit is set and IA32_PEBS_ENABLE.PEBS_EN_FIXEDx is set for the corresponding Fixed counter, an overflow of FixedCtrx results in generation of an adaptive PEBS record with state information based on the selections made in MSR_PEBS_DATA_CFG. If this bit is not set, a basic record is generated.

---
 x86/pmu_pebs.c | 74 ++++++++++++++++++++++++++------------------------
 1 file changed, 38 insertions(+), 36 deletions(-)
x86/pmu_pebs.c +38 -36
index dff1ed26..0e8d60c3 100644
@@ -89,11 +89,11 @@ static u64 counter_start_values[] = {
 	0xffffffffffff,
 };
 
-static unsigned int get_adaptive_pebs_record_size(u64 pebs_data_cfg)
+static unsigned int get_pebs_record_size(u64 pebs_data_cfg, bool use_adaptive)
 {
 	unsigned int sz = sizeof(struct pebs_basic);
 
-	if (!has_baseline)
+	if (!use_adaptive)
 		return sz;
 
 	if (pebs_data_cfg & PEBS_DATACFG_MEMINFO)
@@ -199,10 +199,10 @@ static void free_buffers(void)
 		free_page(pebs_buffer);
 }
 
-static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
+static void pebs_enable(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive)
 {
 	static struct debug_store *ds;
-	u64 baseline_extra_ctrl = 0, fixed_ctr_ctrl = 0;
+	u64 adaptive_ctrl = 0, fixed_ctr_ctrl = 0;
 	unsigned int idx;
 
 	if (has_baseline)
@@ -212,15 +212,15 @@ static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
 	ds->pebs_index = ds->pebs_buffer_base = (unsigned long)pebs_buffer;
 	ds->pebs_absolute_maximum = (unsigned long)pebs_buffer + PAGE_SIZE;
 	ds->pebs_interrupt_threshold = ds->pebs_buffer_base +
-		get_adaptive_pebs_record_size(pebs_data_cfg);
+		get_pebs_record_size(pebs_data_cfg, use_adaptive);
 
 	for (idx = 0; idx < pmu.nr_fixed_counters; idx++) {
 		if (!(BIT_ULL(FIXED_CNT_INDEX + idx) & bitmask))
 			continue;
-		if (has_baseline)
-			baseline_extra_ctrl = BIT(FIXED_CNT_INDEX + idx * 4);
+		if (use_adaptive)
+			adaptive_ctrl = BIT(FIXED_CNT_INDEX + idx * 4);
 		wrmsr(MSR_PERF_FIXED_CTRx(idx), ctr_start_val);
-		fixed_ctr_ctrl |= (0xbULL << (idx * 4) | baseline_extra_ctrl);
+		fixed_ctr_ctrl |= (0xbULL << (idx * 4) | adaptive_ctrl);
 	}
 	if (fixed_ctr_ctrl)
 		wrmsr(MSR_CORE_PERF_FIXED_CTR_CTRL, fixed_ctr_ctrl);
@@ -228,10 +228,10 @@ static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
 	for (idx = 0; idx < max_nr_gp_events; idx++) {
 		if (!(BIT_ULL(idx) & bitmask))
 			continue;
-		if (has_baseline)
-			baseline_extra_ctrl = ICL_EVENTSEL_ADAPTIVE;
+		if (use_adaptive)
+			adaptive_ctrl = ICL_EVENTSEL_ADAPTIVE;
 		wrmsr(MSR_GP_EVENT_SELECTx(idx), EVNTSEL_EN | EVNTSEL_OS | EVNTSEL_USR |
-						 intel_arch_events[idx] | baseline_extra_ctrl);
+						 intel_arch_events[idx] | adaptive_ctrl);
 		wrmsr(MSR_GP_COUNTERx(idx), ctr_start_val);
 	}
 
@@ -268,11 +268,11 @@ static void pebs_disable(unsigned int idx)
 	wrmsr(MSR_CORE_PERF_GLOBAL_CTRL, 0);
 }
 
-static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
+static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive)
 {
 	struct pebs_basic *pebs_rec = (struct pebs_basic *)pebs_buffer;
 	struct debug_store *ds = (struct debug_store *)ds_bufer;
-	unsigned int pebs_record_size = get_adaptive_pebs_record_size(pebs_data_cfg);
+	unsigned int pebs_record_size;
 	unsigned int count = 0;
 	bool expected, pebs_idx_match, pebs_size_match, data_cfg_match;
 	void *cur_record;
@@ -293,12 +293,9 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
 	do {
 		pebs_rec = (struct pebs_basic *)cur_record;
 		pebs_record_size = pebs_rec->format_size >> RECORD_SIZE_OFFSET;
-		pebs_idx_match =
-			pebs_rec->applicable_counters & bitmask;
-		pebs_size_match =
-			pebs_record_size == get_adaptive_pebs_record_size(pebs_data_cfg);
-		data_cfg_match =
-			(pebs_rec->format_size & GENMASK_ULL(47, 0)) == pebs_data_cfg;
+		pebs_idx_match = pebs_rec->applicable_counters & bitmask;
+		pebs_size_match = pebs_record_size == get_pebs_record_size(pebs_data_cfg, use_adaptive);
+		data_cfg_match = (pebs_rec->format_size & GENMASK_ULL(47, 0)) == pebs_data_cfg;
 		expected = pebs_idx_match && pebs_size_match && data_cfg_match;
 		report(expected,
 		       "PEBS record (written seq %d) is verified (including size, counters and cfg).", count);
@@ -311,56 +308,57 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
 			printf("FAIL: The applicable_counters (0x%lx) doesn't match with pmc_bitmask (0x%lx).\n",
 			       pebs_rec->applicable_counters, bitmask);
 		if (!pebs_size_match)
-			printf("FAIL: The pebs_record_size (%d) doesn't match with MSR_PEBS_DATA_CFG (%d).\n",
-			       pebs_record_size, get_adaptive_pebs_record_size(pebs_data_cfg));
+			printf("FAIL: The pebs_record_size (%d) doesn't match with expected record size (%d).\n",
+			       pebs_record_size, get_pebs_record_size(pebs_data_cfg, use_adaptive));
 		if (!data_cfg_match)
-			printf("FAIL: The pebs_data_cfg (0x%lx) doesn't match with MSR_PEBS_DATA_CFG (0x%lx).\n",
-			       pebs_rec->format_size & 0xffffffffffff, pebs_data_cfg);
+			printf("FAIL: The pebs_data_cfg (0x%lx) doesn't match with the effective MSR_PEBS_DATA_CFG (0x%lx).\n",
+			       pebs_rec->format_size & 0xffffffffffff, use_adaptive ? pebs_data_cfg : 0);
 	}
 }
 
-static void check_one_counter(enum pmc_type type,
-			      unsigned int idx, u64 pebs_data_cfg)
+static void check_one_counter(enum pmc_type type, unsigned int idx,
+			      u64 pebs_data_cfg, bool use_adaptive)
 {
 	int pebs_bit = BIT_ULL(type == FIXED ? FIXED_CNT_INDEX + idx : idx);
 
 	report_prefix_pushf("%s counter %d (0x%lx)",
 			    type == FIXED ? "Extended Fixed" : "GP", idx, ctr_start_val);
 	reset_pebs();
-	pebs_enable(pebs_bit, pebs_data_cfg);
+	pebs_enable(pebs_bit, pebs_data_cfg, use_adaptive);
 	workload();
 	pebs_disable(idx);
-	check_pebs_records(pebs_bit, pebs_data_cfg);
+	check_pebs_records(pebs_bit, pebs_data_cfg, use_adaptive);
 	report_prefix_pop();
 }
 
 /* more than one PEBS records will be generated. */
-static void check_multiple_counters(u64 bitmask, u64 pebs_data_cfg)
+static void check_multiple_counters(u64 bitmask, u64 pebs_data_cfg,
+				    bool use_adaptive)
 {
 	reset_pebs();
-	pebs_enable(bitmask, pebs_data_cfg);
+	pebs_enable(bitmask, pebs_data_cfg, use_adaptive);
 	workload2();
 	pebs_disable(0);
-	check_pebs_records(bitmask, pebs_data_cfg);
+	check_pebs_records(bitmask, pebs_data_cfg, use_adaptive);
 }
 
-static void check_pebs_counters(u64 pebs_data_cfg)
+static void check_pebs_counters(u64 pebs_data_cfg, bool use_adaptive)
 {
 	unsigned int idx;
 	u64 bitmask = 0;
 
 	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)
-		check_one_counter(FIXED, idx, pebs_data_cfg);
+		check_one_counter(FIXED, idx, pebs_data_cfg, use_adaptive);
 
 	for (idx = 0; idx < max_nr_gp_events; idx++)
-		check_one_counter(GP, idx, pebs_data_cfg);
+		check_one_counter(GP, idx, pebs_data_cfg, use_adaptive);
 
 	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)
 		bitmask |= BIT_ULL(FIXED_CNT_INDEX + idx);
 	for (idx = 0; idx < max_nr_gp_events; idx += 2)
 		bitmask |= BIT_ULL(idx);
 	report_prefix_pushf("Multiple (0x%lx)", bitmask);
-	check_multiple_counters(bitmask, pebs_data_cfg);
+	check_multiple_counters(bitmask, pebs_data_cfg, use_adaptive);
 	report_prefix_pop();
 }
 
@@ -408,7 +406,7 @@ int main(int ac, char **av)
 
 	for (i = 0; i < ARRAY_SIZE(counter_start_values); i++) {
 		ctr_start_val = counter_start_values[i];
-		check_pebs_counters(0);
+		check_pebs_counters(0, false);
 		if (!has_baseline)
 			continue;
 
@@ -419,7 +417,11 @@ int main(int ac, char **av)
 				pebs_data_cfg |= ((MAX_NUM_LBR_ENTRY -1) << PEBS_DATACFG_LBR_SHIFT);
 
 			report_prefix_pushf("Adaptive (0x%lx)", pebs_data_cfg);
-			check_pebs_counters(pebs_data_cfg);
+			check_pebs_counters(pebs_data_cfg, true);
+			report_prefix_pop();
+
+			report_prefix_pushf("Ignored Adaptive (0x%lx)", pebs_data_cfg);
+			check_pebs_counters(pebs_data_cfg, false);
 			report_prefix_pop();
 		}
 	}
Signature · 1 line
2.44.0.278.ge034bb2e1d-goog
Reviewed-by on v1 3/4
#5 lore raw

Re: [kvm-unit-tests PATCH 3/4] x86/pmu: Test adaptive PEBS without any adaptive counters

Quoted 13 lines · Sean Christopherson

If adaptive PEBS is supported, verify that only basic PEBS records are generated for counters without their ADAPTIVE flag set. Per the SDM, adaptive records are generated if and only if both the per-counter flag *and* the global enable(s) in MSR_PEBS_DATA_CFG are set.

IA32_PERFEVTSELx.Adaptive_Record[34]: If this bit is set and IA32_PEBS_ENABLE.PEBS_EN_PMCx is set for the corresponding GP counter, an overflow of PMCx results in generation of an adaptive PEBS record with state information based on the selections made in MSR_PEBS_DATA_CFG. If this bit is not set, a basic record is generated.

and

Quoted 7 lines

IA32_FIXED_CTR_CTRL.FCx_Adaptive_Record: If this bit is set and IA32_PEBS_ENABLE.PEBS_EN_FIXEDx is set for the corresponding Fixed counter, an overflow of FixedCtrx results in generation of an adaptive PEBS record with state information based on the selections made in MSR_PEBS_DATA_CFG. If this bit is not set, a basic record is generated.

This test case is awesome. Also, I checked that this part of the behaviour has not changed since the first version of the specification description.

Quoted 194 lines
---
  x86/pmu_pebs.c | 74 ++++++++++++++++++++++++++------------------------
  1 file changed, 38 insertions(+), 36 deletions(-)
x86/pmu_pebs.c +38 -36
index dff1ed26..0e8d60c3 100644
@@ -89,11 +89,11 @@ static u64 counter_start_values[] = {
  	0xffffffffffff,
  };
 
-static unsigned int get_adaptive_pebs_record_size(u64 pebs_data_cfg)
+static unsigned int get_pebs_record_size(u64 pebs_data_cfg, bool use_adaptive)
  {
  	unsigned int sz = sizeof(struct pebs_basic);
 
-	if (!has_baseline)
+	if (!use_adaptive)
  		return sz;
 
  	if (pebs_data_cfg & PEBS_DATACFG_MEMINFO)
@@ -199,10 +199,10 @@ static void free_buffers(void)
  		free_page(pebs_buffer);
  }
 
-static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
+static void pebs_enable(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive)
  {
  	static struct debug_store *ds;
-	u64 baseline_extra_ctrl = 0, fixed_ctr_ctrl = 0;
+	u64 adaptive_ctrl = 0, fixed_ctr_ctrl = 0;
  	unsigned int idx;
 
  	if (has_baseline)
@@ -212,15 +212,15 @@ static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
  	ds->pebs_index = ds->pebs_buffer_base = (unsigned long)pebs_buffer;
  	ds->pebs_absolute_maximum = (unsigned long)pebs_buffer + PAGE_SIZE;
  	ds->pebs_interrupt_threshold = ds->pebs_buffer_base +
-		get_adaptive_pebs_record_size(pebs_data_cfg);
+		get_pebs_record_size(pebs_data_cfg, use_adaptive);
 
  	for (idx = 0; idx < pmu.nr_fixed_counters; idx++) {
  		if (!(BIT_ULL(FIXED_CNT_INDEX + idx) & bitmask))
  			continue;
-		if (has_baseline)
-			baseline_extra_ctrl = BIT(FIXED_CNT_INDEX + idx * 4);
+		if (use_adaptive)
+			adaptive_ctrl = BIT(FIXED_CNT_INDEX + idx * 4);
  		wrmsr(MSR_PERF_FIXED_CTRx(idx), ctr_start_val);
-		fixed_ctr_ctrl |= (0xbULL << (idx * 4) | baseline_extra_ctrl);
+		fixed_ctr_ctrl |= (0xbULL << (idx * 4) | adaptive_ctrl);
  	}
  	if (fixed_ctr_ctrl)
  		wrmsr(MSR_CORE_PERF_FIXED_CTR_CTRL, fixed_ctr_ctrl);
@@ -228,10 +228,10 @@ static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
  	for (idx = 0; idx < max_nr_gp_events; idx++) {
  		if (!(BIT_ULL(idx) & bitmask))
  			continue;
-		if (has_baseline)
-			baseline_extra_ctrl = ICL_EVENTSEL_ADAPTIVE;
+		if (use_adaptive)
+			adaptive_ctrl = ICL_EVENTSEL_ADAPTIVE;
  		wrmsr(MSR_GP_EVENT_SELECTx(idx), EVNTSEL_EN | EVNTSEL_OS | EVNTSEL_USR |
-						 intel_arch_events[idx] | baseline_extra_ctrl);
+						 intel_arch_events[idx] | adaptive_ctrl);
  		wrmsr(MSR_GP_COUNTERx(idx), ctr_start_val);
  	}
 
@@ -268,11 +268,11 @@ static void pebs_disable(unsigned int idx)
  	wrmsr(MSR_CORE_PERF_GLOBAL_CTRL, 0);
  }
 
-static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
+static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive)
  {
  	struct pebs_basic *pebs_rec = (struct pebs_basic *)pebs_buffer;
  	struct debug_store *ds = (struct debug_store *)ds_bufer;
-	unsigned int pebs_record_size = get_adaptive_pebs_record_size(pebs_data_cfg);
+	unsigned int pebs_record_size;
  	unsigned int count = 0;
  	bool expected, pebs_idx_match, pebs_size_match, data_cfg_match;
  	void *cur_record;
@@ -293,12 +293,9 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
  	do {
  		pebs_rec = (struct pebs_basic *)cur_record;
  		pebs_record_size = pebs_rec->format_size >> RECORD_SIZE_OFFSET;
-		pebs_idx_match =
-			pebs_rec->applicable_counters & bitmask;
-		pebs_size_match =
-			pebs_record_size == get_adaptive_pebs_record_size(pebs_data_cfg);
-		data_cfg_match =
-			(pebs_rec->format_size & GENMASK_ULL(47, 0)) == pebs_data_cfg;
+		pebs_idx_match = pebs_rec->applicable_counters & bitmask;
+		pebs_size_match = pebs_record_size == get_pebs_record_size(pebs_data_cfg, use_adaptive);
+		data_cfg_match = (pebs_rec->format_size & GENMASK_ULL(47, 0)) == pebs_data_cfg;
  		expected = pebs_idx_match && pebs_size_match && data_cfg_match;
  		report(expected,
  		       "PEBS record (written seq %d) is verified (including size, counters and cfg).", count);
@@ -311,56 +308,57 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
  			printf("FAIL: The applicable_counters (0x%lx) doesn't match with pmc_bitmask (0x%lx).\n",
  			       pebs_rec->applicable_counters, bitmask);
  		if (!pebs_size_match)
-			printf("FAIL: The pebs_record_size (%d) doesn't match with MSR_PEBS_DATA_CFG (%d).\n",
-			       pebs_record_size, get_adaptive_pebs_record_size(pebs_data_cfg));
+			printf("FAIL: The pebs_record_size (%d) doesn't match with expected record size (%d).\n",
+			       pebs_record_size, get_pebs_record_size(pebs_data_cfg, use_adaptive));
  		if (!data_cfg_match)
-			printf("FAIL: The pebs_data_cfg (0x%lx) doesn't match with MSR_PEBS_DATA_CFG (0x%lx).\n",
-			       pebs_rec->format_size & 0xffffffffffff, pebs_data_cfg);
+			printf("FAIL: The pebs_data_cfg (0x%lx) doesn't match with the effective MSR_PEBS_DATA_CFG (0x%lx).\n",
+			       pebs_rec->format_size & 0xffffffffffff, use_adaptive ? pebs_data_cfg : 0);
  	}
  }
 
-static void check_one_counter(enum pmc_type type,
-			      unsigned int idx, u64 pebs_data_cfg)
+static void check_one_counter(enum pmc_type type, unsigned int idx,
+			      u64 pebs_data_cfg, bool use_adaptive)
  {
  	int pebs_bit = BIT_ULL(type == FIXED ? FIXED_CNT_INDEX + idx : idx);
 
  	report_prefix_pushf("%s counter %d (0x%lx)",
  			    type == FIXED ? "Extended Fixed" : "GP", idx, ctr_start_val);
  	reset_pebs();
-	pebs_enable(pebs_bit, pebs_data_cfg);
+	pebs_enable(pebs_bit, pebs_data_cfg, use_adaptive);
  	workload();
  	pebs_disable(idx);
-	check_pebs_records(pebs_bit, pebs_data_cfg);
+	check_pebs_records(pebs_bit, pebs_data_cfg, use_adaptive);
  	report_prefix_pop();
  }
 
  /* more than one PEBS records will be generated. */
-static void check_multiple_counters(u64 bitmask, u64 pebs_data_cfg)
+static void check_multiple_counters(u64 bitmask, u64 pebs_data_cfg,
+				    bool use_adaptive)
  {
  	reset_pebs();
-	pebs_enable(bitmask, pebs_data_cfg);
+	pebs_enable(bitmask, pebs_data_cfg, use_adaptive);
  	workload2();
  	pebs_disable(0);
-	check_pebs_records(bitmask, pebs_data_cfg);
+	check_pebs_records(bitmask, pebs_data_cfg, use_adaptive);
  }
 
-static void check_pebs_counters(u64 pebs_data_cfg)
+static void check_pebs_counters(u64 pebs_data_cfg, bool use_adaptive)
  {
  	unsigned int idx;
  	u64 bitmask = 0;
 
  	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)
-		check_one_counter(FIXED, idx, pebs_data_cfg);
+		check_one_counter(FIXED, idx, pebs_data_cfg, use_adaptive);
 
  	for (idx = 0; idx < max_nr_gp_events; idx++)
-		check_one_counter(GP, idx, pebs_data_cfg);
+		check_one_counter(GP, idx, pebs_data_cfg, use_adaptive);
 
  	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)
  		bitmask |= BIT_ULL(FIXED_CNT_INDEX + idx);
  	for (idx = 0; idx < max_nr_gp_events; idx += 2)
  		bitmask |= BIT_ULL(idx);
  	report_prefix_pushf("Multiple (0x%lx)", bitmask);
-	check_multiple_counters(bitmask, pebs_data_cfg);
+	check_multiple_counters(bitmask, pebs_data_cfg, use_adaptive);
  	report_prefix_pop();
  }
 
@@ -408,7 +406,7 @@ int main(int ac, char **av)
 
  	for (i = 0; i < ARRAY_SIZE(counter_start_values); i++) {
  		ctr_start_val = counter_start_values[i];
-		check_pebs_counters(0);
+		check_pebs_counters(0, false);
  		if (!has_baseline)
  			continue;
 
@@ -419,7 +417,11 @@ int main(int ac, char **av)
  				pebs_data_cfg |= ((MAX_NUM_LBR_ENTRY -1) << PEBS_DATACFG_LBR_SHIFT);
 
  			report_prefix_pushf("Adaptive (0x%lx)", pebs_data_cfg);
-			check_pebs_counters(pebs_data_cfg);
+			check_pebs_counters(pebs_data_cfg, true);
+			report_prefix_pop();
+
+			report_prefix_pushf("Ignored Adaptive (0x%lx)", pebs_data_cfg);
+			check_pebs_counters(pebs_data_cfg, false);
  			report_prefix_pop();
  		}
  	}
Reviewed-by on v1 3/4
#6 lore raw

Re: [kvm-unit-tests PATCH 3/4] x86/pmu: Test adaptive PEBS without any adaptive counters

In reply to Sean Christopherson
Quoted 106 lines · Sean Christopherson

If adaptive PEBS is supported, verify that only basic PEBS records are generated for counters without their ADAPTIVE flag set. Per the SDM, adaptive records are generated if and only if both the per-counter flag *and* the global enable(s) in MSR_PEBS_DATA_CFG are set.

IA32_PERFEVTSELx.Adaptive_Record[34]: If this bit is set and IA32_PEBS_ENABLE.PEBS_EN_PMCx is set for the corresponding GP counter, an overflow of PMCx results in generation of an adaptive PEBS record with state information based on the selections made in MSR_PEBS_DATA_CFG. If this bit is not set, a basic record is generated.

and

IA32_FIXED_CTR_CTRL.FCx_Adaptive_Record: If this bit is set and IA32_PEBS_ENABLE.PEBS_EN_FIXEDx is set for the corresponding Fixed counter, an overflow of FixedCtrx results in generation of an adaptive PEBS record with state information based on the selections made in MSR_PEBS_DATA_CFG. If this bit is not set, a basic record is generated.

---
  x86/pmu_pebs.c | 74 ++++++++++++++++++++++++++------------------------
  1 file changed, 38 insertions(+), 36 deletions(-)
x86/pmu_pebs.c +13 -13
index dff1ed26..0e8d60c3 100644
@@ -89,11 +89,11 @@ static u64 counter_start_values[] = {
  	0xffffffffffff,
  };
 
-static unsigned int get_adaptive_pebs_record_size(u64 pebs_data_cfg)
+static unsigned int get_pebs_record_size(u64 pebs_data_cfg, bool use_adaptive)
  {
  	unsigned int sz = sizeof(struct pebs_basic);
 
-	if (!has_baseline)
+	if (!use_adaptive)
  		return sz;
 
  	if (pebs_data_cfg & PEBS_DATACFG_MEMINFO)
@@ -199,10 +199,10 @@ static void free_buffers(void)
  		free_page(pebs_buffer);
  }
 
-static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
+static void pebs_enable(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive)
  {
  	static struct debug_store *ds;
-	u64 baseline_extra_ctrl = 0, fixed_ctr_ctrl = 0;
+	u64 adaptive_ctrl = 0, fixed_ctr_ctrl = 0;
  	unsigned int idx;
 
  	if (has_baseline)
@@ -212,15 +212,15 @@ static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
  	ds->pebs_index = ds->pebs_buffer_base = (unsigned long)pebs_buffer;
  	ds->pebs_absolute_maximum = (unsigned long)pebs_buffer + PAGE_SIZE;
  	ds->pebs_interrupt_threshold = ds->pebs_buffer_base +
-		get_adaptive_pebs_record_size(pebs_data_cfg);
+		get_pebs_record_size(pebs_data_cfg, use_adaptive);
 
  	for (idx = 0; idx < pmu.nr_fixed_counters; idx++) {
  		if (!(BIT_ULL(FIXED_CNT_INDEX + idx) & bitmask))
  			continue;
-		if (has_baseline)
-			baseline_extra_ctrl = BIT(FIXED_CNT_INDEX + idx * 4);
+		if (use_adaptive)
+			adaptive_ctrl = BIT(FIXED_CNT_INDEX + idx * 4);
  		wrmsr(MSR_PERF_FIXED_CTRx(idx), ctr_start_val);
-		fixed_ctr_ctrl |= (0xbULL << (idx * 4) | baseline_extra_ctrl);
+		fixed_ctr_ctrl |= (0xbULL << (idx * 4) | adaptive_ctrl);
  	}
  	if (fixed_ctr_ctrl)
  		wrmsr(MSR_CORE_PERF_FIXED_CTR_CTRL, fixed_ctr_ctrl);
@@ -228,10 +228,10 @@ static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
  	for (idx = 0; idx < max_nr_gp_events; idx++) {
  		if (!(BIT_ULL(idx) & bitmask))
  			continue;
-		if (has_baseline)
-			baseline_extra_ctrl = ICL_EVENTSEL_ADAPTIVE;
+		if (use_adaptive)
+			adaptive_ctrl = ICL_EVENTSEL_ADAPTIVE;
  		wrmsr(MSR_GP_EVENT_SELECTx(idx), EVNTSEL_EN | EVNTSEL_OS | EVNTSEL_USR |
-						 intel_arch_events[idx] | baseline_extra_ctrl);
+						 intel_arch_events[idx] | adaptive_ctrl);
  		wrmsr(MSR_GP_COUNTERx(idx), ctr_start_val);
  	}
 
@@ -268,11 +268,11 @@ static void pebs_disable(unsigned int idx)
  	wrmsr(MSR_CORE_PERF_GLOBAL_CTRL, 0);
  }
 
-static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
+static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive)
  {
  	struct pebs_basic *pebs_rec = (struct pebs_basic *)pebs_buffer;
  	struct debug_store *ds = (struct debug_store *)ds_bufer;
-	unsigned int pebs_record_size = get_adaptive_pebs_record_size(pebs_data_cfg);
+	unsigned int pebs_record_size;
  	unsigned int count = 0;
  	bool expected, pebs_idx_match, pebs_size_match, data_cfg_match;
  	void *cur_record;
@@ -293,12 +293,9 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
  	do {
  		pebs_rec = (struct pebs_basic *)cur_record;

x86/pmu_pebs.c static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg) · line 298

Quoted 10 lines
x86/pmu_pebs.c +3 -6
  		pebs_record_size = pebs_rec->format_size >> RECORD_SIZE_OFFSET;
-		pebs_idx_match =
-			pebs_rec->applicable_counters & bitmask;
-		pebs_size_match =
-			pebs_record_size == get_adaptive_pebs_record_size(pebs_data_cfg);
-		data_cfg_match =
-			(pebs_rec->format_size & GENMASK_ULL(47, 0)) == pebs_data_cfg;
+		pebs_idx_match = pebs_rec->applicable_counters & bitmask;
+		pebs_size_match = pebs_record_size == get_pebs_record_size(pebs_data_cfg, use_adaptive);
+		data_cfg_match = (pebs_rec->format_size & GENMASK_ULL(47, 0)) == pebs_data_cfg;

Since there is already a macro RECORD_SIZE_OFFSET, we'd better use "RECORD_SIZE_OFFSET - 1" to replace the magic number 47.

Quoted 88 lines
x86/pmu_pebs.c +17 -16
  		expected = pebs_idx_match && pebs_size_match && data_cfg_match;
  		report(expected,
  		       "PEBS record (written seq %d) is verified (including size, counters and cfg).", count);
@@ -311,56 +308,57 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
  			printf("FAIL: The applicable_counters (0x%lx) doesn't match with pmc_bitmask (0x%lx).\n",
  			       pebs_rec->applicable_counters, bitmask);
  		if (!pebs_size_match)
-			printf("FAIL: The pebs_record_size (%d) doesn't match with MSR_PEBS_DATA_CFG (%d).\n",
-			       pebs_record_size, get_adaptive_pebs_record_size(pebs_data_cfg));
+			printf("FAIL: The pebs_record_size (%d) doesn't match with expected record size (%d).\n",
+			       pebs_record_size, get_pebs_record_size(pebs_data_cfg, use_adaptive));
  		if (!data_cfg_match)
-			printf("FAIL: The pebs_data_cfg (0x%lx) doesn't match with MSR_PEBS_DATA_CFG (0x%lx).\n",
-			       pebs_rec->format_size & 0xffffffffffff, pebs_data_cfg);
+			printf("FAIL: The pebs_data_cfg (0x%lx) doesn't match with the effective MSR_PEBS_DATA_CFG (0x%lx).\n",
+			       pebs_rec->format_size & 0xffffffffffff, use_adaptive ? pebs_data_cfg : 0);
  	}
  }
 
-static void check_one_counter(enum pmc_type type,
-			      unsigned int idx, u64 pebs_data_cfg)
+static void check_one_counter(enum pmc_type type, unsigned int idx,
+			      u64 pebs_data_cfg, bool use_adaptive)
  {
  	int pebs_bit = BIT_ULL(type == FIXED ? FIXED_CNT_INDEX + idx : idx);
 
  	report_prefix_pushf("%s counter %d (0x%lx)",
  			    type == FIXED ? "Extended Fixed" : "GP", idx, ctr_start_val);
  	reset_pebs();
-	pebs_enable(pebs_bit, pebs_data_cfg);
+	pebs_enable(pebs_bit, pebs_data_cfg, use_adaptive);
  	workload();
  	pebs_disable(idx);
-	check_pebs_records(pebs_bit, pebs_data_cfg);
+	check_pebs_records(pebs_bit, pebs_data_cfg, use_adaptive);
  	report_prefix_pop();
  }
 
  /* more than one PEBS records will be generated. */
-static void check_multiple_counters(u64 bitmask, u64 pebs_data_cfg)
+static void check_multiple_counters(u64 bitmask, u64 pebs_data_cfg,
+				    bool use_adaptive)
  {
  	reset_pebs();
-	pebs_enable(bitmask, pebs_data_cfg);
+	pebs_enable(bitmask, pebs_data_cfg, use_adaptive);
  	workload2();
  	pebs_disable(0);
-	check_pebs_records(bitmask, pebs_data_cfg);
+	check_pebs_records(bitmask, pebs_data_cfg, use_adaptive);
  }
 
-static void check_pebs_counters(u64 pebs_data_cfg)
+static void check_pebs_counters(u64 pebs_data_cfg, bool use_adaptive)
  {
  	unsigned int idx;
  	u64 bitmask = 0;
 
  	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)
-		check_one_counter(FIXED, idx, pebs_data_cfg);
+		check_one_counter(FIXED, idx, pebs_data_cfg, use_adaptive);
 
  	for (idx = 0; idx < max_nr_gp_events; idx++)
-		check_one_counter(GP, idx, pebs_data_cfg);
+		check_one_counter(GP, idx, pebs_data_cfg, use_adaptive);
 
  	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)
  		bitmask |= BIT_ULL(FIXED_CNT_INDEX + idx);
  	for (idx = 0; idx < max_nr_gp_events; idx += 2)
  		bitmask |= BIT_ULL(idx);
  	report_prefix_pushf("Multiple (0x%lx)", bitmask);
-	check_multiple_counters(bitmask, pebs_data_cfg);
+	check_multiple_counters(bitmask, pebs_data_cfg, use_adaptive);
  	report_prefix_pop();
  }
 
@@ -408,7 +406,7 @@ int main(int ac, char **av)
 
  	for (i = 0; i < ARRAY_SIZE(counter_start_values); i++) {
  		ctr_start_val = counter_start_values[i];
-		check_pebs_counters(0);
+		check_pebs_counters(0, false);
  		if (!has_baseline)
  			continue;
 
@@ -419,7 +417,11 @@ int main(int ac, char **av)
  				pebs_data_cfg |= ((MAX_NUM_LBR_ENTRY -1) << PEBS_DATACFG_LBR_SHIFT);
 

x86/pmu_pebs.c int main(int ac, char **av) · line 427

Quoted 10 lines
x86/pmu_pebs.c +5 -1
  			report_prefix_pushf("Adaptive (0x%lx)", pebs_data_cfg);
-			check_pebs_counters(pebs_data_cfg);
+			check_pebs_counters(pebs_data_cfg, true);
+			report_prefix_pop();
+
+			report_prefix_pushf("Ignored Adaptive (0x%lx)", pebs_data_cfg);
+			check_pebs_counters(pebs_data_cfg, false);
  			report_prefix_pop();
  		}
  	}
author reviewer
#7 lore raw

Re: [kvm-unit-tests PATCH 3/4] x86/pmu: Test adaptive PEBS without any adaptive counters

Quoted 17 lines · Dapeng Mi

x86/pmu_pebs.c static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg) · line 298

Quoted 13 lines · Sean Christopherson
x86/pmu_pebs.c +3 -6
@@ -293,12 +293,9 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
  	do {
  		pebs_rec = (struct pebs_basic *)cur_record;
  		pebs_record_size = pebs_rec->format_size >> RECORD_SIZE_OFFSET;
-		pebs_idx_match =
-			pebs_rec->applicable_counters & bitmask;
-		pebs_size_match =
-			pebs_record_size == get_adaptive_pebs_record_size(pebs_data_cfg);
-		data_cfg_match =
-			(pebs_rec->format_size & GENMASK_ULL(47, 0)) == pebs_data_cfg;
+		pebs_idx_match = pebs_rec->applicable_counters & bitmask;
+		pebs_size_match = pebs_record_size == get_pebs_record_size(pebs_data_cfg, use_adaptive);
+		data_cfg_match = (pebs_rec->format_size & GENMASK_ULL(47, 0)) == pebs_data_cfg;

Since there is already a macro RECORD_SIZE_OFFSET, we'd better use "RECORD_SIZE_OFFSET - 1" to replace the magic number 47.

Very belatedly, I disagree. That the data configuration mask isn't derived from record size. The fact that the record size is the _only_ info that's excluded is coincidental (sort of, that's not quite the right word), i.e. if we want to use a #define, then we should add an explicit define, not abuse RECORD_SIZE_OFFSET.

#8 lore raw

Re: [kvm-unit-tests PATCH 3/4] x86/pmu: Test adaptive PEBS without any adaptive counters

In reply to Sean Christopherson
Quoted 193 lines · Sean Christopherson

If adaptive PEBS is supported, verify that only basic PEBS records are generated for counters without their ADAPTIVE flag set. Per the SDM, adaptive records are generated if and only if both the per-counter flag *and* the global enable(s) in MSR_PEBS_DATA_CFG are set.

IA32_PERFEVTSELx.Adaptive_Record[34]: If this bit is set and IA32_PEBS_ENABLE.PEBS_EN_PMCx is set for the corresponding GP counter, an overflow of PMCx results in generation of an adaptive PEBS record with state information based on the selections made in MSR_PEBS_DATA_CFG. If this bit is not set, a basic record is generated.

and

IA32_FIXED_CTR_CTRL.FCx_Adaptive_Record: If this bit is set and IA32_PEBS_ENABLE.PEBS_EN_FIXEDx is set for the corresponding Fixed counter, an overflow of FixedCtrx results in generation of an adaptive PEBS record with state information based on the selections made in MSR_PEBS_DATA_CFG. If this bit is not set, a basic record is generated.

---
  x86/pmu_pebs.c | 74 ++++++++++++++++++++++++++------------------------
  1 file changed, 38 insertions(+), 36 deletions(-)
x86/pmu_pebs.c +32 -34
index dff1ed26..0e8d60c3 100644
@@ -89,11 +89,11 @@ static u64 counter_start_values[] = {
  	0xffffffffffff,
  };
 
-static unsigned int get_adaptive_pebs_record_size(u64 pebs_data_cfg)
+static unsigned int get_pebs_record_size(u64 pebs_data_cfg, bool use_adaptive)
  {
  	unsigned int sz = sizeof(struct pebs_basic);
 
-	if (!has_baseline)
+	if (!use_adaptive)
  		return sz;
 
  	if (pebs_data_cfg & PEBS_DATACFG_MEMINFO)
@@ -199,10 +199,10 @@ static void free_buffers(void)
  		free_page(pebs_buffer);
  }
 
-static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
+static void pebs_enable(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive)
  {
  	static struct debug_store *ds;
-	u64 baseline_extra_ctrl = 0, fixed_ctr_ctrl = 0;
+	u64 adaptive_ctrl = 0, fixed_ctr_ctrl = 0;
  	unsigned int idx;
 
  	if (has_baseline)
@@ -212,15 +212,15 @@ static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
  	ds->pebs_index = ds->pebs_buffer_base = (unsigned long)pebs_buffer;
  	ds->pebs_absolute_maximum = (unsigned long)pebs_buffer + PAGE_SIZE;
  	ds->pebs_interrupt_threshold = ds->pebs_buffer_base +
-		get_adaptive_pebs_record_size(pebs_data_cfg);
+		get_pebs_record_size(pebs_data_cfg, use_adaptive);
 
  	for (idx = 0; idx < pmu.nr_fixed_counters; idx++) {
  		if (!(BIT_ULL(FIXED_CNT_INDEX + idx) & bitmask))
  			continue;
-		if (has_baseline)
-			baseline_extra_ctrl = BIT(FIXED_CNT_INDEX + idx * 4);
+		if (use_adaptive)
+			adaptive_ctrl = BIT(FIXED_CNT_INDEX + idx * 4);
  		wrmsr(MSR_PERF_FIXED_CTRx(idx), ctr_start_val);
-		fixed_ctr_ctrl |= (0xbULL << (idx * 4) | baseline_extra_ctrl);
+		fixed_ctr_ctrl |= (0xbULL << (idx * 4) | adaptive_ctrl);
  	}
  	if (fixed_ctr_ctrl)
  		wrmsr(MSR_CORE_PERF_FIXED_CTR_CTRL, fixed_ctr_ctrl);
@@ -228,10 +228,10 @@ static void pebs_enable(u64 bitmask, u64 pebs_data_cfg)
  	for (idx = 0; idx < max_nr_gp_events; idx++) {
  		if (!(BIT_ULL(idx) & bitmask))
  			continue;
-		if (has_baseline)
-			baseline_extra_ctrl = ICL_EVENTSEL_ADAPTIVE;
+		if (use_adaptive)
+			adaptive_ctrl = ICL_EVENTSEL_ADAPTIVE;
  		wrmsr(MSR_GP_EVENT_SELECTx(idx), EVNTSEL_EN | EVNTSEL_OS | EVNTSEL_USR |
-						 intel_arch_events[idx] | baseline_extra_ctrl);
+						 intel_arch_events[idx] | adaptive_ctrl);
  		wrmsr(MSR_GP_COUNTERx(idx), ctr_start_val);
  	}
 
@@ -268,11 +268,11 @@ static void pebs_disable(unsigned int idx)
  	wrmsr(MSR_CORE_PERF_GLOBAL_CTRL, 0);
  }
 
-static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
+static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive)
  {
  	struct pebs_basic *pebs_rec = (struct pebs_basic *)pebs_buffer;
  	struct debug_store *ds = (struct debug_store *)ds_bufer;
-	unsigned int pebs_record_size = get_adaptive_pebs_record_size(pebs_data_cfg);
+	unsigned int pebs_record_size;
  	unsigned int count = 0;
  	bool expected, pebs_idx_match, pebs_size_match, data_cfg_match;
  	void *cur_record;
@@ -293,12 +293,9 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
  	do {
  		pebs_rec = (struct pebs_basic *)cur_record;
  		pebs_record_size = pebs_rec->format_size >> RECORD_SIZE_OFFSET;
-		pebs_idx_match =
-			pebs_rec->applicable_counters & bitmask;
-		pebs_size_match =
-			pebs_record_size == get_adaptive_pebs_record_size(pebs_data_cfg);
-		data_cfg_match =
-			(pebs_rec->format_size & GENMASK_ULL(47, 0)) == pebs_data_cfg;
+		pebs_idx_match = pebs_rec->applicable_counters & bitmask;
+		pebs_size_match = pebs_record_size == get_pebs_record_size(pebs_data_cfg, use_adaptive);
+		data_cfg_match = (pebs_rec->format_size & GENMASK_ULL(47, 0)) == pebs_data_cfg;
  		expected = pebs_idx_match && pebs_size_match && data_cfg_match;
  		report(expected,
  		       "PEBS record (written seq %d) is verified (including size, counters and cfg).", count);
@@ -311,56 +308,57 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg)
  			printf("FAIL: The applicable_counters (0x%lx) doesn't match with pmc_bitmask (0x%lx).\n",
  			       pebs_rec->applicable_counters, bitmask);
  		if (!pebs_size_match)
-			printf("FAIL: The pebs_record_size (%d) doesn't match with MSR_PEBS_DATA_CFG (%d).\n",
-			       pebs_record_size, get_adaptive_pebs_record_size(pebs_data_cfg));
+			printf("FAIL: The pebs_record_size (%d) doesn't match with expected record size (%d).\n",
+			       pebs_record_size, get_pebs_record_size(pebs_data_cfg, use_adaptive));
  		if (!data_cfg_match)
-			printf("FAIL: The pebs_data_cfg (0x%lx) doesn't match with MSR_PEBS_DATA_CFG (0x%lx).\n",
-			       pebs_rec->format_size & 0xffffffffffff, pebs_data_cfg);
+			printf("FAIL: The pebs_data_cfg (0x%lx) doesn't match with the effective MSR_PEBS_DATA_CFG (0x%lx).\n",
+			       pebs_rec->format_size & 0xffffffffffff, use_adaptive ? pebs_data_cfg : 0);
  	}
  }
 
-static void check_one_counter(enum pmc_type type,
-			      unsigned int idx, u64 pebs_data_cfg)
+static void check_one_counter(enum pmc_type type, unsigned int idx,
+			      u64 pebs_data_cfg, bool use_adaptive)
  {
  	int pebs_bit = BIT_ULL(type == FIXED ? FIXED_CNT_INDEX + idx : idx);
 
  	report_prefix_pushf("%s counter %d (0x%lx)",
  			    type == FIXED ? "Extended Fixed" : "GP", idx, ctr_start_val);
  	reset_pebs();
-	pebs_enable(pebs_bit, pebs_data_cfg);
+	pebs_enable(pebs_bit, pebs_data_cfg, use_adaptive);
  	workload();
  	pebs_disable(idx);
-	check_pebs_records(pebs_bit, pebs_data_cfg);
+	check_pebs_records(pebs_bit, pebs_data_cfg, use_adaptive);
  	report_prefix_pop();
  }
 
  /* more than one PEBS records will be generated. */
-static void check_multiple_counters(u64 bitmask, u64 pebs_data_cfg)
+static void check_multiple_counters(u64 bitmask, u64 pebs_data_cfg,
+				    bool use_adaptive)
  {
  	reset_pebs();
-	pebs_enable(bitmask, pebs_data_cfg);
+	pebs_enable(bitmask, pebs_data_cfg, use_adaptive);
  	workload2();
  	pebs_disable(0);
-	check_pebs_records(bitmask, pebs_data_cfg);
+	check_pebs_records(bitmask, pebs_data_cfg, use_adaptive);
  }
 
-static void check_pebs_counters(u64 pebs_data_cfg)
+static void check_pebs_counters(u64 pebs_data_cfg, bool use_adaptive)
  {
  	unsigned int idx;
  	u64 bitmask = 0;
 
  	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)
-		check_one_counter(FIXED, idx, pebs_data_cfg);
+		check_one_counter(FIXED, idx, pebs_data_cfg, use_adaptive);
 
  	for (idx = 0; idx < max_nr_gp_events; idx++)
-		check_one_counter(GP, idx, pebs_data_cfg);
+		check_one_counter(GP, idx, pebs_data_cfg, use_adaptive);
 
  	for (idx = 0; has_baseline && idx < pmu.nr_fixed_counters; idx++)
  		bitmask |= BIT_ULL(FIXED_CNT_INDEX + idx);
  	for (idx = 0; idx < max_nr_gp_events; idx += 2)
  		bitmask |= BIT_ULL(idx);
  	report_prefix_pushf("Multiple (0x%lx)", bitmask);
-	check_multiple_counters(bitmask, pebs_data_cfg);
+	check_multiple_counters(bitmask, pebs_data_cfg, use_adaptive);
  	report_prefix_pop();
  }
 

x86/pmu_pebs.c int main(int ac, char **av) · line 408

Quoted 3 lines
x86/pmu_pebs.c +0 -0
@@ -408,7 +406,7 @@ int main(int ac, char **av)
 
  	for (i = 0; i < ARRAY_SIZE(counter_start_values); i++) {
  		ctr_start_val = counter_start_values[i];

After introducing the "Ignored adaptive" check and expanding the pebs_data_cfg, there would be almost 1800 test case, it's not easy to read the test output and locate a certain test case, we'd better add extra print at the beginning of each start_value to distinguish them.

Quoted 18 lines
x86/pmu_pebs.c +6 -2
-		check_pebs_counters(0);
+		check_pebs_counters(0, false);
  		if (!has_baseline)
  			continue;
 
@@ -419,7 +417,11 @@ int main(int ac, char **av)
  				pebs_data_cfg |= ((MAX_NUM_LBR_ENTRY -1) << PEBS_DATACFG_LBR_SHIFT);
 
  			report_prefix_pushf("Adaptive (0x%lx)", pebs_data_cfg);
-			check_pebs_counters(pebs_data_cfg);
+			check_pebs_counters(pebs_data_cfg, true);
+			report_prefix_pop();
+
+			report_prefix_pushf("Ignored Adaptive (0x%lx)", pebs_data_cfg);
+			check_pebs_counters(pebs_data_cfg, false);
  			report_prefix_pop();
  		}
  	}
author reviewer
#9 lore raw

PATCH 4/4 x86/pmu: Add a PEBS test to verify the host LBRs aren't leaked to the guest

In reply to Sean Christopherson

When using adaptive PEBS with LBR entries, verify that the LBR entries are all '0'. If KVM fails to context switch LBRs, e.g. when the guest isn't using LBRs, as is the case in the pmu_pebs test, then adaptive PEBS can be used to read the *host* LBRs as the CPU doesn't enforce the VMX MSR bitmaps when generating PEBS records, i.e. ignores KVM's interception of reads to LBR MSRs.

This testcase is best exercised by simultaneously utilizing LBRs in the host, e.g. by running "perf record -b -e instructions", so that there is non-zero data in the LBR MSRs.

Cc Like Xu <[email protected]>
Cc Mingwei Zhang <[email protected]>
Cc Zhenyu Wang <[email protected]>
Cc Zhang Xiong <[email protected]>
Cc Lv Zhiyuan <[email protected]>
Cc Dapeng Mi <[email protected]>
---
 x86/pmu_pebs.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
x86/pmu_pebs.c +13 -0
index 0e8d60c3..df8e736f 100644
@@ -299,6 +299,19 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive
 		expected = pebs_idx_match && pebs_size_match && data_cfg_match;
 		report(expected,
 		       "PEBS record (written seq %d) is verified (including size, counters and cfg).", count);
+		if (use_adaptive && (pebs_data_cfg & PEBS_DATACFG_LBRS)) {
+			unsigned int lbrs_offset = get_pebs_record_size(pebs_data_cfg & ~PEBS_DATACFG_LBRS, true);
+			struct lbr_entry *pebs_lbrs = cur_record + lbrs_offset;
+			int i;
+
+			for (i = 0; i < MAX_NUM_LBR_ENTRY; i++) {
+				if (!pebs_lbrs[i].from && !pebs_lbrs[i].to)
+					continue;
+
+				report_fail("PEBS LBR record %u isn't empty, got from = '%lx', to = '%lx', info = '%lx'",
+					    i, pebs_lbrs[i].from, pebs_lbrs[i].to, pebs_lbrs[i].info);
+			}
+		}
 		cur_record = cur_record + pebs_record_size;
 		count++;
 	} while (expected && (void *)cur_record < (void *)ds->pebs_index);
Signature · 1 line
2.44.0.278.ge034bb2e1d-goog
Reviewed-by on v1 4/4
#10 lore raw

Re: [kvm-unit-tests PATCH 4/4] x86/pmu: Add a PEBS test to verify the host LBRs aren't leaked to the guest

Quoted 11 lines · Sean Christopherson

When using adaptive PEBS with LBR entries, verify that the LBR entries are all '0'. If KVM fails to context switch LBRs, e.g. when the guest isn't using LBRs, as is the case in the pmu_pebs test, then adaptive PEBS can be used to read the *host* LBRs as the CPU doesn't enforce the VMX MSR bitmaps when generating PEBS records, i.e. ignores KVM's interception of reads to LBR MSRs.

This testcase is best exercised by simultaneously utilizing LBRs in the host, e.g. by running "perf record -b -e instructions", so that there is non-zero data in the LBR MSRs.

Quoted 7 lines
Cc Like Xu <[email protected]>
Cc Mingwei Zhang <[email protected]>
Cc Zhenyu Wang <[email protected]>
Cc Zhang Xiong <[email protected]>
Cc Lv Zhiyuan <[email protected]>
Cc Dapeng Mi <[email protected]>

As I recall, the conclusion of the internal discussions was "low priority."

This issue has been lying on my to-do list for a long time, and finally we have a test case to hit it right in the heart.

Quoted 28 lines
---
  x86/pmu_pebs.c | 13 +++++++++++++
  1 file changed, 13 insertions(+)
x86/pmu_pebs.c +13 -0
index 0e8d60c3..df8e736f 100644
@@ -299,6 +299,19 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive
  		expected = pebs_idx_match && pebs_size_match && data_cfg_match;
  		report(expected,
  		       "PEBS record (written seq %d) is verified (including size, counters and cfg).", count);
+		if (use_adaptive && (pebs_data_cfg & PEBS_DATACFG_LBRS)) {
+			unsigned int lbrs_offset = get_pebs_record_size(pebs_data_cfg & ~PEBS_DATACFG_LBRS, true);
+			struct lbr_entry *pebs_lbrs = cur_record + lbrs_offset;
+			int i;
+
+			for (i = 0; i < MAX_NUM_LBR_ENTRY; i++) {
+				if (!pebs_lbrs[i].from && !pebs_lbrs[i].to)
+					continue;
+
+				report_fail("PEBS LBR record %u isn't empty, got from = '%lx', to = '%lx', info = '%lx'",
+					    i, pebs_lbrs[i].from, pebs_lbrs[i].to, pebs_lbrs[i].info);
+			}
+		}
  		cur_record = cur_record + pebs_record_size;
  		count++;
  	} while (expected && (void *)cur_record < (void *)ds->pebs_index);
Reviewed-by on v1 4/4
#11 lore raw

Re: [kvm-unit-tests PATCH 4/4] x86/pmu: Add a PEBS test to verify the host LBRs aren't leaked to the guest

In reply to Sean Christopherson
Quoted 36 lines · Sean Christopherson

When using adaptive PEBS with LBR entries, verify that the LBR entries are all '0'. If KVM fails to context switch LBRs, e.g. when the guest isn't using LBRs, as is the case in the pmu_pebs test, then adaptive PEBS can be used to read the *host* LBRs as the CPU doesn't enforce the VMX MSR bitmaps when generating PEBS records, i.e. ignores KVM's interception of reads to LBR MSRs.

This testcase is best exercised by simultaneously utilizing LBRs in the host, e.g. by running "perf record -b -e instructions", so that there is non-zero data in the LBR MSRs.

Cc Like Xu <[email protected]>
Cc Mingwei Zhang <[email protected]>
Cc Zhenyu Wang <[email protected]>
Cc Zhang Xiong <[email protected]>
Cc Lv Zhiyuan <[email protected]>
Cc Dapeng Mi <[email protected]>
---
  x86/pmu_pebs.c | 13 +++++++++++++
  1 file changed, 13 insertions(+)
x86/pmu_pebs.c +6 -0
index 0e8d60c3..df8e736f 100644
@@ -299,6 +299,19 @@ static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive
  		expected = pebs_idx_match && pebs_size_match && data_cfg_match;
  		report(expected,
  		       "PEBS record (written seq %d) is verified (including size, counters and cfg).", count);
+		if (use_adaptive && (pebs_data_cfg & PEBS_DATACFG_LBRS)) {
+			unsigned int lbrs_offset = get_pebs_record_size(pebs_data_cfg & ~PEBS_DATACFG_LBRS, true);
+			struct lbr_entry *pebs_lbrs = cur_record + lbrs_offset;
+			int i;
+
+			for (i = 0; i < MAX_NUM_LBR_ENTRY; i++) {

x86/pmu_pebs.c static void check_pebs_records(u64 bitmask, u64 pebs_data_cfg, bool use_adaptive · line 317

Quoted 10 lines
x86/pmu_pebs.c +7 -0
+				if (!pebs_lbrs[i].from && !pebs_lbrs[i].to)
+					continue;
+
+				report_fail("PEBS LBR record %u isn't empty, got from = '%lx', to = '%lx', info = '%lx'",
+					    i, pebs_lbrs[i].from, pebs_lbrs[i].to, pebs_lbrs[i].info);
+			}
+		}
  		cur_record = cur_record + pebs_record_size;
  		count++;
  	} while (expected && (void *)cur_record < (void *)ds->pebs_index);
Tested-by on the v1 series
#12 lore raw
In reply to Sean Christopherson
Quoted 17 lines · Sean Christopherson

One bug fix where pmu_pebs attempts to enable PEBS for fixed counter on CPUs without Extended PEBS, and two new testcases to verify adaptive PEBS functionality.

The new testcases are intended both to demonstrate that adaptive PEBS virtualization is currently broken, and to serve as a gatekeeper for re-enabling adapative PEBS in the future.

https://lore.kernel.org/all/[email protected]

Sean Christopherson (4):
x86/pmu: Enable PEBS on fixed counters iff baseline PEBS is support x86/pmu: Iterate over adaptive PEBS flag combinations x86/pmu: Test adaptive PEBS without any adaptive counters x86/pmu: Add a PEBS test to verify the host LBRs aren't leaked to the
guest

Quoted 3 lines
  lib/x86/pmu.h  |   6 ++-
  x86/pmu_pebs.c | 109 +++++++++++++++++++++++++++----------------------
  2 files changed, 66 insertions(+), 49 deletions(-)
Quoted 2 lines

base-commit: 55dd01b4f066577b49f03fb7146723c4a65822c4

author reviewer Applied
#13 lore raw
In reply to Sean Christopherson
Quoted 9 lines · Sean Christopherson

One bug fix where pmu_pebs attempts to enable PEBS for fixed counter on CPUs without Extended PEBS, and two new testcases to verify adaptive PEBS functionality.

The new testcases are intended both to demonstrate that adaptive PEBS virtualization is currently broken, and to serve as a gatekeeper for re-enabling adapative PEBS in the future.

[...]

Applied to kvm-x86 next. Dapeng, I didn't address your feedback about adding finer grained message prefixes. I'm not opposed to doing so, I'm just extremely short on cycles and want to get the fixes landed.

[1/4] x86/pmu: Enable PEBS on fixed counters iff baseline PEBS is support

      https://github.com/kvm-x86/kvm-unit-tests/commit/79aa106cd427
[2/4] x86/pmu: Iterate over adaptive PEBS flag combinations
      https://github.com/kvm-x86/kvm-unit-tests/commit/fc17d5276b38
[3/4] x86/pmu: Test adaptive PEBS without any adaptive counters
      https://github.com/kvm-x86/kvm-unit-tests/commit/2cb2af7f53db
[4/4] x86/pmu: Add a PEBS test to verify the host LBRs aren't leaked to the guest
      https://github.com/kvm-x86/kvm-unit-tests/commit/8d0f574f4e4d

--
https://github.com/kvm-x86/kvm-unit-tests/tree/next

#14 lore raw
Quoted 20 lines

-----Original Message-----
From: Sean Christopherson <[email protected]>
Sent: Thursday, June 6, 2024 7:21 AM
To: Sean Christopherson <[email protected]>; Paolo Bonzini <[email protected]>
Cc: [email protected]; Like Xu <[email protected]>; Mingwei Zhang <[email protected]>; Zhenyu Wang <[email protected]>; Zhang, Xiong Y <[email protected]>; Lv, Zhiyuan <[email protected]>; Mi, Dapeng1 <[email protected]>
Subject: Re: [kvm-unit-tests PATCH 0/4] x86/pmu: PEBS fixes and new testcases

Quoted 8 lines · Sean Christopherson

One bug fix where pmu_pebs attempts to enable PEBS for fixed counter on CPUs without Extended PEBS, and two new testcases to verify adaptive PEBS functionality.

The new testcases are intended both to demonstrate that adaptive PEBS virtualization is currently broken, and to serve as a gatekeeper for re-enabling adapative PEBS in the future.

Quoted 5 lines
Quoted 1 line

[...]

Applied to kvm-x86 next. Dapeng, I didn't address your feedback about adding finer grained message prefixes. I'm not opposed to doing so, I'm just extremely short on cycles and want to get the fixes landed.

It's fine. I think I can help to add it. 😊

Quoted 12 lines

[1/4] x86/pmu: Enable PEBS on fixed counters iff baseline PEBS is support

      https://github.com/kvm-x86/kvm-unit-tests/commit/79aa106cd427
[2/4] x86/pmu: Iterate over adaptive PEBS flag combinations
      https://github.com/kvm-x86/kvm-unit-tests/commit/fc17d5276b38
[3/4] x86/pmu: Test adaptive PEBS without any adaptive counters
      https://github.com/kvm-x86/kvm-unit-tests/commit/2cb2af7f53db
[4/4] x86/pmu: Add a PEBS test to verify the host LBRs aren't leaked to the guest
      https://github.com/kvm-x86/kvm-unit-tests/commit/8d0f574f4e4d

--
https://github.com/kvm-x86/kvm-unit-tests/tree/next