Skip to contentSkip to conversation

perf/x86: Miscellaneous PMU bug fixes

PATCH Applied Dapeng Mi 3 participants 19 messages Jun 16 – Jun 30

Subsystems (3)
lore
Share & tools
lore

Dapeng Mi

author
#0 lore raw

v4 cover perf/x86: Miscellaneous PMU bug fixes

This series groups several independent PMU fixes to simplify review and backporting.

Changes:
v3 -> v4:
- Patch 5/8: Move intel_pmu_arch_lbr_init() before intel_pmu_check_event_constraints_all() which depends on PMU_FL_BR_CNTR flag which is initialized in intel_pmu_arch_lbr_init() (Sashiko).
- Refine change log, remove markdown format (Peter).

v2 -> v3:
- Patch 2/8: Directly update PERF_EVENT_FLAG_USER_READ_CNT according to rdpmc user disable state (Peter).
- Patch 3/8: Only keep x86_lbr_type check for the detection of hardware branch type decoding (Peter).
- Patch 4/8: Switch from_plm and to_plm variables order and refine comments (Peter).
- Patch 5/8: Move intel_pmu_arch_lbr_init() after model-specfic PMU initialization to avoid extra kmem cache destroy (Peter).
- Patch 6/8: Improve change log to add more details (Peter).

v1 -> v2:
- Fallback to software branch type decoding if hardware decoding is not suppprted (Sashiko patch 4/9).
- Drop kernel IP for PERF_SAMPLE_IP if exclude_kernel attribute is required (Sashiko, patch 8/9).
- Add kernel access check when kernel callchains are requested (Sashiko, patch 9/9)
- Address Zide and Thomas's comments.
- Collect Reviewed-bys.

Patch layout:
- Patch 1/8: Fix anythread_deprecated being overwritten issue.
- Patch 2/8: Fix the issue that cap_user_rdpmc is not updated correctly.
- Patch 3/8: Fallback to software branch type decoding if no hardware decoding.
- Patch 4/8: Fix the kernel address leakage issue in LBR stack.
- Patch 5/8: Fix the issue that the return value of intel_pmu_init_hybrid() is not valiated correctly.
- Patch 6/8: Fix a "unchecked MSR access error" on PEBS_ENABLE MSR.
- Patch 7/8: Prevent a theoretical kernel register data leak in sampling.
- Patch 8/8: Add kernel access check when kernel callchains are requested.

History:
v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/

Dapeng Mi (8):
perf/x86/intel: Remove anythread_deprecated bit from perf_capabilities perf/x86/intel: Keep cap_user_rdpmc in sync with RDPMC user-disable
state
perf/x86/intel: Fallback to sw branch type decoding if no hw decoding perf/x86/intel: Fix kernel address leakages in LBR stack perf/x86/intel: Validate the return value of intel_pmu_init_hybrid() perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS perf/core: Fix kernel register info leak via hardware skid perf/core: Check kernel access when kernel callchains are requested

 arch/x86/events/core.c       |  3 +-
 arch/x86/events/intel/core.c | 60 +++++++++++++++++++++---------------
 arch/x86/events/intel/ds.c   | 13 --------
 arch/x86/events/intel/lbr.c  | 14 ++++++---
 arch/x86/events/perf_event.h |  4 +--
 kernel/events/core.c         | 41 +++++++++++++++++++-----
 6 files changed, 83 insertions(+), 52 deletions(-)

base-commit: 67d27727854def4a7e2b386429941f5c4741ccc4

Signature · 1 line
2.34.1
author
#1 lore raw

PATCH v4 1/8 perf/x86/intel: Remove anythread_deprecated bit from perf_capabilities

AnyThread mode deprecation is enumerated by CPUID.0AH:EDX[15] instead of PERF_CAPABILITIES MSR. It's not a good practice to define a bit to represent "anythread deprecation" in perf_capabilities. It leads to the anythread_deprecated bit could be overwritten by the real value of PERF_CAPABILITIES MSR, just like the below code in update_pmu_cap() does.

if (!intel_pmu_broken_perf_cap()) {
	/* Perf Metric (Bit 15) and PEBS via PT (Bit 16) are hybrid enumeration */
	rdmsrq(MSR_IA32_PERF_CAPABILITIES, hybrid(pmu, intel_cap).capabilities);
}

It leads to the anythread_deprecated bit is cleared to 0 and the "any" attribute is incorrectly shown in the /sys/devices/cpu/format/ folder on these support Perfmon v6 platforms, like Clearwater Forest.

$ grep . /sys/devices/cpu/format/*
/sys/devices/cpu/format/acr_mask:config2:0-63
/sys/devices/cpu/format/any:config:21
/sys/devices/cpu/format/cmask:config:24-31

So remove the anythread_deprecated bit from perf_capabilities structure and directly depends on CPUID.0AH:EDX[15] to judge if anythread is deprecated.

Fixes cadbaa039b99 ("perf/x86/intel: Make anythread filter support conditional")
---
 arch/x86/events/intel/core.c | 10 +++-------
 arch/x86/events/perf_event.h |  2 +-
 2 files changed, 4 insertions(+), 8 deletions(-)
arch/x86/events/intel/core.c +3 -7
index 0217e701aeeb..ea3ab3050a3b 100644
@@ -7946,12 +7946,6 @@ __init int intel_pmu_init(void)
 
 	x86_add_quirk(intel_arch_events_quirk); /* Install first, so it runs last */
 
-	if (version >= 5) {
-		x86_pmu.intel_cap.anythread_deprecated = edx.split.anythread_deprecated;
-		if (x86_pmu.intel_cap.anythread_deprecated)
-			pr_cont(" AnyThread deprecated, ");
-	}
-
 	/* The perf side of core PMU is ready to support the mediated vPMU. */
 	x86_get_pmu(smp_processor_id())->capabilities |= PERF_PMU_CAP_MEDIATED_VPMU;
 
@@ -8828,8 +8822,10 @@ __init int intel_pmu_init(void)
 				      &x86_pmu.intel_ctrl);
 
 	/* AnyThread may be deprecated on arch perfmon v5 or later */
-	if (x86_pmu.intel_cap.anythread_deprecated)
+	if (version >= 5 && edx.split.anythread_deprecated) {
 		x86_pmu.format_attrs = intel_arch_formats_attr;
+		pr_cont("AnyThread deprecated, ");
+	}
 
 	intel_pmu_check_event_constraints_all(NULL);
 
arch/x86/events/perf_event.h +1 -1
index eae24bb35dc1..5902a297daa1 100644
@@ -668,7 +668,7 @@ union perf_capabilities {
 		u64	perf_metrics:1;
 		u64	pebs_output_pt_available:1;
 		u64	pebs_timing_info:1;
-		u64	anythread_deprecated:1;
+		u64	__reserved:1;
 		u64	rdpmc_metrics_clear:1;
 	};
 	u64	capabilities;
Signature · 1 line
2.34.1
merge bot Applied
#2 lore raw

[tip: perf/core] perf/x86/intel: Remove anythread_deprecated bit from perf_capabilities

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     8767b4d73018bd3143f4c55b672064fad292f11b
Gitweb:        https://git.kernel.org/tip/8767b4d73018bd3143f4c55b672064fad292f11b
Author:        Dapeng Mi <[email protected]>
AuthorDate:    Tue, 16 Jun 2026 12:46:47 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 30 Jun 2026 11:46:06 +02:00

perf/x86/intel: Remove anythread_deprecated bit from perf_capabilities

AnyThread mode deprecation is enumerated by CPUID.0AH:EDX[15] instead of PERF_CAPABILITIES MSR. It's not a good practice to define a bit to represent "anythread deprecation" in perf_capabilities. It leads to the anythread_deprecated bit could be overwritten by the real value of PERF_CAPABILITIES MSR, just like the below code in update_pmu_cap() does.

if (!intel_pmu_broken_perf_cap()) {
	/* Perf Metric (Bit 15) and PEBS via PT (Bit 16) are hybrid enumeration */
	rdmsrq(MSR_IA32_PERF_CAPABILITIES, hybrid(pmu, intel_cap).capabilities);
}

It leads to the anythread_deprecated bit is cleared to 0 and the "any" attribute is incorrectly shown in the /sys/devices/cpu/format/ folder on these support Perfmon v6 platforms, like Clearwater Forest.

$ grep . /sys/devices/cpu/format/*
/sys/devices/cpu/format/acr_mask:config2:0-63
/sys/devices/cpu/format/any:config:21
/sys/devices/cpu/format/cmask:config:24-31

So remove the anythread_deprecated bit from perf_capabilities structure and directly depends on CPUID.0AH:EDX[15] to judge if anythread is deprecated.

---
 arch/x86/events/intel/core.c | 10 +++-------
 arch/x86/events/perf_event.h |  2 +-
 2 files changed, 4 insertions(+), 8 deletions(-)
arch/x86/events/intel/core.c +3 -7
index 2b35483..465c414 100644
@@ -7947,12 +7947,6 @@ __init int intel_pmu_init(void)
 
 	x86_add_quirk(intel_arch_events_quirk); /* Install first, so it runs last */
 
-	if (version >= 5) {
-		x86_pmu.intel_cap.anythread_deprecated = edx.split.anythread_deprecated;
-		if (x86_pmu.intel_cap.anythread_deprecated)
-			pr_cont(" AnyThread deprecated, ");
-	}
-
 	/* The perf side of core PMU is ready to support the mediated vPMU. */
 	x86_get_pmu(smp_processor_id())->capabilities |= PERF_PMU_CAP_MEDIATED_VPMU;
 
@@ -8829,8 +8823,10 @@ __init int intel_pmu_init(void)
 				      &x86_pmu.intel_ctrl);
 
 	/* AnyThread may be deprecated on arch perfmon v5 or later */
-	if (x86_pmu.intel_cap.anythread_deprecated)
+	if (version >= 5 && edx.split.anythread_deprecated) {
 		x86_pmu.format_attrs = intel_arch_formats_attr;
+		pr_cont("AnyThread deprecated, ");
+	}
 
 	intel_pmu_check_event_constraints_all(NULL);
 
arch/x86/events/perf_event.h +1 -1
index eae24bb..5902a29 100644
@@ -668,7 +668,7 @@ union perf_capabilities {
 		u64	perf_metrics:1;
 		u64	pebs_output_pt_available:1;
 		u64	pebs_timing_info:1;
-		u64	anythread_deprecated:1;
+		u64	__reserved:1;
 		u64	rdpmc_metrics_clear:1;
 	};
 	u64	capabilities;
author
#3 lore raw

PATCH v4 2/8 perf/x86/intel: Keep cap_user_rdpmc in sync with RDPMC user-disable state

In reply to Dapeng Mi

After introducing the RDPMC user disable feature, user-space RDPMC may return 0 instead of the actual event count. This creates an inconsistency with cap_user_rdpmc, where cap_user_rdpmc is set, but user-space RDPMC only returns 0.

To accurately represent the user-space RDPMC capability, update cap_user_rdpmc (depending on PERF_EVENT_FLAG_USER_READ_CNT) according to the RDPMC user disable state. If RDPMC user disable is enabled, cap_user_rdpmc is updated to false eventually, allowing user-space programs to fall back to the read() syscall to obtain the real event count.

Because PERF_EVENT_FLAG_USER_READ_CNT is evaluated in x86_pmu_event_init(), move intel_pmu_update_rdpmc_user_disable() earlier into intel_pmu_hw_config(). This ensures that the user-disable state is updated before updating PERF_EVENT_FLAG_USER_READ_CNT. Note that since event->ctx is not yet assigned at this stage, use the PERF_ATTACH_TASK flag to detect whether the event is task-attached.

While at it, fix the indentation of x86_pmu_has_rdpmc_user_disable() to adhere to the kernel coding style.

Fixes 59af95e028d4 ("perf/x86/intel: Add support for rdpmc user disable feature")
---
 arch/x86/events/core.c       | 3 ++-
 arch/x86/events/intel/core.c | 6 +++---
 arch/x86/events/perf_event.h | 2 +-
 3 files changed, 6 insertions(+), 5 deletions(-)
arch/x86/events/core.c +2 -1
index 4b9e105309c6..a76a2afae2fb 100644
@@ -2539,7 +2539,8 @@ static int x86_pmu_event_init(struct perf_event *event)
 	}
 
 	if (READ_ONCE(x86_pmu.attr_rdpmc) &&
-	    !(event->hw.flags & PERF_X86_EVENT_LARGE_PEBS))
+	    !(event->hw.flags & PERF_X86_EVENT_LARGE_PEBS) &&
+	    !(event->hw.config & ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE))
 		event->hw.flags |= PERF_EVENT_FLAG_USER_READ_CNT;
 
 	return err;
arch/x86/events/intel/core.c +3 -3
index ea3ab3050a3b..db52e7e53a6c 100644
@@ -3532,7 +3532,7 @@ static void intel_pmu_update_rdpmc_user_disable(struct perf_event *event)
 	 */
 	if (x86_pmu.attr_rdpmc == X86_USER_RDPMC_ALWAYS_ENABLE ||
 	    (x86_pmu.attr_rdpmc == X86_USER_RDPMC_CONDITIONAL_ENABLE &&
-	     event->ctx->task))
+	     (event->attach_state & PERF_ATTACH_TASK)))
 		event->hw.config &= ~ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE;
 	else
 		event->hw.config |= ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE;
@@ -3546,8 +3546,6 @@ static void intel_pmu_enable_event(struct perf_event *event)
 	struct hw_perf_event *hwc = &event->hw;
 	int idx = hwc->idx;
 
-	intel_pmu_update_rdpmc_user_disable(event);
-
 	if (unlikely(event->attr.precise_ip))
 		static_call(x86_pmu_pebs_enable)(event);
 
@@ -5146,6 +5144,8 @@ static int intel_pmu_hw_config(struct perf_event *event)
 		leader->hw.flags |= PERF_X86_EVENT_ACR;
 	}
 
+	intel_pmu_update_rdpmc_user_disable(event);
+
 	if ((event->attr.type == PERF_TYPE_HARDWARE) ||
 	    (event->attr.type == PERF_TYPE_HW_CACHE))
 		return 0;
arch/x86/events/perf_event.h +1 -1
index 5902a297daa1..a8afea8d38f0 100644
@@ -1344,7 +1344,7 @@ static inline u64 x86_pmu_get_event_config(struct perf_event *event)
 static inline bool x86_pmu_has_rdpmc_user_disable(struct pmu *pmu)
 {
 	return !!(hybrid(pmu, config_mask) &
-		 ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE);
+		  ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE);
 }
 
 extern struct event_constraint emptyconstraint;
Signature · 1 line
2.34.1
merge bot Applied
#4 lore raw

[tip: perf/core] perf/x86/intel: Keep cap_user_rdpmc in sync with RDPMC user-disable state

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     3c4ec9b2a5db56b60127bfaf933ecdeea7a1f10a
Gitweb:        https://git.kernel.org/tip/3c4ec9b2a5db56b60127bfaf933ecdeea7a1f10a
Author:        Dapeng Mi <[email protected]>
AuthorDate:    Tue, 16 Jun 2026 12:46:48 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 30 Jun 2026 11:46:06 +02:00

perf/x86/intel: Keep cap_user_rdpmc in sync with RDPMC user-disable state

After introducing the RDPMC user disable feature, user-space RDPMC may return 0 instead of the actual event count. This creates an inconsistency with cap_user_rdpmc, where cap_user_rdpmc is set, but user-space RDPMC only returns 0.

To accurately represent the user-space RDPMC capability, update cap_user_rdpmc (depending on PERF_EVENT_FLAG_USER_READ_CNT) according to the RDPMC user disable state. If RDPMC user disable is enabled, cap_user_rdpmc is updated to false eventually, allowing user-space programs to fall back to the read() syscall to obtain the real event count.

Because PERF_EVENT_FLAG_USER_READ_CNT is evaluated in x86_pmu_event_init(), move intel_pmu_update_rdpmc_user_disable() earlier into intel_pmu_hw_config(). This ensures that the user-disable state is updated before updating PERF_EVENT_FLAG_USER_READ_CNT. Note that since event->ctx is not yet assigned at this stage, use the PERF_ATTACH_TASK flag to detect whether the event is task-attached.

While at it, fix the indentation of x86_pmu_has_rdpmc_user_disable() to adhere to the kernel coding style.

Fixes 59af95e028d4 ("perf/x86/intel: Add support for rdpmc user disable feature")
---
 arch/x86/events/core.c       | 3 ++-
 arch/x86/events/intel/core.c | 6 +++---
 arch/x86/events/perf_event.h | 2 +-
 3 files changed, 6 insertions(+), 5 deletions(-)
arch/x86/events/core.c +2 -1
index d1af33d..af0b67f 100644
@@ -2539,7 +2539,8 @@ static int x86_pmu_event_init(struct perf_event *event)
 	}
 
 	if (READ_ONCE(x86_pmu.attr_rdpmc) &&
-	    !(event->hw.flags & PERF_X86_EVENT_LARGE_PEBS))
+	    !(event->hw.flags & PERF_X86_EVENT_LARGE_PEBS) &&
+	    !(event->hw.config & ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE))
 		event->hw.flags |= PERF_EVENT_FLAG_USER_READ_CNT;
 
 	return err;
arch/x86/events/intel/core.c +3 -3
index 465c414..5116b15 100644
@@ -3533,7 +3533,7 @@ static void intel_pmu_update_rdpmc_user_disable(struct perf_event *event)
 	 */
 	if (x86_pmu.attr_rdpmc == X86_USER_RDPMC_ALWAYS_ENABLE ||
 	    (x86_pmu.attr_rdpmc == X86_USER_RDPMC_CONDITIONAL_ENABLE &&
-	     event->ctx->task))
+	     (event->attach_state & PERF_ATTACH_TASK)))
 		event->hw.config &= ~ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE;
 	else
 		event->hw.config |= ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE;
@@ -3547,8 +3547,6 @@ static void intel_pmu_enable_event(struct perf_event *event)
 	struct hw_perf_event *hwc = &event->hw;
 	int idx = hwc->idx;
 
-	intel_pmu_update_rdpmc_user_disable(event);
-
 	if (unlikely(event->attr.precise_ip))
 		static_call(x86_pmu_pebs_enable)(event);
 
@@ -5147,6 +5145,8 @@ static int intel_pmu_hw_config(struct perf_event *event)
 		leader->hw.flags |= PERF_X86_EVENT_ACR;
 	}
 
+	intel_pmu_update_rdpmc_user_disable(event);
+
 	if ((event->attr.type == PERF_TYPE_HARDWARE) ||
 	    (event->attr.type == PERF_TYPE_HW_CACHE))
 		return 0;
arch/x86/events/perf_event.h +1 -1
index 5902a29..a8afea8 100644
@@ -1344,7 +1344,7 @@ static inline u64 x86_pmu_get_event_config(struct perf_event *event)
 static inline bool x86_pmu_has_rdpmc_user_disable(struct pmu *pmu)
 {
 	return !!(hybrid(pmu, config_mask) &
-		 ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE);
+		  ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE);
 }
 
 extern struct event_constraint emptyconstraint;
author
#5 lore raw

PATCH v4 3/8 perf/x86/intel: Fallback to sw branch type decoding if no hw decoding

In reply to Dapeng Mi

intel_pmu_lbr_filter() currently assumes arch-LBR provides hardware branch-type decoding and skips software decoding on that path.

However, arch-LBR may not always expose branch-type information. In that case, treating entries as hardware-decoded can misclassify sampled branches (for example, defaulting to JCC), which breaks branch-type filtering results.

Fix this by using software branch-type decoding when hardware branch-type decoding is unavailable (that is, when x86_lbr_type is not enabled). This keeps branch classification and filtering behavior correct across arch-LBR configurations.

Since x86_lbr_type is only set to true when arch-LBR is supported, it's unnecessary to check if the CPU has the arch-LBR feature.

---
 arch/x86/events/intel/lbr.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/lbr.c +1 -1
index 72f2adcda7c6..e2657f791e50 100644
@@ -1231,7 +1231,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
 		 * Doesn't support OTHER_BRANCH decoding for now.
 		 * OTHER_BRANCH branch type still rely on software decoding.
 		 */
-		if (static_cpu_has(X86_FEATURE_ARCH_LBR) &&
+		if (static_branch_likely(&x86_lbr_type) &&
 		    type <= ARCH_LBR_BR_TYPE_KNOWN_MAX) {
 			to_plm = kernel_ip(to) ? X86_BR_KERNEL : X86_BR_USER;
 			type = arch_lbr_br_type_map[type] | to_plm;
Signature · 1 line
2.34.1
#6 lore raw

Re: [Patch v4 3/8] perf/x86/intel: Fallback to sw branch type decoding if no hw decoding

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Forcing `X86_BR_TYPE_SAVE` on Arch LBR CPUs without hardware branch-type support causes massive…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Forcing X86_BR_TYPE_SAVE on Arch LBR CPUs without hardware branch-type support causes massive NMI overhead due to an unconditional fallback to software decoding. --

commit f25e57aa8ef5b59fd749c6fcb24f34aa7668326f
Author: Dapeng Mi <[email protected]>

perf/x86/intel: Fallback to sw branch type decoding if no hw decoding

This commit updates the arch-LBR filtering to fall back to software branch-type decoding when the hardware feature is unavailable. This ensures branch classification works correctly across different arch-LBR configurations.

arch/x86/events/intel/lbr.c intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) · line 1234

Quoted 10 lines
arch/x86/events/intel/lbr.c +1 -1
index 72f2adcda7c68..e2657f791e507 100644
@@ -1231,7 +1231,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
 		 * Doesn't support OTHER_BRANCH decoding for now.
 		 * OTHER_BRANCH branch type still rely on software decoding.
 		 */
-		if (static_cpu_has(X86_FEATURE_ARCH_LBR) &&
+		if (static_branch_likely(&x86_lbr_type) &&

[Severity: High]
Does this change unintentionally cause a massive performance regression on systems without hardware branch-type support?

In intel_pmu_setup_hw_lbr_filter(), X86_BR_TYPE_SAVE is always forced for all Arch LBR CPUs:

arch/x86/events/intel/lbr.c:intel_pmu_setup_hw_lbr_filter() {
    ...
    if (static_cpu_has(X86_FEATURE_ARCH_LBR)) {
        reg->config = mask;
        ...
        reg->reg |= X86_BR_TYPE_SAVE;
        return 0;
    }
}

Because X86_BR_TYPE_SAVE is forced, the fast-path early exit in intel_pmu_lbr_filter() is never taken:

arch/x86/events/intel/lbr.c:intel_pmu_lbr_filter() {
    ...
    if (((br_sel & X86_BR_ALL) == X86_BR_ALL) &&
        ((br_sel & X86_BR_TYPE_SAVE) != X86_BR_TYPE_SAVE))
        return;
    ...
}

With the fallback introduced by this patch, won't every sampled branch be forced to undergo high-overhead software disassembly in the NMI context, even when the user requested no specific branch type filtering?

Should intel_pmu_setup_hw_lbr_filter() be updated to conditionally force X86_BR_TYPE_SAVE only if x86_lbr_type is actually enabled?

Quoted 3 lines
arch/x86/events/intel/lbr.c +0 -0
 		    type <= ARCH_LBR_BR_TYPE_KNOWN_MAX) {
 			to_plm = kernel_ip(to) ? X86_BR_KERNEL : X86_BR_USER;
 			type = arch_lbr_br_type_map[type] | to_plm;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
merge bot Applied
#7 lore raw

[tip: perf/core] perf/x86/intel: Fallback to sw branch type decoding if no hw decoding

In reply to Dapeng Mi
The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     170cc6b02e3d5203ccaa49ffea1ee5a7ab08c885
Gitweb:        https://git.kernel.org/tip/170cc6b02e3d5203ccaa49ffea1ee5a7ab08c885
Author:        Dapeng Mi <[email protected]>
AuthorDate:    Tue, 16 Jun 2026 12:46:49 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 30 Jun 2026 11:46:06 +02:00

perf/x86/intel: Fallback to sw branch type decoding if no hw decoding

intel_pmu_lbr_filter() currently assumes arch-LBR provides hardware branch-type decoding and skips software decoding on that path.

However, arch-LBR may not always expose branch-type information. In that case, treating entries as hardware-decoded can misclassify sampled branches (for example, defaulting to JCC), which breaks branch-type filtering results.

Fix this by using software branch-type decoding when hardware branch-type decoding is unavailable (that is, when x86_lbr_type is not enabled). This keeps branch classification and filtering behavior correct across arch-LBR configurations.

Since x86_lbr_type is only set to true when arch-LBR is supported, it's unnecessary to check if the CPU has the arch-LBR feature.

---
 arch/x86/events/intel/lbr.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/lbr.c +1 -1
index cae2e02..688d1df 100644
@@ -1232,7 +1232,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
 		 * Doesn't support OTHER_BRANCH decoding for now.
 		 * OTHER_BRANCH branch type still rely on software decoding.
 		 */
-		if (static_cpu_has(X86_FEATURE_ARCH_LBR) &&
+		if (static_branch_likely(&x86_lbr_type) &&
 		    type <= ARCH_LBR_BR_TYPE_KNOWN_MAX) {
 			to_plm = kernel_ip(to) ? X86_BR_KERNEL : X86_BR_USER;
 			type = arch_lbr_br_type_map[type] | to_plm;
author
#8 lore raw

PATCH v4 4/8 perf/x86/intel: Fix kernel address leakages in LBR stack

In reply to Dapeng Mi

Before Arch LBR gained CPL filtering support, a user-only branch stack could still contain kernel addresses. As a result, kernel branch records may be exposed to user space even when PERF_SAMPLE_BRANCH_USER is requested.

For example, on Intel Tiger Lake, the following command can still report SYSRET/ERET entries with kernel-space from addresses:

$ ./perf record -e cycles:p -o - --branch-filter any,save_type,u -- \
 	./perf bench syscall basic --loop 1000 | \
	./perf script -i - --fields brstack|tr ' ' '\n'| \
	grep -E '0x[89a-f][0-9a-f]{15}'
    Total time: 0.000 [sec]

      0.219000 usecs/op
     4,566,210 ops/sec

[ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.551 MB - ] 0xffffffff93c001c8/0x7f12a2b1d647/P/-/-/16959/SYSRET/- 0xffffffff93c001c8/0x7f12a2b1d5c2/P/-/-/17535/SYSRET/- 0xffffffff93c01928/0x7f12a2861000/P/-/-/6719/ERET/- 0xffffffff93c01928/0x7f12a297a000/P/-/-/8575/ERET/-

The problem is that intel_pmu_lbr_filter() does not fully validate the privilege level of sampled entries. It filters some mismatches based on the branch type and the to address, but it does not reject entries whose from address violates the requested branch privilege filter.

Fix this by extending software filtering to validate both from and to addresses against br_sel. Any LBR entry contains kernel address does not match the requested user filter is dropped. This prevents kernel addresses from appearing in user-only branch stacks.

Fixes 47125db27e47 ("perf/x86/intel/lbr: Support Architectural LBR")
---
 arch/x86/events/intel/lbr.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
arch/x86/events/intel/lbr.c +9 -3
index e2657f791e50..16eef1537816 100644
@@ -1212,7 +1212,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
 {
 	u64 from, to;
 	int br_sel = cpuc->br_sel;
-	int i, j, type, to_plm;
+	int i, j, type, from_plm, to_plm;
 	bool compress = false;
 
 	/* if sampling all branches, then nothing to filter */
@@ -1244,8 +1244,14 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
 				type |= X86_BR_NO_TX;
 		}
 
-		/* if type does not correspond, then discard */
-		if (type == X86_BR_NONE || (br_sel & type) != type) {
+		from_plm = kernel_ip(from) ? X86_BR_KERNEL : X86_BR_USER;
+		/*
+		 * If type does not correspond, then discard.
+		 * Specifically reject entries whose from address is in
+		 * kernel space when only X86_BR_USER is requested.
+		 */
+		if (type == X86_BR_NONE || (br_sel & type) != type ||
+		    (!(br_sel & X86_BR_KERNEL) && (from_plm & X86_BR_KERNEL))) {
 			cpuc->lbr_entries[i].from = 0;
 			compress = true;
 		}
Signature · 1 line
2.34.1
#9 lore raw

Re: [Patch v4 4/8] perf/x86/intel: Fix kernel address leakages in LBR stack

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The kernel address leak fixed in Intel LBR is also present in the AMD LBR and AMD BRS implementations. --

commit 6e58211ef4792cdee63b678a385b927eb7214af9
Author: Dapeng Mi <[email protected]>

perf/x86/intel: Fix kernel address leakages in LBR stack

This commit fixes an information leak in Intel LBR where the software filter failed to validate the from address. By validating both the from and to addresses against the requested privilege mask, it prevents kernel addresses from leaking into user-only branch stacks.

Quoted 4 lines
arch/x86/events/intel/lbr.c +0 -0
index e2657f791e507..16eef15378166 100644

[ ... ]

Quoted 4 lines
arch/x86/events/intel/lbr.c +0 -0
@@ -1244,8 +1244,14 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
 				type |= X86_BR_NO_TX;
 		}
 

arch/x86/events/intel/lbr.c intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) · line 1254

Quoted 10 lines
arch/x86/events/intel/lbr.c +8 -2
-		/* if type does not correspond, then discard */
-		if (type == X86_BR_NONE || (br_sel & type) != type) {
+		from_plm = kernel_ip(from) ? X86_BR_KERNEL : X86_BR_USER;
+		/*
+		 * If type does not correspond, then discard.
+		 * Specifically reject entries whose from address is in
+		 * kernel space when only X86_BR_USER is requested.
+		 */
+		if (type == X86_BR_NONE || (br_sel & type) != type ||
+		    (!(br_sel & X86_BR_KERNEL) && (from_plm & X86_BR_KERNEL))) {

[Severity: High]
This isn't a bug introduced by this patch, but does the same kernel address leak exist in the AMD LBR and BRS implementations?

Looking at arch/x86/events/amd/lbr.c, the software filter sets the branch privilege strictly based on the target address. Kernel-to-user branches might bypass the filter and leak the kernel from address:

arch/x86/events/amd/lbr.c:amd_pmu_lbr_filter() {
    ...
    type = branch_type_fused(from, to, 0, &offset);

    /* If type does not correspond, then discard */
    if (type == X86_BR_NONE || (br_sel & type) != type) {
    ...
}

Similarly, in arch/x86/events/amd/brs.c, the privilege filter only evaluates the target address against the requested privilege mask:

arch/x86/events/amd/brs.c:amd_brs_match_plm() {
    ...
    if (!(type & plm_k) && kernel_ip(to))
        return 0;

    if (!(type & plm_u) && !kernel_ip(to))
        return 0;

    return 1;
}

Because the hardware PMU natively records branches going from kernel space to user space, could unprivileged users requesting user-only branches still sample kernel addresses on AMD through these paths?

Quoted 3 lines
arch/x86/events/intel/lbr.c +0 -0
 			cpuc->lbr_entries[i].from = 0;
 			compress = true;
 		}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
merge bot Applied
#10 lore raw

[tip: perf/core] perf/x86/intel: Fix kernel address leakages in LBR stack

In reply to Dapeng Mi
The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     e2b0575900ff72aa82748af96e7bd564ade5157a
Gitweb:        https://git.kernel.org/tip/e2b0575900ff72aa82748af96e7bd564ade5157a
Author:        Dapeng Mi <[email protected]>
AuthorDate:    Tue, 16 Jun 2026 12:46:50 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 30 Jun 2026 11:46:06 +02:00

perf/x86/intel: Fix kernel address leakages in LBR stack

Before Arch LBR gained CPL filtering support, a user-only branch stack could still contain kernel addresses. As a result, kernel branch records may be exposed to user space even when PERF_SAMPLE_BRANCH_USER is requested.

For example, on Intel Tiger Lake, the following command can still report SYSRET/ERET entries with kernel-space from addresses:

$ ./perf record -e cycles:p -o - --branch-filter any,save_type,u -- \
 	./perf bench syscall basic --loop 1000 | \
	./perf script -i - --fields brstack|tr ' ' '\n'| \
	grep -E '0x[89a-f][0-9a-f]{15}'
    Total time: 0.000 [sec]

      0.219000 usecs/op
     4,566,210 ops/sec

[ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.551 MB - ] 0xffffffff93c001c8/0x7f12a2b1d647/P/-/-/16959/SYSRET/- 0xffffffff93c001c8/0x7f12a2b1d5c2/P/-/-/17535/SYSRET/- 0xffffffff93c01928/0x7f12a2861000/P/-/-/6719/ERET/- 0xffffffff93c01928/0x7f12a297a000/P/-/-/8575/ERET/-

The problem is that intel_pmu_lbr_filter() does not fully validate the privilege level of sampled entries. It filters some mismatches based on the branch type and the to address, but it does not reject entries whose from address violates the requested branch privilege filter.

Fix this by extending software filtering to validate both from and to addresses against br_sel. Any LBR entry contains kernel address does not match the requested user filter is dropped. This prevents kernel addresses from appearing in user-only branch stacks.

Fixes 47125db27e47 ("perf/x86/intel/lbr: Support Architectural LBR")
---
 arch/x86/events/intel/lbr.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
arch/x86/events/intel/lbr.c +9 -3
index 688d1df..f8fadb0 100644
@@ -1213,7 +1213,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
 {
 	u64 from, to;
 	int br_sel = cpuc->br_sel;
-	int i, j, type, to_plm;
+	int i, j, type, from_plm, to_plm;
 	bool compress = false;
 
 	/* if sampling all branches, then nothing to filter */
@@ -1245,8 +1245,14 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
 				type |= X86_BR_NO_TX;
 		}
 
-		/* if type does not correspond, then discard */
-		if (type == X86_BR_NONE || (br_sel & type) != type) {
+		from_plm = kernel_ip(from) ? X86_BR_KERNEL : X86_BR_USER;
+		/*
+		 * If type does not correspond, then discard.
+		 * Specifically reject entries whose from address is in
+		 * kernel space when only X86_BR_USER is requested.
+		 */
+		if (type == X86_BR_NONE || (br_sel & type) != type ||
+		    (!(br_sel & X86_BR_KERNEL) && (from_plm & X86_BR_KERNEL))) {
 			cpuc->lbr_entries[i].from = 0;
 			compress = true;
 		}
author
#11 lore raw

PATCH v4 5/8 perf/x86/intel: Validate the return value of intel_pmu_init_hybrid()

In reply to Dapeng Mi

The intel_pmu_init_hybrid() function allocates memory for the x86_pmu.hybrid_pmu[] array. If this allocation fails under memory pressure, hybrid PMU initialization will fail.

Currently, the caller does not check the return value of intel_pmu_init_hybrid(). This can lead to a null-pointer dereference or invalid memory access when attempting to use the uninitialized array, potentially triggering a system panic.

Fix this by validating the return value of intel_pmu_init_hybrid(). Additionally, reset x86_pmu.num_hybrid_pmus to 0 on failure, and defer intel_pmu_arch_lbr_init() until after hybrid PMU initialization succeeds. This reordering avoids the need to explicitly destroy the kmem cache if the memory allocation fails.

---
 arch/x86/events/intel/core.c | 44 ++++++++++++++++++++++++------------
 1 file changed, 30 insertions(+), 14 deletions(-)
arch/x86/events/intel/core.c +30 -14
index db52e7e53a6c..edf6f8732234 100644
@@ -7678,8 +7678,10 @@ static __always_inline int intel_pmu_init_hybrid(enum hybrid_pmu_type pmus)
 	x86_pmu.num_hybrid_pmus = hweight_long(pmus_mask);
 	x86_pmu.hybrid_pmu = kzalloc_objs(struct x86_hybrid_pmu,
 					  x86_pmu.num_hybrid_pmus);
-	if (!x86_pmu.hybrid_pmu)
+	if (!x86_pmu.hybrid_pmu) {
+		x86_pmu.num_hybrid_pmus = 0;
 		return -ENOMEM;
+	}
 
 	static_branch_enable(&perf_is_hybrid);
 	x86_pmu.filter = intel_pmu_filter;
@@ -7862,14 +7864,14 @@ __init int intel_pmu_init(void)
 	struct attribute **td_attr    = &empty_attrs;
 	struct attribute **mem_attr   = &empty_attrs;
 	struct attribute **tsx_attr   = &empty_attrs;
+	struct x86_hybrid_pmu *pmu;
+	unsigned int fixed_mask;
 	union cpuid10_edx edx;
 	union cpuid10_eax eax;
 	union cpuid10_ebx ebx;
-	unsigned int fixed_mask;
+	int version, i, ret;
 	bool pmem = false;
-	int version, i;
 	char *name;
-	struct x86_hybrid_pmu *pmu;
 
 	/* Architectural Perfmon was introduced starting with Core "Yonah" */
 	if (!cpu_has(&boot_cpu_data, X86_FEATURE_ARCH_PERFMON)) {
@@ -7939,9 +7941,6 @@ __init int intel_pmu_init(void)
 		x86_pmu.lbr_read = intel_pmu_lbr_read_32;
 	}
 
-	if (boot_cpu_has(X86_FEATURE_ARCH_LBR))
-		intel_pmu_arch_lbr_init();
-
 	intel_pebs_init();
 
 	x86_add_quirk(intel_arch_events_quirk); /* Install first, so it runs last */
@@ -8539,7 +8538,9 @@ __init int intel_pmu_init(void)
 		 *
 		 * Initialize the common PerfMon capabilities here.
 		 */
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		x86_pmu.pebs_latency_data = grt_latency_data;
 		x86_pmu.get_event_constraints = adl_get_event_constraints;
@@ -8597,7 +8598,9 @@ __init int intel_pmu_init(void)
 	case INTEL_METEORLAKE:
 	case INTEL_METEORLAKE_L:
 	case INTEL_ARROWLAKE_U:
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		x86_pmu.pebs_latency_data = cmt_latency_data;
 		x86_pmu.get_event_constraints = mtl_get_event_constraints;
@@ -8628,7 +8631,9 @@ __init int intel_pmu_init(void)
 		pr_cont("Pantherlake Hybrid events, ");
 		name = "pantherlake_hybrid";
 
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		/* Initialize big core specific PerfMon capabilities.*/
 		pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_CORE_IDX];
@@ -8643,7 +8648,9 @@ __init int intel_pmu_init(void)
 		pr_cont("Arrowlake Hybrid events, ");
 		name = "arrowlake_hybrid";
 
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		/* Initialize big core specific PerfMon capabilities.*/
 		pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_CORE_IDX];
@@ -8660,7 +8667,9 @@ __init int intel_pmu_init(void)
 		pr_cont("Lunarlake Hybrid events, ");
 		name = "lunarlake_hybrid";
 
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		/* Initialize big core specific PerfMon capabilities.*/
 		pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_CORE_IDX];
@@ -8685,7 +8694,9 @@ __init int intel_pmu_init(void)
 		break;
 
 	case INTEL_ARROWLAKE_H:
-		intel_pmu_init_hybrid(hybrid_big_small_tiny);
+		ret = intel_pmu_init_hybrid(hybrid_big_small_tiny);
+		if (ret)
+			return ret;
 
 		x86_pmu.pebs_latency_data = arl_h_latency_data;
 		x86_pmu.get_event_constraints = arl_h_get_event_constraints;
@@ -8720,7 +8731,9 @@ __init int intel_pmu_init(void)
 	case INTEL_NOVALAKE_L:
 		pr_cont("Novalake Hybrid events, ");
 		name = "novalake_hybrid";
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		x86_pmu.pebs_latency_data = nvl_latency_data;
 		x86_pmu.get_event_constraints = mtl_get_event_constraints;
@@ -8827,6 +8840,9 @@ __init int intel_pmu_init(void)
 		pr_cont("AnyThread deprecated, ");
 	}
 
+	if (boot_cpu_has(X86_FEATURE_ARCH_LBR))
+		intel_pmu_arch_lbr_init();
+
 	intel_pmu_check_event_constraints_all(NULL);
 
 	/*
Signature · 1 line
2.34.1
merge bot Applied
#12 lore raw

[tip: perf/core] perf/x86/intel: Validate the return value of intel_pmu_init_hybrid()

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     01c153956b4436ead05a529dae56abc0ef58beac
Gitweb:        https://git.kernel.org/tip/01c153956b4436ead05a529dae56abc0ef58beac
Author:        Dapeng Mi <[email protected]>
AuthorDate:    Tue, 16 Jun 2026 12:46:51 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 30 Jun 2026 11:46:07 +02:00

perf/x86/intel: Validate the return value of intel_pmu_init_hybrid()

The intel_pmu_init_hybrid() function allocates memory for the x86_pmu.hybrid_pmu[] array. If this allocation fails under memory pressure, hybrid PMU initialization will fail.

Currently, the caller does not check the return value of intel_pmu_init_hybrid(). This can lead to a null-pointer dereference or invalid memory access when attempting to use the uninitialized array, potentially triggering a system panic.

Fix this by validating the return value of intel_pmu_init_hybrid(). Additionally, reset x86_pmu.num_hybrid_pmus to 0 on failure, and defer intel_pmu_arch_lbr_init() until after hybrid PMU initialization succeeds. This reordering avoids the need to explicitly destroy the kmem cache if the memory allocation fails.

---
 arch/x86/events/intel/core.c | 44 +++++++++++++++++++++++------------
 1 file changed, 30 insertions(+), 14 deletions(-)
arch/x86/events/intel/core.c +30 -14
index 5116b15..b39c6ce 100644
@@ -7679,8 +7679,10 @@ static __always_inline int intel_pmu_init_hybrid(enum hybrid_pmu_type pmus)
 	x86_pmu.num_hybrid_pmus = hweight_long(pmus_mask);
 	x86_pmu.hybrid_pmu = kzalloc_objs(struct x86_hybrid_pmu,
 					  x86_pmu.num_hybrid_pmus);
-	if (!x86_pmu.hybrid_pmu)
+	if (!x86_pmu.hybrid_pmu) {
+		x86_pmu.num_hybrid_pmus = 0;
 		return -ENOMEM;
+	}
 
 	static_branch_enable(&perf_is_hybrid);
 	x86_pmu.filter = intel_pmu_filter;
@@ -7863,14 +7865,14 @@ __init int intel_pmu_init(void)
 	struct attribute **td_attr    = &empty_attrs;
 	struct attribute **mem_attr   = &empty_attrs;
 	struct attribute **tsx_attr   = &empty_attrs;
+	struct x86_hybrid_pmu *pmu;
+	unsigned int fixed_mask;
 	union cpuid10_edx edx;
 	union cpuid10_eax eax;
 	union cpuid10_ebx ebx;
-	unsigned int fixed_mask;
+	int version, i, ret;
 	bool pmem = false;
-	int version, i;
 	char *name;
-	struct x86_hybrid_pmu *pmu;
 
 	/* Architectural Perfmon was introduced starting with Core "Yonah" */
 	if (!cpu_has(&boot_cpu_data, X86_FEATURE_ARCH_PERFMON)) {
@@ -7940,9 +7942,6 @@ __init int intel_pmu_init(void)
 		x86_pmu.lbr_read = intel_pmu_lbr_read_32;
 	}
 
-	if (boot_cpu_has(X86_FEATURE_ARCH_LBR))
-		intel_pmu_arch_lbr_init();
-
 	intel_pebs_init();
 
 	x86_add_quirk(intel_arch_events_quirk); /* Install first, so it runs last */
@@ -8540,7 +8539,9 @@ __init int intel_pmu_init(void)
 		 *
 		 * Initialize the common PerfMon capabilities here.
 		 */
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		x86_pmu.pebs_latency_data = grt_latency_data;
 		x86_pmu.get_event_constraints = adl_get_event_constraints;
@@ -8598,7 +8599,9 @@ __init int intel_pmu_init(void)
 	case INTEL_METEORLAKE:
 	case INTEL_METEORLAKE_L:
 	case INTEL_ARROWLAKE_U:
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		x86_pmu.pebs_latency_data = cmt_latency_data;
 		x86_pmu.get_event_constraints = mtl_get_event_constraints;
@@ -8629,7 +8632,9 @@ __init int intel_pmu_init(void)
 		pr_cont("Pantherlake Hybrid events, ");
 		name = "pantherlake_hybrid";
 
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		/* Initialize big core specific PerfMon capabilities.*/
 		pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_CORE_IDX];
@@ -8644,7 +8649,9 @@ __init int intel_pmu_init(void)
 		pr_cont("Arrowlake Hybrid events, ");
 		name = "arrowlake_hybrid";
 
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		/* Initialize big core specific PerfMon capabilities.*/
 		pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_CORE_IDX];
@@ -8661,7 +8668,9 @@ __init int intel_pmu_init(void)
 		pr_cont("Lunarlake Hybrid events, ");
 		name = "lunarlake_hybrid";
 
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		/* Initialize big core specific PerfMon capabilities.*/
 		pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_CORE_IDX];
@@ -8686,7 +8695,9 @@ __init int intel_pmu_init(void)
 		break;
 
 	case INTEL_ARROWLAKE_H:
-		intel_pmu_init_hybrid(hybrid_big_small_tiny);
+		ret = intel_pmu_init_hybrid(hybrid_big_small_tiny);
+		if (ret)
+			return ret;
 
 		x86_pmu.pebs_latency_data = arl_h_latency_data;
 		x86_pmu.get_event_constraints = arl_h_get_event_constraints;
@@ -8721,7 +8732,9 @@ __init int intel_pmu_init(void)
 	case INTEL_NOVALAKE_L:
 		pr_cont("Novalake Hybrid events, ");
 		name = "novalake_hybrid";
-		intel_pmu_init_hybrid(hybrid_big_small);
+		ret = intel_pmu_init_hybrid(hybrid_big_small);
+		if (ret)
+			return ret;
 
 		x86_pmu.pebs_latency_data = nvl_latency_data;
 		x86_pmu.get_event_constraints = mtl_get_event_constraints;
@@ -8828,6 +8841,9 @@ __init int intel_pmu_init(void)
 		pr_cont("AnyThread deprecated, ");
 	}
 
+	if (boot_cpu_has(X86_FEATURE_ARCH_LBR))
+		intel_pmu_arch_lbr_init();
+
 	intel_pmu_check_event_constraints_all(NULL);
 
 	/*
author
#13 lore raw

PATCH v4 6/8 perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS

In reply to Dapeng Mi

On Sapphire Rapids (SPR) guests where pebs_baseline is not advertised, running command

$ ./perf record -e cpu/event=0x00,umask=0x01,i\
	 name=INST_RETIRED.PREC_DIST/p -c 10000 sleep 1

can trigger:

unchecked MSR access error: WRMSR to 0x3f1 ... in\
intel_pmu_pebs_enable_all()

This occurs because SPR-specific PEBS constraints allow fixed-counter scheduling (for example, INST_RETIRED.PREC_DIST on fixed counter 0). In guests lacking pebs_baseline, KVM does not support PEBS sampling on fixed counters, so enabling such events reaches an invalid MSR programming path.

Starting with Icelake, regardless of whether Extended PEBS or Architectural PEBS is in use, all counters (including fixed counters) support PEBS sampling, and the PMU_FL_PEBS_ALL flag is set by default. As long as PMU_FL_PEBS_ALL is set, constraint lookup automatically falls back to the corresponding non-PEBS constraints if no matching entry is found in the PEBS constraints table. Since non-PEBS event constraints already contain the same fixed-counter constraints, it is safe to remove these fixed-counter entries from the PEBS constraints table. The fallback mechanism will ensure that fixed PEBS events are scheduled onto the correct fixed counters.

So, directly drop the fixed-counter entries from the PEBS constraint table. Without pebs_baseline, these fixed-counter PEBS events will now resolve to empty constraints and will not be scheduled or enabled, thereby avoiding the warning and bypassing the broken guest PEBS path.

---
 arch/x86/events/intel/ds.c | 13 -------------
 1 file changed, 13 deletions(-)
arch/x86/events/intel/ds.c +0 -13
index cb72af9b61ce..5db15a92017a 100644
@@ -1447,10 +1447,6 @@ struct event_constraint intel_skl_pebs_event_constraints[] = {
 };
 
 struct event_constraint intel_icl_pebs_event_constraints[] = {
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x01c0, 0x100000000ULL),	/* old INST_RETIRED.PREC_DIST */
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x0100, 0x100000000ULL),	/* INST_RETIRED.PREC_DIST */
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x0400, 0x800000000ULL),	/* SLOTS */
-
 	INTEL_PLD_CONSTRAINT(0x1cd, 0xff),			/* MEM_TRANS_RETIRED.LOAD_LATENCY */
 	INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD(0x11d0, 0xf),	/* MEM_INST_RETIRED.STLB_MISS_LOADS */
 	INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST(0x12d0, 0xf),	/* MEM_INST_RETIRED.STLB_MISS_STORES */
@@ -1473,9 +1469,6 @@ struct event_constraint intel_icl_pebs_event_constraints[] = {
 };
 
 struct event_constraint intel_glc_pebs_event_constraints[] = {
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x100, 0x100000000ULL),	/* INST_RETIRED.PREC_DIST */
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x0400, 0x800000000ULL),
-
 	INTEL_FLAGS_EVENT_CONSTRAINT(0xc0, 0xfe),
 	INTEL_PLD_CONSTRAINT(0x1cd, 0xfe),
 	INTEL_PSD_CONSTRAINT(0x2cd, 0x1),
@@ -1500,9 +1493,6 @@ struct event_constraint intel_glc_pebs_event_constraints[] = {
 };
 
 struct event_constraint intel_lnc_pebs_event_constraints[] = {
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x100, 0x100000000ULL),	/* INST_RETIRED.PREC_DIST */
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x0400, 0x800000000ULL),
-
 	INTEL_FLAGS_UEVENT_CONSTRAINT(0x012a, 0x1),		/* OCR.* events */
 	INTEL_FLAGS_UEVENT_CONSTRAINT(0x012b, 0x1),		/* OCR.* events */
 
@@ -1534,9 +1524,6 @@ struct event_constraint intel_lnc_pebs_event_constraints[] = {
 };
 
 struct event_constraint intel_pnc_pebs_event_constraints[] = {
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x100, 0x100000000ULL),	/* INST_RETIRED.PREC_DIST */
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x0400, 0x800000000ULL),
-
 	INTEL_HYBRID_LDLAT_CONSTRAINT(0x1cd, 0xfc),
 	INTEL_HYBRID_STLAT_CONSTRAINT(0x2cd, 0x3),
 	INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD(0x11d0, 0xf),	/* MEM_INST_RETIRED.STLB_MISS_LOADS */
Signature · 1 line
2.34.1
merge bot Applied
#14 lore raw

[tip: perf/core] perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     a6b5fbc33172509fbe991358d718617a8e33ea7e
Gitweb:        https://git.kernel.org/tip/a6b5fbc33172509fbe991358d718617a8e33ea7e
Author:        Dapeng Mi <[email protected]>
AuthorDate:    Tue, 16 Jun 2026 12:46:52 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 30 Jun 2026 11:46:07 +02:00

perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS

On Sapphire Rapids (SPR) guests where pebs_baseline is not advertised, running command

$ ./perf record -e cpu/event=0x00,umask=0x01,i\
	 name=INST_RETIRED.PREC_DIST/p -c 10000 sleep 1

can trigger:

unchecked MSR access error: WRMSR to 0x3f1 ... in\
intel_pmu_pebs_enable_all()

This occurs because SPR-specific PEBS constraints allow fixed-counter scheduling (for example, INST_RETIRED.PREC_DIST on fixed counter 0). In guests lacking pebs_baseline, KVM does not support PEBS sampling on fixed counters, so enabling such events reaches an invalid MSR programming path.

Starting with Icelake, regardless of whether Extended PEBS or Architectural PEBS is in use, all counters (including fixed counters) support PEBS sampling, and the PMU_FL_PEBS_ALL flag is set by default. As long as PMU_FL_PEBS_ALL is set, constraint lookup automatically falls back to the corresponding non-PEBS constraints if no matching entry is found in the PEBS constraints table. Since non-PEBS event constraints already contain the same fixed-counter constraints, it is safe to remove these fixed-counter entries from the PEBS constraints table. The fallback mechanism will ensure that fixed PEBS events are scheduled onto the correct fixed counters.

So, directly drop the fixed-counter entries from the PEBS constraint table. Without pebs_baseline, these fixed-counter PEBS events will now resolve to empty constraints and will not be scheduled or enabled, thereby avoiding the warning and bypassing the broken guest PEBS path.

---
 arch/x86/events/intel/ds.c | 13 -------------
 1 file changed, 13 deletions(-)
arch/x86/events/intel/ds.c +0 -13
index 91a093d..e86e4ba 100644
@@ -1444,10 +1444,6 @@ struct event_constraint intel_skl_pebs_event_constraints[] = {
 };
 
 struct event_constraint intel_icl_pebs_event_constraints[] = {
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x01c0, 0x100000000ULL),	/* old INST_RETIRED.PREC_DIST */
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x0100, 0x100000000ULL),	/* INST_RETIRED.PREC_DIST */
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x0400, 0x800000000ULL),	/* SLOTS */
-
 	INTEL_PLD_CONSTRAINT(0x1cd, 0xff),			/* MEM_TRANS_RETIRED.LOAD_LATENCY */
 	INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD(0x11d0, 0xf),	/* MEM_INST_RETIRED.STLB_MISS_LOADS */
 	INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST(0x12d0, 0xf),	/* MEM_INST_RETIRED.STLB_MISS_STORES */
@@ -1470,9 +1466,6 @@ struct event_constraint intel_icl_pebs_event_constraints[] = {
 };
 
 struct event_constraint intel_glc_pebs_event_constraints[] = {
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x100, 0x100000000ULL),	/* INST_RETIRED.PREC_DIST */
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x0400, 0x800000000ULL),
-
 	INTEL_FLAGS_EVENT_CONSTRAINT(0xc0, 0xfe),
 	INTEL_PLD_CONSTRAINT(0x1cd, 0xfe),
 	INTEL_PSD_CONSTRAINT(0x2cd, 0x1),
@@ -1497,9 +1490,6 @@ struct event_constraint intel_glc_pebs_event_constraints[] = {
 };
 
 struct event_constraint intel_lnc_pebs_event_constraints[] = {
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x100, 0x100000000ULL),	/* INST_RETIRED.PREC_DIST */
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x0400, 0x800000000ULL),
-
 	INTEL_FLAGS_UEVENT_CONSTRAINT(0x012a, 0x1),		/* OCR.* events */
 	INTEL_FLAGS_UEVENT_CONSTRAINT(0x012b, 0x1),		/* OCR.* events */
 
@@ -1531,9 +1521,6 @@ struct event_constraint intel_lnc_pebs_event_constraints[] = {
 };
 
 struct event_constraint intel_pnc_pebs_event_constraints[] = {
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x100, 0x100000000ULL),	/* INST_RETIRED.PREC_DIST */
-	INTEL_FLAGS_UEVENT_CONSTRAINT(0x0400, 0x800000000ULL),
-
 	INTEL_HYBRID_LDLAT_CONSTRAINT(0x1cd, 0xfc),
 	INTEL_HYBRID_STLAT_CONSTRAINT(0x2cd, 0x3),
 	INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD(0x11d0, 0xf),	/* MEM_INST_RETIRED.STLB_MISS_LOADS */
author
#15 lore raw

PATCH v4 7/8 perf/core: Fix kernel register info leak via hardware skid

In reply to Dapeng Mi

An unprivileged hardware perf event using exclude_kernel=1 can leak kernel register data to user space via PERF_SAMPLE_REGS_INTR or PERF_SAMPLE_IP. Due to hardware skid, a PMI may trigger after the CPU has already entered kernel space (Ring 0), bypassing the perf_allow_kernel() privilege barrier.

This security vulnerability is severely exacerbated by upcoming support for SIMD register sampling via XSAVES, which could expose sensitive kernel FPU states (such as active cryptographic keys).

Fix this by ensuring that sampled register data is dropped if the event's exclude_kernel attribute is set but the PMI catches the CPU in kernel mode.

Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Ian Rogers <[email protected]>
---
 kernel/events/core.c | 37 ++++++++++++++++++++++++++++++-------
 1 file changed, 30 insertions(+), 7 deletions(-)
kernel/events/core.c +30 -7
index 95d806bba654..89f6c9ffb964 100644
@@ -7792,10 +7792,20 @@ unsigned long perf_misc_flags(struct perf_event *event,
 unsigned long perf_instruction_pointer(struct perf_event *event,
 				       struct pt_regs *regs)
 {
-	if (should_sample_guest(event))
-		return perf_guest_get_ip();
+	/*
+	 * Hardware skid can lead to a scenario where a PMI is
+	 * delivered after the CPU has already entered kernel mode.
+	 * In that case, user-space sampling must not expose kernel
+	 * register state.
+	 */
+	if (should_sample_guest(event)) {
+		return event->attr.exclude_kernel &&
+		       !(perf_guest_state() & PERF_GUEST_USER) ?
+			0 : perf_guest_get_ip();
+	}
 
-	return perf_arch_instruction_pointer(regs);
+	return event->attr.exclude_kernel && !user_mode(regs) ?
+		0 : perf_arch_instruction_pointer(regs);
 }
 
 static void
@@ -7829,10 +7839,22 @@ static void perf_sample_regs_user(struct perf_regs *regs_user,
 }
 
 static void perf_sample_regs_intr(struct perf_regs *regs_intr,
-				  struct pt_regs *regs)
+				  struct pt_regs *regs,
+				  bool exclude_kernel)
 {
-	regs_intr->regs = regs;
-	regs_intr->abi  = perf_reg_abi(current);
+	/*
+	 * Hardware skid can lead to a scenario where a PMI is
+	 * delivered after the CPU has already entered kernel mode.
+	 * In that case, user-space sampling must not expose kernel
+	 * register state.
+	 */
+	if (exclude_kernel && !user_mode(regs)) {
+		regs_intr->abi = PERF_SAMPLE_REGS_ABI_NONE;
+		regs_intr->regs = NULL;
+	} else {
+		regs_intr->regs = regs;
+		regs_intr->abi = perf_reg_abi(current);
+	}
 }
 
 
@@ -8723,7 +8745,8 @@ void perf_prepare_sample(struct perf_sample_data *data,
 		/* regs dump ABI info */
 		int size = sizeof(u64);
 
-		perf_sample_regs_intr(&data->regs_intr, regs);
+		perf_sample_regs_intr(&data->regs_intr, regs,
+				      event->attr.exclude_kernel);
 
 		if (data->regs_intr.regs) {
 			u64 mask = event->attr.sample_regs_intr;
Signature · 1 line
2.34.1
merge bot Applied
#16 lore raw

[tip: perf/core] perf/core: Fix kernel register info leak via hardware skid

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     166f10836a653dfa280d4335603b52f685b8b1ef
Gitweb:        https://git.kernel.org/tip/166f10836a653dfa280d4335603b52f685b8b1ef
Author:        Dapeng Mi <[email protected]>
AuthorDate:    Tue, 16 Jun 2026 12:46:53 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 30 Jun 2026 11:46:07 +02:00

perf/core: Fix kernel register info leak via hardware skid

An unprivileged hardware perf event using exclude_kernel=1 can leak kernel register data to user space via PERF_SAMPLE_REGS_INTR or PERF_SAMPLE_IP. Due to hardware skid, a PMI may trigger after the CPU has already entered kernel space (Ring 0), bypassing the perf_allow_kernel() privilege barrier.

This security vulnerability is severely exacerbated by upcoming support for SIMD register sampling via XSAVES, which could expose sensitive kernel FPU states (such as active cryptographic keys).

Fix this by ensuring that sampled register data is dropped if the event's exclude_kernel attribute is set but the PMI catches the CPU in kernel mode.

---
 kernel/events/core.c | 37 ++++++++++++++++++++++++++++++-------
 1 file changed, 30 insertions(+), 7 deletions(-)
kernel/events/core.c +30 -7
index 954c36e..dd3bd9c 100644
@@ -7791,10 +7791,20 @@ unsigned long perf_misc_flags(struct perf_event *event,
 unsigned long perf_instruction_pointer(struct perf_event *event,
 				       struct pt_regs *regs)
 {
-	if (should_sample_guest(event))
-		return perf_guest_get_ip();
+	/*
+	 * Hardware skid can lead to a scenario where a PMI is
+	 * delivered after the CPU has already entered kernel mode.
+	 * In that case, user-space sampling must not expose kernel
+	 * register state.
+	 */
+	if (should_sample_guest(event)) {
+		return event->attr.exclude_kernel &&
+		       !(perf_guest_state() & PERF_GUEST_USER) ?
+			0 : perf_guest_get_ip();
+	}
 
-	return perf_arch_instruction_pointer(regs);
+	return event->attr.exclude_kernel && !user_mode(regs) ?
+		0 : perf_arch_instruction_pointer(regs);
 }
 
 static void
@@ -7828,10 +7838,22 @@ static void perf_sample_regs_user(struct perf_regs *regs_user,
 }
 
 static void perf_sample_regs_intr(struct perf_regs *regs_intr,
-				  struct pt_regs *regs)
+				  struct pt_regs *regs,
+				  bool exclude_kernel)
 {
-	regs_intr->regs = regs;
-	regs_intr->abi  = perf_reg_abi(current);
+	/*
+	 * Hardware skid can lead to a scenario where a PMI is
+	 * delivered after the CPU has already entered kernel mode.
+	 * In that case, user-space sampling must not expose kernel
+	 * register state.
+	 */
+	if (exclude_kernel && !user_mode(regs)) {
+		regs_intr->abi = PERF_SAMPLE_REGS_ABI_NONE;
+		regs_intr->regs = NULL;
+	} else {
+		regs_intr->regs = regs;
+		regs_intr->abi = perf_reg_abi(current);
+	}
 }
 
 
@@ -8722,7 +8744,8 @@ void perf_prepare_sample(struct perf_sample_data *data,
 		/* regs dump ABI info */
 		int size = sizeof(u64);
 
-		perf_sample_regs_intr(&data->regs_intr, regs);
+		perf_sample_regs_intr(&data->regs_intr, regs,
+				      event->attr.exclude_kernel);
 
 		if (data->regs_intr.regs) {
 			u64 mask = event->attr.sample_regs_intr;
author
#17 lore raw

PATCH v4 8/8 perf/core: Check kernel access when kernel callchains are requested

In reply to Dapeng Mi

perf_event_open() currently gates perf_allow_kernel() only on !attr.exclude_kernel.

However, users can still request kernel callchain collection with attr.exclude_callchain_kernel == 0 even when attr.exclude_kernel == 1. That still requires kernel profiling privilege, but the existing check does not enforce it.

Update the permission check to call perf_allow_kernel() when either kernel sampling is requested or kernel callchains are requested.

This keeps permission checks aligned with requested data and prevents unprivileged use of kernel callchain capture.

Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Ian Rogers <[email protected]>
---
 kernel/events/core.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/events/core.c +3 -1
index 89f6c9ffb964..bbd260d9d5b5 100644
@@ -13924,7 +13924,9 @@ SYSCALL_DEFINE5(perf_event_open,
 	if (err)
 		return err;
 
-	if (!attr.exclude_kernel) {
+	if (!attr.exclude_kernel ||
+	    ((attr.sample_type & PERF_SAMPLE_CALLCHAIN) &&
+	     !attr.exclude_callchain_kernel)) {
 		err = perf_allow_kernel();
 		if (err)
 			return err;
Signature · 1 line
2.34.1
merge bot Applied
#18 lore raw

[tip: perf/core] perf/core: Check kernel access when kernel callchains are requested

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     a4573a3838ae4fc73b70019cfa1dac9aaea7cc2f
Gitweb:        https://git.kernel.org/tip/a4573a3838ae4fc73b70019cfa1dac9aaea7cc2f
Author:        Dapeng Mi <[email protected]>
AuthorDate:    Tue, 16 Jun 2026 12:46:54 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 30 Jun 2026 11:46:08 +02:00

perf/core: Check kernel access when kernel callchains are requested

perf_event_open() currently gates perf_allow_kernel() only on !attr.exclude_kernel.

However, users can still request kernel callchain collection with attr.exclude_callchain_kernel == 0 even when attr.exclude_kernel == 1. That still requires kernel profiling privilege, but the existing check does not enforce it.

Update the permission check to call perf_allow_kernel() when either kernel sampling is requested or kernel callchains are requested.

This keeps permission checks aligned with requested data and prevents unprivileged use of kernel callchain capture.

---
 kernel/events/core.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/events/core.c +3 -1
index dd3bd9c..0f29d17 100644
@@ -13932,7 +13932,9 @@ SYSCALL_DEFINE5(perf_event_open,
 	if (err)
 		return err;
 
-	if (!attr.exclude_kernel) {
+	if (!attr.exclude_kernel ||
+	    ((attr.sample_type & PERF_SAMPLE_CALLCHAIN) &&
+	     !attr.exclude_callchain_kernel)) {
 		err = perf_allow_kernel();
 		if (err)
 			return err;