perf/x86: Miscellaneous PMU bug fixes
Subsystems (3)
v2 cover perf/x86: Miscellaneous PMU bug fixes
This series groups several independent PMU fixes to simplify review and backporting.
Changes:
v1 -> v2:
- Fallback to software branch type decoding if hardware decoding is not suppprted (Sashiko patch 4/9).
- Drop kernel IP for PERF_SAMPLE_IP if exclude_kernel attribute is required (Sashiko, patch 8/9).
- Add kernel access check when kernel callchains are requested (Sashiko, patch 9/9)
- Address Zide and Thomas's comments.
- Collect Reviewed-bys.
Patch layout:
- Patch 1/9: Fix anythread_deprecated being overwritten issue.
- Patches 2-3/9: Fix the issue that cap_user_rdpmc is not updated correctly.
- Patch 4/9: Fallback to software branch type decoding if no hardware decoding.
- Patch 5/9: Fix a kernel address leakage issue in LBR stack.
- Patch 6/9: Fix the issue that the return value of intel_pmu_init_hybrid() is not valiated correctly.
- Patch 7/9: Fix a "unchecked MSR access error" on PEBS_ENABLE MSR.
- Patch 8/9: Prevent a theoretical kernel register data leak in sampling.
- Patch 9/9: Add kernel access check when kernel callchains are requested.
History:
v1: https://lore.kernel.org/all/[email protected]/
Dapeng Mi (8):
perf/x86/intel: Remove anythread_deprecated bit from perf_capabilities perf/x86: Update cap_user_rdpmc base on rdpmc user disable state perf/x86/intel: Fallback to sw branch type decoding if no hw decoding perf/x86/intel: Drop LBR entries whose privilege level mismatches
br_sel
perf/x86/intel: Validate return value of intel_pmu_init_hybrid() perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS perf/core: Fix kernel register info leak via hardware skid perf/core: Check kernel access when kernel callchains are requested
Ian Rogers (1):
perf/x86: Introduce is_x86_pmu() helper
arch/x86/events/core.c | 19 +++------------- arch/x86/events/intel/core.c | 43 ++++++++++++++++++++++++------------ arch/x86/events/intel/ds.c | 13 ----------- arch/x86/events/intel/lbr.c | 15 ++++++++++--- arch/x86/events/perf_event.h | 25 +++++++++++++++++---- kernel/events/core.c | 41 +++++++++++++++++++++++++++------- 6 files changed, 98 insertions(+), 58 deletions(-)
Signature · 1 line
2.34.1
PATCH v2 1/9 perf/x86/intel: Remove anythread_deprecated bit from perf_capabilities
AnyThread mode deprecation is enumerated by CPUID.0AH:EDX[15] instead of PERF_CAPABILITIES MSR. It's not a good practice to define a bit to represent "anythread deprecation" in perf_capabilities. It leads to the anythread_deprecated bit could be overwritten by the real value of PERF_CAPABILITIES MSR, just like the below code in update_pmu_cap() does.
```
if (!intel_pmu_broken_perf_cap()) {
/* Perf Metric (Bit 15) and PEBS via PT (Bit 16) are hybrid enumeration */
rdmsrq(MSR_IA32_PERF_CAPABILITIES, hybrid(pmu, intel_cap).capabilities);
}
```It leads to the anythread_deprecated bit is cleared to 0 and the "any" attribute is incorrectly shown in the /sys/devices/cpu/format/ folder on these support Perfmon v6 platforms, like Clearwater Forest.
``` $grep . /sys/devices/cpu/format/* /sys/devices/cpu/format/acr_mask:config2:0-63 /sys/devices/cpu/format/any:config:21 /sys/devices/cpu/format/cmask:config:24-31 ```
So remove the anythread_deprecated bit from perf_capabilities structure and directly depends on CPUID.0AH:EDX[15] to judge if anythread is deprecated.
--- arch/x86/events/intel/core.c | 10 +++------- arch/x86/events/perf_event.h | 2 +- 2 files changed, 4 insertions(+), 8 deletions(-)
arch/x86/events/intel/core.c +3 -7
@@ -7946,12 +7946,6 @@ __init int intel_pmu_init(void)x86_add_quirk (intel_arch_events_quirk);/* Install first, so it runs last */ -if (version >=5 ) { - x86_pmu.intel_cap.anythread_deprecated = edx.split.anythread_deprecated; -if (x86_pmu.intel_cap.anythread_deprecated) -pr_cont (" AnyThread deprecated, " ); - } -/* The perf side of core PMU is ready to support the mediated vPMU. */ x86_get_pmu (smp_processor_id ())->capabilities |=PERF_PMU_CAP_MEDIATED_VPMU ;
@@ -8828,8 +8822,10 @@ __init int intel_pmu_init(void) &x86_pmu.intel_ctrl);/* AnyThread may be deprecated on arch perfmon v5 or later */ -if (x86_pmu.intel_cap.anythread_deprecated) +if (version >=5 && edx.split.anythread_deprecated) { x86_pmu.format_attrs = intel_arch_formats_attr; +pr_cont ("AnyThread deprecated, " ); + }intel_pmu_check_event_constraints_all (NULL );
arch/x86/events/perf_event.h +1 -1
@@ -668,7 +668,7 @@ union perf_capabilities { u64 perf_metrics:1; u64 pebs_output_pt_available:1; u64 pebs_timing_info:1; - u64 anythread_deprecated:1; + u64 __reserved:1; u64 rdpmc_metrics_clear:1; };u64 capabilities;
Signature · 1 line
2.34.1
PATCH v2 2/9 perf/x86: Introduce is_x86_pmu() helper
In reply to Dapeng MiFrom: Ian Rogers <[email protected]>
To facilitate the detection of x86 PMU structures in upcoming patches, the is_x86_pmu() helper is introduced. Additionally, the is_x86_event() helper has been refactored to utilize is_x86_pmu().
No function changes intended.
--- arch/x86/events/core.c | 16 ---------------- arch/x86/events/perf_event.h | 18 +++++++++++++++++- 2 files changed, 17 insertions(+), 17 deletions(-)
arch/x86/events/core.c +0 -16
@@ -774,22 +774,6 @@ void x86_pmu_enable_all(int added) } } -int is_x86_event (struct perf_event *event) -{ -/* -* For a non-hybrid platforms, the type of X86 pmu is -* always PERF_TYPE_RAW. -* For a hybrid platform, the PERF_PMU_CAP_EXTENDED_HW_TYPE -* is a unique capability for the X86 PMU. -* Use them to detect a X86 event. -*/ -if (event->pmu->type ==PERF_TYPE_RAW || - event->pmu->capabilities &PERF_PMU_CAP_EXTENDED_HW_TYPE ) -return true; - -return false; -} -struct pmu *x86_get_pmu (unsigned int cpu) {struct cpu_hw_events *cpuc = &per_cpu (cpu_hw_events, cpu);
arch/x86/events/perf_event.h +17 -1
@@ -115,7 +115,23 @@ static inline bool is_topdown_event(struct perf_event *event)return is_metric_event (event) ||is_slots_event (event); } -int is_x86_event (struct perf_event *event); +static inline bool is_x86_pmu (struct pmu *pmu) +{ +/* +* For a non-hybrid platforms, the type of X86 pmu is +* always PERF_TYPE_RAW. +* For a hybrid platform, the PERF_PMU_CAP_EXTENDED_HW_TYPE +* is a unique capability for the X86 PMU. +* Use them to detect a X86 event. +*/ +return pmu->type ==PERF_TYPE_RAW || + pmu->capabilities &PERF_PMU_CAP_EXTENDED_HW_TYPE ; +} + +static inline bool is_x86_event (struct perf_event *event) +{ +return is_x86_pmu (event->pmu); +}static inline bool check_leader_group (struct perf_event *leader,int flags) {
Signature · 1 line
2.34.1
PATCH v2 3/9 perf/x86: Update cap_user_rdpmc base on rdpmc user disable state
In reply to Dapeng MiAfter introducing the RDPMC user disable feature, user-space RDPMC may return 0 instead of the actual event count. This creates an inconsistency with cap_user_rdpmc, where cap_user_rdpmc is set, but user-space RDPMC only returns 0.
To accurately represent the user-space RDPMC capability, update cap_user_rdpmc based on the RDPMC user disable state. If RDPMC user disable is enabled, cap_user_rdpmc is set to false, allowing user-space programs to fall back to the read() syscall to obtain the real event count.
Since arch_perf_update_userpage() could be called for software events, enhance x86_pmu_has_rdpmc_user_disable() to only check the x86 PMUs.
--- arch/x86/events/core.c | 3 +++ arch/x86/events/perf_event.h | 5 +++-- 2 files changed, 6 insertions(+), 2 deletions(-)
arch/x86/events/core.c +3 -0
@@ -2797,6 +2797,9 @@ void arch_perf_update_userpage(struct perf_event *event, userpg->cap_user_time_zero =0 ; userpg->cap_user_rdpmc = !!(event->hw.flags &PERF_EVENT_FLAG_USER_READ_CNT ); +if (x86_pmu_has_rdpmc_user_disable (event->pmu) && + event->hw.config &ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE ) + userpg->cap_user_rdpmc =0 ; userpg->pmc_width = x86_pmu.cntval_bits;if (!using_native_sched_clock () || !sched_clock_stable ())
arch/x86/events/perf_event.h +3 -2
@@ -1359,8 +1359,9 @@ static inline u64 x86_pmu_get_event_config(struct perf_event *event)static inline bool x86_pmu_has_rdpmc_user_disable (struct pmu *pmu) { -return !!(hybrid (pmu, config_mask) & -ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE ); +return is_x86_pmu (pmu) && + (hybrid (pmu, config_mask) & +ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE ); }extern struct event_constraint emptyconstraint;
Signature · 1 line
2.34.1
PATCH v2 4/9 perf/x86/intel: Fallback to sw branch type decoding if no hw decoding
In reply to Dapeng Miintel_pmu_lbr_filter() currently assumes Arch LBR provides hardware branch-type decoding and skips software decoding on that path.
However, Arch LBR may not always expose branch-type information. In that case, treating entries as hardware-decoded can misclassify sampled branches (for example, defaulting to JCC), which breaks branch-type filtering results.
Fix this by using software branch-type decoding when hardware branch-type decoding is unavailable (that is, when x86_lbr_type is not enabled). This keeps branch classification and filtering behavior correct across Arch LBR configurations.
--- arch/x86/events/intel/lbr.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/events/intel/lbr.c +1 -0
@@ -1232,6 +1232,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)* OTHER_BRANCH branch type still rely on software decoding. */ if (static_cpu_has (X86_FEATURE_ARCH_LBR ) && +static_branch_likely (&x86_lbr_type) && type <=ARCH_LBR_BR_TYPE_KNOWN_MAX ) { to_plm =kernel_ip (to) ?X86_BR_KERNEL :X86_BR_USER ; type = arch_lbr_br_type_map[type] | to_plm;
Signature · 1 line
2.34.1
PATCH v2 5/9 perf/x86/intel: Drop LBR entries whose privilege level mismatches br_sel
In reply to Dapeng MiBefore Arch LBR gained CPL filtering support, a user-only branch stack could still contain kernel addresses. As a result, kernel branch records may be exposed to user space even when PERF_SAMPLE_BRANCH_USER is requested.
For example, on Intel Tiger Lake, the following command can still report SYSRET/ERET entries with kernel-space from addresses:
```
$./perf record -e cycles:p -o - --branch-filter any,save_type,u -- \
./perf bench syscall basic --loop 1000 | \
./perf script -i - --fields brstack|tr ' ' '\n'| \
grep -E '0x[89a-f][0-9a-f]{15}'
Total time: 0.000 [sec]
0.219000 usecs/op
4,566,210 ops/sec
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.551 MB - ]
0xffffffff93c001c8/0x7f12a2b1d647/P/-/-/16959/SYSRET/-
0xffffffff93c001c8/0x7f12a2b1d5c2/P/-/-/17535/SYSRET/-
0xffffffff93c01928/0x7f12a2861000/P/-/-/6719/ERET/-
0xffffffff93c01928/0x7f12a297a000/P/-/-/8575/ERET/-
```The problem is that intel_pmu_lbr_filter() does not fully validate the privilege level of sampled entries. It filters some mismatches based on the branch type and the to address, but it does not reject entries whose from address violates the requested branch privilege filter.
Fix this by extending software filtering to validate both from and to addresses against br_sel. Any LBR entry whose privilege level does not match the requested user/kernel filter is dropped. This prevents kernel addresses from appearing in user-only branch stacks, and likewise drops user entries from kernel-only stacks.
--- arch/x86/events/intel/lbr.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
arch/x86/events/intel/lbr.c +11 -3
@@ -1212,7 +1212,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) {u64 from, to;int br_sel = cpuc->br_sel; -int i, j, type, to_plm; +int i, j, type, to_plm, from_plm;bool compress = false;/* if sampling all branches, then nothing to filter */
@@ -1245,8 +1245,16 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) type |=X86_BR_NO_TX ; } -/* if type does not correspond, then discard */ -if (type ==X86_BR_NONE || (br_sel & type) != type) { + from_plm =kernel_ip (from) ?X86_BR_KERNEL :X86_BR_USER ; +/* +* If type does not correspond, then discard. +* Especially filter out the entries whose from or to address is +* a kernel address while only X86_BR_USER is set. This prevents +* kernel address from being leaked into a user-space-only LBR stack. +*/ +if (type ==X86_BR_NONE || (br_sel & type) != type || + (!(br_sel &X86_BR_KERNEL ) && (from_plm &X86_BR_KERNEL )) || + (!(br_sel &X86_BR_USER ) && (from_plm &X86_BR_USER ))) { cpuc->lbr_entries[i].from =0 ; compress = true; }
Signature · 1 line
2.34.1
PATCH v2 6/9 perf/x86/intel: Validate return value of intel_pmu_init_hybrid()
In reply to Dapeng MiThe memory allocation for the x86_pmu.hybrid_pmu[] array in intel_pmu_init_hybrid() can theoretically fail due to memory shortages. If this occurs, the initialization of the x86 hybrid PMU would fail.
Currently, the code does not check the return value of the intel_pmu_init_hybrid() function, which could lead to attempts to access the uninitialized x86_pmu.hybrid_pmu[] array, potentially causing a system panic.
So, add a check for the return value of intel_pmu_init_hybrid() to prevent invalid memory access in such scenarios. Besides, free the created kmem cache when error occurs.
--- arch/x86/events/intel/core.c | 33 ++++++++++++++++++++++++++------- 1 file changed, 26 insertions(+), 7 deletions(-)
arch/x86/events/intel/core.c +26 -7
@@ -7870,6 +7870,7 @@ __init int intel_pmu_init(void)int version, i;char *name;struct x86_hybrid_pmu *pmu; +int ret;/* Architectural Perfmon was introduced starting with Core "Yonah" */ if (!cpu_has (&boot_cpu_data,X86_FEATURE_ARCH_PERFMON )) {
@@ -8539,7 +8540,9 @@ __init int intel_pmu_init(void)* * Initialize the common PerfMon capabilities here. */ -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret <0 ) + goto err; x86_pmu.pebs_latency_data = grt_latency_data; x86_pmu.get_event_constraints = adl_get_event_constraints;
@@ -8597,7 +8600,9 @@ __init int intel_pmu_init(void)case INTEL_METEORLAKE :case INTEL_METEORLAKE_L :case INTEL_ARROWLAKE_U : -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret <0 ) + goto err; x86_pmu.pebs_latency_data = cmt_latency_data; x86_pmu.get_event_constraints = mtl_get_event_constraints;
@@ -8628,7 +8633,9 @@ __init int intel_pmu_init(void)pr_cont ("Pantherlake Hybrid events, " ); name ="pantherlake_hybrid" ; -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret <0 ) + goto err;/* Initialize big core specific PerfMon capabilities.*/ pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_CORE_IDX ];
@@ -8643,7 +8650,9 @@ __init int intel_pmu_init(void)pr_cont ("Arrowlake Hybrid events, " ); name ="arrowlake_hybrid" ; -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret <0 ) + goto err;/* Initialize big core specific PerfMon capabilities.*/ pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_CORE_IDX ];
@@ -8660,7 +8669,9 @@ __init int intel_pmu_init(void)pr_cont ("Lunarlake Hybrid events, " ); name ="lunarlake_hybrid" ; -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret <0 ) + goto err;/* Initialize big core specific PerfMon capabilities.*/ pmu = &x86_pmu.hybrid_pmu[X86_HYBRID_PMU_CORE_IDX ];
@@ -8685,7 +8696,9 @@ __init int intel_pmu_init(void)break ;case INTEL_ARROWLAKE_H : -intel_pmu_init_hybrid (hybrid_big_small_tiny); + ret =intel_pmu_init_hybrid (hybrid_big_small_tiny); +if (ret <0 ) + goto err; x86_pmu.pebs_latency_data = arl_h_latency_data; x86_pmu.get_event_constraints = arl_h_get_event_constraints;
@@ -8720,7 +8733,9 @@ __init int intel_pmu_init(void)case INTEL_NOVALAKE_L :pr_cont ("Novalake Hybrid events, " ); name ="novalake_hybrid" ; -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret <0 ) + goto err; x86_pmu.pebs_latency_data = nvl_latency_data; x86_pmu.get_event_constraints = mtl_get_event_constraints;
@@ -8885,6 +8900,10 @@ __init int intel_pmu_init(void)intel_aux_output_init ();return 0 ; + +err: +kmem_cache_destroy (x86_get_pmu (smp_processor_id ())->task_ctx_cache); +return ret; }/*
Signature · 1 line
2.34.1
PATCH v2 7/9 perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS
In reply to Dapeng MiOn SPR guests where pebs_baseline is not advertised, running:
$ ./perf record -e cpu/event=0x00,umask=0x01,i\ name=INST_RETIRED.PREC_DIST/p -c 10000 sleep 1
can trigger:
unchecked MSR access error: WRMSR to 0x3f1 ... in\
intel_pmu_pebs_enable_all()
Root cause:
SPR-specific PEBS constraints allow fixed-counter scheduling, for example INST_RETIRED.PREC_DIST on fixed counter 0. In guests without pebs_baseline, KVM does not support PEBS sampling on fixed counters, so enabling such events reaches an invalid MSR programming path.
Fix:
Drop fixed-counter entries from the PEBS constraint table. Without pebs_baseline, those fixed-counter PEBS events now resolve to empty constraints and are not scheduled/enabled, avoiding the warning and the broken guest PEBS path.
This is safe because, in pebs_baseline-capable cases, PEBS constraint lookup already falls back to non-PEBS constraints when needed, and fixed-counter constraints are effectively shared there.
--- arch/x86/events/intel/ds.c | 13 ------------- 1 file changed, 13 deletions(-)
arch/x86/events/intel/ds.c +0 -13
@@ -1447,10 +1447,6 @@ struct event_constraint intel_skl_pebs_event_constraints[] = { };struct event_constraint intel_icl_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x01c0 ,0x100000000ULL ),/* old INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ),/* SLOTS */ -INTEL_PLD_CONSTRAINT (0x1cd ,0xff ),/* MEM_TRANS_RETIRED.LOAD_LATENCY */ INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x11d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_LOADS */ INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x12d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_STORES */
@@ -1473,9 +1469,6 @@ struct event_constraint intel_icl_pebs_event_constraints[] = { };struct event_constraint intel_glc_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ), -INTEL_FLAGS_EVENT_CONSTRAINT (0xc0 ,0xfe ),INTEL_PLD_CONSTRAINT (0x1cd ,0xfe ),INTEL_PSD_CONSTRAINT (0x2cd ,0x1 ),
@@ -1500,9 +1493,6 @@ struct event_constraint intel_glc_pebs_event_constraints[] = { };struct event_constraint intel_lnc_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ), -INTEL_FLAGS_UEVENT_CONSTRAINT (0x012a ,0x1 ),/* OCR.* events */ INTEL_FLAGS_UEVENT_CONSTRAINT (0x012b ,0x1 ),/* OCR.* events */
@@ -1534,9 +1524,6 @@ struct event_constraint intel_lnc_pebs_event_constraints[] = { };struct event_constraint intel_pnc_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ), -INTEL_HYBRID_LDLAT_CONSTRAINT (0x1cd ,0xfc ),INTEL_HYBRID_STLAT_CONSTRAINT (0x2cd ,0x3 ),INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x11d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_LOADS */
Signature · 1 line
2.34.1
PATCH v2 8/9 perf/core: Fix kernel register info leak via hardware skid
In reply to Dapeng MiAn unprivileged hardware perf event using exclude_kernel=1 can leak kernel register data to user space via PERF_SAMPLE_REGS_INTR or PERF_SAMPLE_IP. Due to hardware skid, a PMI may trigger after the CPU has already entered kernel space (Ring 0), bypassing the perf_allow_kernel() privilege barrier.
This security vulnerability is severely exacerbated by upcoming support for SIMD register sampling via XSAVES, which could expose sensitive kernel FPU states (such as active cryptographic keys).
Fix this by ensuring that sampled register data is dropped if the event's exclude_kernel attribute is set but the PMI catches the CPU in kernel mode.
--- kernel/events/core.c | 37 ++++++++++++++++++++++++++++++------- 1 file changed, 30 insertions(+), 7 deletions(-)
kernel/events/core.c +30 -7
@@ -7763,10 +7763,20 @@ unsigned long perf_misc_flags(struct perf_event *event,unsigned long perf_instruction_pointer (struct perf_event *event,struct pt_regs *regs) { -if (should_sample_guest (event)) -return perf_guest_get_ip (); +/* +* Hardware skid can lead to a scenario where a PMI is +* delivered after the CPU has already entered kernel mode. +* In that case, user-space sampling must not expose kernel +* register state. +*/ +if (should_sample_guest (event)) { +return event->attr.exclude_kernel && + !(perf_guest_state () &PERF_GUEST_USER ) ? +0 :perf_guest_get_ip (); + } -return perf_arch_instruction_pointer (regs); +return event->attr.exclude_kernel && !user_mode (regs) ? +0 :perf_arch_instruction_pointer (regs); } static void
@@ -7800,10 +7810,22 @@ static void perf_sample_regs_user(struct perf_regs *regs_user, }static void perf_sample_regs_intr (struct perf_regs *regs_intr, -struct pt_regs *regs) +struct pt_regs *regs, +bool exclude_kernel) { - regs_intr->regs = regs; - regs_intr->abi =perf_reg_abi (current); +/* +* Hardware skid can lead to a scenario where a PMI is +* delivered after the CPU has already entered kernel mode. +* In that case, user-space sampling must not expose kernel +* register state. +*/ +if (exclude_kernel && !user_mode (regs)) { + regs_intr->abi =PERF_SAMPLE_REGS_ABI_NONE ; + regs_intr->regs =NULL ; + }else { + regs_intr->regs = regs; + regs_intr->abi =perf_reg_abi (current); + } }
@@ -8694,7 +8716,8 @@ void perf_prepare_sample(struct perf_sample_data *data,/* regs dump ABI info */ int size =sizeof (u64); -perf_sample_regs_intr (&data->regs_intr, regs); +perf_sample_regs_intr (&data->regs_intr, regs, + event->attr.exclude_kernel);if (data->regs_intr.regs) {u64 mask = event->attr.sample_regs_intr;
Signature · 1 line
2.34.1
PATCH v2 9/9 perf/core: Check kernel access when kernel callchains are requested
In reply to Dapeng Miperf_event_open() currently gates perf_allow_kernel() only on !attr.exclude_kernel.
However, users can still request kernel callchain collection with attr.exclude_callchain_kernel == 0 even when attr.exclude_kernel == 1. That still requires kernel profiling privilege, but the existing check does not enforce it.
Update the permission check to call perf_allow_kernel() when either kernel sampling is requested or kernel callchains are requested.
This keeps permission checks aligned with requested data and prevents unprivileged use of kernel callchain capture.
--- kernel/events/core.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/events/core.c +3 -1
@@ -13895,7 +13895,9 @@ SYSCALL_DEFINE5(perf_event_open,if (err)return err; -if (!attr.exclude_kernel) { +if (!attr.exclude_kernel || + ((attr.sample_type &PERF_SAMPLE_CALLCHAIN ) && + !attr.exclude_callchain_kernel)) { err =perf_allow_kernel ();if (err)return err;
Signature · 1 line
2.34.1
[email protected]
AI reviewRe: [Patch v2 5/9] perf/x86/intel: Drop LBR entries whose privilege level mismatches br_sel
In reply to Dapeng MiThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] The kernel address leak in LBR stacks fixed here for Intel PMUs still exists in AMD PMUs (both AMD LBR and AMD BRS), allowing KASLR bypass. --
commit 955fb4f61b3305a8e4972ee94bb41e752a3dd25c
Author: Dapeng Mi <[email protected]>
perf/x86/intel: Drop LBR entries whose privilege level mismatches br_sel
This commit fixes an issue where kernel branch addresses could be exposed to user space in user-only branch stacks. It extends the software filter in intel_pmu_lbr_filter() to validate both the from and to addresses against the requested branch privilege filter.
Quoted 4 lines
arch/x86/events/intel/lbr.c +0
-0
[ ... ]
Quoted 9 lines
arch/x86/events/intel/lbr.c +3
-2@@ -1246,8 +1246,16 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) type |=X86_BR_NO_TX ; } -/* if type does not correspond, then discard */ -if (type ==X86_BR_NONE || (br_sel & type) != type) { + from_plm =kernel_ip (from) ?X86_BR_KERNEL :X86_BR_USER ; +/* +* If type does not correspond, then discard.
arch/x86/events/intel/lbr.c intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) · line 1261
Quoted 10 lines
arch/x86/events/intel/lbr.c +7
-0+* Especially filter out the entries whose from or to address is +* a kernel address while only X86_BR_USER is set. This prevents +* kernel address from being leaked into a user-space-only LBR stack. +*/ +if (type ==X86_BR_NONE || (br_sel & type) != type || + (!(br_sel &X86_BR_KERNEL ) && (from_plm &X86_BR_KERNEL )) || + (!(br_sel &X86_BR_USER ) && (from_plm &X86_BR_USER ))) { cpuc->lbr_entries[i].from =0 ; compress = true; }
[Severity: High]
This is a pre-existing issue, but does the AMD PMU code suffer from this same vulnerability?
Looking at the AMD LBR software filter, it appears to only check the target privilege level and is missing the from_plm privilege validation:
arch/x86/events/amd/lbr.c:amd_pmu_lbr_filter() {
...
/* If type does not correspond, then discard */
if (type == X86_BR_NONE || (br_sel & type) != type) {
cpuc->lbr_entries[i].from = 0; /* mark invalid */
...
}Similarly, the AMD BRS filter explicitly only validates the to address and ignores the from address:
arch/x86/events/amd/brs.c:amd_brs_match_plm() {
...
if (!(type & plm_k) && kernel_ip(to))
return 0;
if (!(type & plm_u) && !kernel_ip(to))
return 0;
return 1;
}Could kernel addresses from kernel-to-user branches leak into user-space LBR stacks on AMD systems through these paths, bypassing KASLR in the same way?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
[email protected]
AI reviewRe: [Patch v2 4/9] perf/x86/intel: Fallback to sw branch type decoding if no hw decoding
In reply to Dapeng MiThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [High] The fallback to software branch decoding causes massive NMI overhead and drops valid LBR entries when user memory is paged out.
Pre-existing issues:
- [Critical] Page fault in NMI context due to an out-of-bounds direct memory read into unmapped guard pages during LBR software branch type decoding. --
commit 11f8cb2dc702b7450f8fbb74de9eaa30a63a9248
Author: Dapeng Mi <[email protected]>
Subject: perf/x86/intel: Fallback to sw branch type decoding if no hw decoding
This commit updates intel_pmu_lbr_filter() to use software branch-type decoding when hardware branch-type decoding is unavailable (i.e. x86_lbr_type is not enabled). This aims to keep branch classification and filtering behavior correct across Arch LBR configurations.
arch/x86/events/intel/lbr.c intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) · line 1236
Quoted 9 lines
arch/x86/events/intel/lbr.c +1
-0@@ -1233,6 +1233,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)* OTHER_BRANCH branch type still rely on software decoding. */ if (static_cpu_has(X86_FEATURE_ARCH_LBR) && + static_branch_likely(&x86_lbr_type) &&
[Severity: High]
When x86_lbr_type is not enabled, this falls back to software decoding via branch_type(). Because intel_pmu_setup_hw_lbr_filter() unconditionally sets X86_BR_TYPE_SAVE for Arch LBR CPUs:
arch/x86/events/intel/lbr.c:intel_pmu_setup_hw_lbr_filter() {
...
if (static_cpu_has(X86_FEATURE_ARCH_LBR)) {
...
reg->reg |= X86_BR_TYPE_SAVE;
...
}the early return fast-path in intel_pmu_lbr_filter() is bypassed. Does this cause intel_pmu_lbr_filter() to iterate over all LBR entries and perform high-overhead instruction decoding in NMI context even for generic branch requests?
Additionally, if user memory is paged out, copy_from_user_nmi() will fail, causing branch_type() to return X86_BR_NONE. Does this result in valid LBR entries being silently dropped?
[Severity: Critical]
This is a pre-existing issue, but falling back to software decoding exposes a vulnerability in get_branch_type() when processing LBR entries in NMI context:
arch/x86/events/utils.c:get_branch_type() {
...
if (kernel_text_address(from) && !in_gate_area_no_mm(from)) {
addr = (void *)from;
bytes_read = MAX_INSN_SIZE;
...
insn_init(&insn, addr, bytes_read, is64);
ret = decode_branch_type(&insn);
...
}If the from address (which is controlled by the LBR hardware recording a faulting RIP) points to the very last byte of a mapped page, bytes_read is set to 15. The instruction decoder will blindly read past the page boundary into an unmapped guard page without exception table handlers.
Can this trigger a kernel page fault (#PF) and Oops in NMI context when processing the PMU interrupt?
Quoted 3 lines
arch/x86/events/intel/lbr.c +0
-0type <= ARCH_LBR_BR_TYPE_KNOWN_MAX) { to_plm =kernel_ip (to) ?X86_BR_KERNEL :X86_BR_USER ; type = arch_lbr_br_type_map[type] | to_plm;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
[email protected]
AI reviewRe: [Patch v2 9/9] perf/core: Check kernel access when kernel callchains are requested
In reply to Dapeng MiThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Critical] The patch breaks `perf record -g` for unprivileged users with existing `perf` binaries. --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] The patch breaks perf record -g for unprivileged users with existing perf binaries. --
commit ac223ffdf5e79c1aa96a91deecdff94004bbe30f
Author: Dapeng Mi <[email protected]>
perf/core: Check kernel access when kernel callchains are requested
This commit updates perf_event_open to verify kernel access permissions when kernel callchains are requested, even if exclude_kernel is set. This is intended to prevent unprivileged capture of kernel callchains.
Quoted 3 lines
kernel/events/core.c +0
-0
[ ... ]
kernel/events/core.c SYSCALL_DEFINE5(perf_event_open, · line 13910
Quoted 9 lines
kernel/events/core.c +3
-1@@ -13904,7 +13904,9 @@ SYSCALL_DEFINE5(perf_event_open,if (err)return err; -if (!attr.exclude_kernel) { +if (!attr.exclude_kernel || + ((attr.sample_type &PERF_SAMPLE_CALLCHAIN ) && + !attr.exclude_callchain_kernel)) { err =perf_allow_kernel ();
[Severity: Critical]
Does this change break perf record -g for unprivileged users using existing perf binaries?
When an unprivileged user runs perf record -g, the initial syscall request fails with -EACCES. The perf tool's existing fallback logic retries by setting exclude_kernel = 1, but it leaves exclude_callchain_kernel = 0.
Because this new check enforces the privilege requirement for callchains even when exclude_kernel is set, the retried syscall will fail a second time with -EACCES.
Looking at the perf tool's fallback handler:
tools/perf/util/evsel.c:evsel__fallback() {
...
} else if (err == EACCES && !evsel->core.attr.exclude_kernel &&
(paranoid = perf_event_paranoid()) > 1) {
...
}Since exclude_kernel was set to 1 during the first fallback, the tool cannot handle the second -EACCES failure. Will this cause the tool to abort entirely with a permission denied error rather than falling back to user-only callchains?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
[email protected]
AI reviewRe: [Patch v2 6/9] perf/x86/intel: Validate return value of intel_pmu_init_hybrid()
In reply to Dapeng MiThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Unnecessary error handling logic for memory allocation failure in early boot (__init) function --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Unnecessary error handling logic for memory allocation failure in early boot (__init) function --
commit 3dd0c7770a2df131c0ee92d24ad34bc579754b8e
Author: Dapeng Mi <[email protected]>
perf/x86/intel: Validate return value of intel_pmu_init_hybrid()
This commit adds checks for the return value of intel_pmu_init_hybrid() to prevent accessing the uninitialized x86_pmu.hybrid_pmu[] array on allocation failure. It also introduces an error path to free the created task_ctx_cache kmem cache if this failure occurs.
Quoted 4 lines
arch/x86/events/intel/core.c +0
-0
[ ... ]
Quoted 8 lines
arch/x86/events/intel/core.c +3
-1@@ -8540,7 +8541,9 @@ __init int intel_pmu_init(void)* * Initialize the common PerfMon capabilities here. */ -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret <0 ) + goto err;
[ ... ]
arch/x86/events/intel/core.c __init int intel_pmu_init(void) · line 8908
Quoted 9 lines
arch/x86/events/intel/core.c +4
-0@@ -8886,6 +8901,10 @@ __init int intel_pmu_init(void)intel_aux_output_init ();return 0 ; + +err: +kmem_cache_destroy (x86_get_pmu (smp_processor_id ())->task_ctx_cache); +return ret; }
[Severity: Low]
Is this cleanup logic necessary for an __init function?
Memory allocation failures during early boot generally mean the system cannot boot anyway. Functions executed only at this stage typically do not need to handle memory allocation failures gracefully, making this kind of error handling and cleanup logic effectively dead code.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Re: [Patch v2 5/9] perf/x86/intel: Drop LBR entries whose privilege level mismatches br_sel
In reply to [email protected]Quoted 70 lines · On 6/9/2026 1:21 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] The kernel address leak in LBR stacks fixed here for Intel PMUs still exists in AMD PMUs (both AMD LBR and AMD BRS), allowing KASLR bypass. --commit
955fb4f61b3305a8e4972ee94bb41e752a3dd25c
Author: Dapeng Mi <[email protected]>perf/x86/intel: Drop LBR entries whose privilege level mismatches
br_selThis commit fixes an issue where kernel branch addresses could be exposed to user space in user-only branch stacks. It extends the software filter in
intel_pmu_lbr_filter()to validate both the from and to addresses against the requested branch privilege filter.Quoted 4 lines
arch/x86/events/intel/lbr.c +0
-0[ ... ]
Quoted 9 lines
arch/x86/events/intel/lbr.c +3
-2@@ -1246,8 +1246,16 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) type |=X86_BR_NO_TX ; } -/* if type does not correspond, then discard */ -if (type ==X86_BR_NONE || (br_sel & type) != type) { + from_plm =kernel_ip (from) ?X86_BR_KERNEL :X86_BR_USER ; +/* +* If type does not correspond, then discard. arch/x86/events/intel/lbr.c intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) · line 1261
Quoted 10 lines
arch/x86/events/intel/lbr.c +7
-0+* Especially filter out the entries whose from or to address is +* a kernel address while only X86_BR_USER is set. This prevents +* kernel address from being leaked into a user-space-only LBR stack. +*/ +if (type ==X86_BR_NONE || (br_sel & type) != type || + (!(br_sel &X86_BR_KERNEL ) && (from_plm &X86_BR_KERNEL )) || + (!(br_sel &X86_BR_USER ) && (from_plm &X86_BR_USER ))) { cpuc->lbr_entries[i].from =0 ; compress = true; }[Severity: High]
This is a pre-existing issue, but does the AMD PMU code suffer from this same vulnerability?Looking at the AMD LBR software filter, it appears to only check the target privilege level and is missing the
from_plmprivilege validation:arch/x86/events/amd/lbr.c:amd_pmu_lbr_filter() { ... /* If type does not correspond, then discard */ if (type == X86_BR_NONE || (br_sel & type) != type) { cpuc->lbr_entries[i].from = 0; /* mark invalid */ ... }Similarly, the AMD BRS filter explicitly only validates the to address and ignores the from address:
arch/x86/events/amd/brs.c:amd_brs_match_plm() { ... if (!(type & plm_k) && kernel_ip(to)) return 0; if (!(type & plm_u) && !kernel_ip(to)) return 0; return 1; }
Quoted 3 lines
Could kernel addresses from kernel-to-user branches leak into user-space LBR stacks on AMD systems through these paths, bypassing KASLR in the same way?
Yes, this is a known issue and AMD guys would post the fixing patches separately. Thanks.
Quoted 1 line
Re: [Patch v2 6/9] perf/x86/intel: Validate return value of intel_pmu_init_hybrid()
In reply to [email protected]Quoted 42 lines · On 6/9/2026 1:25 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Unnecessary error handling logic for memory allocation failure in early boot (__init) function --commit
3dd0c7770a2df131c0ee92d24ad34bc579754b8e
Author: Dapeng Mi <[email protected]>perf/x86/intel: Validate return value of
intel_pmu_init_hybrid()This commit adds checks for the return value of
intel_pmu_init_hybrid()to prevent accessing the uninitialized x86_pmu.hybrid_pmu[] array on allocation failure. It also introduces an error path to free the createdtask_ctx_cachekmem cache if this failure occurs.Quoted 4 lines
arch/x86/events/intel/core.c +0
-0[ ... ]
Quoted 8 lines
arch/x86/events/intel/core.c +3
-1@@ -8540,7 +8541,9 @@ __init int intel_pmu_init(void)* * Initialize the common PerfMon capabilities here. */ -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret <0 ) + goto err;[ ... ]
arch/x86/events/intel/core.c __init int intel_pmu_init(void) · line 8908
Quoted 9 lines
arch/x86/events/intel/core.c +4
-0@@ -8886,6 +8901,10 @@ __init int intel_pmu_init(void)intel_aux_output_init ();return 0 ; + +err: +kmem_cache_destroy (x86_get_pmu (smp_processor_id ())->task_ctx_cache); +return ret; }[Severity: Low]
Is this cleanup logic necessary for an__initfunction?
Quoted 4 lines
Memory allocation failures during early boot generally mean the system cannot boot anyway. Functions executed only at this stage typically do not need to handle memory allocation failures gracefully, making this kind of error handling and cleanup logic effectively dead code.
IMO, even it's in early boot, it's better not to trigger an invalid memory access error which could cause misleading.
Thanks.
Quoted 1 line
Re: [Patch v2 9/9] perf/core: Check kernel access when kernel callchains are requested
In reply to [email protected]Quoted 48 lines · On 6/9/2026 1:24 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] The patch breaksperf record -gfor unprivileged users with existingperfbinaries. --commit
ac223ffdf5e79c1aa96a91deecdff94004bbe30f
Author: Dapeng Mi <[email protected]>perf/core: Check kernel access when kernel callchains are requested
This commit updates
perf_event_opento verify kernel access permissions when kernel callchains are requested, even ifexclude_kernelis set. This is intended to prevent unprivileged capture of kernel callchains.Quoted 3 lines
kernel/events/core.c +0
-0[ ... ]
kernel/events/core.c SYSCALL_DEFINE5(perf_event_open, · line 13910
Quoted 9 lines
kernel/events/core.c +3
-1@@ -13904,7 +13904,9 @@ SYSCALL_DEFINE5(perf_event_open,if (err)return err; -if (!attr.exclude_kernel) { +if (!attr.exclude_kernel || + ((attr.sample_type &PERF_SAMPLE_CALLCHAIN ) && + !attr.exclude_callchain_kernel)) { err =perf_allow_kernel ();[Severity: Critical]
Does this change break perf record -g for unprivileged users using existing perf binaries?When an unprivileged user runs perf record -g, the initial syscall request fails with
-EACCES. The perf tool's existing fallback logic retries by settingexclude_kernel= 1, but it leavesexclude_callchain_kernel= 0.Because this new check enforces the privilege requirement for callchains even when
exclude_kernelis set, the retried syscall will fail a second time with-EACCES.Looking at the perf tool's fallback handler:
tools/perf/util/evsel.c:evsel__fallback() { ... } else if (err == EACCES && !evsel->core.attr.exclude_kernel && (paranoid = perf_event_paranoid()) > 1) { ... }
Quoted 4 lines
Since
exclude_kernelwas set to 1 during the first fallback, the tool cannot handle the second-EACCESfailure. Will this cause the tool to abort entirely with a permission denied error rather than falling back to user-only callchains?
Yes, this is the expected behavior, we need the similar fallback for exclude_callchain_kernel attribute. The perf tools patch would be posted separately.
Thanks.
Quoted 1 line
Re: [Patch v2 4/9] perf/x86/intel: Fallback to sw branch type decoding if no hw decoding
In reply to [email protected]Quoted 40 lines · On 6/9/2026 1:24 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [High] The fallback to software branch decoding causes massive NMI overhead and drops valid LBR entries when user memory is paged out.Pre-existing issues:
- [Critical] Page fault in NMI context due to an out-of-bounds direct memory read into unmapped guard pages during LBR software branch type decoding. --commit
11f8cb2dc702b7450f8fbb74de9eaa30a63a9248
Author: Dapeng Mi <[email protected]>
Subject: perf/x86/intel: Fallback to sw branch type decoding if no hw decodingThis commit updates
intel_pmu_lbr_filter()to use software branch-type decoding when hardware branch-type decoding is unavailable (i.e.x86_lbr_typeis not enabled). This aims to keep branch classification and filtering behavior correct across Arch LBR configurations.arch/x86/events/intel/lbr.c intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) · line 1236
Quoted 9 lines
arch/x86/events/intel/lbr.c +1
-0@@ -1233,6 +1233,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)* OTHER_BRANCH branch type still rely on software decoding. */ if (static_cpu_has(X86_FEATURE_ARCH_LBR) && + static_branch_likely(&x86_lbr_type) &&[Severity: High]
Whenx86_lbr_typeis not enabled, this falls back to software decoding viabranch_type(). Becauseintel_pmu_setup_hw_lbr_filter()unconditionally setsX86_BR_TYPE_SAVEfor Arch LBR CPUs:arch/x86/events/intel/lbr.c:intel_pmu_setup_hw_lbr_filter() { ... if (static_cpu_has(X86_FEATURE_ARCH_LBR)) { ... reg->reg |= X86_BR_TYPE_SAVE; ... }
Quoted 4 lines
the early return fast-path in
intel_pmu_lbr_filter()is bypassed. Does this causeintel_pmu_lbr_filter()to iterate over all LBR entries and perform high-overhead instruction decoding in NMI context even for generic branch requests?
Generally speaking, the x86_lbr_type should always be true for arch-LBR. Even it's not true in some corner cases (like guest environment), high overhead is better than wrong branch decoding.
Quoted 4 lines
Additionally, if user memory is paged out,
copy_from_user_nmi()will fail, causingbranch_type()to returnX86_BR_NONE. Does this result in valid LBR entries being silently dropped?
I doubt if the issue could really be seeb. Even it happens, it seems there is nothing to do except dropping them.
Thanks.
Quoted 27 lines
[Severity: Critical]
This is a pre-existing issue, but falling back to software decoding exposes a vulnerability inget_branch_type()when processing LBR entries in NMI context:arch/x86/events/utils.c:get_branch_type() { ... if (kernel_text_address(from) && !in_gate_area_no_mm(from)) { addr = (void *)from; bytes_read = MAX_INSN_SIZE; ... insn_init(&insn, addr, bytes_read, is64); ret = decode_branch_type(&insn); ... }If the from address (which is controlled by the LBR hardware recording a faulting RIP) points to the very last byte of a mapped page,
bytes_readis set to 15. The instruction decoder will blindly read past the page boundary into an unmapped guard page without exception table handlers.Can this trigger a kernel page fault (#PF) and Oops in NMI context when processing the PMU interrupt?
Quoted 3 lines
arch/x86/events/intel/lbr.c +0
-0type <= ARCH_LBR_BR_TYPE_KNOWN_MAX) { to_plm =kernel_ip (to) ?X86_BR_KERNEL :X86_BR_USER ; type = arch_lbr_br_type_map[type] | to_plm;
Peter Zijlstra
maintainerRe: [Patch v2 4/9] perf/x86/intel: Fallback to sw branch type decoding if no hw decoding
In reply to Dapeng MiQuoted 23 lines · Dapeng Mi
intel_pmu_lbr_filter()currently assumes Arch LBR provides hardware branch-type decoding and skips software decoding on that path.However, Arch LBR may not always expose branch-type information. In that case, treating entries as hardware-decoded can misclassify sampled branches (for example, defaulting to JCC), which breaks branch-type filtering results.
Fix this by using software branch-type decoding when hardware branch-type decoding is unavailable (that is, when
x86_lbr_typeis not enabled). This keeps branch classification and filtering behavior correct across Arch LBR configurations.Signed-off-by Dapeng Mi <[email protected]>--- arch/x86/events/intel/lbr.c | 1 + 1 file changed, 1 insertion(+)arch/x86/events/intel/lbr.c +0
-0
arch/x86/events/intel/lbr.c intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) · line 1238
Quoted 7 lines
arch/x86/events/intel/lbr.c +1
-0@@ -1232,6 +1232,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)* OTHER_BRANCH branch type still rely on software decoding. */ if (static_cpu_has (X86_FEATURE_ARCH_LBR ) && +static_branch_likely (&x86_lbr_type) && type <=ARCH_LBR_BR_TYPE_KNOWN_MAX ) { to_plm =kernel_ip (to) ?X86_BR_KERNEL :X86_BR_USER ; type = arch_lbr_br_type_map[type] | to_plm;
Now you have two static branches in concert. Best to make sure x86_lbr_type covers both conditions, no?
Peter Zijlstra
maintainerRe: [Patch v2 5/9] perf/x86/intel: Drop LBR entries whose privilege level mismatches br_sel
In reply to Dapeng MiQuoted 50 lines · Dapeng Mi
Before Arch LBR gained CPL filtering support, a user-only branch stack could still contain kernel addresses. As a result, kernel branch records may be exposed to user space even when
PERF_SAMPLE_BRANCH_USERis requested.For example, on Intel Tiger Lake, the following command can still report SYSRET/ERET entries with kernel-space from addresses:
``` $./perf record -e cycles:p -o - --branch-filter any,save_type,u -- \ ./perf bench syscall basic --loop 1000 | \ ./perf script -i - --fields brstack|tr ' ' '\n'| \ grep -E '0x[89a-f][0-9a-f]{15}' Total time: 0.000 [sec] 0.219000 usecs/op 4,566,210 ops/sec [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.551 MB - ] 0xffffffff93c001c8/0x7f12a2b1d647/P/-/-/16959/SYSRET/- 0xffffffff93c001c8/0x7f12a2b1d5c2/P/-/-/17535/SYSRET/- 0xffffffff93c01928/0x7f12a2861000/P/-/-/6719/ERET/- 0xffffffff93c01928/0x7f12a297a000/P/-/-/8575/ERET/- ```The problem is that
intel_pmu_lbr_filter()does not fully validate the privilege level of sampled entries. It filters some mismatches based on the branch type and the to address, but it does not reject entries whose from address violates the requested branch privilege filter.Fix this by extending software filtering to validate both from and to addresses against
br_sel. Any LBR entry whose privilege level does not match the requested user/kernel filter is dropped. This prevents kernel addresses from appearing in user-only branch stacks, and likewise drops user entries from kernel-only stacks.Reported-by Ian Rogers <[email protected]>Fixes 47125db27e47 ("perf/x86/intel/lbr: Support Architectural LBR")Signed-off-by Dapeng Mi <[email protected]>--- arch/x86/events/intel/lbr.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)arch/x86/events/intel/lbr.c +0
-0
arch/x86/events/intel/lbr.c intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) · line 1218
Quoted 8 lines
arch/x86/events/intel/lbr.c +1
-1@@ -1212,7 +1212,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) {u64 from, to;int br_sel = cpuc->br_sel; -int i, j, type, to_plm; +int i, j, type, to_plm, from_plm;bool compress = false;/* if sampling all branches, then nothing to filter */
If there, might as well order those variables in reverse xmas.
Peter Zijlstra
maintainerRe: [Patch v2 3/9] perf/x86: Update cap_user_rdpmc base on rdpmc user disable state
In reply to Dapeng MiQuoted 5 lines · Dapeng Mi
arch/x86/events/core.c +0
-0
arch/x86/events/core.c void arch_perf_update_userpage(struct perf_event *event, · line 2805
Quoted 9 lines
arch/x86/events/core.c +3
-0@@ -2797,6 +2797,9 @@ void arch_perf_update_userpage(struct perf_event *event, userpg->cap_user_time_zero =0 ; userpg->cap_user_rdpmc = !!(event->hw.flags &PERF_EVENT_FLAG_USER_READ_CNT ); +if (x86_pmu_has_rdpmc_user_disable (event->pmu) && + event->hw.config &ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE ) + userpg->cap_user_rdpmc =0 ; userpg->pmc_width = x86_pmu.cntval_bits;if (!using_native_sched_clock () || !sched_clock_stable ())
Remember, this is evaluated very very often. So it makes more sense to ensure the hw.flags state is correct such that this remains a simple expression.
Re: [Patch v2 3/9] perf/x86: Update cap_user_rdpmc base on rdpmc user disable state
Quoted 15 lines · Peter Zijlstra
Quoted 13 lines · Dapeng Mi
arch/x86/events/core.c +3
-0@@ -2797,6 +2797,9 @@ void arch_perf_update_userpage(struct perf_event *event, userpg->cap_user_time_zero =0 ; userpg->cap_user_rdpmc = !!(event->hw.flags &PERF_EVENT_FLAG_USER_READ_CNT ); +if (x86_pmu_has_rdpmc_user_disable (event->pmu) && + event->hw.config &ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE ) + userpg->cap_user_rdpmc =0 ; userpg->pmc_width = x86_pmu.cntval_bits;
Quoted 4 lines
arch/x86/events/core.c void arch_perf_update_userpage(struct perf_event *event, · line 2805
Quoted 1 line
arch/x86/events/core.c +0
-0if (!using_native_sched_clock () || !sched_clock_stable ())Remember, this is evaluated very very often. So it makes more sense to ensure the hw.flags state is correct such that this remains a simple expression.
Yes, directly manipulating PERF_EVENT_FLAG_USER_READ_CNT is a simpler way. Thanks.
arch/x86/events/core.c +0 -0
@@ -2797,9 +2797,6 @@ void arch_perf_update_userpage(struct perf_event *event,
userpg->cap_user_time_zero = 0; userpg->cap_user_rdpmc =
!!(event->hw.flags & PERF_EVENT_FLAG_USER_READ_CNT); - if (x86_pmu_has_rdpmc_user_disable(event->pmu) &&
- event->hw.config & ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE) - userpg->cap_user_rdpmc = 0; userpg->pmc_width = x86_pmu.cntval_bits;
if (!using_native_sched_clock() || !sched_clock_stable())
arch/x86/events/intel/core.c +0 -0
@@ -3532,10 +3532,13 @@ static void
intel_pmu_update_rdpmc_user_disable(struct perf_event *event)
*/
if (x86_pmu.attr_rdpmc == X86_USER_RDPMC_ALWAYS_ENABLE || (x86_pmu.attr_rdpmc == X86_USER_RDPMC_CONDITIONAL_ENABLE && - event->ctx->task))
+ event->ctx->task)) {
event->hw.config &= ~ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE; - else
+ } else {
event->hw.config |= ARCH_PERFMON_EVENTSEL_RDPMC_USER_DISABLE; + if (event->hw.flags & PERF_EVENT_FLAG_USER_READ_CNT) + event->hw.flags &= ~PERF_EVENT_FLAG_USER_READ_CNT; + }
}
Re: [Patch v2 4/9] perf/x86/intel: Fallback to sw branch type decoding if no hw decoding
In reply to Peter ZijlstraQuoted 19 lines · Peter Zijlstra
Quoted 18 lines · Dapeng Mi
intel_pmu_lbr_filter()currently assumes Arch LBR provides hardware branch-type decoding and skips software decoding on that path.However, Arch LBR may not always expose branch-type information. In that case, treating entries as hardware-decoded can misclassify sampled branches (for example, defaulting to JCC), which breaks branch-type filtering results.
Fix this by using software branch-type decoding when hardware branch-type decoding is unavailable (that is, when
x86_lbr_typeis not enabled). This keeps branch classification and filtering behavior correct across Arch LBR configurations.Signed-off-by Dapeng Mi <[email protected]>--- arch/x86/events/intel/lbr.c | 1 + 1 file changed, 1 insertion(+)
Quoted 14 lines
arch/x86/events/intel/lbr.c intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) · line 1238
Quoted 12 lines
arch/x86/events/intel/lbr.c +1
-0@@ -1232,6 +1232,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)* OTHER_BRANCH branch type still rely on software decoding. */ if (static_cpu_has (X86_FEATURE_ARCH_LBR ) && +static_branch_likely (&x86_lbr_type) && type <=ARCH_LBR_BR_TYPE_KNOWN_MAX ) { to_plm =kernel_ip (to) ?X86_BR_KERNEL :X86_BR_USER ; type = arch_lbr_br_type_map[type] | to_plm;Now you have two static branches in concert. Best to make sure
x86_lbr_typecovers both conditions, no?
Oh, yes. Actually x86_lbr_type would be enabled only when there is arch-LBR, so it's unnecessary to check static_cpu_has(X86_FEATURE_ARCH_LBR) again. static_cpu_has(X86_FEATURE_ARCH_LBR) could be simply removed. Thanks.
Quoted 1 line
Re: [Patch v2 5/9] perf/x86/intel: Drop LBR entries whose privilege level mismatches br_sel
In reply to Peter ZijlstraQuoted 58 lines · Peter Zijlstra
Quoted 57 lines · Dapeng Mi
Before Arch LBR gained CPL filtering support, a user-only branch stack could still contain kernel addresses. As a result, kernel branch records may be exposed to user space even when
PERF_SAMPLE_BRANCH_USERis requested.For example, on Intel Tiger Lake, the following command can still report SYSRET/ERET entries with kernel-space from addresses:
``` $./perf record -e cycles:p -o - --branch-filter any,save_type,u -- \ ./perf bench syscall basic --loop 1000 | \ ./perf script -i - --fields brstack|tr ' ' '\n'| \ grep -E '0x[89a-f][0-9a-f]{15}' Total time: 0.000 [sec] 0.219000 usecs/op 4,566,210 ops/sec [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.551 MB - ] 0xffffffff93c001c8/0x7f12a2b1d647/P/-/-/16959/SYSRET/- 0xffffffff93c001c8/0x7f12a2b1d5c2/P/-/-/17535/SYSRET/- 0xffffffff93c01928/0x7f12a2861000/P/-/-/6719/ERET/- 0xffffffff93c01928/0x7f12a297a000/P/-/-/8575/ERET/- ```The problem is that
intel_pmu_lbr_filter()does not fully validate the privilege level of sampled entries. It filters some mismatches based on the branch type and the to address, but it does not reject entries whose from address violates the requested branch privilege filter.Fix this by extending software filtering to validate both from and to addresses against
br_sel. Any LBR entry whose privilege level does not match the requested user/kernel filter is dropped. This prevents kernel addresses from appearing in user-only branch stacks, and likewise drops user entries from kernel-only stacks.Reported-by Ian Rogers <[email protected]>Fixes 47125db27e47 ("perf/x86/intel/lbr: Support Architectural LBR")Signed-off-by Dapeng Mi <[email protected]>--- arch/x86/events/intel/lbr.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)arch/x86/events/intel/lbr.c +1
-1@@ -1212,7 +1212,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) {u64 from, to;int br_sel = cpuc->br_sel; -int i, j, type, to_plm; +int i, j, type, to_plm, from_plm;bool compress = false;
Quoted 2 lines
arch/x86/events/intel/lbr.c intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) · line 1218
Quoted 1 line
arch/x86/events/intel/lbr.c +0
-0/* if sampling all branches, then nothing to filter */ If there, might as well order those variables in reverse xmas.
Sure. Thanks.
Peter Zijlstra
maintainerRe: [Patch v2 6/9] perf/x86/intel: Validate return value of intel_pmu_init_hybrid()
In reply to Dapeng MiWould not something like so work?
I could not find a reason we *have* to init arch lbr that early -- but perhaps I didn't look hard enough?
---
arch/x86/events/intel/core.c +21 -11
@@ -7534,14 +7534,14 @@ __init int intel_pmu_init(void)struct attribute **td_attr = &empty_attrs;struct attribute **mem_attr = &empty_attrs;struct attribute **tsx_attr = &empty_attrs; +struct x86_hybrid_pmu *pmu; +unsigned int fixed_mask;union cpuid10_edx edx;union cpuid10_eax eax;union cpuid10_ebx ebx; -unsigned int fixed_mask; +int version, i, ret;bool pmem = false; -int version, i;char *name; -struct x86_hybrid_pmu *pmu;/* Architectural Perfmon was introduced starting with Core "Yonah" */ if (!cpu_has (&boot_cpu_data,X86_FEATURE_ARCH_PERFMON )) {
@@ -7611,9 +7611,6 @@ __init int intel_pmu_init(void) x86_pmu.lbr_read = intel_pmu_lbr_read_32; } -if (boot_cpu_has (X86_FEATURE_ARCH_LBR )) -intel_pmu_arch_lbr_init (); -intel_pebs_init ();x86_add_quirk (intel_arch_events_quirk);/* Install first, so it runs last */
@@ -8216,7 +8213,9 @@ __init int intel_pmu_init(void)* * Initialize the common PerfMon capabilities here. */ -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret) +return ret; x86_pmu.pebs_latency_data = grt_latency_data; x86_pmu.get_event_constraints = adl_get_event_constraints;
@@ -8274,7 +8273,9 @@ __init int intel_pmu_init(void)case INTEL_METEORLAKE :case INTEL_METEORLAKE_L :case INTEL_ARROWLAKE_U : -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret) +return ret; x86_pmu.pebs_latency_data = cmt_latency_data; x86_pmu.get_event_constraints = mtl_get_event_constraints;
@@ -8313,7 +8314,9 @@ __init int intel_pmu_init(void) name ="lunarlake_hybrid" ; lnl_common: -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret) +return ret; x86_pmu.pebs_latency_data = lnl_latency_data; x86_pmu.get_event_constraints = mtl_get_event_constraints;
@@ -8337,7 +8340,9 @@ __init int intel_pmu_init(void)break ;case INTEL_ARROWLAKE_H : -intel_pmu_init_hybrid (hybrid_big_small_tiny); + ret =intel_pmu_init_hybrid (hybrid_big_small_tiny); +if (ret) +return ret; x86_pmu.pebs_latency_data = arl_h_latency_data; x86_pmu.get_event_constraints = arl_h_get_event_constraints;
@@ -8371,7 +8376,9 @@ __init int intel_pmu_init(void)case INTEL_NOVALAKE_L :pr_cont ("Novalake Hybrid events, " ); name ="novalake_hybrid" ; -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret) +return ret; x86_pmu.pebs_latency_data = nvl_latency_data; x86_pmu.get_event_constraints = mtl_get_event_constraints;
@@ -8478,6 +8485,9 @@ __init int intel_pmu_init(void)intel_pmu_check_event_constraints_all (NULL ); +if (boot_cpu_has (X86_FEATURE_ARCH_LBR )) +intel_pmu_arch_lbr_init (); +/* * Access LBR MSR may cause #GP under certain circumstances. * Check all LBR MSR here.
Peter Zijlstra
maintainerRe: [Patch v2 7/9] perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS
In reply to Dapeng MiQuoted 22 lines · Dapeng Mi
On SPR guests where
pebs_baselineis not advertised, running:$ ./perf record -e cpu/event=0x00,umask=0x01,i\ name=INST_RETIRED.PREC_DIST/p -c 10000 sleep 1can trigger:
unchecked MSR access error: WRMSR to
0x3f1... in\
intel_pmu_pebs_enable_all()Root cause:
SPR-specific PEBS constraints allow fixed-counter scheduling, for example INST_RETIRED.PREC_DIST on fixed counter 0. In guests withoutpebs_baseline, KVM does not support PEBS sampling on fixed counters, so enabling such events reaches an invalid MSR programming path.Fix:
Drop fixed-counter entries from the PEBS constraint table. Withoutpebs_baseline, those fixed-counter PEBS events now resolve to empty constraints and are not scheduled/enabled, avoiding the warning and the broken guest PEBS path.
Quoted 3 lines
This is safe because, in pebs_baseline-capable cases, PEBS constraint lookup already falls back to non-PEBS constraints when needed, and fixed-counter constraints are effectively shared there.
I am confused, this works outside of KVM? (It appears to work fine on my spr).. so removing this to fix some guest only issue seems wrong.
Quoted 54 lines
Reported-by Yi Lai <[email protected]>Signed-off-by Dapeng Mi <[email protected]>--- arch/x86/events/intel/ds.c | 13 ------------- 1 file changed, 13 deletions(-)arch/x86/events/intel/ds.c +0
-13@@ -1447,10 +1447,6 @@ struct event_constraint intel_skl_pebs_event_constraints[] = { };struct event_constraint intel_icl_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x01c0 ,0x100000000ULL ),/* old INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ),/* SLOTS */ -INTEL_PLD_CONSTRAINT (0x1cd ,0xff ),/* MEM_TRANS_RETIRED.LOAD_LATENCY */ INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x11d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_LOADS */ INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x12d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_STORES */ @@ -1473,9 +1469,6 @@ struct event_constraint intel_icl_pebs_event_constraints[] = { };struct event_constraint intel_glc_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ), -INTEL_FLAGS_EVENT_CONSTRAINT (0xc0 ,0xfe ),INTEL_PLD_CONSTRAINT (0x1cd ,0xfe ),INTEL_PSD_CONSTRAINT (0x2cd ,0x1 ),@@ -1500,9 +1493,6 @@ struct event_constraint intel_glc_pebs_event_constraints[] = { };struct event_constraint intel_lnc_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ), -INTEL_FLAGS_UEVENT_CONSTRAINT (0x012a ,0x1 ),/* OCR.* events */ INTEL_FLAGS_UEVENT_CONSTRAINT (0x012b ,0x1 ),/* OCR.* events */ @@ -1534,9 +1524,6 @@ struct event_constraint intel_lnc_pebs_event_constraints[] = { };struct event_constraint intel_pnc_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ), -INTEL_HYBRID_LDLAT_CONSTRAINT (0x1cd ,0xfc ),INTEL_HYBRID_STLAT_CONSTRAINT (0x2cd ,0x3 ),INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x11d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_LOADS */ Signature · 1 line
2.34.1
Peter Zijlstra
maintainerRe: [Patch v2 7/9] perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS
Quoted 23 lines · Peter Zijlstra
Quoted 22 lines · Dapeng Mi
On SPR guests where
pebs_baselineis not advertised, running:$ ./perf record -e cpu/event=0x00,umask=0x01,i\ name=INST_RETIRED.PREC_DIST/p -c 10000 sleep 1can trigger:
unchecked MSR access error: WRMSR to
0x3f1... in\
intel_pmu_pebs_enable_all()Root cause:
SPR-specific PEBS constraints allow fixed-counter scheduling, for example INST_RETIRED.PREC_DIST on fixed counter 0. In guests withoutpebs_baseline, KVM does not support PEBS sampling on fixed counters, so enabling such events reaches an invalid MSR programming path.Fix:
Drop fixed-counter entries from the PEBS constraint table. Withoutpebs_baseline, those fixed-counter PEBS events now resolve to empty constraints and are not scheduled/enabled, avoiding the warning and the broken guest PEBS path.
Quoted 7 lines
Quoted 3 lines
This is safe because, in pebs_baseline-capable cases, PEBS constraint lookup already falls back to non-PEBS constraints when needed, and fixed-counter constraints are effectively shared there.
I am confused, this works outside of KVM? (It appears to work fine on my spr).. so removing this to fix some guest only issue seems wrong.
Also, perf tools guys:
[ perf record: Woken up 1 times to write data ]
ERROR: Trying to write bpf_prog_info without libbpf support.
ERROR: Trying to write btf data without libbpf support. [ perf record: Captured and wrote 0.041 MB perf.data (92 samples) ]
I explicitly build perf like:
make NO_LIBBPF=1 NO_LIBPYTHON=1 NO_LIBLLVM=1 NO_RUST=1
W.T.F does it even attempt to use libbpf and report and error about it ?!?!
(Also, I needed that NO_RUST because the build was full on crashing without it)
Re: [Patch v2 6/9] perf/x86/intel: Validate return value of intel_pmu_init_hybrid()
In reply to Peter ZijlstraQuoted 94 lines · Peter Zijlstra
Would not something like so work?
I could not find a reason we *have* to init arch lbr that early -- but perhaps I didn't look hard enough?
---
arch/x86/events/intel/core.c +18
-11@@ -7534,14 +7534,14 @@ __init int intel_pmu_init(void)struct attribute **td_attr = &empty_attrs;struct attribute **mem_attr = &empty_attrs;struct attribute **tsx_attr = &empty_attrs; +struct x86_hybrid_pmu *pmu; +unsigned int fixed_mask;union cpuid10_edx edx;union cpuid10_eax eax;union cpuid10_ebx ebx; -unsigned int fixed_mask; +int version, i, ret;bool pmem = false; -int version, i;char *name; -struct x86_hybrid_pmu *pmu;/* Architectural Perfmon was introduced starting with Core "Yonah" */ if (!cpu_has (&boot_cpu_data,X86_FEATURE_ARCH_PERFMON )) {@@ -7611,9 +7611,6 @@ __init int intel_pmu_init(void) x86_pmu.lbr_read = intel_pmu_lbr_read_32; } -if (boot_cpu_has (X86_FEATURE_ARCH_LBR )) -intel_pmu_arch_lbr_init (); -intel_pebs_init ();x86_add_quirk (intel_arch_events_quirk);/* Install first, so it runs last */ @@ -8216,7 +8213,9 @@ __init int intel_pmu_init(void)* * Initialize the common PerfMon capabilities here. */ -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret) +return ret; x86_pmu.pebs_latency_data = grt_latency_data; x86_pmu.get_event_constraints = adl_get_event_constraints;@@ -8274,7 +8273,9 @@ __init int intel_pmu_init(void)case INTEL_METEORLAKE :case INTEL_METEORLAKE_L :case INTEL_ARROWLAKE_U : -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret) +return ret; x86_pmu.pebs_latency_data = cmt_latency_data; x86_pmu.get_event_constraints = mtl_get_event_constraints;@@ -8313,7 +8314,9 @@ __init int intel_pmu_init(void) name ="lunarlake_hybrid" ; lnl_common: -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret) +return ret; x86_pmu.pebs_latency_data = lnl_latency_data; x86_pmu.get_event_constraints = mtl_get_event_constraints;@@ -8337,7 +8340,9 @@ __init int intel_pmu_init(void)break ;case INTEL_ARROWLAKE_H : -intel_pmu_init_hybrid (hybrid_big_small_tiny); + ret =intel_pmu_init_hybrid (hybrid_big_small_tiny); +if (ret) +return ret; x86_pmu.pebs_latency_data = arl_h_latency_data; x86_pmu.get_event_constraints = arl_h_get_event_constraints;@@ -8371,7 +8376,9 @@ __init int intel_pmu_init(void)case INTEL_NOVALAKE_L :pr_cont ("Novalake Hybrid events, " ); name ="novalake_hybrid" ; -intel_pmu_init_hybrid (hybrid_big_small); + ret =intel_pmu_init_hybrid (hybrid_big_small); +if (ret) +return ret; x86_pmu.pebs_latency_data = nvl_latency_data; x86_pmu.get_event_constraints = mtl_get_event_constraints;
arch/x86/events/intel/core.c __init int intel_pmu_init(void) · line 8489
Quoted 5 lines
arch/x86/events/intel/core.c +2
-0@@ -8478,6 +8485,9 @@ __init int intel_pmu_init(void)intel_pmu_check_event_constraints_all (NULL ); +if (boot_cpu_has (X86_FEATURE_ARCH_LBR )) +intel_pmu_arch_lbr_init ();
It looks fine to move the arch_lbr_init() after the model-specific initialization, but need a double check. Let me run some tests on SPR and NVL. Thanks.
Quoted 4 lines
arch/x86/events/intel/core.c +1
-0+/* * Access LBR MSR may cause #GP under certain circumstances. * Check all LBR MSR here.
Re: [Patch v2 7/9] perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS
In reply to Peter ZijlstraQuoted 23 lines · Peter Zijlstra
Quoted 22 lines · Dapeng Mi
On SPR guests where
pebs_baselineis not advertised, running:$ ./perf record -e cpu/event=0x00,umask=0x01,i\ name=INST_RETIRED.PREC_DIST/p -c 10000 sleep 1can trigger:
unchecked MSR access error: WRMSR to
0x3f1... in\
intel_pmu_pebs_enable_all()Root cause:
SPR-specific PEBS constraints allow fixed-counter scheduling, for example INST_RETIRED.PREC_DIST on fixed counter 0. In guests withoutpebs_baseline, KVM does not support PEBS sampling on fixed counters, so enabling such events reaches an invalid MSR programming path.Fix:
Drop fixed-counter entries from the PEBS constraint table. Withoutpebs_baseline, those fixed-counter PEBS events now resolve to empty constraints and are not scheduled/enabled, avoiding the warning and the broken guest PEBS path.
Quoted 5 lines
Quoted 3 lines
This is safe because, in pebs_baseline-capable cases, PEBS constraint lookup already falls back to non-PEBS constraints when needed, and fixed-counter constraints are effectively shared there.
I am confused, this works outside of KVM? (It appears to work fine on my spr).. so removing this to fix some guest only issue seems wrong.
The reason that it works on bare metal is currently the constraint lookup would fallback into non-PEBS constraints if there is no matched entry in the PEBS constraints as long as the PEBS supports sampling on all counters including fixed counters (what the flag "PMU_FL_PEBS_ALL" indicates), like the below code shows.
```
struct event_constraint *intel_pebs_constraints(struct perf_event *event)
{
struct event_constraint *pebs_constraints = hybrid(event->pmu,
pebs_constraints);
struct event_constraint *c;
if (!event->attr.precise_ip)
return NULL;
if (pebs_constraints) {
for_each_event_constraint(c, pebs_constraints) {
if (constraint_match(c, event->hw.config)) {
event->hw.flags |= c->flags;
return c;
}
}
}
/*
* Extended PEBS support
* Makes the PEBS code search the normal constraints.
*/
if (x86_pmu.flags & PMU_FL_PEBS_ALL)
return NULL;
return &emptyconstraint;
}
static struct event_constraint *
__intel_get_event_constraints(struct cpu_hw_events *cpuc, int idx,
struct perf_event *event)
{
struct event_constraint *c;
c = intel_vlbr_constraints(event);
if (c)
return c;
c = intel_bts_constraints(event);
if (c)
return c;
c = intel_shared_regs_constraints(cpuc, event);
if (c)
return c;
c = intel_pebs_constraints(event);
if (c)
return c;
return x86_get_event_constraints(cpuc, idx, event);
}
```if intel_pebs_constraints() can't find the matched entry from PEBS constraints and PMU_FL_PEBS_ALL is set, NULL would be returned and it would lead to fallback the non-PEBS constraints lookup.
From Icelake starts, regardless of the extended PEBS or the arch-PEBS, all counters including the fixed counters support PEBS sampling and the flag PMU_FL_PEBS_ALL would set by default.
Since the non-PEBS constraints contain the similar fixed counter constraints, the fixed counter events would still be correctly scheduled on the fixed counters.
for example, here is the fixed counter constraints in intel_icl_event_constraints[],
```
static struct event_constraint intel_icl_event_constraints[] = {
FIXED_EVENT_CONSTRAINT(0x00c0, 0), /* INST_RETIRED.ANY */
FIXED_EVENT_CONSTRAINT(0x01c0, 0), /* old INST_RETIRED.PREC_DIST */
FIXED_EVENT_CONSTRAINT(0x0100, 0), /* pseudo INST_RETIRED.ANY */
FIXED_EVENT_CONSTRAINT(0x003c, 1), /* CPU_CLK_UNHALTED.CORE */
FIXED_EVENT_CONSTRAINT(0x0200, 1), /* pseudo CPU_CLK_UNHALTED.THREAD */
FIXED_EVENT_CONSTRAINT(0x0300, 2), /* pseudo CPU_CLK_UNHALTED.REF_TSC */
FIXED_EVENT_CONSTRAINT(0x0400, 3), /* pseudo TOPDOWN.SLOTS */
......
```Quoted 56 lines
Quoted 56 lines
Reported-by Yi Lai <[email protected]>Signed-off-by Dapeng Mi <[email protected]>--- arch/x86/events/intel/ds.c | 13 ------------- 1 file changed, 13 deletions(-)arch/x86/events/intel/ds.c +0
-13@@ -1447,10 +1447,6 @@ struct event_constraint intel_skl_pebs_event_constraints[] = { };struct event_constraint intel_icl_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x01c0 ,0x100000000ULL ),/* old INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ),/* SLOTS */ -INTEL_PLD_CONSTRAINT (0x1cd ,0xff ),/* MEM_TRANS_RETIRED.LOAD_LATENCY */ INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x11d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_LOADS */ INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x12d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_STORES */ @@ -1473,9 +1469,6 @@ struct event_constraint intel_icl_pebs_event_constraints[] = { };struct event_constraint intel_glc_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ), -INTEL_FLAGS_EVENT_CONSTRAINT (0xc0 ,0xfe ),INTEL_PLD_CONSTRAINT (0x1cd ,0xfe ),INTEL_PSD_CONSTRAINT (0x2cd ,0x1 ),@@ -1500,9 +1493,6 @@ struct event_constraint intel_glc_pebs_event_constraints[] = { };struct event_constraint intel_lnc_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ), -INTEL_FLAGS_UEVENT_CONSTRAINT (0x012a ,0x1 ),/* OCR.* events */ INTEL_FLAGS_UEVENT_CONSTRAINT (0x012b ,0x1 ),/* OCR.* events */ @@ -1534,9 +1524,6 @@ struct event_constraint intel_lnc_pebs_event_constraints[] = { };struct event_constraint intel_pnc_pebs_event_constraints[] = { -INTEL_FLAGS_UEVENT_CONSTRAINT (0x100 ,0x100000000ULL ),/* INST_RETIRED.PREC_DIST */ -INTEL_FLAGS_UEVENT_CONSTRAINT (0x0400 ,0x800000000ULL ), -INTEL_HYBRID_LDLAT_CONSTRAINT (0x1cd ,0xfc ),INTEL_HYBRID_STLAT_CONSTRAINT (0x2cd ,0x3 ),INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x11d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_LOADS */ Signature · 1 line
2.34.1
Peter Zijlstra
maintainerRe: [Patch v2 8/9] perf/core: Fix kernel register info leak via hardware skid
In reply to Dapeng MiQuoted 6 lines · Dapeng Mi
An unprivileged hardware perf event using exclude_kernel=1 can leak kernel register data to user space via
PERF_SAMPLE_REGS_INTRorPERF_SAMPLE_IP. Due to hardware skid, a PMI may trigger after the CPU has already entered kernel space (Ring 0), bypassing theperf_allow_kernel()privilege barrier.
Quoted 6 lines
This security vulnerability is severely exacerbated by upcoming support for SIMD register sampling via XSAVES, which could expose sensitive kernel FPU states (such as active cryptographic keys).
Fix this by ensuring that sampled register data is dropped if the event's
exclude_kernelattribute is set but the PMI catches the CPU in kernel mode.
There is history here, see for example:
https://lore.kernel.org/r/CAP045Ap8cMx6mzSgcQ3n3bnh_8GJuCp7_KZe_5ZTCR_K6cPTLw@mail.gmail.com
So your earlier patches also sanitize the branch stack, but not in generic code.
PHYS_ADDR already requires privileges
ADDR comes from PEBS on Intel, and IBS on AMD (iirc) and should be reliable.
So yeah, I suppose IP and REGS_INTR are the big ones.
Peter Zijlstra
maintainerRe: [Patch v2 7/9] perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS
In reply to Mi, DapengQuoted 31 lines · Dapeng
Quoted 23 lines · Peter Zijlstra
Quoted 22 lines · Dapeng Mi
On SPR guests where
pebs_baselineis not advertised, running:$ ./perf record -e cpu/event=0x00,umask=0x01,i\ name=INST_RETIRED.PREC_DIST/p -c 10000 sleep 1can trigger:
unchecked MSR access error: WRMSR to
0x3f1... in\
intel_pmu_pebs_enable_all()Root cause:
SPR-specific PEBS constraints allow fixed-counter scheduling, for example INST_RETIRED.PREC_DIST on fixed counter 0. In guests withoutpebs_baseline, KVM does not support PEBS sampling on fixed counters, so enabling such events reaches an invalid MSR programming path.Fix:
Drop fixed-counter entries from the PEBS constraint table. Withoutpebs_baseline, those fixed-counter PEBS events now resolve to empty constraints and are not scheduled/enabled, avoiding the warning and the broken guest PEBS path.Quoted 5 lines
Quoted 3 lines
This is safe because, in pebs_baseline-capable cases, PEBS constraint lookup already falls back to non-PEBS constraints when needed, and fixed-counter constraints are effectively shared there.
I am confused, this works outside of KVM? (It appears to work fine on my spr).. so removing this to fix some guest only issue seems wrong.
Quoted 7 lines
The reason that it works on bare metal is currently the constraint lookup would fallback into non-PEBS constraints if there is no matched entry in the PEBS constraints as long as the PEBS supports sampling on all counters including fixed counters (what the flag "
PMU_FL_PEBS_ALL" indicates), like the below code shows.```
Why the markdown nonsense? You're not an LLM, right? And you all know I'm reading this in mutt. Its just visual noise.
Anyway, please clarify the Changelog.
Re: [Patch v2 7/9] perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS
Quoted 36 lines · Peter Zijlstra
Quoted 35 lines · Dapeng
Quoted 23 lines · Peter Zijlstra
Quoted 22 lines · Dapeng Mi
On SPR guests where
pebs_baselineis not advertised, running:$ ./perf record -e cpu/event=0x00,umask=0x01,i\ name=INST_RETIRED.PREC_DIST/p -c 10000 sleep 1can trigger:
unchecked MSR access error: WRMSR to
0x3f1... in\
intel_pmu_pebs_enable_all()Root cause:
SPR-specific PEBS constraints allow fixed-counter scheduling, for example INST_RETIRED.PREC_DIST on fixed counter 0. In guests withoutpebs_baseline, KVM does not support PEBS sampling on fixed counters, so enabling such events reaches an invalid MSR programming path.Fix:
Drop fixed-counter entries from the PEBS constraint table. Withoutpebs_baseline, those fixed-counter PEBS events now resolve to empty constraints and are not scheduled/enabled, avoiding the warning and the broken guest PEBS path.Quoted 5 lines
Quoted 3 lines
This is safe because, in pebs_baseline-capable cases, PEBS constraint lookup already falls back to non-PEBS constraints when needed, and fixed-counter constraints are effectively shared there.
I am confused, this works outside of KVM? (It appears to work fine on my spr).. so removing this to fix some guest only issue seems wrong.
The reason that it works on bare metal is currently the constraint lookup would fallback into non-PEBS constraints if there is no matched entry in the PEBS constraints as long as the PEBS supports sampling on all counters including fixed counters (what the flag "
PMU_FL_PEBS_ALL" indicates), like the below code shows.
Quoted 5 lines
Quoted 1 line
```
Why the markdown nonsense? You're not an LLM, right? And you all know I'm reading this in mutt. Its just visual noise.
Anyway, please clarify the Changelog.
Sorry for the visual noise. I'm using the thunderbird mail client, not sure why the markdown doesn't work.
Sure. I would rewrite the change log and increase more details. Thanks.
Peter Zijlstra
maintainerRe: [Patch v2 7/9] perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS
Quoted 11 lines · Dapeng
Quoted 8 lines
Quoted 4 lines
including fixed counters (what the flag "
PMU_FL_PEBS_ALL" indicates), like the below code shows.```
Why the markdown nonsense? You're not an LLM, right? And you all know I'm reading this in mutt. Its just visual noise.
Anyway, please clarify the Changelog.
Sorry for the visual noise. I'm using the thunderbird mail client, not sure why the markdown doesn't work.
How is it supposed to 'work'? LKML is plain text, there is no markup.
From what I can tell, there is this thunderbug markdown plugin to more 'easily' write HTML mail, but that is explicitly forbidden on LKML.
On top of that, markdown is utter garbage, the idiot that 'invented' it explicitly ignored the pre-existing standard of: *bold*, /italics/, _underline_ and -strike-through-, a convention that has existed longer than markdown is even is a thing (heck it pre-dates computers!).
And yes, clearly I'm a dinosaur. But dammit, we stare at code and comments (and email) in text editors, not some web render bullshit.
</rant>
Re: [Patch v2 8/9] perf/core: Fix kernel register info leak via hardware skid
In reply to Peter ZijlstraQuoted 11 lines · Peter Zijlstra
Quoted 10 lines · Dapeng Mi
An unprivileged hardware perf event using exclude_kernel=1 can leak kernel register data to user space via
PERF_SAMPLE_REGS_INTRorPERF_SAMPLE_IP. Due to hardware skid, a PMI may trigger after the CPU has already entered kernel space (Ring 0), bypassing theperf_allow_kernel()privilege barrier.This security vulnerability is severely exacerbated by upcoming support for SIMD register sampling via XSAVES, which could expose sensitive kernel FPU states (such as active cryptographic keys).
Quoted 5 lines
Quoted 2 lines
Fix this by ensuring that sampled register data is dropped if the event's
exclude_kernelattribute is set but the PMI catches the CPU in kernel mode.There is history here, see for example:
https://lore.kernel.org/r/CAP045Ap8cMx6mzSgcQ3n3bnh_8GJuCp7_KZe_5ZTCR_K6cPTLw@mail.gmail.com
Thanks for the information. What a long discussion!
Quoted 9 lines
So your earlier patches also sanitize the branch stack, but not in generic code.
PHYS_ADDRalready requires privileges
ADDR comes from PEBS on Intel, and IBS on AMD (iirc) and should be reliable.So yeah, I suppose IP and
REGS_INTRare the big ones.
Yes. Besides IP and REGS_INTR, callchains could be another risk. Currently SAMPLE_CALLCHAIN doesn't explicitly require the perf_allow_kernel() check, which could leak to kernel callchains to user space. This would be fixed in next version.
As for the branch_stack, it should be reliable since it comes from LBR which already has correct CPL configuration.
Thanks.