perf/x86/amd/lbr: Compress branch stack in a single pass
Subsystems (2)
PATCH 1/1 perf/x86/amd/lbr: Compress branch stack in a single pass
Software filtering removes rejected branch records by shifting the entire tail of the array down by one for each dropped entry. With k invalid entries out of n, this costs O(k*n) copies. Switch to a two pointer compaction that walks the array once and packs the remaining entries towards the front, moving each entry at most once, for O(n) total. No functional change intended.
--- arch/x86/events/amd/lbr.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-)
arch/x86/events/amd/lbr.c +7 -10
@@ -141,17 +141,14 @@ static void amd_pmu_lbr_filter(void)return ;/* Remove all invalid entries */ -for (i =0 ; i < cpuc->lbr_stack.nr; ) { -if (!cpuc->lbr_entries[i].from) { - j = i; -while (++j < cpuc->lbr_stack.nr) - cpuc->lbr_entries[j -1 ] = cpuc->lbr_entries[j]; - cpuc->lbr_stack.nr--; -if (!cpuc->lbr_entries[i].from) -continue ; - } - i++; +for (i =0 , j =0 ; i < cpuc->lbr_stack.nr; i++) { +if (!cpuc->lbr_entries[i].from) +continue ; +if (j != i) + cpuc->lbr_entries[j] = cpuc->lbr_entries[i]; + j++; } + cpuc->lbr_stack.nr = j; }static const int lbr_spec_map[PERF_BR_SPEC_MAX ] = {
Signature · 1 line
2.53.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] bpf: info leak of uncleared LBR tail entries via branch snapshot API --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] bpf: info leak of uncleared LBR tail entries via branch snapshot API --
commit ed8227763c0f1dd19117eba0f7a4a321639e9390
Author: Sandipan Das <[email protected]>
perf/x86/amd/lbr: Compress branch stack in a single pass
This commit optimizes the software filtering of branch records by packing the remaining valid entries towards the front of the array in a single pass, rather than shifting the tail down for each dropped entry.
Quoted 16 lines
arch/x86/events/amd/lbr.c +0
-8@@ -141,17 +141,14 @@ static void amd_pmu_lbr_filter(void)return ;/* Remove all invalid entries */ -for (i =0 ; i < cpuc->lbr_stack.nr; ) { -if (!cpuc->lbr_entries[i].from) { - j = i; -while (++j < cpuc->lbr_stack.nr) - cpuc->lbr_entries[j -1 ] = cpuc->lbr_entries[j]; - cpuc->lbr_stack.nr--; -if (!cpuc->lbr_entries[i].from) -continue ;
arch/x86/events/amd/lbr.c static void amd_pmu_lbr_filter(void) · line 151
Quoted 10 lines
arch/x86/events/amd/lbr.c +7
-2- } - i++; +for (i =0 , j =0 ; i < cpuc->lbr_stack.nr; i++) { +if (!cpuc->lbr_entries[i].from) +continue ; +if (j != i) + cpuc->lbr_entries[j] = cpuc->lbr_entries[i]; + j++; } + cpuc->lbr_stack.nr = j;
[Severity: Medium]
Does this code leave stale entries at the tail of the cpuc->lbr_entries array?
When a privileged BPF program calls bpf_get_branch_snapshot(), it invokes perf_snapshot_branch_stack(), which calls amd_pmu_v2_snapshot_branch_stack().
Looking at amd_pmu_v2_snapshot_branch_stack() in arch/x86/events/amd/core.c, it unconditionally copies cnt (up to x86_pmu.lbr_nr) entries instead of cpuc->lbr_stack.nr:
memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
Since amd_pmu_lbr_filter() drops invalid entries and shrinks cpuc->lbr_stack.nr but doesn't clear the tail elements, will these uncleared tail elements (which could contain filtered branches or branches from other contexts) be leaked to BPF programs?
Additionally, if cpuc->lbr_users == 0, the LBRs are not read and the entire stale array from a previous task could be leaked.
Should the remaining entries from j to the original cpuc->lbr_stack.nr be zeroed out here, or should the snapshot APIs be updated to only copy cpuc->lbr_stack.nr entries?
Note that Intel's BPF snapshot implementation (__intel_pmu_snapshot_branch_stack in arch/x86/events/intel/core.c) and LBR filter (intel_pmu_lbr_filter in arch/x86/events/intel/lbr.c) appear to have a similar behavior where nr is shrunk without clearing tail elements.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1