Skip to contentSkip to conversation

perf/x86/amd/lbr: Limit branch snapshots to valid entries

PATCH Superseded Sandipan Das 4 participants 8 messages Oct 7 – Oct 14

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 perf/x86/amd/lbr: Limit branch snapshots to valid entries

The branch snapshot callback used by bpf_get_branch_snapshot() copies up to x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the last LBR read kept fewer branches. The entries past lbr_stack.nr are invalid, but they end up reaching BPF programs.

Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when amd_pmu_lbr_read() skips reading the registers, so that a stale count is never reused.

Fixes a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
---
 arch/x86/events/amd/core.c | 2 +-
 arch/x86/events/amd/lbr.c  | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/events/amd/core.c +1 -1
index 49b6b8fce566..bf631f60757c 100644
@@ -944,7 +944,7 @@ static int amd_pmu_v2_snapshot_branch_stack(struct perf_branch_entry *entries, u
 	cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	amd_pmu_lbr_read();
-	cnt = min(cnt, x86_pmu.lbr_nr);
+	cnt = min(cnt, cpuc->lbr_stack.nr);
 	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
 
 	amd_pmu_v2_enable_all(0);
arch/x86/events/amd/lbr.c +3 -1
index 1e1b2f08cb51..a00d3850c4a8 100644
@@ -165,8 +165,10 @@ void amd_pmu_lbr_read(void)
 	struct branch_entry entry;
 	int out = 0, idx, i;
 
-	if (!cpuc->lbr_users)
+	if (!cpuc->lbr_users) {
+		cpuc->lbr_stack.nr = 0;
 		return;
+	}
 
 	for (i = 0; i < x86_pmu.lbr_nr; i++) {
 		entry.from.full	= amd_pmu_lbr_get_from(i);
Signature · 1 line
2.53.0
#1 lore raw
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 5 lines

The branch snapshot callback used by bpf_get_branch_snapshot() copies up to x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the last LBR read kept fewer branches. The entries past lbr_stack.nr are invalid, but they end up reaching BPF programs.

Quoted 8 lines

Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when amd_pmu_lbr_read() skips reading the registers, so that a stale count is never reused.

Fixes a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
maintainer
#2 lore raw
In reply to Sandipan Das
Quoted 37 lines · Sandipan Das

The branch snapshot callback used by bpf_get_branch_snapshot() copies up to x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the last LBR read kept fewer branches. The entries past lbr_stack.nr are invalid, but they end up reaching BPF programs.

Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when amd_pmu_lbr_read() skips reading the registers, so that a stale count is never reused.

Fixes a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
---
 arch/x86/events/amd/core.c | 2 +-
 arch/x86/events/amd/lbr.c  | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/events/amd/core.c +1 -1
index 49b6b8fce566..bf631f60757c 100644
@@ -944,7 +944,7 @@ static int amd_pmu_v2_snapshot_branch_stack(struct perf_branch_entry *entries, u
 	cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	amd_pmu_lbr_read();
-	cnt = min(cnt, x86_pmu.lbr_nr);
+	cnt = min(cnt, cpuc->lbr_stack.nr);
 	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
 
 	amd_pmu_v2_enable_all(0);
arch/x86/events/amd/lbr.c +0 -0
index 1e1b2f08cb51..a00d3850c4a8 100644
@@ -165,8 +165,10 @@ void amd_pmu_lbr_read(void)
 	struct branch_entry entry;

arch/x86/events/amd/lbr.c void amd_pmu_lbr_read(void) · line 174

Quoted 10 lines
arch/x86/events/amd/lbr.c +3 -1
 	int out = 0, idx, i;
 
-	if (!cpuc->lbr_users)
+	if (!cpuc->lbr_users) {
+		cpuc->lbr_stack.nr = 0;
 		return;
+	}
 
 	for (i = 0; i < x86_pmu.lbr_nr; i++) {
 		entry.from.full	= amd_pmu_lbr_get_from(i);

You now seem to deviate from what the Intel code does. Please clarify.

#3 lore raw
Quoted 46 lines · Peter Zijlstra
Quoted 45 lines · Sandipan Das

The branch snapshot callback used by bpf_get_branch_snapshot() copies up to x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the last LBR read kept fewer branches. The entries past lbr_stack.nr are invalid, but they end up reaching BPF programs.

Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when amd_pmu_lbr_read() skips reading the registers, so that a stale count is never reused.

Fixes a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
---
 arch/x86/events/amd/core.c | 2 +-
 arch/x86/events/amd/lbr.c  | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/events/amd/core.c +1 -1
index 49b6b8fce566..bf631f60757c 100644
@@ -944,7 +944,7 @@ static int amd_pmu_v2_snapshot_branch_stack(struct perf_branch_entry *entries, u
 	cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	amd_pmu_lbr_read();
-	cnt = min(cnt, x86_pmu.lbr_nr);
+	cnt = min(cnt, cpuc->lbr_stack.nr);
 	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
 
 	amd_pmu_v2_enable_all(0);
arch/x86/events/amd/lbr.c +3 -1
index 1e1b2f08cb51..a00d3850c4a8 100644
@@ -165,8 +165,10 @@ void amd_pmu_lbr_read(void)
 	struct branch_entry entry;
 	int out = 0, idx, i;
 
-	if (!cpuc->lbr_users)
+	if (!cpuc->lbr_users) {
+		cpuc->lbr_stack.nr = 0;
 		return;
+	}
 
Quoted 4 lines

arch/x86/events/amd/lbr.c void amd_pmu_lbr_read(void) · line 174

Quoted 2 lines
arch/x86/events/amd/lbr.c +0 -0
 	for (i = 0; i < x86_pmu.lbr_nr; i++) {
 		entry.from.full	= amd_pmu_lbr_get_from(i);

You now seem to deviate from what the Intel code does. Please clarify.

Yes, it is a deviation. I thought of proposing the following but didn't proceed because I was not sure about the if condition in the second hunk which considers things other than cpuc->lbr_users for an early return.

arch/x86/events/intel/core.c +1 -1
index 0a34d674df59..5dfb9e9156e8 100644
@@ -2822,7 +2822,7 @@ __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
        struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
        intel_pmu_lbr_read();
-       cnt = min_t(unsigned int, cnt, x86_pmu.lbr_nr);
+       cnt = min_t(unsigned int, cnt, cpuc->lbr_stack.nr);
 
        memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
        intel_pmu_enable_all(0);
arch/x86/events/intel/lbr.c +3 -1
index 0b446d8912e4..6b070490f461 100644
@@ -1026,8 +1026,10 @@ void intel_pmu_lbr_read(void)
         * but this simple approach seems to work for now.
         */
        if (!cpuc->lbr_users || vlbr_exclude_host() ||
-           cpuc->lbr_users == cpuc->lbr_pebs_users)
+           cpuc->lbr_users == cpuc->lbr_pebs_users) {
+               cpuc->lbr_stack.nr = 0;
                return;
+       }
 
        x86_pmu.lbr_read(cpuc);
 
maintainer
#4 lore raw
Quoted 47 lines · Sandipan Das
Quoted 46 lines · Peter Zijlstra
Quoted 45 lines · Sandipan Das

The branch snapshot callback used by bpf_get_branch_snapshot() copies up to x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the last LBR read kept fewer branches. The entries past lbr_stack.nr are invalid, but they end up reaching BPF programs.

Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when amd_pmu_lbr_read() skips reading the registers, so that a stale count is never reused.

Fixes a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
---
 arch/x86/events/amd/core.c | 2 +-
 arch/x86/events/amd/lbr.c  | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/events/amd/core.c +1 -1
index 49b6b8fce566..bf631f60757c 100644
@@ -944,7 +944,7 @@ static int amd_pmu_v2_snapshot_branch_stack(struct perf_branch_entry *entries, u
 	cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	amd_pmu_lbr_read();
-	cnt = min(cnt, x86_pmu.lbr_nr);
+	cnt = min(cnt, cpuc->lbr_stack.nr);
 	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
 
 	amd_pmu_v2_enable_all(0);
arch/x86/events/amd/lbr.c +3 -1
index 1e1b2f08cb51..a00d3850c4a8 100644
@@ -165,8 +165,10 @@ void amd_pmu_lbr_read(void)
 	struct branch_entry entry;
 	int out = 0, idx, i;
 
-	if (!cpuc->lbr_users)
+	if (!cpuc->lbr_users) {
+		cpuc->lbr_stack.nr = 0;
 		return;
+	}
 
Quoted 8 lines
Quoted 4 lines

arch/x86/events/amd/lbr.c void amd_pmu_lbr_read(void) · line 174

Quoted 2 lines
arch/x86/events/amd/lbr.c +0 -0
 	for (i = 0; i < x86_pmu.lbr_nr; i++) {
 		entry.from.full	= amd_pmu_lbr_get_from(i);

You now seem to deviate from what the Intel code does. Please clarify.

Yes, it is a deviation. I thought of proposing the following but didn't proceed because I was not sure about the if condition in the second hunk which considers things other than cpuc->lbr_users for an early return.

Dapeng, can you look into this? I would really like to keep behaviour consistent where possible.

Quoted 29 lines
arch/x86/events/intel/core.c +1 -1
index 0a34d674df59..5dfb9e9156e8 100644
@@ -2822,7 +2822,7 @@ __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
        struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
        intel_pmu_lbr_read();
-       cnt = min_t(unsigned int, cnt, x86_pmu.lbr_nr);
+       cnt = min_t(unsigned int, cnt, cpuc->lbr_stack.nr);
 
        memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
        intel_pmu_enable_all(0);
arch/x86/events/intel/lbr.c +3 -1
index 0b446d8912e4..6b070490f461 100644
@@ -1026,8 +1026,10 @@ void intel_pmu_lbr_read(void)
         * but this simple approach seems to work for now.
         */
        if (!cpuc->lbr_users || vlbr_exclude_host() ||
-           cpuc->lbr_users == cpuc->lbr_pebs_users)
+           cpuc->lbr_users == cpuc->lbr_pebs_users) {
+               cpuc->lbr_stack.nr = 0;
                return;
+       }
 
        x86_pmu.lbr_read(cpuc);
 
#5 lore raw
Quoted 48 lines · Peter Zijlstra
Quoted 47 lines · Sandipan Das
Quoted 46 lines · Peter Zijlstra
Quoted 45 lines · Sandipan Das

The branch snapshot callback used by bpf_get_branch_snapshot() copies up to x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the last LBR read kept fewer branches. The entries past lbr_stack.nr are invalid, but they end up reaching BPF programs.

Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when amd_pmu_lbr_read() skips reading the registers, so that a stale count is never reused.

Fixes a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
---
 arch/x86/events/amd/core.c | 2 +-
 arch/x86/events/amd/lbr.c  | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/events/amd/core.c +1 -1
index 49b6b8fce566..bf631f60757c 100644
@@ -944,7 +944,7 @@ static int amd_pmu_v2_snapshot_branch_stack(struct perf_branch_entry *entries, u
 	cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	amd_pmu_lbr_read();
-	cnt = min(cnt, x86_pmu.lbr_nr);
+	cnt = min(cnt, cpuc->lbr_stack.nr);
 	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
 
 	amd_pmu_v2_enable_all(0);
arch/x86/events/amd/lbr.c +3 -1
index 1e1b2f08cb51..a00d3850c4a8 100644
@@ -165,8 +165,10 @@ void amd_pmu_lbr_read(void)
 	struct branch_entry entry;
 	int out = 0, idx, i;
 
-	if (!cpuc->lbr_users)
+	if (!cpuc->lbr_users) {
+		cpuc->lbr_stack.nr = 0;
 		return;
+	}
 
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

arch/x86/events/amd/lbr.c void amd_pmu_lbr_read(void) · line 174

Quoted 2 lines
arch/x86/events/amd/lbr.c +0 -0
 	for (i = 0; i < x86_pmu.lbr_nr; i++) {
 		entry.from.full	= amd_pmu_lbr_get_from(i);

You now seem to deviate from what the Intel code does. Please clarify.

Yes, it is a deviation. I thought of proposing the following but didn't proceed because I was not sure about the if condition in the second hunk which considers things other than cpuc->lbr_users for an early return.

Dapeng, can you look into this? I would really like to keep behaviour consistent where possible.

Sure.

Quoted 12 lines
Quoted 12 lines
arch/x86/events/intel/core.c +1 -1
index 0a34d674df59..5dfb9e9156e8 100644
@@ -2822,7 +2822,7 @@ __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
        struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
        intel_pmu_lbr_read();
-       cnt = min_t(unsigned int, cnt, x86_pmu.lbr_nr);
+       cnt = min_t(unsigned int, cnt, cpuc->lbr_stack.nr);
 
Quoted 13 lines

arch/x86/events/intel/lbr.c void intel_pmu_lbr_read(void) · line 1030

Quoted 13 lines
arch/x86/events/intel/core.c +0 -0
        memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
        intel_pmu_enable_all(0);
arch/x86/events/intel/lbr.c +2 -1
index 0b446d8912e4..6b070490f461 100644
@@ -1026,8 +1026,10 @@ void intel_pmu_lbr_read(void)
         * but this simple approach seems to work for now.
         */
        if (!cpuc->lbr_users || vlbr_exclude_host() ||
-           cpuc->lbr_users == cpuc->lbr_pebs_users)
+           cpuc->lbr_users == cpuc->lbr_pebs_users) {
+               cpuc->lbr_stack.nr = 0;

I checked the code, and it looks safe to clear cpuc->lbr_stack.nr here. For PEBS events with the LBR group, the LBR stack is populated again for each PEBS record, so this should not affect the LBR trace for PEBS events.

But I’m not fully sure if it's intentional for the bpf_get_branch_snapshot() helper. Clearing lbr_stack.nr to 0 prevents the BPF helper from reading LBR data from PEBS although the LBR data could be stale since it comes from the past PEBS record instead of the runtime hardware LBR state. That seems reasonable and correct, but the side effect is that the BPF helper can't read the LBR stack when all LBR events are PEBS events. Not sure if it's intentional and expected.

I’d like to run some tests for this change tomorrow to confirm the behavior. Thanks.

Quoted 5 lines
Quoted 5 lines
arch/x86/events/intel/lbr.c +1 -0
                return;
+       }
 
        x86_pmu.lbr_read(cpuc);
 
#6 lore raw
Quoted 71 lines · Dapeng
Quoted 48 lines · Peter Zijlstra
Quoted 47 lines · Sandipan Das
Quoted 46 lines · Peter Zijlstra
Quoted 45 lines · Sandipan Das

The branch snapshot callback used by bpf_get_branch_snapshot() copies up to x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the last LBR read kept fewer branches. The entries past lbr_stack.nr are invalid, but they end up reaching BPF programs.

Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when amd_pmu_lbr_read() skips reading the registers, so that a stale count is never reused.

Fixes a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
---
 arch/x86/events/amd/core.c | 2 +-
 arch/x86/events/amd/lbr.c  | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/events/amd/core.c +1 -1
index 49b6b8fce566..bf631f60757c 100644
@@ -944,7 +944,7 @@ static int amd_pmu_v2_snapshot_branch_stack(struct perf_branch_entry *entries, u
 	cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	amd_pmu_lbr_read();
-	cnt = min(cnt, x86_pmu.lbr_nr);
+	cnt = min(cnt, cpuc->lbr_stack.nr);
 	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
 
 	amd_pmu_v2_enable_all(0);
arch/x86/events/amd/lbr.c +3 -1
index 1e1b2f08cb51..a00d3850c4a8 100644
@@ -165,8 +165,10 @@ void amd_pmu_lbr_read(void)
 	struct branch_entry entry;
 	int out = 0, idx, i;
 
-	if (!cpuc->lbr_users)
+	if (!cpuc->lbr_users) {
+		cpuc->lbr_stack.nr = 0;
 		return;
+	}
 
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

arch/x86/events/amd/lbr.c void amd_pmu_lbr_read(void) · line 174

Quoted 2 lines
arch/x86/events/amd/lbr.c +0 -0
 	for (i = 0; i < x86_pmu.lbr_nr; i++) {
 		entry.from.full	= amd_pmu_lbr_get_from(i);

You now seem to deviate from what the Intel code does. Please clarify.

Yes, it is a deviation. I thought of proposing the following but didn't proceed because I was not sure about the if condition in the second hunk which considers things other than cpuc->lbr_users for an early return.

Dapeng, can you look into this? I would really like to keep behaviour consistent where possible.

Sure.

Quoted 11 lines
Quoted 11 lines
arch/x86/events/intel/core.c +1 -1
index 0a34d674df59..5dfb9e9156e8 100644
@@ -2822,7 +2822,7 @@ __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
        struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
        intel_pmu_lbr_read();
-       cnt = min_t(unsigned int, cnt, x86_pmu.lbr_nr);
+       cnt = min_t(unsigned int, cnt, cpuc->lbr_stack.nr);
 
Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
arch/x86/events/intel/core.c +0 -0
        memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
        intel_pmu_enable_all(0);
arch/x86/events/intel/lbr.c +0 -1
index 0b446d8912e4..6b070490f461 100644
--- a/arch/x86/events/intel/lbr.c
Quoted 8 lines

arch/x86/events/intel/lbr.c void intel_pmu_lbr_read(void) · line 1030

Quoted 8 lines

+++ b/arch/x86/events/intel/lbr.c

arch/x86/events/intel/lbr.c +2 -1
@@ -1026,8 +1026,10 @@ void intel_pmu_lbr_read(void)
         * but this simple approach seems to work for now.
         */
        if (!cpuc->lbr_users || vlbr_exclude_host() ||
-           cpuc->lbr_users == cpuc->lbr_pebs_users)
+           cpuc->lbr_users == cpuc->lbr_pebs_users) {
+               cpuc->lbr_stack.nr = 0;

I checked the code, and it looks safe to clear cpuc->lbr_stack.nr here. For PEBS events with the LBR group, the LBR stack is populated again for each PEBS record, so this should not affect the LBR trace for PEBS events.

I test the changes on SPR for the 3 cases, pure LBR PEBS sampling, pure LBR non-PEBS sampling and the PEBS/non-PEBS mixed LBR sampling. No issue is found for the LBR stack sampling.

Sandipan, would you mind to post the fixes for Intel LBR with the AMD fixes together? Thanks.

Quoted 18 lines

But I’m not fully sure if it's intentional for the bpf_get_branch_snapshot() helper. Clearing lbr_stack.nr to 0 prevents the BPF helper from reading LBR data from PEBS although the LBR data could be stale since it comes from the past PEBS record instead of the runtime hardware LBR state. That seems reasonable and correct, but the side effect is that the BPF helper can't read the LBR stack when all LBR events are PEBS events. Not sure if it's intentional and expected.

I’d like to run some tests for this change tomorrow to confirm the behavior. Thanks.

Quoted 5 lines
Quoted 5 lines
arch/x86/events/intel/lbr.c +1 -0
                return;
+       }
 
        x86_pmu.lbr_read(cpuc);
 
#7 lore raw
Quoted 90 lines · Dapeng
Quoted 71 lines · Dapeng
Quoted 48 lines · Peter Zijlstra
Quoted 47 lines · Sandipan Das
Quoted 46 lines · Peter Zijlstra
Quoted 45 lines · Sandipan Das

The branch snapshot callback used by bpf_get_branch_snapshot() copies up to x86_pmu.lbr_nr entries from cpuc->lbr_entries[], even when the last LBR read kept fewer branches. The entries past lbr_stack.nr are invalid, but they end up reaching BPF programs.

Limit copying to lbr_stack.nr entries and also reset lbr_stack.nr when amd_pmu_lbr_read() skips reading the registers, so that a stale count is never reused.

Fixes a4d18112e531 ("perf/x86/amd: Support capturing LBR from software events")
---
 arch/x86/events/amd/core.c | 2 +-
 arch/x86/events/amd/lbr.c  | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/events/amd/core.c +1 -1
index 49b6b8fce566..bf631f60757c 100644
@@ -944,7 +944,7 @@ static int amd_pmu_v2_snapshot_branch_stack(struct perf_branch_entry *entries, u
 	cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	amd_pmu_lbr_read();
-	cnt = min(cnt, x86_pmu.lbr_nr);
+	cnt = min(cnt, cpuc->lbr_stack.nr);
 	memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
 
 	amd_pmu_v2_enable_all(0);
arch/x86/events/amd/lbr.c +3 -1
index 1e1b2f08cb51..a00d3850c4a8 100644
@@ -165,8 +165,10 @@ void amd_pmu_lbr_read(void)
 	struct branch_entry entry;
 	int out = 0, idx, i;
 
-	if (!cpuc->lbr_users)
+	if (!cpuc->lbr_users) {
+		cpuc->lbr_stack.nr = 0;
 		return;
+	}
 
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

arch/x86/events/amd/lbr.c void amd_pmu_lbr_read(void) · line 174

Quoted 2 lines
arch/x86/events/amd/lbr.c +0 -0
 	for (i = 0; i < x86_pmu.lbr_nr; i++) {
 		entry.from.full	= amd_pmu_lbr_get_from(i);

You now seem to deviate from what the Intel code does. Please clarify.

Yes, it is a deviation. I thought of proposing the following but didn't proceed because I was not sure about the if condition in the second hunk which considers things other than cpuc->lbr_users for an early return.

Dapeng, can you look into this? I would really like to keep behaviour consistent where possible.

Sure.

Quoted 11 lines
Quoted 11 lines
arch/x86/events/intel/core.c +1 -1
index 0a34d674df59..5dfb9e9156e8 100644
@@ -2822,7 +2822,7 @@ __intel_pmu_snapshot_branch_stack(struct perf_branch_entry *entries,
        struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
        intel_pmu_lbr_read();
-       cnt = min_t(unsigned int, cnt, x86_pmu.lbr_nr);
+       cnt = min_t(unsigned int, cnt, cpuc->lbr_stack.nr);
 
Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
arch/x86/events/intel/core.c +0 -0
        memcpy(entries, cpuc->lbr_entries, sizeof(struct perf_branch_entry) * cnt);
        intel_pmu_enable_all(0);
arch/x86/events/intel/lbr.c +0 -1
index 0b446d8912e4..6b070490f461 100644
--- a/arch/x86/events/intel/lbr.c
Quoted 8 lines

arch/x86/events/intel/lbr.c void intel_pmu_lbr_read(void) · line 1030

Quoted 8 lines

+++ b/arch/x86/events/intel/lbr.c

arch/x86/events/intel/lbr.c +2 -1
@@ -1026,8 +1026,10 @@ void intel_pmu_lbr_read(void)
         * but this simple approach seems to work for now.
         */
        if (!cpuc->lbr_users || vlbr_exclude_host() ||
-           cpuc->lbr_users == cpuc->lbr_pebs_users)
+           cpuc->lbr_users == cpuc->lbr_pebs_users) {
+               cpuc->lbr_stack.nr = 0;

I checked the code, and it looks safe to clear cpuc->lbr_stack.nr here. For PEBS events with the LBR group, the LBR stack is populated again for each PEBS record, so this should not affect the LBR trace for PEBS events.

Quoted 7 lines

I test the changes on SPR for the 3 cases, pure LBR PEBS sampling, pure LBR non-PEBS sampling and the PEBS/non-PEBS mixed LBR sampling. No issue is found for the LBR stack sampling.

Sandipan, would you mind to post the fixes for Intel LBR with the AMD fixes together? Thanks.

Will post the fixes together. Thanks for testing this.

Quoted 19 lines
Quoted 19 lines

But I’m not fully sure if it's intentional for the bpf_get_branch_snapshot() helper. Clearing lbr_stack.nr to 0 prevents the BPF helper from reading LBR data from PEBS although the LBR data could be stale since it comes from the past PEBS record instead of the runtime hardware LBR state. That seems reasonable and correct, but the side effect is that the BPF helper can't read the LBR stack when all LBR events are PEBS events. Not sure if it's intentional and expected.

I’d like to run some tests for this change tomorrow to confirm the behavior. Thanks.

Quoted 5 lines
Quoted 5 lines
arch/x86/events/intel/lbr.c +1 -0
                return;
+       }
 
        x86_pmu.lbr_read(cpuc);