Skip to contentSkip to conversation

perf test: "object code reading" test fixes

PATCH Superseded James Clark 3 participants 8 messages Apr 9, 2024 – Apr 10, 2024 perf events

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover perf test: "object code reading" test fixes

A few fixes around the object code reading test. The third patch appears to be unrelated, but in this case the data symbol test was broken on Arm N1 by the first commit.

James Clark (3):
perf tests: Apply attributes to all events in object code reading test perf map: Remove kernel map before updating start and end addresses perf tests: Skip "test data symbol" on Neoverse N1

 tools/perf/tests/code-reading.c            | 10 +++++-----
 tools/perf/tests/shell/test_data_symbol.sh |  6 ++++++
 tools/perf/util/machine.c                  |  2 +-
 3 files changed, 12 insertions(+), 6 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH 1/3 perf tests: Apply attributes to all events in object code reading test

PERF_PMU_CAP_EXTENDED_HW_TYPE results in multiple events being opened on heterogeneous systems. Currently this test only sets its required attributes on the first event. Not disabling enable_on_exec on the other events causes the test to fail because the forked objdump processes are sampled. No tracking event is opened so Perf only knows about its own mappings causing the objdump samples to give the following error:

  $ perf test -vvv "object code reading"

Reading object code for memory address: 0xffff9aaa55ec thread__find_map failed
---- end(-1) ----
24: Object code reading : FAILED!

Fixes 251aa040244a ("perf parse-events: Wildcard most "numeric" events")
---
 tools/perf/tests/code-reading.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
tools/perf/tests/code-reading.c +5 -5
index 7a3a7bbbec71..29d2f3ee4e10 100644
@@ -637,11 +637,11 @@ static int do_test_code_reading(bool try_kcore)
 
 		evlist__config(evlist, &opts, NULL);
 
-		evsel = evlist__first(evlist);
-
-		evsel->core.attr.comm = 1;
-		evsel->core.attr.disabled = 1;
-		evsel->core.attr.enable_on_exec = 0;
+		evlist__for_each_entry(evlist, evsel) {
+			evsel->core.attr.comm = 1;
+			evsel->core.attr.disabled = 1;
+			evsel->core.attr.enable_on_exec = 0;
+		}
 
 		ret = evlist__open(evlist);
 		if (ret < 0) {
Signature · 1 line
2.34.1
#2 lore raw

PATCH 2/3 perf map: Remove kernel map before updating start and end addresses

In reply to James Clark

In a debug build there is validation that mmap lists are sorted when taking a lock. In machine__update_kernel_mmap() the start and end addresses are updated resulting in an unsorted list before the map is removed from the list. When the map is removed, the lock is taken which triggers the validation and the failure:

  $ perf test "object code reading"
  --- start ---
  perf: util/maps.c:88: check_invariants: Assertion `map__start(prev) <= map__start(map)' failed.
  Aborted

Fix it by updating the addresses after removal, but before insertion. The bug depends on the ordering and type of debug info on the system and doesn't reproduce everywhere.

Fixes 659ad3492b91 ("perf maps: Switch from rbtree to lazily sorted array for addresses")
---
 tools/perf/util/machine.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/machine.c +1 -1
index 5eb9044bc223..a26c8bea58d0 100644
@@ -1549,8 +1549,8 @@ static int machine__update_kernel_mmap(struct machine *machine,
 	updated = map__get(orig);
 
 	machine->vmlinux_map = updated;
-	machine__set_kernel_mmap(machine, start, end);
 	maps__remove(machine__kernel_maps(machine), orig);
+	machine__set_kernel_mmap(machine, start, end);
 	err = maps__insert(machine__kernel_maps(machine), updated);
 	map__put(orig);
 
Signature · 1 line
2.34.1
#3 lore raw

PATCH 3/3 perf tests: Skip "test data symbol" on Neoverse N1

In reply to James Clark

To prevent anyone from seeing a test failure appear as a regression and thinking that it was caused by their code change, just skip the test on N1.

It can be caused by any unrelated change that shifts the loop into an unfortunate position in the Perf binary which is almost impossible to debug as the root cause of the test failure. Ultimately it's caused by the referenced errata.

Fixes 60abedb8aa90 ("perf test: Introduce script for data symbol testing")
---
 tools/perf/tests/shell/test_data_symbol.sh | 6 ++++++
 1 file changed, 6 insertions(+)
tools/perf/tests/shell/test_data_symbol.sh +6 -0
index 3dfa91832aa8..ffc641d00aa4 100755
@@ -16,6 +16,12 @@ skip_if_no_mem_event() {
 	return 2
 }
 
+# Skip on Arm N1 due to errata 1694299. Bias exists in SPE sampling
+# which can cause the load and store instructions to be skipped
+# entirely. This comes and goes randomly depending on the offset the
+# linker places the datasym loop at in the Perf binary.
+lscpu | grep -q "Neoverse-N1" && exit 2
+
 skip_if_no_mem_event || exit 2
 
 skip_test_missing_symbol buf1
Signature · 1 line
2.34.1
reviewer
#4 lore raw

Re: [PATCH 3/3] perf tests: Skip "test data symbol" on Neoverse N1

Quoted 5 lines · James Clark

To prevent anyone from seeing a test failure appear as a regression and thinking that it was caused by their code change, just skip the test on N1.

Quoted 7 lines

It can be caused by any unrelated change that shifts the loop into an unfortunate position in the Perf binary which is almost impossible to debug as the root cause of the test failure. Ultimately it's caused by the referenced errata.

Fixes 60abedb8aa90 ("perf test: Introduce script for data symbol testing")

This change makes me sad :-( Is there no hope of aligning the loop? We have little enough testing coverage for memory events and even precise events on ARM that anything take away testing coverage feels like we should try to do better.

Which models are we losing coverage for, presumably neoverse-n1 but what about neoverse-v1 and neoverse-n2-v2?

If aligning the loop doesn't work, could we use objdump and check its alignment skipping when it is off? Or run the test and turn fails to skip on neoverse-n1 - so we get some coverage testing.

It would also be nice if the change didn't add a dependency on lscpu for the sake of parsing /proc/cpuinfo, I see another arm test already did this test_arm_callgraph_fp.sh - that case looks like it should be using uname.

Thanks,
Ian

Quoted 24 lines
---
 tools/perf/tests/shell/test_data_symbol.sh | 6 ++++++
 1 file changed, 6 insertions(+)
tools/perf/tests/shell/test_data_symbol.sh +6 -1
index 3dfa91832aa8..ffc641d00aa4 100755
@@ -16,6 +16,12 @@ skip_if_no_mem_event() {
        return 2
 }
 
+# Skip on Arm N1 due to errata 1694299. Bias exists in SPE sampling
+# which can cause the load and store instructions to be skipped
+# entirely. This comes and goes randomly depending on the offset the
+# linker places the datasym loop at in the Perf binary.
+lscpu | grep -q "Neoverse-N1" && exit 2
+
 skip_if_no_mem_event || exit 2
 
 skip_test_missing_symbol buf1
--

2.34.1

maintainer
#5 lore raw

Re: [PATCH 3/3] perf tests: Skip "test data symbol" on Neoverse N1

Hi Ian and James,

Quoted 12 lines · Ian Rogers
Quoted 10 lines · James Clark

To prevent anyone from seeing a test failure appear as a regression and thinking that it was caused by their code change, just skip the test on N1.

It can be caused by any unrelated change that shifts the loop into an unfortunate position in the Perf binary which is almost impossible to debug as the root cause of the test failure. Ultimately it's caused by the referenced errata.

Quoted 7 lines
Quoted 2 lines
Fixes 60abedb8aa90 ("perf test: Introduce script for data symbol testing")

This change makes me sad :-( Is there no hope of aligning the loop? We have little enough testing coverage for memory events and even precise events on ARM that anything take away testing coverage feels like we should try to do better.

Can we just add some noise in the loop?

Thanks,
Namhyung

(quoted diff) +0 -0

b/tools/perf/tests/workloads/datasym.c
index ddd40bc63448..e2514bf393cd 100644

tools/perf/tests/workloads/datasym.c +0 -0
@@ -16,6 +16,8 @@ static int datasym(int argc __maybe_unused, const

char **argv __maybe_unused)

 {
        for (;;) {
                buf1.data1++;
+               if ((buf1.data1 % 100129) == 0)
+                       buf1.data1++;
                buf1.data2 += buf1.data1;
        }
        return 0;
#6 lore raw

Re: [PATCH 3/3] perf tests: Skip "test data symbol" on Neoverse N1

Quoted 23 lines · Namhyung Kim

Hi Ian and James,

Quoted 12 lines · Ian Rogers
Quoted 10 lines · James Clark

To prevent anyone from seeing a test failure appear as a regression and thinking that it was caused by their code change, just skip the test on N1.

It can be caused by any unrelated change that shifts the loop into an unfortunate position in the Perf binary which is almost impossible to debug as the root cause of the test failure. Ultimately it's caused by the referenced errata.

Quoted 7 lines
Quoted 2 lines
Fixes 60abedb8aa90 ("perf test: Introduce script for data symbol testing")

This change makes me sad :-( Is there no hope of aligning the loop? We have little enough testing coverage for memory events and even precise events on ARM that anything take away testing coverage feels like we should try to do better.

Quoted 5 lines

Can we just add some noise in the loop?

Thanks,
Namhyung

Yes that would probably work. I decided to skip rather than touch the test because I didn't want the errata on one specific core to affect testing everywhere else.

But if we don't think that it's too hacky to include that in the test then I can add it. To be honest maybe it makes the test more "realistic" because a very short infinite loop doesn't really represent a real workload.

Quoted 17 lines
tools/perf/tests/workloads/datasym.c +2 -0
index ddd40bc63448..e2514bf393cd 100644
@@ -16,6 +16,8 @@ static int datasym(int argc __maybe_unused, const
 char **argv __maybe_unused)
 {
        for (;;) {
                buf1.data1++;
+               if ((buf1.data1 % 100129) == 0)
+                       buf1.data1++;
                buf1.data2 += buf1.data1;
        }
        return 0;
#7 lore raw

Re: [PATCH 3/3] perf tests: Skip "test data symbol" on Neoverse N1

In reply to Ian Rogers
Quoted 22 lines · Ian Rogers
Quoted 5 lines · James Clark

To prevent anyone from seeing a test failure appear as a regression and thinking that it was caused by their code change, just skip the test on N1.

Quoted 7 lines

It can be caused by any unrelated change that shifts the loop into an unfortunate position in the Perf binary which is almost impossible to debug as the root cause of the test failure. Ultimately it's caused by the referenced errata.

Fixes 60abedb8aa90 ("perf test: Introduce script for data symbol testing")

This change makes me sad :-( Is there no hope of aligning the loop? We have little enough testing coverage for memory events and even precise events on ARM that anything take away testing coverage feels like we should try to do better.

Which models are we losing coverage for, presumably neoverse-n1 but what about neoverse-v1 and neoverse-n2-v2?

Quoted 9 lines

If aligning the loop doesn't work, could we use objdump and check its alignment skipping when it is off? Or run the test and turn fails to skip on neoverse-n1 - so we get some coverage testing.

It would also be nice if the change didn't add a dependency on lscpu for the sake of parsing /proc/cpuinfo, I see another arm test already did this test_arm_callgraph_fp.sh - that case looks like it should be using uname.

I'll make the change to add the noise to the loop, which will drop this lscpu addition. And I'll fix up test_arm_callgraph_fp.sh while I'm at it.

Quoted 27 lines

Thanks,
Ian

Quoted 24 lines
---
 tools/perf/tests/shell/test_data_symbol.sh | 6 ++++++
 1 file changed, 6 insertions(+)
tools/perf/tests/shell/test_data_symbol.sh +6 -1
index 3dfa91832aa8..ffc641d00aa4 100755
@@ -16,6 +16,12 @@ skip_if_no_mem_event() {
        return 2
 }
 
+# Skip on Arm N1 due to errata 1694299. Bias exists in SPE sampling
+# which can cause the load and store instructions to be skipped
+# entirely. This comes and goes randomly depending on the offset the
+# linker places the datasym loop at in the Perf binary.
+lscpu | grep -q "Neoverse-N1" && exit 2
+
 skip_if_no_mem_event || exit 2
 
 skip_test_missing_symbol buf1
--

2.34.1