perf arm-spe: extend Arm SPE test script with regression testing
cover perf arm-spe: extend Arm SPE test script with regression testing
This patch extends the test_arm_spe.sh test script in order to tests for regressions in the decoding flow of Arm SPE samples.
At the time of writing, perf-inject is not working for Arm SPE. In order to test/apply this patch, [1] needs to be resolved first.
[1] https://lore.kernel.org/all/[email protected]/T/#m06a8cc68f7b971b6eb10d11aa6e7385a7f6be29e
German Gomez (1):
perf arm-spe: extend Arm SPE test script with regression testing
tools/perf/tests/shell/test_arm_spe.sh | 32 ++++++++++++++++++++++++ tools/perf/tests/shell/test_arm_spe.tgz | Bin 0 -> 253541 bytes 2 files changed, 32 insertions(+) create mode 100644 tools/perf/tests/shell/test_arm_spe.tgz
Signature · 1 line
2.25.1
PATCH 1/1 perf arm-spe: extend Arm SPE test script with regression testing
Extend the test_arm_spe.sh script to test for regressions in the decoding flow of Arm SPE samples. In order to support the tests, a set of perf.data files has been generated offline and is being hosted under tools/perf/tests/shell/test_arm_spe.tgz:
(See perf report --header-only to see the actual commands used)
- test_arm_spe.tgz
`- arm_spe_decode_contextidr.data
`- arm_spe_decode_contextidr.data.inject
`- arm_spe_decode_switch_events.data
`- arm_spe_decode_switch_events.data.inject
The purpose of each file is as follows:
- arm_spe_decode_contextidr.data is used to test decoding when SPE traces contain CONTEXT packets (for pid/tid tracking).
- arm_spe_decode_switch_events.data is used to test decoding when SPE traces _do not_ contain CONTEXT packets.
- arm_spe_*.data.inject were generated using perf-inject and the above files as inputs.
During testing, events are decoded from the arm_spe_*.data files, and the result are compared against the hosted arm_spe_*.data.inject files. If there are any differences, the test fails.
--- tools/perf/tests/shell/test_arm_spe.sh | 32 ++++++++++++++++++++++++ tools/perf/tests/shell/test_arm_spe.tgz | Bin 0 -> 253541 bytes 2 files changed, 32 insertions(+) create mode 100644 tools/perf/tests/shell/test_arm_spe.tgz
tools/perf/tests/shell/test_arm_spe.sh +32 -0
@@ -18,11 +18,13 @@ skip_if_no_arm_spe_event() { skip_if_no_arm_spe_event || exit 2 perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX) +scratchdir=$(mktemp -d) glb_err=0 cleanup_files() { rm -f ${perfdata} + rm -rf ${scratchdir} exit $glb_err }
@@ -85,5 +87,35 @@ arm_spe_snapshot_test() { arm_spe_report "SPE snapshot testing" $err } +arm_spe_decoder_regression_test() { + echo "Testing event decoding $scratchdir" + + # unpack test data + tar xf ./tests/shell/test_arm_spe.tgz -C $scratchdir + + # generate md5 file of ground truth files + for file in $scratchdir/arm_spe_*.data.inject; do + filename=$(basename $file) + perf script -f --fields event,ip,pid,tid -i $file > $scratchdir/$filename.script + done + + md5sum $scratchdir/*.script > $scratchdir/hash.md5 + rm -f $scratchdir/*.script + + # run Arm SPE decoder + for file in $scratchdir/arm_spe_*.data; do + filename=$(basename $file) + perf inject -f --itrace --strip -i $file -o $scratchdir/$filename.inject + perf script -f --fields event,ip,pid,tid -i $scratchdir/$filename.inject > $scratchdir/$filename.inject.script + done + + # compare decoder output + md5sum -c $scratchdir/hash.md5 + + err=$? + arm_spe_report "SPE event decoding testing" $err +} + arm_spe_snapshot_test +arm_spe_decoder_regression_test exit $glb_err
tools/perf/tests/shell/test_arm_spe.tgz new file binary +0 -0
Signature · 1 line
2.25.1
Quoted 6 lines · German Gomez
Extend the
test_arm_spe.shscript to test for regressions in the decoding flow of Arm SPE samples. In order to support the tests, a set of perf.data files has been generated offline and is being hosted undertools/perf/tests/shell/test_arm_spe.tgz:(See perf report --header-only to see the actual commands used)
Looking back at this patch, I don't think it can be applied as is. I notice that the docs for the --itrace argument aren't up to date (it says the default is --itrace=ibxwpe but lately it has changed ([1] & [2]). The value of --itrace should be locked in the test script.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/perf/Documentation/itrace.txt?id=136057256686de39cc3a07c2e39ef6bc43003ff6#n26
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/perf/util/auxtrace.c?id=136057256686de39cc3a07c2e39ef6bc43003ff6#n1335
Quoted 8 lines
- test_arm_spe.tgz
`- arm_spe_decode_contextidr.data
`- arm_spe_decode_contextidr.data.inject
`- arm_spe_decode_switch_events.data
`- arm_spe_decode_switch_events.data.inject[...]
Hi German,
Quoted 4 lines · German Gomez
Extend the
test_arm_spe.shscript to test for regressions in the decoding flow of Arm SPE samples. In order to support the tests, a set of perf.data files has been generated offline and is being hosted undertools/perf/tests/shell/test_arm_spe.tgz:
Seems to me it's not a good idea to upstream perf data binaries into the mainline kernel. I understood you want to test the perf data with different context tracing modes (using contextidr vs switch events), since these two different modes we cannot capture them with the same kernel Image, I think this is the main reason you upstreamed the perf data binaries in this patch.
On the other hand, like CoreSight smoke testing, by default we can give priority for testing root PID namespace, so you could do the test with below commands, which is assumed that tracing PID in contextidr:
perf record -e arm_spe_0// -- test_program
perf report
perf script
Then, for testing non-root PID namespace, can we use the command "unshare" to create a namespace and run perf tool in a non-root PID namespace? In this way, you could record Arm SPE trace data and decode it on the fly. Finally we can avoid to upstream perf data binaries.
Thanks,
Leo
Hi Leo,
Thanks for the comments.
Quoted 7 lines · Leo Yan
Hi German,
Quoted 4 lines · German Gomez
Extend the
test_arm_spe.shscript to test for regressions in the decoding flow of Arm SPE samples. In order to support the tests, a set of perf.data files has been generated offline and is being hosted undertools/perf/tests/shell/test_arm_spe.tgz:
Quoted 6 lines
Seems to me it's not a good idea to upstream perf data binaries into the mainline kernel. I understood you want to test the perf data with different context tracing modes (using contextidr vs switch events), since these two different modes we cannot capture them with the same kernel Image, I think this is the main reason you upstreamed the perf data binaries in this patch.
For this patch I wanted to test the decoding of SPE events, given fixed inputs, in order to lock the current implementation of the decoder.
Quoted 9 lines
On the other hand, like CoreSight smoke testing, by default we can give priority for testing root PID namespace, so you could do the test with below commands, which is assumed that tracing PID in contextidr:
perf record -e arm_spe_0// --
test_program
perf report
perf script
Quoted 5 lines
Then, for testing non-root PID namespace, can we use the command "unshare" to create a namespace and run perf tool in a non-root PID namespace? In this way, you could record Arm SPE trace data and decode it on the fly. Finally we can avoid to upstream perf data binaries.
I think adding additional smoke testing is a good idea. We could also check if CONTEXTIDR is enabled in the config (either /boot/config... or /proc/config.gz).
Thanks,
German
Quoted 3 lines
Thanks,
Leo