kselftest/arm64: Add coresight test
PATCH 1/1 kselftest/arm64: Add coresight test
Add a script to test the coresight functionalities by performing the perf test 'coresight'.
The script checks the prerequisites and launches a bundle of Coresight-related tests with a 180-second timeout.
--- tools/testing/selftests/arm64/Makefile | 2 +- .../selftests/arm64/coresight/Makefile | 5 +++ .../selftests/arm64/coresight/coresight.sh | 40 +++++++++++++++++++ .../selftests/arm64/coresight/settings | 1 + 4 files changed, 47 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/arm64/coresight/Makefile create mode 100755 tools/testing/selftests/arm64/coresight/coresight.sh create mode 100644 tools/testing/selftests/arm64/coresight/settings
tools/testing/selftests/arm64/Makefile +1 -1
@@ -4,7 +4,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),aarch64 arm64)) -ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi +ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi coresight else ARM64_SUBTARGETS := endif
tools/testing/selftests/arm64/coresight/Makefile new file +5 -0
@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_PROGS := coresight.sh + +include ../../lib.mk
tools/testing/selftests/arm64/coresight/coresight.sh new file +40 -0
@@ -0,0 +1,40 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +skip() { + echo "SKIP: $1" + exit 4 +} + +fail() { + echo "FAIL: $1" + exit 255 +} + +is_coresight_supported() { + if [ -d "/sys/bus/coresight/devices" ]; then + return 0 + fi + return 255 +} + +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + [ "$(id -u)" -ne 0 ] && \ + skip "this test must be run as root." + which perf >/dev/null 2>&1 || \ + skip "perf is not installed." + perf test list 2>&1 | grep -qi 'coresight' || \ + skip "perf doesn't support testing coresight." + is_coresight_supported || \ + skip "coresight is not supported." + + cmd_output=$(perf test -vv 'coresight' 2>&1) + perf_ret=$? + + if [ $perf_ret -ne 0 ]; then + fail "perf command returns non-zero." + elif [[ $cmd_output == *"FAILED!"* ]]; then + echo $cmd_output + fail "perf test 'arm coresight' test failed!" + fi +fi
tools/testing/selftests/arm64/coresight/settings new file +1 -0
@@ -0,0 +1 @@ +timeout=180
Signature · 1 line
2.34.1
Catalin Marinas
maintainerQuoted 7 lines · Jeremy Szu
Add a script to test the coresight functionalities by performing the perf test 'coresight'.
The script checks the prerequisites and launches a bundle of Coresight-related tests with a 180-second timeout.
Signed-off-by Jeremy Szu <[email protected]>
I have not idea how to test coresight, so adding Suzuki as well.
Quoted 89 lines
--- tools/testing/selftests/arm64/Makefile | 2 +- .../selftests/arm64/coresight/Makefile | 5 +++ .../selftests/arm64/coresight/coresight.sh | 40 +++++++++++++++++++ .../selftests/arm64/coresight/settings | 1 + 4 files changed, 47 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/arm64/coresight/Makefile create mode 100755 tools/testing/selftests/arm64/coresight/coresight.sh create mode 100644 tools/testing/selftests/arm64/coresight/settingstools/testing/selftests/arm64/Makefile +1
-1@@ -4,7 +4,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),aarch64 arm64)) -ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi +ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi coresight else ARM64_SUBTARGETS := endiftools/testing/selftests/arm64/coresight/Makefile new file +5
-0@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_PROGS := coresight.sh + +include ../../lib.mktools/testing/selftests/arm64/coresight/coresight.sh new file +40
-0@@ -0,0 +1,40 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +skip() { + echo "SKIP: $1" + exit 4 +} + +fail() { + echo "FAIL: $1" + exit 255 +} + +is_coresight_supported() { + if [ -d "/sys/bus/coresight/devices" ]; then + return 0 + fi + return 255 +} + +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + [ "$(id -u)" -ne 0 ] && \ + skip "this test must be run as root." + which perf >/dev/null 2>&1 || \ + skip "perf is not installed." + perf test list 2>&1 | grep -qi 'coresight' || \ + skip "perf doesn't support testing coresight." + is_coresight_supported || \ + skip "coresight is not supported." + + cmd_output=$(perf test -vv 'coresight' 2>&1) + perf_ret=$? + + if [ $perf_ret -ne 0 ]; then + fail "perf command returns non-zero." + elif [[ $cmd_output == *"FAILED!"* ]]; then + echo $cmd_output + fail "perf test 'arm coresight' test failed!" + fi +fitools/testing/selftests/arm64/coresight/settings new file +1
-0@@ -0,0 +1 @@ +timeout=180Signature · 1 line
2.34.1
Signature · 1 line
Catalin
Quoted 7 lines · Catalin Marinas
Quoted 6 lines · Jeremy Szu
Add a script to test the coresight functionalities by performing the perf test 'coresight'.
The script checks the prerequisites and launches a bundle of Coresight-related tests with a 180-second timeout.
Hi Jeremy,
On the whole I'm not sure running the Perf tests under kself tests is a good fit. We're already running all the Perf tests in various CIs, so this is going to duplicate effort. Especially with setup and parsing of the results output.
There is also no clean line between what's a kernel issue and whats a Perf issue when these fail.
And thirdly why only run the Coresight tests? Most of the Perf tests test some part of the kernel, so if we were going to do this I think it would make sense to make some kind of proper harness and run them all. I have some recollection that someone said it might be something we could do, but I can't remember the discussion.
Ignoring the main issue above I've left some comments about this patch inline below:
Quoted 55 lines
Quoted 1 line
Signed-off-by Jeremy Szu <[email protected]>I have not idea how to test coresight, so adding Suzuki as well.
Quoted 51 lines
--- tools/testing/selftests/arm64/Makefile | 2 +- .../selftests/arm64/coresight/Makefile | 5 +++ .../selftests/arm64/coresight/coresight.sh | 40 +++++++++++++++++++ .../selftests/arm64/coresight/settings | 1 + 4 files changed, 47 insertions(+), 1 deletion(-)create mode 100644
tools/testing/selftests/arm64/coresight/Makefilecreate mode 100755tools/testing/selftests/arm64/coresight/coresight.shcreate mode 100644tools/testing/selftests/arm64/coresight/settingstools/testing/selftests/arm64/Makefile +1
-1@@ -4,7 +4,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),aarch64 arm64)) -ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi +ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi coresight else ARM64_SUBTARGETS := endiftools/testing/selftests/arm64/coresight/Makefile new file +5
-0@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_PROGS := coresight.sh + +include ../../lib.mktools/testing/selftests/arm64/coresight/coresight.sh new file +11
-0@@ -0,0 +1,40 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +skip() { + echo "SKIP: $1" + exit 4 +} + +fail() { + echo "FAIL: $1" + exit 255
Quoted 8 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 19
Quoted 8 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +8
-0+} + +is_coresight_supported() { + if [ -d "/sys/bus/coresight/devices" ]; then + return 0 + fi + return 255 +}
The Perf coresight tests already have a skip mechanism built in so can we rely on that instead of duplicating it here? There are also other scenarios for skipping like Perf not linked with OpenCSD which aren't covered here.
Quoted 8 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 27
Quoted 8 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +8
-0+ +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + [ "$(id -u)" -ne 0 ] && \ + skip "this test must be run as root." + which perf >/dev/null 2>&1 || \ + skip "perf is not installed." + perf test list 2>&1 | grep -qi 'coresight' || \ + skip "perf doesn't support testing coresight."
Can this be an error instead? The coresight tests were added in 5.10 and I don't think new kselftest needs to support such old kernels. This skip risks turning an error with installing the tests into a silent failure.
Also as far as I know a lot of distros will refuse to open Perf unless it matches the kernel version if it was installed from the package manager, so we don't need to worry about old versions.
Quoted 10 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 37
Quoted 10 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +10
-0+ is_coresight_supported || \ + skip "coresight is not supported." + + cmd_output=$(perf test -vv 'coresight' 2>&1) + perf_ret=$? + + if [ $perf_ret -ne 0 ]; then + fail "perf command returns non-zero." + elif [[ $cmd_output == *"FAILED!"* ]]; then + echo $cmd_output
It's probably helpful to print cmd_output in both failure cases.
Quoted 10 lines
tools/testing/selftests/arm64/coresight/settings line 1
Quoted 10 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +3
-0+ fail "perf test 'arm coresight' test failed!" + fi +fitools/testing/selftests/arm64/coresight/settings new file +1
-0@@ -0,0 +1 @@ +timeout=180
I timed 331 seconds on n1sdp, and probably even longer on Juno.
It doesn't need to run for this long and it's an issue with the tests, but currently that's how long it takes so the timeout needs to be longer.
Quoted 3 lines
Quoted 2 lines
Signature · 1 line
2.34.1
Hi James,
Thank you for your reply.
James Clark 於 7/12/24 5:01 PM 寫道:
Quoted 11 lines · Catalin Marinas
Quoted 7 lines · Jeremy Szu
Quoted 6 lines
Add a script to test the coresight functionalities by performing the perf test 'coresight'.
The script checks the prerequisites and launches a bundle of Coresight-related tests with a 180-second timeout.
Quoted 7 lines
Hi Jeremy,
On the whole I'm not sure running the Perf tests under kself tests is a good fit. We're already running all the Perf tests in various CIs, so this is going to duplicate effort. Especially with setup and parsing of the results output.
There is also no clean line between what's a kernel issue and whats a Perf issue when these fail.
And thirdly why only run the Coresight tests? Most of the Perf tests test some part of the kernel, so if we were going to do this I think it would make sense to make some kind of proper harness and run them all. I have some recollection that someone said it might be something we could do, but I can't remember the discussion.
The idea I'm trying to pursue is to use arm64 kselftest to run as many test cases as possible for ARM SoCs across different designs and distros. I believe it could provide an alert if there is an issue, whether it originates from userspace or kernel, similar to how perf is used in other categories.
I'm not sure if all perf tests could be counted in soc (selftests/arm64) category such as some tests may target to storage, memory or devices. I could replace 'arm64/coresight' with 'arm64/perf' if it makes more sense. I believe it could help users verify functionality more conveniently.
Quoted 59 lines
Ignoring the main issue above I've left some comments about this patch inline below:
Quoted 56 lines
Quoted 1 line
Signed-off-by Jeremy Szu <[email protected]>I have not idea how to test coresight, so adding Suzuki as well.
Quoted 52 lines
---
tools/testing/selftests/arm64/Makefile | 2 +- .../selftests/arm64/coresight/Makefile | 5 +++ .../selftests/arm64/coresight/coresight.sh | 40 +++++++++++++++++++ .../selftests/arm64/coresight/settings | 1 +4 files changed, 47 insertions(+), 1 deletion(-)create mode 100644
tools/testing/selftests/arm64/coresight/Makefilecreate mode 100755tools/testing/selftests/arm64/coresight/coresight.shcreate mode 100644tools/testing/selftests/arm64/coresight/settingstools/testing/selftests/arm64/Makefile +1
-1@@ -4,7 +4,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),aarch64 arm64)) -ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi +ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi coresightelse
ARM64_SUBTARGETS:=
endiftools/testing/selftests/arm64/coresight/Makefile new file +5
-0@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_PROGS := coresight.sh + +include ../../lib.mktools/testing/selftests/arm64/coresight/coresight.sh new file +13
-0@@ -0,0 +1,40 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +skip() { + echo "SKIP: $1" + exit 4 +} + +fail() { + echo "FAIL: $1" + exit 255 +} +
Quoted 9 lines
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 19
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +6
-0+is_coresight_supported() { + if [ -d "/sys/bus/coresight/devices" ]; then + return 0 + fi + return 255 +}The Perf coresight tests already have a skip mechanism built in so can we rely on that instead of duplicating it here? There are also other scenarios for skipping like Perf not linked with OpenCSD which aren't covered here.
Will it return the different error code to indicate if it's failed to be executed or the coresight is not supported?
Quoted 11 lines
Quoted 8 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 27
Quoted 8 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +8
-0+ +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + [ "$(id -u)" -ne 0 ] && \ + skip "this test must be run as root." + which perf >/dev/null 2>&1 || \ + skip "perf is not installed." + perf test list 2>&1 | grep -qi 'coresight' || \ + skip "perf doesn't support testing coresight."Can this be an error instead? The coresight tests were added in 5.10 and I don't think new kselftest needs to support such old kernels. This skip risks turning an error with installing the tests into a silent failure.
Quoted 2 lines
Also as far as I know a lot of distros will refuse to open Perf unless it matches the kernel version if it was installed from the package manager, so we don't need to worry about old versions.
The idea to skip it here is because I thought either a distro (custom) kernel doesn't enable the kconfig or the distro built the perf with CORESIGHT=0.
I could make it as an error and put it after "is_coresight_support" if it makes more sense.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +4
-0+ is_coresight_supported || \ + skip "coresight is not supported." + + cmd_output=$(perf test -vv 'coresight' 2>&1)
Quoted 9 lines
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 37
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +6
-0+ perf_ret=$? + + if [ $perf_ret -ne 0 ]; then + fail "perf command returns non-zero." + elif [[ $cmd_output == *"FAILED!"* ]]; then + echo $cmd_outputIt's probably helpful to print
cmd_outputin both failure cases.
ok, will do.
Quoted 13 lines
Quoted 10 lines
tools/testing/selftests/arm64/coresight/settings line 1
Quoted 10 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +3
-0+ fail "perf test 'arm coresight' test failed!" + fi +fitools/testing/selftests/arm64/coresight/settings new file +1
-0@@ -0,0 +1 @@ +timeout=180I timed 331 seconds on n1sdp, and probably even longer on Juno.
Quoted 2 lines
It doesn't need to run for this long and it's an issue with the tests, but currently that's how long it takes so the timeout needs to be longer.
ok, will extend it to 600.
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
Signature · 1 line
2.34.1
Quoted 20 lines · Jeremy Szu
Hi James,
Thank you for your reply.
James Clark 於 7/12/24 5:01 PM 寫道:
Quoted 15 lines · Catalin Marinas
Quoted 7 lines · Jeremy Szu
Quoted 6 lines
Add a script to test the coresight functionalities by performing the perf test 'coresight'.
The script checks the prerequisites and launches a bundle of Coresight-related tests with a 180-second timeout.
Hi Jeremy,
On the whole I'm not sure running the Perf tests under kself tests is a good fit. We're already running all the Perf tests in various CIs, so this is going to duplicate effort. Especially with setup and parsing of the results output.
Quoted 7 lines
Quoted 3 lines
There is also no clean line between what's a kernel issue and whats a Perf issue when these fail.
And thirdly why only run the Coresight tests? Most of the Perf tests test some part of the kernel, so if we were going to do this I think it would make sense to make some kind of proper harness and run them all. I have some recollection that someone said it might be something we could do, but I can't remember the discussion.
The idea I'm trying to pursue is to use arm64 kselftest to run as many test cases as possible for ARM SoCs across different designs and distros. I believe it could provide an alert if there is an issue, whether it originates from userspace or kernel, similar to how perf is used in other categories.
I'm not sure if all perf tests could be counted in soc (selftests/arm64) category such as some tests may target to storage, memory or devices. I
Could we not put the Perf tests in .../selftests/perf.sh, then it doesn't really matter which subsystem they're targeting and we can run all the Perf tests?
Quoted 72 lines
could replace 'arm64/coresight' with 'arm64/perf' if it makes more sense. I believe it could help users verify functionality more conveniently.
Quoted 60 lines
Ignoring the main issue above I've left some comments about this patch inline below:
Quoted 57 lines
Quoted 1 line
Signed-off-by Jeremy Szu <[email protected]>I have not idea how to test coresight, so adding Suzuki as well.
Quoted 53 lines
---
tools/testing/selftests/arm64/Makefile | 2 +- .../selftests/arm64/coresight/Makefile | 5 +++ .../selftests/arm64/coresight/coresight.sh | 40
+++++++++++++++++++
.../selftests/arm64/coresight/settings | 1 +4 files changed, 47 insertions(+), 1 deletion(-)create mode 100644
tools/testing/selftests/arm64/coresight/Makefilecreate mode 100755tools/testing/selftests/arm64/coresight/coresight.shcreate mode 100644tools/testing/selftests/arm64/coresight/settingstools/testing/selftests/arm64/Makefile +1
-1@@ -4,7 +4,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),aarch64 arm64)) -ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi +ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi coresightelse
ARM64_SUBTARGETS:=
endiftools/testing/selftests/arm64/coresight/Makefile new file +5
-0@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_PROGS := coresight.sh + +include ../../lib.mktools/testing/selftests/arm64/coresight/coresight.sh new file +13
-0@@ -0,0 +1,40 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +skip() { + echo "SKIP: $1" + exit 4 +} + +fail() { + echo "FAIL: $1" + exit 255 +} +Quoted 9 lines
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 19
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +6
-0+is_coresight_supported() { + if [ -d "/sys/bus/coresight/devices" ]; then + return 0 + fi + return 255 +}The Perf coresight tests already have a skip mechanism built in so can we rely on that instead of duplicating it here? There are also other scenarios for skipping like Perf not linked with OpenCSD which aren't covered here.
Quoted 2 lines
Will it return the different error code to indicate if it's failed to be executed or the coresight is not supported?
I think the exit code is only used for more serious errors. For things like missing tests, SKIP and FAIL it still exits with 0 but you have to grep for the strings. Actually for a missing test it prints nothing and exits 0.
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 27
Quoted 8 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +8
-0+ +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + [ "$(id -u)" -ne 0 ] && \ + skip "this test must be run as root." + which perf >/dev/null 2>&1 || \ + skip "perf is not installed." + perf test list 2>&1 | grep -qi 'coresight' || \ + skip "perf doesn't support testing coresight."Can this be an error instead? The coresight tests were added in 5.10 and I don't think new kselftest needs to support such old kernels. This skip risks turning an error with installing the tests into a silent failure.
Quoted 7 lines
Quoted 2 lines
Also as far as I know a lot of distros will refuse to open Perf unless it matches the kernel version if it was installed from the package manager, so we don't need to worry about old versions.
The idea to skip it here is because I thought either a distro (custom) kernel doesn't enable the kconfig or the distro built the perf with CORESIGHT=0.
I could make it as an error and put it after "
is_coresight_support" if it makes more sense.
But the Coresight test already checks these things, which is my point. You can grep for "SKIP" which it will print for any case where the coresight test can't be run due to some missing config.
Quoted 37 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +4
-0+ is_coresight_supported || \ + skip "coresight is not supported." + + cmd_output=$(perf test -vv 'coresight' 2>&1)Quoted 9 lines
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 37
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +6
-0+ perf_ret=$? + + if [ $perf_ret -ne 0 ]; then + fail "perf command returns non-zero." + elif [[ $cmd_output == *"FAILED!"* ]]; then + echo $cmd_outputIt's probably helpful to print
cmd_outputin both failure cases.ok, will do.
Quoted 13 lines
Quoted 10 lines
tools/testing/selftests/arm64/coresight/settings line 1
Quoted 10 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +3
-0+ fail "perf test 'arm coresight' test failed!" + fi +fitools/testing/selftests/arm64/coresight/settings new file +1
-0@@ -0,0 +1 @@ +timeout=180I timed 331 seconds on n1sdp, and probably even longer on Juno.
Quoted 2 lines
It doesn't need to run for this long and it's an issue with the tests, but currently that's how long it takes so the timeout needs to be longer.
ok, will extend it to 600.
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
Signature · 1 line
2.34.1
James Clark 於 7/17/24 10:48 PM 寫道:
Quoted 25 lines · Jeremy Szu
Quoted 22 lines
Hi James,
Thank you for your reply.
James Clark 於 7/12/24 5:01 PM 寫道:
Quoted 17 lines · Catalin Marinas
Quoted 7 lines · Jeremy Szu
Quoted 6 lines
Add a script to test the coresight functionalities by performing the perf test 'coresight'.
The script checks the prerequisites and launches a bundle of Coresight-related tests with a 180-second timeout.
Hi Jeremy,
On the whole I'm not sure running the Perf tests under kself tests is a good fit. We're already running all the Perf tests in various CIs, so this is going to duplicate effort. Especially with setup and parsing of the results output.
There is also no clean line between what's a kernel issue and whats a Perf issue when these fail.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
And thirdly why only run the Coresight tests? Most of the Perf tests test some part of the kernel, so if we were going to do this I think it would make sense to make some kind of proper harness and run them all. I have some recollection that someone said it might be something we could do, but I can't remember the discussion.
The idea I'm trying to pursue is to use arm64 kselftest to run as many test cases as possible for ARM SoCs across different designs and distros. I believe it could provide an alert if there is an issue, whether it originates from userspace or kernel, similar to how perf is used in other categories.
I'm not sure if all perf tests could be counted in soc (selftests/arm64) category such as some tests may target to storage, memory or devices. I
Could we not put the Perf tests in
.../selftests/perf.sh, then it doesn't really matter which subsystem they're targeting and we can run all the Perf tests?
The .../sefltests/ seems for the kselftest framework only, not sure if having a new .../selftests/perf will make more sense?
Quoted 90 lines
Quoted 72 lines
could replace 'arm64/coresight' with 'arm64/perf' if it makes more sense. I believe it could help users verify functionality more conveniently.
Quoted 60 lines
Ignoring the main issue above I've left some comments about this patch inline below:
Quoted 57 lines
Quoted 1 line
Signed-off-by Jeremy Szu <[email protected]>I have not idea how to test coresight, so adding Suzuki as well.
Quoted 53 lines
---
tools/testing/selftests/arm64/Makefile | 2 +- .../selftests/arm64/coresight/Makefile | 5 +++ .../selftests/arm64/coresight/coresight.sh | 40
+++++++++++++++++++
.../selftests/arm64/coresight/settings | 1 +4 files changed, 47 insertions(+), 1 deletion(-)create mode 100644
tools/testing/selftests/arm64/coresight/Makefilecreate mode 100755tools/testing/selftests/arm64/coresight/coresight.shcreate mode 100644tools/testing/selftests/arm64/coresight/settingstools/testing/selftests/arm64/Makefile +1
-1@@ -4,7 +4,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),aarch64 arm64)) -ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi +ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi coresightelse
ARM64_SUBTARGETS:=
endiftools/testing/selftests/arm64/coresight/Makefile new file +5
-0@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_PROGS := coresight.sh + +include ../../lib.mktools/testing/selftests/arm64/coresight/coresight.sh new file +13
-0@@ -0,0 +1,40 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +skip() { + echo "SKIP: $1" + exit 4 +} + +fail() { + echo "FAIL: $1" + exit 255 +} +Quoted 9 lines
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 19
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +6
-0+is_coresight_supported() { + if [ -d "/sys/bus/coresight/devices" ]; then + return 0 + fi + return 255 +}The Perf coresight tests already have a skip mechanism built in so can we rely on that instead of duplicating it here? There are also other scenarios for skipping like Perf not linked with OpenCSD which aren't covered here.
Quoted 2 lines
Will it return the different error code to indicate if it's failed to be executed or the coresight is not supported?
I think the exit code is only used for more serious errors. For things like missing tests, SKIP and FAIL it still exits with 0 but you have to grep for the strings. Actually for a missing test it prints nothing and exits 0.
Quoted 14 lines
Quoted 13 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
+
+if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + [ "$(id -u)" -ne 0 ] && \
+ skip "this test must be run as root." + which perf >/dev/null 2>&1 || \
+ skip "perf is not installed."
+ perf test list 2>&1 | grep -qi 'coresight' || \
+ skip "perf doesn't support testing coresight."Can this be an error instead? The coresight tests were added in 5.10 and I don't think new kselftest needs to support such old kernels. This skip risks turning an error with installing the tests into a silent failure.
Also as far as I know a lot of distros will refuse to open Perf unless it matches the kernel version if it was installed from the package manager, so we don't need to worry about old versions.
Quoted 7 lines
Quoted 4 lines
The idea to skip it here is because I thought either a distro (custom) kernel doesn't enable the kconfig or the distro built the perf with CORESIGHT=0.
I could make it as an error and put it after "
is_coresight_support" if it makes more sense.But the Coresight test already checks these things, which is my point. You can grep for "SKIP" which it will print for any case where the coresight test can't be run due to some missing config.
Oh, I guess you mean to rely on something like the perf list cs_etm | grep -q cs_etm || exit 2. Yes, that makes more sense.
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 10 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 37
Quoted 10 lines
+
is_coresight_supported|| \
+ skip "coresight is not supported."tools/testing/selftests/arm64/coresight/coresight.sh new file +8
-0+ + cmd_output=$(perf test -vv 'coresight' 2>&1) + perf_ret=$? + + if [ $perf_ret -ne 0 ]; then + fail "perf command returns non-zero." + elif [[ $cmd_output == *"FAILED!"* ]]; then + echo $cmd_outputIt's probably helpful to print
cmd_outputin both failure cases.
Quoted 3 lines
Quoted 3 lines
ok, will do.
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ fail "perf test 'arm coresight' test failed!"
I think I should remove the is_coresight_supported there and checks the output as a "else" to see if the test is PASS or SKIP. Does it make sense to you?
Quoted 20 lines
Quoted 20 lines
Quoted 12 lines
Quoted 9 lines
tools/testing/selftests/arm64/coresight/settings line 1
Quoted 9 lines
+ fi
tools/testing/selftests/arm64/coresight/coresight.sh new file +1
-0+fitools/testing/selftests/arm64/coresight/settings new file +1
-0@@ -0,0 +1 @@ +timeout=180I timed 331 seconds on n1sdp, and probably even longer on Juno.
Quoted 2 lines
It doesn't need to run for this long and it's an issue with the tests, but currently that's how long it takes so the timeout needs to be longer.
ok, will extend it to 600.
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
Signature · 1 line
2.34.1
Quoted 30 lines · Jeremy Szu
James Clark 於 7/17/24 10:48 PM 寫道:
Quoted 27 lines · Jeremy Szu
Quoted 24 lines
Hi James,
Thank you for your reply.
James Clark 於 7/12/24 5:01 PM 寫道:
Quoted 18 lines · Catalin Marinas
Quoted 7 lines · Jeremy Szu
Quoted 6 lines
Add a script to test the coresight functionalities by performing the perf test 'coresight'.
The script checks the prerequisites and launches a bundle of Coresight-related tests with a 180-second timeout.
Hi Jeremy,
On the whole I'm not sure running the Perf tests under kself tests is a good fit. We're already running all the Perf tests in various CIs, so this is going to duplicate effort. Especially with setup and parsing of the results output.
There is also no clean line between what's a kernel issue and whats a Perf issue when these fail.
And thirdly why only run the Coresight tests? Most of the Perf tests test some part of the kernel, so if we were going to do this I think it would make sense to make some kind of proper harness and run them all. I have some recollection that someone said it might be something we could do, but I can't remember the discussion.
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
The idea I'm trying to pursue is to use arm64 kselftest to run as many test cases as possible for ARM SoCs across different designs and distros. I believe it could provide an alert if there is an issue, whether it originates from userspace or kernel, similar to how perf is used in other categories.
I'm not sure if all perf tests could be counted in soc (selftests/arm64) category such as some tests may target to storage, memory or devices. I
Could we not put the Perf tests in
.../selftests/perf.sh, then it doesn't really matter which subsystem they're targeting and we can run all the Perf tests?The .../sefltests/ seems for the kselftest framework only, not sure if having a new .../selftests/perf will make more sense?
Yeah that sounds better, but it probably requires changing the title of the patch to "[RFC] kselftest: Run Perf tests under kselftest" or something like that.
Quoted 92 lines
Quoted 92 lines
Quoted 72 lines
could replace 'arm64/coresight' with 'arm64/perf' if it makes more sense. I believe it could help users verify functionality more conveniently.
Quoted 60 lines
Ignoring the main issue above I've left some comments about this patch inline below:
Quoted 57 lines
Quoted 1 line
Signed-off-by Jeremy Szu <[email protected]>I have not idea how to test coresight, so adding Suzuki as well.
Quoted 53 lines
---
tools/testing/selftests/arm64/Makefile | 2 +- .../selftests/arm64/coresight/Makefile | 5 +++ .../selftests/arm64/coresight/coresight.sh | 40
+++++++++++++++++++
.../selftests/arm64/coresight/settings | 1 +4 files changed, 47 insertions(+), 1 deletion(-)create mode 100644
tools/testing/selftests/arm64/coresight/Makefilecreate mode 100755tools/testing/selftests/arm64/coresight/coresight.shcreate mode 100644tools/testing/selftests/arm64/coresight/settingstools/testing/selftests/arm64/Makefile +1
-1@@ -4,7 +4,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),aarch64 arm64)) -ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi +ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi coresightelse
ARM64_SUBTARGETS:=
endiftools/testing/selftests/arm64/coresight/Makefile new file +5
-0@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_PROGS := coresight.sh + +include ../../lib.mktools/testing/selftests/arm64/coresight/coresight.sh new file +13
-0@@ -0,0 +1,40 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +skip() { + echo "SKIP: $1" + exit 4 +} + +fail() { + echo "FAIL: $1" + exit 255 +} +Quoted 9 lines
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 19
Quoted 6 lines
tools/testing/selftests/arm64/coresight/coresight.sh new file +6
-0+is_coresight_supported() { + if [ -d "/sys/bus/coresight/devices" ]; then + return 0 + fi + return 255 +}The Perf coresight tests already have a skip mechanism built in so can we rely on that instead of duplicating it here? There are also other scenarios for skipping like Perf not linked with OpenCSD which aren't covered here.
Quoted 2 lines
Will it return the different error code to indicate if it's failed to be executed or the coresight is not supported?
I think the exit code is only used for more serious errors. For things like missing tests, SKIP and FAIL it still exits with 0 but you have to grep for the strings. Actually for a missing test it prints nothing and exits 0.
Quoted 16 lines
Quoted 9 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
+
+if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + [ "$(id -u)" -ne 0 ] && \
+ skip "this test must be run as root." + which perf >/dev/null 2>&1 || \
+ skip "perf is not installed."
+ perf test list 2>&1 | grep -qi 'coresight' || \
+ skip "perf doesn't support testing coresight."Quoted 4 lines
Can this be an error instead? The coresight tests were added in 5.10 and I don't think new kselftest needs to support such old kernels. This skip risks turning an error with installing the tests into a silent failure.
Also as far as I know a lot of distros will refuse to open Perf unless it matches the kernel version if it was installed from the package manager, so we don't need to worry about old versions.
The idea to skip it here is because I thought either a distro (custom) kernel doesn't enable the kconfig or the distro built the perf with CORESIGHT=0.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
I could make it as an error and put it after "
is_coresight_support" if it makes more sense.But the Coresight test already checks these things, which is my point. You can grep for "SKIP" which it will print for any case where the coresight test can't be run due to some missing config.
Oh, I guess you mean to rely on something like the
perf list cs_etm | grep -q cs_etm || exit 2. Yes, that makes more sense.
If we're leaning towards running all the Perf tests then none of this is required anymore.
This is another example of why it's better to run them all. We can't realistically hard code a kself test with loads of logic for each Perf subtest when Perf will happily run all the tests on it's own without any extra work.
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 10 lines
tools/testing/selftests/arm64/coresight/coresight.sh line 37
Quoted 10 lines
+
is_coresight_supported|| \
+ skip "coresight is not supported."tools/testing/selftests/arm64/coresight/coresight.sh new file +8
-0+ + cmd_output=$(perf test -vv 'coresight' 2>&1) + perf_ret=$? + + if [ $perf_ret -ne 0 ]; then + fail "perf command returns non-zero." + elif [[ $cmd_output == *"FAILED!"* ]]; then + echo $cmd_outputIt's probably helpful to print
cmd_outputin both failure cases.
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
ok, will do.
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ fail "perf test 'arm coresight' test failed!"
I think I should remove the
is_coresight_supportedthere and checks the output as a "else" to see if the test is PASS or SKIP. Does it make sense to you?
Same as above applies, the only thing that's really required is parsing for "FAILED".
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
Quoted 12 lines
Quoted 9 lines
tools/testing/selftests/arm64/coresight/settings line 1
Quoted 9 lines
+ fi
tools/testing/selftests/arm64/coresight/coresight.sh new file +1
-0+fitools/testing/selftests/arm64/coresight/settings new file +1
-0@@ -0,0 +1 @@ +timeout=180I timed 331 seconds on n1sdp, and probably even longer on Juno.
Quoted 2 lines
It doesn't need to run for this long and it's an issue with the tests, but currently that's how long it takes so the timeout needs to be longer.
ok, will extend it to 600.
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
Signature · 1 line
2.34.1