Skip to contentSkip to conversation

perf/hw_breakpoint: test: Skip the test if dependencies unmet

PATCH Applied David Gow 5 participants 7 messages Oct 26, 2022 – Nov 2, 2022 perf events

lore
Share & tools
lore

David Gow

author
#0 lore raw

PATCH 1/1 perf/hw_breakpoint: test: Skip the test if dependencies unmet

Running the test currently fails on non-SMP systems, despite being enabled by default. This means that running the test with:

./tools/testing/kunit/kunit.py run --arch x86_64 hw_breakpoint

results in every hw_breakpoint test failing with:

# test_one_cpu: failed to initialize: -22
not ok 1 - test_one_cpu

Instead, use kunit_skip(), which will mark the test as skipped, and give a more comprehensible message:

ok 1 - test_one_cpu # SKIP not enough cpus

This makes it more obvious that the test is not suited to the test environment, and so wasn't run, rather than having run and failed.

---
 kernel/events/hw_breakpoint_test.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/events/hw_breakpoint_test.c +2 -2
index 5ced822df788..c57610f52bb4 100644
@@ -295,11 +295,11 @@ static int test_init(struct kunit *test)
 {
 	/* Most test cases want 2 distinct CPUs. */
 	if (num_online_cpus() < 2)
-		return -EINVAL;
+		kunit_skip(test, "not enough cpus");
 
 	/* Want the system to not use breakpoints elsewhere. */
 	if (hw_breakpoint_is_used())
-		return -EBUSY;
+		kunit_skip(test, "hw breakpoint already in use");
 
 	return 0;
 }
Signature · 1 line
2.38.0.135.g90850a2211-goog
Acked-by on v1 1/1
#1 lore raw
Quoted 14 lines · David Gow

Running the test currently fails on non-SMP systems, despite being enabled by default. This means that running the test with:

./tools/testing/kunit/kunit.py run --arch x86_64 hw_breakpoint

results in every hw_breakpoint test failing with:

# test_one_cpu: failed to initialize: -22
not ok 1 - test_one_cpu

Instead, use kunit_skip(), which will mark the test as skipped, and give a more comprehensible message:

Quoted 6 lines

ok 1 - test_one_cpu # SKIP not enough cpus

This makes it more obvious that the test is not suited to the test environment, and so wasn't run, rather than having run and failed.

Although I still get confused by the fact that skipped tests say "ok" and then need to double check the log that tests weren't skipped.

Quoted 25 lines
---
 kernel/events/hw_breakpoint_test.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/events/hw_breakpoint_test.c +2 -3
index 5ced822df788..c57610f52bb4 100644
@@ -295,11 +295,11 @@ static int test_init(struct kunit *test)
 {
        /* Most test cases want 2 distinct CPUs. */
        if (num_online_cpus() < 2)
-               return -EINVAL;
+               kunit_skip(test, "not enough cpus");
 
        /* Want the system to not use breakpoints elsewhere. */
        if (hw_breakpoint_is_used())
-               return -EBUSY;
+               kunit_skip(test, "hw breakpoint already in use");
 
        return 0;
 }
--

2.38.0.135.g90850a2211-goog

Reviewed-by on v1 1/1
#2 lore raw
In reply to David Gow
Quoted 14 lines · David Gow

Running the test currently fails on non-SMP systems, despite being enabled by default. This means that running the test with:

./tools/testing/kunit/kunit.py run --arch x86_64 hw_breakpoint

results in every hw_breakpoint test failing with:

# test_one_cpu: failed to initialize: -22
not ok 1 - test_one_cpu

Instead, use kunit_skip(), which will mark the test as skipped, and give a more comprehensible message:

Quoted 6 lines

ok 1 - test_one_cpu # SKIP not enough cpus

This makes it more obvious that the test is not suited to the test environment, and so wasn't run, rather than having run and failed.

This patch makes this command pass for me.

$ ./tools/testing/kunit/kunit.py run --arch x86_64
Since this test gets picked up by default, having it pass for common
uses of kunit.py is a priority, IMO.

(Note: if I add --alltests as well, these were the only failures)

I agree with Marco that TAP/KTAP saying "ok" for skipped tests can be confusing at first.
But a SKIP status feels more appropriate than FAIL, so I'd strongly like for this change to go in.

Quoted 9 lines
---
 kernel/events/hw_breakpoint_test.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/events/hw_breakpoint_test.c +0 -0
index 5ced822df788..c57610f52bb4 100644

kernel/events/hw_breakpoint_test.c static int test_init(struct kunit *test) · line 298

Quoted 5 lines
kernel/events/hw_breakpoint_test.c +1 -1
@@ -295,11 +295,11 @@ static int test_init(struct kunit *test)
 {
        /* Most test cases want 2 distinct CPUs. */
        if (num_online_cpus() < 2)
-               return -EINVAL;
+               kunit_skip(test, "not enough cpus");

The only minor nit I have is that I'd personally prefer something like
kunit_skip(test, "need >=2 cpus");
since that makes it clearer
a) that we must only have 1 CPU by default
b) roughly how one might address this.

Note: b) is a bit more complicated than I would like. The final command is something like

$ ./tools/testing/kunit/kunit.py run --arch x86_64 --qemu_args='-smp
2' --kconfig_add='CONFIG_SMP=y'

But that's orthogonal to this patch.

#3 lore raw

Hi David, Daniel,

On Wed, 26 Oct 2022 at 20:31, Daniel Latypov <[email protected]> wrote:
[...]

Quoted 9 lines

kernel/events/hw_breakpoint_test.c static int test_init(struct kunit *test) · line 298

Quoted 2 lines
kernel/events/hw_breakpoint_test.c +1 -1
-               return -EINVAL;
+               kunit_skip(test, "not enough cpus");

The only minor nit I have is that I'd personally prefer something like
kunit_skip(test, "need >=2 cpus");
since that makes it clearer
a) that we must only have 1 CPU by default
b) roughly how one might address this.

Quoted 6 lines

Note: b) is a bit more complicated than I would like. The final command is something like

$ ./tools/testing/kunit/kunit.py run --arch x86_64 --qemu_args='-smp
2' --kconfig_add='CONFIG_SMP=y'

But that's orthogonal to this patch.

Was there going to be a v2 to address (a), or is this patch ready to be picked up?

I assume (unless I hear otherwise), this patch shall also go through -tip?

Thanks,
-- Marco

maintainer
#4 lore raw
Quoted 4 lines · Marco Elver

Was there going to be a v2 to address (a), or is this patch ready to be picked up?

I assume (unless I hear otherwise), this patch shall also go through -tip?

Yes, I've got it queued, I just haven't gotten around to pushing it out to -tip, hopefully later today.

#5 lore raw
In reply to Marco Elver
Quoted 19 lines · Marco Elver

Hi David, Daniel,

On Wed, 26 Oct 2022 at 20:31, Daniel Latypov <[email protected]> wrote:
[...]

Quoted 14 lines

kernel/events/hw_breakpoint_test.c static int test_init(struct kunit *test) · line 298

Quoted 2 lines
kernel/events/hw_breakpoint_test.c +1 -1
-               return -EINVAL;
+               kunit_skip(test, "not enough cpus");

The only minor nit I have is that I'd personally prefer something like
kunit_skip(test, "need >=2 cpus");
since that makes it clearer
a) that we must only have 1 CPU by default
b) roughly how one might address this.

Note: b) is a bit more complicated than I would like. The final command is something like

$ ./tools/testing/kunit/kunit.py run --arch x86_64 --qemu_args='-smp
2' --kconfig_add='CONFIG_SMP=y'
Quoted 6 lines
Quoted 1 line

But that's orthogonal to this patch.

Was there going to be a v2 to address (a), or is this patch ready to be picked up?

I assume (unless I hear otherwise), this patch shall also go through -tip?

Just noting for the record:
I'm totally fine with this version going in, esp. if Peter is already planning on picking it up.

This patch makes it so kunit.py run --arch=x86_64 doesn't have test failures, so I don't want it delayed due to just my small nit.

Daniel

merge bot Applied
#6 lore raw
In reply to David Gow
The following commit has been merged into the perf/urgent branch of tip:

The following commit has been merged into the perf/urgent branch of tip:

Commit-ID:     4b18cb3f74dcfc183c2434e17bfce09ce6302e37
Gitweb:        https://git.kernel.org/tip/4b18cb3f74dcfc183c2434e17bfce09ce6302e37
Author:        David Gow <[email protected]>
AuthorDate:    Wed, 26 Oct 2022 22:10:40 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 02 Nov 2022 12:22:05 +01:00

perf/hw_breakpoint: test: Skip the test if dependencies unmet

Running the test currently fails on non-SMP systems, despite being enabled by default. This means that running the test with:

./tools/testing/kunit/kunit.py run --arch x86_64 hw_breakpoint

results in every hw_breakpoint test failing with:

# test_one_cpu: failed to initialize: -22
not ok 1 - test_one_cpu

Instead, use kunit_skip(), which will mark the test as skipped, and give a more comprehensible message:

ok 1 - test_one_cpu # SKIP not enough cpus

This makes it more obvious that the test is not suited to the test environment, and so wasn't run, rather than having run and failed.

---
 kernel/events/hw_breakpoint_test.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/events/hw_breakpoint_test.c +2 -2
index 5ced822..c57610f 100644
@@ -295,11 +295,11 @@ static int test_init(struct kunit *test)
 {
 	/* Most test cases want 2 distinct CPUs. */
 	if (num_online_cpus() < 2)
-		return -EINVAL;
+		kunit_skip(test, "not enough cpus");
 
 	/* Want the system to not use breakpoints elsewhere. */
 	if (hw_breakpoint_is_used())
-		return -EBUSY;
+		kunit_skip(test, "hw breakpoint already in use");
 
 	return 0;
 }