kunit/cfg80211-ie-generation flaking ?
Subsystems (2)
Jakub Kicinski
maintainerHi!
Do you see any problems with the cfg80211-ie-generation kUnit test?
We hit 3 failures in the last 3 days.
https://netdev.bots.linux.dev/flakes.html?min-flip=0&tn-needle=cfg80211-ie-generation
But the kunit stuff likes to break because of cross-tests corruptions :(
We run:
./tools/testing/kunit/kunit.py run --alltests --json --arch=x86_64
Quoted 3 lines · Jakub Kicinski
Hi!
Do you see any problems with the cfg80211-ie-generation kUnit test?
Nope, never, we must run it dozens of times a day ...
Quoted 1 line
We hit 3 failures in the last 3 days.
Four, actually, it seems? ;-)
Quoted 3 lines
https://netdev.bots.linux.dev/flakes.html?min-flip=0&tn-needle=cfg80211-ie-generation
But the kunit stuff likes to break because of cross-tests corruptions :(
Hmm. Let's say ...
https://netdev-3.bots.linux.dev/kunit/results/987921/kunit-test.log
is your serial console simply too slow?
ok 70 cfg80211-inform-bss
KTAP version 1
# Subtest: cfg80211-ie-generation
# module: cfg80211_tests
1..2
KTAP version 1
# Subtest: test_gen_new_ie
oaction: accept multicast without MFP
should say
"ok 4 public action: accept ..."
instead, I think?
johannes
Jakub Kicinski
maintainerQuoted 3 lines · On Thu, 13 Feb 2025 19:22:04 +0100 Johannes Berg wrote:
Quoted 1 line
We hit 3 failures in the last 3 days.
Four, actually, it seems? ;-)
Yup! I jinxed it, it failed again after I sent the report :)
Quoted 19 lines
Quoted 3 lines
https://netdev.bots.linux.dev/flakes.html?min-flip=0&tn-needle=cfg80211-ie-generation
But the kunit stuff likes to break because of cross-tests corruptions :(
Hmm. Let's say ...
https://netdev-3.bots.linux.dev/kunit/results/987921/kunit-test.log
is your serial console simply too slow?
ok 70 cfg80211-inform-bss
KTAP version 1
# Subtest: cfg80211-ie-generation
# module:cfg80211_tests
1..2
KTAP version 1
# Subtest:test_gen_new_ie
oaction: accept multicast without MFP
Quoted 5 lines
should say
"ok 4 public action: accept ..."
instead, I think?
Oh, that's annoying :( Thanks for investigating.
I think the CI runs when the machine is overloaded by builds. I'll add some safety for that.
Quoted 6 lines · Jakub Kicinski
Quoted 3 lines · On Thu, 13 Feb 2025 19:22:04 +0100 Johannes Berg wrote:
Quoted 1 line
We hit 3 failures in the last 3 days.
Four, actually, it seems? ;-)
Yup! I jinxed it, it failed again after I sent the report :)
:)
It's weird that it happens in this test, or are others similar?
Quoted 15 lines
Quoted 15 lines
https://netdev-3.bots.linux.dev/kunit/results/987921/kunit-test.log
is your serial console simply too slow?
ok 70 cfg80211-inform-bss
KTAP version 1
# Subtest: cfg80211-ie-generation
# module:cfg80211_tests
1..2
KTAP version 1
# Subtest:test_gen_new_ie
oaction: accept multicast without MFPshould say
Quoted 7 lines
Quoted 3 lines
"ok 4 public action: accept ..."
instead, I think?
Oh, that's annoying :( Thanks for investigating.
I think the CI runs when the machine is overloaded by builds. I'll add some safety for that.
It almost feels like it shouldn't matter - couldn't qemu just kind of 'pause' the VM when the serial port isn't keeping up? I think you're using qemu? But I guess I could also see why that might not be something you want in other use cases...
Not sure, but it really seems more related to the output (buffering) than anything else.
Are you using the tools/testing/kunit/kunit.py script?
johannes
Jakub Kicinski
maintainerQuoted 9 lines · On Thu, 13 Feb 2025 20:03:04 +0100 Johannes Berg wrote:
Quoted 3 lines
Oh, that's annoying :( Thanks for investigating.
I think the CI runs when the machine is overloaded by builds. I'll add some safety for that.It almost feels like it shouldn't matter - couldn't qemu just kind of 'pause' the VM when the serial port isn't keeping up? I think you're using qemu? But I guess I could also see why that might not be something you want in other use cases...
Quoted 4 lines
Not sure, but it really seems more related to the output (buffering) than anything else.
Are you using the
tools/testing/kunit/kunit.pyscript?
Yes. We do set:
Environment=PYTHONUNBUFFERED=true
So that the logs reach journald without a huge delay. I guess it disturbs the kunit wrapper.