Skip to contentSkip to conversation

kunit/cfg80211-ie-generation flaking ?

TEST Jakub Kicinski 2 participants 5 messages Feb 13, 2025 – Feb 14, 2025

Subsystems (2)
lore
Share & tools
lore
#1 lore raw
Quoted 3 lines · Jakub Kicinski

Hi!

Do you see any problems with the cfg80211-ie-generation kUnit test?

Nope, never, we must run it dozens of times a day ...

Quoted 1 line

We hit 3 failures in the last 3 days.

Four, actually, it seems? ;-)

Quoted 3 lines

https://netdev.bots.linux.dev/flakes.html?min-flip=0&tn-needle=cfg80211-ie-generation

But the kunit stuff likes to break because of cross-tests corruptions :(

Hmm. Let's say ...

https://netdev-3.bots.linux.dev/kunit/results/987921/kunit-test.log

is your serial console simply too slow?

ok 70 cfg80211-inform-bss
KTAP version 1
# Subtest: cfg80211-ie-generation
# module: cfg80211_tests
1..2
KTAP version 1
# Subtest: test_gen_new_ie
oaction: accept multicast without MFP

should say

"ok 4 public action: accept ..."

instead, I think?

johannes

author maintainer
#2 lore raw
Quoted 3 lines · On Thu, 13 Feb 2025 19:22:04 +0100 Johannes Berg wrote:
Quoted 1 line

We hit 3 failures in the last 3 days.

Four, actually, it seems? ;-)

Yup! I jinxed it, it failed again after I sent the report :)

Quoted 19 lines
Quoted 3 lines

https://netdev.bots.linux.dev/flakes.html?min-flip=0&tn-needle=cfg80211-ie-generation

But the kunit stuff likes to break because of cross-tests corruptions :(

Hmm. Let's say ...

https://netdev-3.bots.linux.dev/kunit/results/987921/kunit-test.log

is your serial console simply too slow?

ok 70 cfg80211-inform-bss
KTAP version 1
# Subtest: cfg80211-ie-generation
# module: cfg80211_tests
1..2
KTAP version 1
# Subtest: test_gen_new_ie
oaction: accept multicast without MFP

Quoted 5 lines

should say

"ok 4 public action: accept ..."

instead, I think?

Oh, that's annoying :( Thanks for investigating.
I think the CI runs when the machine is overloaded by builds. I'll add some safety for that.

#3 lore raw
Quoted 6 lines · Jakub Kicinski
Quoted 3 lines · On Thu, 13 Feb 2025 19:22:04 +0100 Johannes Berg wrote:
Quoted 1 line

We hit 3 failures in the last 3 days.

Four, actually, it seems? ;-)

Yup! I jinxed it, it failed again after I sent the report :)

:)

It's weird that it happens in this test, or are others similar?

Quoted 15 lines
Quoted 15 lines

https://netdev-3.bots.linux.dev/kunit/results/987921/kunit-test.log

is your serial console simply too slow?

ok 70 cfg80211-inform-bss
KTAP version 1
# Subtest: cfg80211-ie-generation
# module: cfg80211_tests
1..2
KTAP version 1
# Subtest: test_gen_new_ie
oaction: accept multicast without MFP

should say

Quoted 7 lines
Quoted 3 lines

"ok 4 public action: accept ..."

instead, I think?

Oh, that's annoying :( Thanks for investigating.
I think the CI runs when the machine is overloaded by builds. I'll add some safety for that.

It almost feels like it shouldn't matter - couldn't qemu just kind of 'pause' the VM when the serial port isn't keeping up? I think you're using qemu? But I guess I could also see why that might not be something you want in other use cases...

Not sure, but it really seems more related to the output (buffering) than anything else.

Are you using the tools/testing/kunit/kunit.py script?

johannes

author maintainer
#4 lore raw
Quoted 9 lines · On Thu, 13 Feb 2025 20:03:04 +0100 Johannes Berg wrote:
Quoted 3 lines

Oh, that's annoying :( Thanks for investigating.
I think the CI runs when the machine is overloaded by builds. I'll add some safety for that.

It almost feels like it shouldn't matter - couldn't qemu just kind of 'pause' the VM when the serial port isn't keeping up? I think you're using qemu? But I guess I could also see why that might not be something you want in other use cases...

Quoted 4 lines

Not sure, but it really seems more related to the output (buffering) than anything else.

Are you using the tools/testing/kunit/kunit.py script?

Yes. We do set:

Environment=PYTHONUNBUFFERED=true

So that the logs reach journald without a huge delay. I guess it disturbs the kunit wrapper.