After the last PR, I was focussing on monitoring simult_flows.sh -- the recent behaviour changes could have affected it -- and forgot to look at the others, sorry about that...

It looks like our CI doesn't have this issue, but the builds are less frequent [1]. I'm going to investigate that ASAP.

[1] https://ci-results.mptcp.dev/flakes.html?branch=export-net

Cheers,
Matt

Signature · 1 line

Sponsored by the NGI0 Core fund.

Matthieu Baerts

maintainer

2025-01-13 16:08 UTC #2 lore raw

Hi Jakub,

Quoted 8 lines · Matthieu Baerts

Hi Jakub,
(+cc MPTCP list)
Quoted 2 lines · Jakub Kicinski
Unfortunately mptcp_connect.sh has started flaking again :( Looks like it started around Dec 30th, so one of the recent PRs

Quoted 8 lines

Thank you for the heads-up!
After the last PR, I was focussing on monitoring simult_flows.sh -- the recent behaviour changes could have affected it -- and forgot to look at the others, sorry about that...
It looks like our CI doesn't have this issue, but the builds are less frequent [1]. I'm going to investigate that ASAP.

It was not easy to reproduce it, but Paolo managed to find a fix for it [1]!

Out of curiosity, is the netdev CI not too overloaded? To reproduce this issue on my side, the host had to be quite busy: not dying with stress-ng using all resources, but still competing with many other processes, e.g. a kernel compilation running in parallel. I'm not complaining here, because this situation helped finding this important issue, but just curious about what to expect, especially for more "sensitive" tests :)

On the MPTCP CI, some unessarry KConfig are disabled and ccache is used to reduce the build time. Also, RETPOLINE is disabled (+ vng --append mitigations=off) to save some CPU cycles during the tests.

[1]
https://lore.kernel.org/netdev/20250113-net-mptcp-connect-st-flakes-v1-0-0d986ee7b1b6@kernel.org/T/

Cheers,
Matt

Signature · 1 line

Sponsored by the NGI0 Core fund.

Jakub Kicinski

author maintainer

2025-01-13 19:44 UTC #3 lore raw

Quoted 9 lines · On Mon, 13 Jan 2025 17:08:02 +0100 Matthieu Baerts wrote:

It was not easy to reproduce it, but Paolo managed to find a fix for it [1]!
Out of curiosity, is the netdev CI not too overloaded? To reproduce this issue on my side, the host had to be quite busy: not dying with stress-ng using all resources, but still competing with many other processes, e.g. a kernel compilation running in parallel. I'm not complaining here, because this situation helped finding this important issue, but just curious about what to expect, especially for more "sensitive" tests :)

The tests my overlap with other workers still building their kernel. ccache is enabled.

Quoted 3 lines

On the MPTCP CI, some unessarry KConfig are disabled and ccache is used to reduce the build time. Also, RETPOLINE is disabled (+ vng --append mitigations=off) to save some CPU cycles during the tests.

Could help. Let's put it on the list of things to do once we migrate the CI to a public set of hosts :)

Matthieu Baerts

maintainer

2025-01-14 14:18 UTC #4 lore raw

Hi Jakub,

Quoted 11 lines · Jakub Kicinski

Quoted 9 lines · On Mon, 13 Jan 2025 17:08:02 +0100 Matthieu Baerts wrote:
It was not easy to reproduce it, but Paolo managed to find a fix for it [1]!
Out of curiosity, is the netdev CI not too overloaded? To reproduce this issue on my side, the host had to be quite busy: not dying with stress-ng using all resources, but still competing with many other processes, e.g. a kernel compilation running in parallel. I'm not complaining here, because this situation helped finding this important issue, but just curious about what to expect, especially for more "sensitive" tests :)

Quoted 2 lines

The tests my overlap with other workers still building their kernel. ccache is enabled.

Good to know, thank you.

Quoted 6 lines

Quoted 3 lines
On the MPTCP CI, some unessarry KConfig are disabled and ccache is used to reduce the build time. Also, RETPOLINE is disabled (+ vng --append mitigations=off) to save some CPU cycles during the tests.
Could help. Let's put it on the list of things to do once we migrate the CI to a public set of hosts :)

Good idea :)

Cheers,
Matt

Signature · 1 line

Sponsored by the NGI0 Core fund.