net-next is OPEN
Subsystems (4)
Jakub Kicinski
maintainerHi,
net-next is open again, and accepting changes for v6.9.
Over the merge window I spent some time stringing together selftest runner for netdev: https://netdev.bots.linux.dev/status.html
It is now connected to patchwork, meaning there should be a check posted to each patch indicating whether selftests have passed or not.
If you authored any net or drivers/net selftests, please look around and see if they are passing. If not - send patches or LMK what I need to do to make them pass on the runner.. Make sure to scroll down to the "Not reporting to patchwork" section.
Simon Horman
reviewerQuoted 8 lines · Jakub Kicinski
Hi,
net-next is open again, and accepting changes for v6.9.
Over the merge window I spent some time stringing together selftest runner for netdev: https://netdev.bots.linux.dev/status.html
It is now connected to patchwork, meaning there should be a check posted to each patch indicating whether selftests have passed or not.
Excellent, I'm really pleased to see this.
Quoted 4 lines
If you authored any net or
drivers/netselftests, please look around and see if they are passing. If not - send patches or LMK what I need to do to make them pass on the runner.. Make sure to scroll down to the "Not reporting to patchwork" section.
Quoted 8 lines · Jakub Kicinski
Hi,
net-next is open again, and accepting changes for v6.9.
Over the merge window I spent some time stringing together selftest runner for netdev: https://netdev.bots.linux.dev/status.html
It is now connected to patchwork, meaning there should be a check posted to each patch indicating whether selftests have passed or not.
Cool! Does it group a couple of patches together and run the tests or run for each patch separately?
Quoted 5 lines
If you authored any net or
drivers/netselftests, please look around and see if they are passing. If not - send patches or LMK what I need to do to make them pass on the runner.. Make sure to scroll down to the "Not reporting to patchwork" section.
Yes, this could push the selftest authors help checking all the failures.
[1] https://patchwork.kernel.org/project/netdevbpf/patch/[email protected]/
Thanks
Hangbin
Quoted 4 lines · Jakub Kicinski
If you authored any net or
drivers/netselftests, please look around and see if they are passing. If not - send patches or LMK what I need to do to make them pass on the runner.. Make sure to scroll down to the "Not reporting to patchwork" section.
A whole bunch of them fail because of no IPv6 support in the runner kernel. E.g. this from bridge-mdb.sh[0]:
# Error: Rule family not supported.
# Error: Rule family not supported.
# sysctl: cannot stat /proc/sys/net/ipv6/conf/all/forwarding: No such file or directory # sysctl: cannot stat /proc/sys/net/ipv6/conf/all/forwarding: No such file or directory # RTNETLINK answers: Operation not supported
I'm surprised any passed at all, it's super common for tests to validate their topology by pinging through, but I guess it's often just IPv4. I think the fix is just this?
$ scripts/config -k -m CONFIG_IPV6
There are also a bunch of missing qdiscs, e.g. in [1], [2]. To fix:
$ scripts/config -k -m CONFIG_NET_SCH_TBF
$ scripts/config -k -m CONFIG_NET_SCH_PRIO
$ scripts/config -k -m CONFIG_NET_SCH_ETSRegarding sch_red.sh[3], I worry the test will be noisy, and suspect it does not make sense to run it in automated fashion. But if you think it's worth a try:
$ scripts/config -k -m CONFIG_NET_SCH_RED
Then there are a bunch of missing netdevices. VXLAN[4]:
$ scripts/config -k -m CONFIG_VXLAN
and GRE [5], which I think needs all of these:
$ scripts/config -k -m CONFIG_NET_IPIP
$ scripts/config -k -m CONFIG_IPV6_GRE
$ scripts/config -k -m CONFIG_NET_IPGRE_DEMUX
$ scripts/config -k -m CONFIG_NET_IPGREAnd TC actions [6]. I think the following will be necessary for some of the tests (we enable BPF as well internally).
$ scripts/config -k -m CONFIG_NET_ACT_GACT
$ scripts/config -k -m CONFIG_NET_ACT_MIRRED
$ scripts/config -k -m CONFIG_NET_ACT_SAMPLE
$ scripts/config -k -m CONFIG_NET_ACT_VLAN
$ scripts/config -k -m CONFIG_NET_ACT_SKBEDIT
$ scripts/config -k -m CONFIG_NET_ACT_PEDIT
$ scripts/config -k -m CONFIG_NET_ACT_POLICEHopefully the above should clean up the results a bit, I can take another sweep afterwards.
[0] https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/14-bridge-mdb-sh/stdout
[1] https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/17-sch-ets-sh/stdout
[2] https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/33-sch-tbf-prio-sh/stdout
[3] https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/21-sch-red-sh/stdout
[4] https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/27-mirror-gre-changes-sh/stdout
[5] https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/18-vxlan-bridge-1d-sh/stdout
[6] https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/42-pedit-l4port-sh/stdout
Quoted 58 lines · Petr Machata
Quoted 4 lines · Jakub Kicinski
If you authored any net or
drivers/netselftests, please look around and see if they are passing. If not - send patches or LMK what I need to do to make them pass on the runner.. Make sure to scroll down to the "Not reporting to patchwork" section.A whole bunch of them fail because of no IPv6 support in the runner kernel. E.g. this from bridge-mdb.sh[0]:
# Error: Rule family not supported.
# Error: Rule family not supported.
# sysctl: cannot stat/proc/sys/net/ipv6/conf/all/forwarding: No such file or directory # sysctl: cannot stat/proc/sys/net/ipv6/conf/all/forwarding: No such file or directory # RTNETLINK answers: Operation not supportedI'm surprised any passed at all, it's super common for tests to validate their topology by pinging through, but I guess it's often just IPv4. I think the fix is just this?
$ scripts/config -k -m CONFIG_IPV6There are also a bunch of missing qdiscs, e.g. in [1], [2]. To fix:
$ scripts/config -k -m CONFIG_NET_SCH_TBF $ scripts/config -k -m CONFIG_NET_SCH_PRIO $ scripts/config -k -m CONFIG_NET_SCH_ETSRegarding sch_red.sh[3], I worry the test will be noisy, and suspect it does not make sense to run it in automated fashion. But if you think it's worth a try:
$ scripts/config -k -m CONFIG_NET_SCH_REDThen there are a bunch of missing netdevices. VXLAN[4]:
$ scripts/config -k -m CONFIG_VXLANand GRE [5], which I think needs all of these:
$ scripts/config -k -m CONFIG_NET_IPIP $ scripts/config -k -m CONFIG_IPV6_GRE $ scripts/config -k -m CONFIG_NET_IPGRE_DEMUX $ scripts/config -k -m CONFIG_NET_IPGREAnd TC actions [6]. I think the following will be necessary for some of the tests (we enable BPF as well internally).
$ scripts/config -k -m CONFIG_NET_ACT_GACT $ scripts/config -k -m CONFIG_NET_ACT_MIRRED $ scripts/config -k -m CONFIG_NET_ACT_SAMPLE $ scripts/config -k -m CONFIG_NET_ACT_VLAN $ scripts/config -k -m CONFIG_NET_ACT_SKBEDIT $ scripts/config -k -m CONFIG_NET_ACT_PEDIT $ scripts/config -k -m CONFIG_NET_ACT_POLICEHopefully the above should clean up the results a bit, I can take another sweep afterwards.
Quoted 8 lines
[0] https://urldefense.com/v3/__https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/14-bridge-mdb-sh/stdout__;!!I9LPvj3b!Eb-NZpjOY2wkTb2sApNj0Hx-II6xvFO688SZN7feUAjC_6RFAz3dmeR3LzssrLqm_Kr7rJEalB7DvzQR$
[1] https://urldefense.com/v3/__https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/17-sch-ets-sh/stdout__;!!I9LPvj3b!Eb-NZpjOY2wkTb2sApNj0Hx-II6xvFO688SZN7feUAjC_6RFAz3dmeR3LzssrLqm_Kr7rJEalDWGO6rP$
[2] https://urldefense.com/v3/__https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/33-sch-tbf-prio-sh/stdout__;!!I9LPvj3b!Eb-NZpjOY2wkTb2sApNj0Hx-II6xvFO688SZN7feUAjC_6RFAz3dmeR3LzssrLqm_Kr7rJEalMVxxlTD$
[3] https://urldefense.com/v3/__https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/21-sch-red-sh/stdout__;!!I9LPvj3b!Eb-NZpjOY2wkTb2sApNj0Hx-II6xvFO688SZN7feUAjC_6RFAz3dmeR3LzssrLqm_Kr7rJEalMfr0cOk$
[4] https://urldefense.com/v3/__https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/27-mirror-gre-changes-sh/stdout__;!!I9LPvj3b!Eb-NZpjOY2wkTb2sApNj0Hx-II6xvFO688SZN7feUAjC_6RFAz3dmeR3LzssrLqm_Kr7rJEalP3lTojb$
[5] https://urldefense.com/v3/__https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/18-vxlan-bridge-1d-sh/stdout__;!!I9LPvj3b!Eb-NZpjOY2wkTb2sApNj0Hx-II6xvFO688SZN7feUAjC_6RFAz3dmeR3LzssrLqm_Kr7rJEalBLEoTPs$
[6] https://urldefense.com/v3/__https://netdev-2.bots.linux.dev/vmksft-forwarding/results/433341/42-pedit-l4port-sh/stdout__;!!I9LPvj3b!Eb-NZpjOY2wkTb2sApNj0Hx-II6xvFO688SZN7feUAjC_6RFAz3dmeR3LzssrLqm_Kr7rJEalOO5rTam$
Hi
Probably also missing is CONFIG_GENEVE
81-l2-tos-ttl-inherit-sh check operation with gre, vxlan and geneve, but modprobes first if the module is actually available. If it isn't, it just continues and doesn't fail.
Also there seems to be something wrong with ending, see https://netdev-2.bots.linux.dev/vmksft-net/results/433200/81-l2-tos-ttl-inherit-sh
The test outputs the results in a table with box drawing characters (┌─┬┐├─┼┤└─┴┘)
BR
Matthias
Quoted 3 lines · Matthias May
Also there seems to be something wrong with ending, see
https://netdev-2.bots.linux.dev/vmksft-net/results/433200/81-l2-tos-ttl-inherit-sh
The test outputs the results in a table with box drawing characters (┌─┬┐├─┼┤└─┴┘)
It looks like whatever is serving the output should use MIME of "text/plain;charset=UTF-8" instead of just "text/plain".
Jakub Kicinski
maintainerQuoted 7 lines · On Tue, 23 Jan 2024 16:45:55 +0800 Hangbin Liu wrote:
Quoted 4 lines
Over the merge window I spent some time stringing together selftest runner for netdev: https://netdev.bots.linux.dev/status.html
It is now connected to patchwork, meaning there should be a check posted to each patch indicating whether selftests have passed or not.Cool! Does it group a couple of patches together and run the tests or run for each patch separately?
It groups all patches outstanding in patchwork (which build cleanly). I'm hoping we could also do HW testing using this setup, so batching is a must. Not 100% sure it's the right direction for SW testing but there's one way to find out :)
Jakub Kicinski
maintainerQuoted 8 lines · On Tue, 23 Jan 2024 14:38:24 +0100 Petr Machata wrote:
Quoted 3 lines · Matthias May
Also there seems to be something wrong with ending, see
https://netdev-2.bots.linux.dev/vmksft-net/results/433200/81-l2-tos-ttl-inherit-sh
The test outputs the results in a table with box drawing characters (┌─┬┐├─┼┤└─┴┘)It looks like whatever is serving the output should use MIME of "text/plain;charset=UTF-8" instead of just "text/plain".
😮️ interesting. The table characters are not part of ASCII, right? So it must be using some extended charset. Firefox hid the option to tweak encoding tho so IDK what it actually uses :S
Jakub Kicinski
maintainerQuoted 7 lines · On Tue, 23 Jan 2024 10:55:09 +0100 Petr Machata wrote:
Quoted 4 lines
If you authored any net or
drivers/netselftests, please look around and see if they are passing. If not - send patches or LMK what I need to do to make them pass on the runner.. Make sure to scroll down to the "Not reporting to patchwork" section.A whole bunch of them fail because of no IPv6 support in the runner kernel. E.g. this from bridge-mdb.sh[0]:
Thanks a lot for investigating! I take it that you're looking at forwarding? Please send a patch to add the missing configs to
tools/testing/selftests/net/forwarding/config
The runner uses that to configure the kernel on top of defconfig.
Unless I'm doing it wrong and the sub-directories are supposed to inherit the parent directory's config? So net/forwarding/ should be built with net/'s config? I could not find the info in docs, does anyone know?
Quoted 11 lines · Jakub Kicinski
Quoted 8 lines · On Tue, 23 Jan 2024 14:38:24 +0100 Petr Machata wrote:
Quoted 3 lines · Matthias May
Also there seems to be something wrong with ending, see
https://netdev-2.bots.linux.dev/vmksft-net/results/433200/81-l2-tos-ttl-inherit-sh
The test outputs the results in a table with box drawing characters (┌─┬┐├─┼┤└─┴┘)It looks like whatever is serving the output should use MIME of "text/plain;charset=UTF-8" instead of just "text/plain".
😮️ interesting. The table characters are not part of ASCII, right?
Yeah, the table is UTF-8.
Quoted 2 lines
So it must be using some extended charset. Firefox hid the option to tweak encoding tho so IDK what it actually uses :S
My guess would be ISO-8859-1, but dunno. The developer console shows Firefox has had to guess the encoding, but not what guess it made.
It seems weird to guess anything but UTF-8 in this day and age. Maybe the logic is that modern web has correct content-type, because it needs UTF-8, so if it doesn't declare encoding, it's something ancient.
Jakub Kicinski
maintainerQuoted 10 lines · On Tue, 23 Jan 2024 17:05:12 +0100 Petr Machata wrote:
Quoted 4 lines
Quoted 2 lines
It looks like whatever is serving the output should use MIME of "text/plain;charset=UTF-8" instead of just "text/plain".
😮️ interesting. The table characters are not part of ASCII, right?
Yeah, the table is UTF-8.
Quoted 2 lines
So it must be using some extended charset. Firefox hid the option to tweak encoding tho so IDK what it actually uses :S
Quoted 6 lines
My guess would be ISO-8859-1, but dunno. The developer console shows Firefox has had to guess the encoding, but not what guess it made.
It seems weird to guess anything but UTF-8 in this day and age. Maybe the logic is that modern web has correct content-type, because it needs UTF-8, so if it doesn't declare encoding, it's something ancient.
Ah, sorry, I read your message backwards. I remember fixing the encoding on the NIPA nginx to make it utf-8 so I thought you said it's utf-8 and should be something else... But the runner has a separate instance, and there the charset is indeed not set. Fixed now!
Quoted 6 lines · Jakub Kicinski
Quoted 5 lines · On Tue, 23 Jan 2024 16:45:55 +0800 Hangbin Liu wrote:
Quoted 4 lines
Over the merge window I spent some time stringing together selftest runner for netdev: https://netdev.bots.linux.dev/status.html
It is now connected to patchwork, meaning there should be a check posted to each patch indicating whether selftests have passed or not.
Quoted 8 lines
Quoted 2 lines
Cool! Does it group a couple of patches together and run the tests or run for each patch separately?
It groups all patches outstanding in patchwork (which build cleanly). I'm hoping we could also do HW testing using this setup, so batching is a must. Not 100% sure it's the right direction for SW testing but there's one way to find out :)
Really cool. Thanks for spending time to make this happen.
Scanning the tests I wrote, I think most of the failures are config related. e.g., fib-nexthops.sh needs MPLS and those config settings are enabled in tools/testing/selftests/net/config.
Another one, fcnal-test, needs nettest built. From
https://netdev-2.bots.linux.dev/vmksft-net/results/432660/36-fcnal-test-sh:
# which: no nettest in (/home/virtme/tools/fs/bin:/home/virtme/tools/fs/sbin:/home/virtme/tools/fs/usr/bin:/home/virtme/tools/fs/usr/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin)
It is in the Makefile, so how should that dependency be defined for this new environment?
Finally, how can people replicate this setup to validate changes to get everything work correctly?
Quoted 6 lines · Jakub Kicinski
Quoted 5 lines · On Tue, 23 Jan 2024 10:55:09 +0100 Petr Machata wrote:
Quoted 4 lines
If you authored any net or
drivers/netselftests, please look around and see if they are passing. If not - send patches or LMK what I need to do to make them pass on the runner.. Make sure to scroll down to the "Not reporting to patchwork" section.
Quoted 7 lines
Quoted 2 lines
A whole bunch of them fail because of no IPv6 support in the runner kernel. E.g. this from bridge-mdb.sh[0]:
Thanks a lot for investigating! I take it that you're looking at forwarding? Please send a patch to add the missing configs to
tools/testing/selftests/net/forwarding/config
OK.
Quoted 6 lines
The runner uses that to configure the kernel on top of defconfig.
Unless I'm doing it wrong and the sub-directories are supposed to inherit the parent directory's config? So
net/forwarding/should be built with net/'s config? I could not find the info in docs, does anyone know?
I don't think they are, net/config defines CONFIG_VXLAN, but then the vxlan tests still complain about unknown device type. Though maybe there's another device type that it's missing...
What do I do to feed the config file to some build script to get a kernel image to test? I can of course just do something like cat config | xargs -n1 scripts/config -m, but I expect there's some automation for it and I just can't find it.
Jakub Kicinski
maintainerQuoted 5 lines · On Tue, 23 Jan 2024 18:04:19 +0100 Petr Machata wrote:
Quoted 4 lines
Unless I'm doing it wrong and the sub-directories are supposed to inherit the parent directory's config? So
net/forwarding/should be built with net/'s config? I could not find the info in docs, does anyone know?
Quoted 8 lines
I don't think they are, net/config defines
CONFIG_VXLAN, but then the vxlan tests still complain about unknown device type. Though maybe there's another device type that it's missing...What do I do to feed the config file to some build script to get a kernel image to test? I can of course just do something like cat config | xargs -n1
scripts/config-m, but I expect there's some automation for it and I just can't find it.
The CI script is based on virtme-ng. So it does this:
# $target is net or net/forwarding or drivers/net/bonding etc. make mrproper
vng -v -b -f tools/testing/selftests/$target
# build the scripts make headers make -C tools/testing/selftests/$target
vng -v -r arch/x86/boot/bzImage --user root
# inside the VM make -C tools/testing/selftests TARGETS=$target run_tests
https://github.com/kuba-moo/nipa/blob/master/contest/remote/vmksft.py#L138
You're right, it definitely does not "inherit" net's config when running forwarding/net. I can easily make it do so, but I'm not clear what the expectation from the kselftest subsystem is. Because if other testers (people testing stable, KernelCI etc. et.c) don't "inherit" we better fill in the config completely so that the tests pass for everyone.
Jakub Kicinski
maintainerQuoted 6 lines · On Tue, 23 Jan 2024 09:51:07 -0700 David Ahern wrote:
Really cool. Thanks for spending time to make this happen.
Scanning the tests I wrote, I think most of the failures are config related. e.g.,
fib-nexthops.shneeds MPLS and those config settings are enabled intools/testing/selftests/net/config.
Quoted 8 lines
Another one, fcnal-test, needs nettest built. From
https://netdev-2.bots.linux.dev/vmksft-net/results/432660/36-fcnal-test-sh:# which: no nettest in (/home/virtme/tools/fs/bin:/home/virtme/tools/fs/sbin:/home/virtme/tools/fs/usr/bin:/home/virtme/tools/fs/usr/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin)
It is in the
Makefile, so how should that dependency be defined for this new environment?
We run the tests with
make -C tools/testing/selftests TARGETS=net run_tests
the binary is there:
$ ls tools/testing/selftests/net/nettest tools/testing/selftests/net/nettest
The script does:
if ! which nettest >/dev/null; then
PATH=$PWD:$PATH
if ! which nettest >/dev/null; then
echo "'nettest' command not found; skipping tests" exit $ksft_skip
fi
fi
But that's not exactly the message that gets printed up top. Is it coming from lib.sh? Not sure..
Quoted 2 lines
Finally, how can people replicate this setup to validate changes to get everything work correctly?
You can run the exact scripts the CI runs, they are pretty standalone. But they don't do any magic, really, I'd start by trying the steps I outlined in
https://lore.kernel.org/all/[email protected]/
and if you can't repro the failure LMK, I'll share more instructions on replicating exactly.
Quoted 13 lines · Jakub Kicinski
Quoted 12 lines · On Tue, 23 Jan 2024 09:51:07 -0700 David Ahern wrote:
Really cool. Thanks for spending time to make this happen.
Scanning the tests I wrote, I think most of the failures are config related. e.g.,
fib-nexthops.shneeds MPLS and those config settings are enabled intools/testing/selftests/net/config.Another one, fcnal-test, needs nettest built. From
https://netdev-2.bots.linux.dev/vmksft-net/results/432660/36-fcnal-test-sh:# which: no nettest in (/home/virtme/tools/fs/bin:/home/virtme/tools/fs/sbin:/home/virtme/tools/fs/usr/bin:/home/virtme/tools/fs/usr/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin)
Quoted 6 lines
Quoted 2 lines
It is in the
Makefile, so how should that dependency be defined for this new environment?We run the tests with
make -C
tools/testing/selftestsTARGETS=netrun_tests
thanks for the tip to direclty run the tests.
Quoted 8 lines
the binary is there:
$ ls tools/testing/selftests/net/nettest tools/testing/selftests/net/nettestThe script does:
Quoted 7 lines
if ! which nettest >/dev/null; then
PATH=$PWD:$PATH
if ! which nettest >/dev/null; then
echo "'nettest' command not found; skipping tests" exit $ksft_skip
fi
fi
This fixes the PATH problem:
(quoted diff) +0 -0
b/tools/testing/selftests/net/fcnal-test.sh
index a8ad92850e63..0c5ac117155d 100755
tools/testing/selftests/net/fcnal-test.sh +2 -0
@@ -37,6 +37,8 @@ # # server / client nomenclature relative to ns-A +PATH=$PWD:$PWD/tools/testing/selftests/net:$PATH + VERBOSE=0 NSA_DEV=eth1
but given the permutations it does, the script needs lot more than 45 seconds. This does the trick, but not sure how to bump the timeout for a specific test.
(quoted diff) +0 -0
b/tools/testing/selftests/kselftest/runner.sh
index cc9c846585f0..9e7dcb728249 100644
tools/testing/selftests/kselftest/runner.sh +1 -1
@@ -9,7 +9,7 @@ export per_test_logging= # Defaults for "settings" file fields: # "timeout" how many seconds to let each test run before failing. -export kselftest_default_timeout=45 +export kselftest_default_timeout=3600 # There isn't a shell-agnostic way to find the path of a sourced file, # so we must rely on BASE_DIR being set to find other tools.
Paolo Abeni
maintainerOn Tue, 2024-01-23 at 22:20 -0700, David Ahern wrote:
[...]
Quoted 3 lines
the script needs lot more than 45
seconds. This does the trick, but not sure how to bump the timeout for a specific test.
You can set a test-group-specific timeout touching the 'settings' file in the relevant directory. Note that for 'net' self-tests the timeout is currently 3600 seconds (for each test).
AFAIK there is no way to set a single-test-specific timeout, without running that specific test individually:
make install TARGETS=net./kselftest_install/run_kselftest.sh -o <timeout in sec> -t net:fcnal-test.sh
Cheers,
Paolo
Jakub Kicinski
maintainerQuoted 2 lines · On Wed, 24 Jan 2024 09:22:15 +0100 Paolo Abeni wrote:
Quoted 1 line · David Ahern
thanks for the tip to direclty run the tests.
Actually, I simplified this one a little bit. What we do now is:
make -C tools/testing/selftests TARGETS={target} TEST_PROGS={prog} TEST_GEN_PROGS="" run_tests
This let's us run tests individually. This way we can put them in a FIFO and have multiple workers run them in parallel.
Quoted 8 lines
Quoted 3 lines
the script needs lot more than 45
seconds. This does the trick, but not sure how to bump the timeout for a specific test.You can set a test-group-specific timeout touching the 'settings' file in the relevant directory. Note that for 'net' self-tests the timeout is currently 3600 seconds (for each test).
Quoted 5 lines
AFAIK there is no way to set a single-test-specific timeout, without running that specific test individually:
make install TARGETS=net
./kselftest_install/run_kselftest.sh-o <timeout in sec> -t net:fcnal-test.sh
Yet another thing run_kselftest.sh can do that make can't :( We should probably bite the bullet and switch to it at some point.
David, I applied your diff locally, hopefully I did it in time for the 7am Pacific run to have it included :)
Quoted 10 lines · David Ahern
Quoted 9 lines · Jakub Kicinski
Quoted 7 lines · On Tue, 23 Jan 2024 16:45:55 +0800 Hangbin Liu wrote:
Quoted 4 lines
Over the merge window I spent some time stringing together selftest runner for netdev: https://netdev.bots.linux.dev/status.html
It is now connected to patchwork, meaning there should be a check posted to each patch indicating whether selftests have passed or not.Cool! Does it group a couple of patches together and run the tests or run for each patch separately?
Quoted 7 lines
Quoted 5 lines
It groups all patches outstanding in patchwork (which build cleanly). I'm hoping we could also do HW testing using this setup, so batching is a must. Not 100% sure it's the right direction for SW testing but there's one way to find out :)
Really cool. Thanks for spending time to make this happen.
Just to add to the choir: this is fantastic, thanks!
Hopefully it will encourage people to add kselftests, kunit tests or other kinds that now get continuous coverage.
Going through the failing ksft-net series on
https://netdev.bots.linux.dev/status.html, all the tests I'm responsible seem to be passing.
Jakub Kicinski
maintainerQuoted 2 lines · On Wed, 24 Jan 2024 07:07:55 -0800 Jakub Kicinski wrote:
David, I applied your diff locally, hopefully I did it in time for the 7am Pacific run to have it included :)
This is the latest run:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/435141/1-fcnal-test-sh/stdout
the nettest warning is indeed gone, but the failures are the same:
$ grep FAIL stdout # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL] # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL]
:(
Jakub Kicinski
maintainerQuoted 8 lines · On Wed, 24 Jan 2024 10:59:36 -0500 Willem de Bruijn wrote:
Quoted 7 lines · David Ahern
Quoted 5 lines · Jakub Kicinski
It groups all patches outstanding in patchwork (which build cleanly). I'm hoping we could also do HW testing using this setup, so batching is a must. Not 100% sure it's the right direction for SW testing but there's one way to find out :)
Quoted 6 lines
Quoted 1 line
Really cool. Thanks for spending time to make this happen.
Just to add to the choir: this is fantastic, thanks!
Hopefully it will encourage people to add kselftests, kunit tests or other kinds that now get continuous coverage.
Fingers crossed :)
Quoted 3 lines
Going through the failing ksft-net series on
https://netdev.bots.linux.dev/status.html, all the tests I'm responsible seem to be passing.
Here's a more handy link filtered down to failures (clicking on the test counts links here):
I have been attributing the udpg[rs]o and timestamp tests to you, but I haven't actually checked.. are they not yours? :)
Quoted 9 lines · Jakub Kicinski
Quoted 2 lines · On Wed, 24 Jan 2024 07:07:55 -0800 Jakub Kicinski wrote:
David, I applied your diff locally, hopefully I did it in time for the 7am Pacific run to have it included :)
This is the latest run:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/435141/1-fcnal-test-sh/stdout
the nettest warning is indeed gone, but the failures are the same:
yep, I will send a formal patch. I see the timeout is high enough, so good there.
Quoted 8 lines
$ grep FAIL stdout # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL] # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL]:(
known problems. I can disable the tests for now so we avoid regressions, and add to the TO-DO list for someone with time.
Jakub Kicinski
maintainerQuoted 8 lines · On Wed, 24 Jan 2024 09:35:09 -0700 David Ahern wrote:
Quoted 5 lines
This is the latest run:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/435141/1-fcnal-test-sh/stdout
the nettest warning is indeed gone, but the failures are the same:
yep, I will send a formal patch. I see the timeout is high enough, so good there.
Well, kinda, to be honest I did bump the time to 4000s locally. The runtime of the entire net suite 1h 10min - that's pretty much the runtime of this one test :) The VMs run the tests without HW virtualization, so they are a bit slower, but it'd be nice if no local hacks were necessary.
I haven't sent a patch to bump it because it may make more sense to split the test into multiple. But as a stop gap we can as well bump the timeout.
Quoted 10 lines
Quoted 7 lines
$ grep FAIL stdout # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL] # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL]:(
known problems. I can disable the tests for now so we avoid regressions, and add to the TO-DO list for someone with time.
Sounds good!
Jakub Kicinski
maintainerQuoted 11 lines · On Wed, 24 Jan 2024 08:22:55 -0800 Jakub Kicinski wrote:
Quoted 3 lines
Going through the failing ksft-net series on
https://netdev.bots.linux.dev/status.html, all the tests I'm responsible seem to be passing.Here's a more handy link filtered down to failures (clicking on the test counts links here):
I have been attributing the udpg[rs]o and timestamp tests to you, but I haven't actually checked.. are they not yours? :)
Ah, BTW, a major source of failures seems to be that iptables is mapping to nftables on the executor. And either nftables doesn't support the functionality the tests expect or we're missing configs :( E.g. the TTL module.
Quoted 22 lines · Jakub Kicinski
Quoted 8 lines · On Wed, 24 Jan 2024 10:59:36 -0500 Willem de Bruijn wrote:
Quoted 7 lines · David Ahern
Quoted 5 lines · Jakub Kicinski
It groups all patches outstanding in patchwork (which build cleanly). I'm hoping we could also do HW testing using this setup, so batching is a must. Not 100% sure it's the right direction for SW testing but there's one way to find out :)
Quoted 6 lines
Quoted 1 line
Really cool. Thanks for spending time to make this happen.
Just to add to the choir: this is fantastic, thanks!
Hopefully it will encourage people to add kselftests, kunit tests or other kinds that now get continuous coverage.
Fingers crossed :)
Quoted 3 lines
Going through the failing ksft-net series on
https://netdev.bots.linux.dev/status.html, all the tests I'm responsible seem to be passing.
Quoted 7 lines
Here's a more handy link filtered down to failures (clicking on the test counts links here):
I have been attributing the udpg[rs]o and timestamp tests to you, but I haven't actually checked.. are they not yours? :)
I just looked at the result file and assumed 0 meant fine. Oops.
Technically udpgro is Paolo. But it is derived from udpgso.
But I'm on the hook for a few more. I added fq-band-pktlimit, our team added gso and txtimestamp I supposed is owned by the maintainers by now.
Jakub Kicinski
maintainerQuoted 9 lines · On Wed, 24 Jan 2024 12:42:46 -0500 Willem de Bruijn wrote:
Quoted 7 lines
Here's a more handy link filtered down to failures (clicking on the test counts links here):
I have been attributing the udpg[rs]o and timestamp tests to you, but I haven't actually checked.. are they not yours? :)
I just looked at the result file and assumed 0 meant fine. Oops.
Sorry about the confusion there, make run_tests apparently always returns 0 and the result file holds the exit code :( It could be improved by I figured, as long as the JSON output is correct, investing the time in the web UI is probably a better choice than massaging the output files.
Quoted 9 lines · Jakub Kicinski
Quoted 8 lines · On Wed, 24 Jan 2024 12:42:46 -0500 Willem de Bruijn wrote:
Quoted 7 lines
Here's a more handy link filtered down to failures (clicking on the test counts links here):
I have been attributing the udpg[rs]o and timestamp tests to you, but I haven't actually checked.. are they not yours? :)
Quoted 7 lines
Quoted 1 line
I just looked at the result file and assumed 0 meant fine. Oops.
Sorry about the confusion there, make
run_testsapparently always returns 0 and the result file holds the exit code :( It could be improved by I figured, as long as the JSON output is correct, investing the time in the web UI is probably a better choice than massaging the output files.
Absolutely. My bad for jumping to conclusions.
Paolo Abeni
maintainerQuoted 12 lines · Willem de Bruijn
Quoted 11 lines · Jakub Kicinski
Quoted 9 lines · On Wed, 24 Jan 2024 12:42:46 -0500 Willem de Bruijn wrote:
Quoted 7 lines
Here's a more handy link filtered down to failures (clicking on the test counts links here):
I have been attributing the udpg[rs]o and timestamp tests to you, but I haven't actually checked.. are they not yours? :)
I just looked at the result file and assumed 0 meant fine. Oops.
Quoted 7 lines
Quoted 5 lines
Sorry about the confusion there, make
run_testsapparently always returns 0 and the result file holds the exit code :( It could be improved by I figured, as long as the JSON output is correct, investing the time in the web UI is probably a better choice than massaging the output files.Absolutely. My bad for jumping to conclusions.
Some tests are failing because they need an xdp program build under the ebpf directory.
Since such program is used even by ebpf tests, I'll send a patch duplicating it under net, and using the existing rule for nat6to4 to build it.
Cheers,
Paolo
Hello,
24 Jan 2024 17:01:24 Jakub Kicinski <[email protected]>:
Quoted 10 lines · On Wed, 24 Jan 2024 08:22:55 -0800 Jakub Kicinski wrote:
Quoted 9 lines
Quoted 3 lines
Going through the failing ksft-net series on
https://netdev.bots.linux.dev/status.html, all the tests I'm responsible seem to be passing.Here's a more handy link filtered down to failures (clicking on the test counts links here):
Quoted 7 lines
Quoted 2 lines
I have been attributing the udpg[rs]o and timestamp tests to you, but I haven't actually checked.. are they not yours? :)
Ah, BTW, a major source of failures seems to be that iptables is mapping to nftables on the executor. And either nftables doesn't support the functionality the tests expect or we're missing configs :( E.g. the TTL module.
I don't know if it is the same issue, but for MPTCP, we use 'iptables-legacy' if available.
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=0c4cd3f86a400
Cheers,
Matt
Jakub Kicinski
maintainerQuoted 9 lines · On Wed, 24 Jan 2024 18:35:14 +0000 (GMT) Matthieu Baerts wrote:
Quoted 4 lines
Ah, BTW, a major source of failures seems to be that iptables is mapping to nftables on the executor. And either nftables doesn't support the functionality the tests expect or we're missing configs :( E.g. the TTL module.
I don't know if it is the same issue, but for MPTCP, we use 'iptables-legacy' if available.
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=0c4cd3f86a400
Great! Thanks for the pointer. I installed the packages now, so folks should be able to fix up their scripts.
Pablo Neira Ayuso
maintainerQuoted 10 lines · Jakub Kicinski
Quoted 9 lines · On Wed, 24 Jan 2024 08:22:55 -0800 Jakub Kicinski wrote:
Quoted 3 lines
Going through the failing ksft-net series on
https://netdev.bots.linux.dev/status.html, all the tests I'm responsible seem to be passing.Here's a more handy link filtered down to failures (clicking on the test counts links here):
Quoted 7 lines
Quoted 2 lines
I have been attributing the udpg[rs]o and timestamp tests to you, but I haven't actually checked.. are they not yours? :)
Ah, BTW, a major source of failures seems to be that iptables is mapping to nftables on the executor. And either nftables doesn't support the functionality the tests expect or we're missing configs :( E.g. the TTL module.
I could only find in the listing above this:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/435141/37-ip-defrag-sh/stdout
which shows:
ip6tables v1.8.8 (nf_tables): Couldn't load match `conntrack':No such file or directory
which seems like setup is broken, ie. it could not find libxt_conntrack.so
What is the issue?
Pablo Neira Ayuso
maintainerQuoted 22 lines · Matthieu Baerts
Hello,
24 Jan 2024 17:01:24 Jakub Kicinski <[email protected]>:
Quoted 10 lines · On Wed, 24 Jan 2024 08:22:55 -0800 Jakub Kicinski wrote:
Quoted 9 lines
Quoted 3 lines
Going through the failing ksft-net series on
https://netdev.bots.linux.dev/status.html, all the tests I'm responsible seem to be passing.Here's a more handy link filtered down to failures (clicking on the test counts links here):
Quoted 7 lines
Quoted 2 lines
I have been attributing the udpg[rs]o and timestamp tests to you, but I haven't actually checked.. are they not yours? :)
Ah, BTW, a major source of failures seems to be that iptables is mapping to nftables on the executor. And either nftables doesn't support the functionality the tests expect or we're missing configs :( E.g. the TTL module.
Quoted 4 lines
I don't know if it is the same issue, but for MPTCP, we use 'iptables-legacy' if available.
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=0c4cd3f86a400
I'd suggest you do the other way around, first check if iptables-nft is available, otherwise fall back to iptables-nft
commit refers to 5.15 already have iptables-nft support, it should work out of the box.
Jakub Kicinski
maintainerQuoted 6 lines · On Wed, 24 Jan 2024 20:16:39 +0100 Pablo Neira Ayuso wrote:
Quoted 4 lines
Ah, BTW, a major source of failures seems to be that iptables is mapping to nftables on the executor. And either nftables doesn't support the functionality the tests expect or we're missing configs :( E.g. the TTL module.
I could only find in the listing above this:
Thanks for taking a look!
Quoted 7 lines
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/435141/37-ip-defrag-sh/stdout
which shows:
ip6tables v1.8.8 (
nf_tables): Couldn't load match `conntrack':No such file or directorywhich seems like setup is broken, ie. it could not find libxt_conntrack.so
Hm, odd, it's there:
$ ls /lib64/xtables/libxt_conntrack.so /lib64/xtables/libxt_conntrack.so
but I set a custom LD_LIBRARY_PATH, let me make sure that /lib64 is in it (normal loaded always scans system paths)!
Quoted 1 line
What is the issue?
A lot of the tests print warning messages like the ones below. Some of them pass some of them fail. Tweaking the kernel config to make sure the right CONFIG_IP_NF_TARGET_* and CONFIG_IP_NF_MATCH_* are included seem to have made no difference, which I concluded was because iptables CLI uses nf_tables here by default..
[435321]$ grep -nrI "Warning: Extension" .
./6-fib-tests-sh/stdout:305:# Warning: Extension MARK revision 0 not supported, missing kernel module? ./6-fib-tests-sh/stdout:308:# Warning: Extension MARK revision 0 not supported, missing kernel module? ./6-fib-tests-sh/stdout:316:# Warning: Extension MARK revision 0 not supported, missing kernel module? ./6-fib-tests-sh/stdout:319:# Warning: Extension MARK revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:12:# No GRO Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:13:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:14:# Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:16:# GRO frag list Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:17:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:18:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:19:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:21:# Warning: Extension DNAT revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:23:# GRO fwd Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:24:# Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:38:# GRO frag list over UDP tunnel Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:39:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:40:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:41:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:43:# Warning: Extension DNAT revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:45:# GRO fwd over UDP tunnel Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:46:# Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:61:# No GRO Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:62:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:63:# Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:65:# GRO frag list Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:66:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:67:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:68:# Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:72:# GRO fwd Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:73:# Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:88:# GRO frag list over UDP tunnel Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:89:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:90:# Warning: Extension udp revision 0 not supported, missing kernel module? ./18-udpgro-fwd-sh/stdout:91:# Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:95:# GRO fwd over UDP tunnel Warning: Extension udp revision 0 not supported, missing kernel module?
./18-udpgro-fwd-sh/stdout:96:# Warning: Extension udp revision 0 not supported, missing kernel module? ./37-big-tcp-sh/stdout:17:# Warning: Extension length revision 0 not supported, missing kernel module? ./37-big-tcp-sh/stdout:19:# Warning: Extension length revision 0 not supported, missing kernel module? ./37-big-tcp-sh/stdout:22:# Warning: Extension length revision 0 not supported, missing kernel module? ./37-big-tcp-sh/stdout:24:# Warning: Extension length revision 0 not supported, missing kernel module? ./56-xfrm-policy-sh/stdout:11:# Warning: Extension policy revision 0 not supported, missing kernel module? ./56-xfrm-policy-sh/stdout:13:# Warning: Extension policy revision 0 not supported, missing kernel module? ./54-amt-sh/stdout:94:# Warning: Extension TTL revision 0 not supported, missing kernel module?
Pablo Neira Ayuso
maintainerQuoted 18 lines · Jakub Kicinski
Quoted 6 lines · On Wed, 24 Jan 2024 20:16:39 +0100 Pablo Neira Ayuso wrote:
Quoted 4 lines
Ah, BTW, a major source of failures seems to be that iptables is mapping to nftables on the executor. And either nftables doesn't support the functionality the tests expect or we're missing configs :( E.g. the TTL module.
I could only find in the listing above this:
Thanks for taking a look!
Quoted 7 lines
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/435141/37-ip-defrag-sh/stdout
which shows:
ip6tables v1.8.8 (
nf_tables): Couldn't load match `conntrack':No such file or directorywhich seems like setup is broken, ie. it could not find libxt_conntrack.so
Quoted 7 lines
Hm, odd, it's there:
$ ls /lib64/xtables/libxt_conntrack.so /lib64/xtables/libxt_conntrack.sobut I set a custom
LD_LIBRARY_PATH, let me make sure that /lib64 is in it (normal loaded always scans system paths)!
Could you also check your ./configure output for iptables? It shows the directory where the .so file are search and found:
...
Xtables extension directory: /usr/lib/xtables
Quoted 7 lines
Quoted 1 line
What is the issue?
A lot of the tests print warning messages like the ones below. Some of them pass some of them fail. Tweaking the kernel config to make sure the right CONFIG_IP_NF_TARGET_* and CONFIG_IP_NF_MATCH_* are included seem to have made no difference, which I concluded was because iptables CLI uses
nf_tableshere by default..
Please, check if the symlink refers to -legacy or -nft via:
$ ls -la /usr/sbin/iptables
Quoted 2 lines
[435321]$ grep -nrI "Warning: Extension" .
./6-fib-tests-sh/stdout:305:# Warning: Extension MARK revision 0 not supported, missing kernel module?
This could come from either legacy or nftables:
libxtables/xtables.c: "Warning: Extension %s revision 0 not supported, missing kernel module?\n", iptables/nft.c: "Warning: Extension %s revision 0 not supported, missing kernel module?\n",
both have the same error.
if that is the nftables backend, it might be also that .config is missing CONFIG_NF_TABLES and CONFIG_NFT_COMPAT there, among other options.
Jakub Kicinski
maintainerQuoted 6 lines · On Wed, 24 Jan 2024 21:02:16 +0100 Pablo Neira Ayuso wrote:
Quoted 5 lines · Jakub Kicinski
Hm, odd, it's there:
$ ls /lib64/xtables/libxt_conntrack.so /lib64/xtables/libxt_conntrack.so
Quoted 8 lines
Quoted 2 lines
but I set a custom
LD_LIBRARY_PATH, let me make sure that /lib64 is in it (normal loaded always scans system paths)!Could you also check your ./configure output for iptables? It shows the directory where the .so file are search and found:
...
Xtables extension directory:/usr/lib/xtables
I'm using the OS iptables, I was hoping that for net/ tests OS iptables should be good enough :S Is there a way to get the info you're after?
$ iptables -V iptables v1.8.8 (nf_tables)
Quoted 11 lines
Quoted 7 lines
Quoted 1 line
What is the issue?
A lot of the tests print warning messages like the ones below. Some of them pass some of them fail. Tweaking the kernel config to make sure the right CONFIG_IP_NF_TARGET_* and CONFIG_IP_NF_MATCH_* are included seem to have made no difference, which I concluded was because iptables CLI uses
nf_tableshere by default..Please, check if the symlink refers to -legacy or -nft via:
$ ls -la /usr/sbin/iptables
Ah, neat:
$ ls -la /etc/alternatives/iptables lrwxrwxrwx. 1 root root 22 Jan 7 22:10 /etc/alternatives/iptables -> /usr/sbin/iptables-nft $ ls -la /usr/sbin/iptables lrwxrwxrwx. 1 root root 26 Jan 7 22:10 /usr/sbin/iptables -> /etc/alternatives/iptables
Quoted 5 lines
Quoted 2 lines
[435321]$ grep -nrI "Warning: Extension" .
./6-fib-tests-sh/stdout:305:# Warning: Extension MARK revision 0 not supported, missing kernel module?This could come from either legacy or nftables:
Quoted 8 lines
libxtables/xtables.c: "Warning: Extension %s revision 0 not supported, missing kernel module?\n", iptables/nft.c: "Warning: Extension %s revision 0 not supported, missing kernel module?\n",both have the same error.
if that is the nftables backend, it might be also that .config is missing
CONFIG_NF_TABLESandCONFIG_NFT_COMPATthere, among other options.
FWIW full config:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/435321/config
CONFIG_NFT_COMPAT was indeed missing! Let's see how it fares with it enabled.
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines · On Wed, 24 Jan 2024 09:35:09 -0700 David Ahern wrote:
Quoted 4 lines
This is the latest run:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/435141/1-fcnal-test-sh/stdout
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
the nettest warning is indeed gone, but the failures are the same:
yep, I will send a formal patch. I see the timeout is high enough, so good there.
Well, kinda, to be honest I did bump the time to 4000s locally. The runtime of the entire net suite 1h 10min - that's pretty much the runtime of this one test :) The VMs run the tests without
one *script* = 900+ tests. :-)
Quoted 6 lines
HW virtualization, so they are a bit slower, but it'd be nice if no local hacks were necessary.
I haven't sent a patch to bump it because it may make more sense to split the test into multiple. But as a stop gap we can as well bump the timeout.
The script has the tests in groups and each group can be run in parallel (with Hangbin's recent namespace changes). I can look at splitting that script into many (or write wrappers to run specific groups), but even then most of those test groups need more than 45 seconds. There are lot of permutations covered (with and without vrf, different address types, different uapis, ...).
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines
$ grep FAIL stdout # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL] # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL]:(
known problems. I can disable the tests for now so we avoid regressions, and add to the TO-DO list for someone with time.
Sent a PR to fix a few things. I did not have to disable any tests - everything passes cleanly with the changes. If tests fail after those are applied, let's compare OS environments - maybe some sysctl is enabled or disabled (or a CONFIG) in your environment.
Jakub Kicinski
maintainerQuoted 9 lines · On Wed, 24 Jan 2024 12:13:43 -0800 Jakub Kicinski wrote:
Quoted 3 lines
if that is the nftables backend, it might be also that .config is missing
CONFIG_NF_TABLESandCONFIG_NFT_COMPATthere, among other options.FWIW full config:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/435321/config
CONFIG_NFT_COMPATwas indeed missing! Let's see how it fares with it enabled.
NFT_COMPAT fixed a lot! One remaining warning comes from using -m length. Which NFT config do we need for that one?
Florian Westphal
maintainerQuoted 5 lines · Jakub Kicinski
Quoted 4 lines · On Wed, 24 Jan 2024 12:13:43 -0800 Jakub Kicinski wrote:
Quoted 3 lines
if that is the nftables backend, it might be also that .config is missing
CONFIG_NF_TABLESandCONFIG_NFT_COMPATthere, among other options.
Quoted 8 lines
Quoted 5 lines
FWIW full config:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/435321/config
CONFIG_NFT_COMPATwas indeed missing! Let's see how it fares with it enabled.
NFT_COMPATfixed a lot! One remaining warning comes from using -m length. Which NFT config do we need for that one?
CONFIG_NETFILTER_XT_MATCH_LENGTH=m|y
Quoted 15 lines · Jakub Kicinski
Quoted 5 lines · On Tue, 23 Jan 2024 18:04:19 +0100 Petr Machata wrote:
Quoted 4 lines
Unless I'm doing it wrong and the sub-directories are supposed to inherit the parent directory's config? So
net/forwarding/should be built with net/'s config? I could not find the info in docs, does anyone know?Quoted 8 lines
I don't think they are, net/config defines
CONFIG_VXLAN, but then the vxlan tests still complain about unknown device type. Though maybe there's another device type that it's missing...What do I do to feed the config file to some build script to get a kernel image to test? I can of course just do something like cat config | xargs -n1
scripts/config-m, but I expect there's some automation for it and I just can't find it.
Quoted 5 lines
The CI script is based on virtme-ng. So it does this:
# $target is net or net/forwarding or
drivers/net/bondingetc. make mrproper
vng -v -b -ftools/testing/selftests/$target
Actually:
# vng -v -b -f tools/testing/selftests/${target}/configDidn't know about vng, it's way cool!
(Despite having used virtme for a long time.)
Quoted 7 lines
# build the scripts make headers make -C tools/testing/selftests/$targetvng -v -r
arch/x86/boot/bzImage--user root# inside the VM make -C tools/testing/selftests TARGETS=$target run_tests
I'm working my way through the selftests.
Quoted 9 lines
https://github.com/kuba-moo/nipa/blob/master/contest/remote/vmksft.py#L138
You're right, it definitely does not "inherit" net's config when running forwarding/net. I can easily make it do so, but I'm not clear
what the expectation from the kselftest subsystem is. Because if other testers (people testing stable, KernelCI etc.
et.c) don't "inherit" we better fill in the config completely so that the tests pass for everyone.
Oh, gotcha, the question was not whether it does, but whether it's supposed to. OK.
IMHO not necessarily. net/config is for net/*.sh, net/forwarding/config for net/forwarding/*.sh. It's not a given that whatever is needed for net/ is needed for net/forwarding/ as well.
It will also lead to simpler patches, where enabling config options in X/ doesn't imply checking for newly useless duplicities in X's child directories.
Pablo Neira Ayuso
maintainerQuoted 5 lines · Jakub Kicinski
Quoted 4 lines · On Wed, 24 Jan 2024 12:13:43 -0800 Jakub Kicinski wrote:
Quoted 3 lines
if that is the nftables backend, it might be also that .config is missing
CONFIG_NF_TABLESandCONFIG_NFT_COMPATthere, among other options.
Quoted 8 lines
Quoted 5 lines
FWIW full config:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/435321/config
CONFIG_NFT_COMPATwas indeed missing! Let's see how it fares with it enabled.
NFT_COMPATfixed a lot! One remaining warning comes from using -m length. Which NFT config do we need for that one?
May I have a look at the logs? How does the error look like?
Jakub Kicinski
maintainerQuoted 4 lines · On Thu, 25 Jan 2024 09:52:11 +0100 Florian Westphal wrote:
Quoted 2 lines
NFT_COMPATfixed a lot! One remaining warning comes from using -m length. Which NFT config do we need for that one?
CONFIG_NETFILTER_XT_MATCH_LENGTH=m|y
Thank you, that seems to solve the remaining problem!
Jakub Kicinski
maintainerQuoted 4 lines · On Thu, 25 Jan 2024 10:29:41 +0100 Pablo Neira Ayuso wrote:
Quoted 2 lines
NFT_COMPATfixed a lot! One remaining warning comes from using -m length. Which NFT config do we need for that one?May I have a look at the logs? How does the error look like?
With the config pointed out by Florian in addition to NFT_COMPAT all the iptables errors in the logs are gone, without switching to legacy. Thank you for the help!
Also LMK if you guys want us to try running netfilter tests. I'm guessing we're unlikely to regress anything in net-next, and you have running netfilter tests covered - but if it's not too hard we can try to hook them up to net-next, too! (Or you can run them on our branch and report back :))
Quoted 8 lines · David Ahern
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
$ grep FAIL stdout # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL] # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL]:(
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/438381/1-fcnal-test-sh/stdout
still shows those 4 tests failing. since they pass on default Ubuntu 23.10, I need some information about the setup. What is the OS image in use and any known changes to the sysctl settings?
Can I get sysctl net > /tmp/sysctl.net ? I will compare to Ubuntu and see if I can figure out the difference and get those added to the script.
Jakub Kicinski
maintainerQuoted 9 lines · On Fri, 26 Jan 2024 17:56:26 -0700 David Ahern wrote:
On 1/24/24 2:48 PM, David Ahern wrote:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/438381/1-fcnal-test-sh/stdoutstill shows those 4 tests failing. since they pass on default Ubuntu 23.10, I need some information about the setup. What is the OS image in use and any known changes to the sysctl settings?
Can I get
sysctl net > /tmp/sysctl.net? I will compare to Ubuntu and see if I can figure out the difference and get those added to the script.
Here's a boot and run of the command (not sure how to export the file form the VM so I captured all of stdout):
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/sysctl-for-david
The OS is Amazon Linux, annoyingly.
Quoted 11 lines · Jakub Kicinski
Quoted 9 lines · On Fri, 26 Jan 2024 17:56:26 -0700 David Ahern wrote:
On 1/24/24 2:48 PM, David Ahern wrote:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/438381/1-fcnal-test-sh/stdoutstill shows those 4 tests failing. since they pass on default Ubuntu 23.10, I need some information about the setup. What is the OS image in use and any known changes to the sysctl settings?
Can I get
sysctl net > /tmp/sysctl.net? I will compare to Ubuntu and see if I can figure out the difference and get those added to the script.
Quoted 6 lines
Here's a boot and run of the command (not sure how to export the file form the VM so I captured all of stdout):
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/sysctl-for-david
The OS is Amazon Linux, annoyingly.
It's a bug in that version of iputils ping. It sets the BINDTODEVICE and then resets it because the source address is not set on the command line (it should not be required).
There are a couple of workarounds - one which might not age well (ie., amazon linux moving forward to newer packages -I <addr> -I <vrf>) and one that bypasses the purpose of the test (ip vrf exec)).
Paolo Abeni
maintainerQuoted 19 lines · David Ahern
Quoted 11 lines · Jakub Kicinski
Quoted 9 lines · On Fri, 26 Jan 2024 17:56:26 -0700 David Ahern wrote:
On 1/24/24 2:48 PM, David Ahern wrote:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/438381/1-fcnal-test-sh/stdoutstill shows those 4 tests failing. since they pass on default Ubuntu 23.10, I need some information about the setup. What is the OS image in use and any known changes to the sysctl settings?
Can I get
sysctl net > /tmp/sysctl.net? I will compare to Ubuntu and see if I can figure out the difference and get those added to the script.Quoted 6 lines
Here's a boot and run of the command (not sure how to export the file form the VM so I captured all of stdout):
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/sysctl-for-david
The OS is Amazon Linux, annoyingly.
Quoted 7 lines
It's a bug in that version of iputils ping. It sets the BINDTODEVICE and then resets it because the source address is not set on the command line (it should not be required).
There are a couple of workarounds - one which might not age well (ie., amazon linux moving forward to newer packages -I <addr> -I <vrf>) and one that bypasses the purpose of the test (ip vrf exec)).
Could the script validate the 'ping' command WRT the bad behavior/bug and eventually skip the related tests?
Cheers,
Paolo
Quoted 4 lines · Jakub Kicinski
If you authored any net or
drivers/netselftests, please look around and see if they are passing. If not - send patches or LMK what I need to do to make them pass on the runner.. Make sure to scroll down to the "Not reporting to patchwork" section.
selftests-net/test-bridge-neigh-suppress-sh should be fixed by:
dnf install ndisc6
selftests-net/test-bridge-backup-port-sh should be fixed by:
https://lore.kernel.org/netdev/[email protected]/
selftests-net/drop-monitor-tests-sh should be fixed by:
dnf install dropwatch
Quoted 5 lines · Ido Schimmel
Quoted 4 lines · Jakub Kicinski
If you authored any net or
drivers/netselftests, please look around and see if they are passing. If not - send patches or LMK what I need to do to make them pass on the runner.. Make sure to scroll down to the "Not reporting to patchwork" section.
Forgot to mention: Thanks a lot for setting this up!
Quoted 12 lines
selftests-net/test-bridge-neigh-suppress-sh should be fixed by:
dnf install ndisc6
selftests-net/test-bridge-backup-port-sh should be fixed by:
https://lore.kernel.org/netdev/[email protected]/
selftests-net/drop-monitor-tests-sh should be fixed by:
dnf install dropwatch
Jakub Kicinski
maintainerQuoted 8 lines · On Mon, 29 Jan 2024 14:46:32 +0200 Ido Schimmel wrote:
Quoted 5 lines · Ido Schimmel
Quoted 4 lines · Jakub Kicinski
If you authored any net or
drivers/netselftests, please look around and see if they are passing. If not - send patches or LMK what I need to do to make them pass on the runner.. Make sure to scroll down to the "Not reporting to patchwork" section.Forgot to mention: Thanks a lot for setting this up!
Finger crossed that it ends up being useful :)
Quoted 11 lines
Quoted 11 lines
selftests-net/test-bridge-neigh-suppress-sh should be fixed by:
dnf install ndisc6
selftests-net/test-bridge-backup-port-sh should be fixed by:
https://lore.kernel.org/netdev/[email protected]/
selftests-net/drop-monitor-tests-sh should be fixed by:
dnf install dropwatch
Installed both (from source) just in time for the
net-next-2024-01-29--15-00 run.. let's see.
Jakub Kicinski
maintainerQuoted 10 lines · On Mon, 29 Jan 2024 10:23:15 +0100 Paolo Abeni wrote:
Quoted 7 lines
It's a bug in that version of iputils ping. It sets the BINDTODEVICE and then resets it because the source address is not set on the command line (it should not be required).
There are a couple of workarounds - one which might not age well (ie., amazon linux moving forward to newer packages -I <addr> -I <vrf>) and one that bypasses the purpose of the test (ip vrf exec)).
Could the script validate the 'ping' command WRT the bad behavior/bug and eventually skip the related tests?
Skipping if the package has a bug would be best, if we can figure that out. The latest version is fixed, right? I can build it locally, just like pretty much everything else..
Quoted 9 lines · Jakub Kicinski
Quoted 8 lines · On Mon, 29 Jan 2024 10:23:15 +0100 Paolo Abeni wrote:
Quoted 7 lines
It's a bug in that version of iputils ping. It sets the BINDTODEVICE and then resets it because the source address is not set on the command line (it should not be required).
There are a couple of workarounds - one which might not age well (ie., amazon linux moving forward to newer packages -I <addr> -I <vrf>) and one that bypasses the purpose of the test (ip vrf exec)).
Quoted 6 lines
Quoted 2 lines
Could the script validate the 'ping' command WRT the bad behavior/bug and eventually skip the related tests?
Skipping if the package has a bug would be best, if we can figure that out. The latest version is fixed, right? I can build it locally, just like pretty much everything else..
yes, that is why Ubuntu 23.10 passes. I downloaded iputils, built ping locally at the Amazon release version and did a side by side comparison of behavior to verify it is the ping command.
Jakub Kicinski
maintainerQuoted 10 lines · On Mon, 29 Jan 2024 08:32:33 -0700 David Ahern wrote:
Quoted 6 lines
Quoted 2 lines
Could the script validate the 'ping' command WRT the bad behavior/bug and eventually skip the related tests?
Skipping if the package has a bug would be best, if we can figure that out. The latest version is fixed, right? I can build it locally, just like pretty much everything else..
yes, that is why Ubuntu 23.10 passes. I downloaded iputils, built ping locally at the Amazon release version and did a side by side comparison of behavior to verify it is the ping command.
Built iputils from git manually.
Could not figure out how to convince mason not to insert /local/ into the install dir path so I copied the binaries manually :/ Seems to work, tho.
Quoted 18 lines · Jakub Kicinski
Quoted 8 lines · On Mon, 29 Jan 2024 14:46:32 +0200 Ido Schimmel wrote:
Quoted 5 lines · Ido Schimmel
Quoted 4 lines · Jakub Kicinski
If you authored any net or
drivers/netselftests, please look around and see if they are passing. If not - send patches or LMK what I need to do to make them pass on the runner.. Make sure to scroll down to the "Not reporting to patchwork" section.Forgot to mention: Thanks a lot for setting this up!
Finger crossed that it ends up being useful :)
Quoted 6 lines
Quoted 6 lines
selftests-net/test-bridge-neigh-suppress-sh should be fixed by:
dnf install ndisc6
selftests-net/test-bridge-backup-port-sh should be fixed by:
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
https://lore.kernel.org/netdev/[email protected]/
selftests-net/drop-monitor-tests-sh should be fixed by:
dnf install dropwatch
Installed both (from source) just in time for the
net-next-2024-01-29--15-00 run.. let's see.
Thanks!
The last two tests look good now, but the first still fails. Can you share the ndisc6 version information? I tested with [1] from [2].
If your copy of ndisc6 indeed works, then I might be missing some sysctl. I will be AFK tomorrow so I will look into it later this week.
[1]
$ ./ndisc6 --version ndisc6: IPv6 Neighbor/Router Discovery userland tool 1.0.8 ($Rev$) built Jan 29 2024 on <redacted> Configured with: ./configure Written by Remi Denis-Courmont
Copyright (C) 2004-2007 Remi Denis-Courmont
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Jakub Kicinski
maintainerQuoted 10 lines · On Mon, 29 Jan 2024 19:00:49 +0200 Ido Schimmel wrote:
Quoted 2 lines
Installed both (from source) just in time for the
net-next-2024-01-29--15-00 run.. let's see.Thanks!
The last two tests look good now, but the first still fails. Can you share the ndisc6 version information? I tested with [1] from [2].
If your copy of ndisc6 indeed works, then I might be missing some sysctl. I will be AFK tomorrow so I will look into it later this week.
Hm. Looks like our versions match. I put the entire tools root dir up on HTTP now: https://netdev-2.bots.linux.dev/tools/fs/ in case you wanna fetch the exact binary, it only links with libc, it seems.
Quoted 9 lines · Jakub Kicinski
Quoted 8 lines · On Mon, 29 Jan 2024 19:00:49 +0200 Ido Schimmel wrote:
Quoted 2 lines
Installed both (from source) just in time for the
net-next-2024-01-29--15-00 run.. let's see.Thanks!
The last two tests look good now, but the first still fails. Can you share the ndisc6 version information? I tested with [1] from [2].
Quoted 6 lines
Quoted 2 lines
If your copy of ndisc6 indeed works, then I might be missing some sysctl. I will be AFK tomorrow so I will look into it later this week.
Hm. Looks like our versions match. I put the entire tools root dir up on HTTP now: https://netdev-2.bots.linux.dev/tools/fs/ in case you wanna fetch the exact binary, it only links with libc, it seems.
I tried with your binary and on other setups and I'm unable to reproduce the failure. From the test output it seems the NS is never sent. If you can, attaching the verbose test output might help:
./test_bridge_neigh_suppress.sh -t neigh_suppress_ns -v
Thanks
Quoted 17 lines · Ido Schimmel
Quoted 9 lines · Jakub Kicinski
Quoted 8 lines · On Mon, 29 Jan 2024 19:00:49 +0200 Ido Schimmel wrote:
Quoted 2 lines
Installed both (from source) just in time for the
net-next-2024-01-29--15-00 run.. let's see.Thanks!
The last two tests look good now, but the first still fails. Can you share the ndisc6 version information? I tested with [1] from [2].
Quoted 6 lines
Quoted 2 lines
If your copy of ndisc6 indeed works, then I might be missing some sysctl. I will be AFK tomorrow so I will look into it later this week.
Hm. Looks like our versions match. I put the entire tools root dir up on HTTP now: https://netdev-2.bots.linux.dev/tools/fs/ in case you wanna fetch the exact binary, it only links with libc, it seems.
Quoted 8 lines
I tried with your binary and on other setups and I'm unable to reproduce the failure. From the test output it seems the NS is never sent. If you can, attaching the verbose test output might help:
./test_bridge_neigh_suppress.sh -t
neigh_suppress_ns-vThanks
Maybe change the subject of the mail thread? At least I'm confused if I read several times a day "net-next is OPEN".
Jakub Kicinski
maintainerRe: [TEST] bridge tests (was: net-next is OPEN)
In reply to Ido SchimmelQuoted 9 lines · On Wed, 31 Jan 2024 15:23:50 +0200 Ido Schimmel wrote:
Quoted 3 lines
Hm. Looks like our versions match. I put the entire tools root dir up on HTTP now: https://netdev-2.bots.linux.dev/tools/fs/ in case you wanna fetch the exact binary, it only links with libc, it seems.
I tried with your binary and on other setups and I'm unable to reproduce the failure. From the test output it seems the NS is never sent. If you can, attaching the verbose test output might help:
./test_bridge_neigh_suppress.sh -t
neigh_suppress_ns-v
FWIW I started two new instances on bare metal one with the same kernel as the nested VM and one with debug options enabled.
selftests-net/test-bridge-neigh-suppress-sh
- fails across all, so must be the OS rather than the "speed"
selftests-net/test-bridge-backup-port-sh
- passes on VM, metal-dbg
- fails on metal :S very reliably / every time:
https://netdev.bots.linux.dev/contest.html?test=test-bridge-backup-port-sh
# TEST: No forwarding out of swp1 [FAIL]
selftests-net/drop-monitor-tests-sh
- passes everywhere now
The info you asked for:
# ./test_bridge_neigh_suppress.sh -t neigh_suppress_ns -v [ 49.621534] eth0: renamed from veth0 [ 49.654673] swp1: renamed from veth1 [ 49.676235] ip (235) used greatest stack depth: 12032 bytes left [ 49.721434] veth0: renamed from veth1 [ 49.739521] ip (241) used greatest stack depth: 11880 bytes left [ 49.760463] eth0: renamed from veth0 [ 49.787523] swp1: renamed from veth1 [ 50.019197] br0: port 1(swp1) entered blocking state [ 50.019610] br0: port 1(swp1) entered disabled state [ 50.019949] swp1: entered allmulticast mode [ 50.020307] swp1: entered promiscuous mode [ 50.021035] br0: port 1(swp1) entered blocking state [ 50.021415] br0: port 1(swp1) entered forwarding state [ 50.076445] br0: port 2(vx0) entered blocking state [ 50.076805] br0: port 2(vx0) entered disabled state [ 50.077139] vx0: entered allmulticast mode [ 50.077492] vx0: entered promiscuous mode [ 50.077812] br0: port 2(vx0) entered blocking state [ 50.078172] br0: port 2(vx0) entered forwarding state [ 50.238364] br0: port 1(swp1) entered blocking state [ 50.238735] br0: port 1(swp1) entered disabled state [ 50.239077] swp1: entered allmulticast mode [ 50.239434] swp1: entered promiscuous mode [ 50.240046] br0: port 1(swp1) entered blocking state [ 50.240395] br0: port 1(swp1) entered forwarding state [ 50.269453] br0: port 2(vx0) entered blocking state [ 50.269812] br0: port 2(vx0) entered disabled state [ 50.270142] vx0: entered allmulticast mode [ 50.270597] vx0: entered promiscuous mode [ 50.270893] br0: port 2(vx0) entered blocking state [ 50.271222] br0: port 2(vx0) entered forwarding state
Per-port NS suppression - VLAN 10
---------------------------------
COMMAND: tc -n sw1-hF6GFk qdisc replace dev vx0 clsact
COMMAND: tc -n sw1-hF6GFk filter replace dev vx0 egress pref 1 handle 101 proto ipv6 flower indev swp1 ip_proto icmpv6 dst_ip ff02::1:ff00:2 src_ip 2001:db8:1::1 type 135 code 0 action pass
[ 55.411592] GACT probability NOT on
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:1::1 -w 5000 2001:db8:1::2 eth0.10 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: bridge -n sw1-hF6GFk link set dev vx0 neigh_suppress on
COMMAND: bridge -n sw1-hF6GFk -d link show dev vx0 | grep "neigh_suppress on"
hairpin off guard off root_block off fastleave off learning off flood on mcast_flood on bcast_flood on mcast_router 1 mcast_to_unicast off neigh_suppress on neigh_vlan_suppress off vlan_tunnel on isolated off locked off mab off mcast_n_groups 0 mcast_max_groups 0
TEST: "neigh_suppress" is on [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:1::1 -w 5000 2001:db8:1::2 eth0.10 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: bridge -n sw1-hF6GFk fdb replace 5a:1d:b4:4b:25:16 dev vx0 master static vlan 10
TEST: FDB entry installation [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:1::1 -w 5000 2001:db8:1::2 eth0.10 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: ip -n sw1-hF6GFk neigh replace 2001:db8:1::2 lladdr 5a:1d:b4:4b:25:16 nud permanent dev br0.10
TEST: Neighbor entry installation [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:1::1 -w 5000 2001:db8:1::2 eth0.10 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: ip -n h2-g0sh0Q link set dev eth0.10 down
TEST: H2 down [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:1::1 -w 5000 2001:db8:1::2 eth0.10 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: ip -n h2-g0sh0Q link set dev eth0.10 up
TEST: H2 up [ OK ]
COMMAND: bridge -n sw1-hF6GFk link set dev vx0 neigh_suppress off
COMMAND: bridge -n sw1-hF6GFk -d link show dev vx0 | grep "neigh_suppress off"
hairpin off guard off root_block off fastleave off learning off flood on mcast_flood on bcast_flood on mcast_router 1 mcast_to_unicast off neigh_suppress off neigh_vlan_suppress off vlan_tunnel on isolated off locked off mab off mcast_n_groups 0 mcast_max_groups 0
TEST: "neigh_suppress" is off [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:1::1 -w 5000 2001:db8:1::2 eth0.10 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: ip -n h2-g0sh0Q link set dev eth0.10 down
TEST: H2 down [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:1::1 -w 5000 2001:db8:1::2 eth0.10 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 2
TEST: NS suppression [FAIL]
rc=1, expected 0
Per-port NS suppression - VLAN 20
---------------------------------
COMMAND: tc -n sw1-hF6GFk qdisc replace dev vx0 clsact
COMMAND: tc -n sw1-hF6GFk filter replace dev vx0 egress pref 1 handle 101 proto ipv6 flower indev swp1 ip_proto icmpv6 dst_ip ff02::1:ff00:2 src_ip 2001:db8:2::1 type 135 code 0 action pass
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:2::1 -w 5000 2001:db8:2::2 eth0.20
Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: bridge -n sw1-hF6GFk link set dev vx0 neigh_suppress on
COMMAND: bridge -n sw1-hF6GFk -d link show dev vx0 | grep "neigh_suppress on"
hairpin off guard off root_block off fastleave off learning off flood on mcast_flood on bcast_flood on mcast_router 1 mcast_to_unicast off neigh_suppress on neigh_vlan_suppress off vlan_tunnel on isolated off locked off mab off mcast_n_groups 0 mcast_max_groups 0
TEST: "neigh_suppress" is on [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:2::1 -w 5000 2001:db8:2::2 eth0.20 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: bridge -n sw1-hF6GFk fdb replace 5a:1d:b4:4b:25:16 dev vx0 master static vlan 20
TEST: FDB entry installation [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:2::1 -w 5000 2001:db8:2::2 eth0.20 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: ip -n sw1-hF6GFk neigh replace 2001:db8:2::2 lladdr 5a:1d:b4:4b:25:16 nud permanent dev br0.20
TEST: Neighbor entry installation [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:2::1 -w 5000 2001:db8:2::2 eth0.20 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: ip -n h2-g0sh0Q link set dev eth0.20 down
TEST: H2 down [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:2::1 -w 5000 2001:db8:2::2 eth0.20 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: ip -n h2-g0sh0Q link set dev eth0.20 up
TEST: H2 up [ OK ]
COMMAND: bridge -n sw1-hF6GFk link set dev vx0 neigh_suppress off
COMMAND: bridge -n sw1-hF6GFk -d link show dev vx0 | grep "neigh_suppress off"
hairpin off guard off root_block off fastleave off learning off flood on mcast_flood on bcast_flood on mcast_router 1 mcast_to_unicast off neigh_suppress off neigh_vlan_suppress off vlan_tunnel on isolated off locked off mab off mcast_n_groups 0 mcast_max_groups 0
TEST: "neigh_suppress" is off [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:2::1 -w 5000 2001:db8:2::2 eth0.20 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
TEST: NS suppression [FAIL]
rc=1, expected 0
COMMAND: ip -n h2-g0sh0Q link set dev eth0.20 down
TEST: H2 down [ OK ]
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:2::1 -w 5000 2001:db8:2::2 eth0.20 Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 2
TEST: NS suppression [FAIL]
rc=1, expected 0
[ 57.883133] br0: port 1(swp1) entered disabled state [ 57.884754] swp1 (unregistering): left allmulticast mode [ 57.885191] swp1 (unregistering): left promiscuous mode [ 57.885580] br0: port 1(swp1) entered disabled state
Tests passed: 14 Tests failed: 28 [ 57.966498] vx0: left allmulticast mode [ 57.966802] vx0: left promiscuous mode [ 57.967091] br0: port 2(vx0) entered disabled state [ 57.992675] br0: port 1(swp1) entered disabled state [ 58.017115] swp1 (unregistering): left allmulticast mode [ 58.017520] swp1 (unregistering): left promiscuous mode [ 58.017898] br0: port 1(swp1) entered disabled state bash-5.2# [ 58.092405] vx0: left allmulticast mode [ 58.092704] vx0: left promiscuous mode [ 58.092972] br0: port 2(vx0) entered disabled state
Re: [TEST] bridge tests (was: net-next is OPEN)
Quoted 5 lines · Jakub Kicinski
FWIW I started two new instances on bare metal one with the same kernel as the nested VM and one with debug options enabled.
selftests-net/test-bridge-neigh-suppress-sh
- fails across all, so must be the OS rather than the "speed"
Yes, it's something related to the OS. From the log below:
```
COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:1::1 -w 5000 2001:db8:1::2 eth0.10
Raw IPv6 socket: Operation not permitted
TEST: ndisc6 [FAIL]
rc=1, expected 0
```The test is supposed to be run as root so I'm not sure what this error is about. Do you have something like AppArmor or SELinux running? The program creates an IPv6 raw socket and requires CAP_NET_RAW.
Quoted 6 lines
selftests-net/test-bridge-backup-port-sh
- passes on VM, metal-dbg
- fails on metal :S very reliably / every time:
https://netdev.bots.linux.dev/contest.html?test=test-bridge-backup-port-sh# TEST: No forwarding out of swp1 [FAIL]
Passes on all setups:
https://netdev-2.bots.linux.dev/vmksft-net-mp/results/446482/12-test-bridge-backup-port-sh/stdout
https://netdev-3.bots.linux.dev/vmksft-net/results/446481/7-test-bridge-backup-port-sh/stdout
https://netdev-3.bots.linux.dev/vmksft-net-dbg/results/446481/12-test-bridge-backup-port-sh/stdout
With this patch:
https://lore.kernel.org/netdev/[email protected]/
Will submit v2 next week.
Quoted 2 lines
selftests-net/drop-monitor-tests-sh
- passes everywhere now
Nice, thanks
Jakub Kicinski
maintainerRe: [TEST] bridge tests (was: net-next is OPEN)
Quoted 12 lines · On Thu, 1 Feb 2024 15:46:23 +0200 Ido Schimmel wrote:
Quoted 2 lines
selftests-net/test-bridge-neigh-suppress-sh
- fails across all, so must be the OS rather than the "speed"Yes, it's something related to the OS. From the log below:
``` COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:1::1 -w 5000 2001:db8:1::2 eth0.10 Raw IPv6 socket: Operation not permitted TEST: ndisc6 [FAIL] rc=1, expected 0 ```
Quoted 3 lines
The test is supposed to be run as root so I'm not sure what this error is about. Do you have something like AppArmor or SELinux running? The program creates an IPv6 raw socket and requires
CAP_NET_RAW.
Ah, ugh, sorry for the misdirection, you're right.
Looks like the binaries have SUID set:
# find tools/fs/ -perm -4000 tools/fs/usr/bin/ndisc6 tools/fs/usr/bin/rdisc6 tools/fs/usr/bin/rltraceroute6
But I install them as a normal user:
# ll tools/fs/usr/bin/ndisc6 -rwsr-xr-x. 1 virtme virtme 53840 Jan 29 14:36 tools/fs/usr/bin/ndisc6
so I guess they intend to SUID themselves into privileges but end up SUIDing out to a lowly user :(
I cleared the SUID bits out, let's see the next run.
Jakub Kicinski
maintainerRe: [TEST] bridge tests (was: net-next is OPEN)
Quoted 13 lines · On Thu, 1 Feb 2024 07:30:25 -0800 Jakub Kicinski wrote:
Quoted 12 lines · On Thu, 1 Feb 2024 15:46:23 +0200 Ido Schimmel wrote:
Quoted 2 lines
selftests-net/test-bridge-neigh-suppress-sh
- fails across all, so must be the OS rather than the "speed"Yes, it's something related to the OS. From the log below:
``` COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:1::1 -w 5000 2001:db8:1::2 eth0.10 Raw IPv6 socket: Operation not permitted TEST: ndisc6 [FAIL] rc=1, expected 0 ```
Quoted 5 lines
Quoted 3 lines
The test is supposed to be run as root so I'm not sure what this error is about. Do you have something like AppArmor or SELinux running? The program creates an IPv6 raw socket and requires
CAP_NET_RAW.Ah, ugh, sorry for the misdirection, you're right.
Confirmed, with the SUID cleared test-bridge-neigh-suppress-sh now passes on everything with the exception of metal+debug kernel.
Hi Pablo,
Thank you for your reply!
Quoted 26 lines · Pablo Neira Ayuso
Quoted 25 lines · Matthieu Baerts
Hello,
24 Jan 2024 17:01:24 Jakub Kicinski <[email protected]>:
Quoted 10 lines · On Wed, 24 Jan 2024 08:22:55 -0800 Jakub Kicinski wrote:
Quoted 9 lines
Quoted 3 lines
Going through the failing ksft-net series on
https://netdev.bots.linux.dev/status.html, all the tests I'm responsible seem to be passing.Here's a more handy link filtered down to failures (clicking on the test counts links here):
Quoted 7 lines
Quoted 2 lines
I have been attributing the udpg[rs]o and timestamp tests to you, but I haven't actually checked.. are they not yours? :)
Ah, BTW, a major source of failures seems to be that iptables is mapping to nftables on the executor. And either nftables doesn't support the functionality the tests expect or we're missing configs :( E.g. the TTL module.
I don't know if it is the same issue, but for MPTCP, we use 'iptables-legacy' if available.
Quoted 7 lines
Quoted 1 line
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=0c4cd3f86a400
I'd suggest you do the other way around, first check if iptables-nft is available, otherwise fall back to iptables-nft
commit refers to 5.15 already have iptables-nft support, it should work out of the box.
Good point, I understand it sounds better to use 'iptables-nft' in new kselftests. I should have added a bit of background and not just a link to this commit: at that time (around ~v6.4), we didn't need to force using 'iptables-legacy' on -net or net-next tree. But we needed that when testing kernels <= v5.15.
When validating (old) stable kernels, the recommended practice is apparently [1] to use the kselftests from the last stable version, e.g. using the kselftests from v6.7.4 when validating kernel v5.15.148. The kselftests are then supposed to support older kernels, e.g. by skipping some parts if a feature is not available. I didn't know about that before, and I don't know if all kselftests devs know about that.
I don't think that's easy to support old kernels, especially in the networking area, where some features/behaviours are not directly exposed to the userspace. Some MPTCP kselftests have to look at /proc/kallsyms or use other (ugly?) workarounds [2] to predict what we are supposed to have, depending on the kernel that is being used. But something has to be done, not to have big kselftests, with many different subtests, always marked as "failed" when validating new stable releases.
Back to the modification to use 'iptables-legacy', maybe a kernel config was missing, but the same kselftest, with the same list of kconfig to add, was not working with the v5.15 kernel, while everything was OK with a v6.4 one. With 'iptables-legacy', the test was running fine on both. I will check if maybe an old kconfig option was not missing.
[1] https://lore.kernel.org/stable/[email protected]/
[2]
https://lore.kernel.org/netdev/20230609-upstream-net-20230610-mptcp-selftests-support-old-kernels-part-3-v1-0-2896fe2ee8a3@tessares.net
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.
Pablo Neira Ayuso
maintainerHi Matthieu,
On Tue, Feb 06, 2024 at 07:31:44PM +0100, Matthieu Baerts wrote:
[...]
Quoted 6 lines
Good point, I understand it sounds better to use 'iptables-nft' in new kselftests. I should have added a bit of background and not just a link to this commit: at that time (around ~v6.4), we didn't need to force using 'iptables-legacy' on -net or net-next tree. But we needed that when testing kernels <= v5.15.
Quoted 6 lines
When validating (old) stable kernels, the recommended practice is apparently [1] to use the kselftests from the last stable version, e.g. using the kselftests from v6.7.4 when validating kernel v5.15.148. The kselftests are then supposed to support older kernels, e.g. by skipping some parts if a feature is not available. I didn't know about that before, and I don't know if all kselftests devs know about that.
We are sending backports to stable kernels, if one stable kernel fails, then we have to fix it.
Quoted 7 lines
I don't think that's easy to support old kernels, especially in the networking area, where some features/behaviours are not directly exposed to the userspace. Some MPTCP kselftests have to look at
/proc/kallsymsor use other (ugly?) workarounds [2] to predict what we are supposed to have, depending on the kernel that is being used. But something has to be done, not to have big kselftests, with many different subtests, always marked as "failed" when validating new stable releases.
iptables-nft is supported in all of the existing stable kernels.
Quoted 5 lines
Back to the modification to use 'iptables-legacy', maybe a kernel config was missing, but the same kselftest, with the same list of kconfig to add, was not working with the v5.15 kernel, while everything was OK with a v6.4 one. With 'iptables-legacy', the test was running fine on both. I will check if maybe an old kconfig option was not missing.
I suspect this is most likely kernel config missing, as it happened to Jakub.
Thanks.
Hi Pablo,
Thank you for your reply!
Quoted 17 lines · Pablo Neira Ayuso
Hi Matthieu,
On Tue, Feb 06, 2024 at 07:31:44PM +0100, Matthieu Baerts wrote:
[...]Quoted 6 lines
Good point, I understand it sounds better to use 'iptables-nft' in new kselftests. I should have added a bit of background and not just a link to this commit: at that time (around ~v6.4), we didn't need to force using 'iptables-legacy' on -net or net-next tree. But we needed that when testing kernels <= v5.15.
Quoted 6 lines
When validating (old) stable kernels, the recommended practice is apparently [1] to use the kselftests from the last stable version, e.g. using the kselftests from v6.7.4 when validating kernel v5.15.148. The kselftests are then supposed to support older kernels, e.g. by skipping some parts if a feature is not available. I didn't know about that before, and I don't know if all kselftests devs know about that.
Quoted 2 lines
We are sending backports to stable kernels, if one stable kernel fails, then we have to fix it.
Do you validate stable kernels by running the kselftests from the same version (e.g. both from v5.15.x) or by using the kselftests from the last stable one (e.g. kernel v5.15.148 validated using the kselftests from v6.7.4)?
Quoted 9 lines
Quoted 7 lines
I don't think that's easy to support old kernels, especially in the networking area, where some features/behaviours are not directly exposed to the userspace. Some MPTCP kselftests have to look at
/proc/kallsymsor use other (ugly?) workarounds [2] to predict what we are supposed to have, depending on the kernel that is being used. But something has to be done, not to have big kselftests, with many different subtests, always marked as "failed" when validating new stable releases.iptables-nft is supported in all of the existing stable kernels.
OK, then we should not have had the bug we had. I thought we were using features that were not supported in v5.15.
Quoted 7 lines
Quoted 5 lines
Back to the modification to use 'iptables-legacy', maybe a kernel config was missing, but the same kselftest, with the same list of kconfig to add, was not working with the v5.15 kernel, while everything was OK with a v6.4 one. With 'iptables-legacy', the test was running fine on both. I will check if maybe an old kconfig option was not missing.
I suspect this is most likely kernel config missing, as it happened to Jakub.
Probably, yes. I just retried by testing a v5.15.148 kernel using the kselftests from the net-next tree and forcing iptables-nft: I no longer have the issue I had one year ago. Not sure why, we already had NFT_COMPAT=m back then. Maybe because we recently added IP_NF_FILTER and similar, because we noticed some CI didn't have them?
Anyway, I will then switch back to iptables-nft. Thanks for the suggestion!
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.
Re: [TEST] bridge tests (was: net-next is OPEN)
In reply to Jakub KicinskiQuoted 14 lines · Jakub Kicinski
Quoted 13 lines · On Thu, 1 Feb 2024 07:30:25 -0800 Jakub Kicinski wrote:
Quoted 12 lines · On Thu, 1 Feb 2024 15:46:23 +0200 Ido Schimmel wrote:
Quoted 2 lines
selftests-net/test-bridge-neigh-suppress-sh
- fails across all, so must be the OS rather than the "speed"Yes, it's something related to the OS. From the log below:
``` COMMAND: ip netns exec h1-n8Aaip ndisc6 -q -r 1 -s 2001:db8:1::1 -w 5000 2001:db8:1::2 eth0.10 Raw IPv6 socket: Operation not permitted TEST: ndisc6 [FAIL] rc=1, expected 0 ```
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
The test is supposed to be run as root so I'm not sure what this error is about. Do you have something like AppArmor or SELinux running? The program creates an IPv6 raw socket and requires
CAP_NET_RAW.Ah, ugh, sorry for the misdirection, you're right.
Confirmed, with the SUID cleared test-bridge-neigh-suppress-sh now passes on everything with the exception of metal+debug kernel.
I'm sorry for bothering you with this, but I checked today's logs and it seems there is a similar problem with arping:
https://netdev-3.bots.linux.dev/vmksft-net-dbg/results/456561/8-test-bridge-neigh-suppress-sh/stdout
https://netdev-3.bots.linux.dev/vmksft-net/results/456562/6-test-bridge-neigh-suppress-sh/stdout
And according to this log the ndisc6 problem resurfaced:
https://netdev-3.bots.linux.dev/vmksft-net/results/456382/6-test-bridge-neigh-suppress-sh/stdout
Any chance that something in the OS changed since last week?
Jakub Kicinski
maintainerRe: [TEST] bridge tests (was: net-next is OPEN)
Quoted 8 lines · On Thu, 8 Feb 2024 18:21:29 +0200 Ido Schimmel wrote:
Quoted 5 lines · Jakub Kicinski
Quoted 1 line · On Thu, 1 Feb 2024 07:30:25 -0800 Jakub Kicinski wrote:
Ah, ugh, sorry for the misdirection, you're right.
Confirmed, with the SUID cleared test-bridge-neigh-suppress-sh now passes on everything with the exception of metal+debug kernel.
I'm sorry for bothering you with this,
No worries at all :)
Quoted 7 lines
but I checked today's logs and it
seems there is a similar problem with arping:
https://netdev-3.bots.linux.dev/vmksft-net-dbg/results/456561/8-test-bridge-neigh-suppress-sh/stdout
https://netdev-3.bots.linux.dev/vmksft-net/results/456562/6-test-bridge-neigh-suppress-sh/stdoutAnd according to this log the ndisc6 problem resurfaced:
https://netdev-3.bots.linux.dev/vmksft-net/results/456382/6-test-bridge-neigh-suppress-sh/stdout
It seems pretty faily:
https://netdev.bots.linux.dev/flakes.html?br-cnt=80&tn-needle=bridge-neigh&min-flip=0
I guess the short green streak does coincide with the fix, tho. Hm.
Quoted 1 line
Any chance that something in the OS changed since last week?
My memory is bad so I started keeping a log:
But it's not the suid:
$ find tools/fs/ -perm -4000 $ ll $(find tools/fs/ -name arping) -rwxr-xr-x. 1 virtme virtme 79360 Jan 29 14:38 tools/fs/usr/bin/arping $ ll $(find tools/fs/ -name ndisc6) -rwxr-xr-x. 1 virtme virtme 53840 Jan 29 14:36 tools/fs/usr/bin/ndisc6
And it occasionally blips to "pass" so can't be completely broken env.
I run the test now with VERBOSE=1 and -q removed from arping and ndisc6:
https://netdev-3.bots.linux.dev/vmksft-net/test_bridge_neigh_suppress/
It just says: Received 0 response(s) :(
Repro is 100%, LMK if you'd like me to try with some custom diff on top..
Pablo Neira Ayuso
maintainerHi,
Sorry for taking a while.
Quoted 22 lines · Matthieu Baerts
Hi Pablo,
Thank you for your reply!
Quoted 17 lines · Pablo Neira Ayuso
Hi Matthieu,
On Tue, Feb 06, 2024 at 07:31:44PM +0100, Matthieu Baerts wrote:
[...]Quoted 12 lines
Good point, I understand it sounds better to use 'iptables-nft' in new kselftests. I should have added a bit of background and not just a link to this commit: at that time (around ~v6.4), we didn't need to force using 'iptables-legacy' on -net or net-next tree. But we needed that when testing kernels <= v5.15.
When validating (old) stable kernels, the recommended practice is apparently [1] to use the kselftests from the last stable version, e.g. using the kselftests from v6.7.4 when validating kernel v5.15.148. The kselftests are then supposed to support older kernels, e.g. by skipping some parts if a feature is not available. I didn't know about that before, and I don't know if all kselftests devs know about that.
Quoted 7 lines
Quoted 2 lines
We are sending backports to stable kernels, if one stable kernel fails, then we have to fix it.
Do you validate stable kernels by running the kselftests from the same version (e.g. both from v5.15.x) or by using the kselftests from the last stable one (e.g. kernel v5.15.148 validated using the kselftests from v6.7.4)?
We have kselftests, but nftables/tests/shell probe for kernel capabilities then it runs tests according to what the kernel supports, this includes packet and control plane path tests. For iptables, there are iptables-tests.py for the control plane path.
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
I don't think that's easy to support old kernels, especially in the networking area, where some features/behaviours are not directly exposed to the userspace. Some MPTCP kselftests have to look at
/proc/kallsymsor use other (ugly?) workarounds [2] to predict what we are supposed to have, depending on the kernel that is being used. But something has to be done, not to have big kselftests, with many different subtests, always marked as "failed" when validating new stable releases.
Quoted 4 lines
Quoted 1 line
iptables-nft is supported in all of the existing stable kernels.
OK, then we should not have had the bug we had. I thought we were using features that were not supported in v5.15.
I don't think so, iptables-nft supports the same features as iptables-legacy.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Back to the modification to use 'iptables-legacy', maybe a kernel config was missing, but the same kselftest, with the same list of kconfig to add, was not working with the v5.15 kernel, while everything was OK with a v6.4 one. With 'iptables-legacy', the test was running fine on both. I will check if maybe an old kconfig option was not missing.
Quoted 8 lines
Quoted 1 line
I suspect this is most likely kernel config missing, as it happened to Jakub.
Probably, yes. I just retried by testing a v5.15.148 kernel using the kselftests from the net-next tree and forcing iptables-nft: I no longer have the issue I had one year ago. Not sure why, we already had NFT_COMPAT=m back then. Maybe because we recently added
IP_NF_FILTERand similar, because we noticed some CI didn't have them?
Anyway, I will then switch back to iptables-nft. Thanks for the suggestion!
Thanks. If you experience any issue, report back to netfilter-devel@
Hi Pablo,
Thank you for your reply!
Quoted 35 lines · Pablo Neira Ayuso
Hi,
Sorry for taking a while.
Quoted 22 lines · Matthieu Baerts
Hi Pablo,
Thank you for your reply!
Quoted 17 lines · Pablo Neira Ayuso
Hi Matthieu,
On Tue, Feb 06, 2024 at 07:31:44PM +0100, Matthieu Baerts wrote:
[...]Quoted 12 lines
Good point, I understand it sounds better to use 'iptables-nft' in new kselftests. I should have added a bit of background and not just a link to this commit: at that time (around ~v6.4), we didn't need to force using 'iptables-legacy' on -net or net-next tree. But we needed that when testing kernels <= v5.15.
When validating (old) stable kernels, the recommended practice is apparently [1] to use the kselftests from the last stable version, e.g. using the kselftests from v6.7.4 when validating kernel v5.15.148. The kselftests are then supposed to support older kernels, e.g. by skipping some parts if a feature is not available. I didn't know about that before, and I don't know if all kselftests devs know about that.
Quoted 7 lines
Quoted 2 lines
We are sending backports to stable kernels, if one stable kernel fails, then we have to fix it.
Do you validate stable kernels by running the kselftests from the same version (e.g. both from v5.15.x) or by using the kselftests from the last stable one (e.g. kernel v5.15.148 validated using the kselftests from v6.7.4)?
Quoted 4 lines
We have kselftests, but nftables/tests/shell probe for kernel capabilities then it runs tests according to what the kernel supports, this includes packet and control plane path tests. For iptables, there are
iptables-tests.pyfor the control plane path.
That's great! It is good to support all the different kernels.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
I don't think that's easy to support old kernels, especially in the networking area, where some features/behaviours are not directly exposed to the userspace. Some MPTCP kselftests have to look at
/proc/kallsymsor use other (ugly?) workarounds [2] to predict what we are supposed to have, depending on the kernel that is being used. But something has to be done, not to have big kselftests, with many different subtests, always marked as "failed" when validating new stable releases.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
iptables-nft is supported in all of the existing stable kernels.
OK, then we should not have had the bug we had. I thought we were using features that were not supported in v5.15.
I don't think so, iptables-nft supports the same features as iptables-legacy.
We were probably unlucky and hit a kernel/userspace bug that has been fixed in between, sorry for the noise!
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Back to the modification to use 'iptables-legacy', maybe a kernel config was missing, but the same kselftest, with the same list of kconfig to add, was not working with the v5.15 kernel, while everything was OK with a v6.4 one. With 'iptables-legacy', the test was running fine on both. I will check if maybe an old kconfig option was not missing.
I suspect this is most likely kernel config missing, as it happened to Jakub.
Quoted 8 lines
Quoted 6 lines
Probably, yes. I just retried by testing a v5.15.148 kernel using the kselftests from the net-next tree and forcing iptables-nft: I no longer have the issue I had one year ago. Not sure why, we already had NFT_COMPAT=m back then. Maybe because we recently added
IP_NF_FILTERand similar, because we noticed some CI didn't have them?
Anyway, I will then switch back to iptables-nft. Thanks for the suggestion!Thanks. If you experience any issue, report back to netfilter-devel@
Will do, thank you!
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.