selftests/bpf: Mount debugfs in setns_by_fd
Subsystems (3)
PATCH 1/1 selftests/bpf: Mount debugfs in setns_by_fd
Jiri reports broken test_progs after recent commit 68f8e3d4b916 ("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount /sys/kernel/debug.
0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1
--- tools/testing/selftests/bpf/network_helpers.c | 4 ++++ 1 file changed, 4 insertions(+)
tools/testing/selftests/bpf/network_helpers.c +4 -0
@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)if (!ASSERT_OK (err,"mount /sys/fs/bpf" ))return err; + err =mount ("debugfs" ,"/sys/kernel/debug" ,"debugfs" ,0 ,NULL ); +if (!ASSERT_OK (err,"mount /sys/kernel/debug" )) +return err; +return 0 ; }
Signature · 1 line
2.38.1.584.g0f3c55d4c2-goog
Quoted 20 lines · Stanislav Fomichev
Jiri reports broken
test_progsafter recent commit68f8e3d4b916("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount/sys/kernel/debug.0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1
Fixes a30338840fa5 ("selftests/bpf: Move open_netns() and close_netns() into network_helpers.c")Reported-by Jiri Olsa <[email protected]>Signed-off-by Stanislav Fomichev <[email protected]>--- tools/testing/selftests/bpf/network_helpers.c | 4 ++++ 1 file changed, 4 insertions(+)tools/testing/selftests/bpf/network_helpers.c +0
-0
tools/testing/selftests/bpf/network_helpers.c static int setns_by_fd(int nsfd) · line 434
Quoted 9 lines
tools/testing/selftests/bpf/network_helpers.c +4
-0@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)if (!ASSERT_OK (err,"mount /sys/fs/bpf" ))return err; + err =mount ("debugfs" ,"/sys/kernel/debug" ,"debugfs" ,0 ,NULL ); +if (!ASSERT_OK (err,"mount /sys/kernel/debug" )) +return err; +return 0 ; }
Thanks.
It fixes part of it but it's still racy.
I see:
do_read:FAIL:open open /sys/fs/bpf/bpf_iter_test1 failed: No such file or directory
I suspect it happens when iter tests are running while test_empty_skb is cleaning the netns.
So I've added:
-void test_empty_skb(void)
+void serial_test_empty_skb(void)
-void test_xdp_do_redirect(void)
+void serial_test_xdp_do_redirect(void)
-void test_xdp_synproxy(void)
+void serial_test_xdp_synproxy(void)
to stop the bleeding and applied.
Quoted 49 lines · Alexei Starovoitov
Quoted 20 lines · Stanislav Fomichev
Jiri reports broken
test_progsafter recent commit68f8e3d4b916("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount/sys/kernel/debug.0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1
Fixes a30338840fa5 ("selftests/bpf: Move open_netns() and close_netns() into network_helpers.c")Reported-by Jiri Olsa <[email protected]>Signed-off-by Stanislav Fomichev <[email protected]>--- tools/testing/selftests/bpf/network_helpers.c | 4 ++++ 1 file changed, 4 insertions(+)tools/testing/selftests/bpf/network_helpers.c +0
-0tools/testing/selftests/bpf/network_helpers.c static int setns_by_fd(int nsfd) · line 434
Quoted 9 lines
tools/testing/selftests/bpf/network_helpers.c +4
-0@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)if (!ASSERT_OK (err,"mount /sys/fs/bpf" ))return err; + err =mount ("debugfs" ,"/sys/kernel/debug" ,"debugfs" ,0 ,NULL ); +if (!ASSERT_OK (err,"mount /sys/kernel/debug" )) +return err; +return 0 ; }Thanks.
It fixes part of it but it's still racy.
I see:
do_read:FAIL:open open/sys/fs/bpf/bpf_iter_test1failed: No such file or directoryI suspect it happens when iter tests are running while
test_empty_skbis cleaning the netns.So I've added:
(quoted diff) +3
-3-void test_empty_skb(void) +void serial_test_empty_skb(void) -void test_xdp_do_redirect(void) +void serial_test_xdp_do_redirect(void) -void test_xdp_synproxy(void) +void serial_test_xdp_synproxy(void)
Quoted 1 line
to stop the bleeding and applied.
Not sure I understand where the race is coming from and no luck reproducing locally :-(
Looks like we run the tests in the forked workers, so that unshare(mountns) shouldn't theoretically affect the rest, but obviously I'm missing something..
Quoted 52 lines · Stanislav Fomichev
Quoted 49 lines · Alexei Starovoitov
Quoted 20 lines · Stanislav Fomichev
Jiri reports broken
test_progsafter recent commit68f8e3d4b916("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount/sys/kernel/debug.0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1
Fixes a30338840fa5 ("selftests/bpf: Move open_netns() and close_netns() into network_helpers.c")Reported-by Jiri Olsa <[email protected]>Signed-off-by Stanislav Fomichev <[email protected]>--- tools/testing/selftests/bpf/network_helpers.c | 4 ++++ 1 file changed, 4 insertions(+)tools/testing/selftests/bpf/network_helpers.c +0
-0tools/testing/selftests/bpf/network_helpers.c static int setns_by_fd(int nsfd) · line 434
Quoted 9 lines
tools/testing/selftests/bpf/network_helpers.c +4
-0@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)if (!ASSERT_OK (err,"mount /sys/fs/bpf" ))return err; + err =mount ("debugfs" ,"/sys/kernel/debug" ,"debugfs" ,0 ,NULL ); +if (!ASSERT_OK (err,"mount /sys/kernel/debug" )) +return err; +return 0 ; }Thanks.
It fixes part of it but it's still racy.
I see:
do_read:FAIL:open open/sys/fs/bpf/bpf_iter_test1failed: No such file or directoryI suspect it happens when iter tests are running while
test_empty_skbis cleaning the netns.So I've added:
(quoted diff) +3
-3-void test_empty_skb(void) +void serial_test_empty_skb(void) -void test_xdp_do_redirect(void) +void serial_test_xdp_do_redirect(void) -void test_xdp_synproxy(void) +void serial_test_xdp_synproxy(void)
Quoted 7 lines
Quoted 1 line
to stop the bleeding and applied.
Not sure I understand where the race is coming from and no luck reproducing locally :-(
Looks like we run the tests in the forked workers, so that unshare(mountns) shouldn't theoretically affect the rest, but obviously I'm missing something..
I'm equally confused.
If what you're describing was the case than the bug of not mounting debugfs wouldn't have caused issues in parallel run. That close_netns is somehow messing with other forked processes.
Quoted 63 lines · Alexei Starovoitov
Quoted 52 lines · Stanislav Fomichev
Quoted 49 lines · Alexei Starovoitov
Quoted 20 lines · Stanislav Fomichev
Jiri reports broken
test_progsafter recent commit68f8e3d4b916("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount/sys/kernel/debug.0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1
Fixes a30338840fa5 ("selftests/bpf: Move open_netns() and close_netns() into network_helpers.c")Reported-by Jiri Olsa <[email protected]>Signed-off-by Stanislav Fomichev <[email protected]>--- tools/testing/selftests/bpf/network_helpers.c | 4 ++++ 1 file changed, 4 insertions(+)tools/testing/selftests/bpf/network_helpers.c +0
-0tools/testing/selftests/bpf/network_helpers.c static int setns_by_fd(int nsfd) · line 434
Quoted 9 lines
tools/testing/selftests/bpf/network_helpers.c +4
-0@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)if (!ASSERT_OK (err,"mount /sys/fs/bpf" ))return err; + err =mount ("debugfs" ,"/sys/kernel/debug" ,"debugfs" ,0 ,NULL ); +if (!ASSERT_OK (err,"mount /sys/kernel/debug" )) +return err; +return 0 ; }Thanks.
It fixes part of it but it's still racy.
I see:
do_read:FAIL:open open/sys/fs/bpf/bpf_iter_test1failed: No such file or directoryI suspect it happens when iter tests are running while
test_empty_skbis cleaning the netns.So I've added:
(quoted diff) +3
-3-void test_empty_skb(void) +void serial_test_empty_skb(void) -void test_xdp_do_redirect(void) +void serial_test_xdp_do_redirect(void) -void test_xdp_synproxy(void) +void serial_test_xdp_synproxy(void)Quoted 7 lines
Quoted 1 line
to stop the bleeding and applied.
Not sure I understand where the race is coming from and no luck reproducing locally :-(
Looks like we run the tests in the forked workers, so that unshare(mountns) shouldn't theoretically affect the rest, but obviously I'm missing something..I'm equally confused.
[..]
Quoted 2 lines
If what you're describing was the case than the bug of not mounting debugfs wouldn't have caused issues in parallel run.
The workers are reused and aren't respawned for every test. If some of them runs empty_skb test, its debugfs is not mounted anymore and any other test that requires debugfs (and lands on this worker) will be broken.
Quoted 1 line
That
close_netnsis somehow messing with other forked processes.
Yeah, agreed, looks like /sys/fs/bpf disappears while bpf_iter test is running. I can try to stick open/close_netns in a bunch of places in test_progs to see if I can trigger the issue. (or maybe have a test in parallel that does while(1) {openns(); closens(); }
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 8 lines · On Wed, 23 Nov 2022 12:08:29 -0800 you wrote:
Jiri reports broken
test_progsafter recent commit68f8e3d4b916("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount/sys/kernel/debug.0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1
[...]
Here is the summary with links:
- [bpf-next] selftests/bpf: Mount debugfs in setns_by_fd
https://git.kernel.org/bpf/bpf-next/c/8ac88eece800
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
Quoted 10 lines · Stanislav Fomichev
Jiri reports broken
test_progsafter recent commit68f8e3d4b916("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount/sys/kernel/debug.0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1
Fixes a30338840fa5 ("selftests/bpf: Move open_netns() and close_netns() into network_helpers.c")Reported-by Jiri Olsa <[email protected]>Signed-off-by Stanislav Fomichev <[email protected]>
thanks,
jirka
Quoted 22 lines
--- tools/testing/selftests/bpf/network_helpers.c | 4 ++++ 1 file changed, 4 insertions(+)tools/testing/selftests/bpf/network_helpers.c +4
-0@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)if (!ASSERT_OK (err,"mount /sys/fs/bpf" ))return err; + err =mount ("debugfs" ,"/sys/kernel/debug" ,"debugfs" ,0 ,NULL ); +if (!ASSERT_OK (err,"mount /sys/kernel/debug" )) +return err; +return 0 ; }Signature · 1 line
2.38.1.584.g0f3c55d4c2-goog