Skip to contentSkip to conversation

selftests/bpf: Mount debugfs in setns_by_fd

PATCH Applied Stanislav Fomichev 4 participants 7 messages Nov 23, 2022 – Nov 24, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 selftests/bpf: Mount debugfs in setns_by_fd

Jiri reports broken test_progs after recent commit 68f8e3d4b916 ("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount /sys/kernel/debug.

0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1

Fixes a30338840fa5 ("selftests/bpf: Move open_netns() and close_netns() into network_helpers.c")
---
 tools/testing/selftests/bpf/network_helpers.c | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/bpf/network_helpers.c +4 -0
index bec15558fd93..1f37adff7632 100644
@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)
 	if (!ASSERT_OK(err, "mount /sys/fs/bpf"))
 		return err;
 
+	err = mount("debugfs", "/sys/kernel/debug", "debugfs", 0, NULL);
+	if (!ASSERT_OK(err, "mount /sys/kernel/debug"))
+		return err;
+
 	return 0;
 }
 
Signature · 1 line
2.38.1.584.g0f3c55d4c2-goog
#1 lore raw
Quoted 20 lines · Stanislav Fomichev

Jiri reports broken test_progs after recent commit 68f8e3d4b916 ("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount /sys/kernel/debug.

0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1

Fixes a30338840fa5 ("selftests/bpf: Move open_netns() and close_netns() into network_helpers.c")
---
 tools/testing/selftests/bpf/network_helpers.c | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/bpf/network_helpers.c +0 -0
index bec15558fd93..1f37adff7632 100644

tools/testing/selftests/bpf/network_helpers.c static int setns_by_fd(int nsfd) · line 434

Quoted 9 lines
tools/testing/selftests/bpf/network_helpers.c +4 -0
@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)
        if (!ASSERT_OK(err, "mount /sys/fs/bpf"))
                return err;
 
+       err = mount("debugfs", "/sys/kernel/debug", "debugfs", 0, NULL);
+       if (!ASSERT_OK(err, "mount /sys/kernel/debug"))
+               return err;
+
        return 0;
 }

Thanks.
It fixes part of it but it's still racy.
I see:
do_read:FAIL:open open /sys/fs/bpf/bpf_iter_test1 failed: No such file or directory

I suspect it happens when iter tests are running while test_empty_skb is cleaning the netns.

So I've added:
-void test_empty_skb(void)
+void serial_test_empty_skb(void)
-void test_xdp_do_redirect(void)
+void serial_test_xdp_do_redirect(void)
-void test_xdp_synproxy(void)
+void serial_test_xdp_synproxy(void)

to stop the bleeding and applied.

#5 lore raw
In reply to Stanislav Fomichev
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 8 lines · On Wed, 23 Nov 2022 12:08:29 -0800 you wrote:

Jiri reports broken test_progs after recent commit 68f8e3d4b916 ("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount /sys/kernel/debug.

0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1

[...]

Here is the summary with links:
- [bpf-next] selftests/bpf: Mount debugfs in setns_by_fd

    https://git.kernel.org/bpf/bpf-next/c/8ac88eece800

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
Acked-by Tested-by on v1 1/1
#6 lore raw
In reply to Stanislav Fomichev
Quoted 10 lines · Stanislav Fomichev

Jiri reports broken test_progs after recent commit 68f8e3d4b916 ("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount /sys/kernel/debug.

0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1

Fixes a30338840fa5 ("selftests/bpf: Move open_netns() and close_netns() into network_helpers.c")

thanks,
jirka

Quoted 22 lines
---
 tools/testing/selftests/bpf/network_helpers.c | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/bpf/network_helpers.c +4 -0
index bec15558fd93..1f37adff7632 100644
@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)
 	if (!ASSERT_OK(err, "mount /sys/fs/bpf"))
 		return err;
 
+	err = mount("debugfs", "/sys/kernel/debug", "debugfs", 0, NULL);
+	if (!ASSERT_OK(err, "mount /sys/kernel/debug"))
+		return err;
+
 	return 0;
 }
 
Signature · 1 line
2.38.1.584.g0f3c55d4c2-goog
#2 lore raw
In reply to Alexei Starovoitov
Quoted 49 lines · Alexei Starovoitov
Quoted 20 lines · Stanislav Fomichev

Jiri reports broken test_progs after recent commit 68f8e3d4b916 ("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount /sys/kernel/debug.

0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1

Fixes a30338840fa5 ("selftests/bpf: Move open_netns() and close_netns() into network_helpers.c")
---
 tools/testing/selftests/bpf/network_helpers.c | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/bpf/network_helpers.c +0 -0
index bec15558fd93..1f37adff7632 100644

tools/testing/selftests/bpf/network_helpers.c static int setns_by_fd(int nsfd) · line 434

Quoted 9 lines
tools/testing/selftests/bpf/network_helpers.c +4 -0
@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)
        if (!ASSERT_OK(err, "mount /sys/fs/bpf"))
                return err;
 
+       err = mount("debugfs", "/sys/kernel/debug", "debugfs", 0, NULL);
+       if (!ASSERT_OK(err, "mount /sys/kernel/debug"))
+               return err;
+
        return 0;
 }

Thanks.
It fixes part of it but it's still racy.
I see:
do_read:FAIL:open open /sys/fs/bpf/bpf_iter_test1 failed: No such file or directory

I suspect it happens when iter tests are running while test_empty_skb is cleaning the netns.

So I've added:

(quoted diff) +3 -3
-void test_empty_skb(void)
+void serial_test_empty_skb(void)
-void test_xdp_do_redirect(void)
+void serial_test_xdp_do_redirect(void)
-void test_xdp_synproxy(void)
+void serial_test_xdp_synproxy(void)
Quoted 1 line

to stop the bleeding and applied.

Not sure I understand where the race is coming from and no luck reproducing locally :-(
Looks like we run the tests in the forked workers, so that unshare(mountns) shouldn't theoretically affect the rest, but obviously I'm missing something..

#3 lore raw
Quoted 52 lines · Stanislav Fomichev
Quoted 49 lines · Alexei Starovoitov
Quoted 20 lines · Stanislav Fomichev

Jiri reports broken test_progs after recent commit 68f8e3d4b916 ("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount /sys/kernel/debug.

0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1

Fixes a30338840fa5 ("selftests/bpf: Move open_netns() and close_netns() into network_helpers.c")
---
 tools/testing/selftests/bpf/network_helpers.c | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/bpf/network_helpers.c +0 -0
index bec15558fd93..1f37adff7632 100644

tools/testing/selftests/bpf/network_helpers.c static int setns_by_fd(int nsfd) · line 434

Quoted 9 lines
tools/testing/selftests/bpf/network_helpers.c +4 -0
@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)
        if (!ASSERT_OK(err, "mount /sys/fs/bpf"))
                return err;
 
+       err = mount("debugfs", "/sys/kernel/debug", "debugfs", 0, NULL);
+       if (!ASSERT_OK(err, "mount /sys/kernel/debug"))
+               return err;
+
        return 0;
 }

Thanks.
It fixes part of it but it's still racy.
I see:
do_read:FAIL:open open /sys/fs/bpf/bpf_iter_test1 failed: No such file or directory

I suspect it happens when iter tests are running while test_empty_skb is cleaning the netns.

So I've added:

(quoted diff) +3 -3
-void test_empty_skb(void)
+void serial_test_empty_skb(void)
-void test_xdp_do_redirect(void)
+void serial_test_xdp_do_redirect(void)
-void test_xdp_synproxy(void)
+void serial_test_xdp_synproxy(void)
Quoted 7 lines
Quoted 1 line

to stop the bleeding and applied.

Not sure I understand where the race is coming from and no luck reproducing locally :-(
Looks like we run the tests in the forked workers, so that unshare(mountns) shouldn't theoretically affect the rest, but obviously I'm missing something..

I'm equally confused.
If what you're describing was the case than the bug of not mounting debugfs wouldn't have caused issues in parallel run. That close_netns is somehow messing with other forked processes.

#4 lore raw
Quoted 63 lines · Alexei Starovoitov
Quoted 52 lines · Stanislav Fomichev
Quoted 49 lines · Alexei Starovoitov
Quoted 20 lines · Stanislav Fomichev

Jiri reports broken test_progs after recent commit 68f8e3d4b916 ("selftests/bpf: Make sure zero-len skbs aren't redirectable"). Apparently we don't remount debugfs when we switch back networking namespace. Let's explicitly mount /sys/kernel/debug.

0: https://lore.kernel.org/bpf/[email protected]/T/#ma66ca9c92e99eee0a25e40f422489b26ee0171c1

Fixes a30338840fa5 ("selftests/bpf: Move open_netns() and close_netns() into network_helpers.c")
---
 tools/testing/selftests/bpf/network_helpers.c | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/bpf/network_helpers.c +0 -0
index bec15558fd93..1f37adff7632 100644

tools/testing/selftests/bpf/network_helpers.c static int setns_by_fd(int nsfd) · line 434

Quoted 9 lines
tools/testing/selftests/bpf/network_helpers.c +4 -0
@@ -426,6 +426,10 @@ static int setns_by_fd(int nsfd)
        if (!ASSERT_OK(err, "mount /sys/fs/bpf"))
                return err;
 
+       err = mount("debugfs", "/sys/kernel/debug", "debugfs", 0, NULL);
+       if (!ASSERT_OK(err, "mount /sys/kernel/debug"))
+               return err;
+
        return 0;
 }

Thanks.
It fixes part of it but it's still racy.
I see:
do_read:FAIL:open open /sys/fs/bpf/bpf_iter_test1 failed: No such file or directory

I suspect it happens when iter tests are running while test_empty_skb is cleaning the netns.

So I've added:

(quoted diff) +3 -3
-void test_empty_skb(void)
+void serial_test_empty_skb(void)
-void test_xdp_do_redirect(void)
+void serial_test_xdp_do_redirect(void)
-void test_xdp_synproxy(void)
+void serial_test_xdp_synproxy(void)
Quoted 7 lines
Quoted 1 line

to stop the bleeding and applied.

Not sure I understand where the race is coming from and no luck reproducing locally :-(
Looks like we run the tests in the forked workers, so that unshare(mountns) shouldn't theoretically affect the rest, but obviously I'm missing something..

I'm equally confused.

[..]

Quoted 2 lines

If what you're describing was the case than the bug of not mounting debugfs wouldn't have caused issues in parallel run.

The workers are reused and aren't respawned for every test. If some of them runs empty_skb test, its debugfs is not mounted anymore and any other test that requires debugfs (and lands on this worker) will be broken.

Quoted 1 line

That close_netns is somehow messing with other forked processes.

Yeah, agreed, looks like /sys/fs/bpf disappears while bpf_iter test is running. I can try to stick open/close_netns in a bunch of places in test_progs to see if I can trigger the issue. (or maybe have a test in parallel that does while(1) {openns(); closens(); }