Skip to contentSkip to conversation

selftests/bpf: Remove "&>" usage in the selftests

PATCH Applied Martin KaFai Lau 3 participants 6 messages Jan 27, 2024 – Jan 29, 2024

Subsystems (4)
lore
Share & tools
lore

Martin KaFai Lau

author reviewer
#0 lore raw

PATCH 1/1 selftests/bpf: Remove "&>" usage in the selftests

From: Martin KaFai Lau <[email protected]>

In s390, CI reported that the sock_iter_batch selftest hits this error very often:

2024-01-26T16:56:49.3091804Z Bind /proc/self/ns/net -> /run/netns/sock_iter_batch_netns failed: No such file or directory
2024-01-26T16:56:49.3149524Z Cannot remove namespace file "/run/netns/sock_iter_batch_netns": No such file or directory
2024-01-26T16:56:49.3772213Z test_sock_iter_batch:FAIL:ip netns add sock_iter_batch_netns unexpected error: 256 (errno 0)

It happens very often in s390 but Manu also noticed it happens very sparsely in other arch also.

It turns out the default dash shell does not recognize "&>" as a redirection operator, so the command went to the background. In the sock_iter_batch selftest, the "ip netns delete" went into background and then race with the following "ip netns add" command.

This patch replaces the "&> /dev/null" usage with ">/dev/null 2>&1" and does this redirection in the SYS_NOFAIL macro instead of doing it individually by its caller. The SYS_NOFAIL callers do not care about failure, so it is no harm to do this redirection even if some of the existing callers do not redirect to /dev/null now.

It touches different test files, so I skipped the Fixes tags in this patch. Some of the changed tests do not use "&>" but they use the SYS_NOFAIL, so these tests are also changed to avoid doing its own redirection because SYS_NOFAIL does it internally now.

---
 .../selftests/bpf/prog_tests/decap_sanity.c    |  2 +-
 .../selftests/bpf/prog_tests/fib_lookup.c      |  2 +-
 .../selftests/bpf/prog_tests/ip_check_defrag.c |  4 ++--
 .../selftests/bpf/prog_tests/lwt_redirect.c    |  2 +-
 .../selftests/bpf/prog_tests/lwt_reroute.c     |  2 +-
 tools/testing/selftests/bpf/prog_tests/mptcp.c |  2 +-
 .../selftests/bpf/prog_tests/sock_destroy.c    |  2 +-
 .../selftests/bpf/prog_tests/sock_iter_batch.c |  4 ++--
 .../selftests/bpf/prog_tests/test_tunnel.c     | 18 +++++++++---------
 tools/testing/selftests/bpf/test_progs.h       |  7 ++++++-
 10 files changed, 25 insertions(+), 20 deletions(-)
tools/testing/selftests/bpf/prog_tests/decap_sanity.c +1 -1
index 5c0ebe6ba866..dcb9e5070cc3 100644
@@ -72,6 +72,6 @@ void test_decap_sanity(void)
 		bpf_tc_hook_destroy(&qdisc_hook);
 		close_netns(nstoken);
 	}
-	SYS_NOFAIL("ip netns del " NS_TEST " &> /dev/null");
+	SYS_NOFAIL("ip netns del " NS_TEST);
 	decap_sanity__destroy(skel);
 }
tools/testing/selftests/bpf/prog_tests/fib_lookup.c +1 -1
index 4ad4cd69152e..3379df2d4cf2 100644
@@ -298,6 +298,6 @@ void test_fib_lookup(void)
 fail:
 	if (nstoken)
 		close_netns(nstoken);
-	SYS_NOFAIL("ip netns del " NS_TEST " &> /dev/null");
+	SYS_NOFAIL("ip netns del " NS_TEST);
 	fib_lookup__destroy(skel);
 }
tools/testing/selftests/bpf/prog_tests/ip_check_defrag.c +2 -2
index 57c814f5f6a7..8dd2af9081f4 100644
@@ -59,9 +59,9 @@ static int setup_topology(bool ipv6)
 	/* Wait for up to 5s for links to come up */
 	for (i = 0; i < 5; ++i) {
 		if (ipv6)
-			up = !system("ip netns exec " NS0 " ping -6 -c 1 -W 1 " VETH1_ADDR6 " &>/dev/null");
+			up = !SYS_NOFAIL("ip netns exec " NS0 " ping -6 -c 1 -W 1 " VETH1_ADDR6);
 		else
-			up = !system("ip netns exec " NS0 " ping -c 1 -W 1 " VETH1_ADDR " &>/dev/null");
+			up = !SYS_NOFAIL("ip netns exec " NS0 " ping -c 1 -W 1 " VETH1_ADDR);
 
 		if (up)
 			break;
tools/testing/selftests/bpf/prog_tests/lwt_redirect.c +1 -1
index 59b38569f310..beeb3ac1c361 100644
@@ -85,7 +85,7 @@ static void ping_dev(const char *dev, bool is_ingress)
 		snprintf(ip, sizeof(ip), "20.0.0.%d", link_index);
 
 	/* We won't get a reply. Don't fail here */
-	SYS_NOFAIL("ping %s -c1 -W1 -s %d >/dev/null 2>&1",
+	SYS_NOFAIL("ping %s -c1 -W1 -s %d",
 		   ip, ICMP_PAYLOAD_SIZE);
 }
 
tools/testing/selftests/bpf/prog_tests/lwt_reroute.c +1 -1
index f4bb2d5fcae0..5610bc76928d 100644
@@ -63,7 +63,7 @@
 static void ping_once(const char *ip)
 {
 	/* We won't get a reply. Don't fail here */
-	SYS_NOFAIL("ping %s -c1 -W1 -s %d >/dev/null 2>&1",
+	SYS_NOFAIL("ping %s -c1 -W1 -s %d",
 		   ip, ICMP_PAYLOAD_SIZE);
 }
 
tools/testing/selftests/bpf/prog_tests/mptcp.c +1 -1
index 7c0be7cf550b..8f8d792307c1 100644
@@ -79,7 +79,7 @@ static void cleanup_netns(struct nstoken *nstoken)
 	if (nstoken)
 		close_netns(nstoken);
 
-	SYS_NOFAIL("ip netns del %s &> /dev/null", NS_TEST);
+	SYS_NOFAIL("ip netns del %s", NS_TEST);
 }
 
 static int verify_tsk(int map_fd, int client_fd)
tools/testing/selftests/bpf/prog_tests/sock_destroy.c +1 -1
index b0583309a94e..9c11938fe597 100644
@@ -214,7 +214,7 @@ void test_sock_destroy(void)
 cleanup:
 	if (nstoken)
 		close_netns(nstoken);
-	SYS_NOFAIL("ip netns del " TEST_NS " &> /dev/null");
+	SYS_NOFAIL("ip netns del " TEST_NS);
 	if (cgroup_fd >= 0)
 		close(cgroup_fd);
 	sock_destroy_prog__destroy(skel);
tools/testing/selftests/bpf/prog_tests/sock_iter_batch.c +2 -2
index 0c365f36c73b..d56e18b25528 100644
@@ -112,7 +112,7 @@ void test_sock_iter_batch(void)
 {
 	struct nstoken *nstoken = NULL;
 
-	SYS_NOFAIL("ip netns del " TEST_NS " &> /dev/null");
+	SYS_NOFAIL("ip netns del " TEST_NS);
 	SYS(done, "ip netns add %s", TEST_NS);
 	SYS(done, "ip -net %s link set dev lo up", TEST_NS);
 
@@ -131,5 +131,5 @@ void test_sock_iter_batch(void)
 	close_netns(nstoken);
 
 done:
-	SYS_NOFAIL("ip netns del " TEST_NS " &> /dev/null");
+	SYS_NOFAIL("ip netns del " TEST_NS);
 }
tools/testing/selftests/bpf/prog_tests/test_tunnel.c +9 -9
index 2b3c6dd66259..5f1fb0a2ea56 100644
@@ -118,9 +118,9 @@ static int config_device(void)
 static void cleanup(void)
 {
 	SYS_NOFAIL("test -f /var/run/netns/at_ns0 && ip netns delete at_ns0");
-	SYS_NOFAIL("ip link del veth1 2> /dev/null");
-	SYS_NOFAIL("ip link del %s 2> /dev/null", VXLAN_TUNL_DEV1);
-	SYS_NOFAIL("ip link del %s 2> /dev/null", IP6VXLAN_TUNL_DEV1);
+	SYS_NOFAIL("ip link del veth1");
+	SYS_NOFAIL("ip link del %s", VXLAN_TUNL_DEV1);
+	SYS_NOFAIL("ip link del %s", IP6VXLAN_TUNL_DEV1);
 }
 
 static int add_vxlan_tunnel(void)
@@ -265,9 +265,9 @@ static int add_ipip_tunnel(enum ipip_encap encap)
 static void delete_ipip_tunnel(void)
 {
 	SYS_NOFAIL("ip -n at_ns0 link delete dev %s", IPIP_TUNL_DEV0);
-	SYS_NOFAIL("ip -n at_ns0 fou del port 5555 2> /dev/null");
+	SYS_NOFAIL("ip -n at_ns0 fou del port 5555");
 	SYS_NOFAIL("ip link delete dev %s", IPIP_TUNL_DEV1);
-	SYS_NOFAIL("ip fou del port 5555 2> /dev/null");
+	SYS_NOFAIL("ip fou del port 5555");
 }
 
 static int add_xfrm_tunnel(void)
@@ -346,13 +346,13 @@ static int add_xfrm_tunnel(void)
 
 static void delete_xfrm_tunnel(void)
 {
-	SYS_NOFAIL("ip xfrm policy delete dir out src %s/32 dst %s/32 2> /dev/null",
+	SYS_NOFAIL("ip xfrm policy delete dir out src %s/32 dst %s/32",
 		   IP4_ADDR_TUNL_DEV1, IP4_ADDR_TUNL_DEV0);
-	SYS_NOFAIL("ip xfrm policy delete dir in src %s/32 dst %s/32 2> /dev/null",
+	SYS_NOFAIL("ip xfrm policy delete dir in src %s/32 dst %s/32",
 		   IP4_ADDR_TUNL_DEV0, IP4_ADDR_TUNL_DEV1);
-	SYS_NOFAIL("ip xfrm state delete src %s dst %s proto esp spi %d 2> /dev/null",
+	SYS_NOFAIL("ip xfrm state delete src %s dst %s proto esp spi %d",
 		   IP4_ADDR_VETH0, IP4_ADDR1_VETH1, XFRM_SPI_IN_TO_OUT);
-	SYS_NOFAIL("ip xfrm state delete src %s dst %s proto esp spi %d 2> /dev/null",
+	SYS_NOFAIL("ip xfrm state delete src %s dst %s proto esp spi %d",
 		   IP4_ADDR1_VETH1, IP4_ADDR_VETH0, XFRM_SPI_OUT_TO_IN);
 }
 
tools/testing/selftests/bpf/test_progs.h +6 -1
index 2f9f6f250f17..80df51244886 100644
@@ -385,10 +385,15 @@ int test__join_cgroup(const char *path);
 			goto goto_label;				\
 	})
 
+#define ALL_TO_DEV_NULL " >/dev/null 2>&1"
+
 #define SYS_NOFAIL(fmt, ...)						\
 	({								\
 		char cmd[1024];						\
-		snprintf(cmd, sizeof(cmd), fmt, ##__VA_ARGS__);		\
+		int n;							\
+		n = snprintf(cmd, sizeof(cmd), fmt, ##__VA_ARGS__);	\
+		if (n < sizeof(cmd) && sizeof(cmd) - n >= sizeof(ALL_TO_DEV_NULL)) \
+			strcat(cmd, ALL_TO_DEV_NULL);			\
 		system(cmd);						\
 	})
 
Signature · 1 line
2.34.1
reviewer
#1 lore raw
Quoted 5 lines · Martin KaFai Lau

From: Martin KaFai Lau <[email protected]>

In s390, CI reported that the sock_iter_batch selftest hits this error very often:

Quoted 8 lines
2024-01-26T16:56:49.3091804Z Bind /proc/self/ns/net -> /run/netns/sock_iter_batch_netns failed: No such file or directory
2024-01-26T16:56:49.3149524Z Cannot remove namespace file "/run/netns/sock_iter_batch_netns": No such file or directory
2024-01-26T16:56:49.3772213Z test_sock_iter_batch:FAIL:ip netns add sock_iter_batch_netns unexpected error: 256 (errno 0)

It happens very often in s390 but Manu also noticed it happens very sparsely in other arch also.

It turns out the default dash shell does not recognize "&>"

Not sure whether it is feasible or not. But is it possible for all our test VMs we run '/bin/bash' before everyting else so we have a uniform bash environment so we do not need to worry about other shells?

Quoted 43 lines

as a redirection operator, so the command went to the background. In the sock_iter_batch selftest, the "ip netns delete" went into background and then race with the following "ip netns add" command.

This patch replaces the "&> /dev/null" usage with ">/dev/null 2>&1" and does this redirection in the SYS_NOFAIL macro instead of doing it individually by its caller. The SYS_NOFAIL callers do not care about failure, so it is no harm to do this redirection even if some of the existing callers do not redirect to /dev/null now.

It touches different test files, so I skipped the Fixes tags in this patch. Some of the changed tests do not use "&>" but they use the SYS_NOFAIL, so these tests are also changed to avoid doing its own redirection because SYS_NOFAIL does it internally now.

---
  .../selftests/bpf/prog_tests/decap_sanity.c    |  2 +-
  .../selftests/bpf/prog_tests/fib_lookup.c      |  2 +-
  .../selftests/bpf/prog_tests/ip_check_defrag.c |  4 ++--
  .../selftests/bpf/prog_tests/lwt_redirect.c    |  2 +-
  .../selftests/bpf/prog_tests/lwt_reroute.c     |  2 +-
  tools/testing/selftests/bpf/prog_tests/mptcp.c |  2 +-
  .../selftests/bpf/prog_tests/sock_destroy.c    |  2 +-
  .../selftests/bpf/prog_tests/sock_iter_batch.c |  4 ++--
  .../selftests/bpf/prog_tests/test_tunnel.c     | 18 +++++++++---------
  tools/testing/selftests/bpf/test_progs.h       |  7 ++++++-
  10 files changed, 25 insertions(+), 20 deletions(-)
tools/testing/selftests/bpf/prog_tests/decap_sanity.c +1 -1
index 5c0ebe6ba866..dcb9e5070cc3 100644
@@ -72,6 +72,6 @@ void test_decap_sanity(void)
  		bpf_tc_hook_destroy(&qdisc_hook);
  		close_netns(nstoken);
  	}
-	SYS_NOFAIL("ip netns del " NS_TEST " &> /dev/null");
+	SYS_NOFAIL("ip netns del " NS_TEST);
  	decap_sanity__destroy(skel);
  }

[...]

Quoted 21 lines
tools/testing/selftests/bpf/test_progs.h +6 -1
index 2f9f6f250f17..80df51244886 100644
@@ -385,10 +385,15 @@ int test__join_cgroup(const char *path);
  			goto goto_label;				\
  	})
 
+#define ALL_TO_DEV_NULL " >/dev/null 2>&1"
+
  #define SYS_NOFAIL(fmt, ...)						\
  	({								\
  		char cmd[1024];						\
-		snprintf(cmd, sizeof(cmd), fmt, ##__VA_ARGS__);		\
+		int n;							\
+		n = snprintf(cmd, sizeof(cmd), fmt, ##__VA_ARGS__);	\
+		if (n < sizeof(cmd) && sizeof(cmd) - n >= sizeof(ALL_TO_DEV_NULL)) \
+			strcat(cmd, ALL_TO_DEV_NULL);			\
  		system(cmd);						\
  	})
 
author reviewer
#4 lore raw
In reply to Martin KaFai Lau

tools/testing/selftests/bpf/prog_tests/lwt_redirect.c static void ping_dev(const char *dev, bool is_ingress) · line 90

Quoted 12 lines · Martin KaFai Lau
tools/testing/selftests/bpf/prog_tests/lwt_redirect.c +1 -1
index 59b38569f310..beeb3ac1c361 100644
@@ -85,7 +85,7 @@ static void ping_dev(const char *dev, bool is_ingress)
  		snprintf(ip, sizeof(ip), "20.0.0.%d", link_index);
 
  	/* We won't get a reply. Don't fail here */
-	SYS_NOFAIL("ping %s -c1 -W1 -s %d >/dev/null 2>&1",
+	SYS_NOFAIL("ping %s -c1 -W1 -s %d",
  		   ip, ICMP_PAYLOAD_SIZE);
  }

The "lwt_redirect/lwt_redirect_normal" is flaky now in s390. I don't see how this change affected it other than moving the ">/dev/null 2>&1" part to SYS_NOFAIL.

====
Error: #142/1 lwt_redirect/lwt_redirect_normal
test_lwt_redirect_run:PASS:netns_create 0 nsec
open_netns:PASS:malloc token 0 nsec
open_netns:PASS:open /proc/self/ns/net 0 nsec
open_netns:PASS:open netns fd 0 nsec
open_netns:PASS:setns 0 nsec
test_lwt_redirect_run:PASS:setns 0 nsec
open_tuntap:PASS:open(/dev/net/tun) 0 nsec
open_tuntap:PASS:ioctl(TUNSETIFF) 0 nsec
open_tuntap:PASS:fcntl(O_NONBLOCK) 0 nsec
setup_redirect_target:PASS:open_tuntap 0 nsec
setup_redirect_target:PASS:if_nametoindex 0 nsec
setup_redirect_target:PASS:ip link add link_err type dummy 0 nsec
setup_redirect_target:PASS:ip link set lo up 0 nsec
setup_redirect_target:PASS:ip addr add dev lo 10.0.0.1/32 0 nsec
setup_redirect_target:PASS:ip link set link_err up 0 nsec
setup_redirect_target:PASS:ip link set tap0 up 0 nsec
setup_redirect_target:PASS:ip route add 10.0.0.0/24 dev link_err encap bpf xmit obj test_lwt_redirect.bpf.o sec redir_ingress 0 nsec setup_redirect_target:PASS:ip route add 20.0.0.0/24 dev link_err encap bpf xmit obj test_lwt_redirect.bpf.o sec redir_egress 0 nsec test_lwt_redirect_normal:PASS:setup_redirect_target 0 nsec
ping_dev:PASS:if_nametoindex 0 nsec
send_and_capture_test_packets:FAIL:wait_for_epacket unexpected wait_for_epacket: actual 0 != expected 1 (/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/lwt_redirect.c:175: errno: Success) test_lwt_redirect_normal egress test fails
====

May be the timeout is too short...

static void send_and_capture_test_packets(const char *test_name, int tap_fd,
const char *target_dev, bool need_mac)

{
	int psock = -1;
	struct timeval timeo = {
		.tv_sec = 0,
		.tv_usec = 250000,
	};

	/* ... */
}

[ ... ]

Quoted 20 lines
tools/testing/selftests/bpf/test_progs.h +6 -1
index 2f9f6f250f17..80df51244886 100644
@@ -385,10 +385,15 @@ int test__join_cgroup(const char *path);
  			goto goto_label;				\
  	})
 
+#define ALL_TO_DEV_NULL " >/dev/null 2>&1"
+
  #define SYS_NOFAIL(fmt, ...)						\
  	({								\
  		char cmd[1024];						\
-		snprintf(cmd, sizeof(cmd), fmt, ##__VA_ARGS__);		\
+		int n;							\
+		n = snprintf(cmd, sizeof(cmd), fmt, ##__VA_ARGS__);	\
+		if (n < sizeof(cmd) && sizeof(cmd) - n >= sizeof(ALL_TO_DEV_NULL)) \
+			strcat(cmd, ALL_TO_DEV_NULL);			\
  		system(cmd);						\
  	})
author reviewer
#2 lore raw
In reply to Yonghong Song
Quoted 14 lines · Yonghong Song
Quoted 12 lines · Martin KaFai Lau

From: Martin KaFai Lau <[email protected]>

In s390, CI reported that the sock_iter_batch selftest hits this error very often:

2024-01-26T16:56:49.3091804Z Bind /proc/self/ns/net -> /run/netns/sock_iter_batch_netns failed: No such file or directory
2024-01-26T16:56:49.3149524Z Cannot remove namespace file "/run/netns/sock_iter_batch_netns": No such file or directory
2024-01-26T16:56:49.3772213Z test_sock_iter_batch:FAIL:ip netns add sock_iter_batch_netns unexpected error: 256 (errno 0)

It happens very often in s390 but Manu also noticed it happens very sparsely in other arch also.

Quoted 6 lines
Quoted 1 line

It turns out the default dash shell does not recognize "&>"

Not sure whether it is feasible or not. But is it possible for all our test VMs we run '/bin/bash' before everyting else so we have a uniform bash environment so we do not need to worry about other shells?

It was my initial thought also. I think it makes sense to use the minimal shell feature such that it is more portable to different developer environments.

reviewer
#3 lore raw
Quoted 15 lines · Martin KaFai Lau
Quoted 14 lines · Yonghong Song
Quoted 12 lines · Martin KaFai Lau

From: Martin KaFai Lau <[email protected]>

In s390, CI reported that the sock_iter_batch selftest hits this error very often:

2024-01-26T16:56:49.3091804Z Bind /proc/self/ns/net -> /run/netns/sock_iter_batch_netns failed: No such file or directory
2024-01-26T16:56:49.3149524Z Cannot remove namespace file "/run/netns/sock_iter_batch_netns": No such file or directory
2024-01-26T16:56:49.3772213Z test_sock_iter_batch:FAIL:ip netns add sock_iter_batch_netns unexpected error: 256 (errno 0)

It happens very often in s390 but Manu also noticed it happens very sparsely in other arch also.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

It turns out the default dash shell does not recognize "&>"

Not sure whether it is feasible or not. But is it possible for all our test VMs we run '/bin/bash' before everyting else so we have a uniform bash environment so we do not need to worry about other shells?

It was my initial thought also. I think it makes sense to use the minimal shell feature such that it is more portable to different developer environments.

I just proposed an alternate solution. Since you have throught about this, I am certainly okay with the current patch.

#5 lore raw
In reply to Martin KaFai Lau
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 10 lines · On Fri, 26 Jan 2024 18:50:17 -0800 you wrote:

From: Martin KaFai Lau <[email protected]>

In s390, CI reported that the sock_iter_batch selftest hits this error very often:

2024-01-26T16:56:49.3091804Z Bind /proc/self/ns/net -> /run/netns/sock_iter_batch_netns failed: No such file or directory
2024-01-26T16:56:49.3149524Z Cannot remove namespace file "/run/netns/sock_iter_batch_netns": No such file or directory
2024-01-26T16:56:49.3772213Z test_sock_iter_batch:FAIL:ip netns add sock_iter_batch_netns unexpected error: 256 (errno 0)

[...]

Here is the summary with links:
- [bpf-next] selftests/bpf: Remove "&>" usage in the selftests

    https://git.kernel.org/bpf/bpf-next/c/fbaf59a9f513

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html