Skip to contentSkip to conversation

Fixes for sock_fields selftests

PATCH Superseded Jakub Sitnicki 2 participants 8 messages Feb 27, 2022 – Mar 3, 2022

Subsystems (5)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover Fixes for sock_fields selftests

This is a respin of a fix for error reporting in sock_fields test.

Fixing the error reporting has uncovered bugs in the recently added test for sk->dst_port access. Series now includes patches that address the broken test.

The series has been tested on x86_64 and s390.

v1 -> v2:
- Limit read_sk_dst_port only to client traffic (patch 2)
- Make read_sk_dst_port pass on litte- and big-endian (patch 3)

v1: https://lore.kernel.org/bpf/[email protected]/

Jakub Sitnicki (3):
selftests/bpf: Fix error reporting from sock_fields programs selftests/bpf: Check dst_port only on the client socket selftests/bpf: Fix test for 4-byte load from dst_port on big-endian

 .../selftests/bpf/progs/test_sock_fields.c    | 30 ++++++++++++++++---
 1 file changed, 26 insertions(+), 4 deletions(-)
Signature · 1 line
2.35.1
#1 lore raw

PATCH v2 1/3 selftests/bpf: Fix error reporting from sock_fields programs

The helper macro that records an error in BPF programs that exercise sock fields access has been inadvertently broken by adaptation work that happened in commit b18c1f0aa477 ("bpf: selftest: Adapt sock_fields test to use skel and global variables").

BPF_NOEXIST flag cannot be used to update BPF_MAP_TYPE_ARRAY. The operation always fails with -EEXIST, which in turn means the error never gets recorded, and the checks for errors always pass.

Revert the change in update flags.

Fixes b18c1f0aa477 ("bpf: selftest: Adapt sock_fields test to use skel and global variables")
---
 tools/testing/selftests/bpf/progs/test_sock_fields.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/progs/test_sock_fields.c +1 -1
index 246f1f001813..3e2e3ee51cc9 100644
@@ -114,7 +114,7 @@ static void tpcpy(struct bpf_tcp_sock *dst,
 
 #define RET_LOG() ({						\
 	linum = __LINE__;					\
-	bpf_map_update_elem(&linum_map, &linum_idx, &linum, BPF_NOEXIST);	\
+	bpf_map_update_elem(&linum_map, &linum_idx, &linum, BPF_ANY);	\
 	return CG_OK;						\
 })
 
Signature · 1 line
2.35.1
#2 lore raw

PATCH v2 2/3 selftests/bpf: Check dst_port only on the client socket

In reply to Jakub Sitnicki

cgroup_skb/egress programs which sock_fields test installs process packets flying in both directions, from the client to the server, and in reverse direction.

Recently added dst_port check relies on the fact that destination port (remote peer port) of the socket which sends the packet is known ahead of time. This holds true only for the client socket, which connects to the known server port.

Filter out any traffic that is not bound to be egressing from the client socket in the test program for reading the dst_port.

Fixes 8f50f16ff39d ("selftests/bpf: Extend verifier and bpf_sock tests for dst_port loads")
---
 .../testing/selftests/bpf/progs/test_sock_fields.c  | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/test_sock_fields.c +11 -2
index 3e2e3ee51cc9..186fed1deaab 100644
@@ -42,6 +42,11 @@ struct {
 	__type(value, struct bpf_spinlock_cnt);
 } sk_pkt_out_cnt10 SEC(".maps");
 
+enum {
+	TCP_SYN_SENT = 2,
+	TCP_LISTEN = 10,
+};
+
 struct bpf_tcp_sock listen_tp = {};
 struct sockaddr_in6 srv_sa6 = {};
 struct bpf_tcp_sock cli_tp = {};
@@ -138,7 +143,7 @@ int egress_read_sock_fields(struct __sk_buff *skb)
 	 * TCP_LISTEN (10) socket will be copied at the ingress side.
 	 */
 	if (sk->family != AF_INET6 || !is_loopback6(sk->src_ip6) ||
-	    sk->state == 10)
+	    sk->state == TCP_LISTEN)
 		return CG_OK;
 
 	if (sk->src_port == bpf_ntohs(srv_sa6.sin6_port)) {
@@ -233,7 +238,7 @@ int ingress_read_sock_fields(struct __sk_buff *skb)
 		return CG_OK;
 
 	/* Only interested in TCP_LISTEN */
-	if (sk->state != 10)
+	if (sk->state != TCP_LISTEN)
 		return CG_OK;
 
 	/* It must be a fullsock for cgroup_skb/ingress prog */
@@ -281,6 +286,10 @@ int read_sk_dst_port(struct __sk_buff *skb)
 	if (!sk)
 		RET_LOG();
 
+	/* Ignore everything but the SYN from the client socket */
+	if (sk->state != TCP_SYN_SENT)
+		return CG_OK;
+
 	if (!sk_dst_port__load_word(sk))
 		RET_LOG();
 	if (!sk_dst_port__load_half(sk))
Signature · 1 line
2.35.1
Acked-by on v2 2/3
#3 lore raw

Re: [PATCH bpf-next v2 2/3] selftests/bpf: Check dst_port only on the client socket

Quoted 23 lines · Jakub Sitnicki

cgroup_skb/egress programs which sock_fields test installs process packets flying in both directions, from the client to the server, and in reverse direction.

Recently added dst_port check relies on the fact that destination port (remote peer port) of the socket which sends the packet is known ahead of time. This holds true only for the client socket, which connects to the known server port.

Filter out any traffic that is not bound to be egressing from the client socket in the test program for reading the dst_port.

Fixes 8f50f16ff39d ("selftests/bpf: Extend verifier and bpf_sock tests for dst_port loads")
---
 .../testing/selftests/bpf/progs/test_sock_fields.c  | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/test_sock_fields.c +0 -0
index 3e2e3ee51cc9..186fed1deaab 100644

tools/testing/selftests/bpf/progs/test_sock_fields.c struct { · line 47

Quoted 6 lines
tools/testing/selftests/bpf/progs/test_sock_fields.c +3 -0
@@ -42,6 +42,11 @@ struct {
 	__type(value, struct bpf_spinlock_cnt);
 } sk_pkt_out_cnt10 SEC(".maps");
 
+enum {
+	TCP_SYN_SENT = 2,
+	TCP_LISTEN = 10,

Thanks for the clean up.

A nit. directly use BPF_TCP_SYN_SENT and BPF_TCP_LISTEN.

Others lgtm.

#4 lore raw

Re: [PATCH bpf-next v2 2/3] selftests/bpf: Check dst_port only on the client socket

Quoted 27 lines · Martin KaFai Lau
Quoted 26 lines · Jakub Sitnicki

cgroup_skb/egress programs which sock_fields test installs process packets flying in both directions, from the client to the server, and in reverse direction.

Recently added dst_port check relies on the fact that destination port (remote peer port) of the socket which sends the packet is known ahead of time. This holds true only for the client socket, which connects to the known server port.

Filter out any traffic that is not bound to be egressing from the client socket in the test program for reading the dst_port.

Fixes 8f50f16ff39d ("selftests/bpf: Extend verifier and bpf_sock tests for dst_port loads")
---
 .../testing/selftests/bpf/progs/test_sock_fields.c  | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/test_sock_fields.c +0 -0
index 3e2e3ee51cc9..186fed1deaab 100644
@@ -42,6 +42,11 @@ struct {
 	__type(value, struct bpf_spinlock_cnt);
 } sk_pkt_out_cnt10 SEC(".maps");
 
Quoted 6 lines

tools/testing/selftests/bpf/progs/test_sock_fields.c struct { · line 47

Quoted 3 lines
tools/testing/selftests/bpf/progs/test_sock_fields.c +3 -0
+enum {
+	TCP_SYN_SENT = 2,
+	TCP_LISTEN = 10,

Thanks for the clean up.

A nit. directly use BPF_TCP_SYN_SENT and BPF_TCP_LISTEN.

Thanks. Completely forgot about those.

#5 lore raw

PATCH v2 3/3 selftests/bpf: Fix test for 4-byte load from dst_port on big-endian

In reply to Jakub Sitnicki

The check for 4-byte load from dst_port offset into bpf_sock is failing on big-endian architecture - s390. The bpf access converter rewrites the 4-byte load to a 2-byte load from sock_common at skc_dport offset, as shown below.

* s390 / llvm-objdump -S --no-show-raw-insn

00000000000002a0 <sk_dst_port__load_word>:

        84:       r1 = *(u32 *)(r1 + 48)
        85:       w0 = 1
        86:       if w1 == 51966 goto +1 <LBB5_2>
        87:       w0 = 0

00000000000002c0 <LBB5_2>:
88: exit

* s390 / bpftool prog dump xlated

_Bool sk_dst_port__load_word(struct bpf_sock * sk):

    35: (69) r1 = *(u16 *)(r1 +12)
    36: (bc) w1 = w1
    37: (b4) w0 = 1
    38: (16) if w1 == 0xcafe goto pc+1
    39: (b4) w0 = 0
    40: (95) exit

* s390 / llvm-objdump -S --no-show-raw-insn

00000000000002a0 <sk_dst_port__load_word>:

        84:       r1 = *(u32 *)(r1 + 48)
        85:       w0 = 1
        86:       if w1 == 65226 goto +1 <LBB5_2>
        87:       w0 = 0

00000000000002c0 <LBB5_2>:
88: exit

* x86_64 / bpftool prog dump xlated

_Bool sk_dst_port__load_word(struct bpf_sock * sk):

    33: (69) r1 = *(u16 *)(r1 +12)
    34: (b4) w0 = 1
    35: (16) if w1 == 0xfeca goto pc+1
    36: (b4) w0 = 0
    37: (95) exit

This leads to surprisings results. On big-endian platforms, the loaded value is as expected. The user observes no difference between a 4-byte load and 2-byte load. However, on little-endian platforms, the access conversion is not what would be expected, that is the result is left shifted after converting the value to the native byte order.

That said, 4-byte loads in BPF from sk->dst_port are not a use case we expect to see, now that the dst_port field is clearly declared as a u16.

Account for the quirky behavior of the access converter in the test case, so that the check passes on both endian variants.

Fixes 8f50f16ff39d ("selftests/bpf: Extend verifier and bpf_sock tests for dst_port loads")
---
 .../selftests/bpf/progs/test_sock_fields.c        | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/progs/test_sock_fields.c +14 -1
index 186fed1deaab..3dddc173070c 100644
@@ -256,10 +256,23 @@ int ingress_read_sock_fields(struct __sk_buff *skb)
 	return CG_OK;
 }
 
+/*
+ * NOTE: 4-byte load from bpf_sock at dst_port offset is quirky. The
+ * result is left shifted on little-endian architectures because the
+ * access is converted to a 2-byte load. The quirky behavior is kept
+ * for backward compatibility.
+ */
 static __noinline bool sk_dst_port__load_word(struct bpf_sock *sk)
 {
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+	const __u8 SHIFT = 16;
+#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+	const __u8 SHIFT = 0;
+#else
+#error "Unrecognized __BYTE_ORDER__"
+#endif
 	__u32 *word = (__u32 *)&sk->dst_port;
-	return word[0] == bpf_htonl(0xcafe0000);
+	return word[0] == bpf_htonl(0xcafe << SHIFT);
 }
 
 static __noinline bool sk_dst_port__load_half(struct bpf_sock *sk)
Signature · 1 line
2.35.1
#6 lore raw

Re: [PATCH bpf-next v2 3/3] selftests/bpf: Fix test for 4-byte load from dst_port on big-endian

Quoted 18 lines · Jakub Sitnicki

The check for 4-byte load from dst_port offset into bpf_sock is failing on big-endian architecture - s390. The bpf access converter rewrites the 4-byte load to a 2-byte load from sock_common at skc_dport offset, as shown below.

* s390 / llvm-objdump -S --no-show-raw-insn

00000000000002a0 <sk_dst_port__load_word>:

        84:       r1 = *(u32 *)(r1 + 48)
        85:       w0 = 1
        86:       if w1 == 51966 goto +1 <LBB5_2>
        87:       w0 = 0

00000000000002c0 <LBB5_2>:
88: exit

* s390 / bpftool prog dump xlated

_Bool sk_dst_port__load_word(struct bpf_sock * sk):

Quoted 8 lines
    35: (69) r1 = *(u16 *)(r1 +12)
    36: (bc) w1 = w1
    37: (b4) w0 = 1
    38: (16) if w1 == 0xcafe goto pc+1
    39: (b4) w0 = 0
    40: (95) exit

* s390 / llvm-objdump -S --no-show-raw-insn

x86_64

Quoted 52 lines

00000000000002a0 <sk_dst_port__load_word>:

        84:       r1 = *(u32 *)(r1 + 48)
        85:       w0 = 1
        86:       if w1 == 65226 goto +1 <LBB5_2>
        87:       w0 = 0

00000000000002c0 <LBB5_2>:
88: exit

* x86_64 / bpftool prog dump xlated

_Bool sk_dst_port__load_word(struct bpf_sock * sk):

    33: (69) r1 = *(u16 *)(r1 +12)
    34: (b4) w0 = 1
    35: (16) if w1 == 0xfeca goto pc+1
    36: (b4) w0 = 0
    37: (95) exit

This leads to surprisings results. On big-endian platforms, the loaded value is as expected. The user observes no difference between a 4-byte load and 2-byte load. However, on little-endian platforms, the access conversion is not what would be expected, that is the result is left shifted after converting the value to the native byte order.

That said, 4-byte loads in BPF from sk->dst_port are not a use case we expect to see, now that the dst_port field is clearly declared as a u16.

Account for the quirky behavior of the access converter in the test case, so that the check passes on both endian variants.

Fixes 8f50f16ff39d ("selftests/bpf: Extend verifier and bpf_sock tests for dst_port loads")
---
 .../selftests/bpf/progs/test_sock_fields.c        | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/progs/test_sock_fields.c +6 -0
index 186fed1deaab..3dddc173070c 100644
@@ -256,10 +256,23 @@ int ingress_read_sock_fields(struct __sk_buff *skb)
 	return CG_OK;
 }
 
+/*
+ * NOTE: 4-byte load from bpf_sock at dst_port offset is quirky. The
+ * result is left shifted on little-endian architectures because the
+ * access is converted to a 2-byte load. The quirky behavior is kept
+ * for backward compatibility.
+ */
 static __noinline bool sk_dst_port__load_word(struct bpf_sock *sk)
 {

tools/testing/selftests/bpf/progs/test_sock_fields.c int ingress_read_sock_fields(struct __sk_buff *skb) · line 275

Quoted 10 lines
tools/testing/selftests/bpf/progs/test_sock_fields.c +8 -1
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+	const __u8 SHIFT = 16;
+#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+	const __u8 SHIFT = 0;
+#else
+#error "Unrecognized __BYTE_ORDER__"
+#endif
 	__u32 *word = (__u32 *)&sk->dst_port;
-	return word[0] == bpf_htonl(0xcafe0000);
+	return word[0] == bpf_htonl(0xcafe << SHIFT);

I believe it should be fine. It is the behavior even before
commit 4421a582718a ("bpf: Make dst_port field in struct bpf_sock 16-bit wide") ?

btw, is it the same as testing "return word[0] == bpf_hton's'(0xcafe);"

Quoted 6 lines
tools/testing/selftests/bpf/progs/test_sock_fields.c +0 -0
 }
 
 static __noinline bool sk_dst_port__load_half(struct bpf_sock *sk)
Signature · 1 line
2.35.1
#7 lore raw

Re: [PATCH bpf-next v2 3/3] selftests/bpf: Fix test for 4-byte load from dst_port on big-endian

Quoted 1 line · Martin KaFai Lau

On Sun, Feb 27, 2022 at 09:27:57PM +0100, Jakub Sitnicki wrote:

[...]

Quoted 18 lines
Quoted 18 lines
tools/testing/selftests/bpf/progs/test_sock_fields.c +10 -0
@@ -256,10 +256,23 @@ int ingress_read_sock_fields(struct __sk_buff *skb)
 	return CG_OK;
 }
 
+/*
+ * NOTE: 4-byte load from bpf_sock at dst_port offset is quirky. The
+ * result is left shifted on little-endian architectures because the
+ * access is converted to a 2-byte load. The quirky behavior is kept
+ * for backward compatibility.
+ */
 static __noinline bool sk_dst_port__load_word(struct bpf_sock *sk)
 {
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+	const __u8 SHIFT = 16;
+#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+	const __u8 SHIFT = 0;
Quoted 8 lines

tools/testing/selftests/bpf/progs/test_sock_fields.c int ingress_read_sock_fields(struct __sk_buff *skb) · line 275

Quoted 6 lines
tools/testing/selftests/bpf/progs/test_sock_fields.c +4 -1
+#else
+#error "Unrecognized __BYTE_ORDER__"
+#endif
 	__u32 *word = (__u32 *)&sk->dst_port;
-	return word[0] == bpf_htonl(0xcafe0000);
+	return word[0] == bpf_htonl(0xcafe << SHIFT);

I believe it should be fine. It is the behavior even before
commit 4421a582718a ("bpf: Make dst_port field in struct bpf_sock 16-bit wide") ?

Yes, exactly. AFAICT there was no change in behavior in commit 4421a582718a, that is:

1. 4-byte load behaves like it did, in its quirky way,
2. 2-byte load at offset dst_port works the same
3. 2-byte load at offset dst_port+2 continues to be rejected.

Quoted 1 line

btw, is it the same as testing "return word[0] == bpf_hton's'(0xcafe);"

Right. Clever observation. I got the impression from the original problem report [1] that the users were failing when trying to do:

bpf_htonl(sk->dst_port) == 0xcafe

Hence I the bpf_htonl() use here.

But perhaps it's better to promote this cleaner pattern in tests.

I will respin it once we hash out the details of what the access should look like on big-endian with Ilya.

[1] https://lore.kernel.org/bpf/[email protected]/

[...]