Skip to contentSkip to conversation

bpf, sockmap complete fixes for avail bytes

PATCH Superseded John Fastabend 2 participants 5 messages Sep 25, 2023 – Sep 26, 2023

Subsystems (7)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

John Fastabend

author maintainer
#0 lore raw

v2 cover bpf, sockmap complete fixes for avail bytes

With e5c6de5fa0258 ("bpf, sockmap: Incorrectly handling copied_seq") we started fixing the available bytes accounting by moving copied_seq to where the user actually reads the bytes.

However we missed handling MSG_PEEK correctly and we need to ensure that we don't kfree_skb() a skb off the receive_queue when the copied_seq number is not incremented by user reads for some time.

v2: drop seq var in tcp_read_skb its no longer necessary per Jakub's
suggestion

John Fastabend (3):
bpf: tcp_read_skb needs to pop skb regardless of seq
bpf: sockmap, do not inc copied_seq when PEEK flag set
bpf: sockmap, add tests for MSG_F_PEEK

 net/ipv4/tcp.c                                |  3 +-
 net/ipv4/tcp_bpf.c                            |  4 +-
 .../selftests/bpf/prog_tests/sockmap_basic.c  | 52 +++++++++++++++++++
 3 files changed, 56 insertions(+), 3 deletions(-)
Signature · 1 line
2.33.0
author maintainer
#1 lore raw

PATCH v2 1/3 bpf: tcp_read_skb needs to pop skb regardless of seq

Before fix e5c6de5fa0258 tcp_read_skb() would increment the tp->copied-seq value. This (as described in the commit) would cause an error for apps because once that is incremented the application might believe there is no data to be read. Then some apps would stall or abort believing no data is available.

However, the fix is incomplete because it introduces another issue in the skb dequeue. The loop does tcp_recv_skb() in a while loop to consume as many skbs as possible. The problem is the call is,

tcp_recv_skb(sk, seq, &offset)

Where 'seq' is

  u32 seq = tp->copied_seq;

Now we can hit a case where we've yet incremented copied_seq from BPF side, but then tcp_recv_skb() fails this test,

if (offset < skb->len || (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN))

so that instead of returning the skb we call tcp_eat_recv_skb() which frees the skb. This is because the routine believes the SKB has been collapsed per comment,

/* This looks weird, but this can happen if TCP collapsing
* splitted a fat GRO packet, while we released socket lock
* in skb_splice_bits()
*/

This can't happen here we've unlinked the full SKB and orphaned it. Anyways it would confuse any BPF programs if the data were suddenly moved underneath it.

To fix this situation do simpler operation and just skb_peek() the data of the queue followed by the unlink. It shouldn't need to check this condition and tcp_read_skb() reads entire skbs so there is no need to handle the 'offset!=0' case as we would see in tcp_read_sock().

Fixes e5c6de5fa0258 ("bpf, sockmap: Incorrectly handling copied_seq")
Fixes 04919bed948dc ("tcp: Introduce tcp_read_skb()")
---
 net/ipv4/tcp.c | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)
net/ipv4/tcp.c +2 -7
index 0c3040a63ebd..235d77af3177 100644
@@ -1622,15 +1622,13 @@ EXPORT_SYMBOL(tcp_read_sock);
 int tcp_read_skb(struct sock *sk, skb_read_actor_t recv_actor)
 {
 	struct tcp_sock *tp = tcp_sk(sk);
-	u32 seq = tp->copied_seq;
 	struct sk_buff *skb;
 	int copied = 0;
-	u32 offset;
 
 	if (sk->sk_state == TCP_LISTEN)
 		return -ENOTCONN;
 
-	while ((skb = tcp_recv_skb(sk, seq, &offset)) != NULL) {
+	while ((skb = skb_peek(&sk->sk_receive_queue)) != NULL) {
 		u8 tcp_flags;
 		int used;
 
@@ -1643,13 +1641,10 @@ int tcp_read_skb(struct sock *sk, skb_read_actor_t recv_actor)
 				copied = used;
 			break;
 		}
-		seq += used;
 		copied += used;
 
-		if (tcp_flags & TCPHDR_FIN) {
-			++seq;
+		if (tcp_flags & TCPHDR_FIN)
 			break;
-		}
 	}
 	return copied;
 }
Signature · 1 line
2.33.0
#2 lore raw

Re: [PATCH bpf v2 1/3] bpf: tcp_read_skb needs to pop skb regardless of seq

kernel test robot noticed the following build warnings:

Hi John,

kernel test robot noticed the following build warnings:

[auto build test WARNING on bpf/master]

url:    https://github.com/intel-lab-lkp/linux/commits/John-Fastabend/bpf-tcp_read_skb-needs-to-pop-skb-regardless-of-seq/20230926-042625
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git master
patch link:    https://lore.kernel.org/r/20230925202448.100920-2-john.fastabend%40gmail.com
patch subject: [PATCH bpf v2 1/3] bpf: tcp_read_skb needs to pop skb regardless of seq
config: m68k-allyesconfig (https://download.01.org/0day-ci/archive/20230926/[email protected]/config)
compiler: m68k-linux-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20230926/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

net/ipv4/tcp.c: In function 'tcp_read_skb':

>> net/ipv4/tcp.c:1624:26: warning: unused variable 'tp' [-Wunused-variable]
    1624 |         struct tcp_sock *tp = tcp_sk(sk);
         |                          ^~

vim +/tp +1624 net/ipv4/tcp.c

^1da177e4c3f41 Linus Torvalds 2005-04-16  1621  
965b57b469a589 Cong Wang      2022-06-15  1622  int tcp_read_skb(struct sock *sk, skb_read_actor_t recv_actor)
04919bed948dc2 Cong Wang      2022-06-15  1623  {
04919bed948dc2 Cong Wang      2022-06-15 @1624  	struct tcp_sock *tp = tcp_sk(sk);
04919bed948dc2 Cong Wang      2022-06-15  1625  	struct sk_buff *skb;
04919bed948dc2 Cong Wang      2022-06-15  1626  	int copied = 0;
04919bed948dc2 Cong Wang      2022-06-15  1627  
04919bed948dc2 Cong Wang      2022-06-15  1628  	if (sk->sk_state == TCP_LISTEN)
04919bed948dc2 Cong Wang      2022-06-15  1629  		return -ENOTCONN;
04919bed948dc2 Cong Wang      2022-06-15  1630  
44bb37a8112f62 John Fastabend 2023-09-25  1631  	while ((skb = skb_peek(&sk->sk_receive_queue)) != NULL) {
db4192a754ebd5 Cong Wang      2022-09-12  1632  		u8 tcp_flags;
db4192a754ebd5 Cong Wang      2022-09-12  1633  		int used;
04919bed948dc2 Cong Wang      2022-06-15  1634  
04919bed948dc2 Cong Wang      2022-06-15  1635  		__skb_unlink(skb, &sk->sk_receive_queue);
96628951869c0d Peilin Ye      2022-09-08  1636  		WARN_ON_ONCE(!skb_set_owner_sk_safe(skb, sk));
db4192a754ebd5 Cong Wang      2022-09-12  1637  		tcp_flags = TCP_SKB_CB(skb)->tcp_flags;
db4192a754ebd5 Cong Wang      2022-09-12  1638  		used = recv_actor(sk, skb);
db4192a754ebd5 Cong Wang      2022-09-12  1639  		if (used < 0) {
db4192a754ebd5 Cong Wang      2022-09-12  1640  			if (!copied)
db4192a754ebd5 Cong Wang      2022-09-12  1641  				copied = used;
db4192a754ebd5 Cong Wang      2022-09-12  1642  			break;
db4192a754ebd5 Cong Wang      2022-09-12  1643  		}
db4192a754ebd5 Cong Wang      2022-09-12  1644  		copied += used;
db4192a754ebd5 Cong Wang      2022-09-12  1645  
44bb37a8112f62 John Fastabend 2023-09-25  1646  		if (tcp_flags & TCPHDR_FIN)
db4192a754ebd5 Cong Wang      2022-09-12  1647  			break;
db4192a754ebd5 Cong Wang      2022-09-12  1648  	}
04919bed948dc2 Cong Wang      2022-06-15  1649  	return copied;
04919bed948dc2 Cong Wang      2022-06-15  1650  }
04919bed948dc2 Cong Wang      2022-06-15  1651  EXPORT_SYMBOL(tcp_read_skb);
04919bed948dc2 Cong Wang      2022-06-15  1652  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author maintainer
#3 lore raw

PATCH v2 2/3 bpf: sockmap, do not inc copied_seq when PEEK flag set

In reply to John Fastabend

When data is peek'd off the receive queue we shouldn't considered it copied from tcp_sock side. When we increment copied_seq this will confuse tcp_data_ready() because copied_seq can be arbitrarily increased. From] application side it results in poll() operations not waking up when expected.

Notice tcp stack without BPF recvmsg programs also does not increment copied_seq.

We broke this when we moved copied_seq into recvmsg to only update when actual copy was happening. But, it wasn't working correctly either before because the tcp_data_ready() tried to use the copied_seq value to see if data was read by user yet. See fixes tags.

Fixes e5c6de5fa0258 ("bpf, sockmap: Incorrectly handling copied_seq")
Fixes 04919bed948dc ("tcp: Introduce tcp_read_skb()")
---
 net/ipv4/tcp_bpf.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
net/ipv4/tcp_bpf.c +3 -1
index 81f0dff69e0b..327268203001 100644
@@ -222,6 +222,7 @@ static int tcp_bpf_recvmsg_parser(struct sock *sk,
 				  int *addr_len)
 {
 	struct tcp_sock *tcp = tcp_sk(sk);
+	int peek = flags & MSG_PEEK;
 	u32 seq = tcp->copied_seq;
 	struct sk_psock *psock;
 	int copied = 0;
@@ -311,7 +312,8 @@ static int tcp_bpf_recvmsg_parser(struct sock *sk,
 		copied = -EAGAIN;
 	}
 out:
-	WRITE_ONCE(tcp->copied_seq, seq);
+	if (!peek)
+		WRITE_ONCE(tcp->copied_seq, seq);
 	tcp_rcv_space_adjust(sk);
 	if (copied > 0)
 		__tcp_cleanup_rbuf(sk, copied);
Signature · 1 line
2.33.0
author maintainer
#4 lore raw

PATCH v2 3/3 bpf: sockmap, add tests for MSG_F_PEEK

In reply to John Fastabend

Test that we can read with MSG_F_PEEK and then still get correct number of available bytes through FIONREAD. The recv() (without PEEK) then returns the bytes as expected. The recv() always worked though because it was just the available byte reporting that was broke before latest fixes.

---
 .../selftests/bpf/prog_tests/sockmap_basic.c  | 52 +++++++++++++++++++
 1 file changed, 52 insertions(+)
tools/testing/selftests/bpf/prog_tests/sockmap_basic.c +52 -0
index 064cc5e8d9ad..e8eee2b8901e 100644
@@ -475,6 +475,56 @@ static void test_sockmap_skb_verdict_fionread(bool pass_prog)
 		test_sockmap_drop_prog__destroy(drop);
 }
 
+
+static void test_sockmap_skb_verdict_peek(void)
+{
+	int err, map, verdict, s, c1, p1, zero = 0, sent, recvd, avail;
+	struct test_sockmap_pass_prog *pass;
+	char snd[256] = "0123456789";
+	char rcv[256] = "0";
+
+	pass = test_sockmap_pass_prog__open_and_load();
+	if (!ASSERT_OK_PTR(pass, "open_and_load"))
+		return;
+	verdict = bpf_program__fd(pass->progs.prog_skb_verdict);
+	map = bpf_map__fd(pass->maps.sock_map_rx);
+
+	err = bpf_prog_attach(verdict, map, BPF_SK_SKB_STREAM_VERDICT, 0);
+	if (!ASSERT_OK(err, "bpf_prog_attach"))
+		goto out;
+
+	s = socket_loopback(AF_INET, SOCK_STREAM);
+	if (!ASSERT_GT(s, -1, "socket_loopback(s)"))
+		goto out;
+
+	err = create_pair(s, AF_INET, SOCK_STREAM, &c1, &p1);
+	if (!ASSERT_OK(err, "create_pairs(s)"))
+		goto out;
+
+	err = bpf_map_update_elem(map, &zero, &c1, BPF_NOEXIST);
+	if (!ASSERT_OK(err, "bpf_map_update_elem(c1)"))
+		goto out_close;
+
+	sent = xsend(p1, snd, sizeof(snd), 0);
+	ASSERT_EQ(sent, sizeof(snd), "xsend(p1)");
+	recvd = recv(c1, rcv, sizeof(rcv), MSG_PEEK);
+	ASSERT_EQ(recvd, sizeof(rcv), "recv(c1)");
+	err = ioctl(c1, FIONREAD, &avail);
+	ASSERT_OK(err, "ioctl(FIONREAD) error");
+	ASSERT_EQ(avail, sizeof(snd), "after peek ioctl(FIONREAD)");
+	recvd = recv(c1, rcv, sizeof(rcv), 0);
+	ASSERT_EQ(recvd, sizeof(rcv), "recv(p0)");
+	err = ioctl(c1, FIONREAD, &avail);
+	ASSERT_OK(err, "ioctl(FIONREAD) error");
+	ASSERT_EQ(avail, 0, "after read ioctl(FIONREAD)");
+
+out_close:
+	close(c1);
+	close(p1);
+out:
+	test_sockmap_pass_prog__destroy(pass);
+}
+
 void test_sockmap_basic(void)
 {
 	if (test__start_subtest("sockmap create_update_free"))
@@ -515,4 +565,6 @@ void test_sockmap_basic(void)
 		test_sockmap_skb_verdict_fionread(true);
 	if (test__start_subtest("sockmap skb_verdict fionread on drop"))
 		test_sockmap_skb_verdict_fionread(false);
+	if (test__start_subtest("sockmap skb_verdict msg_f_peek"))
+		test_sockmap_skb_verdict_peek();
 }
Signature · 1 line
2.33.0