Skip to contentSkip to conversation

tcp: fix skb_copy_ubufs() vs BIG TCP

PATCH Applied Eric Dumazet 3 participants 3 messages Apr 28, 2023 – Apr 30, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 tcp: fix skb_copy_ubufs() vs BIG TCP

David Ahern reported crashes in skb_copy_ubufs() caused by TCP tx zerocopy using hugepages, and skb length bigger than ~68 KB.

skb_copy_ubufs() assumed it could copy all payload using up to MAX_SKB_FRAGS order-0 pages.

This assumption broke when BIG TCP was able to put up to 512 KB per skb.

We did not hit this bug at Google because we use CONFIG_MAX_SKB_FRAGS=45 and limit gso_max_size to 180000.

A solution is to use higher order pages if needed.

v2: add missing __GFP_COMP, or we leak memory.

Fixes 7c4e983c4f3c ("net: allow gso_max_size to exceed 65536")
Cc Xin Long <[email protected]>
Cc Willem de Bruijn <[email protected]>
Cc Coco Li <[email protected]>
---
 net/core/skbuff.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
net/core/skbuff.c +14 -6
index 2112146092bfe24061b24b9e4684bcc031a045b9..26a586007d8b1ae39ab7a09eecf8575e04dadfeb 100644
@@ -1758,7 +1758,7 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask)
 {
 	int num_frags = skb_shinfo(skb)->nr_frags;
 	struct page *page, *head = NULL;
-	int i, new_frags;
+	int i, order, psize, new_frags;
 	u32 d_off;
 
 	if (skb_shared(skb) || skb_unclone(skb, gfp_mask))
@@ -1767,9 +1767,17 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask)
 	if (!num_frags)
 		goto release;
 
-	new_frags = (__skb_pagelen(skb) + PAGE_SIZE - 1) >> PAGE_SHIFT;
+	/* We might have to allocate high order pages, so compute what minimum
+	 * page order is needed.
+	 */
+	order = 0;
+	while ((PAGE_SIZE << order) * MAX_SKB_FRAGS < __skb_pagelen(skb))
+		order++;
+	psize = (PAGE_SIZE << order);
+
+	new_frags = (__skb_pagelen(skb) + psize - 1) >> (PAGE_SHIFT + order);
 	for (i = 0; i < new_frags; i++) {
-		page = alloc_page(gfp_mask);
+		page = alloc_pages(gfp_mask | __GFP_COMP, order);
 		if (!page) {
 			while (head) {
 				struct page *next = (struct page *)page_private(head);
@@ -1796,11 +1804,11 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask)
 			vaddr = kmap_atomic(p);
 
 			while (done < p_len) {
-				if (d_off == PAGE_SIZE) {
+				if (d_off == psize) {
 					d_off = 0;
 					page = (struct page *)page_private(page);
 				}
-				copy = min_t(u32, PAGE_SIZE - d_off, p_len - done);
+				copy = min_t(u32, psize - d_off, p_len - done);
 				memcpy(page_address(page) + d_off,
 				       vaddr + p_off + done, copy);
 				done += copy;
@@ -1816,7 +1824,7 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask)
 
 	/* skb frags point to kernel buffers */
 	for (i = 0; i < new_frags - 1; i++) {
-		__skb_fill_page_desc(skb, i, head, 0, PAGE_SIZE);
+		__skb_fill_page_desc(skb, i, head, 0, psize);
 		head = (struct page *)page_private(head);
 	}
 	__skb_fill_page_desc(skb, new_frags - 1, head, 0, d_off);
Signature · 1 line
2.40.1.495.gc816e09b53d-goog
#1 lore raw
This patch was applied to netdev/net.git (main) by David S. Miller <[email protected]>:

Hello:

This patch was applied to netdev/net.git (main)
by David S. Miller <[email protected]>:

Quoted 9 lines · On Fri, 28 Apr 2023 04:32:31 +0000 you wrote:

David Ahern reported crashes in skb_copy_ubufs() caused by TCP tx zerocopy using hugepages, and skb length bigger than ~68 KB.

skb_copy_ubufs() assumed it could copy all payload using up to MAX_SKB_FRAGS order-0 pages.

This assumption broke when BIG TCP was able to put up to 512 KB per skb.

[...]

Here is the summary with links:
- [v2,net] tcp: fix skb_copy_ubufs() vs BIG TCP

    https://git.kernel.org/netdev/net/c/7e692df39336

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
#2 lore raw
In reply to Eric Dumazet

From: Eric Dumazet

Quoted 1 line

Sent: 28 April 2023 05:33

...

net/core/skbuff.c int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask) · line 1778

Quoted 10 lines
net/core/skbuff.c +9 -1
-	new_frags = (__skb_pagelen(skb) + PAGE_SIZE - 1) >> PAGE_SHIFT;
+	/* We might have to allocate high order pages, so compute what minimum
+	 * page order is needed.
+	 */
+	order = 0;
+	while ((PAGE_SIZE << order) * MAX_SKB_FRAGS < __skb_pagelen(skb))
+		order++;
+	psize = (PAGE_SIZE << order);
+
+	new_frags = (__skb_pagelen(skb) + psize - 1) >> (PAGE_SHIFT + order);

That looks like it will generate quite horrid code. Perhaps something like:

	new_frags = (__skb_pagelen(skb) + PAGE_SIZE - 1) >> PAGE_SHIFT;
	order = 0;
	psize = PAGE_SIZE;
	if (new_frags > MAX_SKB_FRAGS) {
		/* Allocate high order pages to reduce the number of frags. */
		order = ilog2(DIV_ROUNDUP(new_frags, MAX_SKB_FRAGS) - 1) + 1;
		psize <<= order;
	}

There might be an 'off-by-one' in there somewhere though...

David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)