Skip to contentSkip to conversation

net: gso: fix MTU validation of BIG TCP

PATCH In review Mariusz Klimek 5 participants 12 messages Jan 6 – Jan 30

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v2 cover net: gso: fix MTU validation of BIG TCP

This series fixes the MTU validation of BIG TCP jumbograms and removes the existing IP6SKB_FAKEJUMBO work-around that only fixes the issue in one location.

For GSO packets, the length that matters for MTU validation is the segment length, not the total length of the packet. skb_gso_network_seglen is used by skb_gso_validate_network_len to calculate the segment length including the network and transport headers and to then verify that the segment length is below the MTU.

skb_gso_network_seglen assumes that the headers of the segments are identical to those of the unsegmented packet, but that assumption is incorrect for BIG TCP jumbograms which have an added HBH header that is removed upon segmentation. The calculated segment length ends up being 8 bytes more than the actual segment length.

The actual segment length is set according to the MSS, so the segment length calculated by skb_gso_network_seglen is greater than the MTU, causing the skb_gso_validate_network_len check to fail despite the fact that the actual segment length is lower than the MTU.

There is currently a work-around that fixes this bug in some cases:
ip6_xmit sets the IP6SKB_FAKEJUMBO flag for BIG TCP jumbograms, which causes the MTU validation in ip6_finish_output_gso to be skipped (intentionally). However, this work-around doesn't apply to MTU validations performed in other places such as in ip6_forward. BIG TCP jumbograms don't pass the MTU validation when forwarded locally and are therefore dropped, unless the MTU of the originating interface is lower than the MTUs of the rest of the interfaces the packets are forwarded through.

v2:
fix jumbogram check in skb_gso_network_seglen add jumbogram check to skb_gso_mac_seglen as well

v1:

Mariusz Klimek (3):
net: gso: do not include jumbogram HBH header in seglen calculation
ipv6: remove IP6SKB_FAKEJUMBO flag
selftests/net: remove unnecessary MTU config in big_tcp.sh

 include/linux/ipv6.h                   |  1 -
 net/core/gso.c                         | 14 +++++++++-----
 net/ipv6/ip6_output.c                  |  4 +---
 tools/testing/selftests/net/big_tcp.sh |  1 -
 4 files changed, 10 insertions(+), 10 deletions(-)
Signature · 1 line
2.47.3
#1 lore raw

PATCH v2 1/3 net: gso: do not include jumbogram HBH header in seglen calculation

This patch fixes an issue in skb_gso_network_seglen and similar functions where the calculated segment length includes the HBH headers of BIG TCP jumbograms despite these headers being removed before segmentation. These headers are added by GRO or by ip6_xmit for BIG TCP packets and are later removed by GSO. This bug causes MTU validation of BIG TCP jumbograms to fail.

---
 net/core/gso.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
net/core/gso.c +9 -5
index bcd156372f4d..deacd32f644d 100644
@@ -1,4 +1,5 @@
 // SPDX-License-Identifier: GPL-2.0-or-later
+#include <linux/if_vlan.h>
 #include <linux/skbuff.h>
 #include <linux/sctp.h>
 #include <net/gso.h>
@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)
  */
 static unsigned int skb_gso_network_seglen(const struct sk_buff *skb)
 {
-	unsigned int hdr_len = skb_transport_header(skb) -
-			       skb_network_header(skb);
+	unsigned int off = skb_network_offset(skb) + sizeof(struct ipv6hdr);
+	unsigned int hdr_len = skb_network_header_len(skb);
+
+	/* Jumbogram HBH header is removed upon segmentation. */
+	if (skb_protocol(skb, true) == htons(ETH_P_IPV6) &&
+	    skb->len - off > IPV6_MAXPLEN)
+		hdr_len -= sizeof(struct hop_jumbo_hdr);
 
 	return hdr_len + skb_gso_transport_seglen(skb);
 }
@@ -194,9 +200,7 @@ static unsigned int skb_gso_network_seglen(const struct sk_buff *skb)
  */
 static unsigned int skb_gso_mac_seglen(const struct sk_buff *skb)
 {
-	unsigned int hdr_len = skb_transport_header(skb) - skb_mac_header(skb);
-
-	return hdr_len + skb_gso_transport_seglen(skb);
+	return skb_mac_header_len(skb) + skb_gso_network_seglen(skb);
 }
 
 /**
Signature · 1 line
2.47.3
maintainer
#2 lore raw

Re: [PATCH net-next v2 1/3] net: gso: do not include jumbogram HBH header in seglen calculation

net/core/gso.c static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb) · line 187

Quoted 13 lines · Mariusz Klimek
net/core/gso.c +7 -2
@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)
  */
 static unsigned int skb_gso_network_seglen(const struct sk_buff *skb)
 {
-	unsigned int hdr_len = skb_transport_header(skb) -
-			       skb_network_header(skb);
+	unsigned int off = skb_network_offset(skb) + sizeof(struct ipv6hdr);
+	unsigned int hdr_len = skb_network_header_len(skb);
+
+	/* Jumbogram HBH header is removed upon segmentation. */
+	if (skb_protocol(skb, true) == htons(ETH_P_IPV6) &&
+	    skb->len - off > IPV6_MAXPLEN)
+		hdr_len -= sizeof(struct hop_jumbo_hdr);

IIRC there is some ongoing discussion about introducing big tcp support for virtio. Perhaps a DEBUG_NET_WARN_ON_ONCE(SKB_GSO_DODGY) could help keeping this check updated at due time?

@Jason: could you please double check if I'm off WRT virtio support for big TCP?

/P

maintainer
#3 lore raw

Re: [PATCH net-next v2 1/3] net: gso: do not include jumbogram HBH header in seglen calculation

Quoted 19 lines · Paolo Abeni

net/core/gso.c static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb) · line 187

Quoted 13 lines · Mariusz Klimek
net/core/gso.c +7 -2
@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)
  */
 static unsigned int skb_gso_network_seglen(const struct sk_buff *skb)
 {
-	unsigned int hdr_len = skb_transport_header(skb) -
-			       skb_network_header(skb);
+	unsigned int off = skb_network_offset(skb) + sizeof(struct ipv6hdr);
+	unsigned int hdr_len = skb_network_header_len(skb);
+
+	/* Jumbogram HBH header is removed upon segmentation. */
+	if (skb_protocol(skb, true) == htons(ETH_P_IPV6) &&
+	    skb->len - off > IPV6_MAXPLEN)
+		hdr_len -= sizeof(struct hop_jumbo_hdr);

IIRC there is some ongoing discussion about introducing big tcp support for virtio. Perhaps a DEBUG_NET_WARN_ON_ONCE(SKB_GSO_DODGY) could help keeping this check updated at due time?

Quoted 2 lines

@Jason: could you please double check if I'm off WRT virtio support for big TCP?

Reconsidering the above, I think the mentioned check could be added separately, if and when virtio big TCP will come to life.

/P

#4 lore raw

Re: [PATCH net-next v2 1/3] net: gso: do not include jumbogram HBH header in seglen calculation

Quoted 21 lines · Paolo Abeni
Quoted 19 lines · Paolo Abeni

net/core/gso.c static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb) · line 187

Quoted 13 lines · Mariusz Klimek
net/core/gso.c +7 -2
@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)
  */
 static unsigned int skb_gso_network_seglen(const struct sk_buff *skb)
 {
-    unsigned int hdr_len = skb_transport_header(skb) -
-                           skb_network_header(skb);
+    unsigned int off = skb_network_offset(skb) + sizeof(struct ipv6hdr);
+    unsigned int hdr_len = skb_network_header_len(skb);
+
+    /* Jumbogram HBH header is removed upon segmentation. */
+    if (skb_protocol(skb, true) == htons(ETH_P_IPV6) &&
+        skb->len - off > IPV6_MAXPLEN)
+            hdr_len -= sizeof(struct hop_jumbo_hdr);

IIRC there is some ongoing discussion about introducing big tcp support for virtio. Perhaps a DEBUG_NET_WARN_ON_ONCE(SKB_GSO_DODGY) could help keeping this check updated at due time?

Quoted 5 lines
Quoted 2 lines

@Jason: could you please double check if I'm off WRT virtio support for big TCP?

Reconsidering the above, I think the mentioned check could be added separately, if and when virtio big TCP will come to life.

I agree. BIG TCP support for virtio requires spec features like tso_max_size/segs advertisement, jumbogram head flags. And if I am not wrong, HBH header is probably not needed at that time.

Thanks

Quoted 3 lines

/P

maintainer
#5 lore raw

Re: [PATCH net-next v2 1/3] net: gso: do not include jumbogram HBH header in seglen calculation

In reply to Mariusz Klimek

net/core/gso.c static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb) · line 187

Quoted 13 lines · Mariusz Klimek
net/core/gso.c +7 -2
@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)
  */
 static unsigned int skb_gso_network_seglen(const struct sk_buff *skb)
 {
-	unsigned int hdr_len = skb_transport_header(skb) -
-			       skb_network_header(skb);
+	unsigned int off = skb_network_offset(skb) + sizeof(struct ipv6hdr);
+	unsigned int hdr_len = skb_network_header_len(skb);
+
+	/* Jumbogram HBH header is removed upon segmentation. */
+	if (skb_protocol(skb, true) == htons(ETH_P_IPV6) &&
+	    skb->len - off > IPV6_MAXPLEN)
+		hdr_len -= sizeof(struct hop_jumbo_hdr);

I'm sorry for splitting the feedback in multiple replies.

I think the concern I expressed on v1:

https://lore.kernel.org/netdev/[email protected]/

is still not addressed here. What I fear is:

- TCP cooks a plain GSO packet just below the 64K limit.
- Such packet goes trough UDP (or gre) encapsulation, the skb->len size (including outer network header) grows above the 64K limit.
- the above check is satisfied, but no jumbo hop option is present.

I think you could use the ipv6_has_hopopt_jumbo() helper to be on the safe side.

/P

#6 lore raw

Re: [PATCH net-next v2 1/3] net: gso: do not include jumbogram HBH header in seglen calculation

Quoted 19 lines · Paolo Abeni

net/core/gso.c static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb) · line 187

Quoted 13 lines · Mariusz Klimek
net/core/gso.c +7 -2
@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)
  */
 static unsigned int skb_gso_network_seglen(const struct sk_buff *skb)
 {
-	unsigned int hdr_len = skb_transport_header(skb) -
-			       skb_network_header(skb);
+	unsigned int off = skb_network_offset(skb) + sizeof(struct ipv6hdr);
+	unsigned int hdr_len = skb_network_header_len(skb);
+
+	/* Jumbogram HBH header is removed upon segmentation. */
+	if (skb_protocol(skb, true) == htons(ETH_P_IPV6) &&
+	    skb->len - off > IPV6_MAXPLEN)
+		hdr_len -= sizeof(struct hop_jumbo_hdr);

I'm sorry for splitting the feedback in multiple replies.

I think the concern I expressed on v1:

Quoted 9 lines

https://lore.kernel.org/netdev/[email protected]/

is still not addressed here. What I fear is:

- TCP cooks a plain GSO packet just below the 64K limit.
- Such packet goes trough UDP (or gre) encapsulation, the skb->len size (including outer network header) grows above the 64K limit.
- the above check is satisfied, but no jumbo hop option is present.

Could you maybe clarify what you mean?

This check is for the outer IPv6 header. skb->len - off is the IPv6 payload length so the packet contains a hop-by-hop header if and only if skb->len - off > IPV6_MAXPLEN. It doesn't matter what comes after the IPv6 header. If the TCP payload is smaller than 65535 and it is the encapsulation headers that cause the IPv6 payload length to be above 65535, a jumbogram hop-by-hop header is still added (in ip6_xmit).

Quoted 2 lines

I think you could use the ipv6_has_hopopt_jumbo() helper to be on the safe side.

Sure, I could submit another revision with this change if my explanation is unsatisfactory.

Quoted 3 lines

/P

Signature · 1 line
Mariusz K.
#7 lore raw

PATCH v2 2/3 ipv6: remove IP6SKB_FAKEJUMBO flag

In reply to Mariusz Klimek

This patch removes the IP6SKB_FAKEJUMBO flag that is used as a work-around to bypass MTU validation of BIG TCP jumbograms due to a bug in skb_gso_network_seglen. This work-around is no longer required now that the bug is fixed.

---
 include/linux/ipv6.h  | 1 -
 net/ipv6/ip6_output.c | 4 +---
 2 files changed, 1 insertion(+), 4 deletions(-)
include/linux/ipv6.h +0 -1
index 7294e4e89b79..9f076171106e 100644
@@ -155,7 +155,6 @@ struct inet6_skb_parm {
 #define IP6SKB_L3SLAVE         64
 #define IP6SKB_JUMBOGRAM      128
 #define IP6SKB_SEG6	      256
-#define IP6SKB_FAKEJUMBO      512
 #define IP6SKB_MULTIPATH      1024
 #define IP6SKB_MCROUTE        2048
 };
net/ipv6/ip6_output.c +1 -3
index f904739e99b9..9af9ec6bdb8c 100644
@@ -179,8 +179,7 @@ ip6_finish_output_gso_slowpath_drop(struct net *net, struct sock *sk,
 static int ip6_finish_output_gso(struct net *net, struct sock *sk,
 				 struct sk_buff *skb, unsigned int mtu)
 {
-	if (!(IP6CB(skb)->flags & IP6SKB_FAKEJUMBO) &&
-	    !skb_gso_validate_network_len(skb, mtu))
+	if (!skb_gso_validate_network_len(skb, mtu))
 		return ip6_finish_output_gso_slowpath_drop(net, sk, skb, mtu);
 
 	return ip6_finish_output2(net, sk, skb);
@@ -323,7 +322,6 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6,
 
 		proto = IPPROTO_HOPOPTS;
 		seg_len = 0;
-		IP6CB(skb)->flags |= IP6SKB_FAKEJUMBO;
 	}
 
 	skb_push(skb, sizeof(struct ipv6hdr));
Signature · 1 line
2.47.3
#8 lore raw

PATCH v2 3/3 selftests/net: remove unnecessary MTU config in big_tcp.sh

In reply to Mariusz Klimek

This patch removes the manual lowering of the client MTU in big_tcp.sh. The MTU lowering was previously required as a work-around due to a bug in the MTU validation of BIG TCP jumbograms. The MTU was lowered to 1442, but note that 1492 (1500 - 8) would of worked just as well. Now that the bug has been fixed, the manual client MTU modification can be removed entirely.

---
 tools/testing/selftests/net/big_tcp.sh | 1 -
 1 file changed, 1 deletion(-)
tools/testing/selftests/net/big_tcp.sh +0 -1
index 2db9d15cd45f..b5d9145296d3 100755
@@ -32,7 +32,6 @@ setup() {
 	ip -net $ROUTER_NS link add link2 type veth peer name link3 netns $SERVER_NS
 
 	ip -net $CLIENT_NS link set link0 up
-	ip -net $CLIENT_NS link set link0 mtu 1442
 	ip -net $CLIENT_NS addr add $CLIENT_IP4/24 dev link0
 	ip -net $CLIENT_NS addr add $CLIENT_IP6/64 dev link0 nodad
 	ip -net $CLIENT_NS route add $SERVER_IP4 dev link0 via $CLIENT_GW4
Signature · 1 line
2.47.3
#9 lore raw
In reply to Mariusz Klimek
Quoted 3 lines · Mariusz Klimek

This series fixes the MTU validation of BIG TCP jumbograms and removes the existing IP6SKB_FAKEJUMBO work-around that only fixes the issue in one location.

My series removes IPv6 HBH in BIG TCP entirely:

https://lore.kernel.org/netdev/[email protected]/T/

I believe, it makes this fix no longer necessary.

Quoted 48 lines

For GSO packets, the length that matters for MTU validation is the segment length, not the total length of the packet. skb_gso_network_seglen is used by skb_gso_validate_network_len to calculate the segment length including the network and transport headers and to then verify that the segment length is below the MTU.

skb_gso_network_seglen assumes that the headers of the segments are identical to those of the unsegmented packet, but that assumption is incorrect for BIG TCP jumbograms which have an added HBH header that is removed upon segmentation. The calculated segment length ends up being 8 bytes more than the actual segment length.

The actual segment length is set according to the MSS, so the segment length calculated by skb_gso_network_seglen is greater than the MTU, causing the skb_gso_validate_network_len check to fail despite the fact that the actual segment length is lower than the MTU.

There is currently a work-around that fixes this bug in some cases:
ip6_xmit sets the IP6SKB_FAKEJUMBO flag for BIG TCP jumbograms, which causes the MTU validation in ip6_finish_output_gso to be skipped (intentionally). However, this work-around doesn't apply to MTU validations performed in other places such as in ip6_forward. BIG TCP jumbograms don't pass the MTU validation when forwarded locally and are therefore dropped, unless the MTU of the originating interface is lower than the MTUs of the rest of the interfaces the packets are forwarded through.

v2:
fix jumbogram check in skb_gso_network_seglen add jumbogram check to skb_gso_mac_seglen as well

v1:

Mariusz Klimek (3):
net: gso: do not include jumbogram HBH header in seglen calculation
ipv6: remove IP6SKB_FAKEJUMBO flag
selftests/net: remove unnecessary MTU config in big_tcp.sh

 include/linux/ipv6.h                   |  1 -
 net/core/gso.c                         | 14 +++++++++-----
 net/ipv6/ip6_output.c                  |  4 +---
 tools/testing/selftests/net/big_tcp.sh |  1 -
 4 files changed, 10 insertions(+), 10 deletions(-)
Signature · 1 line
2.47.3
#10 lore raw
Quoted 11 lines · Alice Mikityanska
Quoted 3 lines · Mariusz Klimek

This series fixes the MTU validation of BIG TCP jumbograms and removes the existing IP6SKB_FAKEJUMBO work-around that only fixes the issue in one location.

My series removes IPv6 HBH in BIG TCP entirely:

https://lore.kernel.org/netdev/[email protected]/T/

I believe, it makes this fix no longer necessary.

You're right, your patch also fixes this issue by removing HBH headers altogether. Thanks for bringing this up :)

Heads up: I'm working on another patch series that uses some of the code you're removing...

Paolo: should we still merge this or should we just wait for Alice's patch to be accepted? Also, should I already submit my next patch series or should I wait until we're finished with this one?

Signature · 1 line
Mariusz K.
#11 lore raw
Quoted 17 lines · Mariusz Klimek
Quoted 11 lines · Alice Mikityanska
Quoted 3 lines · Mariusz Klimek

This series fixes the MTU validation of BIG TCP jumbograms and removes the existing IP6SKB_FAKEJUMBO work-around that only fixes the issue in one location.

My series removes IPv6 HBH in BIG TCP entirely:

https://lore.kernel.org/netdev/[email protected]/T/

I believe, it makes this fix no longer necessary.

You're right, your patch also fixes this issue by removing HBH headers altogether. Thanks for bringing this up :)

Quoted 6 lines

Heads up: I'm working on another patch series that uses some of the code you're removing...

Paolo: should we still merge this or should we just wait for Alice's patch to be accepted? Also, should I already submit my next patch series or should I wait until we're finished with this one?

CCed the tcpdump folks: I would appreciate if you could take forward either of these pull requests:

https://github.com/the-tcpdump-group/tcpdump/pull/1329
https://github.com/the-tcpdump-group/tcpdump/pull/1396

That would unblock the kernel series and allow Paolo to merge it, and also unblock Mariusz's work.

Thanks,
Alice

Quoted 2 lines

--
Mariusz K.