net: gso: fix MTU validation of BIG TCP
Subsystems (4)
v2 cover net: gso: fix MTU validation of BIG TCP
This series fixes the MTU validation of BIG TCP jumbograms and removes the existing IP6SKB_FAKEJUMBO work-around that only fixes the issue in one location.
For GSO packets, the length that matters for MTU validation is the segment length, not the total length of the packet. skb_gso_network_seglen is used by skb_gso_validate_network_len to calculate the segment length including the network and transport headers and to then verify that the segment length is below the MTU.
skb_gso_network_seglen assumes that the headers of the segments are identical to those of the unsegmented packet, but that assumption is incorrect for BIG TCP jumbograms which have an added HBH header that is removed upon segmentation. The calculated segment length ends up being 8 bytes more than the actual segment length.
The actual segment length is set according to the MSS, so the segment length calculated by skb_gso_network_seglen is greater than the MTU, causing the skb_gso_validate_network_len check to fail despite the fact that the actual segment length is lower than the MTU.
There is currently a work-around that fixes this bug in some cases:ip6_xmit sets the IP6SKB_FAKEJUMBO flag for BIG TCP jumbograms, which causes the MTU validation in ip6_finish_output_gso to be skipped (intentionally). However, this work-around doesn't apply to MTU validations performed in other places such as in ip6_forward. BIG TCP jumbograms don't pass the MTU validation when forwarded locally and are therefore dropped, unless the MTU of the originating interface is lower than the MTUs of the rest of the interfaces the packets are forwarded through.
v2:
fix jumbogram check in skb_gso_network_seglen add jumbogram check to skb_gso_mac_seglen as well
v1:
Mariusz Klimek (3):
net: gso: do not include jumbogram HBH header in seglen calculation
ipv6: remove IP6SKB_FAKEJUMBO flag
selftests/net: remove unnecessary MTU config in big_tcp.sh
include/linux/ipv6.h | 1 - net/core/gso.c | 14 +++++++++----- net/ipv6/ip6_output.c | 4 +--- tools/testing/selftests/net/big_tcp.sh | 1 - 4 files changed, 10 insertions(+), 10 deletions(-)
Signature · 1 line
2.47.3
PATCH v2 1/3 net: gso: do not include jumbogram HBH header in seglen calculation
This patch fixes an issue in skb_gso_network_seglen and similar functions where the calculated segment length includes the HBH headers of BIG TCP jumbograms despite these headers being removed before segmentation. These headers are added by GRO or by ip6_xmit for BIG TCP packets and are later removed by GSO. This bug causes MTU validation of BIG TCP jumbograms to fail.
--- net/core/gso.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
net/core/gso.c +9 -5
@@ -1,4 +1,5 @@// SPDX-License-Identifier: GPL-2.0-or-later +#include <linux/if_vlan.h> #include <linux/skbuff.h> #include <linux/sctp.h> #include <net/gso.h>
@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)*/ static unsigned int skb_gso_network_seglen (const struct sk_buff *skb) { -unsigned int hdr_len =skb_transport_header (skb) - -skb_network_header (skb); +unsigned int off =skb_network_offset (skb) +sizeof (struct ipv6hdr ); +unsigned int hdr_len =skb_network_header_len (skb); + +/* Jumbogram HBH header is removed upon segmentation. */ +if (skb_protocol (skb, true) ==htons (ETH_P_IPV6 ) && + skb->len - off >IPV6_MAXPLEN ) + hdr_len -=sizeof (struct hop_jumbo_hdr );return hdr_len +skb_gso_transport_seglen (skb); }
@@ -194,9 +200,7 @@ static unsigned int skb_gso_network_seglen(const struct sk_buff *skb)*/ static unsigned int skb_gso_mac_seglen (const struct sk_buff *skb) { -unsigned int hdr_len =skb_transport_header (skb) -skb_mac_header (skb); - -return hdr_len +skb_gso_transport_seglen (skb); +return skb_mac_header_len (skb) +skb_gso_network_seglen (skb); }/**
Signature · 1 line
2.47.3
Paolo Abeni
maintainerRe: [PATCH net-next v2 1/3] net: gso: do not include jumbogram HBH header in seglen calculation
net/core/gso.c static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb) · line 187
Quoted 13 lines · Mariusz Klimek
net/core/gso.c +7
-2@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)*/ static unsigned int skb_gso_network_seglen (const struct sk_buff *skb) { -unsigned int hdr_len =skb_transport_header (skb) - -skb_network_header (skb); +unsigned int off =skb_network_offset (skb) +sizeof (struct ipv6hdr ); +unsigned int hdr_len =skb_network_header_len (skb); + +/* Jumbogram HBH header is removed upon segmentation. */ +if (skb_protocol (skb, true) ==htons (ETH_P_IPV6 ) && + skb->len - off >IPV6_MAXPLEN ) + hdr_len -=sizeof (struct hop_jumbo_hdr );
IIRC there is some ongoing discussion about introducing big tcp support for virtio. Perhaps a DEBUG_NET_WARN_ON_ONCE(SKB_GSO_DODGY) could help keeping this check updated at due time?
@Jason: could you please double check if I'm off WRT virtio support for big TCP?
/P
Paolo Abeni
maintainerRe: [PATCH net-next v2 1/3] net: gso: do not include jumbogram HBH header in seglen calculation
Quoted 19 lines · Paolo Abeni
net/core/gso.c static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb) · line 187
Quoted 13 lines · Mariusz Klimek
net/core/gso.c +7
-2@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)*/ static unsigned int skb_gso_network_seglen (const struct sk_buff *skb) { -unsigned int hdr_len =skb_transport_header (skb) - -skb_network_header (skb); +unsigned int off =skb_network_offset (skb) +sizeof (struct ipv6hdr ); +unsigned int hdr_len =skb_network_header_len (skb); + +/* Jumbogram HBH header is removed upon segmentation. */ +if (skb_protocol (skb, true) ==htons (ETH_P_IPV6 ) && + skb->len - off >IPV6_MAXPLEN ) + hdr_len -=sizeof (struct hop_jumbo_hdr );IIRC there is some ongoing discussion about introducing big tcp support for virtio. Perhaps a
DEBUG_NET_WARN_ON_ONCE(SKB_GSO_DODGY)could help keeping this check updated at due time?
Quoted 2 lines
@Jason: could you please double check if I'm off WRT virtio support for big TCP?
Reconsidering the above, I think the mentioned check could be added separately, if and when virtio big TCP will come to life.
/P
Re: [PATCH net-next v2 1/3] net: gso: do not include jumbogram HBH header in seglen calculation
Quoted 21 lines · Paolo Abeni
Quoted 19 lines · Paolo Abeni
net/core/gso.c static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb) · line 187
Quoted 13 lines · Mariusz Klimek
net/core/gso.c +7
-2@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)*/ static unsigned int skb_gso_network_seglen (const struct sk_buff *skb) { -unsigned int hdr_len =skb_transport_header (skb) - -skb_network_header (skb); +unsigned int off =skb_network_offset (skb) +sizeof (struct ipv6hdr ); +unsigned int hdr_len =skb_network_header_len (skb); + +/* Jumbogram HBH header is removed upon segmentation. */ +if (skb_protocol (skb, true) ==htons (ETH_P_IPV6 ) && + skb->len - off >IPV6_MAXPLEN ) + hdr_len -=sizeof (struct hop_jumbo_hdr );IIRC there is some ongoing discussion about introducing big tcp support for virtio. Perhaps a
DEBUG_NET_WARN_ON_ONCE(SKB_GSO_DODGY)could help keeping this check updated at due time?
Quoted 5 lines
Quoted 2 lines
@Jason: could you please double check if I'm off WRT virtio support for big TCP?
Reconsidering the above, I think the mentioned check could be added separately, if and when virtio big TCP will come to life.
I agree. BIG TCP support for virtio requires spec features like tso_max_size/segs advertisement, jumbogram head flags. And if I am not wrong, HBH header is probably not needed at that time.
Thanks
Quoted 3 lines
/P
Paolo Abeni
maintainerRe: [PATCH net-next v2 1/3] net: gso: do not include jumbogram HBH header in seglen calculation
In reply to Mariusz Klimeknet/core/gso.c static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb) · line 187
Quoted 13 lines · Mariusz Klimek
net/core/gso.c +7
-2@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)*/ static unsigned int skb_gso_network_seglen (const struct sk_buff *skb) { -unsigned int hdr_len =skb_transport_header (skb) - -skb_network_header (skb); +unsigned int off =skb_network_offset (skb) +sizeof (struct ipv6hdr ); +unsigned int hdr_len =skb_network_header_len (skb); + +/* Jumbogram HBH header is removed upon segmentation. */ +if (skb_protocol (skb, true) ==htons (ETH_P_IPV6 ) && + skb->len - off >IPV6_MAXPLEN ) + hdr_len -=sizeof (struct hop_jumbo_hdr );
I'm sorry for splitting the feedback in multiple replies.
I think the concern I expressed on v1:
https://lore.kernel.org/netdev/[email protected]/
is still not addressed here. What I fear is:
- TCP cooks a plain GSO packet just below the 64K limit.
- Such packet goes trough UDP (or gre) encapsulation, the skb->len size (including outer network header) grows above the 64K limit.
- the above check is satisfied, but no jumbo hop option is present.
I think you could use the ipv6_has_hopopt_jumbo() helper to be on the safe side.
/P
Re: [PATCH net-next v2 1/3] net: gso: do not include jumbogram HBH header in seglen calculation
Quoted 19 lines · Paolo Abeni
net/core/gso.c static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb) · line 187
Quoted 13 lines · Mariusz Klimek
net/core/gso.c +7
-2@@ -177,8 +178,13 @@ static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)*/ static unsigned int skb_gso_network_seglen (const struct sk_buff *skb) { -unsigned int hdr_len =skb_transport_header (skb) - -skb_network_header (skb); +unsigned int off =skb_network_offset (skb) +sizeof (struct ipv6hdr ); +unsigned int hdr_len =skb_network_header_len (skb); + +/* Jumbogram HBH header is removed upon segmentation. */ +if (skb_protocol (skb, true) ==htons (ETH_P_IPV6 ) && + skb->len - off >IPV6_MAXPLEN ) + hdr_len -=sizeof (struct hop_jumbo_hdr );I'm sorry for splitting the feedback in multiple replies.
I think the concern I expressed on v1:
Quoted 9 lines
https://lore.kernel.org/netdev/[email protected]/
is still not addressed here. What I fear is:
- TCP cooks a plain GSO packet just below the 64K limit.
- Such packet goes trough UDP (or gre) encapsulation, theskb->lensize (including outer network header) grows above the 64K limit.
- the above check is satisfied, but no jumbo hop option is present.
Could you maybe clarify what you mean?
This check is for the outer IPv6 header. skb->len - off is the IPv6 payload length so the packet contains a hop-by-hop header if and only if skb->len - off > IPV6_MAXPLEN. It doesn't matter what comes after the IPv6 header. If the TCP payload is smaller than 65535 and it is the encapsulation headers that cause the IPv6 payload length to be above 65535, a jumbogram hop-by-hop header is still added (in ip6_xmit).
Quoted 2 lines
I think you could use the
ipv6_has_hopopt_jumbo()helper to be on the safe side.
Sure, I could submit another revision with this change if my explanation is unsatisfactory.
Quoted 3 lines
/P
Signature · 1 line
Mariusz K.
PATCH v2 2/3 ipv6: remove IP6SKB_FAKEJUMBO flag
In reply to Mariusz KlimekThis patch removes the IP6SKB_FAKEJUMBO flag that is used as a work-around to bypass MTU validation of BIG TCP jumbograms due to a bug in skb_gso_network_seglen. This work-around is no longer required now that the bug is fixed.
--- include/linux/ipv6.h | 1 - net/ipv6/ip6_output.c | 4 +--- 2 files changed, 1 insertion(+), 4 deletions(-)
include/linux/ipv6.h +0 -1
@@ -155,7 +155,6 @@ struct inet6_skb_parm {#define IP6SKB_L3SLAVE 64#define IP6SKB_JUMBOGRAM 128#define IP6SKB_SEG6 256 -#define IP6SKB_FAKEJUMBO 512#define IP6SKB_MULTIPATH 1024#define IP6SKB_MCROUTE 2048 };
net/ipv6/ip6_output.c +1 -3
@@ -179,8 +179,7 @@ ip6_finish_output_gso_slowpath_drop(struct net *net, struct sock *sk,static int ip6_finish_output_gso (struct net *net,struct sock *sk,struct sk_buff *skb,unsigned int mtu) { -if (!(IP6CB (skb)->flags &IP6SKB_FAKEJUMBO ) && - !skb_gso_validate_network_len (skb, mtu)) +if (!skb_gso_validate_network_len (skb, mtu))return ip6_finish_output_gso_slowpath_drop (net, sk, skb, mtu);return ip6_finish_output2 (net, sk, skb);
@@ -323,7 +322,6 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6, proto =IPPROTO_HOPOPTS ; seg_len =0 ; -IP6CB (skb)->flags |=IP6SKB_FAKEJUMBO ; }skb_push (skb,sizeof (struct ipv6hdr ));
Signature · 1 line
2.47.3
PATCH v2 3/3 selftests/net: remove unnecessary MTU config in big_tcp.sh
In reply to Mariusz KlimekThis patch removes the manual lowering of the client MTU in big_tcp.sh. The MTU lowering was previously required as a work-around due to a bug in the MTU validation of BIG TCP jumbograms. The MTU was lowered to 1442, but note that 1492 (1500 - 8) would of worked just as well. Now that the bug has been fixed, the manual client MTU modification can be removed entirely.
--- tools/testing/selftests/net/big_tcp.sh | 1 - 1 file changed, 1 deletion(-)
tools/testing/selftests/net/big_tcp.sh +0 -1
@@ -32,7 +32,6 @@ setup() { ip -net $ROUTER_NS link add link2 type veth peer name link3 netns $SERVER_NS ip -net $CLIENT_NS link set link0 up - ip -net $CLIENT_NS link set link0 mtu 1442 ip -net $CLIENT_NS addr add $CLIENT_IP4/24 dev link0 ip -net $CLIENT_NS addr add $CLIENT_IP6/64 dev link0 nodad ip -net $CLIENT_NS route add $SERVER_IP4 dev link0 via $CLIENT_GW4
Signature · 1 line
2.47.3
Quoted 3 lines · Mariusz Klimek
This series fixes the MTU validation of BIG TCP jumbograms and removes the existing
IP6SKB_FAKEJUMBOwork-around that only fixes the issue in one location.
My series removes IPv6 HBH in BIG TCP entirely:
https://lore.kernel.org/netdev/[email protected]/T/
I believe, it makes this fix no longer necessary.
Quoted 48 lines
For GSO packets, the length that matters for MTU validation is the segment length, not the total length of the packet.
skb_gso_network_seglenis used byskb_gso_validate_network_lento calculate the segment length including the network and transport headers and to then verify that the segment length is below the MTU.
skb_gso_network_seglenassumes that the headers of the segments are identical to those of the unsegmented packet, but that assumption is incorrect for BIG TCP jumbograms which have an added HBH header that is removed upon segmentation. The calculated segment length ends up being 8 bytes more than the actual segment length.The actual segment length is set according to the MSS, so the segment length calculated by
skb_gso_network_seglenis greater than the MTU, causing theskb_gso_validate_network_lencheck to fail despite the fact that the actual segment length is lower than the MTU.There is currently a work-around that fixes this bug in some cases:
ip6_xmitsets theIP6SKB_FAKEJUMBOflag for BIG TCP jumbograms, which causes the MTU validation inip6_finish_output_gsoto be skipped (intentionally). However, this work-around doesn't apply to MTU validations performed in other places such as inip6_forward. BIG TCP jumbograms don't pass the MTU validation when forwarded locally and are therefore dropped, unless the MTU of the originating interface is lower than the MTUs of the rest of the interfaces the packets are forwarded through.v2:
fix jumbogram check inskb_gso_network_seglenadd jumbogram check toskb_gso_mac_seglenas wellv1:
Mariusz Klimek (3):
net: gso: do not include jumbogram HBH header in seglen calculation
ipv6: removeIP6SKB_FAKEJUMBOflag
selftests/net: remove unnecessary MTU config inbig_tcp.shinclude/linux/ipv6.h | 1 - net/core/gso.c | 14 +++++++++----- net/ipv6/ip6_output.c | 4 +--- tools/testing/selftests/net/big_tcp.sh | 1 - 4 files changed, 10 insertions(+), 10 deletions(-)Signature · 1 line
2.47.3
Quoted 11 lines · Alice Mikityanska
Quoted 3 lines · Mariusz Klimek
This series fixes the MTU validation of BIG TCP jumbograms and removes the existing
IP6SKB_FAKEJUMBOwork-around that only fixes the issue in one location.My series removes IPv6 HBH in BIG TCP entirely:
https://lore.kernel.org/netdev/[email protected]/T/
I believe, it makes this fix no longer necessary.
You're right, your patch also fixes this issue by removing HBH headers altogether. Thanks for bringing this up :)
Heads up: I'm working on another patch series that uses some of the code you're removing...
Paolo: should we still merge this or should we just wait for Alice's patch to be accepted? Also, should I already submit my next patch series or should I wait until we're finished with this one?
Signature · 1 line
Mariusz K.
Quoted 17 lines · Mariusz Klimek
Quoted 11 lines · Alice Mikityanska
Quoted 3 lines · Mariusz Klimek
This series fixes the MTU validation of BIG TCP jumbograms and removes the existing
IP6SKB_FAKEJUMBOwork-around that only fixes the issue in one location.My series removes IPv6 HBH in BIG TCP entirely:
https://lore.kernel.org/netdev/[email protected]/T/
I believe, it makes this fix no longer necessary.
You're right, your patch also fixes this issue by removing HBH headers altogether. Thanks for bringing this up :)
Quoted 6 lines
Heads up: I'm working on another patch series that uses some of the code you're removing...
Paolo: should we still merge this or should we just wait for Alice's patch to be accepted? Also, should I already submit my next patch series or should I wait until we're finished with this one?
CCed the tcpdump folks: I would appreciate if you could take forward either of these pull requests:
https://github.com/the-tcpdump-group/tcpdump/pull/1329
https://github.com/the-tcpdump-group/tcpdump/pull/1396
That would unblock the kernel series and allow Paolo to merge it, and also unblock Mariusz's work.
Thanks,
Alice
Quoted 2 lines
--
Mariusz K.