Skip to contentSkip to conversation

IPv4: Tunnel: Fix effective path mtu calculation

PATCH In review Oliver Herms 3 participants 7 messages Jun 25, 2020 – Jun 30, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 IPv4: Tunnel: Fix effective path mtu calculation

The calculation of the effective tunnel mtu, that is used to create mtu exceptions if necessary, is currently not done correctly. This leads to unnecessary entries in the IPv6 route cache for any packet send through the tunnel.

The root cause is, that "dev->hard_header_len" is subtracted from the tunnel destionations path mtu. Thus subtracting too much, if dev->hard_header_len is filled in. This is that case for SIT tunnels where hard_header_len is the underlyings dev hard_header_len (e.g. 14 for ethernet) + 20 bytes IP header (see net/ipv6/sit.c:1091).

However, the MTU of the path is exclusive of the ethernet header and the 20 bytes for the IP header are being subtracted separately already. Thus hard_header_len is removed from this calculation.

For IPIP and GRE tunnels this doesn't change anything as hard_header_len is zero in those cases anyways.

This patch also corrects the calculation of the payload's packet size.

Fixes c54419321455 ("GRE: Refactor GRE tunneling code.")
---
 net/ipv4/ip_tunnel.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
net/ipv4/ip_tunnel.c +2 -3
index f4f1d11eab50..66565647122d 100644
@@ -492,11 +492,10 @@ static int tnl_update_pmtu(struct net_device *dev, struct sk_buff *skb,
 	int mtu;
 
 	tunnel_hlen = md ? tunnel_hlen : tunnel->hlen;
-	pkt_size = skb->len - tunnel_hlen - dev->hard_header_len;
+	pkt_size = skb->len - tunnel_hlen;
 
 	if (df)
-		mtu = dst_mtu(&rt->dst) - dev->hard_header_len
-					- sizeof(struct iphdr) - tunnel_hlen;
+		mtu = dst_mtu(&rt->dst) - sizeof(struct iphdr) - tunnel_hlen;
 	else
 		mtu = skb_valid_dst(skb) ? dst_mtu(skb_dst(skb)) : dev->mtu;
 
Signature · 1 line
2.25.1
maintainer
#1 lore raw
Quoted 10 lines · On Fri, 26 Jun 2020 00:44:35 +0200 Oliver Herms wrote:

The calculation of the effective tunnel mtu, that is used to create mtu exceptions if necessary, is currently not done correctly. This leads to unnecessary entries in the IPv6 route cache for any packet send through the tunnel.

The root cause is, that "dev->hard_header_len" is subtracted from the tunnel destionations path mtu. Thus subtracting too much, if dev->hard_header_len is filled in. This is that case for SIT tunnels where hard_header_len is the underlyings dev hard_header_len (e.g. 14 for ethernet) + 20 bytes IP header (see net/ipv6/sit.c:1091).

It seems like SIT possibly got missed in evolution of the ip_tunnel code? It seems to duplicate a lot of code, including pmtu checking. Doesn't call ip_tunnel_init()...

My understanding is that for a while now tunnels are not supposed to use dev->hard_header_len to reserve skb space, and use dev->needed_headroom, instead. sit uses hard_header_len and doesn't even copy needed_headroom of the lower device.

Quoted 6 lines

However, the MTU of the path is exclusive of the ethernet header and the 20 bytes for the IP header are being subtracted separately already. Thus hard_header_len is removed from this calculation.

For IPIP and GRE tunnels this doesn't change anything as hard_header_len is zero in those cases anyways.

This statement is definitely not true. Please see the calls to ether_setup() in ip_gre.c, and the implementation of this function.

Quoted 4 lines

This patch also corrects the calculation of the payload's packet size.

Fixes c54419321455 ("GRE: Refactor GRE tunneling code.")

All in all, I think it's the SIT code that needs work, not ip_tunnel.

#2 lore raw
Quoted 12 lines · Jakub Kicinski
Quoted 10 lines · On Fri, 26 Jun 2020 00:44:35 +0200 Oliver Herms wrote:

The calculation of the effective tunnel mtu, that is used to create mtu exceptions if necessary, is currently not done correctly. This leads to unnecessary entries in the IPv6 route cache for any packet send through the tunnel.

The root cause is, that "dev->hard_header_len" is subtracted from the tunnel destionations path mtu. Thus subtracting too much, if dev->hard_header_len is filled in. This is that case for SIT tunnels where hard_header_len is the underlyings dev hard_header_len (e.g. 14 for ethernet) + 20 bytes IP header (see net/ipv6/sit.c:1091).

Quoted 3 lines

It seems like SIT possibly got missed in evolution of the ip_tunnel code? It seems to duplicate a lot of code, including pmtu checking. Doesn't call ip_tunnel_init()...

Are you open for patches cleaning this up?

Quoted 10 lines

My understanding is that for a while now tunnels are not supposed to use dev->hard_header_len to reserve skb space, and use dev->needed_headroom, instead. sit uses hard_header_len and doesn't even copy needed_headroom of the lower device.

Quoted 4 lines

However, the MTU of the path is exclusive of the ethernet header and the 20 bytes for the IP header are being subtracted separately already. Thus hard_header_len is removed from this calculation.

Quoted 5 lines
Quoted 2 lines

For IPIP and GRE tunnels this doesn't change anything as hard_header_len is zero in those cases anyways.

This statement is definitely not true. Please see the calls to ether_setup() in ip_gre.c, and the implementation of this function

Right. I have to admit I've only checked for L3 tunnels using printk on dev->hard_header_len. Showing 0 for IPIP and GRE.

So shall I file a patch that changes hard_header_len for SIT tunnels to 0?

maintainer
#3 lore raw
Quoted 13 lines · On Tue, 30 Jun 2020 12:21:14 +0200 Oliver Herms wrote:
Quoted 12 lines · Jakub Kicinski
Quoted 10 lines · On Fri, 26 Jun 2020 00:44:35 +0200 Oliver Herms wrote:

The calculation of the effective tunnel mtu, that is used to create mtu exceptions if necessary, is currently not done correctly. This leads to unnecessary entries in the IPv6 route cache for any packet send through the tunnel.

The root cause is, that "dev->hard_header_len" is subtracted from the tunnel destionations path mtu. Thus subtracting too much, if dev->hard_header_len is filled in. This is that case for SIT tunnels where hard_header_len is the underlyings dev hard_header_len (e.g. 14 for ethernet) + 20 bytes IP header (see net/ipv6/sit.c:1091).

Quoted 5 lines
Quoted 3 lines

It seems like SIT possibly got missed in evolution of the ip_tunnel code? It seems to duplicate a lot of code, including pmtu checking. Doesn't call ip_tunnel_init()...

Are you open for patches cleaning this up?

Certainly! Maybe some of the oddities are justified, but cleanup / re-aligning with the rest of ip_tunnels would be nice.

Not sure how much of it is qualifying as a bug, so perhaps two series would be needed - one for net / stable with bug fixes and another of pure cleanups for net-next?

#4 lore raw
In reply to Jakub Kicinski

Le 30/06/2020 à 08:22, Jakub Kicinski a écrit :
[snip]

Quoted 4 lines

My understanding is that for a while now tunnels are not supposed to use dev->hard_header_len to reserve skb space, and use dev->needed_headroom, instead. sit uses hard_header_len and doesn't even copy needed_headroom of the lower device.

I missed this. I was wondering why IPv6 tunnels uses hard_header_len, if there was a "good" reason:

$ git grep "hard_header_len.*=" net/ipv6/
net/ipv6/ip6_tunnel.c:                  dev->hard_header_len =
tdev->hard_header_len + t_hlen;
net/ipv6/ip6_tunnel.c:  dev->hard_header_len = LL_MAX_HEADER + t_hlen;
net/ipv6/sit.c:         dev->hard_header_len = tdev->hard_header_len +
sizeof(struct iphdr);
net/ipv6/sit.c: dev->hard_header_len    = LL_MAX_HEADER + t_hlen;

A cleanup would be nice ;-)

maintainer
#5 lore raw
Quoted 18 lines · On Tue, 30 Jun 2020 17:51:41 +0200 Nicolas Dichtel wrote:

Le 30/06/2020 à 08:22, Jakub Kicinski a écrit :
[snip]

Quoted 4 lines

My understanding is that for a while now tunnels are not supposed to use dev->hard_header_len to reserve skb space, and use dev->needed_headroom, instead. sit uses hard_header_len and doesn't even copy needed_headroom of the lower device.

I missed this. I was wondering why IPv6 tunnels uses hard_header_len, if there was a "good" reason:

$ git grep "hard_header_len.*=" net/ipv6/
net/ipv6/ip6_tunnel.c:                  dev->hard_header_len =
tdev->hard_header_len + t_hlen;
net/ipv6/ip6_tunnel.c:  dev->hard_header_len = LL_MAX_HEADER + t_hlen;
net/ipv6/sit.c:         dev->hard_header_len = tdev->hard_header_len +
sizeof(struct iphdr);
net/ipv6/sit.c: dev->hard_header_len    = LL_MAX_HEADER + t_hlen;
Quoted 1 line

A cleanup would be nice ;-)

I did some archaeological investigatin' yesterday, and I saw c95b819ad75b ("gre: Use needed_headroom") which converted GRE. Then I think Pravin used GRE as a base for better ip_tunnel infra and the no-hard_header_len-abuse gospel has spread to other IPv4 tunnels. AFAICT IPv6 tunnels were not as lucky, and SIT just got missed in the IPV4 conversion..

#6 lore raw

Le 30/06/2020 à 19:33, Jakub Kicinski a écrit :

Quoted 19 lines · On Tue, 30 Jun 2020 17:51:41 +0200 Nicolas Dichtel wrote:
Quoted 18 lines

Le 30/06/2020 à 08:22, Jakub Kicinski a écrit :
[snip]

Quoted 4 lines

My understanding is that for a while now tunnels are not supposed to use dev->hard_header_len to reserve skb space, and use dev->needed_headroom, instead. sit uses hard_header_len and doesn't even copy needed_headroom of the lower device.

I missed this. I was wondering why IPv6 tunnels uses hard_header_len, if there was a "good" reason:

$ git grep "hard_header_len.*=" net/ipv6/
net/ipv6/ip6_tunnel.c:                  dev->hard_header_len =
tdev->hard_header_len + t_hlen;
net/ipv6/ip6_tunnel.c:  dev->hard_header_len = LL_MAX_HEADER + t_hlen;
net/ipv6/sit.c:         dev->hard_header_len = tdev->hard_header_len +
sizeof(struct iphdr);
net/ipv6/sit.c: dev->hard_header_len    = LL_MAX_HEADER + t_hlen;
Quoted 4 lines
Quoted 1 line

A cleanup would be nice ;-)

I did some archaeological investigatin' yesterday, and I saw c95b819ad75b ("gre: Use needed_headroom") which converted GRE.

Thanks for the pointer.

Quoted 4 lines

Then I think Pravin used GRE as a base for better ip_tunnel infra and the no-hard_header_len-abuse gospel has spread to other IPv4 tunnels. AFAICT IPv6 tunnels were not as lucky, and SIT just got missed in the IPV4 conversion..

Yep, I agree with you, it's probably "historical".