question: update_pmtu doesn't update dst mtu

Hello,

Is this a place where I can post questions not the patches? If so, can anybody explain me what the problem I have when trying to update dst PMTU value.

So, the scheme is the following:

                  skb->dst                   rt(pmtu)

IN_DEV (MTU 1500) -------> TUNNEL (MTU 1440) -------> OUT_DEV (MTU 1500)

I have a simple tunnel_xmit function that handles all the packets that goes trough the tunnel. So, I have an skb with valid skb->dst value (filled earlier in ip_input_route_noref).

Next, when encapsulation is done, I needed to get an output route for the packet (rt at the scheme). At this point I know that the output route may have a PMTU value that is different from the OUT_DEV->mtu. So, I'm trying to update the input skb->dst route's PMTU with the update_pmtu function. It seems that all is OK, but when I trying to get the dst mtu value (using dst_mtu(skb_dst(skb))) I always get the value of 1440 (TUNNEL's MTU):

tunnel_xmit:

    ...
    pmtu = dst_mtu(&rt->dst) - OVERHEAD; // pmtu = 1300, for example
    skb_dst(skb)->ops->update_pmtu(skb_dst(skb), NULL, skb, pmtu);
    // dst_mtu(skb_dst(skb)) still returns 1440

Looking through the code gives me that rt_pmtu is always 0 for the skb->dst entry and ipv4_mtu that called via the dst->ops->mtu() uses dev->mtu :(

Quoted 2 lines · Ilya V. Matveychikov

Looking through the code gives me that rt_pmtu is always 0 for the skb->dst entry and ipv4_mtu that called via the dst->ops->mtu() uses dev->mtu :(

At this point you have to drop skb_dst and have to relookup the route. During that a new dst will be created which gets the mtu value from the next hop exception, which got created by update_pmtu.

Normally routes are checked with dst_check if they are still valid and a relookup should happen. In your example just do the relookup unconditionally.

Quoted 4 lines · Hannes Frederic Sowa

Quoted 2 lines · Ilya V. Matveychikov

Looking through the code gives me that rt_pmtu is always 0 for the skb->dst entry and ipv4_mtu that called via the dst->ops->mtu() uses dev->mtu :(

Quoted 8 lines

At this point you have to drop skb_dst and have to relookup the route. During that a new dst will be created which gets the mtu value from the next hop exception, which got created by update_pmtu.

Normally routes are checked with dst_check if they are still valid and a relookup should happen. In your example just do the relookup unconditionally.

Does it mean that the next packet must have an updated route without any problems? I meant that if the first packet xmitting leads to updating the route PMTU via the exception creating (or updating) so the next packets must have an updated route? Am I right?

Quoted 14 lines · Ilya V. Matveychikov

Quoted 4 lines · Hannes Frederic Sowa

Quoted 2 lines · Ilya V. Matveychikov

Looking through the code gives me that rt_pmtu is always 0 for the skb->dst entry and ipv4_mtu that called via the dst->ops->mtu() uses dev->mtu :(

Quoted 8 lines

At this point you have to drop skb_dst and have to relookup the route. During that a new dst will be created which gets the mtu value from the next hop exception, which got created by update_pmtu.

Normally routes are checked with dst_check if they are still valid and a relookup should happen. In your example just do the relookup unconditionally.

Quoted 4 lines

Does it mean that the next packet must have an updated route without any problems? I meant that if the first packet xmitting leads to updating the route PMTU via the exception creating (or updating) so the next packets must have an updated route? Am I right?

In case the next packet causes a lookup in the routing table, yes. Or if it does cache the routing lookup in some structure and checks the route with dst_check and conditionally does a relookup (which kernel implementation should already do) then you should also get the updated mtu value.

Quoted 20 lines · Hannes Frederic Sowa

Quoted 18 lines · Ilya V. Matveychikov

Quoted 4 lines · Hannes Frederic Sowa

Quoted 2 lines · Ilya V. Matveychikov

Looking through the code gives me that rt_pmtu is always 0 for the skb->dst entry and ipv4_mtu that called via the dst->ops->mtu() uses dev->mtu :(

Quoted 8 lines

At this point you have to drop skb_dst and have to relookup the route. During that a new dst will be created which gets the mtu value from the next hop exception, which got created by update_pmtu.

Normally routes are checked with dst_check if they are still valid and a relookup should happen. In your example just do the relookup unconditionally.

Does it mean that the next packet must have an updated route without any problems? I meant that if the first packet xmitting leads to updating the route PMTU via the exception creating (or updating) so the next packets must have an updated route? Am I right?

Quoted 6 lines

In case the next packet causes a lookup in the routing table, yes. Or if it does cache the routing lookup in some structure and checks the route with dst_check and conditionally does a relookup (which kernel implementation should already do) then you should also get the updated mtu value.

OK, seems that I understand the logic. Thanks a lot.

Just another related question that gets me into trouble. Imagine that there is an SKB that wants to be transmitted via that tunnel. Let's say that when it comes to the TUNNEL device it has an MTU1 value. Now, someone updates the PMTU for the route and mtu decreasing from MTU1 to MTU2, so MTU2 < MTU1.

Given that, I suppose that our SKB must be (re)fragmented with ip_fragment as it's size might be slightly bigger then the path can pass. The problem is that ip_fragment uses dst_mtu(skb_dst(skb)) to determine the fragment size but it still has MTU1 value as even update_pmtu(MTU2) was called as it doesn't leads to real dst MTU updating.

So the question is do I need to relookup the route or can I use the following hack before ip_fragment:

	// dst_mtu(dst) shows MTU1
	dst->ops->update_pmtu(dst, ..., MTU2)
	...
	skb_rtable(skb)->rt_pmtu = MTU2;

	dst_set_expires(dst, 1);

	...
	// now, ip_fragment knows about real MTU value
	ip_fragment(skb, output...)

Quoted 11 lines · Ilya V. Matveychikov

Just another related question that gets me into trouble. Imagine that there is an SKB that wants to be transmitted via that tunnel. Let's say that when it comes to the TUNNEL device it has an MTU1 value. Now, someone updates the PMTU for the route and mtu decreasing from MTU1 to MTU2, so MTU2 < MTU1.

Given that, I suppose that our SKB must be (re)fragmented with ip_fragment as it's size might be slightly bigger then the path can pass. The problem is that ip_fragment uses dst_mtu(skb_dst(skb)) to determine the fragment size but it still has MTU1 value as even update_pmtu(MTU2) was called as it doesn't leads to real dst MTU updating.

Quoted 7 lines

So the question is do I need to relookup the route or can I use the following hack before ip_fragment:

	// dst_mtu(dst) shows MTU1
	dst->ops->update_pmtu(dst, ..., MTU2)
	...
	skb_rtable(skb)->rt_pmtu = MTU2;

This might be a cached dst and you would alter the mtu for more nexthops than you intended.

Quoted 1 line

	dst_set_expires(dst, 1);

With this you won't get around the time_after_eq check. You would have to tweak it manually to not retrieve dst_metrics value (this is what you intended?).

Quoted 3 lines

	...
	// now, ip_fragment knows about real MTU value
	ip_fragment(skb, output...)

Check if you can something do like
skb_dst_drop(skb);
new_dst = ip_route_output*(..., &fl4, ...);
skb_dst_set(_noref)(skb, new_dst);

This should be a very unlikely path, I assume, so should not degrade performance that much.

I wonder why you update the mtu in the output path.

Works fine, thanks! By the way, could you briefly explain why routes are separated to input and output? What are the benefits?

Quoted 2 lines · Ilya V. Matveychikov

Works fine, thanks! By the way, could you briefly explain why routes are separated to input and output? What are the benefits?

Core routing table is not split by input and output, merely the dst construction and surrounding lookup and policy checks are seperated for input and output.

Output does not have to deal with source address validation e.g. but has to do source address selection.

Quite hard to answer, I guess the design emerged quite naturally. ;)