ppp_async: ioctl to set MTU needed
Some months ago I filed the following bug report (with some sentences corrected) on the Kernel Bug Tracker but have received no reaction reaction so far:
Summary: ppp_async: ioctl to set MTU needed
The MTU of a channel of a Multilink PPP link is set from the MRU of a Configure-Ack LCP packet. However, just because the is able to receive packets of a certain size it doesn't mean that the link or the sender are able to transmit packets of that size, so the received MTU of the channel should not be set to the MRU of the peer.
A practical example is PPP encapsulated in GRE or L2TP over IPv6 (without fragmentation). Peer A sets its MRU to the link MTU minus maximum IP header length minus GRE/L2TP header length to be able to receive packets of maximum length over PPP. Peer B connects to peer A and the maximum payload size, which is derived from the Path MTU, of the tunnel between peers A and B is lower than the MRU of peer A. If peer B sends a packet P smaller than the MRU of peer A and larger than the maximum payload size of the IP packet encapsulating P, it will exceed the Path MTU and thus will be rejected and never arrive at peer A. Such scenario is common for VPNs that use PPP or to transport PPP sessions of broadband/Internet subscribers (e.g. if one ISP manages the physical network and another ISP provides Internet access and is the endpoint of the PPP session that runs over the physical network and the PPP session is transported over IPv6). IPv4 was able to fragment IP packets larger than the Path MTU, so setting the MTU to the MRU of peer a worked. With IPv6 (without fragmentation) this no longer works.
As a workaround I patched pppd and sent a Configure-Nak with the correct MRU in the options of the packet in the hope that the remote peer would adjust its MRU to the MRU suggested in the Configure-Nak and that the PPP driver would in turn set the MTU of the channel correctly. However, this didn't work because the remote peer simply sent the original Configure-Request again and again without adjusting its MRU. Unfortunately, I don't know which PPP implementation the remote peer uses.
I understand that PPP is not as interesting as in the 1990s any longer. Nonetheless I would like to have this issue fixed and I would be happy to create a provide for the kernel module and pppd if I'm certain that the patch would get accepted (with minor modifications). So it would help if somebody in charge of pppd or the kernel module comment on my proposal to introduce an ioctl for this. Thanks!
Regards,
Matthias-Christian
Quoted 5 lines · Matthias-Christian Ott
Some months ago I filed the following bug report (with some sentences corrected) on the Kernel Bug Tracker but have received no reaction reaction so far:
Summary:
ppp_async: ioctl to set MTU needed
It would be nice to see complete configuration options and a trace or debug log of the problem. I followed what you posted, but being 100% sure of the scenario and the issues is pretty important when trying to talk about fixes.
Have you tried just setting the pppd "mtu" option? That should force pppd to set an MTU of that value (unless the peer's MRU is even smaller).
I realize that's not a perfect fix. A perfect fix would have some mechanism for the underlying transport to signal changes in effective MTU (e.g., Path MTU data) so that this value could be altered at the network layer as needed. Unfortunately, there's currently no such mechanism, as far as I know.
Quoted 7 lines
As a workaround I patched pppd and sent a Configure-Nak with the correct MRU in the options of the packet in the hope that the remote peer would adjust its MRU to the MRU suggested in the Configure-Nak and that the PPP driver would in turn set the MTU of the channel correctly. However, this didn't work because the remote peer simply sent the original Configure-Request again and again without adjusting its MRU. Unfortunately, I don't know which PPP implementation the remote peer uses.
I would have been surprised if that had worked. Sending a Configure-Nak for a smaller MRU is (to me, at least) a nonsense request. The peer's offer of a "too large" MRU is just fine and should be accepted. You're under no obligation ever to send a packet as large as the maximum your peer may be willing to accept, so the local MTU can be lower than the peer's MRU. It really only makes sense to negotiate the peer's MRU upwards, not downwards, and then only *if* the advertised value is too small for the desired purpose of the link and if there's some hope that it'll either comply or fail in a manner that makes sense to the user.
So, yes, ignoring your request for a smaller MRU sounds like the behavior of a somewhat reasonable (or not entirely wrong) peer. Better still would be to omit the option on subsequent Configure-Request messages, and thus insist on the default.
Signature · 1 line
James Carlson 42.703N 71.076W <[email protected]>
Hello!
Matthias-Christian Ott schrieb am Tue, 13 Jan 2015 18:33:04 +0100:
Quoted 5 lines
The MTU of a channel of a Multilink PPP link is set from the MRU of a Configure-Ack LCP packet. However, just because the is able to receive packets of a certain size it doesn't mean that the link or the sender are able to transmit packets of that size, so the received MTU of the channel should not be set to the MRU of the peer.
Well, this seems to be not so easy. I just studied the Kernel source code, the pppd source code and the PPP RFCs, so I feel rather well-informed in a way ;-)
What does the "mtu" option really do? In a word: (almost) nothing! It only restricts the MTU of the underlying network interface that is finally created after LCP negotation has completed successfully. You can see that the MTU value is stored in lcp_allowoptions[0].mru:
{ "mtu", o_int, &lcp_allowoptions[0].mru,
"Set our MTU", OPT_LIMITS, NULL, MAXMRU, MINMRU },
(lcp.c, lines 132-133), and it is only used when setting the MTU of the network interface. The non-multilink case:
/*
* Set our MTU to the smaller of the MTU we wanted and
* the MRU our peer wanted. If we negotiated an MRU,
* set our MRU to the larger of value we wanted and
* the value we got in the negotiation.
* Note on the MTU: the link MTU can be the MRU the peer wanted,
* the interface MTU is set to the lowest of that, the
* MTU we want to use, and our link MRU.
*/
mtu = ho->neg_mru? ho->mru: PPP_MRU;
mru = go->neg_mru? MAX(wo->mru, go->mru): PPP_MRU;
#ifdef HAVE_MULTILINK
if (!(multilink && go->neg_mrru && ho->neg_mrru))
#endif /* HAVE_MULTILINK */
netif_set_mtu(f->unit, MIN(MIN(mtu, mru), ao->mru));
^^^ ^^^^^^^(lcp.c, lines 1911-1925) and
/* not doing multilink */
if (go->neg_mrru)
notice("oops, multilink negotiated only for receive");
mtu = ho->neg_mru? ho->mru: PPP_MRU;---> if (mtu > ao->mru)
---> mtu = ao->mru;
(multilink.c, lines 128-133). The multilink case:
mtu = MIN(ho->mrru, ao->mru);
^^^ ^^^^^^^
[...]
netif_set_mtu(0, mtu);(multilink.c, lines 184 and 234). So ao->mru (or the "mtu" value) effectively caps the MTU (due to the various MIN(...) macro invocations), but this does NOT mean that a larger peer MRU cannot be successfully negotiated! You see this in the lcp_reqci() function where CONFREQs are ACKed NAKed or REJected:
case CI_MRU:
if (!ao->neg_mru || /* Allow option? */
cilen != CILEN_SHORT) { /* Check CI length */
orc = CONFREJ; /* Reject CI */
break;
}
GETSHORT(cishort, p); /* Parse MRU */
/*
* He must be able to receive at least our minimum.
* No need to check a maximum. If he sends a large number,
* we'll just ignore it.
*/
if (cishort < MINMRU) {
orc = CONFNAK; /* Nak CI */
PUTCHAR(CI_MRU, nakp);
PUTCHAR(CILEN_SHORT, nakp);
PUTSHORT(MINMRU, nakp); /* Give him a hint */
break;
}
ho->neg_mru = 1; /* Remember he sent MRU */
ho->mru = cishort; /* And remember value */(lcp.c, lines 1556-1577). So the "mtu" value does not have any consequences for the negotiated peer MRU. And, consequently, it does not influence the value the kernel uses for the channel MTU either, as the kernel uses the ACKed value:
/* process the options in the confack */
data += 4;
dlen -= 4;
/* data[0] is code, data[1] is length */
while (dlen >= 2 && dlen >= data[1] && data[1] >= 2) {
switch (data[0]) {
case LCP_MRU:
val = get_unaligned_be16(data + 2);
if (inbound)
ap->mru = val;
else
ap->chan.mtu = val;
break;(Kernel 3.17.8, ppp_async.c, lines 995-1007).
Where are we now? I think your problem lies in this comment in lcp_reqci():
/*
* He must be able to receive at least our minimum.
* No need to check a maximum. If he sends a large number,
---> !! * we'll just ignore it.
*/
You think each PPP implementation has to NAK when a MRU being larger than our MTU is REQuested by the peer. Obviously this isn't handled this way, at least not by pppd. However, I think your problem is twofold. You did not write how big your desired MTU (at the side of peer B) really is. I bet that it is smaller than 1500, as you write it is smaller than "link MTU minus maximum IP header length minus GRE/L2TP header length". However, RFC 1661 requires that each implementation MUST accept packets that are 1500 bytes long:
6.1. Maximum-Receive-Unit (MRU)
[...]
The default value is 1500 octets. If smaller packets are requested, an implementation MUST still be able to receive the full 1500 octet information field in case link synchronization is lost.
So each attempt to force peer A to request a MRU < 1500 is formally wrong as it might be ignored -- even if the peer ACKs this MRU.
Regards,
Christoph Schulz
Quoted 2 lines · Christoph Schulz
(
lcp.c, lines 132-133), and it is only used when setting the MTU of the network interface. The non-multilink case:
[...]
Quoted 2 lines
netif_set_mtu(f->unit, MIN(MIN(mtu, mru), ao->mru));
^^^ ^^^^^^^That part looks good to me. Capping at ao->mru means that the user can specify the "mtu" option and force the actual link MTU to be anything he wants (up to the peer's advertised MRU).
Quoted 6 lines
(
multilink.c, lines 128-133). The multilink case:mtu = MIN(ho->mrru, ao->mru); ^^^ ^^^^^^^ [...] netif_set_mtu(0, mtu);
That also looks good. Again, the user can specify anything he wants with the "mtu" option.
That's why I suggested setting "mtu" in the pppd options as a reasonable work-around.
Quoted 9 lines
6.1. Maximum-Receive-Unit (MRU)
[...]
The default value is 1500 octets. If smaller packets are requested, an implementation MUST still be able to receive the full 1500 octet information field in case link synchronization is lost.So each attempt to force peer A to request a MRU < 1500 is formally wrong as it might be ignored -- even if the peer ACKs this MRU.
Not so much "wrong" as "pointless." Why bother to ask for less? You don't have to pad the packets out to the full MRU, so why do you care what the peer has advertised as a maximum? All that matters is that (by acking the value) you've promised not to send anything bigger. By adhering to a smaller limit, you're certainly in compliance with the peer's limitations.
Note that the "synchronization" bit there is obscure, but refers to the possibility of LCP renegotiation. For some (many?) implementations, LCP renegotiation (unfortunately!) causes link failure, so everyone avoids it.
In truth, the reason you're required to support an MRU of 1500 is that the peer can always just Configure-Reject the option. If he does that, then what do you do? If the peer rejects it, that's the end; the option ends up with the default value, which is 1500.
Signature · 1 line
James Carlson 42.703N 71.076W <[email protected]>
Hello!
James Carlson schrieb am Tue, 13 Jan 2015 16:31:29 -0500:
Quoted 6 lines
Not so much "wrong" as "pointless." Why bother to ask for less? You don't have to pad the packets out to the full MRU, so why do you care what the peer has advertised as a maximum? All that matters is that (by acking the value) you've promised not to send anything bigger. By adhering to a smaller limit, you're certainly in compliance with the peer's limitations.
Yes. However, the problem lies in the mechanism the Linux kernel determines the MTU of a link in a multilink bundle: It simply takes the peer's MRU we ACKed. Obviously this is not correct if we are bound to use a smaller (channel) MTU due to other constraints (tunnel overhead). But there is currently no possibility to tell the Linux kernel what channel MTU to use.
Regards,
Christoph Schulz
Quoted 11 lines · Christoph Schulz
Hello!
James Carlson schrieb am Tue, 13 Jan 2015 16:31:29 -0500:
Quoted 6 lines
Not so much "wrong" as "pointless." Why bother to ask for less? You don't have to pad the packets out to the full MRU, so why do you care what the peer has advertised as a maximum? All that matters is that (by acking the value) you've promised not to send anything bigger. By adhering to a smaller limit, you're certainly in compliance with the peer's limitations.
Quoted 6 lines
Yes. However, the problem lies in the mechanism the Linux kernel determines the MTU of a link in a multilink bundle: It simply takes the peer's MRU we ACKed. Obviously this is not correct if we are bound to use a smaller (channel) MTU due to other constraints (tunnel overhead). But there is currently no possibility to tell the Linux kernel what channel MTU to use.
Ah, ok. Are you referring to the individual link MTU rather than the bundle's overall MTU?
If so, then, yes, that does look like a problem. It should be possible to limit individual link MTU values as desired, but I don't see a good way to do it.
But, for what it's worth, running MP over a tunneled link sounds like extreme weirdness to me. Unless you're trying to set new records for packet loss and link latency, why would you do that?
Signature · 1 line
James Carlson 42.703N 71.076W <[email protected]>
Hello!
James Carlson schrieb am Tue, 13 Jan 2015 17:22:47 -0500:
Quoted 2 lines
Ah, ok. Are you referring to the individual link MTU rather than the bundle's overall MTU?
Yes.
Quoted 4 lines
If so, then, yes, that does look like a problem. It should be possible to limit individual link MTU values as desired, but I don't see a good way to do it.
Me neither, at least without introducing a system-specific ioctl that would have to be explicitly called by pppd. However, perhaps anyone has a better idea...
By the way, how does Solaris solve this problem? I see that PPPIO_MTU is sent in tty_send_config() in sys-solaris.c for setting the link MTU. Is that used when doing multilink PPP? The corresponding function in sys-linux.c does not send the link MTU to the kernel, it only saves the value in a global static variable "link_mtu" (line 1250) which is not used anywhere else.
Quoted 4 lines
But, for what it's worth, running MP over a tunneled link sounds like extreme weirdness to me. Unless you're trying to set new records for packet loss and link latency, why would you do that?
Don't ask me, ask Matthias-Christian ;-)
But nevertheless there is no obvious reason why it should be impossible to make it work.
Regards,
Christoph
Hello!
Christoph Schulz schrieb am Tue, 13 Jan 2015 23:56:25 +0100:
Quoted 1 line
By the way, how does Solaris solve this problem? [...]
Never mind, I forgot that pppd supports multilink PPP only under Linux.
Regards,
Christoph Sculz
Quoted 11 lines · Christoph Schulz
Matthias-Christian Ott schrieb am Tue, 13 Jan 2015 18:33:04 +0100:
Quoted 5 lines
The MTU of a channel of a Multilink PPP link is set from the MRU of a Configure-Ack LCP packet. However, just because the is able to receive packets of a certain size it doesn't mean that the link or the sender are able to transmit packets of that size, so the received MTU of the channel should not be set to the MRU of the peer.
Well, this seems to be not so easy. I just studied the Kernel source code, the pppd source code and the PPP RFCs, so I feel rather well-informed in a way ;-)
I dont't doubt that.
Quoted 24 lines
Where are we now? I think your problem lies in this comment in
lcp_reqci():/*
* He must be able to receive at least our minimum.
* No need to check a maximum. If he sends a large number,
---> !! * we'll just ignore it.*/You think each PPP implementation has to NAK when a MRU being larger than our MTU is REQuested by the peer. Obviously this isn't handled this way, at least not by pppd. However, I think your problem is twofold. You did not write how big your desired MTU (at the side of peer B) really is. I bet that it is smaller than 1500, as you write it is smaller than "link MTU minus maximum IP header length minus GRE/L2TP header length". However, RFC 1661 requires that each implementation MUST accept packets that are 1500 bytes long:
6.1. Maximum-Receive-Unit (MRU)
[...]
The default value is 1500 octets. If smaller packets are requested, an implementation MUST still be able to receive the full 1500 octet information field in case link synchronization is lost.
Quoted 2 lines
So each attempt to force peer A to request a MRU < 1500 is formally wrong as it might be ignored -- even if the peer ACKs this MRU.
It seems that I read over this than and what I wanted to achieve is impossible with PPP. I think setting the channel MTU to the MRU is unaffected by this and is still wrong though. From RFC 1661:
6.1. Maximum-Receive-Unit (MRU)
[...]
This option is used to indicate an implementation capability. The peer is not required to maximize the use of the capacity. For example, when a MRU is indicated which is 2048 octets, the peer is not required to send any packet with 2048 octets. The peer need not Configure-Nak to indicate that it will only send smaller packets, since the implementation will always require support for at least 1500 octets.
Regards,
Matthias-Christian
Quoted 8 lines · Matthias-Christian Ott
It seems that I read over this than and what I wanted to achieve is impossible with PPP. I think setting the channel MTU to the MRU is unaffected by this and is still wrong though. From RFC 1661:
6.1. Maximum-Receive-Unit (MRU)
[...]
Quoted 7 lines
This option is used to indicate an implementation capability. The peer is not required to maximize the use of the capacity. For example, when a MRU is indicated which is 2048 octets, the peer is not required to send any packet with 2048 octets. The peer need not Configure-Nak to indicate that it will only send smaller packets, since the implementation will always require support for at least 1500 octets.
Another aspect is that if Multilink PPP is used, it might be wrong to set the MTU based on the MRU. If both peers know that the link MTU is smaller than 1500 octets and want to encapsulate payloads larger than the link MTU (that is the interface MTU is bigger than the channel MTU and Multilink PPP fragments payloads larger the channel MTU), they can adjust their MRRU and channel MTU accordingly and thus never send PPP packets larger than the link MTU. If the channel MTU is set from the MRU, this is not possible.
Regards,
Matthias-Christian
Quoted 7 lines · Christoph Schulz
Hello!
Christoph Schulz schrieb am Tue, 13 Jan 2015 23:56:25 +0100:
Quoted 1 line
By the way, how does Solaris solve this problem? [...]
Never mind, I forgot that pppd supports multilink PPP only under Linux.
Yes, exactly. At Sun, we had a multilink implementation, but there wasn't much call for it. The best reason to use MP is to bond together the two B channels in Basic-Rate ISDN or two or more B channels in Primary-Rate ISDN. Those links tend to be tightly controlled in terms of delay and error rate, so they work well for MP.
And not many people ended up caring too much about ISDN. (Yeah, I know there are still aficionados out there ...)
If because the segmentation/reassembly process in MP works only as well as the worst-performing link, it tends to be an overall loss if you have links that differ.
A better plan is to use multipath IP routes -- ECMP across multiple independent ordinary (non-MP) PPP links -- if you have links that differ.
Signature · 1 line
James Carlson 42.703N 71.076W <[email protected]>