A patch you wrote some time ago (aka: "[patch 41/54] ICMP: Fix icmp_errors_use_inbound_ifaddr sysctl")
Subsystems (2)
I hope you (or anyone else) can spare half a minute to have a quick look at a patch you wrote a few years ago:
Quoted 1 line
I've been tracking down a case of ICMP Redirects originating from the wrong IPs, and as far I can tell, you patch is the last to touch this code (net/ipv4/icmp.c:507):
Quoted 7 lines
if (
rt->fl.iif&&net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr)dev = dev_get_by_index_rcu(net, rt->fl.iif);if (dev)
saddr = inet_select_addr(dev, 0, RT_SCOPE_LINK); else saddr = 0;
In a plain world this would work, but I have come across a case that seems to be not handled by this.
I have two machines set up with VRRP to act as routers out of a subnet, and they have IPs x.x.x.13/28 and x.x.x.14/28, with VRRP holding on to x.x.x.1/28.
If a node in x.x.x.0/28 needs to get a ICMP redirect from x.x.x.1/28 (to reach another subnet behind a different gateway in x.x.x.0/28), then the source IP on the ICMP redirect is chosen as the primary IP on the interface that the packet arrived at.
This is as far as I can tell from RFCs and colleagues fine for most things after you're routed one hop or more, but in the case of ICMP redirect it means that the redirect is not adhered to by the client, as it will get the reidrect from x.x.x.13/28, not x.x.x.1/28.
inet_select_addr seems to be explicitly looking for the primary IP in all cases (./net/ipv4/devinet.c:875), and in the case of sending ICMP recdirect when in an VRRP setup, that would not work well. It should try to match the actual inbound IP.
Judging by the comments from your patch I am not sure if the source IP that triggers the ICMP redirect is available at this point any more.
The way I understand it should pick adress is this way:
Quoted 10 lines
if (
rt->fl.iif&&net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr)dev = dev_get_by_index_rcu(net, rt->fl.iif);if (dev == fl.iif)
saddr = iph->daddr;if (dev != fl.iif)
saddr = inet_select_addr(dev, 0, RT_SCOPE_LINK); else saddr = 0;
I.e. if we are replying to something that is from a local network segment, then iph->daddr would be a more correct source. My C skill is prehistoric so what I've written likely is far from correct, but the general gist is that there is a special case for replying to something local.
As it stands today (I'm on 2.6.35.11), ICMP redirects when using VRRP are broken, and I'm hoping I may have found out why. :)
mvh,
A
Signature · 3 lines
Alexander Hoogerhuis | http://no.linkedin.com/in/alexh Boxed Solutions AS | +47 908 21 485 - [email protected] "Given enough eyeballs, all bugs are shallow." -Eric S. Raymond
Quoted 4 lines · * Alexander Hoogerhuis ([email protected]) wrote:
I hope you (or anyone else) can spare half a minute to have a quick look at a patch you wrote a few years ago:
Quoted 1 line
I actually did not write that patch, rather added it to the -stable tree. Patrick (CCd) wrote it.
Quoted 66 lines
I've been tracking down a case of ICMP Redirects originating from the wrong IPs, and as far I can tell, you patch is the last to touch this code (
net/ipv4/icmp.c:507):Quoted 7 lines
if (
rt->fl.iif&&net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr)dev = dev_get_by_index_rcu(net, rt->fl.iif);if (dev)
saddr = inet_select_addr(dev, 0, RT_SCOPE_LINK); else saddr = 0;In a plain world this would work, but I have come across a case that seems to be not handled by this.
I have two machines set up with VRRP to act as routers out of a subnet, and they have IPs x.x.x.13/28 and x.x.x.14/28, with VRRP holding on to x.x.x.1/28.
If a node in x.x.x.0/28 needs to get a ICMP redirect from x.x.x.1/28 (to reach another subnet behind a different gateway in x.x.x.0/28), then the source IP on the ICMP redirect is chosen as the primary IP on the interface that the packet arrived at.
This is as far as I can tell from RFCs and colleagues fine for most things after you're routed one hop or more, but in the case of ICMP redirect it means that the redirect is not adhered to by the client, as it will get the reidrect from x.x.x.13/28, not x.x.x.1/28.
inet_select_addrseems to be explicitly looking for the primary IP in all cases (./net/ipv4/devinet.c:875), and in the case of sending ICMP recdirect when in an VRRP setup, that would not work well. It should try to match the actual inbound IP.Judging by the comments from your patch I am not sure if the source IP that triggers the ICMP redirect is available at this point any more.
The way I understand it should pick adress is this way:
Quoted 10 lines
if (
rt->fl.iif&&net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr)dev = dev_get_by_index_rcu(net, rt->fl.iif);if (dev == fl.iif)
saddr = iph->daddr;if (dev != fl.iif)
saddr = inet_select_addr(dev, 0, RT_SCOPE_LINK); else saddr = 0;I.e. if we are replying to something that is from a local network segment, then
iph->daddrwould be a more correct source. My C skill is prehistoric so what I've written likely is far from correct, but the general gist is that there is a special case for replying to something local.As it stands today (I'm on 2.6.35.11), ICMP redirects when using VRRP are broken, and I'm hoping I may have found out why. :)
mvh,
ASignature · 3 lines
Alexander Hoogerhuis | http://no.linkedin.com/in/alexh Boxed Solutions AS | +47 908 21 485 - [email protected] "Given enough eyeballs, all bugs are shallow." -Eric S. Raymond
Quoted 8 lines · Chris Wright
Quoted 4 lines · * Alexander Hoogerhuis ([email protected]) wrote:
I hope you (or anyone else) can spare half a minute to have a quick look at a patch you wrote a few years ago:
Quoted 1 line
I actually did not write that patch, rather added it to the -stable tree. Patrick (CCd) wrote it.
I actually only fixed it, it was added in 1c2fb7f9 by J. Simonetti <[email protected]>. Anyways ...
Quoted 29 lines
Quoted 29 lines
I've been tracking down a case of ICMP Redirects originating from the wrong IPs, and as far I can tell, you patch is the last to touch this code (
net/ipv4/icmp.c:507):Quoted 7 lines
if (
rt->fl.iif&&net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr)dev = dev_get_by_index_rcu(net, rt->fl.iif);if (dev)
saddr = inet_select_addr(dev, 0, RT_SCOPE_LINK); else saddr = 0;In a plain world this would work, but I have come across a case that seems to be not handled by this.
I have two machines set up with VRRP to act as routers out of a subnet, and they have IPs x.x.x.13/28 and x.x.x.14/28, with VRRP holding on to x.x.x.1/28.
If a node in x.x.x.0/28 needs to get a ICMP redirect from x.x.x.1/28 (to reach another subnet behind a different gateway in x.x.x.0/28), then the source IP on the ICMP redirect is chosen as the primary IP on the interface that the packet arrived at.
This is as far as I can tell from RFCs and colleagues fine for most things after you're routed one hop or more, but in the case of ICMP redirect it means that the redirect is not adhered to by the client, as it will get the reidrect from x.x.x.13/28, not x.x.x.1/28.
Quoted 6 lines
Quoted 4 lines
inet_select_addrseems to be explicitly looking for the primary IP in all cases (./net/ipv4/devinet.c:875), and in the case of sending ICMP recdirect when in an VRRP setup, that would not work well. It should try to match the actual inbound IP.From what I understand, its explicitly meant to behave this way.
This is what the original commit stated:
The new behaviour (when the sysctl variable is toggled on), it will send the message with the ip of the interface that received the packet that caused the icmp error. This is the behaviour network administrators will expect from a router. It makes debugging complicated network layouts much easier. Also, all 'vendor routers' I know of have the later behaviour.
Quoted 17 lines
Quoted 17 lines
Judging by the comments from your patch I am not sure if the source IP that triggers the ICMP redirect is available at this point any more.
The way I understand it should pick adress is this way:
Quoted 10 lines
if (
rt->fl.iif&&net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr)dev = dev_get_by_index_rcu(net, rt->fl.iif);if (dev == fl.iif)
saddr = iph->daddr;if (dev != fl.iif)
saddr = inet_select_addr(dev, 0, RT_SCOPE_LINK); else saddr = 0;
Quoted 5 lines
Quoted 5 lines
I.e. if we are replying to something that is from a local network segment, then
iph->daddrwould be a more correct source. My C skill is prehistoric so what I've written likely is far from correct, but the general gist is that there is a special case for replying to something local.
That might be a possibility to fix this for your case. But I'm wondering why you're turning this on at all and not have routing decide the correct source address?
Quoted 4 lines · Patrick McHardy
That might be a possibility to fix this for your case. But I'm wondering why you're turning this on at all and not have routing decide the correct source address?
Not a whole lot of tuning, but trying to figure why this would not work as any other VRRP implementation would work on other routers/OSes.
My case seems to be a general problem for ICMP errors, as the IP stack tends to want to listen more to advice coming back with the source IP of the gateway, not a third party.
If you have two machines (A and B) run VRRP and share an IP (C), then any ICMP redirect should have the VRRP IP as source (C), and the way it works today (with or without sysctl_icmp_errors_use_inbound_ifaddr) is that it will have the source set to the primary IP of the source interface.
I suspect this holds for any other ICMP message sent back to hosts in the connected network as well, such as PMTU-related issues, etc.
In my case nodes in the connected subnet would get ICMP redirects from the primary IPs, and thus not listen to them as they are arriving from nodes not listen in the list of known gateways.
It would make more sense when returning ICMP messages the source IP would be the actual IP it is recveied on, not the primary IP of the interface.
mvh,
A
Signature · 3 lines
Alexander Hoogerhuis | http://no.linkedin.com/in/alexh Boxed Solutions AS | +47 908 21 485 - [email protected] "Given enough eyeballs, all bugs are shallow." -Eric S. Raymond
Also, in a quick followup on myself, this is ambiguous, as it does not state that it will select the primary IP. I've
The second part is that RFC 5798 (VRRP v3) lists in section 8.1.1:
Quoted 2 lines
The IPv4 source address of an ICMP redirect should be the address that the end-host used when making its next-hop routing decision.
I.e. the settings in linux, with or without the sysctl flag set, would run against this.
mvh,
A
Signature · 3 lines
Alexander Hoogerhuis | http://no.linkedin.com/in/alexh Boxed Solutions AS | +47 908 21 485 - [email protected] "Given enough eyeballs, all bugs are shallow." -Eric S. Raymond
Excuse the top post; copying netdev where the right people will see this.
Quoted 66 lines · Alexander Hoogerhuis
I hope you (or anyone else) can spare half a minute to have a quick look at a patch you wrote a few years ago:
Quoted 1 line
I've been tracking down a case of ICMP Redirects originating from the wrong IPs, and as far I can tell, you patch is the last to touch this code (
net/ipv4/icmp.c:507):Quoted 7 lines
if (
rt->fl.iif&&net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr) dev =dev_get_by_index_rcu(net, rt->fl.iif);if (dev)
saddr =inet_select_addr(dev, 0, RT_SCOPE_LINK);
else
saddr = 0;In a plain world this would work, but I have come across a case that seems to be not handled by this.
I have two machines set up with VRRP to act as routers out of a subnet, and they have IPs x.x.x.13/28 and x.x.x.14/28, with VRRP holding on to x.x.x.1/28.
If a node in x.x.x.0/28 needs to get a ICMP redirect from x.x.x.1/28 (to reach another subnet behind a different gateway in x.x.x.0/28), then the source IP on the ICMP redirect is chosen as the primary IP on the interface that the packet arrived at.
This is as far as I can tell from RFCs and colleagues fine for most things after you're routed one hop or more, but in the case of ICMP redirect it means that the redirect is not adhered to by the client, as it will get the reidrect from x.x.x.13/28, not x.x.x.1/28.
inet_select_addrseems to be explicitly looking for the primary IP in all cases (./net/ipv4/devinet.c:875), and in the case of sending ICMP recdirect when in an VRRP setup, that would not work well. It should try to match the actual inbound IP.Judging by the comments from your patch I am not sure if the source IP that triggers the ICMP redirect is available at this point any more.
The way I understand it should pick adress is this way:
Quoted 10 lines
if (
rt->fl.iif&&net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr) dev =dev_get_by_index_rcu(net, rt->fl.iif);if (dev == fl.iif)
saddr =iph->daddr;if (dev != fl.iif)
saddr =inet_select_addr(dev, 0, RT_SCOPE_LINK);
else
saddr = 0;I.e. if we are replying to something that is from a local network segment, then
iph->daddrwould be a more correct source. My C skill is prehistoric so what I've written likely is far from correct, but the general gist is that there is a special case for replying to something local.As it stands today (I'm on 2.6.35.11), ICMP redirects when using VRRP are broken, and I'm hoping I may have found out why. :)
mvh,
A