Configure ICMP error source address
Subsystems (3)
Hi,
It is possible to change the source address of ICMP error messages generated by the kernel via /proc/sys/net/ipv4/icmp_errors_use_inbound_ifaddr. This is currently the only way to influence the source address as ICMP errors do not travel through the NAT table (for obvious reasons).
We have the situation that our routers use RFC1918 addresses on their transfer networks (which should be quite common nowadays to save on public IPv4 addresses). ICMP errors are generated with RFC1918 source addresses and therefor never reach the original sender.
Every router has its public IP address bound to dev lo to be reachable even if any one interface is down. Routing protocols assure that.
Is it a good idea to develop a kernel patch that makes it possible to select the first IPv4 address on dev lo with scope global as the source address for ICMP errors? Would that do any harm to the Internet at large?
Regards
Signature · 12 lines
Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Don't put a public address on a lo device use a dummy eth interface instead. Any IP address and it's subnet assigned to a lo device is marked as a marcian address and the traffic is dropped if it tries to leave the lo device.
I know that there is som old documentation out there (for example quagga's documentation) that says you can do it but it's been wrong since the 2.4 version off the kernel. Linux treats the lo device differently that what routers call a loopback device. The dummy driver is the linux equivalent of what routers call a loopback device. Original Message
From: Robert Sander
Sent: Friday, January 8, 2016 04:32
To: [email protected]; [email protected]
Subject: Configure ICMP error source address
Hi,
It is possible to change the source address of ICMP error messages generated by the kernel via /proc/sys/net/ipv4/icmp_errors_use_inbound_ifaddr. This is currently the only way to influence the source address as ICMP errors do not travel through the NAT table (for obvious reasons).
We have the situation that our routers use RFC1918 addresses on their transfer networks (which should be quite common nowadays to save on public IPv4 addresses). ICMP errors are generated with RFC1918 source addresses and therefor never reach the original sender.
Every router has its public IP address bound to dev lo to be reachable even if any one interface is down. Routing protocols assure that.
Is it a good idea to develop a kernel patch that makes it possible to select the first IPv4 address on dev lo with scope global as the source address for ICMP errors? Would that do any harm to the Internet at large?
Regards
Signature · 12 lines
Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Quoted 1 line · On 08.01.2016 16:24, [email protected] wrote:
Don't put a public address on a lo device use a dummy eth interface instead. Any IP address and it's subnet assigned to a lo device is marked as a marcian address and the traffic is dropped if it tries to leave the lo device.
O_o
Quoted 2 lines
I know that there is som old documentation out there (for example quagga's documentation) that says you can do it but it's been wrong since the 2.4 version off the kernel. Linux treats the lo device differently that what routers call a loopback device. The dummy driver is the linux equivalent of what routers call a loopback device.
What you write seems odd to me, we don't treat lo devices differently to dummy devices in respect if you bind a public ip address on it.
Bye,
Hannes
The reason why is in the kernel documentation distributed with the source code.
Its not all immediately obvious why and where the kernel does this unless you were involved in the original debates about it. The problems with binding a public ip address to a loopback address are more recently is partialy selinux but mostly of it is because of how routing changed when iproute2 was introduced in the 2.2 kernel (if I remember the versions correctly it was so long ago). The reason is security, many processes bind to the loopback interface for IPC and just grab the first address on the first lo device they find, some of the developers of the applications weren't even aware they were doing it because in some cases it was C libraries several layers deep doing it because odd loopback addresses were common on UNIX ( I remember an AIX admin back in the early 90's telling me I should always use addresses like 127.1.0.2 on the loopback because it was valid made it harder for script kiddies to attack processes ). This was fine when additional addresses would need additional alias on the interface for ifconfig (lo:1 for example) but with iproute2 you could now assign multiple ip addresses to an interface without creating an alias (commonly seen on the keepalived mailing list "why doesn't my VIP show up in ifconfig?") So there was a decision made to declare all subnets assigned to the loopback interface as unroutable to prevent a lot of very common security holes that were being seen in the wild.
That said you can make it work but you have to do a lot of things you should never do on a firewall like turn off selinux. Or you can simply use the dummy driver and bind your public IP's to that and the problems magically go away. And it makes a lot of sense why else would there be a need for the dummy network driver?
Original Message
From: Hannes Frederic Sowa
Sent: Friday, January 8, 2016 11:11
To: [email protected]; Robert Sander; [email protected]; [email protected]
Subject: Re: Configure ICMP error source address
Quoted 1 line · On 08.01.2016 16:24, [email protected] wrote:
Don't put a public address on a lo device use a dummy eth interface instead. Any IP address and it's subnet assigned to a lo device is marked as a marcian address and the traffic is dropped if it tries to leave the lo device.
O_o
Quoted 2 lines
I know that there is som old documentation out there (for example quagga's documentation) that says you can do it but it's been wrong since the 2.4 version off the kernel. Linux treats the lo device differently that what routers call a loopback device. The dummy driver is the linux equivalent of what routers call a loopback device.
What you write seems odd to me, we don't treat lo devices differently to dummy devices in respect if you bind a public ip address on it.
Bye,
Hannes
Quoted 2 lines · On 09.01.2016 04:57, [email protected] wrote:
The reason why is in the kernel documentation distributed with the source code.
Its not all immediately obvious why and where the kernel does this
unless you were involved in the original debates about it. The problems
with binding a public ip address to a loopback address are more
recently is partialy selinux but mostly of it is because of how routing
changed when iproute2 was introduced in the 2.2 kernel (if I remember
the versions correctly it was so long ago). The reason is security,
many processes bind to the loopback interface for IPC and just grab the
first address on the first lo device they find, some of the developers
of the applications weren't even aware they were doing it because in
some cases it was C libraries several layers deep doing it because odd
loopback addresses were common on UNIX ( I remember an AIX admin back
in the early 90's telling me I should always use addresses like
127.1.0.2 on the loopback because it was valid made it harder for
script kiddies to attack processes ). This was fine when additional
addresses would need additiona
l alias on the interface for ifconfig (lo:1 for example) but with iproute2 you could now assign multiple ip addresses to an interface without creating an alias (commonly seen on the keepalived mailing list "why doesn't my VIP show up in ifconfig?")
Quoted 1 line
So there was a decision made to declare all subnets assigned to the loopback interface as unroutable to prevent a lot of very common security holes that were being seen in the wild.
selinux can only affect what happens between user space and kernel space, it cannot install policy what happens during forwarding if no user space is involved.
In general the kernel makes sure nothing leaves or enters the system with an address in the 127.0.0.0/8 subnet. Albeit we have a sysctl nowadays to lift this restriction, sysctl_route_localnet.
Besides that I don't see any differences, as I bind addresses to the loopback interface from time to time and it works.
One must just be careful if the kernel is switched from weak end mode into a stong end mode by enabling more strict arp filters, but this would also break setups with dummy interfaces.
Quoted 1 line
That said you can make it work but you have to do a lot of things you should never do on a firewall like turn off selinux. Or you can simply use the dummy driver and bind your public IP's to that and the problems magically go away. And it makes a lot of sense why else would there be a need for the dummy network driver?
I would also use dummy interfaces in production systems, merely to split the statistics from dummy.
Thanks,
Hannes
Hi,
Am 09.01.2016 um 10:57 schrieb Hannes Frederic Sowa:
Quoted 3 lines
I would also use dummy interfaces in production systems, merely to split the statistics from dummy.
Thank you for discussing the merits of dummy interfaces. I will consider your arguments. But unfortunately this did not answer my question.
Let me rephrase it:
Is it a good idea to set a specific global IPv4 address as source address for outgoing ICMP error messages?
Would it be OK to create a /proc/sys/net/ipv4/icmp_errors_source where you could write an arbitrary IPv4 address into? And that would get used as the source address of ICMP errors?
My questions did contain the loopback interface as I first thought it a good source of a globally routable IPv4 address (at least in our case).
Secound thought: Instead of writing an IPv4 address to /proc/sys/net/ipv4/icmp_errors_source write an interface name to that file and take the first global IPv4 address from that interface as source for ICMP errors. Then you could create a dummy interface for that use case, too.
Still: Is it a good idea to do so?
Regards
Signature · 12 lines
Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Quoted 3 lines · Robert Sander
Is it a good idea to set a specific global IPv4 address as source address for outgoing ICMP error messages?
Why only for ICMP error messages and not all outgoing packets sent to a global destination address ?
Quoted 9 lines · Robert Sander
Hi,
Am 09.01.2016 um 10:57 schrieb Hannes Frederic Sowa:
Quoted 3 lines
I would also use dummy interfaces in production systems, merely to split the statistics from dummy.
Thank you for discussing the merits of dummy interfaces. I will consider your arguments. But unfortunately this did not answer my question.
Yes, I know. :) I tried to answer it in the other reply.
Quoted 4 lines
Let me rephrase it:
Is it a good idea to set a specific global IPv4 address as source address for outgoing ICMP error messages?
Not sure if this generic enough.
So my idea was to have a specific routing table and ip rule you can install merely for selecting the source address of an icmp errors.
Not sure yet how complicated that is, it would require a match in the rule lookup logic to specifically use another routing table when the source address for an icmp packet is generated. We already supply the protocol in the flow4 information, maybe this can be used plus another input/flag in the flowi4 struct?
I can see situations were it is necessary to actually select the source address depending of the interface.
Quoted 4 lines
Would it be OK to create a
/proc/sys/net/ipv4/icmp_errors_sourcewhere you could write an arbitrary IPv4 address into? And that would get used as the source address of ICMP errors?
Quoted 8 lines
My questions did contain the loopback interface as I first thought it a good source of a globally routable IPv4 address (at least in our case).
Secound thought: Instead of writing an IPv4 address to
/proc/sys/net/ipv4/icmp_errors_sourcewrite an interface name to that file and take the first global IPv4 address from that interface as source for ICMP errors. Then you could create a dummy interface for that use case, too.
I am not a fan of such implicit assumptions. I would prefer the direct specification of the source ip address over writing interface information to a procfs file.
Quoted 1 line
Still: Is it a good idea to do so?
I agree, there should be a solution for this as this is a common setup for BGP routers.
Bye,
Hannes
Am 10.01.2016 um 00:01 schrieb Hannes Frederic Sowa:
Quoted 3 lines
I am not a fan of such implicit assumptions. I would prefer the direct specification of the source ip address over writing interface information to a procfs file.
I tried that but as I am not a seasoned kernel hacker introducing a new sysctl including the validation of an IPv4 address was a bit too much.
Instead I created this patch (applicable against kernel version 3.2):
net/ipv4/icmp.c +18 -1
@@ -567,7 +567,7 @@ void icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info)rcu_read_lock ();if (rt_is_input_route (rt) && - net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr) + net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr ==1 ) dev =dev_get_by_index_rcu (net, rt->rt_iif);if (dev)
@@ -577,6 +577,23 @@ void icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info)rcu_read_unlock (); } +/* +* Set source in case of error reply +*/ + +if (icmp_pointers[type].error && net->ipv4.sysctl_icmp_errors_use_inbound_ifaddr ==2 ) { +struct net_device *dev =NULL ; +__be32 tmpaddr =0 ; + +rcu_read_lock (); + dev =dev_get_by_name_rcu (net,"lo" ); +if (dev) + tmpaddr =inet_select_addr (dev,0 ,RT_SCOPE_UNIVERSE ); +if (tmpaddr) + saddr = tmpaddr; +rcu_read_unlock (); + } + tos = icmp_pointers[type].error ? ((iph->tos &IPTOS_TOS_MASK ) |IPTOS_PREC_INTERNETCONTROL ) : iph->tos;
It currently works in my testbed (Debian wheezy based, therefor kernel 3.2).
Maybe there is someone more experienced with introducing new sysctl files and handling strings in kernel space than me that is able to pick up this idea and implement it properly.
Regards
Signature · 12 lines
Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Quoted 3 lines · Robert Sander
Is it a good idea to develop a kernel patch that makes it possible to select the first IPv4 address on dev lo with scope global as the source address for ICMP errors? Would that do any harm to the Internet at large?
I think the way to go is to have a ip rule option in which the admin can add routes which get consulted only by the icmp source address determination logic. I can assume that some other installations use other interface number logic or multiple public addresses. This kind of lookup seems to allow all possible lookup scenarios.
Thoughts?
Hannes
Quoted 4 lines · Robert Sander
We have the situation that our routers use RFC1918 addresses on their transfer networks (which should be quite common nowadays to save on public IPv4 addresses). ICMP errors are generated with RFC1918 source addresses and therefor never reach the original sender.
A follow-up to this request:
It is much easier to accomplish when settings routes with a source address that is a public IP like this:
ip route add default via 10.10.10.1 src 192.0.2.24
ICMP errors will then use 192.0.2.24 as the source address.
No need for a kernel patch. Thanks for all the input.
Regards
Signature · 12 lines
Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin