Why linux keeps connected routes when link goes down
Subsystems (2)
Hello!
Why kernel keeps connected routes when link is not in RUNNING state? It's not only linux behavior, some other OSes do so, but why?
It's interesting for me because I have a problem with setting up Quagga for OSPF support. When link goes down (!IFF_RUNNING), Quagga updates only its own routing table. I must use ifplugd to update routes in kernel (actually to delete connected route), but I think it is not a right way.
So the main question is: why linux keeps connected routes when link goes down?
PS: ``ip monitor'' reports about link fail after a minute or longer (on 8139too and 3c905x). On some Acorp cards it doesn't notice link failure. Is it a bad driver or crappy cards? On r8169 I have no problems
--
Konstantin
E-mail [email protected]
Jabber [email protected]
David Miller
maintainerFrom: Ханкин Константин <[email protected]>
Date: Thu, 19 Feb 2009 02:03:40 +0500
Quoted 2 lines
So the main question is: why linux keeps connected routes when link goes down?
Because the IP addresses are still assigned to the interface.
IP addresses are owned by the "host" rather than specific interfaces under Linux. So just bringing an interface down does not disable IP addresses configured to that interface.
If you want all the routes to go away, explicitly delete the IP addresses.
Quoted 11 lines · On Thu, 19 Feb 2009 02:03:40 +0500 Ханкин Константин <[email protected]>…
Hello!
Why kernel keeps connected routes when link is not in RUNNING state? It's not only linux behavior, some other OSes do so, but why?
It's interesting for me because I have a problem with setting up Quagga for OSPF support. When link goes down (!IFF_RUNNING), Quagga updates only its own routing table. I must use ifplugd to update routes in kernel (actually to delete connected route), but I think it is not a right way.
So the main question is: why linux keeps connected routes when link goes down?
PS: ``ip monitor'' reports about link fail after a minute or longer (on 8139too and 3c905x). On some Acorp cards it doesn't notice link failure. Is it a bad driver or crappy cards? On r8169 I have no problems
There are patches in Vyatta version of Quagga to what you want. They have been submitted to quagga-developers but the community process on quagga is much slower than the kernel.
Quoted 3 lines
IP addresses are owned by the "host" rather than specific interfaces under Linux. So just bringing an interface down does not disable IP addresses configured to that interface.
Well. Why routes are not in kernel table when interface is DOWN? And why routes are still alive if interface is UP, but not RUNNING?
I think it's right when I can't send anything through interface when there is no link. But now I have no link and a route to some network with the highest metric
--
Konstantin
E-mail [email protected]
Jabber [email protected]
Quoted 8 lines · David Miller
Because the IP addresses are still assigned to the interface.
IP addresses are owned by the "host" rather than specific interfaces under Linux. So just bringing an interface down does not disable IP addresses configured to that interface.
If you want all the routes to go away, explicitly delete the IP addresses.
An interface IP address and the router to the connected subnet are not the same thing. Keep the IP address but drop the route to the subnet you can no longer reach.
By elliminating the route to stuff you can no longer reach, quagga can do its job to provide an alternative way to reach the subnet. It can not do so as long as the kernel insist on providing a now useless route.
Signature · 1 line
Len Sorensen
David Miller
maintainerFrom: [email protected] (Lennart Sorensen)
Date: Fri, 20 Feb 2009 12:19:12 -0500
Quoted 2 lines
By elliminating the route to stuff you can no longer reach, quagga can do its job to provide an alternative way to reach the subnet.
This is not necessarily true.
The subnet might be reachable by one of the other interfaces. Just the same as someone might be able to reach the down interface's IP address via another one of our interfaces.
The route is tied to the IP address, and the route is therefore created and destroyed alongside the IP address.
As long as the IP address is active and associated with the system, so is the route.
Quoted 5 lines · David Miller
This is not necessarily true.
The subnet might be reachable by one of the other interfaces. Just the same as someone might be able to reach the down interface's IP address via another one of our interfaces.
If you assign IPs for the same subnet to two different interfaces in linux, the results are not particularly useful. The route contains the interface associated with the creation of the route, and since it doesn't go away even if you had the same route for two interfaecs, it will never use the second one.
Quoted 2 lines
The route is tied to the IP address, and the route is therefore created and destroyed alongside the IP address.
That's how it works now. It wasn't how it worked in linux a long time ago. Long ago you had to create the route yourself in userspace (which was actually to some extent better than the current setup). The current method is simpler to understand for your typical user though.
Quoted 2 lines
As long as the IP address is active and associated with the system, so is the route.
Exactly, and that's the problem when trying to be a router. You still want to be able to reach the system through any of the IPs assigned to it, and you would prefer to still be able to reach any networks through alternate routes if they exist. The way linux, BSD and other unix systems behave is counter productive to routing, and hence not the way real router OSs work.
Fortunately patching some sense into the linux kernel isn't very hard.
Signature · 1 line
Len Sorensen
Quoted 1 line
Fortunately patching some sense into the linux kernel isn't very hard.
Can I hope that it will be done some day?
Linux is used as router in many places, such behavior is a suicide. Of course, one can use ifplugd (so do I), but it's a crap for network OS, don't you think so?
Who made linux network subsystem? Can Linus Torvalds say some words about this problem?
I'm not a developer, but network administrator. So I can't make a patch. I can only find a solution or make it. So please make my work a bit easier
--
Konstantin
E-mail [email protected]
Jabber [email protected]
Quoted 1 line · ???????????? ????????????????????
Can I hope that it will be done some day?
I don't have much hope that the kernel will ever be changed to behave as desired for a router.
Quoted 3 lines
Linux is used as router in many places, such behavior is a suicide. Of course, one can use ifplugd (so do I), but it's a crap for network OS, don't you think so?
Well you will need some user space support to handle static routes, unless you want to make major changes to the kernel route management. We use zebra to manage static routes when the link goes up or down, with the kernel simply patched to delete the subnet route when an interface goes down, and re add it when it comes back up. All other routes are handled by zebra.
With the current patch I am using all routes assigned to an interface goes away when the link goes down, but the kernel only recreates the directly connected network's route when it comes up. Any other static routes have to be recreated by user space, hence the user of zebra for that.
Better and more friendly would be if routes could be marked inactive and ignored by the kernel for routing decisions when a link went down. That way multiple routes with different metrics could be used with the best one who's link is up would be in use. I find this too much to try and put into the kernel though, so I will stick to zebra for that task.
Quoted 5 lines
Who made linux network subsystem? Can Linus Torvalds say some words about this problem?
I'm not a developer, but network administrator. So I can't make a patch. I can only find a solution or make it. So please make my work a bit easier
Signature · 1 line
Len Sorensen
Quoted 22 lines · On Mon, 23 Feb 2009 16:14:11 -0500 [email protected] (Lennart…
Quoted 1 line · ???????????? ????????????????????
Can I hope that it will be done some day?
I don't have much hope that the kernel will ever be changed to behave as desired for a router.
Quoted 3 lines
Linux is used as router in many places, such behavior is a suicide. Of course, one can use ifplugd (so do I), but it's a crap for network OS, don't you think so?
Well you will need some user space support to handle static routes, unless you want to make major changes to the kernel route management. We use zebra to manage static routes when the link goes up or down, with the kernel simply patched to delete the subnet route when an interface goes down, and re add it when it comes back up. All other routes are handled by zebra.
With the current patch I am using all routes assigned to an interface goes away when the link goes down, but the kernel only recreates the directly connected network's route when it comes up. Any other static routes have to be recreated by user space, hence the user of zebra for that.
Quoted 12 lines
Better and more friendly would be if routes could be marked inactive and ignored by the kernel for routing decisions when a link went down. That way multiple routes with different metrics could be used with the best one who's link is up would be in use. I find this too much to try and put into the kernel though, so I will stick to zebra for that task.
Quoted 5 lines
Who made linux network subsystem? Can Linus Torvalds say some words about this problem?
I'm not a developer, but network administrator. So I can't make a patch. I can only find a solution or make it. So please make my work a bit easier
Like I said already, it is possible to fix Quagga; we have done it the patches are available. unfortunately quagga is without a real software maintenance infrastructure at this point.
David Miller
maintainerFrom: [email protected] (Lennart Sorensen)
Date: Mon, 23 Feb 2009 09:38:30 -0500
Quoted 8 lines · David Miller
Quoted 2 lines
The route is tied to the IP address, and the route is therefore created and destroyed alongside the IP address.
That's how it works now. It wasn't how it worked in linux a long time ago. Long ago you had to create the route yourself in userspace (which was actually to some extent better than the current setup). The current method is simpler to understand for your typical user though.
I don't remember Linux ever not creating a subnet route when an IP address was added to an interface. And I also don't remember Linux ever deleting an IP address on interface down.
Maybe it happened back in 0.99pl5 times, but we're talking at least 10 years ago.
David Miller
maintainerFrom: Ханкин Константин <[email protected]>
Date: Mon, 23 Feb 2009 23:02:55 +0500
Quoted 3 lines
Linux is used as router in many places, such behavior is a suicide. Of course, one can use ifplugd (so do I), but it's a crap for network OS, don't you think so?
The default behavior for a general purpose operating system is to increase the likelyhood of successful communication.
And the way to maximize successful communication is to associate addresses with the host rather than the interface.
This decision was made at least 10 years ago, and if you think what we have now is broken just imagine how much struff would explode if we changed things.
Quoted 7 lines · David Miller
From: [email protected] (Lennart Sorensen)
Date: Mon, 23 Feb 2009 09:38:30 -0500Quoted 4 lines · David Miller
Quoted 2 lines
The route is tied to the IP address, and the route is therefore created and destroyed alongside the IP address.
Quoted 8 lines
Quoted 4 lines
That's how it works now. It wasn't how it worked in linux a long time ago. Long ago you had to create the route yourself in userspace (which was actually to some extent better than the current setup). The current method is simpler to understand for your typical user though.
I don't remember Linux ever not creating a subnet route when an IP address was added to an interface. And I also don't remember Linux ever deleting an IP address on interface down.
Not IP address. And no it never deleted the route either.
Quoted 2 lines
Maybe it happened back in 0.99pl5 times, but we're talking at least 10 years ago.
I did run 0.99something on my first linux system, so I remember the transition happening at some point. I remember distribution's network scripts creating routes after assigning the IP to an interface. I don't remember how long ago that was though.
Now it had nothing to do with link state, it was simply a matter that setting an ip and netmask on an interface did not create a route the way newer kernels have done (where newer probably covers most years of linux kernels). All route management was userspace's job back then. In the case of running quagga and trying to do link detect and route management based on link state, having the one route the kernel auto creates there is a pain in the ass and it would be nice if it wasn't there based on some sysctl option. I haven't found one though. I would be happy to have zebra be responsible for the network route for all interfaces for consistency. That way all route management without exception would be a userspace problem. This would of course not be the default, since for most normal users this is not the desired behaviour, but for an advanced router it very much is desired behaviour.
Signature · 1 line
Len Sorensen