Routing tables associated with VLANs dissappear when parent ethX down/up
Subsystems (2)
Hello!
Found something strange today.
Suppose I have eth2 with VLAN 2 (eth2.2) associated with it. I add a routing table that is associated with eth2.2 (table 76 in my case).
Now, I run:
ifconfig eth2 down
ifconfig eth2 up
All of the routes in routing table 76 are now gone! I can understand how it might be useful to remove the routes from the eth2 table (as is also done), but it seems pretty extreme to muck with the VLAN device's tables as well.
[root@lanforge-33-46 local]# ip route show table 76 27.1.1.0/24 via 27.1.1.2 dev eth2.2
default via 27.1.1.1 dev eth2.2
[root@lanforge-33-46 local]# ifconfig eth2.2
eth2.2 Link encap:Ethernet HWaddr 00:03:2D:08:33:47
inet addr:27.1.1.2 Bcast:27.1.1.255 Mask:255.255.255.0 UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
[root@lanforge-33-46 local]# ifconfig eth2 down;ifconfig eth2 up [root@lanforge-33-46 local]# ip route show table 76
Signature · 2 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
For consideration, this patch seems to work for me. I'm not sure why we ever listed to these events. I've only tested on a NIC that doesn't support hw-accel at the moment..will test with e1000 later.
Thanks,
Ben
Signature · 2 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
net/8021q/vlan.c +6 -0
@@ -615,6 +615,11 @@ static int vlan_device_event(struct notifier_block *unused, unsigned long event, }break ; +#if 0 +/* Don't propagate management state from base dev to VLANs. If you do this, +* then if you 'ifconfig eth0 down; ifconfig eth0 up', you also lose all the +* routes for eth0.* VLANs. --Ben +*/ case NETDEV_DOWN :/* Put all VLANs for this dev in the down state too. */ for (i =0 ; i <VLAN_GROUP_ARRAY_LEN ; i++) {
@@ -644,6 +649,7 @@ static int vlan_device_event(struct notifier_block *unused, unsigned long event, dev_change_flags(vlandev, flgs | IFF_UP); }break ; +#endif case NETDEV_UNREGISTER :/* Delete all VLANs for this dev. */
Quoted 7 lines · On Wed, 21 Nov 2007 11:51:43 -0800 Ben Greear <[email protected]> wrote:
For consideration, this patch seems to work for me. I'm not sure why we ever listed to these events. I've only tested on a NIC that doesn't support hw-accel at the moment..will test with e1000 later.
Thanks,
Ben
But then if you are doing bonding or bridging of vlan's and you bring down the root network device, the upper layer is not notified (for failover).
Signature · 1 line
Stephen Hemminger <[email protected]>
Quoted 8 lines · Stephen Hemminger
Quoted 4 lines · On Wed, 21 Nov 2007 11:51:43 -0800 Ben Greear <[email protected]> wrote:
For consideration, this patch seems to work for me. I'm not sure why we ever listed to these events. I've only tested on a NIC that doesn't support hw-accel at the moment..will test with e1000 later.
Quoted 8 lines
Quoted 4 lines
Thanks,
BenBut then if you are doing bonding or bridging of vlan's and you bring down the root network device, the upper layer is not notified (for failover).
operstate should be enough for this I guess. Ben, what does iproute show for the vlan device when the lower device is down?
Quoted 14 lines · Patrick McHardy
Quoted 13 lines · Stephen Hemminger
Quoted 8 lines · On Wed, 21 Nov 2007 11:51:43 -0800 Ben Greear <[email protected]> wrote:
For consideration, this patch seems to work for me. I'm not sure why we ever listed to these events. I've only tested on a NIC that doesn't support hw-accel at the moment..will test with e1000 later.
Thanks,
Ben
Quoted 6 lines
Quoted 3 lines
But then if you are doing bonding or bridging of vlan's and you bring down the root network device, the upper layer is not notified (for failover).
operstate should be enough for this I guess. Ben, what does iproute show for the vlan device when the lower device is down?
It looks like it knows, assuming M-DOWN is useful information. Eth2 is un-plugged, by the way.
[root@lanforge-33-46 ~]# ifconfig eth2 up
[root@lanforge-33-46 ~]# ip link show eth2.2
125: eth2.2@eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue
link/ether 00:03:2d:08:33:47 brd ff:ff:ff:ff:ff:ff
[root@lanforge-33-46 ~]# ifconfig eth2 down
[root@lanforge-33-46 ~]# ip link show eth2.2
125: eth2.2@eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP,M-DOWN> mtu 1500 qdisc noqueue
link/ether 00:03:2d:08:33:47 brd ff:ff:ff:ff:ff:ff[root@lanforge-33-46 ~]#
Thanks,
Ben
Signature · 2 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 16 lines · Ben Greear
Quoted 8 lines · Patrick McHardy
Quoted 4 lines · Stephen Hemminger
But then if you are doing bonding or bridging of vlan's and you bring down the root network device, the upper layer is not notified (for failover).
operstate should be enough for this I guess. Ben, what does iproute show for the vlan device when the lower device is down?
It looks like it knows, assuming M-DOWN is useful information. Eth2 is un-plugged, by the way.
[root@lanforge-33-46 ~]# ifconfig eth2 up
[root@lanforge-33-46 ~]# ip link show eth2.2
125: eth2.2@eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue
Quoted 6 lines
link/ether 00:03:2d:08:33:47 brd ff:ff:ff:ff:ff:ff [root@lanforge-33-46 ~]# ifconfig eth2 down [root@lanforge-33-46 ~]# ip link show eth2.2 125: eth2.2@eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP,M-DOWN> mtu 1500 qdisc noqueue link/ether 00:03:2d:08:33:47 brd ff:ff:ff:ff:ff:ff[root@lanforge-33-46 ~]#
That comes from iproute itself, but the missing LOWER-UP flag indicates it and that should be enough for bridging and bonding. I'm unsure about this though since its still a big difference in userspace visible behaviour, people might just as well manually configure failover once routing disappears or the device goes down, or just have routing fall through to different routes. All this wouldn't work anymore.
Maybe we can make this optional somehow without too much uglyness?
Quoted 32 lines · Patrick McHardy
Quoted 16 lines · Ben Greear
Quoted 8 lines · Patrick McHardy
Quoted 4 lines · Stephen Hemminger
But then if you are doing bonding or bridging of vlan's and you bring down the root network device, the upper layer is not notified (for failover).
operstate should be enough for this I guess. Ben, what does iproute show for the vlan device when the lower device is down?
It looks like it knows, assuming M-DOWN is useful information. Eth2 is un-plugged, by the way.
[root@lanforge-33-46 ~]# ifconfig eth2 up
[root@lanforge-33-46 ~]# ip link show eth2.2
125: eth2.2@eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueueQuoted 6 lines
link/ether 00:03:2d:08:33:47 brd ff:ff:ff:ff:ff:ff [root@lanforge-33-46 ~]# ifconfig eth2 down [root@lanforge-33-46 ~]# ip link show eth2.2 125: eth2.2@eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP,M-DOWN> mtu 1500 qdisc noqueue link/ether 00:03:2d:08:33:47 brd ff:ff:ff:ff:ff:ff[root@lanforge-33-46 ~]#
That comes from iproute itself, but the missing LOWER-UP flag indicates it and that should be enough for bridging and bonding. I'm unsure about this though since its still a big difference in userspace visible behaviour, people might just as well manually configure failover once routing disappears or the device goes down, or just have routing fall through to different routes. All this wouldn't work anymore.
Quoted 1 line
Maybe we can make this optional somehow without too much uglyness?
I'm fine with that..we can just add a new vlan-device flag similar to the reorder-header flag.
With the current code, on 'UP' of the underlying
code, all of the VLANs will also go UP, even if the user had previously put them DOWN. That seems like it could be quite dangerous/unexpected to me..but I guess it's required if we are going to automatically DOWN them...
One other thought: Maybe we could tell a small lie and say that we have NO-CARRIER on the VLAN when the underlying device is down OR has no carrier?
That way we keep normal link up/down semantics w/out having to change the admin state of the VLANs...
Thanks,
Ben
Signature · 2 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 10 lines · Ben Greear
Quoted 9 lines · Patrick McHardy
That comes from iproute itself, but the missing LOWER-UP flag indicates it and that should be enough for bridging and bonding. I'm unsure about this though since its still a big difference in userspace visible behaviour, people might just as well manually configure failover once routing disappears or the device goes down, or just have routing fall through to different routes. All this wouldn't work anymore.
Quoted 4 lines
Quoted 1 line
Maybe we can make this optional somehow without too much uglyness?
I'm fine with that..we can just add a new vlan-device flag similar to the reorder-header flag.
An alternative to this would be something like Julian Anastasov static routes patch. Not sure if it has ever been considered for merging, but its a cleaner way than doing per-device hacks.
Quoted 5 lines
With the current code, on 'UP' of the underlying
code, all of the VLANs will also go UP, even if the user had previously put them DOWN. That seems like it could be quite dangerous/unexpected to me..but I guess it's required if we are going to automatically DOWN them...
Yeah, I too never liked this behaviour.
Quoted 6 lines
One other thought: Maybe we could tell a small lie and say that we have NO-CARRIER on the VLAN when the underlying device is down OR has no carrier?
That way we keep normal link up/down semantics w/out having to change the admin state of the VLANs...
Thats pretty much what the operstate is doing, it should go to IF_OPER_LOWERLAYERDOWN when the lower device is down. But as I said above, people could actually rely on routes disappearing.