primary and secondary ip addresses
Subsystems (4)
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
[Cc'ing netdev, since nobody replied since november]
Quoted 8 lines · Henrik Nordstrom
Quoted 5 lines · Neil Horman
Can anyone tell me why when removing a primary ip address from an interface, all of its associated secondary ip addresses (those addresses on the same subnet), are also removed? Theres a comment in the kernel thats pretty clear on the subject indicating it needs to be done, but I don't quite grasp the reasoning behind why.
Quoted 7 lines
A more valid question is why one can not add more than one primary in the same subnet.
ip addr add x.x.x.x/y dev eth0 primary
would make sense to me, clearly indicating that these addresses are maintained separately.
agreed.
Quoted 2 lines
or put in another way why the addresses automatically become secondary addresses to the first added address.
... and why does removing the primary address remove all secondary addresses. This makes it complicated if you have one interface with multiple ip addresse, that change over time (failover, let's say.) You don't know yet, which address you need to remove, because you don't know which of your peers fails.
So you have all this complicated userspace magic that checks whether the address that is about to be deleted is the primary, and if yes, re-add all the other addresses. If if is a secondary, you're happy and only need to delete that one.
Can anyone comment what the idea of all this was?
Thanks.
Signature · 3 lines
- Harald Welte <[email protected]> http://www.gnumonks.org/ ============================================================================ Programming is like sex: One mistake and you have to support it your lifetime
Quoted 11 lines · Harald Welte
[Cc'ing netdev, since nobody replied since november]
Quoted 7 lines · Henrik Nordstrom
A more valid question is why one can not add more than one primary in the same subnet.
ip addr add x.x.x.x/y dev eth0 primary
would make sense to me, clearly indicating that these addresses are maintained separately.
agreed.
And why I can't even choose which address is primary?
Quoted 9 lines
Quoted 2 lines
or put in another way why the addresses automatically become secondary addresses to the first added address.
... and why does removing the primary address remove all secondary addresses. This makes it complicated if you have one interface with multiple ip addresse, that change over time (failover, let's say.) You don't know yet, which address you need to remove, because you don't know which of your peers fails.
Quoted 6 lines
So you have all this complicated userspace magic that checks whether the address that is about to be deleted is the primary, and if yes, re-add all the other addresses. If if is a secondary, you're happy and only need to delete that one.
Can anyone comment what the idea of all this was?
This reminds me related issue ... Actually there is concept in Junos software I'd love to see in Linux as well. There is "preferred address" and "primary address" in Junos. Quoting Junos documentation:
**************************************************************************
Primary
=======
Configure this address to be the primary address of the protocol on the interface. If the logical unit has more than one address, the primary address is used by default as the source address when packets originate from the interface and the destination does not indicate the subnet (ie. multicast destination for example).
Default
For unicast traffic, the primary address is the lowest non-127 preferred address on the unit.
Preferred
=========
Configure this address to be the preferred address on the interface. If you configure more than one address on the same subnet, the preferred source address is chosen by default as the source address when you originate packets to destinations on the subnet.
Default
The lowest numbered address on the subnet is the preferred address.
**************************************************************************
Would it be overkill for Linux? From the routers point of view it does make perfect sense. And of course you can manually override defaults in Junos.
Signature · 3 lines
Hasso Tepper Elion Enterprises Ltd. WAN administrator
Quoted 1 line · Hasso Tepper
And why I can't even choose which address is primary?
It is the first you add in a subnet.
Quoted 8 lines
Primary
=======Configure this address to be the primary address of the protocol on the interface. If the logical unit has more than one address, the primary address is used by default as the source address when packets originate from the interface and the destination does not indicate the subnet (ie. multicast destination for example).
This is what the primary addresses is used for, indirectly via the automatically created route entries.
Quoted 7 lines
Preferred
=========Configure this address to be the preferred address on the interface. If you configure more than one address on the same subnet, the preferred source address is chosen by default as the source address when you originate packets to destinations on the subnet.
What is the difference from primary here?
Broadcast traffic using the primary, the rest using preferred?
Regards
Henrik
Quoted 4 lines · Henrik Nordstrom
Quoted 1 line · Hasso Tepper
And why I can't even choose which address is primary?
It is the first you add in a subnet.
Yes, but to change the primary I have remove old primary (and therefore all secondaries) and assign new primary to the interface and then all secondaries etc. I can't just tell "make this address which is secondary at the moment, primary".
Quoted 11 lines
Quoted 8 lines
Primary
=======Configure this address to be the primary address of the protocol on the interface. If the logical unit has more than one address, the primary address is used by default as the source address when packets originate from the interface and the destination does not indicate the subnet (ie. multicast destination for example).
This is what the primary addresses is used for, indirectly via the automatically created route entries.
No. There is only one primary address per interface and it is used if destination address doesn't indicate which source address to use. Ie. if packet is sent over eth1 to the address 224.0.0.6 and eth1 has addresses 192.168.0.1/24 and 10.10.10.1/24 which one choose for source address?
Quoted 9 lines
Quoted 7 lines
Preferred
=========Configure this address to be the preferred address on the interface. If you configure more than one address on the same subnet, the preferred source address is chosen by default as the source address when you originate packets to destinations on the subnet.
What is the difference from primary here?
This has same meaning as primary in the Linux. There is as many preferred addresses on the interfaces as there is subnets - every subnet has one preferred address. Preferred aadress is used if next hop indicates subnet - if packet is sent to the 192.168.0.2, 192.168.0.1 is used as source; if packet is sent to the 10.10.10.28, 10.10.10.1 is used as source.
Signature · 3 lines
Hasso Tepper Elion Enterprises Ltd. WAN administrator
Quoted 2 lines · Hasso Tepper
No. There is only one primary address per interface and it is used if destination address doesn't indicate which source address to use.
In the linux world your first primary is used for this purpose if I am not mistaken.
But this shouldn't happen much in any sane network, and when it does it is not that unlikely the return traffic won't know how to reach you correctly..
Regards
Henrik
Yes, using the same concept used in Junos would make things much easier. On top of this, when I change the primary IP, it takes about 500ms to see the change applied which is critical for real time applications. Using more than one primary IP would solve this as well. Also it seems that if I have a primary address and a secondary one (alias), even though I make the primary address invalid (trying to force the kernel to use the secondary first), the kernel still tries the primary first regardless of its validity (I set the primary IP to 0.0.0.0). Anybody has any comments about this?
Can anyone point out to me where is the code for primary/secondary IPs in the kernel?
Thanks a lot,
Andrea
Quoted 54 lines · Hasso Tepper
Quoted 9 lines · Henrik Nordstrom
Quoted 3 lines · Hasso Tepper
And why I can't even choose which address is primary?
It is the first you add in a subnet.
Yes, but to change the primary I have remove old primary (and therefore all secondaries) and assign new primary to the interface and then all secondaries etc. I can't just tell "make this address which is secondary at the moment, primary".
Quoted 10 lines
Quoted 10 lines
Primary
=======Configure this address to be the primary address of the protocol on the interface. If the logical unit has more than one address, the primary address is used by default as the source address when packets originate from the interface and the destination does not indicate the subnet (ie. multicast destination for example).
Quoted 4 lines
This is what the primary addresses is used for, indirectly via the automatically created route entries.
No. There is only one primary address per interface and it is used if destination address doesn't indicate which source address to use. Ie. if packet is sent over eth1 to the address 224.0.0.6 and eth1 has addresses 192.168.0.1/24 and 10.10.10.1/24 which one choose for source address?
Quoted 12 lines
Quoted 9 lines
Preferred
=========Configure this address to be the preferred address on the interface. If you configure more than one address on the same subnet, the preferred source address is chosen by default as the source address when you originate packets to destinations on the subnet.
What is the difference from primary here?
This has same meaning as primary in the Linux. There is as many preferred addresses on the interfaces as there is subnets - every subnet has one preferred address. Preferred aadress is used if next hop indicates subnet - if packet is sent to the 192.168.0.2, 192.168.0.1 is used as source; if packet is sent to the 10.10.10.28, 10.10.10.1 is used as source.
Quoted 3 lines · Andrea G Forte
Yes, using the same concept used in Junos would make things much easier. On top of this, when I change the primary IP, it takes about 500ms to see the change applied which is critical for real time applications. Using more than one primary IP would solve this as well.
Also it seems that if I have a primary address and a secondary one (alias), even though I make the primary address invalid (trying to force the kernel to use the secondary first), the kernel still tries the primary first regardless of its validity (I set the primary IP to 0.0.0.0).
Which source IP is used by the kernel is determined primary by your routing tables.
The requirements for an IP address to be allowed to be used in the routing table is that the IP address does exists on any of your interfaces, either as primary or secondary.
When you add/delete a primary address to a interface the kernel automatically adds/deletes routes accordingly, including source IP address selection.
If the routing table does not have information about which source IP address to use for this traffic then the kernel searches the interface for a valid primary address.
Regards
Henrik
Quoted 6 lines
Which source IP is used by the kernel is determined primary by your routing tables.
The requirements for an IP address to be allowed to be used in the routing table is that the IP address does exists on any of your interfaces, either as primary or secondary.
When you add/delete a primary address to a interface the kernel automatically adds/deletes routes accordingly, including source IP address selection.
This does not help, since if I want to use my secondary IP address instead of my primary, I cannot delete the primary otherwise all of my secondary IPs are lost as well (and since I can only have only one primary IP address).
Quoted 2 lines
If the routing table does not have information about which source IP address to use for this traffic then the kernel searches the interface for a valid primary address.
I update all the routing entries and eventually things start to work again. The problem is that:
-If I use a secondary IP and try to invalidate the primary (i.e. by removing its routing table entry), it takes about 500ms for the actual change (data packets sent on the secondary IP instead of the primary) to take effect. -If I try to update the primary address directly without creating any secondary IP, then it still takes about 300ms for the change to take place.
I honestly do not understand what harm could do to have more than one primary address, especially on different subnets.
Cheers,
Andrea
Quoted 2 lines
Regards
Henrik
Quoted 1 line · Andrea G Forte
This does not help, since if I want to use my secondary IP address instead of my primary, I cannot delete the primary otherwise all of my secondary IPs are lost as well (and since I can only have only one primary IP address).
Why change the primary address? What is wrong with simply changing the route to use the other source IP?
Quoted 2 lines
-If I use a secondary IP and try to invalidate the primary (i.e. by removing
its routing table entry), it takes about 500ms for the actual change (data packets sent on the secondary IP instead of the primary) to take effect.
This is most likely the routing cache or something.
Quoted 1 line
I honestly do not understand what harm could do to have more than one primary address, especially on different subnets.
How it works today is that the first IP you add in a subnet becomes a primary, any additional IPs you add in the same subnet becomes secondary. You can have any number of primary IPs with each any number of secondary IPs, the primary IPs just can't be in the same subnet.
Regards
Henrik
Quoted 3 lines
How it works today is that the first IP you add in a subnet becomes a primary, any additional IPs you add in the same subnet becomes secondary. You can have any number of primary IPs with each any number of secondary IPs, the primary IPs just can't be in the same subnet.
Well, when I assign a new IP (for a new subnet) to the same interface (using ip route add...), even though the routing table is updated, it still takes about 500ms for the change to happen. This new IP should be detected as primary, right? Also, am I right to say that any IP address assigned to an alias interface is a secondary IP? Do you know where the code for the routing cache is?
Thanks for your help,
Andrea
Quoted 2 lines
Regards
Henrik
Quoted 1 line · Andrea G Forte
Well, when I assign a new IP (for a new subnet) to the same interface (using ip route add...), even though the routing table is updated, it still takes about 500ms for the change to happen. This new IP should be detected as primary, right?
If it is on a new subnet yes.
Quoted 1 line
Also, am I right to say that any IP address assigned to an alias interface is a secondary IP?
It is irrelevant if you label the address or not. The "alias" labels are just a compatibility layer to work with old userspace programs (mainly ifconfig), not interfaces of their own.
See /sbin/ip addr for all the details.
Regards
Henrik
Quoted 8 lines · Henrik Nordstrom
Quoted 4 lines · Andrea G Forte
This does not help, since if I want to use my secondary IP address instead of my primary, I cannot delete the primary otherwise all of my secondary IPs are lost as well (and since I can only have only one primary IP address).
Why change the primary address? What is wrong with simply changing the route to use the other source IP?
There is no support for it in most of user space software. None of the routing protocols suites support it etc.
Signature · 3 lines
Hasso Tepper Elion Enterprises Ltd. WAN administrator
Hello Hasso and Andrea,
We've gotten a little far afield from Neil Horman's initial question about why there are primary and secondary IPs, and I can't address your concern Andrea about the (route cache?) 500ms latency between the time that an address is added (or removed) from an interface and the time that the address is actually used. Even so, the Linux routing code allows the kernel to suggest an IP with the "src" keyword.
: > Why change the primary address? What is wrong with simply changing the : > route to use the other source IP?
:
: There is no support for it in most of user space software.
: None of the routing protocols suites support it etc.
Though some software provides support for explicit configuration of source address for initiated sockets, you can use INADDR_ANY and let the kernel perform source address selection for you.
Linux select an IP based on the routing table. [0] Example:
# ip route show 192.168.90.0/24 192.168.90.0/24 dev eth0 scope link src 192.168.90.250 # ip route change 192.168.88.0/24 dev eth0 scope link src $SECONDARY
If you want to be fancy about it, you can have a higher preference routing table (make sure there's an entry in /etc/iproute2/rt_tables for $SECONDARY_TABLE). Then you can add and remove tables in this routing table instead of changing the route in the main routing table.
# ip rule add prio table $SECONDARY_TABLE # ip route add table $SECONDARY_TABLE $DESTNET dev $REALDEV src $SECONDARY
Best of luck!
-Martin
[0] http://linux-ip.net/gl/ip-cref/node155.html
Signature · 1 line
Martin A. Brown --- SecurePipe, Inc. --- [email protected]
Ühel kenal päeval (reede 17 detsember 2004 20:37) kirjutas Martin A. Brown:
Quoted 8 lines
Hello Hasso and Andrea,
We've gotten a little far afield from Neil Horman's initial question about why there are primary and secondary IPs, and I can't address your concern Andrea about the (route cache?) 500ms latency between the time that an address is added (or removed) from an interface and the time that the address is actually used. Even so, the Linux routing code allows the kernel to suggest an IP with the "src" keyword.
I know.
Quoted 9 lines
: > Why change the primary address? What is wrong with simply changing : > the route to use the other source IP?
:
: There is no support for it in most of user space software. : None of the routing protocols suites support it etc.Though some software provides support for explicit configuration of source address for initiated sockets, you can use
INADDR_ANYand let the kernel perform source address selection for you.
Well, that's the point - we want to have full control over this selection process without doing fancy things in user space.
Quoted 6 lines
Linux select an IP based on the routing table. [0] Example:
# ip route show 192.168.90.0/24 192.168.90.0/24 dev eth0 scope link src 192.168.90.250 # ip route change 192.168.88.0/24 dev eth0 scope link src $SECONDARY
Quoted 8 lines
If you want to be fancy about it, you can have a higher preference routing table (make sure there's an entry in
/etc/iproute2/rt_tablesfor $SECONDARY_TABLE). Then you can add and remove tables in this routing table instead of changing the route in the main routing table.# ip rule add prio table $SECONDARY_TABLE # ip route add table $SECONDARY_TABLE $DESTNET dev $REALDEV src $SECONDARY
All these tricks don't help if you are using dynamic routing.
Signature · 3 lines
Hasso Tepper Elion Enterprises Ltd. WAN administrator
Quoted 1 line · Hasso Tepper
All these tricks don't help if you are using dynamic routing.
Are you seriously saying you are doing dynamic routing for your locally attached lans?
source address assignment for routed traffic via gateways is automatically derived by the source address assignment for traffic addressed to the gateway itself. So even with routing protocols etc you can control the source address assignment simply by setting up routing to use the correct source address to speak to your gateways, the added routes will then inherit the intended source address.
But I am not sure about how you can control this on multicast or broadcast traffic in a reasonable manner, mostly because I rarely have to care about this on such traffic.
Regards
Henrik
Quoted 9 lines · Henrik Nordstrom
Quoted 1 line · Hasso Tepper
All these tricks don't help if you are using dynamic routing.
Are you seriously saying you are doing dynamic routing for your locally attached lans?
source address assignment for routed traffic via gateways is automatically derived by the source address assignment for traffic addressed to the gateway itself.
So? Router learns via rip that default route should go via 10.0.0.1 and it has 10.0.0.2/24 and 10.0.0.3/24 addresses on eth0.
Quoted 4 lines
So even with routing protocols etc you
can control the source address assignment simply by setting up routing to use the correct source address to speak to your gateways, the added routes will then inherit the intended source address.
No. It would be true in very trivial case only - ie. "use address I'm using for announcing these routes as gateway for these routes". All routing protocols can carry nexthop information. Moreover, addresses routing protocols use for transport don't have to have anything common with addresses used for routing. You can even carry IPv4 routing info with routing protocol using IPv6 for transport (ospfv3) or iso (is-is).
Signature · 3 lines
Hasso Tepper Elion Enterprises Ltd. WAN administrator
Quoted 6 lines · Martin A. Brown
Hello Hasso and Andrea,
We've gotten a little far afield from Neil Horman's initial question about why there are primary and secondary IPs, and I can't address your concern Andrea about the (route cache?) 500ms latency between the time that an address is added (or removed) from an interface and the time that the address is actually used. Even so, the Linux routing code allows the kernel to suggest an IP with the "src" keyword.
I apologize if I moved the topic too far from its original scope.
Quoted 9 lines
: > Why change the primary address? What is wrong with simply changing the : > route to use the other source IP?
: : There is no support for it in most of user space software. : None of the routing protocols suites support it etc.Though some software provides support for explicit configuration of source address for initiated sockets, you can use
INADDR_ANYand let the kernel perform source address selection for you.Linux select an IP based on the routing table. [0] Example:
Quoted 11 lines
# ip route show 192.168.90.0/24 192.168.90.0/24 dev eth0 scope link src 192.168.90.250 # ip route change 192.168.88.0/24 dev eth0 scope link src $SECONDARYIf you want to be fancy about it, you can have a higher preference routing table (make sure there's an entry in
/etc/iproute2/rt_tablesfor $SECONDARY_TABLE). Then you can add and remove tables in this routing table instead of changing the route in the main routing table.# ip rule add prio table $SECONDARY_TABLE # ip route add table $SECONDARY_TABLE $DESTNET dev $REALDEV src $SECONDARY
I will give it a try and see if I get any improvement.
However, as last question (after this I will stop moving this topic further away from its original scope), do you guys know where I can find the code related to the routing cache in the kernel? I would like to see if there is a kind of timer for the update of such a cache.
Thank you again for all of your help.
Andrea
Quoted 8 lines
Best of luck!
-Martin
David S. Miller
maintainerQuoted 4 lines · On Fri, 17 Dec 2004 10:58:10 -0500 Andrea G Forte <[email protected]>…
This does not help, since if I want to use my secondary IP address instead of my primary, I cannot delete the primary otherwise all of my secondary IPs are lost as well (and since I can only have only one primary IP address).
By definition, a secondary IP address on an interface is not to
be used as a source. It is the whole reason for the distinction
between primary and secondary IP addresses, and it is why all secondaries are deleted once the primary is removed (because there are no valid source addresses to choose from any longer, therefore IP valid communications are no longer possible).
Quoted 1 line · David S. Miller
By definition, a secondary IP address on an interface is not to be used as a source.
But you can, but adding a route with such address as source or applications excplicitly binding to this source address. And it is highly useful to be able to use different source addresses in the same subnet for different purposes.
Quoted 1 line
It is the whole reason for the distinction between primary and secondary IP addresses, and it is why all secondaries are deleted once the primary is removed (because there are no valid source addresses to choose from any longer, therefore IP valid communications are no longer possible).
Which is a false assumption in very many situations.
Regards
Henrik
Quoted 7 lines · Henrik Nordstrom
Quoted 2 lines · David S. Miller
By definition, a secondary IP address on an interface is not to be used as a source.
But you can, but adding a route with such address as source or applications excplicitly binding to this source address.
Even when not bound, a secondary address could be used as src if within a fitting scope+mask.
Quoted 3 lines
And it is
highly useful to be able to use different source addresses in the same subnet for different purposes.
You can.
You can also have multiple primary addresses each on different subnets and scope. And each primary can have multiple secondary addresses.
Quoted 6 lines
Quoted 4 lines
It is the whole reason for the distinction between primary and secondary IP addresses, and it is why all secondaries are deleted once the primary is removed (because there are no valid source addresses to choose from any longer, therefore IP valid communications are no longer possible).
Which is a false assumption in very many situations.
The operative term is "IP valid communications are no longer possible". When you attach an IP address for the first time on a port/interface thats a signal "IP communication using this device is now Valid". Its like ifconfiging up a device - but only for IP processing purposes. When you delete that IP address that created that signal i.e the primary address/first address attached, you are signaling "IP communication using this device is now no longer Valid". Thats why those secondary addresses are deleted. Someone please take note of this somewhere in some FAQ since it has been an issue of contention for a long time. A routing protocol implementation
MUST take the above into consideration.
Having said the above, I think it would make sense to have a "promotion" scheme so that in the case a primary address is deleted, one could promote the next secondary address in line. But that should be optional. Now where is the fireman who wants to do this? I could help cheering since i know the code.
cheers,
jamal
Quoted 3 lines · jamal
Having said the above, I think it would make sense to have a "promotion" scheme so that in the case a primary address is deleted, one could promote the next secondary address in line. But that should be optional.
Oh yes, please. This would save a lot of headache. I'm much in favour of such a proposal.
Quoted 2 lines
Now where is the fireman who wants to do this? I could help cheering since i know the code.
how would you think it fits best into the current netlink messages?
Quoted 2 lines
cheers,
jamal
Signature · 3 lines
- Harald Welte <[email protected]> http://www.gnumonks.org/ ============================================================================ Programming is like sex: One mistake and you have to support it your lifetime
* Harald Welte <[email protected]> 2004-12-19 22:41
Quoted 8 lines · jamal
Quoted 4 lines
Having said the above, I think it would make sense to have a "promotion" scheme so that in the case a primary address is deleted, one could promote the next secondary address in line. But that should be optional.
Oh yes, please. This would save a lot of headache. I'm much in favour of such a proposal.
Agreed, would be nice to have.
Quoted 4 lines
Quoted 2 lines
Now where is the fireman who wants to do this? I could help cheering since i know the code.
how would you think it fits best into the current netlink messages?
1) IFA_F_PROM_CAND flag and have inet_del_ifa* iterate over its secondary addresses and elect the first with the flag set.
2) IFA_PROM_PRIO TLV of type u32 holding a priority where 0 means no candiate. inet_del_ifa* iterates over its secondary addresses and elects the one with the highest prio as new primary address or deletes all addresses if none is found.
* respectively the equivalent function of the other address families.
Second variant requires more work but is more flexible so it's definitely my favourite. I'm willing to put some effort into this, I'm not familiar with all address families though.
I had something much simpler in mind.
Basically, promote the next one in line. This would be cleanly backward compatible and would be an improvement over whats there (however medievial it is). Let me see if i can whip something that at least compiles. Unfortunately i wont have time to chase it to completion of testing until around xmas when i have time off from work.
cheers,
jamal
Quoted 31 lines · Thomas Graf
* Harald Welte <[email protected]> 2004-12-19 22:41
Quoted 8 lines · jamal
Quoted 4 lines
Having said the above, I think it would make sense to have a "promotion" scheme so that in the case a primary address is deleted, one could promote the next secondary address in line. But that should be optional.
Oh yes, please. This would save a lot of headache. I'm much in favour of such a proposal.
Agreed, would be nice to have.
Quoted 4 lines
Quoted 2 lines
Now where is the fireman who wants to do this? I could help cheering since i know the code.
how would you think it fits best into the current netlink messages?
1)
IFA_F_PROM_CANDflag and have inet_del_ifa* iterate over its secondary addresses and elect the first with the flag set.2)
IFA_PROM_PRIOTLV of type u32 holding a priority where 0 means no candiate. inet_del_ifa* iterates over its secondary addresses and elects the one with the highest prio as new primary address or deletes all addresses if none is found.* respectively the equivalent function of the other address families.
Second variant requires more work but is more flexible so it's definitely my favourite. I'm willing to put some effort into this, I'm not familiar with all address families though.
Harald,
My (stoopid) ISP doesnt like your email address. In any case, attached patch of what i was alluding to. Maybe be missing some things. Compiles - not tested and extremely dangerous becdause of side effects to arp and forwarding - so needs a lot of testing.
damn spent my tim hortons coffee on this patch and too cold to go out.
cheers,
jamal
Quoted 7 lines · jamal
I had something much simpler in mind.
Basically, promote the next one in line. This would be cleanly backward compatible and would be an improvement over whats there (however medievial it is). Let me see if i can whip something that at least compiles. Unfortunately i wont have time to chase it to completion of testing until around xmas when i have time off from work.
Quoted 35 lines
cheers,
jamalQuoted 31 lines · Thomas Graf
* Harald Welte <[email protected]> 2004-12-19 22:41
Quoted 8 lines · jamal
Quoted 4 lines
Having said the above, I think it would make sense to have a "promotion" scheme so that in the case a primary address is deleted, one could promote the next secondary address in line. But that should be optional.
Oh yes, please. This would save a lot of headache. I'm much in favour of such a proposal.
Agreed, would be nice to have.
Quoted 4 lines
Quoted 2 lines
Now where is the fireman who wants to do this? I could help cheering since i know the code.
how would you think it fits best into the current netlink messages?
1)
IFA_F_PROM_CANDflag and have inet_del_ifa* iterate over its secondary addresses and elect the first with the flag set.2)
IFA_PROM_PRIOTLV of type u32 holding a priority where 0 means no candiate. inet_del_ifa* iterates over its secondary addresses and elects the one with the highest prio as new primary address or deletes all addresses if none is found.* respectively the equivalent function of the other address families.
Second variant requires more work but is more flexible so it's definitely my favourite. I'm willing to put some effort into this, I'm not familiar with all address families though.
include/linux/sysctl.h renamed from include/linux/bak.sysctl.h +1 -0
@@ -395,6 +395,7 @@NET_IPV4_CONF_FORCE_IGMP_VERSION =17 ,NET_IPV4_CONF_ARP_ANNOUNCE =18 ,NET_IPV4_CONF_ARP_IGNORE =19 , + NET_IPV4_CONF_PROMOTE_ALIASES=20, };/* /proc/sys/net/ipv4/netfilter */
net/ipv4/devinet.c renamed from net/ipv4/bak.devinet.c +30 -5
@@ -230,11 +230,15 @@static void inet_del_ifa (struct in_device *in_dev,struct in_ifaddr **ifap,int destroy) { +struct in_ifaddr *promote =NULL ;struct in_ifaddr *ifa1 = *ifap;ASSERT_RTNL (); -/* 1. Deleting primary ifaddr forces deletion all secondaries */ +/* 1. Deleting primary ifaddr forces deletion all secondaries +* unless sysctl aliaspromo not set +* ipv4_devconf.aliaspromo +**/ if (!(ifa1->ifa_flags &IFA_F_SECONDARY )) {struct in_ifaddr *ifa;
@@ -248,11 +252,16 @@continue ; } - *ifap1 = ifa->ifa_next; +if (!IN_DEV_PROMOTE_ALIASES (in_dev)) { + *ifap1 = ifa->ifa_next; -rtmsg_ifa (RTM_DELADDR , ifa); -notifier_call_chain (&inetaddr_chain,NETDEV_DOWN , ifa); -inet_free_ifa (ifa); +rtmsg_ifa (RTM_DELADDR , ifa); +notifier_call_chain (&inetaddr_chain,NETDEV_DOWN , ifa); +inet_free_ifa (ifa); + }else { + promote = ifa; +break ; + } } }
@@ -278,6 +287,13 @@if (!in_dev->ifa_list)inetdev_destroy (in_dev); } + +if (promote &&IN_DEV_PROMOTE_ALIASES (in_dev)) { +/* not sure if we should send a delete notify first? */ + promote->ifa_flags &= ~IFA_F_SECONDARY ; +rtmsg_ifa (RTM_NEWADDR , promote); +notifier_call_chain (&inetaddr_chain,NETDEV_UP , promote); + } }static int inet_insert_ifa (struct in_ifaddr *ifa)
@@ -1374,6 +1390,15 @@ .proc_handler = &ipv4_doint_and_flush, .strategy = &ipv4_doint_and_flush_strategy, }, + { + .ctl_name = NET_IPV4_CONF_PROMOTE_ALIASES, + .procname = "promote_aliases", + .data = &ipv4_devconf.promote_aliases, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &ipv4_doint_and_flush, + .strategy = &ipv4_doint_and_flush_strategy, + }, }, .devinet_dev = { {
include/linux/inetdevice.h renamed from include/linux/bak.inetdevice.h +2 -0
@@ -29,6 +29,7 @@int no_xfrm;int no_policy;int force_igmp_version; +int promote_aliases;void *sysctl; };
@@ -71,6 +72,7 @@#define IN_DEV_SEC_REDIRECTS (in_dev) (ipv4_devconf.secure_redirects || (in_dev)->cnf.secure_redirects)#define IN_DEV_IDTAG (in_dev) ((in_dev)->cnf.tag)#define IN_DEV_MEDIUM_ID (in_dev) ((in_dev)->cnf.medium_id) +#define IN_DEV_PROMOTE_ALIASES (in_dev) (ipv4_devconf.promote_aliases || (in_dev)->cnf.promote_aliases)#define IN_DEV_RX_REDIRECTS (in_dev) \ ((IN_DEV_FORWARD(in_dev) && \
Quoted 5 lines · jamal
Harald,
My (stoopid) ISP doesnt like your email address. In any case, attached patch of what i was alluding to. Maybe be missing some things. Compiles - not tested
Didnt boot ;->
A small silly magic number i missed. Now boots - but doesnt mean it works. And i dont have much time to spare chasing it.
cheers,
jamal
net/ipv4/devinet.c renamed from net/ipv4/bak.devinet.c +31 -7
@@ -230,11 +230,14 @@static void inet_del_ifa (struct in_device *in_dev,struct in_ifaddr **ifap,int destroy) { +struct in_ifaddr *promote =NULL ;struct in_ifaddr *ifa1 = *ifap;ASSERT_RTNL (); -/* 1. Deleting primary ifaddr forces deletion all secondaries */ +/* 1. Deleting primary ifaddr forces deletion all secondaries +* unless alias promotion is set +**/ if (!(ifa1->ifa_flags &IFA_F_SECONDARY )) {struct in_ifaddr *ifa;
@@ -248,11 +251,16 @@continue ; } - *ifap1 = ifa->ifa_next; +if (!IN_DEV_PROMOTE_ALIASES (in_dev)) { + *ifap1 = ifa->ifa_next; -rtmsg_ifa (RTM_DELADDR , ifa); -notifier_call_chain (&inetaddr_chain,NETDEV_DOWN , ifa); -inet_free_ifa (ifa); +rtmsg_ifa (RTM_DELADDR , ifa); +notifier_call_chain (&inetaddr_chain,NETDEV_DOWN , ifa); +inet_free_ifa (ifa); + }else { + promote = ifa; +break ; + } } }
@@ -278,6 +286,13 @@if (!in_dev->ifa_list)inetdev_destroy (in_dev); } + +if (promote &&IN_DEV_PROMOTE_ALIASES (in_dev)) { +/* not sure if we should send a delete notify first? */ + promote->ifa_flags &= ~IFA_F_SECONDARY ; +rtmsg_ifa (RTM_NEWADDR , promote); +notifier_call_chain (&inetaddr_chain,NETDEV_UP , promote); + } }static int inet_insert_ifa (struct in_ifaddr *ifa)
@@ -1209,10 +1224,10 @@return 1 ; } - +#define DEVINET_SIZE 21 static struct devinet_sysctl_table { struct ctl_table_header *sysctl_header; - ctl_table devinet_vars[20]; + ctl_table devinet_vars[DEVINET_SIZE]; ctl_table devinet_dev[2]; ctl_table devinet_conf_dir[2]; ctl_table devinet_proto_dir[2];
@@ -1374,6 +1389,15 @@ .proc_handler = &ipv4_doint_and_flush, .strategy = &ipv4_doint_and_flush_strategy, }, + { + .ctl_name = NET_IPV4_CONF_PROMOTE_ALIASES, + .procname = "promote_aliases", + .data = &ipv4_devconf.promote_aliases, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &ipv4_doint_and_flush, + .strategy = &ipv4_doint_and_flush_strategy, + }, }, .devinet_dev = { {
include/linux/sysctl.h renamed from include/linux/bak.sysctl.h +1 -0
@@ -395,6 +395,7 @@NET_IPV4_CONF_FORCE_IGMP_VERSION =17 ,NET_IPV4_CONF_ARP_ANNOUNCE =18 ,NET_IPV4_CONF_ARP_IGNORE =19 , + NET_IPV4_CONF_PROMOTE_ALIASES=20, };/* /proc/sys/net/ipv4/netfilter */
include/linux/inetdevice.h renamed from include/linux/bak.inetdevice.h +2 -0
@@ -29,6 +29,7 @@int no_xfrm;int no_policy;int force_igmp_version; +int promote_aliases;void *sysctl; };
@@ -71,6 +72,7 @@#define IN_DEV_SEC_REDIRECTS (in_dev) (ipv4_devconf.secure_redirects || (in_dev)->cnf.secure_redirects)#define IN_DEV_IDTAG (in_dev) ((in_dev)->cnf.tag)#define IN_DEV_MEDIUM_ID (in_dev) ((in_dev)->cnf.medium_id) +#define IN_DEV_PROMOTE_ALIASES (in_dev) (ipv4_devconf.promote_aliases || (in_dev)->cnf.promote_aliases)#define IN_DEV_RX_REDIRECTS (in_dev) \ ((IN_DEV_FORWARD(in_dev) && \
Quoted 3 lines · jamal
Didnt boot ;->
A small silly magic number i missed. Now boots - but doesnt mean it works. And i dont have much time to spare chasing it.
Don't bother, I'll give it some testing here.
Thanks again.
Quoted 2 lines
cheers,
jamal
Signature · 3 lines
- Harald Welte <[email protected]> http://www.gnumonks.org/ ============================================================================ Programming is like sex: One mistake and you have to support it your lifetime
Quoted 3 lines · jamal
Didnt boot ;->
A small silly magic number i missed. Now boots - but doesnt mean it works. And i dont have much time to spare chasing it.
Due to a customer having again trouble with this issue, I was forced to actually spend some time testing it (and merging it to a current kernel). At least in my simple tests, it worked like a charm :)
I've changed your wording 'ALIAS' to the (as I think) more apropriate 'SECONDARY', and did some minor cosmetic changes to make it apply against 2.6.12-rc2.
Dave, would you consider adding this patch to your tree?
Signature · 4 lines
- Harald Welte <[email protected]> http://gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
include/linux/inetdevice.h +2 -0
@@ -29,6 +29,7 @@int no_xfrm;int no_policy;int force_igmp_version; +int promote_secondaries;void *sysctl; };
@@ -71,6 +72,7 @@#define IN_DEV_SEC_REDIRECTS (in_dev) (ipv4_devconf.secure_redirects || (in_dev)->cnf.secure_redirects)#define IN_DEV_IDTAG (in_dev) ((in_dev)->cnf.tag)#define IN_DEV_MEDIUM_ID (in_dev) ((in_dev)->cnf.medium_id) +#define IN_DEV_PROMOTE_SECONDARIES (in_dev) (ipv4_devconf.promote_secondaries || (in_dev)->cnf.promote_secondaries)#define IN_DEV_RX_REDIRECTS (in_dev) \ ((IN_DEV_FORWARD(in_dev) && \
include/linux/sysctl.h +1 -0
@@ -399,6 +399,7 @@NET_IPV4_CONF_FORCE_IGMP_VERSION =17 ,NET_IPV4_CONF_ARP_ANNOUNCE =18 ,NET_IPV4_CONF_ARP_IGNORE =19 , +NET_IPV4_CONF_PROMOTE_SECONDARIES =20 , __NET_IPV4_CONF_MAX };
net/ipv4/devinet.c +29 -5
@@ -233,11 +233,14 @@static void inet_del_ifa (struct in_device *in_dev,struct in_ifaddr **ifap,int destroy) { +struct in_ifaddr *promote =NULL ;struct in_ifaddr *ifa1 = *ifap;ASSERT_RTNL (); -/* 1. Deleting primary ifaddr forces deletion all secondaries */ +/* 1. Deleting primary ifaddr forces deletion all secondaries +* unless alias promotion is set +**/ if (!(ifa1->ifa_flags &IFA_F_SECONDARY )) {struct in_ifaddr *ifa;
@@ -251,11 +254,16 @@continue ; } - *ifap1 = ifa->ifa_next; +if (!IN_DEV_PROMOTE_SECONDARIES (in_dev)) { + *ifap1 = ifa->ifa_next; -rtmsg_ifa (RTM_DELADDR , ifa); -notifier_call_chain (&inetaddr_chain,NETDEV_DOWN , ifa); -inet_free_ifa (ifa); +rtmsg_ifa (RTM_DELADDR , ifa); +notifier_call_chain (&inetaddr_chain,NETDEV_DOWN , ifa); +inet_free_ifa (ifa); + }else { + promote = ifa; +break ; + } } }
@@ -281,6 +289,13 @@if (!in_dev->ifa_list)inetdev_destroy (in_dev); } + +if (promote &&IN_DEV_PROMOTE_SECONDARIES (in_dev)) { +/* not sure if we should send a delete notify first? */ + promote->ifa_flags &= ~IFA_F_SECONDARY ; +rtmsg_ifa (RTM_NEWADDR , promote); +notifier_call_chain (&inetaddr_chain,NETDEV_UP , promote); + } }static int inet_insert_ifa (struct in_ifaddr *ifa)
@@ -1383,6 +1398,15 @@ .proc_handler = &ipv4_doint_and_flush, .strategy = &ipv4_doint_and_flush_strategy, }, + { + .ctl_name = NET_IPV4_CONF_PROMOTE_SECONDARIES, + .procname = "promote_secondaries", + .data = &ipv4_devconf.promote_secondaries, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &ipv4_doint_and_flush, + .strategy = &ipv4_doint_and_flush_strategy, + }, }, .devinet_dev = { {
Quoted 9 lines · Harald Welte
Quoted 3 lines · jamal
Didnt boot ;->
A small silly magic number i missed. Now boots - but doesnt mean it works. And i dont have much time to spare chasing it.Due to a customer having again trouble with this issue, I was forced to actually spend some time testing it (and merging it to a current kernel). At least in my simple tests, it worked like a charm :)
I'm using it for some weeks in production and haven't noticed any issues either. Would be really good to see it kernel.
regards,
Signature · 3 lines
Hasso Tepper Elion Enterprises Ltd. WAN administrator
David, would you consider applying that patch to mainline? I think there was concensus on this solution, and it has now received some amount of testing by Hasso and me.
Thanks!
Quoted 22 lines · Hasso Tepper
Quoted 9 lines · Harald Welte
Quoted 3 lines · jamal
Didnt boot ;->
A small silly magic number i missed. Now boots - but doesnt mean it works. And i dont have much time to spare chasing it.Due to a customer having again trouble with this issue, I was forced to actually spend some time testing it (and merging it to a current kernel). At least in my simple tests, it worked like a charm :)
I'm using it for some weeks in production and haven't noticed any issues either. Would be really good to see it kernel.
regards,
Signature · 3 lines
Hasso Tepper Elion Enterprises Ltd. WAN administrator
Signature · 4 lines
- Harald Welte <[email protected]> http://gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
include/linux/inetdevice.h +2 -0
@@ -29,6 +29,7 @@int no_xfrm;int no_policy;int force_igmp_version; +int promote_secondaries;void *sysctl; };
@@ -71,6 +72,7 @@#define IN_DEV_SEC_REDIRECTS (in_dev) (ipv4_devconf.secure_redirects || (in_dev)->cnf.secure_redirects)#define IN_DEV_IDTAG (in_dev) ((in_dev)->cnf.tag)#define IN_DEV_MEDIUM_ID (in_dev) ((in_dev)->cnf.medium_id) +#define IN_DEV_PROMOTE_SECONDARIES (in_dev) (ipv4_devconf.promote_secondaries || (in_dev)->cnf.promote_secondaries)#define IN_DEV_RX_REDIRECTS (in_dev) \ ((IN_DEV_FORWARD(in_dev) && \
include/linux/sysctl.h +1 -0
@@ -399,6 +399,7 @@NET_IPV4_CONF_FORCE_IGMP_VERSION =17 ,NET_IPV4_CONF_ARP_ANNOUNCE =18 ,NET_IPV4_CONF_ARP_IGNORE =19 , +NET_IPV4_CONF_PROMOTE_SECONDARIES =20 , __NET_IPV4_CONF_MAX };
net/ipv4/devinet.c +29 -5
@@ -233,11 +233,14 @@static void inet_del_ifa (struct in_device *in_dev,struct in_ifaddr **ifap,int destroy) { +struct in_ifaddr *promote =NULL ;struct in_ifaddr *ifa1 = *ifap;ASSERT_RTNL (); -/* 1. Deleting primary ifaddr forces deletion all secondaries */ +/* 1. Deleting primary ifaddr forces deletion all secondaries +* unless alias promotion is set +**/ if (!(ifa1->ifa_flags &IFA_F_SECONDARY )) {struct in_ifaddr *ifa;
@@ -251,11 +254,16 @@continue ; } - *ifap1 = ifa->ifa_next; +if (!IN_DEV_PROMOTE_SECONDARIES (in_dev)) { + *ifap1 = ifa->ifa_next; -rtmsg_ifa (RTM_DELADDR , ifa); -notifier_call_chain (&inetaddr_chain,NETDEV_DOWN , ifa); -inet_free_ifa (ifa); +rtmsg_ifa (RTM_DELADDR , ifa); +notifier_call_chain (&inetaddr_chain,NETDEV_DOWN , ifa); +inet_free_ifa (ifa); + }else { + promote = ifa; +break ; + } } }
@@ -281,6 +289,13 @@if (!in_dev->ifa_list)inetdev_destroy (in_dev); } + +if (promote &&IN_DEV_PROMOTE_SECONDARIES (in_dev)) { +/* not sure if we should send a delete notify first? */ + promote->ifa_flags &= ~IFA_F_SECONDARY ; +rtmsg_ifa (RTM_NEWADDR , promote); +notifier_call_chain (&inetaddr_chain,NETDEV_UP , promote); + } }static int inet_insert_ifa (struct in_ifaddr *ifa)
@@ -1383,6 +1398,15 @@ .proc_handler = &ipv4_doint_and_flush, .strategy = &ipv4_doint_and_flush_strategy, }, + { + .ctl_name = NET_IPV4_CONF_PROMOTE_SECONDARIES, + .procname = "promote_secondaries", + .data = &ipv4_devconf.promote_secondaries, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &ipv4_doint_and_flush, + .strategy = &ipv4_doint_and_flush_strategy, + }, }, .devinet_dev = { {
* Harald Welte <[email protected]> 2005-05-26 20:11
Quoted 3 lines
David, would you consider applying that patch to mainline? I think there was concensus on this solution, and it has now received some amount of testing by Hasso and me.
I agree, I've been running this patch for 3 weeks now without any problems. Two comments below.
net/ipv4/devinet.c line 293
Quoted 6 lines
net/ipv4/devinet.c +2
-0@@ -281,6 +289,13 @@if (!in_dev->ifa_list)inetdev_destroy (in_dev); } + +if (promote &&IN_DEV_PROMOTE_SECONDARIES (in_dev)) {
promote can only be !=NULL if promotion is enabled, no?
net/ipv4/devinet.c line 296
Quoted 3 lines
net/ipv4/devinet.c +3
-0+/* not sure if we should send a delete notify first? */ + promote->ifa_flags &= ~IFA_F_SECONDARY ; +rtmsg_ifa (RTM_NEWADDR , promote);
This can be improved, however sending a delete/add seems inappropriate. I've patch prepared to add a change mask which will give us the proper methods to do this right but for now this is just fine I guess.
David S. Miller
maintainerFrom: Harald Welte <[email protected]>
Date: Thu, 26 May 2005 20:11:18 +0200
Quoted 3 lines
David, would you consider applying that patch to mainline? I think there was concensus on this solution, and it has now received some amount of testing by Hasso and me.
Since it retains the default behavior by default, this change seems fine. I've added it to my tree, thanks.
Quoted 1 line · Harald Welte
... and why does removing the primary address remove all secondary addresses.
Because applications may have state which relies on the implied source address selection, I'd hazard a guess. If you dont remove the secondaries, you might break the apps - but i dont know.
(certainly, the primary/secondary thing in kernel is useful - though, it possibly doesnt need to be in kernel.)
Quoted 1 line
This makes it complicated if you have one interface with multiple ip addresse, that change over time (failover, let's say.) You don't know yet, which address you need to remove, because you don't know which of your peers fails.
That's bad planning. ;)
You should assign your failover addresses seperately to your primary. Ie, the primary should never fail over - its unique to the machine/interface.
Quoted 1 line
So you have all this complicated userspace magic that checks whether the address that is about to be deleted is the primary, and if yes, re-add all the other addresses. If if is a secondary, you're happy and only need to delete that one.
Ick ;)
Quoted 1 line
Can anyone comment what the idea of all this was?
Source address selection. ?
Quoted 1 line
Thanks.
regards,
Signature · 5 lines
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A Fortune: Prizes are for children. -- Charles Ives, upon being given, but refusing, the Pulitzer prize