Skip to contentSkip to conversation

ipv6: do not merge differe type and protocol routes

PATCH In review Hangbin Liu 3 participants 17 messages Aug 30, 2023 – Sep 15, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ipv6: do not merge differe type and protocol routes

Different with IPv4, IPv6 will auto merge the same metric routes into multipath routes. But the different type and protocol routes are also merged, which will lost user's configure info. e.g.

+ ip route add local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100
+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip -6 route show table 100
local 2001:db8:103::/64 metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

+ ip route add 2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel table 200
+ ip route append 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp table 200
+ ip -6 route show table 200
2001:db8:104::/64 proto kernel metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

So let's skip counting the different type and protocol routes as siblings. After update, the different type/protocol routes will not be merged.

+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

+ ip -6 route show table 200
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel metric 1024 pref medium 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp metric 1024 pref medium

---
All fib test passed:
Tests passed: 203
Tests failed: 0

---
 net/ipv6/ip6_fib.c | 5 +++++
 1 file changed, 5 insertions(+)
net/ipv6/ip6_fib.c +5 -0
index 28b01a068412..f60f5d14f034 100644
@@ -1133,6 +1133,11 @@ static int fib6_add_rt2node(struct fib6_node *fn, struct fib6_info *rt,
 							rt->fib6_pmtu);
 				return -EEXIST;
 			}
+
+			if (iter->fib6_type != rt->fib6_type ||
+			    iter->fib6_protocol != rt->fib6_protocol)
+				goto next_iter;
+
 			/* If we have the same destination and the same metric,
 			 * but not the same gateway, then the route we try to
 			 * add is sibling to this route, increment our counter
Signature · 1 line
2.41.0
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 29 lines · Hangbin Liu

Different with IPv4, IPv6 will auto merge the same metric routes into multipath routes. But the different type and protocol routes are also merged, which will lost user's configure info. e.g.

+ ip route add local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100
+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip -6 route show table 100
local 2001:db8:103::/64 metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

+ ip route add 2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel table 200
+ ip route append 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp table 200
+ ip -6 route show table 200
2001:db8:104::/64 proto kernel metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

So let's skip counting the different type and protocol routes as siblings. After update, the different type/protocol routes will not be merged.

+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

+ ip -6 route show table 200
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel metric 1024 pref medium 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp metric 1024 pref medium

Quoted 7 lines

---
All fib test passed:
Tests passed: 203
Tests failed: 0

Please add the above tests for this case to the fib tests script.

Quoted 10 lines
---
 net/ipv6/ip6_fib.c | 5 +++++
 1 file changed, 5 insertions(+)
net/ipv6/ip6_fib.c +0 -0
index 28b01a068412..f60f5d14f034 100644
@@ -1133,6 +1133,11 @@ static int fib6_add_rt2node(struct fib6_node *fn, struct fib6_info *rt,
 							rt->fib6_pmtu);

net/ipv6/ip6_fib.c static int fib6_add_rt2node(struct fib6_node *fn, struct fib6_info *rt, · line 1143

Quoted 10 lines
net/ipv6/ip6_fib.c +5 -0
 				return -EEXIST;
 			}
+
+			if (iter->fib6_type != rt->fib6_type ||
+			    iter->fib6_protocol != rt->fib6_protocol)
+				goto next_iter;
+
 			/* If we have the same destination and the same metric,
 			 * but not the same gateway, then the route we try to
 			 * add is sibling to this route, increment our counter
#2 lore raw
Quoted 38 lines · David Ahern
Quoted 29 lines · Hangbin Liu

Different with IPv4, IPv6 will auto merge the same metric routes into multipath routes. But the different type and protocol routes are also merged, which will lost user's configure info. e.g.

+ ip route add local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100
+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip -6 route show table 100
local 2001:db8:103::/64 metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

+ ip route add 2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel table 200
+ ip route append 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp table 200
+ ip -6 route show table 200
2001:db8:104::/64 proto kernel metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

So let's skip counting the different type and protocol routes as siblings. After update, the different type/protocol routes will not be merged.

+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

+ ip -6 route show table 200
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel metric 1024 pref medium 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp metric 1024 pref medium

Quoted 7 lines

---
All fib test passed:
Tests passed: 203
Tests failed: 0

Quoted 1 line

Please add the above tests for this case to the fib tests script.

OK, I will add them and re-post after net-next open.

Thanks
Hangbin

#3 lore raw
In reply to Hangbin Liu

Le 30/08/2023 à 08:15, Hangbin Liu a écrit :

Quoted 21 lines

Different with IPv4, IPv6 will auto merge the same metric routes into multipath routes. But the different type and protocol routes are also merged, which will lost user's configure info. e.g.

+ ip route add local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100
+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip -6 route show table 100
local 2001:db8:103::/64 metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

+ ip route add 2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel table 200
+ ip route append 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp table 200
+ ip -6 route show table 200
2001:db8:104::/64 proto kernel metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

So let's skip counting the different type and protocol routes as siblings. After update, the different type/protocol routes will not be merged.

Quoted 7 lines

+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

+ ip -6 route show table 200
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel metric 1024 pref medium 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp metric 1024 pref medium

This seems wrong. The goal of 'ip route append' is to add a next hop, not to create a new route. Ok, it adds a new route if no route exists, but it seems wrong to me to use it by default, instead of 'add', to make things work magically.

It seems more correct to return an error in these cases, but this will change the uapi and it may break existing setups.

Before this patch, both next hops could be used by the kernel. After it, one route will be ignored (the former or the last one?). This is confusing and also seems wrong.

Quoted 3 lines

Please, don't put private link. The bug entry is not public.

Can you explain what is the initial problem?

Quoted 25 lines

---
All fib test passed:
Tests passed: 203
Tests failed: 0

---
 net/ipv6/ip6_fib.c | 5 +++++
 1 file changed, 5 insertions(+)
net/ipv6/ip6_fib.c +5 -0
index 28b01a068412..f60f5d14f034 100644
@@ -1133,6 +1133,11 @@ static int fib6_add_rt2node(struct fib6_node *fn, struct fib6_info *rt,
 							rt->fib6_pmtu);
 				return -EEXIST;
 			}
+
+			if (iter->fib6_type != rt->fib6_type ||
+			    iter->fib6_protocol != rt->fib6_protocol)
+				goto next_iter;
+
 			/* If we have the same destination and the same metric,
 			 * but not the same gateway, then the route we try to
 			 * add is sibling to this route, increment our counter
maintainer
#4 lore raw
Quoted 26 lines · Nicolas Dichtel

Le 30/08/2023 à 08:15, Hangbin Liu a écrit :

Quoted 25 lines

Different with IPv4, IPv6 will auto merge the same metric routes into multipath routes. But the different type and protocol routes are also merged, which will lost user's configure info. e.g.

+ ip route add local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100
+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip -6 route show table 100
local 2001:db8:103::/64 metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

+ ip route add 2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel table 200
+ ip route append 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp table 200
+ ip -6 route show table 200
2001:db8:104::/64 proto kernel metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

So let's skip counting the different type and protocol routes as siblings. After update, the different type/protocol routes will not be merged.

+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

Quoted 7 lines
Quoted 3 lines

+ ip -6 route show table 200
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel metric 1024 pref medium 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp metric 1024 pref medium

This seems wrong. The goal of 'ip route append' is to add a next hop, not to create a new route. Ok, it adds a new route if no route exists, but it seems wrong to me to use it by default, instead of 'add', to make things work magically.

Legacy API; nothing can be done about that (ie., that append makes a new route when none exists).

Quoted 7 lines

It seems more correct to return an error in these cases, but this will change the uapi and it may break existing setups.

Before this patch, both next hops could be used by the kernel. After it, one route will be ignored (the former or the last one?). This is confusing and also seems wrong.

Append should match all details of a route to add to an existing entry and make it multipath. If there is a difference (especially the type - protocol difference is arguable) in attributes, then they are different routes.

#5 lore raw

Le 30/08/2023 à 20:57, David Ahern a écrit :

Quoted 42 lines · Nicolas Dichtel
Quoted 26 lines

Le 30/08/2023 à 08:15, Hangbin Liu a écrit :

Quoted 25 lines

Different with IPv4, IPv6 will auto merge the same metric routes into multipath routes. But the different type and protocol routes are also merged, which will lost user's configure info. e.g.

+ ip route add local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100
+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip -6 route show table 100
local 2001:db8:103::/64 metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

+ ip route add 2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel table 200
+ ip route append 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp table 200
+ ip -6 route show table 200
2001:db8:104::/64 proto kernel metric 1024 pref medium

        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1

So let's skip counting the different type and protocol routes as siblings. After update, the different type/protocol routes will not be merged.

+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

Quoted 7 lines
Quoted 3 lines

+ ip -6 route show table 200
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel metric 1024 pref medium 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp metric 1024 pref medium

This seems wrong. The goal of 'ip route append' is to add a next hop, not to create a new route. Ok, it adds a new route if no route exists, but it seems wrong to me to use it by default, instead of 'add', to make things work magically.

Legacy API; nothing can be done about that (ie., that append makes a new route when none exists).

Quoted 4 lines

It seems more correct to return an error in these cases, but this will change the uapi and it may break existing setups.

Quoted 9 lines
Quoted 3 lines

Before this patch, both next hops could be used by the kernel. After it, one route will be ignored (the former or the last one?). This is confusing and also seems wrong.

Append should match all details of a route to add to an existing entry and make it multipath. If there is a difference (especially the type - protocol difference is arguable) in attributes, then they are different routes.

As you said, the protocol difference is arguable. It's not a property of the route, just a hint.
I think the 'append' should match a route whatever the protocol is. 'ip route change' for example does not use the protocol to find the existing route, it will update it:

$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 metric 1024 pref medium
$ ip -6 route change 2003:1:2:3::/64 via 2001::2 dev eth1 protocol bgp
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 proto bgp metric 1024 pref medium
$ ip -6 route change 2003:1:2:3::/64 via 2001::2 dev eth1 protocol kernel
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 proto kernel metric 1024 pref medium

Why would 'append' selects route differently?

This patch breaks the legacy API.

#6 lore raw

Hi Nicolas,

Quoted 38 lines · Nicolas Dichtel
Quoted 22 lines
Quoted 7 lines
Quoted 7 lines

So let's skip counting the different type and protocol routes as siblings. After update, the different type/protocol routes will not be merged.

+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

Quoted 7 lines
Quoted 3 lines

+ ip -6 route show table 200
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel metric 1024 pref medium 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp metric 1024 pref medium

This seems wrong. The goal of 'ip route append' is to add a next hop, not to create a new route. Ok, it adds a new route if no route exists, but it seems wrong to me to use it by default, instead of 'add', to make things work magically.

Legacy API; nothing can be done about that (ie., that append makes a new route when none exists).

Quoted 4 lines

It seems more correct to return an error in these cases, but this will change the uapi and it may break existing setups.

Quoted 9 lines
Quoted 3 lines

Before this patch, both next hops could be used by the kernel. After it, one route will be ignored (the former or the last one?). This is confusing and also seems wrong.

Append should match all details of a route to add to an existing entry and make it multipath. If there is a difference (especially the type - protocol difference is arguable) in attributes, then they are different routes.

As you said, the protocol difference is arguable. It's not a property of the route, just a hint.
I think the 'append' should match a route whatever the protocol is. 'ip route change' for example does not use the protocol to find the existing route, it will update it:

Quoted 9 lines
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 metric 1024 pref medium
$ ip -6 route change 2003:1:2:3::/64 via 2001::2 dev eth1 protocol bgp
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 proto bgp metric 1024 pref medium
$ ip -6 route change 2003:1:2:3::/64 via 2001::2 dev eth1 protocol kernel
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 proto kernel metric 1024 pref medium

Not sure if I understand correctly, ip route replace should able to replace all other field other than dest and dev. It's for changing the route, not only nexthop.

Quoted 2 lines

Why would 'append' selects route differently?

The append should also works for a single route, not only for append nexthop, no?

Quoted 2 lines

This patch breaks the legacy API.

As the patch's description. Who would expect different type/protocol route should be merged as multipath route? I don't think the old API is correct.

Thanks
Hangbin

#7 lore raw

Le 31/08/2023 à 12:14, Hangbin Liu a écrit :

Quoted 50 lines

Hi Nicolas,

Quoted 38 lines · Nicolas Dichtel
Quoted 22 lines
Quoted 7 lines
Quoted 7 lines

So let's skip counting the different type and protocol routes as siblings. After update, the different type/protocol routes will not be merged.

+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

Quoted 7 lines
Quoted 3 lines

+ ip -6 route show table 200
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel metric 1024 pref medium 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp metric 1024 pref medium

This seems wrong. The goal of 'ip route append' is to add a next hop, not to create a new route. Ok, it adds a new route if no route exists, but it seems wrong to me to use it by default, instead of 'add', to make things work magically.

Legacy API; nothing can be done about that (ie., that append makes a new route when none exists).

Quoted 4 lines

It seems more correct to return an error in these cases, but this will change the uapi and it may break existing setups.

Quoted 9 lines
Quoted 3 lines

Before this patch, both next hops could be used by the kernel. After it, one route will be ignored (the former or the last one?). This is confusing and also seems wrong.

Append should match all details of a route to add to an existing entry and make it multipath. If there is a difference (especially the type - protocol difference is arguable) in attributes, then they are different routes.

As you said, the protocol difference is arguable. It's not a property of the route, just a hint.
I think the 'append' should match a route whatever the protocol is. 'ip route change' for example does not use the protocol to find the existing route, it will update it:

Quoted 9 lines
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 metric 1024 pref medium
$ ip -6 route change 2003:1:2:3::/64 via 2001::2 dev eth1 protocol bgp
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 proto bgp metric 1024 pref medium
$ ip -6 route change 2003:1:2:3::/64 via 2001::2 dev eth1 protocol kernel
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 proto kernel metric 1024 pref medium
Quoted 7 lines

Not sure if I understand correctly, ip route replace should able to replace all other field other than dest and dev. It's for changing the route, not only nexthop.

Quoted 2 lines

Why would 'append' selects route differently?

The append should also works for a single route, not only for append nexthop, no?

I don't think so. The 'append' should 'join', not add. Adding more cases where a route is added instead of appended doesn't make the API clearer.

With this patch, it will be possible to add a new route with the 'append' command when the 'add' command fails:

$ ip -6 route add local 2003:1:2:3::/64 via 2001::2 dev eth1 table 200
$ ip -6 route add unicast 2003:1:2:3::/64 via 2001::2 dev eth1 table 200
RTNETLINK answers: File exists
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1 protocol bgp table 200
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1 protocol kernel table 200
RTNETLINK answers: File exists

This makes the API more confusing and complex. And I don't understand how it will be used later. There will be 2 routes on the system, but only one will be used, which one? This is confusing.

Quoted 6 lines
Quoted 3 lines

This patch breaks the legacy API.

As the patch's description. Who would expect different type/protocol route should be merged as multipath route? I don't think the old API is correct.

The question is not 'who expect', but 'is there some systems somewhere that rely on this (deliberately or not)'.
Frankly, the protocol is just informative, so I don't see why it is a problem to ignore it with the 'append' command.
For the type, it is weird, for sure. Rejecting the command seems better than duplicating routes. Which route is used by the stack?

Regards,
Nicolas

maintainer
#8 lore raw
Quoted 79 lines · Nicolas Dichtel

Le 31/08/2023 à 12:14, Hangbin Liu a écrit :

Quoted 50 lines

Hi Nicolas,

Quoted 38 lines · Nicolas Dichtel
Quoted 22 lines
Quoted 7 lines
Quoted 7 lines

So let's skip counting the different type and protocol routes as siblings. After update, the different type/protocol routes will not be merged.

+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

Quoted 7 lines
Quoted 3 lines

+ ip -6 route show table 200
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel metric 1024 pref medium 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp metric 1024 pref medium

This seems wrong. The goal of 'ip route append' is to add a next hop, not to create a new route. Ok, it adds a new route if no route exists, but it seems wrong to me to use it by default, instead of 'add', to make things work magically.

Legacy API; nothing can be done about that (ie., that append makes a new route when none exists).

Quoted 4 lines

It seems more correct to return an error in these cases, but this will change the uapi and it may break existing setups.

Quoted 9 lines
Quoted 3 lines

Before this patch, both next hops could be used by the kernel. After it, one route will be ignored (the former or the last one?). This is confusing and also seems wrong.

Append should match all details of a route to add to an existing entry and make it multipath. If there is a difference (especially the type - protocol difference is arguable) in attributes, then they are different routes.

As you said, the protocol difference is arguable. It's not a property of the route, just a hint.
I think the 'append' should match a route whatever the protocol is. 'ip route change' for example does not use the protocol to find the existing route, it will update it:

Quoted 9 lines
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 metric 1024 pref medium
$ ip -6 route change 2003:1:2:3::/64 via 2001::2 dev eth1 protocol bgp
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 proto bgp metric 1024 pref medium
$ ip -6 route change 2003:1:2:3::/64 via 2001::2 dev eth1 protocol kernel
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 proto kernel metric 1024 pref medium
Quoted 7 lines

Not sure if I understand correctly, ip route replace should able to replace all other field other than dest and dev. It's for changing the route, not only nexthop.

Quoted 2 lines

Why would 'append' selects route differently?

The append should also works for a single route, not only for append nexthop, no?

I don't think so. The 'append' should 'join', not add. Adding more cases where a route is added instead of appended doesn't make the API clearer.

With this patch, it will be possible to add a new route with the 'append' command when the 'add' command fails:

$ ip -6 route add local 2003:1:2:3::/64 via 2001::2 dev eth1 table 200
$ ip -6 route add unicast 2003:1:2:3::/64 via 2001::2 dev eth1 table 200
RTNETLINK answers: File exists
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1 protocol bgp table 200
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1 protocol kernel table 200
RTNETLINK answers: File exists

This makes the API more confusing and complex. And I don't understand how it will be used later. There will be 2 routes on the system, but only one will be used, which one? This is confusing.

Quoted 4 lines
Quoted 3 lines

This patch breaks the legacy API.

Quoted 10 lines
Quoted 2 lines

As the patch's description. Who would expect different type/protocol route should be merged as multipath route? I don't think the old API is correct.

The question is not 'who expect', but 'is there some systems somewhere that rely on this (deliberately or not)'.
Frankly, the protocol is just informative, so I don't see why it is a problem to ignore it with the 'append' command.
For the type, it is weird, for sure. Rejecting the command seems better than duplicating routes. Which route is used by the stack?

Part of my intent with fib_tests.sh was to document the legacy meaning of 'append, prepend, replace, and change' options while also providing a test script to detect changes that cause a regression.

I do agree now that protocol is informative (passthrough from the kernel perspective) so not really part of the route. That should be dropped from the patch leaving just a check on rt_type as to whether the routes are different. From there the append, prepend, replace and change semantics should decide what happens (ie., how the route is inserted).

#9 lore raw

Le 31/08/2023 à 20:27, David Ahern a écrit :

Quoted 91 lines · Nicolas Dichtel
Quoted 79 lines

Le 31/08/2023 à 12:14, Hangbin Liu a écrit :

Quoted 50 lines

Hi Nicolas,

Quoted 38 lines · Nicolas Dichtel
Quoted 22 lines
Quoted 7 lines
Quoted 7 lines

So let's skip counting the different type and protocol routes as siblings. After update, the different type/protocol routes will not be merged.

+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

Quoted 7 lines
Quoted 3 lines

+ ip -6 route show table 200
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel metric 1024 pref medium 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp metric 1024 pref medium

This seems wrong. The goal of 'ip route append' is to add a next hop, not to create a new route. Ok, it adds a new route if no route exists, but it seems wrong to me to use it by default, instead of 'add', to make things work magically.

Legacy API; nothing can be done about that (ie., that append makes a new route when none exists).

Quoted 4 lines

It seems more correct to return an error in these cases, but this will change the uapi and it may break existing setups.

Quoted 9 lines
Quoted 3 lines

Before this patch, both next hops could be used by the kernel. After it, one route will be ignored (the former or the last one?). This is confusing and also seems wrong.

Append should match all details of a route to add to an existing entry and make it multipath. If there is a difference (especially the type - protocol difference is arguable) in attributes, then they are different routes.

As you said, the protocol difference is arguable. It's not a property of the route, just a hint.
I think the 'append' should match a route whatever the protocol is. 'ip route change' for example does not use the protocol to find the existing route, it will update it:

Quoted 9 lines
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 metric 1024 pref medium
$ ip -6 route change 2003:1:2:3::/64 via 2001::2 dev eth1 protocol bgp
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 proto bgp metric 1024 pref medium
$ ip -6 route change 2003:1:2:3::/64 via 2001::2 dev eth1 protocol kernel
$ ip -6 route
2003:1:2:3::/64 via 2001::2 dev eth1 proto kernel metric 1024 pref medium
Quoted 7 lines

Not sure if I understand correctly, ip route replace should able to replace all other field other than dest and dev. It's for changing the route, not only nexthop.

Quoted 2 lines

Why would 'append' selects route differently?

The append should also works for a single route, not only for append nexthop, no?

I don't think so. The 'append' should 'join', not add. Adding more cases where a route is added instead of appended doesn't make the API clearer.

With this patch, it will be possible to add a new route with the 'append' command when the 'add' command fails:

$ ip -6 route add local 2003:1:2:3::/64 via 2001::2 dev eth1 table 200
$ ip -6 route add unicast 2003:1:2:3::/64 via 2001::2 dev eth1 table 200
RTNETLINK answers: File exists
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1 protocol bgp table 200
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1 protocol kernel table 200
RTNETLINK answers: File exists

This makes the API more confusing and complex. And I don't understand how it will be used later. There will be 2 routes on the system, but only one will be used, which one? This is confusing.

Quoted 4 lines
Quoted 3 lines

This patch breaks the legacy API.

Quoted 10 lines
Quoted 2 lines

As the patch's description. Who would expect different type/protocol route should be merged as multipath route? I don't think the old API is correct.

The question is not 'who expect', but 'is there some systems somewhere that rely on this (deliberately or not)'.
Frankly, the protocol is just informative, so I don't see why it is a problem to ignore it with the 'append' command.
For the type, it is weird, for sure. Rejecting the command seems better than duplicating routes. Which route is used by the stack?

Quoted 3 lines

Part of my intent with fib_tests.sh was to document the legacy meaning of 'append, prepend, replace, and change' options while also providing a test script to detect changes that cause a regression.

Yes, it's a good idea for sure.

Quoted 6 lines

I do agree now that protocol is informative (passthrough from the kernel perspective) so not really part of the route. That should be dropped from the patch leaving just a check on rt_type as to whether the routes are different. From there the append, prepend, replace and change semantics should decide what happens (ie., how the route is inserted).

Right. What can guide us is the meaning/concept/benefit of having this kind of routing table:
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

I don't understand how this is used/useful. It's why I ask for the use case/goal of this patch.
How does the user know which route is used?

Regards,
Nicolas

#10 lore raw
Quoted 2 lines · Nicolas Dichtel
Quoted 2 lines

I do agree now that protocol is informative (passthrough from the kernel perspective) so not really part of the route. That should be dropped

I'm not sure. Is there any user space route daemon will use this info? e.g. some BGP route daemon?

Quoted 11 lines
Quoted 3 lines

from the patch leaving just a check on rt_type as to whether the routes are different. From there the append, prepend, replace and change semantics should decide what happens (ie., how the route is inserted).

Right. What can guide us is the meaning/concept/benefit of having this kind of routing table:
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

I don't understand how this is used/useful. It's why I ask for the use case/goal of this patch.
How does the user know which route is used?

I'm not sure how user will use it. Maybe just block/forward some traffic to local first and remove the local route later to unblock them. IPv4 can also do like this. e.g.

+ ip link add dummy1 up type dummy
+ ip link add dummy2 up type dummy
+ ip addr add 192.168.0.1/24 dev dummy1
+ ip addr add 192.168.0.2/24 dev dummy2
+ ip route add local 192.168.3.0/24 dev dummy1 table 100
+ ip route append unicast 192.168.3.0/24 dev dummy1 table 100
+ ip route append unicast 192.168.3.0/24 dev dummy2 table 100
+ ip route show table 100
local 192.168.3.0/24 dev dummy1 scope host 192.168.3.0/24 dev dummy1 scope link
192.168.3.0/24 dev dummy2 scope link

Thanks
Hangbin

maintainer
#11 lore raw
Quoted 6 lines · Hangbin Liu
Quoted 2 lines · Nicolas Dichtel
Quoted 2 lines

I do agree now that protocol is informative (passthrough from the kernel perspective) so not really part of the route. That should be dropped

I'm not sure. Is there any user space route daemon will use this info? e.g. some BGP route daemon?

It is passthrough information for userspace to track who installed the route. It is not part of the route itself, but metadata passed in and returned.

#12 lore raw
Quoted 11 lines · David Ahern
Quoted 6 lines · Hangbin Liu
Quoted 2 lines · Nicolas Dichtel
Quoted 2 lines

I do agree now that protocol is informative (passthrough from the kernel perspective) so not really part of the route. That should be dropped

I'm not sure. Is there any user space route daemon will use this info? e.g. some BGP route daemon?

It is passthrough information for userspace to track who installed the route. It is not part of the route itself, but metadata passed in and returned.

Thanks. I will drop this info.

Hangbin

#13 lore raw
In reply to Nicolas Dichtel
Quoted 10 lines · Nicolas Dichtel
Quoted 1 line

The append should also works for a single route, not only for append nexthop, no?

I don't think so. The 'append' should 'join', not add. Adding more cases where a route is added instead of appended doesn't make the API clearer.

With this patch, it will be possible to add a new route with the 'append' command when the 'add' command fails:

$ ip -6 route add local 2003:1:2:3::/64 via 2001::2 dev eth1 table 200
$ ip -6 route add unicast 2003:1:2:3::/64 via 2001::2 dev eth1 table 200
RTNETLINK answers: File exists
Quoted 7 lines
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1 protocol bgp table 200
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1 protocol kernel table 200
RTNETLINK answers: File exists

This makes the API more confusing and complex. And I don't understand how it will be used later. There will be 2 routes on the system, but only one will be used, which one? This is confusing.

Just to makeit it clear, the new patch will not add two route with only different type/protocol. Here is the result with my patch.

+ ip -6 route flush table 300
+ ip link add dummy1 up type dummy
+ ip link add dummy2 up type dummy
+ ip addr add 2001:db8:101::1/64 dev dummy1
+ ip addr add 2001:db8:101::2/64 dev dummy2
+ ip route add local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100
+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100 RTNETLINK answers: File exists

^^ here the append still failed

+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium
+ ip route add 2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel table 200
+ ip route append 2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto bgp table 200 RTNETLINK answers: File exists

^^ And here

+ ip route append 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp table 200
+ ip -6 route show table 200
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel metric 1024 pref medium 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp metric 1024 pref medium

Thanks
Hangbin

#14 lore raw

Le 01/09/2023 à 05:58, Hangbin Liu a écrit :

Quoted 4 lines · Nicolas Dichtel
Quoted 3 lines
Quoted 1 line

The append should also works for a single route, not only for append nexthop, no?

I don't think so. The 'append' should 'join', not add. Adding more cases where a route is added instead of appended doesn't make the API clearer.

It was clear to me. I distinguish the route part and the next hop part.

Quoted 27 lines
Quoted 7 lines

With this patch, it will be possible to add a new route with the 'append' command when the 'add' command fails:

$ ip -6 route add local 2003:1:2:3::/64 via 2001::2 dev eth1 table 200
$ ip -6 route add unicast 2003:1:2:3::/64 via 2001::2 dev eth1 table 200
RTNETLINK answers: File exists
Quoted 7 lines
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1 protocol bgp table 200
$ ip -6 route add 2003:1:2:3::/64 via 2001::2 dev eth1 protocol kernel table 200
RTNETLINK answers: File exists

This makes the API more confusing and complex. And I don't understand how it will be used later. There will be 2 routes on the system, but only one will be used, which one? This is confusing.

Just to makeit it clear, the new patch will not add two route with only different type/protocol. Here is the result with my patch.

+ ip -6 route flush table 300
+ ip link add dummy1 up type dummy
+ ip link add dummy2 up type dummy
+ ip addr add 2001:db8:101::1/64 dev dummy1
+ ip addr add 2001:db8:101::2/64 dev dummy2
+ ip route add local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100
+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100 RTNETLINK answers: File exists

Quoted 1 line

^^ here the append still failed

And if I play a little bit of the devil's advocate: why is it rejected? It's not the same route, the types differ :)

Quoted 5 lines

+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

What is the purpose of such a routing table? How is the gateway of a local route used by the kernel? Which route will be used when a packet enters the stack?

Regards,
Nicolas

#15 lore raw
Quoted 9 lines · Nicolas Dichtel
Quoted 9 lines

+ ip -6 route flush table 300
+ ip link add dummy1 up type dummy
+ ip link add dummy2 up type dummy
+ ip addr add 2001:db8:101::1/64 dev dummy1
+ ip addr add 2001:db8:101::2/64 dev dummy2
+ ip route add local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100
+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100 RTNETLINK answers: File exists

Quoted 3 lines
Quoted 1 line

^^ here the append still failed

And if I play a little bit of the devil's advocate: why is it rejected? It's not the same route, the types differ :)

This is actually for compatibility...

Quoted 8 lines
Quoted 6 lines

+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

What is the purpose of such a routing table?

As I replied in last mail. I don't have a clear purpose. A user may use the local route to block traffic temporary.

Quoted 1 line

How is the gateway of a local route used by the kernel?

I don't know. IPv6 support this since beginning...

Quoted 1 line

Which route will be used when a packet enters the stack?

The first one it find?

Thanks
Hangbin

#16 lore raw

Le 15/09/2023 à 05:49, Hangbin Liu a écrit :

Quoted 1 line · Nicolas Dichtel

[snip]

Quoted 10 lines
Quoted 6 lines
Quoted 4 lines

+ ip route append unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip -6 route show table 100
local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 metric 1024 pref medium 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 metric 1024 pref medium

What is the purpose of such a routing table?

As I replied in last mail. I don't have a clear purpose. A user may use the local route to block traffic temporary.

Quoted 7 lines
Quoted 1 line

How is the gateway of a local route used by the kernel?

I don't know. IPv6 support this since beginning...

Quoted 1 line

Which route will be used when a packet enters the stack?

The first one it find?

Frankly, it seems strange to me to try to fix a scenario without a clear view of what is expected. I agree that the current behavior is questionable, but it enables to keep the system (the routing table) in a clear state.

My two cents,
Nicolas