nexthop: Resilient next-hop groups
Subsystems (4)
RFC PATCH 1/13 nexthop: Pass nh_config to replace_nexthop()
In reply to Petr MachataCurrently, replace assumes that the new group that is given is a fully-formed object. But mpath groups really only have one attribute, and that is the constituent next hop configuration. This may not be universally true. From the usability perspective, it is desirable to allow the replace operation to adjust just the constituent next hop configuration and leave the group attributes as such intact.
But the object that keeps track of whether an attribute was or was not given is the nh_config object, not the next hop or next-hop group. To allow (selective) attribute updates during NH group replacement, propagate `cfg' to replace_nexthop() and further to replace_nexthop_grp().
--- net/ipv4/nexthop.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
net/ipv4/nexthop.c +5 -4
@@ -1107,7 +1107,7 @@ static void nh_rt_cache_flush(struct net *net, struct nexthop *nh) }static int replace_nexthop_grp (struct net *net,struct nexthop *old, -struct nexthop *new, +struct nexthop *new,const struct nh_config *cfg,struct netlink_ext_ack *extack) {struct nh_group *oldg, *newg;
@@ -1276,7 +1276,8 @@ static void nexthop_replace_notify(struct net *net, struct nexthop *nh, }static int replace_nexthop (struct net *net,struct nexthop *old, -struct nexthop *new,struct netlink_ext_ack *extack) +struct nexthop *new,const struct nh_config *cfg, +struct netlink_ext_ack *extack) {bool new_is_reject = false;struct nh_grp_entry *nhge;
@@ -1319,7 +1320,7 @@ static int replace_nexthop(struct net *net, struct nexthop *old, }if (old->is_group) - err =replace_nexthop_grp (net, old, new, extack); + err =replace_nexthop_grp (net, old, new, cfg, extack);else err =replace_nexthop_single (net, old, new, extack);
@@ -1361,7 +1362,7 @@ static int insert_nexthop(struct net *net, struct nexthop *new_nh, }else if (new_id > nh->id) { pp = &next->rb_right; }else if (replace) { - rc =replace_nexthop (net, nh, new_nh, extack); + rc =replace_nexthop (net, nh, new_nh, cfg, extack);if (!rc) { new_nh = nh;/* send notification with old nh */ replace_notify =1 ;
Signature · 1 line
2.26.2
cover nexthop: Resilient next-hop groups
At this moment, there is only one type of next-hop group: an mpath group. Mpath groups implement the hash-threshold algorithm, described in RFC 2992[1].
To select a next hop, hash-threshold algorithm first assigns a range of hashes to each next hop in the group, and then selects the next hop by comparing the SKB hash with the individual ranges. When a next hop is removed from the group, the ranges are recomputed, which leads to reassignment of parts of hash space from one next hop to another. RFC 2992 illustrates it thus:
+-------+-------+-------+-------+-------+
| 1 | 2 | 3 | 4 | 5 |
+-------+-+-----+---+---+-----+-+-------+
| 1 | 2 | 4 | 5 |
+---------+---------+---------+---------+Before and after deletion of next hop 3 under the hash-threshold algorithm.
Note how next hop 2 gave up part of the hash space in favor of next hop 1, and 4 in favor of 5. While there will usually be some overlap between the previous and the new distribution, some traffic flows change the next hop that they resolve to.
If a multipath group is used for load-balancing between multiple servers, this hash space reassignment causes an issue that packets from a single flow suddenly end up arriving at a server that does not expect them, Which may lead to TCP reset.
If a multipath group is used for load-balancing among available paths to the same server, the issue is that different latencies and reordering along the way causes the packets to arrive in wrong order.
Resilient hashing is a technique to address the above problem. Resilient next-hop group has another layer of indirection between the group itself and its constituent next hops: a hash table. The selection algorithm uses a straightforward modulo operation to choose a hash bucket, and then reads the next hop that this bucket contains, and forwards traffic there.
This indirection brings an important feature. In the hash-threshold algorithm, the range of hashes associated with a next hop must be continuous. With a hash table, mapping between the hash table buckets and the individual next hops is arbitrary. Therefore when a next hop is deleted the buckets that held it are simply reassigned to other next hops:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1|1|1|1|2|2|2|2|3|3|3|3|4|4|4|4|5|5|5|5|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
v v v v
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1|1|1|1|2|2|2|2|1|2|4|5|4|4|4|4|5|5|5|5|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Before and after deletion of next hop 3
under the resilient hashing algorithm.When weights of next hops in a group are altered, it may be possible to choose a subset of buckets that are currently not used for forwarding traffic, and use those to satisfy the new next-hop distribution demands, keeping the "busy" buckets intact. This way, established flows are ideally kept being forwarded to the same endpoints through the same paths as before the next-hop group change.
This patch set adds the implementation of resilient next hop group.
In a nutshell, the algorithm works as follows. Each next hop has a number of buckets that it wants to have, according to its weight and the number of buckets in the hash table. In case of an event that might cause bucket allocation change, the numbers for individual next hops are updated, similarly to how ranges are updated for mpath group next hops. Following that, a new "upkeep" algorithm runs, and for idle buckets that belong to a next hop that is currently occupying more buckets than it wants (it is "overweight"), it migrates the buckets to one of the next hops that has fewer buckets than it wants (it is "underweight"). If, after this, there are still underweight next hops, another upkeep run is scheduled to a future time.
Chances are there are not enough "idle" buckets to satisfy the new demands. The algorithm has knobs to select both what it means for a bucket to be idle, and for whether and when to forcefully migrate buckets if there keeps being an insufficient number of idle buckets.
To illustrate the usage, consider the following commands:
# ip nexthop add id 1 via 192.0.2.2 dev dummy1 # ip nexthop add id 2 via 192.0.2.3 dev dummy1 # ip nexthop add id 10 group 1/2 type resilient \ buckets 8 idle_timer 60 unbalanced_timer 300
The last command creates a resilient next hop group. It will have 8 buckets, each bucket will be considered idle when no traffic hits it for at least 60 seconds, and if the table remains out of balance for 300 seconds, it will be forcefully brought into balance. (If not present in netlink message, the idle timer defaults to 120 seconds, and there is no unbalanced timer, meaning the group may remain unbalanced indefinitely.)
Unbalanced time, i.e. how long since the last time that all nexthops had as many buckets as they should according to their weights, is reported when the group is dumped:
# ip nexthop show id 10 id 10 group 1/2 type resilient buckets 8 idle_timer 60 unbalanced_timer 300 unbalanced_time 0
When replacing next hops or changing weights, if one does not specify some parameters, their value is left as it was:
# ip nexthop replace id 10 group 1,2/2 type resilient # ip nexthop show id 10 id 10 group 1,2/2 type resilient buckets 8 idle_timer 60 unbalanced_timer 300 unbalanced_time 0
It is also possible to do a dump of individual buckets (and now you know why there were only 8 of them in the table):
# ip nexthop bucket show id 10 id 10 index 0 idle_time 5.59 nhid 1 id 10 index 1 idle_time 5.59 nhid 1 id 10 index 2 idle_time 8.74 nhid 2 id 10 index 3 idle_time 8.74 nhid 2 id 10 index 4 idle_time 8.74 nhid 1 id 10 index 5 idle_time 8.74 nhid 1 id 10 index 6 idle_time 8.74 nhid 1 id 10 index 7 idle_time 8.74 nhid 1
Note the two buckets that have a shorter idle time. Those are the ones that were migrated after the nexthop replace command to satisfy the new demand that nexthop 1 be given 6 buckets instead of 4.
The patchset proceeds as follows:
- Patches #1 and #2 are small refactoring patches.
- Patch #3 contains defines of new UAPI attributes and the new next-hop group type. At this point, the nexthop code is made to bounce the new type. Is the resilient hashing code is gradually added in the following patch sets, it will remain dead. The last patch will make it accessible.
This patch also adds a suite of new messages related to next hop buckets. This approach was taken instead of overloading the information on the existing RTM_{NEW,DEL,GET}NEXTHOP messages for the following reasons.
First, a next-hop group can contain a large number of next-hop buckets (4k is not unheard of). This imposes limits on the amount of information that can be encoded for each next-hop bucket given a netlink message is limited to 64k bytes.
Second, while RTM_NEWNEXTHOPBUCKET is only used for notifications at this point, in the future it can be extended to provide user space with control over next-hop buckets configuration.
- Patch #4 contains the meat of the resilient next hop group support.
- Patches #5 and #6 implement support for notifications towards the drivers.
- Patch #7 adds an interface for the drivers to report resilient hash table bucket activity. Drivers will be able to report through this interface whether traffic is hitting a given bucket.
- Patch #8 adds an interface for the drivers to report whether a given hash table bucket is offloaded or trapping traffic.
- In patches #9, #10, #11 and #12, UAPI is implemented. This includes all the code necessary for creation of resilient groups, bucket dumping and getting, and bucket migration notifications.
- In patch #13 the next hop groups are finally made available.
The overall plan is to contribute approximately the following patchsets:
1) Nexthop policy refactoring (already pushed)
2) Preparations for resilient next hop groups (already pushed)
3) Implementation of resilient next hop group (this patchset)
4) Netdevsim offload plus a suite of selftests
5) Preparations for mlxsw offload of resilient next-hop groups
6) mlxsw offload including selftests
Interested parties can look at the complete code at [2].
[1] https://tools.ietf.org/html/rfc2992
[2] https://github.com/idosch/linux/commits/submit/res_integ_v1
Ido Schimmel (4):
nexthop: Add netlink defines and enumerators for resilient NH groups
nexthop: Add data structures for resilient group notifications
nexthop: Allow setting "offload" and "trap" indication of nexthop
buckets
nexthop: Allow reporting activity of nexthop buckets
Petr Machata (9):
nexthop: Pass nh_config to replace_nexthop()
nexthop: __nh_notifier_single_info_init(): Make nh_info an argument
nexthop: Add implementation of resilient next-hop groups
nexthop: Implement notifiers for resilient nexthop groups
nexthop: Add netlink handlers for resilient nexthop groups
nexthop: Add netlink handlers for bucket dump
nexthop: Add netlink handlers for bucket get
nexthop: Notify userspace about bucket migrations
nexthop: Enable resilient next-hop groups
include/net/nexthop.h | 71 +- include/uapi/linux/nexthop.h | 43 + include/uapi/linux/rtnetlink.h | 7 + net/ipv4/nexthop.c | 1521 ++++++++++++++++++++++++++++++-- security/selinux/nlmsgtab.c | 5 +- 5 files changed, 1593 insertions(+), 54 deletions(-)
Signature · 1 line
2.26.2
RFC PATCH 4/13 nexthop: Add implementation of resilient next-hop groups
At this moment, there is only one type of next-hop group: an mpath group, which implements the hash-threshold algorithm.
To select a next hop, hash-threshold algorithm first assigns a range of hashes to each next hop in the group, and then selects the next hop by comparing the SKB hash with the individual ranges. When a next hop is removed from the group, the ranges are recomputed, which leads to reassignment of parts of hash space from one next hop to another. While there will usually be some overlap between the previous and the new distribution, some traffic flows change the next hop that they resolve to. That causes problems e.g. as established TCP connections are reset, because the traffic is forwarded to a server that is not familiar with the connection.
Resilient hashing is a technique to address the above problem. Resilient next-hop group has another layer of indirection between the group itself and its constituent next hops: a hash table. The selection algorithm uses a straightforward modulo operation to choose a hash bucket, and then reads the next hop that this bucket contains, and forwards traffic there.
This indirection brings an important feature. In the hash-threshold algorithm, the range of hashes associated with a next hop must be continuous. With a hash table, mapping between the hash table buckets and the individual next hops is arbitrary. Therefore when a next hop is deleted the buckets that held it are simply reassigned to other next hops. When weights of next hops in a group are altered, it may be possible to choose a subset of buckets that are currently not used for forwarding traffic, and use those to satisfy the new next-hop distribution demands, keeping the "busy" buckets intact. This way, established flows are ideally kept being forwarded to the same endpoints through the same paths as before the next-hop group change.
In a nutshell, the algorithm works as follows. Each next hop has a number of buckets that it wants to have, according to its weight and the number of buckets in the hash table. In case of an event that might cause bucket allocation change, the numbers for individual next hops are updated, similarly to how ranges are updated for mpath group next hops. Following that, a new "upkeep" algorithm runs, and for idle buckets that belong to a next hop that is currently occupying more buckets than it wants (it is "overweight"), it migrates the buckets to one of the next hops that has fewer buckets than it wants (it is "underweight"). If, after this, there are still underweight next hops, another upkeep run is scheduled to a future time.
Chances are there are not enough "idle" buckets to satisfy the new demands. The algorithm has knobs to select both what it means for a bucket to be idle, and for whether and when to forcefully migrate buckets if there keeps being an insufficient number of idle buckets.
There are three users of the resilient data structures.
- The forwarding code accesses them under RCU, and does not modify them except for updating the time a selected bucket was last used.
- Netlink code, running under RTNL, which may modify the data.
- The delayed upkeep code, which may modify the data. This runs unlocked, and mutual exclusion between the RTNL code and the delayed upkeep is maintained by canceling the delayed work synchronously before the RTNL code touches anything. Later it restarts the delayed work if necessary.
The RTNL code has to implement next-hop group replacement, next hop removal, etc. For removal, the mpath code uses a neat trick of having a backup next hop group structure, doing the necessary changes offline, and then RCU-swapping them in. However, the hash tables for resilient hashing are about an order of magnitude larger than the groups themselves (the size might be e.g. 4K entries), and it was felt that keeping two of them is an overkill. Both the primary next-hop group and the spare therefore use the same resilient table, and writers are careful to keep all references valid for the forwarding code. The hash table references next-hop group entries from the next-hop group that is currently in the primary role (i.e. not spare). During the transition from primary to spare, the table references a mix of both the primary group and the spare. When a next hop is deleted, the corresponding buckets are not set to NULL, but instead marked as empty, so that the pointer is valid and can be used by the forwarding code. The buckets are then migrated to a new next-hop group entry during upkeep. The only times that the hash table is invalid is the very beginning and very end of its lifetime. Between those points, it is always kept valid.
This patch introduces the core support code itself. It does not handle notifications towards drivers, which are kept as if the group were an mpath one. It does not handle netlink either. The only bit currently exposed to user space is the new next-hop group type, and that is currently bounced. There is therefore no way to actually access this code.
--- include/net/nexthop.h | 48 +++- net/ipv4/nexthop.c | 521 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 551 insertions(+), 18 deletions(-)
include/net/nexthop.h +45 -3
@@ -40,6 +40,12 @@ struct nh_config {struct nlattr *nh_grp;u16 nh_grp_type; +u32 nh_grp_res_num_buckets; +unsigned long nh_grp_res_idle_timer; +unsigned long nh_grp_res_unbalanced_timer; +bool nh_grp_res_has_num_buckets; +bool nh_grp_res_has_idle_timer; +bool nh_grp_res_has_unbalanced_timer;struct nlattr *nh_encap;u16 nh_encap_type;
@@ -63,6 +69,32 @@ struct nh_info { }; }; +struct nh_res_bucket { +struct nh_grp_entry __rcu *nh_entry; +atomic_long_t used_time; +unsigned long migrated_time; +bool occupied; +u8 nh_flags; +}; + +struct nh_res_table { +struct net *net; +u32 nhg_id; +struct delayed_work upkeep_dw; + +/* List of NHGEs that have too few buckets ("uw" for underweight). +* Reclaimed buckets will be given to entries in this list. +*/ +struct list_head uw_nh_entries; +unsigned long unbalanced_since; + +u32 idle_timer; +u32 unbalanced_timer; + +u32 num_nh_buckets; +struct nh_res_bucket nh_buckets[]; +}; +struct nh_grp_entry {struct nexthop *nh;u8 weight;
@@ -71,6 +103,13 @@ struct nh_grp_entry {struct {atomic_t upper_bound; } mpath; +struct { +/* Member on uw_nh_entries. */ +struct list_head uw_nh_entry; + +u32 count_buckets; +u32 wants_buckets; + } res; };struct list_head nh_list;
@@ -81,8 +120,11 @@ struct nh_group {struct nh_group *spare;/* spare group for removals */ u16 num_nh;bool mpath; +bool resilient;bool fdb_nh;bool has_v4; + +struct nh_res_table __rcu *res_table;struct nh_grp_entry nh_entries[]; };
@@ -212,7 +254,7 @@ static inline bool nexthop_is_multipath(const struct nexthop *nh)struct nh_group *nh_grp; nh_grp =rcu_dereference_rtnl (nh->nh_grp); -return nh_grp->mpath; +return nh_grp->mpath || nh_grp->resilient; }return false; }
@@ -227,7 +269,7 @@ static inline unsigned int nexthop_num_path(const struct nexthop *nh)struct nh_group *nh_grp; nh_grp =rcu_dereference_rtnl (nh->nh_grp); -if (nh_grp->mpath) +if (nh_grp->mpath || nh_grp->resilient) rc = nh_grp->num_nh; }
@@ -308,7 +350,7 @@ struct fib_nh_common *nexthop_fib_nhc(struct nexthop *nh, int nhsel)struct nh_group *nh_grp; nh_grp =rcu_dereference_rtnl (nh->nh_grp); -if (nh_grp->mpath) { +if (nh_grp->mpath || nh_grp->resilient) { nh =nexthop_mpath_select (nh_grp, nhsel);if (!nh)return NULL ;
net/ipv4/nexthop.c +506 -15
661 lines not on this page · raw message
Signature · 1 line
2.26.2
RFC PATCH 2/13 nexthop: __nh_notifier_single_info_init(): Make nh_info an argument
In reply to Petr MachataThe cited function currently uses rtnl_dereference() to get nh_info from a handed-in nexthop. However, under the resilient hashing scheme, this function will not always be called under RTNL, sometimes the mutual exclusion will be achieved differently. Therefore move the nh_info extraction from the function to its callers to make it possible to use a different synchronization guarantee.
--- net/ipv4/nexthop.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
net/ipv4/nexthop.c +7 -5
@@ -52,10 +52,8 @@ static bool nexthop_notifiers_is_empty(struct net *net)static void __nh_notifier_single_info_init (struct nh_notifier_single_info *nh_info, -const struct nexthop *nh) +const struct nh_info *nhi) { -struct nh_info *nhi =rtnl_dereference (nh->nh_info); - nh_info->dev = nhi->fib_nhc.nhc_dev; nh_info->gw_family = nhi->fib_nhc.nhc_gw_family;if (nh_info->gw_family ==AF_INET )
@@ -71,12 +69,14 @@ __nh_notifier_single_info_init(struct nh_notifier_single_info *nh_info,static int nh_notifier_single_info_init (struct nh_notifier_info *info,const struct nexthop *nh) { +struct nh_info *nhi =rtnl_dereference (nh->nh_info); + info->type =NH_NOTIFIER_INFO_TYPE_SINGLE ; info->nh =kzalloc (sizeof (*info->nh),GFP_KERNEL );if (!info->nh)return -ENOMEM ; -__nh_notifier_single_info_init (info->nh, nh); +__nh_notifier_single_info_init (info->nh, nhi);return 0 ; }
@@ -103,11 +103,13 @@ static int nh_notifier_mp_info_init(struct nh_notifier_info *info,for (i =0 ; i < num_nh; i++) {struct nh_grp_entry *nhge = &nhg->nh_entries[i]; +struct nh_info *nhi; + nhi =rtnl_dereference (nhge->nh->nh_info); info->nh_grp->nh_entries[i].id = nhge->nh->id; info->nh_grp->nh_entries[i].weight = nhge->weight;__nh_notifier_single_info_init (&info->nh_grp->nh_entries[i].nh, - nhge->nh); + nhi); }return 0 ;
Signature · 1 line
2.26.2
RFC PATCH 3/13 nexthop: Add netlink defines and enumerators for resilient NH groups
In reply to Petr MachataFrom: Ido Schimmel <[email protected]>
- RTM_NEWNEXTHOP et.al. that handle resilient groups will have a new nested attribute, NHA_RES_GROUP, whose elements are attributes NHA_RES_GROUP_*.
- RTM_NEWNEXTHOPBUCKET et.al. is a suite of new messages that will currently serve only for dumping of individual buckets of resilient next hop groups. For nexthop group buckets, these messages will carry a nested attribute NHA_RES_BUCKET, whose elements are attributes NHA_RES_BUCKET_*.
There are several reasons why a new suite of messages is created for nexthop buckets instead of overloading the information on the existing RTM_{NEW,DEL,GET}NEXTHOP messages.
First, a nexthop group can contain a large number of nexthop buckets (4k is not unheard of). This imposes limits on the amount of information that can be encoded for each nexthop bucket given a netlink message is limited to 64k bytes.
Second, while RTM_NEWNEXTHOPBUCKET is only used for notifications at this point, in the future it can be extended to provide user space with control over nexthop buckets configuration.
- The new group type is NEXTHOP_GRP_TYPE_RES. Note that nexthop code is adjusted to bounce groups with that type for now.
--- include/uapi/linux/nexthop.h | 43 ++++++++++++++++++++++++++++++++++ include/uapi/linux/rtnetlink.h | 7 ++++++ net/ipv4/nexthop.c | 2 ++ security/selinux/nlmsgtab.c | 5 +++- 4 files changed, 56 insertions(+), 1 deletion(-)
include/uapi/linux/nexthop.h +43 -0
@@ -22,6 +22,7 @@ struct nexthop_grp {enum {NEXTHOP_GRP_TYPE_MPATH ,/* default type if not specified */ +NEXTHOP_GRP_TYPE_RES ,/* resilient nexthop group */ __NEXTHOP_GRP_TYPE_MAX, };
@@ -52,8 +53,50 @@ enum {NHA_FDB ,/* flag; nexthop belongs to a bridge fdb */ /* if NHA_FDB is added, OIF, BLACKHOLE, ENCAP cannot be set */ +/* nested; resilient nexthop group attributes */ +NHA_RES_GROUP , +/* nested; nexthop bucket attributes */ +NHA_RES_BUCKET , + __NHA_MAX, };#define NHA_MAX (__NHA_MAX - 1) + +enum { +NHA_RES_GROUP_UNSPEC , +/* Pad attribute for 64-bit alignment. */ +NHA_RES_GROUP_PAD =NHA_RES_GROUP_UNSPEC , + +/* u32; number of nexthop buckets in a resilient nexthop group */ +NHA_RES_GROUP_BUCKETS , +/* clock_t as u32; nexthop bucket idle timer (per-group) */ +NHA_RES_GROUP_IDLE_TIMER , +/* clock_t as u32; nexthop unbalanced timer */ +NHA_RES_GROUP_UNBALANCED_TIMER , +/* clock_t as u64; nexthop unbalanced time */ +NHA_RES_GROUP_UNBALANCED_TIME , + + __NHA_RES_GROUP_MAX, +}; + +#define NHA_RES_GROUP_MAX (__NHA_RES_GROUP_MAX - 1) + +enum { +NHA_RES_BUCKET_UNSPEC , +/* Pad attribute for 64-bit alignment. */ +NHA_RES_BUCKET_PAD =NHA_RES_BUCKET_UNSPEC , + +/* u32; nexthop bucket index */ +NHA_RES_BUCKET_INDEX , +/* clock_t as u64; nexthop bucket idle time */ +NHA_RES_BUCKET_IDLE_TIME , +/* u32; nexthop id assigned to the nexthop bucket */ +NHA_RES_BUCKET_NH_ID , + + __NHA_RES_BUCKET_MAX, +}; + +#define NHA_RES_BUCKET_MAX (__NHA_RES_BUCKET_MAX - 1) +#endif
include/uapi/linux/rtnetlink.h +7 -0
@@ -178,6 +178,13 @@ enum { RTM_GETVLAN,#define RTM_GETVLAN RTM_GETVLAN + RTM_NEWNEXTHOPBUCKET = 116, +#define RTM_NEWNEXTHOPBUCKET RTM_NEWNEXTHOPBUCKET + RTM_DELNEXTHOPBUCKET, +#define RTM_DELNEXTHOPBUCKET RTM_DELNEXTHOPBUCKET + RTM_GETNEXTHOPBUCKET, +#define RTM_GETNEXTHOPBUCKET RTM_GETNEXTHOPBUCKET + __RTM_MAX,#define RTM_MAX (((__RTM_MAX + 3) & ~3) - 1) };
net/ipv4/nexthop.c +2 -0
@@ -1486,6 +1486,8 @@ static struct nexthop *nexthop_create_group(struct net *net,if (cfg->nh_grp_type ==NEXTHOP_GRP_TYPE_MPATH ) nhg->mpath =1 ; +else if (cfg->nh_grp_type ==NEXTHOP_GRP_TYPE_RES ) + goto out_no_nh;WARN_ON_ONCE (nhg->mpath !=1 );
security/selinux/nlmsgtab.c +4 -1
@@ -88,6 +88,9 @@ static const struct nlmsg_perm nlmsg_route_perms[] = { RTM_NEWVLAN, NETLINK_ROUTE_SOCKET__NLMSG_WRITE }, { RTM_DELVLAN, NETLINK_ROUTE_SOCKET__NLMSG_WRITE }, { RTM_GETVLAN, NETLINK_ROUTE_SOCKET__NLMSG_READ }, + { RTM_NEWNEXTHOPBUCKET, NETLINK_ROUTE_SOCKET__NLMSG_WRITE }, + { RTM_DELNEXTHOPBUCKET, NETLINK_ROUTE_SOCKET__NLMSG_WRITE }, + { RTM_GETNEXTHOPBUCKET, NETLINK_ROUTE_SOCKET__NLMSG_READ }, }; static const struct nlmsg_perm nlmsg_tcpdiag_perms[] =
@@ -171,7 +174,7 @@ int selinux_nlmsg_lookup(u16 sclass, u16 nlmsg_type, u32 *perm)* structures at the top of this file with the new mappings * before updating the BUILD_BUG_ON() macro! */ -BUILD_BUG_ON (RTM_MAX != (RTM_NEWVLAN +3 )); +BUILD_BUG_ON (RTM_MAX != (RTM_NEWNEXTHOPBUCKET +3 )); err =nlmsg_perm (nlmsg_type, perm, nlmsg_route_perms,sizeof (nlmsg_route_perms));break ;
Signature · 1 line
2.26.2
RFC PATCH 5/13 nexthop: Add data structures for resilient group notifications
In reply to Petr MachataFrom: Ido Schimmel <[email protected]>
Add data structures that will be used for in-kernel notifications about addition / deletion of a resilient nexthop group and about changes to a hash bucket within a resilient group.
--- include/net/nexthop.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
include/net/nexthop.h +19 -0
@@ -154,11 +154,15 @@ struct nexthop {enum nexthop_event_type {NEXTHOP_EVENT_DEL ,NEXTHOP_EVENT_REPLACE , +NEXTHOP_EVENT_RES_TABLE_PRE_REPLACE , +NEXTHOP_EVENT_BUCKET_REPLACE , };enum nh_notifier_info_type {NH_NOTIFIER_INFO_TYPE_SINGLE ,NH_NOTIFIER_INFO_TYPE_GRP , +NH_NOTIFIER_INFO_TYPE_RES_TABLE , +NH_NOTIFIER_INFO_TYPE_RES_BUCKET , };struct nh_notifier_single_info {
@@ -185,6 +189,19 @@ struct nh_notifier_grp_info {struct nh_notifier_grp_entry_info nh_entries[]; }; +struct nh_notifier_res_bucket_info { +u32 bucket_index; +unsigned int idle_timer_ms; +bool force; +struct nh_notifier_single_info old_nh; +struct nh_notifier_single_info new_nh; +}; + +struct nh_notifier_res_table_info { +u32 num_nh_buckets; +struct nh_notifier_single_info nhs[]; +}; +struct nh_notifier_info {struct net *net;struct netlink_ext_ack *extack;
@@ -193,6 +210,8 @@ struct nh_notifier_info {union {struct nh_notifier_single_info *nh;struct nh_notifier_grp_info *nh_grp; +struct nh_notifier_res_table_info *nh_res_table; +struct nh_notifier_res_bucket_info *nh_res_bucket; }; };
Signature · 1 line
2.26.2
RFC PATCH 7/13 nexthop: Allow setting "offload" and "trap" indication of nexthop buckets
In reply to Petr MachataFrom: Ido Schimmel <[email protected]>
Add a function that can be called by device drivers to set "offload" or "trap" indication on nexthop buckets following nexthop notifications and other changes such as a neighbour becoming invalid.
--- include/net/nexthop.h | 2 ++ net/ipv4/nexthop.c | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+)
include/net/nexthop.h +2 -0
@@ -219,6 +219,8 @@ int register_nexthop_notifier(struct net *net, struct notifier_block *nb, struct netlink_ext_ack *extack);int unregister_nexthop_notifier (struct net *net,struct notifier_block *nb);void nexthop_set_hw_flags (struct net *net,u32 id,bool offload,bool trap); +void nexthop_bucket_set_hw_flags (struct net *net,u32 id,u32 bucket_index, +bool offload,bool trap);/* caller is holding rcu or rtnl; no reference taken to nexthop */ struct nexthop *nexthop_find_by_id (struct net *net,u32 id);
net/ipv4/nexthop.c +34 -0
@@ -3065,6 +3065,40 @@ void nexthop_set_hw_flags(struct net *net, u32 id, bool offload, bool trap) }EXPORT_SYMBOL (nexthop_set_hw_flags); +void nexthop_bucket_set_hw_flags (struct net *net,u32 id,u32 bucket_index, +bool offload,bool trap) +{ +struct nh_res_table *res_table; +struct nh_res_bucket *bucket; +struct nexthop *nexthop; +struct nh_group *nhg; + +rcu_read_lock (); + + nexthop =nexthop_find_by_id (net, id); +if (!nexthop || !nexthop->is_group) + goto out; + + nhg =rcu_dereference (nexthop->nh_grp); +if (!nhg->resilient) + goto out; + +if (bucket_index >= nhg->res_table->num_nh_buckets) + goto out; + + res_table =rcu_dereference (nhg->res_table); + bucket = &res_table->nh_buckets[bucket_index]; + bucket->nh_flags &= ~(RTNH_F_OFFLOAD |RTNH_F_TRAP ); +if (offload) + bucket->nh_flags |=RTNH_F_OFFLOAD ; +if (trap) + bucket->nh_flags |=RTNH_F_TRAP ; + +out: +rcu_read_unlock (); +} +EXPORT_SYMBOL (nexthop_bucket_set_hw_flags); + static void __net_exit nexthop_net_exit(struct net *net) {rtnl_lock ();
Signature · 1 line
2.26.2
RFC PATCH 6/13 nexthop: Implement notifiers for resilient nexthop groups
In reply to Petr MachataImplement the following notifications towards drivers:
- NEXTHOP_EVENT_REPLACE, when a resilient nexthop group is created.
- NEXTHOP_EVENT_BUCKET_REPLACE any time there is a change in assignment of next hops to hash table buckets. That includes replacements, deletions, and delayed upkeep cycles. Some bucket notifications can be vetoed by the driver, to make it possible to propagate bucket busy-ness flags from the HW back to the algorithm. Some are however forced, e.g. if a next hop is deleted, all buckets that use this next hop simply must be migrated, whether the HW wishes so or not.
- NEXTHOP_EVENT_RES_TABLE_PRE_REPLACE, before a resilient nexthop group is replaced. Usually the driver will get the bucket notifications as well, and could veto those. But in some cases, a bucket may not be migrated immediately, but during delayed upkeep, and that is too late to roll the transaction back. This notification allows the driver to take a look and veto the new proposed group up front, before anything is committed.
--- net/ipv4/nexthop.c | 320 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 308 insertions(+), 12 deletions(-)
net/ipv4/nexthop.c +308 -12
@@ -115,6 +115,37 @@ static int nh_notifier_mp_info_init(struct nh_notifier_info *info,return 0 ; } +static int nh_notifier_res_table_info_init (struct nh_notifier_info *info, +struct nh_group *nhg) +{ +struct nh_res_table *res_table =rtnl_dereference (nhg->res_table); +u32 num_nh_buckets = res_table->num_nh_buckets; +unsigned long size; +u32 i; + + info->type =NH_NOTIFIER_INFO_TYPE_RES_TABLE ; + size =struct_size (info->nh_res_table, nhs, num_nh_buckets); + info->nh_res_table =__vmalloc (size,GFP_KERNEL | __GFP_ZERO | + __GFP_NOWARN); +if (!info->nh_res_table) +return -ENOMEM ; + + info->nh_res_table->num_nh_buckets = num_nh_buckets; + +for (i =0 ; i < num_nh_buckets; i++) { +struct nh_res_bucket *bucket = &res_table->nh_buckets[i]; +struct nh_grp_entry *nhge; +struct nh_info *nhi; + + nhge =rtnl_dereference (bucket->nh_entry); + nhi =rtnl_dereference (nhge->nh->nh_info); +__nh_notifier_single_info_init (&info->nh_res_table->nhs[i], + nhi); + } + +return 0 ; +} +static int nh_notifier_grp_info_init (struct nh_notifier_info *info,const struct nexthop *nh) {
@@ -122,6 +153,8 @@ static int nh_notifier_grp_info_init(struct nh_notifier_info *info,if (nhg->mpath)return nh_notifier_mp_info_init (info, nhg); +else if (nhg->resilient) +return nh_notifier_res_table_info_init (info, nhg);return -EINVAL ; }
@@ -132,6 +165,8 @@ static void nh_notifier_grp_info_fini(struct nh_notifier_info *info,if (nhg->mpath)kfree (info->nh_grp); +else if (nhg->resilient) +vfree (info->nh_res_table); } static int nh_notifier_info_init(struct nh_notifier_info *info,
@@ -183,6 +218,107 @@ static int call_nexthop_notifiers(struct net *net,return notifier_to_errno (err); } +static int +nh_notifier_res_bucket_idle_timer_get (const struct nh_notifier_info *info, +bool force,unsigned int *p_idle_timer_ms) +{ +struct nh_res_table *res_table; +struct nh_group *nhg; +struct nexthop *nh; +int err =0 ; + +/* When 'force' is false, nexthop bucket replacement is performed +* because the bucket was deemed to be idle. In this case, capable +* listeners can choose to perform an atomic replacement: The bucket is +* only replaced if it is inactive. However, if the idle timer interval +* is smaller than the interval in which a listener is querying +* buckets' activity from the device, then atomic replacement should +* not be tried. Pass the idle timer value to listeners, so that they +* could determine which type of replacement to perform. +*/ +if (force) { + *p_idle_timer_ms =0 ; +return 0 ; + } + +rcu_read_lock (); + + nh =nexthop_find_by_id (info->net, info->id); +if (!nh) { + err = -EINVAL ; + goto out; + } + + nhg =rcu_dereference (nh->nh_grp); + res_table =rcu_dereference (nhg->res_table); + *p_idle_timer_ms =jiffies_to_msecs (res_table->idle_timer); + +out: +rcu_read_unlock (); + +return err; +} + +static int nh_notifier_res_bucket_info_init (struct nh_notifier_info *info, +u32 bucket_index,bool force, +struct nh_info *oldi, +struct nh_info *newi) +{ +unsigned int idle_timer_ms; +int err; + + err =nh_notifier_res_bucket_idle_timer_get (info, force, + &idle_timer_ms); +if (err) +return err; + + info->type =NH_NOTIFIER_INFO_TYPE_RES_BUCKET ; + info->nh_res_bucket =kzalloc (sizeof (*info->nh_res_bucket), +GFP_KERNEL ); +if (!info->nh_res_bucket) +return -ENOMEM ; + + info->nh_res_bucket->bucket_index = bucket_index; + info->nh_res_bucket->idle_timer_ms = idle_timer_ms; + info->nh_res_bucket->force = force; +__nh_notifier_single_info_init (&info->nh_res_bucket->old_nh, oldi); +__nh_notifier_single_info_init (&info->nh_res_bucket->new_nh, newi); +return 0 ; +} + +static void nh_notifier_res_bucket_info_fini (struct nh_notifier_info *info) +{ +kfree (info->nh_res_bucket); +} + +static int __call_nexthop_res_bucket_notifiers (struct net *net,u32 nhg_id, +u32 bucket_index,bool force, +struct nh_info *oldi, +struct nh_info *newi, +struct netlink_ext_ack *extack) +{ +struct nh_notifier_info info = { + .net = net, + .extack = extack, + .id = nhg_id, + }; +int err; + +if (nexthop_notifiers_is_empty (net)) +return 0 ; + + err =nh_notifier_res_bucket_info_init (&info, bucket_index, force, + oldi, newi); +if (err) +return err; + + err =blocking_notifier_call_chain (&net->nexthop.notifier_chain, +NEXTHOP_EVENT_BUCKET_REPLACE , &info); +nh_notifier_res_bucket_info_fini (&info); + +return notifier_to_errno (err); +} +/* There are three users of RES_TABLE, and NHs etc. referenced from there: * * 1) a collection of callbacks for NH maintenance. This operates under
@@ -207,6 +343,53 @@ static int call_nexthop_notifiers(struct net *net,*/ #define nh_res_dereference (p) (rcu_dereference_raw(p)) +static int call_nexthop_res_bucket_notifiers (struct net *net,u32 nhg_id, +u32 bucket_index,bool force, +struct nexthop *old_nh, +struct nexthop *new_nh, +struct netlink_ext_ack *extack) +{ +struct nh_info *oldi =nh_res_dereference (old_nh->nh_info); +struct nh_info *newi =nh_res_dereference (new_nh->nh_info); + +return __call_nexthop_res_bucket_notifiers (net, nhg_id, bucket_index, + force, oldi, newi, extack); +} + +static int call_nexthop_res_table_notifiers (struct net *net,struct nexthop *nh, +struct netlink_ext_ack *extack) +{ +struct nh_notifier_info info = { + .net = net, + .extack = extack, + }; +struct nh_group *nhg; +int err; + +ASSERT_RTNL (); + +if (nexthop_notifiers_is_empty (net)) +return 0 ; + +/* At this point, the nexthop buckets are still not populated. Only +* emit a notification with the logical nexthops, so that a listener +* could potentially veto it in case of unsupported configuration. +*/ + nhg =rtnl_dereference (nh->nh_grp); + err =nh_notifier_mp_info_init (&info, nhg); +if (err) { +NL_SET_ERR_MSG (extack,"Failed to initialize nexthop notifier info" ); +return err; + } + + err =blocking_notifier_call_chain (&net->nexthop.notifier_chain, +NEXTHOP_EVENT_RES_TABLE_PRE_REPLACE , + &info); +kfree (info.nh_grp); + +return notifier_to_errno (err); +} + static int call_nexthop_notifier(struct notifier_block *nb, struct net *net, enum nexthop_event_type event_type, struct nexthop *nh,
@@ -1144,10 +1327,12 @@ static bool nh_res_bucket_should_migrate(struct nh_res_table *res_table, }static bool nh_res_bucket_migrate (struct nh_res_table *res_table, -u32 bucket_index,bool force) +u32 bucket_index,bool notify,bool force) {struct nh_res_bucket *bucket = &res_table->nh_buckets[bucket_index];struct nh_grp_entry *new_nhge; +struct netlink_ext_ack extack; +int err; new_nhge = list_first_entry_or_null(&res_table->uw_nh_entries, struct nh_grp_entry,
@@ -1160,6 +1345,28 @@ static bool nh_res_bucket_migrate(struct nh_res_table *res_table,*/ return false; +if (notify) { +struct nh_grp_entry *old_nhge; + + old_nhge =nh_res_dereference (bucket->nh_entry); + err =call_nexthop_res_bucket_notifiers (res_table->net, + res_table->nhg_id, + bucket_index, force, + old_nhge->nh, + new_nhge->nh, &extack); +if (err) { +pr_err_ratelimited ("%s\n" , extack._msg); +if (!force) +return false; +/* It is not possible to veto a forced replacement, so +* just clear the hardware flags from the nexthop +* bucket to indicate to user space that this bucket is +* not correctly populated in hardware. +*/ + bucket->nh_flags &= ~(RTNH_F_OFFLOAD |RTNH_F_TRAP ); + } + } +nh_res_bucket_set_nh (bucket, new_nhge);nh_res_bucket_set_idle (res_table, bucket);
@@ -1170,7 +1377,7 @@ static bool nh_res_bucket_migrate(struct nh_res_table *res_table,#define NH_RES_UPKEEP_DW_MINIMUM_INTERVAL (HZ / 2) -static void nh_res_table_upkeep (struct nh_res_table *res_table) +static void nh_res_table_upkeep (struct nh_res_table *res_table,bool notify) {unsigned long now = jiffies;unsigned long deadline;
@@ -1194,7 +1401,8 @@ static void nh_res_table_upkeep(struct nh_res_table *res_table)if (nh_res_bucket_should_migrate (res_table, bucket, &deadline, &force)) { -if (!nh_res_bucket_migrate (res_table, i, force)) { +if (!nh_res_bucket_migrate (res_table, i, notify, + force)) {unsigned long idle_point;/* A driver can override the migration
@@ -1235,7 +1443,7 @@ static void nh_res_table_upkeep_dw(struct work_struct *work)struct nh_res_table *res_table; res_table = container_of(dw, struct nh_res_table, upkeep_dw); -nh_res_table_upkeep (res_table); +nh_res_table_upkeep (res_table, true); }static void nh_res_table_cancel_upkeep (struct nh_res_table *res_table)
@@ -1323,7 +1531,7 @@ static void replace_nexthop_grp_res(struct nh_group *oldg,nh_res_group_rebalance (newg, old_res_table);if (prev_has_uw && !list_empty (&old_res_table->uw_nh_entries)) old_res_table->unbalanced_since = prev_unbalanced_since; -nh_res_table_upkeep (old_res_table); +nh_res_table_upkeep (old_res_table, true); }static void nh_mp_group_rebalance (struct nh_group *nhg)
@@ -1406,9 +1614,15 @@ static void remove_nh_grp_entry(struct net *net, struct nh_grp_entry *nhge,list_del (&nhge->nh_list);nexthop_put (nhge->nh); - err =call_nexthop_notifiers (net,NEXTHOP_EVENT_REPLACE , nhp, &extack); -if (err) -pr_err ("%s\n" , extack._msg); +/* Removal of a NH from a resilient group is notified through +* bucket notifications. +*/ +if (newg->mpath) { + err =call_nexthop_notifiers (net,NEXTHOP_EVENT_REPLACE , nhp, + &extack); +if (err) +pr_err ("%s\n" , extack._msg); + }if (nlinfo)nexthop_notify (RTM_NEWNEXTHOP , nhp, nlinfo);
@@ -1561,6 +1775,16 @@ static int replace_nexthop_grp(struct net *net, struct nexthop *old,return -EINVAL ; } +/* Emit a pre-replace notification so that listeners could veto +* a potentially unsupported configuration. Otherwise, +* individual bucket replacement notifications would need to be +* vetoed, which is something that should only happen if the +* bucket is currently active. +*/ + err =call_nexthop_res_table_notifiers (net, new, extack); +if (err) +return err; +if (cfg->nh_grp_res_has_idle_timer) old_res_table->idle_timer = cfg->nh_grp_res_idle_timer;if (cfg->nh_grp_res_has_unbalanced_timer)
@@ -1610,6 +1834,71 @@ static void nh_group_v4_update(struct nh_group *nhg) nhg->has_v4 = has_v4; } +static int replace_nexthop_single_notify_res (struct net *net, +struct nh_res_table *res_table, +struct nexthop *old, +struct nh_info *oldi, +struct nh_info *newi, +struct netlink_ext_ack *extack) +{ +u32 nhg_id = res_table->nhg_id; +int err; +u32 i; + +for (i =0 ; i < res_table->num_nh_buckets; i++) { +struct nh_res_bucket *bucket = &res_table->nh_buckets[i]; +struct nh_grp_entry *nhge; + + nhge =rtnl_dereference (bucket->nh_entry); +if (nhge->nh == old) { + err =__call_nexthop_res_bucket_notifiers (net, nhg_id, + i, true, + oldi, newi, + extack); +if (err) + goto err_notify; + } + } + +return 0 ; + +err_notify: +while (i-- >0 ) { +struct nh_res_bucket *bucket = &res_table->nh_buckets[i]; +struct nh_grp_entry *nhge; + + nhge =rtnl_dereference (bucket->nh_entry); +if (nhge->nh == old) +__call_nexthop_res_bucket_notifiers (net, nhg_id, i, + true, newi, oldi, + extack); + } +return err; +} + +static int replace_nexthop_single_notify (struct net *net, +struct nexthop *group_nh, +struct nexthop *old, +struct nh_info *oldi, +struct nh_info *newi, +struct netlink_ext_ack *extack) +{ +struct nh_group *nhg =rtnl_dereference (group_nh->nh_grp); +struct nh_res_table *res_table; + +if (nhg->mpath) { +return call_nexthop_notifiers (net,NEXTHOP_EVENT_REPLACE , + group_nh, extack); + }else if (nhg->resilient) { + res_table =rtnl_dereference (nhg->res_table); +return replace_nexthop_single_notify_res (net, res_table, + old, oldi, newi, + extack); + } + +return -EINVAL ; +} +static int replace_nexthop_single (struct net *net,struct nexthop *old,struct nexthop *new,struct netlink_ext_ack *extack)
@@ -1652,8 +1941,8 @@ static int replace_nexthop_single(struct net *net, struct nexthop *old,list_for_each_entry (nhge, &old->grp_list, nh_list) {struct nexthop *nhp = nhge->nh_parent; - err =call_nexthop_notifiers (net,NEXTHOP_EVENT_REPLACE , nhp, - extack); + err =replace_nexthop_single_notify (net, nhp, old, oldi, newi, + extack);if (err) goto err_notify; }
@@ -1683,7 +1972,7 @@ static int replace_nexthop_single(struct net *net, struct nexthop *old,list_for_each_entry_continue_reverse (nhge, &old->grp_list, nh_list) {struct nexthop *nhp = nhge->nh_parent; -call_nexthop_notifiers (net,NEXTHOP_EVENT_REPLACE , nhp, extack); +replace_nexthop_single_notify (net, nhp, old, newi, oldi,NULL ); }call_nexthop_notifiers (net,NEXTHOP_EVENT_REPLACE , old, extack);return err;
@@ -1851,13 +2140,20 @@ static int insert_nexthop(struct net *net, struct nexthop *new_nh, }nh_res_group_rebalance (nhg, res_table); -nh_res_table_upkeep (res_table); + +/* Do not send bucket notifications, we do full +* notification below. +*/ +nh_res_table_upkeep (res_table, false); } }rb_link_node_rcu (&new_nh->rb_node, parent, pp);rb_insert_color (&new_nh->rb_node, root); +/* The initial insertion is a full notification for mpath as well +* as resilient groups. +*/ rc =call_nexthop_notifiers (net,NEXTHOP_EVENT_REPLACE , new_nh, extack);if (rc)rb_erase (&new_nh->rb_node, &net->nexthop.rb_root);
Signature · 1 line
2.26.2
RFC PATCH 8/13 nexthop: Allow reporting activity of nexthop buckets
In reply to Petr MachataFrom: Ido Schimmel <[email protected]>
The kernel periodically checks the idle time of nexthop buckets to determine if they are idle and can be re-populated with a new nexthop.
When the resilient nexthop group is offloaded to hardware, the kernel will not see activity on nexthop buckets unless it is reported from hardware.
Add a function that can be periodically called by device drivers to report activity on nexthop buckets after querying it from the underlying device.
--- include/net/nexthop.h | 2 ++ net/ipv4/nexthop.c | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+)
include/net/nexthop.h +2 -0
@@ -221,6 +221,8 @@ int unregister_nexthop_notifier(struct net *net, struct notifier_block *nb);void nexthop_set_hw_flags (struct net *net,u32 id,bool offload,bool trap);void nexthop_bucket_set_hw_flags (struct net *net,u32 id,u32 bucket_index,bool offload,bool trap); +void nexthop_res_grp_activity_update (struct net *net,u32 id,u32 num_buckets, +unsigned long *activity);/* caller is holding rcu or rtnl; no reference taken to nexthop */ struct nexthop *nexthop_find_by_id (struct net *net,u32 id);
net/ipv4/nexthop.c +35 -0
@@ -3099,6 +3099,41 @@ void nexthop_bucket_set_hw_flags(struct net *net, u32 id, u32 bucket_index, }EXPORT_SYMBOL (nexthop_bucket_set_hw_flags); +void nexthop_res_grp_activity_update (struct net *net,u32 id,u32 num_buckets, +unsigned long *activity) +{ +struct nh_res_table *res_table; +struct nexthop *nexthop; +struct nh_group *nhg; +u32 i; + +rcu_read_lock (); + + nexthop =nexthop_find_by_id (net, id); +if (!nexthop || !nexthop->is_group) + goto out; + + nhg =rcu_dereference (nexthop->nh_grp); +if (!nhg->resilient) + goto out; + +/* Instead of silently ignoring some buckets, demand that the sizes +* be the same. +*/ +if (num_buckets != nhg->res_table->num_nh_buckets) + goto out; + + res_table =rcu_dereference (nhg->res_table); +for (i =0 ; i < num_buckets; i++) { +if (test_bit (i, activity)) +nh_res_bucket_set_busy (&res_table->nh_buckets[i]); + } + +out: +rcu_read_unlock (); +} +EXPORT_SYMBOL (nexthop_res_grp_activity_update); + static void __net_exit nexthop_net_exit(struct net *net) {rtnl_lock ();
Signature · 1 line
2.26.2
RFC PATCH 13/13 nexthop: Enable resilient next-hop groups
In reply to Petr MachataNow that all the code is in place, stop rejecting requests to create resilient next-hop groups.
--- net/ipv4/nexthop.c | 4 ---- 1 file changed, 4 deletions(-)
net/ipv4/nexthop.c +0 -4
@@ -2437,10 +2437,6 @@ static struct nexthop *nexthop_create_group(struct net *net, }else if (cfg->nh_grp_type ==NEXTHOP_GRP_TYPE_RES ) {struct nh_res_table *res_table; -/* Bounce resilient groups for now. */ - err = -EINVAL ; - goto out_no_nh; - res_table =nexthop_res_table_alloc (net, cfg->nh_id, cfg);if (!res_table) { err = -ENOMEM ;
Signature · 1 line
2.26.2
RFC PATCH 12/13 nexthop: Notify userspace about bucket migrations
In reply to Petr MachataNexthop replacements et.al. are notified through netlink, but if a delayed work migrates buckets on the background, userspace will stay oblivious. Notify these as RTM_NEWNEXTHOPBUCKET events.
--- net/ipv4/nexthop.c | 45 +++++++++++++++++++++++++++++++++++++++------ 1 file changed, 39 insertions(+), 6 deletions(-)
net/ipv4/nexthop.c +39 -6
@@ -957,6 +957,34 @@ static int nh_fill_res_bucket(struct sk_buff *skb, struct nexthop *nh,return -EMSGSIZE ; } +static void nexthop_bucket_notify (struct nh_res_table *res_table, +u32 bucket_index) +{ +struct nh_res_bucket *bucket = &res_table->nh_buckets[bucket_index]; +struct nh_grp_entry *nhge =nh_res_dereference (bucket->nh_entry); +struct nexthop *nh = nhge->nh_parent; +struct sk_buff *skb; +int err = -ENOBUFS ; + + skb =alloc_skb (NLMSG_GOODSIZE ,GFP_KERNEL ); +if (!skb) + goto errout; + + err =nh_fill_res_bucket (skb, nh, bucket, bucket_index, +RTM_NEWNEXTHOPBUCKET ,0 ,0 ,NLM_F_REPLACE , +NULL ); +if (err <0 ) { +kfree_skb (skb); + goto errout; + } + +rtnl_notify (skb, nh->net,0 ,RTNLGRP_NEXTHOP ,NULL ,GFP_KERNEL ); +return ; +errout: +if (err <0 ) +rtnl_set_sk_err (nh->net,RTNLGRP_NEXTHOP , err); +} +static bool valid_group_nh (struct nexthop *nh,unsigned int npaths,bool *is_fdb,struct netlink_ext_ack *extack) {
@@ -1470,7 +1498,8 @@ static bool nh_res_bucket_should_migrate(struct nh_res_table *res_table, }static bool nh_res_bucket_migrate (struct nh_res_table *res_table, -u32 bucket_index,bool notify,bool force) +u32 bucket_index,bool notify, +bool notify_nl,bool force) {struct nh_res_bucket *bucket = &res_table->nh_buckets[bucket_index];struct nh_grp_entry *new_nhge;
@@ -1513,6 +1542,9 @@ static bool nh_res_bucket_migrate(struct nh_res_table *res_table,nh_res_bucket_set_nh (bucket, new_nhge);nh_res_bucket_set_idle (res_table, bucket); +if (notify_nl) +nexthop_bucket_notify (res_table, bucket_index); +if (nh_res_nhge_is_balanced (new_nhge))list_del (&new_nhge->res.uw_nh_entry);return true;
@@ -1520,7 +1552,8 @@ static bool nh_res_bucket_migrate(struct nh_res_table *res_table,#define NH_RES_UPKEEP_DW_MINIMUM_INTERVAL (HZ / 2) -static void nh_res_table_upkeep (struct nh_res_table *res_table,bool notify) +static void nh_res_table_upkeep (struct nh_res_table *res_table, +bool notify,bool notify_nl) {unsigned long now = jiffies;unsigned long deadline;
@@ -1545,7 +1578,7 @@ static void nh_res_table_upkeep(struct nh_res_table *res_table, bool notify)if (nh_res_bucket_should_migrate (res_table, bucket, &deadline, &force)) {if (!nh_res_bucket_migrate (res_table, i, notify, - force)) { + notify_nl, force)) {unsigned long idle_point;/* A driver can override the migration
@@ -1586,7 +1619,7 @@ static void nh_res_table_upkeep_dw(struct work_struct *work)struct nh_res_table *res_table; res_table = container_of(dw, struct nh_res_table, upkeep_dw); -nh_res_table_upkeep (res_table, true); +nh_res_table_upkeep (res_table, true, true); }static void nh_res_table_cancel_upkeep (struct nh_res_table *res_table)
@@ -1674,7 +1707,7 @@ static void replace_nexthop_grp_res(struct nh_group *oldg,nh_res_group_rebalance (newg, old_res_table);if (prev_has_uw && !list_empty (&old_res_table->uw_nh_entries)) old_res_table->unbalanced_since = prev_unbalanced_since; -nh_res_table_upkeep (old_res_table, true); +nh_res_table_upkeep (old_res_table, true, false); }static void nh_mp_group_rebalance (struct nh_group *nhg)
@@ -2287,7 +2320,7 @@ static int insert_nexthop(struct net *net, struct nexthop *new_nh,/* Do not send bucket notifications, we do full * notification below. */ -nh_res_table_upkeep (res_table, false); +nh_res_table_upkeep (res_table, false, false); } }
Signature · 1 line
2.26.2
RFC PATCH 9/13 nexthop: Add netlink handlers for resilient nexthop groups
In reply to Petr MachataImplement the netlink messages that allow creation and dumping of resilient nexthop groups.
--- net/ipv4/nexthop.c | 150 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 145 insertions(+), 5 deletions(-)
net/ipv4/nexthop.c +145 -5
@@ -16,6 +16,9 @@#include <net/route.h> #include <net/sock.h> +#define NH_RES_DEFAULT_IDLE_TIMER (120 * HZ) +#define NH_RES_DEFAULT_UNBALANCED_TIMER 0/* No forced rebalancing. */ +static void remove_nexthop (struct net *net,struct nexthop *nh,struct nl_info *nlinfo);
@@ -32,6 +35,7 @@ static const struct nla_policy rtm_nh_policy_new[] = { [NHA_ENCAP_TYPE] = { .type = NLA_U16 }, [NHA_ENCAP] = { .type = NLA_NESTED }, [NHA_FDB] = { .type = NLA_FLAG }, + [NHA_RES_GROUP] = { .type = NLA_NESTED }, };static const struct nla_policy rtm_nh_policy_get[] = {
@@ -45,6 +49,12 @@ static const struct nla_policy rtm_nh_policy_dump[] = { [NHA_FDB] = { .type = NLA_FLAG }, }; +static const struct nla_policy rtm_nh_res_policy_new[] = { + [NHA_RES_GROUP_BUCKETS ] = { .type =NLA_U32 }, + [NHA_RES_GROUP_IDLE_TIMER ] = { .type =NLA_U32 }, + [NHA_RES_GROUP_UNBALANCED_TIMER ] = { .type =NLA_U32 }, +}; +static bool nexthop_notifiers_is_empty (struct net *net) {return !net->nexthop.notifier_chain.head;
@@ -588,6 +598,41 @@ static void nh_res_time_set_deadline(unsigned long next_time, *deadline = next_time; } +static clock_t nh_res_table_unbalanced_time (struct nh_res_table *res_table) +{ +if (list_empty (&res_table->uw_nh_entries)) +return 0 ; +return jiffies_delta_to_clock_t (jiffies - res_table->unbalanced_since); +} + +static int nla_put_nh_group_res (struct sk_buff *skb,struct nh_group *nhg) +{ +struct nh_res_table *res_table =rtnl_dereference (nhg->res_table); +struct nlattr *nest; + + nest =nla_nest_start (skb,NHA_RES_GROUP ); +if (!nest) +return -EMSGSIZE ; + +if (nla_put_u32 (skb,NHA_RES_GROUP_BUCKETS , + res_table->num_nh_buckets) || +nla_put_u32 (skb,NHA_RES_GROUP_IDLE_TIMER , +jiffies_to_clock_t (res_table->idle_timer)) || +nla_put_u32 (skb,NHA_RES_GROUP_UNBALANCED_TIMER , +jiffies_to_clock_t (res_table->unbalanced_timer)) || +nla_put_u64_64bit (skb,NHA_RES_GROUP_UNBALANCED_TIME , +nh_res_table_unbalanced_time (res_table), +NHA_RES_GROUP_PAD )) + goto nla_put_failure; + +nla_nest_end (skb, nest); +return 0 ; + +nla_put_failure: +nla_nest_cancel (skb, nest); +return -EMSGSIZE ; +} +static int nla_put_nh_group (struct sk_buff *skb,struct nh_group *nhg) {struct nexthop_grp *p;
@@ -598,6 +643,8 @@ static int nla_put_nh_group(struct sk_buff *skb, struct nh_group *nhg)if (nhg->mpath) group_type =NEXTHOP_GRP_TYPE_MPATH ; +else if (nhg->resilient) + group_type =NEXTHOP_GRP_TYPE_RES ;if (nla_put_u16 (skb,NHA_GROUP_TYPE , group_type)) goto nla_put_failure;
@@ -613,6 +660,9 @@ static int nla_put_nh_group(struct sk_buff *skb, struct nh_group *nhg) p +=1 ; } +if (nhg->resilient &&nla_put_nh_group_res (skb, nhg)) + goto nla_put_failure; +return 0 ; nla_put_failure:
@@ -700,13 +750,26 @@ static int nh_fill_node(struct sk_buff *skb, struct nexthop *nh,return -EMSGSIZE ; } +static size_t nh_nlmsg_size_grp_res (struct nh_group *nhg) +{ +return nla_total_size (0 ) +/* NHA_RES_GROUP */ +nla_total_size (4 ) +/* NHA_RES_GROUP_BUCKETS */ +nla_total_size (4 ) +/* NHA_RES_GROUP_IDLE_TIMER */ +nla_total_size (4 ) +/* NHA_RES_GROUP_UNBALANCED_TIMER */ +nla_total_size_64bit (8 );/* NHA_RES_GROUP_UNBALANCED_TIME */ +} +static size_t nh_nlmsg_size_grp (struct nexthop *nh) {struct nh_group *nhg =rtnl_dereference (nh->nh_grp);size_t sz =sizeof (struct nexthop_grp ) * nhg->num_nh; +size_t tot =nla_total_size (sz) + +nla_total_size (2 );/* NHA_GROUP_TYPE */ + +if (nhg->resilient) + tot +=nh_nlmsg_size_grp_res (nhg); -return nla_total_size (sz) + -nla_total_size (2 );/* NHA_GROUP_TYPE */ +return tot; }static size_t nh_nlmsg_size_single (struct nexthop *nh)
@@ -876,7 +939,7 @@ static int nh_check_attr_fdb_group(struct nexthop *nh, u8 *nh_family,static int nh_check_attr_group (struct net *net,struct nlattr *tb[],size_t tb_size, -struct netlink_ext_ack *extack) +u16 nh_grp_type,struct netlink_ext_ack *extack) {unsigned int len =nla_len (tb[NHA_GROUP ]);u8 nh_family =AF_UNSPEC ;
@@ -937,8 +1000,14 @@ static int nh_check_attr_group(struct net *net,for (i =NHA_GROUP_TYPE +1 ; i < tb_size; ++i) {if (!tb[i])continue ; -if (i ==NHA_FDB ) +switch (i) { +case NHA_FDB :continue ; +case NHA_RES_GROUP : +if (nh_grp_type ==NEXTHOP_GRP_TYPE_RES ) +continue ; +break ; + }NL_SET_ERR_MSG (extack,"No other attributes can be set in nexthop groups" );return -EINVAL ;
@@ -2468,6 +2537,70 @@ static struct nexthop *nexthop_add(struct net *net, struct nh_config *cfg,return nh; } +static int rtm_nh_get_timer (struct nlattr *attr,unsigned long fallback, +unsigned long *timer_p,bool *has_p, +struct netlink_ext_ack *extack) +{ +unsigned long timer; +u32 value; + +if (!attr) { + *timer_p = fallback; + *has_p = false; +return 0 ; + } + + value =nla_get_u32 (attr); + timer =clock_t_to_jiffies (value); +if (timer == ~0UL ) { +NL_SET_ERR_MSG (extack,"Timer value too large" ); +return -EINVAL ; + } + + *timer_p = timer; + *has_p = true; +return 0 ; +} + +static int rtm_to_nh_config_grp_res (struct nlattr *res,struct nh_config *cfg, +struct netlink_ext_ack *extack) +{ +struct nlattr *tb[ARRAY_SIZE (rtm_nh_res_policy_new)] = {}; +int err; + +if (res) { + err =nla_parse_nested (tb, +ARRAY_SIZE (rtm_nh_res_policy_new) -1 , + res, rtm_nh_res_policy_new, extack); +if (err <0 ) +return err; + } + +if (tb[NHA_RES_GROUP_BUCKETS ]) { + cfg->nh_grp_res_num_buckets = +nla_get_u32 (tb[NHA_RES_GROUP_BUCKETS ]); + cfg->nh_grp_res_has_num_buckets = true; +if (!cfg->nh_grp_res_num_buckets) { +NL_SET_ERR_MSG (extack,"Number of buckets needs to be non-0" ); +return -EINVAL ; + } + } + + err =rtm_nh_get_timer (tb[NHA_RES_GROUP_IDLE_TIMER ], +NH_RES_DEFAULT_IDLE_TIMER , + &cfg->nh_grp_res_idle_timer, + &cfg->nh_grp_res_has_idle_timer, + extack); +if (err) +return err; + +return rtm_nh_get_timer (tb[NHA_RES_GROUP_UNBALANCED_TIMER ], +NH_RES_DEFAULT_UNBALANCED_TIMER , + &cfg->nh_grp_res_unbalanced_timer, + &cfg->nh_grp_res_has_unbalanced_timer, + extack); +} +static int rtm_to_nh_config (struct net *net,struct sk_buff *skb,struct nlmsghdr *nlh,struct nh_config *cfg,struct netlink_ext_ack *extack)
@@ -2546,7 +2679,14 @@ static int rtm_to_nh_config(struct net *net, struct sk_buff *skb,NL_SET_ERR_MSG (extack,"Invalid group type" ); goto out; } - err =nh_check_attr_group (net, tb,ARRAY_SIZE (tb), extack); + err =nh_check_attr_group (net, tb,ARRAY_SIZE (tb), + cfg->nh_grp_type, extack); +if (err) + goto out; + +if (cfg->nh_grp_type ==NEXTHOP_GRP_TYPE_RES ) + err =rtm_to_nh_config_grp_res (tb[NHA_RES_GROUP ], + cfg, extack);/* no other attributes should be set */ goto out;
Signature · 1 line
2.26.2
RFC PATCH 10/13 nexthop: Add netlink handlers for bucket dump
In reply to Petr MachataAdd a dump handler for resilient next hop buckets. When next-hop group ID is given, it walks buckets of that group, otherwise it walks buckets of all groups. It then dumps the buckets whose next hops match the given filtering criteria.
--- net/ipv4/nexthop.c | 283 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 283 insertions(+)
net/ipv4/nexthop.c +283 -0
@@ -55,6 +55,17 @@ static const struct nla_policy rtm_nh_res_policy_new[] = { [NHA_RES_GROUP_UNBALANCED_TIMER] = { .type = NLA_U32 }, }; +static const struct nla_policy rtm_nh_policy_dump_bucket[] = { + [NHA_ID ] = { .type =NLA_U32 }, + [NHA_OIF ] = { .type =NLA_U32 }, + [NHA_MASTER ] = { .type =NLA_U32 }, + [NHA_RES_BUCKET ] = { .type =NLA_NESTED }, +}; + +static const struct nla_policy rtm_nh_res_bucket_policy_dump[] = { + [NHA_RES_BUCKET_NH_ID ] = { .type =NLA_U32 }, +}; +static bool nexthop_notifiers_is_empty (struct net *net) {return !net->nexthop.notifier_chain.head;
@@ -883,6 +894,60 @@ static void nh_res_bucket_set_busy(struct nh_res_bucket *bucket)atomic_long_set (&bucket->used_time, (long )jiffies); } +static clock_t nh_res_bucket_idle_time (const struct nh_res_bucket *bucket) +{ +unsigned long used_time =nh_res_bucket_used_time (bucket); + +return jiffies_delta_to_clock_t (jiffies - used_time); +} + +static int nh_fill_res_bucket (struct sk_buff *skb,struct nexthop *nh, +struct nh_res_bucket *bucket,u32 bucket_index, +int event,u32 portid,u32 seq, +unsigned int nlflags, +struct netlink_ext_ack *extack) +{ +struct nh_grp_entry *nhge =nh_res_dereference (bucket->nh_entry); +struct nlmsghdr *nlh; +struct nlattr *nest; +struct nhmsg *nhm; + + nlh =nlmsg_put (skb, portid, seq, event,sizeof (*nhm), nlflags); +if (!nlh) +return -EMSGSIZE ; + + nhm =nlmsg_data (nlh); + nhm->nh_family =AF_UNSPEC ; + nhm->nh_flags = bucket->nh_flags; + nhm->nh_protocol = nh->protocol; + nhm->nh_scope =0 ; + nhm->resvd =0 ; + +if (nla_put_u32 (skb,NHA_ID , nh->id)) + goto nla_put_failure; + + nest =nla_nest_start (skb,NHA_RES_BUCKET ); +if (!nest) + goto nla_put_failure; + +if (nla_put_u32 (skb,NHA_RES_BUCKET_INDEX , bucket_index) || +nla_put_u32 (skb,NHA_RES_BUCKET_NH_ID , nhge->nh->id) || +nla_put_u64_64bit (skb,NHA_RES_BUCKET_IDLE_TIME , +nh_res_bucket_idle_time (bucket), +NHA_RES_BUCKET_PAD )) + goto nla_put_failure_nest; + +nla_nest_end (skb, nest); +nlmsg_end (skb, nlh); +return 0 ; + +nla_put_failure_nest: +nla_nest_cancel (skb, nest); +nla_put_failure: +nlmsg_cancel (skb, nlh); +return -EMSGSIZE ; +} +static bool valid_group_nh (struct nexthop *nh,unsigned int npaths,bool *is_fdb,struct netlink_ext_ack *extack) {
@@ -2911,10 +2976,12 @@ static int rtm_get_nexthop(struct sk_buff *in_skb, struct nlmsghdr *nlh, }struct nh_dump_filter { +u32 nh_id;int dev_idx;int master_idx;bool group_filter;bool fdb_filter; +u32 res_bucket_nh_id; }; static bool nh_dump_filtered(struct nexthop *nh,
@@ -3094,6 +3161,219 @@ static int rtm_dump_nexthop(struct sk_buff *skb, struct netlink_callback *cb)return err; } +static struct nexthop * +nexthop_find_group_resilient (struct net *net,u32 id, +struct netlink_ext_ack *extack) +{ +struct nh_group *nhg; +struct nexthop *nh; + + nh =nexthop_find_by_id (net, id); +if (!nh) +return ERR_PTR (-ENOENT ); + +if (!nh->is_group) { +NL_SET_ERR_MSG (extack,"Not a nexthop group" ); +return ERR_PTR (-EINVAL ); + } + + nhg =rtnl_dereference (nh->nh_grp); +if (!nhg->resilient) { +NL_SET_ERR_MSG (extack,"Nexthop group not of type resilient" ); +return ERR_PTR (-EINVAL ); + } + +return nh; +} + +static int nh_valid_dump_nhid (struct nlattr *attr,u32 *nh_id_p, +struct netlink_ext_ack *extack) +{ +u32 idx; + +if (attr) { + idx =nla_get_u32 (attr); +if (!idx) { +NL_SET_ERR_MSG (extack,"Invalid nexthop id" ); +return -EINVAL ; + } + *nh_id_p = idx; + }else { + *nh_id_p =0 ; + } + +return 0 ; +} + +static int nh_valid_dump_bucket_req (const struct nlmsghdr *nlh, +struct nh_dump_filter *filter, +struct netlink_callback *cb) +{ +struct nlattr *res_tb[ARRAY_SIZE (rtm_nh_res_bucket_policy_dump)]; +struct nlattr *tb[ARRAY_SIZE (rtm_nh_policy_dump_bucket)]; +int err; + + err =nlmsg_parse (nlh,sizeof (struct nhmsg ), tb, +ARRAY_SIZE (rtm_nh_policy_dump_bucket) -1 , + rtm_nh_policy_dump_bucket,NULL ); +if (err <0 ) +return err; + + err =nh_valid_dump_nhid (tb[NHA_ID ], &filter->nh_id, cb->extack); +if (err) +return err; + +if (tb[NHA_RES_BUCKET ]) { +size_t max =ARRAY_SIZE (rtm_nh_res_bucket_policy_dump) -1 ; + + err =nla_parse_nested (res_tb, max, + tb[NHA_RES_BUCKET ], + rtm_nh_res_bucket_policy_dump, + cb->extack); +if (err <0 ) +return err; + + err =nh_valid_dump_nhid (res_tb[NHA_RES_BUCKET_NH_ID ], + &filter->res_bucket_nh_id, + cb->extack); +if (err) +return err; + } + +return __nh_valid_dump_req (nlh, tb, filter, cb->extack); +} + +struct rtm_dump_res_bucket_ctx { +struct rtm_dump_nh_ctx nh; +u32 bucket_index; +u32 done_nh_idx;/* 1 + the index of the last fully processed NH. */ +}; + +static struct rtm_dump_res_bucket_ctx * +rtm_dump_res_bucket_ctx (struct netlink_callback *cb) +{ +struct rtm_dump_res_bucket_ctx *ctx = (void *)cb->ctx; + +BUILD_BUG_ON (sizeof (*ctx) >sizeof (cb->ctx)); +return ctx; +} + +struct rtm_dump_nexthop_bucket_data { +struct rtm_dump_res_bucket_ctx *ctx; +struct nh_dump_filter filter; +}; + +static int rtm_dump_nexthop_bucket_nh (struct sk_buff *skb, +struct netlink_callback *cb, +struct nexthop *nh, +struct rtm_dump_nexthop_bucket_data *dd) +{ +u32 portid =NETLINK_CB (cb->skb).portid; +struct nhmsg *nhm =nlmsg_data (cb->nlh); +struct nh_res_table *res_table; +struct nh_group *nhg; +u32 bucket_index; +int err; + +if (dd->ctx->nh.idx < dd->ctx->done_nh_idx) +return 0 ; + + nhg =rtnl_dereference (nh->nh_grp); + res_table =rtnl_dereference (nhg->res_table); +for (bucket_index = dd->ctx->bucket_index; + bucket_index < res_table->num_nh_buckets; + bucket_index++) { +struct nh_res_bucket *bucket; +struct nh_grp_entry *nhge; + + bucket = &res_table->nh_buckets[bucket_index]; + nhge =rtnl_dereference (bucket->nh_entry); +if (nh_dump_filtered (nhge->nh, &dd->filter, nhm->nh_family)) +continue ; + +if (dd->filter.res_bucket_nh_id && + dd->filter.res_bucket_nh_id != nhge->nh->id) +continue ; + + err =nh_fill_res_bucket (skb, nh, bucket, bucket_index, +RTM_NEWNEXTHOPBUCKET , portid, + cb->nlh->nlmsg_seq,NLM_F_MULTI , + cb->extack); +if (err <0 ) { +if (likely (skb->len)) + goto out; + goto out_err; + } + } + + dd->ctx->done_nh_idx = dd->ctx->nh.idx +1 ; + bucket_index =0 ; + +out: + err = skb->len; +out_err: + dd->ctx->bucket_index = bucket_index; +return err; +} + +static int rtm_dump_nexthop_bucket_cb (struct sk_buff *skb, +struct netlink_callback *cb, +struct nexthop *nh,void *data) +{ +struct rtm_dump_nexthop_bucket_data *dd = data; +struct nh_group *nhg; + +if (!nh->is_group) +return 0 ; + + nhg =rtnl_dereference (nh->nh_grp); +if (!nhg->resilient) +return 0 ; + +return rtm_dump_nexthop_bucket_nh (skb, cb, nh, dd); +} + +/* rtnl */ +static int rtm_dump_nexthop_bucket (struct sk_buff *skb, +struct netlink_callback *cb) +{ +struct rtm_dump_res_bucket_ctx *ctx =rtm_dump_res_bucket_ctx (cb); +struct rtm_dump_nexthop_bucket_data dd = { .ctx = ctx }; +struct net *net =sock_net (skb->sk); +struct nexthop *nh; +int err; + + err =nh_valid_dump_bucket_req (cb->nlh, &dd.filter, cb); +if (err) +return err; + +if (dd.filter.nh_id) { + nh =nexthop_find_group_resilient (net, dd.filter.nh_id, + cb->extack); +if (IS_ERR (nh)) +return PTR_ERR (nh); + err =rtm_dump_nexthop_bucket_nh (skb, cb, nh, &dd); + }else { +struct rb_root *root = &net->nexthop.rb_root; + + err =rtm_dump_walk_nexthops (skb, cb, root, &ctx->nh, + &rtm_dump_nexthop_bucket_cb, &dd); + } + +if (err <0 ) { +if (likely (skb->len)) + goto out; + goto out_err; + } + +out: + err = skb->len; +out_err: + cb->seq = net->nexthop.seq; +nl_dump_check_consistent (cb,nlmsg_hdr (skb)); +return err; +} +static void nexthop_sync_mtu (struct net_device *dev,u32 orig_mtu) {unsigned int hash =nh_dev_hashfn (dev->ifindex);
@@ -3317,6 +3597,9 @@ static int __init nexthop_init(void)rtnl_register (PF_INET6 ,RTM_NEWNEXTHOP , rtm_new_nexthop,NULL ,0 );rtnl_register (PF_INET6 ,RTM_GETNEXTHOP ,NULL , rtm_dump_nexthop,0 ); +rtnl_register (PF_UNSPEC ,RTM_GETNEXTHOPBUCKET ,NULL , + rtm_dump_nexthop_bucket,0 ); +return 0 ; }subsys_initcall (nexthop_init);
Signature · 1 line
2.26.2
RFC PATCH 11/13 nexthop: Add netlink handlers for bucket get
In reply to Petr MachataAllow getting (but not setting) individual buckets to inspect the next hop mapped therein, idle time, and flags.
--- net/ipv4/nexthop.c | 110 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 109 insertions(+), 1 deletion(-)
net/ipv4/nexthop.c +109 -1
@@ -66,6 +66,15 @@ static const struct nla_policy rtm_nh_res_bucket_policy_dump[] = { [NHA_RES_BUCKET_NH_ID] = { .type = NLA_U32 }, }; +static const struct nla_policy rtm_nh_policy_get_bucket[] = { + [NHA_ID ] = { .type =NLA_U32 }, + [NHA_RES_BUCKET ] = { .type =NLA_NESTED }, +}; + +static const struct nla_policy rtm_nh_res_bucket_policy_get[] = { + [NHA_RES_BUCKET_INDEX ] = { .type =NLA_U32 }, +}; +static bool nexthop_notifiers_is_empty (struct net *net) {return !net->nexthop.notifier_chain.head;
@@ -3374,6 +3383,105 @@ static int rtm_dump_nexthop_bucket(struct sk_buff *skb,return err; } +static int nh_valid_get_bucket_req_res_bucket (struct nlattr *res, +u32 *bucket_index, +struct netlink_ext_ack *extack) +{ +struct nlattr *tb[ARRAY_SIZE (rtm_nh_res_bucket_policy_get)]; +int err; + + err =nla_parse_nested (tb,ARRAY_SIZE (rtm_nh_res_bucket_policy_get) -1 , + res, rtm_nh_res_bucket_policy_get, extack); +if (err <0 ) +return err; + +if (!tb[NHA_RES_BUCKET_INDEX ]) { +NL_SET_ERR_MSG (extack,"Bucket index is missing" ); +return -EINVAL ; + } + + *bucket_index =nla_get_u32 (tb[NHA_RES_BUCKET_INDEX ]); +return 0 ; +} + +static int nh_valid_get_bucket_req (const struct nlmsghdr *nlh, +u32 *id,u32 *bucket_index, +struct netlink_ext_ack *extack) +{ +struct nlattr *tb[ARRAY_SIZE (rtm_nh_policy_get_bucket)]; +int err; + + err =nlmsg_parse (nlh,sizeof (struct nhmsg ), tb, +ARRAY_SIZE (rtm_nh_policy_get_bucket) -1 , + rtm_nh_policy_get_bucket, extack); +if (err <0 ) +return err; + + err =__nh_valid_get_del_req (nlh, tb, id, extack); +if (err) +return err; + +if (!tb[NHA_RES_BUCKET ]) { +NL_SET_ERR_MSG (extack,"Bucket information is missing" ); +return -EINVAL ; + } + + err =nh_valid_get_bucket_req_res_bucket (tb[NHA_RES_BUCKET ], + bucket_index, extack); +if (err) +return err; + +return 0 ; +} + +/* rtnl */ +static int rtm_get_nexthop_bucket (struct sk_buff *in_skb,struct nlmsghdr *nlh, +struct netlink_ext_ack *extack) +{ +struct net *net =sock_net (in_skb->sk); +struct nh_res_table *res_table; +struct sk_buff *skb =NULL ; +struct nh_group *nhg; +struct nexthop *nh; +u32 bucket_index; +int err; +u32 id; + + err =nh_valid_get_bucket_req (nlh, &id, &bucket_index, extack); +if (err) +return err; + + nh =nexthop_find_group_resilient (net, id, extack); +if (IS_ERR (nh)) +return PTR_ERR (nh); + + nhg =rtnl_dereference (nh->nh_grp); + res_table =rtnl_dereference (nhg->res_table); +if (bucket_index >= res_table->num_nh_buckets) { +NL_SET_ERR_MSG (extack,"Bucket index out of bounds" ); +return -ENOENT ; + } + + skb =alloc_skb (NLMSG_GOODSIZE ,GFP_KERNEL ); +if (!skb) +return -ENOBUFS ; + + err =nh_fill_res_bucket (skb, nh, &res_table->nh_buckets[bucket_index], + bucket_index,RTM_NEWNEXTHOPBUCKET , +NETLINK_CB (in_skb).portid, nlh->nlmsg_seq, +0 , extack); +if (err <0 ) { +WARN_ON (err == -EMSGSIZE ); + goto errout_free; + } + +return rtnl_unicast (skb, net,NETLINK_CB (in_skb).portid); + +errout_free: +kfree_skb (skb); +return err; +} +static void nexthop_sync_mtu (struct net_device *dev,u32 orig_mtu) {unsigned int hash =nh_dev_hashfn (dev->ifindex);
@@ -3597,7 +3705,7 @@ static int __init nexthop_init(void)rtnl_register (PF_INET6 ,RTM_NEWNEXTHOP , rtm_new_nexthop,NULL ,0 );rtnl_register (PF_INET6 ,RTM_GETNEXTHOP ,NULL , rtm_dump_nexthop,0 ); -rtnl_register (PF_UNSPEC ,RTM_GETNEXTHOPBUCKET ,NULL , +rtnl_register (PF_UNSPEC ,RTM_GETNEXTHOPBUCKET , rtm_get_nexthop_bucket, rtm_dump_nexthop_bucket,0 );return 0 ;
Signature · 1 line
2.26.2
Quoted 7 lines · Petr Machata
To illustrate the usage, consider the following commands:
# ip nexthop add id 1 via 192.0.2.2 dev dummy1 # ip nexthop add id 2 via 192.0.2.3 dev dummy1 # ip nexthop add id 10 group 1/2 type resilient \ buckets 8 idle_timer 60 unbalanced_timer 300
Quoted 6 lines
The last command creates a resilient next hop group. It will have 8 buckets, each bucket will be considered idle when no traffic hits it for at least 60 seconds, and if the table remains out of balance for 300 seconds, it will be forcefully brought into balance. (If not present in netlink message, the idle timer defaults to 120 seconds, and there is no unbalanced timer, meaning the group may remain unbalanced indefinitely.)
How did you come up with the default timer of 120 seconds?
overall this looks really good.
Quoted 8 lines · David Ahern
Quoted 7 lines · Petr Machata
To illustrate the usage, consider the following commands:
# ip nexthop add id 1 via 192.0.2.2 dev dummy1 # ip nexthop add id 2 via 192.0.2.3 dev dummy1 # ip nexthop add id 10 group 1/2 type resilient \ buckets 8 idle_timer 60 unbalanced_timer 300
Quoted 8 lines
Quoted 6 lines
The last command creates a resilient next hop group. It will have 8 buckets, each bucket will be considered idle when no traffic hits it for at least 60 seconds, and if the table remains out of balance for 300 seconds, it will be forcefully brought into balance. (If not present in netlink message, the idle timer defaults to 120 seconds, and there is no unbalanced timer, meaning the group may remain unbalanced indefinitely.)
How did you come up with the default timer of 120 seconds?
It is the default in the Cumulus Linux implementation (deployed for several years already), so we figured it should be OK.
Re: [RFC PATCH 03/13] nexthop: Add netlink defines and enumerators for resilient NH groups
In reply to Petr MachataQuoted 11 lines · Petr Machata
include/uapi/linux/nexthop.h +5
-0@@ -52,8 +53,50 @@ enum {NHA_FDB ,/* flag; nexthop belongs to a bridge fdb */ /* if NHA_FDB is added, OIF, BLACKHOLE, ENCAP cannot be set */ +/* nested; resilient nexthop group attributes */ +NHA_RES_GROUP , +/* nested; nexthop bucket attributes */ +NHA_RES_BUCKET , + __NHA_MAX, };
include/uapi/linux/nexthop.h enum { · line 72
Quoted 10 lines
include/uapi/linux/nexthop.h +8
-0#define NHA_MAX (__NHA_MAX - 1) + +enum { +NHA_RES_GROUP_UNSPEC , +/* Pad attribute for 64-bit alignment. */ +NHA_RES_GROUP_PAD =NHA_RES_GROUP_UNSPEC , + +/* u32; number of nexthop buckets in a resilient nexthop group */ +NHA_RES_GROUP_BUCKETS ,
u32 is overkill; arguably u16 (64k) should be more than enough buckets for any real use case.
Quoted 16 lines · Ido Schimmel
Quoted 15 lines · David Ahern
Quoted 13 lines · Petr Machata
To illustrate the usage, consider the following commands:
# ip nexthop add id 1 via 192.0.2.2 dev dummy1 # ip nexthop add id 2 via 192.0.2.3 dev dummy1 # ip nexthop add id 10 group 1/2 type resilient \ buckets 8 idle_timer 60 unbalanced_timer 300The last command creates a resilient next hop group. It will have 8 buckets, each bucket will be considered idle when no traffic hits it for at least 60 seconds, and if the table remains out of balance for 300 seconds, it will be forcefully brought into balance. (If not present in netlink message, the idle timer defaults to 120 seconds, and there is no unbalanced timer, meaning the group may remain unbalanced indefinitely.)
Quoted 5 lines
Quoted 1 line
How did you come up with the default timer of 120 seconds?
It is the default in the Cumulus Linux implementation (deployed for several years already), so we figured it should be OK.
Add that to the commit log.
Re: [RFC PATCH 04/13] nexthop: Add implementation of resilient next-hop groups
In reply to Petr Machatainclude/net/nexthop.h static inline bool nexthop_is_multipath(const struct nexthop *nh) · line 257
Quoted 6 lines · Petr Machata
include/net/nexthop.h +1
-1@@ -212,7 +254,7 @@ static inline bool nexthop_is_multipath(const struct nexthop *nh)struct nh_group *nh_grp; nh_grp =rcu_dereference_rtnl (nh->nh_grp); -return nh_grp->mpath; +return nh_grp->mpath || nh_grp->resilient;
That pattern shows up multiple times; since this is datapath, it would be worth adding a new bool for it (is_multipath or has_nexthops or something else along those lines)
Re: [RFC PATCH 04/13] nexthop: Add implementation of resilient next-hop groups
In reply to Petr Machatanet/ipv4/nexthop.c> @@ -734,6 +834,22 @@ static struct nexthop
Quoted 4 lines · Petr Machata
net/ipv4/nexthop.c> @@ -734,6 +834,22 @@ static struct nexthop renamed from net/ipv4/nexthop.c +0
-0
*nexthop_select_path_mp(struct nh_group *nhg, int hash)
net/ipv4/nexthop.c> @@ -734,6 +834,22 @@ static struct nexthop
Quoted 7 lines
net/ipv4/nexthop.c> @@ -734,6 +834,22 @@ static struct nexthop renamed from net/ipv4/nexthop.c +4
-0return rc; } +static struct nexthop *nexthop_select_path_res(struct nh_group *nhg, int hash) +{ + struct nh_res_table *res_table = rcu_dereference(nhg->res_table); + u32 bucket_index = hash % res_table->num_nh_buckets;
Have you considered requiring the number of buckets to be a power of 2 to avoid the modulo in the hot path? Seems like those are the more likely size options.
Quoted 17 lines · David Ahern
Quoted 16 lines · Ido Schimmel
Quoted 15 lines · David Ahern
Quoted 13 lines · Petr Machata
To illustrate the usage, consider the following commands:
# ip nexthop add id 1 via 192.0.2.2 dev dummy1 # ip nexthop add id 2 via 192.0.2.3 dev dummy1 # ip nexthop add id 10 group 1/2 type resilient \ buckets 8 idle_timer 60 unbalanced_timer 300The last command creates a resilient next hop group. It will have 8 buckets, each bucket will be considered idle when no traffic hits it for at least 60 seconds, and if the table remains out of balance for 300 seconds, it will be forcefully brought into balance. (If not present in netlink message, the idle timer defaults to 120 seconds, and there is no unbalanced timer, meaning the group may remain unbalanced indefinitely.)
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
How did you come up with the default timer of 120 seconds?
It is the default in the Cumulus Linux implementation (deployed for several years already), so we figured it should be OK.
Add that to the commit log.
OK, will add
Re: [RFC PATCH 03/13] nexthop: Add netlink defines and enumerators for resilient NH groups
In reply to David AhernQuoted 13 lines · David Ahern
Quoted 12 lines · Petr Machata
include/uapi/linux/nexthop.h +5
-0@@ -52,8 +53,50 @@ enum {NHA_FDB ,/* flag; nexthop belongs to a bridge fdb */ /* if NHA_FDB is added, OIF, BLACKHOLE, ENCAP cannot be set */ +/* nested; resilient nexthop group attributes */ +NHA_RES_GROUP , +/* nested; nexthop bucket attributes */ +NHA_RES_BUCKET , + __NHA_MAX, };
Quoted 12 lines
include/uapi/linux/nexthop.h enum { · line 72
Quoted 9 lines
include/uapi/linux/nexthop.h +8
-0#define NHA_MAX (__NHA_MAX - 1) + +enum { +NHA_RES_GROUP_UNSPEC , +/* Pad attribute for 64-bit alignment. */ +NHA_RES_GROUP_PAD =NHA_RES_GROUP_UNSPEC , + +/* u32; number of nexthop buckets in a resilient nexthop group */ +NHA_RES_GROUP_BUCKETS ,u32 is overkill; arguably u16 (64k) should be more than enough buckets for any real use case.
We wanted to make it future-proof, but I think we can live with 64k. At least in Spectrum the maximum is 4k. I don't know about other devices, but I guess it is not more than 64k.
Re: [RFC PATCH 04/13] nexthop: Add implementation of resilient next-hop groups
In reply to David AhernQuoted 9 lines · David Ahern
net/ipv4/nexthop.c> @@ -734,6 +834,22 @@ static struct nexthop
Quoted 4 lines · Petr Machata
net/ipv4/nexthop.c> @@ -734,6 +834,22 @@ static struct nexthop renamed from net/ipv4/nexthop.c +0
-0*nexthop_select_path_mp(struct
nh_group*nhg, int hash)Quoted 3 lines
return rc;
}
Quoted 8 lines
net/ipv4/nexthop.c> @@ -734,6 +834,22 @@ static struct nexthop
Quoted 4 lines
net/ipv4/nexthop.c> @@ -734,6 +834,22 @@ static struct nexthop renamed from net/ipv4/nexthop.c +4
-0+static struct nexthop *nexthop_select_path_res(struct nh_group *nhg, int hash) +{ + struct nh_res_table *res_table = rcu_dereference(nhg->res_table); + u32 bucket_index = hash % res_table->num_nh_buckets;Have you considered requiring the number of buckets to be a power of 2 to avoid the modulo in the hot path? Seems like those are the more likely size options.
We thought about it, but I think it is overly limiting. Even in Spectrum (for some sizes) it does not have to be a power of 2.
Re: [RFC PATCH 03/13] nexthop: Add netlink defines and enumerators for resilient NH groups
In reply to Ido SchimmelQuoted 27 lines · Ido Schimmel
Quoted 13 lines · David Ahern
Quoted 12 lines · Petr Machata
include/uapi/linux/nexthop.h +5
-0@@ -52,8 +53,50 @@ enum {NHA_FDB ,/* flag; nexthop belongs to a bridge fdb */ /* if NHA_FDB is added, OIF, BLACKHOLE, ENCAP cannot be set */ +/* nested; resilient nexthop group attributes */ +NHA_RES_GROUP , +/* nested; nexthop bucket attributes */ +NHA_RES_BUCKET , + __NHA_MAX, };Quoted 12 lines
include/uapi/linux/nexthop.h enum { · line 72
Quoted 9 lines
include/uapi/linux/nexthop.h +8
-0#define NHA_MAX (__NHA_MAX - 1) + +enum { +NHA_RES_GROUP_UNSPEC , +/* Pad attribute for 64-bit alignment. */ +NHA_RES_GROUP_PAD =NHA_RES_GROUP_UNSPEC , + +/* u32; number of nexthop buckets in a resilient nexthop group */ +NHA_RES_GROUP_BUCKETS ,u32 is overkill; arguably u16 (64k) should be more than enough buckets for any real use case.
Quoted 3 lines
We wanted to make it future-proof, but I think we can live with 64k. At least in Spectrum the maximum is 4k. I don't know about other devices, but I guess it is not more than 64k.
OK, no problem. I was thinking of keeping the API as u32, and tracking as u16 internally, but let's not add baggage at this stage already. Push comes to shove there can be another u32 attribute mutexed with this one.
Re: [RFC PATCH 04/13] nexthop: Add implementation of resilient next-hop groups
In reply to David AhernQuoted 11 lines · David Ahern
include/net/nexthop.h static inline bool nexthop_is_multipath(const struct nexthop *nh) · line 257
Quoted 6 lines · Petr Machata
include/net/nexthop.h +1
-1@@ -212,7 +254,7 @@ static inline bool nexthop_is_multipath(const struct nexthop *nh)struct nh_group *nh_grp; nh_grp =rcu_dereference_rtnl (nh->nh_grp); -return nh_grp->mpath; +return nh_grp->mpath || nh_grp->resilient;That pattern shows up multiple times; since this is datapath, it would be worth adding a new bool for it (
is_multipathorhas_nexthopsor something else along those lines)
OK.