Skip to content

net: Enable nexthop objects with IPv4 and IPv6 routes

David Ahern netdev 4 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest20 patches 2 repliesIn review
  2. v320 patches 2 repliesSuperseded
  3. v220 patches 2 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v119 patches6 repliesSuperseded

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/20nexthops: Add ipv6 helper to walk all fib6_nh in a nexthop structinclude/net/nexthop.h net/ipv4/nexthop.c+35 -00 replies
2/20ipv6: Handle all fib6_nh in a nexthop in fib6_drop_pcpu_fromnet/ipv6/ip6_fib.c+27 -40 replies
3/20ipv6: Handle all fib6_nh in a nexthop in rt6_device_matchnet/ipv6/route.c+52 -20 replies
4/20ipv6: Handle all fib6_nh in a nexthop in __find_rr_leafnet/ipv6/route.c+47 -20 replies
5/20ipv6: Handle all fib6_nh in a nexthop in rt6_nlmsg_sizenet/ipv6/route.c+37 -120 replies
6/20ipv6: Handle all fib6_nh in a nexthop in fib6_info_uses_devnet/ipv6/route.c+18 -00 replies
7/20ipv6: Handle all fib6_nh in a nexthop in exception handlingnet/ipv6/route.c+108 -30 replies
8/20ipv6: Handle all fib6_nh in a nexthop in __ip6_route_redirectnet/ipv6/route.c+35 -40 replies
9/20ipv6: Handle all fib6_nh in a nexthop in rt6_do_redirectnet/ipv6/route.c+19 -10 replies
10/20ipv6: Handle all fib6_nh in a nexthop in mtu updatesnet/ipv6/route.c+28 -10 replies
11/20ipv4: Allow routes to use nexthop objectsinclude/net/ip_fib.h net/ipv4/fib_frontend.c net/ipv4/fib_semantics.c+35 -00 replies
12/20ipv4: Optimization for fib_info lookup with nexthopsnet/ipv4/fib_semantics.c+65 -60 replies
13/20ipv6: Allow routes to use nexthop objectsinclude/net/ip6_fib.h net/ipv6/route.c+82 -80 replies
14/20nexthops: add support for replacenet/ipv4/nexthop.c+214 -50 replies
15/20selftests: pmtu: Move running of test into a new functiontools/testing/selftests/net/pmtu.sh+36 -270 replies
16/20selftests: pmtu: Move route installs to a new functiontools/testing/selftests/net/pmtu.sh+15 -110 replies
17/20selftests: pmtu: Add support for routing via nexthop objectstools/testing/selftests/net/pmtu.sh+126 -320 replies
18/20selftests: icmp_redirect: Add support for routing via nexthop objectstools/testing/selftests/net/icmp_redirect.sh+49 -00 replies
19/20selftests: Add test with multiple prefixes using single nexthoptools/testing/selftests/net/fib_nexthop_multiprefix.sh+290 -00 replies
20/20selftests: Add version of router_multipath.sh using nexthop objectstools/testing/selftests/net/forwarding/router_mpath_nh.sh+359 -00 replies

Reviews trailers per patch

3 of 20 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchStefano BrivioWei Wang
SeriesR-b
1/20
2/20
3/20
4/20
5/20
6/20
7/20
8/20
9/20
10/20
11/20
12/20
13/20
14/20
15/20R-b
16/20R-b
17/20R-b
18/20
19/20
20/20

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [PATCH v4 net-next 00/20] net: Enable nexthop objects with IPv4 and IPv6 routes

From: David Ahern <[email protected]>

This is the final set of the initial nexthop object work. When I started this idea almost 2 years ago, it took 18 seconds to inject 700k+ IPv4 routes with 1 hop and about 28 seconds for 4-paths. Some of that time was due to inefficiencies in 'ip', but most of it was kernel side with excessive synchronize_rcu calls in ipv4, and redundant processing validating a nexthop spec (device, gateway, encap). Worse, the time increased dramatically as the number of legs in the routes increased; for example, taking over 72 seconds for 16-path routes.

After this set, with increased dirty memory limits (fib_sync_mem sysctl), an improved ip and nexthop objects a full internet fib (743,799 routes based on a pull in January 2019) can be pushed to the kernel in 4.3 seconds. Even better, the time to insert is "almost" constant with increasing number of paths. The 'almost constant' time is due to expanding the nexthop definitions when generating notifications. A follow on patch will be sent adding a sysctl that allows an admin to avoid the nexthop expansion and truly get constant route insert time regardless of the number of paths in a route! (Useful once all programs used for a deployment that care about routes understand nexthop objects).

To be clear, 'ip' is used for benchmarking for no other reason than 'ip -batch' is a trivial to use for the tests. FRR, for example, better manages nexthops and route changes and the way those are pushed to the kernel and thus will have less userspace processing times than 'ip -batch'.

Patches 1-10 iterate over fib6_nh with a nexthop invoke a processing function per fib6_nh. Prior to nexthop objects, a fib6_info referenced a single fib6_nh. Multipath routes were added as separate fib6_info for each leg of the route and linked as siblings:

    f6i -> sibling -> sibling ... -> sibling
     |                                   |
     +--------- multipath route ---------+

With nexthop objects a single fib6_info references an external nexthop which may have a series of fib6_nh:

     f6i ---> nexthop ---> fib6_nh
                           ...
                           fib6_nh

making IPv6 routes similar to IPv4. The side effect is that a single fib6_info now indirectly references a series of fib6_nh so the code needs to walk each entry and call the local, per-fib6_nh processing function.

Patches 11 and 13 wire up use of nexthops with fib entries for IPv4 and IPv6. With these commits you can actually use nexthops with routes.

Patch 12 is an optimization for IPv4 when using nexthops in the most predominant use case (no metrics).

Patches 14 handles replace of a nexthop config.

Patches 15-18 add update pmtu and redirect tests to use both old and new routing.

Patches 19 and 20 add new tests for the nexthop infrastructure. The first is single nexthop is used by multiple prefixes to communicate with remote hosts. This is on top of the functional tests already committed. The second verifies multipath selection.

v4
- changed return to 'goto out' in patch 9 since the rcu_read_lock is held (noticed by Wei)

v3
- removed found arg in patch 7 and changed rt6_nh_remove_exception_rt to return 1 when a match is found for an exception

v2
- changed ++i to i++ in patches 1 and 14 as noticed by DaveM
- improved commit message for patch 14 (nexthop replace)
- removed the skip_fib argument to remove_nexthop; vestige of an older design

David Ahern (20):
nexthops: Add ipv6 helper to walk all fib6_nh in a nexthop struct
ipv6: Handle all fib6_nh in a nexthop in fib6_drop_pcpu_from
ipv6: Handle all fib6_nh in a nexthop in rt6_device_match
ipv6: Handle all fib6_nh in a nexthop in __find_rr_leaf
ipv6: Handle all fib6_nh in a nexthop in rt6_nlmsg_size
ipv6: Handle all fib6_nh in a nexthop in fib6_info_uses_dev
ipv6: Handle all fib6_nh in a nexthop in exception handling
ipv6: Handle all fib6_nh in a nexthop in __ip6_route_redirect
ipv6: Handle all fib6_nh in a nexthop in rt6_do_redirect
ipv6: Handle all fib6_nh in a nexthop in mtu updates
ipv4: Allow routes to use nexthop objects
ipv4: Optimization for fib_info lookup with nexthops
ipv6: Allow routes to use nexthop objects
nexthops: add support for replace
selftests: pmtu: Move running of test into a new function
selftests: pmtu: Move route installs to a new function
selftests: pmtu: Add support for routing via nexthop objects
selftests: icmp_redirect: Add support for routing via nexthop objects
selftests: Add test with multiple prefixes using single nexthop
selftests: Add version of router_multipath.sh using nexthop objects

 include/net/ip6_fib.h                              |   1 +
 include/net/ip_fib.h                               |   1 +
 include/net/nexthop.h                              |   4 +
 net/ipv4/fib_frontend.c                            |  19 +
 net/ipv4/fib_semantics.c                           |  86 +++-
 net/ipv4/nexthop.c                                 | 250 ++++++++++-
 net/ipv6/ip6_fib.c                                 |  31 +-
 net/ipv6/route.c                                   | 458 +++++++++++++++++++--
 .../selftests/net/fib_nexthop_multiprefix.sh       | 290 +++++++++++++
 .../selftests/net/forwarding/router_mpath_nh.sh    | 359 ++++++++++++++++
 tools/testing/selftests/net/icmp_redirect.sh       |  49 +++
 tools/testing/selftests/net/pmtu.sh                | 237 ++++++++---
 12 files changed, 1672 insertions(+), 113 deletions(-)
 create mode 100755 tools/testing/selftests/net/fib_nexthop_multiprefix.sh
 create mode 100755 tools/testing/selftests/net/forwarding/router_mpath_nh.sh
Signature · 1 line
2.11.0