Skip to contentSkip to conversation

net: add struct nexthop to fib{6}_info

PATCH NAK David Ahern 7 participants 29 messages May 30, 2019 – Jun 3, 2019

Subsystems (8)
lore
Share & tools
lore

David Ahern

author maintainer
#0 lore raw

cover net: add struct nexthop to fib{6}_info

From: David Ahern <[email protected]>

This sets adds 'struct nexthop' to fib_info and fib6_info. IPv4 already handles multiple fib_nh entries in a single fib_info, so the conversion to use a nexthop struct is fairly mechanical. IPv6 using a nexthop struct with a fib6_info impacts a lot of core logic which is built around the assumption of a single, builtin fib6_nh per fib6_info. To make this easier to review, this set adds nexthop to fib6_info and adds checks in most places fib6_info is used. The next set finishes the IPv6 conversion, walking through the places that need to consider all fib6_nh within a nexthop struct.

Offload drivers - mlx5, mlxsw and rocker - are changed to fail FIB entries using nexthop objects. That limitation can be removed once the drivers are updated to properly support separate nexthops.

This set starts by adding accessors for fib_nh and fib_nhs in a fib_info. This makes it easier to extract the number of nexthops in the fib entry and a specific fib_nh once the entry references a struct nexthop. Patch 2 converts more of IPv4 code to use fib_nh_common allowing a struct nexthop to use a fib6_nh with an IPv4 entry.

Patches 3 and 4 add 'struct nexthop' to fib{6}_info and update references to both take a different path when it is set. New exported functions are added to the nexthop code to validate a nexthop struct when configured for use with a fib entry. IPv4 is allowed to use a nexthop with either v4 or v6 entries. IPv6 is limited to v6 entries only. In both cases list_heads track the fib entries using a nexthop struct for fast correlation on events (e.g., device events or nexthop events like delete or replace).

The last 3 patches add hooks to drivers listening for FIB notificationas. All 3 of them reject the routes as unsupported, returning an error message to the user via extack. For mlxsw at least this is a stop gap measure until the driver is updated for proper support.

David Ahern (7):
ipv4: Use accessors for fib_info nexthop data
ipv4: Prepare for fib6_nh from a nexthop object
ipv4: Plumb support for nexthop object in a fib_info
ipv6: Plumb support for nexthop object in a fib6_info
mlxsw: Fail attempts to use routes with nexthop objects
mlx5: Fail attempts to use routes with nexthop objects
rocker: Fail attempts to use routes with nexthop objects

 drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c   |  33 ++-
 .../net/ethernet/mellanox/mlxsw/spectrum_router.c  |  33 ++-
 drivers/net/ethernet/rocker/rocker_main.c          |   4 +
 drivers/net/ethernet/rocker/rocker_ofdpa.c         |  25 +-
 include/net/ip6_fib.h                              |  11 +-
 include/net/ip6_route.h                            |  13 +-
 include/net/ip_fib.h                               |  25 +-
 include/net/nexthop.h                              | 113 +++++++++
 net/core/filter.c                                  |   3 +-
 net/ipv4/fib_frontend.c                            |  15 +-
 net/ipv4/fib_lookup.h                              |   1 +
 net/ipv4/fib_rules.c                               |   8 +-
 net/ipv4/fib_semantics.c                           | 257 ++++++++++++++-------
 net/ipv4/fib_trie.c                                |  38 ++-
 net/ipv4/nexthop.c                                 | 111 ++++++++-
 net/ipv4/route.c                                   |   5 +-
 net/ipv6/addrconf.c                                |   5 +
 net/ipv6/ip6_fib.c                                 |  22 +-
 net/ipv6/ndisc.c                                   |   3 +-
 net/ipv6/route.c                                   | 156 +++++++++++--
 20 files changed, 706 insertions(+), 175 deletions(-)
Signature · 1 line
2.11.0
author maintainer
#2 lore raw

PATCH 2/7 ipv4: Prepare for fib6_nh from a nexthop object

From: David Ahern <[email protected]>

Convert more IPv4 code to use fib_nh_common over fib_nh to enable routes to use a fib6_nh based nexthop. In the end, only code not using a nexthop object in a fib_info should directly access fib_nh in a fib_info without checking the famiy and going through fib_nh_common. Those functions will be marked when it is not directly evident.

---
 include/net/ip_fib.h     | 15 +++++++++----
 net/ipv4/fib_frontend.c  | 12 +++++------
 net/ipv4/fib_rules.c     |  4 ++--
 net/ipv4/fib_semantics.c | 55 +++++++++++++++++++++++++++++++++---------------
 net/ipv4/fib_trie.c      | 15 +++++++------
 net/ipv4/nexthop.c       |  3 ++-
 net/ipv4/route.c         |  2 +-
 7 files changed, 69 insertions(+), 37 deletions(-)
include/net/ip_fib.h +11 -4
index 42b1a806f6f5..7da8ea784029 100644
@@ -195,8 +195,8 @@ struct fib_result_nl {
 #define FIB_TABLE_HASHSZ 2
 #endif
 
-__be32 fib_info_update_nh_saddr(struct net *net, struct fib_nh *nh,
-				unsigned char scope);
+__be32 fib_info_update_nhc_saddr(struct net *net, struct fib_nh_common *nhc,
+				 unsigned char scope);
 __be32 fib_result_prefsrc(struct net *net, struct fib_result *res);
 
 #define FIB_RES_NHC(res)		((res).nhc)
@@ -455,11 +455,18 @@ static inline void fib_combine_itag(u32 *itag, const struct fib_result *res)
 {
 #ifdef CONFIG_IP_ROUTE_CLASSID
 	struct fib_nh_common *nhc = res->nhc;
-	struct fib_nh *nh = container_of(nhc, struct fib_nh, nh_common);
 #ifdef CONFIG_IP_MULTIPLE_TABLES
 	u32 rtag;
 #endif
-	*itag = nh->nh_tclassid << 16;
+	if (nhc->nhc_family == AF_INET) {
+		struct fib_nh *nh;
+
+		nh = container_of(nhc, struct fib_nh, nh_common);
+		*itag = nh->nh_tclassid << 16;
+	} else {
+		*itag = 0;
+	}
+
 #ifdef CONFIG_IP_MULTIPLE_TABLES
 	rtag = res->tclassid;
 	if (*itag == 0)
net/ipv4/fib_frontend.c +6 -6
index ab369959ce0b..8e49baa00d20 100644
@@ -235,9 +235,9 @@ static inline unsigned int __inet_dev_addr_type(struct net *net,
 	if (table) {
 		ret = RTN_UNICAST;
 		if (!fib_table_lookup(table, &fl4, &res, FIB_LOOKUP_NOREF)) {
-			struct fib_nh *nh = fib_info_nh(res.fi, 0);
+			struct fib_nh_common *nhc = fib_info_nhc(res.fi, 0);
 
-			if (!dev || dev == nh->fib_nh_dev)
+			if (!dev || dev == nhc->nhc_dev)
 				ret = res.type;
 		}
 	}
@@ -325,18 +325,18 @@ bool fib_info_nh_uses_dev(struct fib_info *fi, const struct net_device *dev)
 	int ret;
 
 	for (ret = 0; ret < fib_info_num_path(fi); ret++) {
-		const struct fib_nh *nh = fib_info_nh(fi, ret);
+		const struct fib_nh_common *nhc = fib_info_nhc(fi, ret);
 
-		if (nh->fib_nh_dev == dev) {
+		if (nhc->nhc_dev == dev) {
 			dev_match = true;
 			break;
-		} else if (l3mdev_master_ifindex_rcu(nh->fib_nh_dev) == dev->ifindex) {
+		} else if (l3mdev_master_ifindex_rcu(nhc->nhc_dev) == dev->ifindex) {
 			dev_match = true;
 			break;
 		}
 	}
 #else
-	if (fib_info_nh(fi, 0)->fib_nh_dev == dev)
+	if (fib_info_nhc(fi, 0)->nhc_dev == dev)
 		dev_match = true;
 #endif
 
net/ipv4/fib_rules.c +2 -2
index ab06fd73b343..88807c138df4 100644
@@ -147,9 +147,9 @@ static bool fib4_rule_suppress(struct fib_rule *rule, struct fib_lookup_arg *arg
 	struct net_device *dev = NULL;
 
 	if (result->fi) {
-		struct fib_nh *nh = fib_info_nh(result->fi, 0);
+		struct fib_nh_common *nhc = fib_info_nhc(result->fi, 0);
 
-		dev = nh->fib_nh_dev;
+		dev = nhc->nhc_dev;
 	}
 
 	/* do not accept result if the route does
net/ipv4/fib_semantics.c +38 -17

113 lines not on this page · raw message

net/ipv4/fib_trie.c +9 -6
index 5c8a4d21b8e0..d704d1606b8f 100644
@@ -2724,9 +2724,9 @@ static unsigned int fib_flag_trans(int type, __be32 mask, struct fib_info *fi)
 	if (type == RTN_UNREACHABLE || type == RTN_PROHIBIT)
 		flags = RTF_REJECT;
 	if (fi) {
-		const struct fib_nh *nh = fib_info_nh(fi, 0);
+		const struct fib_nh_common *nhc = fib_info_nhc(fi, 0);
 
-		if (nh->fib_nh_gw4)
+		if (nhc->nhc_gw.ipv4)
 			flags |= RTF_GATEWAY;
 	}
 	if (mask == htonl(0xFFFFFFFF))
@@ -2773,14 +2773,17 @@ static int fib_route_seq_show(struct seq_file *seq, void *v)
 		seq_setwidth(seq, 127);
 
 		if (fi) {
-			struct fib_nh *nh = fib_info_nh(fi, 0);
+			struct fib_nh_common *nhc = fib_info_nhc(fi, 0);
+			__be32 gw = 0;
+
+			if (nhc->nhc_gw_family == AF_INET)
+				gw = nhc->nhc_gw.ipv4;
 
 			seq_printf(seq,
 				   "%s\t%08X\t%08X\t%04X\t%d\t%u\t"
 				   "%d\t%08X\t%d\t%u\t%u",
-				   nh->fib_nh_dev ? nh->fib_nh_dev->name : "*",
-				   prefix,
-				   nh->fib_nh_gw4, flags, 0, 0,
+				   nhc->nhc_dev ? nhc->nhc_dev->name : "*",
+				   prefix, gw, flags, 0, 0,
 				   fi->fib_priority,
 				   mask,
 				   (fi->fib_advmss ?
net/ipv4/nexthop.c +2 -1
index 1af8a329dacb..d426829b7d32 100644
@@ -815,7 +815,8 @@ static int nh_create_ipv4(struct net *net, struct nexthop *nh,
 	err = fib_check_nh(net, fib_nh, tb_id, 0, extack);
 	if (!err) {
 		nh->nh_flags = fib_nh->fib_nh_flags;
-		fib_info_update_nh_saddr(net, fib_nh, fib_nh->fib_nh_scope);
+		fib_info_update_nhc_saddr(net, &fib_nh->nh_common,
+					  fib_nh->fib_nh_scope);
 	} else {
 		fib_nh_release(net, fib_nh);
 	}
net/ipv4/route.c +1 -1
index 05a6a8ecb574..4a1168451f3a 100644
@@ -1585,7 +1585,7 @@ static void rt_set_nexthop(struct rtable *rt, __be32 daddr,
 		ip_dst_init_metrics(&rt->dst, fi->fib_metrics);
 
 #ifdef CONFIG_IP_ROUTE_CLASSID
-		{
+		if (nhc->nhc_family == AF_INET) {
 			struct fib_nh *nh;
 
 			nh = container_of(nhc, struct fib_nh, nh_common);
Signature · 1 line
2.11.0
author maintainer
#1 lore raw

PATCH 1/7 ipv4: Use accessors for fib_info nexthop data

In reply to David Ahern

From: David Ahern <[email protected]>

Use helpers to access fib_nh and fib_nhs fields of a fib_info. Drop the fib_dev macro which is an alias for the first nexthop. Replacements:

  fib_dev        --> fib_info_nh(fi, 0)->fib_nh_dev
  fi->fib_nh     --> fib_info_nh(fi, 0)
  fi->fib_nh[i]  --> fib_info_nh(fi, i)
  fi->fib_nhs    --> fib_info_num_path(fi)

Move the existing fib_info_nhc to nexthop.h and define the new ones there. A later patch adds a check if a fib_info uses a nexthop object, and defining the helpers in nexthop.h avoid circular header dependencies.

After this all remaining open coded references to fi->fib_nhs and fi->fib_nh are in:
- fib_create_info and helpers used to lookup an existing fib_info entry, and
- the netdev event functions fib_sync_down_dev and fib_sync_up.

The latter two will not be reused for nexthops, and the fib_create_info will be updated to handle a nexthop in a fib_info.

---
 drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c   | 29 ++++++----
 .../net/ethernet/mellanox/mlxsw/spectrum_router.c  | 19 ++++---
 drivers/net/ethernet/rocker/rocker_ofdpa.c         | 25 +++++---
 include/net/ip_fib.h                               |  6 --
 include/net/nexthop.h                              | 15 +++++
 net/core/filter.c                                  |  3 +-
 net/ipv4/fib_frontend.c                            | 11 ++--
 net/ipv4/fib_lookup.h                              |  1 +
 net/ipv4/fib_rules.c                               |  8 ++-
 net/ipv4/fib_semantics.c                           | 66 ++++++++++++----------
 net/ipv4/fib_trie.c                                | 26 +++++----
 net/ipv4/route.c                                   |  3 +-
 12 files changed, 132 insertions(+), 80 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c +19 -10
index 8212bfd05733..2cbfaa8da7fc 100644
@@ -2,6 +2,7 @@
 /* Copyright (c) 2019 Mellanox Technologies. */
 
 #include <linux/netdevice.h>
+#include <net/nexthop.h>
 #include "lag.h"
 #include "lag_mp.h"
 #include "mlx5_core.h"
@@ -110,6 +111,8 @@ static void mlx5_lag_fib_route_event(struct mlx5_lag *ldev,
 				     struct fib_info *fi)
 {
 	struct lag_mp *mp = &ldev->lag_mp;
+	struct fib_nh *fib_nh0, *fib_nh1;
+	unsigned int nhs;
 
 	/* Handle delete event */
 	if (event == FIB_EVENT_ENTRY_DEL) {
@@ -120,9 +123,11 @@ static void mlx5_lag_fib_route_event(struct mlx5_lag *ldev,
 	}
 
 	/* Handle add/replace event */
-	if (fi->fib_nhs == 1) {
+	nhs = fib_info_num_path(fi);
+	if (nhs == 1) {
 		if (__mlx5_lag_is_active(ldev)) {
-			struct net_device *nh_dev = fi->fib_nh[0].fib_nh_dev;
+			struct fib_nh *nh = fib_info_nh(fi, 0);
+			struct net_device *nh_dev = nh->fib_nh_dev;
 			int i = mlx5_lag_dev_get_netdev_idx(ldev, nh_dev);
 
 			mlx5_lag_set_port_affinity(ldev, ++i);
@@ -130,14 +135,16 @@ static void mlx5_lag_fib_route_event(struct mlx5_lag *ldev,
 		return;
 	}
 
-	if (fi->fib_nhs != 2)
+	if (nhs != 2)
 		return;
 
 	/* Verify next hops are ports of the same hca */
-	if (!(fi->fib_nh[0].fib_nh_dev == ldev->pf[0].netdev &&
-	      fi->fib_nh[1].fib_nh_dev == ldev->pf[1].netdev) &&
-	    !(fi->fib_nh[0].fib_nh_dev == ldev->pf[1].netdev &&
-	      fi->fib_nh[1].fib_nh_dev == ldev->pf[0].netdev)) {
+	fib_nh0 = fib_info_nh(fi, 0);
+	fib_nh1 = fib_info_nh(fi, 1);
+	if (!(fib_nh0->fib_nh_dev == ldev->pf[0].netdev &&
+	      fib_nh1->fib_nh_dev == ldev->pf[1].netdev) &&
+	    !(fib_nh0->fib_nh_dev == ldev->pf[1].netdev &&
+	      fib_nh1->fib_nh_dev == ldev->pf[0].netdev)) {
 		mlx5_core_warn(ldev->pf[0].dev, "Multipath offload require two ports of the same HCA\n");
 		return;
 	}
@@ -174,7 +181,7 @@ static void mlx5_lag_fib_nexthop_event(struct mlx5_lag *ldev,
 			mlx5_lag_set_port_affinity(ldev, i);
 		}
 	} else if (event == FIB_EVENT_NH_ADD &&
-		   fi->fib_nhs == 2) {
+		   fib_info_num_path(fi) == 2) {
 		mlx5_lag_set_port_affinity(ldev, 0);
 	}
 }
@@ -238,6 +245,7 @@ static int mlx5_lag_fib_event(struct notifier_block *nb,
 	struct mlx5_fib_event_work *fib_work;
 	struct fib_entry_notifier_info *fen_info;
 	struct fib_nh_notifier_info *fnh_info;
+	struct net_device *fib_dev;
 	struct fib_info *fi;
 
 	if (info->family != AF_INET)
@@ -254,8 +262,9 @@ static int mlx5_lag_fib_event(struct notifier_block *nb,
 		fen_info = container_of(info, struct fib_entry_notifier_info,
 					info);
 		fi = fen_info->fi;
-		if (fi->fib_dev != ldev->pf[0].netdev &&
-		    fi->fib_dev != ldev->pf[1].netdev) {
+		fib_dev = fib_info_nh(fen_info->fi, 0)->fib_nh_dev;
+		if (fib_dev != ldev->pf[0].netdev &&
+		    fib_dev != ldev->pf[1].netdev) {
 			return NOTIFY_DONE;
 		}
 		fib_work = mlx5_lag_init_fib_work(ldev, event);
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c +11 -8
index 0ec52be7cc33..4f781358aef1 100644
@@ -21,6 +21,7 @@
 #include <net/arp.h>
 #include <net/ip_fib.h>
 #include <net/ip6_fib.h>
+#include <net/nexthop.h>
 #include <net/fib_rules.h>
 #include <net/ip_tunnels.h>
 #include <net/l3mdev.h>
@@ -3816,23 +3817,25 @@ static void mlxsw_sp_nexthop_rif_gone_sync(struct mlxsw_sp *mlxsw_sp,
 }
 
 static bool mlxsw_sp_fi_is_gateway(const struct mlxsw_sp *mlxsw_sp,
-				   const struct fib_info *fi)
+				   struct fib_info *fi)
 {
-	return fi->fib_nh->fib_nh_scope == RT_SCOPE_LINK ||
-	       mlxsw_sp_nexthop4_ipip_type(mlxsw_sp, fi->fib_nh, NULL);
+	const struct fib_nh *nh = fib_info_nh(fi, 0);
+
+	return nh->fib_nh_scope == RT_SCOPE_LINK ||
+	       mlxsw_sp_nexthop4_ipip_type(mlxsw_sp, nh, NULL);
 }
 
 static struct mlxsw_sp_nexthop_group *
 mlxsw_sp_nexthop4_group_create(struct mlxsw_sp *mlxsw_sp, struct fib_info *fi)
 {
+	unsigned int nhs = fib_info_num_path(fi);
 	struct mlxsw_sp_nexthop_group *nh_grp;
 	struct mlxsw_sp_nexthop *nh;
 	struct fib_nh *fib_nh;
 	int i;
 	int err;
 
-	nh_grp = kzalloc(struct_size(nh_grp, nexthops, fi->fib_nhs),
-			 GFP_KERNEL);
+	nh_grp = kzalloc(struct_size(nh_grp, nexthops, nhs), GFP_KERNEL);
 	if (!nh_grp)
 		return ERR_PTR(-ENOMEM);
 	nh_grp->priv = fi;
@@ -3840,11 +3843,11 @@ mlxsw_sp_nexthop4_group_create(struct mlxsw_sp *mlxsw_sp, struct fib_info *fi)
 	nh_grp->neigh_tbl = &arp_tbl;
 
 	nh_grp->gateway = mlxsw_sp_fi_is_gateway(mlxsw_sp, fi);
-	nh_grp->count = fi->fib_nhs;
+	nh_grp->count = nhs;
 	fib_info_hold(fi);
 	for (i = 0; i < nh_grp->count; i++) {
 		nh = &nh_grp->nexthops[i];
-		fib_nh = &fi->fib_nh[i];
+		fib_nh = fib_info_nh(fi, i);
 		err = mlxsw_sp_nexthop4_init(mlxsw_sp, nh_grp, nh, fib_nh);
 		if (err)
 			goto err_nexthop4_init;
@@ -4282,9 +4285,9 @@ mlxsw_sp_fib4_entry_type_set(struct mlxsw_sp *mlxsw_sp,
 			     const struct fib_entry_notifier_info *fen_info,
 			     struct mlxsw_sp_fib_entry *fib_entry)
 {
+	struct net_device *dev = fib_info_nh(fen_info->fi, 0)->fib_nh_dev;
 	union mlxsw_sp_l3addr dip = { .addr4 = htonl(fen_info->dst) };
 	u32 tb_id = mlxsw_sp_fix_tb_id(fen_info->tb_id);
-	struct net_device *dev = fen_info->fi->fib_dev;
 	struct mlxsw_sp_ipip_entry *ipip_entry;
 	struct fib_info *fi = fen_info->fi;
 
drivers/net/ethernet/rocker/rocker_ofdpa.c +16 -9
index 30a49802fb51..47ed9d41047f 100644
@@ -22,6 +22,7 @@
 #include <net/neighbour.h>
 #include <net/switchdev.h>
 #include <net/ip_fib.h>
+#include <net/nexthop.h>
 #include <net/arp.h>
 
 #include "rocker.h"
@@ -2286,8 +2287,8 @@ static int ofdpa_port_fib_ipv4(struct ofdpa_port *ofdpa_port,  __be32 dst,
 
 	/* XXX support ECMP */
 
-	nh = fi->fib_nh;
-	nh_on_port = (fi->fib_dev == ofdpa_port->dev);
+	nh = fib_info_nh(fi, 0);
+	nh_on_port = (nh->fib_nh_dev == ofdpa_port->dev);
 	has_gw = !!nh->fib_nh_gw4;
 
 	if (has_gw && nh_on_port) {
@@ -2737,11 +2738,13 @@ static int ofdpa_fib4_add(struct rocker *rocker,
 {
 	struct ofdpa *ofdpa = rocker->wpriv;
 	struct ofdpa_port *ofdpa_port;
+	struct fib_nh *nh;
 	int err;
 
 	if (ofdpa->fib_aborted)
 		return 0;
-	ofdpa_port = ofdpa_port_dev_lower_find(fen_info->fi->fib_dev, rocker);
+	nh = fib_info_nh(fen_info->fi, 0);
+	ofdpa_port = ofdpa_port_dev_lower_find(nh->fib_nh_dev, rocker);
 	if (!ofdpa_port)
 		return 0;
 	err = ofdpa_port_fib_ipv4(ofdpa_port, htonl(fen_info->dst),
@@ -2749,7 +2752,7 @@ static int ofdpa_fib4_add(struct rocker *rocker,
 				  fen_info->tb_id, 0);
 	if (err)
 		return err;
-	fen_info->fi->fib_nh->fib_nh_flags |= RTNH_F_OFFLOAD;
+	nh->fib_nh_flags |= RTNH_F_OFFLOAD;
 	return 0;
 }
 
@@ -2758,13 +2761,15 @@ static int ofdpa_fib4_del(struct rocker *rocker,
 {
 	struct ofdpa *ofdpa = rocker->wpriv;
 	struct ofdpa_port *ofdpa_port;
+	struct fib_nh *nh;
 
 	if (ofdpa->fib_aborted)
 		return 0;
-	ofdpa_port = ofdpa_port_dev_lower_find(fen_info->fi->fib_dev, rocker);
+	nh = fib_info_nh(fen_info->fi, 0);
+	ofdpa_port = ofdpa_port_dev_lower_find(nh->fib_nh_dev, rocker);
 	if (!ofdpa_port)
 		return 0;
-	fen_info->fi->fib_nh->fib_nh_flags &= ~RTNH_F_OFFLOAD;
+	nh->fib_nh_flags &= ~RTNH_F_OFFLOAD;
 	return ofdpa_port_fib_ipv4(ofdpa_port, htonl(fen_info->dst),
 				   fen_info->dst_len, fen_info->fi,
 				   fen_info->tb_id, OFDPA_OP_FLAG_REMOVE);
@@ -2784,14 +2789,16 @@ static void ofdpa_fib4_abort(struct rocker *rocker)
 
 	spin_lock_irqsave(&ofdpa->flow_tbl_lock, flags);
 	hash_for_each_safe(ofdpa->flow_tbl, bkt, tmp, flow_entry, entry) {
+		struct fib_nh *nh;
+
 		if (flow_entry->key.tbl_id !=
 		    ROCKER_OF_DPA_TABLE_ID_UNICAST_ROUTING)
 			continue;
-		ofdpa_port = ofdpa_port_dev_lower_find(flow_entry->fi->fib_dev,
-						       rocker);
+		nh = fib_info_nh(flow_entry->fi, 0);
+		ofdpa_port = ofdpa_port_dev_lower_find(nh->fib_nh_dev, rocker);
 		if (!ofdpa_port)
 			continue;
-		flow_entry->fi->fib_nh->fib_nh_flags &= ~RTNH_F_OFFLOAD;
+		nh->fib_nh_flags &= ~RTNH_F_OFFLOAD;
 		ofdpa_flow_tbl_del(ofdpa_port, OFDPA_OP_FLAG_REMOVE,
 				   flow_entry);
 	}
include/net/ip_fib.h +0 -6
index 70ba0302c8c9..42b1a806f6f5 100644
@@ -153,7 +153,6 @@ struct fib_info {
 	bool			nh_updated;
 	struct rcu_head		rcu;
 	struct fib_nh		fib_nh[0];
-#define fib_dev		fib_nh[0].fib_nh_dev
 };
 
 
@@ -190,11 +189,6 @@ struct fib_result_nl {
 	int             err;
 };
 
-static inline struct fib_nh_common *fib_info_nhc(struct fib_info *fi, int nhsel)
-{
-	return &fi->fib_nh[nhsel].nh_common;
-}
-
 #ifdef CONFIG_IP_MULTIPLE_TABLES
 #define FIB_TABLE_HASHSZ 256
 #else
include/net/nexthop.h +15 -0
index 6e1b8f53624c..e501d77b82c8 100644
@@ -192,4 +192,19 @@ static inline bool nexthop_is_blackhole(const struct nexthop *nh)
 	nhi = rcu_dereference_rtnl(nh->nh_info);
 	return nhi->reject_nh;
 }
+
+static inline unsigned int fib_info_num_path(const struct fib_info *fi)
+{
+	return fi->fib_nhs;
+}
+
+static inline struct fib_nh_common *fib_info_nhc(struct fib_info *fi, int nhsel)
+{
+	return &fi->fib_nh[nhsel].nh_common;
+}
+
+static inline struct fib_nh *fib_info_nh(struct fib_info *fi, int nhsel)
+{
+	return &fi->fib_nh[nhsel];
+}
 #endif
net/core/filter.c +2 -1
index 55bfc941d17a..2ae72bbfa6d2 100644
@@ -66,6 +66,7 @@
 #include <net/inet_hashtables.h>
 #include <net/inet6_hashtables.h>
 #include <net/ip_fib.h>
+#include <net/nexthop.h>
 #include <net/flow.h>
 #include <net/arp.h>
 #include <net/ipv6.h>
@@ -4674,7 +4675,7 @@ static int bpf_ipv4_fib_lookup(struct net *net, struct bpf_fib_lookup *params,
 	if (res.type != RTN_UNICAST)
 		return BPF_FIB_LKUP_RET_NOT_FWDED;
 
-	if (res.fi->fib_nhs > 1)
+	if (fib_info_num_path(res.fi) > 1)
 		fib_select_path(net, &res, &fl4, NULL);
 
 	if (check_mtu) {
net/ipv4/fib_frontend.c +7 -4
index 76055c66326a..ab369959ce0b 100644
@@ -43,6 +43,7 @@
 #include <net/sock.h>
 #include <net/arp.h>
 #include <net/ip_fib.h>
+#include <net/nexthop.h>
 #include <net/rtnetlink.h>
 #include <net/xfrm.h>
 #include <net/l3mdev.h>
@@ -234,7 +235,9 @@ static inline unsigned int __inet_dev_addr_type(struct net *net,
 	if (table) {
 		ret = RTN_UNICAST;
 		if (!fib_table_lookup(table, &fl4, &res, FIB_LOOKUP_NOREF)) {
-			if (!dev || dev == res.fi->fib_dev)
+			struct fib_nh *nh = fib_info_nh(res.fi, 0);
+
+			if (!dev || dev == nh->fib_nh_dev)
 				ret = res.type;
 		}
 	}
@@ -321,8 +324,8 @@ bool fib_info_nh_uses_dev(struct fib_info *fi, const struct net_device *dev)
 #ifdef CONFIG_IP_ROUTE_MULTIPATH
 	int ret;
 
-	for (ret = 0; ret < fi->fib_nhs; ret++) {
-		struct fib_nh *nh = &fi->fib_nh[ret];
+	for (ret = 0; ret < fib_info_num_path(fi); ret++) {
+		const struct fib_nh *nh = fib_info_nh(fi, ret);
 
 		if (nh->fib_nh_dev == dev) {
 			dev_match = true;
@@ -333,7 +336,7 @@ bool fib_info_nh_uses_dev(struct fib_info *fi, const struct net_device *dev)
 		}
 	}
 #else
-	if (fi->fib_nh[0].fib_nh_dev == dev)
+	if (fib_info_nh(fi, 0)->fib_nh_dev == dev)
 		dev_match = true;
 #endif
 
net/ipv4/fib_lookup.h +1 -0
index 7945f0534db7..a68b5e21ec51 100644
@@ -5,6 +5,7 @@
 #include <linux/types.h>
 #include <linux/list.h>
 #include <net/ip_fib.h>
+#include <net/nexthop.h>
 
 struct fib_alias {
 	struct hlist_node	fa_list;
net/ipv4/fib_rules.c +6 -2
index cfec3af54c8d..ab06fd73b343 100644
@@ -31,6 +31,7 @@
 #include <net/route.h>
 #include <net/tcp.h>
 #include <net/ip_fib.h>
+#include <net/nexthop.h>
 #include <net/fib_rules.h>
 
 struct fib4_rule {
@@ -145,8 +146,11 @@ static bool fib4_rule_suppress(struct fib_rule *rule, struct fib_lookup_arg *arg
 	struct fib_result *result = (struct fib_result *) arg->result;
 	struct net_device *dev = NULL;
 
-	if (result->fi)
-		dev = result->fi->fib_dev;
+	if (result->fi) {
+		struct fib_nh *nh = fib_info_nh(result->fi, 0);
+
+		dev = nh->fib_nh_dev;
+	}
 
 	/* do not accept result if the route does
 	 * not meet the required prefix length
net/ipv4/fib_semantics.c +37 -29

209 lines not on this page · raw message

net/ipv4/fib_trie.c +16 -10
index b53ecef89d59..5c8a4d21b8e0 100644
@@ -1469,7 +1469,7 @@ int fib_table_lookup(struct fib_table *tb, const struct flowi4 *flp,
 		}
 		if (fi->fib_flags & RTNH_F_DEAD)
 			continue;
-		for (nhsel = 0; nhsel < fi->fib_nhs; nhsel++) {
+		for (nhsel = 0; nhsel < fib_info_num_path(fi); nhsel++) {
 			struct fib_nh_common *nhc = fib_info_nhc(fi, nhsel);
 
 			if (nhc->nhc_flags & RTNH_F_DEAD)
@@ -2717,14 +2717,18 @@ static void fib_route_seq_stop(struct seq_file *seq, void *v)
 	rcu_read_unlock();
 }
 
-static unsigned int fib_flag_trans(int type, __be32 mask, const struct fib_info *fi)
+static unsigned int fib_flag_trans(int type, __be32 mask, struct fib_info *fi)
 {
 	unsigned int flags = 0;
 
 	if (type == RTN_UNREACHABLE || type == RTN_PROHIBIT)
 		flags = RTF_REJECT;
-	if (fi && fi->fib_nh->fib_nh_gw4)
-		flags |= RTF_GATEWAY;
+	if (fi) {
+		const struct fib_nh *nh = fib_info_nh(fi, 0);
+
+		if (nh->fib_nh_gw4)
+			flags |= RTF_GATEWAY;
+	}
 	if (mask == htonl(0xFFFFFFFF))
 		flags |= RTF_HOST;
 	flags |= RTF_UP;
@@ -2755,7 +2759,7 @@ static int fib_route_seq_show(struct seq_file *seq, void *v)
 	prefix = htonl(l->key);
 
 	hlist_for_each_entry_rcu(fa, &l->leaf, fa_list) {
-		const struct fib_info *fi = fa->fa_info;
+		struct fib_info *fi = fa->fa_info;
 		__be32 mask = inet_make_mask(KEYLENGTH - fa->fa_slen);
 		unsigned int flags = fib_flag_trans(fa->fa_type, mask, fi);
 
@@ -2768,26 +2772,28 @@ static int fib_route_seq_show(struct seq_file *seq, void *v)
 
 		seq_setwidth(seq, 127);
 
-		if (fi)
+		if (fi) {
+			struct fib_nh *nh = fib_info_nh(fi, 0);
+
 			seq_printf(seq,
 				   "%s\t%08X\t%08X\t%04X\t%d\t%u\t"
 				   "%d\t%08X\t%d\t%u\t%u",
-				   fi->fib_dev ? fi->fib_dev->name : "*",
+				   nh->fib_nh_dev ? nh->fib_nh_dev->name : "*",
 				   prefix,
-				   fi->fib_nh->fib_nh_gw4, flags, 0, 0,
+				   nh->fib_nh_gw4, flags, 0, 0,
 				   fi->fib_priority,
 				   mask,
 				   (fi->fib_advmss ?
 				    fi->fib_advmss + 40 : 0),
 				   fi->fib_window,
 				   fi->fib_rtt >> 3);
-		else
+		} else {
 			seq_printf(seq,
 				   "*\t%08X\t%08X\t%04X\t%d\t%u\t"
 				   "%d\t%08X\t%d\t%u\t%u",
 				   prefix, 0, flags, 0, 0, 0,
 				   mask, 0, 0, 0);
-
+		}
 		seq_pad(seq, '\n');
 	}
 
net/ipv4/route.c +2 -1
index 11ddc276776e..05a6a8ecb574 100644
@@ -99,6 +99,7 @@
 #include <net/inetpeer.h>
 #include <net/sock.h>
 #include <net/ip_fib.h>
+#include <net/nexthop.h>
 #include <net/arp.h>
 #include <net/tcp.h>
 #include <net/icmp.h>
@@ -1950,7 +1951,7 @@ static int ip_mkroute_input(struct sk_buff *skb,
 			    struct flow_keys *hkeys)
 {
 #ifdef CONFIG_IP_ROUTE_MULTIPATH
-	if (res->fi && res->fi->fib_nhs > 1) {
+	if (res->fi && fib_info_num_path(res->fi) > 1) {
 		int h = fib_multipath_hash(res->fi->fib_net, NULL, skb, hkeys);
 
 		fib_select_multipath(res, h);
Signature · 1 line
2.11.0
author maintainer
#5 lore raw

PATCH 5/7 mlxsw: Fail attempts to use routes with nexthop objects

In reply to David Ahern

From: David Ahern <[email protected]>

Fail attempts to use nexthop objects with routes until support can be properly added.

---
 drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c +14 -0
index 4f781358aef1..23f17ea52061 100644
@@ -6122,6 +6122,20 @@ static int mlxsw_sp_router_fib_event(struct notifier_block *nb,
 				NL_SET_ERR_MSG_MOD(info->extack, "IPv6 gateway with IPv4 route is not supported");
 				return notifier_from_errno(-EINVAL);
 			}
+			if (fen_info->fi->nh) {
+				NL_SET_ERR_MSG_MOD(info->extack, "IPv4 route with nexthop objects is not supported");
+				return notifier_from_errno(-EINVAL);
+			}
+		} else if (info->family == AF_INET6) {
+			struct fib6_entry_notifier_info *fen6_info;
+
+			fen6_info = container_of(info,
+						 struct fib6_entry_notifier_info,
+						 info);
+			if (fen6_info->rt->nh) {
+				NL_SET_ERR_MSG_MOD(info->extack, "IPv6 route with nexthop objects is not supported");
+				return notifier_from_errno(-EINVAL);
+			}
 		}
 		break;
 	}
Signature · 1 line
2.11.0
author maintainer
#3 lore raw

PATCH 3/7 ipv4: Plumb support for nexthop object in a fib_info

In reply to David Ahern

From: David Ahern <[email protected]>

Add 'struct nexthop' and nh_list list_head to fib_info. nh_list is the fib_info side of the nexthop <-> fib_info relationship.

Add fi_list list_head to 'struct nexthop' to track fib_info entries using a nexthop instance. Add __remove_nexthop_fib and add it to __remove_nexthop to walk the new list_head and mark those fib entries as dead when the nexthop is deleted.

Add a few nexthop helpers for use when a nexthop is added to fib_info. Update the fib_info functions to check for fi->nh and take a different path as needed.

The bulk of the changes are in fib_semantics.c and most of that is moving the existing change_nexthops into an else branch.

---
 include/net/ip_fib.h     |   4 ++
 include/net/nexthop.h    |  48 ++++++++++++++++
 net/ipv4/fib_semantics.c | 142 +++++++++++++++++++++++++++++++++++------------
 net/ipv4/fib_trie.c      |   7 +++
 net/ipv4/nexthop.c       |  64 +++++++++++++++++++++
 5 files changed, 229 insertions(+), 36 deletions(-)
include/net/ip_fib.h +4 -0
index 7da8ea784029..071d280de389 100644
@@ -129,9 +129,12 @@ struct fib_nh {
  * This structure contains data shared by many of routes.
  */
 
+struct nexthop;
+
 struct fib_info {
 	struct hlist_node	fib_hash;
 	struct hlist_node	fib_lhash;
+	struct list_head	nh_list;
 	struct net		*fib_net;
 	int			fib_treeref;
 	refcount_t		fib_clntref;
@@ -151,6 +154,7 @@ struct fib_info {
 	int			fib_nhs;
 	bool			fib_nh_is_v6;
 	bool			nh_updated;
+	struct nexthop		*nh;
 	struct rcu_head		rcu;
 	struct fib_nh		fib_nh[0];
 };
include/net/nexthop.h +48 -0

78 lines not on this page · raw message

net/ipv4/fib_semantics.c +106 -36

259 lines not on this page · raw message

net/ipv4/fib_trie.c +7 -0
index d704d1606b8f..716f2d66cb3f 100644
@@ -1461,6 +1461,7 @@ int fib_table_lookup(struct fib_table *tb, const struct flowi4 *flp,
 		fib_alias_accessed(fa);
 		err = fib_props[fa->fa_type].error;
 		if (unlikely(err < 0)) {
+out_reject:
 #ifdef CONFIG_IP_FIB_TRIE_STATS
 			this_cpu_inc(stats->semantic_match_passed);
 #endif
@@ -1469,6 +1470,12 @@ int fib_table_lookup(struct fib_table *tb, const struct flowi4 *flp,
 		}
 		if (fi->fib_flags & RTNH_F_DEAD)
 			continue;
+
+		if (unlikely(fi->nh && nexthop_is_blackhole(fi->nh))) {
+			err = fib_props[RTN_BLACKHOLE].error;
+			goto out_reject;
+		}
+
 		for (nhsel = 0; nhsel < fib_info_num_path(fi); nhsel++) {
 			struct fib_nh_common *nhc = fib_info_nhc(fi, nhsel);
 
net/ipv4/nexthop.c +64 -0

85 lines not on this page · raw message

Signature · 1 line
2.11.0
author maintainer
#7 lore raw

PATCH 6/7 mlx5: Fail attempts to use routes with nexthop objects

In reply to David Ahern

From: David Ahern <[email protected]>

Fail attempts to use nexthop objects with routes until support can be properly added.

---
 drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c +4 -0
index 2cbfaa8da7fc..e69766393990 100644
@@ -262,6 +262,10 @@ static int mlx5_lag_fib_event(struct notifier_block *nb,
 		fen_info = container_of(info, struct fib_entry_notifier_info,
 					info);
 		fi = fen_info->fi;
+		if (fi->nh) {
+			NL_SET_ERR_MSG_MOD(info->extack, "IPv4 route with nexthop objects is not supported");
+			return notifier_from_errno(-EINVAL);
+		}
 		fib_dev = fib_info_nh(fen_info->fi, 0)->fib_nh_dev;
 		if (fib_dev != ldev->pf[0].netdev &&
 		    fib_dev != ldev->pf[1].netdev) {
Signature · 1 line
2.11.0
author maintainer
#4 lore raw

PATCH 4/7 ipv6: Plumb support for nexthop object in a fib6_info

In reply to David Ahern

From: David Ahern <[email protected]>

Add struct nexthop and nh_list list_head to fib6_info. nh_list is the fib6_info side of the nexthop <-> fib_info relationship. Since a fib6_info referencing a nexthop object can not have 'sibling' entries (the old way of doing multipath routes), the nh_list is a union with fib6_siblings.

Add f6i_list list_head to 'struct nexthop' to track fib6_info entries using a nexthop instance. Update __remove_nexthop_fib to walk f6_list and delete fib entries using the nexthop.

Add a few nexthop helpers for use when a nexthop is added to fib6_info. Update the fib6_info references to check for nh and take a different path as needed.

---
 include/net/ip6_fib.h   |  11 ++--
 include/net/ip6_route.h |  13 +++-
 include/net/nexthop.h   |  50 ++++++++++++++++
 net/ipv4/nexthop.c      |  44 ++++++++++++++
 net/ipv6/addrconf.c     |   5 ++
 net/ipv6/ip6_fib.c      |  22 +++++--
 net/ipv6/ndisc.c        |   3 +-
 net/ipv6/route.c        | 156 +++++++++++++++++++++++++++++++++++++++++-------
 8 files changed, 268 insertions(+), 36 deletions(-)
include/net/ip6_fib.h +5 -6
index ebe5d65f97e0..1a8acd51b277 100644
@@ -146,7 +146,10 @@ struct fib6_info {
 	 * destination, but not the same gateway. nsiblings is just a cache
 	 * to speed up lookup.
 	 */
-	struct list_head		fib6_siblings;
+	union {
+		struct list_head	fib6_siblings;
+		struct list_head	nh_list;
+	};
 	unsigned int			fib6_nsiblings;
 
 	refcount_t			fib6_ref;
@@ -170,6 +173,7 @@ struct fib6_info {
 					unused:3;
 
 	struct rcu_head			rcu;
+	struct nexthop			*nh;
 	struct fib6_nh			fib6_nh[0];
 };
 
@@ -441,11 +445,6 @@ void rt6_get_prefsrc(const struct rt6_info *rt, struct in6_addr *addr)
 	rcu_read_unlock();
 }
 
-static inline struct net_device *fib6_info_nh_dev(const struct fib6_info *f6i)
-{
-	return f6i->fib6_nh->fib_nh_dev;
-}
-
 int fib6_nh_init(struct net *net, struct fib6_nh *fib6_nh,
 		 struct fib6_config *cfg, gfp_t gfp_flags,
 		 struct netlink_ext_ack *extack);
include/net/ip6_route.h +11 -2
index a6ce6ea856b9..7375a165fd98 100644
@@ -27,6 +27,7 @@ struct route_info {
 #include <linux/ip.h>
 #include <linux/ipv6.h>
 #include <linux/route.h>
+#include <net/nexthop.h>
 
 #define RT6_LOOKUP_F_IFACE		0x00000001
 #define RT6_LOOKUP_F_REACHABLE		0x00000002
@@ -66,10 +67,13 @@ static inline bool rt6_need_strict(const struct in6_addr *daddr)
 		(IPV6_ADDR_MULTICAST | IPV6_ADDR_LINKLOCAL | IPV6_ADDR_LOOPBACK);
 }
 
+/* fib entries using a nexthop object can not be coalesced into
+ * a multipath route
+ */
 static inline bool rt6_qualify_for_ecmp(const struct fib6_info *f6i)
 {
 	/* the RTF_ADDRCONF flag filters out RA's */
-	return !(f6i->fib6_flags & RTF_ADDRCONF) &&
+	return !(f6i->fib6_flags & RTF_ADDRCONF) && !f6i->nh &&
 		f6i->fib6_nh->fib_nh_gw_family;
 }
 
@@ -275,8 +279,13 @@ static inline struct in6_addr *rt6_nexthop(struct rt6_info *rt,
 
 static inline bool rt6_duplicate_nexthop(struct fib6_info *a, struct fib6_info *b)
 {
-	struct fib6_nh *nha = a->fib6_nh, *nhb = b->fib6_nh;
+	struct fib6_nh *nha, *nhb;
+
+	if (a->nh || b->nh)
+		return nexthop_cmp(a->nh, b->nh);
 
+	nha = a->fib6_nh;
+	nhb = b->fib6_nh;
 	return nha->fib_nh_dev == nhb->fib_nh_dev &&
 	       ipv6_addr_equal(&nha->fib_nh_gw6, &nhb->fib_nh_gw6) &&
 	       !lwtunnel_cmp_encap(nha->fib_nh_lws, nhb->fib_nh_lws);
include/net/nexthop.h +50 -0

66 lines not on this page · raw message

net/ipv4/nexthop.c +44 -0

68 lines not on this page · raw message

net/ipv6/addrconf.c +5 -0
index 683613e7355b..bc2a33f1a6ce 100644
@@ -2421,6 +2421,10 @@ static struct fib6_info *addrconf_get_prefix_route(const struct in6_addr *pfx,
 		goto out;
 
 	for_each_fib6_node_rt_rcu(fn) {
+		/* prefix routes only use builtin fib6_nh */
+		if (rt->nh)
+			continue;
+
 		if (rt->fib6_nh->fib_nh_dev->ifindex != dev->ifindex)
 			continue;
 		if (no_gw && rt->fib6_nh->fib_nh_gw_family)
@@ -6341,6 +6345,7 @@ void addrconf_disable_policy_idev(struct inet6_dev *idev, int val)
 	list_for_each_entry(ifa, &idev->addr_list, if_list) {
 		spin_lock(&ifa->lock);
 		if (ifa->rt) {
+			/* host routes only use builtin fib6_nh */
 			struct fib6_nh *nh = ifa->rt->fib6_nh;
 			int cpu;
 
net/ipv6/ip6_fib.c +18 -4
index cdfb8500ccae..02feda73a98e 100644
@@ -159,6 +159,7 @@ struct fib6_info *fib6_info_alloc(gfp_t gfp_flags, bool with_fib6_nh)
 	if (!f6i)
 		return NULL;
 
+	/* fib6_siblings is a union with nh_list, so this initializes both */
 	INIT_LIST_HEAD(&f6i->fib6_siblings);
 	refcount_set(&f6i->fib6_ref, 1);
 
@@ -171,7 +172,11 @@ void fib6_info_destroy_rcu(struct rcu_head *head)
 
 	WARN_ON(f6i->fib6_node);
 
-	fib6_nh_release(f6i->fib6_nh);
+	if (f6i->nh)
+		nexthop_put(f6i->nh);
+	else
+		fib6_nh_release(f6i->fib6_nh);
+
 	ip_fib_metrics_put(f6i->fib6_metrics);
 	kfree(f6i);
 }
@@ -927,6 +932,9 @@ static void fib6_purge_rt(struct fib6_info *rt, struct fib6_node *fn,
 
 	fib6_drop_pcpu_from(rt, table);
 
+	if (rt->nh && !list_empty(&rt->nh_list))
+		list_del_init(&rt->nh_list);
+
 	if (refcount_read(&rt->fib6_ref) != 1) {
 		/* This route is used as dummy address holder in some split
 		 * nodes. It is not leaked, but it still holds other resources,
@@ -1334,6 +1342,8 @@ int fib6_add(struct fib6_node *root, struct fib6_info *rt,
 
 	err = fib6_add_rt2node(fn, rt, info, extack);
 	if (!err) {
+		if (rt->nh)
+			list_add(&rt->nh_list, &rt->nh->f6i_list);
 		__fib6_update_sernum_upto_root(rt, sernum);
 		fib6_start_gc(info->nl_net, rt);
 	}
@@ -2295,9 +2305,13 @@ static int ipv6_route_seq_show(struct seq_file *seq, void *v)
 {
 	struct fib6_info *rt = v;
 	struct ipv6_route_iter *iter = seq->private;
+	struct fib6_nh *fib6_nh = rt->fib6_nh;
 	unsigned int flags = rt->fib6_flags;
 	const struct net_device *dev;
 
+	if (rt->nh)
+		fib6_nh = nexthop_fib6_nh(rt->nh);
+
 	seq_printf(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen);
 
 #ifdef CONFIG_IPV6_SUBTREES
@@ -2305,14 +2319,14 @@ static int ipv6_route_seq_show(struct seq_file *seq, void *v)
 #else
 	seq_puts(seq, "00000000000000000000000000000000 00 ");
 #endif
-	if (rt->fib6_nh->fib_nh_gw_family) {
+	if (fib6_nh->fib_nh_gw_family) {
 		flags |= RTF_GATEWAY;
-		seq_printf(seq, "%pi6", &rt->fib6_nh->fib_nh_gw6);
+		seq_printf(seq, "%pi6", &fib6_nh->fib_nh_gw6);
 	} else {
 		seq_puts(seq, "00000000000000000000000000000000");
 	}
 
-	dev = rt->fib6_nh->fib_nh_dev;
+	dev = fib6_nh->fib_nh_dev;
 	seq_printf(seq, " %08x %08x %08x %08x %8s\n",
 		   rt->fib6_metric, refcount_read(&rt->fib6_ref), 0,
 		   flags, dev ? dev->name : "");
net/ipv6/ndisc.c +1 -2
index f874dde1ee85..6e3c51109c83 100644
@@ -1289,9 +1289,8 @@ static void ndisc_router_discovery(struct sk_buff *skb)
 	    !in6_dev->cnf.accept_ra_rtr_pref)
 		pref = ICMPV6_ROUTER_PREF_MEDIUM;
 #endif
-
+	/* routes added from RAs do not use nexthop objects */
 	rt = rt6_get_dflt_router(net, &ipv6_hdr(skb)->saddr, skb->dev);
-
 	if (rt) {
 		neigh = ip6_neigh_lookup(&rt->fib6_nh->fib_nh_gw6,
 					 rt->fib6_nh->fib_nh_dev, NULL,
net/ipv6/route.c +134 -22

302 lines not on this page · raw message

Signature · 1 line
2.11.0
author maintainer
#8 lore raw

PATCH 7/7 rocker: Fail attempts to use routes with nexthop objects

In reply to David Ahern

From: David Ahern <[email protected]>

Fail attempts to use nexthop objects with routes until support can be properly added.

---
 drivers/net/ethernet/rocker/rocker_main.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/net/ethernet/rocker/rocker_main.c +4 -0
index 7ae6c124bfe9..45b3325c3a38 100644
@@ -2214,6 +2214,10 @@ static int rocker_router_fib_event(struct notifier_block *nb,
 				NL_SET_ERR_MSG_MOD(info->extack, "IPv6 gateway with IPv4 route is not supported");
 				return notifier_from_errno(-EINVAL);
 			}
+			if (fen_info->fi->nh) {
+				NL_SET_ERR_MSG_MOD(info->extack, "IPv4 route with nexthop objects is not supported");
+				return notifier_from_errno(-EINVAL);
+			}
 		}
 
 		memcpy(&fib_work->fen_info, ptr, sizeof(fib_work->fen_info));
Signature · 1 line
2.11.0
Nacked-by on the v1 series NAK
#9 lore raw
In reply to David Ahern
Quoted 62 lines · David Ahern

From: David Ahern <[email protected]>

This sets adds 'struct nexthop' to fib_info and fib6_info. IPv4 already handles multiple fib_nh entries in a single fib_info, so the conversion to use a nexthop struct is fairly mechanical. IPv6 using a nexthop struct with a fib6_info impacts a lot of core logic which is built around the assumption of a single, builtin fib6_nh per fib6_info. To make this easier to review, this set adds nexthop to fib6_info and adds checks in most places fib6_info is used. The next set finishes the IPv6 conversion, walking through the places that need to consider all fib6_nh within a nexthop struct.

Offload drivers - mlx5, mlxsw and rocker - are changed to fail FIB entries using nexthop objects. That limitation can be removed once the drivers are updated to properly support separate nexthops.

This set starts by adding accessors for fib_nh and fib_nhs in a fib_info. This makes it easier to extract the number of nexthops in the fib entry and a specific fib_nh once the entry references a struct nexthop. Patch 2 converts more of IPv4 code to use fib_nh_common allowing a struct nexthop to use a fib6_nh with an IPv4 entry.

Patches 3 and 4 add 'struct nexthop' to fib{6}_info and update references to both take a different path when it is set. New exported functions are added to the nexthop code to validate a nexthop struct when configured for use with a fib entry. IPv4 is allowed to use a nexthop with either v4 or v6 entries. IPv6 is limited to v6 entries only. In both cases list_heads track the fib entries using a nexthop struct for fast correlation on events (e.g., device events or nexthop events like delete or replace).

The last 3 patches add hooks to drivers listening for FIB notificationas. All 3 of them reject the routes as unsupported, returning an error message to the user via extack. For mlxsw at least this is a stop gap measure until the driver is updated for proper support.

David Ahern (7):
ipv4: Use accessors for fib_info nexthop data
ipv4: Prepare for fib6_nh from a nexthop object
ipv4: Plumb support for nexthop object in a fib_info
ipv6: Plumb support for nexthop object in a fib6_info
mlxsw: Fail attempts to use routes with nexthop objects
mlx5: Fail attempts to use routes with nexthop objects
rocker: Fail attempts to use routes with nexthop objects

 drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c   |  33 ++-
 .../net/ethernet/mellanox/mlxsw/spectrum_router.c  |  33 ++-
 drivers/net/ethernet/rocker/rocker_main.c          |   4 +
 drivers/net/ethernet/rocker/rocker_ofdpa.c         |  25 +-
 include/net/ip6_fib.h                              |  11 +-
 include/net/ip6_route.h                            |  13 +-
 include/net/ip_fib.h                               |  25 +-
 include/net/nexthop.h                              | 113 +++++++++
 net/core/filter.c                                  |   3 +-
 net/ipv4/fib_frontend.c                            |  15 +-
 net/ipv4/fib_lookup.h                              |   1 +
 net/ipv4/fib_rules.c                               |   8 +-
 net/ipv4/fib_semantics.c                           | 257 ++++++++++++++-------
Quoted 8 lines
 net/ipv4/fib_trie.c                                |  38 ++-
 net/ipv4/nexthop.c                                 | 111 ++++++++-
 net/ipv4/route.c                                   |   5 +-
 net/ipv6/addrconf.c                                |   5 +
 net/ipv6/ip6_fib.c                                 |  22 +-
 net/ipv6/ndisc.c                                   |   3 +-
 net/ipv6/route.c                                   | 156 +++++++++++--
 20 files changed, 706 insertions(+), 175 deletions(-)

Huge number of core changes and zero tests.

#10 lore raw
Quoted 1 line · Alexei Starovoitov

Huge number of core changes and zero tests.

As mentioned in a past response, there are a number of tests under selftests that exercise the code paths affected by this change.

#11 lore raw
Quoted 6 lines · David Ahern
Quoted 1 line · Alexei Starovoitov

Huge number of core changes and zero tests.

As mentioned in a past response, there are a number of tests under selftests that exercise the code paths affected by this change.

I see zero new tests added.

maintainer
#12 lore raw

From: Alexei Starovoitov <[email protected]>
Date: Thu, 30 May 2019 08:18:10 -0700

Quoted 9 lines · David Ahern
Quoted 6 lines · Alexei Starovoitov
Quoted 1 line

Huge number of core changes and zero tests.

As mentioned in a past response, there are a number of tests under selftests that exercise the code paths affected by this change.

I see zero new tests added.

If the existing tests give sufficient coverage, your objections are not reasonable Alexei.

#13 lore raw
Quoted 9 lines · David Miller

From: Alexei Starovoitov <[email protected]>
Date: Thu, 30 May 2019 08:18:10 -0700

Quoted 5 lines · David Ahern
Quoted 4 lines · Alexei Starovoitov
Quoted 1 line

Huge number of core changes and zero tests.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

As mentioned in a past response, there are a number of tests under selftests that exercise the code paths affected by this change.

I see zero new tests added.

If the existing tests give sufficient coverage, your objections are not reasonable Alexei.

I completely disagree. Existing tests are not sufficient.
It is a new feature for the kernel with corresponding iproute2 new features, yet there are zero tests.

#14 lore raw
Quoted 13 lines · Alexei Starovoitov
Quoted 12 lines · David Miller

From: Alexei Starovoitov <[email protected]>
Date: Thu, 30 May 2019 08:18:10 -0700

Quoted 8 lines · David Ahern
Quoted 6 lines · Alexei Starovoitov
Quoted 1 line

Huge number of core changes and zero tests.

As mentioned in a past response, there are a number of tests under selftests that exercise the code paths affected by this change.

Quoted 9 lines
Quoted 4 lines
Quoted 1 line

I see zero new tests added.

If the existing tests give sufficient coverage, your objections are not reasonable Alexei.

I completely disagree. Existing tests are not sufficient.
It is a new feature for the kernel with corresponding iproute2 new features, yet there are zero tests.

Your objection was based on changes to core code ("Huge number of core changes and zero tests"), not new code paths.

The nexthop code paths are not live yet. More changes are needed before that can happen. I have been sending the patches in small, reviewable increments to be kind to reviewers than a single 27 patch set with everything (the remaining set which is over the limit BTW).

Once iproute2 has the nexthop command (patches sent to the list) AND the RTA_NHID patches are in, I have this as the final set:

8c0b06b9813e selftests: Add test cases for nexthop objects ea5c19e4dc7c selftests: Add nexthop objects to router_multipath.sh 3be7b15d1e56 selftests: pmtu: Move running of test into a new function a896b2206ea5 selftests: pmtu: Move route installs to a new function cfa48193d0b8 selftests: pmtu: Add support for routing via nexthop objects 3d09a79208b9 selftests: icmp_redirect: Add support for routing via nexthop objects

That includes:
1. a test script doing functional validation of the nexthop code (net/ipv4/nexthop.c), along with stack integration (e.g., v6 nexthop with v4 routes) and traffic (ping).

2. updates to existing exception tests (pmtu and redirect) with the nexthop obects used for routing

3. updates to the existing router_multipath script with the nexthop obects used for routing and validating balanced selection in paths.

As always, my patches can be viewed by anyone:
https://github.com/dsahern/linux

Latest branch is:
https://github.com/dsahern/linux/tree/5.2-next-nexthops-v14

In addition to the functional and runtime tests listed above:
1. I have all kinds of MPLS and network functional tests that I run. I have already committed to sending those for inclusion, it is a matter of time.

2. The FRR team has had a kernel with these patches for 5+ months adding support for nexthops to FRR. They have their own test suites used for their development.

3. The patches are included in our 4.19 kernel and subjected to our suite of smoke and validation tests.

maintainer
#28 lore raw
In reply to Alexei Starovoitov

From: Alexei Starovoitov <[email protected]>
Date: Thu, 30 May 2019 11:27:23 -0700

Quoted 13 lines · David Miller
Quoted 12 lines

From: Alexei Starovoitov <[email protected]>
Date: Thu, 30 May 2019 08:18:10 -0700

Quoted 8 lines · David Ahern
Quoted 6 lines · Alexei Starovoitov
Quoted 1 line

Huge number of core changes and zero tests.

As mentioned in a past response, there are a number of tests under selftests that exercise the code paths affected by this change.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

I see zero new tests added.

If the existing tests give sufficient coverage, your objections are not reasonable Alexei.

I completely disagree. Existing tests are not sufficient.
It is a new feature for the kernel with corresponding iproute2 new features, yet there are zero tests.

Ok, that's tree, and I agree.

maintainer
#15 lore raw
In reply to David Ahern

From: David Ahern <[email protected]>
Date: Thu, 30 May 2019 12:52:43 -0600

Quoted 4 lines

The nexthop code paths are not live yet. More changes are needed before that can happen. I have been sending the patches in small, reviewable increments to be kind to reviewers than a single 27 patch set with everything (the remaining set which is over the limit BTW).

We definitely need tests using the new netlink attributes even if the code paths aren't live yet.

#16 lore raw
Quoted 11 lines · David Miller

From: David Ahern <[email protected]>
Date: Thu, 30 May 2019 12:52:43 -0600

Quoted 4 lines

The nexthop code paths are not live yet. More changes are needed before that can happen. I have been sending the patches in small, reviewable increments to be kind to reviewers than a single 27 patch set with everything (the remaining set which is over the limit BTW).

We definitely need tests using the new netlink attributes even if the code paths aren't live yet.

https://github.com/dsahern/linux/commit/8c0b06b9813e74c561c8626de862fb649f615cb1

#17 lore raw
In reply to David Ahern
Quoted 36 lines · David Ahern
Quoted 13 lines · Alexei Starovoitov
Quoted 12 lines · David Miller

From: Alexei Starovoitov <[email protected]>
Date: Thu, 30 May 2019 08:18:10 -0700

Quoted 8 lines · David Ahern
Quoted 6 lines · Alexei Starovoitov
Quoted 1 line

Huge number of core changes and zero tests.

As mentioned in a past response, there are a number of tests under selftests that exercise the code paths affected by this change.

Quoted 9 lines
Quoted 4 lines
Quoted 1 line

I see zero new tests added.

If the existing tests give sufficient coverage, your objections are not reasonable Alexei.

I completely disagree. Existing tests are not sufficient.
It is a new feature for the kernel with corresponding iproute2 new features, yet there are zero tests.

Your objection was based on changes to core code ("Huge number of core changes and zero tests"), not new code paths.

The nexthop code paths are not live yet. More changes are needed before that can happen. I have been sending the patches in small, reviewable increments to be kind to reviewers than a single 27 patch set with everything (the remaining set which is over the limit BTW).

Once iproute2 has the nexthop command (patches sent to the list) AND the RTA_NHID patches are in, I have this as the final set:

Quoted 7 lines

8c0b06b9813e selftests: Add test cases for nexthop objects ea5c19e4dc7c selftests: Add nexthop objects to router_multipath.sh 3be7b15d1e56 selftests: pmtu: Move running of test into a new function a896b2206ea5 selftests: pmtu: Move route installs to a new function cfa48193d0b8 selftests: pmtu: Add support for routing via nexthop objects 3d09a79208b9 selftests: icmp_redirect: Add support for routing via nexthop objects

cool. that's exactly what I was asking about.
Could you please post them to the mailing list as RFC or some not-yet-to-be-merged tag?
And in the main patch set, just say in cover letter:
"here look at this set posted separately, because of such and such"

Quoted 2 lines

As always, my patches can be viewed by anyone:
https://github.com/dsahern/linux

sure, but it's counter to the standard code review procedure we have. Pointing a link to your tree doesn't make them reviewable on the public mailing list.
I very much prefer to do code reviews by looking at the tests first. It gives me a better idea on api, use cases, etc.

Quoted 10 lines

That includes:
1. a test script doing functional validation of the nexthop code (net/ipv4/nexthop.c), along with stack integration (e.g., v6 nexthop with v4 routes) and traffic (ping).

2. updates to existing exception tests (pmtu and redirect) with the nexthop obects used for routing

3. updates to the existing router_multipath script with the nexthop obects used for routing and validating balanced selection in paths.

great.

Quoted 8 lines

Latest branch is:

   https://github.com/dsahern/linux/tree/5.2-next-nexthops-v14

In addition to the functional and runtime tests listed above:
1. I have all kinds of MPLS and network functional tests that I run. I have already committed to sending those for inclusion, it is a matter of time.

cool. first time I hear.

Quoted 3 lines

2. The FRR team has had a kernel with these patches for 5+ months adding support for nexthops to FRR. They have their own test suites used for their development.

what is FRR team? First time I hear about them as well.

Quoted 2 lines

3. The patches are included in our 4.19 kernel and subjected to our suite of smoke and validation tests.

it's also great to hear, but your kernel tree is not a substitute for netdev community with many companies contributing and running those kernels, syzbots, buildbots, etc.
I have no doubt that you test your patches, but these patches also touch the very core of networking that we all are care deeply about. Hence we want to be able to run all those tests on our kernels too.

maintainer
#18 lore raw

David, can you add some supplementary information to your cover letter et al. which seems to be part of what Alexei is asking for and seems quite reasonable?

Thanks.

#19 lore raw
Quoted 4 lines · David Miller

David, can you add some supplementary information to your cover letter et al. which seems to be part of what Alexei is asking for and seems quite reasonable?

It is not clear to me what more is wanted in the cover letter. His complaints were on lack of tests. I sent those separately; they really aren't tied to this specific set (most of them apply to the previous set which provided the uapi and core implementation). This set is mostly mechanical in adding wrappers to 2 fields and shoving existing code into the else branch of 'if (fi->nh) { new code } else { existing code }'

#20 lore raw
Quoted 6 lines · David Ahern
Quoted 4 lines · David Miller

David, can you add some supplementary information to your cover letter et al. which seems to be part of what Alexei is asking for and seems quite reasonable?

Quoted 7 lines

It is not clear to me what more is wanted in the cover letter. His complaints were on lack of tests. I sent those separately; they really aren't tied to this specific set (most of them apply to the previous set which provided the uapi and core implementation). This set is mostly mechanical in adding wrappers to 2 fields and shoving existing code into the else branch of 'if (fi->nh) { new code } else { existing code }'

Hi David

I have a bunch (about 15 ) of syzbot reports, probably caused to your latest patch series.

Do we want to stabilize first, or do you expect this new patch series to fix these issues ?

#21 lore raw
Quoted 6 lines · Eric Dumazet

I have a bunch (about 15 ) of syzbot reports, probably caused to your latest patch series.

Do we want to stabilize first, or do you expect this new patch series to fix these issues ?

Please forward. I will take a look.

#22 lore raw
Quoted 10 lines · David Ahern
Quoted 6 lines · Eric Dumazet

I have a bunch (about 15 ) of syzbot reports, probably caused to your latest patch series.

Do we want to stabilize first, or do you expect this new patch series to fix these issues ?

Please forward. I will take a look.

I will release one of them, I suspect they are duplicates.

This is why I was looking at rt6_get_pcpu_route()

I thought we were lacking a READ_ONCE() but this does not seem needed.

https://patchwork.ozlabs.org/patch/1108632/

#23 lore raw
Quoted 14 lines · Eric Dumazet
Quoted 10 lines · David Ahern
Quoted 6 lines · Eric Dumazet

I have a bunch (about 15 ) of syzbot reports, probably caused to your latest patch series.

Do we want to stabilize first, or do you expect this new patch series to fix these issues ?

Please forward. I will take a look.

Quoted 8 lines

I will release one of them, I suspect they are duplicates.

This is why I was looking at rt6_get_pcpu_route()

I thought we were lacking a READ_ONCE() but this does not seem needed.

https://patchwork.ozlabs.org/patch/1108632/

I will release a second report, please not they are no repro yet.

#24 lore raw
In reply to David Ahern
Quoted 7 lines · David Ahern
Quoted 4 lines · David Miller

David, can you add some supplementary information to your cover letter et al. which seems to be part of what Alexei is asking for and seems quite reasonable?

Quoted 6 lines

It is not clear to me what more is wanted in the cover letter. His complaints were on lack of tests. I sent those separately; they really aren't tied to this specific set (most of them apply to the previous set which provided the uapi and core implementation). This set is mostly mechanical in adding wrappers to 2 fields and shoving existing code into the else branch of 'if (fi->nh) { new code } else { existing code }'

Are you talking about this one?
https://patchwork.ozlabs.org/patch/1107892/
From single sentence of commit log it's not clear at all whether they're related to this thread.
Will they fail if run w/o this set?

#25 lore raw
Quoted 4 lines · Alexei Starovoitov

From single sentence of commit log it's not clear at all whether they're related to this thread.
Will they fail if run w/o this set?

New code in this set (if (fi->nh) {}) can not be tested yet. It can not be tested until the final patch that wires up nexthops with fib entries.

Old code in this set (else {}) is tested by existing selftests.

#26 lore raw
Quoted 9 lines · David Ahern
Quoted 4 lines · Alexei Starovoitov

From single sentence of commit log it's not clear at all whether they're related to this thread.
Will they fail if run w/o this set?

New code in this set (if (fi->nh) {}) can not be tested yet. It can not be tested until the final patch that wires up nexthops with fib entries.

Could you post the whole thing please?
This set to review/apply and whatever number of
supporting sets _including_ final tests as RFC ?

#27 lore raw
In reply to David Ahern
Quoted 8 lines · David Ahern
Quoted 4 lines · David Miller

David, can you add some supplementary information to your cover letter et al. which seems to be part of what Alexei is asking for and seems quite reasonable?

It is not clear to me what more is wanted in the cover letter.

A bit late and people in this thread probably already do know the context.

But for those like me that did not: a summary or pointer to the overall goal and design would be informative. A search dug up the RFC from last year. That definitely helped me understand the intent a bit better.

net: Improve route scalability via support for nexthop objects
https://lwn.net/Articles/763950/