Skip to contentSkip to conversation

net: adjust HW GRO and LRO feature handling

PATCH In review Jakub Kicinski 6 participants 16 messages Nov 18 – Nov 19

Subsystems (11)
lore
Share & tools
lore

Jakub Kicinski

author maintainer
#0 lore raw

cover net: adjust HW GRO and LRO feature handling

XDP has no understanding of GSO frames. GSO frame looks to XDP like a large packet. Pushing / popping headers cannot be reflected in GSO metadata. XDP redirect is completely blind to GSO since struct xdp_frame carries no GSO fields.

For those reasons most (read: all well behaved?) drivers disable LRO and HW GRO when XDP is attached.

This series lifts the auto-clearing of HW-GRO and LRO from drivers up to netdev core. This gives us uniform behavior - previously some drivers would reject XDP attach w/ HW-GRO and others would auto- -clear the HW-GRO features.

More importantly (for me) after this series the HW-GRO/LRO features should consistently remain in wanted_features, and get re-enabled after XDP is detached. I started looking at this because XDP generic tests in NIPA unintentionally clear HW-GRO, which in turn makes devmem tests fail / skip on mlx5.

Note that XDP generic mangling GSO state is its own can of worms this series doesn't try to address. There are multiple paths (xdpgeneric, tun, veth...) which end up calling XDP on skbs, with varying level of assurance that the skb is not a GSO super-frame...

Jakub Kicinski (8):
net: stop making HW-GRO depend on SW GRO
selftests: drv-net: drop the generic XDP hack from the HW GRO setup
net: state the "XDP excludes HW-GRO" rule in the core
net: state the "XDP excludes LRO" rule in the core
eth: mlx5e: drop the now-dead HW-GRO and LRO vs XDP handling
eth: bnxt_en: don't clear HW-GRO and LRO for XDP in ndo_fix_features
eth: qede: drop the now-dead HW-GRO vs XDP handling
eth: gve: drop the now-dead HW-GRO vs XDP interlock

 .../networking/net_cachelines/net_device.rst  |   1 +
 include/linux/netdev_features.h               |   6 +
 include/linux/netdevice.h                     |   6 +
 .../net/ethernet/broadcom/bnx2x/bnx2x_cmn.c   |   2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |   5 +-
 drivers/net/ethernet/google/gve/gve_main.c    |  12 --
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  32 +-----
 .../net/ethernet/qlogic/qede/qede_filter.c    |  12 +-
 drivers/net/ethernet/qlogic/qede/qede_main.c  |   1 -
 net/core/dev.c                                | 107 ++++++++++++++----
 .../testing/selftests/drivers/net/gro_lib.py  |  21 +---
 .../selftests/drivers/net/hw/gro_stats.py     |  18 ---
 12 files changed, 104 insertions(+), 119 deletions(-)
Signature · 1 line
2.56.1
author maintainer
#1 lore raw

PATCH 1/8 net: stop making HW-GRO depend on SW GRO

bnxt, qede and bnx2x clear NETIF_F_GRO_HW whenever NETIF_F_GRO is off. Nothing in the stack needs that. HW-GRO'd skbs arrive already coalesced with gso_size set, exactly like LRO frames, and LRO never depended on the software GRO bit. netif_elide_gro() only gates the software path, and the in-driver consumers of the offload key off the feature bit itself, not off NETIF_F_GRO.

The coupling gets in the way of exercising the offload on its own:
turning SW GRO off to isolate HW GRO takes HW GRO down with it on these three drivers, which is why the GRO selftests carry a generic XDP hack. The core only ever constrained GRO_HW on RXCSUM, RXFCS and LRO, and the other HW-GRO capable drivers do not couple the two either.

User visible change: "ethtool -K $dev gro off" no longer implies "rx-gro-hw off" on bnxt, qede and bnx2x.

---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c | 2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt.c       | 3 ---
 drivers/net/ethernet/qlogic/qede/qede_filter.c  | 3 +--
 3 files changed, 2 insertions(+), 6 deletions(-)
drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c +1 -1
index 6fb0619731ae..5686c4ecb944 100644
@@ -4934,7 +4934,7 @@ netdev_features_t bnx2x_fix_features(struct net_device *dev,
 	if (!(features & NETIF_F_RXCSUM))
 		features &= ~NETIF_F_LRO;
 
-	if (!(features & NETIF_F_GRO) || !bnx2x_mtu_allows_gro(dev->mtu))
+	if (!bnx2x_mtu_allows_gro(dev->mtu))
 		features &= ~NETIF_F_GRO_HW;
 	if (features & NETIF_F_GRO_HW)
 		features &= ~NETIF_F_LRO;
drivers/net/ethernet/broadcom/bnxt/bnxt.c +0 -3
index 6846a27bfef4..27c53db2c9e4 100644
@@ -14092,9 +14092,6 @@ static netdev_features_t bnxt_fix_features(struct net_device *dev,
 	if ((bp->flags & BNXT_FLAG_NO_AGG_RINGS) || bp->xdp_prog)
 		features &= ~(NETIF_F_LRO | NETIF_F_GRO_HW);
 
-	if (!(features & NETIF_F_GRO))
-		features &= ~NETIF_F_GRO_HW;
-
 	if (features & NETIF_F_GRO_HW)
 		features &= ~NETIF_F_LRO;
 
drivers/net/ethernet/qlogic/qede/qede_filter.c +1 -2
index 5bf32f1034e0..d0e203167e35 100644
@@ -915,8 +915,7 @@ netdev_features_t qede_fix_features(struct net_device *dev,
 {
 	struct qede_dev *edev = netdev_priv(dev);
 
-	if (edev->xdp_prog || edev->ndev->mtu > PAGE_SIZE ||
-	    !(features & NETIF_F_GRO))
+	if (edev->xdp_prog || edev->ndev->mtu > PAGE_SIZE)
 		features &= ~NETIF_F_GRO_HW;
 
 	return features;
Signature · 1 line
2.56.1
author maintainer
#2 lore raw

PATCH 2/8 selftests: drv-net: drop the generic XDP hack from the HW GRO setup

In reply to Jakub Kicinski

The HW GRO tests want SW GRO out of the way so that what they observe can only have come from the device. On drivers which keyed HW GRO off NETIF_F_GRO that was not expressible with ethtool alone, so the setup fell back to attaching a generic XDP program, which suppresses SW GRO as a side effect.

Those drivers no longer tie the two features together, so ask for the feature combination directly.

---
 .../testing/selftests/drivers/net/gro_lib.py  | 21 +------------------
 .../selftests/drivers/net/hw/gro_stats.py     | 18 ----------------
 2 files changed, 1 insertion(+), 38 deletions(-)
tools/testing/selftests/drivers/net/gro_lib.py +1 -20
index 875a715b2ae1..6eff19db3541 100644
@@ -263,26 +263,7 @@ def _run_gro_bin(cfg, test_name, protocol=None, num_flows=None,
         set_ethtool_feat(cfg.ifname, cfg.feat,
                          {"generic-receive-offload": False,
                           "rx-gro-hw": True,
-                          "large-receive-offload": False},
-                          check=False)
-
-        # Some NICs treat HW GRO as a GRO sub-feature so disabling GRO
-        # will also clear HW GRO. Use a hack of installing XDP generic
-        # to skip SW GRO, even when enabled.
-        feat = ethtool(f"-k {cfg.ifname}", json=True)[0]
-        if not feat["rx-gro-hw"]["active"]:
-            ksft_pr("Driver clears HW GRO and SW GRO is cleared, using generic XDP workaround")
-            prog = cfg.net_lib_dir / "xdp_dummy.bpf.o"
-            ip(f"link set dev {cfg.ifname} xdpgeneric obj {prog} sec xdp")
-            defer(ip, f"link set dev {cfg.ifname} xdpgeneric off")
-
-            # Attaching XDP may change features, fetch the latest state
-            feat = ethtool(f"-k {cfg.ifname}", json=True)[0]
-
-            set_ethtool_feat(cfg.ifname, feat,
-                             {"generic-receive-offload": True,
-                              "rx-gro-hw": True,
-                              "large-receive-offload": False})
+                          "large-receive-offload": False})
     elif mode == "lro":
         # netdevsim advertises LRO for feature inheritance testing with
         # bonding/team tests but it doesn't actually perform the offload
tools/testing/selftests/drivers/net/hw/gro_stats.py +0 -18
index 70e76e3888bd..569ca81368e6 100755
@@ -173,24 +173,6 @@ def _run_gro_test(cfg, test_name, num_flows=None, ignore_fail=False,
                        "rx-gro-hw": True,
                        "large-receive-offload": False})
 
-    # Some NICs treat HW GRO as a GRO sub-feature so disabling GRO
-    # will also clear HW GRO. Use a hack of installing XDP generic
-    # to skip SW GRO, even when enabled.
-    feat = ethtool(f"-k {cfg.ifname}", json=True)[0]
-    if not feat["rx-gro-hw"]["active"]:
-        ksft_pr("Driver clears HW GRO when SW GRO is cleared, using generic XDP workaround")
-        prog = cfg.net_lib_dir / "xdp_dummy.bpf.o"
-        ip(f"link set dev {cfg.ifname} xdpgeneric obj {prog} sec xdp")
-        defer(ip, f"link set dev {cfg.ifname} xdpgeneric off")
-
-        # Attaching XDP may change features, fetch the latest state
-        feat = ethtool(f"-k {cfg.ifname}", json=True)[0]
-
-        _set_ethtool_feat(cfg, feat,
-                          {"generic-receive-offload": True,
-                           "rx-gro-hw": True,
-                           "large-receive-offload": False})
-
 
 def _check_gro_stats(cfg, test_queue, stats_before,
                      expect_rx, expect_gro, expect_wire):
Signature · 1 line
2.56.1
#3 lore raw

Re: [PATCH net-next 2/8] selftests: drv-net: drop the generic XDP hack from the HW GRO setup

Quoted 63 lines · Jakub Kicinski

The HW GRO tests want SW GRO out of the way so that what they observe can only have come from the device. On drivers which keyed HW GRO off NETIF_F_GRO that was not expressible with ethtool alone, so the setup fell back to attaching a generic XDP program, which suppresses SW GRO as a side effect.

Those drivers no longer tie the two features together, so ask for the feature combination directly.

---
 .../testing/selftests/drivers/net/gro_lib.py  | 21 +------------------
 .../selftests/drivers/net/hw/gro_stats.py     | 18 ----------------
 2 files changed, 1 insertion(+), 38 deletions(-)
tools/testing/selftests/drivers/net/gro_lib.py +1 -20
index 875a715b2ae1..6eff19db3541 100644
@@ -263,26 +263,7 @@ def _run_gro_bin(cfg, test_name, protocol=None, num_flows=None,
         set_ethtool_feat(cfg.ifname, cfg.feat,
                          {"generic-receive-offload": False,
                           "rx-gro-hw": True,
-                          "large-receive-offload": False},
-                          check=False)
-
-        # Some NICs treat HW GRO as a GRO sub-feature so disabling GRO
-        # will also clear HW GRO. Use a hack of installing XDP generic
-        # to skip SW GRO, even when enabled.
-        feat = ethtool(f"-k {cfg.ifname}", json=True)[0]
-        if not feat["rx-gro-hw"]["active"]:
-            ksft_pr("Driver clears HW GRO and SW GRO is cleared, using generic XDP workaround")
-            prog = cfg.net_lib_dir / "xdp_dummy.bpf.o"
-            ip(f"link set dev {cfg.ifname} xdpgeneric obj {prog} sec xdp")
-            defer(ip, f"link set dev {cfg.ifname} xdpgeneric off")
-
-            # Attaching XDP may change features, fetch the latest state
-            feat = ethtool(f"-k {cfg.ifname}", json=True)[0]
-
-            set_ethtool_feat(cfg.ifname, feat,
-                             {"generic-receive-offload": True,
-                              "rx-gro-hw": True,
-                              "large-receive-offload": False})
+                          "large-receive-offload": False})
     elif mode == "lro":
         # netdevsim advertises LRO for feature inheritance testing with
         # bonding/team tests but it doesn't actually perform the offload
tools/testing/selftests/drivers/net/hw/gro_stats.py +0 -8
index 70e76e3888bd..569ca81368e6 100755
@@ -173,24 +173,6 @@ def _run_gro_test(cfg, test_name, num_flows=None, ignore_fail=False,
                        "rx-gro-hw": True,
                        "large-receive-offload": False})
 
-    # Some NICs treat HW GRO as a GRO sub-feature so disabling GRO
-    # will also clear HW GRO. Use a hack of installing XDP generic
-    # to skip SW GRO, even when enabled.
-    feat = ethtool(f"-k {cfg.ifname}", json=True)[0]
-    if not feat["rx-gro-hw"]["active"]:
-        ksft_pr("Driver clears HW GRO when SW GRO is cleared, using generic XDP workaround")
-        prog = cfg.net_lib_dir / "xdp_dummy.bpf.o"
-        ip(f"link set dev {cfg.ifname} xdpgeneric obj {prog} sec xdp")

tools/testing/selftests/drivers/net/hw/gro_stats.py def _run_gro_test(cfg, test_name, num_flows=None, ignore_fail=False, · line 193

Quoted 10 lines
tools/testing/selftests/drivers/net/hw/gro_stats.py +0 -10
-        defer(ip, f"link set dev {cfg.ifname} xdpgeneric off")
-
-        # Attaching XDP may change features, fetch the latest state
-        feat = ethtool(f"-k {cfg.ifname}", json=True)[0]
-
-        _set_ethtool_feat(cfg, feat,
-                          {"generic-receive-offload": True,
-                           "rx-gro-hw": True,
-                           "large-receive-offload": False})
-

Not sure how likely these driver edge cases would be to reach, but FYI:

gro_stats.py uses its own _set_ethtool_feat(), which, unlike the shared set_ethtool_feat() in lib/py/feat.py, never reads the features back after ethtool -K. This removed if not feat["rx-gro-hw"]["active"] was the only check that HW GRO actually turned on. Drivers still drop GRO_HW in ndo_fix_features for reasons unrelated to SW GRO:
- qede with MTU > PAGE_SIZE;
- bnx2x when the MTU doesn't allow GRO;
- bnxt with BNXT_FLAG_NO_AGG_RINGS;
- mlx5 with legacy RQ or CQE compression;
- and, after 3/8, any attached XDP program.
In those cases the test now runs with both SW and HW GRO off and fails later in _check_gro_stats() with a confusing count mismatch, instead of skipping. gro_lib.py got that check by dropping check=False, so the two tests now behave differently. Either switch _setup_hw_gro() to the shared set_ethtool_feat() (and delete the local copy), or keep a short read-back

Quoted 6 lines
tools/testing/selftests/drivers/net/hw/gro_stats.py +0 -0
 def _check_gro_stats(cfg, test_queue, stats_before,
                      expect_rx, expect_gro, expect_wire):
Signature · 1 line
2.56.1
author maintainer
#4 lore raw

PATCH 3/8 net: state the "XDP excludes HW-GRO" rule in the core

In reply to Jakub Kicinski

XDP has to see the frames as the wire delivered them, so HW-GRO must be off while a program is attached. Drivers currently open-code that policy, and they diverge: mlx5, gve and virtio_net refuse the attach, bnxt and qede clear the feature from ndo_fix_features(), and idpf does nothing at all and happily runs RSC under XDP.

Clear HW-GRO in __netdev_update_features(), and make sure XDP setup paths refresh the features.

Take extra care to make sure the disablement actually works, some drivers add the feature back in ndo_fix_features, even when removed from wanted features (mlx5 does for header-data-split).

This change also removes the warn-only check dev_disable_gro_hw() performed. On mlx5 with tcp-data-split on, attaching a multi-buf generic XDP program reached generic_xdp_install(), whose dev_disable_gro_hw() cleared wanted_features, recomputed, saw mlx5e_fix_features() put NETIF_F_GRO_HW straight back, and splatted:

  ------------[ cut here ]------------
  netdevice: eth0: failed to disable GRO_HW!
  WARNING: net/core/dev.c:1880 at generic_xdp_install+0x135/0x190, CPU#53: ip/667
  RIP: 0010:generic_xdp_install+0x17a/0x190
  Call Trace:
   <TASK>
   dev_xdp_install+0x2aa/0x380
   dev_xdp_attach+0x41f/0x460
   dev_change_xdp_fd+0xbb/0x110
   do_setlink+0xfb9/0x1030
   rtnl_newlink+0x9f5/0xaf0
   rtnetlink_rcv_msg+0x442/0x490
   netlink_rcv_skb+0x16a/0x1a0
   netlink_unicast+0x1fc/0x320
   netlink_sendmsg+0x2d6/0x380

and then attached anyway, leaving XDP looking at coalesced frames. The device also lost the bit from wanted_features for good, so HW-GRO never came back after detach - on mlx5 that took header-data split with it, and with it devmem and io_uring zero-copy Rx.

---
 .../networking/net_cachelines/net_device.rst  |  1 +
 include/linux/netdevice.h                     |  6 ++
 net/core/dev.c                                | 97 ++++++++++++++-----
 3 files changed, 81 insertions(+), 23 deletions(-)
Documentation/networking/net_cachelines/net_device.rst +1 -0
index 7620d3fcb8dc..25cf02a184ec 100644
@@ -169,6 +169,7 @@ unsigned_long:1                     see_all_hwtstamp_requests
 unsigned_long:1                     change_proto_down
 unsigned_long:1                     netns_immutable
 unsigned_long:1                     fcoe_mtu
+unsigned_long:1                     xdp_pending
 unsigned_long:1                     xdp_from_upper
 struct list_head                    net_notifier_list
 struct macsec_ops*                  macsec_ops
include/linux/netdevice.h +6 -0
index 0325c6f2a7d1..049538b13e61 100644
@@ -2531,6 +2531,12 @@ struct net_device {
 	unsigned long		change_proto_down:1;
 	unsigned long		netns_immutable:1;
 	unsigned long		fcoe_mtu:1;
+	/**
+	 * @xdp_pending: an XDP program is being installed, but is not in
+	 *	@xdp_state yet. Most users should access it implicitly via
+	 *	helpers like dev_xdp_prog_count(), rather than directly.
+	 */
+	unsigned long		xdp_pending:1;
 	/**
 	 * @xdp_from_upper: the program in @xdp_state was installed by an
 	 *	upper device with netif_xdp_propagate(); it belongs to the
net/core/dev.c +74 -23

170 lines not on this page · raw message

Signature · 1 line
2.56.1
#5 lore raw

Re: [PATCH net-next 3/8] net: state the "XDP excludes HW-GRO" rule in the core

Quoted 246 lines · Jakub Kicinski

XDP has to see the frames as the wire delivered them, so HW-GRO must be off while a program is attached. Drivers currently open-code that policy, and they diverge: mlx5, gve and virtio_net refuse the attach, bnxt and qede clear the feature from ndo_fix_features(), and idpf does nothing at all and happily runs RSC under XDP.

Clear HW-GRO in __netdev_update_features(), and make sure XDP setup paths refresh the features.

Take extra care to make sure the disablement actually works, some drivers add the feature back in ndo_fix_features, even when removed from wanted features (mlx5 does for header-data-split).

This change also removes the warn-only check dev_disable_gro_hw() performed. On mlx5 with tcp-data-split on, attaching a multi-buf generic XDP program reached generic_xdp_install(), whose dev_disable_gro_hw() cleared wanted_features, recomputed, saw mlx5e_fix_features() put NETIF_F_GRO_HW straight back, and splatted:

  ------------[ cut here ]------------
  netdevice: eth0: failed to disable GRO_HW!
  WARNING: net/core/dev.c:1880 at generic_xdp_install+0x135/0x190, CPU#53: ip/667
  RIP: 0010:generic_xdp_install+0x17a/0x190
  Call Trace:
   <TASK>
   dev_xdp_install+0x2aa/0x380
   dev_xdp_attach+0x41f/0x460
   dev_change_xdp_fd+0xbb/0x110
   do_setlink+0xfb9/0x1030
   rtnl_newlink+0x9f5/0xaf0
   rtnetlink_rcv_msg+0x442/0x490
   netlink_rcv_skb+0x16a/0x1a0
   netlink_unicast+0x1fc/0x320
   netlink_sendmsg+0x2d6/0x380

and then attached anyway, leaving XDP looking at coalesced frames. The device also lost the bit from wanted_features for good, so HW-GRO never came back after detach - on mlx5 that took header-data split with it, and with it devmem and io_uring zero-copy Rx.

---
 .../networking/net_cachelines/net_device.rst  |  1 +
 include/linux/netdevice.h                     |  6 ++
 net/core/dev.c                                | 97 ++++++++++++++-----
 3 files changed, 81 insertions(+), 23 deletions(-)
Documentation/networking/net_cachelines/net_device.rst +1 -0
index 7620d3fcb8dc..25cf02a184ec 100644
@@ -169,6 +169,7 @@ unsigned_long:1                     see_all_hwtstamp_requests
 unsigned_long:1                     change_proto_down
 unsigned_long:1                     netns_immutable
 unsigned_long:1                     fcoe_mtu
+unsigned_long:1                     xdp_pending
 unsigned_long:1                     xdp_from_upper
 struct list_head                    net_notifier_list
 struct macsec_ops*                  macsec_ops
include/linux/netdevice.h +6 -0
index 0325c6f2a7d1..049538b13e61 100644
@@ -2531,6 +2531,12 @@ struct net_device {
 	unsigned long		change_proto_down:1;
 	unsigned long		netns_immutable:1;
 	unsigned long		fcoe_mtu:1;
+	/**
+	 * @xdp_pending: an XDP program is being installed, but is not in
+	 *	@xdp_state yet. Most users should access it implicitly via
+	 *	helpers like dev_xdp_prog_count(), rather than directly.
+	 */
+	unsigned long		xdp_pending:1;
 	/**
 	 * @xdp_from_upper: the program in @xdp_state was installed by an
 	 *	upper device with netif_xdp_propagate(); it belongs to the
net/core/dev.c +66 -23
index d88af07aea29..c0bb5eb9fdea 100644
@@ -1901,23 +1901,6 @@ void netif_disable_lro(struct net_device *dev)
 	}
 }
 
-/**
- *	dev_disable_gro_hw - disable HW Generic Receive Offload on a device
- *	@dev: device
- *
- *	Disable HW Generic Receive Offload (GRO_HW) on a net device.  Must be
- *	called under RTNL.  This is needed if Generic XDP is installed on
- *	the device.
- */
-static void dev_disable_gro_hw(struct net_device *dev)
-{
-	dev->wanted_features &= ~NETIF_F_GRO_HW;
-	netdev_update_features(dev);
-
-	if (unlikely(dev->features & NETIF_F_GRO_HW))
-		netdev_WARN(dev, "failed to disable GRO_HW!\n");
-}
-
 const char *netdev_cmd_to_name(enum netdev_cmd cmd)
 {
 #define N(val) 						\
@@ -6652,7 +6635,6 @@ static int generic_xdp_install(struct net_device *dev, struct netdev_bpf *xdp)
 		} else if (new && !old) {
 			static_branch_inc(&generic_xdp_needed_key);
 			netif_disable_lro(dev);
-			dev_disable_gro_hw(dev);
 		}
 		break;
 
@@ -10571,7 +10553,7 @@ u8 dev_xdp_prog_count(struct net_device *dev)
 	for (i = 0; i < __MAX_XDP_MODE; i++)
 		if (dev->xdp_state[i].prog || dev->xdp_state[i].link)
 			count++;
-	return count;
+	return count ?: !!dev->xdp_pending;
 }
 EXPORT_SYMBOL_GPL(dev_xdp_prog_count);
 
@@ -10612,6 +10594,32 @@ static void dev_xdp_set_prog(struct net_device *dev, enum bpf_xdp_mode mode,
 	dev->xdp_state[mode].prog = prog;
 }
 
+/* Try to clear HW-GRO before the driver reconfigures for XDP.
+ * Driver may add features back in via ndo_fix_features(), due to feature
+ * dependencies. In that case we can't run XDP at all.
+ */
+static int dev_xdp_clear_rx_agg(struct net_device *dev,
+				struct netlink_ext_ack *extack)
+{
+	if (!(dev->features & NETIF_F_GRO_HW))
+		return 0;
+	if (!(dev->hw_features & NETIF_F_GRO_HW)) {
+		NL_SET_ERR_MSG(extack,
+			       "unable to disable HW-GRO, fixed on");
+		return -EOPNOTSUPP;
+	}
+
+	netdev_update_features(dev);
+
+	if (dev->features & NETIF_F_GRO_HW) {
+		NL_SET_ERR_MSG(extack,
+			       "unable to disable HW-GRO, required by another feature");
+		return -EBUSY;
+	}
+
+	return 0;
+}
+
 int netif_xdp_propagate(struct net_device *dev, struct netdev_bpf *bpf)
 {
 	struct bpf_prog *old_prog;
@@ -10645,11 +10653,17 @@ int netif_xdp_propagate(struct net_device *dev, struct netdev_bpf *bpf)
 				       "unable to propagate XDP to device using memory provider");
 			return -EBUSY;
 		}
+
+		dev->xdp_pending = 1;
+
+		err = dev_xdp_clear_rx_agg(dev, bpf->extack);
+		if (err)
+			goto err_clear_pending;
 	}
 
 	err = dev->netdev_ops->ndo_bpf(dev, bpf);
 	if (err)
-		return err;
+		goto err_clear_pending;
 	if (bpf->prog)
 		bpf_prog_inc(bpf->prog);
 
@@ -10662,11 +10676,22 @@ int netif_xdp_propagate(struct net_device *dev, struct netdev_bpf *bpf)
 	old_prog = dev_xdp_prog(dev, XDP_MODE_DRV);
 	bpf_prog_change_xdp(old_prog, bpf->prog);
 	dev_xdp_set_prog(dev, XDP_MODE_DRV, bpf->prog);
+	dev->xdp_pending = 0;
 	dev->xdp_from_upper = !!bpf->prog;
-	if (old_prog)
+
+	if (old_prog) {
 		bpf_prog_put(old_prog);
+		/* The last program is gone, HW-GRO may be available again */
+		if (!bpf->prog)
+			netdev_update_features(dev);
+	}
 
 	return 0;
+
+err_clear_pending:
+	dev->xdp_pending = 0;
+	netdev_update_features(dev);
+	return err;
 }
 EXPORT_SYMBOL_GPL(netif_xdp_propagate);
 
@@ -10875,19 +10900,36 @@ static int dev_xdp_attach(struct net_device *dev, struct netlink_ext_ack *extack
 			return -EOPNOTSUPP;
 		}
 
+		if (new_prog) {
+			dev->xdp_pending = 1;
+			err = dev_xdp_clear_rx_agg(dev, extack);
+			if (err)
+				goto err_clear_pending;
+		}
+
 		err = dev_xdp_install(dev, mode, bpf_op, extack, flags, new_prog);
 		if (err)
-			return err;
+			goto err_clear_pending;
 	}
 
 	if (link)
 		dev_xdp_set_link(dev, mode, link);
 	else
 		dev_xdp_set_prog(dev, mode, new_prog);
-	if (cur_prog)
+	dev->xdp_pending = 0;
+	if (cur_prog) {
 		bpf_prog_put(cur_prog);
+		/* The last program is gone, HW-GRO may be available again */
+		if (!new_prog)
+			netdev_update_features(dev);
+	}
 
 	return 0;
+
+err_clear_pending:
+	dev->xdp_pending = 0;
+	netdev_update_features(dev);
+	return err;
 }
 
 static int dev_xdp_attach_link(struct net_device *dev,
@@ -10913,6 +10955,7 @@ static int dev_xdp_detach_link(struct net_device *dev,
 	bpf_op = dev_xdp_bpf_op(dev, mode);
 	WARN_ON(dev_xdp_install(dev, mode, bpf_op, NULL, 0, NULL));
 	dev_xdp_set_link(dev, mode, NULL);
+	netdev_update_features(dev);
 	return 0;
 }
 
@@ -11385,6 +11428,14 @@ int __netdev_update_features(struct net_device *dev)
 

net/core/dev.c int __netdev_update_features(struct net_device *dev) · line 11438

Quoted 10 lines
net/core/dev.c +8 -0
 	features = netdev_get_wanted_features(dev);
 
+	/* GRO_HW may be added back in via ndo_fix_features e.g. if driver needs
+	 * it to honor the HDS config, clear it before the fix features call.
+	 */
+	if ((features & NETIF_F_GRO_HW) && dev_xdp_prog_count(dev)) {
+		netdev_dbg(dev, "Dropping HW-GRO feature since XDP is attached.\n");
+		features &= ~NETIF_F_GRO_HW;
+	}
+

Apparently drivers can enable HW_GRO in some edge case (that can be plugged itself?):

The rule is only enforced at attach time (dev_xdp_clear_rx_agg). Later recomputes don't re-check after ndo_fix_features. Example:
1. Attach a multi-buf XDP program to mlx5 with HDS off. HW-GRO goes off.
2. Enable tcp-data-split. rings.c:253 only refuses this for single-buffer XDP, via dev_xdp_sb_prog_count().
3. The next feature recompute runs mlx5e_fix_features(), and its "Keeping HW-GRO enabled" block (en_main.c:4643) sets GRO_HW again, after this strip. HW-GRO is now on with XDP attached.
This already happens today, but once 5/8 removes mlx5's own XDP check the core is the only guard, and it misses this case. Either add a post-fix_features check here (warn and strip, or refuse the change), or make the HDS-enable path in rings.c refuse while any XDP program is attached and the driver ties HDS to HW-GRO.

Quoted 6 lines
net/core/dev.c +0 -0
 	if (dev->netdev_ops->ndo_fix_features)
 		features = dev->netdev_ops->ndo_fix_features(dev, features);
 
Signature · 1 line
2.56.1
author maintainer
#6 lore raw

PATCH 4/8 net: state the "XDP excludes LRO" rule in the core

In reply to Jakub Kicinski

LRO is the older take on the same thing as HW-GRO. And similarly to HW-GRO XDP cannot be handed post-LRO frames. Drivers open-code the exclusion just like they did for HW-GRO, and disagree just the
same: mlx5 and netvsc refuse the attach, bnxt and ixgbe clear the feature from ndo_fix_features(), vmxnet3 writes dev->features behind the core's back, and atlantic only minds single-buffer programs. Clear LRO next to HW-GRO, with the same ordering and the same refusal if a driver insists on keeping it.

atlantic loses LRO under a multi-buffer program now as well. XDP_TX and redirect can't carry the GSO metadata of a coalesced frame, however many buffers it spans.

Drop netif_disable_lro() from generic_xdp_install(), like the previous patch dropped dev_disable_gro_hw(). The core covers the device itself, and LRO no longer stays off after the detach.

This changes the behavior for lower devices. netif_disable_lro() walked them, dev_disable_gro_hw() never did. And unlike HW-GRO, LRO is in NETIF_F_UPPER_DISABLES, an upper which loses it clears it from the wanted_features of its lowers for good. bond and team carry LRO only to hand it down, so a program on a bond still turns LRO off on the slaves, and they don't get it back after the detach. Making sure lowers don't coalesce is a separate problem, for HW-GRO and LRO alike.

LRO state with the program [attached / after it's detached], netdevsim as the single device, bond of two vmxnet3 ports:

  program on    device   w/o patch    w/ patch
  ------------  -------  -----------  ---------
  dev, native   dev      on  / on `1  off / on
  dev, generic  dev      off / off    off / on
  bond, native  bond     on  / on     off / on
                slaves   off / off`2  off / off
  bond, generic bond     off / off    off / on
                slaves   off / off    off / off

`1 netdevsim ignores LRO under XDP, real drivers turned it off one
way or another
`2 vmxnet3 clears LRO from dev->features by itself and nothing turns
it back on, bnxt slaves would get it back

Ideally we'd want [off / on] (XDP disables LRO, LRO comes back once XDP is cleared). But the two "off"s after detach on slaves are not worth the complexity IMHO.

---
 include/linux/netdev_features.h |  6 ++++++
 net/core/dev.c                  | 36 ++++++++++++++++++++-------------
 2 files changed, 28 insertions(+), 14 deletions(-)
include/linux/netdev_features.h +6 -0
index 8eb6b8033606..62a05df8ee68 100644
@@ -234,6 +234,12 @@ static inline int find_next_netdev_feature(u64 feature, unsigned long start)
  */
 #define NETIF_F_UPPER_DISABLES	NETIF_F_LRO
 
+/*
+ * Receive offloads which coalesce frames. XDP has to see the frames as the
+ * wire delivered them, so these are disabled while a program is attached.
+ */
+#define NETIF_F_XDP_DISABLES	(NETIF_F_GRO_HW | NETIF_F_LRO)
+
 /* changeable features with no special hardware requirements */
 #define NETIF_F_SOFT_FEATURES	(NETIF_F_GSO | NETIF_F_GRO)
 
net/core/dev.c +22 -14
index c0bb5eb9fdea..2b9c7aac0969 100644
@@ -6634,7 +6634,6 @@ static int generic_xdp_install(struct net_device *dev, struct netdev_bpf *xdp)
 			static_branch_dec(&generic_xdp_needed_key);
 		} else if (new && !old) {
 			static_branch_inc(&generic_xdp_needed_key);
-			netif_disable_lro(dev);
 		}
 		break;
 
@@ -10594,26 +10593,35 @@ static void dev_xdp_set_prog(struct net_device *dev, enum bpf_xdp_mode mode,
 	dev->xdp_state[mode].prog = prog;
 }
 
-/* Try to clear HW-GRO before the driver reconfigures for XDP.
+static const char *dev_xdp_rx_agg_name(netdev_features_t features)
+{
+	return features & NETIF_F_GRO_HW ? "HW-GRO" : "LRO";
+}
+
+/* Try to clear HW-GRO and LRO before the driver reconfigures for XDP.
  * Driver may add features back in via ndo_fix_features(), due to feature
  * dependencies. In that case we can't run XDP at all.
  */
 static int dev_xdp_clear_rx_agg(struct net_device *dev,
 				struct netlink_ext_ack *extack)
 {
-	if (!(dev->features & NETIF_F_GRO_HW))
+	netdev_features_t agg = dev->features & NETIF_F_XDP_DISABLES;
+
+	if (!agg)
 		return 0;
-	if (!(dev->hw_features & NETIF_F_GRO_HW)) {
-		NL_SET_ERR_MSG(extack,
-			       "unable to disable HW-GRO, fixed on");
+	if (agg & ~dev->hw_features) {
+		NL_SET_ERR_MSG_FMT(extack, "unable to disable %s, fixed on",
+				   dev_xdp_rx_agg_name(agg & ~dev->hw_features));
 		return -EOPNOTSUPP;
 	}
 
 	netdev_update_features(dev);
 
-	if (dev->features & NETIF_F_GRO_HW) {
-		NL_SET_ERR_MSG(extack,
-			       "unable to disable HW-GRO, required by another feature");
+	agg = dev->features & NETIF_F_XDP_DISABLES;
+	if (agg) {
+		NL_SET_ERR_MSG_FMT(extack,
+				   "unable to disable %s, required by another feature",
+				   dev_xdp_rx_agg_name(agg));
 		return -EBUSY;
 	}
 
@@ -10681,7 +10689,7 @@ int netif_xdp_propagate(struct net_device *dev, struct netdev_bpf *bpf)
 
 	if (old_prog) {
 		bpf_prog_put(old_prog);
-		/* The last program is gone, HW-GRO may be available again */
+		/* No program left, HW-GRO and LRO may be available again */
 		if (!bpf->prog)
 			netdev_update_features(dev);
 	}
@@ -10919,7 +10927,7 @@ static int dev_xdp_attach(struct net_device *dev, struct netlink_ext_ack *extack
 	dev->xdp_pending = 0;
 	if (cur_prog) {
 		bpf_prog_put(cur_prog);
-		/* The last program is gone, HW-GRO may be available again */
+		/* No program left, HW-GRO and LRO may be available again */
 		if (!new_prog)
 			netdev_update_features(dev);
 	}
@@ -11431,9 +11439,9 @@ int __netdev_update_features(struct net_device *dev)
 	/* GRO_HW may be added back in via ndo_fix_features e.g. if driver needs
 	 * it to honor the HDS config, clear it before the fix features call.
 	 */
-	if ((features & NETIF_F_GRO_HW) && dev_xdp_prog_count(dev)) {
-		netdev_dbg(dev, "Dropping HW-GRO feature since XDP is attached.\n");
-		features &= ~NETIF_F_GRO_HW;
+	if ((features & NETIF_F_XDP_DISABLES) && dev_xdp_prog_count(dev)) {
+		netdev_dbg(dev, "Dropping HW-GRO and LRO features since XDP is attached.\n");
+		features &= ~NETIF_F_XDP_DISABLES;
 	}
 
 	if (dev->netdev_ops->ndo_fix_features)
Signature · 1 line
2.56.1
Reviewed-by on v1 4/8
#7 lore raw

Re: [PATCH net-next 4/8] net: state the "XDP excludes LRO" rule in the core

Quoted 42 lines · Jakub Kicinski

LRO is the older take on the same thing as HW-GRO. And similarly to HW-GRO XDP cannot be handed post-LRO frames. Drivers open-code the exclusion just like they did for HW-GRO, and disagree just the
same: mlx5 and netvsc refuse the attach, bnxt and ixgbe clear the feature from ndo_fix_features(), vmxnet3 writes dev->features behind the core's back, and atlantic only minds single-buffer programs. Clear LRO next to HW-GRO, with the same ordering and the same refusal if a driver insists on keeping it.

atlantic loses LRO under a multi-buffer program now as well. XDP_TX and redirect can't carry the GSO metadata of a coalesced frame, however many buffers it spans.

Drop netif_disable_lro() from generic_xdp_install(), like the previous patch dropped dev_disable_gro_hw(). The core covers the device itself, and LRO no longer stays off after the detach.

This changes the behavior for lower devices. netif_disable_lro() walked them, dev_disable_gro_hw() never did. And unlike HW-GRO, LRO is in NETIF_F_UPPER_DISABLES, an upper which loses it clears it from the wanted_features of its lowers for good. bond and team carry LRO only to hand it down, so a program on a bond still turns LRO off on the slaves, and they don't get it back after the detach. Making sure lowers don't coalesce is a separate problem, for HW-GRO and LRO alike.

LRO state with the program [attached / after it's detached], netdevsim as the single device, bond of two vmxnet3 ports:

  program on    device   w/o patch    w/ patch
  ------------  -------  -----------  ---------
  dev, native   dev      on  / on `1  off / on
  dev, generic  dev      off / off    off / on
  bond, native  bond     on  / on     off / on
                slaves   off / off`2  off / off
  bond, generic bond     off / off    off / on
                slaves   off / off    off / off

`1 netdevsim ignores LRO under XDP, real drivers turned it off one
way or another
`2 vmxnet3 clears LRO from dev->features by itself and nothing turns
it back on, bnxt slaves would get it back

Quoted 5 lines

Ideally we'd want [off / on] (XDP disables LRO, LRO comes back once XDP is cleared). But the two "off"s after detach on slaves are not worth the complexity IMHO.

#8 lore raw

Re: [PATCH net-next 4/8] net: state the "XDP excludes LRO" rule in the core

In reply to Jakub Kicinski
Thank you for your contribution! Sashiko AI review found no new issues in this patch, but noticed 1 related pre-existing issue it does not introduce: - [High] net/mlx5e: HDS state desync and NULL…

Thank you for your contribution! Sashiko AI review found no new issues in this patch, but noticed 1 related pre-existing issue it does not introduce:
- [High] net/mlx5e: HDS state desync and NULL pointer dereference in mlx5e_fix_features() [1]
--

commit 8d89a545dc54012efb30f6483506602bf2681942
Author: Jakub Kicinski <[email protected]>

net: state the "XDP excludes LRO" rule in the core

This commit clears NETIF_F_LRO alongside NETIF_F_GRO_HW in the networking core when an XDP program is attached, and drops netif_disable_lro() from generic_xdp_install().

Quoted 4 lines
net/core/dev.c +0 -0
index c0bb5eb9fdea0..2b9c7aac09693 100644

[ ... ]

net/core/dev.c int __netdev_update_features(struct net_device *dev) · line 11447

Quoted 13 lines
net/core/dev.c +3 -3
@@ -11431,9 +11439,9 @@ int __netdev_update_features(struct net_device *dev)
 	/* GRO_HW may be added back in via ndo_fix_features e.g. if driver needs
 	 * it to honor the HDS config, clear it before the fix features call.
 	 */
-	if ((features & NETIF_F_GRO_HW) && dev_xdp_prog_count(dev)) {
-		netdev_dbg(dev, "Dropping HW-GRO feature since XDP is attached.\n");
-		features &= ~NETIF_F_GRO_HW;
+	if ((features & NETIF_F_XDP_DISABLES) && dev_xdp_prog_count(dev)) {
+		netdev_dbg(dev, "Dropping HW-GRO and LRO features since XDP is attached.\n");
+		features &= ~NETIF_F_XDP_DISABLES;
 	}
 
 	if (dev->netdev_ops->ndo_fix_features)

[Severity: High]
This problem wasn't introduced by this patch, but can mlx5e_fix_features() re-enabling NETIF_F_GRO_HW for HDS cause an HDS state desync or a NULL pointer dereference [1]?

Because mlx5e_fix_features() runs before netdev_fix_features() without enforcing NETIF_F_RXCSUM, disabling NETIF_F_RXCSUM causes netdev_fix_features() to drop NETIF_F_GRO_HW. That turns off SHAMPO while cfg->hds_config stays ETHTOOL_TCP_DATA_SPLIT_ENABLED, blocking subsequent ring parameter changes.

In addition, mlx5e_fix_features() forces NETIF_F_GRO_HW back on for HDS even after clearing it earlier for a legacy cyclic RQ:

drivers/net/ethernet/mellanox/mlx5/core/en_main.c:mlx5e_fix_features() {
    ...
	if (!MLX5E_GET_PFLAG(params, MLX5E_PFLAG_RX_STRIDING_RQ)) {
    ...
		if (features & NETIF_F_GRO_HW) {
			netdev_warn(netdev, "Disabling HW-GRO, not supported in legacy RQ\n");
			features &= ~NETIF_F_GRO_HW;
		}
	}
    ...
	if (cfg && cfg->hds_config == ETHTOOL_TCP_DATA_SPLIT_ENABLED &&
	    !(features & NETIF_F_GRO_HW)) {
		netdev_warn(netdev, "Keeping HW-GRO enabled, TCP header-data split depends on it\n");
		features |= NETIF_F_GRO_HW;
	}
    ...
}

If rx_striding_rq is disabled while SHAMPO is off and NETIF_F_RXCSUM is then re-enabled, SHAMPO is enabled on a cyclic RQ. mlx5e_open_rq() sets MLX5E_RQ_STATE_SHAMPO while mlx5e_alloc_rq() skips mlx5_rq_shampo_alloc(), and mlx5e_create_rq() dereferences rq->mpwqe.shampo while NULL.

[1] https://sashiko.dev/#/bug/linux-5e98dbaa-290f-464e-8d13-a0e3360d73b7

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
author maintainer
#9 lore raw

PATCH 5/8 eth: mlx5e: drop the now-dead HW-GRO and LRO vs XDP handling

In reply to Jakub Kicinski

The core clears HW-GRO and LRO before ndo_bpf runs and keeps them off for as long as a program is attached. mlx5e_fix_features() never sees either with params->xdp_prog set, and packet merge is always off by the time mlx5e_xdp_set() gets called, packet_merge.type only follows the two feature handlers. If tcp-data-split keeps HW-GRO on, the core refuses the attach with -EBUSY before calling the driver.

The AF_XDP check stays, the core doesn't know about XSK pools.

This does change user visible behavior: attaching XDP with HW-GRO or LRO on used to fail with -EINVAL and now succeeds with them turned off.

---
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 32 ++-----------------
 1 file changed, 3 insertions(+), 29 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +3 -29
index d8563f863844..e462dfc725ae 100644
@@ -4585,17 +4585,6 @@ static netdev_features_t mlx5e_fix_features(struct net_device *netdev,
 		}
 	}
 
-	if (params->xdp_prog) {
-		if (features & NETIF_F_LRO) {
-			netdev_warn(netdev, "LRO is incompatible with XDP\n");
-			features &= ~NETIF_F_LRO;
-		}
-		if (features & NETIF_F_GRO_HW) {
-			netdev_warn(netdev, "HW GRO is incompatible with XDP\n");
-			features &= ~NETIF_F_GRO_HW;
-		}
-	}
-
 	if (priv->xsk.refcnt) {
 		if (features & NETIF_F_LRO) {
 			netdev_warn(netdev, "LRO is incompatible with AF_XDP (%u XSKs are active)\n",
@@ -5177,20 +5166,6 @@ static void mlx5e_tx_timeout(struct net_device *dev, unsigned int txqueue)
 	queue_work(priv->wq, &priv->tx_timeout_work);
 }
 
-static int mlx5e_xdp_allowed(struct net_device *netdev, struct mlx5_core_dev *mdev,
-			     struct mlx5e_params *params)
-{
-	if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) {
-		netdev_warn(netdev, "can't set XDP while HW-GRO/LRO is on, disable them first\n");
-		return -EINVAL;
-	}
-
-	if (!mlx5e_params_validate_xdp(netdev, mdev, params))
-		return -EINVAL;
-
-	return 0;
-}
-
 static void mlx5e_rq_replace_xdp_prog(struct mlx5e_rq *rq, struct bpf_prog *prog)
 {
 	struct bpf_prog *old_prog;
@@ -5215,10 +5190,9 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog)
 	new_params = priv->channels.params;
 	new_params.xdp_prog = prog;
 
-	if (prog) {
-		err = mlx5e_xdp_allowed(netdev, priv->mdev, &new_params);
-		if (err)
-			goto unlock;
+	if (prog && !mlx5e_params_validate_xdp(netdev, priv->mdev, &new_params)) {
+		err = -EINVAL;
+		goto unlock;
 	}
 
 	/* no need for full reset when exchanging programs */
Signature · 1 line
2.56.1
#10 lore raw

Re: [PATCH net-next 5/8] eth: mlx5e: drop the now-dead HW-GRO and LRO vs XDP handling

Quoted 7 lines · Jakub Kicinski

The core clears HW-GRO and LRO before ndo_bpf runs and keeps them off for as long as a program is attached. mlx5e_fix_features() never sees either with params->xdp_prog set, and packet merge is always off by the time mlx5e_xdp_set() gets called, packet_merge.type only follows the two feature handlers. If tcp-data-split keeps HW-GRO on, the core refuses the attach with -EBUSY before calling the driver.

Quoted 5 lines

The AF_XDP check stays, the core doesn't know about XSK pools.

This does change user visible behavior: attaching XDP with HW-GRO or LRO on used to fail with -EINVAL and now succeeds with them turned off.

Thanks for the cleanup!

Quoted 52 lines
---
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 32 ++-----------------
 1 file changed, 3 insertions(+), 29 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -25
index d8563f863844..e462dfc725ae 100644
@@ -4585,17 +4585,6 @@ static netdev_features_t mlx5e_fix_features(struct net_device *netdev,
 		}
 	}
 
-	if (params->xdp_prog) {
-		if (features & NETIF_F_LRO) {
-			netdev_warn(netdev, "LRO is incompatible with XDP\n");
-			features &= ~NETIF_F_LRO;
-		}
-		if (features & NETIF_F_GRO_HW) {
-			netdev_warn(netdev, "HW GRO is incompatible with XDP\n");
-			features &= ~NETIF_F_GRO_HW;
-		}
-	}
-
 	if (priv->xsk.refcnt) {
 		if (features & NETIF_F_LRO) {
 			netdev_warn(netdev, "LRO is incompatible with AF_XDP (%u XSKs are active)\n",
@@ -5177,20 +5166,6 @@ static void mlx5e_tx_timeout(struct net_device *dev, unsigned int txqueue)
 	queue_work(priv->wq, &priv->tx_timeout_work);
 }
 
-static int mlx5e_xdp_allowed(struct net_device *netdev, struct mlx5_core_dev *mdev,
-			     struct mlx5e_params *params)
-{
-	if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) {
-		netdev_warn(netdev, "can't set XDP while HW-GRO/LRO is on, disable them first\n");
-		return -EINVAL;
-	}
-
-	if (!mlx5e_params_validate_xdp(netdev, mdev, params))
-		return -EINVAL;
-
-	return 0;
-}
-
 static void mlx5e_rq_replace_xdp_prog(struct mlx5e_rq *rq, struct bpf_prog *prog)
 {
 	struct bpf_prog *old_prog;
@@ -5215,10 +5190,9 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog)
 	new_params = priv->channels.params;
 	new_params.xdp_prog = prog;
 

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) · line 5198

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +3 -4
-	if (prog) {
-		err = mlx5e_xdp_allowed(netdev, priv->mdev, &new_params);
-		if (err)
-			goto unlock;
+	if (prog && !mlx5e_params_validate_xdp(netdev, priv->mdev, &new_params)) {
+		err = -EINVAL;
+		goto unlock;
 	}
 
 	/* no need for full reset when exchanging programs */

There's a netdev_update_features() at the end of mlx5e_xdp_set(). It doesn't seem necessary anyore.

Also, there's a check left in mlx5e_set_xdp_feature().

Thanks,
Dragos

author maintainer
#11 lore raw

PATCH 6/8 eth: bnxt_en: don't clear HW-GRO and LRO for XDP in ndo_fix_features

In reply to Jakub Kicinski

The core keeps HW-GRO and LRO off while a program is attached, from before ndo_bpf is called until it's gone. The xdp_prog check in bnxt_fix_features() never changes the outcome any more. Without agg rings, which single-buffer XDP at a small MTU implies, neither offload works, so that part stays.

---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1 -1
index 27c53db2c9e4..a5d22198aa11 100644
@@ -14089,7 +14089,7 @@ static netdev_features_t bnxt_fix_features(struct net_device *dev,
 	    bp->tx_ring_size < 2 * BNXT_SW_USO_MAX_DESCS)
 		features &= ~NETIF_F_GSO_UDP_L4;
 
-	if ((bp->flags & BNXT_FLAG_NO_AGG_RINGS) || bp->xdp_prog)
+	if (bp->flags & BNXT_FLAG_NO_AGG_RINGS)
 		features &= ~(NETIF_F_LRO | NETIF_F_GRO_HW);
 
 	if (features & NETIF_F_GRO_HW)
Signature · 1 line
2.56.1
author maintainer
#12 lore raw

PATCH 7/8 eth: qede: drop the now-dead HW-GRO vs XDP handling

In reply to Jakub Kicinski

The core clears HW-GRO before ndo_bpf runs and keeps it off for as long as a program is attached, which makes three bits of qede dead:

- the xdp_prog check in qede_fix_features();
- qede_alloc_mem_rxq() clearing NETIF_F_GRO_HW from dev->features directly, the feature is already off by the time the reload for the program allocates the rings;
- qede_set_features() skipping the reload when a program is attached, HW-GRO can't change while edev->xdp_prog is set any more.

The direct write in qede_alloc_mem_rxq() is also why HW-GRO used to stay off after the program was removed - nothing recomputed the features. The core does that on detach now.

---
 drivers/net/ethernet/qlogic/qede/qede_filter.c | 11 ++---------
 drivers/net/ethernet/qlogic/qede/qede_main.c   |  1 -
 2 files changed, 2 insertions(+), 10 deletions(-)
drivers/net/ethernet/qlogic/qede/qede_filter.c +2 -9
index d0e203167e35..9c68579c5b33 100644
@@ -915,7 +915,7 @@ netdev_features_t qede_fix_features(struct net_device *dev,
 {
 	struct qede_dev *edev = netdev_priv(dev);
 
-	if (edev->xdp_prog || edev->ndev->mtu > PAGE_SIZE)
+	if (edev->ndev->mtu > PAGE_SIZE)
 		features &= ~NETIF_F_GRO_HW;
 
 	return features;
@@ -936,15 +936,8 @@ int qede_set_features(struct net_device *dev, netdev_features_t features)
 		args.u.features = features;
 		args.func = &qede_set_features_reload;
 
-		/* Make sure that we definitely need to reload.
-		 * In case of an eBPF attached program, there will be no FW
-		 * aggregations, so no need to actually reload.
-		 */
 		__qede_lock(edev);
-		if (edev->xdp_prog)
-			args.func(edev, &args);
-		else
-			qede_reload(edev, &args, true);
+		qede_reload(edev, &args, true);
 		__qede_unlock(edev);
 
 		return 1;
drivers/net/ethernet/qlogic/qede/qede_main.c +0 -1
index 7ed17faced54..b24baf0e808e 100644
@@ -1571,7 +1571,6 @@ static int qede_alloc_mem_rxq(struct qede_dev *edev, struct qede_rx_queue *rxq)
 		rxq->rx_buf_seg_size = roundup_pow_of_two(size);
 	} else {
 		rxq->rx_buf_seg_size = PAGE_SIZE;
-		edev->ndev->features &= ~NETIF_F_GRO_HW;
 	}
 
 	/* Allocate the parallel driver ring for Rx buffers */
Signature · 1 line
2.56.1
author maintainer
#13 lore raw

PATCH 8/8 eth: gve: drop the now-dead HW-GRO vs XDP interlock

In reply to Jakub Kicinski

The core clears NETIF_F_GRO_HW before ndo_bpf runs and refuses to turn it back on while a program is attached, so neither of these can fire any more. That holds for a program an upper device pushes down as well, which is why this comes last - gve_verify_xdp_configuration() was the only thing standing between a bonded gve port and XDP running with RSC still on.

The user visible behavior changes as intended: attaching XDP to a gve device with HW-GRO on used to fail with -EOPNOTSUPP and now succeeds with HW-GRO turned off, which is what bnxt and qede have always done. The header-data split refusal right below keys off priv->header_split_enabled and stays.

---
 drivers/net/ethernet/google/gve/gve_main.c | 12 ------------
 1 file changed, 12 deletions(-)
drivers/net/ethernet/google/gve/gve_main.c +0 -12
index 0ee69bc11d83..8b65b8119662 100644
@@ -1798,12 +1798,6 @@ static int gve_verify_xdp_configuration(struct net_device *dev,
 	struct gve_priv *priv = netdev_priv(dev);
 	u16 max_xdp_mtu;
 
-	if (dev->features & NETIF_F_GRO_HW) {
-		NL_SET_ERR_MSG_MOD(extack,
-				   "XDP is not supported when HW-GRO is on.");
-		return -EOPNOTSUPP;
-	}
-
 	if (priv->header_split_enabled) {
 		NL_SET_ERR_MSG_MOD(extack,
 				   "XDP is not supported when header-data split is enabled.");
@@ -2212,12 +2206,6 @@ static int gve_set_features(struct net_device *netdev,
 	if ((netdev->features & NETIF_F_GRO_HW) !=
 	    (features & NETIF_F_GRO_HW)) {
 		netdev->features ^= NETIF_F_GRO_HW;
-		if (priv->xdp_prog && (netdev->features & NETIF_F_GRO_HW)) {
-			netdev_warn(netdev,
-				    "HW-GRO is not supported when XDP is on.");
-			err = -EOPNOTSUPP;
-			goto revert_features;
-		}
 		if (netif_running(netdev)) {
 			err = gve_adjust_config(priv, &tx_alloc_cfg, &rx_alloc_cfg);
 			if (err)
Signature · 1 line
2.56.1
Reviewed-by on v1 8/8
#14 lore raw

Re: [PATCH net-next 8/8] eth: gve: drop the now-dead HW-GRO vs XDP interlock

Quoted 8 lines · Jakub Kicinski

The core clears NETIF_F_GRO_HW before ndo_bpf runs and refuses to turn it back on while a program is attached, so neither of these can fire any more. That holds for a program an upper device pushes down as well, which is why this comes last - gve_verify_xdp_configuration() was the only thing standing between a bonded gve port and XDP running with RSC still on.

Quoted 7 lines

The user visible behavior changes as intended: attaching XDP to a gve device with HW-GRO on used to fail with -EOPNOTSUPP and now succeeds with HW-GRO turned off, which is what bnxt and qede have always done. The header-data split refusal right below keys off priv->header_split_enabled and stays.

Quoted 38 lines
---
 drivers/net/ethernet/google/gve/gve_main.c | 12 ------------
 1 file changed, 12 deletions(-)
drivers/net/ethernet/google/gve/gve_main.c +0 -13
index 0ee69bc11d83..8b65b8119662 100644
@@ -1798,12 +1798,6 @@ static int gve_verify_xdp_configuration(struct net_device *dev,
        struct gve_priv *priv = netdev_priv(dev);
        u16 max_xdp_mtu;
 
-       if (dev->features & NETIF_F_GRO_HW) {
-               NL_SET_ERR_MSG_MOD(extack,
-                                  "XDP is not supported when HW-GRO is on.");
-               return -EOPNOTSUPP;
-       }
-
        if (priv->header_split_enabled) {
                NL_SET_ERR_MSG_MOD(extack,
                                   "XDP is not supported when header-data split is enabled.");
@@ -2212,12 +2206,6 @@ static int gve_set_features(struct net_device *netdev,
        if ((netdev->features & NETIF_F_GRO_HW) !=
            (features & NETIF_F_GRO_HW)) {
                netdev->features ^= NETIF_F_GRO_HW;
-               if (priv->xdp_prog && (netdev->features & NETIF_F_GRO_HW)) {
-                       netdev_warn(netdev,
-                                   "HW-GRO is not supported when XDP is on.");
-                       err = -EOPNOTSUPP;
-                       goto revert_features;
-               }
                if (netif_running(netdev)) {
                        err = gve_adjust_config(priv, &tx_alloc_cfg, &rx_alloc_cfg);
                        if (err)
--

2.56.1

maintainer Reviewed-by on v1 8/8
#15 lore raw

Re: [PATCH net-next 8/8] eth: gve: drop the now-dead HW-GRO vs XDP interlock

In reply to Jakub Kicinski
Quoted 8 lines · Jakub Kicinski

The core clears NETIF_F_GRO_HW before ndo_bpf runs and refuses to turn it back on while a program is attached, so neither of these can fire any more. That holds for a program an upper device pushes down as well, which is why this comes last - gve_verify_xdp_configuration() was the only thing standing between a bonded gve port and XDP running with RSC still on.

Quoted 7 lines

The user visible behavior changes as intended: attaching XDP to a gve device with HW-GRO on used to fail with -EOPNOTSUPP and now succeeds with HW-GRO turned off, which is what bnxt and qede have always done. The header-data split refusal right below keys off priv->header_split_enabled and stays.

Thanks!

Quoted 37 lines
---
 drivers/net/ethernet/google/gve/gve_main.c | 12 ------------
 1 file changed, 12 deletions(-)
drivers/net/ethernet/google/gve/gve_main.c +0 -13
index 0ee69bc11d83..8b65b8119662 100644
@@ -1798,12 +1798,6 @@ static int gve_verify_xdp_configuration(struct net_device *dev,
        struct gve_priv *priv = netdev_priv(dev);
        u16 max_xdp_mtu;
 
-       if (dev->features & NETIF_F_GRO_HW) {
-               NL_SET_ERR_MSG_MOD(extack,
-                                  "XDP is not supported when HW-GRO is on.");
-               return -EOPNOTSUPP;
-       }
-
        if (priv->header_split_enabled) {
                NL_SET_ERR_MSG_MOD(extack,
                                   "XDP is not supported when header-data split is enabled.");
@@ -2212,12 +2206,6 @@ static int gve_set_features(struct net_device *netdev,
        if ((netdev->features & NETIF_F_GRO_HW) !=
            (features & NETIF_F_GRO_HW)) {
                netdev->features ^= NETIF_F_GRO_HW;
-               if (priv->xdp_prog && (netdev->features & NETIF_F_GRO_HW)) {
-                       netdev_warn(netdev,
-                                   "HW-GRO is not supported when XDP is on.");
-                       err = -EOPNOTSUPP;
-                       goto revert_features;
-               }
                if (netif_running(netdev)) {
                        err = gve_adjust_config(priv, &tx_alloc_cfg, &rx_alloc_cfg);
                        if (err)
--

2.56.1