update xdp_features flag according to NIC re-configuration

PATCH Superseded Lorenzo Bianconi 5 participants 18 messages Mar 7, 2023 – Mar 9, 2023

Subsystems (10)

Networking Network drivers BPF Mellanox mlx5 core +6 more subsystems

lore

Share & tools

lore

Lorenzo Bianconi

author

2023-03-07 15:03 UTC #0 lore raw

cover update xdp_features flag according to NIC re-configuration

Take into account possible NIC configuration changes (e.g. LRO or tx/rx queues reconfiguration) setting device xdp_features flag.
Introduce xdp_set_features_flag utility routine.

Lorenzo Bianconi (7):
tools: ynl: fix render-max for flags definition
tools: ynl: fix get_mask utility routine
xdp: add xdp_set_features_flag utility routine
net: thunderx: take into account xdp_features setting tx/rx queues
net: ena: take into account xdp_features setting tx/rx queues
veth: take into account device reconfiguration for xdp_features flag net/mlx5e: take into account device reconfiguration for xdp_features
flag

Matteo Croce (1):
mvpp2: take care of xdp_features when reconfiguring queues

 Documentation/netlink/specs/netdev.yaml       |  1 +
 drivers/net/ethernet/amazon/ena/ena_ethtool.c | 15 +++++--
 drivers/net/ethernet/amazon/ena/ena_netdev.c  |  6 ++-
 .../ethernet/cavium/thunder/nicvf_ethtool.c   | 17 ++++---
 .../net/ethernet/cavium/thunder/nicvf_main.c  |  4 +-
 .../net/ethernet/marvell/mvpp2/mvpp2_main.c   | 15 +++++--
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  1 +
 .../ethernet/mellanox/mlx5/core/en_ethtool.c  | 10 ++++-
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++---
 .../net/ethernet/mellanox/mlx5/core/en_rep.c  |  3 ++
 drivers/net/veth.c                            | 42 +++++++++++++++--
 include/net/xdp.h                             | 11 +++++
 include/uapi/linux/netdev.h                   |  2 +
 net/core/xdp.c                                | 26 ++++++++---
 tools/include/uapi/linux/netdev.h             |  2 +
 tools/net/ynl/ynl-gen-c.py                    | 17 ++++---
 16 files changed, 176 insertions(+), 41 deletions(-)

Signature · 1 line

2.39.2

Lorenzo Bianconi

author

2023-03-07 15:03 UTC #1 lore raw

PATCH 1/8 tools: ynl: fix render-max for flags definition

Properly manage render-max property for flags definition type introducing mask value and setting it to (last_element << 1) - 1 instead of adding max value set to last_element + 1

Fixes be5bea1cc0bf ("net: add basic C code generators for Netlink")

Signed-off-by Lorenzo Bianconi <[email protected]>

---
 tools/net/ynl/ynl-gen-c.py | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

tools/net/ynl/ynl-gen-c.py +8 -3

index 274e9c566f61..f2e41dd962d4 100755
@@ -1995,9 +1995,14 @@ def render_uapi(family, cw):
 
             if const.get('render-max', False):
                 cw.nl()
-                max_name = c_upper(name_pfx + 'max')
-                cw.p('__' + max_name + ',')
-                cw.p(max_name + ' = (__' + max_name + ' - 1)')
+                if const['type'] == 'flags':
+                    max_name = c_upper(name_pfx + 'mask')
+                    max_val = f' = {(entry.user_value() << 1) - 1},'
+                    cw.p(max_name + max_val)
+                else:
+                    max_name = c_upper(name_pfx + 'max')
+                    cw.p('__' + max_name + ',')
+                    cw.p(max_name + ' = (__' + max_name + ' - 1)')
             cw.block_end(line=';')
             cw.nl()
         elif const['type'] == 'const':

Signature · 1 line

2.39.2

Jakub Kicinski

maintainer

2023-03-08 08:07 UTC #2 lore raw

Re: [PATCH net-next 1/8] tools: ynl: fix render-max for flags definition

Quoted 15 lines · On Tue, 7 Mar 2023 15:53:58 +0100 Lorenzo Bianconi wrote:

Properly manage render-max property for flags definition type introducing mask value and setting it to (last_element << 1) - 1 instead of adding max value set to last_element + 1
Fixes be5bea1cc0bf ("net: add basic C code generators for Netlink")
Signed-off-by Lorenzo Bianconi <[email protected]>
---
 tools/net/ynl/ynl-gen-c.py | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
tools/net/ynl/ynl-gen-c.py +0 -0
index 274e9c566f61..f2e41dd962d4 100755

tools/net/ynl/ynl-gen-c.py def render_uapi(family, cw): · line 2000

Quoted 9 lines

tools/net/ynl/ynl-gen-c.py +3 -3

@@ -1995,9 +1995,14 @@ def render_uapi(family, cw):
 
             if const.get('render-max', False):
                 cw.nl()
-                max_name = c_upper(name_pfx + 'max')
-                cw.p('__' + max_name + ',')
-                cw.p(max_name + ' = (__' + max_name + ' - 1)')
+                if const['type'] == 'flags':
+                    max_name = c_upper(name_pfx + 'mask')
+                    max_val = f' = {(entry.user_value() << 1) - 1},'

Hm, why not use const.get_mask() here? Rather than the last entry?

Quoted 8 lines

tools/net/ynl/ynl-gen-c.py +5 -0

+                    cw.p(max_name + max_val)
+                else:
+                    max_name = c_upper(name_pfx + 'max')
+                    cw.p('__' + max_name + ',')
+                    cw.p(max_name + ' = (__' + max_name + ' - 1)')
             cw.block_end(line=';')
             cw.nl()
         elif const['type'] == 'const':

Lorenzo Bianconi

author

2023-03-08 10:28 UTC #3 lore raw

Re: [PATCH net-next 1/8] tools: ynl: fix render-max for flags definition

Quoted 17 lines · On Tue, 7 Mar 2023 15:53:58 +0100 Lorenzo Bianconi wrote:

Quoted 16 lines
Properly manage render-max property for flags definition type introducing mask value and setting it to (last_element << 1) - 1 instead of adding max value set to last_element + 1
Fixes be5bea1cc0bf ("net: add basic C code generators for Netlink")
Signed-off-by Lorenzo Bianconi <[email protected]>
---
 tools/net/ynl/ynl-gen-c.py | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
tools/net/ynl/ynl-gen-c.py +0 -0
index 274e9c566f61..f2e41dd962d4 100755
@@ -1995,9 +1995,14 @@ def render_uapi(family, cw):
 

Quoted 10 lines

tools/net/ynl/ynl-gen-c.py def render_uapi(family, cw): · line 2000

Quoted 8 lines

tools/net/ynl/ynl-gen-c.py +3 -3

             if const.get('render-max', False):
                 cw.nl()
-                max_name = c_upper(name_pfx + 'max')
-                cw.p('__' + max_name + ',')
-                cw.p(max_name + ' = (__' + max_name + ' - 1)')
+                if const['type'] == 'flags':
+                    max_name = c_upper(name_pfx + 'mask')
+                    max_val = f' = {(entry.user_value() << 1) - 1},'

Hm, why not use const.get_mask() here? Rather than the last entry?

actually I did this change but it ended up in patch 3/8. I will fix it in v2.

Regards,
Lorenzo

Quoted 10 lines

Quoted 9 lines

tools/net/ynl/ynl-gen-c.py +5 -0

+                    cw.p(max_name + max_val)
+                else:
+                    max_name = c_upper(name_pfx + 'max')
+                    cw.p('__' + max_name + ',')
+                    cw.p(max_name + ' = (__' + max_name + ' - 1)')
             cw.block_end(line=';')
             cw.nl()
         elif const['type'] == 'const':

Lorenzo Bianconi

author

2023-03-07 15:03 UTC #4 lore raw

PATCH 2/8 tools: ynl: fix get_mask utility routine

In reply to Lorenzo Bianconi

Fix get_mask utility routine in order to take into account possible gaps in the elements list.

Fixes be5bea1cc0bf ("net: add basic C code generators for Netlink")

Signed-off-by Lorenzo Bianconi <[email protected]>

---
 tools/net/ynl/ynl-gen-c.py | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

tools/net/ynl/ynl-gen-c.py +2 -4

index f2e41dd962d4..0d6df9414aa9 100755
@@ -652,10 +652,8 @@ class EnumSet:
 
     def get_mask(self):
         mask = 0
-        idx = self.yaml.get('value-start', 0)
-        for _ in self.entry_list:
-            mask |= 1 << idx
-            idx += 1
+        for e in self.entry_list:
+            mask += e.user_value()
         return mask

Signature · 1 line

2.39.2

Lorenzo Bianconi

author

2023-03-07 15:03 UTC #5 lore raw

PATCH 3/8 xdp: add xdp_set_features_flag utility routine

In reply to Lorenzo Bianconi

Introduce xdp_set_features_flag utility routine in order to update dynamically xdp_features according to the dynamic hw configuration via ethtool (e.g. changing number of hw rx/tx queues).
Add xdp_clear_features_flag() in order to clear all xdp_feature flag.

Signed-off-by Lorenzo Bianconi <[email protected]>

---
 Documentation/netlink/specs/netdev.yaml |  1 +
 include/net/xdp.h                       | 11 +++++++++++
 include/uapi/linux/netdev.h             |  2 ++
 net/core/xdp.c                          | 26 ++++++++++++++++++-------
 tools/include/uapi/linux/netdev.h       |  2 ++
 tools/net/ynl/ynl-gen-c.py              |  2 +-
 6 files changed, 36 insertions(+), 8 deletions(-)

Documentation/netlink/specs/netdev.yaml +1 -0

index cffef09729f1..7e4a5b4e7162 100644
@@ -7,6 +7,7 @@ definitions:
   -
     type: flags
     name: xdp-act
+    render-max: true
     entries:
       -
         name: basic

include/net/xdp.h +11 -0

index d517bfac937b..41c57b8b1671 100644
@@ -428,12 +428,18 @@ MAX_XDP_METADATA_KFUNC,
 #ifdef CONFIG_NET
 u32 bpf_xdp_metadata_kfunc_id(int id);
 bool bpf_dev_bound_kfunc_id(u32 btf_id);
+void xdp_set_features_flag(struct net_device *dev, xdp_features_t val);
 void xdp_features_set_redirect_target(struct net_device *dev, bool support_sg);
 void xdp_features_clear_redirect_target(struct net_device *dev);
 #else
 static inline u32 bpf_xdp_metadata_kfunc_id(int id) { return 0; }
 static inline bool bpf_dev_bound_kfunc_id(u32 btf_id) { return false; }
 
+static inline void
+xdp_set_features_flag(struct net_device *dev, xdp_features_t val)
+{
+}
+
 static inline void
 xdp_features_set_redirect_target(struct net_device *dev, bool support_sg)
 {

@@ -445,4 +451,9 @@ xdp_features_clear_redirect_target(struct net_device *dev)
 }
 #endif
 
+static inline void xdp_clear_features_flag(struct net_device *dev)
+{
+	xdp_set_features_flag(dev, 0);
+}
+
 #endif /* __LINUX_NET_XDP_H__ */

include/uapi/linux/netdev.h +2 -0

index 588391447bfb..497cfc93f2e3 100644
@@ -33,6 +33,8 @@ enum netdev_xdp_act {
 	NETDEV_XDP_ACT_HW_OFFLOAD = 16,
 	NETDEV_XDP_ACT_RX_SG = 32,
 	NETDEV_XDP_ACT_NDO_XMIT_SG = 64,
+
+	NETDEV_XDP_ACT_MASK = 127,
 };
 
 enum {

net/core/xdp.c +19 -7

index 8c92fc553317..87e654b7d06c 100644
@@ -774,20 +774,32 @@ static int __init xdp_metadata_init(void)
 }
 late_initcall(xdp_metadata_init);
 
-void xdp_features_set_redirect_target(struct net_device *dev, bool support_sg)
+void xdp_set_features_flag(struct net_device *dev, xdp_features_t val)
 {
-	dev->xdp_features |= NETDEV_XDP_ACT_NDO_XMIT;
-	if (support_sg)
-		dev->xdp_features |= NETDEV_XDP_ACT_NDO_XMIT_SG;
+	val &= NETDEV_XDP_ACT_MASK;
+	if (dev->xdp_features == val)
+		return;
 
+	dev->xdp_features = val;
 	call_netdevice_notifiers(NETDEV_XDP_FEAT_CHANGE, dev);
 }
+EXPORT_SYMBOL_GPL(xdp_set_features_flag);
+
+void xdp_features_set_redirect_target(struct net_device *dev, bool support_sg)
+{
+	xdp_features_t val = (dev->xdp_features | NETDEV_XDP_ACT_NDO_XMIT);
+
+	if (support_sg)
+		val |= NETDEV_XDP_ACT_NDO_XMIT_SG;
+	xdp_set_features_flag(dev, val);
+}
 EXPORT_SYMBOL_GPL(xdp_features_set_redirect_target);
 
 void xdp_features_clear_redirect_target(struct net_device *dev)
 {
-	dev->xdp_features &= ~(NETDEV_XDP_ACT_NDO_XMIT |
-			       NETDEV_XDP_ACT_NDO_XMIT_SG);
-	call_netdevice_notifiers(NETDEV_XDP_FEAT_CHANGE, dev);
+	xdp_features_t val = dev->xdp_features;
+
+	val &= ~(NETDEV_XDP_ACT_NDO_XMIT | NETDEV_XDP_ACT_NDO_XMIT_SG);
+	xdp_set_features_flag(dev, val);
 }
 EXPORT_SYMBOL_GPL(xdp_features_clear_redirect_target);

tools/include/uapi/linux/netdev.h +2 -0

index 588391447bfb..497cfc93f2e3 100644
@@ -33,6 +33,8 @@ enum netdev_xdp_act {
 	NETDEV_XDP_ACT_HW_OFFLOAD = 16,
 	NETDEV_XDP_ACT_RX_SG = 32,
 	NETDEV_XDP_ACT_NDO_XMIT_SG = 64,
+
+	NETDEV_XDP_ACT_MASK = 127,
 };
 
 enum {

tools/net/ynl/ynl-gen-c.py +1 -1

index 0d6df9414aa9..feb86f043e3b 100755
@@ -1995,7 +1995,7 @@ def render_uapi(family, cw):
                 cw.nl()
                 if const['type'] == 'flags':
                     max_name = c_upper(name_pfx + 'mask')
-                    max_val = f' = {(entry.user_value() << 1) - 1},'
+                    max_val = f' = {enum.get_mask()},'
                     cw.p(max_name + max_val)
                 else:
                     max_name = c_upper(name_pfx + 'max')

Signature · 1 line

2.39.2

Lorenzo Bianconi

author

2023-03-07 15:03 UTC #6 lore raw

PATCH 4/8 net: thunderx: take into account xdp_features setting tx/rx queues

In reply to Lorenzo Bianconi

thunderx nic allows xdp just if enough hw queues are available for XDP. Take into account queues configuration setting xdp_features.

Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")

Signed-off-by Lorenzo Bianconi <[email protected]>

---
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c | 17 +++++++++++------
 .../net/ethernet/cavium/thunder/nicvf_main.c    |  4 +++-
 2 files changed, 14 insertions(+), 7 deletions(-)

drivers/net/ethernet/cavium/thunder/nicvf_ethtool.c +11 -6

index e5c71f907852..d8d71bf97983 100644
@@ -735,12 +735,17 @@ static int nicvf_set_channels(struct net_device *dev,
 	if (channel->tx_count > nic->max_queues)
 		return -EINVAL;
 
-	if (nic->xdp_prog &&
-	    ((channel->tx_count + channel->rx_count) > nic->max_queues)) {
-		netdev_err(nic->netdev,
-			   "XDP mode, RXQs + TXQs > Max %d\n",
-			   nic->max_queues);
-		return -EINVAL;
+	if (channel->tx_count + channel->rx_count > nic->max_queues) {
+		if (nic->xdp_prog) {
+			netdev_err(nic->netdev,
+				   "XDP mode, RXQs + TXQs > Max %d\n",
+				   nic->max_queues);
+			return -EINVAL;
+		}
+
+		xdp_clear_features_flag(nic->netdev);
+	} else if (!pass1_silicon(nic->pdev)) {
+		xdp_set_features_flag(dev, NETDEV_XDP_ACT_BASIC);
 	}
 
 	if (if_up)

drivers/net/ethernet/cavium/thunder/nicvf_main.c +3 -1

index 8b25313c7f6b..eff350e0bc2a 100644
@@ -2218,7 +2218,9 @@ static int nicvf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 	netdev->netdev_ops = &nicvf_netdev_ops;
 	netdev->watchdog_timeo = NICVF_TX_TIMEOUT;
 
-	netdev->xdp_features = NETDEV_XDP_ACT_BASIC;
+	if (!pass1_silicon(nic->pdev) &&
+	    nic->rx_queues + nic->tx_queues <= nic->max_queues)
+		netdev->xdp_features = NETDEV_XDP_ACT_BASIC;
 
 	/* MTU range: 64 - 9200 */
 	netdev->min_mtu = NIC_HW_MIN_FRS;

Signature · 1 line

2.39.2

Lorenzo Bianconi

author

2023-03-07 15:03 UTC #7 lore raw

PATCH 5/8 net: ena: take into account xdp_features setting tx/rx queues

In reply to Lorenzo Bianconi

ena nic allows xdp just if enough hw queues are available for XDP. Take into account queues configuration setting xdp_features.

Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")

Signed-off-by Lorenzo Bianconi <[email protected]>

---
 drivers/net/ethernet/amazon/ena/ena_ethtool.c | 15 ++++++++++++---
 drivers/net/ethernet/amazon/ena/ena_netdev.c  |  6 ++++--
 2 files changed, 16 insertions(+), 5 deletions(-)

drivers/net/ethernet/amazon/ena/ena_ethtool.c +12 -3

index 8da79eedc057..1d4f2f4d10f2 100644
@@ -850,11 +850,20 @@ static int ena_set_channels(struct net_device *netdev,
 	struct ena_adapter *adapter = netdev_priv(netdev);
 	u32 count = channels->combined_count;
 	/* The check for max value is already done in ethtool */
-	if (count < ENA_MIN_NUM_IO_QUEUES ||
-	    (ena_xdp_present(adapter) &&
-	    !ena_xdp_legal_queue_count(adapter, count)))
+	if (count < ENA_MIN_NUM_IO_QUEUES)
 		return -EINVAL;
 
+	if (!ena_xdp_legal_queue_count(adapter, count)) {
+		if (ena_xdp_present(adapter))
+			return -EINVAL;
+
+		xdp_clear_features_flag(netdev);
+	} else {
+		xdp_set_features_flag(netdev,
+				      NETDEV_XDP_ACT_BASIC |
+				      NETDEV_XDP_ACT_REDIRECT);
+	}
+
 	return ena_update_queue_count(adapter, count);
 }

drivers/net/ethernet/amazon/ena/ena_netdev.c +4 -2

index d3999db7c6a2..cbfe7f977270 100644
@@ -4105,8 +4105,6 @@ static void ena_set_conf_feat_params(struct ena_adapter *adapter,
 	/* Set offload features */
 	ena_set_dev_offloads(feat, netdev);
 
-	netdev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT;
-
 	adapter->max_mtu = feat->dev_attr.max_mtu;
 	netdev->max_mtu = adapter->max_mtu;
 	netdev->min_mtu = ENA_MIN_MTU;

@@ -4393,6 +4391,10 @@ static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 	ena_config_debug_area(adapter);
 
+	if (ena_xdp_legal_queue_count(adapter, adapter->num_io_queues))
+		netdev->xdp_features = NETDEV_XDP_ACT_BASIC |
+				       NETDEV_XDP_ACT_REDIRECT;
+
 	memcpy(adapter->netdev->perm_addr, adapter->mac_addr, netdev->addr_len);
 
 	netif_carrier_off(netdev);

Signature · 1 line

2.39.2

Shay Agroskin

Reviewed-by on v1 5/8

2023-03-08 10:29 UTC #8 lore raw

Re: [PATCH net-next 5/8] net: ena: take into account xdp_features setting tx/rx queues

Quoted 11 lines · Lorenzo Bianconi

ena nic allows xdp just if enough hw queues are available for XDP. Take into account queues configuration setting xdp_features.
Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")
Signed-off-by Lorenzo Bianconi <[email protected]>
---
 drivers/net/ethernet/amazon/ena/ena_ethtool.c | 15 
 ++++++++++++---
 drivers/net/ethernet/amazon/ena/ena_netdev.c  |  6 ++++--
 2 files changed, 16 insertions(+), 5 deletions(-)

For the ENA driver changes and the non-driver changes

Reviewed-by Shay Agroskin <[email protected]>

Thank you for doing that (:

Lorenzo Bianconi

author

2023-03-07 15:03 UTC #9 lore raw

PATCH 6/8 veth: take into account device reconfiguration for xdp_features flag

In reply to Lorenzo Bianconi

Take into account tx/rx queues reconfiguration setting device xdp_features flag. Moreover consider NETIF_F_GRO flag in order to enable ndo_xdp_xmit callback.

Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")

Signed-off-by Lorenzo Bianconi <[email protected]>

---
 drivers/net/veth.c | 42 ++++++++++++++++++++++++++++++++++++++----
 1 file changed, 38 insertions(+), 4 deletions(-)

drivers/net/veth.c +38 -4

index 1bb54de7124d..293dc3b2c84a 100644
@@ -1257,6 +1257,26 @@ static int veth_enable_range_safe(struct net_device *dev, int start, int end)
 	return 0;
 }
 
+static void veth_set_xdp_features(struct net_device *dev)
+{
+	struct veth_priv *priv = netdev_priv(dev);
+	struct net_device *peer;
+
+	peer = rcu_dereference(priv->peer);
+	if (peer && peer->real_num_tx_queues <= dev->real_num_rx_queues) {
+		xdp_features_t val = NETDEV_XDP_ACT_BASIC |
+				     NETDEV_XDP_ACT_REDIRECT |
+				     NETDEV_XDP_ACT_RX_SG;
+
+		if (priv->_xdp_prog || veth_gro_requested(dev))
+			val |= NETDEV_XDP_ACT_NDO_XMIT |
+			       NETDEV_XDP_ACT_NDO_XMIT_SG;
+		xdp_set_features_flag(dev, val);
+	} else {
+		xdp_clear_features_flag(dev);
+	}
+}
+
 static int veth_set_channels(struct net_device *dev,
 			     struct ethtool_channels *ch)
 {

@@ -1323,6 +1343,12 @@ static int veth_set_channels(struct net_device *dev,
 		if (peer)
 			netif_carrier_on(peer);
 	}
+
+	/* update XDP supported features */
+	veth_set_xdp_features(dev);
+	if (peer)
+		veth_set_xdp_features(peer);
+
 	return err;
 
 revert:

@@ -1489,7 +1515,10 @@ static int veth_set_features(struct net_device *dev,
 		err = veth_napi_enable(dev);
 		if (err)
 			return err;
+
+		xdp_features_set_redirect_target(dev, true);
 	} else {
+		xdp_features_clear_redirect_target(dev);
 		veth_napi_del(dev);
 	}
 	return 0;

@@ -1570,10 +1599,15 @@ static int veth_xdp_set(struct net_device *dev, struct bpf_prog *prog,
 			peer->hw_features &= ~NETIF_F_GSO_SOFTWARE;
 			peer->max_mtu = max_mtu;
 		}
+
+		xdp_features_set_redirect_target(dev, true);
 	}
 
 	if (old_prog) {
 		if (!prog) {
+			if (!veth_gro_requested(dev))
+				xdp_features_clear_redirect_target(dev);
+
 			if (dev->flags & IFF_UP)
 				veth_disable_xdp(dev);

@@ -1686,10 +1720,6 @@ static void veth_setup(struct net_device *dev)
 	dev->hw_enc_features = VETH_FEATURES;
 	dev->mpls_features = NETIF_F_HW_CSUM | NETIF_F_GSO_SOFTWARE;
 	netif_set_tso_max_size(dev, GSO_MAX_SIZE);
-
-	dev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT |
-			    NETDEV_XDP_ACT_NDO_XMIT | NETDEV_XDP_ACT_RX_SG |
-			    NETDEV_XDP_ACT_NDO_XMIT_SG;
 }
 
 /*

@@ -1857,6 +1887,10 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
 		goto err_queues;
 
 	veth_disable_gro(dev);
+	/* update XDP supported features */
+	veth_set_xdp_features(dev);
+	veth_set_xdp_features(peer);
+
 	return 0;
 
 err_queues:

Signature · 1 line

2.39.2

Lorenzo Bianconi

author

2023-03-07 15:03 UTC #10 lore raw

PATCH 7/8 net/mlx5e: take into account device reconfiguration for xdp_features flag

In reply to Lorenzo Bianconi

Take into account LRO and GRO configuration setting device xdp_features flag. Moreover consider channel rq_wq_type enabling rx scatter-gatter support in xdp_features flag.

Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")

Signed-off-by Lorenzo Bianconi <[email protected]>

---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  1 +
 .../ethernet/mellanox/mlx5/core/en_ethtool.c  | 10 ++++-
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++---
 .../net/ethernet/mellanox/mlx5/core/en_rep.c  |  3 ++
 4 files changed, 51 insertions(+), 8 deletions(-)

drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -0

index 88460b7796e5..4276c6eb6820 100644
@@ -1243,6 +1243,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16
 void mlx5e_rx_dim_work(struct work_struct *work);
 void mlx5e_tx_dim_work(struct work_struct *work);
 
+void mlx5e_set_xdp_feature(struct net_device *netdev);
 netdev_features_t mlx5e_features_check(struct sk_buff *skb,
 				       struct net_device *netdev,
 				       netdev_features_t features);

drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +9 -1

index 7708acc9b2ab..79fd21ecb9cb 100644
@@ -1985,6 +1985,7 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	struct mlx5_core_dev *mdev = priv->mdev;
 	struct mlx5e_params new_params;
+	int err;
 
 	if (enable) {
 		/* Checking the regular RQ here; mlx5e_validate_xsk_param called

@@ -2005,7 +2006,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
 	MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ, enable);
 	mlx5e_set_rq_type(mdev, &new_params);
 
-	return mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
+	err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
+	if (err)
+		return err;
+
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
+	return 0;
 }
 
 static int set_pflag_rx_no_csum_complete(struct net_device *netdev, bool enable)

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +38 -7

79 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/en_rep.c +3 -0

index 9b9203443085..43fd12fb87b8 100644
@@ -747,6 +747,9 @@ static void mlx5e_build_rep_params(struct net_device *netdev)
 	/* RQ */
 	mlx5e_build_rq_params(mdev, params);
 
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
 	/* CQ moderation params */
 	params->rx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation);
 	mlx5e_set_rx_cq_mode_params(params, cq_period_mode);

Signature · 1 line

2.39.2

Tariq Toukan

2023-03-08 10:34 UTC #11 lore raw

Re: [PATCH net-next 7/8] net/mlx5e: take into account device reconfiguration for xdp_features flag

Quoted 58 lines · Lorenzo Bianconi

Take into account LRO and GRO configuration setting device xdp_features flag. Moreover consider channel rq_wq_type enabling rx scatter-gatter support in xdp_features flag.

Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")

Signed-off-by Lorenzo Bianconi <[email protected]>

---
  drivers/net/ethernet/mellanox/mlx5/core/en.h  |  1 +
  .../ethernet/mellanox/mlx5/core/en_ethtool.c  | 10 ++++-
  .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++---
  .../net/ethernet/mellanox/mlx5/core/en_rep.c  |  3 ++
  4 files changed, 51 insertions(+), 8 deletions(-)

drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -0

index 88460b7796e5..4276c6eb6820 100644
@@ -1243,6 +1243,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16
  void mlx5e_rx_dim_work(struct work_struct *work);
  void mlx5e_tx_dim_work(struct work_struct *work);
 
+void mlx5e_set_xdp_feature(struct net_device *netdev);
  netdev_features_t mlx5e_features_check(struct sk_buff *skb,
  				       struct net_device *netdev,
  				       netdev_features_t features);

drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +9 -1

index 7708acc9b2ab..79fd21ecb9cb 100644
@@ -1985,6 +1985,7 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
  	struct mlx5e_priv *priv = netdev_priv(netdev);
  	struct mlx5_core_dev *mdev = priv->mdev;
  	struct mlx5e_params new_params;
+	int err;
 
  	if (enable) {
  		/* Checking the regular RQ here; mlx5e_validate_xsk_param called

@@ -2005,7 +2006,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
  	MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ, enable);
  	mlx5e_set_rq_type(mdev, &new_params);
 
-	return mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
+	err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
+	if (err)
+		return err;
+
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
+	return 0;
  }
 
  static int set_pflag_rx_no_csum_complete(struct net_device *netdev, bool enable)

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -0

index 76a9c5194a70..1b68dd2be2c5 100644

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_handle_feature(struct net_device *netdev, · line 4010

Quoted 7 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +4 -0

@@ -4004,6 +4004,30 @@ static int mlx5e_handle_feature(struct net_device *netdev,
  	return 0;
  }
 
+void mlx5e_set_xdp_feature(struct net_device *netdev)
+{
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	bool ndo_xmit = test_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state);

Our driver doesn't require loading a dummy XDP program to have the redirect-in ability. It's always there.

I actually have a bug fix under internal review with Saeed that addresses this.

In addition, it cleans up the NETDEV_XDP_ACT_NDO_XMIT_SG as we do not support it yet. I have a series that's adding support and will submit it soon.

Any reason you're submitting these fixes to net-next rather than net? Maybe it'd be better if we integrate the patches, here's my fix (still under review...):

Author: Tariq Toukan <[email protected]>
Date:   Thu Feb 23 08:58:04 2023 +0200

net/mlx5e: Fix exposed xdp_features

Always declare NETDEV_XDP_ACT_NDO_XMIT as the ndo_xdp_xmit callback is always functional per our design, and does not require loading a dummy xdp program.

Although non-linear XDP buffer is supported for XDP_TX flow, do not declare NETDEV_XDP_ACT_NDO_XMIT_SG as it is yet supported for redirected-in frames.

Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")

Signed-off-by Tariq Toukan <[email protected]>

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -7

index 53feb0529943..9a5d3ce1fbcd 100644
@@ -4741,13 +4741,6 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog)
        if (old_prog)
                bpf_prog_put(old_prog);
 
-       if (reset) {
-               if (prog)
-                       xdp_features_set_redirect_target(netdev, true);
-               else
-                       xdp_features_clear_redirect_target(netdev);
-       }
-
        if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset)
                goto unlock;

@@ -5144,6 +5137,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
        netdev->features         |= NETIF_F_HW_VLAN_STAG_FILTER;
 
        netdev->xdp_features = NETDEV_XDP_ACT_BASIC |

NETDEV_XDP_ACT_REDIRECT |
+ NETDEV_XDP_ACT_NDO_XMIT |

                               NETDEV_XDP_ACT_XSK_ZEROCOPY |
                               NETDEV_XDP_ACT_RX_SG;

Quoted 6 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +6 -0

+	struct mlx5e_params *params = &priv->channels.params;
+	xdp_features_t val;
+
+	if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) {
+		xdp_clear_features_flag(netdev);
+		return;

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_handle_feature(struct net_device *netdev, · line 4026

Quoted 10 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +10 -0

+	}
+
+	val = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT |
+	      NETDEV_XDP_ACT_XSK_ZEROCOPY;
+	if (ndo_xmit)
+		val |= NETDEV_XDP_ACT_NDO_XMIT;
+	if (params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC) {
+		val |= NETDEV_XDP_ACT_RX_SG;
+		if (ndo_xmit)
+			val |= NETDEV_XDP_ACT_NDO_XMIT_SG;

This NETDEV_XDP_ACT_NDO_XMIT_SG capability is not related to the RQ type. It's still not supported at this point.

BTW, I have a series completing all the missing capabilities (multibuf on Striding + multibuf redirect-in), should be submitted in this kernel.

Quoted 18 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +7 -0

+	}
+	xdp_set_features_flag(netdev, val);
+}
+
  int mlx5e_set_features(struct net_device *netdev, netdev_features_t features)
  {
  	netdev_features_t oper_features = features;

@@ -4030,6 +4054,9 @@ int mlx5e_set_features(struct net_device *netdev, netdev_features_t features)
  		return -EINVAL;
  	}
 
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
  	return 0;
  }

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) · line 4795

Quoted 10 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +4 -3

@@ -4762,10 +4789,14 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog)
  		bpf_prog_put(old_prog);
 
  	if (reset) {
-		if (prog)
-			xdp_features_set_redirect_target(netdev, true);
-		else
+		if (prog) {
+			bool xmit_sg;
+
+			xmit_sg = new_params.rq_wq_type == MLX5_WQ_TYPE_CYCLIC;

Same, not related. Still not supported at this point.

Quoted 46 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +7 -4

+			xdp_features_set_redirect_target(netdev, xmit_sg);
+		} else {
  			xdp_features_clear_redirect_target(netdev);
+		}
  	}
 
  	if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset)

@@ -5163,13 +5194,10 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
  	netdev->features         |= NETIF_F_HIGHDMA;
  	netdev->features         |= NETIF_F_HW_VLAN_STAG_FILTER;
 
-	netdev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT |
-			       NETDEV_XDP_ACT_XSK_ZEROCOPY |
-			       NETDEV_XDP_ACT_RX_SG;
-
  	netdev->priv_flags       |= IFF_UNICAST_FLT;
 
  	netif_set_tso_max_size(netdev, GSO_MAX_SIZE);
+	mlx5e_set_xdp_feature(netdev);
  	mlx5e_set_netdev_dev_addr(netdev);
  	mlx5e_macsec_build_netdev(priv);
  	mlx5e_ipsec_build_netdev(priv);

@@ -5241,6 +5269,9 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev,
  		mlx5_core_err(mdev, "TLS initialization failed, %d\n", err);
 
  	mlx5e_health_create_reporters(priv);
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
  	return 0;
  }

drivers/net/ethernet/mellanox/mlx5/core/en_rep.c +3 -0

index 9b9203443085..43fd12fb87b8 100644
@@ -747,6 +747,9 @@ static void mlx5e_build_rep_params(struct net_device *netdev)
  	/* RQ */
  	mlx5e_build_rq_params(mdev, params);
 
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
  	/* CQ moderation params */
  	params->rx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation);
  	mlx5e_set_rx_cq_mode_params(params, cq_period_mode);

Lorenzo Bianconi

2023-03-08 15:49 UTC #12 lore raw

Re: [PATCH net-next 7/8] net/mlx5e: take into account device reconfiguration for xdp_features flag

Quoted 67 lines · Lorenzo Bianconi

Quoted 54 lines

Take into account LRO and GRO configuration setting device xdp_features flag. Moreover consider channel rq_wq_type enabling rx scatter-gatter support in xdp_features flag.

Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")

Signed-off-by Lorenzo Bianconi <[email protected]>

---
  drivers/net/ethernet/mellanox/mlx5/core/en.h  |  1 +
  .../ethernet/mellanox/mlx5/core/en_ethtool.c  | 10 ++++-
  .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++---
  .../net/ethernet/mellanox/mlx5/core/en_rep.c  |  3 ++
  4 files changed, 51 insertions(+), 8 deletions(-)

drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -0

index 88460b7796e5..4276c6eb6820 100644
@@ -1243,6 +1243,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16
  void mlx5e_rx_dim_work(struct work_struct *work);
  void mlx5e_tx_dim_work(struct work_struct *work);
+void mlx5e_set_xdp_feature(struct net_device *netdev);
  netdev_features_t mlx5e_features_check(struct sk_buff *skb,
  				       struct net_device *netdev,
  				       netdev_features_t features);

drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +9 -1

index 7708acc9b2ab..79fd21ecb9cb 100644
@@ -1985,6 +1985,7 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
  	struct mlx5e_priv *priv = netdev_priv(netdev);
  	struct mlx5_core_dev *mdev = priv->mdev;
  	struct mlx5e_params new_params;
+	int err;
  	if (enable) {
  		/* Checking the regular RQ here; mlx5e_validate_xsk_param called

@@ -2005,7 +2006,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
  	MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ, enable);
  	mlx5e_set_rq_type(mdev, &new_params);
-	return mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
+	err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
+	if (err)
+		return err;
+
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
+	return 0;
  }
  static int set_pflag_rx_no_csum_complete(struct net_device *netdev, bool enable)

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -0

index 76a9c5194a70..1b68dd2be2c5 100644

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_handle_feature(struct net_device *netdev, · line 4009

Quoted 6 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +4 -0

@@ -4004,6 +4004,30 @@ static int mlx5e_handle_feature(struct net_device *netdev,
  	return 0;
  }
+void mlx5e_set_xdp_feature(struct net_device *netdev)
+{
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	bool ndo_xmit = test_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state);

Our driver doesn't require loading a dummy XDP program to have the redirect-in ability. It's always there.

Quoted 8 lines

I actually have a bug fix under internal review with Saeed that addresses this.
In addition, it cleans up the NETDEV_XDP_ACT_NDO_XMIT_SG as we do not support it yet. I have a series that's adding support and will submit it soon.
Any reason you're submitting these fixes to net-next rather than net?

Hi Tariq,

I am fine to repost this series for net instead of net-next. Any downsides about it?

Quoted 46 lines

Maybe it'd be better if we integrate the patches, here's my fix (still under review...):
Author: Tariq Toukan <[email protected]>
Date:   Thu Feb 23 08:58:04 2023 +0200
net/mlx5e: Fix exposed xdp_features
Always declare NETDEV_XDP_ACT_NDO_XMIT as the ndo_xdp_xmit callback is always functional per our design, and does not require loading a dummy xdp program.
Although non-linear XDP buffer is supported for XDP_TX flow, do not declare NETDEV_XDP_ACT_NDO_XMIT_SG as it is yet supported for redirected-in frames.
Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")
Signed-off-by Tariq Toukan <[email protected]>
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -7
index 53feb0529943..9a5d3ce1fbcd 100644
@@ -4741,13 +4741,6 @@ static int mlx5e_xdp_set(struct net_device *netdev,
 struct bpf_prog *prog)
        if (old_prog)
                bpf_prog_put(old_prog);
 
-       if (reset) {
-               if (prog)
-                       xdp_features_set_redirect_target(netdev, true);
-               else
-                       xdp_features_clear_redirect_target(netdev);
-       }
-
        if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset)
                goto unlock;
@@ -5144,6 +5137,7 @@ static void mlx5e_build_nic_netdev(struct net_device
 *netdev)
        netdev->features         |= NETIF_F_HW_VLAN_STAG_FILTER;
 
        netdev->xdp_features = NETDEV_XDP_ACT_BASIC |
NETDEV_XDP_ACT_REDIRECT |
+ NETDEV_XDP_ACT_NDO_XMIT |

Quoted 2 lines

                               NETDEV_XDP_ACT_XSK_ZEROCOPY |
                               NETDEV_XDP_ACT_RX_SG;

I am fine to drop this my patch and rely on the one you provided but it depends on the eta about the described patches because otherwise real capabilities and xdp-features will not be aligned. Any inputs on it?

Quoted 13 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +11 -0

+	struct mlx5e_params *params = &priv->channels.params;
+	xdp_features_t val;
+
+	if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) {
+		xdp_clear_features_flag(netdev);
+		return;
+	}
+
+	val = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT |
+	      NETDEV_XDP_ACT_XSK_ZEROCOPY;
+	if (ndo_xmit)

Quoted 8 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_handle_feature(struct net_device *netdev, · line 4025
Quoted 5 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +5 -0
+		val |= NETDEV_XDP_ACT_NDO_XMIT;
+	if (params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC) {
+		val |= NETDEV_XDP_ACT_RX_SG;
+		if (ndo_xmit)
+			val |= NETDEV_XDP_ACT_NDO_XMIT_SG;
This NETDEV_XDP_ACT_NDO_XMIT_SG capability is not related to the RQ type. It's still not supported at this point.

ack, I will fix it.

Quoted 3 lines

BTW, I have a series completing all the missing capabilities (multibuf on Striding + multibuf redirect-in), should be submitted in this kernel.

cool :)

Regards,
Lorenzo

Quoted 70 lines

Quoted 17 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +7 -0

+	}
+	xdp_set_features_flag(netdev, val);
+}
+
  int mlx5e_set_features(struct net_device *netdev, netdev_features_t features)
  {
  	netdev_features_t oper_features = features;

@@ -4030,6 +4054,9 @@ int mlx5e_set_features(struct net_device *netdev, netdev_features_t features)
  		return -EINVAL;
  	}
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
  	return 0;
  }

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) · line 4794

Quoted 9 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +4 -3

@@ -4762,10 +4789,14 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog)
  		bpf_prog_put(old_prog);
  	if (reset) {
-		if (prog)
-			xdp_features_set_redirect_target(netdev, true);
-		else
+		if (prog) {
+			bool xmit_sg;
+
+			xmit_sg = new_params.rq_wq_type == MLX5_WQ_TYPE_CYCLIC;

Same, not related. Still not supported at this point.

Quoted 40 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +7 -4

+			xdp_features_set_redirect_target(netdev, xmit_sg);
+		} else {
  			xdp_features_clear_redirect_target(netdev);
+		}
  	}
  	if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset)

@@ -5163,13 +5194,10 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
  	netdev->features         |= NETIF_F_HIGHDMA;
  	netdev->features         |= NETIF_F_HW_VLAN_STAG_FILTER;
-	netdev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT |
-			       NETDEV_XDP_ACT_XSK_ZEROCOPY |
-			       NETDEV_XDP_ACT_RX_SG;
-
  	netdev->priv_flags       |= IFF_UNICAST_FLT;
  	netif_set_tso_max_size(netdev, GSO_MAX_SIZE);
+	mlx5e_set_xdp_feature(netdev);
  	mlx5e_set_netdev_dev_addr(netdev);
  	mlx5e_macsec_build_netdev(priv);
  	mlx5e_ipsec_build_netdev(priv);

@@ -5241,6 +5269,9 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev,
  		mlx5_core_err(mdev, "TLS initialization failed, %d\n", err);
  	mlx5e_health_create_reporters(priv);
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
  	return 0;
  }

drivers/net/ethernet/mellanox/mlx5/core/en_rep.c +3 -0

index 9b9203443085..43fd12fb87b8 100644
@@ -747,6 +747,9 @@ static void mlx5e_build_rep_params(struct net_device *netdev)
  	/* RQ */
  	mlx5e_build_rq_params(mdev, params);
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
  	/* CQ moderation params */
  	params->rx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation);
  	mlx5e_set_rx_cq_mode_params(params, cq_period_mode);

Tariq Toukan

2023-03-09 07:23 UTC #13 lore raw

Re: [PATCH net-next 7/8] net/mlx5e: take into account device reconfiguration for xdp_features flag

Quoted 74 lines · Lorenzo Bianconi

Quoted 54 lines

Take into account LRO and GRO configuration setting device xdp_features flag. Moreover consider channel rq_wq_type enabling rx scatter-gatter support in xdp_features flag.

Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")

Signed-off-by Lorenzo Bianconi <[email protected]>

---
   drivers/net/ethernet/mellanox/mlx5/core/en.h  |  1 +
   .../ethernet/mellanox/mlx5/core/en_ethtool.c  | 10 ++++-
   .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++---
   .../net/ethernet/mellanox/mlx5/core/en_rep.c  |  3 ++
   4 files changed, 51 insertions(+), 8 deletions(-)

drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -0

index 88460b7796e5..4276c6eb6820 100644
@@ -1243,6 +1243,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16
   void mlx5e_rx_dim_work(struct work_struct *work);
   void mlx5e_tx_dim_work(struct work_struct *work);
+void mlx5e_set_xdp_feature(struct net_device *netdev);
   netdev_features_t mlx5e_features_check(struct sk_buff *skb,
   				       struct net_device *netdev,
   				       netdev_features_t features);

drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +9 -1

index 7708acc9b2ab..79fd21ecb9cb 100644
@@ -1985,6 +1985,7 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
   	struct mlx5e_priv *priv = netdev_priv(netdev);
   	struct mlx5_core_dev *mdev = priv->mdev;
   	struct mlx5e_params new_params;
+	int err;
   	if (enable) {
   		/* Checking the regular RQ here; mlx5e_validate_xsk_param called

@@ -2005,7 +2006,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
   	MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ, enable);
   	mlx5e_set_rq_type(mdev, &new_params);
-	return mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
+	err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
+	if (err)
+		return err;
+
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
+	return 0;
   }
   static int set_pflag_rx_no_csum_complete(struct net_device *netdev, bool enable)

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -0

index 76a9c5194a70..1b68dd2be2c5 100644

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_handle_feature(struct net_device *netdev, · line 4009

Quoted 6 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c +4 -0

@@ -4004,6 +4004,30 @@ static int mlx5e_handle_feature(struct net_device *netdev,
   	return 0;
   }
+void mlx5e_set_xdp_feature(struct net_device *netdev)
+{
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	bool ndo_xmit = test_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state);

Our driver doesn't require loading a dummy XDP program to have the redirect-in ability. It's always there.

I actually have a bug fix under internal review with Saeed that addresses this.

In addition, it cleans up the NETDEV_XDP_ACT_NDO_XMIT_SG as we do not support it yet. I have a series that's adding support and will submit it soon.

Quoted 6 lines

Quoted 1 line
Any reason you're submitting these fixes to net-next rather than net?
Hi Tariq,
I am fine to repost this series for net instead of net-next. Any downsides about it?

Let's repost to net.
It's a fixes series, and 6.3 is still in its RCs.
If you don't post it to net then the xdp-features in 6.3 will be broken.

Quoted 44 lines

Quoted 44 lines
Maybe it'd be better if we integrate the patches, here's my fix (still under review...):
Author: Tariq Toukan <[email protected]>
Date:   Thu Feb 23 08:58:04 2023 +0200
net/mlx5e: Fix exposed xdp_features
Always declare NETDEV_XDP_ACT_NDO_XMIT as the ndo_xdp_xmit callback is always functional per our design, and does not require loading a dummy xdp program.
Although non-linear XDP buffer is supported for XDP_TX flow, do not declare NETDEV_XDP_ACT_NDO_XMIT_SG as it is yet supported for redirected-in frames.
Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")
Signed-off-by Tariq Toukan <[email protected]>
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -7
index 53feb0529943..9a5d3ce1fbcd 100644
@@ -4741,13 +4741,6 @@ static int mlx5e_xdp_set(struct net_device *netdev,
 struct bpf_prog *prog)
         if (old_prog)
                 bpf_prog_put(old_prog);
 
-       if (reset) {
-               if (prog)
-                       xdp_features_set_redirect_target(netdev, true);
-               else
-                       xdp_features_clear_redirect_target(netdev);
-       }
-
         if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset)
                 goto unlock;
@@ -5144,6 +5137,7 @@ static void mlx5e_build_nic_netdev(struct net_device
 *netdev)
         netdev->features         |= NETIF_F_HW_VLAN_STAG_FILTER;
 

Quoted 10 lines

Quoted 5 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -0
         netdev->xdp_features = NETDEV_XDP_ACT_BASIC |
NETDEV_XDP_ACT_REDIRECT |
+ NETDEV_XDP_ACT_NDO_XMIT |
                                NETDEV_XDP_ACT_XSK_ZEROCOPY |
                                NETDEV_XDP_ACT_RX_SG;
I am fine to drop this my patch and rely on the one you provided but it depends on the eta about the described patches because otherwise real capabilities and xdp-features will not be aligned. Any inputs on it?

My patch doesn't replace yours, as it doesn't fix the missing features_update according to striding RQ and HW LRO/GRO.

I think we should combine them, either take mine as-is into your series, or squash it into this patch. I'm fine with both.

Quoted 33 lines

Quoted 13 lines
Quoted 13 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +11 -0
+	struct mlx5e_params *params = &priv->channels.params;
+	xdp_features_t val;
+
+	if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) {
+		xdp_clear_features_flag(netdev);
+		return;
+	}
+
+	val = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT |
+	      NETDEV_XDP_ACT_XSK_ZEROCOPY;
+	if (ndo_xmit)
Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_handle_feature(struct net_device *netdev, · line 4025
Quoted 5 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +5 -0
+		val |= NETDEV_XDP_ACT_NDO_XMIT;
+	if (params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC) {
+		val |= NETDEV_XDP_ACT_RX_SG;
+		if (ndo_xmit)
+			val |= NETDEV_XDP_ACT_NDO_XMIT_SG;
This NETDEV_XDP_ACT_NDO_XMIT_SG capability is not related to the RQ type. It's still not supported at this point.
ack, I will fix it.
Quoted 3 lines
BTW, I have a series completing all the missing capabilities (multibuf on Striding + multibuf redirect-in), should be submitted in this kernel.
cool :)
Regards,
Lorenzo

Jakub Kicinski

maintainer

2023-03-09 07:33 UTC #14 lore raw

Re: [PATCH net-next 7/8] net/mlx5e: take into account device reconfiguration for xdp_features flag

Quoted 8 lines · On Thu, 9 Mar 2023 09:23:10 +0200 Tariq Toukan wrote:

Quoted 4 lines
Hi Tariq,
I am fine to repost this series for net instead of net-next. Any downsides about it?
Let's repost to net.
It's a fixes series, and 6.3 is still in its RCs.
If you don't post it to net then the xdp-features in 6.3 will be broken.

minor heads up - patch 2 will now apply to lib/nlspec.py I just moved the enum classes there as part of another fix but the code and the changes should be identical

Lorenzo Bianconi

2023-03-09 08:43 UTC #15 lore raw

Re: [PATCH net-next 7/8] net/mlx5e: take into account device reconfiguration for xdp_features flag

Quoted 6 lines · On Thu, 9 Mar 2023 09:23:10 +0200 Tariq Toukan wrote:

Quoted 5 lines
Quoted 4 lines
Hi Tariq,
I am fine to repost this series for net instead of net-next. Any downsides about it?

Quoted 8 lines

Quoted 3 lines
Let's repost to net.
It's a fixes series, and 6.3 is still in its RCs.
If you don't post it to net then the xdp-features in 6.3 will be broken.
minor heads up - patch 2 will now apply to lib/nlspec.py I just moved the enum classes there as part of another fix but the code and the changes should be identical

ack, fine. I will fix the conflicts rebasing on net tree.

Regards,
Lorenzo

Lorenzo Bianconi

2023-03-09 08:45 UTC #16 lore raw

Re: [PATCH net-next 7/8] net/mlx5e: take into account device reconfiguration for xdp_features flag

In reply to Tariq Toukan

Quoted 79 lines · Lorenzo Bianconi

Quoted 76 lines
Quoted 75 lines · Lorenzo Bianconi
Quoted 54 lines
Take into account LRO and GRO configuration setting device xdp_features flag. Moreover consider channel rq_wq_type enabling rx scatter-gatter support in xdp_features flag.
Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")
Signed-off-by Lorenzo Bianconi <[email protected]>
---
   drivers/net/ethernet/mellanox/mlx5/core/en.h  |  1 +
   .../ethernet/mellanox/mlx5/core/en_ethtool.c  | 10 ++++-
   .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++---
   .../net/ethernet/mellanox/mlx5/core/en_rep.c  |  3 ++
   4 files changed, 51 insertions(+), 8 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -0
index 88460b7796e5..4276c6eb6820 100644
@@ -1243,6 +1243,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16
   void mlx5e_rx_dim_work(struct work_struct *work);
   void mlx5e_tx_dim_work(struct work_struct *work);
+void mlx5e_set_xdp_feature(struct net_device *netdev);
   netdev_features_t mlx5e_features_check(struct sk_buff *skb,
   				       struct net_device *netdev,
   				       netdev_features_t features);
drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +9 -1
index 7708acc9b2ab..79fd21ecb9cb 100644
@@ -1985,6 +1985,7 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
   	struct mlx5e_priv *priv = netdev_priv(netdev);
   	struct mlx5_core_dev *mdev = priv->mdev;
   	struct mlx5e_params new_params;
+	int err;
   	if (enable) {
   		/* Checking the regular RQ here; mlx5e_validate_xsk_param called
@@ -2005,7 +2006,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
   	MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ, enable);
   	mlx5e_set_rq_type(mdev, &new_params);
-	return mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
+	err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
+	if (err)
+		return err;
+
+	/* update XDP supported features */
+	mlx5e_set_xdp_feature(netdev);
+
+	return 0;
   }
   static int set_pflag_rx_no_csum_complete(struct net_device *netdev, bool enable)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -0
index 76a9c5194a70..1b68dd2be2c5 100644
drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_handle_feature(struct net_device *netdev, · line 4009
Quoted 6 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +4 -0
@@ -4004,6 +4004,30 @@ static int mlx5e_handle_feature(struct net_device *netdev,
   	return 0;
   }
+void mlx5e_set_xdp_feature(struct net_device *netdev)
+{
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	bool ndo_xmit = test_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state);
Our driver doesn't require loading a dummy XDP program to have the redirect-in ability. It's always there.
I actually have a bug fix under internal review with Saeed that addresses this.
In addition, it cleans up the NETDEV_XDP_ACT_NDO_XMIT_SG as we do not support it yet. I have a series that's adding support and will submit it soon.
Any reason you're submitting these fixes to net-next rather than net?

Quoted 8 lines

Quoted 4 lines
Hi Tariq,
I am fine to repost this series for net instead of net-next. Any downsides about it?
Let's repost to net.
It's a fixes series, and 6.3 is still in its RCs.
If you don't post it to net then the xdp-features in 6.3 will be broken.

ack, fine.

Quoted 56 lines

Quoted 45 lines
Quoted 45 lines
Maybe it'd be better if we integrate the patches, here's my fix (still under review...):
Author: Tariq Toukan <[email protected]>
Date:   Thu Feb 23 08:58:04 2023 +0200
net/mlx5e: Fix exposed xdp_features
Always declare NETDEV_XDP_ACT_NDO_XMIT as the ndo_xdp_xmit callback is always functional per our design, and does not require loading a dummy xdp program.
Although non-linear XDP buffer is supported for XDP_TX flow, do not declare NETDEV_XDP_ACT_NDO_XMIT_SG as it is yet supported for redirected-in frames.
Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")
Signed-off-by Tariq Toukan <[email protected]>
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -7
index 53feb0529943..9a5d3ce1fbcd 100644
@@ -4741,13 +4741,6 @@ static int mlx5e_xdp_set(struct net_device *netdev,
 struct bpf_prog *prog)
         if (old_prog)
                 bpf_prog_put(old_prog);
 
-       if (reset) {
-               if (prog)
-                       xdp_features_set_redirect_target(netdev, true);
-               else
-                       xdp_features_clear_redirect_target(netdev);
-       }
-
         if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset)
                 goto unlock;
@@ -5144,6 +5137,7 @@ static void mlx5e_build_nic_netdev(struct net_device
 *netdev)
         netdev->features         |= NETIF_F_HW_VLAN_STAG_FILTER;
 
Quoted 10 lines
Quoted 5 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -0
         netdev->xdp_features = NETDEV_XDP_ACT_BASIC |
NETDEV_XDP_ACT_REDIRECT |
+ NETDEV_XDP_ACT_NDO_XMIT |
                                NETDEV_XDP_ACT_XSK_ZEROCOPY |
                                NETDEV_XDP_ACT_RX_SG;
I am fine to drop this my patch and rely on the one you provided but it depends on the eta about the described patches because otherwise real capabilities and xdp-features will not be aligned. Any inputs on it?

Quoted 5 lines

My patch doesn't replace yours, as it doesn't fix the missing features_update according to striding RQ and HW LRO/GRO.
I think we should combine them, either take mine as-is into your series, or squash it into this patch. I'm fine with both.

ack fine, I will squash your changes into the patch posting a new version, thx.

Regards,
Lorenzo

Quoted 35 lines

Quoted 34 lines
Quoted 14 lines
Quoted 14 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +11 -0
+	struct mlx5e_params *params = &priv->channels.params;
+	xdp_features_t val;
+
+	if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) {
+		xdp_clear_features_flag(netdev);
+		return;
+	}
+
+	val = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT |
+	      NETDEV_XDP_ACT_XSK_ZEROCOPY;
+	if (ndo_xmit)
Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c static int mlx5e_handle_feature(struct net_device *netdev, · line 4025
Quoted 5 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +5 -0
+		val |= NETDEV_XDP_ACT_NDO_XMIT;
+	if (params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC) {
+		val |= NETDEV_XDP_ACT_RX_SG;
+		if (ndo_xmit)
+			val |= NETDEV_XDP_ACT_NDO_XMIT_SG;
This NETDEV_XDP_ACT_NDO_XMIT_SG capability is not related to the RQ type. It's still not supported at this point.
ack, I will fix it.
Quoted 3 lines
BTW, I have a series completing all the missing capabilities (multibuf on Striding + multibuf redirect-in), should be submitted in this kernel.
cool :)
Regards,
Lorenzo

Lorenzo Bianconi

author

2023-03-07 15:03 UTC #17 lore raw

PATCH 8/8 mvpp2: take care of xdp_features when reconfiguring queues

In reply to Lorenzo Bianconi

From: Matteo Croce <[email protected]>

XDP is supported only if enough queues are present, so when reconfiguring the queues set xdp_features accordingly.

Fixes 66c0e13ad236 ("drivers: net: turn on XDP features")

Signed-off-by Matteo Croce <[email protected]>

Signed-off-by Lorenzo Bianconi <[email protected]>

---
 drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +12 -3

index 9b4ecbe4f36d..3ea00bc9b91c 100644
@@ -4996,6 +4996,14 @@ static int mvpp2_bm_switch_buffers(struct mvpp2 *priv, bool percpu)
 
 	for (i = 0; i < priv->port_count; i++) {
 		port = priv->port_list[i];
+		if (percpu && port->ntxqs >= num_possible_cpus() * 2)
+			xdp_set_features_flag(port->dev,
+					      NETDEV_XDP_ACT_BASIC |
+					      NETDEV_XDP_ACT_REDIRECT |
+					      NETDEV_XDP_ACT_NDO_XMIT);
+		else
+			xdp_clear_features_flag(port->dev);
+
 		mvpp2_swf_bm_pool_init(port);
 		if (status[i])
 			mvpp2_open(port->dev);

@@ -6863,13 +6871,14 @@ static int mvpp2_port_probe(struct platform_device *pdev,
 
 	if (!port->priv->percpu_pools)
 		mvpp2_set_hw_csum(port, port->pool_long->id);
+	else if (port->ntxqs >= num_possible_cpus() * 2)
+		dev->xdp_features = NETDEV_XDP_ACT_BASIC |
+				    NETDEV_XDP_ACT_REDIRECT |
+				    NETDEV_XDP_ACT_NDO_XMIT;
 
 	dev->vlan_features |= features;
 	netif_set_tso_max_segs(dev, MVPP2_MAX_TSO_SEGS);
 
-	dev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT |
-			    NETDEV_XDP_ACT_NDO_XMIT;
-
 	dev->priv_flags |= IFF_UNICAST_FLT;
 
 	/* MTU range: 68 - 9704 */

Signature · 1 line

2.39.2