Skip to contentSkip to conversation

bnxt_en: XDP redirect fixes

PATCH Applied Michael Chan 2 participants 5 messages Apr 2, 2022 – Apr 4, 2022

Subsystems (5)
Networking Network drivers BPF XDP +1 more subsystems
lore
Share & tools
lore

Michael Chan

author maintainer
#0 lore raw

cover bnxt_en: XDP redirect fixes

This series includes 3 fixes related to the XDP redirect code path in the driver. The first one adds locking when the number of TX XDP rings is less than the number of CPUs. The second one adjusts the maximum MTU that can support XDP with enough tail room in the buffer. The 3rd one fixes a race condition between TX ring shutdown and the XDP redirect path.

Andy Gospodarek (1):
bnxt_en: reserve space inside receive page for skb_shared_info

Pavan Chebbi (1):
bnxt_en: Synchronize tx when xdp redirects happen on same ring

Ray Jui (1):
bnxt_en: Prevent XDP redirect from running when stopping TX queue

 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |  7 +++++++
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     |  5 ++++-
 drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 14 ++++++++++++--
 drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h |  2 ++
 4 files changed, 25 insertions(+), 3 deletions(-)
Signature · 1 line
2.18.1
author maintainer
#1 lore raw

PATCH 1/3 bnxt_en: Synchronize tx when xdp redirects happen on same ring

From: Pavan Chebbi <[email protected]>

If there are more CPUs than the number of TX XDP rings, multiple XDP redirects can select the same TX ring based on the CPU on which XDP redirect is called. Add locking when needed and use static key to decide whether to take the lock.

Fixes f18c2b77b2e4 ("bnxt_en: optimized XDP_REDIRECT support")
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     | 7 +++++++
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     | 2 ++
 drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 8 ++++++++
 drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h | 2 ++
 4 files changed, 19 insertions(+)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +7 -0
index 1c28495875cf..874fad0a5cf8 100644
@@ -3253,6 +3253,7 @@ static int bnxt_alloc_tx_rings(struct bnxt *bp)
 		}
 		qidx = bp->tc_to_qidx[j];
 		ring->queue_id = bp->q_info[qidx].queue_id;
+		spin_lock_init(&txr->xdp_tx_lock);
 		if (i < bp->tx_nr_rings_xdp)
 			continue;
 		if (i % bp->tx_nr_rings_per_tc == (bp->tx_nr_rings_per_tc - 1))
@@ -10338,6 +10339,12 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)
 	if (irq_re_init)
 		udp_tunnel_nic_reset_ntf(bp->dev);
 
+	if (bp->tx_nr_rings_xdp < num_possible_cpus()) {
+		if (!static_key_enabled(&bnxt_xdp_locking_key))
+			static_branch_enable(&bnxt_xdp_locking_key);
+	} else if (static_key_enabled(&bnxt_xdp_locking_key)) {
+		static_branch_disable(&bnxt_xdp_locking_key);
+	}
 	set_bit(BNXT_STATE_OPEN, &bp->state);
 	bnxt_enable_int(bp);
 	/* Enable TX queues */
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -0
index 61aa3e8c5952..b4d3d051463b 100644
@@ -800,6 +800,8 @@ struct bnxt_tx_ring_info {
 	u32			dev_state;
 
 	struct bnxt_ring_struct	tx_ring_struct;
+	/* Synchronize simultaneous xdp_xmit on same ring */
+	spinlock_t		xdp_tx_lock;
 };
 
 #define BNXT_LEGACY_COAL_CMPL_PARAMS					\
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +8 -0
index 52fad0fdeacf..c0541ff00ac8 100644
@@ -20,6 +20,8 @@
 #include "bnxt.h"
 #include "bnxt_xdp.h"
 
+DEFINE_STATIC_KEY_FALSE(bnxt_xdp_locking_key);
+
 struct bnxt_sw_tx_bd *bnxt_xmit_bd(struct bnxt *bp,
 				   struct bnxt_tx_ring_info *txr,
 				   dma_addr_t mapping, u32 len)
@@ -227,6 +229,9 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,
 	ring = smp_processor_id() % bp->tx_nr_rings_xdp;
 	txr = &bp->tx_ring[ring];
 
+	if (static_branch_unlikely(&bnxt_xdp_locking_key))
+		spin_lock(&txr->xdp_tx_lock);
+
 	for (i = 0; i < num_frames; i++) {
 		struct xdp_frame *xdp = frames[i];
 
@@ -250,6 +255,9 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,
 		bnxt_db_write(bp, &txr->tx_db, txr->tx_prod);
 	}
 
+	if (static_branch_unlikely(&bnxt_xdp_locking_key))
+		spin_unlock(&txr->xdp_tx_lock);
+
 	return nxmit;
 }
 
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h +2 -0
index 0df40c3beb05..067bb5e821f5 100644
@@ -10,6 +10,8 @@
 #ifndef BNXT_XDP_H
 #define BNXT_XDP_H
 
+DECLARE_STATIC_KEY_FALSE(bnxt_xdp_locking_key);
+
 struct bnxt_sw_tx_bd *bnxt_xmit_bd(struct bnxt *bp,
 				   struct bnxt_tx_ring_info *txr,
 				   dma_addr_t mapping, u32 len);
Signature · 1 line
2.18.1
author maintainer
#2 lore raw

PATCH 2/3 bnxt_en: reserve space inside receive page for skb_shared_info

In reply to Michael Chan

From: Andy Gospodarek <[email protected]>

Insufficient space was being reserved in the page used for packet reception, so the interface MTU could be set too large to still have room for the contents of the packet when doing XDP redirect. This resulted in the following message when redirecting a packet between 3520 and 3822 bytes with an MTU of 3822:

[311815.561880] XDP_WARN: xdp_update_frame_from_buff(line:200): Driver BUG: missing reserved tailroom
Fixes f18c2b77b2e4 ("bnxt_en: optimized XDP_REDIRECT support")
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.h +2 -1
index b4d3d051463b..98453a78cbd0 100644
@@ -593,7 +593,8 @@ struct nqe_cn {
 #define BNXT_MAX_MTU		9500
 #define BNXT_MAX_PAGE_MODE_MTU	\
 	((unsigned int)PAGE_SIZE - VLAN_ETH_HLEN - NET_IP_ALIGN -	\
-	 XDP_PACKET_HEADROOM)
+	 XDP_PACKET_HEADROOM - \
+	 SKB_DATA_ALIGN((unsigned int)sizeof(struct skb_shared_info)))
 
 #define BNXT_MIN_PKT_SIZE	52
 
Signature · 1 line
2.18.1
author maintainer
#3 lore raw

PATCH 3/3 bnxt_en: Prevent XDP redirect from running when stopping TX queue

In reply to Michael Chan

From: Ray Jui <[email protected]>

Add checks in the XDP redirect callback to prevent XDP from running when the TX ring is undergoing shutdown.

Also remove redundant checks in the XDP redirect callback to validate the txr and the flag that indicates the ring supports XDP. The modulo arithmetic on 'tx_nr_rings_xdp' already guarantees the derived TX ring is an XDP ring. txr is also guaranteed to be valid after checking BNXT_STATE_OPEN and within RCU grace period.

Fixes f18c2b77b2e4 ("bnxt_en: optimized XDP_REDIRECT support")
---
 drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +4 -2
index c0541ff00ac8..03b1d6c04504 100644
@@ -229,14 +229,16 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,
 	ring = smp_processor_id() % bp->tx_nr_rings_xdp;
 	txr = &bp->tx_ring[ring];
 
+	if (READ_ONCE(txr->dev_state) == BNXT_DEV_STATE_CLOSING)
+		return -EINVAL;
+
 	if (static_branch_unlikely(&bnxt_xdp_locking_key))
 		spin_lock(&txr->xdp_tx_lock);
 
 	for (i = 0; i < num_frames; i++) {
 		struct xdp_frame *xdp = frames[i];
 
-		if (!txr || !bnxt_tx_avail(bp, txr) ||
-		    !(bp->bnapi[ring]->flags & BNXT_NAPI_FLAG_XDP))
+		if (!bnxt_tx_avail(bp, txr))
 			break;
 
 		mapping = dma_map_single(&pdev->dev, xdp->data, xdp->len,
Signature · 1 line
2.18.1
#4 lore raw
In reply to Michael Chan
This series was applied to netdev/net.git (master) by David S. Miller <[email protected]>:

Hello:

This series was applied to netdev/net.git (master) by David S. Miller <[email protected]>:

Quoted 10 lines · On Fri, 1 Apr 2022 20:21:09 -0400 you wrote:

This series includes 3 fixes related to the XDP redirect code path in the driver. The first one adds locking when the number of TX XDP rings is less than the number of CPUs. The second one adjusts the maximum MTU that can support XDP with enough tail room in the buffer. The 3rd one fixes a race condition between TX ring shutdown and the XDP redirect path.

Andy Gospodarek (1):
bnxt_en: reserve space inside receive page for skb_shared_info

[...]

Here is the summary with links:
- [net,1/3] bnxt_en: Synchronize tx when xdp redirects happen on same ring

    https://git.kernel.org/netdev/net/c/4f81def272de
  - [net,2/3] bnxt_en: reserve space inside receive page for skb_shared_info
    https://git.kernel.org/netdev/net/c/facc173cf700
  - [net,3/3] bnxt_en: Prevent XDP redirect from running when stopping TX queue
    https://git.kernel.org/netdev/net/c/27d4073f8d9a

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html