AF_XDP patches for zero-copy in igc driver
Subsystems (4)
v2 cover AF_XDP patches for zero-copy in igc driver
From: Jeff Evanson <[email protected]>
Patch 1/2
In igc_xdp_xmit_zc, the local variable ntu is initialized from ring->next_to_use without holding the __netif_tx_lock. If another thread already held the lock, the ntu value is potentially incorrect. Various bad behaviors were observed ranging from transmit timeouts to an outright hard-lock of the host. The behavior is corrected by simply initializing ntu while __netif_tx_lock is held.
Patch 2/2
In igc_xsk_wakeup, only the q_vector interrupt for the passed queue_id was being triggered. This worked fine so long as both the tx and rx queues shared the same interrupt. If the tx and rx queues use separate interrupts, only the rx interrupt would be triggered, regardless of whether flags contained XDP_WAKEUP_TX. This patch changes the behavior so that q_vectors (rxq_vector and txq_vector) are looked up from the referenced tx and/or rx queues instead of from the adapter q_vector array. If only XDP_WAKEUP_RX is set, rxq_vector is triggered. If only XDP_WAKEUP_TX is set txq_vector is triggered. If both are set, rxq_vector is triggered and txq_vector is triggered only if it is not equal to rxq_vector.
The bad behavior here is apparent when packets are queued up to an AF_XDP socket and then sendmsg is called on the socket. The packets would not be sent until the q_vector was triggered via some other mechanism (IE igc_xmit_frame).
Jeff Evanson (2):
igc: Fix race in igc_xdp_xmit_zc
igc: Trigger proper interrupts in igc_xsk_wakeup
drivers/net/ethernet/intel/igc/igc_main.c | 44 +++++++++++++++++------ 1 file changed, 34 insertions(+), 10 deletions(-)
Signature · 1 line
2.17.1
PATCH v2 1/2 igc: Fix race in igc_xdp_xmit_zc
From: Jeff Evanson <[email protected]>
In igc_xdp_xmit_zc, initialize next_to_use while holding the netif_tx_lock to prevent racing with other users of the tx ring.
--- drivers/net/ethernet/intel/igc/igc_main.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/igc/igc_main.c +3 -1
@@ -2598,7 +2598,7 @@ static void igc_xdp_xmit_zc(struct igc_ring *ring)struct netdev_queue *nq =txring_txq (ring);union igc_adv_tx_desc *tx_desc =NULL ;int cpu =smp_processor_id (); -u16 ntu = ring->next_to_use; +u16 ntu;struct xdp_desc xdp_desc;u16 budget;
@@ -2607,6 +2607,8 @@ static void igc_xdp_xmit_zc(struct igc_ring *ring)__netif_tx_lock (nq, cpu); + ntu = ring->next_to_use; + budget =igc_desc_unused (ring);while (xsk_tx_peek_desc (pool, &xdp_desc) && budget--) {
Signature · 1 line
2.17.1
PATCH v2 2/2 igc: Trigger proper interrupts in igc_xsk_wakeup
In reply to [email protected]From: Jeff Evanson <[email protected]>
In igc_xsk_wakeup, trigger the proper interrupt based on whether flags contains XDP_WAKEUP_RX and/or XDP_WAKEUP_TX.
Consider a scenario where the transmit queue interrupt is mapped to a different irq from the receive queue. If XDP_WAKEUP_TX is set in the flags argument, the interrupt for transmit queue must be triggered, otherwise the transmit queue's napi_struct will never be scheduled.
In the case where both XDP_WAKEUP_TX and XDP_WAKEUP_RX are both set, the receive interrupt should always be triggered, but the transmit interrupt should only be triggered if its q_vector differs from the receive queue's interrupt.
--- drivers/net/ethernet/intel/igc/igc_main.c | 40 ++++++++++++++++++----- 1 file changed, 31 insertions(+), 9 deletions(-)
drivers/net/ethernet/intel/igc/igc_main.c +31 -9
@@ -6072,8 +6072,8 @@ static void igc_trigger_rxtxq_interrupt(struct igc_adapter *adapter,int igc_xsk_wakeup (struct net_device *dev,u32 queue_id,u32 flags) { +struct igc_q_vector *txq_vector =NULL , *rxq_vector =NULL ;struct igc_adapter *adapter =netdev_priv (dev); -struct igc_q_vector *q_vector;struct igc_ring *ring;if (test_bit (__IGC_DOWN, &adapter->state))
@@ -6082,17 +6082,39 @@ int igc_xsk_wakeup(struct net_device *dev, u32 queue_id, u32 flags)if (!igc_xdp_is_enabled (adapter))return -ENXIO ; -if (queue_id >= adapter->num_rx_queues) -return -EINVAL ; +if (flags &XDP_WAKEUP_RX ) { +if (queue_id >= adapter->num_rx_queues) +return -EINVAL ; - ring = adapter->rx_ring[queue_id]; + ring = adapter->rx_ring[queue_id]; +if (!ring->xsk_pool) +return -ENXIO ; -if (!ring->xsk_pool) -return -ENXIO ; + rxq_vector = ring->q_vector; + } + +if (flags &XDP_WAKEUP_TX ) { +if (queue_id >= adapter->num_tx_queues) +return -EINVAL ; + + ring = adapter->tx_ring[queue_id]; +if (!ring->xsk_pool) +return -ENXIO ; + + txq_vector = ring->q_vector; + } + +if (rxq_vector !=NULL && + !napi_if_scheduled_mark_missed (&rxq_vector->napi)) +igc_trigger_rxtxq_interrupt (adapter, rxq_vector); - q_vector = adapter->q_vector[queue_id]; -if (!napi_if_scheduled_mark_missed (&q_vector->napi)) -igc_trigger_rxtxq_interrupt (adapter, q_vector); +/* only trigger tx interrupt if the receive interrupt was not +* triggered or if its irq differs from the receive queue's irq +*/ +if (txq_vector !=NULL && + txq_vector != rxq_vector && + !napi_if_scheduled_mark_missed (&txq_vector->napi)) +igc_trigger_rxtxq_interrupt (adapter, txq_vector);return 0 ; }
Signature · 1 line
2.17.1
Tony Nguyen
maintainerRe: [PATCH v2 2/2] igc: Trigger proper interrupts in igc_xsk_wakeup
Quoted 10 lines · jeff.evanson at gmail.com
From: Jeff Evanson <[email protected]>
In
igc_xsk_wakeup, trigger the proper interrupt based on whether flags containsXDP_WAKEUP_RXand/orXDP_WAKEUP_TX.Consider a scenario where the transmit queue interrupt is mapped to a different irq from the receive queue. If
XDP_WAKEUP_TXis set in the flags argument, the interrupt for transmit queue must be triggered, otherwise the transmit queue'snapi_structwill never be scheduled.
Quoted 6 lines
In the case where both
XDP_WAKEUP_TXandXDP_WAKEUP_RXare both set, the receive interrupt should always be triggered, but the transmit interrupt should only be triggered if itsq_vectordiffers from the receive queue's interrupt.Fixes fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy")
You're missing your sign-off; same for patch 1. Also, there's a handful of issues being reported by checkpatch [1]
Thanks,
Tony
[1]
https://patchwork.kernel.org/project/netdevbpf/patch/20220420012635.13733-3-jeff.evanson at qsc.com/