wifi: ath12k: isolate RXDMA page-fragment caches
v2 cover wifi: ath12k: isolate RXDMA page-fragment caches
Hi,
This series adapts Mark Ruvald Pedersen's ath11k page-fragment cache changes to ath12k. On QCN9274, sustained WAN traffic
can exhaust memory when Ethernet and Wi-Fi run on the same CPU. Long-lived RXDMA buffers share per-CPU page-fragment caches with other network users and can pin mostly unused 32-KiB backing pages.
Patch 1 makes external IRQ transitions follow the NAPI state so that crash recovery can safely quiesce RXDMA before teardown. Patch 2 gives regular and monitor RXDMA rings private caches and drains them after their buffers have been released. It also fixes alignment of empty skbs and preserves monitor status buffer capacity after alignment.
The series is based on ath.git main at:388d3c42e8aba1cfc3c6d2c52877a46d8cf740db (Linux 7.3-rc5). Runtime measurements were made on Linux 6.18.54, with Ethernet and Wi-Fi IRQs on CPU0:
Hardware: QCN9274 hw2.0 PCI
Firmware: WLAN.WBE.1.6-01243-QCAHKSWPL_SILICONZ-1
Two 45-second downloads showed peak order-3 memory growth of 54 and 68 MiB. An earlier unpatched 45-second test grew by 187 MiB, and longer unpatched downloads reached OOM.
A five-minute download at about 660 Mbps transferred 24.8 GB without OOM. Peak growth was 105 MiB, falling to 35 MiB above baseline after one minute idle.
Thanks,
Nikolai
Changes in v2:
- Include linux/bug.h before linux/page_frag_cache.h in dp.c to make WARN_ON_ONCE available on Alpha. This fixes the build error reported by the kernel test robot:
https://lore.kernel.org/oe-kbuild-all/[email protected]/
Nikolai Semendiaev (2):
wifi: ath12k: make external IRQ control idempotent
wifi: ath12k: use private page-frag caches for RXDMA rings
drivers/net/wireless/ath/ath12k/ahb.c | 4 +-- drivers/net/wireless/ath/ath12k/core.c | 3 ++ drivers/net/wireless/ath/ath12k/dp.c | 4 +++ drivers/net/wireless/ath/ath12k/dp.h | 3 ++ drivers/net/wireless/ath/ath12k/dp_mon.c | 29 +++++++++------ drivers/net/wireless/ath/ath12k/dp_rx.c | 45 +++++++++++++++++++++--- drivers/net/wireless/ath/ath12k/dp_rx.h | 2 ++ drivers/net/wireless/ath/ath12k/pci.c | 6 ++-- 8 files changed, 74 insertions(+), 22 deletions(-)
base-commit: 388d3c42e8aba1cfc3c6d2c52877a46d8cf740db
Signature · 1 line
2.43.0
PATCH v2 1/2 wifi: ath12k: make external IRQ control idempotent
The AHB external IRQ helpers change the hardware IRQ state even when NAPI is already in the requested state. PCI has a device-wide enabled bit guard, but also changes each group's IRQ state outside its NAPI state check.
Tie each group's hardware IRQ transition to its NAPI state transition. Mask IRQs before disabling NAPI and enable NAPI before unmasking IRQs. This permits an extra IRQ disable before crash-recovery RXDMA teardown without increasing the IRQ disable depth.
Based on Mark Ruvald Pedersen's ath11k IRQ lifecycle fix.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.6-01243-QCAHKSWPL_SILICONZ-1
--- drivers/net/wireless/ath/ath12k/ahb.c | 4 ++-- drivers/net/wireless/ath/ath12k/pci.c | 6 ++---- 2 files changed, 4 insertions(+), 6 deletions(-)
drivers/net/wireless/ath/ath12k/ahb.c +2 -2
@@ -182,8 +182,8 @@ static void __ath12k_ahb_ext_irq_disable(struct ath12k_base *ab)for (i =0 ; i <ATH12K_EXT_IRQ_GRP_NUM_MAX ; i++) {struct ath12k_ext_irq_grp *irq_grp = &ab->ext_irq_grp[i]; -ath12k_ahb_ext_grp_disable (irq_grp);if (irq_grp->napi_enabled) { +ath12k_ahb_ext_grp_disable (irq_grp);napi_synchronize (&irq_grp->napi);napi_disable (&irq_grp->napi); irq_grp->napi_enabled = false;
@@ -326,8 +326,8 @@ static void ath12k_ahb_ext_irq_enable(struct ath12k_base *ab)if (!irq_grp->napi_enabled) {napi_enable (&irq_grp->napi); irq_grp->napi_enabled = true; +ath12k_ahb_ext_grp_enable (irq_grp); } -ath12k_ahb_ext_grp_enable (irq_grp); } }
drivers/net/wireless/ath/ath12k/pci.c +2 -4
@@ -458,9 +458,8 @@ static void __ath12k_pci_ext_irq_disable(struct ath12k_base *ab)for (i =0 ; i <ATH12K_EXT_IRQ_GRP_NUM_MAX ; i++) {struct ath12k_ext_irq_grp *irq_grp = &ab->ext_irq_grp[i]; -ath12k_pci_ext_grp_disable (irq_grp); -if (irq_grp->napi_enabled) { +ath12k_pci_ext_grp_disable (irq_grp);napi_synchronize (&irq_grp->napi);napi_disable (&irq_grp->napi); irq_grp->napi_enabled = false;
@@ -1108,9 +1107,8 @@ void ath12k_pci_ext_irq_enable(struct ath12k_base *ab)if (!irq_grp->napi_enabled) {napi_enable (&irq_grp->napi); irq_grp->napi_enabled = true; +ath12k_pci_ext_grp_enable (irq_grp); } - -ath12k_pci_ext_grp_enable (irq_grp); }set_bit (ATH12K_FLAG_EXT_IRQ_ENABLED , &ab->dev_flags);
Signature · 1 line
2.43.0
PATCH v2 2/2 wifi: ath12k: use private page-frag caches for RXDMA rings
In reply to Nikolai SemendiaevRXDMA buffers allocated with dev_alloc_skb() share per-CPU page-fragment caches with other network users. Long-lived ring buffers can pin mostly unused 32-KiB backing pages as short-lived network allocations cycle through those caches. On QCN9274, OOM was observed during sustained WAN downloads when Ethernet and ath12k processing shared a CPU.
Give regular and monitor RXDMA rings private page-fragment caches. The existing refill SRNG locks serialize cache access. Preserve NET_SKB_PAD, allocation flags and the DMA mapping and unmapping operations.
Drain each monitor cache after releasing its IDR-owned buffers, and the regular RX cache after releasing its cookie-conversion descriptors. Disable external IRQ/NAPI processing before non-reset crash teardown; the reset worker already does so.
Use skb_reserve() for DMA alignment because skb_pull() cannot advance an empty skb. Allocate extra alignment space for monitor status buffers to preserve at least RX_MON_STATUS_BUF_SIZE of DMA capacity.
Based on Mark Ruvald Pedersen's ath11k per-ring page-fragment isolation.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.6-01243-QCAHKSWPL_SILICONZ-1
--- drivers/net/wireless/ath/ath12k/core.c | 3 ++ drivers/net/wireless/ath/ath12k/dp.c | 4 +++ drivers/net/wireless/ath/ath12k/dp.h | 3 ++ drivers/net/wireless/ath/ath12k/dp_mon.c | 29 +++++++++------ drivers/net/wireless/ath/ath12k/dp_rx.c | 45 +++++++++++++++++++++--- drivers/net/wireless/ath/ath12k/dp_rx.h | 2 ++ 6 files changed, 70 insertions(+), 16 deletions(-)
drivers/net/wireless/ath/ath12k/core.c +3 -0
@@ -1390,6 +1390,9 @@ static int ath12k_core_reconfigure_on_crash(struct ath12k_base *ab) {int ret, total_vdev; +if (!ab->is_reset) +ath12k_hif_irq_disable (ab); +mutex_lock (&ab->core_lock);ath12k_link_sta_rhash_tbl_destroy (ab);ath12k_thermal_unregister (ab);
drivers/net/wireless/ath/ath12k/dp.c +4 -0
@@ -4,6 +4,8 @@* Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries. */ +#include <linux/bug.h> +#include <linux/page_frag_cache.h> #include "core.h" #include "dp_tx.h" #include "hif.h"
@@ -1020,6 +1022,8 @@ static void ath12k_dp_cc_cleanup(struct ath12k_base *ab)spin_unlock_bh (&dp->rx_desc_lock); +page_frag_cache_drain (&dp->rx_refill_buf_ring.frag_cache); +/* TX Descriptor cleanup */ for (i =0 ; i <ATH12K_HW_MAX_QUEUES ; i++) {spin_lock_bh (&dp->tx_desc_lock[i]);
drivers/net/wireless/ath/ath12k/dp.h +3 -0
@@ -11,6 +11,7 @@#include "hw.h" #include "dp_htt.h" #include "dp_cmn.h" +#include <linux/mm_types.h> #include <linux/rhashtable.h> #define MAX_RXDMA_PER_PDEV 2
@@ -51,6 +52,7 @@ struct dp_srng {struct dp_rxdma_mon_ring {struct dp_srng refill_buf_ring; +struct page_frag_cache frag_cache;struct idr bufs_idr;/* Protects bufs_idr */ spinlock_t idr_lock;
@@ -59,6 +61,7 @@ struct dp_rxdma_mon_ring {struct dp_rxdma_ring {struct dp_srng refill_buf_ring; +struct page_frag_cache frag_cache;int bufs_max; };
drivers/net/wireless/ath/ath12k/dp_mon.c +18 -11
@@ -5,6 +5,7 @@*/ #include "dp_mon.h" +#include "dp_rx.h" #include "debug.h" #include "dp_tx.h" #include "peer.h"
@@ -44,15 +45,17 @@ struct sk_buffstruct sk_buff *skb;dma_addr_t paddr; - skb =dev_alloc_skb (RX_MON_STATUS_BUF_SIZE ); + skb =ath12k_dp_rx_alloc_skb (&rx_ring->frag_cache, +RX_MON_STATUS_BUF_SIZE + +RX_MON_STATUS_BUF_ALIGN );if (!skb) goto fail_alloc_skb;if (!IS_ALIGNED ((unsigned long )skb->data,RX_MON_STATUS_BUF_ALIGN )) { -skb_pull (skb,PTR_ALIGN (skb->data,RX_MON_STATUS_BUF_ALIGN ) - - skb->data); +skb_reserve (skb,PTR_ALIGN (skb->data,RX_MON_STATUS_BUF_ALIGN ) - + skb->data); } paddr = dma_map_single(ab->dev, skb->data,
@@ -598,14 +601,16 @@ int ath12k_dp_mon_buf_replenish(struct ath12k_base *ab,ath12k_hal_srng_access_begin (ab, srng);while (req_entries >0 ) { - skb =dev_alloc_skb (DP_RX_BUFFER_SIZE +DP_RX_BUFFER_ALIGN_SIZE ); + skb =ath12k_dp_rx_alloc_skb (&buf_ring->frag_cache, +DP_RX_BUFFER_SIZE + +DP_RX_BUFFER_ALIGN_SIZE );if (unlikely (!skb)) goto fail_alloc_skb;if (!IS_ALIGNED ((unsigned long )skb->data,DP_RX_BUFFER_ALIGN_SIZE )) { -skb_pull (skb, -PTR_ALIGN (skb->data,DP_RX_BUFFER_ALIGN_SIZE ) - - skb->data); +skb_reserve (skb, +PTR_ALIGN (skb->data,DP_RX_BUFFER_ALIGN_SIZE ) - + skb->data); } paddr = dma_map_single(ab->dev, skb->data,
@@ -687,15 +692,17 @@ int ath12k_dp_mon_status_bufs_replenish(struct ath12k_base *ab, num_remain = req_entries;while (num_remain >0 ) { - skb =dev_alloc_skb (RX_MON_STATUS_BUF_SIZE ); + skb =ath12k_dp_rx_alloc_skb (&rx_ring->frag_cache, +RX_MON_STATUS_BUF_SIZE + +RX_MON_STATUS_BUF_ALIGN );if (!skb)break ;if (!IS_ALIGNED ((unsigned long )skb->data,RX_MON_STATUS_BUF_ALIGN )) { -skb_pull (skb, -PTR_ALIGN (skb->data,RX_MON_STATUS_BUF_ALIGN ) - - skb->data); +skb_reserve (skb, +PTR_ALIGN (skb->data,RX_MON_STATUS_BUF_ALIGN ) - + skb->data); } paddr = dma_map_single(ab->dev, skb->data,
drivers/net/wireless/ath/ath12k/dp_rx.c +40 -5
77 lines not on this page · raw message
drivers/net/wireless/ath/ath12k/dp_rx.h +2 -0
@@ -232,6 +232,8 @@ int ath12k_dp_rx_bufs_replenish(struct ath12k_dp *dp,struct dp_rxdma_ring *rx_ring, struct list_head *used_list, int req_entries); +struct sk_buff *ath12k_dp_rx_alloc_skb (struct page_frag_cache *cache, +unsigned int len);int ath12k_dp_rx_pdev_mon_attach (struct ath12k *ar);int ath12k_dp_rx_peer_frag_setup (struct ath12k *ar,const u8 *peer_mac,int vdev_id);
Signature · 1 line
2.43.0
Re: [PATCH v2 1/2] wifi: ath12k: make external IRQ control idempotent
In reply to Nikolai SemendiaevQuoted 10 lines · Nikolai Semendiaev
The AHB external IRQ helpers change the hardware IRQ state even when NAPI is already in the requested state. PCI has a device-wide enabled bit guard, but also changes each group's IRQ state outside its NAPI state check.
Tie each group's hardware IRQ transition to its NAPI state transition. Mask IRQs before disabling NAPI and enable NAPI before unmasking IRQs. This permits an extra IRQ disable before crash-recovery RXDMA teardown without increasing the IRQ disable depth.
first of all, did you see any issues with current code?
second, the idempotency predicate is bound to the per-group irq_grp->napi_enabled, which answers "is NAPI poll enabled?", not "is the IRQ line enabled?", so semantically this is not proper. PCI already has a semantically correct and atomic state bit for exactly this, ATH12K_FLAG_EXT_IRQ_ENABLED, why not just use it? The current approach leaves two sources of truth — per-group napi_enabled and device-level EXT_IRQ_ENABLED — that must stay perfectly consistent, which is fragile. As for AHB, can we also use ATH12K_FLAG_EXT_IRQ_ENABLED?
Re: [PATCH v2 1/2] wifi: ath12k: make external IRQ control idempotent
Quoted 1 line
first of all, did you see any issues with current code?
I didn't reproduce a separate IRQ issue. What I saw on QCN9274 was memory growing until OOM. I added this patch because patch 2 calls ath12k_hif_irq_disable() again before freeing the RXDMA buffers.
Quoted 2 lines
PCI already has a semantically correct and atomic state bit for exactly this,
ATH12K_FLAG_EXT_IRQ_ENABLED, why not just use it?
You're right, napi_enabled is the wrong guard here. PCI already handles repeated disables, so I'll drop those changes.
Quoted 1 line
As for AHB, can we also use
ATH12K_FLAG_EXT_IRQ_ENABLED?
Yes, I'll use that flag for AHB in v3.
Re: [PATCH v2 2/2] wifi: ath12k: use private page-frag caches for RXDMA rings
In reply to Nikolai SemendiaevQuoted 19 lines · Nikolai Semendiaev
RXDMA buffers allocated with
dev_alloc_skb()share per-CPU page-fragment caches with other network users. Long-lived ring buffers can pin mostly unused 32-KiB backing pages as short-lived network allocations cycle through those caches. On QCN9274, OOM was observed during sustained WAN downloads when Ethernet and ath12k processing shared a CPU.Give regular and monitor RXDMA rings private page-fragment caches. The existing refill SRNG locks serialize cache access. Preserve
NET_SKB_PAD, allocation flags and the DMA mapping and unmapping operations.Drain each monitor cache after releasing its IDR-owned buffers, and the regular RX cache after releasing its cookie-conversion descriptors. Disable external IRQ/NAPI processing before non-reset crash teardown; the reset worker already does so.
Use
skb_reserve()for DMA alignment becauseskb_pull()cannot advance an empty skb. Allocate extra alignment space for monitor status buffers to preserve at leastRX_MON_STATUS_BUF_SIZEof DMA capacity.
Quoted 6 lines
Based on Mark Ruvald Pedersen's ath11k per-ring page-fragment isolation.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.6-01243-QCAHKSWPL_SILICONZ-1
Signed-off-by Nikolai Semendiaev <[email protected]>
This conflicts with a different proposal:
https://lore.kernel.org/all/[email protected]/