ath11k/ath12k: implement TX flow control
cover ath11k/ath12k: implement TX flow control
This series implements custom wake_tx_queue operations for ath11k and ath12k drivers to prevent hardware ring overflow issues under heavy traffic.
Without proper flow control, both drivers experience -ENOMEM errors ("failed to transmit frame -12") when the hardware TCL ring fills up. Additionally, ath12k can hang under sustained high throughput. These issues are more commonly observed in VMs with PCIe passthrough but also occur on bare metal systems.
The implementation follows the pattern used in the iwlwifi driver, checking hardware ring space before dequeuing packets from mac80211. Per-packet locking serializes ring access while allowing TX completions to run, preventing deadlocks.
Testing shows stable operation with eliminated -ENOMEM errors, no hangs, and improved throughput under heavy traffic conditions.
Jose Ignacio Tornos Martinez (2):
wifi: ath11k: implement custom wake_tx_queue with flow control
wifi: ath12k: implement custom wake_tx_queue with flow control
drivers/net/wireless/ath/ath11k/dp.c | 1 + drivers/net/wireless/ath/ath11k/dp.h | 2 + drivers/net/wireless/ath/ath11k/mac.c | 49 ++++++++++++++++++++- drivers/net/wireless/ath/ath12k/dp.c | 1 + drivers/net/wireless/ath/ath12k/dp.h | 2 + drivers/net/wireless/ath/ath12k/hal.c | 1 + drivers/net/wireless/ath/ath12k/wifi7/hw.c | 50 +++++++++++++++++++++- 7 files changed, 104 insertions(+), 2 deletions(-)
Signature · 1 line
2.54.0
PATCH 1/2 wifi: ath11k: implement custom wake_tx_queue with flow control
Under heavy traffic, ath11k experiences frequent -ENOMEM errors ("failed to transmit frame -12") when the hardware TCL ring fills up. This issue is more commonly observed in VMs with PCIe passthrough but also occurs on bare metal systems. It is particularly problematic on devices with a single shared TCL ring where all traffic classes compete for the same 512 descriptor slots.
Implement a custom wake_tx_queue operation that:
1. Checks hardware ring space before dequeuing packets from mac80211
2. Uses per-packet locking to serialize ring access and prevent races
3. Syncs with hardware state to get accurate free slot count
4. Returns early during firmware crash in the same way as other tx paths
This approach follows the pattern used in the iwlwifi driver, adapted for ath11k's hardware ring architecture.
This eliminates -ENOMEM errors and improves throughput by optimizing resource usage and preventing unnecessary packet drops.
--- drivers/net/wireless/ath/ath11k/dp.c | 1 + drivers/net/wireless/ath/ath11k/dp.h | 2 ++ drivers/net/wireless/ath/ath11k/mac.c | 49 ++++++++++++++++++++++++++- 3 files changed, 51 insertions(+), 1 deletion(-)
drivers/net/wireless/ath/ath11k/dp.c +1 -0
@@ -1087,6 +1087,7 @@ int ath11k_dp_alloc(struct ath11k_base *ab)for (i =0 ; i < ab->hw_params.hal_params->num_tx_rings; i++) {idr_init (&dp->tx_ring[i].txbuf_idr);spin_lock_init (&dp->tx_ring[i].tx_idr_lock); +spin_lock_init (&dp->tx_ring[i].wake_tx_lock); dp->tx_ring[i].tcl_data_ring_id = i; dp->tx_ring[i].tx_status_head =0 ;
drivers/net/wireless/ath/ath11k/dp.h +2 -0
@@ -87,6 +87,8 @@ struct dp_tx_ring {struct idr txbuf_idr;/* Protects txbuf_idr and num_pending */ spinlock_t tx_idr_lock; +/* Serializes wake_tx_queue operations for this ring */ +spinlock_t wake_tx_lock;struct hal_wbm_release_ring *tx_status;int tx_status_head;int tx_status_tail;
drivers/net/wireless/ath/ath11k/mac.c +48 -1
57 lines not on this page · raw message
Signature · 1 line
2.54.0
PATCH 1/2 wifi: ath11k: implement custom wake_tx_queue with flow control
Hi Jose,
In wake_tx_queue:ring_id = txq->ac % ar->ab->hw_params.hal_params->num_tx_rings;
In ath11k_dp_tx which is called by ath11k_mac_op_tx:ring_selector = ab->hw_params.hw_ops->get_ring_selector(skb); ti.ring_id = ring_selector % num_tx_rings;
Are you sure ring_id will be the same?
Also mgmt frames use a different path.
Best regards,
Zhi-Jun
Re: [PATCH 1/2] wifi: ath11k: implement custom wake_tx_queue with flow control
Hi Zhi-Jun,
Thank you for the review.
Quoted 8 lines
In
wake_tx_queue:ring_id=txq->ac%ar->ab->hw_params.hal_params->num_tx_rings;In
ath11k_dp_txwhich is called byath11k_mac_op_tx:ring_selector=ab->hw_params.hw_ops->get_ring_selector(skb); ti.ring_id =ring_selector%num_tx_rings;Are you sure
ring_idwill be the same?
You're right — txq->ac % num_tx_rings doesn't match get_ring_selector(skb) % num_tx_rings on all platforms.
In v1 the ring identification was only correct for platforms where the ring selector happens to coincide with the AC, but not for platforms.
I will send a v2 trying to solve this globally for all the platforms.
Quoted 1 line
Also mgmt frames use a different path.
Correct — management frames go through WMI, not the TCL data rings, so they are not affected. The flow control applies to data frames, which are the ones causing the problem since they use different TCL rings depending on the platform's ring selector.
Best regards,
Jose Ignacio
Re: [PATCH 1/2] wifi: ath11k: implement custom wake_tx_queue with flow control
Quoted 20 lines · Jose Ignacio Tornos Martinez
Hi Zhi-Jun,
Thank you for the review.
Quoted 8 lines
In
wake_tx_queue:ring_id=txq->ac%ar->ab->hw_params.hal_params->num_tx_rings;In
ath11k_dp_txwhich is called byath11k_mac_op_tx:ring_selector=ab->hw_params.hw_ops->get_ring_selector(skb); ti.ring_id =ring_selector%num_tx_rings;Are you sure
ring_idwill be the same?You're right —
txq->ac%num_tx_ringsdoesn't matchget_ring_selector(skb)%num_tx_ringson all platforms.
In v1 the ring identification was only correct for platforms where the ring selector happens to coincide with the AC, but not for platforms.I will send a v2 trying to solve this globally for all the platforms.
Quoted 9 lines
Quoted 1 line
Also mgmt frames use a different path.
Correct — management frames go through WMI, not the TCL data rings, so they are not affected. The flow control applies to data frames, which are the ones causing the problem since they use different TCL rings depending on the platform's ring selector.
Best regards,
Jose Ignacio
Hi Jose,
While you are reworking it would you mind looking at the previous attempt?
It tried to follow the behaviour in ath10k but lacks throttling mechanism.
https://lore.kernel.org/linux-wireless/[email protected]/
Best regards,
Zhi-Jun
Re: [PATCH 1/2] wifi: ath11k: implement custom wake_tx_queue with flow control
Hi Zhi-Jun,
Quoted 5 lines
While you are reworking it would you mind looking at the previous attempt?
It tried to follow the behaviour in ath10k but lacks throttling mechanism.
Thanks for the pointer.
I have looked at Tamizh's patch and the main difference with this approach is that the ring free space check is done before dequeuing, so the skb is never consumed when the ring is full. This avoids frame drops and the -ENOMEM errors we were seeing.
I think no additional push mechanism from TX completions is needed since wake_tx_queue is called frequently enough under traffic to resume dequeuing when ring space becomes available.
Best regards,
Jose Ignacio
PATCH 2/2 wifi: ath12k: implement custom wake_tx_queue with flow control
In reply to Jose Ignacio Tornos MartinezUnder heavy traffic, ath12k can hang and experiences -ENOMEM errors ("failed to transmit frame -12") when the hardware TCL ring fills up. This issue is more commonly observed in VMs with PCIe passthrough but also occurs on bare metal systems.
Implement a custom wake_tx_queue operation that:
1. Checks hardware ring space before dequeuing packets from mac80211
2. Uses per-packet locking to serialize ring access and prevent races
3. Syncs with hardware state to get accurate free slot count
4. Returns early during firmware crash in the same way as other tx paths
This approach follows the pattern used in the iwlwifi driver, adapted for ath12k's hardware ring architecture.
This prevents hangs, eliminates -ENOMEM errors, and improves throughput by optimizing resource usage and preventing unnecessary packet drops.
--- drivers/net/wireless/ath/ath12k/dp.c | 1 + drivers/net/wireless/ath/ath12k/dp.h | 2 + drivers/net/wireless/ath/ath12k/hal.c | 1 + drivers/net/wireless/ath/ath12k/wifi7/hw.c | 50 +++++++++++++++++++++- 4 files changed, 53 insertions(+), 1 deletion(-)
drivers/net/wireless/ath/ath12k/dp.c +1 -0
@@ -1539,6 +1539,7 @@ static int ath12k_dp_setup(struct ath12k_base *ab) }for (i =0 ; i < ab->hw_params->max_tx_ring; i++) { +spin_lock_init (&dp->tx_ring[i].wake_tx_lock); dp->tx_ring[i].tcl_data_ring_id = i; dp->tx_ring[i].tx_status_head =0 ;
drivers/net/wireless/ath/ath12k/dp.h +2 -0
@@ -58,6 +58,8 @@ struct dp_tx_ring {u8 tcl_data_ring_id;struct dp_srng tcl_data_ring;struct dp_srng tcl_comp_ring; +/* Serializes wake_tx_queue operations for this ring */ +spinlock_t wake_tx_lock;struct hal_wbm_completion_ring_tx *tx_status;int tx_status_head;int tx_status_tail;
drivers/net/wireless/ath/ath12k/hal.c +1 -0
@@ -390,6 +390,7 @@ int ath12k_hal_srng_src_num_free(struct ath12k_base *ab, struct hal_srng *srng,else return ((srng->ring_size - hp + tp) / srng->entry_size) -1 ; } +EXPORT_SYMBOL (ath12k_hal_srng_src_num_free);void *ath12k_hal_srng_src_next_peek (struct ath12k_base *ab,struct hal_srng *srng)
drivers/net/wireless/ath/ath12k/wifi7/hw.c +49 -1
58 lines not on this page · raw message
Signature · 1 line
2.54.0
Re: [PATCH 2/2] wifi: ath12k: implement custom wake_tx_queue with flow control
Quoted 12 lines · Jose Ignacio Tornos Martinez
Under heavy traffic, ath12k can hang and experiences
-ENOMEMerrors ("failed to transmit frame -12") when the hardware TCL ring fills up. This issue is more commonly observed in VMs with PCIe passthrough but also occurs on bare metal systems.Implement a custom
wake_tx_queueoperation that:1. Checks hardware ring space before dequeuing packets from mac80211
2. Uses per-packet locking to serialize ring access and prevent races
3. Syncs with hardware state to get accurate free slot count
4. Returns early during firmware crash in the same way as other tx paths
Quoted 7 lines
This approach follows the pattern used in the iwlwifi driver, adapted for ath12k's hardware ring architecture.
This prevents hangs, eliminates
-ENOMEMerrors, and improves throughput by optimizing resource usage and preventing unnecessary packet drops.Signed-off-by Jose Ignacio Tornos Martinez <[email protected]>
Just based upon the description I doubt this will be accepted. The ath12k datapath team is striving to have a lockless fastpath, so adding new locks is problematic. I've BCC'ed the internal developers list so that the team knows to give more constructive feedback.
/jeff
Re: [PATCH 2/2] wifi: ath12k: implement custom wake_tx_queue with flow control
Hi Jeff
Thank you for your feedback and for routing to the team.
I will try to ensure the approach fits all the ath12k datapath requirements.
Best regards,
Jose Ignacio
Re: [PATCH 2/2] wifi: ath12k: implement custom wake_tx_queue with flow control
In reply to Jose Ignacio Tornos MartinezQuoted 70 lines · Jose Ignacio Tornos Martinez
Under heavy traffic, ath12k can hang and experiences
-ENOMEMerrors ("failed to transmit frame -12") when the hardware TCL ring fills up. This issue is more commonly observed in VMs with PCIe passthrough but also occurs on bare metal systems.Implement a custom
wake_tx_queueoperation that:1. Checks hardware ring space before dequeuing packets from mac80211
2. Uses per-packet locking to serialize ring access and prevent races
3. Syncs with hardware state to get accurate free slot count
4. Returns early during firmware crash in the same way as other tx pathsThis approach follows the pattern used in the iwlwifi driver, adapted for ath12k's hardware ring architecture.
This prevents hangs, eliminates
-ENOMEMerrors, and improves throughput by optimizing resource usage and preventing unnecessary packet drops.Signed-off-by Jose Ignacio Tornos Martinez <[email protected]>--- drivers/net/wireless/ath/ath12k/dp.c | 1 + drivers/net/wireless/ath/ath12k/dp.h | 2 + drivers/net/wireless/ath/ath12k/hal.c | 1 + drivers/net/wireless/ath/ath12k/wifi7/hw.c | 50 +++++++++++++++++++++- 4 files changed, 53 insertions(+), 1 deletion(-)drivers/net/wireless/ath/ath12k/dp.c +1
-0@@ -1539,6 +1539,7 @@ static int ath12k_dp_setup(struct ath12k_base *ab) }for (i =0 ; i < ab->hw_params->max_tx_ring; i++) { +spin_lock_init (&dp->tx_ring[i].wake_tx_lock); dp->tx_ring[i].tcl_data_ring_id = i; dp->tx_ring[i].tx_status_head =0 ;drivers/net/wireless/ath/ath12k/dp.h +2
-0@@ -58,6 +58,8 @@ struct dp_tx_ring {u8 tcl_data_ring_id;struct dp_srng tcl_data_ring;struct dp_srng tcl_comp_ring; +/* Serializes wake_tx_queue operations for this ring */ +spinlock_t wake_tx_lock;struct hal_wbm_completion_ring_tx *tx_status;int tx_status_head;int tx_status_tail;drivers/net/wireless/ath/ath12k/hal.c +1
-0@@ -390,6 +390,7 @@ int ath12k_hal_srng_src_num_free(struct ath12k_base *ab, struct hal_srng *srng,else return ((srng->ring_size - hp + tp) / srng->entry_size) -1 ; } +EXPORT_SYMBOL (ath12k_hal_srng_src_num_free);void *ath12k_hal_srng_src_next_peek (struct ath12k_base *ab,struct hal_srng *srng)drivers/net/wireless/ath/ath12k/wifi7/hw.c +0
-0@@ -1100,9 +1100,57 @@ static void ath12k_wifi7_mac_op_tx(struct ieee80211_hw *hw, }
drivers/net/wireless/ath/ath12k/wifi7/hw.c static void ath12k_wifi7_mac_op_tx(struct ieee80211_hw *hw, · line 1110
Quoted 10 lines
drivers/net/wireless/ath/ath12k/wifi7/hw.c +8
-0} +static void ath12k_wifi7_mac_op_wake_tx_queue (struct ieee80211_hw *hw, +struct ieee80211_txq *txq) +{ +struct ath12k_hw *ah =ath12k_hw_to_ah (hw); +struct ieee80211_tx_control control = { + .sta = txq->sta, + }; +struct ath12k *ar = ah->radio;
This assignment is wrong and all traffic is incorrectly steered to radio[0] as ah->radio is a flexible array, not a pointer
Quoted 5 lines
drivers/net/wireless/ath/ath12k/wifi7/hw.c +5
-0+struct dp_tx_ring *tx_ring; +struct hal_srng *tcl_ring; +struct ath12k_dp *dp; +struct sk_buff *skb; +int num_free;
drivers/net/wireless/ath/ath12k/wifi7/hw.c static void ath12k_wifi7_mac_op_tx(struct ieee80211_hw *hw, · line 1125
Quoted 10 lines
drivers/net/wireless/ath/ath12k/wifi7/hw.c +10
-0+ +if (!ar) +return ; + +if (unlikely (test_bit (ATH12K_FLAG_CRASH_FLUSH , &ar->ab->dev_flags))) +return ; + + dp = ar->ab->dp; + tx_ring = &dp->tx_ring[txq->ac % dp->hw_params->max_tx_ring]; + tcl_ring = &dp->hal->srng_list[tx_ring->tcl_data_ring.ring_id];
tx_ring/tcl_ring selection should be corrected.
drivers/net/wireless/ath/ath12k/wifi7/hw.c static void ath12k_wifi7_mac_op_tx(struct ieee80211_hw *hw, · line 1128
Quoted 3 lines
drivers/net/wireless/ath/ath12k/wifi7/hw.c +3
-0+ +while (1 ) { +spin_lock_bh (&tx_ring->wake_tx_lock);
Do we need this spin_lock_bh?
drivers/net/wireless/ath/ath12k/wifi7/hw.c static void ath12k_wifi7_mac_op_tx(struct ieee80211_hw *hw, · line 1133
Quoted 5 lines
drivers/net/wireless/ath/ath12k/wifi7/hw.c +5
-0+ +spin_lock (&tcl_ring->lock); + num_free =ath12k_hal_srng_src_num_free (ar->ab, tcl_ring, true); +spin_unlock (&tcl_ring->lock); +
Do we need this check and spin_lock here? already ath12k_wifi7_dp_tx() has this lock and fetches the next entry. Can we check those return value here and break the loop?
Quoted 25 lines
drivers/net/wireless/ath/ath12k/wifi7/hw.c +18
-2+if (num_free ==0 ) { +spin_unlock_bh (&tx_ring->wake_tx_lock); +break ; + } + + skb =ieee80211_tx_dequeue (hw, txq); +if (!skb) { +spin_unlock_bh (&tx_ring->wake_tx_lock); +break ; + } + +ath12k_wifi7_mac_op_tx (hw, &control, skb); + +spin_unlock_bh (&tx_ring->wake_tx_lock); + } +} +static const struct ieee80211_ops ath12k_ops_wifi7 = { .tx = ath12k_wifi7_mac_op_tx, - .wake_tx_queue = ieee80211_handle_wake_tx_queue, + .wake_tx_queue = ath12k_wifi7_mac_op_wake_tx_queue, .start = ath12k_mac_op_start, .stop = ath12k_mac_op_stop, .reconfig_complete = ath12k_mac_op_reconfig_complete, --
Quoted 3 lines
2.54.0
--
- Tamizh.
Re: [PATCH 2/2] wifi: ath12k: implement custom wake_tx_queue with flow control
Hi Tamizh,
Thank you for the review.
Quoted 2 lines
This assignment is wrong and all traffic is incorrectly steered to radio[0] as
ah->radiois a flexible array, not a pointer
You are right, I will fix it in v2.
Quoted 1 line
tx_ring/tcl_ring selection should be corrected.
Yes, as also pointed out by Zhi-Jun You in the ath11k review, the ring selection in wake_tx_queue needs to match dp_tx in a general way. I will address this for both ath11k and ath12k in v2.
Quoted 1 line
Do we need this
spin_lock_bh?
Yes. Without wake_tx_lock, there is a TOCTOU race between checking ring space and dequeuing:
CPU A: num_free = 1 (sees 1 free slot)
CPU B: num_free = 1 (sees same slot)
CPU A: dequeue -> dp_tx (succeeds, fills last slot) CPU B: dequeue -> dp_tx (fails -> -ENOMEM, frame dropped)
The tcl_ring->lock inside dp_tx only protects the descriptor write, not the check-before-dequeue sequence.
Quoted 3 lines
Do we need this check and
spin_lockhere? alreadyath12k_wifi7_dp_tx()has this lock and fetches the next entry. Can we check those return value here and break the loop?
We need to check before dequeuing. Once ieee80211_tx_dequeue removes the skb from mac80211, a dp_tx failure means a dropped frame. Checking num_free first avoids dequeuing when the ring is full, which is the whole point of the flow control.
In addition, tcl_ring->lock is required by ath12k_hal_srng_src_num_free itself, which asserts the lock is held.
The brief lock also ensures an accurate count synchronized with dp_tx on other CPUs.
Best regards,
Jose Ignacio