wifi: mt76: mt7603: bound the frames a station keeps in the hardware
Subsystems (2)
cover wifi: mt76: mt7603: bound the frames a station keeps in the hardware
mt7603 completes a frame to mac80211 as soon as DMA has copied it, so AQL never sees the frames the hardware still holds for a station. On a TP-Link Archer XR500v (EN751221 + MT7603E) with one client, counters added to upstream mt76 showed the hardware holding about 200 frames for a single downloading client, nearly the whole 256-entry tx ring, and up to 740 under bidirectional load.
With a client that toggles PM often (here a Samsung Galaxy A56 with Bluetooth on, which enters power save several times per second under load), every PS entry loops all of those frames back to the host, and the PS queue keeps only 64 of them. The frames freed per second roughly matched the client's TCP retransmissions per second.
The series holds a station's tx queues while the hardware owes it too many TX statuses (a new MT_WCID_FLAG_TX_HOLD, honoured by the mt76 tx scheduler like the non-AQL limit), and sizes the PS queue so that what the hardware loops back fits. It depends on "wifi: mt76: mt7603: don't drop short frames looped back on PS entry" [1], without which the count leaks one frame for every BlockAck request that is looped back.
This is an RFC because I am not sure a driver-side hold is the approach you want. Questions:
- Would you rather keep the airtime of each frame accounted until its TX status, so that AQL sees what the hardware holds, instead of a count in the driver?
- The marks (hold at 96, release at 48) and the PS queue size (tx ring minus 64) were tuned with one client only.
- If a held station sees neither a TX status nor a loop-back return for a second, its count is reset. In the tests below this happened 0 to 3 times in 30 minutes, probably when the client slept for more than a second with frames still in the hardware. Is there a better signal?
- The raw re-send of the PS queue on wake-up bypasses the hold, so the count can briefly exceed the high mark (up to 134 here).
Either change alone makes things worse, which is why the cap and the PS queue size are one patch. In a 30 minute bidirectional test (iperf3 -P 4, one client, a ping from a wired host every 200 ms) that cycled through the four combinations every minute, about 7 minutes each:
upstream bigger PSQ cap cap + PSQ
frames in hw (max) 748 844 98 134 PSQ frees per second 92 213 51 0 downlink Mbit/s 12.2 15.2 10.6 17.0 seconds without downlink 45 108 111 10 longest stall (s) 10 53 16 1 retransmits per second 15.4 8.9 10.2 3.0 uplink Mbit/s 64.0 60.7 63.8 59.3
ping median/p99 (ms) 119/339 145/2915 125/482 130/292
pings lost 6.0% 13.7% 7.5% 0.13%
The stalls come in bursts of PM toggling, with up to 22-28 PS entries per second. In the worst stretch, the minute with the cap and the bigger queue had 2 seconds without downlink progress, against 20, 41 and 39 in the neighbouring minutes of the other combinations.
Two shorter runs toggled the cap alone (PS queue of 64) every 30 seconds, over 60 seconds of download and 300 of bidirectional traffic, with a 2.4 GHz wireless headset dongle nearby. In the download phases, with the cap: PSQ frees went from 59.5 to 7.3 and from 54.5 to 5.3 per second, retransmits from 54.9 to 8.2 and from 58.8 to 9.1 per second, the ping median from 38 to 23 and from 32 to 25 ms, and throughput from 56 to 61 and from 59 to 63 Mbit/s. Bidirectional retransmits went from 13.5 to 6.0 and from 13.8 to 8.3 per second.
Testing: OpenWrt with kernel 6.18.41 and the mac80211 backport of 6.18.39, openwrt/mt76 at be5ce79105 plus [1], this series, the board's local patches (EEPROM file from DT, two crash fixes, beacon stall recovery, fixes for the mt76x2e radio) and debugfs counters that are not part of the series. AP on channel 1, HT20. One device and one client
only: no MT7628/MT7688, and nothing with several busy stations. The series is against wireless-next; on openwrt/mt76, where it was built and tested, patch 1 only differs in context (mt76_wcid_primary() in the non-AQL check).
[1] https://lore.kernel.org/linux-wireless/[email protected]/
Tools: an AI coding assistant (Claude Opus 5.5 in Claude Code) wrote the instrumentation, analyzed the data, and drafted these patches and this letter; the tests ran on my device.
Cristian Papa (2):
wifi: mt76: let a driver hold a station's tx queues
wifi: mt76: mt7603: bound the frames a station keeps in the hardware
drivers/net/wireless/mediatek/mt76/mt76.h | 1 + .../wireless/mediatek/mt76/mt7603/debugfs.c | 32 ++++++++++ .../net/wireless/mediatek/mt76/mt7603/dma.c | 7 ++- .../net/wireless/mediatek/mt76/mt7603/init.c | 2 + .../net/wireless/mediatek/mt76/mt7603/mac.c | 58 +++++++++++++++++++ .../net/wireless/mediatek/mt76/mt7603/main.c | 20 +++++-- .../wireless/mediatek/mt76/mt7603/mt7603.h | 51 ++++++++++++++++ drivers/net/wireless/mediatek/mt76/tx.c | 7 +++ 8 files changed, 172 insertions(+), 6 deletions(-)
base-commit: 10cfa109c880092df32e396647b4afdca9be8350
prerequisite-patch-id: b2c7fb85fb032bec084c6e325be32ea1172af916
Signature · 1 line
2.47.3
RFC PATCH 1/2 wifi: mt76: let a driver hold a station's tx queues
Some drivers know how many frames a station has inside the hardware better than AQL does. mt7603 completes a frame to mac80211 as soon as DMA has copied it, so AQL releases its airtime right away while the hardware may keep hundreds of frames for the station.
Add MT_WCID_FLAG_TX_HOLD. While it is set, the tx scheduler leaves the station's queues alone, like it does for a station at the non-AQL limit, and stops a burst that filled the hardware. The driver clears the flag and schedules the queues again when the hardware has drained.
No functional change for drivers that do not set the flag.
--- drivers/net/wireless/mediatek/mt76/mt76.h | 1 + drivers/net/wireless/mediatek/mt76/tx.c | 7 +++++++ 2 files changed, 8 insertions(+)
drivers/net/wireless/mediatek/mt76/mt76.h +1 -0
@@ -362,6 +362,7 @@ enum mt76_wcid_flags {MT_WCID_FLAG_4ADDR ,MT_WCID_FLAG_HDR_TRANS ,MT_WCID_FLAG_TDLS_PEER , + MT_WCID_FLAG_TX_HOLD, };#define MT76_N_WCIDS 1088
drivers/net/wireless/mediatek/mt76/tx.c +7 -0
@@ -534,6 +534,10 @@ mt76_txq_send_burst(struct mt76_phy *phy, struct mt76_queue *q,if (stop ||mt76_txq_stopped (q))break ; +/* The previous frame may have filled the hardware. */ +if (test_bit (MT_WCID_FLAG_TX_HOLD , &wcid->flags)) +break ; + skb =mt76_txq_dequeue (phy, mtxq);if (!skb)break ;
@@ -605,6 +609,9 @@ mt76_txq_schedule_list(struct mt76_phy *phy, enum mt76_txq_id qid)if (atomic_read (&wcid->non_aql_packets) >=MT_MAX_NON_AQL_PKT )continue ; +/* The driver schedules the queues again when it lifts the hold. */ +if (test_bit (MT_WCID_FLAG_TX_HOLD , &wcid->flags)) +continue ;if (dev->queue_ops->tx_cleanup && q->queued +2 *MT_TXQ_FREE_THR >= q->ndesc) { dev->queue_ops->tx_cleanup (dev, q, false);
Signature · 1 line
2.47.3
RFC PATCH 2/2 wifi: mt76: mt7603: bound the frames a station keeps in the hardware
In reply to Cristian Papamt7603 completes a frame to mac80211 once DMA has copied it into the hardware, so AQL does not see the frames the hardware still holds for a station. With one client downloading, the hardware held about 200 frames for it, nearly the whole 256-entry tx ring, and up to about 750 under bidirectional load, when frames also wait in the PSE.
Each time the station enters power save, the PSE loops all of them back to the host. The PS queue keeps at most 64 of them and frees the oldest to make room. A client that toggles PM often, like a phone with Bluetooth coexistence, turns this into a steady loss of data frames:
with one client downloading, the frames freed per second roughly matched its TCP retransmissions per second.
Count, per station, the frames queued to the hardware that still owe a TX status: counted when queued, settled by the TX status or by the loop-back return. Hold the station's tx queues with MT_WCID_FLAG_TX_HOLD at 96 frames and release them at 48, and let the PS queue keep up to the tx ring size minus 64 frames, so that what the hardware loops back fits and is sent again on wake-up. Either change alone makes bursts of PM toggling worse: without the cap, a larger queue lets the hardware loop back hundreds of frames over and over, and with the cap, the smaller queue still frees the frames above 64.
The marks can be changed, or the cap turned off with 0, in debugfs (hw_cap_high, hw_cap_low); hw_cap shows the count per station.
Every frame counted ends in a TX status or a loop-back return. If a held station sees neither for a second, reset its count and release it rather than keep it on hold.
In a 30 minute bidirectional test that cycled through the four combinations every minute, the cap with the larger queue freed no frames from the PS queue (92 per second without either change), and the downlink stalled for 10 seconds (45), with 3 TCP retransmissions per second (15) and 0.1% of pings lost (6%).
--- .../wireless/mediatek/mt76/mt7603/debugfs.c | 32 ++++++++++ .../net/wireless/mediatek/mt76/mt7603/dma.c | 7 ++- .../net/wireless/mediatek/mt76/mt7603/init.c | 2 + .../net/wireless/mediatek/mt76/mt7603/mac.c | 58 +++++++++++++++++++ .../net/wireless/mediatek/mt76/mt7603/main.c | 20 +++++-- .../wireless/mediatek/mt76/mt7603/mt7603.h | 51 ++++++++++++++++ 6 files changed, 164 insertions(+), 6 deletions(-)
drivers/net/wireless/mediatek/mt76/mt7603/debugfs.c +32 -0
@@ -93,6 +93,34 @@ mt7603_ampdu_stat_show(struct seq_file *file, void *data)DEFINE_SHOW_ATTRIBUTE (mt7603_ampdu_stat); +static int +mt7603_hw_cap_read (struct seq_file *s,void *data) +{ +struct mt7603_dev *dev =dev_get_drvdata (s->private); +int i; + +seq_printf (s,"high %u low %u resync %u\n" , +READ_ONCE (dev->hw_cap_high),READ_ONCE (dev->hw_cap_low), +READ_ONCE (dev->hw_cap_resync)); + +rcu_read_lock (); +for (i =0 ; i <MT7603_WTBL_STA ; i++) { +struct mt76_wcid *wcid =mt76_wcid_ptr (dev, i); +struct mt7603_sta *msta; + +if (!wcid || !wcid->sta) +continue ; + + msta = container_of(wcid, struct mt7603_sta, wcid); +seq_printf (s,"wcid %d pending %d hold %d\n" , i, +atomic_read (&msta->hw_pending), +test_bit (MT_WCID_FLAG_TX_HOLD , &wcid->flags)); + } +rcu_read_unlock (); + +return 0 ; +} +void mt7603_init_debugfs (struct mt7603_dev *dev) {struct dentry *dir;
@@ -115,4 +143,8 @@ void mt7603_init_debugfs(struct mt7603_dev *dev) &dev->sensitivity_limit);debugfs_create_bool ("dynamic_sensitivity" ,0600 , dir, &dev->dynamic_sensitivity); +debugfs_create_u32 ("hw_cap_high" ,0600 , dir, &dev->hw_cap_high); +debugfs_create_u32 ("hw_cap_low" ,0600 , dir, &dev->hw_cap_low); +debugfs_create_devm_seqfile (dev->mt76.dev,"hw_cap" , dir, + mt7603_hw_cap_read); }
drivers/net/wireless/mediatek/mt76/mt7603/dma.c +6 -1
@@ -49,6 +49,11 @@ mt7603_rx_loopback_skb(struct mt7603_dev *dev, struct sk_buff *skb) priv = msta = container_of(wcid, struct mt7603_sta, wcid); +/* The frame left the hardware: its TX status will not arrive. */ +if (wcid->sta && +FIELD_GET (MT_TXD5_PID ,le32_to_cpu (txd[5 ])) !=MT_PACKET_ID_NO_ACK ) +mt7603_hw_pending_dec (dev, msta); + sta = container_of(priv, struct ieee80211_sta, drv_priv); hdr = (struct ieee80211_hdr *)&skb->data[MT_TXD_SIZE ];
@@ -97,7 +102,7 @@ mt7603_rx_loopback_skb(struct mt7603_dev *dev, struct sk_buff *skb)spin_lock_bh (&dev->ps_lock);__skb_queue_tail (&msta->psq, skb); -if (skb_queue_len (&msta->psq) >=64 ) { +if (skb_queue_len (&msta->psq) >=MT7603_PSQ_MAX ) { skb =__skb_dequeue (&msta->psq);dev_kfree_skb (skb); }
drivers/net/wireless/mediatek/mt76/mt7603/init.c +2 -0
@@ -518,6 +518,8 @@ int mt7603_register_device(struct mt7603_dev *dev) dev->slottime =9 ; dev->sensitivity_limit =28 ; dev->dynamic_sensitivity = true; + dev->hw_cap_high =MT7603_HW_CAP_HIGH ; + dev->hw_cap_low =MT7603_HW_CAP_LOW ; ret =mt7603_init_hardware (dev);if (ret)
drivers/net/wireless/mediatek/mt76/mt7603/mac.c +58 -0
82 lines not on this page · raw message
drivers/net/wireless/mediatek/mt76/mt7603/main.c +15 -5
@@ -339,6 +339,8 @@ mt7603_sta_add(struct mt76_dev *mdev, struct ieee80211_vif *vif,INIT_LIST_HEAD (&msta->wcid.poll_list);__skb_queue_head_init (&msta->psq); +atomic_set (&msta->hw_pending,0 ); + msta->hw_settled = jiffies; msta->ps = ~0 ; msta->smps = ~0 ; msta->wcid.sta =1 ;
@@ -391,14 +393,22 @@ mt7603_sta_remove(struct mt76_dev *mdev, struct ieee80211_vif *vif, }static void -mt7603_ps_tx_list (struct mt7603_dev *dev,struct sk_buff_head *list) +mt7603_ps_tx_list (struct mt7603_dev *dev,struct mt7603_sta *msta, +struct sk_buff_head *list) {struct sk_buff *skb;while ((skb =__skb_dequeue (list)) !=NULL ) {int qid =skb_get_queue_mapping (skb); - -mt76_tx_queue_skb_raw (dev, dev->mphy.q_tx[qid], skb,0 ); +bool txs =le32_get_bits (((__le32 *)skb->data)[5 ], +MT_TXD5_PID ) !=MT_PACKET_ID_NO_ACK ; + +/* Count it before the TX status can arrive. */ +if (txs) +mt7603_hw_pending_inc (dev, msta); +if (mt76_tx_queue_skb_raw (dev, dev->mphy.q_tx[qid], skb,0 ) && + txs) +mt7603_hw_pending_dec (dev, msta); } }
@@ -420,7 +430,7 @@ mt7603_sta_ps(struct mt76_dev *mdev, struct ieee80211_sta *sta, bool ps)skb_queue_splice_tail_init (&msta->psq, &list);spin_unlock_bh (&dev->ps_lock); -mt7603_ps_tx_list (dev, &list); +mt7603_ps_tx_list (dev, msta, &list); } static struct ieee80211_hdr *
@@ -529,7 +539,7 @@ mt7603_release_buffered_frames(struct ieee80211_hw *hw, !ieee80211_is_data_qos (mt7603_ps_skb_hdr (last)->frame_control); last_tid =__fls (tids); -mt7603_ps_tx_list (dev, &list); +mt7603_ps_tx_list (dev, msta, &list);if (eosp_null)ieee80211_send_eosp_nullfunc (sta, last_tid);
drivers/net/wireless/mediatek/mt76/mt7603/mt7603.h +51 -0
71 lines not on this page · raw message
Signature · 1 line
2.47.3