mac80211: Switch to a virtual time-based airtime scheduler
PATCH v9 1/1 mac80211: Switch to a virtual time-based airtime scheduler
This switches the airtime scheduler in mac80211 to use a virtual time-based scheduler instead of the round-robin scheduler used before. This has a couple of advantages:
- No need to sync up the round-robin scheduler in firmware/hardware with the round-robin airtime scheduler.
- If several stations are eligible for transmission we can schedule both of them; no need to hard-block the scheduling rotation until the head of the queue has used up its quantum.
- The check of whether a station is eligible for transmission becomes simpler (in ieee80211_txq_may_transmit()).
The drawback is that scheduling becomes slightly more expensive, as we need to maintain an rbtree of TXQs sorted by virtual time. This means that ieee80211_register_airtime() becomes O(logN) in the number of currently scheduled TXQs because it can change the order of the scheduled stations. We mitigate this overhead by only resorting when a station changes position in the tree, and hopefully N rarely grows too big (it's only TXQs currently backlogged, not all associated stations), so it shouldn't be too big of an issue.
To prevent divisions in the fast path, we maintain both station sums and pre-computed reciprocals of the sums. This turns the fast-path operation into a multiplication, with divisions only happening as the number of active stations change (to re-compute the current sum of all active station weights). To prevent this re-computation of the reciprocal from happening too frequently, we use a time-based notion of station activity, instead of updating the weight every time a station gets scheduled or de-scheduled. As queues can oscillate between empty and occupied quite frequently, this can significantly cut down on the number of re-computations. It also has the added benefit of making the station airtime calculation independent on whether the queue happened to have drained at the time an airtime value was accounted.
---
Changelog:
v9:
Rebase to newest mac80211-next.
Reflow commit message to 72 chars and remove some long lines in the code.
Refactor the sta parameter parsing to a separate sta_apply_airtime_params()
v8:
Fix locking issue pointed out by kbot
v7:
Rebase on latest mac80211-next, fix a kbot compile warning.
Add back a hard limit on how far a station call fall behind in addition to the activity-based catchup, to avoid the case where a station manages to get a single packet scheduled every 100ms and thus stays "active" indefinitely.
v6:
Rebase on latest mac80211-next.
Fix issue with scheduling hanging because the schedule position was not cleared when starting a new scheduling round.
Switch the reciprocal calculation to use u32 (split 19/13) for per-station weights and a u64 only for the weight sum (to cut down on the number of 64-bit operations performed)
Introduce the notion of time-based station activity when calculating weight sums. This also gets rid of the need for a "grace time" when catching up stations, since we now have a direct notion of when a station has been inactive for a while.
Add a debugfs file for the netdev so users can inspect the airtime usage of the vif TXQ.
v5:
Rebase on latest mac80211-next.
Squash everything into a single patch again, and add back Yibo's co-developed-by tag and signoff.
Move all the per-ac airtime-related data into separate structs.
Use pre-calculated reciprocals to avoid divisions in the fast path.
Record airtime usage for the vif TXQ to avoid it always being scheduled first.
v4:
Change schedule_pos to previous node once it has chance to be moved/removed from current position in the tree in loop scenario and bring back schedule_round in case that same node is to be scheduled again in the mean time.
Increase airtime grace period to 2000 us in the first patch.
Put per-AC station weight checking in its lock during configuration from application.
v3:
Changes station airtime weight to be per-AC based to avoid sync issue Remove Co-developed-by and Toke's sign-off as Toke suggested
v2:
Modify the author of Co-developed-by as Johannes suggested
include/net/mac80211.h | 17 +-
net/mac80211/cfg.c | 35 +++- net/mac80211/debugfs.c | 70 +++++-- net/mac80211/debugfs_netdev.c | 32 ++- net/mac80211/debugfs_sta.c | 24 +-- net/mac80211/ieee80211_i.h | 182 +++++++++++++++-- net/mac80211/iface.c | 3 + net/mac80211/main.c | 11 +- net/mac80211/rx.c | 6 +- net/mac80211/sta_info.c | 67 +++++-- net/mac80211/sta_info.h | 11 +- net/mac80211/status.c | 19 ++ net/mac80211/tx.c | 366 ++++++++++++++++++++++++---------- 13 files changed, 653 insertions(+), 190 deletions(-)
include/net/mac80211.h +3 -14
@@ -6605,9 +6605,6 @@ static inline void ieee80211_txq_schedule_end(struct ieee80211_hw *hw, u8 ac) { } -void __ieee80211_schedule_txq (struct ieee80211_hw *hw, -struct ieee80211_txq *txq,bool force); -/** * ieee80211_schedule_txq - schedule a TXQ for transmission *
@@ -6620,11 +6617,7 @@ void __ieee80211_schedule_txq(struct ieee80211_hw *hw,* The driver may call this function if it has buffered packets for * this TXQ internally. */ -static inline void -ieee80211_schedule_txq (struct ieee80211_hw *hw,struct ieee80211_txq *txq) -{ -__ieee80211_schedule_txq (hw, txq, true); -} +void ieee80211_schedule_txq (struct ieee80211_hw *hw,struct ieee80211_txq *txq);/** * ieee80211_return_txq - return a TXQ previously acquired by ieee80211_next_txq()
@@ -6636,12 +6629,8 @@ ieee80211_schedule_txq(struct ieee80211_hw *hw, struct ieee80211_txq *txq)* The driver may set force=true if it has buffered packets for this TXQ * internally. */ -static inline void -ieee80211_return_txq (struct ieee80211_hw *hw,struct ieee80211_txq *txq, -bool force) -{ -__ieee80211_schedule_txq (hw, txq, force); -} +void ieee80211_return_txq (struct ieee80211_hw *hw,struct ieee80211_txq *txq, +bool force);/** * ieee80211_txq_may_transmit - check whether TXQ is allowed to transmit
net/mac80211/cfg.c +34 -1
@@ -1442,6 +1442,38 @@ static void sta_apply_mesh_params(struct ieee80211_local *local,#endif } +static void sta_apply_airtime_params (struct ieee80211_local *local, +struct sta_info *sta, +struct station_parameters *params) +{ +u8 ac; + +for (ac =0 ; ac <IEEE80211_NUM_ACS ; ac++) { +struct airtime_sched_info *air_sched = &local->airtime[ac]; +struct airtime_info *air_info = &sta->airtime[ac]; +struct txq_info *txqi; +u8 tid; + +spin_lock_bh (&air_sched->lock); +for (tid =0 ; tid <IEEE80211_NUM_TIDS +1 ; tid++) { +if (air_info->weight == params->airtime_weight || + !sta->sta.txq[tid] || + ac !=ieee80211_ac_from_tid (tid)) +continue ; + +airtime_weight_set (air_info, params->airtime_weight); + + txqi =to_txq_info (sta->sta.txq[tid]); +if (RB_EMPTY_NODE (&txqi->schedule_order)) +continue ; + +ieee80211_update_airtime_weight (local, air_sched, +0 , true); + } +spin_unlock_bh (&air_sched->lock); + } +} +static int sta_apply_parameters (struct ieee80211_local *local,struct sta_info *sta,struct station_parameters *params)
@@ -1629,7 +1661,8 @@ static int sta_apply_parameters(struct ieee80211_local *local,sta_apply_mesh_params (local, sta, params);if (params->airtime_weight) - sta->airtime_weight = params->airtime_weight; +sta_apply_airtime_params (local, sta, params); +/* set the STA state after all sta info from usermode has been set */ if (test_sta_flag(sta, WLAN_STA_TDLS_PEER) ||
net/mac80211/debugfs.c +57 -13
95 lines not on this page · raw message
net/mac80211/debugfs_netdev.c +31 -1
@@ -512,6 +512,34 @@ static ssize_t ieee80211_if_fmt_aqm( }IEEE80211_IF_FILE_R (aqm); +static ssize_t ieee80211_if_fmt_airtime ( +const struct ieee80211_sub_if_data *sdata,char *buf,int buflen) +{ +struct ieee80211_local *local = sdata->local; +struct ieee80211_txq *txq = sdata->vif.txq; +struct airtime_info *air_info; +int len; + +if (!txq) +return 0 ; + +spin_lock_bh (&local->airtime[txq->ac].lock); + air_info =to_airtime_info (txq); + len =scnprintf (buf, + buflen, +"RX: %llu us\nTX: %llu us\nWeight: %u\n" +"Virt-T: %lld us\n" , + air_info->rx_airtime, + air_info->tx_airtime, + air_info->weight, + air_info->v_t); +spin_unlock_bh (&local->airtime[txq->ac].lock); + +return len; +} + +IEEE80211_IF_FILE_R (airtime); +IEEE80211_IF_FILE (multicast_to_unicast, u.ap.multicast_to_unicast,HEX );/* IBSS attributes */
@@ -657,8 +685,10 @@ static void add_common_files(struct ieee80211_sub_if_data *sdata)if (sdata->local->ops->wake_tx_queue && sdata->vif.type !=NL80211_IFTYPE_P2P_DEVICE && - sdata->vif.type !=NL80211_IFTYPE_NAN ) + sdata->vif.type !=NL80211_IFTYPE_NAN ) {DEBUGFS_ADD (aqm); +DEBUGFS_ADD (airtime); + } }static void add_sta_files (struct ieee80211_sub_if_data *sdata)
net/mac80211/debugfs_sta.c +12 -12
@@ -202,7 +202,7 @@ static ssize_t sta_airtime_read(struct file *file, char __user *userbuf,size_t bufsz =400 ;char *buf =kzalloc (bufsz,GFP_KERNEL ), *p = buf;u64 rx_airtime =0 , tx_airtime =0 ; -s64 deficit[IEEE80211_NUM_ACS ]; +u64 v_t[IEEE80211_NUM_ACS ];ssize_t rv;int ac;
@@ -210,18 +210,18 @@ static ssize_t sta_airtime_read(struct file *file, char __user *userbuf,return -ENOMEM ;for (ac =0 ; ac <IEEE80211_NUM_ACS ; ac++) { -spin_lock_bh (&local->active_txq_lock[ac]); +spin_lock_bh (&local->airtime[ac].lock); rx_airtime += sta->airtime[ac].rx_airtime; tx_airtime += sta->airtime[ac].tx_airtime; - deficit[ac] = sta->airtime[ac].deficit; -spin_unlock_bh (&local->active_txq_lock[ac]); + v_t[ac] = sta->airtime[ac].v_t; +spin_unlock_bh (&local->airtime[ac].lock); } p +=scnprintf (p, bufsz + buf - p,"RX: %llu us\nTX: %llu us\nWeight: %u\n" -"Deficit: VO: %lld us VI: %lld us BE: %lld us BK: %lld us\n" , - rx_airtime, tx_airtime, sta->airtime_weight, - deficit[0 ], deficit[1 ], deficit[2 ], deficit[3 ]); +"Virt-T: VO: %lld us VI: %lld us BE: %lld us BK: %lld us\n" , + rx_airtime, tx_airtime, sta->airtime[0 ].weight, + v_t[0 ], v_t[1 ], v_t[2 ], v_t[3 ]); rv =simple_read_from_buffer (userbuf, count, ppos, buf, p - buf);kfree (buf);
@@ -236,11 +236,11 @@ static ssize_t sta_airtime_write(struct file *file, const char __user *userbuf,int ac;for (ac =0 ; ac <IEEE80211_NUM_ACS ; ac++) { -spin_lock_bh (&local->active_txq_lock[ac]); +spin_lock_bh (&local->airtime[ac].lock); sta->airtime[ac].rx_airtime =0 ; sta->airtime[ac].tx_airtime =0 ; - sta->airtime[ac].deficit = sta->airtime_weight; -spin_unlock_bh (&local->active_txq_lock[ac]); + sta->airtime[ac].v_t =0 ; +spin_unlock_bh (&local->airtime[ac].lock); }return count;
@@ -263,10 +263,10 @@ static ssize_t sta_aql_read(struct file *file, char __user *userbuf,return -ENOMEM ;for (ac =0 ; ac <IEEE80211_NUM_ACS ; ac++) { -spin_lock_bh (&local->active_txq_lock[ac]); +spin_lock_bh (&local->airtime[ac].lock); q_limit_l[ac] = sta->airtime[ac].aql_limit_low; q_limit_h[ac] = sta->airtime[ac].aql_limit_high; -spin_unlock_bh (&local->active_txq_lock[ac]); +spin_unlock_bh (&local->airtime[ac].lock); q_depth[ac] =atomic_read (&sta->airtime[ac].aql_tx_pending); }
net/mac80211/ieee80211_i.h +170 -12
227 lines not on this page · raw message
net/mac80211/iface.c +3 -0
@@ -1977,6 +1977,9 @@ int ieee80211_if_add(struct ieee80211_local *local, const char *name, } } +for (i =0 ; i <IEEE80211_NUM_ACS ; i++) +init_airtime_info (&sdata->airtime[i], &local->airtime[i]); +ieee80211_set_default_queues (sdata); sdata->ap_power_level =IEEE80211_UNSET_POWER_LEVEL ;
net/mac80211/main.c +7 -4
@@ -705,10 +705,13 @@ struct ieee80211_hw *ieee80211_alloc_hw_nm(size_t priv_data_len,spin_lock_init (&local->queue_stop_reason_lock);for (i =0 ; i <IEEE80211_NUM_ACS ; i++) { -INIT_LIST_HEAD (&local->active_txqs[i]); -spin_lock_init (&local->active_txq_lock[i]); - local->aql_txq_limit_low[i] =IEEE80211_DEFAULT_AQL_TXQ_LIMIT_L ; - local->aql_txq_limit_high[i] = +struct airtime_sched_info *air_sched = &local->airtime[i]; + + air_sched->active_txqs =RB_ROOT_CACHED ; +INIT_LIST_HEAD (&air_sched->active_list); +spin_lock_init (&air_sched->lock); + air_sched->aql_txq_limit_low =IEEE80211_DEFAULT_AQL_TXQ_LIMIT_L ; + air_sched->aql_txq_limit_high =IEEE80211_DEFAULT_AQL_TXQ_LIMIT_H ; }
net/mac80211/rx.c +1 -5
@@ -1578,12 +1578,8 @@ static void sta_ps_start(struct sta_info *sta)for (tid =0 ; tid <IEEE80211_NUM_TIDS ; tid++) {struct ieee80211_txq *txq = sta->sta.txq[tid]; -struct txq_info *txqi =to_txq_info (txq); -spin_lock (&local->active_txq_lock[txq->ac]); -if (!list_empty (&txqi->schedule_order)) -list_del_init (&txqi->schedule_order); -spin_unlock (&local->active_txq_lock[txq->ac]); +ieee80211_unschedule_txq (&local->hw, txq, false);if (txq_has_queue (txq))set_bit (tid, &sta->txq_buffered_tids);
net/mac80211/sta_info.c +49 -18
95 lines not on this page · raw message
net/mac80211/sta_info.h +8 -3
@@ -135,18 +135,25 @@ enum ieee80211_agg_stop_reason {#define AIRTIME_USE_TX BIT(0)#define AIRTIME_USE_RX BIT(1) +struct airtime_info {u64 rx_airtime;u64 tx_airtime; -s64 deficit; +u64 v_t; +u64 last_scheduled; +struct list_head list;atomic_t aql_tx_pending;/* Estimated airtime for frames pending */ u32 aql_limit_low;u32 aql_limit_high; +u32 weight_reciprocal; +u16 weight; };void ieee80211_sta_update_pending_airtime (struct ieee80211_local *local,struct sta_info *sta,u8 ac,u16 tx_airtime,bool tx_completed); +void ieee80211_register_airtime (struct ieee80211_txq *txq, +u32 tx_airtime,u32 rx_airtime);struct sta_info ;
@@ -515,7 +522,6 @@ struct ieee80211_fragment_cache {* @tid_seq: per-TID sequence numbers for sending to this STA * @airtime: per-AC struct airtime_info describing airtime statistics for this * station -* @airtime_weight: station weight for airtime fairness calculation purposes * @ampdu_mlme: A-MPDU state machine state * @mesh: mesh STA information * @debugfs_dir: debug filesystem directory dentry
@@ -646,7 +652,6 @@ struct sta_info {u16 tid_seq[IEEE80211_QOS_CTL_TID_MASK +1 ];struct airtime_info airtime[IEEE80211_NUM_ACS ]; -u16 airtime_weight;/* * Aggregation information, locked with lock.
net/mac80211/status.c +19 -0
@@ -970,6 +970,25 @@ static void __ieee80211_tx_status(struct ieee80211_hw *hw,if (!(info->flags &IEEE80211_TX_CTL_INJECTED ) && acked)ieee80211_frame_acked (sta, skb); + }else if (wiphy_ext_feature_isset (local->hw.wiphy, +NL80211_EXT_FEATURE_AIRTIME_FAIRNESS )) { +struct ieee80211_sub_if_data *sdata; +struct ieee80211_txq *txq; +u32 airtime; + +/* Account airtime to multicast queue */ + sdata =ieee80211_sdata_from_skb (local, skb); + +if (sdata && (txq = sdata->vif.txq)) { + airtime = info->status.tx_time ?: +ieee80211_calc_expected_tx_airtime (hw, + &sdata->vif, +NULL , + skb->len, + false); + +ieee80211_register_airtime (txq, airtime,0 ); + } }/* SNMP counters
net/mac80211/tx.c +259 -107
458 lines not on this page · raw message
Signature · 1 line
2.32.0