RFC sched,fair: flatten CPU controller runqueues
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v5 RFC Latest15 patches 0 repliesNew
- v4 RFC15 patches 30 repliesSuperseded
- v3 RFC13 of 14 patches 13 repliesSuperseded
Earlier versions · 2
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v2 RFC | 10 patches | 14 replies | Superseded | |
| v1 | 8 patches | 15 replies | Superseded |
Changes in v5 from the cover letter
- drop patch that removes __sched_slice (Vincent Guittot)
- remove se->depth (Dietmar Eggemann)
- various other small cleanups and improvements suggested by people
Patches in v5 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/15 | sched: introduce task_se_h_load helperkernel/ | +14 | 0 replies | |
| 2/15 | sched: change /proc/sched_debug fieldskernel/ | +2 | 0 replies | |
| 3/15 | sched,fair: redefine runnable_load_avg as the sum of task_h_loadinclude/ kernel/ kernel/ +3 more | +56 | 0 replies | |
| 4/15 | sched,fair: move runnable_load_avg to cfs_rqinclude/ kernel/ kernel/ +1 more | +6 | 0 replies | |
| 5/15 | sched,fair: remove cfs_rqs from leaf_cfs_rq_list bottom upkernel/ | +34 | 0 replies | |
| 6/15 | sched,cfs: use explicit cfs_rq of parent se helperkernel/ | +13 | 0 replies | |
| 7/15 | sched,cfs: fix zero length timeslice calculationkernel/ | +7 | 0 replies | |
| 8/15 | sched,fair: refactor enqueue/dequeue_entitykernel/ | +42 | 0 replies | |
| 9/15 | sched,fair: add helper functions for flattened runqueuekernel/ | +31 | 0 replies | |
| 10/15 | sched,fair: flatten hierarchical runqueuesinclude/ kernel/ kernel/ +2 more | +170 | 0 replies | |
| 11/15 | sched,fair: flatten update_curr functionalitykernel/ | +12 | 0 replies | |
| 12/15 | sched,fair: propagate sum_exec_runtime up the hierarchyinclude/ kernel/ kernel/ | +22 | 0 replies | |
| 13/15 | sched,fair: ramp up task_se_h_weight quicklykernel/ | +6 | 0 replies | |
| 14/15 | sched,fair: scale vdiff in wakeup_preempt_entitykernel/ | +1 | 0 replies | |
| 15/15 | sched,fair: remove se->depthinclude/ kernel/ | +2 | 0 replies |
Reviews trailers per patch
3 of 15 patches have a Reviewed-by or Acked-by.
| Patch | Josef Bacik | Rik van Riel | Vincent Guittot |
|---|---|---|---|
| 1/15 | R-b | ||
| 2/15 | R-b | ||
| 3/15 | |||
| 4/15 | |||
| 5/15 | A-b | ||
| 6/15 | |||
| 7/15 | |||
| 8/15 | |||
| 9/15 | |||
| 10/15 | |||
| 11/15 | |||
| 12/15 | |||
| 13/15 | Not-yet-signed-off-by | ||
| 14/15 | |||
| 15/15 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.
Cover letter [PATCH RFC v5 0/15] sched,fair: flatten CPU controller runqueues
The current implementation of the CPU controller uses hierarchical runqueues, where on wakeup a task is enqueued on its group's runqueue, the group is enqueued on the runqueue of the group above it, etc.
This increases a fairly large amount of overhead for workloads that do a lot of wakeups a second, especially given that the default systemd hierarchy is 2 or 3 levels deep.
This patch series is an attempt at reducing that overhead, by placing all the tasks on the same runqueue, and scaling the task priority by the priority of the group, which is calculated periodically.
My main TODO items for the next period of time are likely going to be testing, testing, and testing. I hope to find and flush out any corner case I can find, and make sure performance does not regress with any workloads, and hopefully improves some.
Other TODO items:
- More code cleanups.
- Remove some more now unused code.
- Reimplement CONFIG_CFS_BANDWIDTH.
Plan for the CONFIG_CFS_BANDWIDTH reimplementation:
- When a cgroup gets throttled, mark the cgroup and its children as throttled.
- When pick_next_entity finds a task that is on a throttled cgroup, stash it on the cgroup runqueue (which is not used for runnable tasks any more). Leave the vruntime unchanged, and adjust that runqueue's vruntime to be that of the left-most task.
- When a cgroup gets unthrottled, and has tasks on it, place it on a vruntime ordered heap separate from the main runqueue.
- Have pick_next_task_fair grab one task off that heap every time it is called, and the min vruntime of that heap is lower than the vruntime of the CPU's cfs_rq (or the CPU has no other runnable tasks).
- Place that selected task on the CPU's cfs_rq, renormalizing its vruntime with the GENTLE_FAIR_SLEEPERS logic. That should help interleave the already runnable tasks with the recently unthrottled group, and prevent thundering herd issues.
- If the group gets throttled again before all of its task had a chance to run, vruntime sorting ensures all the tasks in the throttled cgroup get a chance to run over time.
Changes from v4:
- drop patch that removes __sched_slice (Vincent Guittot)
- remove se->depth (Dietmar Eggemann)
- various other small cleanups and improvements suggested by people
Changes from v3:
- replace max_h_load with another hacky idea to ramp up the task_se_h_weight; I believe this new idea is wrong as well, but it will hopefully inspire a better solution (thanks to Peter Zijlstra)
- fix the ordering inside enqueue_task_fair to get task weights set up right (thanks to Peter Zijlstra)
- change wakeup_preempt_entity to reduce the number of task preemptions, hopefully resulting in behavior closer to what people configure in sysctl
- various other small cleanups and fixes
Changes from v2:
- fixed the web server performance regression, in a way vaguely similar to what Josef Bacik suggested (blame me for the implementation)
- removed some code duplication so the diffstat is redder than before
- propagate sum_exec_runtime up the tree, in preparation for CFS_BANDWIDTH
- small cleanups left and right
Changes from v1:
- use task_se_h_weight instead of task_se_h_load in calc_delta_fair and sched_slice, this seems to improve performance a little, but I still have some remaining regression to chase with our web server workload
- implement a number of the changes suggested by Dietmar Eggemann (still holding out for a better name for group_cfs_rq_of_parent)
This series applies on top of 5.2
include/linux/sched.h | 8 kernel/sched/core.c | 3 kernel/sched/debug.c | 15 kernel/sched/fair.c | 821 ++++++++++++++++++++------------------------------ kernel/sched/pelt.c | 68 +--- kernel/sched/pelt.h | 2 kernel/sched/sched.h | 9 7 files changed, 367 insertions(+), 559 deletions(-)