Skip to content

RFC Core scheduling v4

Vineeth Remanan Pillai lkml 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 RFC Latest19 patches 64 repliesIn review
  2. v3 RFC16 patches 133 repliesSuperseded
  3. v2 RFC17 patches 91 repliesSuperseded

Changes in v4 from the cover letter

-------------
- Implement a core wide min_vruntime for vruntime comparison of tasks across cpus in a core.
- Fixes a typo bug in setting the forced_idle cpu.

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/19stop_machine: Fix stop_cpus_in_progress orderingkernel/stop_machine.c+2 -00 replies
2/19sched: Fix kerneldoc comment for ia64_set_curr_taskkernel/sched/core.c+1 -10 replies
3/19sched: Wrap rq::lock accesskernel/sched/core.c kernel/sched/cpuacct.c kernel/sched/deadline.c +7 more+128 -1250 replies
4/19sched/{rt,deadline}: Fix set_next_task vs pick_next_taskkernel/sched/deadline.c kernel/sched/rt.c+26 -240 replies
5/19sched: Add task_struct pointer to sched_class::set_curr_taskkernel/sched/core.c kernel/sched/deadline.c kernel/sched/fair.c +4 more+49 -460 replies
6/19sched/fair: Export newidle_balance()kernel/sched/fair.c kernel/sched/sched.h+12 -100 replies
7/19sched: Allow put_prev_task() to drop rq->lockkernel/sched/core.c kernel/sched/deadline.c kernel/sched/fair.c +4 more+32 -80 replies
8/19sched: Rework pick_next_task() slow-pathkernel/sched/core.c kernel/sched/deadline.c kernel/sched/fair.c +4 more+34 -730 replies
9/19sched: Introduce sched_class::pick_task()kernel/sched/deadline.c kernel/sched/fair.c kernel/sched/idle.c +3 more+92 -190 replies
10/19sched: Core-wide rq->lockkernel/Kconfig.preempt kernel/sched/core.c kernel/sched/sched.h+148 -20 replies
11/19sched: Basic tracking of matching tasksinclude/linux/sched.h kernel/sched/core.c kernel/sched/fair.c +1 more+208 -470 replies
12/19sched: A quick and dirty cgroup tagging interfacekernel/sched/core.c kernel/sched/sched.h+82 -00 replies
13/19sched: Add core wide task selection and scheduling.kernel/sched/core.c kernel/sched/sched.h+274 -30 replies
14/19sched/fair: Add a few assertionskernel/sched/fair.c+10 -20 replies
15/19sched: Trivial forced-newidle balancerinclude/linux/sched.h kernel/sched/core.c kernel/sched/idle.c +1 more+138 -10 replies
16/19sched: Debug bits...kernel/sched/core.c+42 -20 replies
17/19sched/fair: wrapper for cfs_rq->min_vruntimekernel/sched/fair.c+16 -110 replies
18/19sched/fair: core wide vruntime comparisonkernel/sched/core.c kernel/sched/fair.c kernel/sched/sched.h+102 -140 replies
19/19sched/fair : Wake up forced idle siblings if neededkernel/sched/fair.c+40 -00 replies

Reviews trailers per patch

None of the 19 patches has a Reviewed-by or Acked-by.

PatchPeter Zijlstra (Intel)
1/19
2/19
3/19
4/19
5/19
6/19
7/19
8/19
9/19
10/19
11/19
12/19
13/19
14/19
15/19
16/19Not-Signed-off-by
17/19
18/19
19/19

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [RFC PATCH v4 00/19] Core scheduling v4

Fourth iteration of the Core-Scheduling feature.

This version was aiming mostly at addressing the vruntime comparison issues with v3. The main issue seen in v3 was the starvation of interactive tasks when competing with cpu intensive tasks. This issue is mitigated to a large extent.

We have tested and verified that incompatible processes are not selected during schedule. In terms of performance, the impact depends on the workload:
- on CPU intensive applications that use all the logical CPUs with SMT enabled, enabling core scheduling performs better than nosmt.
- on mixed workloads with considerable io compared to cpu usage, nosmt seems to perform better than core scheduling.

v4 is rebased on top of 5.3.5(dc073f193b70):
https://github.com/digitalocean/linux-coresched/tree/coresched/v4-v5.3.5

Changes in v4
-------------
- Implement a core wide min_vruntime for vruntime comparison of tasks across cpus in a core.
- Fixes a typo bug in setting the forced_idle cpu.

Changes in v3
-------------
- Fixes the issue of sibling picking up an incompatible task
- Aaron Lu
- Vineeth Pillai
- Julien Desfossez
- Fixes the issue of starving threads due to forced idle
- Peter Zijlstra
- Fixes the refcounting issue when deleting a cgroup with tag
- Julien Desfossez
- Fixes a crash during cpu offline/online with coresched enabled
- Vineeth Pillai
- Fixes a comparison logic issue in sched_core_find
- Aaron Lu

Changes in v2
-------------
- Fixes for couple of NULL pointer dereference crashes
- Subhra Mazumdar
- Tim Chen
- Improves priority comparison logic for process in different cpus
- Peter Zijlstra
- Aaron Lu
- Fixes a hard lockup in rq locking
- Vineeth Pillai
- Julien Desfossez
- Fixes a performance issue seen on IO heavy workloads
- Vineeth Pillai
- Julien Desfossez
- Fix for 32bit build
- Aubrey Li

TODO
----
- Decide on the API for exposing the feature to userland
- Investigate the source of the overhead even when no tasks are tagged:
https://lkml.org/lkml/2019/10/29/242
- Investigate the performance scaling issue when we have a high number of tagged threads: https://lkml.org/lkml/2019/10/29/248
- Try to optimize the performance for IO-demanding applications:
https://lkml.org/lkml/2019/10/29/261

---

Aaron Lu (3):
sched/fair: wrapper for cfs_rq->min_vruntime sched/fair: core wide vruntime comparison sched/fair : Wake up forced idle siblings if needed

Peter Zijlstra (16):
stop_machine: Fix stop_cpus_in_progress ordering
sched: Fix kerneldoc comment for ia64_set_curr_task
sched: Wrap rq::lock access
sched/{rt,deadline}: Fix set_next_task vs pick_next_task
sched: Add task_struct pointer to sched_class::set_curr_task sched/fair: Export newidle_balance()
sched: Allow put_prev_task() to drop rq->lock
sched: Rework pick_next_task() slow-path
sched: Introduce sched_class::pick_task()
sched: Core-wide rq->lock
sched: Basic tracking of matching tasks
sched: A quick and dirty cgroup tagging interface
sched: Add core wide task selection and scheduling. sched/fair: Add a few assertions
sched: Trivial forced-newidle balancer
sched: Debug bits...

 include/linux/sched.h    |   9 +-
 kernel/Kconfig.preempt   |   6 +
 kernel/sched/core.c      | 847 +++++++++++++++++++++++++++++++++++++--
 kernel/sched/cpuacct.c   |  12 +-
 kernel/sched/deadline.c  |  99 +++--
 kernel/sched/debug.c     |   4 +-
 kernel/sched/fair.c      | 346 +++++++++++-----
 kernel/sched/idle.c      |  42 +-
 kernel/sched/pelt.h      |   2 +-
 kernel/sched/rt.c        |  96 ++---
 kernel/sched/sched.h     | 246 +++++++++---
 kernel/sched/stop_task.c |  35 +-
 kernel/sched/topology.c  |   4 +-
 kernel/stop_machine.c    |   2 +
 14 files changed, 1399 insertions(+), 351 deletions(-)
Signature · 1 line
2.17.1