Skip to contentSkip to conversation

scheduler changes for v5.5

GIT PULL Merged Ingo Molnar 5 participants 9 messages Nov 25, 2019 – Nov 27, 2019

lore
Share & tools
lore
#0 lore raw

Linus,

Please pull the latest sched-core-for-linus git tree from:

git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git sched-core-for-linus

# HEAD: de881a341c4143650fa50ce95cf450a5c94faa9f Merge branch 'sched/rt' into sched/core, to pick up commit

The biggest changes in this cycle were:

- Make kcpustat vtime aware (Frederic Weisbecker)

- Rework the CFS load_balance() logic (Vincent Guittot)

- Misc cleanups, smaller enhancements, fixes.

The load-balancing rework is the most intrusive change: it replaces the old heuristics that have become less meaningful after the introduction of the PELT metrics, with a grounds-up load-balancing algorithm.

As such it's not really an iterative series, but replaces the old load-balancing logic with the new one. We hope there are no performance regressions left - but statistically it's highly probable that there *is* going to be some workload that is hurting from these chnages. If so then we'd prefer to have a look at that workload and fix its scheduling, instead of reverting the changes.

Thanks,

Ingo

------------------>
Frederic Weisbecker (22):
sched/cputime: Rename vtime_account_system() to vtime_account_kernel() sched/cputime: Spare a seqcount lock/unlock cycle on context switch
sched/vtime: Record CPU under seqcount for kcpustat needs
sched/cputime: Add vtime idle task state
sched/cputime: Add vtime guest task state
context_tracking: Remove context_tracking_active()
context_tracking: Rename context_tracking_is_enabled() => context_tracking_enabled()
context_tracking: Rename context_tracking_is_cpu_enabled() to context_tracking_enabled_this_cpu()
context_tracking: Introduce context_tracking_enabled_cpu()
sched/vtime: Rename vtime_accounting_cpu_enabled() to vtime_accounting_enabled_this_cpu() sched/vtime: Introduce vtime_accounting_enabled_cpu()
context_tracking: Check static key on context_tracking_enabled_*cpu() sched/kcpustat: Introduce vtime-aware kcpustat accessor for CPUTIME_SYSTEM
procfs: Use vtime aware kcpustat accessor to fetch CPUTIME_SYSTEM
cpufreq: Use vtime aware kcpustat accessor to fetch CPUTIME_SYSTEM
leds: Use vtime aware kcpustat accessor to fetch CPUTIME_SYSTEM
sched/cputime: Support other fields on kcpustat_field()
sched/vtime: Bring up complete kcpustat accessor
procfs: Use all-in-one vtime aware kcpustat accessor
cpufreq: Use vtime aware kcpustat accessors for user time
leds: Use all-in-one vtime aware kcpustat accessor
rackmeter: Use vtime aware kcpustat accessor

Patrick Bellasi (1):
sched/fair/util_est: Implement faster ramp-up EWMA on utilization increases

Peter Zijlstra (6):
sched/fair: Better document newidle_balance() sched/core: Make pick_next_task_idle() more consistent sched/core: Optimize pick_next_task() sched/core: Simplify sched_class::pick_next_task() sched/fair: Use mul_u32_u32()
sched/core: Further clarify sched_class::set_next_task()

Srivatsa S. Bhat (VMware) (1):
sched/Kconfig: Fix spelling mistake in user-visible help text

Valentin Schneider (2):
sched/topology: Don't set SD_BALANCE_WAKE on cpuset domain relax sched/uclamp: Fix overzealous type replacement

Vincent Guittot (14):
sched/fair: Clean up asym packing
sched/fair: Rename sg_lb_stats::sum_nr_running to sum_h_nr_running sched/fair: Remove meaningless imbalance calculation sched/fair: Rework load_balance()
sched/fair: Use rq->nr_running when balancing load sched/fair: Use load instead of runnable load in load_balance() sched/fair: Spread out tasks evenly when not overloaded sched/fair: Use utilization to select misfit task
sched/fair: Use load instead of runnable load in wakeup path sched/fair: Optimize find_idlest_group()
sched/fair: Rework find_idlest_group()
sched/fair: Fix rework of find_idlest_group()
sched/fair: Add comments for group_type and balancing at SD_NUMA level sched/cpufreq: Move the cfs_rq_util_change() call to cpufreq_update_util()

 arch/ia64/kernel/time.c                 |    4 +-
 arch/powerpc/kernel/time.c              |    6 +-
 arch/s390/kernel/vtime.c                |    4 +-
 arch/x86/entry/calling.h                |    2 +-
 drivers/cpufreq/cpufreq.c               |   17 +-
 drivers/cpufreq/cpufreq_governor.c      |    6 +-
 drivers/leds/trigger/ledtrig-activity.c |   14 +-
 drivers/macintosh/rack-meter.c          |    7 +-
 fs/proc/stat.c                          |   56 +-
 include/linux/context_tracking.h        |   30 +-
 include/linux/context_tracking_state.h  |   21 +-
 include/linux/kernel_stat.h             |   18 +
 include/linux/sched.h                   |    9 +-
 include/linux/tick.h                    |    2 +-
 include/linux/vtime.h                   |   59 +-
 kernel/Kconfig.preempt                  |    2 +-
 kernel/context_tracking.c               |    6 +-
 kernel/sched/core.c                     |   18 +-
 kernel/sched/cputime.c                  |  288 ++++++-
 kernel/sched/deadline.c                 |   12 +-
 kernel/sched/fair.c                     | 1437 +++++++++++++++++++------------
 kernel/sched/features.h                 |    1 +
 kernel/sched/idle.c                     |   10 +-
 kernel/sched/rt.c                       |   12 +-
 kernel/sched/sched.h                    |   25 +-
 kernel/sched/stop_task.c                |    9 +-
 kernel/sched/topology.c                 |    9 +-
 kernel/time/tick-sched.c                |    2 +-
 28 files changed, 1331 insertions(+), 755 deletions(-)
#1 lore raw

On 25/11/2019 12:59, Ingo Molnar wrote:

So I really don't want to be labeled as "that annoying scheduler PR guy", but some patches in Vincent's rework should be squashed to avoid being performance bisection honeypots.

Quoted 3 lines

Vincent Guittot (14):
sched/fair: Remove meaningless imbalance calculation sched/fair: Rework load_balance()

These two ^ (were split for ease of reviewing, [1])

Quoted 2 lines

sched/fair: Rework find_idlest_group() sched/fair: Fix rework of find_idlest_group()

And these two ^ (Mel voiced similar concerns at [2])

Sorry for being annoying :/

[1]: https://lore.kernel.org/lkml/CAKfTPtCqLGJDNHgG099wQ7Gy6A+63yP36cxAR_somNPEcJMxGA@mail.gmail.com/
[2]: https://lore.kernel.org/lkml/[email protected]/

#2 lore raw
Quoted 5 lines · Valentin Schneider

On 25/11/2019 12:59, Ingo Molnar wrote:

So I really don't want to be labeled as "that annoying scheduler PR guy", but some patches in Vincent's rework should be squashed to avoid being performance bisection honeypots.

I really didn't want to do an intrusive rebasing of the Git tree though, especially as we didn't know whether the improvements a) improved things
b) were stable:

Quoted 10 lines
Quoted 3 lines

Vincent Guittot (14):
sched/fair: Remove meaningless imbalance calculation sched/fair: Rework load_balance()

These two ^ (were split for ease of reviewing, [1])

Quoted 2 lines

sched/fair: Rework find_idlest_group() sched/fair: Fix rework of find_idlest_group()

And these two ^ (Mel voiced similar concerns at [2])

We can give testers a linearized tree for testing, should this come up (which I doubt it will ...), ok?

Thanks,

Ingo

#3 lore raw
Quoted 3 lines · Ingo Molnar

We can give testers a linearized tree for testing, should this come up (which I doubt it will ...), ok?

My worry (and I think Mel's) is on performance bisection of the mainline tree (not specifically on the load balancer rework), by LKP or else. It's not something I personally do (nor expect to do in the foreseeable future), so Mel is much better positioned than I to argue for/against this.

Still, I was under the impression that not introducing (scheduler) regressions in the git history was valuable. If I'm misguided, feel free to ignore.

Quoted 4 lines

Thanks,

Ingo

#4 lore raw
Quoted 10 lines · Valentin Schneider
Quoted 3 lines · Ingo Molnar

We can give testers a linearized tree for testing, should this come up (which I doubt it will ...), ok?

My worry (and I think Mel's) is on performance bisection of the mainline tree (not specifically on the load balancer rework), by LKP or else. It's not something I personally do (nor expect to do in the foreseeable future), so Mel is much better positioned than I to argue for/against this.

This was a concern. If there is a regression then a bisection may point to one of the earlier patches. That will be ok as long as people remember to look at the whole series instead of just the patch the bisection identifies. The second source of confusion may be that LKP reports a regression followed by a gain that will lack a comparison with the baseline so we may not be able to rely on LKP to detect regressions/gains from the series. It's inconvenient but not critical enough to dump the testing the existing branch has.

Signature · 2 lines
Mel Gorman
SUSE Labs
#5 lore raw
In reply to Valentin Schneider
Quoted 11 lines · Valentin Schneider

On 25/11/2019 12:59, Ingo Molnar wrote:

So I really don't want to be labeled as "that annoying scheduler PR guy", but some patches in Vincent's rework should be squashed to avoid being performance bisection honeypots.

Quoted 3 lines

Vincent Guittot (14):
sched/fair: Remove meaningless imbalance calculation sched/fair: Rework load_balance()

Quoted 6 lines

These two ^ (were split for ease of reviewing, [1])

Quoted 2 lines

sched/fair: Rework find_idlest_group() sched/fair: Fix rework of find_idlest_group()

And these two ^ (Mel voiced similar concerns at [2])

If they were split for ease of reviewing, then they should be split in the history too.

I worry a lot less about some possible (temporary!) performance dip than about a hard bug, and if the code is easier to review in two steps then it's going to be easier to find the bug in two steps too.

Linus

#6 lore raw
Quoted 18 lines · Linus Torvalds
Quoted 16 lines · Valentin Schneider

On 25/11/2019 12:59, Ingo Molnar wrote:

So I really don't want to be labeled as "that annoying scheduler PR guy", but some patches in Vincent's rework should be squashed to avoid being performance bisection honeypots.

Quoted 3 lines

Vincent Guittot (14):
sched/fair: Remove meaningless imbalance calculation sched/fair: Rework load_balance()

These two ^ (were split for ease of reviewing, [1])

Quoted 2 lines

sched/fair: Rework find_idlest_group() sched/fair: Fix rework of find_idlest_group()

Quoted 9 lines
Quoted 1 line

And these two ^ (Mel voiced similar concerns at [2])

If they were split for ease of reviewing, then they should be split in the history too.

I worry a lot less about some possible (temporary!) performance dip than about a hard bug, and if the code is easier to review in two steps then it's going to be easier to find the bug in two steps too.

Fair enough, lesson learned, sorry for the noise.

#7 lore raw
In reply to Ingo Molnar
Quoted 7 lines · Ingo Molnar

As such it's not really an iterative series, but replaces the old load-balancing logic with the new one. We hope there are no performance regressions left - but statistically it's highly probable that there *is* going to be some workload that is hurting from these chnages. If so then we'd prefer to have a look at that workload and fix its scheduling, instead of reverting the changes.

We may not have that opportunity. A lot of loads are various private loads and you'll just get "this regressed by a huge amount".

So please do keep in mind that reverting this may be the only reasonable model if it turns out to have bad effects.

Linus

merge bot Merged
#8 lore raw
In reply to Ingo Molnar
The pull request you sent on Mon, 25 Nov 2019 13:59:44 +0100:

The pull request you sent on Mon, 25 Nov 2019 13:59:44 +0100:

Quoted 1 line

git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git sched-core-for-linus

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/77a05940eee7e9891cd6add7a690a3e762ee21b0

Thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker