cpuidle: Always stop scheduler tick on adaptive-tick CPUs
PATCH 1/1 cpuidle: Always stop scheduler tick on adaptive-tick CPUs
From: Rafael J. Wysocki <[email protected]>
Running the scheduler tick on idle adaptive-tick CPUs is not useful and it may also be not expected by users (as reported by Thomas), so add a check to cpuidle_idle_call() to always stop the tick on them regardless of the idle duration predicted by the governor.
--- kernel/sched/idle.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/sched/idle.c +2 -1
@@ -191,7 +191,8 @@ static void cpuidle_idle_call(void)*/ next_state =cpuidle_select (drv, dev, &stop_tick); -if (stop_tick ||tick_nohz_tick_stopped ()) +if (stop_tick ||tick_nohz_tick_stopped () || + !housekeeping_cpu (dev->cpu,HK_FLAG_TICK ))tick_nohz_idle_stop_tick ();else tick_nohz_idle_retain_tick ();
Quoted 3 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
Running the scheduler tick on idle adaptive-tick CPUs is not useful
Judging by the below change, you mean full dynticks, right?
Quoted 17 lines
and it may also be not expected by users (as reported by Thomas), so add a check to
cpuidle_idle_call()to always stop the tick on them regardless of the idle duration predicted by the governor.Fixes 554c8aa8ecad ("sched: idle: Select idle state before stopping the tick")Reported-by Thomas Lindroth <[email protected]>Tested-by Thomas Lindroth <[email protected]>Signed-off-by Rafael J. Wysocki <[email protected]>--- kernel/sched/idle.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)Index:
linux-pm/kernel/sched/idle.c
===================================================================
kernel/sched/idle.c static void cpuidle_idle_call(void) · line 195
Quoted 6 lines
kernel/sched/idle.c +2
-1@@ -191,7 +191,8 @@ static void cpuidle_idle_call(void)*/ next_state =cpuidle_select (drv, dev, &stop_tick); -if (stop_tick ||tick_nohz_tick_stopped ()) +if (stop_tick ||tick_nohz_tick_stopped () || + !housekeeping_cpu (dev->cpu,HK_FLAG_TICK ))
But tick_nohz_tick_stopped() also works on full dynticks CPUs. If the tick isn't stopped on a full dynticks CPU by the time we reach this path, it means that the conditions for the tick to be stopped are not met anyway (eg: more than one task and sched tick is needed, perf event requires the tick, posix CPU timer, etc...)
Or am I missing something else?
Thanks.
Quoted 6 lines
kernel/sched/idle.c +0
-0tick_nohz_idle_stop_tick (); elsetick_nohz_idle_retain_tick ();
Quoted 7 lines · Frederic Weisbecker
Quoted 3 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
Running the scheduler tick on idle adaptive-tick CPUs is not useful
Judging by the below change, you mean full dynticks, right?
Right.
Quoted 20 lines
Quoted 20 lines
and it may also be not expected by users (as reported by Thomas), so add a check to
cpuidle_idle_call()to always stop the tick on them regardless of the idle duration predicted by the governor.Fixes 554c8aa8ecad ("sched: idle: Select idle state before stopping the tick")Reported-by Thomas Lindroth <[email protected]>Tested-by Thomas Lindroth <[email protected]>Signed-off-by Rafael J. Wysocki <[email protected]>--- kernel/sched/idle.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)Index:
linux-pm/kernel/sched/idle.c
===================================================================kernel/sched/idle.c +0
-0@@ -191,7 +191,8 @@ static void cpuidle_idle_call(void)*/ next_state =cpuidle_select (drv, dev, &stop_tick);
Quoted 9 lines
kernel/sched/idle.c static void cpuidle_idle_call(void) · line 195
Quoted 3 lines
kernel/sched/idle.c +2
-1-if (stop_tick ||tick_nohz_tick_stopped ()) +if (stop_tick ||tick_nohz_tick_stopped () || + !housekeeping_cpu (dev->cpu,HK_FLAG_TICK ))But
tick_nohz_tick_stopped()also works on full dynticks CPUs. If the tick isn't stopped on a full dynticks CPU by the time we reach this path, it means that the conditions for the tick to be stopped are not met anyway (eg: more than one task and sched tick is needed, perf event requires the tick, posix CPU timer, etc...)
First of all, according to Thomas, the patch does make a difference, so evidently on his system(s) the full dynticks CPUs enter the idle loop with running tick.
This means that, indeed, the conditions for the tick to be stopped have not been met up to that point, but if the (full dynticks) CPU becomes idle, that's because it has been made idle on purpose (presumably by a user-space "orchestrator" or the sysadmin), so the kernel can assume that it will remain idle indefinitely. That, in turn, is when the tick would be stopped on it regardless of everything else (even if it wasn't a full dynticks CPU).
I guess I should add the above to the changelog.
Quoted 1 line
Or am I missing something else?
Well, if full dynticks CPUs are expected to always enter the idle loop with stopped tick, then something appears to be amiss, but I'm not sure if that expectation is entirely realistic.
Cheers!
Quoted 46 lines · Rafael J. Wysocki
Quoted 7 lines · Frederic Weisbecker
Quoted 3 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
Running the scheduler tick on idle adaptive-tick CPUs is not useful
Judging by the below change, you mean full dynticks, right?
Right.
Quoted 20 lines
Quoted 20 lines
and it may also be not expected by users (as reported by Thomas), so add a check to
cpuidle_idle_call()to always stop the tick on them regardless of the idle duration predicted by the governor.Fixes 554c8aa8ecad ("sched: idle: Select idle state before stopping the tick")Reported-by Thomas Lindroth <[email protected]>Tested-by Thomas Lindroth <[email protected]>Signed-off-by Rafael J. Wysocki <[email protected]>--- kernel/sched/idle.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)Index:
linux-pm/kernel/sched/idle.c
===================================================================kernel/sched/idle.c +0
-0@@ -191,7 +191,8 @@ static void cpuidle_idle_call(void)*/ next_state =cpuidle_select (drv, dev, &stop_tick);Quoted 9 lines
kernel/sched/idle.c static void cpuidle_idle_call(void) · line 195
Quoted 3 lines
kernel/sched/idle.c +2
-1-if (stop_tick ||tick_nohz_tick_stopped ()) +if (stop_tick ||tick_nohz_tick_stopped () || + !housekeeping_cpu (dev->cpu,HK_FLAG_TICK ))But
tick_nohz_tick_stopped()also works on full dynticks CPUs. If the tick isn't stopped on a full dynticks CPU by the time we reach this path, it means that the conditions for the tick to be stopped are not met anyway (eg: more than one task and sched tick is needed, perf event requires the tick, posix CPU timer, etc...)First of all, according to Thomas, the patch does make a difference, so evidently on his system(s) the full dynticks CPUs enter the idle loop with running tick.
Quoted 7 lines
This means that, indeed, the conditions for the tick to be stopped have not been met up to that point, but if the (full dynticks) CPU becomes idle, that's because it has been made idle on purpose (presumably by a user-space "orchestrator" or the sysadmin), so the kernel can assume that it will remain idle indefinitely. That, in turn, is when the tick would be stopped on it regardless of everything else (even if it wasn't a full dynticks CPU).
Well I think we disagree on that assumption that if a nohz_full CPU is put idle, it will remain there indefinitely. Nohz_full CPUs aren't really special in this regard, they can sleep on an IO, wait for a short event just like any other CPU.
The only difference with other CPUs is that they _might_ enter the idle loop with the tick already stopped. Ok that should be the case most of the time with regular full dynticks usecases, but there can be initialization work or stuff that make the CPU run with periodic tick for some time.
Thanks.
Quoted 10 lines
I guess I should add the above to the changelog.
Quoted 1 line
Or am I missing something else?
Well, if full dynticks CPUs are expected to always enter the idle loop with stopped tick, then something appears to be amiss, but I'm not sure if that expectation is entirely realistic.
Cheers!
Quoted 56 lines · Frederic Weisbecker
Quoted 46 lines · Rafael J. Wysocki
Quoted 7 lines · Frederic Weisbecker
Quoted 3 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
Running the scheduler tick on idle adaptive-tick CPUs is not useful
Judging by the below change, you mean full dynticks, right?
Right.
Quoted 20 lines
Quoted 20 lines
and it may also be not expected by users (as reported by Thomas), so add a check to
cpuidle_idle_call()to always stop the tick on them regardless of the idle duration predicted by the governor.Fixes 554c8aa8ecad ("sched: idle: Select idle state before stopping the tick")Reported-by Thomas Lindroth <[email protected]>Tested-by Thomas Lindroth <[email protected]>Signed-off-by Rafael J. Wysocki <[email protected]>--- kernel/sched/idle.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)Index:
linux-pm/kernel/sched/idle.c
===================================================================kernel/sched/idle.c +0
-0@@ -191,7 +191,8 @@ static void cpuidle_idle_call(void)*/ next_state =cpuidle_select (drv, dev, &stop_tick);Quoted 9 lines
kernel/sched/idle.c static void cpuidle_idle_call(void) · line 195
Quoted 3 lines
kernel/sched/idle.c +2
-1-if (stop_tick ||tick_nohz_tick_stopped ()) +if (stop_tick ||tick_nohz_tick_stopped () || + !housekeeping_cpu (dev->cpu,HK_FLAG_TICK ))But
tick_nohz_tick_stopped()also works on full dynticks CPUs. If the tick isn't stopped on a full dynticks CPU by the time we reach this path, it means that the conditions for the tick to be stopped are not met anyway (eg: more than one task and sched tick is needed, perf event requires the tick, posix CPU timer, etc...)First of all, according to Thomas, the patch does make a difference, so evidently on his system(s) the full dynticks CPUs enter the idle loop with running tick.
Quoted 7 lines
This means that, indeed, the conditions for the tick to be stopped have not been met up to that point, but if the (full dynticks) CPU becomes idle, that's because it has been made idle on purpose (presumably by a user-space "orchestrator" or the sysadmin), so the kernel can assume that it will remain idle indefinitely. That, in turn, is when the tick would be stopped on it regardless of everything else (even if it wasn't a full dynticks CPU).
Quoted 4 lines
Well I think we disagree on that assumption that if a
nohz_fullCPU is put idle, it will remain there indefinitely.Nohz_fullCPUs aren't really special in this regard, they can sleep on an IO, wait for a short event just like any other CPU.
Fair enough.
This means that the governor (or rather governors) will need to be modified to address the issue reported by Thomas.
Fortunately, I have a patch going in that direction too. :-)
Cheers!
Quoted 8 lines · Rafael J. Wysocki
Quoted 6 lines · Frederic Weisbecker
On Wed, Jul 17, 2019 at 09:55:08AM +0200, Rafael J. Wysocki wrote:
Well I think we disagree on that assumption that if anohz_fullCPU is put idle, it will remain there indefinitely.Nohz_fullCPUs aren't really special in this regard, they can sleep on an IO, wait for a short event just like any other CPU.
Quoted 6 lines
Fair enough.
This means that the governor (or rather governors) will need to be modified to address the issue reported by Thomas.
Fortunately, I have a patch going in that direction too. :-)
Good to hear, please also Cc me on that one, thanks!