sched/fair: fix nohz next idle balance
Vincent Guittot
maintainerPATCH v2 1/1 sched/fair: fix nohz next idle balance
With commit:
'b7031a02ec75 ("sched/fair: Add NOHZ_STATS_KICK")'rebalance_domains of the local cfs_rq happens before others idle cpus have updated nohz.next_balance and its value is overwritten.
Move the update of nohz.next_balance for other idles cpus before balancing and updating the next_balance of local cfs_rq.
Also, the nohz.next_balance is now updated only if all idle cpus got a chance to rebalance their domains and the idle balance has not been aborted because of new activities on the CPU. In case of need_resched, the idle load balance will be kick the next jiffie in order to address remaining ilb.
---
- Changes v2:
- use jiffies instead of incrementing nohz.next_balance to be sure of the value. The change slipped out of the previous version.
kernel/sched/fair.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-)
kernel/sched/fair.c +14 -9
@@ -10017,7 +10017,12 @@ static void kick_ilb(unsigned int flags) {int ilb_cpu; - nohz.next_balance++; +/* +* Increase nohz.next_balance only when if full ilb is triggered but +* not if we only update stats. +*/ +if (flags &NOHZ_BALANCE_KICK ) + nohz.next_balance = jiffies+1 ; ilb_cpu =find_new_ilb ();
@@ -10338,6 +10343,14 @@ static bool _nohz_idle_balance(struct rq *this_rq, unsigned int flags, } } +/* +* next_balance will be updated only when there is a need. +* When the CPU is attached to null domain for ex, it will not be +* updated. +*/ +if (likely (update_next_balance)) + nohz.next_balance = next_balance; +/* Newly idle CPU doesn't need an update */ if (idle !=CPU_NEWLY_IDLE ) {update_blocked_averages (this_cpu);
@@ -10358,14 +10371,6 @@ static bool _nohz_idle_balance(struct rq *this_rq, unsigned int flags,if (has_blocked_load)WRITE_ONCE (nohz.has_blocked,1 ); -/* -* next_balance will be updated only when there is a need. -* When the CPU is attached to null domain for ex, it will not be -* updated. -*/ -if (likely (update_next_balance)) - nohz.next_balance = next_balance; -return ret; }
Signature · 1 line
2.17.1
Peter Zijlstra
maintainerQuoted 8 lines · Vincent Guittot
With commit:
'b7031a02ec75("sched/fair: AddNOHZ_STATS_KICK")'rebalance_domainsof the localcfs_rqhappens before others idle cpus have updated nohz.next_balance and its value is overwritten.Move the update of nohz.next_balance for other idles cpus before balancing and updating the
next_balanceof localcfs_rq.
Quoted 8 lines
Also, the nohz.next_balance is now updated only if all idle cpus got a chance to rebalance their domains and the idle balance has not been aborted because of new activities on the CPU. In case of
need_resched, the idle load balance will be kick the next jiffie in order to address remaining ilb.Reported-by Peng Liu <[email protected]>Signed-off-by Vincent Guittot <[email protected]>
Thanks
Quoted 8 lines · Vincent Guittot
With commit:
'b7031a02ec75("sched/fair: AddNOHZ_STATS_KICK")'rebalance_domainsof the localcfs_rqhappens before others idle cpus have updated nohz.next_balance and its value is overwritten.Move the update of nohz.next_balance for other idles cpus before balancing and updating the
next_balanceof localcfs_rq.
Quoted 8 lines
Also, the nohz.next_balance is now updated only if all idle cpus got a chance to rebalance their domains and the idle balance has not been aborted because of new activities on the CPU. In case of
need_resched, the idle load balance will be kick the next jiffie in order to address remaining ilb.Reported-by Peng Liu <[email protected]>Signed-off-by Vincent Guittot <[email protected]>
FWIW:
Do we want a Fixes: tag for this? I'm thinkingb7031a02ec75 ("sched/fair: Add NOHZ_STATS_KICK")
Peter Zijlstra
maintainerQuoted 16 lines · Valentin Schneider
Quoted 14 lines · Vincent Guittot
With commit:
'b7031a02ec75("sched/fair: AddNOHZ_STATS_KICK")'rebalance_domainsof the localcfs_rqhappens before others idle cpus have updated nohz.next_balance and its value is overwritten.Move the update of nohz.next_balance for other idles cpus before balancing and updating the
next_balanceof localcfs_rq.Also, the nohz.next_balance is now updated only if all idle cpus got a chance to rebalance their domains and the idle balance has not been aborted because of new activities on the CPU. In case of
need_resched, the idle load balance will be kick the next jiffie in order to address remaining ilb.
Quoted 8 lines
Quoted 2 lines
Reported-by Peng Liu <[email protected]>Signed-off-by Vincent Guittot <[email protected]>FWIW:
Reviewed-by Valentin Schneider <[email protected]>Do we want a Fixes: tag for this? I'm thinking
b7031a02ec75("sched/fair: AddNOHZ_STATS_KICK")
Done, thanks!
Quoted 8 lines · Vincent Guittot
With commit:
'b7031a02ec75("sched/fair: AddNOHZ_STATS_KICK")'rebalance_domainsof the localcfs_rqhappens before others idle cpus have updated nohz.next_balance and its value is overwritten.Move the update of nohz.next_balance for other idles cpus before balancing and updating the
next_balanceof localcfs_rq.
Quoted 8 lines
Also, the nohz.next_balance is now updated only if all idle cpus got a chance to rebalance their domains and the idle balance has not been aborted because of new activities on the CPU. In case of
need_resched, the idle load balance will be kick the next jiffie in order to address remaining ilb.Reported-by Peng Liu <[email protected]>Signed-off-by Vincent Guittot <[email protected]>
Signature · 2 lines
Mel Gorman SUSE Labs
[tip: sched/core] sched/fair: Fix NOHZ next idle balance
In reply to Vincent GuittotThe following commit has been merged into the sched/core branch of tip:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 3ea2f097b17e13a8280f1f9386c331b326a3dbef Gitweb: https://git.kernel.org/tip/3ea2f097b17e13a8280f1f9386c331b326a3dbef Author: Vincent Guittot <[email protected]> AuthorDate: Tue, 09 Jun 2020 14:37:48 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 15 Jun 2020 14:10:04 +02:00
sched/fair: Fix NOHZ next idle balance
With commit:
'b7031a02ec75 ("sched/fair: Add NOHZ_STATS_KICK")'rebalance_domains of the local cfs_rq happens before others idle cpus have updated nohz.next_balance and its value is overwritten.
Move the update of nohz.next_balance for other idles cpus before balancing and updating the next_balance of local cfs_rq.
Also, the nohz.next_balance is now updated only if all idle cpus got a chance to rebalance their domains and the idle balance has not been aborted because of new activities on the CPU. In case of need_resched, the idle load balance will be kick the next jiffie in order to address remaining ilb.
--- kernel/sched/fair.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-)
kernel/sched/fair.c +14 -9
@@ -10022,7 +10022,12 @@ static void kick_ilb(unsigned int flags) {int ilb_cpu; - nohz.next_balance++; +/* +* Increase nohz.next_balance only when if full ilb is triggered but +* not if we only update stats. +*/ +if (flags &NOHZ_BALANCE_KICK ) + nohz.next_balance = jiffies+1 ; ilb_cpu =find_new_ilb ();
@@ -10343,6 +10348,14 @@ static bool _nohz_idle_balance(struct rq *this_rq, unsigned int flags, } } +/* +* next_balance will be updated only when there is a need. +* When the CPU is attached to null domain for ex, it will not be +* updated. +*/ +if (likely (update_next_balance)) + nohz.next_balance = next_balance; +/* Newly idle CPU doesn't need an update */ if (idle !=CPU_NEWLY_IDLE ) {update_blocked_averages (this_cpu);
@@ -10363,14 +10376,6 @@ abort:if (has_blocked_load)WRITE_ONCE (nohz.has_blocked,1 ); -/* -* next_balance will be updated only when there is a need. -* When the CPU is attached to null domain for ex, it will not be -* updated. -*/ -if (likely (update_next_balance)) - nohz.next_balance = next_balance; -return ret; }