sched: Update cpu_load before nohz ilb
RFC PATCH 1/1 sched: Update cpu_load before nohz ilb
Update the cpu_load of the kickee in run_rebalance_domains before calling rebalance_domains if the kickee is in nohz_idle. This avoids using stale cpu_load for the subsequent load_balance.
cpu_load in the nohz path is updated on tick_nohz_idle_exit or if a kickee performances balancing on the behalf of tickless cpus in nohz_idle_balance. However, the kickee's cpu_load is never updated if it doesn't exit nohz_idle. This causes stale cpu_load to be used which causes incorrect load balancing decisions.
Tested on k3.10.
--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)
kernel/sched/fair.c +0 -0
@@ -7140,6 +7140,13 @@ static void run_rebalance_domains(struct
softirq_action *h)
enum cpu_idle_type idle = this_rq->idle_balance ?
CPU_IDLE : CPU_NOT_IDLE;+ if (test_bit(NOHZ_BALANCE_KICK, nohz_flags(this_rq->cpu))) {
+ raw_spin_lock_irq(&this_rq->lock);
+ update_rq_clock(this_rq);
+ update_idle_cpu_load(this_rq);
+ raw_spin_unlock_irq(&this_rq->lock);
+ }
+
rebalance_domains(this_rq, idle);
/*Signature · 1 line
1.8.1.5
+ More people.
Quoted 37 lines · Sai
Update the
cpu_loadof the kickee inrun_rebalance_domainsbefore callingrebalance_domainsif the kickee is innohz_idle. This avoids using stalecpu_loadfor the subsequentload_balance.
cpu_loadin the nohz path is updated ontick_nohz_idle_exitor if a kickee performances balancing on the behalf of tickless cpus innohz_idle_balance. However, the kickee'scpu_loadis never updated if it doesn't exitnohz_idle. This causes stalecpu_loadto be used which causes incorrect load balancing decisions.Tested on k3.10.
Signed-off-by Sai Gurrappadi <[email protected]>--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)kernel/sched/fair.c +7
-0@@ -7140,6 +7140,13 @@ static void run_rebalance_domains(struct softirq_action *h)enum cpu_idle_type idle = this_rq->idle_balance ?CPU_IDLE :CPU_NOT_IDLE ; +if (test_bit (NOHZ_BALANCE_KICK ,nohz_flags (this_rq->cpu))) { +raw_spin_lock_irq (&this_rq->lock); +update_rq_clock (this_rq); +update_idle_cpu_load (this_rq); +raw_spin_unlock_irq (&this_rq->lock); + } +rebalance_domains (this_rq, idle);/*