sched/fair: Active balancer RT/DL preemption fix
v2 cover sched/fair: Active balancer RT/DL preemption fix
Vincent's load balance rework [1] got me thinking about how and where we use rq.nr_running vs rq.cfs.h_nr_running checks, and this lead me to stare intently at the active load balancer.
I haven't seen it happen (yet), but from reading the code it really looks like we can have some scenarios where the cpu_stopper ends up preempting a > CFS class task, since we never actually look at what's the remote rq's running task.
This series shuffles things around the CFS active load balancer to prevent this from happening.
- Patch 1 is a freebie cleanup
- Patch 2 is a preparatory code move
- Patch 3 adds h_nr_running checks
- Patch 4 adds a sched class check + detach_one_task() to the active balance
This is based on top of today's tip/sched/core:
a46d14eca7b7 ("sched/fair: Use rq_lock/unlock in online_fair_sched_group")
v1 -> v2:
- (new patch) Added need_active_balance() cleanup
- Tweaked active balance code move to respect existing
sd->nr_balance_failed modifications
- Added explicit checks of active_load_balance()'s return value
- Added an h_nr_running < 1 check before kicking the cpu_stopper
- Added a detach_one_task() call in active_load_balance() when the remote rq's running task is > CFS
[1]: https://lore.kernel.org/lkml/[email protected]/
Valentin Schneider (4):
sched/fair: Make need_active_balance() return bools sched/fair: Move active balance logic to its own function sched/fair: Check for CFS tasks before detach_one_task() sched/fair: Prevent active LB from preempting higher sched classes
kernel/sched/fair.c | 151 ++++++++++++++++++++++++++++---------------- 1 file changed, 95 insertions(+), 56 deletions(-)
--
2.22.0
PATCH v2 1/4 sched/fair: Make need_active_balance() return bools
That function has no need to return ints, make it return bools.
--- kernel/sched/fair.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/fair.c +2 -2
@@ -8734,12 +8734,12 @@ voluntary_active_balance(struct lb_env *env)return 0 ; } -static int need_active_balance (struct lb_env *env) +static bool need_active_balance (struct lb_env *env) {struct sched_domain *sd = env->sd;if (voluntary_active_balance (env)) -return 1 ; +return true;return unlikely (sd->nr_balance_failed > sd->cache_nice_tries+2 ); }
Signature · 1 line
2.22.0
PATCH v2 2/4 sched/fair: Move active balance logic to its own function
In reply to Valentin SchneiderThe logic to trigger an active balance is already quite lengthy, and as I'm about to add yet another unlock condition it's probably better to give it its own corner.
The only annoying requirement is that we need to branch to out_one_pinned when the active balance is cancelled due to the running task's affinity. Something like < 0, == 0 and > 0 return values could suffice, but I went for an enum for clarity.
No change in functionality intended.
Note that the final
status != cancelled_affinity
check is there to maintain the existing behaviour, i.e. bump sd->nr_balance_failed both when the cpu_stopper has been kicked and when it hasn't, which doesn't sound entirely right to me.
--- kernel/sched/fair.c | 126 ++++++++++++++++++++++++++------------------ 1 file changed, 74 insertions(+), 52 deletions(-)
kernel/sched/fair.c +74 -52
@@ -8746,6 +8746,72 @@ static bool need_active_balance(struct lb_env *env)static int active_load_balance_cpu_stop (void *data); +/* Active load balance */ +enum alb_status { + cancelled =0 , + cancelled_affinity, + started +}; + +/* Attempt to move a remote rq's running task to env's dst_cpu */ +static inline enum alb_status active_load_balance (struct lb_env *env) +{ +enum alb_status status = cancelled; +struct sched_domain *sd = env->sd; +struct rq *busiest = env->src_rq; +unsigned long flags; + +schedstat_inc (sd->lb_failed[env->idle]); +/* +* Increment the failure counter only on periodic balance. +* We do not want newidle balance, which can be very frequent, pollute +* the failure counter causing excessive cache_hot migrations and +* active balances. +*/ +if (env->idle !=CPU_NEWLY_IDLE ) + sd->nr_balance_failed++; + +if (!need_active_balance (env)) + goto out; + +raw_spin_lock_irqsave (&busiest->lock, flags); + +/* +* Don't kick the active_load_balance_cpu_stop, if the curr task on +* busiest CPU can't be moved to dst_cpu: +*/ +if (!cpumask_test_cpu (env->dst_cpu, busiest->curr->cpus_ptr)) { + env->flags |=LBF_ALL_PINNED ; + status = cancelled_affinity; + goto unlock; + } + +/* +* ->active_balance synchronizes accesses to ->active_balance_work. +* Once set, it's cleared only after active load balance is finished. +*/ +if (!busiest->active_balance) { + busiest->active_balance =1 ; + busiest->push_cpu = env->dst_cpu; + status = started; + } + +unlock: +raw_spin_unlock_irqrestore (&busiest->lock, flags); + +if (status == started) +stop_one_cpu_nowait (cpu_of (busiest), + active_load_balance_cpu_stop, busiest, + &busiest->active_balance_work); + +/* We've kicked active balancing, force task migration. */ +if (status != cancelled_affinity) + sd->nr_balance_failed = sd->cache_nice_tries +1 ; + +out: +return status; +} +static int should_we_balance (struct lb_env *env) {struct sched_group *sg = env->sd->groups;
@@ -8792,7 +8858,8 @@ static int load_balance(int this_cpu, struct rq *this_rq, struct sched_domain *sd, enum cpu_idle_type idle, int *continue_balancing) { - int ld_moved, cur_ld_moved, active_balance = 0; + enum alb_status active_balance = cancelled; +int ld_moved, cur_ld_moved;struct sched_domain *sd_parent = sd->parent;struct sched_group *group;struct rq *busiest;
@@ -8950,61 +9017,16 @@ static int load_balance(int this_cpu, struct rq *this_rq, } } -if (!ld_moved) { -schedstat_inc (sd->lb_failed[idle]); -/* -* Increment the failure counter only on periodic balance. -* We do not want newidle balance, which can be very -* frequent, pollute the failure counter causing -* excessive cache_hot migrations and active balances. -*/ -if (idle !=CPU_NEWLY_IDLE ) - sd->nr_balance_failed++; - -if (need_active_balance (&env)) { -unsigned long flags; - -raw_spin_lock_irqsave (&busiest->lock, flags); - -/* -* Don't kick the active_load_balance_cpu_stop, -* if the curr task on busiest CPU can't be -* moved to this_cpu: -*/ -if (!cpumask_test_cpu (this_cpu, busiest->curr->cpus_ptr)) { -raw_spin_unlock_irqrestore (&busiest->lock, - flags); - env.flags |=LBF_ALL_PINNED ; - goto out_one_pinned; - } - -/* -* ->active_balance synchronizes accesses to -* ->active_balance_work. Once set, it's cleared -* only after active load balance is finished. -*/ -if (!busiest->active_balance) { - busiest->active_balance =1 ; - busiest->push_cpu = this_cpu; - active_balance =1 ; - } -raw_spin_unlock_irqrestore (&busiest->lock, flags); - -if (active_balance) { -stop_one_cpu_nowait (cpu_of (busiest), - active_load_balance_cpu_stop, busiest, - &busiest->active_balance_work); - } - -/* We've kicked active balancing, force task migration. */ - sd->nr_balance_failed = sd->cache_nice_tries+1 ; - } - }else +if (!ld_moved) + active_balance =active_load_balance (&env); +else sd->nr_balance_failed =0 ; -if (likely (!active_balance) ||voluntary_active_balance (&env)) { +if (likely (active_balance == cancelled) ||voluntary_active_balance (&env)) {/* We were unbalanced, so reset the balancing interval */ sd->balance_interval = sd->min_interval; + }else if (active_balance == cancelled_affinity) { + goto out_one_pinned; }else {/* * If we've begun active balancing, start to back off. This
Signature · 1 line
2.22.0
Re: [PATCH v2 2/4] sched/fair: Move active balance logic to its own function
kernel/sched/fair.c static bool need_active_balance(struct lb_env *env) · line 8809
Quoted 11 lines
kernel/sched/fair.c +11
-0+unlock: +raw_spin_unlock_irqrestore (&busiest->lock, flags); + +if (status == started) +stop_one_cpu_nowait (cpu_of (busiest), + active_load_balance_cpu_stop, busiest, + &busiest->active_balance_work); + +/* We've kicked active balancing, force task migration. */ +if (status != cancelled_affinity) + sd->nr_balance_failed = sd->cache_nice_tries +1 ;
Should we really update nr_balance_failed if status is cancelled? I do understand this behaviour was present even before this change. But still dont understand why we need to update if the current operation didn't kick active_load_balance.
Signature · 2 lines
Thanks and Regards Srikar Dronamraju
Re: [PATCH v2 2/4] sched/fair: Move active balance logic to its own function
Quoted 8 lines · Srikar Dronamraju
Quoted 8 lines
(quoted diff) +8
-0+unlock: + raw_spin_unlock_irqrestore(&busiest->lock, flags); + + if (status == started) + stop_one_cpu_nowait(cpu_of(busiest), + active_load_balance_cpu_stop, busiest, + &busiest->active_balance_work); +
Quoted 9 lines
Review
Quoted 3 lines
(quoted diff) +3
-0+ /* We've kicked active balancing, force task migration. */ + if (status != cancelled_affinity) + sd->nr_balance_failed = sd->cache_nice_tries + 1;Should we really update
nr_balance_failedif status is cancelled? I do understand this behaviour was present even before this change. But still dont understand why we need to update if the current operation didn't kickactive_load_balance.
Agreed, I kept it as is to keep this as pure a code movement as possible, but I don't see why this wouldn't be valid wouldn't be valid (PoV of the current code):
---
kernel/sched/fair.c +3 -3
@@ -9023,10 +9023,10 @@ static int load_balance(int this_cpu, struct rq *this_rq,stop_one_cpu_nowait (cpu_of (busiest), active_load_balance_cpu_stop, busiest, &busiest->active_balance_work); - } -/* We've kicked active balancing, force task migration. */ - sd->nr_balance_failed = sd->cache_nice_tries+1 ; +/* We've kicked active balancing, force task migration. */ + sd->nr_balance_failed = sd->cache_nice_tries+1 ; + } } }else sd->nr_balance_failed =0 ;
---
Or even better, fold it in active_load_balance_cpu_stop(). I could add that after the move.
PATCH v2 3/4 sched/fair: Check for CFS tasks before detach_one_task()
In reply to Valentin Schneiderdetach_one_task() (and the CFS load-balancer in general) can only ever pull CFS tasks, so we should check if there's any before invoking the cpu stopper.
Likewise, when the cpu stopper is already up, we can bail out slightly earlier when we see there's no CFS task to detach.
Check for CFS tasks at the top of need_active_balance(), and add a similar check in active_load_balance_cpu_stop().
---
Note that this only touches the active load balance paths. IMO load_balance() could go through a similar treatment (s/rq.nr_running/rq.cfs.h_nr_running/ before going to detach_tasks()) but that conflicts with Vincent's rework, so I've not included it here.
--- kernel/sched/fair.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
kernel/sched/fair.c +6 -2
@@ -8776,6 +8776,10 @@ static inline enum alb_status active_load_balance(struct lb_env *env)raw_spin_lock_irqsave (&busiest->lock, flags); +/* Is there actually anything to pull? */ +if (busiest->cfs.h_nr_running <1 ) + goto unlock; +/* * Don't kick the active_load_balance_cpu_stop, if the curr task on * busiest CPU can't be moved to dst_cpu:
@@ -9142,8 +9146,8 @@ static int active_load_balance_cpu_stop(void *data) !busiest_rq->active_balance)) goto out_unlock; -/* Is there any task to move? */ -if (busiest_rq->nr_running <=1 ) +/* Is there any CFS task to move? */ +if (busiest_rq->cfs.h_nr_running <1 ) goto out_unlock;/*
Signature · 1 line
2.22.0
PATCH v2 4/4 sched/fair: Prevent active LB from preempting higher sched classes
In reply to Valentin SchneiderThe CFS load balancer can cause the cpu_stopper to run a function to try and steal a remote rq's running task. However, it so happens that while only CFS tasks will ever be migrated by that function, we can end up preempting higher sched class tasks, since it is executed by the cpu_stopper.
This can potentially occur whenever a rq's running task is > CFS but the rq has runnable CFS tasks.
Check the sched class of the remote rq's running task after we've grabbed its lock. If it's CFS, carry on, otherwise run detach_one_task() locally since we don't need the cpu_stopper (that !CFS task is doing the exact same thing).
--- kernel/sched/fair.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
kernel/sched/fair.c +14 -1
@@ -8759,6 +8759,7 @@ static inline enum alb_status active_load_balance(struct lb_env *env)enum alb_status status = cancelled;struct sched_domain *sd = env->sd;struct rq *busiest = env->src_rq; +struct task_struct *p =NULL ;unsigned long flags;schedstat_inc (sd->lb_failed[env->idle]);
@@ -8780,6 +8781,16 @@ static inline enum alb_status active_load_balance(struct lb_env *env)if (busiest->cfs.h_nr_running <1 ) goto unlock; +/* +* If busiest->curr isn't CFS, then there's no point in running the +* cpu_stopper. We can try to nab one CFS task though, since they're +* all ready to be plucked. +*/ +if (busiest->curr->sched_class != &fair_sched_class) { + p =detach_one_task (env); + goto unlock; + } +/* * Don't kick the active_load_balance_cpu_stop, if the curr task on * busiest CPU can't be moved to dst_cpu:
@@ -8803,7 +8814,9 @@ static inline enum alb_status active_load_balance(struct lb_env *env) unlock:raw_spin_unlock_irqrestore (&busiest->lock, flags); -if (status == started) +if (p) +attach_one_task (env->dst_rq, p); +else if (status == started)stop_one_cpu_nowait (cpu_of (busiest), active_load_balance_cpu_stop, busiest, &busiest->active_balance_work);
Signature · 1 line
2.22.0
Vincent Guittot
maintainerRe: [PATCH v2 4/4] sched/fair: Prevent active LB from preempting higher sched classes
Quoted 7 lines · Valentin Schneider
The CFS load balancer can cause the
cpu_stopperto run a function to try and steal a remote rq's running task. However, it so happens that while only CFS tasks will ever be migrated by that function, we can end up preempting higher sched class tasks, since it is executed by thecpu_stopper.
Quoted 7 lines
This can potentially occur whenever a rq's running task is > CFS but the rq has runnable CFS tasks.
Check the sched class of the remote rq's running task after we've grabbed its lock. If it's CFS, carry on, otherwise run
detach_one_task()locally since we don't need thecpu_stopper(that !CFS task is doing the exact same thing).
AFAICT, this doesn't prevent from preempting !CFS task but only reduce the window.
As soon as you unlock, !CFS task can preempt CFS before you start stop thread
testing busiest->cfs.h_nr_running < 1 and/or busiest->curr->sched_class != &fair_sched_class
in need_active_balance() will do almost the same and is much simpler than this patchset IMO.
Quoted 49 lines
Signed-off-by Valentin Schneider <[email protected]>--- kernel/sched/fair.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)kernel/sched/fair.c +14
-2@@ -8759,6 +8759,7 @@ static inline enum alb_status active_load_balance(struct lb_env *env)enum alb_status status = cancelled;struct sched_domain *sd = env->sd;struct rq *busiest = env->src_rq; +struct task_struct *p =NULL ;unsigned long flags;schedstat_inc (sd->lb_failed[env->idle]);@@ -8780,6 +8781,16 @@ static inline enum alb_status active_load_balance(struct lb_env *env)if (busiest->cfs.h_nr_running <1 ) goto unlock; +/* +* If busiest->curr isn't CFS, then there's no point in running the +* cpu_stopper. We can try to nab one CFS task though, since they're +* all ready to be plucked. +*/ +if (busiest->curr->sched_class != &fair_sched_class) { + p =detach_one_task (env); + goto unlock; + } +/* * Don't kick the active_load_balance_cpu_stop, if the curr task on * busiest CPU can't be moved to dst_cpu: @@ -8803,7 +8814,9 @@ static inline enum alb_status active_load_balance(struct lb_env *env) unlock:raw_spin_unlock_irqrestore (&busiest->lock, flags); -if (status == started) +if (p) +attach_one_task (env->dst_rq, p); +else if (status == started)stop_one_cpu_nowait (cpu_of (busiest), active_load_balance_cpu_stop, busiest, &busiest->active_balance_work); --2.22.0
Re: [PATCH v2 4/4] sched/fair: Prevent active LB from preempting higher sched classes
Quoted 12 lines · Vincent Guittot
Quoted 10 lines · Valentin Schneider
The CFS load balancer can cause the
cpu_stopperto run a function to try and steal a remote rq's running task. However, it so happens that while only CFS tasks will ever be migrated by that function, we can end up preempting higher sched class tasks, since it is executed by thecpu_stopper.This can potentially occur whenever a rq's running task is > CFS but the rq has runnable CFS tasks.
Quoted 9 lines
Quoted 4 lines
Check the sched class of the remote rq's running task after we've grabbed its lock. If it's CFS, carry on, otherwise run
detach_one_task()locally since we don't need thecpu_stopper(that !CFS task is doing the exact same thing).AFAICT, this doesn't prevent from preempting !CFS task but only reduce the window.
As soon as you unlock, !CFS task can preempt CFS before you start stop thread
Right, if we end up kicking the cpu_stopper this can still happen (since we drop the lock). Thing is, you can't detect it on the cpu_stopper side, since the currently running is obviously not going to be CFS (and it's too late anyway, we already preempted whatever was running there). Though I should probably change the name of the patch to reflect that it's not a 100% cure.
I tweaked the nr_running check of the cpu_stop callback in patch 3/4 to try to bail out early, but AFAICT that's the best we can do without big changes elsewhere.
If we wanted to prevent those preemptions at all cost, I suppose we'd want the equivalent of a sched class sitting between CFS and RT: have the callback only run when there's no runnable > CFS tasks. But then by the time we execute it we may no longer need to balance anything...
At the very least, what I'm proposing here alleviates *some* of the preemption cases without swinging the wrecking ball too hard (and without delaying the balancing either).
Quoted 5 lines
testing
busiest->cfs.h_nr_running< 1 and/orbusiest->curr->sched_class!=&fair_sched_class
inneed_active_balance()will do almost the same and is much simpler than this patchset IMO.
I had this initially but convinced myself out of it: since we hold no lock in need_active_balance(), the information we get on the current task (and, arguably, on the h_nr_running) is too volatile to be of any use.
I do believe those checks have their place in active_load_balance()'s critical section, as that's the most accurate we're going to get. On the plus side, if we *do* detect the remote rq's current task isn't CFS, we can run detach_one_task() locally, which is an improvement IMO.
Vincent Guittot
maintainerRe: [PATCH v2 4/4] sched/fair: Prevent active LB from preempting higher sched classes
Quoted 31 lines · Valentin Schneider
Quoted 12 lines · Vincent Guittot
Quoted 10 lines · Valentin Schneider
The CFS load balancer can cause the
cpu_stopperto run a function to try and steal a remote rq's running task. However, it so happens that while only CFS tasks will ever be migrated by that function, we can end up preempting higher sched class tasks, since it is executed by thecpu_stopper.This can potentially occur whenever a rq's running task is > CFS but the rq has runnable CFS tasks.
Quoted 9 lines
Quoted 4 lines
Check the sched class of the remote rq's running task after we've grabbed its lock. If it's CFS, carry on, otherwise run
detach_one_task()locally since we don't need thecpu_stopper(that !CFS task is doing the exact same thing).AFAICT, this doesn't prevent from preempting !CFS task but only reduce the window.
As soon as you unlock, !CFS task can preempt CFS before you start stop threadRight, if we end up kicking the
cpu_stopperthis can still happen (since we drop the lock). Thing is, you can't detect it on thecpu_stopperside, since the currently running is obviously not going to be CFS (and it's too late anyway, we already preempted whatever was running there). Though I should probably change the name of the patch to reflect that it's not a 100% cure.
Quoted 5 lines
I tweaked the
nr_runningcheck of thecpu_stopcallback in patch 3/4 to try to bail out early, but AFAICT that's the best we can do without big changes elsewhere.If we wanted to prevent those preemptions at all cost, I suppose we'd want
I'm not sure that it's worth the effort and the complexity
Quoted 14 lines
the equivalent of a sched class sitting between CFS and RT: have the callback only run when there's no runnable > CFS tasks. But then by the time we execute it we may no longer need to balance anything...
At the very least, what I'm proposing here alleviates *some* of the preemption cases without swinging the wrecking ball too hard (and without delaying the balancing either).
Quoted 5 lines
testing
busiest->cfs.h_nr_running< 1 and/orbusiest->curr->sched_class!=&fair_sched_class
inneed_active_balance()will do almost the same and is much simpler than this patchset IMO.
Quoted 3 lines
I had this initially but convinced myself out of it: since we hold no lock in
need_active_balance(), the information we get on the current task (and, arguably, on theh_nr_running) is too volatile to be of any use.
But since the lock is released anyway, everything will always be too volatile in this case.
Quoted 5 lines
I do believe those checks have their place in active_load_balance()'s critical section, as that's the most accurate we're going to get. On the plus side, if we *do* detect the remote rq's current task isn't CFS, we can run
detach_one_task()locally, which is an improvement IMO.
This add complexity in the code by adding another path to detach attach task(s). We could simply bail out and wait the next load balance (which is already the case sometime) or if you really want to detach a task jump back to more_balance
Re: [PATCH v2 4/4] sched/fair: Prevent active LB from preempting higher sched classes
On 29/08/2019 15:19, Vincent Guittot wrote:
[...]
Quoted 7 lines
Quoted 7 lines
Right, if we end up kicking the
cpu_stopperthis can still happen (since we drop the lock). Thing is, you can't detect it on thecpu_stopperside, since the currently running is obviously not going to be CFS (and it's too late anyway, we already preempted whatever was running there). Though I should probably change the name of the patch to reflect that it's not a 100% cure.
Quoted 8 lines
Quoted 5 lines
I tweaked the
nr_runningcheck of thecpu_stopcallback in patch 3/4 to try to bail out early, but AFAICT that's the best we can do without big changes elsewhere.If we wanted to prevent those preemptions at all cost, I suppose we'd want
I'm not sure that it's worth the effort and the complexity
My point exactly :)
[...]
Quoted 6 lines
Quoted 3 lines
I had this initially but convinced myself out of it: since we hold no lock in
need_active_balance(), the information we get on the current task (and, arguably, on theh_nr_running) is too volatile to be of any use.But since the lock is released anyway, everything will always be too volatile in this case.
We do release the lock if we go kick the cpu_stopper, but can nevertheless make a decision with the most up to date information. I'd say it's for similar reasons that we check busiest->curr->cpus_ptr right before kicking the cpu_stopper rather than in need_active_balance().
The majority of the checks in need_active_balance() (all but one) depend on env/sd stats which aren't volatile.
Quoted 7 lines
Quoted 5 lines
I do believe those checks have their place in active_load_balance()'s critical section, as that's the most accurate we're going to get. On the plus side, if we *do* detect the remote rq's current task isn't CFS, we can run
detach_one_task()locally, which is an improvement IMO.This add complexity in the code by adding another path to detach attach task(s).
Note that it's not a new detach/attach per se, rather it's about doing it in active_load_balance() rather than active_load_balance_cpu_stop() in some cases.
Quoted 4 lines
We could simply bail out and wait the next load balance (which is already the case sometime) or if you really want to detach a task jump back to
more_balance
A simple bail-out is what I had in v1, but following Qais' comments I figured I could add the detach_one_tasks().
Jumping back to more_balance is quite different than doing a detach_one_task().
(expanded the Cc list)
RT/DL folks, any thought on the thing?
Quoted 46 lines · Valentin Schneider
Vincent's load balance rework [1] got me thinking about how and where we use rq.nr_running vs rq.cfs.h_nr_running checks, and this lead me to stare intently at the active load balancer.
I haven't seen it happen (yet), but from reading the code it really looks like we can have some scenarios where the
cpu_stopperends up preempting a > CFS class task, since we never actually look at what's the remote rq's running task.This series shuffles things around the CFS active load balancer to prevent this from happening.
- Patch 1 is a freebie cleanup
- Patch 2 is a preparatory code move
- Patch 3 addsh_nr_runningchecks
- Patch 4 adds a sched class check +detach_one_task()to the active balanceThis is based on top of today's tip/sched/core:
a46d14eca7b7("sched/fair: Use rq_lock/unlock inonline_fair_sched_group")v1 -> v2:
- (new patch) Addedneed_active_balance()cleanup- Tweaked active balance code move to respect existing
sd->nr_balance_failed modifications- Added explicit checks of active_load_balance()'s return value
- Added an
h_nr_running< 1 check before kicking thecpu_stopper- Added a
detach_one_task()call inactive_load_balance()when the remote rq's running task is > CFS[1]: https://lore.kernel.org/lkml/[email protected]/
Valentin Schneider (4):
sched/fair: Makeneed_active_balance()return bools sched/fair: Move active balance logic to its own function sched/fair: Check for CFS tasks beforedetach_one_task()sched/fair: Prevent active LB from preempting higher sched classeskernel/sched/fair.c | 151 ++++++++++++++++++++++++++++---------------- 1 file changed, 95 insertions(+), 56 deletions(-)--
2.22.0
Juri Lelli
maintainerHi Valentin,
Quoted 2 lines · Valentin Schneider
(expanded the Cc list)
RT/DL folks, any thought on the thing?
Even if I like your idea and it looks theoretically the right thing to do, I'm not sure we want it in practice if it adds complexity to CFS.
I personally never noticed this kind of interference from CFS, but, at the same time, for RT we usually like more to be safe than sorry. However, since this doesn't seem to be bullet-proof (as you also say), I guess it all boils down again to complexity vs. practical benefits.
Best,
Juri
Hi Juri,
Quoted 6 lines · Juri Lelli
Hi Valentin,
Quoted 2 lines · Valentin Schneider
(expanded the Cc list)
RT/DL folks, any thought on the thing?
Quoted 8 lines
Even if I like your idea and it looks theoretically the right thing to do, I'm not sure we want it in practice if it adds complexity to CFS.
I personally never noticed this kind of interference from CFS, but, at the same time, for RT we usually like more to be safe than sorry. However, since this doesn't seem to be bullet-proof (as you also say), I guess it all boils down again to complexity vs. practical benefits.
Thanks for having a look.
IMO worst part is the local detach_one_task() thing, I added that after v1 following Qais' comments but perhaps it doesn't gain us much.
I'll try to cook something up with rt-app and see if I can get sensible numbers.
Quoted 4 lines
Best,
Juri