sched/uclamp: Avoid getting unreasonable ucalmp value when rq is idle
PATCH v2 1/1 sched/uclamp: Avoid getting unreasonable ucalmp value when rq is idle
From: Xuewen Yan <[email protected]>
Now in uclamp_rq_util_with(), when the task != NULL, the uclamp_max as following:uc_rq_max = rq->uclamp[UCLAMP_MAX].value;uc_eff_max = uclamp_eff_value(p, UCLAMP_MAX);uclamp_max = max{uc_rq_max, uc_eff_max};
Consider the following scenario:
(1)the rq is idle, the uc_rq_max is last runnable task's UCLAMP_MAX; (2)the p's uc_eff_max < uc_rq_max.
As a result, the uclamp_max = uc_rq_max instead of uc_eff_max, it is unreasonable.
The scenario often happens in find_energy_efficient_cpu(), when the task has smaller UCLAMP_MAX.
When rq has UCLAMP_FLAG_IDLE flag, enqueuing the task will lift UCLAMP_FLAG_IDLE and set the rq clamp as the task's via uclamp_idle_reset(). It doesn't need to read the rq clamp. And it can also avoid the problems described above.
---
change v2:
*add Fixes(Valentin Schneider); *ignore all rq clamp when idle (Valentin Schneider) --- kernel/sched/sched.h | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)
kernel/sched/sched.h +14 -7
@@ -2818,20 +2818,27 @@ static __always_inlineunsigned long uclamp_rq_util_with (struct rq *rq,unsigned long util,struct task_struct *p) { -unsigned long min_util; -unsigned long max_util; +unsigned long min_util =0 ; +unsigned long max_util =0 ;if (!static_branch_likely (&sched_uclamp_used))return util; - min_util =READ_ONCE (rq->uclamp[UCLAMP_MIN ].value); - max_util =READ_ONCE (rq->uclamp[UCLAMP_MAX ].value); -if (p) { - min_util =max (min_util,uclamp_eff_value (p,UCLAMP_MIN )); - max_util =max (max_util,uclamp_eff_value (p,UCLAMP_MAX )); + min_util =uclamp_eff_value (p,UCLAMP_MIN ); + max_util =uclamp_eff_value (p,UCLAMP_MAX ); + +/* +* Ignore last runnable task's max clamp, as this task will +* reset it. Similarly, no need to read the rq's min clamp. +*/ +if (rq->uclamp_flags &UCLAMP_FLAG_IDLE ) + goto out; } + min_util = max_t(unsigned long, min_util, READ_ONCE(rq->uclamp[UCLAMP_MIN].value)); + max_util = max_t(unsigned long, max_util, READ_ONCE(rq->uclamp[UCLAMP_MAX].value)); +out:/* * Since CPU's {min,max}_util clamps are MAX aggregated considering * RUNNABLE tasks with _different_ clamps, we can end up with an
Signature · 1 line
2.25.1
On the subject: s/ucalmp/uclamp/
Quoted 15 lines · Xuewen Yan
From: Xuewen Yan <[email protected]>
Now in
uclamp_rq_util_with(), when the task != NULL, theuclamp_maxas following:uc_rq_max= rq->uclamp[UCLAMP_MAX].value;uc_eff_max=uclamp_eff_value(p, UCLAMP_MAX);uclamp_max= max{uc_rq_max,uc_eff_max};Consider the following scenario:
(1)the rq is idle, theuc_rq_maxis last runnable task'sUCLAMP_MAX; (2)the p'suc_eff_max<uc_rq_max.As a result, the
uclamp_max=uc_rq_maxinstead ofuc_eff_max, it is unreasonable.The scenario often happens in
find_energy_efficient_cpu(), when the task has smallerUCLAMP_MAX.
Quoted 8 lines
When rq has
UCLAMP_FLAG_IDLEflag, enqueuing the task will liftUCLAMP_FLAG_IDLEand set the rq clamp as the task's viauclamp_idle_reset(). It doesn't need to read the rq clamp. And it can also avoid the problems described above.Fixes 9d20ad7dfc9a ("sched/uclamp: Add uclamp_util_with()")Signed-off-by Xuewen Yan <[email protected]>
Thanks!
Quoted 52 lines · Xuewen Yan
From: Xuewen Yan <[email protected]>
Now in
uclamp_rq_util_with(), when the task != NULL, theuclamp_maxas following:uc_rq_max= rq->uclamp[UCLAMP_MAX].value;uc_eff_max=uclamp_eff_value(p, UCLAMP_MAX);uclamp_max= max{uc_rq_max,uc_eff_max};Consider the following scenario:
(1)the rq is idle, theuc_rq_maxis last runnable task'sUCLAMP_MAX; (2)the p'suc_eff_max<uc_rq_max.As a result, the
uclamp_max=uc_rq_maxinstead ofuc_eff_max, it is unreasonable.The scenario often happens in
find_energy_efficient_cpu(), when the task has smallerUCLAMP_MAX.When rq has
UCLAMP_FLAG_IDLEflag, enqueuing the task will liftUCLAMP_FLAG_IDLEand set the rq clamp as the task's viauclamp_idle_reset(). It doesn't need to read the rq clamp. And it can also avoid the problems described above.Fixes 9d20ad7dfc9a ("sched/uclamp: Add uclamp_util_with()")Signed-off-by Xuewen Yan <[email protected]>---
change v2:*add Fixes(Valentin Schneider); *ignore all rq clamp when idle (Valentin Schneider) --- kernel/sched/sched.h | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)kernel/sched/sched.h +2
-6@@ -2818,20 +2818,27 @@ static __always_inlineunsigned long uclamp_rq_util_with (struct rq *rq,unsigned long util,struct task_struct *p) { -unsigned long min_util; -unsigned long max_util; +unsigned long min_util =0 ; +unsigned long max_util =0 ;if (!static_branch_likely (&sched_uclamp_used))return util; - min_util =READ_ONCE (rq->uclamp[UCLAMP_MIN ].value); - max_util =READ_ONCE (rq->uclamp[UCLAMP_MAX ].value); -if (p) { - min_util =max (min_util,uclamp_eff_value (p,UCLAMP_MIN ));
kernel/sched/sched.h static __always_inline · line 2836
Quoted 10 lines
kernel/sched/sched.h +9
-1- max_util =max (max_util,uclamp_eff_value (p,UCLAMP_MAX )); + min_util =uclamp_eff_value (p,UCLAMP_MIN ); + max_util =uclamp_eff_value (p,UCLAMP_MAX ); + +/* +* Ignore last runnable task's max clamp, as this task will +* reset it. Similarly, no need to read the rq's min clamp. +*/ +if (rq->uclamp_flags &UCLAMP_FLAG_IDLE ) + goto out;
We read rq->uclamp_flags without locks here. Me thinks this needs READ_ONCE(). But since we care only about a single bit, I can't see any risk for inconsistency, so we're fine.
Thanks!
--
Qais Yousef
Quoted 11 lines
kernel/sched/sched.h +3
-0} + min_util = max_t(unsigned long, min_util, READ_ONCE(rq->uclamp[UCLAMP_MIN].value)); + max_util = max_t(unsigned long, max_util, READ_ONCE(rq->uclamp[UCLAMP_MAX].value)); +out:/* * Since CPU's {min,max}_util clamps are MAX aggregated considering * RUNNABLE tasks with _different_ clamps, we can end up with an Signature · 1 line
2.25.1
Peter Zijlstra
maintainerQuoted 15 lines · Xuewen Yan
From: Xuewen Yan <[email protected]>
Now in
uclamp_rq_util_with(), when the task != NULL, theuclamp_maxas following:uc_rq_max= rq->uclamp[UCLAMP_MAX].value;uc_eff_max=uclamp_eff_value(p, UCLAMP_MAX);uclamp_max= max{uc_rq_max,uc_eff_max};Consider the following scenario:
(1)the rq is idle, theuc_rq_maxis last runnable task'sUCLAMP_MAX; (2)the p'suc_eff_max<uc_rq_max.As a result, the
uclamp_max=uc_rq_maxinstead ofuc_eff_max, it is unreasonable.The scenario often happens in
find_energy_efficient_cpu(), when the task has smallerUCLAMP_MAX.
Quoted 7 lines
When rq has
UCLAMP_FLAG_IDLEflag, enqueuing the task will liftUCLAMP_FLAG_IDLEand set the rq clamp as the task's viauclamp_idle_reset(). It doesn't need to read the rq clamp. And it can also avoid the problems described above.Fixes 9d20ad7dfc9a ("sched/uclamp: Add uclamp_util_with()")Signed-off-by Xuewen Yan <[email protected]>
Valentin, Qais, can either of you write a Changelog/comment for this, I can't seem to make any sense of it.
Is this about wake-from-idle, where the first task's uclamp goes amis because the rq->uclamp values haven't been updated yet?
Quoted 46 lines
--- kernel/sched/sched.h | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)kernel/sched/sched.h +14
-7@@ -2818,20 +2818,27 @@ static __always_inlineunsigned long uclamp_rq_util_with (struct rq *rq,unsigned long util,struct task_struct *p) { -unsigned long min_util; -unsigned long max_util; +unsigned long min_util =0 ; +unsigned long max_util =0 ;if (!static_branch_likely (&sched_uclamp_used))return util; - min_util =READ_ONCE (rq->uclamp[UCLAMP_MIN ].value); - max_util =READ_ONCE (rq->uclamp[UCLAMP_MAX ].value); -if (p) { - min_util =max (min_util,uclamp_eff_value (p,UCLAMP_MIN )); - max_util =max (max_util,uclamp_eff_value (p,UCLAMP_MAX )); + min_util =uclamp_eff_value (p,UCLAMP_MIN ); + max_util =uclamp_eff_value (p,UCLAMP_MAX ); + +/* +* Ignore last runnable task's max clamp, as this task will +* reset it. Similarly, no need to read the rq's min clamp. +*/ +if (rq->uclamp_flags &UCLAMP_FLAG_IDLE ) + goto out; } + min_util = max_t(unsigned long, min_util, READ_ONCE(rq->uclamp[UCLAMP_MIN].value)); + max_util = max_t(unsigned long, max_util, READ_ONCE(rq->uclamp[UCLAMP_MAX].value)); +out:/* * Since CPU's {min,max}_util clamps are MAX aggregated considering * RUNNABLE tasks with _different_ clamps, we can end up with an Signature · 1 line
2.25.1
Quoted 20 lines · Peter Zijlstra
Quoted 19 lines · Xuewen Yan
From: Xuewen Yan <[email protected]>
Now in
uclamp_rq_util_with(), when the task != NULL, theuclamp_maxas following:uc_rq_max= rq->uclamp[UCLAMP_MAX].value;uc_eff_max=uclamp_eff_value(p, UCLAMP_MAX);uclamp_max= max{uc_rq_max,uc_eff_max};Consider the following scenario:
(1)the rq is idle, theuc_rq_maxis last runnable task'sUCLAMP_MAX; (2)the p'suc_eff_max<uc_rq_max.As a result, the
uclamp_max=uc_rq_maxinstead ofuc_eff_max, it is unreasonable.The scenario often happens in
find_energy_efficient_cpu(), when the task has smallerUCLAMP_MAX.When rq has
UCLAMP_FLAG_IDLEflag, enqueuing the task will liftUCLAMP_FLAG_IDLEand set the rq clamp as the task's viauclamp_idle_reset(). It doesn't need to read the rq clamp. And it can also avoid the problems described above.
Quoted 6 lines
Quoted 3 lines
Fixes 9d20ad7dfc9a ("sched/uclamp: Add uclamp_util_with()")Signed-off-by Xuewen Yan <[email protected]>Valentin, Qais, can either of you write a Changelog/comment for this, I can't seem to make any sense of it.
Err, yeah I think I've been staring at uclamp for too long. It could be clearer.
Quoted 3 lines
Is this about wake-from-idle, where the first task's uclamp goes amis because the
rq->uclampvalues haven't been updated yet?
Yep. How about the below?
--->8---
sched/uclamp: Ignore max aggregation if rq is idle
When a task wakes up on an idle rq, uclamp_rq_util_with() would max aggregate with rq value. But since there is no task enqueued yet, the values are stale based on the last task that was running. When the new task actually wakes up and enqueued, then the rq uclamp values should reflect that of the newly woken up task effective uclamp values.
This is a problem particularly for uclamp_max because it default to 1024. If a task p with uclamp_max = 512 wakes up, then max aggregation would ignore the capping that should apply when this task is enqueued, which is wrong.
Fix that by ignoring max aggregation if the rq is idle since in that case the effective uclamp value of the rq will be the ones of the task that will wake up.
--->8---
Thanks
--
Qais Yousef
Peter Zijlstra
maintainerQuoted 17 lines · Qais Yousef
Yep. How about the below?
--->8---
sched/uclamp: Ignore max aggregation if rq is idle
When a task wakes up on an idle rq,
uclamp_rq_util_with()would max aggregate with rq value. But since there is no task enqueued yet, the values are stale based on the last task that was running. When the new task actually wakes up and enqueued, then the rq uclamp values should reflect that of the newly woken up task effective uclamp values.This is a problem particularly for
uclamp_maxbecause it default to 1024. If a task p withuclamp_max= 512 wakes up, then max aggregation would ignore the capping that should apply when this task is enqueued, which is wrong.
Quoted 5 lines
Fix that by ignoring max aggregation if the rq is idle since in that case the effective uclamp value of the rq will be the ones of the task that will wake up.
--->8---
Much better, I've updated it. Thanks!
Quoted 5 lines · Qais Yousef
sched/uclamp: Ignore max aggregation if rq is idle
When a task wakes up on an idle rq,
uclamp_rq_util_with()would max aggregate with rq value. But since there is no task enqueued yet, the values are stale based on the last task that was running. When the new
Nit: those values are "intentionally stale" for UCLAMP_MAX, per
e496187da710 ("sched/uclamp: Enforce last task's UCLAMP_MAX")
for UCLAMP_MIN we'll set uclamp_none(UCLAMP_MIN) == 0 upon dequeueing the last runnable task, which DTRT.
Quoted 4 lines
task actually wakes up and enqueued, then the rq uclamp values should reflect that of the newly woken up task effective uclamp values.
This is a problem particularly for
uclamp_maxbecause it default to
^^^^^^^^^^^^
Per the above, it's "only" a problem for UCLAMP_MAX.
Quoted 8 lines
1024. If a task p with
uclamp_max= 512 wakes up, then max aggregation would ignore the capping that should apply when this task is enqueued, which is wrong.Fix that by ignoring max aggregation if the rq is idle since in that case the effective uclamp value of the rq will be the ones of the task that will wake up.
Quoted 15 lines · Valentin Schneider
Quoted 5 lines · Qais Yousef
sched/uclamp: Ignore max aggregation if rq is idle
When a task wakes up on an idle rq,
uclamp_rq_util_with()would max aggregate with rq value. But since there is no task enqueued yet, the values are stale based on the last task that was running. When the newNit: those values are "intentionally stale" for
UCLAMP_MAX, per
e496187da710("sched/uclamp: Enforce last task'sUCLAMP_MAX")for
UCLAMP_MINwe'll setuclamp_none(UCLAMP_MIN)== 0 upon dequeueing the last runnable task, which DTRT.
Quoted 15 lines
Quoted 4 lines
task actually wakes up and enqueued, then the rq uclamp values should reflect that of the newly woken up task effective uclamp values.
This is a problem particularly for
uclamp_maxbecause it default to^^^^^^^^^^^^Per the above, it's "only" a problem for
UCLAMP_MAX.Quoted 8 lines
1024. If a task p with
uclamp_max= 512 wakes up, then max aggregation would ignore the capping that should apply when this task is enqueued, which is wrong.Fix that by ignoring max aggregation if the rq is idle since in that case the effective uclamp value of the rq will be the ones of the task that will wake up.
Thanks!
xuewen
[tip: sched/urgent] sched/uclamp: Ignore max aggregation if rq is idle
In reply to Xuewen YanThe following commit has been merged into the sched/urgent branch of tip:
The following commit has been merged into the sched/urgent branch of tip:
Commit-ID: 3e1493f46390618ea78607cb30c58fc19e2a5035 Gitweb: https://git.kernel.org/tip/3e1493f46390618ea78607cb30c58fc19e2a5035 Author: Xuewen Yan <[email protected]> AuthorDate: Wed, 30 Jun 2021 22:12:04 +08:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Fri, 02 Jul 2021 15:58:24 +02:00
sched/uclamp: Ignore max aggregation if rq is idle
When a task wakes up on an idle rq, uclamp_rq_util_with() would max aggregate with rq value. But since there is no task enqueued yet, the values are stale based on the last task that was running. When the new task actually wakes up and enqueued, then the rq uclamp values should reflect that of the newly woken up task effective uclamp values.
This is a problem particularly for uclamp_max because it default to 1024. If a task p with uclamp_max = 512 wakes up, then max aggregation would ignore the capping that should apply when this task is enqueued, which is wrong.
Fix that by ignoring max aggregation if the rq is idle since in that case the effective uclamp value of the rq will be the ones of the task that will wake up.
--- kernel/sched/sched.h | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)
kernel/sched/sched.h +14 -7
@@ -2818,20 +2818,27 @@ static __always_inlineunsigned long uclamp_rq_util_with (struct rq *rq,unsigned long util,struct task_struct *p) { -unsigned long min_util; -unsigned long max_util; +unsigned long min_util =0 ; +unsigned long max_util =0 ;if (!static_branch_likely (&sched_uclamp_used))return util; - min_util =READ_ONCE (rq->uclamp[UCLAMP_MIN ].value); - max_util =READ_ONCE (rq->uclamp[UCLAMP_MAX ].value); -if (p) { - min_util =max (min_util,uclamp_eff_value (p,UCLAMP_MIN )); - max_util =max (max_util,uclamp_eff_value (p,UCLAMP_MAX )); + min_util =uclamp_eff_value (p,UCLAMP_MIN ); + max_util =uclamp_eff_value (p,UCLAMP_MAX ); + +/* +* Ignore last runnable task's max clamp, as this task will +* reset it. Similarly, no need to read the rq's min clamp. +*/ +if (rq->uclamp_flags &UCLAMP_FLAG_IDLE ) + goto out; } + min_util = max_t(unsigned long, min_util, READ_ONCE(rq->uclamp[UCLAMP_MIN].value)); + max_util = max_t(unsigned long, max_util, READ_ONCE(rq->uclamp[UCLAMP_MAX].value)); +out:/* * Since CPU's {min,max}_util clamps are MAX aggregated considering * RUNNABLE tasks with _different_ clamps, we can end up with an