sched/pelt: Fix the attach_entity_load_avg calculate method
Subsystems (2)
PATCH v3 1/1 sched/pelt: Fix the attach_entity_load_avg calculate method
From: kuyo chang <[email protected]>
I meet the warning message at cfs_rq_is_decayed at below code.
SCHED_WARN_ON(cfs_rq->avg.load_avg ||
cfs_rq->avg.util_avg || cfs_rq->avg.runnable_avg)
Following is the calltrace.
Call trace:
__update_blocked_fairupdate_blocked_averagesnewidle_balancepick_next_task_fair__schedule
schedulepipe_readvfs_readksys_read
After code analyzing and some debug messages, I found it exits a corner case at attach_entity_load_avg which will cause load_sum is null but load_avg is not.
Consider se_weight is 88761 according by sched_prio_to_weight table. And assume the get_pelt_divider() is 47742, se->avg.load_avg is 1. By the calculating for se->avg.load_sum as following will become zero as following.se->avg.load_sum =
div_u64(se->avg.load_avg * se->avg.load_sum, se_weight(se));
se->avg.load_sum = 1*47742/88761 = 0.
After enqueue_load_avg code as below. cfs_rq->avg.load_avg += se->avg.load_avg; cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
Then the load_sum for cfs_rq will be 1 while the load_sum for cfs_rq is 0. So it will hit the warning message.
In order to fix the corner case, make sure the se->load_avg|sum is correct before enqueue_load_avg.
After long time testing, the kernel warning was gone and the system runs as well as before.
---
v1->v2:
(1)Thanks for suggestion from Peter Zijlstra & Vincent Guittot. (2)By suggestion from Vincent Guittot,
rework the se->load_sum calculation method for fix the corner case, make sure the se->load_avg|sum is correct before enqueue_load_avg. (3)Rework changlog.
v2->v3:
(1)Rename Subject.
(1)Add fix tag.
kernel/sched/fair.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
kernel/sched/fair.c +5 -3
@@ -3829,10 +3829,12 @@ static void attach_entity_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *s se->avg.runnable_sum = se->avg.runnable_avg * divider; - se->avg.load_sum = divider; -if (se_weight (se)) { + se->avg.load_sum = se->avg.load_avg * divider; +if (se_weight (se) < se->avg.load_sum) { se->avg.load_sum = -div_u64 (se->avg.load_avg * se->avg.load_sum,se_weight (se)); +div_u64 (se->avg.load_sum,se_weight (se)); + }else { + se->avg.load_sum =1 ; }enqueue_load_avg (cfs_rq, se);
Signature · 1 line
2.18.0
I've taken the liberty of carrying over the tags from v2 and reworked the Changelog a little.
---
Subject: sched/pelt: Fix attach_entity_load_avg() corner case
From: kuyo chang <[email protected]>
Date: Thu, 14 Apr 2022 17:02:20 +0800
From: kuyo chang <[email protected]>
The warning in cfs_rq_is_decayed() triggered:
SCHED_WARN_ON(cfs_rq->avg.load_avg || cfs_rq->avg.util_avg || cfs_rq->avg.runnable_avg)
There exists a corner case in attach_entity_load_avg() which will cause load_sum to be zero while load_avg will not be.
Consider se_weight is 88761 as per the sched_prio_to_weight[] table. Further assume the get_pelt_divider() is 47742, this gives:se->avg.load_avg is 1.
However, calculating load_sum results in 0:
se->avg.load_sum = div_u64(se->avg.load_avg * se->avg.load_sum, se_weight(se)); se->avg.load_sum = 1*47742/88761 = 0.
Then enqueue_load_avg() adds this to the cfs_rq totals:
cfs_rq->avg.load_avg += se->avg.load_avg; cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
Resulting in load_avg being 1 with load_sum is 0, which will trigger the WARN.
--- kernel/sched/fair.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
kernel/sched/fair.c +5 -5
@@ -3829,11 +3829,11 @@ static void attach_entity_load_avg(struc se->avg.runnable_sum = se->avg.runnable_avg * divider; - se->avg.load_sum = divider; -if (se_weight (se)) { - se->avg.load_sum = -div_u64 (se->avg.load_avg * se->avg.load_sum,se_weight (se)); - } + se->avg.load_sum = se->avg.load_avg * divider; +if (se_weight (se) < se->avg.load_sum) + se->avg.load_sum =div_u64 (se->avg.load_sum,se_weight (se)); +else + se->avg.load_sum =1 ;enqueue_load_avg (cfs_rq, se); cfs_rq->avg.util_avg += se->avg.util_avg;
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Quoted 2 lines · Peter Zijlstra
I've taken the liberty of carrying over the tags from v2 and reworked the Changelog a little.
Thank you for all your assistance.
Quoted 69 lines
---
Subject: sched/pelt: Fixattach_entity_load_avg()corner case
From: kuyo chang <[email protected]>
Date: Thu, 14 Apr 2022 17:02:20 +0800From: kuyo chang <[email protected]>
The warning in
cfs_rq_is_decayed()triggered:SCHED_WARN_ON(cfs_rq->avg.load_avg || cfs_rq->avg.util_avg || cfs_rq->avg.runnable_avg)There exists a corner case in
attach_entity_load_avg()which will causeload_sumto be zero whileload_avgwill not be.Consider
se_weightis 88761 as per the sched_prio_to_weight[] table. Further assume theget_pelt_divider()is 47742, this gives:se->avg.load_avgis 1.However, calculating
load_sumresults in 0:
se->avg.load_sum= div_u64(se->avg.load_avg *se->avg.load_sum,se_weight(se));
se->avg.load_sum= 1*47742/88761 = 0.Then
enqueue_load_avg()adds this to thecfs_rqtotals:cfs_rq->avg.load_avg += se->avg.load_avg; cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;Resulting in
load_avgbeing 1 withload_sumis 0, which will trigger the WARN.Fixes f207934fb79d ("sched/fair: Align PELT windows between cfs_rqand its se")
Signed-off-by kuyo chang <[email protected]>[peterz: massage changelog]Signed-off-by Peter Zijlstra (Intel) <[email protected]>Reviewed-by Vincent Guittot <[email protected]>Tested-by Dietmar Eggemann <[email protected]>--- kernel/sched/fair.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)kernel/sched/fair.c +5
-5@@ -3829,11 +3829,11 @@ static void attach_entity_load_avg(struc se->avg.runnable_sum = se->avg.runnable_avg * divider; - se->avg.load_sum = divider; -if (se_weight (se)) { - se->avg.load_sum = -div_u64 (se->avg.load_avg * se->avg.load_sum, se_weight(se)); - } + se->avg.load_sum = se->avg.load_avg * divider; +if (se_weight (se) < se->avg.load_sum) + se->avg.load_sum =div_u64 (se->avg.load_sum,se_weight (se)); +else + se->avg.load_sum =1 ;kernel/sched/fair.c +0
-0enqueue_load_avg (cfs_rq, se); cfs_rq->avg.util_avg += se->avg.util_avg;
[tip: sched/urgent] sched/pelt: Fix attach_entity_load_avg() corner case
In reply to Kuyo ChangThe following commit has been merged into the sched/urgent branch of tip:
The following commit has been merged into the sched/urgent branch of tip:
Commit-ID: 40f5aa4c5eaebfeaca4566217cb9c468e28ed682 Gitweb: https://git.kernel.org/tip/40f5aa4c5eaebfeaca4566217cb9c468e28ed682 Author: kuyo chang <[email protected]> AuthorDate: Thu, 14 Apr 2022 17:02:20 +08:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Tue, 19 Apr 2022 21:15:41 +02:00
sched/pelt: Fix attach_entity_load_avg() corner case
The warning in cfs_rq_is_decayed() triggered:
SCHED_WARN_ON(cfs_rq->avg.load_avg || cfs_rq->avg.util_avg || cfs_rq->avg.runnable_avg)
There exists a corner case in attach_entity_load_avg() which will cause load_sum to be zero while load_avg will not be.
Consider se_weight is 88761 as per the sched_prio_to_weight[] table. Further assume the get_pelt_divider() is 47742, this gives:se->avg.load_avg is 1.
However, calculating load_sum:
se->avg.load_sum = div_u64(se->avg.load_avg * se->avg.load_sum, se_weight(se)); se->avg.load_sum = 1*47742/88761 = 0.
Then enqueue_load_avg() adds this to the cfs_rq totals:
cfs_rq->avg.load_avg += se->avg.load_avg; cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
Resulting in load_avg being 1 with load_sum is 0, which will trigger the WARN.
--- kernel/sched/fair.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
kernel/sched/fair.c +5 -5
@@ -3829,11 +3829,11 @@ static void attach_entity_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *s se->avg.runnable_sum = se->avg.runnable_avg * divider; - se->avg.load_sum = divider; -if (se_weight (se)) { - se->avg.load_sum = -div_u64 (se->avg.load_avg * se->avg.load_sum,se_weight (se)); - } + se->avg.load_sum = se->avg.load_avg * divider; +if (se_weight (se) < se->avg.load_sum) + se->avg.load_sum =div_u64 (se->avg.load_sum,se_weight (se)); +else + se->avg.load_sum =1 ;enqueue_load_avg (cfs_rq, se); cfs_rq->avg.util_avg += se->avg.util_avg;