Skip to contentSkip to conversation

Patch "sched/fair: sanitize vruntime of entity being placed" has been added to the 4.14-stable tree

DISCUSSION Zhang Qiao 2 participants 8 messages Mar 6, 2023 – Mar 28, 2023 1 message not in the archive Stable branch

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

Zhang Qiao

author
#1 lore raw
Quoted 11 lines · Sasha Levin

This is a note to let you know that I've just added the patch titled

sched/fair: sanitize vruntime of entity being placed

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
sched-fair-sanitize-vruntime-of-entity-being-placed.patch
and it can be found in the queue-4.14 subdirectory.

Quoted 6 lines

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

commit 38247e1de3305a6ef644404ac818bc6129440eae

Hi,
This patch has significant impact on the hackbench.throughput [1]. Please don't backport this patch.

[1] https://lore.kernel.org/lkml/[email protected]/T/#u

Thanks.
Zhang Qiao.

Quoted 64 lines
Author: Zhang Qiao <[email protected]>
Date:   Mon Jan 30 13:22:16 2023 +0100

sched/fair: sanitize vruntime of entity being placed

[ Upstream commit 829c1651e9c4a6f78398d3e67651cef9bb6b42cc ]

When a scheduling entity is placed onto cfs_rq, its vruntime is pulled to the base level (around cfs_rq->min_vruntime), so that the entity doesn't gain extra boost when placed backwards.

However, if the entity being placed wasn't executed for a long time, its vruntime may get too far behind (e.g. while cfs_rq was executing a low-weight hog), which can inverse the vruntime comparison due to s64 overflow. This results in the entity being placed with its original vruntime way forwards, so that it will effectively never get to the cpu.

To prevent that, ignore the vruntime of the entity being placed if it didn't execute for much longer than the characteristic sheduler time scale.

kernel/sched/fair.c +13 -2
index 3ff60230710c9..afa21e43477fa 100644
@@ -3615,6 +3615,7 @@ static void
 place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
 {
 	u64 vruntime = cfs_rq->min_vruntime;
+	u64 sleep_time;
 
 	/*
 	 * The 'current' period is already promised to the current tasks,
@@ -3639,8 +3640,18 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
 		vruntime -= thresh;
 	}
 
-	/* ensure we never gain time by being placed backwards. */
-	se->vruntime = max_vruntime(se->vruntime, vruntime);
+	/*
+	 * Pull vruntime of the entity being placed to the base level of
+	 * cfs_rq, to prevent boosting it if placed backwards.  If the entity
+	 * slept for a long time, don't even try to compare its vruntime with
+	 * the base as it may be too far off and the comparison may get
+	 * inversed due to s64 overflow.
+	 */
+	sleep_time = rq_clock_task(rq_of(cfs_rq)) - se->exec_start;
+	if ((s64)sleep_time > 60LL * NSEC_PER_SEC)
+		se->vruntime = vruntime;
+	else
+		se->vruntime = max_vruntime(se->vruntime, vruntime);
 }
 
 static void check_enqueue_throttle(struct cfs_rq *cfs_rq);

.

maintainer
#2 lore raw
Quoted 19 lines · Zhang Qiao
Quoted 16 lines · Sasha Levin

This is a note to let you know that I've just added the patch titled

sched/fair: sanitize vruntime of entity being placed

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
sched-fair-sanitize-vruntime-of-entity-being-placed.patch
and it can be found in the queue-4.14 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 7 lines
Quoted 1 line

commit 38247e1de3305a6ef644404ac818bc6129440eae

Hi,
This patch has significant impact on the hackbench.throughput [1]. Please don't backport this patch.

[1] https://lore.kernel.org/lkml/[email protected]/T/#u

This link says it made hackbench.throughput faster, not slower, so why would we NOT want it?

confused,

greg k-h

author
#3 lore raw
Quoted 22 lines · Greg KH
Quoted 21 lines · Zhang Qiao
Quoted 17 lines · Sasha Levin

This is a note to let you know that I've just added the patch titled

sched/fair: sanitize vruntime of entity being placed

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
sched-fair-sanitize-vruntime-of-entity-being-placed.patch
and it can be found in the queue-4.14 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

commit 38247e1de3305a6ef644404ac818bc6129440eae

Quoted 8 lines
Quoted 5 lines

Hi,
This patch has significant impact on the hackbench.throughput [1]. Please don't backport this patch.

[1] https://lore.kernel.org/lkml/[email protected]/T/#u

This link says it made hackbench.throughput faster, not slower, so why would we NOT want it?

Please see this section. In some cases, this patch reset task's vruntime by mistake and will lead to wrong results.

Quoted 13 lines · kernel test robot

FYI, In addition to that, the commit also has significant impact on the following tests:

+------------------+--------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -8.1% regression |
| test machine     | 104 threads 2 sockets (Skylake) with 192G memory |
| test parameters  | cpufreq_governor=performance                     |
|                  | ipc=socket                                       |
|                  | iterations=4                                     |
|                  | mode=process                                     |
|                  | nr_threads=100%                                  |
+------------------+--------------------------------------------------+
Quoted 1 line

Details are as below:

Thanks.
Zhang Qiao.

Quoted 6 lines

confused,

greg k-h
.

maintainer
#4 lore raw
Quoted 31 lines · Zhang Qiao
Quoted 28 lines · Greg KH
Quoted 26 lines · Zhang Qiao
Quoted 11 lines · Sasha Levin

This is a note to let you know that I've just added the patch titled

sched/fair: sanitize vruntime of entity being placed

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
sched-fair-sanitize-vruntime-of-entity-being-placed.patch
and it can be found in the queue-4.14 subdirectory.

Quoted 6 lines

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

commit 38247e1de3305a6ef644404ac818bc6129440eae

Hi,
This patch has significant impact on the hackbench.throughput [1]. Please don't backport this patch.

[1] https://lore.kernel.org/lkml/[email protected]/T/#u

Quoted 22 lines
Quoted 2 lines

This link says it made hackbench.throughput faster, not slower, so why would we NOT want it?

Please see this section. In some cases, this patch reset task's vruntime by mistake and will lead to wrong results.

Quoted 14 lines · kernel test robot

FYI, In addition to that, the commit also has significant impact on the following tests:

+------------------+--------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -8.1% regression |
| test machine     | 104 threads 2 sockets (Skylake) with 192G memory |
| test parameters  | cpufreq_governor=performance                     |
|                  | ipc=socket                                       |
|                  | iterations=4                                     |
|                  | mode=process                                     |
|                  | nr_threads=100%                                  |
+------------------+--------------------------------------------------+

Details are as below:

So one benchmark did better, by a lot, and one did less, by a little? Which one matters "more"?

So Linus's tree now has a regression? Or not? I'm confused. We are just matching what is in Linus's tree, if it's wrong here, in a stable tree, it should be wrong there too. If not, please explain why not?

thanks,

greg k-h

author
#5 lore raw
Quoted 53 lines · Greg KH
Quoted 52 lines · Zhang Qiao
Quoted 22 lines · Greg KH
Quoted 21 lines · Zhang Qiao
Quoted 17 lines · Sasha Levin

This is a note to let you know that I've just added the patch titled

sched/fair: sanitize vruntime of entity being placed

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
sched-fair-sanitize-vruntime-of-entity-being-placed.patch
and it can be found in the queue-4.14 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

commit 38247e1de3305a6ef644404ac818bc6129440eae

Quoted 8 lines
Quoted 5 lines

Hi,
This patch has significant impact on the hackbench.throughput [1]. Please don't backport this patch.

[1] https://lore.kernel.org/lkml/[email protected]/T/#u

This link says it made hackbench.throughput faster, not slower, so why would we NOT want it?

Please see this section. In some cases, this patch reset task's vruntime by mistake and will lead to wrong results.

Quoted 13 lines · kernel test robot

FYI, In addition to that, the commit also has significant impact on the following tests:

+------------------+--------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -8.1% regression |
| test machine     | 104 threads 2 sockets (Skylake) with 192G memory |
| test parameters  | cpufreq_governor=performance                     |
|                  | ipc=socket                                       |
|                  | iterations=4                                     |
|                  | mode=process                                     |
|                  | nr_threads=100%                                  |
+------------------+--------------------------------------------------+
Quoted 6 lines
Quoted 1 line
Quoted 1 line

Details are as below:

So one benchmark did better, by a lot, and one did less, by a little? Which one matters "more">

So Linus's tree now has a regression? Or not? I'm confused. We are

Yes, Linus's tree also has a regression, and i have sent a patch[1] for fix this regression.

[1]: https://lore.kernel.org/lkml/[email protected]/

thanks.
Zhang qiao.

Quoted 10 lines

just matching what is in Linus's tree, if it's wrong here, in a stable tree, it should be wrong there too. If not, please explain why not?

thanks,

greg k-h
.

maintainer
#6 lore raw
Quoted 63 lines · Zhang Qiao
Quoted 53 lines · Greg KH
Quoted 52 lines · Zhang Qiao
Quoted 22 lines · Greg KH
Quoted 21 lines · Zhang Qiao
Quoted 17 lines · Sasha Levin

This is a note to let you know that I've just added the patch titled

sched/fair: sanitize vruntime of entity being placed

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
sched-fair-sanitize-vruntime-of-entity-being-placed.patch
and it can be found in the queue-4.14 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

commit 38247e1de3305a6ef644404ac818bc6129440eae

Quoted 8 lines
Quoted 5 lines

Hi,
This patch has significant impact on the hackbench.throughput [1]. Please don't backport this patch.

[1] https://lore.kernel.org/lkml/[email protected]/T/#u

This link says it made hackbench.throughput faster, not slower, so why would we NOT want it?

Please see this section. In some cases, this patch reset task's vruntime by mistake and will lead to wrong results.

Quoted 13 lines · kernel test robot

FYI, In addition to that, the commit also has significant impact on the following tests:

+------------------+--------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -8.1% regression |
| test machine     | 104 threads 2 sockets (Skylake) with 192G memory |
| test parameters  | cpufreq_governor=performance                     |
|                  | ipc=socket                                       |
|                  | iterations=4                                     |
|                  | mode=process                                     |
|                  | nr_threads=100%                                  |
+------------------+--------------------------------------------------+
Quoted 6 lines
Quoted 1 line
Quoted 1 line

Details are as below:

So one benchmark did better, by a lot, and one did less, by a little? Which one matters "more">

So Linus's tree now has a regression? Or not? I'm confused. We are

Quoted 7 lines

Yes, Linus's tree also has a regression, and i have sent a patch[1] for fix this regression.

[1]: https://lore.kernel.org/lkml/[email protected]/

thanks.
Zhang qiao.

Ok, I've dropped this from all stable queues now. Please let us know when we can pick it up again and what the fixup commit id in Linus's tree is when it lands there.

thanks,

greg k-h

author
#7 lore raw
Quoted 67 lines · Greg KH
Quoted 66 lines · Zhang Qiao
Quoted 53 lines · Greg KH
Quoted 52 lines · Zhang Qiao
Quoted 22 lines · Greg KH
Quoted 21 lines · Zhang Qiao
Quoted 17 lines · Sasha Levin

This is a note to let you know that I've just added the patch titled

sched/fair: sanitize vruntime of entity being placed

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
sched-fair-sanitize-vruntime-of-entity-being-placed.patch
and it can be found in the queue-4.14 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

commit 38247e1de3305a6ef644404ac818bc6129440eae

Quoted 8 lines
Quoted 5 lines

Hi,
This patch has significant impact on the hackbench.throughput [1]. Please don't backport this patch.

[1] https://lore.kernel.org/lkml/[email protected]/T/#u

This link says it made hackbench.throughput faster, not slower, so why would we NOT want it?

Please see this section. In some cases, this patch reset task's vruntime by mistake and will lead to wrong results.

Quoted 13 lines · kernel test robot

FYI, In addition to that, the commit also has significant impact on the following tests:

+------------------+--------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -8.1% regression |
| test machine     | 104 threads 2 sockets (Skylake) with 192G memory |
| test parameters  | cpufreq_governor=performance                     |
|                  | ipc=socket                                       |
|                  | iterations=4                                     |
|                  | mode=process                                     |
|                  | nr_threads=100%                                  |
+------------------+--------------------------------------------------+
Quoted 6 lines
Quoted 1 line
Quoted 1 line

Details are as below:

So one benchmark did better, by a lot, and one did less, by a little? Which one matters "more">

So Linus's tree now has a regression? Or not? I'm confused. We are

Yes, Linus's tree also has a regression, and i have sent a patch[1] for fix this regression.

Quoted 8 lines
Quoted 4 lines

[1]: https://lore.kernel.org/lkml/[email protected]/

thanks.
Zhang qiao.

Ok, I've dropped this from all stable queues now. Please let us know when we can pick it up again and what the fixup commit id in Linus's tree is when it lands there.

Hi,

The fixup patch has been merged into Linus's tree, its commit id is:
a53ce18cacb477dd0513c607f187d16f0fa96f71 ("sched/fair: Sanitize vruntime of entity being migrated")

thanks,
Zhangqiao.

Quoted 6 lines

thanks,

greg k-h
.

maintainer
#8 lore raw
Quoted 75 lines · Zhang Qiao
Quoted 72 lines · Greg KH
Quoted 70 lines · Zhang Qiao
Quoted 53 lines · Greg KH
Quoted 52 lines · Zhang Qiao
Quoted 22 lines · Greg KH
Quoted 21 lines · Zhang Qiao
Quoted 17 lines · Sasha Levin

This is a note to let you know that I've just added the patch titled

sched/fair: sanitize vruntime of entity being placed

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
sched-fair-sanitize-vruntime-of-entity-being-placed.patch
and it can be found in the queue-4.14 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

commit 38247e1de3305a6ef644404ac818bc6129440eae

Quoted 8 lines
Quoted 5 lines

Hi,
This patch has significant impact on the hackbench.throughput [1]. Please don't backport this patch.

[1] https://lore.kernel.org/lkml/[email protected]/T/#u

This link says it made hackbench.throughput faster, not slower, so why would we NOT want it?

Please see this section. In some cases, this patch reset task's vruntime by mistake and will lead to wrong results.

Quoted 13 lines · kernel test robot

FYI, In addition to that, the commit also has significant impact on the following tests:

+------------------+--------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -8.1% regression |
| test machine     | 104 threads 2 sockets (Skylake) with 192G memory |
| test parameters  | cpufreq_governor=performance                     |
|                  | ipc=socket                                       |
|                  | iterations=4                                     |
|                  | mode=process                                     |
|                  | nr_threads=100%                                  |
+------------------+--------------------------------------------------+
Quoted 6 lines
Quoted 1 line
Quoted 1 line

Details are as below:

So one benchmark did better, by a lot, and one did less, by a little? Which one matters "more">

So Linus's tree now has a regression? Or not? I'm confused. We are

Yes, Linus's tree also has a regression, and i have sent a patch[1] for fix this regression.

[1]: https://lore.kernel.org/lkml/[email protected]/

thanks.
Zhang qiao.

Quoted 8 lines
Quoted 3 lines

Ok, I've dropped this from all stable queues now. Please let us know when we can pick it up again and what the fixup commit id in Linus's tree is when it lands there.

Hi,

The fixup patch has been merged into Linus's tree, its commit id is:
a53ce18cacb477dd0513c607f187d16f0fa96f71 ("sched/fair: Sanitize vruntime of entity being migrated")

Wonderful, I have queued both of these up now, thank you for letting me know.

greg k-h