Skip to contentSkip to conversation

sched/fair: Optimize select_idle_core

PATCH Applied Srikar Dronamraju 4 participants 5 messages Dec 6, 2019 – Jan 29, 2020 Scheduler

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 sched/fair: Optimize select_idle_core

Currently we loop through all threads of a core to evaluate if the core is idle or not. This is unnecessary. If a thread of a core is not idle, skip evaluating other threads of a core. Also while clearing the cpumask, bits of all CPUs of a core can be cleared in one-shot.

Collecting ticks on a Power 9 SMT 8 system around select_idle_core while running schbench shows us

(units are in ticks, hence lesser is better)
Without patch
N Min Max Median Avg Stddev
x 130 151 1083 284 322.72308 144.41494

With patch
N Min Max Median Avg Stddev Improvement
x 164 88 610 201 225.79268 106.78943 30.03%

---
Changelog v1->v2:
Updated patch description with measurements made from a debug patch.

 kernel/sched/fair.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
kernel/sched/fair.c +4 -2
index 69a81a5709ff..b9d628128cfc 100644
@@ -5872,10 +5872,12 @@ static int select_idle_core(struct task_struct *p, struct sched_domain *sd, int
 		bool idle = true;
 
 		for_each_cpu(cpu, cpu_smt_mask(core)) {
-			__cpumask_clear_cpu(cpu, cpus);
-			if (!available_idle_cpu(cpu))
+			if (!available_idle_cpu(cpu)) {
 				idle = false;
+				break;
+			}
 		}
+		cpumask_andnot(cpus, cpus, cpu_smt_mask(core));
 
 		if (idle)
 			return core;
Signature · 1 line
2.18.1
Acked-by on v2 1/1
#1 lore raw
Quoted 14 lines · Srikar Dronamraju

Currently we loop through all threads of a core to evaluate if the core is idle or not. This is unnecessary. If a thread of a core is not idle, skip evaluating other threads of a core. Also while clearing the cpumask, bits of all CPUs of a core can be cleared in one-shot.

Collecting ticks on a Power 9 SMT 8 system around select_idle_core while running schbench shows us

(units are in ticks, hence lesser is better)
Without patch
N Min Max Median Avg Stddev
x 130 151 1083 284 322.72308 144.41494

Quoted 8 lines

With patch
N Min Max Median Avg Stddev Improvement
x 164 88 610 201 225.79268 106.78943 30.03%

I'm a bit surprised to not see this in linux-next or tip. Did this get rejected or did it accidentally get overlooked because the subject is so similar to 60588bfa223f ("sched/fair: Optimize select_idle_cpu") ?

Signature · 2 lines
Mel Gorman
SUSE Labs
#2 lore raw
Quoted 7 lines · Mel Gorman

I'm a bit surprised to not see this in linux-next or tip. Did this get rejected or did it accidentally get overlooked because the subject is so similar to 60588bfa223f ("sched/fair: Optimize select_idle_cpu") ?

Most likely the latter; I also had completely forgotten about it and had to retrace where and when I reviewed that...

#3 lore raw
In reply to Srikar Dronamraju

Hey Peter, Ingo,

Quoted 22 lines · Srikar Dronamraju

Currently we loop through all threads of a core to evaluate if the core is idle or not. This is unnecessary. If a thread of a core is not idle, skip evaluating other threads of a core. Also while clearing the cpumask, bits of all CPUs of a core can be cleared in one-shot.

Collecting ticks on a Power 9 SMT 8 system around select_idle_core while running schbench shows us

(units are in ticks, hence lesser is better)
Without patch
N Min Max Median Avg Stddev
x 130 151 1083 284 322.72308 144.41494

With patch
N Min Max Median Avg Stddev Improvement
x 164 88 610 201 225.79268 106.78943 30.03%

Quoted 6 lines

---
Changelog v1->v2:
Updated patch description with measurements made from a debug patch.

 kernel/sched/fair.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

Do you have any comments or improvements to be done for this patch?

Signature · 2 lines
Thanks and Regards
Srikar Dronamraju
#4 lore raw

[tip: sched/core] sched/fair: Optimize select_idle_core()

In reply to Srikar Dronamraju
The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     bec2860a2bd6cd38ea34434d04f4033eb32f0f31
Gitweb:        https://git.kernel.org/tip/bec2860a2bd6cd38ea34434d04f4033eb32f0f31
Author:        Srikar Dronamraju <[email protected]>
AuthorDate:    Fri, 06 Dec 2019 22:54:22 +05:30
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Tue, 28 Jan 2020 21:37:08 +01:00

sched/fair: Optimize select_idle_core()

Currently we loop through all threads of a core to evaluate if the core is idle or not. This is unnecessary. If a thread of a core is not idle, skip evaluating other threads of a core. Also while clearing the cpumask, bits of all CPUs of a core can be cleared in one-shot.

Collecting ticks on a Power 9 SMT 8 system around select_idle_core while running schbench shows us

(units are in ticks, hence lesser is better)
Without patch
N Min Max Median Avg Stddev
x 130 151 1083 284 322.72308 144.41494

With patch
N Min Max Median Avg Stddev Improvement
x 164 88 610 201 225.79268 106.78943 30.03%

---
 kernel/sched/fair.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
kernel/sched/fair.c +4 -2
index 25dffc0..1a0ce83 100644
@@ -5787,10 +5787,12 @@ static int select_idle_core(struct task_struct *p, struct sched_domain *sd, int 
 		bool idle = true;
 
 		for_each_cpu(cpu, cpu_smt_mask(core)) {
-			__cpumask_clear_cpu(cpu, cpus);
-			if (!available_idle_cpu(cpu))
+			if (!available_idle_cpu(cpu)) {
 				idle = false;
+				break;
+			}
 		}
+		cpumask_andnot(cpus, cpus, cpu_smt_mask(core));
 
 		if (idle)
 			return core;