Skip to contentSkip to conversation

sched: add discrete weighted cpu load function

PATCH In review Con Kolivas 3 participants 10 messages Mar 13, 2006 – Mar 14, 2006 Scheduler

lore
Share & tools
lore
#0 lore raw

PATCH 2/4 sched: add discrete weighted cpu load function

When the type of weighting is known to be zero we can use a simpler version of source_load with a weighted_cpuload() function. Export that function to allow relative weighted cpu load to be used by other subsystems if desired.

---
 include/linux/sched.h |    1 +
 kernel/sched.c        |   10 ++++++++--
 2 files changed, 9 insertions(+), 2 deletions(-)
include/linux/sched.h +1 -0
Index: linux-2.6.16-rc6-mm1/include/linux/sched.h
===================================================================
@@ -102,6 +102,7 @@ extern int nr_processes(void);
 extern unsigned long nr_running(void);
 extern unsigned long nr_uninterruptible(void);
 extern unsigned long nr_iowait(void);
+extern unsigned long weighted_cpuload(const int cpu);
 
 #include <linux/time.h>
 #include <linux/param.h>
kernel/sched.c +8 -2
Index: linux-2.6.16-rc6-mm1/kernel/sched.c
===================================================================
@@ -914,6 +914,12 @@ inline int task_curr(const task_t *p)
 	return cpu_curr(task_cpu(p)) == p;
 }
 
+/* Used instead of source_load when we know the type == 0 */
+unsigned long weighted_cpuload(const int cpu)
+{
+	return (cpu_rq(cpu)->raw_weighted_load);
+}
+
 #ifdef CONFIG_SMP
 typedef struct {
 	struct list_head list;
@@ -1114,7 +1120,7 @@ find_idlest_cpu(struct sched_group *grou
 	cpus_and(tmp, group->cpumask, p->cpus_allowed);
 
 	for_each_cpu_mask(i, tmp) {
-		load = source_load(i, 0);
+		load = weighted_cpuload(i);
 
 		if (load < min_load || (load == min_load && i == this_cpu)) {
 			min_load = load;
@@ -2186,7 +2192,7 @@ static runqueue_t *find_busiest_queue(st
 	int i;
 
 	for_each_cpu_mask(i, group->cpumask) {
-		load = source_load(i, 0);
+		load = weighted_cpuload(i);
 
 		if (load > max_load) {
 			max_load = load;
Acked-by on v1 2/4
#1 lore raw

kernel/sched.c inline int task_curr(const task_t *p) · line 920

Quoted 4 lines · Con Kolivas
kernel/sched.c +4 -0
+/* Used instead of source_load when we know the type == 0 */
+unsigned long weighted_cpuload(const int cpu)
+{
+	return (cpu_rq(cpu)->raw_weighted_load);

no braces in return please. Looks good to me otherwise.

Ingo

#2 lore raw
Quoted 9 lines · Ingo Molnar

Review

Quoted 4 lines · Con Kolivas
(quoted diff) +4 -0
+/* Used instead of source_load when we know the type == 0 */
+unsigned long weighted_cpuload(const int cpu)
+{
+	return (cpu_rq(cpu)->raw_weighted_load);

no braces in return please. Looks good to me otherwise.

Thanks. Respin with that one change below.

Cheers,
Con
---
When the type of weighting is known to be zero we can use a simpler version of source_load with a weighted_cpuload() function. Export that function to allow relative weighted cpu load to be used by other subsystems if desired.

---
 include/linux/sched.h |    1 +
 kernel/sched.c        |   10 ++++++++--
 2 files changed, 9 insertions(+), 2 deletions(-)
include/linux/sched.h +1 -0
Index: linux-2.6.16-rc6-mm1/include/linux/sched.h
===================================================================
@@ -102,6 +102,7 @@ extern int nr_processes(void);
 extern unsigned long nr_running(void);
 extern unsigned long nr_uninterruptible(void);
 extern unsigned long nr_iowait(void);
+extern unsigned long weighted_cpuload(const int cpu);
 
 #include <linux/time.h>
 #include <linux/param.h>
kernel/sched.c +8 -2
Index: linux-2.6.16-rc6-mm1/kernel/sched.c
===================================================================
@@ -914,6 +914,12 @@ inline int task_curr(const task_t *p)
 	return cpu_curr(task_cpu(p)) == p;
 }
 
+/* Used instead of source_load when we know the type == 0 */
+unsigned long weighted_cpuload(const int cpu)
+{
+	return cpu_rq(cpu)->raw_weighted_load;
+}
+
 #ifdef CONFIG_SMP
 typedef struct {
 	struct list_head list;
@@ -1114,7 +1120,7 @@ find_idlest_cpu(struct sched_group *grou
 	cpus_and(tmp, group->cpumask, p->cpus_allowed);
 
 	for_each_cpu_mask(i, tmp) {
-		load = source_load(i, 0);
+		load = weighted_cpuload(i);
 
 		if (load < min_load || (load == min_load && i == this_cpu)) {
 			min_load = load;
@@ -2186,7 +2192,7 @@ static runqueue_t *find_busiest_queue(st
 	int i;
 
 	for_each_cpu_mask(i, group->cpumask) {
-		load = source_load(i, 0);
+		load = weighted_cpuload(i);
 
 		if (load > max_load) {
 			max_load = load;
#3 lore raw
In reply to Con Kolivas
Quoted 29 lines · Con Kolivas

When the type of weighting is known to be zero we can use a simpler version of source_load with a weighted_cpuload() function. Export that function to allow relative weighted cpu load to be used by other subsystems if desired.

---
 include/linux/sched.h |    1 +
 kernel/sched.c        |   10 ++++++++--
 2 files changed, 9 insertions(+), 2 deletions(-)

Index: linux-2.6.16-rc6-mm1/include/linux/sched.h
===================================================================

include/linux/sched.h +1 -0
@@ -102,6 +102,7 @@ extern int nr_processes(void);
 extern unsigned long nr_running(void);
 extern unsigned long nr_uninterruptible(void);
 extern unsigned long nr_iowait(void);
+extern unsigned long weighted_cpuload(const int cpu);
 
 #include <linux/time.h>
 #include <linux/param.h>

Index: linux-2.6.16-rc6-mm1/kernel/sched.c
===================================================================

kernel/sched.c inline int task_curr(const task_t *p) · line 922

Quoted 9 lines
kernel/sched.c +6 -0
@@ -914,6 +914,12 @@ inline int task_curr(const task_t *p)
 	return cpu_curr(task_cpu(p)) == p;
 }
 
+/* Used instead of source_load when we know the type == 0 */
+unsigned long weighted_cpuload(const int cpu)
+{
+	return (cpu_rq(cpu)->raw_weighted_load);
+}
+

Wouldn't this be a candidate for inlining?

Quoted 26 lines
kernel/sched.c +2 -3
 #ifdef CONFIG_SMP
 typedef struct {
 	struct list_head list;
@@ -1114,7 +1120,7 @@ find_idlest_cpu(struct sched_group *grou
 	cpus_and(tmp, group->cpumask, p->cpus_allowed);
 
 	for_each_cpu_mask(i, tmp) {
-		load = source_load(i, 0);
+		load = weighted_cpuload(i);
 
 		if (load < min_load || (load == min_load && i == this_cpu)) {
 			min_load = load;
@@ -2186,7 +2192,7 @@ static runqueue_t *find_busiest_queue(st
 	int i;
 
 	for_each_cpu_mask(i, group->cpumask) {
-		load = source_load(i, 0);
+		load = weighted_cpuload(i);
 
 		if (load > max_load) {
 			max_load = load;
-

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 4 lines
Peter Williams                                   [email protected]

"Learning, n. The kind of ignorance distinguishing the studious."
 -- Ambrose Bierce
#4 lore raw
Quoted 8 lines · Peter Williams

kernel/sched.c inline int task_curr(const task_t *p) · line 922

Quoted 5 lines · Con Kolivas
kernel/sched.c +5 -0
+unsigned long weighted_cpuload(const int cpu)
+{
+	return (cpu_rq(cpu)->raw_weighted_load);
+}
+

Wouldn't this be a candidate for inlining?

That would make it unsuitable for exporting via sched.h.

Cheers,
Con

#5 lore raw
Quoted 14 lines · Con Kolivas
Quoted 10 lines · Peter Williams

kernel/sched.c inline int task_curr(const task_t *p) · line 922

Quoted 5 lines · Con Kolivas
kernel/sched.c +5 -0
+unsigned long weighted_cpuload(const int cpu)
+{
+    return (cpu_rq(cpu)->raw_weighted_load);
+}
+

Wouldn't this be a candidate for inlining?

Quoted 1 line

That would make it unsuitable for exporting via sched.h.

Right, no cpu_rq() out there.

Sorry, for the noise,
Peter

Signature · 4 lines
Peter Williams                                   [email protected]

"Learning, n. The kind of ignorance distinguishing the studious."
 -- Ambrose Bierce
#6 lore raw
In reply to Con Kolivas
Quoted 14 lines · Con Kolivas
Quoted 10 lines · Peter Williams

kernel/sched.c inline int task_curr(const task_t *p) · line 922

Quoted 5 lines · Con Kolivas
kernel/sched.c +5 -0
+unsigned long weighted_cpuload(const int cpu)
+{
+    return (cpu_rq(cpu)->raw_weighted_load);
+}
+

Wouldn't this be a candidate for inlining?

Quoted 1 line

That would make it unsuitable for exporting via sched.h.

If above_background_load() were implemented inside sched.c instead of in sched.h there would be no need to export weighted_cpuload() would there?
This would allow weighted_cpuload() to be inline and the efficiency
would be better as above_background_load() doesn't gain a lot by being inline as having weighted_cpulpad() non inline means that it's doing a function call several times in a loop i.e. it may save one function call by being inline but requires (up to) one function call for every CPU.

The other way around the cost would be just one function call.

Peter

Signature · 4 lines
Peter Williams                                   [email protected]

"Learning, n. The kind of ignorance distinguishing the studious."
 -- Ambrose Bierce
#7 lore raw
Quoted 11 lines · Peter Williams
Quoted 10 lines · Con Kolivas
Quoted 8 lines · Peter Williams

kernel/sched.c inline int task_curr(const task_t *p) · line 922

Quoted 5 lines · Con Kolivas
kernel/sched.c +5 -0
+unsigned long weighted_cpuload(const int cpu)
+{
+    return (cpu_rq(cpu)->raw_weighted_load);
+}
+

Wouldn't this be a candidate for inlining?

Quoted 7 lines
Quoted 1 line

That would make it unsuitable for exporting via sched.h.

If above_background_load() were implemented inside sched.c instead of in sched.h there would be no need to export weighted_cpuload() would there?
This would allow weighted_cpuload() to be inline and the efficiency
would be better as above_background_load() doesn't gain a lot by being inline

I don't care about above_background_load() being inline; that's done because all functions in header files need to be static inline to not become a mess.

Quoted 3 lines

as having weighted_cpulpad() non inline means that it's doing a function call several times in a loop i.e. it may save one function call by being inline but requires (up to) one function call for every CPU.

I haven't checked but gcc may well inline weighted_cpuload anyway? We're moving away from inlining most things manually since the compiler is doing it well these days.

Quoted 1 line

The other way around the cost would be just one function call.

The way you're suggesting adds a function that is never used by anything but swap prefetch which would then need to be 'ifdef'ed out to not be needlessly built on every system. Adding ifdefs is frowned upon already, and to have an mm/ specific ifdef in sched.c would be rather ugly.

Cheers,
Con

#8 lore raw
Quoted 25 lines · Con Kolivas
Quoted 14 lines · Peter Williams
Quoted 12 lines · Con Kolivas
Quoted 9 lines · Peter Williams

kernel/sched.c inline int task_curr(const task_t *p) · line 922

Quoted 5 lines · Con Kolivas
kernel/sched.c +5 -0
+unsigned long weighted_cpuload(const int cpu)
+{
+    return (cpu_rq(cpu)->raw_weighted_load);
+}
+

Wouldn't this be a candidate for inlining?

Quoted 7 lines
Quoted 1 line

That would make it unsuitable for exporting via sched.h.

If above_background_load() were implemented inside sched.c instead of in sched.h there would be no need to export weighted_cpuload() would there?
This would allow weighted_cpuload() to be inline and the efficiency
would be better as above_background_load() doesn't gain a lot by being inline

Quoted 8 lines

I don't care about above_background_load() being inline; that's done because all functions in header files need to be static inline to not become a mess.

Quoted 2 lines

as having weighted_cpulpad() non inline means that it's doing a function call several times in a loop i.e. it may save one function call by being inline but requires (up to) one function call for every CPU.

I haven't checked but gcc may well inline weighted_cpuload anyway?

It may be doing so for internal uses inside sched.c but I'm pretty sure that it won't for external calls.

Quoted 2 lines

We're moving away from inlining most things manually since the compiler is doing it well these days.

OK. Even without explicit inlining you need to give the compiler a chance to optimize function call overhead away. It cant do that with the present arrangement.

Quoted 5 lines
Quoted 2 lines

The other way around the cost would be just one function call.

The way you're suggesting adds a function that is never used by anything but swap prefetch which would then need to be 'ifdef'ed out to not be needlessly built on every system. Adding ifdefs is frowned upon already, and to have an mm/ specific ifdef in sched.c would be rather ugly.

Sometimes ugliness is the best option.

Peter

Signature · 4 lines
Peter Williams                                   [email protected]

"Learning, n. The kind of ignorance distinguishing the studious."
 -- Ambrose Bierce
#9 lore raw
Quoted 5 lines · Peter Williams
Quoted 1 line · Con Kolivas

I haven't checked but gcc may well inline weighted_cpuload anyway?

It may be doing so for internal uses inside sched.c but I'm pretty sure that it won't for external calls.

Hmm I investigated briefly and only C99 inlining (whatever that means) will allow me to locally inline and export as well. It would do so if I specified -finline-functions which is not our default at all in the kernel (we only recently disable -fnoinline-functions I believe). Anyway checking positively shows me this on gcc 4.1.0:

0xc0111283 <find_busiest_queue+83>: call 0xc0110dc0 <weighted_cpuload>

So no, it doesn't get inlined.

Quoted 6 lines
Quoted 4 lines

The way you're suggesting adds a function that is never used by anything but swap prefetch which would then need to be 'ifdef'ed out to not be needlessly built on every system. Adding ifdefs is frowned upon already, and to have an mm/ specific ifdef in sched.c would be rather ugly.

Sometimes ugliness is the best option.

I spent quite some time trying to find the least cost way to do this without uglifying code. I don't feel strongly about just how to do it though. Comments from Andrew and Ingo would be most welcome on this matter.

Cheers,
Con