Skip to contentSkip to conversation

use for_each_cpu

PATCH In review Anton Blanchard 3 participants 6 messages Aug 1, 2004 – Aug 2, 2004

lore
Share & tools
lore

Anton Blanchard

author
#0 lore raw

PATCH 1/1 use for_each_cpu

Hi,

The per cpu schedule counters need to be summed up over all possible cpus. When testing hotplug cpu remove I saw the sum of the online cpu count for nr_uninterruptible go negative which made the load average go nuts.

Anton

kernel/sched.c renamed from kernel/sched.c~debug_nr_running +3 -3
diff -puN kernel/sched.c~debug_nr_running kernel/sched.c
@@ -1095,7 +1095,7 @@ unsigned long nr_uninterruptible(void)
 {
 	unsigned long i, sum = 0;
 
-	for_each_online_cpu(i)
+	for_each_cpu(i)
 		sum += cpu_rq(i)->nr_uninterruptible;
 
 	return sum;

@@ -1105,7 +1105,7 @@ unsigned long long nr_context_switches(v
 {
 	unsigned long long i, sum = 0;
 
-	for_each_online_cpu(i)
+	for_each_cpu(i)
 		sum += cpu_rq(i)->nr_switches;
 
 	return sum;

@@ -1115,7 +1115,7 @@ unsigned long nr_iowait(void)
 {
 	unsigned long i, sum = 0;
 
-	for_each_online_cpu(i)
+	for_each_cpu(i)
 		sum += atomic_read(&cpu_rq(i)->nr_iowait);
 
 	return sum;
#1 lore raw
Quoted 4 lines · Anton Blanchard

The per cpu schedule counters need to be summed up over all possible cpus.
When testing hotplug cpu remove I saw the sum of the online cpu count for nr_uninterruptible go negative which made the load average go nuts.

I think the preferred approach here is to transfer the count over to the current CPU in the CPU_DEAD handler.

author
#2 lore raw
Quoted 7 lines
Quoted 4 lines

The per cpu schedule counters need to be summed up over all possible cpus.
When testing hotplug cpu remove I saw the sum of the online cpu count for nr_uninterruptible go negative which made the load average go nuts.

I think the preferred approach here is to transfer the count over to the current CPU in the CPU_DEAD handler.

They only look to be called out of proc, and once every 5 seconds for loadaverage calculations. Is it worth adding complexity to the cpu notifiers vs just using for_each_cpu?

Anton

#3 lore raw
Quoted 11 lines · Anton Blanchard
Quoted 8 lines
Quoted 5 lines

The per cpu schedule counters need to be summed up over all possible cpus.
When testing hotplug cpu remove I saw the sum of the online cpu count for nr_uninterruptible go negative which made the load average go nuts.

I think the preferred approach here is to transfer the count over to the current CPU in the CPU_DEAD handler.

They only look to be called out of proc, and once every 5 seconds for loadaverage calculations.

OK.

Quoted 2 lines

Is it worth adding complexity to the cpu
notifiers vs just using for_each_cpu?

yup ;) It's only six lines, and it follows the same pattern as is used in, say, page_alloc_cpu_notify(). Doing the same thing the same way in multiple places is to be preferred, yes?

author
#4 lore raw
Quoted 4 lines

yup ;) It's only six lines, and it follows the same pattern as is used in, say, page_alloc_cpu_notify(). Doing the same thing the same way in multiple places is to be preferred, yes?

If the data structure contains allocated memory, like per cpu pages I agree. But for percpu stuff that is straight stats (eg nr_running), I thought the aim was to just total all all possible cpus. bh_accounting is another one that falls into this group.

Anton

#5 lore raw
In reply to Andrew Morton
Quoted 6 lines · Andrew Morton
Quoted 2 lines

Is it worth adding complexity to the cpu
notifiers vs just using for_each_cpu?

yup ;) It's only six lines, and it follows the same pattern as is used in, say, page_alloc_cpu_notify(). Doing the same thing the same way in multiple places is to be preferred, yes?

Yes, and that way is "for_each_cpu".

Andrew, take his patch. You know for_each_cpu is preferred over for_each_online_cpu unless there's a good reason for the latter.

Rusty.

Signature · 1 line
Anyone who quotes me in their signature is an idiot -- Rusty Russell