Skip to contentSkip to conversation

x86_64: Avoid too many remote cpu references due to /proc/stat

PATCH In review Ravikiran G Thirumalai 3 participants 5 messages Jul 13, 2007 Proc filesystem

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86_64: Avoid too many remote cpu references due to /proc/stat

Too many remote cpu references due to /proc/stat.

On x86_64, with newer kernel versions, kstat_irqs is a bit of a problem. On every call to kstat_irqs, the process brings in per-cpu data from all online cpus. Doing this for NR_IRQS, which is now 256 + 32 * NR_CPUS results in (256+32*63) * 63 remote cpu references on a 64 cpu config. /proc/stat is parsed by common commands like top, who etc, causing lots of cacheline transfers

This statistic seems useless. Other 'big iron' arches disable this. Can we disable computing/reporting this statistic? This piece of statistic is not human readable on x86_64 anymore,

If not, can we optimize computing this statistic so as to avoid too many remote references (patch to follow)

fs/proc/proc_misc.c +2 -1
Index: linux-2.6.22/fs/proc/proc_misc.c
===================================================================
@@ -498,7 +498,8 @@ static int show_stat(struct seq_file *p,
 	}
 	seq_printf(p, "intr %llu", (unsigned long long)sum);
 
-#if !defined(CONFIG_PPC64) && !defined(CONFIG_ALPHA) && !defined(CONFIG_IA64)
+#if !defined(CONFIG_PPC64) && !defined(CONFIG_ALPHA) && !defined(CONFIG_IA64) \
+					&& !defined(CONFIG_X86_64)
 	for (i = 0; i < NR_IRQS; i++)
 		seq_printf(p, " %u", kstat_irqs(i));
 #endif
#1 lore raw

PATCH 1/1 Avoid too many remote cpu references due to /proc/stat

Quoted 9 lines · Ravikiran G Thirumalai

Too many remote cpu references due to /proc/stat.

On x86_64, with newer kernel versions, kstat_irqs is a bit of a problem. On every call to kstat_irqs, the process brings in per-cpu data from all online cpus. Doing this for NR_IRQS, which is now 256 + 32 * NR_CPUS results in (256+32*63) * 63 remote cpu references on a 64 cpu config. /proc/stat is parsed by common commands like top, who etc, causing lots of cacheline transfers

Quoted 6 lines

This statistic seems useless. Other 'big iron' arches disable this. Can we disable computing/reporting this statistic? This piece of statistic is not human readable on x86_64 anymore,

If not, can we optimize computing this statistic so as to avoid too many remote references (patch to follow)

Optimize show_stat to collect per-irq information just once.

On x86_64, with newer kernel versions, kstat_irqs is a bit of a problem. On every call to kstat_irqs, the process brings in per-cpu data from all online cpus. Doing this for NR_IRQS, which is now 256 + 32 * NR_CPUS results in (256+32*63) * 63 remote cpu references on a 64 cpu config. Considering the fact that we already compute this value per-cpu, we can save on the remote references as below.

fs/proc/proc_misc.c +12 -3
Index: linux-2.6.22/fs/proc/proc_misc.c
===================================================================
@@ -443,6 +443,11 @@ static int show_stat(struct seq_file *p,
 	unsigned long jif;
 	cputime64_t user, nice, system, idle, iowait, irq, softirq, steal;
 	u64 sum = 0;
+	unsigned int *per_irq_sum;
+
+	per_irq_sum = kzalloc(sizeof(unsigned int)*NR_IRQS, GFP_KERNEL);
+	if (!per_irq_sum)
+		return -ENOMEM;
 
 	user = nice = system = idle = iowait =
 		irq = softirq = steal = cputime64_zero;
@@ -461,8 +466,11 @@ static int show_stat(struct seq_file *p,
 		irq = cputime64_add(irq, kstat_cpu(i).cpustat.irq);
 		softirq = cputime64_add(softirq, kstat_cpu(i).cpustat.softirq);
 		steal = cputime64_add(steal, kstat_cpu(i).cpustat.steal);
-		for (j = 0 ; j < NR_IRQS ; j++)
-			sum += kstat_cpu(i).irqs[j];
+		for (j = 0 ; j < NR_IRQS ; j++) {
+			unsigned int temp = kstat_cpu(i).irqs[j];
+			sum += temp;
+			per_irq_sum[j] += temp;
+		}
 	}
 
 	seq_printf(p, "cpu  %llu %llu %llu %llu %llu %llu %llu %llu\n",
@@ -500,7 +508,7 @@ static int show_stat(struct seq_file *p,
 
 #if !defined(CONFIG_PPC64) && !defined(CONFIG_ALPHA) && !defined(CONFIG_IA64)
 	for (i = 0; i < NR_IRQS; i++)
-		seq_printf(p, " %u", kstat_irqs(i));
+		seq_printf(p, " %u", per_irq_sum[i]);
 #endif
 
 	seq_printf(p,
@@ -515,6 +523,7 @@ static int show_stat(struct seq_file *p,
 		nr_running(),
 		nr_iowait());
 
+	kfree(per_irq_sum);
 	return 0;
 }
 
#2 lore raw
In reply to Ravikiran G Thirumalai
Quoted 8 lines · On Thu, 12 Jul 2007 17:06:16 -0700 Ravikiran G Thirumalai <[email protected]>…

Too many remote cpu references due to /proc/stat.

On x86_64, with newer kernel versions, kstat_irqs is a bit of a problem. On every call to kstat_irqs, the process brings in per-cpu data from all online cpus. Doing this for NR_IRQS, which is now 256 + 32 * NR_CPUS results in (256+32*63) * 63 remote cpu references on a 64 cpu config. /proc/stat is parsed by common commands like top, who etc, causing lots of cacheline transfers

(256+32*63) * 63 = 143136

Do you have any actual numbers for how much this hurts?

Quoted 3 lines

This statistic seems useless. Other 'big iron' arches disable this. Can we disable computing/reporting this statistic? This piece of statistic is not human readable on x86_64 anymore,

Did you consider using percpu_counters (or such) at interrupt-time? (warning: percpu_counters aren't presently interrupt safe).

Quoted 2 lines

If not, can we optimize computing this statistic so as to avoid too many remote references (patch to follow)

You other patch is a straightforward optimisation and should just be merged.

But afaict it will only provide a 2x speedup which I doubt is sufficient?

Another thought is: how many of the NR_IRQS counters are actually non-zero? Because a pretty obvious optimisation would be to have a global bitmap[NR_IRQS] and do

	if (!bitmap[irq])
		bitmap[irq] = 1;

at interrupt-time, then just print a "0" for the interrupts which have never occurred within show_stats().

#3 lore raw
Quoted 11 lines · Andrew Morton
Quoted 8 lines · On Thu, 12 Jul 2007 17:06:16 -0700 Ravikiran G Thirumalai <[email protected]>…

Too many remote cpu references due to /proc/stat.

On x86_64, with newer kernel versions, kstat_irqs is a bit of a problem. On every call to kstat_irqs, the process brings in per-cpu data from all online cpus. Doing this for NR_IRQS, which is now 256 + 32 * NR_CPUS results in (256+32*63) * 63 remote cpu references on a 64 cpu config. /proc/stat is parsed by common commands like top, who etc, causing lots of cacheline transfers

Quoted 3 lines

(256+32*63) * 63 = 143136

Do you have any actual numbers for how much this hurts?

Under certain load conditions with 2.6.21/22, this hurts a *lot* infact, I have noticed delays in seconds when /proc/stat is read (48P box). However, the reason for this is not the transfer of per-cpu kstat, but dereferencing of the percpu data offset from a remote cpu pda! Since the cpu pda is kmalloced, with 2.6.21, the zone pcp structures happen to share the same page (and in some cases the internode cacheline as well) with the cpu_pda. Excessive page allocation activities on the remote node causes reads to the cpu pda of the remote cpus to result in cache misses.

Quoted 7 lines
Quoted 4 lines

This statistic seems useless. Other 'big iron' arches disable this. Can we disable computing/reporting this statistic? This piece of statistic is not human readable on x86_64 anymore,

Did you consider using percpu_counters (or such) at interrupt-time? (warning: percpu_counters aren't presently interrupt safe).

No, but given that alloc_percpu of a counter wastes so much memory, an array of NR_IRQS percpu_counters will only cause more bloat no?

Quoted 6 lines
Quoted 3 lines

If not, can we optimize computing this statistic so as to avoid too many remote references (patch to follow)

You other patch is a straightforward optimisation and should just be merged.

Sure! But I still don't see this statistic _really_ being useful anywhere; /proc/stat output with NR_CPUS of 64 or 96 is ugly, but I might be missing something

Quoted 1 line

But afaict it will only provide a 2x speedup which I doubt is sufficient?

No, it provides much more than 2x speedup with the 'hurting' workload. This is because the optimized patch goes like this:

for_each_possible_cpu(i) {
	...
	for (j = 0 ; j < NR_IRQS ; j++) {
		unsigned int temp = kstat_cpu(i).irqs[j];
	...
	}
}

But the existing code with kstat_irqs goes as:

for (j = 0 ; j < NR_IRQS ; j++) {
	for_each_possible_cpu(cpu) {
		sum += kstat_cpu(cpu).irqs[irq]
	...
}

Former case causes less cache misses to the pda per-cpu dataoffset than the latter by a huge margin.

Cache miss itself is not so bad per miss, but the number of misses in the latter case adds up!

Yes, the cpu pda needs to be on an internode boundary by itself, I will be submitting a patch for the same soon. I am thinking of page aligning cpu pda and using the 12 lower bits in %gs to store the cpu processor id maybe :)

Quoted 7 lines

Another thought is: how many of the NR_IRQS counters are actually non-zero? Because a pretty obvious optimisation would be to have a global bitmap[NR_IRQS] and do

Quoted 5 lines
	if (!bitmap[irq])
		bitmap[irq] = 1;

at interrupt-time, then just print a "0" for the interrupts which have never occurred within show_stats().

Yep. That would work too.

#4 lore raw
In reply to Ravikiran G Thirumalai
Quoted 9 lines · Ravikiran G Thirumalai

Too many remote cpu references due to /proc/stat.

On x86_64, with newer kernel versions, kstat_irqs is a bit of a problem. On every call to kstat_irqs, the process brings in per-cpu data from all online cpus. Doing this for NR_IRQS, which is now 256 + 32 * NR_CPUS results in (256+32*63) * 63 remote cpu references on a 64 cpu config. /proc/stat is parsed by common commands like top, who etc, causing lots of cacheline transfers

Quoted 3 lines

This statistic seems useless. Other 'big iron' arches disable this. Can we disable computing/reporting this statistic? This piece of statistic is not human readable on x86_64 anymore,

At which point we might just remove it completely..