x86_64: Avoid too many remote cpu references due to /proc/stat
PATCH 1/1 x86_64: Avoid too many remote cpu references due to /proc/stat
Too many remote cpu references due to /proc/stat.
On x86_64, with newer kernel versions, kstat_irqs is a bit of a problem. On every call to kstat_irqs, the process brings in per-cpu data from all online cpus. Doing this for NR_IRQS, which is now 256 + 32 * NR_CPUS results in (256+32*63) * 63 remote cpu references on a 64 cpu config. /proc/stat is parsed by common commands like top, who etc, causing lots of cacheline transfers
This statistic seems useless. Other 'big iron' arches disable this. Can we disable computing/reporting this statistic? This piece of statistic is not human readable on x86_64 anymore,
If not, can we optimize computing this statistic so as to avoid too many remote references (patch to follow)
fs/proc/proc_misc.c +2 -1
@@ -498,7 +498,8 @@ static int show_stat(struct seq_file *p, }seq_printf (p,"intr %llu" , (unsigned long long )sum); -#if !defined(CONFIG_PPC64 ) && !defined(CONFIG_ALPHA ) && !defined(CONFIG_IA64 ) +#if !defined(CONFIG_PPC64 ) && !defined(CONFIG_ALPHA ) && !defined(CONFIG_IA64 ) \ + && !defined(CONFIG_X86_64 )for (i =0 ; i <NR_IRQS ; i++)seq_printf (p," %u" ,kstat_irqs (i));#endif
PATCH 1/1 Avoid too many remote cpu references due to /proc/stat
Quoted 9 lines · Ravikiran G Thirumalai
Too many remote cpu references due to
/proc/stat.On
x86_64, with newer kernel versions,kstat_irqsis a bit of a problem. On every call tokstat_irqs, the process brings in per-cpu data from all online cpus. Doing this forNR_IRQS, which is now 256 + 32 *NR_CPUSresults in (256+32*63) * 63 remote cpu references on a 64 cpu config./proc/statis parsed by common commands like top, who etc, causing lots of cacheline transfers
Quoted 6 lines
This statistic seems useless. Other 'big iron' arches disable this. Can we disable computing/reporting this statistic? This piece of statistic is not human readable on
x86_64anymore,If not, can we optimize computing this statistic so as to avoid too many remote references (patch to follow)
Optimize show_stat to collect per-irq information just once.
On x86_64, with newer kernel versions, kstat_irqs is a bit of a problem. On every call to kstat_irqs, the process brings in per-cpu data from all online cpus. Doing this for NR_IRQS, which is now 256 + 32 * NR_CPUS results in (256+32*63) * 63 remote cpu references on a 64 cpu config. Considering the fact that we already compute this value per-cpu, we can save on the remote references as below.
fs/proc/proc_misc.c +12 -3
@@ -443,6 +443,11 @@ static int show_stat(struct seq_file *p,unsigned long jif;cputime64_t user, nice, system, idle, iowait, irq, softirq, steal;u64 sum =0 ; +unsigned int *per_irq_sum; + + per_irq_sum =kzalloc (sizeof (unsigned int )*NR_IRQS ,GFP_KERNEL ); +if (!per_irq_sum) +return -ENOMEM ; user = nice = system = idle = iowait = irq = softirq = steal = cputime64_zero;
@@ -461,8 +466,11 @@ static int show_stat(struct seq_file *p, irq =cputime64_add (irq,kstat_cpu (i).cpustat.irq); softirq =cputime64_add (softirq,kstat_cpu (i).cpustat.softirq); steal =cputime64_add (steal,kstat_cpu (i).cpustat.steal); -for (j =0 ; j <NR_IRQS ; j++) - sum +=kstat_cpu (i).irqs[j]; +for (j =0 ; j <NR_IRQS ; j++) { +unsigned int temp =kstat_cpu (i).irqs[j]; + sum += temp; + per_irq_sum[j] += temp; + } } seq_printf(p, "cpu %llu %llu %llu %llu %llu %llu %llu %llu\n",
@@ -500,7 +508,7 @@ static int show_stat(struct seq_file *p,#if !defined(CONFIG_PPC64 ) && !defined(CONFIG_ALPHA ) && !defined(CONFIG_IA64 )for (i =0 ; i <NR_IRQS ; i++) -seq_printf (p," %u" ,kstat_irqs (i)); +seq_printf (p," %u" , per_irq_sum[i]);#endif seq_printf(p,
@@ -515,6 +523,7 @@ static int show_stat(struct seq_file *p,nr_running (), nr_iowait()); +kfree (per_irq_sum);return 0 ; }
Quoted 8 lines · On Thu, 12 Jul 2007 17:06:16 -0700 Ravikiran G Thirumalai <[email protected]>…
Too many remote cpu references due to
/proc/stat.On
x86_64, with newer kernel versions,kstat_irqsis a bit of a problem. On every call tokstat_irqs, the process brings in per-cpu data from all online cpus. Doing this forNR_IRQS, which is now 256 + 32 *NR_CPUSresults in (256+32*63) * 63 remote cpu references on a 64 cpu config./proc/statis parsed by common commands like top, who etc, causing lots of cacheline transfers
(256+32*63) * 63 = 143136
Do you have any actual numbers for how much this hurts?
Quoted 3 lines
This statistic seems useless. Other 'big iron' arches disable this. Can we disable computing/reporting this statistic? This piece of statistic is not human readable on
x86_64anymore,
Did you consider using percpu_counters (or such) at interrupt-time? (warning: percpu_counters aren't presently interrupt safe).
Quoted 2 lines
If not, can we optimize computing this statistic so as to avoid too many remote references (patch to follow)
You other patch is a straightforward optimisation and should just be merged.
But afaict it will only provide a 2x speedup which I doubt is sufficient?
Another thought is: how many of the NR_IRQS counters are actually non-zero? Because a pretty obvious optimisation would be to have a global bitmap[NR_IRQS] and do
if (!bitmap[irq]) bitmap[irq] = 1;
at interrupt-time, then just print a "0" for the interrupts which have never occurred within show_stats().
Quoted 11 lines · Andrew Morton
Quoted 8 lines · On Thu, 12 Jul 2007 17:06:16 -0700 Ravikiran G Thirumalai <[email protected]>…
Too many remote cpu references due to
/proc/stat.On
x86_64, with newer kernel versions,kstat_irqsis a bit of a problem. On every call tokstat_irqs, the process brings in per-cpu data from all online cpus. Doing this forNR_IRQS, which is now 256 + 32 *NR_CPUSresults in (256+32*63) * 63 remote cpu references on a 64 cpu config./proc/statis parsed by common commands like top, who etc, causing lots of cacheline transfers
Quoted 3 lines
(256+32*63) * 63 = 143136
Do you have any actual numbers for how much this hurts?
Under certain load conditions with 2.6.21/22, this hurts a *lot* infact, I have noticed delays in seconds when /proc/stat is read (48P box). However, the reason for this is not the transfer of per-cpu kstat, but dereferencing of the percpu data offset from a remote cpu pda! Since the cpu pda is kmalloced, with 2.6.21, the zone pcp structures happen to share the same page (and in some cases the internode cacheline as well) with the cpu_pda. Excessive page allocation activities on the remote node causes reads to the cpu pda of the remote cpus to result in cache misses.
Quoted 7 lines
Quoted 4 lines
This statistic seems useless. Other 'big iron' arches disable this. Can we disable computing/reporting this statistic? This piece of statistic is not human readable on
x86_64anymore,Did you consider using
percpu_counters(or such) at interrupt-time? (warning:percpu_countersaren't presently interrupt safe).
No, but given that alloc_percpu of a counter wastes so much memory, an array of NR_IRQS percpu_counters will only cause more bloat no?
Quoted 6 lines
Quoted 3 lines
If not, can we optimize computing this statistic so as to avoid too many remote references (patch to follow)
You other patch is a straightforward optimisation and should just be merged.
Sure! But I still don't see this statistic _really_ being useful anywhere; /proc/stat output with NR_CPUS of 64 or 96 is ugly, but I might be missing something
Quoted 1 line
But afaict it will only provide a 2x speedup which I doubt is sufficient?
No, it provides much more than 2x speedup with the 'hurting' workload. This is because the optimized patch goes like this:
for_each_possible_cpu(i) {
...
for (j = 0 ; j < NR_IRQS ; j++) {
unsigned int temp = kstat_cpu(i).irqs[j];
...
}
}But the existing code with kstat_irqs goes as:
for (j = 0 ; j < NR_IRQS ; j++) {
for_each_possible_cpu(cpu) {
sum += kstat_cpu(cpu).irqs[irq]
...
}Former case causes less cache misses to the pda per-cpu dataoffset than the latter by a huge margin.
Cache miss itself is not so bad per miss, but the number of misses in the latter case adds up!
Yes, the cpu pda needs to be on an internode boundary by itself, I will be submitting a patch for the same soon. I am thinking of page aligning cpu pda and using the 12 lower bits in %gs to store the cpu processor id maybe :)
Quoted 7 lines
Another thought is: how many of the
NR_IRQScounters are actually non-zero? Because a pretty obvious optimisation would be to have a global bitmap[NR_IRQS] and do
Quoted 5 lines
if (!bitmap[irq]) bitmap[irq] = 1;at interrupt-time, then just print a "0" for the interrupts which have never occurred within
show_stats().
Yep. That would work too.
Quoted 9 lines · Ravikiran G Thirumalai
Too many remote cpu references due to
/proc/stat.On
x86_64, with newer kernel versions,kstat_irqsis a bit of a problem. On every call tokstat_irqs, the process brings in per-cpu data from all online cpus. Doing this forNR_IRQS, which is now 256 + 32 *NR_CPUSresults in (256+32*63) * 63 remote cpu references on a 64 cpu config./proc/statis parsed by common commands like top, who etc, causing lots of cacheline transfers
Quoted 3 lines
This statistic seems useless. Other 'big iron' arches disable this. Can we disable computing/reporting this statistic? This piece of statistic is not human readable on
x86_64anymore,
At which point we might just remove it completely..