Skip to content

x86: Reduce Memory Usage and Inter-Node message traffic (v3)

[email protected] linux-mm linuxppc-dev lkml sparclinux 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest10 patches 7 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/10x86: remove x86_cpu_to_log_apicid array (v3)arch/x86_64/kernel/genapic.c arch/x86_64/kernel/genapic_flat.c arch/x86_64/kernel/smpboot.c +1 more+0 -50 replies
2/10x86: fix cpu_to_node references (v3)arch/x86_64/kernel/vsyscall.c arch/x86_64/mm/numa.c arch/x86_64/mm/srat.c+5 -50 replies
3/10x86: Convert cpu_core_map to be a per cpu variable (v3)arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c arch/i386/kernel/cpu/cpufreq/powernow-k8.c arch/i386/kernel/cpu/proc.c +9 more+64 -460 replies
4/10x86: Convert cpu_sibling_map to be a per cpu variable (v3)arch/i386/kernel/cpu/cpufreq/p4-clockmod.c arch/i386/kernel/cpu/cpufreq/speedstep-ich.c arch/i386/kernel/io_apic.c +10 more+50 -480 replies
5/10x86: Convert x86_cpu_to_apicid to be a per cpu variable (v3)arch/i386/kernel/acpi/boot.c arch/i386/kernel/smp.c arch/i386/kernel/smpboot.c +9 more+80 -220 replies
6/10x86: Convert cpu_llc_id to be a per cpu variable (v3)arch/i386/kernel/cpu/intel_cacheinfo.c arch/i386/kernel/smpboot.c arch/x86_64/kernel/smpboot.c +2 more+17 -140 replies
7/10x86: acpi-use-cpu_physical_id (v3)arch/x86_64/kernel/mpparse.c drivers/acpi/processor_core.c+3 -70 replies
8/10ia64: Convert cpu_sibling_map to a per_cpu data array (v3)arch/ia64/kernel/setup.c arch/ia64/kernel/smpboot.c arch/ia64/mm/contig.c +2 more+18 -142 replies
9/10ppc64: Convert cpu_sibling_map to a per_cpu data array (v3)arch/powerpc/kernel/setup-common.c arch/powerpc/kernel/smp.c arch/powerpc/platforms/cell/cbe_cpufreq.c +2 more+9 -73 replies
10/10sparc64: Convert cpu_sibling_map to a per_cpu data array (v3)arch/sparc64/kernel/smp.c include/asm-sparc64/smp.h include/asm-sparc64/topology.h+11 -110 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 00/10] x86: Reduce Memory Usage and Inter-Node message traffic (v3)

Note:

This patch consolidates all the previous patches regarding the conversion of static arrays sized by NR_CPUS into per_cpu data arrays and is referenced against 2.6.23-rc6 .

v1 Intro:

In x86_64 and i386 architectures most arrays that are sized using NR_CPUS lay in local memory on node 0. Not only will most (99%?) of the systems not use all the slots in these arrays, particularly when NR_CPUS is increased to accommodate future very high cpu count systems, but a number of cache lines are passed unnecessarily on the system bus when these arrays are referenced by cpus on other nodes.

Typically, the values in these arrays are referenced by the cpu accessing it's own values, though when passing IPI interrupts, the cpu does access the data relevant to the targeted cpu/node. Of course, if the referencing cpu is not on node 0, then the reference will still require cross node exchanges of cache lines. A common use of this is for an interrupt service routine to pass the interrupt to other cpus local to that node.

Ideally, all the elements in these arrays should be moved to the per_cpu data area. In some cases (such as x86_cpu_to_apicid) the array is referenced before the per_cpu data areas are setup. In this case, a static array is declared in the __initdata area and initialized by the booting cpu (BSP). The values are then moved to the per_cpu area after it is initialized and the original static array is freed with the rest of the __initdata. This patch is referenced against 2.6.23-rc6.
--

Changes for version v2:

Quoted 5 lines
Quoted 2 lines

Note the addtional change of the cpu_llc_id type from u8 to int for ARCH x86_64 to correspond with ARCH i386.

At least currently it cannot be more than 8 bit. So why waste memory? It would be better to change i386

Done. (x86_64 type => u8).

Quoted 7 lines
Quoted 3 lines

Fix four instances where cpu_to_node is referenced

Quoted 2 lines

by array instead of via the cpu_to_node macro. This is preparation to moving it to the per_cpu data area.

Shouldn't this patch be logically before the per cpu conversion (which is 3). This way the result would be git bisectable.

Done. (Moved to PATCH 1).

Quoted 9 lines
Quoted 6 lines

processor_core.c currently tries to determine the apicid by special casing

Quoted 5 lines

for IA64 and x86. The desired information is readily available via

cpu_physical_id()

on IA64, i386 and x86_64.

Have you tried this with a !CONFIG_SMP build? The drivers/dma code was doing the same and running into problems because it wasn't defined there.

Fixed. (New export in PATCH 1).
--

Changes for version v3:

cpu_sibling_map has been converted to a per_cpu data array to fix build errors on ia64, ppc64 and sparc64 to accomodate references in block/blktrace.c and kernel/sched.c when CONFIG_SCHED_SMT is defined.

Warning: ppc64 and sparc64 have not yet been built nor tested. --

Signature · 5 lines
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Read the cover letter in its thread