percpu: convert most archs to dynamic percpu
Subsystems (24)
Tejun Heo
maintainerPATCH 1/1 percpu: convert most archs to dynamic percpu
Hello,
Please do NOT pull before maintainers of alpha and s390 ack. Upon ack, please pull from the following git tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc.git tj-percpu
This patchset is combination of the last patch of percpu-use-dynamic-percpu-allocator-as-default patchset[1] and patches inspired by alpha conversion patch[2] posted by Ivan Kokshaysky.
0001-percpu-use-dynamic-percpu-allocator-as-the-default.patch 0002-linker-script-throw-away-.discard-section.patch 0003-percpu-clean-up-percpu-variable-definitions.patch 0004-percpu-enforce-global-uniqueness-and-disallow-in-fu.patch 0005-alpha-kill-unnecessary-__used-attribute-in-PER_CPU_.patch 0006-alpha-switch-to-dynamic-percpu-allocator.patch 0007-s390-switch-to-dynamic-percpu-allocator.patch
0001 is mostly unchanged except for added Acked-by's.
0002-0003 prepare for percpu variable definition update - .discard is thrown away on all archs when linking kernel and modules, static percpu variables in functions are moved outside of the function and all static percpu variables are given unique names.
0004 updates percpu variable definition such that all percpu variables in the same link unit have unique names regardless of its scope (static or global) and disallow in-function static definitions. The new definition uses dummy variables in .discard to enforce the declaration/definition semantics and always defines percpu variables as globals. This allows archs to add 'weak' attribute if necessary.
0005-0006 converts alpha to use 'weak' attribute instead of inline assembly to generate GOT based reference when referencing percpu variables from modules, which in turn allows conversion to dynamic percpu allocator.
0007 does the same for s390.
The new DECLARE/DEFINE macros implement the following behaviors.
a. DECLARE + static DEFINE -> compile error
b. multiple non-static DEFINEs with the same name -> compile error
c. multiple static/global DEFINEs with the same name -> compile error
d. in function static DEFINEs -> compile error
#a and #b are expected behaviors on DECLARE/DEFINE macros in general. #c and #d are necessary to allow 'weak' attribute on certain archs. Please note that the rules are applied consistently whether 'weak' attribute is specified or not. This eases catching mistakes and makes things less confusing.
Test config boots and works fine on x86_64. alpha and s390 are compile tested and verified to generate necessary code for external references but I don't have any way to actually verify them, so please test them. :-)
all-yes-config on x86_64 is building now. Ergggh... I want more memory. Okay, completed successfully.
This patchset is on top of
the last #tj-percpu (e1b9aa3f47242e757c776a3771bb6613e675bf9c)
+ linus#master (279e677faa775ad16e75c32e1bf4a37f8158bc61)
and contains the following changes.
Makefile | 2 arch/alpha/include/asm/percpu.h | 101 ++------------------------------- arch/alpha/include/asm/tlbflush.h | 1 arch/alpha/kernel/vmlinux.lds.S | 1 arch/arm/kernel/vmlinux.lds.S | 1 arch/avr32/kernel/vmlinux.lds.S | 1 arch/blackfin/kernel/vmlinux.lds.S | 1 arch/cris/include/asm/mmu_context.h | 2 arch/cris/kernel/vmlinux.lds.S | 1 arch/frv/kernel/vmlinux.lds.S | 2 arch/h8300/kernel/vmlinux.lds.S | 1 arch/ia64/Kconfig | 3 arch/ia64/kernel/vmlinux.lds.S | 1 arch/m32r/kernel/vmlinux.lds.S | 1 arch/m68k/kernel/vmlinux-std.lds | 1 arch/m68k/kernel/vmlinux-sun3.lds | 1 arch/m68knommu/kernel/vmlinux.lds.S | 1 arch/microblaze/kernel/vmlinux.lds.S | 2 arch/mips/kernel/vmlinux.lds.S | 1 arch/mn10300/kernel/vmlinux.lds.S | 1 arch/parisc/kernel/vmlinux.lds.S | 1 arch/powerpc/Kconfig | 3 arch/powerpc/kernel/vmlinux.lds.S | 1 arch/s390/include/asm/percpu.h | 32 ++-------- arch/s390/kernel/vmlinux.lds.S | 1 arch/sh/kernel/vmlinux_32.lds.S | 1 arch/sh/kernel/vmlinux_64.lds.S | 1 arch/sparc/Kconfig | 3 arch/sparc/kernel/vmlinux.lds.S | 1 arch/um/kernel/dyn.lds.S | 2 arch/um/kernel/uml.lds.S | 2 arch/x86/Kconfig | 3 arch/xtensa/kernel/vmlinux.lds.S | 1 block/as-iosched.c | 10 +-- block/cfq-iosched.c | 10 +-- drivers/cpufreq/cpufreq_conservative.c | 12 +-- drivers/cpufreq/cpufreq_ondemand.c | 15 ++-- drivers/xen/events.c | 9 +- include/linux/percpu-defs.h | 34 ++++++++--- include/linux/percpu.h | 12 ++- init/main.c | 24 ------- kernel/module.c | 6 - mm/Makefile | 2 mm/allocpercpu.c | 28 +++++++++ mm/page-writeback.c | 5 - mm/percpu.c | 40 ++++++++++++- net/ipv4/syncookies.c | 4 - net/ipv6/syncookies.c | 4 - scripts/module-common.lds | 8 ++ 49 files changed, 206 insertions(+), 195 deletions(-)
Thanks.
--
tejun
[1] http://thread.gmane.org/gmane.linux.kernel/804631/focus=804627
[2] http://thread.gmane.org/gmane.linux.kernel/820849/focus=820926
Tejun Heo
maintainerPATCH 1/7 percpu: use dynamic percpu allocator as the default percpu allocator
This patch makes most !CONFIG_HAVE_SETUP_PER_CPU_AREA archs use dynamic percpu allocator. The first chunk is allocated using embedding helper and 8k is reserved for modules. This ensures that the new allocator behaves almost identically to the original allocator as long as static percpu variables are concerned, so it shouldn't introduce much breakage.
s390 and alpha use custom SHIFT_PERCPU_PTR() to work around addressing range limit the addressing model imposes. Unfortunately, this breaks if the address is specified using a variable, so for now, the two archs aren't converted.
The following architectures are affected by this change.
* sh
* arm
* cris
* mips
* sparc(32)
* blackfin
* avr32
* parisc
* m32r
* powerpc(32)
As this change makes the dynamic allocator the default one, CONFIG_HAVE_DYNAMIC_PER_CPU_AREA is replaced with its invert - CONFIG_HAVE_LEGACY_PER_CPU_AREA, which is added to yet-to-be converted archs. These archs implement their own setup_per_cpu_areas() and the conversion is not trivial.
* powerpc(64)
* sparc(64)
* ia64
* alpha
* s390
Boot and batch alloc/free tests on x86_32 with debug code (x86_32 doesn't use default first chunk initialization). Compile tested on sparc(32), powerpc(32), arm and alpha.
[ Impact: use dynamic allocator for most archs w/o custom percpu setup ]
--- arch/alpha/Kconfig | 3 +++ arch/ia64/Kconfig | 3 +++ arch/powerpc/Kconfig | 3 +++ arch/s390/Kconfig | 3 +++ arch/sparc/Kconfig | 3 +++ arch/x86/Kconfig | 3 --- include/linux/percpu.h | 12 +++++++++--- init/main.c | 24 ------------------------ kernel/module.c | 6 +++--- mm/Makefile | 2 +- mm/allocpercpu.c | 28 ++++++++++++++++++++++++++++ mm/percpu.c | 40 +++++++++++++++++++++++++++++++++++++++- 12 files changed, 95 insertions(+), 35 deletions(-)
arch/alpha/Kconfig +3 -0
@@ -70,6 +70,9 @@ config AUTO_IRQ_AFFINITY depends on SMP default y +config HAVE_LEGACY_PER_CPU_AREA + def_bool y + source "init/Kconfig" source "kernel/Kconfig.freezer"
arch/ia64/Kconfig +3 -0
@@ -88,6 +88,9 @@ config GENERIC_TIME_VSYSCALL bool default y +config HAVE_LEGACY_PER_CPU_AREA + def_bool y + config HAVE_SETUP_PER_CPU_AREA def_bool y
arch/powerpc/Kconfig +3 -0
@@ -42,6 +42,9 @@ config GENERIC_HARDIRQS bool default y +config HAVE_LEGACY_PER_CPU_AREA + def_bool PPC64 + config HAVE_SETUP_PER_CPU_AREA def_bool PPC64
arch/s390/Kconfig +3 -0
@@ -75,6 +75,9 @@ config VIRT_CPU_ACCOUNTING config ARCH_SUPPORTS_DEBUG_PAGEALLOC def_bool y +config HAVE_LEGACY_PER_CPU_AREA + def_bool y + mainmenu "Linux Kernel Configuration" config S390
arch/sparc/Kconfig +3 -0
@@ -90,6 +90,9 @@ config AUDIT_ARCH bool default y +config HAVE_LEGACY_PER_CPU_AREA + def_bool y if SPARC64 + config HAVE_SETUP_PER_CPU_AREA def_bool y if SPARC64
arch/x86/Kconfig +0 -3
@@ -141,9 +141,6 @@ config ARCH_HAS_CACHE_LINE_SIZE config HAVE_SETUP_PER_CPU_AREA def_bool y -config HAVE_DYNAMIC_PER_CPU_AREA - def_bool y - config HAVE_CPUMASK_OF_CPU_MAP def_bool X86_64_SMP
include/linux/percpu.h +9 -3
@@ -34,7 +34,7 @@#ifdef CONFIG_SMP -#ifdef CONFIG_HAVE_DYNAMIC_PER_CPU_AREA +#ifndef CONFIG_HAVE_LEGACY_PER_CPU_AREA /* minimum unit size, also is the maximum supported allocation size */ #define PCPU_MIN_UNIT_SIZE PFN_ALIGN(64 << 10)
@@ -80,7 +80,7 @@ extern ssize_t __init pcpu_embed_first_chunk( extern void *__alloc_reserved_percpu(size_t size, size_t align); -#else /* CONFIG_HAVE_DYNAMIC_PER_CPU_AREA */ +#else /* CONFIG_HAVE_LEGACY_PER_CPU_AREA */ struct percpu_data { void *ptrs[1];
@@ -94,11 +94,15 @@ struct percpu_data { (__typeof__ (ptr ))__p->ptrs[(cpu)]; \ }) -#endif /* CONFIG_HAVE_DYNAMIC_PER_CPU_AREA */ +#endif /* CONFIG_HAVE_LEGACY_PER_CPU_AREA */ extern void *__alloc_percpu (size_t size,size_t align);extern void free_percpu (void *__pdata); +#ifndef CONFIG_HAVE_SETUP_PER_CPU_AREA +extern void __init setup_per_cpu_areas (void ); +#endif +#else /* CONFIG_SMP */ #define per_cpu_ptr (ptr, cpu) ({ (void)(cpu); (ptr); })
@@ -119,6 +123,8 @@ static inline void free_percpu(void *p) kfree(p); } +static inline void __init setup_per_cpu_areas (void ) { } +#endif /* CONFIG_SMP */ #define alloc_percpu (type) (type *)__alloc_percpu(sizeof(type), \
init/main.c +0 -24
@@ -355,7 +355,6 @@ static void __init smp_init(void)#define smp_init () do { } while (0)#endif -static inline void setup_per_cpu_areas (void ) { }static inline void setup_nr_cpu_ids (void ) { }static inline void smp_prepare_cpus (unsigned int maxcpus) { }
@@ -376,29 +375,6 @@ static void __init setup_nr_cpu_ids(void) nr_cpu_ids =find_last_bit (cpumask_bits (cpu_possible_mask),NR_CPUS ) +1 ; } -#ifndef CONFIG_HAVE_SETUP_PER_CPU_AREA -unsigned long __per_cpu_offset[NR_CPUS ]__read_mostly ; - -EXPORT_SYMBOL (__per_cpu_offset); - -static void __init setup_per_cpu_areas (void ) -{ -unsigned long size, i; -char *ptr; -unsigned long nr_possible_cpus =num_possible_cpus (); - -/* Copy section for each CPU (we discard the original) */ - size =ALIGN (PERCPU_ENOUGH_ROOM ,PAGE_SIZE ); - ptr =alloc_bootmem_pages (size * nr_possible_cpus); - -for_each_possible_cpu (i) { - __per_cpu_offset[i] = ptr - __per_cpu_start; -memcpy (ptr, __per_cpu_start, __per_cpu_end - __per_cpu_start); - ptr += size; - } -} -#endif /* CONFIG_HAVE_SETUP_PER_CPU_AREA */ -/* Called by boot processor to activate the rest. */ static void __init smp_init (void ) {
kernel/module.c +3 -3
@@ -359,7 +359,7 @@ EXPORT_SYMBOL_GPL(find_module);#ifdef CONFIG_SMP -#ifdef CONFIG_HAVE_DYNAMIC_PER_CPU_AREA +#ifndef CONFIG_HAVE_LEGACY_PER_CPU_AREA static void *percpu_modalloc (unsigned long size,unsigned long align,const char *name)
@@ -384,7 +384,7 @@ static void percpu_modfree(void *freeme) free_percpu(freeme); } -#else /* ... !CONFIG_HAVE_DYNAMIC_PER_CPU_AREA */ +#else /* ... CONFIG_HAVE_LEGACY_PER_CPU_AREA */ /* Number of blocks used and allocated. */ static unsigned int pcpu_num_used, pcpu_num_allocated;
@@ -519,7 +519,7 @@ static int percpu_modinit(void) }__initcall (percpu_modinit); -#endif /* CONFIG_HAVE_DYNAMIC_PER_CPU_AREA */ +#endif /* CONFIG_HAVE_LEGACY_PER_CPU_AREA */ static unsigned int find_pcpusec(Elf_Ehdr *hdr, Elf_Shdr *sechdrs,
mm/Makefile +1 -1
@@ -31,7 +31,7 @@ obj-$(CONFIG_FAILSLAB) += failslab.o obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o obj-$(CONFIG_FS_XIP) += filemap_xip.o obj-$(CONFIG_MIGRATION) += migrate.o -ifdef CONFIG_HAVE_DYNAMIC_PER_CPU_AREA +ifndef CONFIG_HAVE_LEGACY_PER_CPU_AREA obj-$(CONFIG_SMP) += percpu.o else obj-$(CONFIG_SMP) += allocpercpu.o
mm/allocpercpu.c +28 -0
@@ -5,6 +5,8 @@*/ #include <linux/mm.h> #include <linux/module.h> +#include <linux/bootmem.h> +#include <asm/sections.h> #ifndef cache_line_size#define cache_line_size () L1_CACHE_BYTES
@@ -147,3 +149,29 @@ void free_percpu(void *__pdata)kfree (__percpu_disguise (__pdata)); }EXPORT_SYMBOL_GPL (free_percpu); + +/* +* Generic percpu area setup. +*/ +#ifndef CONFIG_HAVE_SETUP_PER_CPU_AREA +unsigned long __per_cpu_offset[NR_CPUS ]__read_mostly ; + +EXPORT_SYMBOL (__per_cpu_offset); + +void __init setup_per_cpu_areas (void ) +{ +unsigned long size, i; +char *ptr; +unsigned long nr_possible_cpus =num_possible_cpus (); + +/* Copy section for each CPU (we discard the original) */ + size =ALIGN (PERCPU_ENOUGH_ROOM ,PAGE_SIZE ); + ptr =alloc_bootmem_pages (size * nr_possible_cpus); + +for_each_possible_cpu (i) { + __per_cpu_offset[i] = ptr - __per_cpu_start; +memcpy (ptr, __per_cpu_start, __per_cpu_end - __per_cpu_start); + ptr += size; + } +} +#endif /* CONFIG_HAVE_SETUP_PER_CPU_AREA */
mm/percpu.c +39 -1
@@ -43,7 +43,7 @@* * To use this allocator, arch code should do the followings. * -* - define CONFIG_HAVE_DYNAMIC_PER_CPU_AREA +* - drop CONFIG_HAVE_LEGACY_PER_CPU_AREA * * - define __addr_to_pcpu_ptr() and __pcpu_ptr_to_addr() to translate * regular address to percpu pointer and back if they need to be
@@ -1271,3 +1271,41 @@ ssize_t __init pcpu_embed_first_chunk(size_t static_size, size_t reserved_size, reserved_size, dyn_size, pcpue_unit_size, pcpue_ptr, NULL); } + +/* +* Generic percpu area setup. +* +* The embedding helper is used because its behavior closely resembles +* the original non-dynamic generic percpu area setup. This is +* important because many archs have addressing restrictions and might +* fail if the percpu area is located far away from the previous +* location. As an added bonus, in non-NUMA cases, embedding is +* generally a good idea TLB-wise because percpu area can piggy back +* on the physical linear memory mapping which uses large page +* mappings on applicable archs. +*/ +#ifndef CONFIG_HAVE_SETUP_PER_CPU_AREA +unsigned long __per_cpu_offset[NR_CPUS ]__read_mostly ; +EXPORT_SYMBOL (__per_cpu_offset); + +void __init setup_per_cpu_areas (void ) +{ +size_t static_size = __per_cpu_end - __per_cpu_start; +ssize_t unit_size; +unsigned long delta; +unsigned int cpu; + +/* +* Always reserve area for module percpu variables. That's +* what the legacy allocator did. +*/ + unit_size =pcpu_embed_first_chunk (static_size,PERCPU_MODULE_RESERVE , +PERCPU_DYNAMIC_RESERVE ,-1 ); +if (unit_size <0 ) +panic ("Failed to initialized percpu areas." ); + + delta = (unsigned long )pcpu_base_addr - (unsigned long )__per_cpu_start; +for_each_possible_cpu (cpu) + __per_cpu_offset[cpu] = delta + cpu * unit_size; +} +#endif /* CONFIG_HAVE_SETUP_PER_CPU_AREA */
Signature · 1 line
1.6.0.2
Tejun Heo
maintainerPATCH 2/7 linker script: throw away .discard section
In reply to Tejun Heox86 throws away .discard section but no other archs do. Also, .discard is not thrown away while linking modules. Make every arch and module linking throw it away. This will be used to define dummy variables for percpu declarations and definitions.
This patch is based on Ivan Kokshaysky's alpha percpu patch.
[ Impact: always throw away everything in .discard ]
--- Makefile | 2 +- arch/alpha/kernel/vmlinux.lds.S | 1 + arch/arm/kernel/vmlinux.lds.S | 1 + arch/avr32/kernel/vmlinux.lds.S | 1 + arch/blackfin/kernel/vmlinux.lds.S | 1 + arch/cris/kernel/vmlinux.lds.S | 1 + arch/frv/kernel/vmlinux.lds.S | 2 ++ arch/h8300/kernel/vmlinux.lds.S | 1 + arch/ia64/kernel/vmlinux.lds.S | 1 + arch/m32r/kernel/vmlinux.lds.S | 1 + arch/m68k/kernel/vmlinux-std.lds | 1 + arch/m68k/kernel/vmlinux-sun3.lds | 1 + arch/m68knommu/kernel/vmlinux.lds.S | 1 + arch/microblaze/kernel/vmlinux.lds.S | 2 ++ arch/mips/kernel/vmlinux.lds.S | 1 + arch/mn10300/kernel/vmlinux.lds.S | 1 + arch/parisc/kernel/vmlinux.lds.S | 1 + arch/powerpc/kernel/vmlinux.lds.S | 1 + arch/s390/kernel/vmlinux.lds.S | 1 + arch/sh/kernel/vmlinux_32.lds.S | 1 + arch/sh/kernel/vmlinux_64.lds.S | 1 + arch/sparc/kernel/vmlinux.lds.S | 1 + arch/um/kernel/dyn.lds.S | 2 ++ arch/um/kernel/uml.lds.S | 2 ++ arch/xtensa/kernel/vmlinux.lds.S | 1 + scripts/module-common.lds | 8 ++++++++ 26 files changed, 37 insertions(+), 1 deletions(-) create mode 100644 scripts/module-common.lds
Makefile +1 -1
@@ -335,7 +335,7 @@ CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \ MODFLAGS = -DMODULE CFLAGS_MODULE = $(MODFLAGS) AFLAGS_MODULE = $(MODFLAGS) -LDFLAGS_MODULE = +LDFLAGS_MODULE = -T $(srctree)/scripts/module-common.lds CFLAGS_KERNEL = AFLAGS_KERNEL =
arch/alpha/kernel/vmlinux.lds.S +1 -0
@@ -139,6 +139,7 @@ SECTIONS EXIT_TEXT EXIT_DATA *(.exitcall.exit) + *(.discard) } .mdebug 0 : {
arch/arm/kernel/vmlinux.lds.S +1 -0
@@ -82,6 +82,7 @@ SECTIONS EXIT_TEXT EXIT_DATA *(.exitcall.exit) + *(.discard) *(.ARM.exidx.exit.text) *(.ARM.extab.exit.text) #ifndef CONFIG_MMU
arch/avr32/kernel/vmlinux.lds.S +1 -0
@@ -131,6 +131,7 @@ SECTIONS /DISCARD/ : { EXIT_DATA *(.exitcall.exit) + *(.discard) } DWARF_DEBUG
arch/blackfin/kernel/vmlinux.lds.S +1 -0
@@ -262,5 +262,6 @@ SECTIONS EXIT_TEXT EXIT_DATA *(.exitcall.exit) + *(.discard) } }
arch/cris/kernel/vmlinux.lds.S +1 -0
@@ -145,6 +145,7 @@ SECTIONS EXIT_TEXT EXIT_DATA *(.exitcall.exit) + *(.discard) } dram_end = dram_start + (CONFIG_ETRAX_DRAM_SIZE - __CONFIG_ETRAX_VMEM_SIZE)*1024*1024;
arch/frv/kernel/vmlinux.lds.S +2 -0
@@ -177,6 +177,8 @@ SECTIONS .debug_ranges 0 : { *(.debug_ranges) } .comment 0 : { *(.comment) } + + /DISCARD/ : { *(.discard) } } __kernel_image_size_no_bss = __bss_start - __kernel_image_start;
arch/h8300/kernel/vmlinux.lds.S +1 -0
@@ -154,6 +154,7 @@ SECTIONS } /DISCARD/ : { *(.exitcall.exit) + *(.discard) } .romfs : {
arch/ia64/kernel/vmlinux.lds.S +1 -0
@@ -29,6 +29,7 @@ SECTIONS EXIT_TEXT EXIT_DATA *(.exitcall.exit) + *(.discard) *(.IA_64.unwind.exit.text) *(.IA_64.unwind_info.exit.text) }
arch/m32r/kernel/vmlinux.lds.S +1 -0
@@ -125,6 +125,7 @@ SECTIONS EXIT_TEXT EXIT_DATA *(.exitcall.exit) + *(.discard) } /* Stabs debugging sections. */
arch/m68k/kernel/vmlinux-std.lds +1 -0
@@ -87,6 +87,7 @@ SECTIONS EXIT_TEXT EXIT_DATA *(.exitcall.exit) + *(.discard) } /* Stabs debugging sections. */
arch/m68k/kernel/vmlinux-sun3.lds +1 -0
@@ -82,6 +82,7 @@ __init_begin = .; EXIT_TEXT EXIT_DATA *(.exitcall.exit) + *(.discard) } .crap : {
arch/m68knommu/kernel/vmlinux.lds.S +1 -0
@@ -188,6 +188,7 @@ SECTIONS { EXIT_TEXT EXIT_DATA *(.exitcall.exit) + *(.discard) } .bss : {
arch/microblaze/kernel/vmlinux.lds.S +2 -0
@@ -160,4 +160,6 @@ SECTIONS { } . = ALIGN(4096); _end = .; + + /DISCARD/ : { *(.discard) } }
arch/mips/kernel/vmlinux.lds.S +1 -0
@@ -179,6 +179,7 @@ SECTIONS /* Sections to be discarded */ /DISCARD/ : { *(.exitcall.exit) + *(.discard) /* ABI crap starts here */ *(.MIPS.options)
arch/mn10300/kernel/vmlinux.lds.S +1 -0
@@ -146,6 +146,7 @@ SECTIONS /* Sections to be discarded */ /DISCARD/ : { *(.exitcall.exit) + *(.discard) } STABS_DEBUG
arch/parisc/kernel/vmlinux.lds.S +1 -0
@@ -240,6 +240,7 @@ SECTIONS /* Sections to be discarded */ /DISCARD/ : { *(.exitcall.exit) + *(.discard) #ifdef CONFIG_64BIT /* temporary hack until binutils is fixed to not emit these * for static binaries
arch/powerpc/kernel/vmlinux.lds.S +1 -0
@@ -40,6 +40,7 @@ SECTIONS /* Sections to be discarded. */ /DISCARD/ : { *(.exitcall.exit) + *(.discard) EXIT_DATA }
arch/s390/kernel/vmlinux.lds.S +1 -0
@@ -160,6 +160,7 @@ SECTIONS /DISCARD/ : { EXIT_DATA *(.exitcall.exit) + *(.discard) } /* Debugging sections. */
arch/sh/kernel/vmlinux_32.lds.S +1 -0
@@ -147,6 +147,7 @@ SECTIONS */ /DISCARD/ : { *(.exitcall.exit) + *(.discard) } STABS_DEBUG
arch/sh/kernel/vmlinux_64.lds.S +1 -0
@@ -156,6 +156,7 @@ SECTIONS */ /DISCARD/ : { *(.exitcall.exit) + *(.discard) } STABS_DEBUG
arch/sparc/kernel/vmlinux.lds.S +1 -0
@@ -175,6 +175,7 @@ SECTIONS EXIT_TEXT EXIT_DATA *(.exitcall.exit) + *(.discard) } STABS_DEBUG
arch/um/kernel/dyn.lds.S +2 -0
@@ -156,4 +156,6 @@ SECTIONS STABS_DEBUG DWARF_DEBUG + + /DISCARD/ : { *(.discard) } }
arch/um/kernel/uml.lds.S +2 -0
@@ -100,4 +100,6 @@ SECTIONS STABS_DEBUG DWARF_DEBUG + + /DISCARD/ : { *(.discard) } }
arch/xtensa/kernel/vmlinux.lds.S +1 -0
@@ -287,6 +287,7 @@ SECTIONS EXIT_TEXT EXIT_DATA *(.exitcall.exit) + *(.discard) } .xt.lit : { *(.xt.lit) }
scripts/module-common.lds new file +8 -0
@@ -0,0 +1,8 @@ +/* + * Common module linker script, always used when linking a module. + * Archs are free to supply their own linker scripts. ld will + * combine them automatically. + */ +SECTIONS { + /DISCARD/ : { *(.discard) } +}
Signature · 1 line
1.6.0.2
Re: [PATCH 2/7] linker script: throw away .discard section
Quoted 4 lines · Tejun Heo
x86 throws away .discard section but no other archs do. Also, .discard is not thrown away while linking modules. Make every arch and module linking throw it away. This will be used to define dummy variables for percpu declarations and definitions.
seems like a good time to introduce a common DISCARD define to include/asm-generic/vmlinux.lds.h. considering multiple /DISCARD/ entries are allowed in a linker script, the define should be pretty straightforward:
#define DISCARDS \
/DISCARD/ : { \
EXIT_TEXT \
EXIT_DATA \
*(.exitcall.exit) \
*(.discard) \
}-mike
Tejun Heo
maintainerRe: [PATCH 2/7] linker script: throw away .discard section
Quoted 10 lines · Mike Frysinger
Quoted 4 lines · Tejun Heo
x86 throws away .discard section but no other archs do. Also, .discard is not thrown away while linking modules. Make every arch and module linking throw it away. This will be used to define dummy variables for percpu declarations and definitions.
seems like a good time to introduce a common DISCARD define to
include/asm-generic/vmlinux.lds.h. considering multiple /DISCARD/ entries are allowed in a linker script, the define should be pretty straightforward:
Quoted 7 lines
#define DISCARDS \
/DISCARD/ : { \
EXIT_TEXT \
EXIT_DATA \
*(.exitcall.exit) \
*(.discard) \
}Hmmm... indeed, that would be nice. I'll update the patch.
Thanks.
Signature · 1 line
tejun
Tejun Heo
maintainerRe: [PATCH 2/7] linker script: throw away .discard section
Quoted 18 lines · Tejun Heo
Quoted 9 lines · Mike Frysinger
Quoted 4 lines · Tejun Heo
x86 throws away .discard section but no other archs do. Also, .discard is not thrown away while linking modules. Make every arch and module linking throw it away. This will be used to define dummy variables for percpu declarations and definitions.
seems like a good time to introduce a common DISCARD define to
include/asm-generic/vmlinux.lds.h. considering multiple /DISCARD/ entries are allowed in a linker script, the define should be pretty straightforward:Quoted 7 lines
#define DISCARDS \ /DISCARD/ : { \ EXIT_TEXT \ EXIT_DATA \ *(.exitcall.exit) \ *(.discard) \ }
Quoted 1 line
Hmmm... indeed, that would be nice. I'll update the patch.
Eh... handling of EXIT_TEXT and DATA aren't uniform across archs. I think I'll leave the /DISCARD/ unification for the next time.
Thanks.
Signature · 1 line
tejun
Re: [PATCH 2/7] linker script: throw away .discard section
Quoted 19 lines · Tejun Heo
Quoted 18 lines · Tejun Heo
Quoted 16 lines · Mike Frysinger
Quoted 4 lines · Tejun Heo
x86 throws away .discard section but no other archs do. Also, .discard is not thrown away while linking modules. Make every arch and module linking throw it away. This will be used to define dummy variables for percpu declarations and definitions.
seems like a good time to introduce a common DISCARD define to
include/asm-generic/vmlinux.lds.h. considering multiple /DISCARD/ entries are allowed in a linker script, the define should be pretty straightforward:
#define DISCARDS \
/DISCARD/ : { \
EXIT_TEXT\
EXIT_DATA\
*(.exitcall.exit) \
*(.discard) \
}
Quoted 4 lines
Quoted 1 line
Hmmm... indeed, that would be nice. I'll update the patch.
Eh... handling of
EXIT_TEXTand DATA aren't uniform across archs. I think I'll leave the /DISCARD/ unification for the next time.
right, i think that's a bug where people simply forgot to declare them. if you stick DISCARDS at the end (and only replace when an arch already has it at the end), there should be no change in behavior. -mike
Tejun Heo
maintainerPATCH 3/7 percpu: clean up percpu variable definitions
In reply to Tejun HeoPercpu variable definition is about to be updated such that
* percpu symbols must be unique even the static ones
* in-function static definition is not allowed
Update percpu variable definitions accoringly.
* as,cfq: rename ioc_count uniquely
* cpufreq: rename cpu_dbs_info uniquely
* xen: move nesting_count out of xen_evtchn_do_upcall() and rename it
* mm: move ratelimits out of balance_dirty_pages_ratelimited_nr() and rename it
* ipv4,6: rename cookie_scratch uniquely
While at it, make cris:use DECLARE_PER_CPU() instead of extern volatile DEFINE_PER_CPU() for declaration.
[ Impact: percpu usage cleanups, no duplicate static percpu var names ]
--- arch/cris/include/asm/mmu_context.h | 2 +- block/as-iosched.c | 10 +++++----- block/cfq-iosched.c | 10 +++++----- drivers/cpufreq/cpufreq_conservative.c | 12 ++++++------ drivers/cpufreq/cpufreq_ondemand.c | 15 ++++++++------- drivers/xen/events.c | 9 +++++---- mm/page-writeback.c | 5 +++-- net/ipv4/syncookies.c | 4 ++-- net/ipv6/syncookies.c | 4 ++-- 9 files changed, 37 insertions(+), 34 deletions(-)
arch/cris/include/asm/mmu_context.h +1 -1
@@ -17,7 +17,7 @@ extern void switch_mm(struct mm_struct *prev, struct mm_struct *next,* registers like cr3 on the i386 */ -extern volatile DEFINE_PER_CPU(pgd_t *,current_pgd); /* defined in arch/cris/mm/fault.c */ +DECLARE_PER_CPU(pgd_t *,current_pgd); /* defined in arch/cris/mm/fault.c */static inline void enter_lazy_tlb (struct mm_struct *mm,struct task_struct *tsk) {
block/as-iosched.c +5 -5
@@ -146,7 +146,7 @@ enum arq_state {#define RQ_STATE (rq) ((enum arq_state)(rq)->elevator_private2)#define RQ_SET_STATE (rq, state) ((rq)->elevator_private2 = (void *) state) -static DEFINE_PER_CPU(unsigned long, ioc_count); +static DEFINE_PER_CPU(unsigned long, as_ioc_count);static struct completion *ioc_gone;static DEFINE_SPINLOCK (ioc_gone_lock);
@@ -161,7 +161,7 @@ static void as_antic_stop(struct as_data *ad);static void free_as_io_context (struct as_io_context *aic) {kfree (aic); -elv_ioc_count_dec (ioc_count); +elv_ioc_count_dec (as_ioc_count);if (ioc_gone) {/* * AS scheduler is exiting, grab exit lock and check
@@ -169,7 +169,7 @@ static void free_as_io_context(struct as_io_context *aic)* complete ioc_gone and set it back to NULL. */ spin_lock (&ioc_gone_lock); -if (ioc_gone && !elv_ioc_count_read (ioc_count)) { +if (ioc_gone && !elv_ioc_count_read (as_ioc_count)) {complete (ioc_gone); ioc_gone =NULL ; }
@@ -211,7 +211,7 @@ static struct as_io_context *alloc_as_io_context(void) ret->seek_total =0 ; ret->seek_samples =0 ; ret->seek_mean =0 ; -elv_ioc_count_inc (ioc_count); +elv_ioc_count_inc (as_ioc_count); }return ret;
@@ -1509,7 +1509,7 @@ static void __exit as_exit(void) ioc_gone = &all_gone;/* ioc_gone's update must be visible before reading ioc_count */ smp_wmb (); -if (elv_ioc_count_read (ioc_count)) +if (elv_ioc_count_read (as_ioc_count))wait_for_completion (&all_gone);synchronize_rcu (); }
block/cfq-iosched.c +5 -5
@@ -48,7 +48,7 @@ static int cfq_slice_idle = HZ / 125;static struct kmem_cache *cfq_pool;static struct kmem_cache *cfq_ioc_pool; -static DEFINE_PER_CPU(unsigned long, ioc_count); +static DEFINE_PER_CPU(unsigned long, cfq_ioc_count);static struct completion *ioc_gone;static DEFINE_SPINLOCK (ioc_gone_lock);
@@ -1423,7 +1423,7 @@ static void cfq_cic_free_rcu(struct rcu_head *head) cic = container_of(head, struct cfq_io_context, rcu_head);kmem_cache_free (cfq_ioc_pool, cic); -elv_ioc_count_dec (ioc_count); +elv_ioc_count_dec (cfq_ioc_count);if (ioc_gone) {/*
@@ -1432,7 +1432,7 @@ static void cfq_cic_free_rcu(struct rcu_head *head)* complete ioc_gone and set it back to NULL */ spin_lock (&ioc_gone_lock); -if (ioc_gone && !elv_ioc_count_read (ioc_count)) { +if (ioc_gone && !elv_ioc_count_read (cfq_ioc_count)) {complete (ioc_gone); ioc_gone =NULL ; }
@@ -1558,7 +1558,7 @@ cfq_alloc_io_context(struct cfq_data *cfqd, gfp_t gfp_mask)INIT_HLIST_NODE (&cic->cic_list); cic->dtor = cfq_free_io_context; cic->exit = cfq_exit_io_context; -elv_ioc_count_inc (ioc_count); +elv_ioc_count_inc (cfq_ioc_count); }return cic;
@@ -2663,7 +2663,7 @@ static void __exit cfq_exit(void)* this also protects us from entering cfq_slab_kill() with * pending RCU callbacks */ -if (elv_ioc_count_read (ioc_count)) +if (elv_ioc_count_read (cfq_ioc_count))wait_for_completion (&all_gone);cfq_slab_kill (); }
drivers/cpufreq/cpufreq_conservative.c +6 -6
@@ -80,7 +80,7 @@ struct cpu_dbs_info_s {int cpu; unsigned int enable:1; }; -static DEFINE_PER_CPU(struct cpu_dbs_info_s, cpu_dbs_info); +static DEFINE_PER_CPU(struct cpu_dbs_info_s, cs_cpu_dbs_info);static unsigned int dbs_enable;/* number of CPUs using this policy */
@@ -150,7 +150,7 @@ dbs_cpufreq_notifier(struct notifier_block *nb, unsigned long val, void *data) {struct cpufreq_freqs *freq = data; -struct cpu_dbs_info_s *this_dbs_info = &per_cpu (cpu_dbs_info, +struct cpu_dbs_info_s *this_dbs_info = &per_cpu (cs_cpu_dbs_info, freq->cpu);struct cpufreq_policy *policy;
@@ -323,7 +323,7 @@ static ssize_t store_ignore_nice_load(struct cpufreq_policy *policy,/* we need to re-evaluate prev_cpu_idle */ for_each_online_cpu (j) {struct cpu_dbs_info_s *dbs_info; - dbs_info = &per_cpu (cpu_dbs_info, j); + dbs_info = &per_cpu (cs_cpu_dbs_info, j); dbs_info->prev_cpu_idle =get_cpu_idle_time (j, &dbs_info->prev_cpu_wall);if (dbs_tuners_ins.ignore_nice)
@@ -413,7 +413,7 @@ static void dbs_check_cpu(struct cpu_dbs_info_s *this_dbs_info)cputime64_t cur_wall_time, cur_idle_time;unsigned int idle_time, wall_time; - j_dbs_info = &per_cpu (cpu_dbs_info, j); + j_dbs_info = &per_cpu (cs_cpu_dbs_info, j); cur_idle_time =get_cpu_idle_time (j, &cur_wall_time);
@@ -553,7 +553,7 @@ static int cpufreq_governor_dbs(struct cpufreq_policy *policy,unsigned int j;int rc; - this_dbs_info = &per_cpu (cpu_dbs_info, cpu); + this_dbs_info = &per_cpu (cs_cpu_dbs_info, cpu);switch (event) {case CPUFREQ_GOV_START :
@@ -573,7 +573,7 @@ static int cpufreq_governor_dbs(struct cpufreq_policy *policy,for_each_cpu (j, policy->cpus) {struct cpu_dbs_info_s *j_dbs_info; - j_dbs_info = &per_cpu (cpu_dbs_info, j); + j_dbs_info = &per_cpu (cs_cpu_dbs_info, j); j_dbs_info->cur_policy = policy; j_dbs_info->prev_cpu_idle = get_cpu_idle_time(j,
drivers/cpufreq/cpufreq_ondemand.c +8 -7
@@ -87,7 +87,7 @@ struct cpu_dbs_info_s { unsigned int enable:1, sample_type:1; }; -static DEFINE_PER_CPU(struct cpu_dbs_info_s, cpu_dbs_info); +static DEFINE_PER_CPU(struct cpu_dbs_info_s, od_cpu_dbs_info);static unsigned int dbs_enable;/* number of CPUs using this policy */
@@ -162,7 +162,8 @@ static unsigned int powersave_bias_target(struct cpufreq_policy *policy,unsigned int freq_hi, freq_lo;unsigned int index =0 ;unsigned int jiffies_total, jiffies_hi, jiffies_lo; -struct cpu_dbs_info_s *dbs_info = &per_cpu (cpu_dbs_info, policy->cpu); +struct cpu_dbs_info_s *dbs_info = &per_cpu (od_cpu_dbs_info, + policy->cpu);if (!dbs_info->freq_table) { dbs_info->freq_lo =0 ;
@@ -207,7 +208,7 @@ static void ondemand_powersave_bias_init(void) {int i;for_each_online_cpu (i) { -struct cpu_dbs_info_s *dbs_info = &per_cpu (cpu_dbs_info, i); +struct cpu_dbs_info_s *dbs_info = &per_cpu (od_cpu_dbs_info, i); dbs_info->freq_table =cpufreq_frequency_get_table (i); dbs_info->freq_lo =0 ; }
@@ -322,7 +323,7 @@ static ssize_t store_ignore_nice_load(struct cpufreq_policy *policy,/* we need to re-evaluate prev_cpu_idle */ for_each_online_cpu (j) {struct cpu_dbs_info_s *dbs_info; - dbs_info = &per_cpu (cpu_dbs_info, j); + dbs_info = &per_cpu (od_cpu_dbs_info, j); dbs_info->prev_cpu_idle =get_cpu_idle_time (j, &dbs_info->prev_cpu_wall);if (dbs_tuners_ins.ignore_nice)
@@ -416,7 +417,7 @@ static void dbs_check_cpu(struct cpu_dbs_info_s *this_dbs_info)unsigned int load, load_freq;int freq_avg; - j_dbs_info = &per_cpu (cpu_dbs_info, j); + j_dbs_info = &per_cpu (od_cpu_dbs_info, j); cur_idle_time =get_cpu_idle_time (j, &cur_wall_time);
@@ -573,7 +574,7 @@ static int cpufreq_governor_dbs(struct cpufreq_policy *policy,unsigned int j;int rc; - this_dbs_info = &per_cpu (cpu_dbs_info, cpu); + this_dbs_info = &per_cpu (od_cpu_dbs_info, cpu);switch (event) {case CPUFREQ_GOV_START :
@@ -595,7 +596,7 @@ static int cpufreq_governor_dbs(struct cpufreq_policy *policy,for_each_cpu (j, policy->cpus) {struct cpu_dbs_info_s *j_dbs_info; - j_dbs_info = &per_cpu (cpu_dbs_info, j); + j_dbs_info = &per_cpu (od_cpu_dbs_info, j); j_dbs_info->cur_policy = policy; j_dbs_info->prev_cpu_idle = get_cpu_idle_time(j,
drivers/xen/events.c +5 -4
@@ -596,6 +596,8 @@ irqreturn_t xen_debug_interrupt(int irq, void *dev_id)return IRQ_HANDLED ; } +static DEFINE_PER_CPU(unsigned, xed_nesting_count); +/* * Search the CPUs pending events bitmasks. For each one found, map * the event number to an irq, and feed it into do_IRQ() for
@@ -611,7 +613,6 @@ void xen_evtchn_do_upcall(struct pt_regs *regs)struct pt_regs *old_regs =set_irq_regs (regs);struct shared_info *s = HYPERVISOR_shared_info;struct vcpu_info *vcpu_info =__get_cpu_var (xen_vcpu); - static DEFINE_PER_CPU(unsigned, nesting_count);unsigned count;exit_idle ();
@@ -622,7 +623,7 @@ void xen_evtchn_do_upcall(struct pt_regs *regs) vcpu_info->evtchn_upcall_pending =0 ; -if (__get_cpu_var (nesting_count)++) +if (__get_cpu_var (xed_nesting_count)++) goto out;#ifndef CONFIG_X86 /* No need for a barrier -- XCHG is a barrier on x86. */
@@ -647,8 +648,8 @@ void xen_evtchn_do_upcall(struct pt_regs *regs)BUG_ON (!irqs_disabled ()); - count =__get_cpu_var (nesting_count); -__get_cpu_var (nesting_count) =0 ; + count =__get_cpu_var (xed_nesting_count); +__get_cpu_var (xed_nesting_count) =0 ; }while (count !=1 ); out:
mm/page-writeback.c +3 -2
@@ -606,6 +606,8 @@ void set_page_dirty_balance(struct page *page, int page_mkwrite) } } +static DEFINE_PER_CPU(unsigned long, bdp_ratelimits) = 0; +/** * balance_dirty_pages_ratelimited_nr - balance dirty memory state * @mapping: address_space which was dirtied
@@ -623,7 +625,6 @@ void set_page_dirty_balance(struct page *page, int page_mkwrite)void balance_dirty_pages_ratelimited_nr (struct address_space *mapping,unsigned long nr_pages_dirtied) { - static DEFINE_PER_CPU(unsigned long, ratelimits) = 0;unsigned long ratelimit;unsigned long *p;
@@ -636,7 +637,7 @@ void balance_dirty_pages_ratelimited_nr(struct address_space *mapping,* tasks in balance_dirty_pages(). Period. */ preempt_disable (); - p = &__get_cpu_var (ratelimits); + p = &__get_cpu_var (bdp_ratelimits); *p += nr_pages_dirtied;if (unlikely (*p >= ratelimit)) { *p =0 ;
net/ipv4/syncookies.c +2 -2
@@ -37,12 +37,12 @@ __initcall(init_syncookies);#define COOKIEBITS 24/* Upper bits store count */ #define COOKIEMASK (((__u32)1 << COOKIEBITS) - 1) -static DEFINE_PER_CPU(__u32, cookie_scratch)[16 + 5 + SHA_WORKSPACE_WORDS]; +static DEFINE_PER_CPU(__u32, ipv4_cookie_scratch)[16 + 5 + SHA_WORKSPACE_WORDS];static u32 cookie_hash (__be32 saddr,__be32 daddr,__be16 sport,__be16 dport,u32 count,int c) { -__u32 *tmp =__get_cpu_var (cookie_scratch); +__u32 *tmp =__get_cpu_var (ipv4_cookie_scratch);memcpy (tmp +4 , syncookie_secret[c],sizeof (syncookie_secret[c])); tmp[0 ] = (__force u32 )saddr;
net/ipv6/syncookies.c +2 -2
@@ -74,12 +74,12 @@ static inline struct sock *get_cookie_sock(struct sock *sk, struct sk_buff *skb,return child; } -static DEFINE_PER_CPU(__u32, cookie_scratch)[16 + 5 + SHA_WORKSPACE_WORDS]; +static DEFINE_PER_CPU(__u32, ipv6_cookie_scratch)[16 + 5 + SHA_WORKSPACE_WORDS];static u32 cookie_hash (struct in6_addr *saddr,struct in6_addr *daddr,__be16 sport,__be16 dport,u32 count,int c) { -__u32 *tmp =__get_cpu_var (cookie_scratch); +__u32 *tmp =__get_cpu_var (ipv6_cookie_scratch);/* * we have 320 bits of information to hash, copy in the remaining
Signature · 1 line
1.6.0.2
Re: [PATCH 3/7] percpu: clean up percpu variable definitions
Quoted 17 lines · Tejun Heo
Percpu variable definition is about to be updated such that
* percpu symbols must be unique even the static ones
* in-function static definition is not allowed
Update percpu variable definitions accoringly.
* as,cfq: rename
ioc_countuniquely* cpufreq: rename
cpu_dbs_infouniquely* xen: move
nesting_countout ofxen_evtchn_do_upcall()and rename it* mm: move ratelimits out of
balance_dirty_pages_ratelimited_nr()and rename it
Quoted 6 lines
* ipv4,6: rename
cookie_scratchuniquelyWhile at it, make cris:use
DECLARE_PER_CPU()instead of extern volatileDEFINE_PER_CPU()for declaration.[ Impact: percpu usage cleanups, no duplicate static percpu var names ]
The block bits looks fine.
Quoted 374 lines
Signed-off-by Tejun Heo <[email protected]>Cc Ivan Kokshaysky <[email protected]>Cc Jens Axboe <[email protected]>Cc Dave Jones <[email protected]>Cc Jeremy Fitzhardinge <[email protected]>Cc linux-mm <[email protected]>Cc David S. Miller <[email protected]>--- arch/cris/include/asm/mmu_context.h | 2 +- block/as-iosched.c | 10 +++++----- block/cfq-iosched.c | 10 +++++----- drivers/cpufreq/cpufreq_conservative.c | 12 ++++++------ drivers/cpufreq/cpufreq_ondemand.c | 15 ++++++++------- drivers/xen/events.c | 9 +++++---- mm/page-writeback.c | 5 +++-- net/ipv4/syncookies.c | 4 ++-- net/ipv6/syncookies.c | 4 ++-- 9 files changed, 37 insertions(+), 34 deletions(-)arch/cris/include/asm/mmu_context.h +1
-1@@ -17,7 +17,7 @@ extern void switch_mm(struct mm_struct *prev, struct mm_struct *next,* registers like cr3 on the i386 */ -extern volatile DEFINE_PER_CPU(pgd_t *,current_pgd); /* defined in arch/cris/mm/fault.c */ +DECLARE_PER_CPU(pgd_t *,current_pgd); /* defined in arch/cris/mm/fault.c */static inline void enter_lazy_tlb (struct mm_struct *mm,struct task_struct *tsk) {block/as-iosched.c +5
-5@@ -146,7 +146,7 @@ enum arq_state {#define RQ_STATE (rq) ((enum arq_state)(rq)->elevator_private2)#define RQ_SET_STATE (rq, state) ((rq)->elevator_private2 = (void *) state) -static DEFINE_PER_CPU(unsigned long, ioc_count); +static DEFINE_PER_CPU(unsigned long, as_ioc_count);static struct completion *ioc_gone;static DEFINE_SPINLOCK (ioc_gone_lock);@@ -161,7 +161,7 @@ static void as_antic_stop(struct as_data *ad);static void free_as_io_context (struct as_io_context *aic) {kfree (aic); -elv_ioc_count_dec (ioc_count); +elv_ioc_count_dec (as_ioc_count);if (ioc_gone) {/* * AS scheduler is exiting, grab exit lock and check @@ -169,7 +169,7 @@ static void free_as_io_context(struct as_io_context *aic)* complete ioc_gone and set it back to NULL. */ spin_lock (&ioc_gone_lock); -if (ioc_gone && !elv_ioc_count_read (ioc_count)) { +if (ioc_gone && !elv_ioc_count_read (as_ioc_count)) {complete (ioc_gone); ioc_gone =NULL ; }@@ -211,7 +211,7 @@ static struct as_io_context *alloc_as_io_context(void) ret->seek_total =0 ; ret->seek_samples =0 ; ret->seek_mean =0 ; -elv_ioc_count_inc (ioc_count); +elv_ioc_count_inc (as_ioc_count); }return ret;@@ -1509,7 +1509,7 @@ static void __exit as_exit(void) ioc_gone = &all_gone;/* ioc_gone's update must be visible before reading ioc_count */ smp_wmb (); -if (elv_ioc_count_read (ioc_count)) +if (elv_ioc_count_read (as_ioc_count))wait_for_completion (&all_gone);synchronize_rcu (); }block/cfq-iosched.c +5
-5@@ -48,7 +48,7 @@ static int cfq_slice_idle = HZ / 125;static struct kmem_cache *cfq_pool;static struct kmem_cache *cfq_ioc_pool; -static DEFINE_PER_CPU(unsigned long, ioc_count); +static DEFINE_PER_CPU(unsigned long, cfq_ioc_count);static struct completion *ioc_gone;static DEFINE_SPINLOCK (ioc_gone_lock);@@ -1423,7 +1423,7 @@ static void cfq_cic_free_rcu(struct rcu_head *head) cic = container_of(head, struct cfq_io_context, rcu_head);kmem_cache_free (cfq_ioc_pool, cic); -elv_ioc_count_dec (ioc_count); +elv_ioc_count_dec (cfq_ioc_count);if (ioc_gone) {/* @@ -1432,7 +1432,7 @@ static void cfq_cic_free_rcu(struct rcu_head *head)* complete ioc_gone and set it back to NULL */ spin_lock (&ioc_gone_lock); -if (ioc_gone && !elv_ioc_count_read (ioc_count)) { +if (ioc_gone && !elv_ioc_count_read (cfq_ioc_count)) {complete (ioc_gone); ioc_gone =NULL ; }@@ -1558,7 +1558,7 @@ cfq_alloc_io_context(struct cfq_data *cfqd, gfp_t gfp_mask)INIT_HLIST_NODE (&cic->cic_list); cic->dtor = cfq_free_io_context; cic->exit = cfq_exit_io_context; -elv_ioc_count_inc (ioc_count); +elv_ioc_count_inc (cfq_ioc_count); }return cic;@@ -2663,7 +2663,7 @@ static void __exit cfq_exit(void)* this also protects us from entering cfq_slab_kill() with * pending RCU callbacks */ -if (elv_ioc_count_read (ioc_count)) +if (elv_ioc_count_read (cfq_ioc_count))wait_for_completion (&all_gone);cfq_slab_kill (); }drivers/cpufreq/cpufreq_conservative.c +6
-6@@ -80,7 +80,7 @@ struct cpu_dbs_info_s {int cpu; unsigned int enable:1; }; -static DEFINE_PER_CPU(struct cpu_dbs_info_s, cpu_dbs_info); +static DEFINE_PER_CPU(struct cpu_dbs_info_s, cs_cpu_dbs_info);static unsigned int dbs_enable;/* number of CPUs using this policy */ @@ -150,7 +150,7 @@ dbs_cpufreq_notifier(struct notifier_block *nb, unsigned long val, void *data) {struct cpufreq_freqs *freq = data; -struct cpu_dbs_info_s *this_dbs_info = &per_cpu (cpu_dbs_info, +struct cpu_dbs_info_s *this_dbs_info = &per_cpu (cs_cpu_dbs_info, freq->cpu);struct cpufreq_policy *policy;@@ -323,7 +323,7 @@ static ssize_t store_ignore_nice_load(struct cpufreq_policy *policy,/* we need to re-evaluate prev_cpu_idle */ for_each_online_cpu (j) {struct cpu_dbs_info_s *dbs_info; - dbs_info = &per_cpu (cpu_dbs_info, j); + dbs_info = &per_cpu (cs_cpu_dbs_info, j); dbs_info->prev_cpu_idle =get_cpu_idle_time (j, &dbs_info->prev_cpu_wall);if (dbs_tuners_ins.ignore_nice)@@ -413,7 +413,7 @@ static void dbs_check_cpu(struct cpu_dbs_info_s *this_dbs_info)cputime64_t cur_wall_time, cur_idle_time;unsigned int idle_time, wall_time; - j_dbs_info = &per_cpu (cpu_dbs_info, j); + j_dbs_info = &per_cpu (cs_cpu_dbs_info, j); cur_idle_time =get_cpu_idle_time (j, &cur_wall_time);@@ -553,7 +553,7 @@ static int cpufreq_governor_dbs(struct cpufreq_policy *policy,unsigned int j;int rc; - this_dbs_info = &per_cpu (cpu_dbs_info, cpu); + this_dbs_info = &per_cpu (cs_cpu_dbs_info, cpu);switch (event) {case CPUFREQ_GOV_START :@@ -573,7 +573,7 @@ static int cpufreq_governor_dbs(struct cpufreq_policy *policy,for_each_cpu (j, policy->cpus) {struct cpu_dbs_info_s *j_dbs_info; - j_dbs_info = &per_cpu (cpu_dbs_info, j); + j_dbs_info = &per_cpu (cs_cpu_dbs_info, j); j_dbs_info->cur_policy = policy; j_dbs_info->prev_cpu_idle = get_cpu_idle_time(j,drivers/cpufreq/cpufreq_ondemand.c +8
-7@@ -87,7 +87,7 @@ struct cpu_dbs_info_s { unsigned int enable:1, sample_type:1; }; -static DEFINE_PER_CPU(struct cpu_dbs_info_s, cpu_dbs_info); +static DEFINE_PER_CPU(struct cpu_dbs_info_s, od_cpu_dbs_info);static unsigned int dbs_enable;/* number of CPUs using this policy */ @@ -162,7 +162,8 @@ static unsigned int powersave_bias_target(struct cpufreq_policy *policy,unsigned int freq_hi, freq_lo;unsigned int index =0 ;unsigned int jiffies_total, jiffies_hi, jiffies_lo; -struct cpu_dbs_info_s *dbs_info = &per_cpu (cpu_dbs_info, policy->cpu); +struct cpu_dbs_info_s *dbs_info = &per_cpu (od_cpu_dbs_info, + policy->cpu);if (!dbs_info->freq_table) { dbs_info->freq_lo =0 ;@@ -207,7 +208,7 @@ static void ondemand_powersave_bias_init(void) {int i;for_each_online_cpu (i) { -struct cpu_dbs_info_s *dbs_info = &per_cpu (cpu_dbs_info, i); +struct cpu_dbs_info_s *dbs_info = &per_cpu (od_cpu_dbs_info, i); dbs_info->freq_table =cpufreq_frequency_get_table (i); dbs_info->freq_lo =0 ; }@@ -322,7 +323,7 @@ static ssize_t store_ignore_nice_load(struct cpufreq_policy *policy,/* we need to re-evaluate prev_cpu_idle */ for_each_online_cpu (j) {struct cpu_dbs_info_s *dbs_info; - dbs_info = &per_cpu (cpu_dbs_info, j); + dbs_info = &per_cpu (od_cpu_dbs_info, j); dbs_info->prev_cpu_idle =get_cpu_idle_time (j, &dbs_info->prev_cpu_wall);if (dbs_tuners_ins.ignore_nice)@@ -416,7 +417,7 @@ static void dbs_check_cpu(struct cpu_dbs_info_s *this_dbs_info)unsigned int load, load_freq;int freq_avg; - j_dbs_info = &per_cpu (cpu_dbs_info, j); + j_dbs_info = &per_cpu (od_cpu_dbs_info, j); cur_idle_time =get_cpu_idle_time (j, &cur_wall_time);@@ -573,7 +574,7 @@ static int cpufreq_governor_dbs(struct cpufreq_policy *policy,unsigned int j;int rc; - this_dbs_info = &per_cpu (cpu_dbs_info, cpu); + this_dbs_info = &per_cpu (od_cpu_dbs_info, cpu);switch (event) {case CPUFREQ_GOV_START :@@ -595,7 +596,7 @@ static int cpufreq_governor_dbs(struct cpufreq_policy *policy,for_each_cpu (j, policy->cpus) {struct cpu_dbs_info_s *j_dbs_info; - j_dbs_info = &per_cpu (cpu_dbs_info, j); + j_dbs_info = &per_cpu (od_cpu_dbs_info, j); j_dbs_info->cur_policy = policy; j_dbs_info->prev_cpu_idle = get_cpu_idle_time(j,drivers/xen/events.c +5
-4@@ -596,6 +596,8 @@ irqreturn_t xen_debug_interrupt(int irq, void *dev_id)return IRQ_HANDLED ; } +static DEFINE_PER_CPU(unsigned, xed_nesting_count); +/* * Search the CPUs pending events bitmasks. For each one found, map * the event number to an irq, and feed it into do_IRQ() for @@ -611,7 +613,6 @@ void xen_evtchn_do_upcall(struct pt_regs *regs)struct pt_regs *old_regs =set_irq_regs (regs);struct shared_info *s = HYPERVISOR_shared_info;struct vcpu_info *vcpu_info =__get_cpu_var (xen_vcpu); - static DEFINE_PER_CPU(unsigned, nesting_count);unsigned count;exit_idle ();@@ -622,7 +623,7 @@ void xen_evtchn_do_upcall(struct pt_regs *regs) vcpu_info->evtchn_upcall_pending =0 ; -if (__get_cpu_var (nesting_count)++) +if (__get_cpu_var (xed_nesting_count)++) goto out;#ifndef CONFIG_X86 /* No need for a barrier -- XCHG is a barrier on x86. */ @@ -647,8 +648,8 @@ void xen_evtchn_do_upcall(struct pt_regs *regs)BUG_ON (!irqs_disabled ()); - count =__get_cpu_var (nesting_count); -__get_cpu_var (nesting_count) =0 ; + count =__get_cpu_var (xed_nesting_count); +__get_cpu_var (xed_nesting_count) =0 ; }while (count !=1 ); out:mm/page-writeback.c +3
-2@@ -606,6 +606,8 @@ void set_page_dirty_balance(struct page *page, int page_mkwrite) } } +static DEFINE_PER_CPU(unsigned long, bdp_ratelimits) = 0; +/** * balance_dirty_pages_ratelimited_nr - balance dirty memory state * @mapping: address_space which was dirtied @@ -623,7 +625,6 @@ void set_page_dirty_balance(struct page *page, int page_mkwrite)void balance_dirty_pages_ratelimited_nr (struct address_space *mapping,unsigned long nr_pages_dirtied) { - static DEFINE_PER_CPU(unsigned long, ratelimits) = 0;unsigned long ratelimit;unsigned long *p;@@ -636,7 +637,7 @@ void balance_dirty_pages_ratelimited_nr(struct address_space *mapping,* tasks in balance_dirty_pages(). Period. */ preempt_disable (); - p = &__get_cpu_var (ratelimits); + p = &__get_cpu_var (bdp_ratelimits); *p += nr_pages_dirtied;if (unlikely (*p >= ratelimit)) { *p =0 ;net/ipv4/syncookies.c +2
-2@@ -37,12 +37,12 @@ __initcall(init_syncookies);#define COOKIEBITS 24/* Upper bits store count */ #define COOKIEMASK (((__u32)1 << COOKIEBITS) - 1) -static DEFINE_PER_CPU(__u32, cookie_scratch)[16 + 5 + SHA_WORKSPACE_WORDS]; +static DEFINE_PER_CPU(__u32, ipv4_cookie_scratch)[16 + 5 + SHA_WORKSPACE_WORDS];static u32 cookie_hash (__be32 saddr,__be32 daddr,__be16 sport,__be16 dport,u32 count,int c) { -__u32 *tmp =__get_cpu_var (cookie_scratch); +__u32 *tmp =__get_cpu_var (ipv4_cookie_scratch);memcpy (tmp +4 , syncookie_secret[c],sizeof (syncookie_secret[c])); tmp[0 ] = (__force u32 )saddr;net/ipv6/syncookies.c +2
-2@@ -74,12 +74,12 @@ static inline struct sock *get_cookie_sock(struct sock *sk, struct sk_buff *skb,return child; } -static DEFINE_PER_CPU(__u32, cookie_scratch)[16 + 5 + SHA_WORKSPACE_WORDS]; +static DEFINE_PER_CPU(__u32, ipv6_cookie_scratch)[16 + 5 + SHA_WORKSPACE_WORDS];static u32 cookie_hash (struct in6_addr *saddr,struct in6_addr *daddr,__be16 sport,__be16 dport,u32 count,int c) { -__u32 *tmp =__get_cpu_var (cookie_scratch); +__u32 *tmp =__get_cpu_var (ipv6_cookie_scratch);/* * we have 320 bits of information to hash, copy in the remaining Signature · 7 lines
1.6.0.2 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Signature · 7 lines
Jens Axboe -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH 3/7] percpu: clean up percpu variable definitions
In reply to Tejun HeoQuoted 5 lines · Tejun Heo
Percpu variable definition is about to be updated such that
* percpu symbols must be unique even the static ones
* in-function static definition is not allowed
That spluttering noise is be choking on the title of this patch :)
Making these pseudo statics is in no way a cleanup. How about we just say "they can't be static" and do something like:
/* Sorry, can't be static: that breaks archs which need these weak. */ #define DEFINE_PER_CPU(type, var) \
extern typeof(type) var; DEFINE_PER_CPU_SECTION(type, name, "")
Thanks,
Rusty.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Tejun Heo
maintainerRe: [PATCH 3/7] percpu: clean up percpu variable definitions
Quoted 7 lines · Rusty Russell
Quoted 5 lines · Tejun Heo
Percpu variable definition is about to be updated such that
* percpu symbols must be unique even the static ones
* in-function static definition is not allowed
Quoted 8 lines
That spluttering noise is be choking on the title of this patch :)
Making these pseudo statics is in no way a cleanup. How about we just say "they can't be static" and do something like:
/* Sorry, can't be static: that breaks archs which need these weak. */ #define
DEFINE_PER_CPU(type, var)\
extern typeof(type) var;DEFINE_PER_CPU_SECTION(type, name, "")
Heh... well, even though I authored the patch, I kind of agree with you. Maybe it would be better to simply disallow static declaration / definition at all. I wanted to give a go at the original idea as it seemed to have some potential. The result isn't too disappointing but I can't really say there are distinctively compelling advantages to justify the added complexity and subtlety.
What do others think? Is everyone happy with going extern only?
Thanks.
Signature · 7 lines
tejun -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Tejun Heo
maintainerPATCH 4/7 percpu: enforce global uniqueness and disallow in-function statics
In reply to Tejun HeoSome archs (alpha and s390) need to add 'weak' attribute to percpu variable definitions so that the compiler generates external references for them. To allow this, all percpu variables should be unique regardless of scope and in-function static definitions can't be allowed.
This patch updates DECLARE_PER_CPU_SECTION() and DEFINE_PER_CPU_SECTION() such that percpu symbol itself is always defined global so that weak can be added via PER_CPU_ATTRIBUTES. The scope and uniqueness rules are enforced using two dummy variables, both of which are thrown away during linking. The multiline definition combined with section specification prevents in-function static definitions.
This patch is inspired by Ivan Kokshaysky's alpha percpu patch.
[ Impact: stricter rules for percpu variable definitions ]
--- include/linux/percpu-defs.h | 34 +++++++++++++++++++++++++--------- 1 files changed, 25 insertions(+), 9 deletions(-)
include/linux/percpu-defs.h +25 -9
@@ -10,21 +10,37 @@/* * Base implementations of per-CPU variable declarations and definitions, where * the section in which the variable is to be placed is provided by the -* 'section' argument. This may be used to affect the parameters governing the +* 'sec' argument. This may be used to affect the parameters governing the * variable's storage. * +* Percpu variable declaration and definition decouple scope (static or extern) +* specification from the actual percpu variable and use two dummy variables +* to ensure DECLARE_PER_CPU_SECTION() is not coupled with static definition +* and each percpu symbol is unique regardless of scope. This is for archs +* which need to add 'weak' attribute to per cpu variables to force the +* compiler to generate external references for percpu variables (alpha and +* s390). This also means that in-function static percpu variables aren't +* allowed. +* * NOTE! The sections for the DECLARE and for the DEFINE must match, lest * linkage errors occur due the compiler generating the wrong code to access * that section. */ -#define DECLARE_PER_CPU_SECTION (type, name, section) \ - extern \ - __attribute__((__section__(PER_CPU_BASE_SECTION section))) \ - PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name - -#define DEFINE_PER_CPU_SECTION (type, name, section) \ - __attribute__((__section__(PER_CPU_BASE_SECTION section))) \ - PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name +#define __PCPU_ATTRS (sec) \ + __attribute__((section(PER_CPU_BASE_SECTION sec))) \ + PER_CPU_ATTRIBUTES + +#define __PCPU_DUMMY_ATTRS \ + __attribute__((section(".discard"), unused)) + +#define DECLARE_PER_CPU_SECTION (type, name, sec) \ + extern __PCPU_DUMMY_ATTRS char __pcpu_scope_##name; \ + extern __PCPU_ATTRS(sec) __typeof__(type) per_cpu__##name + +#define DEFINE_PER_CPU_SECTION (type, name, sec) \ + __PCPU_DUMMY_ATTRS char __pcpu_scope_##name; \ + __PCPU_DUMMY_ATTRS char __pcpu_unique_##name; \ + __PCPU_ATTRS(sec) __typeof__(type) per_cpu__##name/* * Variant on the per-CPU variable declaration/definition theme used for
Signature · 1 line
1.6.0.2
Tejun Heo
maintainerPATCH 5/7 alpha: kill unnecessary __used attribute in PER_CPU_ATTRIBUTES
In reply to Tejun HeoWith the previous percpu variable definition change, all percpu variables are global and there's no need to specify __used, which only triggers on recent compilers anyway. Kill it.
[ Impact: remove unnecessary percpu attribute ]
--- arch/alpha/include/asm/percpu.h | 5 ----- 1 files changed, 0 insertions(+), 5 deletions(-)
arch/alpha/include/asm/percpu.h +0 -5
@@ -30,7 +30,6 @@ extern unsigned long __per_cpu_offset[NR_CPUS];#ifndef MODULE #define SHIFT_PERCPU_PTR (var, offset) RELOC_HIDE(&per_cpu_var(var), (offset)) -#define PER_CPU_ATTRIBUTES #else /* * To calculate addresses of locally defined variables, GCC uses 32-bit
@@ -49,8 +48,6 @@ extern unsigned long __per_cpu_offset[NR_CPUS]; : "=&r"(__ptr), "=&r"(tmp_gp)); \ (typeof(&per_cpu_var(var)))(__ptr + (offset)); }) -#define PER_CPU_ATTRIBUTES __used -#endif /* MODULE */ /*
@@ -71,8 +68,6 @@ extern unsigned long __per_cpu_offset[NR_CPUS];#define __get_cpu_var (var) per_cpu_var(var)#define __raw_get_cpu_var (var) per_cpu_var(var) -#define PER_CPU_ATTRIBUTES -#endif /* SMP */ #ifdef CONFIG_SMP
Signature · 1 line
1.6.0.2
Tejun Heo
maintainerPATCH 6/7 alpha: switch to dynamic percpu allocator
In reply to Tejun HeoAlpha implements custom SHIFT_PERCPU_PTR for modules because percpu area can be located far away from the 4G area where the module text is located. The custom SHIFT_PERCPU_PTR forces GOT usage using ldq instruction with literal relocation; however, the relocation can't be used with dynamically allocatd percpu variables. Fortunately, similar result can be achieved using weak attribute on percpu variable declarations and definitions, which is allowed with previous changes.
This patch makes alpha use weak attribute instead and switch to dynamic percpu allocator.
asm/tlbflush.h was getting linux/sched.h via asm/percpu.h which no longer needs it. Include linux/sched.h directly in asm/tlbflush.h.
Compile tested. Generation of litereal relocation verified.
This patch is based on Ivan Kokshaysky's alpha percpu patch.
[ Impact: use dynamic percpu allocator ]
--- arch/alpha/Kconfig | 3 - arch/alpha/include/asm/percpu.h | 96 ++++--------------------------------- arch/alpha/include/asm/tlbflush.h | 1 + 3 files changed, 10 insertions(+), 90 deletions(-)
arch/alpha/Kconfig +0 -3
@@ -70,9 +70,6 @@ config AUTO_IRQ_AFFINITY depends on SMP default y -config HAVE_LEGACY_PER_CPU_AREA - def_bool y - source "init/Kconfig" source "kernel/Kconfig.freezer"
arch/alpha/include/asm/percpu.h +9 -87
106 lines not on this page · raw message
arch/alpha/include/asm/tlbflush.h +1 -0
@@ -2,6 +2,7 @@#define _ALPHA_TLBFLUSH_H#include <linux/mm.h> +#include <linux/sched.h> #include <asm/compiler.h> #include <asm/pgalloc.h>
Signature · 1 line
1.6.0.2
Tejun Heo
maintainerPATCH 7/7 s390: switch to dynamic percpu allocator
In reply to Tejun Heo64bit s390 shares the same problem with percpu symbol addressing from modules. It needs assembly magic to force GOTENT reference when building module as the percpu address will be outside the usual 4G range from the module text. Simiarly to alpha, this can be solved by using the weak attribute.
This patch makes s390 use weak attribute instead and switch to dynamic percpu allocator. Please note that weak attribute is not added if !SMP as percpu variables behave exactly the same as normal variables on UP.
Compile tested. Generation of GOTENT reference verified.
This patch is based on Ivan Kokshaysky's alpha percpu patch.
[ Impact: use dynamic percpu allocator ]
--- arch/s390/Kconfig | 3 --- arch/s390/include/asm/percpu.h | 32 ++++++++------------------------ 2 files changed, 8 insertions(+), 27 deletions(-)
arch/s390/Kconfig +0 -3
@@ -75,9 +75,6 @@ config VIRT_CPU_ACCOUNTING config ARCH_SUPPORTS_DEBUG_PAGEALLOC def_bool y -config HAVE_LEGACY_PER_CPU_AREA - def_bool y - mainmenu "Linux Kernel Configuration" config S390
arch/s390/include/asm/percpu.h +8 -24
@@ -1,37 +1,21 @@#ifndef __ARCH_S390_PERCPU__#define __ARCH_S390_PERCPU__ -#include <linux/compiler.h> -#include <asm/lowcore.h> -/* * s390 uses its own implementation for per cpu data, the offset of * the cpu local data area is cached in the cpu's lowcore memory. -* For 64 bit module code s390 forces the use of a GOT slot for the -* address of the per cpu variable. This is needed because the module -* may be more than 4G above the per cpu area. */ -#if defined(__s390x__) && defined(MODULE ) - -#define SHIFT_PERCPU_PTR (ptr,offset) (({ \ - extern int simple_identifier_##var(void); \ - unsigned long *__ptr; \ - asm ( "larl %0, %1@GOTENT" \ - : "=a" (__ptr) : "X" (ptr) ); \ - (typeof(ptr))((*__ptr) + (offset)); })) - -#else - -#define SHIFT_PERCPU_PTR (ptr, offset) (({ \ - extern int simple_identifier_##var(void); \ - unsigned long __ptr; \ - asm ( "" : "=a" (__ptr) : "0" (ptr) ); \ - (typeof(ptr)) (__ptr + (offset)); })) +#define __my_cpu_offset S390_lowcore.percpu_offset +/* +* For 64 bit module code, the module may be more than 4G above the +* per cpu area, use "weak" attribute to force the compiler to +* generate an external reference. +*/ +#if defined(CONFIG_SMP ) && defined(__s390x__) && defined(MODULE ) +#define PER_CPU_ATTRIBUTES __attribute__((weak))#endif -#define __my_cpu_offset S390_lowcore.percpu_offset -#include <asm-generic/percpu.h> #endif /* __ARCH_S390_PERCPU__ */
Signature · 1 line
1.6.0.2
Quoted 56 lines · On Wed, 20 May 2009 16:37:32 +0900 Tejun Heo <[email protected]> wrote:
Hello,
Please do NOT pull before maintainers of alpha and s390 ack. Upon ack, please pull from the following git tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc.gittj-percpuThis patchset is combination of the last patch of percpu-use-dynamic-percpu-allocator-as-default patchset[1] and patches inspired by alpha conversion patch[2] posted by Ivan Kokshaysky.
0001-percpu-use-dynamic-percpu-allocator-as-the-default.patch 0002-linker-script-throw-away-.discard-section.patch 0003-percpu-clean-up-percpu-variable-definitions.patch 0004-percpu-enforce-global-uniqueness-and-disallow-in-fu.patch 0005-alpha-kill-unnecessary-__used-attribute-in-PER_CPU_.patch 0006-alpha-switch-to-dynamic-percpu-allocator.patch 0007-s390-switch-to-dynamic-percpu-allocator.patch
0001 is mostly unchanged except for added Acked-by's.
0002-0003 prepare for percpu variable definition update - .discard is thrown away on all archs when linking kernel and modules, static percpu variables in functions are moved outside of the function and all static percpu variables are given unique names.
0004 updates percpu variable definition such that all percpu variables in the same link unit have unique names regardless of its scope (static or global) and disallow in-function static definitions. The new definition uses dummy variables in .discard to enforce the declaration/definition semantics and always defines percpu variables as globals. This allows archs to add 'weak' attribute if necessary.
0005-0006 converts alpha to use 'weak' attribute instead of inline assembly to generate GOT based reference when referencing percpu variables from modules, which in turn allows conversion to dynamic percpu allocator.
0007 does the same for s390.
The new DECLARE/DEFINE macros implement the following behaviors.
a. DECLARE + static DEFINE -> compile error
b. multiple non-static DEFINEs with the same name -> compile error
c. multiple static/global DEFINEs with the same name -> compile error
d. in function static DEFINEs -> compile error
#a and #b are expected behaviors on DECLARE/DEFINE macros in general. #c and #d are necessary to allow 'weak' attribute on certain archs. Please note that the rules are applied consistently whether 'weak' attribute is specified or not. This eases catching mistakes and makes things less confusing.
Quoted 7 lines
Test config boots and works fine on
x86_64. alpha and s390 are compile tested and verified to generate necessary code for external references but I don't have any way to actually verify them, so please test them. :-)all-yes-config on
x86_64is building now. Ergggh... I want more memory. Okay, completed successfully.
Just tested the patch-series on s390. Works fine:
Kudos to Tejun and Ivan :-)
Signature · 4 lines
blue skies, Martin. "Reality continues to ruin my life." - Calvin.
Quoted 4 lines · Tejun Heo
Test config boots and works fine on
x86_64. alpha and s390 are compile tested and verified to generate necessary code for external references but I don't have any way to actually verify them, so please test them. :-)
Works fine on alpha, thanks!
Ivan.
Tejun Heo
maintainerQuoted 6 lines · Tejun Heo
Hello,
Please do NOT pull before maintainers of alpha and s390 ack. Upon ack, please pull from the following git tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc.gittj-percpu
Ingo, I added Martin's and Ivan's acks. Please go ahead and pull into core/percpu.
Thanks.
Signature · 1 line
tejun