x86,percpu: fix pageattr handling with remap allocator
Subsystems (5)
Tejun Heo
maintainerPATCH 1/1 x86,percpu: fix pageattr handling with remap allocator
Hello,
Upon ack, please pull from the following git tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc.git x86-percpu-pageattr
This patchset fixes subtile bug in pageattr handling when remap percpu first chunk allocator is in use and implements percpu_alloc kernel parameter so that allocator can be selected from boot prompt.
This problem was spotted by Jan Beulich.
The remap allocator allocates a PMD page per cpu, returns whatever is unnecessary to the page allocator and remaps the PMD page into vmalloc area to construct the first percpu chunk. This is to take advantage of large page mapping. However this creates active aliases for the recycled pages. When some user allocates the recycled pages and tries to change pageattr on it, remapped PMD alias might end up having different attributes with the regular page mapped addresses which can lead to subtle data corruption according to Andi.
Similar problem exists for the high mapped area of x86_64 kernel which is handled by detecting the alias and splitting up the high map PMD and applying the same attributes there. This patch implements the same workaround for remapped first chunk.
Also, it's still unclear whether the remap allocator is worth the trouble. Andi thinks it would be better to simply use 4k mapping on NUMA machines as PMD TLB is much more precious resource than PTE TLB. To ease testing, this patch implements percpu_alloc kernel parameter which allows explicitly selecting which allocator to use.
This patchset contains the following four patches.
0001-x86-prepare-setup_pcpu_remap-for-pageattr-fix.patch 0002-x86-simplify-cpa_process_alias.patch 0003-x86-fix-pageattr-handling-for-remap-percpu-allocato.patch 0004-x86-implement-percpu_alloc-kernel-parameter.patch
0001-0002 preps for pageattr fix. 0003 fixes the pageattr bug. 0004 implements percpu_alloc kernel parameter.
This patchset is on top of the current linux-2.6#master (210af919c949a7d6bd330916ef376cec2907d81e) and contains the following changes.
Documentation/kernel-parameters.txt | 6 + arch/x86/include/asm/percpu.h | 9 + arch/x86/kernel/setup_percpu.c | 186 ++++++++++++++++++++++++++---------- arch/x86/mm/pageattr.c | 59 +++++++---- mm/percpu.c | 13 +- 5 files changed, 197 insertions(+), 76 deletions(-)
Thanks.
--
tejun
Tejun Heo
maintainerPATCH 1/4 x86: prepare setup_pcpu_remap() for pageattr fix
Make the following changes in preparation of coming pageattr updates.
* Define and use array of struct pcpur_ent instead of array of pointers. The only difference is ->cpu field which is set but unused yet.
* Rename variables according to the above change.
* Rename local variable vm to pcpur_vm and move it out of the function.
[ Impact: no functional difference ]
--- arch/x86/kernel/setup_percpu.c | 58 ++++++++++++++++++++++----------------- 1 files changed, 33 insertions(+), 25 deletions(-)
arch/x86/kernel/setup_percpu.c +33 -25
@@ -137,8 +137,14 @@ static void * __init pcpu_alloc_bootmem(unsigned int cpu, unsigned long size,* better than only using 4k mappings while still being NUMA friendly. */ #ifdef CONFIG_NEED_MULTIPLE_NODES +struct pcpur_ent { +unsigned int cpu; +void *ptr; +}; +static size_t pcpur_size__initdata ; -static void **pcpur_ptrs__initdata ; +static struct pcpur_ent *pcpur_map__initdata ; +static struct vm_struct pcpur_vm;static struct page *__init pcpur_get_page (unsigned int cpu,int pageno) {
@@ -147,13 +153,12 @@ static struct page * __init pcpur_get_page(unsigned int cpu, int pageno)if (off >= pcpur_size)return NULL ; -return virt_to_page (pcpur_ptrs[cpu] + off); +return virt_to_page (pcpur_map[cpu].ptr + off); }static ssize_t __init setup_pcpu_remap (size_t static_size) { -static struct vm_struct vm; -size_t ptrs_size, dyn_size; +size_t map_size, dyn_size;unsigned int cpu;ssize_t ret;
@@ -178,12 +183,14 @@ static ssize_t __init setup_pcpu_remap(size_t static_size) dyn_size = pcpur_size - static_size -PERCPU_FIRST_CHUNK_RESERVE ;/* allocate pointer array and alloc large pages */ - ptrs_size =PFN_ALIGN (num_possible_cpus () *sizeof (pcpur_ptrs[0 ])); - pcpur_ptrs =alloc_bootmem (ptrs_size); + map_size =PFN_ALIGN (num_possible_cpus () *sizeof (pcpur_map[0 ])); + pcpur_map =alloc_bootmem (map_size);for_each_possible_cpu (cpu) { - pcpur_ptrs[cpu] =pcpu_alloc_bootmem (cpu,PMD_SIZE ,PMD_SIZE ); -if (!pcpur_ptrs[cpu]) + pcpur_map[cpu].cpu = cpu; + pcpur_map[cpu].ptr =pcpu_alloc_bootmem (cpu,PMD_SIZE , +PMD_SIZE ); +if (!pcpur_map[cpu].ptr) goto enomem;/*
@@ -194,42 +201,43 @@ static ssize_t __init setup_pcpu_remap(size_t static_size)* not well-specified to have a PAT-incompatible area * (unmapped RAM, device memory, etc.) in that hole. */ -free_bootmem (__pa (pcpur_ptrs[cpu] + pcpur_size), +free_bootmem (__pa (pcpur_map[cpu].ptr + pcpur_size),PMD_SIZE - pcpur_size); - memcpy(pcpur_ptrs[cpu], __per_cpu_load, static_size); + memcpy(pcpur_map[cpu].ptr, __per_cpu_load, static_size); }/* allocate address and map */ - vm.flags =VM_ALLOC ; - vm.size =num_possible_cpus () *PMD_SIZE ; -vm_area_register_early (&vm,PMD_SIZE ); + pcpur_vm.flags =VM_ALLOC ; + pcpur_vm.size =num_possible_cpus () *PMD_SIZE ; +vm_area_register_early (&pcpur_vm,PMD_SIZE );for_each_possible_cpu (cpu) { -pmd_t *pmd; +pmd_t *pmd, pmd_v; - pmd =populate_extra_pmd ((unsigned long )vm.addr - + cpu *PMD_SIZE ); -set_pmd (pmd,pfn_pmd (page_to_pfn (virt_to_page (pcpur_ptrs[cpu])), -PAGE_KERNEL_LARGE )); + pmd =populate_extra_pmd ((unsigned long )pcpur_vm.addr + + cpu *PMD_SIZE ); + pmd_v =pfn_pmd (page_to_pfn (virt_to_page (pcpur_map[cpu].ptr)), +PAGE_KERNEL_LARGE ); +set_pmd (pmd, pmd_v); }/* we're ready, commit */ pr_info ("PERCPU: Remapped at %p with large pages, static data " -"%zu bytes\n" , vm.addr, static_size); +"%zu bytes\n" , pcpur_vm.addr, static_size); ret =pcpu_setup_first_chunk (pcpur_get_page, static_size,PERCPU_FIRST_CHUNK_RESERVE , dyn_size, -PMD_SIZE , vm.addr,NULL ); - goto out_free_ar; +PMD_SIZE , pcpur_vm.addr,NULL ); + goto out_free_map; enomem:for_each_possible_cpu (cpu) -if (pcpur_ptrs[cpu]) -free_bootmem (__pa (pcpur_ptrs[cpu]),PMD_SIZE ); +if (pcpur_map[cpu].ptr) +free_bootmem (__pa (pcpur_map[cpu].ptr),PMD_SIZE ); ret = -ENOMEM ; -out_free_ar: -free_bootmem (__pa (pcpur_ptrs), ptrs_size); +out_free_map: +free_bootmem (__pa (pcpur_map), map_size);return ret; }#else
Signature · 1 line
1.6.0.2
Tejun Heo
maintainerPATCH 2/4 x86: simplify cpa_process_alias()
In reply to Tejun HeoThe two existing alias conditions in cpa_process_alias() are mutually exclusive and future ones are likely to be exclusive too. Simplify control flow to ease adding other alias cases.
The within(vaddr, (unsigned long)_text, _brk_end) test is removed as it's guaranteed to be false if vaddr is in the page mapped area.
[ Impact: cleanup, no functional difference ]
--- arch/x86/mm/pageattr.c | 42 ++++++++++++++++++------------------------ 1 files changed, 18 insertions(+), 24 deletions(-)
arch/x86/mm/pageattr.c +18 -24
@@ -686,7 +686,6 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int checkalias);static int cpa_process_alias (struct cpa_data *cpa) {struct cpa_data alias_cpa; -int ret =0 ;unsigned long temp_cpa_vaddr, vaddr;if (cpa->pfn >= max_pfn_mapped)
@@ -715,38 +714,33 @@ static int cpa_process_alias(struct cpa_data *cpa) alias_cpa.vaddr = &temp_cpa_vaddr; alias_cpa.flags &= ~(CPA_PAGES_ARRAY |CPA_ARRAY ); - - ret =__change_page_attr_set_clr (&alias_cpa,0 ); +return __change_page_attr_set_clr (&alias_cpa,0 ); } -#ifdef CONFIG_X86_64 -if (ret) -return ret; -/* -* No need to redo, when the primary call touched the high -* mapping already: -*/ -if (within (vaddr, (unsigned long ) _text, _brk_end)) -return 0 ; +/* vaddr is in page mapped area */ +#ifdef CONFIG_X86_64 /* * If the physical address is inside the kernel map, we need * to touch the high mapped kernel as well: */ -if (!within (cpa->pfn,highmap_start_pfn (),highmap_end_pfn ())) -return 0 ; - - alias_cpa = *cpa; - temp_cpa_vaddr = (cpa->pfn <<PAGE_SHIFT ) + __START_KERNEL_map - phys_base; - alias_cpa.vaddr = &temp_cpa_vaddr; - alias_cpa.flags &= ~(CPA_PAGES_ARRAY |CPA_ARRAY ); +if (within (cpa->pfn,highmap_start_pfn (),highmap_end_pfn ())) { + alias_cpa = *cpa; + temp_cpa_vaddr = (cpa->pfn <<PAGE_SHIFT ) + + __START_KERNEL_map - phys_base; + alias_cpa.vaddr = &temp_cpa_vaddr; + alias_cpa.flags &= ~(CPA_PAGES_ARRAY |CPA_ARRAY ); -/* -* The high mapping range is imprecise, so ignore the return value. -*/ -__change_page_attr_set_clr (&alias_cpa,0 ); +/* +* The high mapping range is imprecise, so ignore the +* return value. +*/ +__change_page_attr_set_clr (&alias_cpa,0 ); +return 0 ; + }#endif -return ret; + +return 0 ; }static int __change_page_attr_set_clr (struct cpa_data *cpa,int checkalias)
Signature · 1 line
1.6.0.2
Re: [PATCH 2/4] x86: simplify cpa_process_alias()
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Tejun Heo <[email protected]> 14.05.09 14:49 >>>
The two existing alias conditions in
cpa_process_alias()are mutually exclusive and future ones are likely to be exclusive too. Simplify control flow to ease adding other alias cases.The
within(vaddr, (unsigned long)_text, _brk_end)test is removed as it's guaranteed to be false if vaddr is in the page mapped area.
I don't think that's correct - just consider the case where the originally passed in virtual address is from the vmalloc area: In that case, both the 1:1 mapping *and* the kernel mapping need to be checked.
Jan
Tejun Heo
maintainerRe: [PATCH 2/4] x86: simplify cpa_process_alias()
Hello,
Quoted 5 lines · Jan Beulich
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Tejun Heo <[email protected]> 14.05.09 14:49 >>>
The two existing alias conditions in
cpa_process_alias()are mutually exclusive and future ones are likely to be exclusive too. Simplify control flow to ease adding other alias cases.
Quoted 7 lines
Quoted 2 lines
The
within(vaddr, (unsigned long)_text, _brk_end)test is removed as it's guaranteed to be false if vaddr is in the page mapped area.I don't think that's correct - just consider the case where the originally passed in virtual address is from the vmalloc area: In that case, both the 1:1 mapping *and* the kernel mapping need to be checked.
Yeap, you're right. I was reading the second condition in reversal. Will update and repost. Thanks for spotting it.
Signature · 1 line
tejun
PATCH 2/4 x86: reorganize cpa_process_alias()
Reorganize cpa_process_alias() so that new alias condition can be added easily.
Jan Beulich spotted problem in the original cleanup thread which incorrectly assumed the two existing conditions were mutially exclusive.
[ Impact: code reorganization ]
--- arch/x86/mm/pageattr.c | 44 ++++++++++++++++++++------------------------ 1 files changed, 20 insertions(+), 24 deletions(-)
arch/x86/mm/pageattr.c +20 -24
@@ -686,8 +686,8 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int checkalias);static int cpa_process_alias (struct cpa_data *cpa) {struct cpa_data alias_cpa; -int ret =0 ;unsigned long temp_cpa_vaddr, vaddr; +int ret;if (cpa->pfn >= max_pfn_mapped)return 0 ;
@@ -715,38 +715,34 @@ static int cpa_process_alias(struct cpa_data *cpa) alias_cpa.vaddr = &temp_cpa_vaddr; alias_cpa.flags &= ~(CPA_PAGES_ARRAY |CPA_ARRAY ); - ret =__change_page_attr_set_clr (&alias_cpa,0 ); +if (ret) +return ret; }#ifdef CONFIG_X86_64 -if (ret) -return ret;/* -* No need to redo, when the primary call touched the high -* mapping already: -*/ -if (within (vaddr, (unsigned long ) _text, _brk_end)) -return 0 ; - -/* -* If the physical address is inside the kernel map, we need +* If the primary call didn't touch the high mapping already +* and the physical address is inside the kernel map, we need * to touch the high mapped kernel as well: */ -if (!within (cpa->pfn,highmap_start_pfn (),highmap_end_pfn ())) -return 0 ; - - alias_cpa = *cpa; - temp_cpa_vaddr = (cpa->pfn <<PAGE_SHIFT ) + __START_KERNEL_map - phys_base; - alias_cpa.vaddr = &temp_cpa_vaddr; - alias_cpa.flags &= ~(CPA_PAGES_ARRAY |CPA_ARRAY ); +if (!within (vaddr, (unsigned long )_text, _brk_end) && +within (cpa->pfn,highmap_start_pfn (),highmap_end_pfn ())) { + alias_cpa = *cpa; + temp_cpa_vaddr = (cpa->pfn <<PAGE_SHIFT ) + + __START_KERNEL_map - phys_base; + alias_cpa.vaddr = &temp_cpa_vaddr; + alias_cpa.flags &= ~(CPA_PAGES_ARRAY |CPA_ARRAY ); -/* -* The high mapping range is imprecise, so ignore the return value. -*/ -__change_page_attr_set_clr (&alias_cpa,0 ); +/* +* The high mapping range is imprecise, so ignore the +* return value. +*/ +__change_page_attr_set_clr (&alias_cpa,0 ); + }#endif -return ret; + +return 0 ; }static int __change_page_attr_set_clr (struct cpa_data *cpa,int checkalias)
Signature · 1 line
1.6.0.2
Tejun Heo
maintainerPATCH 3/4 x86: fix pageattr handling for remap percpu allocator
In reply to Tejun HeoRemap allocator aliases a PMD page for each cpu and returns whatever is unused to the page allocator. When the pageattr of the recycled pages are changed, this makes the two aliases point to the overlapping regions with different attributes which isn't allowed and known to cause subtle data corruption in certain cases.
This can be handled in simliar manner to the x86_64 highmap alias. pageattr code should detect if the target pages have PMD alias and split the PMD alias and synchronize the attributes.
pcpur allocator is updated to keep the allocated PMD pages map sorted in ascending address order and provide pcpu_pmd_remapped() function which binary searches the array to determine whether the given address is aliased and if so to which address. pageattr is updated to use pcpu_pmd_remapped() to detect the PMD alias and split it up as necessary from cpa_process_alias().
This problem has been spotted by Jan Beulich.
[ Impact: fix subtle pageattr bug ]
--- arch/x86/include/asm/percpu.h | 9 +++++ arch/x86/kernel/setup_percpu.c | 68 ++++++++++++++++++++++++++++++++++++--- arch/x86/mm/pageattr.c | 21 ++++++++++++ 3 files changed, 92 insertions(+), 6 deletions(-)
arch/x86/include/asm/percpu.h +9 -0
@@ -155,6 +155,15 @@ do { \/* We can use this directly for local CPU (faster). */ DECLARE_PER_CPU(unsigned long, this_cpu_off); +#ifdef CONFIG_NEED_MULTIPLE_NODES +void *pcpu_pmd_remapped (void *kaddr); +#else +static inline void *pcpu_pmd_remapped (void *kaddr) +{ +return NULL ; +} +#endif +#endif /* !__ASSEMBLY__ */ #ifdef CONFIG_SMP
arch/x86/kernel/setup_percpu.c +62 -6
92 lines not on this page · raw message
arch/x86/mm/pageattr.c +21 -0
@@ -11,6 +11,7 @@#include <linux/interrupt.h> #include <linux/seq_file.h> #include <linux/debugfs.h> +#include <linux/pfn.h> #include <asm/e820.h> #include <asm/processor.h>
@@ -687,6 +688,7 @@ static int cpa_process_alias(struct cpa_data *cpa) {struct cpa_data alias_cpa;unsigned long temp_cpa_vaddr, vaddr; +void *remapped;if (cpa->pfn >= max_pfn_mapped)return 0 ;
@@ -740,6 +742,25 @@ static int cpa_process_alias(struct cpa_data *cpa) }#endif +/* +* If the PMD page was partially used for per-cpu remapping, +* the remapped area needs to be split and modified. Note +* that the partial recycling only happens at the tail of a +* partially used PMD page, so touching single PMD page is +* always enough. +*/ + remapped =pcpu_pmd_remapped ((void *)vaddr); +if (remapped) { +int max_pages =PFN_DOWN (PMD_SIZE - (vaddr & ~PMD_MASK )); + + alias_cpa = *cpa; + temp_cpa_vaddr = (unsigned long )remapped; + alias_cpa.vaddr = &temp_cpa_vaddr; + alias_cpa.numpages =min (alias_cpa.numpages, max_pages); + alias_cpa.flags &= ~(CPA_PAGES_ARRAY |CPA_ARRAY ); +return __change_page_attr_set_clr (&alias_cpa,0 ); + } +return 0 ; }
Signature · 1 line
1.6.0.2
Tejun Heo
maintainerRe: [PATCH UPDATED 3/4] x86: fix pageattr handling for remap percpu allocator
Remap allocator aliases a PMD page for each cpu and returns whatever is unused to the page allocator. When the pageattr of the recycled pages are changed, this makes the two aliases point to the overlapping regions with different attributes which isn't allowed and known to cause subtle data corruption in certain cases.
This can be handled in simliar manner to the x86_64 highmap alias. pageattr code should detect if the target pages have PMD alias and split the PMD alias and synchronize the attributes.
pcpur allocator is updated to keep the allocated PMD pages map sorted in ascending address order and provide pcpu_pmd_remapped() function which binary searches the array to determine whether the given address is aliased and if so to which address. pageattr is updated to use pcpu_pmd_remapped() to detect the PMD alias and split it up as necessary from cpa_process_alias().
This problem has been spotted by Jan Beulich.
[ Impact: fix subtle pageattr bug ]
--- arch/x86/include/asm/percpu.h | 9 +++++ arch/x86/kernel/setup_percpu.c | 68 ++++++++++++++++++++++++++++++++++++--- arch/x86/mm/pageattr.c | 23 +++++++++++++ 3 files changed, 94 insertions(+), 6 deletions(-)
arch/x86/include/asm/percpu.h +9 -0
@@ -155,6 +155,15 @@ do { \/* We can use this directly for local CPU (faster). */ DECLARE_PER_CPU(unsigned long, this_cpu_off); +#ifdef CONFIG_NEED_MULTIPLE_NODES +void *pcpu_pmd_remapped (void *kaddr); +#else +static inline void *pcpu_pmd_remapped (void *kaddr) +{ +return NULL ; +} +#endif +#endif /* !__ASSEMBLY__ */ #ifdef CONFIG_SMP
arch/x86/kernel/setup_percpu.c +62 -6
92 lines not on this page · raw message
arch/x86/mm/pageattr.c +23 -0
@@ -11,6 +11,7 @@#include <linux/interrupt.h> #include <linux/seq_file.h> #include <linux/debugfs.h> +#include <linux/pfn.h> #include <asm/e820.h> #include <asm/processor.h>
@@ -687,6 +688,7 @@ static int cpa_process_alias(struct cpa_data *cpa) {struct cpa_data alias_cpa;unsigned long temp_cpa_vaddr, vaddr; +void *remapped;int ret;if (cpa->pfn >= max_pfn_mapped)
@@ -742,6 +744,27 @@ static int cpa_process_alias(struct cpa_data *cpa) }#endif +/* +* If the PMD page was partially used for per-cpu remapping, +* the remapped area needs to be split and modified. Note +* that the partial recycling only happens at the tail of a +* partially used PMD page, so touching single PMD page is +* always enough. +*/ + remapped =pcpu_pmd_remapped ((void *)vaddr); +if (remapped) { +int max_pages =PFN_DOWN (PMD_SIZE - (vaddr & ~PMD_MASK )); + + alias_cpa = *cpa; + temp_cpa_vaddr = (unsigned long )remapped; + alias_cpa.vaddr = &temp_cpa_vaddr; + alias_cpa.numpages =min (alias_cpa.numpages, max_pages); + alias_cpa.flags &= ~(CPA_PAGES_ARRAY |CPA_ARRAY ); + ret =__change_page_attr_set_clr (&alias_cpa,0 ); +if (ret) +return ret; + } +return 0 ; }
Signature · 1 line
1.6.0.2
Tejun Heo
maintainerPATCH 4/4 x86: implement percpu_alloc kernel parameter
In reply to Tejun HeoAccording to Andi, it isn't clear whether remap allocator is worth the trouble as there are many processors where PMD TLB is far scarcer than PTE TLB. The advantage or disadvantage probably depends on the actual size of percpu area and specific processor. As performance degradation due to TLB pressure tends to be highly workload specific and subtle, it is difficult to decide which way to go without more data.
This patch implements percpu_alloc kernel parameter to allow selecting which first chunk allocator to use to ease debugging and testing.
While at it, make sure all the failure paths report why something failed to help determining why certain allocator isn't working. Also, kill the "Great future plan" comment which had already been realized quite some time ago.
[ Impact: allow explicit percpu first chunk allocator selection ]
--- Documentation/kernel-parameters.txt | 6 +++ arch/x86/kernel/setup_percpu.c | 68 +++++++++++++++++++++++------------ mm/percpu.c | 13 +++++-- 3 files changed, 60 insertions(+), 27 deletions(-)
Documentation/kernel-parameters.txt +6 -0
@@ -1837,6 +1837,12 @@ and is between 256 and 4096 characters. It is defined in the file Format: { 0 | 1 } See arch/parisc/kernel/pdc_chassis.c + percpu_alloc= [X86] Select which percpu first chunk allocator to use. + Allowed values are one of "remap", "embed" and "4k". + See comments in arch/x86/kernel/setup_percpu.c for + details on each allocator. This parameter is primarily + for debugging and performance comparison. + pf. [PARIDE] See Documentation/blockdev/paride.txt.
arch/x86/kernel/setup_percpu.c +45 -23
113 lines not on this page · raw message
mm/percpu.c +9 -4
@@ -1287,6 +1287,7 @@ static struct page * __init pcpue_get_page(unsigned int cpu, int pageno)ssize_t __init pcpu_embed_first_chunk (size_t static_size,size_t reserved_size,ssize_t dyn_size,ssize_t unit_size) { +size_t chunk_size;unsigned int cpu;/* determine parameters and allocate */
@@ -1301,11 +1302,15 @@ ssize_t __init pcpu_embed_first_chunk(size_t static_size, size_t reserved_size, }else pcpue_unit_size =max_t (size_t, pcpue_size,PCPU_MIN_UNIT_SIZE ); - pcpue_ptr =__alloc_bootmem_nopanic ( -num_possible_cpus () * pcpue_unit_size, -PAGE_SIZE ,__pa (MAX_DMA_ADDRESS )); -if (!pcpue_ptr) + chunk_size = pcpue_unit_size *num_possible_cpus (); + + pcpue_ptr =__alloc_bootmem_nopanic (chunk_size,PAGE_SIZE , +__pa (MAX_DMA_ADDRESS )); +if (!pcpue_ptr) { +pr_warning ("PERCPU: failed to allocate %zu bytes for " +"embedding\n" , chunk_size);return -ENOMEM ; + }/* return the leftover and copy */ for_each_possible_cpu (cpu) {
Signature · 1 line
1.6.0.2
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Tejun Heo <[email protected]> 14.05.09 14:49 >>>
The remap allocator allocates a PMD page per cpu, returns whatever is unnecessary to the page allocator and remaps the PMD page into vmalloc area to construct the first percpu chunk. This is to take advantage of large page mapping. However this creates active aliases for the recycled pages. When some user allocates the recycled pages and tries to change pageattr on it, remapped PMD alias might end up having different attributes with the regular page mapped addresses which can lead to subtle data corruption according to Andi.
In order to reduce the amount of work to do during lookup as well as the chance of having a collision at all, wouldn't it be reasonable to use as much of an allocated 2/4M page as possible rather than returning whatever is left after a single CPU got its per-CPU memory chunk from it? I.e. you'd return only those (few) pages that either don't fit another CPU's chunk anymore or that are left after running through all CPUs.
Or is there some hidden requirement that each CPU's per-CPU area must start on a PMD boundary?
This would additionally address a potential problem on 32-bits - currently, for a 32-CPU system you consume half of the vmalloc space with PAE (on non-PAE you'd even exhaust it, but I think it's unreasonable to expect a system having 32 CPUs to not need PAE).
Jan
Tejun Heo
maintainerHello, Jan.
Quoted 10 lines · Jan Beulich
In order to reduce the amount of work to do during lookup as well as the chance of having a collision at all, wouldn't it be reasonable to use as much of an allocated 2/4M page as possible rather than returning whatever is left after a single CPU got its per-CPU memory chunk from it? I.e. you'd return only those (few) pages that either don't fit another CPU's chunk anymore or that are left after running through all CPUs.
Or is there some hidden requirement that each CPU's per-CPU area must start on a PMD boundary?
The whole point of doing the remapping is giving each CPU its own PMD mapping for perpcu area, so, yeah, that's the requirement. I don't think the requirement is hidden tho.
How hot is the cpa path? On my test systems, there were only few calls during init and then nothing. Does it become very hot if, for example, GEM is used? But I really don't think the log2 binary search overhead would be anything noticeable compared to TLB shootdown and all other stuff going on there.
Quoted 4 lines
This would additionally address a potential problem on 32-bits - currently, for a 32-CPU system you consume half of the vmalloc space with PAE (on non-PAE you'd even exhaust it, but I think it's unreasonable to expect a system having 32 CPUs to not need PAE).
I recall having about the same conversation before. Looking up...
-- QUOTE --
Actually, I've been looking at the numbers and I'm not sure if the concern is valid. On x86_32, the practical number of maximum processors would be around 16 so it will end up 32M, which isn't nice and it would probably a good idea to introduce a parameter to select which allocator to use but still it's far from consuming all the VM area. On x86_64, the vmalloc area is obscenely large at 245, ie 32 terabytes. Even with 4096 processors, single chunk is measly 0.02%.
If it's a problem for other archs or extreme x86_32 configurations, we can add some safety measures but in general I don't think it is a problem.
-- END OF QUOTE --
So, yeah, if there are 32bit 32-way NUMA machines out there, it would be wise to skip remap allocator on such machines. Maybe we can implement a heuristic - something like "if vm area consumption goes over 25%, don't use remap".
Thanks.
Signature · 1 line
tejun
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
Tejun Heo <[email protected]> 14.05.09 17:55 >>>
In order to reduce the amount of work to do during lookup as well as the chance of having a collision at all, wouldn't it be reasonable to use as much of an allocated 2/4M page as possible rather than returning whatever is left after a single CPU got its per-CPU memory chunk from it? I.e. you'd return only those (few) pages that either don't fit another CPU's chunk anymore or that are left after running through all CPUs.
Quoted 6 lines
Quoted 2 lines
Or is there some hidden requirement that each CPU's per-CPU area must start on a PMD boundary?
The whole point of doing the remapping is giving each CPU its own PMD mapping for perpcu area, so, yeah, that's the requirement. I don't think the requirement is hidden tho.
No, from looking at the code the requirement seems to only be that you get memory allocated from the correct node and mapped by a large page. There's nothing said why the final virtual address would need to be large page aligned. I.e., with a slight modification to take the NUMA requirement into account (I noticed I ignored that aspect after I had already sent that mail), the previous suggestion would still appear usable to me.
Quoted 5 lines
How hot is the cpa path? On my test systems, there were only few calls during init and then nothing. Does it become very hot if, for example, GEM is used? But I really don't think the log2 binary search overhead would be anything noticeable compared to TLB shootdown and all other stuff going on there.
I would view cutting down on that only as a nice side effect, not a primary reason to do the change. The primary reason is this:
Quoted 7 lines
Quoted 4 lines
This would additionally address a potential problem on 32-bits - currently, for a 32-CPU system you consume half of the vmalloc space with PAE (on non-PAE you'd even exhaust it, but I think it's unreasonable to expect a system having 32 CPUs to not need PAE).
I recall having about the same conversation before. Looking up...
Quoted 9 lines
-- QUOTE --
Actually, I've been looking at the numbers and I'm not sure if the concern is valid. Onx86_32, the practical number of maximum processors would be around 16 so it will end up 32M, which isn't nice and it would probably a good idea to introduce a parameter to select which allocator to use but still it's far from consuming all the VM area. Onx86_64, the vmalloc area is obscenely large at 245, ie 32 terabytes. Even with 4096 processors, single chunk is measly 0.02%.
Just to note - there must be a reason we (SuSE/Novell) build our default 32-bit kernel with support for 128 CPUs, which now is simply broken.
Quoted 9 lines
If it's a problem for other archs or extreme
x86_32configurations, we can add some safety measures but in general I don't think it is a problem.
-- END OF QUOTE --So, yeah, if there are 32bit 32-way NUMA machines out there, it would be wise to skip remap allocator on such machines. Maybe we can implement a heuristic - something like "if vm area consumption goes over 25%, don't use remap".
Possibly, as a secondary consideration on top of the suggested reduction of virtual address space consumption.
Jan
Tejun Heo
maintainerHello,
Quoted 10 lines · Jan Beulich
Quoted 3 lines
The whole point of doing the remapping is giving each CPU its own PMD mapping for perpcu area, so, yeah, that's the requirement. I don't think the requirement is hidden tho.
No, from looking at the code the requirement seems to only be that you get memory allocated from the correct node and mapped by a large page. There's nothing said why the final virtual address would need to be large page aligned. I.e., with a slight modification to take the NUMA requirement into account (I noticed I ignored that aspect after I had already sent that mail), the previous suggestion would still appear usable to me.
The requirement is having separate PMD mapping per NUMA node. What has been implemented is the simplest form of that - one mapping per CPU. Sure it can be further improved with more knowledge of the topology. If you're interested, please go ahead.
Quoted 16 lines
Quoted 6 lines
Quoted 4 lines
This would additionally address a potential problem on 32-bits - currently, for a 32-CPU system you consume half of the vmalloc space with PAE (on non-PAE you'd even exhaust it, but I think it's unreasonable to expect a system having 32 CPUs to not need PAE).
I recall having about the same conversation before. Looking up...
Quoted 9 lines
-- QUOTE --
Actually, I've been looking at the numbers and I'm not sure if the concern is valid. Onx86_32, the practical number of maximum processors would be around 16 so it will end up 32M, which isn't nice and it would probably a good idea to introduce a parameter to select which allocator to use but still it's far from consuming all the VM area. Onx86_64, the vmalloc area is obscenely large at 245, ie 32 terabytes. Even with 4096 processors, single chunk is measly 0.02%.
Quoted 2 lines
Just to note - there must be a reason we (SuSE/Novell) build our default 32-bit kernel with support for 128 CPUs, which now is simply broken.
It's not broken, it will just fall back to 4k allocator. Also, please take a look at the refreshed patchset, remap allocator is not used anymore if it's gonna occupy more than 20% (random number from the top of my head) of vmalloc area.
Quoted 7 lines
Quoted 4 lines
So, yeah, if there are 32bit 32-way NUMA machines out there, it would be wise to skip remap allocator on such machines. Maybe we can implement a heuristic - something like "if vm area consumption goes over 25%, don't use remap".
Possibly, as a secondary consideration on top of the suggested reduction of virtual address space consumption.
Yeah, further improvements welcome. No objection whatsoever there.
Thanks.
Signature · 1 line
tejun
Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Tejun Heo <[email protected]> 15.05.09 10:11 >>>
Quoted 4 lines
This would additionally address a potential problem on 32-bits - currently, for a 32-CPU system you consume half of the vmalloc space with PAE (on non-PAE you'd even exhaust it, but I think it's unreasonable to expect a system having 32 CPUs to not need PAE).
I recall having about the same conversation before. Looking up...
-- QUOTE --
Actually, I've been looking at the numbers and I'm not sure if the concern is valid. Onx86_32, the practical number of maximum processors would be around 16 so it will end up 32M, which isn't nice and it would probably a good idea to introduce a parameter to select which allocator to use but still it's far from consuming all the VM area. Onx86_64, the vmalloc area is obscenely large at 245, ie 32 terabytes. Even with 4096 processors, single chunk is measly 0.02%.
Quoted 4 lines
Quoted 2 lines
Just to note - there must be a reason we (SuSE/Novell) build our default 32-bit kernel with support for 128 CPUs, which now is simply broken.
It's not broken, it will just fall back to 4k allocator. Also, please
I'm afraid I have to disagree: There's no check (not even in vm_area_register_early()) whether the vmalloc area is actually large enough to fulfill the request.
Quoted 3 lines
take a look at the refreshed patchset, remap allocator is not used anymore if it's gonna occupy more than 20% (random number from the top of my head) of vmalloc area.
Yeah, I saw this only after following this thread.
Jan
Tejun Heo
maintainerQuoted 16 lines · Jan Beulich
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Tejun Heo <[email protected]> 15.05.09 10:11 >>>
Quoted 4 lines
This would additionally address a potential problem on 32-bits - currently, for a 32-CPU system you consume half of the vmalloc space with PAE (on non-PAE you'd even exhaust it, but I think it's unreasonable to expect a system having 32 CPUs to not need PAE).
I recall having about the same conversation before. Looking up...
-- QUOTE --
Actually, I've been looking at the numbers and I'm not sure if the concern is valid. Onx86_32, the practical number of maximum processors would be around 16 so it will end up 32M, which isn't nice and it would probably a good idea to introduce a parameter to select which allocator to use but still it's far from consuming all the VM area. Onx86_64, the vmalloc area is obscenely large at 245, ie 32 terabytes. Even with 4096 processors, single chunk is measly 0.02%.
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Just to note - there must be a reason we (SuSE/Novell) build our default 32-bit kernel with support for 128 CPUs, which now is simply broken.
It's not broken, it will just fall back to 4k allocator. Also, please
I'm afraid I have to disagree: There's no check (not even in
vm_area_register_early()) whether the vmalloc area is actually large enough to fulfill the request.
Hah... indeed. Well, it's solved now.
Thanks.
Signature · 1 line
tejun
Quoted 5 lines · Tejun Heo
Hello,
Upon ack, please pull from the following git tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc.gitx86-percpu-pageattr
git tree updated to contain two updated patches. The new commit ID is 42fec16c045bc5259002faacdcd23e4530b6482e.
Thanks.
Signature · 1 line
tejun
Quoted 9 lines · Tejun Heo
Quoted 5 lines · Tejun Heo
Hello,
Upon ack, please pull from the following git tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc.gitx86-percpu-pageattrgit tree updated to contain two updated patches. The new commit ID is
42fec16c045bc5259002faacdcd23e4530b6482e.
Will soon post an updated patchset. Please wait a bit.
Thanks.
Signature · 1 line
tejun
You might want to drop linux-kernel-owner from the CC:, I already get enough copies :-)
Quoted 2 lines · David Miller
You might want to drop linux-kernel-owner from the CC:, I already get enough copies :-)
Hehe.. Where did that come from? Sorry about that. :-)
Signature · 1 line
tejun
Quoted 10 lines · Tejun Heo
Hello,
Upon ack, please pull from the following git tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc.gitx86-percpu-pageattrThis patchset fixes subtile bug in pageattr handling when remap percpu first chunk allocator is in use and implements
percpu_allockernel parameter so that allocator can be selected from boot prompt.
Quoted 6 lines
This problem was spotted by Jan Beulich.
The remap allocator allocates a PMD page per cpu, returns whatever is unnecessary to the page allocator and remaps the PMD page into vmalloc area to construct the first percpu chunk. This is to take advantage of large page mapping.
Tejun, Can you please educate me why we need to map this first percpu chunk (which is pre-allocated during boot and is physically contiguous) into vmalloc area? Perhaps even for the other dynamically allocated secondary chunks? (as far as I can see, all the chunk allocations seems to be physically contiguous and later mapped into vmalloc area)..
That should simplify these things quite a bit(atleast for first percpu chunk). I am missing something obvious I guess.
thanks,
suresh
Tejun Heo
maintainerHello, Suresh.
Quoted 3 lines · Suresh Siddha
Tejun, Can you please educate me why we need to map this first percpu chunk (which is pre-allocated during boot and is physically contiguous) into vmalloc area?
To make areas for each cpu congruent such that the address offset of a percpu symbol for CPU N is always the same from the address for CPU 0.
Quoted 6 lines
Perhaps even for the other dynamically allocated secondary chunks? (as far as I can see, all the chunk allocations seems to be physically contiguous and later mapped into vmalloc area)..
That should simplify these things quite a bit(atleast for first percpu chunk). I am missing something obvious I guess.
Hmm... Sorry I don't really follow. Can you please elaborate the question?
thanks.
Signature · 1 line
tejun
Quoted 9 lines · Tejun Heo
Hello, Suresh.
Quoted 3 lines · Suresh Siddha
Tejun, Can you please educate me why we need to map this first percpu chunk (which is pre-allocated during boot and is physically contiguous) into vmalloc area?
To make areas for each cpu congruent such that the address offset of a percpu symbol for CPU N is always the same from the address for CPU 0.
But for the first percpu chunk, isn't it the case that the physical address allocations for a particular cpu is contiguous (as you are using one bootmem allocation for whole PMD_SIZE for any given cpu)? So both the kernel direct mapping aswell as the vmalloc mappings are contiguous for the first chunk, on any given cpu. Right?
Quoted 9 lines
Quoted 6 lines
Perhaps even for the other dynamically allocated secondary chunks? (as far as I can see, all the chunk allocations seems to be physically contiguous and later mapped into vmalloc area)..
That should simplify these things quite a bit(atleast for first percpu chunk). I am missing something obvious I guess.
Hmm... Sorry I don't really follow. Can you please elaborate the question?
For the first percpu chunk, we can use the kernel direct mapping and avoid the vmalloc mapping of PMD_SIZE. And avoid the vmap address aliasing problem (wrt to free pages that we have given back to -mm) that we are trying to avoid with this patchset (as the existing cpa code already takes care of the kernel direct mappings).
thanks,
suresh
Tejun Heo
maintainerHello, Suresh.
Quoted 7 lines · Suresh Siddha
Quoted 6 lines · Tejun Heo
Hello, Suresh.
Quoted 3 lines · Suresh Siddha
Tejun, Can you please educate me why we need to map this first percpu chunk (which is pre-allocated during boot and is physically contiguous) into vmalloc area?
Quoted 8 lines
Quoted 2 lines
To make areas for each cpu congruent such that the address offset of a percpu symbol for CPU N is always the same from the address for CPU 0.
But for the first percpu chunk, isn't it the case that the physical address allocations for a particular cpu is contiguous (as you are using one bootmem allocation for whole
PMD_SIZEfor any given cpu)? So both the kernel direct mapping aswell as the vmalloc mappings are contiguous for the first chunk, on any given cpu. Right?
Hmmm... okay. Percpu areas are composed of multiple chunks. A single chunk is composed of multiple units, one unit for each CPU. Units in a single chunk should be contiguous and of the same size such that unit_addr_for_cpu_N == chunk_addr + N * unit_size whereas chunks don't need to have any special address relation to other chunks. Combined, this results in percpu addresses for CPU N are always offset by N * unit_size from the percpu adresses for CPU 0 which can be efficiently determined using some extra resource in the processor (segment register on x86 for example).
For remap first chunk allocator, each unit for each CPU is allocated separately using the bootmem allocator. Each unit is continuous but they still need to be assembed into a single contiguous area to be used as the first chunk, which is where the remapping comes in. So, the extra requirement is that units in the same chunk need to be contiguous and NUMA allocation means units will be spread according to NUMA configuration, so they need to be put together by remapping them.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Perhaps even for the other dynamically allocated secondary chunks? (as far as I can see, all the chunk allocations seems to be physically contiguous and later mapped into vmalloc area)..
That should simplify these things quite a bit(atleast for first percpu chunk). I am missing something obvious I guess.
Quoted 8 lines
Quoted 2 lines
Hmm... Sorry I don't really follow. Can you please elaborate the question?
For the first percpu chunk, we can use the kernel direct mapping and avoid the vmalloc mapping of
PMD_SIZE. And avoid the vmap address aliasing problem (wrt to free pages that we have given back to -mm) that we are trying to avoid with this patchset (as the existing cpa code already takes care of the kernel direct mappings).
Hmmm.... If you can show me how to use the linear mapping directly, I'll be happy as a clam.
Thanks.
Signature · 1 line
tejun
Quoted 4 lines · Tejun Heo
Units in
a single chunk should be contiguous and of the same size such thatunit_addr_for_cpu_N==chunk_addr+ N *unit_sizewhereas chunks don't need to have any special address relation to other chunks.
And the size of the unit is same across chunks.
This is what I was missing :)
Can we don't use PERCPU_DYNAMIC_RESERVE in the first chunk and for dynamic per_cpu_ptr's we can use some other offset such as per_cpu_ptr_offset() or some such thing?
Then we can separate the static and dynamic chunks. And use large page kernel-direct mappings for fast access for critical common things and use small page accesses for dynamic and not so common accesses.
Just checking to see if we can reduce the complexity of setting up the percpu areas (different versions for embed, non-embed etc) and handling all these aliases with simple code, rather than making it complex.
thanks,
suresh
H. Peter Anvin
reviewerQuoted 5 lines · Suresh Siddha
Can we don't use
PERCPU_DYNAMIC_RESERVEin the first chunk and for dynamic per_cpu_ptr's we can use some other offset such asper_cpu_ptr_offset()or some such thing?
Quoted 8 lines
Then we can separate the static and dynamic chunks. And use large page kernel-direct mappings for fast access for critical common things and use small page accesses for dynamic and not so common accesses.
Just checking to see if we can reduce the complexity of setting up the percpu areas (different versions for embed, non-embed etc) and handling all these aliases with simple code, rather than making it complex.
I'm confused what you're suggesting here. The whole point of the percpu unification work is that we can use %gs:absolute type references to hit a variable right away. Although in theory we could use both %fs and %gs for pointers, setting up both would greatly increase the cost of entering the kernel, especially on 64 bits.
This means all percpu data has to have a constant (virtual) offset from the beginning of the static percpu area.
-hpa
Quoted 15 lines · H. Peter Anvin
Quoted 5 lines · Suresh Siddha
Can we don't use
PERCPU_DYNAMIC_RESERVEin the first chunk and for dynamic per_cpu_ptr's we can use some other offset such asper_cpu_ptr_offset()or some such thing?Quoted 8 lines
Then we can separate the static and dynamic chunks. And use large page kernel-direct mappings for fast access for critical common things and use small page accesses for dynamic and not so common accesses.
Just checking to see if we can reduce the complexity of setting up the percpu areas (different versions for embed, non-embed etc) and handling all these aliases with simple code, rather than making it complex.
Quoted 8 lines
I'm confused what you're suggesting here. The whole point of the percpu unification work is that we can use %gs:absolute type references to hit a variable right away. Although in theory we could use both %fs and %gs for pointers, setting up both would greatly increase the cost of entering the kernel, especially on 64 bits.
This means all percpu data has to have a constant (virtual) offset from the beginning of the static percpu area.
This %gs:absolute type accesses are for static percpu data.
But what I was referring to is the dynamic percpu data(accessed through per_cpu_ptr()). Instead of combining some part of the dynamic percpu data into the static percpu data(first percpu chunk), we can use different chunks for dynamic percpu data and governed by a different per_cpu_dynamic_offset[NR_CPUS] array.
Then we can use large-page kernel direct mappings for static percpu data (or %gs:offset) and small-page vmalloc mappings for dynamic percpu data.
thanks,
suresh
Tejun Heo
maintainerHello, Suresh.
Quoted 10 lines · Suresh Siddha
This %gs:absolute type accesses are for static percpu data.
But what I was referring to is the dynamic percpu
data(accessed through per_cpu_ptr()). Instead of combining some part of the dynamic percpu data into the static percpu data(first percpu chunk), we can use different chunks for dynamic percpu data and governed by a different per_cpu_dynamic_offset[NR_CPUS] array.Then we can use large-page kernel direct mappings for static percpu data (or %gs:offset) and small-page vmalloc mappings for dynamic percpu data.
Hmmm... I can't really follow what you're suggesting. Can you please explain it in more detailed way?
Thanks.
Signature · 1 line
tejun
Quoted 11 lines · Tejun Heo
Hello, Suresh.
Quoted 8 lines · Suresh Siddha
This %gs:absolute type accesses are for static percpu data.
But what I was referring to is the dynamic percpu
data(accessed through per_cpu_ptr()). Instead of combining some part of the dynamic percpu data into the static percpu data(first percpu chunk), we can use different chunks for dynamic percpu data and governed by a different per_cpu_dynamic_offset[NR_CPUS] array.
Quoted 5 lines
Quoted 2 lines
Then we can use large-page kernel direct mappings for static percpu data (or %gs:offset) and small-page vmalloc mappings for dynamic percpu data.
Hmmm... I can't really follow what you're suggesting. Can you please explain it in more detailed way?
Ok. Before I make another attempt walking that hill :)
I was talking to Peter and it seems there are some requests to change the first percpu unit allocation for each possible cpu using bootmem allocator, to allocating the corresponding unit at the cpu online time.
Do you have plans to change this? If we do this allocation during the corresponding cpu online time and don't end up using big pages, then also we avoid all these aliasing issues...
thanks,
suresh
Tejun Heo
maintainerHello,
Quoted 4 lines · Suresh Siddha
Quoted 2 lines
Hmmm... I can't really follow what you're suggesting. Can you please explain it in more detailed way?
Ok. Before I make another attempt walking that hill :)
Heh... the problem is that unless I understand what you're trying to achieve (and vice-versa), our discussion is likely to be riddled with confusions, and currently either you're misunderstanding the whole thing or I'm being slow (not too unusual :-). It would be nice to determine which way it is.
Quoted 5 lines
I was talking to Peter and it seems there are some requests to change the first percpu unit allocation for each possible cpu using bootmem allocator, to allocating the corresponding unit at the cpu online time.
Do you have plans to change this?
Yeap, once the remaining archs are converted, that's the next stop. With recently posted patchset, only three remain - sparc64, powerpc64 and sparc64. Davem is working on sparc64. I'm planning on doing powerpc64. ia64 is a bit tricky as it already remaps percpu areas but I'm sure we'll find a way around it. After that, yeap, the dynamic online thing.
Quoted 3 lines
If we do this allocation during the corresponding cpu online time and don't end up using big pages, then also we avoid all these aliasing issues...
The dynamic onlining will probably use 4k pages so, yeah, it won't have the alias issues but that's not the issue here, right? You can already avoid aliasing that way by simply using 4k allocator from the get-go.
Thanks.
Signature · 1 line
tejun
Quoted 5 lines · Tejun Heo
Heh... the problem is that unless I understand what you're trying to achieve (and vice-versa), our discussion is likely to be riddled with confusions, and currently either you're misunderstanding the whole thing or I'm being slow (not too unusual :-). It would be nice to determine which way it is.
At the start of this, I was trying to see a way where we can use large page mappings for percpu areas and at the same time, avoid the complexity of cpa() that we are adding in this patchset. So, I was going down the path of exploiting your bootmem allocation for the first percpu chunk (as kernel identity mappings are already taken care by cpa()).
Quoted 4 lines
The dynamic onlining will probably use 4k pages so, yeah, it won't have the alias issues but that's not the issue here, right? You can already avoid aliasing that way by simply using 4k allocator from the get-go.
But now that I learnt about dynamic online allocation, can we avoid the complexity brought by this patchset, by simply using 4k allocator from get-go.
i.e., can we drop this remap pageattr handling patchset and simply use 4k mapping for now? And move to dynamic allocation at a later point.
This will simplify quite a bit of code.
thanks,
suresh
Tejun Heo
maintainerHello,
Quoted 8 lines · Suresh Siddha
Quoted 4 lines
The dynamic onlining will probably use 4k pages so, yeah, it won't have the alias issues but that's not the issue here, right? You can already avoid aliasing that way by simply using 4k allocator from the get-go.
But now that I learnt about dynamic online allocation, can we avoid the complexity brought by this patchset, by simply using 4k allocator from get-go.
Sure, we can.
Quoted 2 lines
i.e., can we drop this remap pageattr handling patchset and simply use 4k mapping for now? And move to dynamic allocation at a later point.
4k or not, x86 is already on dynamic allocation. The only difference is how the first chunk is allocated.
Quoted 1 line
This will simplify quite a bit of code.
Yes it will. The question is which way would be better. Till now, there hasn't been any actual data on how remap compares to 4k. The only thing we know is that, on UMA, embed should behave exactly the same for static percpu variables as before the whole dynamic allocator.
On NUMA, both remap and 4k add some level of TLB pressure. remap will waste one more PMD TLB entry (dup) while 4k adds a bunch of 4k ones (non-dup but what used to be accessed by PMD TLB is now accessed with PTE TLB). Some say using one more PMD TLB is better while others disagree. So, the best course of action here seems to offer both and easy way to select between them so that data can be gathered, which is what this patchset does.
I don't think the added complexity for cpa() justifies dropping remap without further testing. The added complexity isn't that big. Most of the confusion in this patchset came from my ignorance on the subject. cpa() is a fragile thing but we need it anyway, so...
Thanks.
Signature · 1 line
tejun
Quoted 4 lines · Tejun Heo
On NUMA, both remap and 4k add some level of TLB pressure. remap will waste one more PMD TLB entry (dup) while 4k adds a bunch of 4k ones (non-dup but what used to be accessed by PMD TLB is now accessed with PTE TLB).
Correction: 4k TLBs are dups too.
Thanks.
Signature · 1 line
tejun
Quoted 2 lines · Tejun Heo
Yes it will. The question is which way would be better. Till now, there hasn't been any actual data on how remap compares to 4k.
I am not sure if we see any measurable difference. Even if we use 4k entries, it will be few entries that kernel will be referring to frequently.
Quoted 7 lines
On NUMA, both remap and 4k add some level of TLB pressure. remap will waste one more PMD TLB entry (dup) while 4k adds a bunch of 4k ones (non-dup but what used to be accessed by PMD TLB is now accessed with PTE TLB). Some say using one more PMD TLB is better while others disagree. So, the best course of action here seems to offer both and easy way to select between them so that data can be gathered, which is what this patchset does.
So with the planned future change of percpu unit allocation during cpu online, you are planning to try large page allocation first and then fallback to 4k pages, if that doesn't succeed. And then populate new percpu ptr accordingly and then sort wrt to other cpu ptr's, so that we can keep aliases in sync for future(and in parallel) cpa()'s that might be happening.
There is nothing wrong with all this. Just the code complexity (and maintenance) for what we are trying to gain ;)
Quoted 8 lines
I don't think the added complexity for
cpa()justifies dropping remap without further testing. The added complexity isn't that big. Most of the confusion in this patchset came from my ignorance on the subject.cpa()is a fragile thing but we need it anyway, so...Thanks.
Tejun Heo
maintainerHello, Suresh.
Quoted 7 lines · Suresh Siddha
Quoted 2 lines · Tejun Heo
Yes it will. The question is which way would be better. Till now, there hasn't been any actual data on how remap compares to 4k.
I am not sure if we see any measurable difference. Even if we use 4k entries, it will be few entries that kernel will be referring to frequently.
Yeah, I hope so too but I *really* want to see some numbers before taking further actions. Remap is chosen as the default because it deviates less from the original behavior but I'll be excited to drop it if 4k works just fine.
Quoted 15 lines
Quoted 7 lines
On NUMA, both remap and 4k add some level of TLB pressure. remap will waste one more PMD TLB entry (dup) while 4k adds a bunch of 4k ones (non-dup but what used to be accessed by PMD TLB is now accessed with PTE TLB). Some say using one more PMD TLB is better while others disagree. So, the best course of action here seems to offer both and easy way to select between them so that data can be gathered, which is what this patchset does.
So with the planned future change of percpu unit allocation during cpu online, you are planning to try large page allocation first and then fallback to 4k pages, if that doesn't succeed. And then populate new percpu ptr accordingly and then sort wrt to other cpu ptr's, so that we can keep aliases in sync for future(and in parallel) cpa()'s that might be happening.
Quoted 2 lines
There is nothing wrong with all this. Just the code complexity (and maintenance) for what we are trying to gain ;)
No, I'll let the first chunk allocation happen the same way for cpus available on boot and then just do 4k allocations for whatever necessary afterward. The needed code change in percpu proper isn't that big. What would take more effort is auditing all percpu users.
Thanks.
Signature · 1 line
tejun
Quoted 4 lines · Tejun Heo
No, I'll let the first chunk allocation happen the same way for cpus available on boot and then just do 4k allocations for whatever necessary afterward. The needed code change in percpu proper isn't that big.
Ok. It will be more cleaner this way.
thanks,
suresh
Ingo, Ping.
Signature · 1 line
tejun
Quoted 1 line · Tejun Heo
Ingo, Ping.
hpa is handling this - Peter will reply if he has any updates.
Ingo
Quoted 5 lines · Ingo Molnar
Quoted 1 line · Tejun Heo
Ingo, Ping.
hpa is handling this - Peter will reply if he has any updates.
Okay, but please ignore this thread. I thought I was replying to take#2 thread. At any rate, the git vector is the same and contains the final version.
Thanks.
Signature · 1 line
tejun