perf_event: Provide vmalloc() based mmap() backing
Peter Zijlstra
maintainerPATCH 1/1 perf_event: Provide vmalloc() based mmap() backing
OK, so I rewrote it a few more times, does the below make everybody feel warm and fuzzy?
Dave, I preserved your ACK, since you don't seem to care about the particular form, as long as the problem gets solved. Feel free to kick me if I got that wrong ;-)
---
Subject: perf_event: Provide vmalloc() based mmap() backing
From: Peter Zijlstra <[email protected]>
Date: Mon, 21 Sep 2009 16:08:49 +0200
Some architectures such as Sparc, ARM and MIPS (basically everything with flush_dcache_page()) need to deal with dcache aliases by carefully placing pages in both kernel and user maps.
These architectures typically have to use vmalloc_user() for this.
However, on other architectures, vmalloc() is not needed and has the downsides of being more restricted and slower than regular allocations.
--- arch/sparc/Kconfig | 2 include/linux/perf_event.h | 5 init/Kconfig | 18 +++ kernel/perf_event.c | 248 +++++++++++++++++++++++++++++++++------------ tools/perf/design.txt | 3 5 files changed, 214 insertions(+), 62 deletions(-)
include/linux/perf_event.h +5 -0
@@ -442,6 +442,7 @@ enum perf_callchain_context {#include <linux/hrtimer.h> #include <linux/fs.h> #include <linux/pid_namespace.h> +#include <linux/workqueue.h> #include <asm/atomic.h> #define PERF_MAX_STACK_DEPTH 255
@@ -513,6 +514,10 @@ struct file;struct perf_mmap_data {struct rcu_head rcu_head; +#ifdef CONFIG_PERF_USE_VMALLOC +struct work_struct work; +#endif +int data_order;int nr_pages;/* nr of data pages */ int writable;/* are we writable */ int nr_locked;/* nr pages mlocked */
kernel/perf_event.c +186 -62
348 lines not on this page · raw message
arch/sparc/Kconfig +2 -0
@@ -26,6 +26,7 @@ config SPARC select RTC_CLASS select RTC_DRV_M48T59 select HAVE_PERF_EVENTS + select PERF_USE_VMALLOC select HAVE_DMA_ATTRS select HAVE_DMA_API_DEBUG
@@ -48,6 +49,7 @@ config SPARC64 select RTC_DRV_SUN4V select RTC_DRV_STARFIRE select HAVE_PERF_EVENTS + select PERF_USE_VMALLOC config ARCH_DEFCONFIG string
init/Kconfig +18 -0
@@ -930,6 +930,11 @@ config HAVE_PERF_EVENTS help See tools/perf/design.txt for details. +config PERF_USE_VMALLOC + bool + help + See tools/perf/design.txt for details + menu "Kernel Performance Events And Counters" config PERF_EVENTS
@@ -985,6 +990,19 @@ config PERF_COUNTERS Say N if unsure. +config DEBUG_PERF_USE_VMALLOC + default n + bool "Debug: use vmalloc to back perf mmap() buffers" + depends on PERF_EVENTS && DEBUG_KERNEL + select PERF_USE_VMALLOC + help + Use vmalloc memory to back perf mmap() buffers. + + Mostly useful for debugging the vmalloc code on platforms + that don't require it. + + Say N if unsure. + endmenu config VM_EVENT_COUNTERS
tools/perf/design.txt +3 -0
@@ -455,3 +455,6 @@ will need at least this: If your architecture does have hardware capabilities, you can override the weak stub hw_perf_event_init() to register hardware counters. + +Architectures that have d-cache aliassing issues, such as Sparc and ARM, +should select PERF_USE_VMALLOC in order to avoid these for perf mmap().
Quoted 2 lines · Peter Zijlstra
OK, so I rewrote it a few more times, does the below make everybody feel warm and fuzzy?
Yeah, looks perfect to me - applied to tip:perf/urgent, thanks Peter!
Dave, could you please test it on Sparc too? The Git coordinates are at:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git perf/urgent
Ingo
David Miller
maintainerFrom: Ingo Molnar <[email protected]>
Date: Tue, 6 Oct 2009 14:23:59 +0200
Quoted 3 lines
Dave, could you please test it on Sparc too? The Git coordinates are at:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.gitperf/urgent
I'll find some time to give it a spin, thanks!
David Miller
maintainerFrom: Ingo Molnar <[email protected]>
Date: Tue, 6 Oct 2009 14:23:59 +0200
Quoted 11 lines · Peter Zijlstra
Quoted 3 lines
OK, so I rewrote it a few more times, does the below make everybody feel warm and fuzzy?
Yeah, looks perfect to me - applied to tip:perf/urgent, thanks Peter!
Dave, could you please test it on Sparc too? The Git coordinates are at:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.gitperf/urgent
Works fine here, I forced the vmalloc stuff on using the debug option. Once your stuff goes in I'll add the select statement to arch/sparc/Kconfig
Andrew also needs to marge the vmalloc_user() patch. I almost forgot that to apply to my tree when testing :-)
Andrew Morton
maintainerQuoted 13 lines · On Wed, 07 Oct 2009 00:13:21 -0700 (PDT) David Miller <[email protected]>…
From: Ingo Molnar <[email protected]>
Date: Tue, 6 Oct 2009 14:23:59 +0200Quoted 10 lines · Peter Zijlstra
Quoted 3 lines
OK, so I rewrote it a few more times, does the below make everybody feel warm and fuzzy?
Yeah, looks perfect to me - applied to tip:perf/urgent, thanks Peter!
Dave, could you please test it on Sparc too? The Git coordinates are at:
Quoted 8 lines
Quoted 1 line
git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.gitperf/urgentWorks fine here, I forced the vmalloc stuff on using the debug option. Once your stuff goes in I'll add the select statement to
arch/sparc/KconfigAndrew also needs to marge the
vmalloc_user()patch. I almost forgot that to apply to my tree when testing :-)
I seem to have lost it. We might as well keep the two patches together so can you please handle it?
David Miller
maintainerFrom: Andrew Morton <[email protected]>
Date: Wed, 7 Oct 2009 00:32:02 -0700
Quoted 2 lines
I seem to have lost it. We might as well keep the two patches together so can you please handle it?
Ingo, in case you don't have a copy handy, I attach it below. Please add it to these perf mmap bits in your tree, thanks!
mm: Make vmalloc_user() align base kernel virtual address to SHMLBA.
When a vmalloc'd area is mmap'd into userspace, some kind of co-ordination is necessary for this to work on platforms with cpu D-caches which can have aliases.
Otherwise kernel side writes won't be seen properly in userspace and vice versa.
If the kernel side mapping and the user side one have the same alignment, modulo SHMLBA, this can work as long as VM_SHARED is shared of VMA and for all current users this is true. VM_SHARED will force SHMLBA alignment of the user side mmap on platforms with D-cache aliasing matters.
The bulk of this patch is just making it so that a specific alignment can be passed down into __get_vm_area_node(). All existing callers pass in '1' which preserves existing behavior. vmalloc_user() gives SHMLBA for the alignment.
As a side effect this should get the video media drivers and other vmalloc_user() users into more working shape on such systems.
mm/vmalloc.c +26 -22
@@ -29,6 +29,7 @@#include <asm/atomic.h> #include <asm/uaccess.h> #include <asm/tlbflush.h> +#include <asm/shmparam.h> /*** Page table manipulation functions ***/
@@ -1156,12 +1157,11 @@ static void insert_vmalloc_vm(struct vm_struct *vm, struct vmap_area *va, }static struct vm_struct *__get_vm_area_node (unsigned long size, -unsigned long flags,unsigned long start,unsigned long end, -int node,gfp_t gfp_mask,void *caller) +unsigned long align,unsigned long flags,unsigned long start, +unsigned long end,int node,gfp_t gfp_mask,void *caller) {static struct vmap_area *va;struct vm_struct *area; -unsigned long align =1 ;BUG_ON (in_interrupt ());if (flags &VM_IOREMAP ) {
@@ -1201,7 +1201,7 @@ static struct vm_struct *__get_vm_area_node(unsigned long size,struct vm_struct *__get_vm_area (unsigned long size,unsigned long flags,unsigned long start,unsigned long end) { -return __get_vm_area_node (size, flags, start, end,-1 ,GFP_KERNEL , +return __get_vm_area_node (size,1 , flags, start, end,-1 ,GFP_KERNEL ,__builtin_return_address (0 )); }EXPORT_SYMBOL_GPL (__get_vm_area);
@@ -1210,7 +1210,7 @@ struct vm_struct *__get_vm_area_caller(unsigned long size, unsigned long flags, unsigned long start, unsigned long end, void *caller) { - return __get_vm_area_node(size, flags, start, end, -1, GFP_KERNEL, + return __get_vm_area_node(size, 1, flags, start, end, -1, GFP_KERNEL,caller ); }
@@ -1225,22 +1225,22 @@ struct vm_struct *__get_vm_area_caller(unsigned long size, unsigned long flags,*/ struct vm_struct *get_vm_area (unsigned long size,unsigned long flags) { -return __get_vm_area_node (size, flags,VMALLOC_START ,VMALLOC_END , +return __get_vm_area_node (size,1 , flags,VMALLOC_START ,VMALLOC_END ,-1 ,GFP_KERNEL ,__builtin_return_address (0 )); }struct vm_struct *get_vm_area_caller (unsigned long size,unsigned long flags,void *caller) { -return __get_vm_area_node (size, flags,VMALLOC_START ,VMALLOC_END , +return __get_vm_area_node (size,1 , flags,VMALLOC_START ,VMALLOC_END ,-1 ,GFP_KERNEL , caller); }struct vm_struct *get_vm_area_node (unsigned long size,unsigned long flags,int node,gfp_t gfp_mask) { -return __get_vm_area_node (size, flags,VMALLOC_START ,VMALLOC_END , node, - gfp_mask,__builtin_return_address (0 )); +return __get_vm_area_node (size,1 , flags,VMALLOC_START ,VMALLOC_END , + node, gfp_mask,__builtin_return_address (0 )); }static struct vm_struct *find_vm_area (const void *addr)
@@ -1403,7 +1403,8 @@ void *vmap(struct page **pages, unsigned int count, }EXPORT_SYMBOL (vmap); -static void *__vmalloc_node (unsigned long size,gfp_t gfp_mask,pgprot_t prot, +static void *__vmalloc_node (unsigned long size,unsigned long align, +gfp_t gfp_mask,pgprot_t prot,int node,void *caller);static void *__vmalloc_area_node (struct vm_struct *area,gfp_t gfp_mask,pgprot_t prot,int node,void *caller)
@@ -1417,7 +1418,7 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, area->nr_pages = nr_pages;/* Please note that the recursion is strictly bounded. */ if (array_size >PAGE_SIZE ) { - pages =__vmalloc_node (array_size, gfp_mask | __GFP_ZERO, + pages =__vmalloc_node (array_size,1 , gfp_mask | __GFP_ZERO,PAGE_KERNEL , node, caller); area->flags |=VM_VPAGES ; }else {
@@ -1476,6 +1477,7 @@ void *__vmalloc_area(struct vm_struct *area, gfp_t gfp_mask, pgprot_t prot)/** * __vmalloc_node - allocate virtually contiguous memory * @size: allocation size +* @align: desired alignment * @gfp_mask: flags for the page level allocator * @prot: protection mask for the allocated pages * @node: node to use for allocation or -1
@@ -1485,8 +1487,9 @@ void *__vmalloc_area(struct vm_struct *area, gfp_t gfp_mask, pgprot_t prot)* allocator with @gfp_mask flags. Map them into contiguous * kernel virtual space, using a pagetable protection of @prot. */ -static void *__vmalloc_node (unsigned long size,gfp_t gfp_mask,pgprot_t prot, -int node,void *caller) +static void *__vmalloc_node (unsigned long size,unsigned long align, +gfp_t gfp_mask,pgprot_t prot, +int node,void *caller) {struct vm_struct *area;void *addr;
@@ -1496,8 +1499,8 @@ static void *__vmalloc_node(unsigned long size, gfp_t gfp_mask, pgprot_t prot,if (!size || (size >>PAGE_SHIFT ) > totalram_pages)return NULL ; - area =__get_vm_area_node (size,VM_ALLOC ,VMALLOC_START ,VMALLOC_END , - node, gfp_mask, caller); + area =__get_vm_area_node (size, align,VM_ALLOC ,VMALLOC_START , +VMALLOC_END , node, gfp_mask, caller);if (!area)return NULL ;
@@ -1516,7 +1519,7 @@ static void *__vmalloc_node(unsigned long size, gfp_t gfp_mask, pgprot_t prot,void *__vmalloc (unsigned long size,gfp_t gfp_mask,pgprot_t prot) { -return __vmalloc_node (size, gfp_mask, prot,-1 , +return __vmalloc_node (size,1 , gfp_mask, prot,-1 ,__builtin_return_address (0 )); }EXPORT_SYMBOL (__vmalloc);
@@ -1532,7 +1535,7 @@ EXPORT_SYMBOL(__vmalloc);*/ void *vmalloc (unsigned long size) { -return __vmalloc_node (size,GFP_KERNEL | __GFP_HIGHMEM,PAGE_KERNEL , +return __vmalloc_node (size,1 ,GFP_KERNEL | __GFP_HIGHMEM,PAGE_KERNEL ,-1 ,__builtin_return_address (0 )); }EXPORT_SYMBOL (vmalloc);
@@ -1549,7 +1552,8 @@ void *vmalloc_user(unsigned long size)struct vm_struct *area;void *ret; - ret =__vmalloc_node (size,GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO, + ret =__vmalloc_node (size,SHMLBA , +GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO,PAGE_KERNEL ,-1 ,__builtin_return_address (0 ));if (ret) { area =find_vm_area (ret);
@@ -1572,7 +1576,7 @@ EXPORT_SYMBOL(vmalloc_user);*/ void *vmalloc_node (unsigned long size,int node) { -return __vmalloc_node (size,GFP_KERNEL | __GFP_HIGHMEM,PAGE_KERNEL , +return __vmalloc_node (size,1 ,GFP_KERNEL | __GFP_HIGHMEM,PAGE_KERNEL , node,__builtin_return_address (0 )); }EXPORT_SYMBOL (vmalloc_node);
@@ -1595,7 +1599,7 @@ EXPORT_SYMBOL(vmalloc_node);void *vmalloc_exec (unsigned long size) { -return __vmalloc_node (size,GFP_KERNEL | __GFP_HIGHMEM,PAGE_KERNEL_EXEC , +return __vmalloc_node (size,1 ,GFP_KERNEL | __GFP_HIGHMEM,PAGE_KERNEL_EXEC ,-1 ,__builtin_return_address (0 )); }
@@ -1616,7 +1620,7 @@ void *vmalloc_exec(unsigned long size)*/ void *vmalloc_32 (unsigned long size) { -return __vmalloc_node (size,GFP_VMALLOC32 ,PAGE_KERNEL , +return __vmalloc_node (size,1 ,GFP_VMALLOC32 ,PAGE_KERNEL ,-1 ,__builtin_return_address (0 )); }EXPORT_SYMBOL (vmalloc_32);
@@ -1633,7 +1637,7 @@ void *vmalloc_32_user(unsigned long size)struct vm_struct *area;void *ret; - ret =__vmalloc_node (size,GFP_VMALLOC32 | __GFP_ZERO,PAGE_KERNEL , + ret =__vmalloc_node (size,1 ,GFP_VMALLOC32 | __GFP_ZERO,PAGE_KERNEL ,-1 ,__builtin_return_address (0 ));if (ret) { area =find_vm_area (ret);
Commit-ID: 906010b2134e14a2e377decbadd357b3d0ab9c6a Gitweb: http://git.kernel.org/tip/906010b2134e14a2e377decbadd357b3d0ab9c6a Author: Peter Zijlstra <[email protected]> AuthorDate: Mon, 21 Sep 2009 16:08:49 +0200 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 6 Oct 2009 14:21:50 +0200
perf_event: Provide vmalloc() based mmap() backing
Some architectures such as Sparc, ARM and MIPS (basically everything with flush_dcache_page()) need to deal with dcache aliases by carefully placing pages in both kernel and user maps.
These architectures typically have to use vmalloc_user() for this.
However, on other architectures, vmalloc() is not needed and has the downsides of being more restricted and slower than regular allocations.
LKML-Reference: <1254830228.21044.272.camel@laptop>
--- arch/sparc/Kconfig | 2 + include/linux/perf_event.h | 5 + init/Kconfig | 18 +++ kernel/perf_event.c | 248 +++++++++++++++++++++++++++++++++----------- tools/perf/design.txt | 3 + 5 files changed, 214 insertions(+), 62 deletions(-)
arch/sparc/Kconfig +2 -0
@@ -26,6 +26,7 @@ config SPARC select RTC_CLASS select RTC_DRV_M48T59 select HAVE_PERF_EVENTS + select PERF_USE_VMALLOC select HAVE_DMA_ATTRS select HAVE_DMA_API_DEBUG
@@ -48,6 +49,7 @@ config SPARC64 select RTC_DRV_SUN4V select RTC_DRV_STARFIRE select HAVE_PERF_EVENTS + select PERF_USE_VMALLOC config ARCH_DEFCONFIG string
include/linux/perf_event.h +5 -0
@@ -442,6 +442,7 @@ enum perf_callchain_context {#include <linux/hrtimer.h> #include <linux/fs.h> #include <linux/pid_namespace.h> +#include <linux/workqueue.h> #include <asm/atomic.h> #define PERF_MAX_STACK_DEPTH 255
@@ -513,6 +514,10 @@ struct file;struct perf_mmap_data {struct rcu_head rcu_head; +#ifdef CONFIG_PERF_USE_VMALLOC +struct work_struct work; +#endif +int data_order;int nr_pages;/* nr of data pages */ int writable;/* are we writable */ int nr_locked;/* nr pages mlocked */
init/Kconfig +18 -0
@@ -921,6 +921,11 @@ config HAVE_PERF_EVENTS help See tools/perf/design.txt for details. +config PERF_USE_VMALLOC + bool + help + See tools/perf/design.txt for details + menu "Kernel Performance Events And Counters" config PERF_EVENTS
@@ -976,6 +981,19 @@ config PERF_COUNTERS Say N if unsure. +config DEBUG_PERF_USE_VMALLOC + default n + bool "Debug: use vmalloc to back perf mmap() buffers" + depends on PERF_EVENTS && DEBUG_KERNEL + select PERF_USE_VMALLOC + help + Use vmalloc memory to back perf mmap() buffers. + + Mostly useful for debugging the vmalloc code on platforms + that don't require it. + + Say N if unsure. + endmenu config VM_EVENT_COUNTERS
kernel/perf_event.c +186 -62
348 lines not on this page · raw message
tools/perf/design.txt +3 -0
@@ -455,3 +455,6 @@ will need at least this: If your architecture does have hardware capabilities, you can override the weak stub hw_perf_event_init() to register hardware counters. + +Architectures that have d-cache aliassing issues, such as Sparc and ARM, +should select PERF_USE_VMALLOC in order to avoid these for perf mmap().
David Miller
maintainerFrom: Peter Zijlstra <[email protected]>
Date: Tue, 06 Oct 2009 13:57:08 +0200
Quoted 3 lines
Dave, I preserved your ACK, since you don't seem to care about the particular form, as long as the problem gets solved. Feel free to kick me if I got that wrong ;-)
That's perfectly fine, thanks!