perf_counter: Track all mmaps, heap and stack extensions
Subsystems (4)
Hi,
Right now perf_counter only logs executable mmaps. While this is enough for instruction profiling, at some stage we are also going to want to do data profiling. This will require us to log non-executable mmaps as well as stack and heap extensions.
Why would we care about heap and stack? A few examples:
1. We can monitor TLB miss rates to suggest what regions of memory should be put into hugepages.
2. We can look into various TLB miss issues. On PowerPC a data prefetch that goes to an unmapped area takes a significant amount of time (it initiates a tablewalk that may take 40+ cycles). With accurate mapping data we can catch areas of code with bad prefetch instructions.
Taking it a bit further, since in some sense perf_counter is a channel for getting events out to userspace, I wonder if we could solve Roland's RDMA address space unmap issue with perf_counter:
http://patchwork.kernel.org/patch/37267/
Below is a dodgy hack I've been using to prototype tracking of all mmaps and heap/stack extensions. Naturally we'd want a perf_counter feature to turn this on and keep instruction profiles more compact. We'd also want munmap events.
The only tricky part is having to set start_stack before we call expand_stack for the first time, so we can easily identify it in perf_counter_mmap_event.
Thoughts?
Anton
---
mm/mmap.c +2 -0
@@ -1680,6 +1680,7 @@ static int expand_downwards(struct vm_arif (!error) { vma->vm_start = address; vma->vm_pgoff -= grow; +perf_counter_mmap (vma); } }anon_vma_unlock (vma);
@@ -2071,6 +2072,7 @@ unsigned long do_brk(unsigned long addr, vma->vm_page_prot =vm_get_page_prot (flags);vma_link (mm, vma, prev, rb_link, rb_parent); out: +perf_counter_mmap (vma); mm->total_vm += len >>PAGE_SHIFT ;if (flags &VM_LOCKED ) {if (!mlock_vma_pages_range (vma, addr, addr + len))
include/linux/perf_counter.h +1 -2
@@ -702,8 +702,7 @@ extern void __perf_counter_mmap(struct vstatic inline void perf_counter_mmap (struct vm_area_struct *vma) { -if (vma->vm_flags &VM_EXEC ) -__perf_counter_mmap (vma); +__perf_counter_mmap (vma); }extern void perf_counter_comm (struct task_struct *tsk);
kernel/perf_counter.c +8 -0
@@ -3142,6 +3142,14 @@ static void perf_counter_mmap_event(struif (!vma->vm_mm) { name =strncpy (tmp,"[vdso]" ,sizeof (tmp)); goto got_name; + }else if (vma->vm_start <= vma->vm_mm->start_brk && + vma->vm_end >= vma->vm_mm->brk) { + name =strncpy (tmp,"[heap]" ,sizeof (tmp)); + goto got_name; + }else if (vma->vm_start <= vma->vm_mm->start_stack && + vma->vm_end >= vma->vm_mm->start_stack) { + name =strncpy (tmp,"[stack]" ,sizeof (tmp)); + goto got_name; } name =strncpy (tmp,"//anon" ,sizeof (tmp));
fs/exec.c +1 -0
@@ -631,6 +631,7 @@ int setup_arg_pages(struct linux_binprm#else stack_base = vma->vm_start -EXTRA_STACK_VM_PAGES *PAGE_SIZE ;#endif + current->mm->start_stack = bprm->p; ret =expand_stack (vma, stack_base);if (ret) ret = -EFAULT ;
Quoted 9 lines · Anton Blanchard
Hi,
Right now
perf_counteronly logs executable mmaps. While this is enough for instruction profiling, at some stage we are also going to want to do data profiling. This will require us to log non-executable mmaps as well as stack and heap extensions.Why would we care about heap and stack? A few examples:
Quoted 7 lines
1. We can monitor TLB miss rates to suggest what regions of memory should be put into hugepages.
2. We can look into various TLB miss issues. On PowerPC a data prefetch that goes to an unmapped area takes a significant amount of time (it initiates a tablewalk that may take 40+ cycles). With accurate mapping data we can catch areas of code with bad prefetch instructions.
Agreed.
Quoted 5 lines
Taking it a bit further, since in some sense
perf_counteris a channel for getting events out to userspace, I wonder if we could solve Roland's RDMA address space unmap issue withperf_counter:
Yeah, saw some of that, Andrew has some good points there. Haven't had time to see the reply yet.
Not sure its a good match, but if so, very nice.
Quoted 4 lines
Below is a dodgy hack I've been using to prototype tracking of all mmaps and heap/stack extensions. Naturally we'd want a
perf_counterfeature to turn this on and keep instruction profiles more compact. We'd also want munmap events.
I used to have an munmap hook in there at some point. We could restore that.
Quoted 1 line
Thoughts?
Seems like a good direction, go for it ;-)
Quoted 36 lines · Peter Zijlstra
Quoted 9 lines · Anton Blanchard
Hi,
Right now
perf_counteronly logs executable mmaps. While this is enough for instruction profiling, at some stage we are also going to want to do data profiling. This will require us to log non-executable mmaps as well as stack and heap extensions.Why would we care about heap and stack? A few examples:
Quoted 7 lines
1. We can monitor TLB miss rates to suggest what regions of memory should be put into hugepages.
2. We can look into various TLB miss issues. On PowerPC a data prefetch that goes to an unmapped area takes a significant amount of time (it initiates a tablewalk that may take 40+ cycles). With accurate mapping data we can catch areas of code with bad prefetch instructions.
Agreed.
Quoted 5 lines
Taking it a bit further, since in some sense
perf_counteris a channel for getting events out to userspace, I wonder if we could solve Roland's RDMA address space unmap issue withperf_counter:Yeah, saw some of that, Andrew has some good points there. Haven't had time to see the reply yet.
Not sure its a good match, but if so, very nice.
Quoted 4 lines
Below is a dodgy hack I've been using to prototype tracking of all mmaps and heap/stack extensions. Naturally we'd want a
perf_counterfeature to turn this on and keep instruction profiles more compact. We'd also want munmap events.
Quoted 6 lines
I used to have an munmap hook in there at some point. We could restore that.
Quoted 1 line
Thoughts?
Seems like a good direction, go for it ;-)
Seconded! Please submit the full patch once you are happy with it, with some tools/perf/ variant/feature that makes use of it and we are golden.
Ingo