Skip to contentSkip to conversation

perf_counter: Track all mmaps, heap and stack extensions

PATCH Anton Blanchard 3 participants 3 messages Jul 28, 2009 – Aug 2, 2009

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

Hi,

Right now perf_counter only logs executable mmaps. While this is enough for instruction profiling, at some stage we are also going to want to do data profiling. This will require us to log non-executable mmaps as well as stack and heap extensions.

Why would we care about heap and stack? A few examples:

1. We can monitor TLB miss rates to suggest what regions of memory should be put into hugepages.

2. We can look into various TLB miss issues. On PowerPC a data prefetch that goes to an unmapped area takes a significant amount of time (it initiates a tablewalk that may take 40+ cycles). With accurate mapping data we can catch areas of code with bad prefetch instructions.

Taking it a bit further, since in some sense perf_counter is a channel for getting events out to userspace, I wonder if we could solve Roland's RDMA address space unmap issue with perf_counter:

http://patchwork.kernel.org/patch/37267/

Below is a dodgy hack I've been using to prototype tracking of all mmaps and heap/stack extensions. Naturally we'd want a perf_counter feature to turn this on and keep instruction profiles more compact. We'd also want munmap events.

The only tricky part is having to set start_stack before we call expand_stack for the first time, so we can easily identify it in perf_counter_mmap_event.

Thoughts?

Anton
---

mm/mmap.c +2 -0
Index: linux.trees.git/mm/mmap.c
===================================================================
@@ -1680,6 +1680,7 @@ static int expand_downwards(struct vm_ar
 		if (!error) {
 			vma->vm_start = address;
 			vma->vm_pgoff -= grow;
+			perf_counter_mmap(vma);
 		}
 	}
 	anon_vma_unlock(vma);
@@ -2071,6 +2072,7 @@ unsigned long do_brk(unsigned long addr,
 	vma->vm_page_prot = vm_get_page_prot(flags);
 	vma_link(mm, vma, prev, rb_link, rb_parent);
 out:
+	perf_counter_mmap(vma);
 	mm->total_vm += len >> PAGE_SHIFT;
 	if (flags & VM_LOCKED) {
 		if (!mlock_vma_pages_range(vma, addr, addr + len))
include/linux/perf_counter.h +1 -2
Index: linux.trees.git/include/linux/perf_counter.h
===================================================================
@@ -702,8 +702,7 @@ extern void __perf_counter_mmap(struct v
 
 static inline void perf_counter_mmap(struct vm_area_struct *vma)
 {
-	if (vma->vm_flags & VM_EXEC)
-		__perf_counter_mmap(vma);
+	__perf_counter_mmap(vma);
 }
 
 extern void perf_counter_comm(struct task_struct *tsk);
kernel/perf_counter.c +8 -0
Index: linux.trees.git/kernel/perf_counter.c
===================================================================
@@ -3142,6 +3142,14 @@ static void perf_counter_mmap_event(stru
 		if (!vma->vm_mm) {
 			name = strncpy(tmp, "[vdso]", sizeof(tmp));
 			goto got_name;
+		} else if (vma->vm_start <= vma->vm_mm->start_brk &&
+				vma->vm_end >= vma->vm_mm->brk) {
+			name = strncpy(tmp, "[heap]", sizeof(tmp));
+			goto got_name;
+		} else if (vma->vm_start <= vma->vm_mm->start_stack &&
+				vma->vm_end >= vma->vm_mm->start_stack) {
+			name = strncpy(tmp, "[stack]", sizeof(tmp));
+			goto got_name;
 		}
 
 		name = strncpy(tmp, "//anon", sizeof(tmp));
fs/exec.c +1 -0
Index: linux.trees.git/fs/exec.c
===================================================================
@@ -631,6 +631,7 @@ int setup_arg_pages(struct linux_binprm 
 #else
 	stack_base = vma->vm_start - EXTRA_STACK_VM_PAGES * PAGE_SIZE;
 #endif
+	current->mm->start_stack = bprm->p;
 	ret = expand_stack(vma, stack_base);
 	if (ret)
 		ret = -EFAULT;
#1 lore raw
Quoted 9 lines · Anton Blanchard

Hi,

Right now perf_counter only logs executable mmaps. While this is enough for instruction profiling, at some stage we are also going to want to do data profiling. This will require us to log non-executable mmaps as well as stack and heap extensions.

Why would we care about heap and stack? A few examples:

Quoted 7 lines

1. We can monitor TLB miss rates to suggest what regions of memory should be put into hugepages.

2. We can look into various TLB miss issues. On PowerPC a data prefetch that goes to an unmapped area takes a significant amount of time (it initiates a tablewalk that may take 40+ cycles). With accurate mapping data we can catch areas of code with bad prefetch instructions.

Agreed.

Quoted 5 lines

Taking it a bit further, since in some sense perf_counter is a channel for getting events out to userspace, I wonder if we could solve Roland's RDMA address space unmap issue with perf_counter:

http://patchwork.kernel.org/patch/37267/

Yeah, saw some of that, Andrew has some good points there. Haven't had time to see the reply yet.

Not sure its a good match, but if so, very nice.

Quoted 4 lines

Below is a dodgy hack I've been using to prototype tracking of all mmaps and heap/stack extensions. Naturally we'd want a perf_counter feature to turn this on and keep instruction profiles more compact. We'd also want munmap events.

I used to have an munmap hook in there at some point. We could restore that.

Quoted 1 line

Thoughts?

Seems like a good direction, go for it ;-)

#2 lore raw
Quoted 36 lines · Peter Zijlstra
Quoted 9 lines · Anton Blanchard

Hi,

Right now perf_counter only logs executable mmaps. While this is enough for instruction profiling, at some stage we are also going to want to do data profiling. This will require us to log non-executable mmaps as well as stack and heap extensions.

Why would we care about heap and stack? A few examples:

Quoted 7 lines

1. We can monitor TLB miss rates to suggest what regions of memory should be put into hugepages.

2. We can look into various TLB miss issues. On PowerPC a data prefetch that goes to an unmapped area takes a significant amount of time (it initiates a tablewalk that may take 40+ cycles). With accurate mapping data we can catch areas of code with bad prefetch instructions.

Agreed.

Quoted 5 lines

Taking it a bit further, since in some sense perf_counter is a channel for getting events out to userspace, I wonder if we could solve Roland's RDMA address space unmap issue with perf_counter:

http://patchwork.kernel.org/patch/37267/

Yeah, saw some of that, Andrew has some good points there. Haven't had time to see the reply yet.

Not sure its a good match, but if so, very nice.

Quoted 4 lines

Below is a dodgy hack I've been using to prototype tracking of all mmaps and heap/stack extensions. Naturally we'd want a perf_counter feature to turn this on and keep instruction profiles more compact. We'd also want munmap events.

Quoted 6 lines

I used to have an munmap hook in there at some point. We could restore that.

Quoted 1 line

Thoughts?

Seems like a good direction, go for it ;-)

Seconded! Please submit the full patch once you are happy with it, with some tools/perf/ variant/feature that makes use of it and we are golden.

Ingo