Skip to contentSkip to conversation

uprobes: Fix limiting un-nested return probes

PATCH In review Hemant Kumar Shaw 3 participants 5 messages Sep 3, 2013 – Sep 12, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 uprobes: Fix limiting un-nested return probes

Here is a sample program which shows a problem in uretprobes:
#include <stdlib.h>

int some_work(int num)

{
        while (num != 0)
                num--;
        return 0;
};

int main(int argc, char **argv)

{
        if (argc != 2)
                return EXIT_FAILURE;

        int num = atoi(argv[1]);
        while(num != 0) {
                some_work(100);
                num--;
        };
        return EXIT_SUCCESS;
}
$ gcc -o sample sample.c

- Add probe for returning from some_work():

$ sudo perf probe -x ./sample -a ret=some_work%return

Added new event:

     probe_sample:ret     (on 0x530%return)

You can now use it in all perf tools, such as:

perf record -e probe_sample:ret -aR sleep 1

- Record events :

$ sudo perf record -e probe_sample:ret -aR ./sample 134

- View report :

$ sudo perf report --stdio
# captured on: Wed Aug 14 17:03:42 2013
# hostname : hemant-fedora
# os release : 3.11.0-rc3+
# perf version : 3.9.4-200.fc18.x86_64
# arch : x86_64
# nrcpus online : 2
# nrcpus avail : 2
# cpudesc : QEMU Virtual CPU version 1.2.2
# cpuid : GenuineIntel,6,2,3
# total memory : 2051912 kB
# cmdline : /usr/bin/perf record -e probe_sample:ret -aR ./sample 134
# event : name = probe_sample:ret, type = 2, config = 0x38c, config1 =
0x0, config2 = 0x0,
# HEADER_CPU_TOPOLOGY info available, use -I to display
# HEADER_NUMA_TOPOLOGY info available, use -I to display
# pmu mappings: software = 1, tracepoint = 2, breakpoint = 5
# ========
#
# Samples: 64  of event 'probe_sample:ret'
# Event count (approx.): 64
#
# Overhead  Command  Shared Object    Symbol
# ........  .......  .............  ........
#
      100.00%   sample  sample         [.] main

#
# (For a higher level overview, try: perf report --sort comm,dso) #

Quoted 1 line

From report we can see that there were only 64 return events, but

actually it should be 134. It looks like uprobes identified return events as recursive events and used restrictions for number of such events. So, here is a patch which fixes this issue.

--->8---

There exists a limit to the number of nested return probes. The current limit is 64. However this limit is getting enforced on even non nested return probes. Hence, registering 64 independent non nested return probes results in failure of return probes on the same task. The problem is utask->depth is getting incremented unconditionally but decremented only if chained. So, utask->depth should be incremented only if chained. This should fix the issue.

---
 kernel/events/uprobes.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/events/uprobes.c +2 -1
index f356974..4fb20fe 100644
@@ -1442,7 +1442,8 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs)
 	ri->orig_ret_vaddr = orig_ret_vaddr;
 	ri->chained = chained;
 
-	utask->depth++;
+	if (chained)
+		utask->depth++;
 
 	/* add instance to the stack */
 	ri->next = utask->return_instances;
maintainer
#1 lore raw

Sorry for delay, vacation.

Quoted 6 lines · Hemant Kumar Shaw

There exists a limit to the number of nested return probes. The current limit is 64. However this limit is getting enforced on even non nested return probes. Hence, registering 64 independent non nested return probes results in failure of return probes on the same task. The problem is utask->depth is getting incremented unconditionally but decremented only if chained.

Hmm. I'll try to recheck later, but at first glance this logic is indeed wrong, thanks.

kernel/events/uprobes.c static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) · line 1446

Quoted 9 lines
kernel/events/uprobes.c +2 -1
@@ -1442,7 +1442,8 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs)
 	ri->orig_ret_vaddr = orig_ret_vaddr;
 	ri->chained = chained;
 
-	utask->depth++;
+	if (chained)
+		utask->depth++;

Not sure, but I can be easily wrong... afaics we need something like below, no?

Anton?

Oleg.

kernel/events/uprobes.c +1 -3
@@ -1682,12 +1682,10 @@ static bool handle_trampoline(struct pt_
 		tmp = ri;
 		ri = ri->next;
 		kfree(tmp);
+		utask->depth--;
 
 		if (!chained)
 			break;
-
-		utask->depth--;
-
 		BUG_ON(!ri);
 	}
 
#2 lore raw
Quoted 19 lines · Oleg Nesterov

Sorry for delay, vacation.

Quoted 6 lines · Hemant Kumar Shaw

There exists a limit to the number of nested return probes. The current limit is 64. However this limit is getting enforced on even non nested return probes. Hence, registering 64 independent non nested return probes results in failure of return probes on the same task. The problem is utask->depth is getting incremented unconditionally but decremented only if chained.

Hmm. I'll try to recheck later, but at first glance this logic is indeed wrong, thanks.

Quoted 6 lines
kernel/events/uprobes.c +0 -0
@@ -1442,7 +1442,8 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs)
 	ri->orig_ret_vaddr = orig_ret_vaddr;
 	ri->chained = chained;
 
Quoted 6 lines

kernel/events/uprobes.c static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) · line 1446

Quoted 3 lines
kernel/events/uprobes.c +2 -1
-	utask->depth++;
+	if (chained)
+		utask->depth++;

Not sure, but I can be easily wrong... afaics we need something like below, no? Anton?

Oleg, your guess is correct.

My original intention was to limit by depth the chained only probes. But later, after your review, we've decided /based on safety concerns/ to limit it hard.

The decrement 'utask->depth--;' in my own tree is above the 'if (!chained)' check. I think it got mangled somehow when I rebased the code before I sent it to lkml.

Anton.

Quoted 17 lines

Oleg.

kernel/events/uprobes.c +1 -3
@@ -1682,12 +1682,10 @@ static bool handle_trampoline(struct pt_
 		tmp = ri;
 		ri = ri->next;
 		kfree(tmp);
+		utask->depth--;
 
 		if (!chained)
 			break;
-
-		utask->depth--;
-
 		BUG_ON(!ri);
 	}
maintainer
#3 lore raw
Quoted 10 lines · Anton Arapov
Quoted 3 lines · Oleg Nesterov

Not sure, but I can be easily wrong... afaics we need something like below, no? Anton?

Oleg, your guess is correct.

My original intention was to limit by depth the chained only probes. But later, after your review, we've decided /based on safety concerns/ to limit it hard.

Chained or not, we allocate return_instance every time, so we certainly need to account to limit the depth unconditionally. Unless we reuse the same return_instance if chained, but this is another story.

Quoted 3 lines

The decrement 'utask->depth--;' in my own tree is above the 'if (!chained)' check. I think it got mangled somehow when I rebased the code before I sent it to lkml.

OK, thanks, I'll write the changelog and re-send the patch below.

Quoted 20 lines

Anton.

Quoted 17 lines

Oleg.

kernel/events/uprobes.c +1 -3
@@ -1682,12 +1682,10 @@ static bool handle_trampoline(struct pt_
 		tmp = ri;
 		ri = ri->next;
 		kfree(tmp);
+		utask->depth--;
 
 		if (!chained)
 			break;
-
-		utask->depth--;
-
 		BUG_ON(!ri);
 	}
author
#4 lore raw

Hi Oleg,

Quoted 11 lines · Oleg Nesterov
Quoted 7 lines · Anton Arapov
Quoted 2 lines · Oleg Nesterov

Not sure, but I can be easily wrong... afaics we need something like below, no? Anton?

Oleg, your guess is correct.

My original intention was to limit by depth the chained only probes. But later, after your review, we've decided /based on safety concerns/ to limit it hard.

Chained or not, we allocate return_instance every time, so we certainly need to account to limit the depth unconditionally. Unless we reuse the same return_instance if chained, but this is another story.

Hmm, agreed. Thanks for the description.

Quoted 26 lines
Quoted 4 lines

The decrement 'utask->depth--;' in my own tree is above the 'if (!chained)' check. I think it got mangled somehow when I rebased the code before I sent it to lkml.

OK, thanks, I'll write the changelog and re-send the patch below.

Quoted 20 lines

Anton.

Quoted 17 lines

Oleg.

kernel/events/uprobes.c +1 -3
@@ -1682,12 +1682,10 @@ static bool handle_trampoline(struct pt_
  		tmp = ri;
  		ri = ri->next;
  		kfree(tmp);
+		utask->depth--;
 
  		if (!chained)
  			break;
-
-		utask->depth--;
-
  		BUG_ON(!ri);
  	}
Signature · 2 lines
Thanks
Hemant