uprobes: Fix limiting un-nested return probes
Subsystems (2)
PATCH 1/1 uprobes: Fix limiting un-nested return probes
Here is a sample program which shows a problem in uretprobes:
#include <stdlib.h>
int some_work(int num)
{
while (num != 0)
num--;
return 0;
};int main(int argc, char **argv)
{
if (argc != 2)
return EXIT_FAILURE;
int num = atoi(argv[1]);
while(num != 0) {
some_work(100);
num--;
};
return EXIT_SUCCESS;
}$ gcc -o sample sample.c
- Add probe for returning from some_work():
$ sudo perf probe -x ./sample -a ret=some_work%return
Added new event:
probe_sample:ret (on 0x530%return)
You can now use it in all perf tools, such as:
perf record -e probe_sample:ret -aR sleep 1
- Record events :
$ sudo perf record -e probe_sample:ret -aR ./sample 134
- View report :
$ sudo perf report --stdio
# captured on: Wed Aug 14 17:03:42 2013
# hostname : hemant-fedora
# os release : 3.11.0-rc3+
# perf version : 3.9.4-200.fc18.x86_64
# arch : x86_64
# nrcpus online : 2
# nrcpus avail : 2
# cpudesc : QEMU Virtual CPU version 1.2.2
# cpuid : GenuineIntel,6,2,3
# total memory : 2051912 kB
# cmdline : /usr/bin/perf record -e probe_sample:ret -aR ./sample 134
# event : name = probe_sample:ret, type = 2, config = 0x38c, config1 =
0x0, config2 = 0x0,
# HEADER_CPU_TOPOLOGY info available, use -I to display
# HEADER_NUMA_TOPOLOGY info available, use -I to display
# pmu mappings: software = 1, tracepoint = 2, breakpoint = 5
# ========
#
# Samples: 64 of event 'probe_sample:ret'
# Event count (approx.): 64
#
# Overhead Command Shared Object Symbol
# ........ ....... ............. ........
#
100.00% sample sample [.] main#
# (For a higher level overview, try: perf report --sort comm,dso) #
Quoted 1 line
From report we can see that there were only 64 return events, but
actually it should be 134. It looks like uprobes identified return events as recursive events and used restrictions for number of such events. So, here is a patch which fixes this issue.
--->8---
There exists a limit to the number of nested return probes. The current limit is 64. However this limit is getting enforced on even non nested return probes. Hence, registering 64 independent non nested return probes results in failure of return probes on the same task. The problem is utask->depth is getting incremented unconditionally but decremented only if chained. So, utask->depth should be incremented only if chained. This should fix the issue.
--- kernel/events/uprobes.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/events/uprobes.c +2 -1
@@ -1442,7 +1442,8 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) ri->orig_ret_vaddr = orig_ret_vaddr; ri->chained = chained; - utask->depth++; +if (chained) + utask->depth++;/* add instance to the stack */ ri->next = utask->return_instances;
Oleg Nesterov
maintainerSorry for delay, vacation.
Quoted 6 lines · Hemant Kumar Shaw
There exists a limit to the number of nested return probes. The current limit is 64. However this limit is getting enforced on even non nested return probes. Hence, registering 64 independent non nested return probes results in failure of return probes on the same task. The problem is
utask->depthis getting incremented unconditionally but decremented only if chained.
Hmm. I'll try to recheck later, but at first glance this logic is indeed wrong, thanks.
kernel/events/uprobes.c static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) · line 1446
Quoted 9 lines
kernel/events/uprobes.c +2
-1@@ -1442,7 +1442,8 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) ri->orig_ret_vaddr = orig_ret_vaddr; ri->chained = chained; - utask->depth++; +if (chained) + utask->depth++;
Not sure, but I can be easily wrong... afaics we need something like below, no?
Anton?
Oleg.
kernel/events/uprobes.c +1 -3
@@ -1682,12 +1682,10 @@ static bool handle_trampoline(struct pt_ tmp = ri; ri = ri->next;kfree (tmp); + utask->depth--;if (!chained)break ; - - utask->depth--; -BUG_ON (!ri); }
Quoted 19 lines · Oleg Nesterov
Sorry for delay, vacation.
Quoted 6 lines · Hemant Kumar Shaw
There exists a limit to the number of nested return probes. The current limit is 64. However this limit is getting enforced on even non nested return probes. Hence, registering 64 independent non nested return probes results in failure of return probes on the same task. The problem is
utask->depthis getting incremented unconditionally but decremented only if chained.Hmm. I'll try to recheck later, but at first glance this logic is indeed wrong, thanks.
Quoted 6 lines
kernel/events/uprobes.c +0
-0@@ -1442,7 +1442,8 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) ri->orig_ret_vaddr = orig_ret_vaddr; ri->chained = chained;
Quoted 6 lines
kernel/events/uprobes.c static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) · line 1446
Quoted 3 lines
kernel/events/uprobes.c +2
-1- utask->depth++; +if (chained) + utask->depth++;Not sure, but I can be easily wrong... afaics we need something like below, no? Anton?
Oleg, your guess is correct.
My original intention was to limit by depth the chained only probes. But later, after your review, we've decided /based on safety concerns/ to limit it hard.
The decrement 'utask->depth--;' in my own tree is above the 'if (!chained)' check. I think it got mangled somehow when I rebased the code before I sent it to lkml.
Anton.
Quoted 17 lines
Oleg.
kernel/events/uprobes.c +1
-3@@ -1682,12 +1682,10 @@ static bool handle_trampoline(struct pt_ tmp = ri; ri = ri->next;kfree (tmp); + utask->depth--;if (!chained)break ; - - utask->depth--; -BUG_ON (!ri); }
Oleg Nesterov
maintainerQuoted 10 lines · Anton Arapov
Quoted 3 lines · Oleg Nesterov
Not sure, but I can be easily wrong... afaics we need something like below, no? Anton?
Oleg, your guess is correct.
My original intention was to limit by depth the chained only probes. But later, after your review, we've decided /based on safety concerns/ to limit it hard.
Chained or not, we allocate return_instance every time, so we certainly need to account to limit the depth unconditionally. Unless we reuse the same return_instance if chained, but this is another story.
Quoted 3 lines
The decrement 'utask->depth--;' in my own tree is above the 'if (!chained)' check. I think it got mangled somehow when I rebased the code before I sent it to lkml.
OK, thanks, I'll write the changelog and re-send the patch below.
Quoted 20 lines
Anton.
Quoted 17 lines
Oleg.
kernel/events/uprobes.c +1
-3@@ -1682,12 +1682,10 @@ static bool handle_trampoline(struct pt_ tmp = ri; ri = ri->next;kfree (tmp); + utask->depth--;if (!chained)break ; - - utask->depth--; -BUG_ON (!ri); }
Hi Oleg,
Quoted 11 lines · Oleg Nesterov
Quoted 7 lines · Anton Arapov
Quoted 2 lines · Oleg Nesterov
Not sure, but I can be easily wrong... afaics we need something like below, no? Anton?
Oleg, your guess is correct.
My original intention was to limit by depth the chained only probes. But later, after your review, we've decided /based on safety concerns/ to limit it hard.
Chained or not, we allocate
return_instanceevery time, so we certainly need to account to limit the depth unconditionally. Unless we reuse the samereturn_instanceif chained, but this is another story.
Hmm, agreed. Thanks for the description.
Quoted 26 lines
Quoted 4 lines
The decrement 'utask->depth--;' in my own tree is above the 'if (!chained)' check. I think it got mangled somehow when I rebased the code before I sent it to lkml.
OK, thanks, I'll write the changelog and re-send the patch below.
Quoted 20 lines
Anton.
Quoted 17 lines
Oleg.
kernel/events/uprobes.c +1
-3@@ -1682,12 +1682,10 @@ static bool handle_trampoline(struct pt_ tmp = ri; ri = ri->next;kfree (tmp); + utask->depth--;if (!chained)break ; - - utask->depth--; -BUG_ON (!ri); }
Signature · 2 lines
Thanks Hemant