perf probe: Fix memory leaking on faiulre by clearing all probe_trace_events
PATCH 1/1 perf probe: Fix memory leaking on faiulre by clearing all probe_trace_events
Fix memory leaking on failure path in debuginfo__find_trace_events() which frees an array of probe_trace_events but doesn't clear all allocated sub-structures and strings.
So, before doing zfree(tevs), clear all the array elements which can have allocated resources.
--- tools/perf/util/probe-finder.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
tools/perf/util/probe-finder.c +3 -1
@@ -1246,7 +1246,7 @@ int debuginfo__find_trace_events(struct debuginfo *dbg,struct trace_event_finder tf = { .pf = {.pev = pev, .callback = add_probe_trace_event}, .max_tevs = probe_conf.max_probes, .mod = dbg->mod}; -int ret; +int ret, i;/* Allocate result tevs array */ *tevs =zalloc (sizeof (struct probe_trace_event ) * tf.max_tevs);
@@ -1258,6 +1258,8 @@ int debuginfo__find_trace_events(struct debuginfo *dbg, ret =debuginfo__find_probes (dbg, &tf.pf);if (ret <0 ) { +for (i =0 ; i < tf.ntevs; i++) +clear_probe_trace_event (&tf.tevs[i]);zfree (tevs);return ret; }
Hi Masami,
Today I remember the reason why I introduced patch [1]. Although your patch is correct, either [1] or [2] is still required, but they are both need to be fixed.
Here is a bug:
A segfault raises if use glob matching and argument together and one of probe point failed to get that argument:
# ./perf probe -v -n 'SyS_dup? oldfd' probe-definition(0): SyS_dup? oldfd symbol:SyS_dup? file:(null) line:0 offset:0 return:0 lazy:(null) parsing arg: oldfd into oldfd 1 arguments Looking at the vmlinux_path (7 entries long) Using /lib/modules/4.3.0-rc4+/build/vmlinux for symbols Open Debuginfo file: /lib/modules/4.3.0-rc4+/build/vmlinux Try to find probe point from debuginfo. Matched function: SyS_dup3 found inline addr: 0xffffffff812095c0 Probe point found: SyS_dup3+0 Searching 'oldfd' variable in context. Converting variable oldfd into trace event. oldfd type is long int. found inline addr: 0xffffffff812096d4 Probe point found: SyS_dup2+36 Searching 'oldfd' variable in context. Failed to find 'oldfd' in this function. Matched function: SyS_dup3 Probe point found: SyS_dup3+0 Searching 'oldfd' variable in context. Converting variable oldfd into trace event. oldfd type is long int. Matched function: SyS_dup2 Probe point found: SyS_dup2+0 Searching 'oldfd' variable in context. Converting variable oldfd into trace event. oldfd type is long int. Found 4 probe_trace_events. Opening /sys/kernel/debug/tracing//kprobe_events write=1 Writing event: p:probe/SyS_dup3 _text+2135488 oldfd=%di:s64 Segmentation fault (core dumped)
Here is how the segfault happen:
In following call stack:
add_probe_trace_eventcall_probe_finderprobe_point_search_cb
??dwarf_getfuncsfind_probe_point_by_funcdebuginfo__find_probesdebuginfo__find_trace_eventstry_to_find_probe_trace_eventsconvert_to_probe_trace_eventsconvert_perf_probe_eventsperf_add_probe_events
add_probe_trace_event get called and failed due to failure of find_variable(). In this case tev->args is not freed and tev->nargs are positive (1 in this case). Error from find_variable() will be passed down along the call stack through return values until get to probe_point_search_cb(). The problem is that, probe_point_search_cb doesn't return error if the function name is a glob:
/* Inlined function: search instances */
param->retval = die_walk_instances(sp_die,
probe_point_inline_cb, (void *)pf);
/* This could be a non-existed inline definition */
if (param->retval == -ENOENT && strisglob(pp->function))
param->retval = 0;So the error won't be transfer to debuginfo__find_trace_events() from debuginfo__find_probes(), and your patch won't take effect.
With patch [1], we set argument number to 0 and frees tev->args. With this fix we can save that tev. Here's the final result:
# ./perf probe 'SyS_dup? oldfd' Failed to find 'oldfd' in this function. Added new events: probe:SyS_dup3 (on SyS_dup? with oldfd) probe:SyS_dup3_1 (on SyS_dup? with oldfd) probe:SyS_dup3_2 (on SyS_dup? with oldfd) probe:SyS_dup2 (on SyS_dup? with oldfd)
You can now use it in all perf tools, such as:
perf record -e probe:SyS_dup2 -aR sleep 1
# PAGER=cat ./perf probe -l
probe:SyS_dup2 (on SyS_dup2@linux-hydrogen/fs/file.c with oldfd) probe:SyS_dup3 (on SyS_dup3@linux-hydrogen/fs/file.c with oldfd) probe:SyS_dup3_1 (on SYSC_dup2:12@linux-hydrogen/fs/file.c) probe:SyS_dup3_2 (on SyS_dup3@linux-hydrogen/fs/file.c with oldfd)
Perf can't find oldfd in the context of SyS_dup3_1. With [1]'s fix it still probe at it but doesn't create argument fetcher.
[2]'s fix totally clear this tev. It is incorrect because it forgets to adjust tf->ntevs. With that appended the final result becomes:
# ./perf probe 'SyS_dup? oldfd' Failed to find 'oldfd' in this function. Added new events: probe:SyS_dup3 (on SyS_dup? with oldfd) probe:SyS_dup3_1 (on SyS_dup? with oldfd) probe:SyS_dup2 (on SyS_dup? with oldfd)
You can now use it in all perf tools, such as:
perf record -e probe:SyS_dup2 -aR sleep 1
# PAGER=cat ./perf probe -l
probe:SyS_dup2 (on SyS_dup2@linux-hydrogen/fs/file.c with oldfd) probe:SyS_dup3 (on SyS_dup3@linux-hydrogen/fs/file.c with oldfd) probe:SyS_dup3_1 (on SyS_dup3@linux-hydrogen/fs/file.c with oldfd)
Here only those probe point we can get 'oldfd' is probed.
I suggest [2]'s fix because I think if user provide argument he or she would get interested with it, and we have already noticed user the argument is unabled to be found in one probe point.
I'll provide another patch based on [2]. Please have a look.
Thank you.
[1] http://lkml.kernel.org/g/[email protected]
[2] http://lkml.kernel.org/g/[email protected]
Quoted 38 lines · Masami Hiramatsu
Fix memory leaking on failure path in
debuginfo__find_trace_events()which frees an array ofprobe_trace_eventsbut doesn't clear all allocated sub-structures and strings.
So, before doing zfree(tevs), clear all the array elements which can have allocated resources.Signed-off-by Masami Hiramatsu <[email protected]>Reported-by Wang Nan <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Cc Zefan Li <[email protected]>--- tools/perf/util/probe-finder.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)tools/perf/util/probe-finder.c +3
-1@@ -1246,7 +1246,7 @@ int debuginfo__find_trace_events(struct debuginfo *dbg,struct trace_event_finder tf = { .pf = {.pev = pev, .callback = add_probe_trace_event}, .max_tevs = probe_conf.max_probes, .mod = dbg->mod}; -int ret; +int ret, i;/* Allocate result tevs array */ *tevs =zalloc (sizeof (struct probe_trace_event ) * tf.max_tevs);@@ -1258,6 +1258,8 @@ int debuginfo__find_trace_events(struct debuginfo *dbg, ret =debuginfo__find_probes (dbg, &tf.pf);if (ret <0 ) { +for (i =0 ; i < tf.ntevs; i++) +clear_probe_trace_event (&tf.tevs[i]);zfree (tevs);return ret; }
From: Wangnan (F) [mailto:[email protected]]
Quoted 131 lines
Hi Masami,
Today I remember the reason why I introduced patch [1]. Although your patch is correct,
either [1] or [2] is still required, but they are both need to be fixed.Here is a bug:
A segfault raises if use glob matching and argument together and one of probe point
failed to get that argument:# ./perf probe -v -n 'SyS_dup? oldfd' probe-definition(0): SyS_dup? oldfd symbol:SyS_dup? file:(null) line:0 offset:0 return:0 lazy:(null) parsing arg: oldfd into oldfd 1 arguments Looking at the vmlinux_path (7 entries long) Using /lib/modules/4.3.0-rc4+/build/vmlinux for symbols Open Debuginfo file: /lib/modules/4.3.0-rc4+/build/vmlinux Try to find probe point from debuginfo. Matched function: SyS_dup3 found inline addr: 0xffffffff812095c0 Probe point found: SyS_dup3+0 Searching 'oldfd' variable in context. Converting variable oldfd into trace event. oldfd type is long int. found inline addr: 0xffffffff812096d4 Probe point found: SyS_dup2+36 Searching 'oldfd' variable in context. Failed to find 'oldfd' in this function. Matched function: SyS_dup3 Probe point found: SyS_dup3+0 Searching 'oldfd' variable in context. Converting variable oldfd into trace event. oldfd type is long int. Matched function: SyS_dup2 Probe point found: SyS_dup2+0 Searching 'oldfd' variable in context. Converting variable oldfd into trace event. oldfd type is long int. Found 4 probe_trace_events. Opening /sys/kernel/debug/tracing//kprobe_events write=1 Writing event: p:probe/SyS_dup3 _text+2135488 oldfd=%di:s64 Segmentation fault (core dumped)Here is how the segfault happen:
In following call stack:
add_probe_trace_eventcall_probe_finderprobe_point_search_cb
??dwarf_getfuncsfind_probe_point_by_funcdebuginfo__find_probesdebuginfo__find_trace_eventstry_to_find_probe_trace_eventsconvert_to_probe_trace_eventsconvert_perf_probe_eventsperf_add_probe_events
add_probe_trace_eventget called and failed due to failure offind_variable(). In this casetev->argsis not freed andtev->nargsare positive (1 in this case). Error fromfind_variable()
will be passed down along the call stack through return values until get toprobe_point_search_cb(). The problem is that,probe_point_search_cbdoesn't return error
if the function name is a glob:/* Inlined function: search instances */ param->retval = die_walk_instances(sp_die, probe_point_inline_cb, (void *)pf); /* This could be a non-existed inline definition */ if (param->retval == -ENOENT && strisglob(pp->function)) param->retval = 0;So the error won't be transfer to
debuginfo__find_trace_events()fromdebuginfo__find_probes(),
and your patch won't take effect.With patch [1], we set argument number to 0 and frees
tev->args. With this fix we can save that
tev. Here's the final result:# ./perf probe 'SyS_dup? oldfd' Failed to find 'oldfd' in this function. Added new events: probe:SyS_dup3 (on SyS_dup? with oldfd) probe:SyS_dup3_1 (on SyS_dup? with oldfd) probe:SyS_dup3_2 (on SyS_dup? with oldfd) probe:SyS_dup2 (on SyS_dup? with oldfd)You can now use it in all perf tools, such as:
perf record -e probe:SyS_dup2 -aR sleep 1
# PAGER=cat ./perf probe -l
probe:SyS_dup2 (on SyS_dup2@linux-hydrogen/fs/file.c with oldfd) probe:SyS_dup3 (on SyS_dup3@linux-hydrogen/fs/file.c with oldfd) probe:SyS_dup3_1 (on SYSC_dup2:12@linux-hydrogen/fs/file.c) probe:SyS_dup3_2 (on SyS_dup3@linux-hydrogen/fs/file.c with oldfd)Perf can't find oldfd in the context of
SyS_dup3_1. With [1]'s fix it still probe at it but doesn't create argument fetcher.[2]'s fix totally clear this tev. It is incorrect because it forgets to adjust
tf->ntevs. With that appended the final result becomes:# ./perf probe 'SyS_dup? oldfd' Failed to find 'oldfd' in this function. Added new events: probe:SyS_dup3 (on SyS_dup? with oldfd) probe:SyS_dup3_1 (on SyS_dup? with oldfd) probe:SyS_dup2 (on SyS_dup? with oldfd)You can now use it in all perf tools, such as:
perf record -e probe:SyS_dup2 -aR sleep 1
# PAGER=cat ./perf probe -l
probe:SyS_dup2 (on SyS_dup2@linux-hydrogen/fs/file.c with oldfd) probe:SyS_dup3 (on SyS_dup3@linux-hydrogen/fs/file.c with oldfd) probe:SyS_dup3_1 (on SyS_dup3@linux-hydrogen/fs/file.c with oldfd)
Quoted 6 lines
Here only those probe point we can get 'oldfd' is probed.
I suggest [2]'s fix because I think if user provide argument he or she would get interested with it, and we have already noticed user the argument is unabled
to be found in one probe point.
OK, this may allow user to probe only where the variables can be accessed. I also think we'd better offer another option to fail all probe if one of them cannot fulfill the required arguments so that they can decide not to trace it or change probe point.
And at least this error message
Quoted 1 line
Failed to find 'oldfd' in this function.
should be fixed ...
Thanks!
Quoted 43 lines
I'll provide another patch based on [2]. Please have a look.
Thank you.
[1]
http://lkml.kernel.org/g/[email protected]
[2]
http://lkml.kernel.org/g/[email protected]Quoted 30 lines · Masami Hiramatsu
Fix memory leaking on failure path in
debuginfo__find_trace_events()which frees an array ofprobe_trace_eventsbut doesn't clear all allocated sub-structures and strings.
So, before doing zfree(tevs), clear all the array elements which can have allocated resources.Signed-off-by Masami Hiramatsu <[email protected]>Reported-by Wang Nan <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Cc Zefan Li <[email protected]>--- tools/perf/util/probe-finder.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)tools/perf/util/probe-finder.c +1
-1@@ -1246,7 +1246,7 @@ int debuginfo__find_trace_events(struct debuginfo *dbg,struct trace_event_finder tf = { .pf = {.pev = pev, .callback = add_probe_trace_event}, .max_tevs = probe_conf.max_probes, .mod = dbg->mod}; -int ret; +int ret, i;/* Allocate result tevs array */ *tevs =zalloc (sizeof (struct probe_trace_event ) * tf.max_tevs);@@ -1258,6 +1258,8 @@ int debuginfo__find_trace_events(struct debuginfo *dbg,
Quoted 9 lines
tools/perf/util/probe-finder.c int debuginfo__find_trace_events(struct debuginfo *dbg, · line 1265
Quoted 8 lines
tools/perf/util/probe-finder.c +2
-0ret =debuginfo__find_probes (dbg, &tf.pf);if (ret <0 ) { +for (i =0 ; i < tf.ntevs; i++) +clear_probe_trace_event (&tf.tevs[i]);zfree (tevs);return ret; }
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�