Skip to contentSkip to conversation

trace/uprobes: fix output issue with address randomization

PATCH In review Thomas Richter 4 participants 4 messages Dec 15, 2017 – Dec 19, 2017 Tracing

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 trace/uprobes: fix output issue with address randomization

Commit ad67b74d2469 ("printk: hash addresses printed with %p") changed %p to hash pointers in order to avoid leaking kernel addresses.

This breaks the tool perf probe.

To set a uprobe on a function named inet_pton in libc library, obtain the address of the symbol inet_pton using command nm and then use the following command to set the uprobe:

 # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
	> /sys/kernel/debug/tracing/uprobe_events

However the output shows a randomized address:

 # cat /sys/kernel/debug/tracing/uprobe_events
 p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x000000002d0f8952

The displayed address 0x000000002d0f8952 is incorrect and breaks tools post processing it:

 # linux/tools/perf/perf probe -l
 Failed to find debug information for address 2d0f8952
   probe_libc:inet_pton (on 0x2d0f8952 in /usr/lib64/libc-2.26.so)

Using the %px printk format string fixes this issue for root and shows the correct address allowing the perf probe tool to resolve the address to the symbol:

 # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
	> /sys/kernel/debug/tracing/uprobe_events
 # cat /sys/kernel/debug/tracing/uprobe_events
 p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x0000000000142060
 # linux/tools/perf/perf probe -l
  probe_libc:inet_pton (on __inet_pton@resolv/inet_pton.c
					in /usr/lib64/libc-2.26.so)
---
 kernel/trace/trace_uprobe.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/trace/trace_uprobe.c +1 -1
index 40592e7b3568..268029ae1be6 100644
@@ -608,7 +608,7 @@ static int probes_seq_show(struct seq_file *m, void *v)
 
 	/* Don't print "0x  (null)" when offset is 0 */
 	if (tu->offset) {
-		seq_printf(m, "0x%p", (void *)tu->offset);
+		seq_printf(m, "0x%px", (void *)tu->offset);
 	} else {
 		switch (sizeof(void *)) {
 		case 4:
Signature · 1 line
2.13.4
#1 lore raw

I'm not in fact the maintainer covering this file..

Quoted 47 lines · Thomas Richter

Commit ad67b74d2469 ("printk: hash addresses printed with %p") changed %p to hash pointers in order to avoid leaking kernel addresses.

This breaks the tool perf probe.

To set a uprobe on a function named inet_pton in libc library, obtain the address of the symbol inet_pton using command nm and then use the following command to set the uprobe:

 # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
	> /sys/kernel/debug/tracing/uprobe_events

However the output shows a randomized address:

 # cat /sys/kernel/debug/tracing/uprobe_events
 p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x000000002d0f8952

The displayed address 0x000000002d0f8952 is incorrect and breaks tools post processing it:

 # linux/tools/perf/perf probe -l
 Failed to find debug information for address 2d0f8952
   probe_libc:inet_pton (on 0x2d0f8952 in /usr/lib64/libc-2.26.so)

Using the %px printk format string fixes this issue for root and shows the correct address allowing the perf probe tool to resolve the address to the symbol:

 # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
	> /sys/kernel/debug/tracing/uprobe_events
 # cat /sys/kernel/debug/tracing/uprobe_events
 p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x0000000000142060
 # linux/tools/perf/perf probe -l
  probe_libc:inet_pton (on __inet_pton@resolv/inet_pton.c
					in /usr/lib64/libc-2.26.so)
---
 kernel/trace/trace_uprobe.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/trace/trace_uprobe.c +0 -0
index 40592e7b3568..268029ae1be6 100644

kernel/trace/trace_uprobe.c static int probes_seq_show(struct seq_file *m, void *v) · line 614

Quoted 8 lines
kernel/trace/trace_uprobe.c +1 -1
@@ -608,7 +608,7 @@ static int probes_seq_show(struct seq_file *m, void *v)
 
 	/* Don't print "0x  (null)" when offset is 0 */
 	if (tu->offset) {
-		seq_printf(m, "0x%p", (void *)tu->offset);
+		seq_printf(m, "0x%px", (void *)tu->offset);
 	} else {
 		switch (sizeof(void *)) {
 		case 4:

So I had a wee peek and afaict this ends up at:

        trace_create_file("uprobe_events", 0644, d_tracer,
                                    NULL, &uprobe_events_ops);

which is a world readable file. Doesn't that leak the kaslr offset?

maintainer
#2 lore raw
Quoted 14 lines · On Fri, 15 Dec 2017 18:31:56 +0100 Peter Zijlstra <[email protected]> wrote:

kernel/trace/trace_uprobe.c static int probes_seq_show(struct seq_file *m, void *v) · line 614

Quoted 11 lines
kernel/trace/trace_uprobe.c +1 -1
@@ -608,7 +608,7 @@ static int probes_seq_show(struct seq_file *m, void *v)
 
 	/* Don't print "0x  (null)" when offset is 0 */
 	if (tu->offset) {
-		seq_printf(m, "0x%p", (void *)tu->offset);
+		seq_printf(m, "0x%px", (void *)tu->offset);
 	} else {
 		switch (sizeof(void *)) {
 		case 4:  

So I had a wee peek and afaict this ends up at:

Quoted 4 lines
        trace_create_file("uprobe_events", 0644, d_tracer,
                                    NULL, &uprobe_events_ops);

which is a world readable file. Doesn't that leak the kaslr offset?

Usually the tracefs directory is mounted and is not world readable.

I just checked, and sure enough, after mounting tracefs, I can't read it from non-root:

# mount -t tracefs nodev /sys/kernel/tracing/
# ls /sys/kernel/tracing/
available_events            max_graph_depth      stack_max_size
available_filter_functions  options              stack_trace
available_tracers           per_cpu              stack_trace_filter
buffer_size_kb              printk_formats       trace
buffer_total_size_kb        README               trace_clock
current_tracer              saved_cmdlines       trace_marker
dyn_ftrace_total_info       saved_cmdlines_size  trace_marker_raw
enabled_functions           saved_tgids          trace_options
eval_map                    set_event            trace_pipe
events                      set_event_pid        trace_stat
free_buffer                 set_ftrace_filter    tracing_cpumask
function_profile_enabled    set_ftrace_notrace   tracing_max_latency
hwlat_detector              set_ftrace_pid       tracing_on
instances                   set_graph_function   tracing_thresh
kprobe_events               set_graph_notrace    uprobe_events
kprobe_profile              snapshot             uprobe_profile
# exit
$ ls /sys/kernel/tracing/
ls: cannot open directory '/sys/kernel/tracing/': Permission denied

-- Steve

maintainer
#3 lore raw
In reply to Peter Zijlstra
Quoted 62 lines · On Fri, 15 Dec 2017 18:31:56 +0100 Peter Zijlstra <[email protected]> wrote:

I'm not in fact the maintainer covering this file..

Quoted 47 lines · Thomas Richter

Commit ad67b74d2469 ("printk: hash addresses printed with %p") changed %p to hash pointers in order to avoid leaking kernel addresses.

This breaks the tool perf probe.

To set a uprobe on a function named inet_pton in libc library, obtain the address of the symbol inet_pton using command nm and then use the following command to set the uprobe:

 # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
	> /sys/kernel/debug/tracing/uprobe_events

However the output shows a randomized address:

 # cat /sys/kernel/debug/tracing/uprobe_events
 p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x000000002d0f8952

The displayed address 0x000000002d0f8952 is incorrect and breaks tools post processing it:

 # linux/tools/perf/perf probe -l
 Failed to find debug information for address 2d0f8952
   probe_libc:inet_pton (on 0x2d0f8952 in /usr/lib64/libc-2.26.so)

Using the %px printk format string fixes this issue for root and shows the correct address allowing the perf probe tool to resolve the address to the symbol:

 # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
	> /sys/kernel/debug/tracing/uprobe_events
 # cat /sys/kernel/debug/tracing/uprobe_events
 p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x0000000000142060
 # linux/tools/perf/perf probe -l
  probe_libc:inet_pton (on __inet_pton@resolv/inet_pton.c
					in /usr/lib64/libc-2.26.so)
---
 kernel/trace/trace_uprobe.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/trace/trace_uprobe.c +0 -0
index 40592e7b3568..268029ae1be6 100644

kernel/trace/trace_uprobe.c static int probes_seq_show(struct seq_file *m, void *v) · line 614

Quoted 8 lines
kernel/trace/trace_uprobe.c +1 -1
@@ -608,7 +608,7 @@ static int probes_seq_show(struct seq_file *m, void *v)
 
 	/* Don't print "0x  (null)" when offset is 0 */
 	if (tu->offset) {
-		seq_printf(m, "0x%p", (void *)tu->offset);
+		seq_printf(m, "0x%px", (void *)tu->offset);
 	} else {
 		switch (sizeof(void *)) {
 		case 4:

So I had a wee peek and afaict this ends up at:

Quoted 4 lines
        trace_create_file("uprobe_events", 0644, d_tracer,
                                    NULL, &uprobe_events_ops);

which is a world readable file. Doesn't that leak the kaslr offset?

As I correctly understand, this fix shows the correct offset in the file of the probe point, this means we fixes that leakage of the aslr offset, doesn't it?

Thank you,

Signature · 1 line
Masami Hiramatsu <[email protected]>