Skip to contentSkip to conversation

perf tool: Ensure forward progress unwinding frames

PATCH In review David Ahern 2 participants 5 messages Dec 2, 2013 – Dec 9, 2013 perf events

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 perf tool: Ensure forward progress unwinding frames

The dwarf option for callstacks is getting stuck on the unwind and showing repeated frames. For example,

qemu-system-x86 13759 [001] 287663.084957: syscalls:sys_enter_futex: uaddr: 0x7f770a958700, op: 0x00000081, val: 0x00000001, utime: 0x7f75ea

            7f7705fd23ea __lll_unlock_wake (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            ...
            (Last frame repeats 126 times)

Catch that the same frame is hit multiple times and break out of the loop.

With this patch:

qemu-system-x86 13759 [001] 287663.084967: syscalls:sys_exit_futex: 0x1

            7f7705fd23ea __lll_unlock_wake (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
---
 tools/perf/util/unwind.c | 5 +++++
 1 file changed, 5 insertions(+)
tools/perf/util/unwind.c +5 -0
index 0efd5393de85..817ffc1f3a00 100644
@@ -563,6 +563,7 @@ static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb,
 {
 	unw_addr_space_t addr_space;
 	unw_cursor_t c;
+	unw_word_t prev_ip = 0;
 	int ret;
 
 	addr_space = unw_create_addr_space(&accessors, 0);
@@ -579,6 +580,10 @@ static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb,
 		unw_word_t ip;
 
 		unw_get_reg(&c, UNW_REG_IP, &ip);
+		if (ip == prev_ip)
+			break;
+		prev_ip = ip;
+
 		ret = entry(ip, ui->thread, ui->machine, cb, arg);
 	}
 
Signature · 1 line
1.8.3.4 (Apple Git-47)
#1 lore raw

ping.. Jiri, any thoughts on this?

Quoted 51 lines · David Ahern

The dwarf option for callstacks is getting stuck on the unwind and showing repeated frames. For example,

qemu-system-x86 13759 [001] 287663.084957: syscalls:sys_enter_futex: uaddr: 0x7f770a958700, op: 0x00000081, val: 0x00000001, utime: 0x7f75ea

             7f7705fd23ea __lll_unlock_wake (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
             ...
             (Last frame repeats 126 times)

Catch that the same frame is hit multiple times and break out of the loop.

With this patch:

qemu-system-x86 13759 [001] 287663.084967: syscalls:sys_exit_futex: 0x1

             7f7705fd23ea __lll_unlock_wake (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
---
  tools/perf/util/unwind.c | 5 +++++
  1 file changed, 5 insertions(+)
tools/perf/util/unwind.c +5 -0
index 0efd5393de85..817ffc1f3a00 100644
@@ -563,6 +563,7 @@ static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb,
  {
  	unw_addr_space_t addr_space;
  	unw_cursor_t c;
+	unw_word_t prev_ip = 0;
  	int ret;
 
  	addr_space = unw_create_addr_space(&accessors, 0);
@@ -579,6 +580,10 @@ static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb,
  		unw_word_t ip;
 
  		unw_get_reg(&c, UNW_REG_IP, &ip);
+		if (ip == prev_ip)
+			break;
+		prev_ip = ip;
+
  		ret = entry(ip, ui->thread, ui->machine, cb, arg);
  	}
 
#2 lore raw
Quoted 1 line · David Ahern

ping.. Jiri, any thoughts on this?

oops, missed that.. sry

Quoted 13 lines · David Ahern
Quoted 11 lines

The dwarf option for callstacks is getting stuck on the unwind and showing repeated frames. For example,

qemu-system-x86 13759 [001] 287663.084957: syscalls:sys_enter_futex: uaddr: 0x7f770a958700, op: 0x00000081, val: 0x00000001, utime: 0x7f75ea

            7f7705fd23ea __lll_unlock_wake (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            ...

I wonder this is us or libunwind bug (or cfi code) ;-)

looks like you can reproduce..? are you using the latest libunwind?

Quoted 27 lines
Quoted 27 lines

(Last frame repeats 126 times)

Catch that the same frame is hit multiple times and break out of the loop.

With this patch:

qemu-system-x86 13759 [001] 287663.084967: syscalls:sys_exit_futex: 0x1

            7f7705fd23ea __lll_unlock_wake (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
---
 tools/perf/util/unwind.c | 5 +++++
 1 file changed, 5 insertions(+)
tools/perf/util/unwind.c +1 -0
index 0efd5393de85..817ffc1f3a00 100644
@@ -563,6 +563,7 @@ static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb,
 {
 	unw_addr_space_t addr_space;
 	unw_cursor_t c;
+	unw_word_t prev_ip = 0;
 	int ret;
 
Quoted 8 lines

tools/perf/util/unwind.c static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb, · line 585

Quoted 8 lines
tools/perf/util/unwind.c +3 -0
 	addr_space = unw_create_addr_space(&accessors, 0);
@@ -579,6 +580,10 @@ static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb,
 		unw_word_t ip;
 
 		unw_get_reg(&c, UNW_REG_IP, &ip);
+		if (ip == prev_ip)
+			break;
+		prev_ip = ip;

hum, how about recursion?

how about some counter and break after some insane value and maybe display some warning..

qemu-system-x86 13759 [001] 287663.084967: syscalls:sys_exit_futex: 0x1

             7f7705fd23ea __lll_unlock_wake (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
             warning: unwind broken

jirka

#3 lore raw
Quoted 13 lines · Jiri Olsa
Quoted 12 lines · David Ahern
Quoted 11 lines

The dwarf option for callstacks is getting stuck on the unwind and showing repeated frames. For example,

qemu-system-x86 13759 [001] 287663.084957: syscalls:sys_enter_futex: uaddr: 0x7f770a958700, op: 0x00000081, val: 0x00000001, utime: 0x7f75ea

             7f7705fd23ea __lll_unlock_wake (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
             ...
Quoted 3 lines

I wonder this is us or libunwind bug (or cfi code) ;-)

looks like you can reproduce..? are you using the latest libunwind?

$ rpm -q libunwind
libunwind-0.99-2.20110424git1e10c293.fc16.x86_64
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines

(Last frame repeats 126 times)

Catch that the same frame is hit multiple times and break out of the loop.

With this patch:

qemu-system-x86 13759 [001] 287663.084967: syscalls:sys_exit_futex: 0x1

             7f7705fd23ea __lll_unlock_wake (/lib64/libpthread-2.14.90.so)
             7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
---
  tools/perf/util/unwind.c | 5 +++++
  1 file changed, 5 insertions(+)
tools/perf/util/unwind.c +1 -0
index 0efd5393de85..817ffc1f3a00 100644
@@ -563,6 +563,7 @@ static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb,
  {
  	unw_addr_space_t addr_space;
  	unw_cursor_t c;
+	unw_word_t prev_ip = 0;
  	int ret;
 
  	addr_space = unw_create_addr_space(&accessors, 0);
@@ -579,6 +580,10 @@ static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb,
  		unw_word_t ip;
 
Quoted 8 lines
Quoted 4 lines

tools/perf/util/unwind.c static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb, · line 585

Quoted 4 lines
tools/perf/util/unwind.c +3 -0
  		unw_get_reg(&c, UNW_REG_IP, &ip);
+		if (ip == prev_ip)
+			break;
+		prev_ip = ip;

hum, how about recursion?

how about some counter and break after some insane value

I chose > 1 as the insane value. Do you have a suggestion on something else? 5? 10?

Quoted 6 lines

and maybe display some warning..

   qemu-system-x86 13759 [001] 287663.084967: syscalls:sys_exit_futex: 0x1
              7f7705fd23ea __lll_unlock_wake (/lib64/libpthread-2.14.90.so)
              7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
              warning: unwind broken

sure.

David

#4 lore raw
Quoted 14 lines · David Ahern
Quoted 13 lines · Jiri Olsa
Quoted 12 lines · David Ahern
Quoted 11 lines

The dwarf option for callstacks is getting stuck on the unwind and showing repeated frames. For example,

qemu-system-x86 13759 [001] 287663.084957: syscalls:sys_enter_futex: uaddr: 0x7f770a958700, op: 0x00000081, val: 0x00000001, utime: 0x7f75ea

            7f7705fd23ea __lll_unlock_wake (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            7f7705fcf045 _L_unlock_649 (/lib64/libpthread-2.14.90.so)
            ...
Quoted 6 lines
Quoted 3 lines

I wonder this is us or libunwind bug (or cfi code) ;-)

looks like you can reproduce..? are you using the latest libunwind?

$ rpm -q libunwind
libunwind-0.99-2.20110424git1e10c293.fc16.x86_64

[jolsa@krava libunwind]$ rpm -q libunwind
libunwind-1.1-2.fc19.x86_64

;-)