Skip to contentSkip to conversation

riscv: Add perf callchain support

PATCH Superseded Mao Han 2 participants 9 messages May 17, 2019 – Aug 24, 2019

Subsystems (2)

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
author
#2 lore raw

PATCH v3 2/3 riscv: Add support for perf registers sampling

In reply to Mao Han

This patch implements the perf registers sampling and validation API for riscv arch. The valid registers and their register ID are defined in perf_regs.h. Perf tool can backtrace in userspace with unwind library and the registers/user stack dump support.

CC Palmer Dabbelt <[email protected]>
CC linux-riscv <[email protected]>
CC Christoph Hellwig <[email protected]>
CC Guo Ren <[email protected]>
---
 arch/riscv/Kconfig                      |  2 ++
 arch/riscv/include/uapi/asm/perf_regs.h | 42 +++++++++++++++++++++++++++++++
 arch/riscv/kernel/Makefile              |  1 +
 arch/riscv/kernel/perf_regs.c           | 44 +++++++++++++++++++++++++++++++++
 4 files changed, 89 insertions(+)
 create mode 100644 arch/riscv/include/uapi/asm/perf_regs.h
 create mode 100644 arch/riscv/kernel/perf_regs.c
arch/riscv/Kconfig +2 -0
index eb56c82..effd157 100644
@@ -33,6 +33,8 @@ config RISCV
 	select HAVE_DMA_CONTIGUOUS
 	select HAVE_FUTEX_CMPXCHG if FUTEX
 	select HAVE_PERF_EVENTS
+	select HAVE_PERF_REGS
+	select HAVE_PERF_USER_STACK_DUMP
 	select HAVE_SYSCALL_TRACEPOINTS
 	select IRQ_DOMAIN
 	select RISCV_ISA_A if SMP
arch/riscv/include/uapi/asm/perf_regs.h new file +42 -0

42 lines not on this page · raw message

arch/riscv/kernel/Makefile +1 -0
index dd2ba44..024badc 100644
@@ -39,5 +39,6 @@ obj-$(CONFIG_DYNAMIC_FTRACE)	+= mcount-dyn.o
 
 obj-$(CONFIG_PERF_EVENTS)	+= perf_event.o
 obj-$(CONFIG_PERF_EVENTS)	+= perf_callchain.o
+obj-$(CONFIG_HAVE_PERF_REGS)	+= perf_regs.o
 
 clean:
arch/riscv/kernel/perf_regs.c new file +44 -0

44 lines not on this page · raw message

Signature · 1 line
2.7.4
author
#1 lore raw

PATCH v3 1/3 riscv: Add perf callchain support

In reply to Mao Han

This patch add support for perf callchain sampling on riscv platform. The return address of leaf function is retrieved from pt_regs as it is not saved in the outmost frame.

CC Palmer Dabbelt <[email protected]>
CC linux-riscv <[email protected]>
CC Christoph Hellwig <[email protected]>
CC Guo Ren <[email protected]>
---
 arch/riscv/Makefile                |   3 +
 arch/riscv/kernel/Makefile         |   3 +-
 arch/riscv/kernel/perf_callchain.c | 113 +++++++++++++++++++++++++++++++++++++
 3 files changed, 118 insertions(+), 1 deletion(-)
 create mode 100644 arch/riscv/kernel/perf_callchain.c
arch/riscv/Makefile +3 -0
index c6342e6..4396abc 100644
@@ -59,6 +59,9 @@ endif
 ifeq ($(CONFIG_MODULE_SECTIONS),y)
 	KBUILD_LDFLAGS_MODULE += -T $(srctree)/arch/riscv/kernel/module.lds
 endif
+ifeq ($(CONFIG_PERF_EVENTS),y)
+        KBUILD_CFLAGS += -fno-omit-frame-pointer
+endif
 
 KBUILD_CFLAGS_MODULE += $(call cc-option,-mno-relax)
 
arch/riscv/kernel/Makefile +2 -1
index 5985681..dd2ba44 100644
@@ -37,6 +37,7 @@ obj-$(CONFIG_MODULE_SECTIONS)	+= module-sections.o
 obj-$(CONFIG_FUNCTION_TRACER)	+= mcount.o ftrace.o
 obj-$(CONFIG_DYNAMIC_FTRACE)	+= mcount-dyn.o
 
-obj-$(CONFIG_PERF_EVENTS)      += perf_event.o
+obj-$(CONFIG_PERF_EVENTS)	+= perf_event.o
+obj-$(CONFIG_PERF_EVENTS)	+= perf_callchain.o
 
 clean:
arch/riscv/kernel/perf_callchain.c new file +113 -0

113 lines not on this page · raw message

Signature · 1 line
2.7.4

Mao Han

author
#0 lore raw

v3 cover riscv: Add perf callchain support

This patch set add perf callchain(FP/DWARF) support for RISC-V. It comes from the csky version callchain support with some slight modifications. The patchset base on Linux 5.1.

CC Palmer Dabbelt <[email protected]>
CC linux-riscv <[email protected]>
CC Christoph Hellwig <[email protected]>
CC Guo Ren <[email protected]>

Changes since v2:
- fix inconsistent comment
- force to build kernel with -fno-omit-frame-pointer if perf event is enabled

Changes since v1:
- simplify implementation and code convention

Mao Han (3):
riscv: Add perf callchain support
riscv: Add support for perf registers sampling
riscv: Add support for libdw

 arch/riscv/Kconfig                            |   2 +
 arch/riscv/Makefile                           |   3 +
 arch/riscv/include/uapi/asm/perf_regs.h       |  42 ++++++++++
 arch/riscv/kernel/Makefile                    |   4 +-
 arch/riscv/kernel/perf_callchain.c            | 113 ++++++++++++++++++++++++++
 arch/riscv/kernel/perf_regs.c                 |  44 ++++++++++
 tools/arch/riscv/include/uapi/asm/perf_regs.h |  42 ++++++++++
 tools/perf/Makefile.config                    |   6 +-
 tools/perf/arch/riscv/Build                   |   1 +
 tools/perf/arch/riscv/Makefile                |   3 +
 tools/perf/arch/riscv/include/perf_regs.h     |  96 ++++++++++++++++++++++
 tools/perf/arch/riscv/util/Build              |   2 +
 tools/perf/arch/riscv/util/dwarf-regs.c       |  72 ++++++++++++++++
 tools/perf/arch/riscv/util/unwind-libdw.c     |  57 +++++++++++++
 14 files changed, 485 insertions(+), 2 deletions(-)
 create mode 100644 arch/riscv/include/uapi/asm/perf_regs.h
 create mode 100644 arch/riscv/kernel/perf_callchain.c
 create mode 100644 arch/riscv/kernel/perf_regs.c
 create mode 100644 tools/arch/riscv/include/uapi/asm/perf_regs.h
 create mode 100644 tools/perf/arch/riscv/Build
 create mode 100644 tools/perf/arch/riscv/Makefile
 create mode 100644 tools/perf/arch/riscv/include/perf_regs.h
 create mode 100644 tools/perf/arch/riscv/util/Build
 create mode 100644 tools/perf/arch/riscv/util/dwarf-regs.c
 create mode 100644 tools/perf/arch/riscv/util/unwind-libdw.c
Signature · 1 line
2.7.4
author
#3 lore raw

PATCH v3 3/3 riscv: Add support for libdw

This patch add support for DWARF register mappings and libdw registers initialization, which is used by perf callchain analyzing when --call-graph=dwarf is given.

CC Palmer Dabbelt <[email protected]>
CC linux-riscv <[email protected]>
CC Christoph Hellwig <[email protected]>
CC Guo Ren <[email protected]>
---
 tools/arch/riscv/include/uapi/asm/perf_regs.h | 42 ++++++++++++
 tools/perf/Makefile.config                    |  6 +-
 tools/perf/arch/riscv/Build                   |  1 +
 tools/perf/arch/riscv/Makefile                |  3 +
 tools/perf/arch/riscv/include/perf_regs.h     | 96 +++++++++++++++++++++++++++
 tools/perf/arch/riscv/util/Build              |  2 +
 tools/perf/arch/riscv/util/dwarf-regs.c       | 72 ++++++++++++++++++++
 tools/perf/arch/riscv/util/unwind-libdw.c     | 57 ++++++++++++++++
 8 files changed, 278 insertions(+), 1 deletion(-)
 create mode 100644 tools/arch/riscv/include/uapi/asm/perf_regs.h
 create mode 100644 tools/perf/arch/riscv/Build
 create mode 100644 tools/perf/arch/riscv/Makefile
 create mode 100644 tools/perf/arch/riscv/include/perf_regs.h
 create mode 100644 tools/perf/arch/riscv/util/Build
 create mode 100644 tools/perf/arch/riscv/util/dwarf-regs.c
 create mode 100644 tools/perf/arch/riscv/util/unwind-libdw.c
tools/arch/riscv/include/uapi/asm/perf_regs.h new file +42 -0
new file mode 100644
index 0000000..df1a581
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2019 Hangzhou C-SKY Microsystems co.,ltd. */
+
+#ifndef _ASM_RISCV_PERF_REGS_H
+#define _ASM_RISCV_PERF_REGS_H
+
+enum perf_event_riscv_regs {
+	PERF_REG_RISCV_PC,
+	PERF_REG_RISCV_RA,
+	PERF_REG_RISCV_SP,
+	PERF_REG_RISCV_GP,
+	PERF_REG_RISCV_TP,
+	PERF_REG_RISCV_T0,
+	PERF_REG_RISCV_T1,
+	PERF_REG_RISCV_T2,
+	PERF_REG_RISCV_S0,
+	PERF_REG_RISCV_S1,
+	PERF_REG_RISCV_A0,
+	PERF_REG_RISCV_A1,
+	PERF_REG_RISCV_A2,
+	PERF_REG_RISCV_A3,
+	PERF_REG_RISCV_A4,
+	PERF_REG_RISCV_A5,
+	PERF_REG_RISCV_A6,
+	PERF_REG_RISCV_A7,
+	PERF_REG_RISCV_S2,
+	PERF_REG_RISCV_S3,
+	PERF_REG_RISCV_S4,
+	PERF_REG_RISCV_S5,
+	PERF_REG_RISCV_S6,
+	PERF_REG_RISCV_S7,
+	PERF_REG_RISCV_S8,
+	PERF_REG_RISCV_S9,
+	PERF_REG_RISCV_S10,
+	PERF_REG_RISCV_S11,
+	PERF_REG_RISCV_T3,
+	PERF_REG_RISCV_T4,
+	PERF_REG_RISCV_T5,
+	PERF_REG_RISCV_T6,
+	PERF_REG_RISCV_MAX,
+};
+#endif /* _ASM_RISCV_PERF_REGS_H */
tools/perf/Makefile.config +5 -1
index fe3f97e..8f2e6d3 100644
@@ -59,6 +59,10 @@ ifeq ($(SRCARCH),arm64)
   LIBUNWIND_LIBS = -lunwind -lunwind-aarch64
 endif
 
+ifeq ($(SRCARCH),riscv)
+  NO_PERF_REGS := 0
+endif
+
 ifeq ($(ARCH),s390)
   NO_PERF_REGS := 0
   NO_SYSCALL_TABLE := 0
@@ -77,7 +81,7 @@ endif
 # Disable it on all other architectures in case libdw unwind
 # support is detected in system. Add supported architectures
 # to the check.
-ifneq ($(SRCARCH),$(filter $(SRCARCH),x86 arm arm64 powerpc s390))
+ifneq ($(SRCARCH),$(filter $(SRCARCH),x86 arm arm64 powerpc s390 riscv))
   NO_LIBDW_DWARF_UNWIND := 1
 endif
 
tools/perf/arch/riscv/Build new file +1 -0
new file mode 100644
index 0000000..e4e5f33
@@ -0,0 +1 @@
+perf-y += util/
tools/perf/arch/riscv/Makefile new file +3 -0
new file mode 100644
index 0000000..7fbca17
@@ -0,0 +1,3 @@
+ifndef NO_DWARF
+PERF_HAVE_DWARF_REGS := 1
+endif
tools/perf/arch/riscv/include/perf_regs.h new file +96 -0

96 lines not on this page · raw message

tools/perf/arch/riscv/util/Build new file +2 -0
new file mode 100644
index 0000000..1160bb2
@@ -0,0 +1,2 @@
+perf-$(CONFIG_DWARF) += dwarf-regs.o
+perf-$(CONFIG_LIBDW_DWARF_UNWIND) += unwind-libdw.o
tools/perf/arch/riscv/util/dwarf-regs.c new file +72 -0

72 lines not on this page · raw message

tools/perf/arch/riscv/util/unwind-libdw.c new file +57 -0

57 lines not on this page · raw message

Signature · 7 lines
2.7.4


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#4 lore raw
In reply to Mao Han

PING?

Quoted 56 lines · Mao Han

This patch set add perf callchain(FP/DWARF) support for RISC-V. It comes from the csky version callchain support with some slight modifications. The patchset base on Linux 5.1.

CC Palmer Dabbelt <[email protected]>
CC linux-riscv <[email protected]>
CC Christoph Hellwig <[email protected]>
CC Guo Ren <[email protected]>

Changes since v2:
- fix inconsistent comment
- force to build kernel with -fno-omit-frame-pointer if perf event is enabled

Changes since v1:
- simplify implementation and code convention

Mao Han (3):
riscv: Add perf callchain support
riscv: Add support for perf registers sampling
riscv: Add support for libdw

 arch/riscv/Kconfig                            |   2 +
 arch/riscv/Makefile                           |   3 +
 arch/riscv/include/uapi/asm/perf_regs.h       |  42 ++++++++++
 arch/riscv/kernel/Makefile                    |   4 +-
 arch/riscv/kernel/perf_callchain.c            | 113 ++++++++++++++++++++++++++
 arch/riscv/kernel/perf_regs.c                 |  44 ++++++++++
 tools/arch/riscv/include/uapi/asm/perf_regs.h |  42 ++++++++++
 tools/perf/Makefile.config                    |   6 +-
 tools/perf/arch/riscv/Build                   |   1 +
 tools/perf/arch/riscv/Makefile                |   3 +
 tools/perf/arch/riscv/include/perf_regs.h     |  96 ++++++++++++++++++++++
 tools/perf/arch/riscv/util/Build              |   2 +
 tools/perf/arch/riscv/util/dwarf-regs.c       |  72 ++++++++++++++++
 tools/perf/arch/riscv/util/unwind-libdw.c     |  57 +++++++++++++
 14 files changed, 485 insertions(+), 2 deletions(-)
 create mode 100644 arch/riscv/include/uapi/asm/perf_regs.h
 create mode 100644 arch/riscv/kernel/perf_callchain.c
 create mode 100644 arch/riscv/kernel/perf_regs.c
 create mode 100644 tools/arch/riscv/include/uapi/asm/perf_regs.h
 create mode 100644 tools/perf/arch/riscv/Build
 create mode 100644 tools/perf/arch/riscv/Makefile
 create mode 100644 tools/perf/arch/riscv/include/perf_regs.h
 create mode 100644 tools/perf/arch/riscv/util/Build
 create mode 100644 tools/perf/arch/riscv/util/dwarf-regs.c
 create mode 100644 tools/perf/arch/riscv/util/unwind-libdw.c
Signature · 7 lines
2.7.4


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#5 lore raw
In reply to Mao Han

Hello Mao Han,

Quoted 8 lines · Mao Han

This patch set add perf callchain(FP/DWARF) support for RISC-V. It comes from the csky version callchain support with some slight modifications. The patchset base on Linux 5.1.

CC Palmer Dabbelt <[email protected]>
CC linux-riscv <[email protected]>
CC Christoph Hellwig <[email protected]>
CC Guo Ren <[email protected]>

I tried these patches on v5.3-rc4, both on the HiFive Unleashed board with a Debian-based rootfs and QEMU rv64 with a Fedora-based rootfs. For QEMU, I used defconfig, and for the HiFive Unleashed, I added a few more Kconfig directives; and on both, I enabled CONFIG_PERF_EVENTS. I built the perf tools from the kernel tree.

Upon running "/root/bin/perf record -e cpu-clock --call-graph fp /bin/ls", I see the backtraces below. The first is on the HiFive Unleashed, the second is on QEMU.

Could you take a look and tell me if you see similar issues? And if not, could you please walk me through your process for testing these patches on rv64, so I can reproduce it here?

thanks,

- Paul

# /root/bin/perf record -e cpu-clock --call-graph fp /bin/ls
[  213.258888] Unable to handle kernel paging request at virtual address ffffffd2d979000c
[  213.266025] Oops [#1]
[  213.268278] Modules linked in:
[  213.271323] CPU: 2 PID: 165 Comm: ls Not tainted 5.3.0-rc4-00006-g2615b94055fc #164
[  213.278961] sepc: ffffffe0005bde76 ra : ffffffe0005bde84 sp : ffffffe1f609d870
[  213.286166]  gp : ffffffe000dbe6c0 tp : ffffffe1f60fa600 t0 : 0000000000046000
[  213.293372]  t1 : 0000000000000002 t2 : 0000002aaaac4890 s0 : ffffffe1f609d8b0
[  213.300579]  s1 : ffffffe1f609d8d8 a0 : 0000000000000004 a1 : 0000000000000000
[  213.307785]  a2 : 10ef8552d9790004 a3 : 0000000000000018 a4 : 0000000000000018
[  213.314991]  a5 : 0000000000000003 a6 : 0000000000000001 a7 : ffffffe1f7b685e0
[  213.322198]  s2 : 10ef8552d9790014 s3 : 0000003fffffffff s4 : 0000000000001ff8
[  213.329404]  s5 : b7e9595db7f9e15f s6 : ffffffe000dbe128 s7 : ffffffe1f8d6f820
[  213.336610]  s8 : 00000031a394a3e8 s9 : ffffffe000656150 s10: 0000000200000120
[  213.343816]  s11: 0000002aaaabc938 t3 : ffffffe1f7b685c0 t4 : 0000000000000001
[  213.351021]  t5 : 0000000000000008 t6 : 0000000000000000
[  213.356319] sstatus: 0000000200000100 sbadaddr: ffffffd2d979000c scause: 000000000000000d
[  213.364482] ---[ end trace f325867b8e0786aa ]---
[  213.369083] Kernel panic - not syncing: Fatal exception in interrupt
[  213.375421] SMP: stopping secondary CPUs
[  213.379336] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
# /root/bin/perf record -e cpu-clock --call-graph fp /bin/ls
[ 3465.349655] Unable to handle kernel paging request at virtual address d9bfd43b47718688
[ 3465.352356] Oops [#1]
[ 3465.353375] Modules linked in:
[ 3465.354496] CPU: 0 PID: 9501 Comm: ls Not tainted 5.3.0-rc4-00004-g3bfea1b43363 #165
[ 3465.355904] sepc: ffffffe000039a0a ra : ffffffe000039a18 sp : ffffffe078c5b5e0
[ 3465.361041]  gp : ffffffe000837760 tp : ffffffe078d88940 t0 : 0000000000046000
[ 3465.366104]  t1 : 0000000000000002 t2 : 000000004387194a s0 : ffffffe078c5b620
[ 3465.366839]  s1 : ffffffe078c5b648 a0 : 0000000000000004 a1 : 0000000000000000
[ 3465.369101]  a2 : d9bfd43b47718680 a3 : 0000000000000690 a4 : 0000000000000018
[ 3465.370949]  a5 : 0000000000000003 a6 : 0000000000000001 a7 : ffffffe0743b16d0
[ 3465.371650]  s2 : d9bfd43b47718690 s3 : 0000003fffffffff s4 : 0000000000001ff8
[ 3465.372658]  s5 : e85970cf9e9fbbc8 s6 : ffffffe0008371c8 s7 : ffffffe07a1527a0
[ 3465.373570]  s8 : 00000326c7b3d418 s9 : ffffffe0000d3326 s10: 0000000000000120
[ 3465.377102]  s11: ffffffe078cb4880 t3 : ffffffe0743b16c0 t4 : 0000000000000001
[ 3465.378125]  t5 : 0000000000000008 t6 : ffffffe078c5bd60
[ 3465.380603] sstatus: 0000000000000100 sbadaddr: d9bfd43b47718688 scause: 000000000000000d
[ 3465.382872] ---[ end trace f0ff9969d564eb82 ]---
[ 3465.383998] Kernel panic - not syncing: Fatal exception in interrupt
[ 3465.385812] SMP: stopping secondary CPUs
[ 3465.394408] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
author
#6 lore raw

Hi Paul,

Quoted 19 lines · Paul Walmsley

Hello Mao Han,

Quoted 8 lines · Mao Han

This patch set add perf callchain(FP/DWARF) support for RISC-V. It comes from the csky version callchain support with some slight modifications. The patchset base on Linux 5.1.

CC Palmer Dabbelt <[email protected]>
CC linux-riscv <[email protected]>
CC Christoph Hellwig <[email protected]>
CC Guo Ren <[email protected]>

I tried these patches on v5.3-rc4, both on the HiFive Unleashed board with a Debian-based rootfs and QEMU rv64 with a Fedora-based rootfs. For QEMU, I used defconfig, and for the HiFive Unleashed, I added a few more Kconfig directives; and on both, I enabled CONFIG_PERF_EVENTS. I built the perf tools from the kernel tree.

Quoted 8 lines

Upon running "/root/bin/perf record -e cpu-clock --call-graph fp /bin/ls", I see the backtraces below. The first is on the HiFive Unleashed, the second is on QEMU.

Could you take a look and tell me if you see similar issues? And if not, could you please walk me through your process for testing these patches on rv64, so I can reproduce it here?

I'v tried the command line above and got similar issues with probability. unwind_frame_kernel can not stop unwind when fp is a quite large value(like 0x70aac93ff0eff584) which can pass the simple stack check.

        if (kstack_end((void *)frame->fp))
                return -EPERM;
        if (frame->fp & 0x3 || frame->fp < TASK_SIZE)
                return -EPERM;

handle_exception from arch/riscv/kernel/entry.S will use s0(fp) as temp register. The context for this frame is unpredictable. We may add more strict check in unwind_frame_kernel or keep s0 always 0 in handle_exception to fix this issue.

Breakpoint 1, unwind_frame_kernel (frame=0xffffffe0000057a0 <rcu_init+118>)
at arch/riscv/kernel/perf_callchain.c:14
14 {
1: /x *frame = {fp = 0xffffffe000005ee0, ra = 0xffffffe000124c90} (gdb)
Continuing.

Breakpoint 1, unwind_frame_kernel (frame=0xffffffe0000057a0 <rcu_init+118>)
at arch/riscv/kernel/perf_callchain.c:14
14 {
1: /x *frame = {fp = 0xffffffe000124c74, ra = 0xffffffe000036cba} (gdb)
Continuing.

Breakpoint 1, unwind_frame_kernel (frame=0xffffffe0000057a0 <rcu_init+118>)
at arch/riscv/kernel/perf_callchain.c:14
14 {
1: /x *frame = {fp = 0x70aac57ff0eff584, ra = 0x8082614d64ea740a} (gdb)

Some time perf record can output perf.data correct. something like:

# perf record -e cpu-clock --call-graph fp ls

perf.data
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.012 MB perf.data (102 samples) ] #

# perf report
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 102  of event 'cpu-clock'
# Event count (approx.): 25500000
#
# Children      Self  Command  Shared Object      Symbol                           
# ........  ........  .......  .................  .................................
#
    30.39%     0.00%  ls       [kernel.kallsyms]  [k] ret_from_exception
            |
            ---ret_from_exception
               |          
               |--16.67%--do_page_fault
               |          handle_mm_fault
               |          __handle_mm_fault
               |          |          
               |          |--9.80%--filemap_map_pages
               |          |          |          
               |          |          |--3.92%--alloc_set_pte
               |          |          |          page_add_file_rmap
               |          |          |          
               |          |           --3.92%--filemap_map_pages
               |          |          
               |           --0.98%--__do_fault
               |          
               |--8.82%--schedule
               |          __sched_text_start
               |          finish_task_switch
               |          
                --4.90%--__kprobes_text_start
                          irq_exit
                          irq_exit
    28.43%     0.00%  ls       [kernel.kallsyms]  [k] ret_from_syscall
            |
            ---ret_from_syscall
               |          
               |--15.69%--__se_sys_execve
               |          __do_execve_file
               |          search_binary_handler.part.7

I previous tested these patch with a program(callchain_test.c) mostly running in userspace, so I didn't got this issues.

Thanks,
Mao Han

#include <stdio.h>

void test_4(void)
{
volatile int i, j;

for(i = 0; i < 10000000; i++)

    j=i; 
}

void test_3(void)
{
volatile int i, j;
test_4();
for(i = 0; i < 3000; i++)

    j=i;
}

void test_2(void)
{
volatile int i, j;
test_3();
for(i = 0; i < 3000; i++)

    j=i;
}

void test_1(void)
{
volatile int i, j;
test_2();
for(i = 0; i < 3000; i++)

    j=i;

}

int main(void)
{
test_1();
return 0;
}

author
#7 lore raw
Quoted 33 lines · Mao Han

Hi Paul,

Quoted 19 lines · Paul Walmsley

Hello Mao Han,

Quoted 8 lines · Mao Han

This patch set add perf callchain(FP/DWARF) support for RISC-V. It comes from the csky version callchain support with some slight modifications. The patchset base on Linux 5.1.

CC Palmer Dabbelt <[email protected]>
CC linux-riscv <[email protected]>
CC Christoph Hellwig <[email protected]>
CC Guo Ren <[email protected]>

I tried these patches on v5.3-rc4, both on the HiFive Unleashed board with a Debian-based rootfs and QEMU rv64 with a Fedora-based rootfs. For QEMU, I used defconfig, and for the HiFive Unleashed, I added a few more Kconfig directives; and on both, I enabled CONFIG_PERF_EVENTS. I built the perf tools from the kernel tree.

Quoted 8 lines

Upon running "/root/bin/perf record -e cpu-clock --call-graph fp /bin/ls", I see the backtraces below. The first is on the HiFive Unleashed, the second is on QEMU.

Could you take a look and tell me if you see similar issues? And if not, could you please walk me through your process for testing these patches on rv64, so I can reproduce it here?

I'v tried the command line above and got similar issues with probability. unwind_frame_kernel can not stop unwind when fp is a quite large value(like 0x70aac93ff0eff584) which can pass the simple stack check.

Quoted 9 lines
        if (kstack_end((void *)frame->fp))
                return -EPERM;
        if (frame->fp & 0x3 || frame->fp < TASK_SIZE)
                return -EPERM;

handle_exception from arch/riscv/kernel/entry.S will use s0(fp) as temp register. The context for this frame is unpredictable. We may add more strict check in unwind_frame_kernel or keep s0 always 0 in handle_exception to fix this issue.

perf record -e cpu-clock --call-graph fp /bin/ls seems can work stably with this change applied.

arch/riscv/kernel/perf_callchain.c +2 -0
index 8b57903..dd27c67 100644
@@ -16,6 +16,8 @@ static int unwind_frame_kernel(struct stackframe *frame)
                return -EPERM;
        if (frame->fp & 0x3 || frame->fp < TASK_SIZE)
                return -EPERM;
+        if (frame->fp < CONFIG_PAGE_OFFSET)
+                return -EPERM;
 
        *frame = *((struct stackframe *)frame->fp - 1);
        if (__kernel_text_address(frame->ra)) {

PS: I got some compile error while compiling glibc 2.30 with linux v5.3-rc4 header. vfork.S include linux/sched.h(./include/uapi/linux/sched.h) which has a struct clone_args inside, added by 7f192e3cd316ba58c88dfa26796cf77789dd9872.

#8 lore raw
Quoted 4 lines · Mao Han

PS: I got some compile error while compiling glibc 2.30 with linux v5.3-rc4 header. vfork.S include linux/sched.h(./include/uapi/linux/sched.h) which has a struct clone_args inside, added by 7f192e3cd316ba58c88dfa26796cf77789dd9872.

Noticed that also. Probably the sched.h uapi kernel header file needs an "#ifndef __ASSEMBLY__" around the struct clone_args...

- Paul