Skip to contentSkip to conversation

riscv: Add perf callchain support

PATCH Superseded Mao Han 3 participants 8 messages Aug 23, 2019 – Aug 27, 2019

Subsystems (3)

v5 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Mao Han

author
#0 lore raw

v5 cover riscv: Add perf callchain support

This patch set add perf callchain(FP/DWARF) support for RISC-V. It comes from the csky version callchain support with some slight modifications. The patchset base on Linux 5.3.

Changes since v4:
- Add missing PERF_HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET verified with extra CFLAGS(-Wall -Werror)

Changes since v3:
- Add more strict check for unwind_frame_kernel
- update for kernel 5.3

Changes since v2:
- fix inconsistent comment
- force to build kernel with -fno-omit-frame-pointer if perf event is enabled

Changes since v1:
- simplify implementation and code convention

Mao Han (3):
riscv: Add perf callchain support
riscv: Add support for perf registers sampling
riscv: Add support for libdw

 arch/riscv/Kconfig                            |   2 +
 arch/riscv/Makefile                           |   3 +
 arch/riscv/include/uapi/asm/perf_regs.h       |  42 ++++++++++
 arch/riscv/kernel/Makefile                    |   4 +-
 arch/riscv/kernel/perf_callchain.c            | 115 ++++++++++++++++++++++++++
 arch/riscv/kernel/perf_regs.c                 |  44 ++++++++++
 tools/arch/riscv/include/uapi/asm/perf_regs.h |  42 ++++++++++
 tools/perf/Makefile.config                    |   6 +-
 tools/perf/arch/riscv/Build                   |   1 +
 tools/perf/arch/riscv/Makefile                |   4 +
 tools/perf/arch/riscv/include/perf_regs.h     |  96 +++++++++++++++++++++
 tools/perf/arch/riscv/util/Build              |   2 +
 tools/perf/arch/riscv/util/dwarf-regs.c       |  72 ++++++++++++++++
 tools/perf/arch/riscv/util/unwind-libdw.c     |  57 +++++++++++++
 14 files changed, 488 insertions(+), 2 deletions(-)
 create mode 100644 arch/riscv/include/uapi/asm/perf_regs.h
 create mode 100644 arch/riscv/kernel/perf_callchain.c
 create mode 100644 arch/riscv/kernel/perf_regs.c
 create mode 100644 tools/arch/riscv/include/uapi/asm/perf_regs.h
 create mode 100644 tools/perf/arch/riscv/Build
 create mode 100644 tools/perf/arch/riscv/Makefile
 create mode 100644 tools/perf/arch/riscv/include/perf_regs.h
 create mode 100644 tools/perf/arch/riscv/util/Build
 create mode 100644 tools/perf/arch/riscv/util/dwarf-regs.c
 create mode 100644 tools/perf/arch/riscv/util/unwind-libdw.c
Signature · 1 line
2.7.4
author
#1 lore raw

PATCH v5 1/3 riscv: Add perf callchain support

This patch add support for perf callchain sampling on riscv platform. The return address of leaf function is retrieved from pt_regs as it is not saved in the outmost frame.

Cc Paul Walmsley <[email protected]>
Cc Greentime Hu <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc linux-riscv <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Guo Ren <[email protected]>
---
 arch/riscv/Makefile                |   3 +
 arch/riscv/kernel/Makefile         |   3 +-
 arch/riscv/kernel/perf_callchain.c | 115 +++++++++++++++++++++++++++++++++++++
 3 files changed, 120 insertions(+), 1 deletion(-)
 create mode 100644 arch/riscv/kernel/perf_callchain.c
arch/riscv/Makefile +3 -0
index 7a117be..946565b 100644
@@ -54,6 +54,9 @@ endif
 ifeq ($(CONFIG_MODULE_SECTIONS),y)
 	KBUILD_LDFLAGS_MODULE += -T $(srctree)/arch/riscv/kernel/module.lds
 endif
+ifeq ($(CONFIG_PERF_EVENTS),y)
+        KBUILD_CFLAGS += -fno-omit-frame-pointer
+endif
 
 KBUILD_CFLAGS_MODULE += $(call cc-option,-mno-relax)
 
arch/riscv/kernel/Makefile +2 -1
index 2420d37..b1bea89 100644
@@ -38,6 +38,7 @@ obj-$(CONFIG_MODULE_SECTIONS)	+= module-sections.o
 obj-$(CONFIG_FUNCTION_TRACER)	+= mcount.o ftrace.o
 obj-$(CONFIG_DYNAMIC_FTRACE)	+= mcount-dyn.o
 
-obj-$(CONFIG_PERF_EVENTS)      += perf_event.o
+obj-$(CONFIG_PERF_EVENTS)	+= perf_event.o
+obj-$(CONFIG_PERF_EVENTS)	+= perf_callchain.o
 
 clean:
arch/riscv/kernel/perf_callchain.c new file +115 -0

115 lines not on this page · raw message

Signature · 1 line
2.7.4
author
#6 lore raw

PATCH v5 2/3 riscv: Add support for perf registers sampling

In reply to Mao Han

This patch implements the perf registers sampling and validation API for riscv arch. The valid registers and their register ID are defined in perf_regs.h. Perf tool can backtrace in userspace with unwind library and the registers/user stack dump support.

Cc Paul Walmsley <[email protected]>
Cc Greentime Hu <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc linux-riscv <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Guo Ren <[email protected]>
---
 arch/riscv/Kconfig                      |  2 ++
 arch/riscv/include/uapi/asm/perf_regs.h | 42 +++++++++++++++++++++++++++++++
 arch/riscv/kernel/Makefile              |  1 +
 arch/riscv/kernel/perf_regs.c           | 44 +++++++++++++++++++++++++++++++++
 4 files changed, 89 insertions(+)
 create mode 100644 arch/riscv/include/uapi/asm/perf_regs.h
 create mode 100644 arch/riscv/kernel/perf_regs.c
arch/riscv/Kconfig +2 -0
index 59a4727..4bc976d 100644
@@ -35,6 +35,8 @@ config RISCV
 	select HAVE_DMA_CONTIGUOUS
 	select HAVE_FUTEX_CMPXCHG if FUTEX
 	select HAVE_PERF_EVENTS
+	select HAVE_PERF_REGS
+	select HAVE_PERF_USER_STACK_DUMP
 	select HAVE_SYSCALL_TRACEPOINTS
 	select IRQ_DOMAIN
 	select SPARSE_IRQ
arch/riscv/include/uapi/asm/perf_regs.h new file +42 -0

42 lines not on this page · raw message

arch/riscv/kernel/Makefile +1 -0
index b1bea89..696020f 100644
@@ -40,5 +40,6 @@ obj-$(CONFIG_DYNAMIC_FTRACE)	+= mcount-dyn.o
 
 obj-$(CONFIG_PERF_EVENTS)	+= perf_event.o
 obj-$(CONFIG_PERF_EVENTS)	+= perf_callchain.o
+obj-$(CONFIG_HAVE_PERF_REGS)	+= perf_regs.o
 
 clean:
arch/riscv/kernel/perf_regs.c new file +44 -0

44 lines not on this page · raw message

Signature · 1 line
2.7.4
author
#7 lore raw

PATCH v5 3/3 riscv: Add support for libdw

In reply to Mao Han

This patch add support for DWARF register mappings and libdw registers initialization, which is used by perf callchain analyzing when --call-graph=dwarf is given.

Cc Paul Walmsley <[email protected]>
Cc Greentime Hu <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc linux-riscv <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Guo Ren <[email protected]>
---
 tools/arch/riscv/include/uapi/asm/perf_regs.h | 42 ++++++++++++
 tools/perf/Makefile.config                    |  6 +-
 tools/perf/arch/riscv/Build                   |  1 +
 tools/perf/arch/riscv/Makefile                |  4 ++
 tools/perf/arch/riscv/include/perf_regs.h     | 96 +++++++++++++++++++++++++++
 tools/perf/arch/riscv/util/Build              |  2 +
 tools/perf/arch/riscv/util/dwarf-regs.c       | 72 ++++++++++++++++++++
 tools/perf/arch/riscv/util/unwind-libdw.c     | 57 ++++++++++++++++
 8 files changed, 279 insertions(+), 1 deletion(-)
 create mode 100644 tools/arch/riscv/include/uapi/asm/perf_regs.h
 create mode 100644 tools/perf/arch/riscv/Build
 create mode 100644 tools/perf/arch/riscv/Makefile
 create mode 100644 tools/perf/arch/riscv/include/perf_regs.h
 create mode 100644 tools/perf/arch/riscv/util/Build
 create mode 100644 tools/perf/arch/riscv/util/dwarf-regs.c
 create mode 100644 tools/perf/arch/riscv/util/unwind-libdw.c
tools/arch/riscv/include/uapi/asm/perf_regs.h new file +42 -0
new file mode 100644
index 0000000..df1a581
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2019 Hangzhou C-SKY Microsystems co.,ltd. */
+
+#ifndef _ASM_RISCV_PERF_REGS_H
+#define _ASM_RISCV_PERF_REGS_H
+
+enum perf_event_riscv_regs {
+	PERF_REG_RISCV_PC,
+	PERF_REG_RISCV_RA,
+	PERF_REG_RISCV_SP,
+	PERF_REG_RISCV_GP,
+	PERF_REG_RISCV_TP,
+	PERF_REG_RISCV_T0,
+	PERF_REG_RISCV_T1,
+	PERF_REG_RISCV_T2,
+	PERF_REG_RISCV_S0,
+	PERF_REG_RISCV_S1,
+	PERF_REG_RISCV_A0,
+	PERF_REG_RISCV_A1,
+	PERF_REG_RISCV_A2,
+	PERF_REG_RISCV_A3,
+	PERF_REG_RISCV_A4,
+	PERF_REG_RISCV_A5,
+	PERF_REG_RISCV_A6,
+	PERF_REG_RISCV_A7,
+	PERF_REG_RISCV_S2,
+	PERF_REG_RISCV_S3,
+	PERF_REG_RISCV_S4,
+	PERF_REG_RISCV_S5,
+	PERF_REG_RISCV_S6,
+	PERF_REG_RISCV_S7,
+	PERF_REG_RISCV_S8,
+	PERF_REG_RISCV_S9,
+	PERF_REG_RISCV_S10,
+	PERF_REG_RISCV_S11,
+	PERF_REG_RISCV_T3,
+	PERF_REG_RISCV_T4,
+	PERF_REG_RISCV_T5,
+	PERF_REG_RISCV_T6,
+	PERF_REG_RISCV_MAX,
+};
+#endif /* _ASM_RISCV_PERF_REGS_H */
tools/perf/Makefile.config +5 -1
index 89ac5a1..eaf25ee 100644
@@ -60,6 +60,10 @@ ifeq ($(SRCARCH),arm64)
   LIBUNWIND_LIBS = -lunwind -lunwind-aarch64
 endif
 
+ifeq ($(SRCARCH),riscv)
+  NO_PERF_REGS := 0
+endif
+
 ifeq ($(SRCARCH),csky)
   NO_PERF_REGS := 0
 endif
@@ -82,7 +86,7 @@ endif
 # Disable it on all other architectures in case libdw unwind
 # support is detected in system. Add supported architectures
 # to the check.
-ifneq ($(SRCARCH),$(filter $(SRCARCH),x86 arm arm64 powerpc s390 csky))
+ifneq ($(SRCARCH),$(filter $(SRCARCH),x86 arm arm64 powerpc s390 csky riscv))
   NO_LIBDW_DWARF_UNWIND := 1
 endif
 
tools/perf/arch/riscv/Build new file +1 -0
new file mode 100644
index 0000000..e4e5f33
@@ -0,0 +1 @@
+perf-y += util/
tools/perf/arch/riscv/Makefile new file +4 -0
new file mode 100644
index 0000000..1aa9dd7
@@ -0,0 +1,4 @@
+ifndef NO_DWARF
+PERF_HAVE_DWARF_REGS := 1
+endif
+PERF_HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET := 1
tools/perf/arch/riscv/include/perf_regs.h new file +96 -0

96 lines not on this page · raw message

tools/perf/arch/riscv/util/Build new file +2 -0
new file mode 100644
index 0000000..1160bb2
@@ -0,0 +1,2 @@
+perf-$(CONFIG_DWARF) += dwarf-regs.o
+perf-$(CONFIG_LIBDW_DWARF_UNWIND) += unwind-libdw.o
tools/perf/arch/riscv/util/dwarf-regs.c new file +72 -0

72 lines not on this page · raw message

tools/perf/arch/riscv/util/unwind-libdw.c new file +57 -0

57 lines not on this page · raw message

Signature · 1 line
2.7.4
#2 lore raw
In reply to Mao Han

Hi Mao,

Mao Han <[email protected]> 於 2019年8月23日 週五 下午2:16寫道:

Quoted 12 lines

This patch add support for perf callchain sampling on riscv platform. The return address of leaf function is retrieved from pt_regs as it is not saved in the outmost frame.

Cc Paul Walmsley <[email protected]>
Cc Greentime Hu <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc linux-riscv <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Guo Ren <[email protected]>
Quoted 6 lines
---
 arch/riscv/Makefile                |   3 +
 arch/riscv/kernel/Makefile         |   3 +-
 arch/riscv/kernel/perf_callchain.c | 115 +++++++++++++++++++++++++++++++++++++
 3 files changed, 120 insertions(+), 1 deletion(-)
 create mode 100644 arch/riscv/kernel/perf_callchain.c

I just tested "./perf record -e cpu-clock --call-graph fp ls" on Unleashed board and I got this failure.
I take a look at it. It seem failed in here. Do you have any idea? It seems fine in Qemu.

1 *frame = *((struct stackframe *)frame->fp - 1);
ffffffe0001a198c: 00863a83 ld s5,8(a2)
ffffffe0001a1990: ff093903 ld s2,-16(s2)

./perf record -e cpu-clock --call-graph fp ls

[ 9619.423884] hrtimer: interrupt took 733000 ns
[ 9619.977017] Unable to handle kernel paging request at virtual
address ffffffffffffff94
[ 9620.214736] Oops [#1]
[ 9620.289893] Modules linked in:
[ 9620.391378] CPU: 0 PID: 264 Comm: ls Not tainted
5.3.0-rc5-00003-gb008f6bcd67c #4
[ 9620.640176] sepc: ffffffe0001a198c ra : ffffffe0001a199a sp :
ffffffe000093720
[ 9620.880366] gp : ffffffe00097dad8 tp : ffffffe000082e40 t0 : 0000000000046000
[ 9621.120564] t1 : 0000000000000002 t2 : 0000000000000007 s0 : ffffffe000093760
[ 9621.360768] s1 : ffffffe000093788 a0 : 0000000000000003 a1 : 0000000000000000
[ 9621.600991] a2 : ffffffffffffff8c a3 : 0000000000001fa0 a4 : 0000000000000010
[ 9621.841181] a5 : 0000000000000002 a6 : 0000000000000001 a7 : ffffffe079b34e10
[ 9622.081400] s2 : ffffffffffffff9c s3 : ffffffe000000000 s4 : 0000000000001ff8
[ 9622.321618] s5 : ffffffe000093da0 s6 : ffffffe00097d540 s7 : ffffffe07a1517a0
[ 9622.561811] s8 : 000008bf01c7ff60 s9 : ffffffe000235b2a s10: 0000000200000120
[ 9622.802015] s11: 0000000000000001 t3 : ffffffe079b34e00 t4 : 0000000000000001
[ 9623.042194] t5 : 0000000000000008 t6 : ffffffe0009208d0
[ 9623.218785] sstatus: 0000000200000100 sbadaddr: ffffffffffffff94
scause: 000000000000000d
[ 9623.490850] ---[ end trace 49043f28e856d84d ]---
[ 9623.644217] Kernel panic - not syncing: Fatal exception in interrupt
[ 9623.855470] SMP: stopping secondary CPUs
[ 9623.985955] ---[ end Kernel panic - not syncing: Fatal exception in

interrupt ]---

#3 lore raw

Please check CONFIG_FRAME_POINTER

1 *frame = *((struct stackframe *)frame->fp - 1);
This code is origionally from riscv/kernel/stacktrace.c: walk_stackframe

In linux/Makefile it'll involve the options for gcc to definitely store ra & prev_fp in fp pointer.
ifdef CONFIG_FRAME_POINTER
KBUILD_CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls

So --call-graph fp need depends on CONFIG_FRAME_POINTER.

Quoted 60 lines · Greentime Hu

Hi Mao,

Mao Han <[email protected]> 於 2019年8月23日 週五 下午2:16寫道:

Quoted 12 lines

This patch add support for perf callchain sampling on riscv platform. The return address of leaf function is retrieved from pt_regs as it is not saved in the outmost frame.

Cc Paul Walmsley <[email protected]>
Cc Greentime Hu <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc linux-riscv <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Guo Ren <[email protected]>
Quoted 6 lines
---
 arch/riscv/Makefile                |   3 +
 arch/riscv/kernel/Makefile         |   3 +-
 arch/riscv/kernel/perf_callchain.c | 115 +++++++++++++++++++++++++++++++++++++
 3 files changed, 120 insertions(+), 1 deletion(-)
 create mode 100644 arch/riscv/kernel/perf_callchain.c

I just tested "./perf record -e cpu-clock --call-graph fp ls" on Unleashed board and I got this failure.
I take a look at it. It seem failed in here. Do you have any idea? It seems fine in Qemu.

1 *frame = *((struct stackframe *)frame->fp - 1);
ffffffe0001a198c: 00863a83 ld s5,8(a2)
ffffffe0001a1990: ff093903 ld s2,-16(s2)

./perf record -e cpu-clock --call-graph fp ls

[ 9619.423884] hrtimer: interrupt took 733000 ns
[ 9619.977017] Unable to handle kernel paging request at virtual
address ffffffffffffff94
[ 9620.214736] Oops [#1]
[ 9620.289893] Modules linked in:
[ 9620.391378] CPU: 0 PID: 264 Comm: ls Not tainted
5.3.0-rc5-00003-gb008f6bcd67c #4
[ 9620.640176] sepc: ffffffe0001a198c ra : ffffffe0001a199a sp :
ffffffe000093720
[ 9620.880366] gp : ffffffe00097dad8 tp : ffffffe000082e40 t0 : 0000000000046000
[ 9621.120564] t1 : 0000000000000002 t2 : 0000000000000007 s0 : ffffffe000093760
[ 9621.360768] s1 : ffffffe000093788 a0 : 0000000000000003 a1 : 0000000000000000
[ 9621.600991] a2 : ffffffffffffff8c a3 : 0000000000001fa0 a4 : 0000000000000010
[ 9621.841181] a5 : 0000000000000002 a6 : 0000000000000001 a7 : ffffffe079b34e10
[ 9622.081400] s2 : ffffffffffffff9c s3 : ffffffe000000000 s4 : 0000000000001ff8
[ 9622.321618] s5 : ffffffe000093da0 s6 : ffffffe00097d540 s7 : ffffffe07a1517a0
[ 9622.561811] s8 : 000008bf01c7ff60 s9 : ffffffe000235b2a s10: 0000000200000120
[ 9622.802015] s11: 0000000000000001 t3 : ffffffe079b34e00 t4 : 0000000000000001
[ 9623.042194] t5 : 0000000000000008 t6 : ffffffe0009208d0
[ 9623.218785] sstatus: 0000000200000100 sbadaddr: ffffffffffffff94
scause: 000000000000000d
[ 9623.490850] ---[ end trace 49043f28e856d84d ]---
[ 9623.644217] Kernel panic - not syncing: Fatal exception in interrupt
[ 9623.855470] SMP: stopping secondary CPUs
[ 9623.985955] ---[ end Kernel panic - not syncing: Fatal exception in

interrupt ]---

Signature · 9 lines
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#4 lore raw

Hi Guo,

Guo Ren <[email protected]> 於 2019年8月24日 週六 上午8:54寫道:

Quoted 6 lines

Please check CONFIG_FRAME_POINTER

1 *frame = *((struct stackframe *)frame->fp - 1);
This code is origionally from riscv/kernel/stacktrace.c: walk_stackframe

Quoted 7 lines

In linux/Makefile it'll involve the options for gcc to definitely store ra & prev_fp in fp pointer.
ifdef CONFIG_FRAME_POINTER
KBUILD_CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls

So --call-graph fp need depends on CONFIG_FRAME_POINTER.

I am pretty sure CONFIG_FRAME_POINTER is Y

# zcat /proc/config.gz |grep CONFIG_FRAME_POINTER
CONFIG_FRAME_POINTER=y

This is not going to go wrong every time, the probability of error is about one tenth or one quarter. randomly
There may be some conditions that we have not considered.

I add one more condition to check if it is a valid virtual address and it( ./perf record -e cpu-clock --call-graph fp ls) passes 1000 times without failure in Unleashed board based on 5.3-rc5.
Here is my patch. Please have a look at it. I am not sure if it is a good solution.

(quoted diff) +0 -0

b/arch/riscv/kernel/perf_callchain.c
index d75d15c13dc7..4717942435df 100644

arch/riscv/kernel/perf_callchain.c +2 -0
@@ -18,6 +18,8 @@ static int unwind_frame_kernel(struct stackframe *frame)
                return -EPERM;
        if (frame->fp < CONFIG_PAGE_OFFSET)
                return -EPERM;
+       if (!virt_addr_valid(frame->fp))
+               return -EPERM;
 
        *frame = *((struct stackframe *)frame->fp - 1);
        if (__kernel_text_address(frame->ra)) {

It could catch cases called in this way.

[ 1381.936586] frame->fp=:ffffffff00547550
[ 1382.038542] CPU: 1 PID: 135 Comm: ls Not tainted
5.3.0-rc5-00003-gb008f6bcd67c-dirty #14
[ 1382.307440] Call Trace:
[ 1382.388947] [<ffffffe0002a2d8e>] walk_stackframe+0x0/0x9a
[ 1382.568053] [<ffffffe0002a2f5a>] show_stack+0x2a/0x34
[ 1382.735960] [<ffffffe00083dcd6>] dump_stack+0x62/0x7c
[ 1382.903863] [<ffffffe0002a49e0>] perf_callchain_kernel+0xd8/0x102
[ 1383.106558] [<ffffffe000340a6e>] get_perf_callchain+0x136/0x1f2
[ 1383.303128] [<ffffffe00033d480>] perf_callchain+0x52/0x6e
[ 1383.482553] [<ffffffe00033d50a>] perf_prepare_sample+0x6e/0x476
[ 1383.679357] [<ffffffe00033d92e>] perf_event_output_forward+0x1c/0x50
[ 1383.890633] [<ffffffe000338b4c>] __perf_event_overflow+0x6a/0xa4
[ 1384.090279] [<ffffffe000338c40>] perf_swevent_hrtimer+0xba/0x106
[ 1384.290094] [<ffffffe0002f307c>] __hrtimer_run_queues+0x84/0x108
[ 1384.489694] [<ffffffe0002f36b8>] hrtimer_interrupt+0xca/0x1ce
[ 1384.680974] [<ffffffe00072f572>] riscv_timer_interrupt+0x32/0x3a
[ 1384.880449] [<ffffffe000854b34>] do_IRQ+0x64/0xbe
[ 1385.036698] [<ffffffe0002a1c5c>] ret_from_exception+0x0/0xc
[13915.697989] frame->fp=:fffffffffffff000
[13915.799937] CPU: 2 PID: 663 Comm: ls Not tainted
5.3.0-rc5-00003-gb008f6bcd67c-dirty #14
[13916.068832] Call Trace:
[13916.150380] [<ffffffe0002a2d8e>] walk_stackframe+0x0/0x9a
[13916.329450] [<ffffffe0002a2f5a>] show_stack+0x2a/0x34
[13916.497360] [<ffffffe00083dcd6>] dump_stack+0x62/0x7c
[13916.665265] [<ffffffe0002a49e0>] perf_callchain_kernel+0xd8/0x102
[13916.867949] [<ffffffe000340a6e>] get_perf_callchain+0x136/0x1f2
[13917.064526] [<ffffffe00033d480>] perf_callchain+0x52/0x6e
[13917.243950] [<ffffffe00033d50a>] perf_prepare_sample+0x6e/0x476
[13917.440759] [<ffffffe00033d92e>] perf_event_output_forward+0x1c/0x50
[13917.652021] [<ffffffe000338b4c>] __perf_event_overflow+0x6a/0xa4
[13917.851683] [<ffffffe000338c40>] perf_swevent_hrtimer+0xba/0x106
[13918.051494] [<ffffffe0002f307c>] __hrtimer_run_queues+0x84/0x108
[13918.251094] [<ffffffe0002f36b8>] hrtimer_interrupt+0xca/0x1ce
[13918.442379] [<ffffffe00072f572>] riscv_timer_interrupt+0x32/0x3a
[13918.641840] [<ffffffe000854b34>] do_IRQ+0x64/0xbe
[13918.798082] [<ffffffe0002a1c5c>] ret_from_exception+0x0/0xc
#5 lore raw

We need know the values of *regs, eg: regs->sepc, regs->ra, regs->fp, regs->sp, regs->tp

void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
struct pt_regs *regs)
{
...
walk_stackframe(&fr, entry);

// May be we could detect error here and print the regs' value }

Quoted 88 lines · Greentime Hu

Hi Guo,

Guo Ren <[email protected]> 於 2019年8月24日 週六 上午8:54寫道:

Quoted 6 lines

Please check CONFIG_FRAME_POINTER

1 *frame = *((struct stackframe *)frame->fp - 1);
This code is origionally from riscv/kernel/stacktrace.c: walk_stackframe

Quoted 7 lines

In linux/Makefile it'll involve the options for gcc to definitely store ra & prev_fp in fp pointer.
ifdef CONFIG_FRAME_POINTER
KBUILD_CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls

So --call-graph fp need depends on CONFIG_FRAME_POINTER.

I am pretty sure CONFIG_FRAME_POINTER is Y

# zcat /proc/config.gz |grep CONFIG_FRAME_POINTER
CONFIG_FRAME_POINTER=y

This is not going to go wrong every time, the probability of error is about one tenth or one quarter. randomly
There may be some conditions that we have not considered.

I add one more condition to check if it is a valid virtual address and it( ./perf record -e cpu-clock --call-graph fp ls) passes 1000 times without failure in Unleashed board based on 5.3-rc5.
Here is my patch. Please have a look at it. I am not sure if it is a good solution.

arch/riscv/kernel/perf_callchain.c +2 -0
index d75d15c13dc7..4717942435df 100644
@@ -18,6 +18,8 @@ static int unwind_frame_kernel(struct stackframe *frame)
                return -EPERM;
        if (frame->fp < CONFIG_PAGE_OFFSET)
                return -EPERM;
+       if (!virt_addr_valid(frame->fp))
+               return -EPERM;
 
        *frame = *((struct stackframe *)frame->fp - 1);
        if (__kernel_text_address(frame->ra)) {

It could catch cases called in this way.

[ 1381.936586] frame->fp=:ffffffff00547550
[ 1382.038542] CPU: 1 PID: 135 Comm: ls Not tainted
5.3.0-rc5-00003-gb008f6bcd67c-dirty #14
[ 1382.307440] Call Trace:
[ 1382.388947] [<ffffffe0002a2d8e>] walk_stackframe+0x0/0x9a
[ 1382.568053] [<ffffffe0002a2f5a>] show_stack+0x2a/0x34
[ 1382.735960] [<ffffffe00083dcd6>] dump_stack+0x62/0x7c
[ 1382.903863] [<ffffffe0002a49e0>] perf_callchain_kernel+0xd8/0x102
[ 1383.106558] [<ffffffe000340a6e>] get_perf_callchain+0x136/0x1f2
[ 1383.303128] [<ffffffe00033d480>] perf_callchain+0x52/0x6e
[ 1383.482553] [<ffffffe00033d50a>] perf_prepare_sample+0x6e/0x476
[ 1383.679357] [<ffffffe00033d92e>] perf_event_output_forward+0x1c/0x50
[ 1383.890633] [<ffffffe000338b4c>] __perf_event_overflow+0x6a/0xa4
[ 1384.090279] [<ffffffe000338c40>] perf_swevent_hrtimer+0xba/0x106
[ 1384.290094] [<ffffffe0002f307c>] __hrtimer_run_queues+0x84/0x108
[ 1384.489694] [<ffffffe0002f36b8>] hrtimer_interrupt+0xca/0x1ce
[ 1384.680974] [<ffffffe00072f572>] riscv_timer_interrupt+0x32/0x3a
[ 1384.880449] [<ffffffe000854b34>] do_IRQ+0x64/0xbe
[ 1385.036698] [<ffffffe0002a1c5c>] ret_from_exception+0x0/0xc
[13915.697989] frame->fp=:fffffffffffff000
[13915.799937] CPU: 2 PID: 663 Comm: ls Not tainted
5.3.0-rc5-00003-gb008f6bcd67c-dirty #14
[13916.068832] Call Trace:
[13916.150380] [<ffffffe0002a2d8e>] walk_stackframe+0x0/0x9a
[13916.329450] [<ffffffe0002a2f5a>] show_stack+0x2a/0x34
[13916.497360] [<ffffffe00083dcd6>] dump_stack+0x62/0x7c
[13916.665265] [<ffffffe0002a49e0>] perf_callchain_kernel+0xd8/0x102
[13916.867949] [<ffffffe000340a6e>] get_perf_callchain+0x136/0x1f2
[13917.064526] [<ffffffe00033d480>] perf_callchain+0x52/0x6e
[13917.243950] [<ffffffe00033d50a>] perf_prepare_sample+0x6e/0x476
[13917.440759] [<ffffffe00033d92e>] perf_event_output_forward+0x1c/0x50
[13917.652021] [<ffffffe000338b4c>] __perf_event_overflow+0x6a/0xa4
[13917.851683] [<ffffffe000338c40>] perf_swevent_hrtimer+0xba/0x106
[13918.051494] [<ffffffe0002f307c>] __hrtimer_run_queues+0x84/0x108
[13918.251094] [<ffffffe0002f36b8>] hrtimer_interrupt+0xca/0x1ce
[13918.442379] [<ffffffe00072f572>] riscv_timer_interrupt+0x32/0x3a
[13918.641840] [<ffffffe000854b34>] do_IRQ+0x64/0xbe
[13918.798082] [<ffffffe0002a1c5c>] ret_from_exception+0x0/0xc
Signature · 4 lines
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/