Skip to contentSkip to conversation

perf evsel: Improve callchain warning for s390

PATCH In review Jens Remus 3 participants 5 messages Aug 21 – Sep 22

Subsystems (2)
lore
Share & tools
lore

Jens Remus

author
#0 lore raw

PATCH v2 1/1 perf evsel: Improve callchain warning for s390

On s390 the kernel uses s390 back chain instead of frame pointers for stack tracing of user space since v6.7 commit aa44433ac4ee ("s390: add USER_STACKTRACE support"). This is because frame pointers on s390 cannot be used for stack tracing. [1]

This requires user space to maintain a s390 back chain. For instance user space to be built with compiler option '-mbackchain' (instead of '-fno-omit-frame-pointer' used on other architectures, which should better not be used on s390 [1]). Only few distributions and users build user space with '-mbackchain'. Therefore '--call-graph fp' may in practice not produce the expected results.

Commit ca76fb67ebdd ("perf evlist: Improve default event for s390") changed the default '-g' option to 'dwarf' on s390 and added a warning for s390 that wrongly claimed that "Framepointer unwinding lacks kernel support". This warning resulted from a misinterpretation of comments in s390 cpumsf_pmu_event_init().

The restriction in cpumsf_pmu_event_init() applies to callchain sampling with the s390 CPU Measurement Sampling Facility (CPUMSF) hardware PMU. CPUMSF events, such as 'cycles', do not support callchain sampling. This is independent of whether perf uses the 'fp' or 'dwarf' call-graph mode. The CPUMSF PMU provides samples collected asynchronously, making it impossible to associate a callchain to the historic IPs. Therefore callchains can only be used with software events on s390.

Remove the incorrect warning. The kernel supports the 'fp' call-graph mode on s390 by walking the back chain. Do not replace it with a hint to use 'dwarf' when the resulting callchain is incomplete. Such a suggestion could imply that 'fp' is inherently inferior. The same general limitation exists on other architectures when user space is not built with frame pointers (i.e. '-fno-omit-frame-pointers').

Instead, warn on s390 when callchain sampling is requested with an event provided by the CPUMSF hardware PMU, because that combination is not supported.

Update the perf record documentation to state that 'dwarf' is the default call-graph mode on s390 and that 'fp' uses back chain instead of frame pointers on s390.

Note that '--call-graph fp' may also be useful for other applications, such as OpenJDK maintaining a s390 back chain (does not require JVM option '-XX:+PreserveFramePointer' on s390):

  $ perf record --call-graph fp ... -- \
    java -XX:+UnlockDiagnosticVMOptions -XX:+DumpPerfMapAtExit ...

[1]: s390: Stack tracing using Frame Pointer, Back Chain, and SFrame,

     https://conf.gnu-tools-cauldron.org/opo25/talk/Y3CVHY/
Fixes ca76fb67ebdd ("perf evlist: Improve default event for s390")
---
 tools/perf/Documentation/perf-record.txt | 7 ++++++-
 tools/perf/util/evsel.c                  | 5 ++---
 2 files changed, 8 insertions(+), 4 deletions(-)
tools/perf/Documentation/perf-record.txt +6 -1
index 178f483140ed..dc72606cb25d 100644
@@ -293,7 +293,7 @@ OPTIONS
 
 --call-graph::
 	Setup and enable call-graph (stack chain/backtrace) recording,
-	implies -g.  Default is "fp" (for user space).
+	implies -g.  Default is "fp" for user space (s390 uses "dwarf").
 
 	The unwinding method used for kernel space is dependent on the
 	unwinder used by the active kernel configuration, i.e
@@ -329,6 +329,11 @@ OPTIONS
 	enable deferred user callchain which will collect user-space callchains
 	when the thread returns to the user space.
 
+	On s390, the "fp" method uses back chain instead of frame pointers,
+	which requires user space to maintain a back chain (e.g. compiler
+	option -mbackchain).  Without a maintained back chain this produces
+	bogus call graphs.  In that case use "dwarf" instead.
+
 -q::
 --quiet::
 	Don't print any warnings or messages, useful for scripting.
tools/perf/util/evsel.c +2 -3
index ea9fa04429f0..3b5c3654d9d7 100644
@@ -1078,9 +1078,8 @@ static void __evsel__config_callchain(struct evsel *evsel, const struct record_o
 	bool function = evsel__is_function_event(evsel);
 	struct perf_event_attr *attr = &evsel->core.attr;
 
-	if (EM_HOST == EM_S390 && param->record_mode == CALLCHAIN_FP) {
-		pr_warning_once(
-			"Framepointer unwinding lacks kernel support. Use '--call-graph dwarf'\n");
+	if (EM_HOST == EM_S390 && evsel->pmu && !perf_pmu__is_software(evsel->pmu)) {
+		pr_warning_once("Cannot use hardware CPUMSF event with -g and --call-graph.\n");
 	}
 
 	evsel__set_sample_bit(evsel, CALLCHAIN);

base-commit: 8d3ae59288f1e7d58d76558a6ee96d533bc5019f

Signature · 1 line
2.53.0
#1 lore raw
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 17 lines

On s390 the kernel uses s390 back chain instead of frame pointers for stack tracing of user space since v6.7 commit aa44433ac4ee ("s390: add USER_STACKTRACE support"). This is because frame pointers on s390 cannot be used for stack tracing. [1]

This requires user space to maintain a s390 back chain. For instance user space to be built with compiler option '-mbackchain' (instead of '-fno-omit-frame-pointer' used on other architectures, which should better not be used on s390 [1]). Only few distributions and users build user space with '-mbackchain'. Therefore '--call-graph fp' may in practice not produce the expected results.

Commit ca76fb67ebdd ("perf evlist: Improve default event for s390") changed the default '-g' option to 'dwarf' on s390 and added a warning for s390 that wrongly claimed that "Framepointer unwinding lacks kernel [ ... ]
[1]: s390: Stack tracing using Frame Pointer, Back Chain, and SFrame,

Quoted 4 lines
     https://conf.gnu-tools-cauldron.org/opo25/talk/Y3CVHY/
Fixes ca76fb67ebdd ("perf evlist: Improve default event for s390")

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author
#2 lore raw
In reply to Jens Remus
Quoted 7 lines · Jens Remus

Instead, warn on s390 when callchain sampling is requested with an event provided by the CPUMSF hardware PMU, because that combination is not supported.

tools/perf/util/evsel.c static void __evsel__config_callchain(struct evsel *evsel, const struct record_o · line 1082

Quoted 8 lines
tools/perf/util/evsel.c +2 -3
@@ -1078,9 +1078,8 @@ static void __evsel__config_callchain(struct evsel *evsel, const struct record_o
 	bool function = evsel__is_function_event(evsel);
 	struct perf_event_attr *attr = &evsel->core.attr;
 
-	if (EM_HOST == EM_S390 && param->record_mode == CALLCHAIN_FP) {
-		pr_warning_once(
-			"Framepointer unwinding lacks kernel support. Use '--call-graph dwarf'\n");
+	if (EM_HOST == EM_S390 && evsel->pmu && !perf_pmu__is_software(evsel->pmu)) {
+		pr_warning_once("Cannot use hardware CPUMSF event with -g and --call-graph.\n");

Jan, Thomas, could you please do me a favor and test whether this works as expected on LPAR with CPUMSF? Time was running out before I leave on vacation.

tools/perf/util/evsel.c static void __evsel__config_callchain(struct evsel *evsel, const struct record_o · line 1085

Quoted 3 lines
tools/perf/util/evsel.c +0 -0
 	}
 
 	evsel__set_sample_bit(evsel, CALLCHAIN);

Thanks and regards,
Jens

Signature · 6 lines
Jens Remus
Linux on Z Development (D3303)
[email protected] / [email protected]

IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats: Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft: Ehningen; Registergericht: Amtsgericht Stuttgart, HRB 243294
IBM Data Privacy Statement: https://www.ibm.com/privacy/
reviewer
#3 lore raw
Quoted 13 lines · Jens Remus
Quoted 10 lines · Jens Remus

Instead, warn on s390 when callchain sampling is requested with an event provided by the CPUMSF hardware PMU, because that combination is not supported.

tools/perf/util/evsel.c +0 -0
@@ -1078,9 +1078,8 @@ static void __evsel__config_callchain(struct evsel *evsel, const struct record_o
      bool function = evsel__is_function_event(evsel);
      struct perf_event_attr *attr = &evsel->core.attr;
 
Quoted 9 lines

tools/perf/util/evsel.c static void __evsel__config_callchain(struct evsel *evsel, const struct record_o · line 1082

Quoted 5 lines
tools/perf/util/evsel.c +2 -3
-     if (EM_HOST == EM_S390 && param->record_mode == CALLCHAIN_FP) {
-             pr_warning_once(
-                     "Framepointer unwinding lacks kernel support. Use '--call-graph dwarf'\n");
+     if (EM_HOST == EM_S390 && evsel->pmu && !perf_pmu__is_software(evsel->pmu)) {
+             pr_warning_once("Cannot use hardware CPUMSF event with -g and --call-graph.\n");

Jan, Thomas, could you please do me a favor and test whether this works as expected on LPAR with CPUMSF? Time was running out before I leave on vacation.

Thomas, the change looks good to me. What are your thoughts?

Thanks,
Ian

Quoted 13 lines

tools/perf/util/evsel.c static void __evsel__config_callchain(struct evsel *evsel, const struct record_o · line 1085

Quoted 3 lines
tools/perf/util/evsel.c +0 -0
      }
 
      evsel__set_sample_bit(evsel, CALLCHAIN);

Thanks and regards,
Jens
--
Jens Remus
Linux on Z Development (D3303)
[email protected] / [email protected]

IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats: Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft: Ehningen; Registergericht: Amtsgericht Stuttgart, HRB 243294 IBM Data Privacy Statement: https://www.ibm.com/privacy/

author
#4 lore raw
Quoted 24 lines · Ian Rogers
Quoted 13 lines · Jens Remus
Quoted 10 lines · Jens Remus

Instead, warn on s390 when callchain sampling is requested with an event provided by the CPUMSF hardware PMU, because that combination is not supported.

tools/perf/util/evsel.c +0 -0
@@ -1078,9 +1078,8 @@ static void __evsel__config_callchain(struct evsel *evsel, const struct record_o
      bool function = evsel__is_function_event(evsel);
      struct perf_event_attr *attr = &evsel->core.attr;
 
Quoted 9 lines

tools/perf/util/evsel.c static void __evsel__config_callchain(struct evsel *evsel, const struct record_o · line 1082

Quoted 5 lines
tools/perf/util/evsel.c +2 -3
-     if (EM_HOST == EM_S390 && param->record_mode == CALLCHAIN_FP) {
-             pr_warning_once(
-                     "Framepointer unwinding lacks kernel support. Use '--call-graph dwarf'\n");
+     if (EM_HOST == EM_S390 && evsel->pmu && !perf_pmu__is_software(evsel->pmu)) {
+             pr_warning_once("Cannot use hardware CPUMSF event with -g and --call-graph.\n");

Jan, Thomas, could you please do me a favor and test whether this works as expected on LPAR with CPUMSF? Time was running out before I leave on vacation.

Quoted 1 line

Thomas, the change looks good to me. What are your thoughts?

Sent a v3 with updated warning message.

Quoted 4 lines
Quoted 4 lines

tools/perf/util/evsel.c static void __evsel__config_callchain(struct evsel *evsel, const struct record_o · line 1085

Quoted 4 lines
tools/perf/util/evsel.c +0 -0
      }
 
      evsel__set_sample_bit(evsel, CALLCHAIN);

Regards,
Jens

Signature · 6 lines
Jens Remus
Linux on Z Development (D3303)
[email protected] / [email protected]

IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats: Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft: Ehningen; Registergericht: Amtsgericht Stuttgart, HRB 243294
IBM Data Privacy Statement: https://www.ibm.com/privacy/