perf record: Implement off-cpu profiling with BPF (v2)
Subsystems (2)
Namhyung Kim
maintainercover perf record: Implement off-cpu profiling with BPF (v2)
Hello,
This is the second version of off-cpu profiling support. Together with (PMU-based) cpu profiling, it can show holistic view of the performance characteristics of your application or system.
Changes in v2)
* change sched_switch argument handling (Andrii)
* use task local storage (Hao)
* fix build error on !BUILD_BPF_SKEL (kernel test robot)
* add documentation regard fp callstack (Milian)
With BPF, it can aggregate scheduling stats for interested tasks and/or states and convert the data into a form of perf sample records. I chose the bpf-output event which is a software event supposed to be consumed by BPF programs and renamed it as "offcpu-time". So it requires no change on the perf report side except for setting sample types of bpf-output event.
Basically it collects userspace callstack for tasks as it's what users want mostly. Maybe we can add support for the kernel stacks but I'm afraid that it'd cause more overhead. So the offcpu-time event will always have callchains regardless of the command line option, and it enables the children mode in perf report by default.
It adds --off-cpu option to perf record like below:
$ sudo perf record -a --off-cpu -- perf bench sched messaging -l 1000 # Running 'sched/messaging' benchmark: # 20 sender and receiver processes per group # 10 groups == 400 processes run
Total time: 1.518 [sec]
[ perf record: Woken up 9 times to write data ]
[ perf record: Captured and wrote 5.313 MB perf.data (53341 samples) ]
Then we can run perf report as usual. The below is just to skip less important parts.
$ sudo perf report --stdio --call-graph=no --percent-limit=2
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 52K of event 'cycles'
# Event count (approx.): 42522453276
#
# Children Self Command Shared Object Symbol
# ........ ........ ............... ................ ..................................
#
9.58% 9.58% sched-messaging [kernel.vmlinux] [k] audit_filter_rules.constprop.0
8.46% 8.46% sched-messaging [kernel.vmlinux] [k] audit_filter_syscall
4.54% 4.54% sched-messaging [kernel.vmlinux] [k] copy_user_enhanced_fast_string
2.94% 2.94% sched-messaging [kernel.vmlinux] [k] unix_stream_read_generic
2.45% 2.45% sched-messaging [kernel.vmlinux] [k] memcg_slab_free_hook # Samples: 983 of event 'offcpu-time'
# Event count (approx.): 684538813464
#
# Children Self Command Shared Object Symbol
# ........ ........ ............... .................... ..........................
#
83.86% 0.00% sched-messaging libc-2.33.so [.] __libc_start_main
83.86% 0.00% sched-messaging perf [.] cmd_bench
83.86% 0.00% sched-messaging perf [.] main
83.86% 0.00% sched-messaging perf [.] run_builtin
83.64% 0.00% sched-messaging perf [.] bench_sched_messaging
41.35% 41.35% sched-messaging libpthread-2.33.so [.] __read
38.88% 38.88% sched-messaging libpthread-2.33.so [.] __write
3.41% 3.41% sched-messaging libc-2.33.so [.] __pollThe perf bench sched messaging created 400 processes to send/receive messages through unix sockets. It spent a large portion of cpu cycles for audit filter and read/copy the messages while most of the offcpu-time was in read and write calls.
You can get the code from 'perf/offcpu-v2' branch in my tree at
git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
Enjoy! :)
Thanks,
Namhyung
Namhyung Kim (4):
perf report: Do not extend sample type of bpf-output event perf record: Enable off-cpu analysis with BPF perf record: Implement basic filtering for off-cpu perf record: Handle argument change in sched_switch
tools/perf/Documentation/perf-record.txt | 10 + tools/perf/Makefile.perf | 1 + tools/perf/builtin-record.c | 21 ++ tools/perf/util/Build | 1 + tools/perf/util/bpf_off_cpu.c | 298 +++++++++++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 209 ++++++++++++++++ tools/perf/util/evsel.c | 4 +- tools/perf/util/off_cpu.h | 24 ++ 8 files changed, 566 insertions(+), 2 deletions(-) create mode 100644 tools/perf/util/bpf_off_cpu.c create mode 100644 tools/perf/util/bpf_skel/off_cpu.bpf.c create mode 100644 tools/perf/util/off_cpu.h
base-commit: 33cd6928039c6bf18cf0baec936924d908e6c89b
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
Namhyung Kim
maintainerPATCH 1/4 perf report: Do not extend sample type of bpf-output event
Currently evsel__new_idx() sets more sample_type bits when it finds a BPF-output event. But it should honor what's recorded in the perf data file rather than blindly sets the bits. Otherwise it could lead to a parse error when it recorded with a modified sample_type.
--- tools/perf/util/evsel.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/evsel.c +2 -2
@@ -269,8 +269,8 @@ struct evsel *evsel__new_idx(struct perf_event_attr *attr, int idx)return NULL ;evsel__init (evsel, attr, idx); -if (evsel__is_bpf_output (evsel)) { - evsel->core.attr.sample_type |= (PERF_SAMPLE_RAW |PERF_SAMPLE_TIME | +if (evsel__is_bpf_output (evsel) && !attr->sample_type) { + evsel->core.attr.sample_type = (PERF_SAMPLE_RAW |PERF_SAMPLE_TIME |PERF_SAMPLE_CPU |PERF_SAMPLE_PERIOD ), evsel->core.attr.sample_period =1 ; }
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 1/4] perf report: Do not extend sample type of bpf-output event
Em Fri, May 06, 2022 at 01:16:24PM -0700, Namhyung Kim escreveu:
Quoted 4 lines
Currently
evsel__new_idx()sets moresample_typebits when it finds a BPF-output event. But it should honor what's recorded in the perf data file rather than blindly sets the bits. Otherwise it could lead to a parse error when it recorded with a modifiedsample_type.
Can you please try to provide a Fixes: tag for this? This way reviewing gets simpler by looking at who introduced this, if there was some reason or if it was a plain oversight.
It also helps to get this fix propabated to the stable trees.
- Arnaldo
Quoted 22 lines
Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/evsel.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/perf/util/evsel.c +2
-2@@ -269,8 +269,8 @@ struct evsel *evsel__new_idx(struct perf_event_attr *attr, int idx)return NULL ;evsel__init (evsel, attr, idx); -if (evsel__is_bpf_output (evsel)) { - evsel->core.attr.sample_type |= (PERF_SAMPLE_RAW |PERF_SAMPLE_TIME | +if (evsel__is_bpf_output (evsel) && !attr->sample_type) { + evsel->core.attr.sample_type = (PERF_SAMPLE_RAW |PERF_SAMPLE_TIME |PERF_SAMPLE_CPU |PERF_SAMPLE_PERIOD ), evsel->core.attr.sample_period =1 ; }Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
Signature · 1 line
- Arnaldo
Namhyung Kim
maintainerRe: [PATCH 1/4] perf report: Do not extend sample type of bpf-output event
Hi Arnaldo,
Quoted 7 lines · Arnaldo Carvalho de Melo
Em Fri, May 06, 2022 at 01:16:24PM -0700, Namhyung Kim escreveu:
Quoted 4 lines
Currently
evsel__new_idx()sets moresample_typebits when it finds a BPF-output event. But it should honor what's recorded in the perf data file rather than blindly sets the bits. Otherwise it could lead to a parse error when it recorded with a modifiedsample_type.
Quoted 5 lines
Can you please try to provide a Fixes: tag for this? This way reviewing gets simpler by looking at who introduced this, if there was some reason or if it was a plain oversight.
It also helps to get this fix propabated to the stable trees.
Well.. actually this is not a fix. I've realized it adds some sample types when it creates a new evsel regardless of the perf_event_attr.
This was not a problem so far (as nobody touched it), but when I changed some sample types during record for this change, perf report sees a different value and rejects the data.
Thanks,
namhyung
Namhyung Kim
maintainerPATCH 2/4 perf record: Enable off-cpu analysis with BPF
In reply to Namhyung KimAdd --off-cpu option to enable the off-cpu profiling with BPF. It'd use a bpf_output event and rename it to "offcpu-time". Samples will be synthesized at the end of the record session using data from a BPF map which contains the aggregated off-cpu time at context switches. So it needs root privilege to get the off-cpu profiling.
Each sample will have a separate user stacktrace so it will skip kernel threads. The sample ip will be set from the stacktrace and other sample data will be updated accordingly. Currently it only handles some basic sample types.
The sample timestamp is set to a dummy value just not to bother with other events during the sorting. So it has a very big initial value and increase it on processing each samples.
Good thing is that it can be used together with regular profiling like cpu cycles. If you don't want to that, you can use a dummy event to enable off-cpu profiling only.
Example output:
$ sudo perf record --off-cpu perf bench sched messaging -l 1000
$ sudo perf report --stdio --call-graph=no # Total Lost Samples: 0 # # Samples: 41K of event 'cycles' # Event count (approx.): 42137343851 ...
# Samples: 1K of event 'offcpu-time'
# Event count (approx.): 587990831640
#
# Children Self Command Shared Object Symbol
# ........ ........ ............... .................. .........................
#
81.66% 0.00% sched-messaging libc-2.33.so [.] __libc_start_main
81.66% 0.00% sched-messaging perf [.] cmd_bench
81.66% 0.00% sched-messaging perf [.] main
81.66% 0.00% sched-messaging perf [.] run_builtin
81.43% 0.00% sched-messaging perf [.] bench_sched_messaging
40.86% 40.86% sched-messaging libpthread-2.33.so [.] __read
37.66% 37.66% sched-messaging libpthread-2.33.so [.] __write
2.91% 2.91% sched-messaging libc-2.33.so [.] __poll...
As you can see it spent most of off-cpu time in read and write in bench_sched_messaging(). The --call-graph=no was added just to make the output concise here.
It uses perf hooks facility to control BPF program during the record session rather than adding new BPF/off-cpu specific calls.
--- tools/perf/Documentation/perf-record.txt | 10 ++ tools/perf/Makefile.perf | 1 + tools/perf/builtin-record.c | 21 +++ tools/perf/util/Build | 1 + tools/perf/util/bpf_off_cpu.c | 208 +++++++++++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 139 +++++++++++++++ tools/perf/util/off_cpu.h | 22 +++ 7 files changed, 402 insertions(+) create mode 100644 tools/perf/util/bpf_off_cpu.c create mode 100644 tools/perf/util/bpf_skel/off_cpu.bpf.c create mode 100644 tools/perf/util/off_cpu.h
tools/perf/Documentation/perf-record.txt +10 -0
@@ -758,6 +758,16 @@ include::intel-hybrid.txt[] If the URLs is not specified, the value of DEBUGINFOD_URLS system environment variable is used. +--off-cpu:: + Enable off-cpu profiling with BPF. The BPF program will collect + task scheduling information with (user) stacktrace and save them + as sample data of a software event named "offcpu-time". The + sample period will have the time the task slept in nano-second. + + Note that BPF can collect stacktrace using frame pointer ("fp") + only, as of now. So the applications built without the frame + pointer might see bogus addresses. + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1], linkperf:perf-intel-pt[1]
tools/perf/Makefile.perf +1 -0
@@ -1038,6 +1038,7 @@ SKEL_TMP_OUT := $(abspath $(SKEL_OUT)/.tmp) SKELETONS := $(SKEL_OUT)/bpf_prog_profiler.skel.h SKELETONS += $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h +SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_TMP_OUT) $(LIBBPF_OUTPUT): $(Q)$(MKDIR) -p $@
tools/perf/builtin-record.c +21 -0
@@ -49,6 +49,7 @@#include "util/clockid.h" #include "util/pmu-hybrid.h" #include "util/evlist-hybrid.h" +#include "util/off_cpu.h" #include "asm/bug.h" #include "perf.h" #include "cputopo.h"
@@ -162,6 +163,7 @@ struct record {bool buildid_mmap;bool timestamp_filename;bool timestamp_boundary; +bool off_cpu;struct switch_output switch_output;unsigned long long samples;unsigned long output_max_size;/* = 0: unlimited */
@@ -903,6 +905,11 @@ static int record__config_text_poke(struct evlist *evlist)return 0 ; } +static int record__config_off_cpu (struct record *rec) +{ +return off_cpu_prepare (rec->evlist); +} +static bool record__kcore_readable (struct machine *machine) {char kcore[PATH_MAX ];
@@ -2600,6 +2607,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) }else status = err; +if (rec->off_cpu) + rec->bytes_written +=off_cpu_write (rec->session); +record__synthesize (rec, true);/* this will be recalculated during process_buildids() */ rec->samples =0 ;
@@ -3324,6 +3334,9 @@ static struct option __record_options[] = { OPT_CALLBACK_OPTARG(0, "threads", &record.opts, NULL, "spec", "write collected trace data into several data files using parallel threads", record__parse_threads), +#ifdef HAVE_BPF_SKEL +OPT_BOOLEAN (0 ,"off-cpu" , &record.off_cpu,"Enable off-cpu analysis" ), +#endifOPT_END () };
@@ -3981,6 +3994,14 @@ int cmd_record(int argc, const char **argv) } } +if (rec->off_cpu) { + err =record__config_off_cpu (rec); +if (err) { +pr_err ("record__config_off_cpu failed, error %d\n" , err); + goto out; + } + } +if (record_opts__config (&rec->opts)) { err = -EINVAL ; goto out;
tools/perf/util/Build +1 -0
@@ -147,6 +147,7 @@ perf-$(CONFIG_LIBBPF) += bpf_map.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter_cgroup.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_ftrace.o +perf-$(CONFIG_PERF_BPF_SKEL) += bpf_off_cpu.o perf-$(CONFIG_BPF_PROLOGUE) += bpf-prologue.o perf-$(CONFIG_LIBELF) += symbol-elf.o perf-$(CONFIG_LIBELF) += probe-file.o
tools/perf/util/bpf_off_cpu.c new file +208 -0
208 lines not on this page · raw message
tools/perf/util/bpf_skel/off_cpu.bpf.c new file +139 -0
139 lines not on this page · raw message
tools/perf/util/off_cpu.h new file +22 -0
@@ -0,0 +1,22 @@ +#ifndef PERF_UTIL_OFF_CPU_H +#define PERF_UTIL_OFF_CPU_H + +struct evlist ; +struct perf_session ; + +#ifdef HAVE_BPF_SKEL +int off_cpu_prepare (struct evlist *evlist); +int off_cpu_write (struct perf_session *session); +#else +static inline int off_cpu_prepare (struct evlist *evlist__maybe_unused ) +{ +return -1 ; +} + +static inline int off_cpu_write (struct perf_session *session__maybe_unused ) +{ +return -1 ; +} +#endif + +#endif /* PERF_UTIL_OFF_CPU_H */
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 2/4] perf record: Enable off-cpu analysis with BPF
Em Fri, May 06, 2022 at 01:16:25PM -0700, Namhyung Kim escreveu:
Quoted 71 lines
Add --off-cpu option to enable the off-cpu profiling with BPF. It'd use a
bpf_outputevent and rename it to "offcpu-time". Samples will be synthesized at the end of the record session using data from a BPF map which contains the aggregated off-cpu time at context switches. So it needs root privilege to get the off-cpu profiling.Each sample will have a separate user stacktrace so it will skip kernel threads. The sample ip will be set from the stacktrace and other sample data will be updated accordingly. Currently it only handles some basic sample types.
The sample timestamp is set to a dummy value just not to bother with other events during the sorting. So it has a very big initial value and increase it on processing each samples.
Good thing is that it can be used together with regular profiling like cpu cycles. If you don't want to that, you can use a dummy event to enable off-cpu profiling only.
Example output:
$ sudo perf record --off-cpu perf bench sched messaging -l 1000$ sudo perf report --stdio --call-graph=no # Total Lost Samples: 0 # # Samples: 41K of event 'cycles' # Event count (approx.): 42137343851 ...# Samples: 1K of event 'offcpu-time'
# Event count (approx.): 587990831640
## Children Self Command Shared Object Symbol # ........ ........ ............... .................. ......................... # 81.66% 0.00% sched-messaging libc-2.33.so [.] __libc_start_main 81.66% 0.00% sched-messaging perf [.] cmd_bench 81.66% 0.00% sched-messaging perf [.] main 81.66% 0.00% sched-messaging perf [.] run_builtin 81.43% 0.00% sched-messaging perf [.] bench_sched_messaging 40.86% 40.86% sched-messaging libpthread-2.33.so [.] __read 37.66% 37.66% sched-messaging libpthread-2.33.so [.] __write 2.91% 2.91% sched-messaging libc-2.33.so [.] __poll...
As you can see it spent most of off-cpu time in read and write in
bench_sched_messaging(). The --call-graph=no was added just to make the output concise here.It uses perf hooks facility to control BPF program during the record session rather than adding new BPF/off-cpu specific calls.
Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/Documentation/perf-record.txt | 10 ++ tools/perf/Makefile.perf | 1 + tools/perf/builtin-record.c | 21 +++ tools/perf/util/Build | 1 + tools/perf/util/bpf_off_cpu.c | 208 +++++++++++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 139 +++++++++++++++ tools/perf/util/off_cpu.h | 22 +++ 7 files changed, 402 insertions(+) create mode 100644 tools/perf/util/bpf_off_cpu.c create mode 100644 tools/perf/util/bpf_skel/off_cpu.bpf.c create mode 100644 tools/perf/util/off_cpu.htools/perf/Documentation/perf-record.txt +0
-0
tools/perf/Documentation/perf-record.txt include::intel-hybrid.txt[] · line 765
Quoted 8 lines
tools/perf/Documentation/perf-record.txt +5
-0@@ -758,6 +758,16 @@ include::intel-hybrid.txt[] If the URLs is not specified, the value of DEBUGINFOD_URLS system environment variable is used. +--off-cpu:: + Enable off-cpu profiling with BPF. The BPF program will collect + task scheduling information with (user) stacktrace and save them + as sample data of a software event named "offcpu-time". The + sample period will have the time the task slept in nano-second.
nanoseconds.
tools/perf/Documentation/perf-record.txt include::intel-hybrid.txt[] · line 767
Quoted 2 lines
tools/perf/Documentation/perf-record.txt +2
-0+ + Note that BPF can collect stacktrace using frame pointer ("fp")
stack traces
tools/perf/Documentation/perf-record.txt include::intel-hybrid.txt[] · line 769
Quoted 2 lines
tools/perf/Documentation/perf-record.txt +2
-0+ only, as of now. So the applications built without the frame + pointer might see bogus addresses.
One possible improvement is to check if debugging info is available in the traced program and check if -fnoomit-frame-pointer is present:
⬢[acme@toolbox perf]$ readelf -wi ~/bin/perf | grep DW_AT_producer -m1
<d> DW_AT_producer : (indirect string, offset: 0x6627): GNU C99 11.2.1 20220127 (Red Hat 11.2.1-9) -mtune=generic -march=x86-64 -ggdb3 -O6 -std=gnu99 -fno-omit-frame-pointer -funwind-tables -fstack-protector-all
⬢[acme@toolbox perf]$
If debugging info is available and -fno-omit-frame-pointer isn't in the DW_AT_producer string, then we can print a flashing warning and even don't use the stack traces.
That or some other more robust method to detect that frame pointers are valid.
Some more comments below.
Quoted 59 lines
tools/perf/Documentation/perf-record.txt +1
-0+ SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1], linkperf:perf-intel-pt[1]tools/perf/Makefile.perf +1
-0@@ -1038,6 +1038,7 @@ SKEL_TMP_OUT := $(abspath $(SKEL_OUT)/.tmp) SKELETONS := $(SKEL_OUT)/bpf_prog_profiler.skel.h SKELETONS += $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h +SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_TMP_OUT) $(LIBBPF_OUTPUT): $(Q)$(MKDIR) -p $@tools/perf/builtin-record.c +10
-0@@ -49,6 +49,7 @@#include "util/clockid.h" #include "util/pmu-hybrid.h" #include "util/evlist-hybrid.h" +#include "util/off_cpu.h" #include "asm/bug.h" #include "perf.h" #include "cputopo.h" @@ -162,6 +163,7 @@ struct record {bool buildid_mmap;bool timestamp_filename;bool timestamp_boundary; +bool off_cpu;struct switch_output switch_output;unsigned long long samples;unsigned long output_max_size;/* = 0: unlimited */ @@ -903,6 +905,11 @@ static int record__config_text_poke(struct evlist *evlist)return 0 ; } +static int record__config_off_cpu (struct record *rec) +{ +return off_cpu_prepare (rec->evlist); +} +static bool record__kcore_readable (struct machine *machine) {char kcore[PATH_MAX ];@@ -2600,6 +2607,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) }else status = err; +if (rec->off_cpu) + rec->bytes_written +=off_cpu_write (rec->session); +record__synthesize (rec, true);/* this will be recalculated during process_buildids() */ rec->samples =0 ;
tools/perf/builtin-record.c static struct option __record_options[] = { · line 3339
Quoted 6 lines
tools/perf/builtin-record.c +3
-0@@ -3324,6 +3334,9 @@ static struct option __record_options[] = { OPT_CALLBACK_OPTARG(0, "threads", &record.opts, NULL, "spec", "write collected trace data into several data files using parallel threads", record__parse_threads), +#ifdef HAVE_BPF_SKEL + OPT_BOOLEAN(0, "off-cpu", &record.off_cpu, "Enable off-cpu analysis"), +#endif
Since this is in the documentation, I think we should have an #else clause and state that --off-cpu needs building with some specific make command line.
Quoted 81 lines
tools/perf/builtin-record.c +8
-0OPT_END() };@@ -3981,6 +3994,14 @@ int cmd_record(int argc, const char **argv) } } +if (rec->off_cpu) { + err =record__config_off_cpu (rec); +if (err) { +pr_err ("record__config_off_cpu failed, error %d\n" , err); + goto out; + } + } +if (record_opts__config (&rec->opts)) { err = -EINVAL ; goto out;tools/perf/util/Build +1
-0@@ -147,6 +147,7 @@ perf-$(CONFIG_LIBBPF) += bpf_map.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter_cgroup.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_ftrace.o +perf-$(CONFIG_PERF_BPF_SKEL) += bpf_off_cpu.o perf-$(CONFIG_BPF_PROLOGUE) += bpf-prologue.o perf-$(CONFIG_LIBELF) += symbol-elf.o perf-$(CONFIG_LIBELF) += probe-file.otools/perf/util/bpf_off_cpu.c new file +45
-0@@ -0,0 +1,208 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "util/bpf_counter.h" +#include "util/debug.h" +#include "util/evsel.h" +#include "util/evlist.h" +#include "util/off_cpu.h" +#include "util/perf-hooks.h" +#include "util/session.h" +#include <bpf/bpf.h> + +#include "bpf_skel/off_cpu.skel.h" + +#define MAX_STACKS 32 +/* we don't need actual timestamp, just want to put the samples at last */ +#define OFF_CPU_TIMESTAMP (~0ull << 32) + +static struct off_cpu_bpf *skel; + +struct off_cpu_key { +u32 pid; +u32 tgid; +u32 stack_id; +u32 state; +u64 cgroup_id; +}; + +union off_cpu_data { +struct perf_event_header hdr; +u64 array[1024 /sizeof (u64)]; +}; + +static int off_cpu_config (struct evlist *evlist) +{ +struct evsel *evsel; +struct perf_event_attr attr = { + .type =PERF_TYPE_SOFTWARE , + .config =PERF_COUNT_SW_BPF_OUTPUT , + .size =sizeof (attr),/* to capture ABI version */ + }; + + evsel =evsel__new (&attr); +if (!evsel) +return -ENOMEM ; + + evsel->core.attr.freq =1 ;
tools/perf/util/bpf_off_cpu.c line 55
Quoted 10 lines
tools/perf/util/bpf_off_cpu.c new file +10
-0+ evsel->core.attr.sample_period =1 ; +/* off-cpu analysis depends on stack trace */ + evsel->core.attr.sample_type =PERF_SAMPLE_CALLCHAIN ; + +evlist__add (evlist, evsel); + +free (evsel->name); + evsel->name =strdup ("offcpu-time" ); +if (evsel->name ==NULL ) +return -ENOMEM ;
But at this point the evsel was left in the evlist? Shouldn't we start it all by trying to allocate this string and don't bother to call evsel__new() on failure, etc?
Quoted 19 lines
tools/perf/util/bpf_off_cpu.c new file +19
-0+ +return 0 ; +} + +static void off_cpu_start (void *arg__maybe_unused ) +{ + skel->bss->enabled =1 ; +} + +static void off_cpu_finish (void *arg__maybe_unused ) +{ + skel->bss->enabled =0 ; +off_cpu_bpf__destroy (skel); +} + +int off_cpu_prepare (struct evlist *evlist) +{ +int err; +
tools/perf/util/bpf_off_cpu.c line 84
Quoted 10 lines
tools/perf/util/bpf_off_cpu.c new file +10
-0+if (off_cpu_config (evlist) <0 ) { +pr_err ("Failed to config off-cpu BPF event\n" ); +return -1 ; + } + +set_max_rlimit (); + + skel =off_cpu_bpf__open_and_load (); +if (!skel) { +pr_err ("Failed to open off-cpu skeleton\n" );
"BPF sleketon", to give a bit more context to users?
tools/perf/util/bpf_off_cpu.c line 90
Quoted 6 lines
tools/perf/util/bpf_off_cpu.c new file +6
-0+return -1 ; + } + + err =off_cpu_bpf__attach (skel); +if (err) { +pr_err ("Failed to attach off-cpu skeleton\n" );
Ditto.
Quoted 31 lines
tools/perf/util/bpf_off_cpu.c new file +31
-0+ goto out; + } + +if (perf_hooks__set_hook ("record_start" , off_cpu_start,NULL ) || +perf_hooks__set_hook ("record_end" , off_cpu_finish,NULL )) { +pr_err ("Failed to attach off-cpu skeleton\n" ); + goto out; + } + +return 0 ; + +out: +off_cpu_bpf__destroy (skel); +return -1 ; +} + +int off_cpu_write (struct perf_session *session) +{ +int bytes =0 , size; +int fd, stack; +bool found = false; +u64 sample_type, val, sid =0 ; +struct evsel *evsel; +struct perf_data_file *file = &session->data->file; +struct off_cpu_key prev, key; +union off_cpu_data data = { + .hdr = { + .type =PERF_RECORD_SAMPLE , + .misc =PERF_RECORD_MISC_USER , + }, + };
tools/perf/util/bpf_off_cpu.c line 131
Quoted 10 lines
tools/perf/util/bpf_off_cpu.c new file +10
-0+u64 tstamp =OFF_CPU_TIMESTAMP ; + + skel->bss->enabled =0 ; + +evlist__for_each_entry (session->evlist, evsel) { +if (!strcmp (evsel__name (evsel),"offcpu-time" )) { + found = true; +break ; + } + }
Don't we have some evlist__find_evsel function?
struct evsel *evlist__find_evsel_by_str(struct evlist *evlist, const char *str)
{
struct evsel *evsel;
evlist__for_each_entry(evlist, evsel) {
if (!evsel->name)
continue;
if (strcmp(str, evsel->name) == 0)
return evsel;
}
return NULL;
}Quoted 252 lines
tools/perf/util/bpf_off_cpu.c new file +77
-0+ +if (!found) { +pr_err ("offcpu-time evsel not found\n" ); +return 0 ; + } + + sample_type = evsel->core.attr.sample_type; + +if (sample_type & (PERF_SAMPLE_ID |PERF_SAMPLE_IDENTIFIER )) { +if (evsel->core.id) + sid = evsel->core.id[0 ]; + } + + fd =bpf_map__fd (skel->maps.off_cpu); + stack =bpf_map__fd (skel->maps.stacks); +memset (&prev,0 ,sizeof (prev)); + +while (!bpf_map_get_next_key (fd, &prev, &key)) { +int n =1 ;/* start from perf_event_header */ +int ip_pos =-1 ; + +bpf_map_lookup_elem (fd, &key, &val); + +if (sample_type &PERF_SAMPLE_IDENTIFIER ) + data.array[n++] = sid; +if (sample_type &PERF_SAMPLE_IP ) { + ip_pos = n; + data.array[n++] =0 ;/* will be updated */ + } +if (sample_type &PERF_SAMPLE_TID ) + data.array[n++] = (u64 )key.pid <<32 | key.tgid; +if (sample_type &PERF_SAMPLE_TIME ) + data.array[n++] = tstamp; +if (sample_type &PERF_SAMPLE_ID ) + data.array[n++] = sid; +if (sample_type &PERF_SAMPLE_CPU ) + data.array[n++] =0 ; +if (sample_type &PERF_SAMPLE_PERIOD ) + data.array[n++] = val; +if (sample_type &PERF_SAMPLE_CALLCHAIN ) { +int len =0 ; + +/* data.array[n] is callchain->nr (updated later) */ + data.array[n +1 ] =PERF_CONTEXT_USER ; + data.array[n +2 ] =0 ; + +bpf_map_lookup_elem (stack, &key.stack_id, &data.array[n +2 ]); +while (data.array[n +2 + len]) + len++; + +/* update length of callchain */ + data.array[n] = len +1 ; + +/* update sample ip with the first callchain entry */ +if (ip_pos >=0 ) + data.array[ip_pos] = data.array[n +2 ]; + +/* calculate sample callchain data array length */ + n += len +2 ; + } +/* TODO: handle more sample types */ + + size = n *sizeof (u64); + data.hdr.size = size; + bytes += size; + +if (perf_data_file__write (file, &data, size) <0 ) { +pr_err ("failed to write perf data, error: %m\n" ); +return bytes; + } + + prev = key; +/* increase dummy timestamp to sort later samples */ + tstamp++; + } +return bytes; +}tools/perf/util/bpf_skel/off_cpu.bpf.c new file +139
-0@@ -0,0 +1,139 @@ +// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +// Copyright (c) 2022 Google +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> +#include <bpf/bpf_core_read.h> + +/* task->flags for off-cpu analysis */ +#define PF_KTHREAD 0x00200000/* I am a kernel thread */ + +/* task->state for off-cpu analysis */ +#define TASK_INTERRUPTIBLE 0x0001 +#define TASK_UNINTERRUPTIBLE 0x0002 + +#define MAX_STACKS 32 +#define MAX_ENTRIES 102400 + +struct tstamp_data { +__u32 stack_id; +__u32 state; +__u64 timestamp; +}; + +struct offcpu_key { +__u32 pid; +__u32 tgid; +__u32 stack_id; +__u32 state; +}; + +struct { + __uint(type, BPF_MAP_TYPE_STACK_TRACE); + __uint(key_size, sizeof(__u32)); + __uint(value_size, MAX_STACKS * sizeof(__u64)); + __uint(max_entries, MAX_ENTRIES); +} stacks SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_TASK_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, struct tstamp_data); +} tstamp SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(key_size, sizeof(struct offcpu_key)); + __uint(value_size, sizeof(__u64)); + __uint(max_entries, MAX_ENTRIES); +} off_cpu SEC(".maps"); + +/* old kernel task_struct definition */ +struct task_struct___old { +long state; +} __attribute__((preserve_access_index)); + +int enabled =0 ; + +/* +* Old kernel used to call it task_struct->state and now it's '__state'. +* Use BPF CO-RE "ignored suffix rule" to deal with it like below: +* +* https://nakryiko.com/posts/bpf-core-reference-guide/#handling-incompatible-field-and-type-changes +*/ +static inline int get_task_state (struct task_struct *t) +{ +if (bpf_core_field_exists (t->__state)) +return BPF_CORE_READ (t, __state); + +/* recast pointer to capture task_struct___old type for compiler */ +struct task_struct___old *t_old = (void *)t; + +/* now use old "state" name of the field */ +return BPF_CORE_READ (t_old, state); +} + +SEC("tp_btf/sched_switch") +int on_switch (u64 *ctx) +{ +__u64 ts; +int state; +__u32 stack_id; +struct task_struct *prev, *next; +struct tstamp_data *pelem; + +if (!enabled) +return 0 ; + + prev = (struct task_struct *)ctx[1 ]; + next = (struct task_struct *)ctx[2 ]; + state =get_task_state (prev); + + ts =bpf_ktime_get_ns (); + +if (prev->flags &PF_KTHREAD ) + goto next; +if (state !=TASK_INTERRUPTIBLE && + state !=TASK_UNINTERRUPTIBLE ) + goto next; + + stack_id =bpf_get_stackid (ctx, &stacks, +BPF_F_FAST_STACK_CMP |BPF_F_USER_STACK ); + + pelem =bpf_task_storage_get (&tstamp, prev,NULL , +BPF_LOCAL_STORAGE_GET_F_CREATE ); +if (!pelem) + goto next; + + pelem->timestamp = ts; + pelem->state = state; + pelem->stack_id = stack_id; + +next: + pelem =bpf_task_storage_get (&tstamp, next,NULL ,0 ); + +if (pelem && pelem->timestamp) { +struct offcpu_key key = { + .pid = next->pid, + .tgid = next->tgid, + .stack_id = pelem->stack_id, + .state = pelem->state, + }; +__u64 delta = ts - pelem->timestamp; +__u64 *total; + + total =bpf_map_lookup_elem (&off_cpu, &key); +if (total) + *total += delta; +else +bpf_map_update_elem (&off_cpu, &key, &delta,BPF_ANY ); + +/* prevent to reuse the timestamp later */ + pelem->timestamp =0 ; + } + +return 0 ; +} + +char LICENSE[] SEC("license") = "Dual BSD/GPL";tools/perf/util/off_cpu.h new file +22
-0@@ -0,0 +1,22 @@ +#ifndef PERF_UTIL_OFF_CPU_H +#define PERF_UTIL_OFF_CPU_H + +struct evlist ; +struct perf_session ; + +#ifdef HAVE_BPF_SKEL +int off_cpu_prepare (struct evlist *evlist); +int off_cpu_write (struct perf_session *session); +#else +static inline int off_cpu_prepare (struct evlist *evlist__maybe_unused ) +{ +return -1 ; +} + +static inline int off_cpu_write (struct perf_session *session__maybe_unused ) +{ +return -1 ; +} +#endif + +#endif /* PERF_UTIL_OFF_CPU_H */ Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
Signature · 1 line
- Arnaldo
Namhyung Kim
maintainerRe: [PATCH 2/4] perf record: Enable off-cpu analysis with BPF
Quoted 2 lines · Arnaldo Carvalho de Melo
Em Fri, May 06, 2022 at 01:16:25PM -0700, Namhyung Kim escreveu:
[SNIP]
Quoted 8 lines
Quoted 8 lines
tools/perf/Documentation/perf-record.txt +0
-0@@ -758,6 +758,16 @@ include::intel-hybrid.txt[] If the URLs is not specified, the value of DEBUGINFOD_URLS system environment variable is used.
Quoted 7 lines
tools/perf/Documentation/perf-record.txt include::intel-hybrid.txt[] · line 765
Quoted 5 lines
tools/perf/Documentation/perf-record.txt +5
-0+--off-cpu:: + Enable off-cpu profiling with BPF. The BPF program will collect + task scheduling information with (user) stacktrace and save them + as sample data of a software event named "offcpu-time". The + sample period will have the time the task slept in nano-second.nanoseconds.
ok
Quoted 4 lines
tools/perf/Documentation/perf-record.txt include::intel-hybrid.txt[] · line 767
Quoted 2 lines
tools/perf/Documentation/perf-record.txt +2
-0+ + Note that BPF can collect stacktrace using frame pointer ("fp")stack traces
ok
Quoted 7 lines
tools/perf/Documentation/perf-record.txt include::intel-hybrid.txt[] · line 769
Quoted 3 lines
tools/perf/Documentation/perf-record.txt +2
-0+ only, as of now. So the applications built without the frame + pointer might see bogus addresses.One possible improvement is to check if debugging info is available in the traced program and check if -fnoomit-frame-pointer is present:
Quoted 7 lines
⬢[acme@toolbox perf]$ readelf -wi ~/bin/perf | grep
DW_AT_producer-m1<d> DW_AT_producer : (indirect string, offset: 0x6627): GNU C99 11.2.1 20220127 (Red Hat 11.2.1-9) -mtune=generic -march=x86-64 -ggdb3 -O6 -std=gnu99 -fno-omit-frame-pointer -funwind-tables -fstack-protector-all⬢[acme@toolbox perf]$
If debugging info is available and -fno-omit-frame-pointer isn't in the
DW_AT_producerstring, then we can print a flashing warning and even don't use the stack traces.
Yeah, but checking the binary only would not be enough as it will call other libraries. Also the libc would be the most frequent user of any sleeping syscalls, I guess.
Quoted 68 lines
That or some other more robust method to detect that frame pointers are valid.
Some more comments below.
Quoted 62 lines
tools/perf/Documentation/perf-record.txt +1
-0+ SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1], linkperf:perf-intel-pt[1]tools/perf/Makefile.perf +1
-0@@ -1038,6 +1038,7 @@ SKEL_TMP_OUT := $(abspath $(SKEL_OUT)/.tmp) SKELETONS := $(SKEL_OUT)/bpf_prog_profiler.skel.h SKELETONS += $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h +SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_TMP_OUT) $(LIBBPF_OUTPUT): $(Q)$(MKDIR) -p $@tools/perf/builtin-record.c +10
-0@@ -49,6 +49,7 @@#include "util/clockid.h" #include "util/pmu-hybrid.h" #include "util/evlist-hybrid.h" +#include "util/off_cpu.h" #include "asm/bug.h" #include "perf.h" #include "cputopo.h" @@ -162,6 +163,7 @@ struct record {bool buildid_mmap;bool timestamp_filename;bool timestamp_boundary; +bool off_cpu;struct switch_output switch_output;unsigned long long samples;unsigned long output_max_size;/* = 0: unlimited */ @@ -903,6 +905,11 @@ static int record__config_text_poke(struct evlist *evlist)return 0 ; } +static int record__config_off_cpu (struct record *rec) +{ +return off_cpu_prepare (rec->evlist); +} +static bool record__kcore_readable (struct machine *machine) {char kcore[PATH_MAX ];@@ -2600,6 +2607,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) }else status = err; +if (rec->off_cpu) + rec->bytes_written +=off_cpu_write (rec->session); +record__synthesize (rec, true);/* this will be recalculated during process_buildids() */ rec->samples =0 ;@@ -3324,6 +3334,9 @@ static struct option __record_options[] = {OPT_CALLBACK_OPTARG (0 ,"threads" , &record.opts,NULL ,"spec" ,"write collected trace data into several data files using parallel threads" , record__parse_threads),
Quoted 8 lines
tools/perf/builtin-record.c static struct option __record_options[] = { · line 3339
Quoted 3 lines
tools/perf/builtin-record.c +3
-0+#ifdef HAVE_BPF_SKEL + OPT_BOOLEAN(0, "off-cpu", &record.off_cpu, "Enable off-cpu analysis"), +#endif Since this is in the documentation, I think we should have an #else clause and state that --off-cpu needs building with some specific make command line.
I'd rather remove the #ifdef here and put it after parse_options to show the warning.
Quoted 88 lines
Quoted 88 lines
tools/perf/builtin-record.c +8
-0OPT_END() };@@ -3981,6 +3994,14 @@ int cmd_record(int argc, const char **argv) } } +if (rec->off_cpu) { + err =record__config_off_cpu (rec); +if (err) { +pr_err ("record__config_off_cpu failed, error %d\n" , err); + goto out; + } + } +if (record_opts__config (&rec->opts)) { err = -EINVAL ; goto out;tools/perf/util/Build +1
-0@@ -147,6 +147,7 @@ perf-$(CONFIG_LIBBPF) += bpf_map.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter_cgroup.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_ftrace.o +perf-$(CONFIG_PERF_BPF_SKEL) += bpf_off_cpu.o perf-$(CONFIG_BPF_PROLOGUE) += bpf-prologue.o perf-$(CONFIG_LIBELF) += symbol-elf.o perf-$(CONFIG_LIBELF) += probe-file.otools/perf/util/bpf_off_cpu.c new file +51
-0@@ -0,0 +1,208 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "util/bpf_counter.h" +#include "util/debug.h" +#include "util/evsel.h" +#include "util/evlist.h" +#include "util/off_cpu.h" +#include "util/perf-hooks.h" +#include "util/session.h" +#include <bpf/bpf.h> + +#include "bpf_skel/off_cpu.skel.h" + +#define MAX_STACKS 32 +/* we don't need actual timestamp, just want to put the samples at last */ +#define OFF_CPU_TIMESTAMP (~0ull << 32) + +static struct off_cpu_bpf *skel; + +struct off_cpu_key { +u32 pid; +u32 tgid; +u32 stack_id; +u32 state; +u64 cgroup_id; +}; + +union off_cpu_data { +struct perf_event_header hdr; +u64 array[1024 /sizeof (u64)]; +}; + +static int off_cpu_config (struct evlist *evlist) +{ +struct evsel *evsel; +struct perf_event_attr attr = { + .type =PERF_TYPE_SOFTWARE , + .config =PERF_COUNT_SW_BPF_OUTPUT , + .size =sizeof (attr),/* to capture ABI version */ + }; + + evsel =evsel__new (&attr); +if (!evsel) +return -ENOMEM ; + + evsel->core.attr.freq =1 ; + evsel->core.attr.sample_period =1 ; +/* off-cpu analysis depends on stack trace */ + evsel->core.attr.sample_type =PERF_SAMPLE_CALLCHAIN ; + +evlist__add (evlist, evsel); +
Quoted 8 lines
tools/perf/util/bpf_off_cpu.c line 55
Quoted 4 lines
tools/perf/util/bpf_off_cpu.c new file +4
-0+free (evsel->name); + evsel->name =strdup ("offcpu-time" ); +if (evsel->name ==NULL ) +return -ENOMEM ;But at this point the evsel was left in the evlist? Shouldn't we start it all by trying to allocate this string and don't bother to call
evsel__new()on failure, etc?
Sounds good, will change.
Quoted 24 lines
Quoted 24 lines
tools/perf/util/bpf_off_cpu.c new file +23
-0+ +return 0 ; +} + +static void off_cpu_start (void *arg__maybe_unused ) +{ + skel->bss->enabled =1 ; +} + +static void off_cpu_finish (void *arg__maybe_unused ) +{ + skel->bss->enabled =0 ; +off_cpu_bpf__destroy (skel); +} + +int off_cpu_prepare (struct evlist *evlist) +{ +int err; + +if (off_cpu_config (evlist) <0 ) { +pr_err ("Failed to config off-cpu BPF event\n" ); +return -1 ; + }
Quoted 8 lines
tools/perf/util/bpf_off_cpu.c line 84
Quoted 6 lines
tools/perf/util/bpf_off_cpu.c new file +6
-0+ +set_max_rlimit (); + + skel =off_cpu_bpf__open_and_load (); +if (!skel) { +pr_err ("Failed to open off-cpu skeleton\n" );"BPF sleketon", to give a bit more context to users?
ok
Quoted 9 lines
tools/perf/util/bpf_off_cpu.c line 90
Quoted 7 lines
tools/perf/util/bpf_off_cpu.c new file +6
-0+return -1 ; + } + + err =off_cpu_bpf__attach (skel); +if (err) { +pr_err ("Failed to attach off-cpu skeleton\n" );Ditto.
ok
Quoted 46 lines
Quoted 32 lines
tools/perf/util/bpf_off_cpu.c new file +31
-0+ goto out; + } + +if (perf_hooks__set_hook ("record_start" , off_cpu_start,NULL ) || +perf_hooks__set_hook ("record_end" , off_cpu_finish,NULL )) { +pr_err ("Failed to attach off-cpu skeleton\n" ); + goto out; + } + +return 0 ; + +out: +off_cpu_bpf__destroy (skel); +return -1 ; +} + +int off_cpu_write (struct perf_session *session) +{ +int bytes =0 , size; +int fd, stack; +bool found = false; +u64 sample_type, val, sid =0 ; +struct evsel *evsel; +struct perf_data_file *file = &session->data->file; +struct off_cpu_key prev, key; +union off_cpu_data data = { + .hdr = { + .type =PERF_RECORD_SAMPLE , + .misc =PERF_RECORD_MISC_USER , + }, + };tools/perf/util/bpf_off_cpu.c line 131
Quoted 10 lines
tools/perf/util/bpf_off_cpu.c new file +10
-0+u64 tstamp =OFF_CPU_TIMESTAMP ; + + skel->bss->enabled =0 ; + +evlist__for_each_entry (session->evlist, evsel) { +if (!strcmp (evsel__name (evsel),"offcpu-time" )) { + found = true; +break ; + } + }Don't we have some
evlist__find_evselfunction?struct evsel *evlist__find_evsel_by_str(struct evlist *evlist, const char *str)
Quoted 12 lines
{
struct evsel *evsel;
evlist__for_each_entry(evlist, evsel) {
if (!evsel->name)
continue;
if (strcmp(str, evsel->name) == 0)
return evsel;
}
return NULL;
}Nice, I'll use it instead.
Thanks for your review!
Namhyung
Namhyung Kim
maintainerPATCH 3/4 perf record: Implement basic filtering for off-cpu
In reply to Namhyung KimIt should honor cpu and task filtering with -a, -C or -p, -t options.
--- tools/perf/builtin-record.c | 2 +- tools/perf/util/bpf_off_cpu.c | 78 +++++++++++++++++++++++--- tools/perf/util/bpf_skel/off_cpu.bpf.c | 52 +++++++++++++++-- tools/perf/util/off_cpu.h | 6 +- 4 files changed, 123 insertions(+), 15 deletions(-)
tools/perf/builtin-record.c +1 -1
@@ -907,7 +907,7 @@ static int record__config_text_poke(struct evlist *evlist)static int record__config_off_cpu (struct record *rec) { -return off_cpu_prepare (rec->evlist); +return off_cpu_prepare (rec->evlist, &rec->opts.target); }static bool record__kcore_readable (struct machine *machine)
tools/perf/util/bpf_off_cpu.c +70 -8
115 lines not on this page · raw message
tools/perf/util/bpf_skel/off_cpu.bpf.c +48 -4
76 lines not on this page · raw message
tools/perf/util/off_cpu.h +4 -2
@@ -2,13 +2,15 @@#define PERF_UTIL_OFF_CPU_H struct evlist ; +struct target ;struct perf_session ;#ifdef HAVE_BPF_SKEL -int off_cpu_prepare (struct evlist *evlist); +int off_cpu_prepare (struct evlist *evlist,struct target *target);int off_cpu_write (struct perf_session *session);#else -static inline int off_cpu_prepare (struct evlist *evlist__maybe_unused ) +static inline int off_cpu_prepare (struct evlist *evlist__maybe_unused , +struct target *target__maybe_unused ) {return -1 ; }
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
Namhyung Kim
maintainerPATCH 4/4 perf record: Handle argument change in sched_switch
In reply to Namhyung KimRecently sched_switch tracepoint added a new argument for prev_state, but it's hard to handle the change in a BPF program. Instead, we can check the function prototype in BTF before loading the program.
Thus I make two copies of the tracepoint handler and select one based on the BTF info.
--- tools/perf/util/bpf_off_cpu.c | 28 +++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 48 ++++++++++++++++++++------ 2 files changed, 65 insertions(+), 11 deletions(-)
tools/perf/util/bpf_off_cpu.c +28 -0
@@ -86,6 +86,33 @@ static void off_cpu_finish(void *arg __maybe_unused)off_cpu_bpf__destroy (skel); } +/* recent kernel added prev_state arg, so it needs to call the proper function */ +static void check_sched_switch_args (void ) +{ +const struct btf *btf =bpf_object__btf (skel->obj); +const struct btf_type *t1, *t2, *t3; +u32 type_id; + + type_id =btf__find_by_name_kind (btf,"bpf_trace_sched_switch" , +BTF_KIND_TYPEDEF ); +if ((s32 )type_id <0 ) +return ; + + t1 =btf__type_by_id (btf, type_id); +if (t1 ==NULL ) +return ; + + t2 =btf__type_by_id (btf, t1->type); +if (t2 ==NULL || !btf_is_ptr (t2)) +return ; + + t3 =btf__type_by_id (btf, t2->type); +if (t3 &&btf_is_func_proto (t3) &&btf_vlen (t3) ==4 ) { +/* new format: pass prev_state as 2nd arg */ + skel->rodata->has_prev_state = true; + } +} +int off_cpu_prepare (struct evlist *evlist,struct target *target) {int err, fd, i;
@@ -114,6 +141,7 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target) }set_max_rlimit (); +check_sched_switch_args (); err =off_cpu_bpf__load (skel);if (err) {
tools/perf/util/bpf_skel/off_cpu.bpf.c +37 -11
69 lines not on this page · raw message
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog