perf record: Implement off-cpu profiling with BPF (v1)
Subsystems (2)
Namhyung Kim
maintainercover perf record: Implement off-cpu profiling with BPF (v1)
Hello,
(Resending with a quick fix for a missing header.)
This is the first version of off-cpu profiling support. Together with (PMU-based) cpu profiling, it can show holistic view of the performance characteristics of your application or system.
With BPF, it can aggregate scheduling stats for interested tasks and/or states and convert the data into a form of perf sample records. I chose the bpf-output event which is a software event supposed to be consumed by BPF programs and renamed it as "offcpu-time". So it requires no change on the perf report side except for setting sample types of bpf-output event.
Basically it collects userspace callstack for tasks as it's what users want mostly. Maybe we can add support for the kernel stacks but I'm afraid that it'd cause more overhead. So the offcpu-time event will always have callchains regardless of the command line option, and it enables the children mode in perf report by default.
It adds --off-cpu option to perf record like below:
$ sudo perf record -a --off-cpu -- perf bench sched messaging -l 1000 # Running 'sched/messaging' benchmark: # 20 sender and receiver processes per group # 10 groups == 400 processes run
Total time: 1.518 [sec]
[ perf record: Woken up 9 times to write data ]
[ perf record: Captured and wrote 5.313 MB perf.data (53341 samples) ]
Then we can run perf report as usual. The below is just to skip less important parts.
$ sudo perf report --stdio --call-graph=no --percent-limit=2
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 52K of event 'cycles'
# Event count (approx.): 42522453276
#
# Children Self Command Shared Object Symbol
# ........ ........ ............... ................ ..................................
#
9.58% 9.58% sched-messaging [kernel.vmlinux] [k] audit_filter_rules.constprop.0
8.46% 8.46% sched-messaging [kernel.vmlinux] [k] audit_filter_syscall
4.54% 4.54% sched-messaging [kernel.vmlinux] [k] copy_user_enhanced_fast_string
2.94% 2.94% sched-messaging [kernel.vmlinux] [k] unix_stream_read_generic
2.45% 2.45% sched-messaging [kernel.vmlinux] [k] memcg_slab_free_hook # Samples: 983 of event 'offcpu-time'
# Event count (approx.): 684538813464
#
# Children Self Command Shared Object Symbol
# ........ ........ ............... .................... ..........................
#
83.86% 0.00% sched-messaging libc-2.33.so [.] __libc_start_main
83.86% 0.00% sched-messaging perf [.] cmd_bench
83.86% 0.00% sched-messaging perf [.] main
83.86% 0.00% sched-messaging perf [.] run_builtin
83.64% 0.00% sched-messaging perf [.] bench_sched_messaging
41.35% 41.35% sched-messaging libpthread-2.33.so [.] __read
38.88% 38.88% sched-messaging libpthread-2.33.so [.] __write
3.41% 3.41% sched-messaging libc-2.33.so [.] __pollThe perf bench sched messaging created 400 processes to send/receive messages through unix sockets. It spent a large portion of cpu cycles for audit filter and read/copy the messages while most of the offcpu-time was in read and write calls.
You can get the code from 'perf/offcpu-v1' branch in my tree at
git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
Enjoy! :)
Thanks,
Namhyung
Namhyung Kim (4):
perf report: Do not extend sample type of bpf-output event perf record: Enable off-cpu analysis with BPF perf record: Implement basic filtering for off-cpu perf record: Handle argument change in sched_switch
tools/perf/Makefile.perf | 1 + tools/perf/builtin-record.c | 21 ++ tools/perf/util/Build | 1 + tools/perf/util/bpf_off_cpu.c | 301 +++++++++++++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 214 ++++++++++++++++++ tools/perf/util/evsel.c | 4 +- 6 files changed, 540 insertions(+), 2 deletions(-) create mode 100644 tools/perf/util/bpf_off_cpu.c create mode 100644 tools/perf/util/bpf_skel/off_cpu.bpf.c
base-commit: 41204da4c16071be9090940b18f566832d46becc
Signature · 27 lines
2.36.0.rc2.479.g8af0fa9b8e-goog *** BLURB HERE *** Namhyung Kim (4): perf report: Do not extend sample type of bpf-output event perf record: Enable off-cpu analysis with BPF perf record: Implement basic filtering for off-cpu perf record: Handle argument change in sched_switch tools/perf/Makefile.perf | 1 + tools/perf/builtin-record.c | 21 ++ tools/perf/util/Build | 1 + tools/perf/util/bpf_off_cpu.c | 302 +++++++++++++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 214 ++++++++++++++++++ tools/perf/util/evsel.c | 4 +- tools/perf/util/off_cpu.h | 24 ++ 7 files changed, 565 insertions(+), 2 deletions(-) create mode 100644 tools/perf/util/bpf_off_cpu.c create mode 100644 tools/perf/util/bpf_skel/off_cpu.bpf.c create mode 100644 tools/perf/util/off_cpu.h base-commit: 41204da4c16071be9090940b18f566832d46becc -- 2.36.0.rc2.479.g8af0fa9b8e-goog
Namhyung Kim
maintainerPATCH 1/4 perf report: Do not extend sample type of bpf-output event
Currently evsel__new_idx() sets more sample_type bits when it finds a BPF-output event. But it should honor what's recorded in the perf data file rather than blindly sets the bits. Otherwise it could lead to a parse error when it recorded with a modified sample_type.
--- tools/perf/util/evsel.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/evsel.c +2 -2
@@ -269,8 +269,8 @@ struct evsel *evsel__new_idx(struct perf_event_attr *attr, int idx)return NULL ;evsel__init (evsel, attr, idx); -if (evsel__is_bpf_output (evsel)) { - evsel->core.attr.sample_type |= (PERF_SAMPLE_RAW |PERF_SAMPLE_TIME | +if (evsel__is_bpf_output (evsel) && !attr->sample_type) { + evsel->core.attr.sample_type = (PERF_SAMPLE_RAW |PERF_SAMPLE_TIME |PERF_SAMPLE_CPU |PERF_SAMPLE_PERIOD ), evsel->core.attr.sample_period =1 ; }
Signature · 1 line
2.36.0.rc2.479.g8af0fa9b8e-goog
Namhyung Kim
maintainerPATCH 2/4 perf record: Enable off-cpu analysis with BPF
In reply to Namhyung KimAdd --off-cpu option to enable the off-cpu profiling with BPF. It'd use a bpf_output event and rename it to "offcpu-time". Samples will be synthesized at the end of the record session using data from a BPF map which contains the aggregated off-cpu time at context switches. So it needs root privilege to get the off-cpu profiling.
Each sample will have a separate user stacktrace so it will skip kernel threads. The sample ip will be set from the stacktrace and other sample data will be updated accordingly. Currently it only handles some basic sample types.
The sample timestamp is set to a dummy value just not to bother with other events during the sorting. So it has a very big initial value and increase it on processing each samples.
Good thing is that it can be used together with regular profiling like cpu cycles. If you don't want to that, you can use a dummy event to enable off-cpu profiling only.
Example output:
$ sudo perf record --off-cpu perf bench sched messaging -l 1000
$ sudo perf report --stdio --call-graph=no # Total Lost Samples: 0 # # Samples: 41K of event 'cycles' # Event count (approx.): 42137343851 ...
# Samples: 1K of event 'offcpu-time'
# Event count (approx.): 587990831640
#
# Children Self Command Shared Object Symbol
# ........ ........ ............... .................. .........................
#
81.66% 0.00% sched-messaging libc-2.33.so [.] __libc_start_main
81.66% 0.00% sched-messaging perf [.] cmd_bench
81.66% 0.00% sched-messaging perf [.] main
81.66% 0.00% sched-messaging perf [.] run_builtin
81.43% 0.00% sched-messaging perf [.] bench_sched_messaging
40.86% 40.86% sched-messaging libpthread-2.33.so [.] __read
37.66% 37.66% sched-messaging libpthread-2.33.so [.] __write
2.91% 2.91% sched-messaging libc-2.33.so [.] __poll...
As you can see it spent most of off-cpu time in read and write in bench_sched_messaging(). The --call-graph=no was added just to make the output concise here.
It uses perf hooks facility to control BPF program during the record session rather than adding new BPF/off-cpu specific calls.
--- tools/perf/Makefile.perf | 1 + tools/perf/builtin-record.c | 21 +++ tools/perf/util/Build | 1 + tools/perf/util/bpf_off_cpu.c | 208 +++++++++++++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 137 ++++++++++++++++ tools/perf/util/off_cpu.h | 22 +++ 6 files changed, 390 insertions(+) create mode 100644 tools/perf/util/bpf_off_cpu.c create mode 100644 tools/perf/util/bpf_skel/off_cpu.bpf.c create mode 100644 tools/perf/util/off_cpu.h
tools/perf/Makefile.perf +1 -0
@@ -1041,6 +1041,7 @@ SKEL_TMP_OUT := $(abspath $(SKEL_OUT)/.tmp) SKELETONS := $(SKEL_OUT)/bpf_prog_profiler.skel.h SKELETONS += $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h +SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_TMP_OUT) $(LIBBPF_OUTPUT): $(Q)$(MKDIR) -p $@
tools/perf/builtin-record.c +21 -0
@@ -49,6 +49,7 @@#include "util/clockid.h" #include "util/pmu-hybrid.h" #include "util/evlist-hybrid.h" +#include "util/off_cpu.h" #include "asm/bug.h" #include "perf.h" #include "cputopo.h"
@@ -162,6 +163,7 @@ struct record {bool buildid_mmap;bool timestamp_filename;bool timestamp_boundary; +bool off_cpu;struct switch_output switch_output;unsigned long long samples;unsigned long output_max_size;/* = 0: unlimited */
@@ -903,6 +905,11 @@ static int record__config_text_poke(struct evlist *evlist)return 0 ; } +static int record__config_off_cpu (struct record *rec) +{ +return off_cpu_prepare (rec->evlist); +} +static bool record__kcore_readable (struct machine *machine) {char kcore[PATH_MAX ];
@@ -2596,6 +2603,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) }else status = err; +if (rec->off_cpu) + rec->bytes_written +=off_cpu_write (rec->session); +record__synthesize (rec, true);/* this will be recalculated during process_buildids() */ rec->samples =0 ;
@@ -3320,6 +3330,9 @@ static struct option __record_options[] = { OPT_CALLBACK_OPTARG(0, "threads", &record.opts, NULL, "spec", "write collected trace data into several data files using parallel threads", record__parse_threads), +#ifdef HAVE_BPF_SKEL +OPT_BOOLEAN (0 ,"off-cpu" , &record.off_cpu,"Enable off-cpu analysis" ), +#endifOPT_END () };
@@ -3968,6 +3981,14 @@ int cmd_record(int argc, const char **argv) } } +if (rec->off_cpu) { + err =record__config_off_cpu (rec); +if (err) { +pr_err ("record__config_off_cpu failed, error %d\n" , err); + goto out; + } + } +if (record_opts__config (&rec->opts)) { err = -EINVAL ; goto out;
tools/perf/util/Build +1 -0
@@ -147,6 +147,7 @@ perf-$(CONFIG_LIBBPF) += bpf_map.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter_cgroup.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_ftrace.o +perf-$(CONFIG_PERF_BPF_SKEL) += bpf_off_cpu.o perf-$(CONFIG_BPF_PROLOGUE) += bpf-prologue.o perf-$(CONFIG_LIBELF) += symbol-elf.o perf-$(CONFIG_LIBELF) += probe-file.o
tools/perf/util/bpf_off_cpu.c new file +208 -0
208 lines not on this page · raw message
tools/perf/util/bpf_skel/off_cpu.bpf.c new file +137 -0
137 lines not on this page · raw message
tools/perf/util/off_cpu.h new file +22 -0
@@ -0,0 +1,22 @@ +#ifndef PERF_UTIL_OFF_CPU_H +#define PERF_UTIL_OFF_CPU_H + +struct evlist ; +struct perf_session ; + +#ifdef HAVE_BPF_SKEL +int off_cpu_prepare (struct evlist *evlist); +int off_cpu_write (struct perf_session *session); +#else +static inline int off_cpu_prepare (struct evlist *evlist__maybe_unused ); +{ +return -1 ; +} + +static inline int off_cpu_write (struct perf_session *session__maybe_unused ) +{ +return -1 ; +} +#endif + +#endif /* PERF_UTIL_OFF_CPU_H */
Signature · 1 line
2.36.0.rc2.479.g8af0fa9b8e-goog
Re: [PATCH 2/4] perf record: Enable off-cpu analysis with BPF
Hi Namhyung,
Quoted 1 line · Namhyung Kim
[...]
Quoted 14 lines
Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/Makefile.perf | 1 + tools/perf/builtin-record.c | 21 +++ tools/perf/util/Build | 1 + tools/perf/util/bpf_off_cpu.c | 208 +++++++++++++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 137 ++++++++++++++++ tools/perf/util/off_cpu.h | 22 +++ 6 files changed, 390 insertions(+) create mode 100644 tools/perf/util/bpf_off_cpu.c create mode 100644 tools/perf/util/bpf_skel/off_cpu.bpf.c create mode 100644 tools/perf/util/off_cpu.h
[...]
tools/perf/util/bpf_skel/off_cpu.bpf.c line 43
Quoted 12 lines
tools/perf/util/bpf_skel/off_cpu.bpf.c new file +6
-0+struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(struct tstamp_data)); + __uint(max_entries, MAX_ENTRIES); +} tstamp SEC(".maps");
I think using task local storage for this tstamp would be more efficient. There is an example in tools/bpf/runqslower/runqslower.bpf.c
Namhyung Kim
maintainerRe: [PATCH 2/4] perf record: Enable off-cpu analysis with BPF
Hi Hao,
Quoted 31 lines · Hao Luo
Hi Namhyung,
Quoted 1 line · Namhyung Kim
[...]
Quoted 14 lines
Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/Makefile.perf | 1 + tools/perf/builtin-record.c | 21 +++ tools/perf/util/Build | 1 + tools/perf/util/bpf_off_cpu.c | 208 +++++++++++++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 137 ++++++++++++++++ tools/perf/util/off_cpu.h | 22 +++ 6 files changed, 390 insertions(+) create mode 100644 tools/perf/util/bpf_off_cpu.c create mode 100644 tools/perf/util/bpf_skel/off_cpu.bpf.c create mode 100644 tools/perf/util/off_cpu.h[...]
Quoted 6 lines
tools/perf/util/bpf_skel/off_cpu.bpf.c new file +0
-0
Quoted 10 lines
tools/perf/util/bpf_skel/off_cpu.bpf.c line 43
Quoted 6 lines
tools/perf/util/bpf_skel/off_cpu.bpf.c new file +6
-0+struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(struct tstamp_data)); + __uint(max_entries, MAX_ENTRIES); +} tstamp SEC(".maps");I think using task local storage for this tstamp would be more efficient. There is an example in
tools/bpf/runqslower/runqslower.bpf.c
Right, will change in v2.
Thanks,
Namhyung
Namhyung Kim
maintainerPATCH 3/4 perf record: Implement basic filtering for off-cpu
In reply to Namhyung KimIt should honor cpu and task filtering with -a, -C or -p, -t options.
--- tools/perf/builtin-record.c | 2 +- tools/perf/util/bpf_off_cpu.c | 78 +++++++++++++++++++++++--- tools/perf/util/bpf_skel/off_cpu.bpf.c | 52 +++++++++++++++-- tools/perf/util/off_cpu.h | 6 +- 4 files changed, 123 insertions(+), 15 deletions(-)
tools/perf/builtin-record.c +1 -1
@@ -907,7 +907,7 @@ static int record__config_text_poke(struct evlist *evlist)static int record__config_off_cpu (struct record *rec) { -return off_cpu_prepare (rec->evlist); +return off_cpu_prepare (rec->evlist, &rec->opts.target); }static bool record__kcore_readable (struct machine *machine)
tools/perf/util/bpf_off_cpu.c +70 -8
115 lines not on this page · raw message
tools/perf/util/bpf_skel/off_cpu.bpf.c +48 -4
76 lines not on this page · raw message
tools/perf/util/off_cpu.h +4 -2
@@ -2,13 +2,15 @@#define PERF_UTIL_OFF_CPU_H struct evlist ; +struct target ;struct perf_session ;#ifdef HAVE_BPF_SKEL -int off_cpu_prepare (struct evlist *evlist); +int off_cpu_prepare (struct evlist *evlist,struct target *target);int off_cpu_write (struct perf_session *session);#else -static inline int off_cpu_prepare (struct evlist *evlist__maybe_unused ); +static inline int off_cpu_prepare (struct evlist *evlist__maybe_unused , +struct target *target__maybe_unused ); {return -1 ; }
Signature · 1 line
2.36.0.rc2.479.g8af0fa9b8e-goog
Namhyung Kim
maintainerPATCH 4/4 perf record: Handle argument change in sched_switch
In reply to Namhyung KimRecently sched_switch tracepoint added a new argument for prev_state, but it's hard to handle the change in a BPF program. Instead, we can check the function prototype in BTF before loading the program.
Thus I make two copies of the tracepoint handler and select one based on the BTF info.
--- tools/perf/util/bpf_off_cpu.c | 32 +++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 55 ++++++++++++++++++++------ 2 files changed, 76 insertions(+), 11 deletions(-)
tools/perf/util/bpf_off_cpu.c +32 -0
@@ -86,6 +86,37 @@ static void off_cpu_finish(void *arg __maybe_unused)off_cpu_bpf__destroy (skel); } +/* recent kernel added prev_state arg, so it needs to call the proper function */ +static void check_sched_switch_args (void ) +{ +const struct btf *btf =bpf_object__btf (skel->obj); +const struct btf_type *t1, *t2, *t3; +u32 type_id; + + type_id =btf__find_by_name_kind (btf,"bpf_trace_sched_switch" , +BTF_KIND_TYPEDEF ); +if ((s32 )type_id <0 ) + goto old_format; + + t1 =btf__type_by_id (btf, type_id); +if (t1 ==NULL ) + goto old_format; + + t2 =btf__type_by_id (btf, t1->type); +if (t2 ==NULL || !btf_is_ptr (t2)) + goto old_format; + + t3 =btf__type_by_id (btf, t2->type); +if (t3 &&btf_is_func_proto (t3) &&btf_vlen (t3) ==4 ) { +/* new format: disable old functions */ +bpf_program__set_autoload (skel->progs.on_switch3, false); +return ; + } + +old_format: +bpf_program__set_autoload (skel->progs.on_switch4, false); +} +int off_cpu_prepare (struct evlist *evlist,struct target *target) {int err, fd, i;
@@ -114,6 +145,7 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target) }set_max_rlimit (); +check_sched_switch_args (); err =off_cpu_bpf__load (skel);if (err) {
tools/perf/util/bpf_skel/off_cpu.bpf.c +44 -11
70 lines not on this page · raw message
Signature · 1 line
2.36.0.rc2.479.g8af0fa9b8e-goog
Re: [PATCH 4/4] perf record: Handle argument change in sched_switch
Quoted 14 lines · Namhyung Kim
Recently
sched_switchtracepoint added a new argument forprev_state, but it's hard to handle the change in a BPF program. Instead, we can check the function prototype in BTF before loading the program.Thus I make two copies of the tracepoint handler and select one based on the BTF info.
Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/bpf_off_cpu.c | 32 +++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 55 ++++++++++++++++++++------ 2 files changed, 76 insertions(+), 11 deletions(-)
[...]
Quoted 6 lines
tools/perf/util/bpf_skel/off_cpu.bpf.c +5
-0+SEC("tp_btf/sched_switch") +int on_switch3 (u64 *ctx) +{ +struct task_struct *prev, *next; +int state;
tools/perf/util/bpf_skel/off_cpu.bpf.c int on_switch(u64 *ctx) · line 186
Quoted 10 lines
tools/perf/util/bpf_skel/off_cpu.bpf.c +10
-0+ +if (!enabled) +return 0 ; + +/* +* TP_PROTO(bool preempt, struct task_struct *prev, +* struct task_struct *next) +*/ + prev = (struct task_struct *)ctx[1 ]; + next = (struct task_struct *)ctx[2 ];
you don't have to have two BPF programs for this, you can use read-only variable to make this choice.
On BPF side
const volatile bool has_prev_state = false;
...
if (has_prev_state) {
prev = (struct task_struct *)ctx[2];
next = (struct task_struct *)ctx[3];
} else {
prev = (struct task_struct *)ctx[1];
next = (struct task_struct *)ctx[2];
}And from user-space side you do your detection and before skeleton is loaded:
skel->rodata->has_prev_state = <whatever you detected>
But I'm still hoping that this prev_state argument can be moved to the end ([0]) to make all this unnecessary.
Quoted 6 lines
tools/perf/util/bpf_skel/off_cpu.bpf.c +6
-0+ + state =get_task_state (prev); + +return on_switch (ctx, prev, next, state); +} +
[...]
Namhyung Kim
maintainerRe: [PATCH 4/4] perf record: Handle argument change in sched_switch
Hello,
Quoted 55 lines · Andrii Nakryiko
Quoted 14 lines · Namhyung Kim
Recently
sched_switchtracepoint added a new argument forprev_state, but it's hard to handle the change in a BPF program. Instead, we can check the function prototype in BTF before loading the program.Thus I make two copies of the tracepoint handler and select one based on the BTF info.
Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/bpf_off_cpu.c | 32 +++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 55 ++++++++++++++++++++------ 2 files changed, 76 insertions(+), 11 deletions(-)[...]
Quoted 6 lines
tools/perf/util/bpf_skel/off_cpu.bpf.c +5
-0+SEC("tp_btf/sched_switch") +int on_switch3 (u64 *ctx) +{ +struct task_struct *prev, *next; +int state;tools/perf/util/bpf_skel/off_cpu.bpf.c int on_switch(u64 *ctx) · line 186
Quoted 10 lines
tools/perf/util/bpf_skel/off_cpu.bpf.c +10
-0+ +if (!enabled) +return 0 ; + +/* +* TP_PROTO(bool preempt, struct task_struct *prev, +* struct task_struct *next) +*/ + prev = (struct task_struct *)ctx[1 ]; + next = (struct task_struct *)ctx[2 ];you don't have to have two BPF programs for this, you can use read-only variable to make this choice.
On BPF side
const volatile bool
has_prev_state= false;...
if (has_prev_state) { prev = (struct task_struct *)ctx[2]; next = (struct task_struct *)ctx[3]; } else { prev = (struct task_struct *)ctx[1]; next = (struct task_struct *)ctx[2]; }
Quoted 3 lines
And from user-space side you do your detection and before skeleton is loaded:
skel->rodata->has_prev_state= <whatever you detected>
Nice, thanks for the tip!
Actually I tried something similar but it was with a variable (in bss) so the verifier in an old kernel rejected it due to invalid arg access.
I guess now the const makes the verifier ignore the branch as if it's dead but the compiler still generates the code, right?
Quoted 5 lines
But I'm still hoping that this
prev_stateargument can be moved to the end ([0]) to make all this unnecessary.
Yeah, that would make life easier. :)
Thanks,
Namhyung
Re: [PATCH 4/4] perf record: Handle argument change in sched_switch
Quoted 64 lines · Namhyung Kim
Hello,
Quoted 55 lines · Andrii Nakryiko
Quoted 14 lines · Namhyung Kim
Recently
sched_switchtracepoint added a new argument forprev_state, but it's hard to handle the change in a BPF program. Instead, we can check the function prototype in BTF before loading the program.Thus I make two copies of the tracepoint handler and select one based on the BTF info.
Signed-off-by Namhyung Kim <[email protected]>--- tools/perf/util/bpf_off_cpu.c | 32 +++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 55 ++++++++++++++++++++------ 2 files changed, 76 insertions(+), 11 deletions(-)[...]
Quoted 6 lines
tools/perf/util/bpf_skel/off_cpu.bpf.c +5
-0+SEC("tp_btf/sched_switch") +int on_switch3 (u64 *ctx) +{ +struct task_struct *prev, *next; +int state;tools/perf/util/bpf_skel/off_cpu.bpf.c int on_switch(u64 *ctx) · line 186
Quoted 10 lines
tools/perf/util/bpf_skel/off_cpu.bpf.c +10
-0+ +if (!enabled) +return 0 ; + +/* +* TP_PROTO(bool preempt, struct task_struct *prev, +* struct task_struct *next) +*/ + prev = (struct task_struct *)ctx[1 ]; + next = (struct task_struct *)ctx[2 ];you don't have to have two BPF programs for this, you can use read-only variable to make this choice.
On BPF side
const volatile bool
has_prev_state= false;...
if (has_prev_state) { prev = (struct task_struct *)ctx[2]; next = (struct task_struct *)ctx[3]; } else { prev = (struct task_struct *)ctx[1]; next = (struct task_struct *)ctx[2]; }Quoted 3 lines
And from user-space side you do your detection and before skeleton is loaded:
skel->rodata->has_prev_state= <whatever you detected>
Quoted 7 lines
Nice, thanks for the tip!
Actually I tried something similar but it was with a variable (in bss) so the verifier in an old kernel rejected it due to invalid arg access.
I guess now the const makes the verifier ignore the branch as if it's dead but the compiler still generates the code, right?
yes, exactly
Quoted 11 lines
Quoted 6 lines
But I'm still hoping that this
prev_stateargument can be moved to the end ([0]) to make all this unnecessary.Yeah, that would make life easier. :)
Thanks,
Namhyung
Namhyung Kim
maintainerRe: [PATCH 4/4] perf record: Handle argument change in sched_switch
Quoted 10 lines · Andrii Nakryiko
Quoted 5 lines · Namhyung Kim
Actually I tried something similar but it was with a variable (in bss) so the verifier in an old kernel rejected it due to invalid arg access.
I guess now the const makes the verifier ignore the branch as if it's dead but the compiler still generates the code, right?
yes, exactly
Then I'm curious how it'd work on newer kernels. The verifier sees the false branch and detects type mismatch for the second argument then it'd reject the program?
Thanks,
Namhyung
Re: [PATCH 4/4] perf record: Handle argument change in sched_switch
Quoted 12 lines · Namhyung Kim
Quoted 9 lines · Andrii Nakryiko
Quoted 5 lines · Namhyung Kim
Actually I tried something similar but it was with a variable (in bss) so the verifier in an old kernel rejected it due to invalid arg access.
I guess now the const makes the verifier ignore the branch as if it's dead but the compiler still generates the code, right?
Quoted 6 lines
Quoted 1 line
yes, exactly
Then I'm curious how it'd work on newer kernels. The verifier sees the false branch and detects type mismatch for the second argument then it'd reject the program?
Verifier will know which branch is never taken, and will just ignore and remove any code in it as dead code.
Quoted 2 lines
Thanks,
Namhyung