perf: generate events for BPF metadata
Subsystems (2)
v2 cover perf: generate events for BPF metadata
Commit ffa915f46193 ("Merge branch 'bpf_metadata'"), from September 2020, added support to the kernel, libbpf, and bpftool to treat read-only BPF variables that have names starting with 'bpf_metadata_' specially. This patch series updates perf to handle these variables similarly, allowing a perf.data file to capture relevant information about BPF programs on the system being profiled.
When it encounters a BPF program, it reads the program's maps to find an '.rodata' map with 'bpf_metadata_' variables. If it finds one, it extracts their values as strings, and creates a new PERF_RECORD_BPF_METADATA synthetic event using that data. It does this both for BPF programs that were loaded when a 'perf record' starts, as well as for programs that are loaded while the profile is running. For the latter case, it stores the metadata for the duration of the profile, and then dumps it at the end of the profile, where it's in a better context to do so.
The PERF_RECORD_BPF_METADATA event holds an array of key-value pairs, where the key is the variable name (minus the "bpf_metadata_" prefix) and the value is the variable's value, formatted as a string. There is one such event generated for each BPF subprogram. Generating it per subprogram rather than per program allows it to be correlated with PERF_RECORD_KSYMBOL events; the metadata event's "prog_name" is designed to be identical to the "name" field of a perf_record_ksymbol. This allows specific BPF metadata to be associated with each BPF address range in the collection.
Changes:
* v1 -> v2:
- Split out libbpf change and send it to the bpf tree.
- Add feature detection to perf to detect the libbpf change.
- Allow the feature to be skipped if the libbpf support is not found.
- Add an example of a PERF_RECORD_BPF_METADATA record.
- Change calloc() calls to zalloc().
- Don't check for NULL before calling free().
- Update the perf_event header when it is created, rather than storing the event size and updating it later.
- Add a BPF metadata variable (with the perf version) to all perf BPF programs.
- Update the selftest to look for the new perf_version variable.
- Split out the selftest into its own patch.
- Link to v1:
https://lore.kernel.org/linux-perf-users/[email protected]/T/#t
Blake Jones (4):
perf: detect support for libbpf's emit_strings option
perf: collect BPF metadata from existing BPF programs
perf: collect BPF metadata from new programs, and display the new
event
perf: add test for PERF_RECORD_BPF_METADATA collection
tools/build/Makefile.feature | 1 + tools/build/feature/Makefile | 4 + tools/build/feature/test-libbpf-strings.c | 10 + tools/lib/perf/include/perf/event.h | 18 + tools/perf/Documentation/perf-check.txt | 1 + tools/perf/Makefile.config | 12 + tools/perf/Makefile.perf | 3 +- tools/perf/builtin-check.c | 1 + tools/perf/builtin-inject.c | 1 + tools/perf/builtin-record.c | 8 + tools/perf/builtin-script.c | 15 +- tools/perf/tests/shell/test_bpf_metadata.sh | 76 ++++ tools/perf/util/bpf-event.c | 378 ++++++++++++++++++++ tools/perf/util/bpf-event.h | 13 + tools/perf/util/bpf_skel/perf_version.h | 17 + tools/perf/util/env.c | 19 +- tools/perf/util/env.h | 4 + tools/perf/util/event.c | 21 ++ tools/perf/util/event.h | 1 + tools/perf/util/header.c | 1 + tools/perf/util/session.c | 4 + tools/perf/util/synthetic-events.h | 2 + tools/perf/util/tool.c | 14 + tools/perf/util/tool.h | 3 +- 24 files changed, 622 insertions(+), 5 deletions(-) create mode 100644 tools/build/feature/test-libbpf-strings.c create mode 100755 tools/perf/tests/shell/test_bpf_metadata.sh create mode 100644 tools/perf/util/bpf_skel/perf_version.h
Signature · 1 line
2.50.0.rc0.604.gd4ff7b7c86-goog
PATCH v2 1/4 perf: detect support for libbpf's emit_strings option
This creates a config option that detects libbpf's ability to display character arrays as strings, which was just added to the BPF tree
(https://git.kernel.org/bpf/bpf-next/c/87c9c79a02b4).
To test this change, I built perf (from later in this patch set) with:
- static libbpf (default, using source from kernel tree)
- dynamic libbpf (LIBBPF_DYNAMIC=1 LIBBPF_INCLUDE=/usr/local/include)
For both the static and dynamic versions, I used headers with and without the ".emit_strings" option.
I verified that of the four resulting binaries, the two with ".emit_strings" would successfully record BPF_METADATA events, and the two without wouldn't. All four binaries would successfully display BPF_METADATA events, because the relevant bit of libbpf code is only used during "perf record".
--- tools/build/Makefile.feature | 1 + tools/build/feature/Makefile | 4 ++++ tools/build/feature/test-libbpf-strings.c | 10 ++++++++++ tools/perf/Documentation/perf-check.txt | 1 + tools/perf/Makefile.config | 12 ++++++++++++ tools/perf/builtin-check.c | 1 + 6 files changed, 29 insertions(+) create mode 100644 tools/build/feature/test-libbpf-strings.c
tools/build/Makefile.feature +1 -0
@@ -126,6 +126,7 @@ FEATURE_TESTS_EXTRA := \ llvm \ clang \ libbpf \ + libbpf-strings \ libbpf-btf__load_from_kernel_by_id \ libbpf-bpf_prog_load \ libbpf-bpf_object__next_program \
tools/build/feature/Makefile +4 -0
@@ -59,6 +59,7 @@ FILES= \ test-lzma.bin \ test-bpf.bin \ test-libbpf.bin \ + test-libbpf-strings.bin \ test-get_cpuid.bin \ test-sdt.bin \ test-cxx.bin \
@@ -360,6 +361,9 @@ $(OUTPUT)test-libbpf-bpf_program__set_insns.bin: $(OUTPUT)test-libbpf-btf__raw_data.bin: $(BUILD) -lbpf +$(OUTPUT)test-libbpf-strings.bin: + $(BUILD) + $(OUTPUT)test-sdt.bin: $(BUILD)
tools/build/feature/test-libbpf-strings.c new file +10 -0
@@ -0,0 +1,10 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <bpf/btf.h> + +int main (void ) +{ +struct btf_dump_type_data_opts opts; + + opts.emit_strings =0 ; +return opts.emit_strings; +}
tools/perf/Documentation/perf-check.txt +1 -0
@@ -52,6 +52,7 @@ feature:: dwarf-unwind / HAVE_DWARF_UNWIND_SUPPORT auxtrace / HAVE_AUXTRACE_SUPPORT libbfd / HAVE_LIBBFD_SUPPORT + libbpf-strings / HAVE_LIBBPF_STRINGS_SUPPORT libcapstone / HAVE_LIBCAPSTONE_SUPPORT libcrypto / HAVE_LIBCRYPTO_SUPPORT libdw-dwarf-unwind / HAVE_LIBDW_SUPPORT
tools/perf/Makefile.config +12 -0
@@ -595,8 +595,20 @@ ifndef NO_LIBELF LIBBPF_STATIC := 1 $(call detected,CONFIG_LIBBPF) CFLAGS += -DHAVE_LIBBPF_SUPPORT + ifneq ($(OUTPUT),) + LIBBPF_INCLUDE = $(abspath $(OUTPUT))/libbpf/include + else + LIBBPF_INCLUDE = $(CURDIR)/libbpf/include + endif endif endif + + FEATURE_CHECK_CFLAGS-libbpf-strings="-I$(LIBBPF_INCLUDE)" + $(call feature_check,libbpf-strings) + ifeq ($(feature-libbpf-strings), 1) + $(call detected,CONFIG_LIBBPF_STRINGS) + CFLAGS += -DHAVE_LIBBPF_STRINGS_SUPPORT + endif endif endif # NO_LIBBPF endif # NO_LIBELF
tools/perf/builtin-check.c +1 -0
@@ -43,6 +43,7 @@ struct feature_status supported_features[] = {FEATURE_STATUS ("dwarf-unwind" ,HAVE_DWARF_UNWIND_SUPPORT ),FEATURE_STATUS ("auxtrace" ,HAVE_AUXTRACE_SUPPORT ),FEATURE_STATUS_TIP ("libbfd" ,HAVE_LIBBFD_SUPPORT ,"Deprecated, license incompatibility, use BUILD_NONDISTRO=1 and install binutils-dev[el]" ), +FEATURE_STATUS ("libbpf-strings" ,HAVE_LIBBPF_STRINGS_SUPPORT ),FEATURE_STATUS ("libcapstone" ,HAVE_LIBCAPSTONE_SUPPORT ),FEATURE_STATUS ("libcrypto" ,HAVE_LIBCRYPTO_SUPPORT ), FEATURE_STATUS("libdw-dwarf-unwind", HAVE_LIBDW_SUPPORT),
Signature · 1 line
2.50.0.rc0.604.gd4ff7b7c86-goog
Namhyung Kim
maintainerRe: [PATCH v2 1/4] perf: detect support for libbpf's emit_strings option
Quoted 34 lines · Blake Jones
This creates a config option that detects libbpf's ability to display character arrays as strings, which was just added to the BPF tree
(https://git.kernel.org/bpf/bpf-next/c/87c9c79a02b4).To test this change, I built perf (from later in this patch set) with:
- static libbpf (default, using source from kernel tree)
- dynamic libbpf (LIBBPF_DYNAMIC=1 LIBBPF_INCLUDE=/usr/local/include)For both the static and dynamic versions, I used headers with and without the ".emit_strings" option.
I verified that of the four resulting binaries, the two with ".emit_strings" would successfully record
BPF_METADATAevents, and the two without wouldn't. All four binaries would successfully displayBPF_METADATAevents, because the relevant bit of libbpf code is only used during "perf record".Signed-off-by Blake Jones <[email protected]>--- tools/build/Makefile.feature | 1 + tools/build/feature/Makefile | 4 ++++ tools/build/feature/test-libbpf-strings.c | 10 ++++++++++ tools/perf/Documentation/perf-check.txt | 1 + tools/perf/Makefile.config | 12 ++++++++++++ tools/perf/builtin-check.c | 1 + 6 files changed, 29 insertions(+) create mode 100644 tools/build/feature/test-libbpf-strings.ctools/build/Makefile.feature +0
-0
tools/build/Makefile.feature FEATURE_TESTS_EXTRA := \ · line 132
Quoted 7 lines
tools/build/Makefile.feature +1
-0@@ -126,6 +126,7 @@ FEATURE_TESTS_EXTRA := \ llvm \ clang \ libbpf \ + libbpf-strings \ libbpf-btf__load_from_kernel_by_id \ libbpf-bpf_prog_load \ libbpf-bpf_object__next_program \
Please check out tmp.perf-tools-next branch which made some changes in the area.
Thanks,
Namhyung
Quoted 90 lines
tools/build/feature/Makefile +4
-0@@ -59,6 +59,7 @@ FILES= \ test-lzma.bin \ test-bpf.bin \ test-libbpf.bin \ + test-libbpf-strings.bin \ test-get_cpuid.bin \ test-sdt.bin \ test-cxx.bin \@@ -360,6 +361,9 @@ $(OUTPUT)test-libbpf-bpf_program__set_insns.bin: $(OUTPUT)test-libbpf-btf__raw_data.bin: $(BUILD) -lbpf +$(OUTPUT)test-libbpf-strings.bin: + $(BUILD) + $(OUTPUT)test-sdt.bin: $(BUILD)tools/build/feature/test-libbpf-strings.c new file +10
-0@@ -0,0 +1,10 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <bpf/btf.h> + +int main (void ) +{ +struct btf_dump_type_data_opts opts; + + opts.emit_strings =0 ; +return opts.emit_strings; +}tools/perf/Documentation/perf-check.txt +1
-0@@ -52,6 +52,7 @@ feature:: dwarf-unwind / HAVE_DWARF_UNWIND_SUPPORT auxtrace / HAVE_AUXTRACE_SUPPORT libbfd / HAVE_LIBBFD_SUPPORT + libbpf-strings / HAVE_LIBBPF_STRINGS_SUPPORT libcapstone / HAVE_LIBCAPSTONE_SUPPORT libcrypto / HAVE_LIBCRYPTO_SUPPORT libdw-dwarf-unwind / HAVE_LIBDW_SUPPORTtools/perf/Makefile.config +12
-0@@ -595,8 +595,20 @@ ifndef NO_LIBELF LIBBPF_STATIC := 1 $(call detected,CONFIG_LIBBPF) CFLAGS += -DHAVE_LIBBPF_SUPPORT + ifneq ($(OUTPUT),) + LIBBPF_INCLUDE = $(abspath $(OUTPUT))/libbpf/include + else + LIBBPF_INCLUDE = $(CURDIR)/libbpf/include + endif endif endif + + FEATURE_CHECK_CFLAGS-libbpf-strings="-I$(LIBBPF_INCLUDE)" + $(call feature_check,libbpf-strings) + ifeq ($(feature-libbpf-strings), 1) + $(call detected,CONFIG_LIBBPF_STRINGS) + CFLAGS += -DHAVE_LIBBPF_STRINGS_SUPPORT + endif endif endif # NO_LIBBPF endif # NO_LIBELFtools/perf/builtin-check.c +1
-0@@ -43,6 +43,7 @@ struct feature_status supported_features[] = {FEATURE_STATUS ("dwarf-unwind" ,HAVE_DWARF_UNWIND_SUPPORT ),FEATURE_STATUS ("auxtrace" ,HAVE_AUXTRACE_SUPPORT ),FEATURE_STATUS_TIP ("libbfd" ,HAVE_LIBBFD_SUPPORT ,"Deprecated, license incompatibility, use BUILD_NONDISTRO=1 and install binutils-dev[el]" ), +FEATURE_STATUS ("libbpf-strings" ,HAVE_LIBBPF_STRINGS_SUPPORT ),FEATURE_STATUS ("libcapstone" ,HAVE_LIBCAPSTONE_SUPPORT ),FEATURE_STATUS ("libcrypto" ,HAVE_LIBCRYPTO_SUPPORT ), FEATURE_STATUS("libdw-dwarf-unwind", HAVE_LIBDW_SUPPORT),Signature · 1 line
2.50.0.rc0.604.gd4ff7b7c86-goog
Re: [PATCH v2 1/4] perf: detect support for libbpf's emit_strings option
Hi Namhyung,
Quoted 2 lines · Namhyung Kim
Please check out tmp.perf-tools-next branch which made some changes in the area.
Done - v3 is synced with that branch. I'll send that out now.
Blake
PATCH v2 2/4 perf: collect BPF metadata from existing BPF programs
In reply to Blake JonesLook for .rodata maps, find ones with 'bpf_metadata_' variables, extract their values as strings, and create a new PERF_RECORD_BPF_METADATA synthetic event using that data. The code gets invoked from the existing routine perf_event__synthesize_one_bpf_prog().
For example, a BPF program with the following variables:
const char bpf_metadata_version[] SEC(".rodata") = "3.14159";
int bpf_metadata_value[] SEC(".rodata") = 42;would generate a PERF_RECORD_BPF_METADATA record with:
.prog_name = <BPF program name, e.g. "bpf_prog_a1b2c3_foo">
.nr_entries = 2
.entries[0].key = "version"
.entries[0].value = "3.14159"
.entries[1].key = "value"
.entries[1].value = "42"Each of the BPF programs and subprograms that share those variables would get a distinct PERF_RECORD_BPF_METADATA record, with the ".prog_name" showing the name of each program or subprogram. The prog_name is deliberately the same as the ".name" field in the corresponding PERF_RECORD_KSYMBOL record.
This code only gets invoked if support for displaying BTF char arrays as strings is detected.
--- tools/lib/perf/include/perf/event.h | 18 ++ tools/perf/util/bpf-event.c | 332 ++++++++++++++++++++++++++++ tools/perf/util/bpf-event.h | 12 + 3 files changed, 362 insertions(+)
tools/lib/perf/include/perf/event.h +18 -0
@@ -467,6 +467,22 @@ struct perf_record_compressed2 {char data[]; }; +#define BPF_METADATA_KEY_LEN 64 +#define BPF_METADATA_VALUE_LEN 256 +#define BPF_PROG_NAME_LEN KSYM_NAME_LEN + +struct perf_record_bpf_metadata_entry { +char key[BPF_METADATA_KEY_LEN ]; +char value[BPF_METADATA_VALUE_LEN ]; +}; + +struct perf_record_bpf_metadata { +struct perf_event_header header; +char prog_name[BPF_PROG_NAME_LEN ]; +__u64 nr_entries; +struct perf_record_bpf_metadata_entry entries[]; +}; +enum perf_user_event_type {/* above any possible kernel type */ PERF_RECORD_USER_TYPE_START =64 ,PERF_RECORD_HEADER_ATTR =64 ,
@@ -489,6 +505,7 @@ enum perf_user_event_type { /* above any possible kernel type */PERF_RECORD_COMPRESSED =81 ,PERF_RECORD_FINISHED_INIT =82 ,PERF_RECORD_COMPRESSED2 =83 , +PERF_RECORD_BPF_METADATA =84 , PERF_RECORD_HEADER_MAX };
@@ -530,6 +547,7 @@ union perf_event {struct perf_record_header_feature feat;struct perf_record_compressed pack;struct perf_record_compressed2 pack2; + struct perf_record_bpf_metadata bpf_metadata; };#endif /* __LIBPERF_EVENT_H */
tools/perf/util/bpf-event.c +332 -0
369 lines not on this page · raw message
tools/perf/util/bpf-event.h +12 -0
@@ -17,6 +17,12 @@ struct record_opts;struct evlist ;struct target ; +struct bpf_metadata { +union perf_event *event; +char **prog_names; +__u64 nr_prog_names; +}; +struct bpf_prog_info_node {struct perf_bpil *info_linear;struct rb_node rb_node;
@@ -36,6 +42,7 @@ int evlist__add_bpf_sb_event(struct evlist *evlist, struct perf_env *env);void __bpf_event__print_bpf_prog_info (struct bpf_prog_info *info,struct perf_env *env,FILE *fp); +void bpf_metadata_free (struct bpf_metadata *metadata);#else static inline int machine__process_bpf(struct machine *machine __maybe_unused, union perf_event *event __maybe_unused,
@@ -55,6 +62,11 @@ static inline void __bpf_event__print_bpf_prog_info(struct bpf_prog_info *info _ FILE *fp __maybe_unused) { +} + +static inline void bpf_metadata_free (struct bpf_metadata *metadata) +{ + }#endif // HAVE_LIBBPF_SUPPORT #endif
Signature · 1 line
2.50.0.rc0.604.gd4ff7b7c86-goog
PATCH v2 3/4 perf: collect BPF metadata from new programs, and display the new event
In reply to Blake JonesThis collects metadata for any BPF programs that were loaded during a "perf record" run, and emits it at the end of the run. It also adds support for displaying the new PERF_RECORD_BPF_METADATA type.
Here's some example "perf script -D" output for the new event type. The ": unhandled!" message is from tool.c, analogous to other behavior there. I've elided some rows with all NUL characters for brevity, and I wrapped one of the >75-column lines to fit in the commit guidelines.
[email protected] [0x260]: event: 84
.
. ... raw event: size 608 bytes
. 0000: 54 00 00 00 00 00 60 02 62 70 66 5f 70 72 6f 67 T.....`.bpf_prog . 0010: 5f 31 65 30 61 32 65 33 36 36 65 35 36 66 31 61 _1e0a2e366e56f1a . 0020: 32 5f 70 65 72 66 5f 73 61 6d 70 6c 65 5f 66 69 2_perf_sample_fi . 0030: 6c 74 65 72 00 00 00 00 00 00 00 00 00 00 00 00 lter............ . 0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [...] . 0110: 74 65 73 74 5f 76 61 6c 75 65 00 00 00 00 00 00 test_value...... . 0120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [...] . 0150: 34 32 00 00 00 00 00 00 00 00 00 00 00 00 00 00 42.............. . 0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[...]
0 0x50fc8 [0x260]: PERF_RECORD_BPF_METADATA \
prog bpf_prog_1e0a2e366e56f1a2_perf_sample_filter entry 0: test_value = 42
: unhandled!
--- tools/perf/builtin-inject.c | 1 + tools/perf/builtin-record.c | 8 ++++++ tools/perf/builtin-script.c | 15 ++++++++-- tools/perf/util/bpf-event.c | 46 ++++++++++++++++++++++++++++++ tools/perf/util/bpf-event.h | 1 + tools/perf/util/env.c | 19 +++++++++++- tools/perf/util/env.h | 4 +++ tools/perf/util/event.c | 21 ++++++++++++++ tools/perf/util/event.h | 1 + tools/perf/util/header.c | 1 + tools/perf/util/session.c | 4 +++ tools/perf/util/synthetic-events.h | 2 ++ tools/perf/util/tool.c | 14 +++++++++ tools/perf/util/tool.h | 3 +- 14 files changed, 136 insertions(+), 4 deletions(-)
tools/perf/builtin-inject.c +1 -0
@@ -2530,6 +2530,7 @@ int cmd_inject(int argc, const char **argv) inject.tool.finished_init = perf_event__repipe_op2_synth; inject.tool.compressed = perf_event__repipe_op4_synth; inject.tool.auxtrace = perf_event__repipe_auxtrace; + inject.tool.bpf_metadata = perf_event__repipe_op2_synth; inject.tool.dont_split_sample_group = true; inject.session =__perf_session__new (&data, &inject.tool,/*trace_event_repipe=*/ inject.output.is_pipe);
tools/perf/builtin-record.c +8 -0
@@ -2161,6 +2161,12 @@ static int record__synthesize(struct record *rec, bool tail)return err; } +static void record__synthesize_final_bpf_metadata (struct record *rec) +{ +perf_event__synthesize_final_bpf_metadata (rec->session, + process_synthesized_event); +} +static int record__process_signal_event (union perf_event *event__maybe_unused ,void *data) {struct record *rec = data;
@@ -2806,6 +2812,8 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)trigger_off (&auxtrace_snapshot_trigger);trigger_off (&switch_output_trigger); +record__synthesize_final_bpf_metadata (rec); +if (opts->auxtrace_snapshot_on_exit)record__auxtrace_snapshot_exit (rec);
tools/perf/builtin-script.c +13 -2
@@ -38,6 +38,7 @@#include "print_insn.h" #include "archinsn.h" #include <linux/bitmap.h> +#include <linux/compiler.h> #include <linux/kernel.h> #include <linux/stringify.h> #include <linux/time64.h>
@@ -50,6 +51,7 @@#include <errno.h> #include <inttypes.h> #include <signal.h> +#include <stdio.h> #include <sys/param.h> #include <sys/types.h> #include <sys/stat.h>
@@ -2755,6 +2757,14 @@ process_bpf_events(const struct perf_tool *tool __maybe_unused, sample->tid); } +static int +process_bpf_metadata_event (struct perf_session *session__maybe_unused , +union perf_event *event) +{ +perf_event__fprintf (event,NULL , stdout); +return 0 ; +} + static int process_text_poke_events(const struct perf_tool *tool, union perf_event *event, struct perf_sample *sample,
@@ -2877,8 +2887,9 @@ static int __cmd_script(struct perf_script *script) script->tool.finished_round = process_finished_round_event; }if (script->show_bpf_events) { - script->tool.ksymbol = process_bpf_events; - script->tool.bpf = process_bpf_events; + script->tool.ksymbol = process_bpf_events; + script->tool.bpf = process_bpf_events; + script->tool.bpf_metadata = process_bpf_metadata_event; }if (script->show_text_poke_events) { script->tool.ksymbol = process_bpf_events;
tools/perf/util/bpf-event.c +46 -0
70 lines not on this page · raw message
tools/perf/util/bpf-event.h +1 -0
@@ -25,6 +25,7 @@ struct bpf_metadata {struct bpf_prog_info_node {struct perf_bpil *info_linear; +struct bpf_metadata *metadata;struct rb_node rb_node; };
tools/perf/util/env.c +18 -1
@@ -3,8 +3,10 @@#include "debug.h" #include "env.h" #include "util/header.h" -#include "linux/compiler.h" +#include "util/rwsem.h" +#include <linux/compiler.h> #include <linux/ctype.h> +#include <linux/rbtree.h> #include <linux/string.h> #include <linux/zalloc.h> #include "cgroup.h"
@@ -89,6 +91,20 @@ struct bpf_prog_info_node *perf_env__find_bpf_prog_info(struct perf_env *env,return node; } +void perf_env__iterate_bpf_prog_info (struct perf_env *env, +void (*cb)(struct bpf_prog_info_node *node, +void *data), +void *data) +{ +struct rb_node *first; + +down_read (&env->bpf_progs.lock); + first =rb_first (&env->bpf_progs.infos); +for (struct rb_node *node = first; node !=NULL ; node =rb_next (node)) + (*cb)(rb_entry(node, struct bpf_prog_info_node, rb_node), data); +up_read (&env->bpf_progs.lock); +} +bool perf_env__insert_btf (struct perf_env *env,struct btf_node *btf_node) {bool ret;
@@ -174,6 +190,7 @@ static void perf_env__purge_bpf(struct perf_env *env) next =rb_next (&node->rb_node);rb_erase (&node->rb_node, root);zfree (&node->info_linear); +bpf_metadata_free (node->metadata);free (node); }
tools/perf/util/env.h +4 -0
@@ -180,6 +180,10 @@ bool perf_env__insert_bpf_prog_info(struct perf_env *env, struct bpf_prog_info_node *info_node);struct bpf_prog_info_node *perf_env__find_bpf_prog_info (struct perf_env *env,__u32 prog_id); +void perf_env__iterate_bpf_prog_info (struct perf_env *env, +void (*cb)(struct bpf_prog_info_node *node, +void *data), +void *data);bool perf_env__insert_btf (struct perf_env *env,struct btf_node *btf_node);bool __perf_env__insert_btf (struct perf_env *env,struct btf_node *btf_node);struct btf_node *perf_env__find_btf (struct perf_env *env,__u32 btf_id);
tools/perf/util/event.c +21 -0
@@ -1,9 +1,12 @@#include <errno.h> #include <fcntl.h> #include <inttypes.h> +#include <linux/compiler.h> #include <linux/kernel.h> #include <linux/types.h> #include <perf/cpumap.h> +#include <perf/event.h> +#include <stdio.h> #include <sys/types.h> #include <sys/stat.h> #include <unistd.h>
@@ -78,6 +81,7 @@ static const char *perf_event__names[] = { [PERF_RECORD_COMPRESSED] = "COMPRESSED", [PERF_RECORD_FINISHED_INIT] = "FINISHED_INIT", [PERF_RECORD_COMPRESSED2] = "COMPRESSED2", + [PERF_RECORD_BPF_METADATA] = "BPF_METADATA", };const char *perf_event__name (unsigned int id)
@@ -504,6 +508,20 @@ size_t perf_event__fprintf_bpf(union perf_event *event, FILE *fp) event->bpf.type, event->bpf.flags, event->bpf.id); } +size_t perf_event__fprintf_bpf_metadata (union perf_event *event,FILE *fp) +{ +struct perf_record_bpf_metadata *metadata = &event->bpf_metadata; +size_t ret; + + ret =fprintf (fp," prog %s\n" , metadata->prog_name); +for (__u32 i =0 ; i < metadata->nr_entries; i++) { + ret +=fprintf (fp," entry %d: %20s = %s\n" , i, + metadata->entries[i].key, + metadata->entries[i].value); + } +return ret; +} +static int text_poke_printer (enum binary_printer_ops op,unsigned int val,void *extra,FILE *fp) {
@@ -601,6 +619,9 @@ size_t perf_event__fprintf(union perf_event *event, struct machine *machine, FILcase PERF_RECORD_AUX_OUTPUT_HW_ID : ret +=perf_event__fprintf_aux_output_hw_id (event, fp);break ; +case PERF_RECORD_BPF_METADATA : + ret +=perf_event__fprintf_bpf_metadata (event, fp); +break ;default : ret +=fprintf (fp,"\n" ); }
tools/perf/util/event.h +1 -0
@@ -370,6 +370,7 @@ size_t perf_event__fprintf_namespaces(union perf_event *event, FILE *fp);size_t perf_event__fprintf_cgroup (union perf_event *event,FILE *fp);size_t perf_event__fprintf_ksymbol (union perf_event *event,FILE *fp);size_t perf_event__fprintf_bpf (union perf_event *event,FILE *fp); +size_t perf_event__fprintf_bpf_metadata (union perf_event *event,FILE *fp);size_t perf_event__fprintf_text_poke (union perf_event *event,struct machine *machine,FILE *fp);size_t perf_event__fprintf (union perf_event *event,struct machine *machine,FILE *fp);
tools/perf/util/header.c +1 -0
@@ -3161,6 +3161,7 @@ static int process_bpf_prog_info(struct feat_fd *ff, void *data __maybe_unused)/* after reading from file, translate offset to address */ bpil_offs_to_addr (info_linear); info_node->info_linear = info_linear; + info_node->metadata =NULL ;if (!__perf_env__insert_bpf_prog_info (env, info_node)) {free (info_linear);free (info_node);
tools/perf/util/session.c +4 -0
@@ -12,6 +12,7 @@#include <sys/types.h> #include <sys/mman.h> #include <perf/cpumap.h> +#include <perf/event.h> #include "map_symbol.h" #include "branch.h"
@@ -1491,6 +1492,9 @@ static s64 perf_session__process_user_event(struct perf_session *session,case PERF_RECORD_FINISHED_INIT : err = tool->finished_init (session, event);break ; +case PERF_RECORD_BPF_METADATA : + err = tool->bpf_metadata (session, event); +break ;default : err = -EINVAL ;break ;
tools/perf/util/synthetic-events.h +2 -0
@@ -92,6 +92,8 @@ int perf_event__synthesize_threads(const struct perf_tool *tool, perf_event__hanint perf_event__synthesize_tracing_data (const struct perf_tool *tool,int fd,struct evlist *evlist,perf_event__handler_t process);int perf_event__synth_time_conv (const struct perf_event_mmap_page *pc,const struct perf_tool *tool,perf_event__handler_t process,struct machine *machine);pid_t perf_event__synthesize_comm (const struct perf_tool *tool,union perf_event *event,pid_t pid,perf_event__handler_t process,struct machine *machine); +void perf_event__synthesize_final_bpf_metadata (struct perf_session *session, +perf_event__handler_t process);int perf_tool__process_synth_event (const struct perf_tool *tool,union perf_event *event,struct machine *machine,perf_event__handler_t process);
tools/perf/util/tool.c +14 -0
@@ -1,12 +1,15 @@// SPDX-License-Identifier: GPL-2.0 #include "data.h" #include "debug.h" +#include "event.h" #include "header.h" #include "session.h" #include "stat.h" #include "tool.h" #include "tsc.h" +#include <linux/compiler.h> #include <sys/mman.h> +#include <stddef.h> #include <unistd.h> #ifdef HAVE_ZSTD_SUPPORT
@@ -237,6 +240,16 @@ static int perf_session__process_compressed_event_stub(struct perf_session *sessreturn 0 ; } +static int perf_event__process_bpf_metadata_stub (struct perf_session *perf_session__maybe_unused , +union perf_event *event) +{ +if (dump_trace) +perf_event__fprintf_bpf_metadata (event, stdout); + +dump_printf (": unhandled!\n" ); +return 0 ; +} +void perf_tool__init (struct perf_tool *tool,bool ordered_events) { tool->ordered_events = ordered_events;
@@ -293,6 +306,7 @@ void perf_tool__init(struct perf_tool *tool, bool ordered_events) tool->compressed = perf_session__process_compressed_event_stub;#endif tool->finished_init = process_event_op2_stub; + tool->bpf_metadata = perf_event__process_bpf_metadata_stub; }bool perf_tool__compressed_is_stub (const struct perf_tool *tool)
tools/perf/util/tool.h +2 -1
@@ -77,7 +77,8 @@ struct perf_tool { stat, stat_round, feature, - finished_init; + finished_init, + bpf_metadata;event_op4 compressed;event_op3 auxtrace;bool ordered_events;
Signature · 1 line
2.50.0.rc0.604.gd4ff7b7c86-goog
Namhyung Kim
maintainerRe: [PATCH v2 3/4] perf: collect BPF metadata from new programs, and display the new event
Hi Blake,
Quoted 3 lines · Blake Jones
This collects metadata for any BPF programs that were loaded during a "perf record" run, and emits it at the end of the run. It also adds support for displaying the new
PERF_RECORD_BPF_METADATAtype.
Can you please split the final synthesis and the processing/display parts? I prefer commits doing one at a time.
Quoted 20 lines
Here's some example "perf script -D" output for the new event type. The ": unhandled!" message is from
tool.c, analogous to other behavior there. I've elided some rows with all NUL characters for brevity, and I wrapped one of the >75-column lines to fit in the commit guidelines.[email protected] [
0x260]: event: 84
.
. ... raw event: size 608 bytes. 0000: 54 00 00 00 00 00 60 02 62 70 66 5f 70 72 6f 67 T.....`.bpf_prog . 0010: 5f 31 65 30 61 32 65 33 36 36 65 35 36 66 31 61 _1e0a2e366e56f1a . 0020: 32 5f 70 65 72 66 5f 73 61 6d 70 6c 65 5f 66 69 2_perf_sample_fi . 0030: 6c 74 65 72 00 00 00 00 00 00 00 00 00 00 00 00 lter............ . 0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [...] . 0110: 74 65 73 74 5f 76 61 6c 75 65 00 00 00 00 00 00 test_value...... . 0120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [...] . 0150: 34 32 00 00 00 00 00 00 00 00 00 00 00 00 00 00 42.............. . 0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
Quoted 6 lines
[...]
0 0x50fc8 [0x260]: PERF_RECORD_BPF_METADATA \ prog bpf_prog_1e0a2e366e56f1a2_perf_sample_filterentry 0:
test_value= 42
: unhandled!
Looks good, thanks!
Namhyung
Re: [PATCH v2 3/4] perf: collect BPF metadata from new programs, and display the new event
Hi Namhyung,
Quoted 2 lines · Namhyung Kim
Can you please split the final synthesis and the processing/display parts? I prefer commits doing one at a time.
Sorry about that - when I was doing development these two parts seemed inextricably tied up, but when I went back over them just now they were easy to separate. v3 will have these split out.
Blake
PATCH v2 4/4 perf: add test for PERF_RECORD_BPF_METADATA collection
In reply to Blake JonesThis is an end-to-end test for the PERF_RECORD_BPF_METADATA support. It adds a new "bpf_metadata_perf_version" variable to perf's BPF programs, so that when they are loaded, there will be at least one BPF program with some metadata to parse. The test invokes "perf record" in a way that loads one of those BPF programs, and then sifts through the output to find its BPF metadata.
--- tools/perf/Makefile.perf | 3 +- tools/perf/tests/shell/test_bpf_metadata.sh | 76 +++++++++++++++++++++ tools/perf/util/bpf_skel/perf_version.h | 17 +++++ 3 files changed, 95 insertions(+), 1 deletion(-) create mode 100755 tools/perf/tests/shell/test_bpf_metadata.sh create mode 100644 tools/perf/util/bpf_skel/perf_version.h
tools/perf/Makefile.perf +2 -1
@@ -1250,8 +1250,9 @@ else $(Q)cp "$(VMLINUX_H)" $@ endif -$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h | $(SKEL_TMP_OUT) +$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(OUTPUT)PERF-VERSION-FILE util/bpf_skel/perf_version.h $(LIBBPF) $(SKEL_OUT)/vmlinux.h | $(SKEL_TMP_OUT) $(QUIET_CLANG)$(CLANG) -g -O2 --target=bpf $(CLANG_OPTIONS) $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \ + -include $(OUTPUT)PERF-VERSION-FILE -include util/bpf_skel/perf_version.h \ -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@ $(SKEL_OUT)/%.skel.h: $(SKEL_TMP_OUT)/%.bpf.o | $(BPFTOOL)
tools/perf/tests/shell/test_bpf_metadata.sh new file +76 -0
76 lines not on this page · raw message
tools/perf/util/bpf_skel/perf_version.h new file +17 -0
@@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) */ + +#ifndef __PERF_VERSION_H__ +#define __PERF_VERSION_H__ + +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> + +/* +* This is used by tests/shell/record_bpf_metadata.sh +* to verify that BPF metadata generation works. +* +* PERF_VERSION is defined by a build rule at compile time. +*/ +const char bpf_metadata_perf_version[] SEC(".rodata") = PERF_VERSION; + +#endif /* __PERF_VERSION_H__ */
Signature · 1 line
2.50.0.rc0.604.gd4ff7b7c86-goog
Ian Rogers
reviewerRe: [PATCH v2 4/4] perf: add test for PERF_RECORD_BPF_METADATA collection
Quoted 38 lines · Blake Jones
This is an end-to-end test for the
PERF_RECORD_BPF_METADATAsupport. It adds a new "bpf_metadata_perf_version" variable to perf's BPF programs, so that when they are loaded, there will be at least one BPF program with some metadata to parse. The test invokes "perf record" in a way that loads one of those BPF programs, and then sifts through the output to find its BPF metadata.Signed-off-by Blake Jones <[email protected]>--- tools/perf/Makefile.perf | 3 +- tools/perf/tests/shell/test_bpf_metadata.sh | 76 +++++++++++++++++++++ tools/perf/util/bpf_skel/perf_version.h | 17 +++++ 3 files changed, 95 insertions(+), 1 deletion(-) create mode 100755 tools/perf/tests/shell/test_bpf_metadata.sh create mode 100644 tools/perf/util/bpf_skel/perf_version.htools/perf/Makefile.perf +2
-1@@ -1250,8 +1250,9 @@ else $(Q)cp "$(VMLINUX_H)" $@ endif -$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h | $(SKEL_TMP_OUT) +$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(OUTPUT)PERF-VERSION-FILE util/bpf_skel/perf_version.h $(LIBBPF) $(SKEL_OUT)/vmlinux.h | $(SKEL_TMP_OUT) $(QUIET_CLANG)$(CLANG) -g -O2 --target=bpf $(CLANG_OPTIONS) $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \ + -include $(OUTPUT)PERF-VERSION-FILE -include util/bpf_skel/perf_version.h \ -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@ $(SKEL_OUT)/%.skel.h: $(SKEL_TMP_OUT)/%.bpf.o | $(BPFTOOL)tools/perf/tests/shell/test_bpf_metadata.sh new file +0
-0
tools/perf/tests/shell/test_bpf_metadata.sh line 2
Quoted 2 lines
tools/perf/tests/shell/test_bpf_metadata.sh new file +2
-0@@ -0,0 +1,76 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0
The 2nd line in a shell test script is taken to be the name of the test, so
``` $ perf test list 108 108: SPDX-License-Identifier: GPL-2.0 ```
tools/perf/tests/shell/test_bpf_metadata.sh line 4
Quoted 2 lines
tools/perf/tests/shell/test_bpf_metadata.sh new file +2
-0+# +# BPF metadata collection test.
This should be on line 2 instead.
Thanks,
Ian
Quoted 98 lines
tools/perf/tests/shell/test_bpf_metadata.sh new file +72
-0+ +set -e + +err=0 +perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX) + +cleanup() { + rm -f "${perfdata}" + rm -f "${perfdata}".old + trap - EXIT TERM INT +} + +trap_cleanup() { + cleanup + exit 1 +} +trap trap_cleanup EXIT TERM INT + +test_bpf_metadata() { + echo "Checking BPF metadata collection" + + if ! perf check -q feature libbpf-strings ; then + echo "Basic BPF metadata test [skipping - not supported]" + err=0 + return + fi + + # This is a basic invocation of perf record + # that invokes the perf_sample_filter BPF program. + if ! perf record -e task-clock --filter 'ip > 0' \ + -o "${perfdata}" sleep 1 2> /dev/null + then + echo "Basic BPF metadata test [Failed record]" + err=1 + return + fi + + # The BPF programs that ship with "perf" all have the following + # variable defined at compile time: + # + # const char bpf_metadata_perf_version[] SEC(".rodata") = <...>; + # + # This invocation looks for a PERF_RECORD_BPF_METADATA event, + # and checks that its content contains the string given by + # "perf version". + VERS=$(perf version | awk '{print $NF}') + if ! perf script --show-bpf-events -i "${perfdata}" | awk ' + /PERF_RECORD_BPF_METADATA.*perf_sample_filter/ { + header = 1; + } + /^ *entry/ { + if (header) { header = 0; entry = 1; } + } + $0 !~ /^ *entry/ { + entry = 0; + } + /perf_version/ { + if (entry) print $NF; + } + ' | egrep "$VERS" > /dev/null + then + echo "Basic BPF metadata test [Failed invalid output]" + err=1 + return + fi + echo "Basic BPF metadata test [Success]" +} + +test_bpf_metadata + +cleanup +exit $errtools/perf/util/bpf_skel/perf_version.h new file +17
-1@@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) */ + +#ifndef __PERF_VERSION_H__ +#define __PERF_VERSION_H__ + +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> + +/* +* This is used by tests/shell/record_bpf_metadata.sh +* to verify that BPF metadata generation works. +* +* PERF_VERSION is defined by a build rule at compile time. +*/ +const char bpf_metadata_perf_version[] SEC(".rodata") = PERF_VERSION; + +#endif /* __PERF_VERSION_H__ */ --2.50.0.rc0.604.gd4ff7b7c86-goog
Re: [PATCH v2 4/4] perf: add test for PERF_RECORD_BPF_METADATA collection
Quoted 11 lines · Ian Rogers
tools/perf/tests/shell/test_bpf_metadata.sh line 2
Quoted 4 lines
+++
b/tools/perf/tests/shell/test_bpf_metadata.shtools/perf/tests/shell/test_bpf_metadata.sh new file +2
-0@@ -0,0 +1,76 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0The 2nd line in a shell test script is taken to be the name of the test, so
``` $ perf test list 108 108: SPDX-License-Identifier: GPL-2.0 ```
Quoted 4 lines
tools/perf/tests/shell/test_bpf_metadata.sh line 4
Quoted 2 lines
tools/perf/tests/shell/test_bpf_metadata.sh new file +2
-0+# +# BPF metadata collection test.This should be on line 2 instead.
Oof, that sure wasn't on my radar. Should I do a followup patch, or is it not worth bothering?
Ian Rogers
reviewerRe: [PATCH v2 4/4] perf: add test for PERF_RECORD_BPF_METADATA collection
Quoted 13 lines · Blake Jones
Quoted 11 lines · Ian Rogers
tools/perf/tests/shell/test_bpf_metadata.sh line 2
Quoted 4 lines
+++
b/tools/perf/tests/shell/test_bpf_metadata.shtools/perf/tests/shell/test_bpf_metadata.sh new file +2
-0@@ -0,0 +1,76 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0The 2nd line in a shell test script is taken to be the name of the test, so
``` $ perf test list 108 108: SPDX-License-Identifier: GPL-2.0 ```
Quoted 7 lines
Quoted 4 lines
tools/perf/tests/shell/test_bpf_metadata.sh line 4
Quoted 2 lines
tools/perf/tests/shell/test_bpf_metadata.sh new file +2
-0+# +# BPF metadata collection test.This should be on line 2 instead.
Oof, that sure wasn't on my radar. Should I do a followup patch, or is it not worth bothering?
The patch has been in perf-tools-next for a few weeks:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/commit/?h=perf-tools-next&id=edf2cadf01e8f2620af25b337d15ebc584911b46
so modifying it is probably not a good idea (it'd need a forced push and break people downstream). If you could send a follow up, that'd be great just so that we have >1 person in the author/reviewer/signed-off-by tag!
Thanks,
Ian
Re: [PATCH v2 4/4] perf: add test for PERF_RECORD_BPF_METADATA collection
I just sent https://lore.kernel.org/linux-perf-users/[email protected]/T/#u
to address this issue. Sorry for the delay.
Quoted 32 lines · Ian Rogers
Quoted 13 lines · Blake Jones
Quoted 11 lines · Ian Rogers
tools/perf/tests/shell/test_bpf_metadata.sh line 2
Quoted 4 lines
+++
b/tools/perf/tests/shell/test_bpf_metadata.shtools/perf/tests/shell/test_bpf_metadata.sh new file +2
-0@@ -0,0 +1,76 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0The 2nd line in a shell test script is taken to be the name of the test, so
``` $ perf test list 108 108: SPDX-License-Identifier: GPL-2.0 ```Quoted 7 lines
Quoted 4 lines
tools/perf/tests/shell/test_bpf_metadata.sh line 4
Quoted 2 lines
tools/perf/tests/shell/test_bpf_metadata.sh new file +2
-0+# +# BPF metadata collection test.This should be on line 2 instead.
Oof, that sure wasn't on my radar. Should I do a followup patch, or is it not worth bothering?
The patch has been in perf-tools-next for a few weeks:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/commit/?h=perf-tools-next&id=edf2cadf01e8f2620af25b337d15ebc584911b46
so modifying it is probably not a good idea (it'd need a forced push and break people downstream). If you could send a follow up, that'd be great just so that we have >1 person in the author/reviewer/signed-off-by tag!Thanks,
Ian