Skip to contentSkip to conversation

Building perf with BUILD_BPF_SKEL=1 by default

DISCUSSION Arnaldo Carvalho de Melo 3 participants 16 messages Dec 3, 2021 – Dec 7, 2021 perf events

lore
Share & tools
lore

Arnaldo Carvalho de Melo

author maintainer
#0 lore raw

Hi Song,

So I'm changing all my containers to build with BUILD_BPF_SKEL=1
to then make this the default, so far older containers fail either because the clang available is too old, so I've added a NO_BUILD_BPF_SKEL=1 env var to disable that in those containers and then there is this other case where clang is recent enough but:

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'
__uint(map_flags, BPF_F_PRESERVE_ELEMS);

Because the system's /usr/include/linux/bpf.h doesn't have that BPF_F_PRESERVE_ELEMS enum entry.

These are enums to make them available via BTF, but then I can't use the:

#ifdef BPF_F_PRESERVE_ELEMS
#define BPF_F_PRESERVE_ELEMS (1U << 11)
#endif

approach.

But then we _have_ it in the tools/include/uapi/linux/bpf.h we ship:

$ grep BPF_F_PRESERVE_ELEMS tools/include/uapi/linux/bpf.h
	BPF_F_PRESERVE_ELEMS	= (1U << 11),
$

so we need to switch to using it somehow, this way we can build in more systems and make bperf and other BPF enabled features.

From a quick look I couldn't find where to add $(sourcedir)/tools/include/uapi/ to the include path used to build util/bpf_skel/bperf_leader.bpf.c, should be easy, can you take a look?

Thanks,

- Arnaldo

author maintainer
#1 lore raw

Em Fri, Dec 03, 2021 at 10:10:42AM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 29 lines

Hi Song,

So I'm changing all my containers to build with BUILD_BPF_SKEL=1
to then make this the default, so far older containers fail either because the clang available is too old, so I've added a NO_BUILD_BPF_SKEL=1 env var to disable that in those containers and then there is this other case where clang is recent enough but:

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'
__uint(map_flags, BPF_F_PRESERVE_ELEMS);

Because the system's /usr/include/linux/bpf.h doesn't have that BPF_F_PRESERVE_ELEMS enum entry.

These are enums to make them available via BTF, but then I can't use the:

#ifdef BPF_F_PRESERVE_ELEMS
#define BPF_F_PRESERVE_ELEMS (1U << 11)
#endif

approach.

But then we _have_ it in the tools/include/uapi/linux/bpf.h we ship:

$ grep BPF_F_PRESERVE_ELEMS tools/include/uapi/linux/bpf.h
	BPF_F_PRESERVE_ELEMS	= (1U << 11),
$
Quoted 6 lines

so we need to switch to using it somehow, this way we can build in more systems and make bperf and other BPF enabled features.

From a quick look I couldn't find where to add $(sourcedir)/tools/include/uapi/ to the include path used to build util/bpf_skel/bperf_leader.bpf.c, should be easy, can you take a look?

The failure for fedora:32, which seems to be on the threshold for clang to be considered recent enough for building skels fails differently:

clang version 10.0.1 (Fedora 10.0.1-3.fc32)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/10 Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/10 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/10 Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
+ '[' '!' ']'
+ rm -rf /tmp/build/perf
+ mkdir /tmp/build/perf
+ make ARCH= CROSS_COMPILE= EXTRA_CFLAGS= BUILD_BPF_SKEL=1 -C tools/perf O=/tmp/build/perf
make: Entering directory '/git/perf-5.16.0-rc3/tools/perf'
BUILD: Doing 'make -j32' parallel build

  HOSTCC  /tmp/build/perf/fixdep.o
  HOSTLD  /tmp/build/perf/fixdep-in.o
  LINK    /tmp/build/perf/fixdep

/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `expr bison (GNU Bison) 3.5 \>\= 371'
Makefile.config:997: No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-dev[el]/libbabeltrace-ctf-dev

Auto-detecting system features:

...                         dwarf: [ on  ]
...            dwarf_getlocations: [ on  ]
...                         glibc: [ on  ]
...                        libbfd: [ on  ]
...                libbfd-buildid: [ on  ]
...                        libcap: [ on  ]
...                        libelf: [ on  ]
...                       libnuma: [ on  ]
...        numa_num_possible_cpus: [ on  ]
...                       libperl: [ on  ]
...                     libpython: [ on  ]
...                     libcrypto: [ on  ]
...                     libunwind: [ on  ]
...            libdw-dwarf-unwind: [ on  ]
...                          zlib: [ on  ]
...                          lzma: [ on  ]
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]
...                        libaio: [ on  ]
...                       libzstd: [ on  ]
...        disassembler-four-args: [ on  ]

GEN /tmp/build/perf/common-cmds.h
PERF_VERSION = 5.16.rc3.g0c338bcdfdb1

  CC      /tmp/build/perf/exec-cmd.o
  CC      /tmp/build/perf/help.o
  CC      /tmp/build/perf/cpu.o
  CC      /tmp/build/perf/pager.o
  CC      /tmp/build/perf/debug.o
  GEN     perf-archive
  CC      /tmp/build/perf/parse-options.o
  CC      /tmp/build/perf/str_error_r.o
  GEN     perf-with-kcore
  CC      /tmp/build/perf/run-command.o
  CC      /tmp/build/perf/sigchain.o
  GEN     perf-iostat
  MKDIR   /tmp/build/perf/fd/
  MKDIR   /tmp/build/perf/fs/
  CC      /tmp/build/perf/subcmd-config.o
  MKDIR   /tmp/build/perf/jvmti/
  MKDIR   /tmp/build/perf/fs/
  MKDIR   /tmp/build/perf/jvmti/
  CC      /tmp/build/perf/dlfilters/dlfilter-test-api-v0.o
  CC      /tmp/build/perf/dlfilters/dlfilter-show-cycles.o
  MKDIR   /tmp/build/perf/jvmti/
  CC      /tmp/build/perf/fd/array.o
  MKDIR   /tmp/build/perf/fs/
  CC      /tmp/build/perf/fs/fs.o
  MKDIR   /tmp/build/perf/jvmti/
  MKDIR   /tmp/build/perf/pmu-events/
  CC      /tmp/build/perf/event-parse.o
  MKDIR   /tmp/build/perf/pmu-events/
  CC      /tmp/build/perf/jvmti/libjvmti.o
  CC      /tmp/build/perf/jvmti/jvmti_agent.o
  CC      /tmp/build/perf/fs/tracing_path.o
  CC      /tmp/build/perf/jvmti/libstring.o
  CC      /tmp/build/perf/fs/cgroup.o
  HOSTCC  /tmp/build/perf/pmu-events/json.o
  CC      /tmp/build/perf/jvmti/libctype.o
  CC      /tmp/build/perf/core.o
  HOSTCC  /tmp/build/perf/pmu-events/jsmn.o
  MKDIR   /tmp/build/perf/pmu-events/
  HOSTCC  /tmp/build/perf/pmu-events/jevents.o
  CC      /tmp/build/perf/cpumap.o
  CC      /tmp/build/perf/plugin_jbd2.o
  CC      /tmp/build/perf/plugin_hrtimer.o
  CC      /tmp/build/perf/plugin_kmem.o
  CC      /tmp/build/perf/threadmap.o
  CC      /tmp/build/perf/plugin_kvm.o
  CC      /tmp/build/perf/evsel.o
  CC      /tmp/build/perf/plugin_mac80211.o
  LINK    /tmp/build/perf/dlfilters/dlfilter-show-cycles.so
  CC      /tmp/build/perf/plugin_sched_switch.o
make[3]: *** No rule to make target 'kernelversion'.  Stop.
  CC      /tmp/build/perf/plugin_function.o
  LD      /tmp/build/perf/fd/libapi-in.o
  CC      /tmp/build/perf/event-plugin.o
  GEN     /tmp/build/perf/libbpf/bpf_helper_defs.h
  CC      /tmp/build/perf/plugin_futex.o
  CC      /tmp/build/perf/plugin_xen.o
  CC      /tmp/build/perf/plugin_scsi.o
  CC      /tmp/build/perf/evlist.o
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf.h
  LD      /tmp/build/perf/plugin_jbd2-in.o
  CC      /tmp/build/perf/plugin_cfg80211.o
  INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf.h
  LD      /tmp/build/perf/plugin_hrtimer-in.o
  LD      /tmp/build/perf/fs/libapi-in.o
  LD      /tmp/build/perf/plugin_kmem-in.o
  CC      /tmp/build/perf/plugin_tlb.o
  LINK    /tmp/build/perf/dlfilters/dlfilter-test-api-v0.so
  CC      /tmp/build/perf/mmap.o
  CC      /tmp/build/perf/zalloc.o
  LINK    /tmp/build/perf/plugin_jbd2.so
  LINK    /tmp/build/perf/plugin_hrtimer.so
  INSTALL /tmp/build/perf/libbpf/include/bpf/btf.h
  CC      /tmp/build/perf/xyarray.o
  CC      /tmp/build/perf/trace-seq.o
  LD      /tmp/build/perf/plugin_mac80211-in.o
  LD      /tmp/build/perf/libapi-in.o
  LD      /tmp/build/perf/plugin_kvm-in.o
  CC      /tmp/build/perf/parse-filter.o
  CC      /tmp/build/perf/parse-utils.o
  LD      /tmp/build/perf/plugin_sched_switch-in.o
  CC      /tmp/build/perf/kbuffer-parse.o
  CC      /tmp/build/perf/tep_strerror.o
  HOSTLD  /tmp/build/perf/pmu-events/jevents-in.o
  INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf_common.h
  INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf_legacy.h
  CC      /tmp/build/perf/event-parse-api.o
  INSTALL /tmp/build/perf/libbpf/include/bpf/xsk.h
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_helpers.h
  AR      /tmp/build/perf/libapi.a
  LD      /tmp/build/perf/plugin_function-in.o
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_tracing.h
  LINK    /tmp/build/perf/plugin_kmem.so
  LINK    /tmp/build/perf/plugin_mac80211.so
  LINK    /tmp/build/perf/plugin_kvm.so
  LD      /tmp/build/perf/plugin_xen-in.o
  LINK    /tmp/build/perf/plugin_sched_switch.so
  LD      /tmp/build/perf/plugin_cfg80211-in.o
  CC      /tmp/build/perf/lib.o
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_endian.h
  LD      /tmp/build/perf/plugin_futex-in.o
  LD      /tmp/build/perf/plugin_scsi-in.o
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_core_read.h
  LINK    /tmp/build/perf/pmu-events/jevents
  LD      /tmp/build/perf/plugin_tlb-in.o
  LINK    /tmp/build/perf/plugin_function.so
  LINK    /tmp/build/perf/plugin_xen.so
  INSTALL /tmp/build/perf/libbpf/include/bpf/skel_internal.h
  LINK    /tmp/build/perf/plugin_cfg80211.so
  INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf_version.h
  LINK    /tmp/build/perf/plugin_futex.so
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h
  LINK    /tmp/build/perf/plugin_scsi.so
  LD      /tmp/build/perf/jvmti/jvmti-in.o
  LINK    /tmp/build/perf/plugin_tlb.so
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  LD      /tmp/build/perf/libsubcmd-in.o
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  CC      /tmp/build/perf/libbpf/staticobjs/xsk.o
  CC      /tmp/build/perf/libbpf/staticobjs/libbpf.o
  LINK    /tmp/build/perf/libperf-jvmti.so
  CC      /tmp/build/perf/libbpf/staticobjs/bpf.o
  CC      /tmp/build/perf/libbpf/staticobjs/nlattr.o
  CC      /tmp/build/perf/libbpf/staticobjs/btf.o
  CC      /tmp/build/perf/libbpf/staticobjs/libbpf_errno.o
  CC      /tmp/build/perf/libbpf/staticobjs/hashmap.o
  CC      /tmp/build/perf/libbpf/staticobjs/str_error.o
  CC      /tmp/build/perf/libbpf/staticobjs/netlink.o
  CC      /tmp/build/perf/libbpf/staticobjs/btf_dump.o
  CC      /tmp/build/perf/libbpf/staticobjs/bpf_prog_linfo.o
  CC      /tmp/build/perf/libbpf/staticobjs/libbpf_probes.o
  CC      /tmp/build/perf/libbpf/staticobjs/ringbuf.o
  CC      /tmp/build/perf/libbpf/staticobjs/strset.o
  GEN     /tmp/build/perf/libtraceevent-dynamic-list
  CC      /tmp/build/perf/libbpf/staticobjs/linker.o
  CC      /tmp/build/perf/libbpf/staticobjs/gen_loader.o
  CC      /tmp/build/perf/libbpf/staticobjs/relo_core.o
  AR      /tmp/build/perf/libsubcmd.a
  LD      /tmp/build/perf/libtraceevent-in.o
  GEN     /tmp/build/perf/pmu-events/pmu-events.c
  LINK    /tmp/build/perf/libtraceevent.a
  LD      /tmp/build/perf/libperf-in.o
  AR      /tmp/build/perf/libperf.a
  GEN     /tmp/build/perf/python/perf.so
  CC      /tmp/build/perf/pmu-events/pmu-events.o

Auto-detecting system features:

...                        libbfd: [ on  ]
...        disassembler-four-args: [ on  ]
...                          zlib: [ on  ]
...                        libcap: [ on  ]
...               clang-bpf-co-re: [ on  ]
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/hashmap.h
  GEN     /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/bpf_helper_defs.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/btf.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf_common.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf_legacy.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/xsk.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_helpers.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_tracing.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_endian.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_core_read.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/skel_internal.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf_version.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_helper_defs.h
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/btf_dump.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/bpf.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/nlattr.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/btf.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/ringbuf.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf_errno.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/str_error.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/netlink.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/bpf_prog_linfo.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/strset.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf_probes.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/xsk.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/hashmap.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/linker.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/gen_loader.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/relo_core.o
  LD      /tmp/build/perf/pmu-events/pmu-events-in.o
  LD      /tmp/build/perf/libbpf/staticobjs/libbpf-in.o
  LINK    /tmp/build/perf/libbpf/libbpf.a
  LD      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf-in.o
  LINK    /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/libbpf.a
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/common.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/main.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/json_writer.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/gen.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/xlated_dumper.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/btf.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/btf_dumper.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/disasm.o
  LINK    /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/bpftool
  GEN     /tmp/build/perf/util/bpf_skel/vmlinux.h
  CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bpf_prog_profiler.bpf.o
  CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_leader.bpf.o
  CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_follower.bpf.o
  CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_cgroup.bpf.o

GENSKEL /tmp/build/perf/util/bpf_skel/bpf_prog_profiler.skel.h
GENSKEL /tmp/build/perf/util/bpf_skel/bperf_leader.skel.h
libbpf: elf: skipping unrecognized data section(9) .eh_frame
libbpf: elf: skipping relo section(15) .rel.eh_frame for section(9) .eh_frame
libbpf: elf: skipping unrecognized data section(7) .eh_frame
libbpf: elf: skipping relo section(12) .rel.eh_frame for section(7) .eh_frame
libbpf: map 'prev_readings': unexpected def kind var.
Error: failed to open BPF object file: Invalid argument
make[2]: *** [Makefile.perf:1076: /tmp/build/perf/util/bpf_skel/bperf_leader.skel.h] Error 255 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [Makefile.perf:240: sub-make] Error 2
make: *** [Makefile:70: all] Error 2
make: Leaving directory '/git/perf-5.16.0-rc3/tools/perf'
+ exit 1
[perfbuilder@five 32]$

author maintainer
#2 lore raw

Em Fri, Dec 03, 2021 at 10:28:00AM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 33 lines

Em Fri, Dec 03, 2021 at 10:10:42AM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 31 lines

Hi Song,

So I'm changing all my containers to build with BUILD_BPF_SKEL=1
to then make this the default, so far older containers fail either because the clang available is too old, so I've added a NO_BUILD_BPF_SKEL=1 env var to disable that in those containers and then there is this other case where clang is recent enough but:

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'
__uint(map_flags, BPF_F_PRESERVE_ELEMS);

Because the system's /usr/include/linux/bpf.h doesn't have that BPF_F_PRESERVE_ELEMS enum entry.

These are enums to make them available via BTF, but then I can't use the:

#ifdef BPF_F_PRESERVE_ELEMS
#define BPF_F_PRESERVE_ELEMS (1U << 11)
#endif

approach.

But then we _have_ it in the tools/include/uapi/linux/bpf.h we ship:

$ grep BPF_F_PRESERVE_ELEMS tools/include/uapi/linux/bpf.h
	BPF_F_PRESERVE_ELEMS	= (1U << 11),
$

so we need to switch to using it somehow, this way we can build in more systems and make bperf and other BPF enabled features.

Quoted 8 lines
Quoted 3 lines

From a quick look I couldn't find where to add $(sourcedir)/tools/include/uapi/ to the include path used to build util/bpf_skel/bperf_leader.bpf.c, should be easy, can you take a look?

The failure for fedora:32, which seems to be on the threshold for clang to be considered recent enough for building skels fails differently:

clang version 10.0.1 (Fedora 10.0.1-3.fc32)

Amazon Linux 2 fails differently:

clang version 11.1.0 (Amazon Linux 2 11.1.0-1.amzn2.0.2)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7 Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7 Selected GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
+ '[' '!' ']'
+ rm -rf /tmp/build/perf
+ mkdir /tmp/build/perf
+ make ARCH= CROSS_COMPILE= EXTRA_CFLAGS= BUILD_BPF_SKEL=1 -C tools/perf O=/tmp/build/perf
make: Entering directory `/git/perf-5.16.0-rc3/tools/perf'
BUILD: Doing 'make -j32' parallel build

  HOSTCC  /tmp/build/perf/fixdep.o
  HOSTLD  /tmp/build/perf/fixdep-in.o
  LINK    /tmp/build/perf/fixdep

Makefile.config:780: Missing perl devel files. Disabling perl scripting support, please install perl-ExtUtils-Embed/libperl-dev Makefile.config:997: No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-dev[el]/libbabeltrace-ctf-dev

Auto-detecting system features:

...                         dwarf: [ on  ]
...            dwarf_getlocations: [ on  ]
...                         glibc: [ on  ]
...                        libbfd: [ on  ]
...                libbfd-buildid: [ on  ]
...                        libcap: [ on  ]
...                        libelf: [ on  ]
...                       libnuma: [ on  ]
...        numa_num_possible_cpus: [ on  ]
...                       libperl: [ OFF ]
...                     libpython: [ on  ]
...                     libcrypto: [ on  ]
...                     libunwind: [ on  ]
...            libdw-dwarf-unwind: [ on  ]
...                          zlib: [ on  ]
...                          lzma: [ on  ]
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]
...                        libaio: [ on  ]
...                       libzstd: [ on  ]
...        disassembler-four-args: [ on  ]

GEN /tmp/build/perf/common-cmds.h
PERF_VERSION = 5.16.rc3.g0c338bcdfdb1

  GEN     perf-archive
  CC      /tmp/build/perf/cpu.o
  GEN     perf-with-kcore
  MKDIR   /tmp/build/perf/fd/
  CC      /tmp/build/perf/exec-cmd.o
  MKDIR   /tmp/build/perf/fs/
  MKDIR   /tmp/build/perf/jvmti/
  CC      /tmp/build/perf/debug.o
  CC      /tmp/build/perf/help.o
  MKDIR   /tmp/build/perf/fs/
  GEN     perf-iostat
  CC      /tmp/build/perf/str_error_r.o
  CC      /tmp/build/perf/fd/array.o
  CC      /tmp/build/perf/pager.o
  CC      /tmp/build/perf/fs/fs.o
  CC      /tmp/build/perf/jvmti/libjvmti.o
  CC      /tmp/build/perf/parse-options.o
  CC      /tmp/build/perf/fs/tracing_path.o
  CC      /tmp/build/perf/dlfilters/dlfilter-test-api-v0.o
  MKDIR   /tmp/build/perf/jvmti/
  CC      /tmp/build/perf/run-command.o
  MKDIR   /tmp/build/perf/fs/
  CC      /tmp/build/perf/event-parse.o
  CC      /tmp/build/perf/jvmti/libstring.o
  CC      /tmp/build/perf/sigchain.o
  MKDIR   /tmp/build/perf/pmu-events/
  CC      /tmp/build/perf/event-plugin.o
  CC      /tmp/build/perf/subcmd-config.o
  CC      /tmp/build/perf/jvmti/jvmti_agent.o
  MKDIR   /tmp/build/perf/pmu-events/
  CC      /tmp/build/perf/trace-seq.o
  CC      /tmp/build/perf/fs/cgroup.o
  CC      /tmp/build/perf/jvmti/libctype.o
  HOSTCC  /tmp/build/perf/pmu-events/json.o
  CC      /tmp/build/perf/core.o
  HOSTCC  /tmp/build/perf/pmu-events/jsmn.o
  CC      /tmp/build/perf/cpumap.o
  CC      /tmp/build/perf/plugin_jbd2.o
  CC      /tmp/build/perf/dlfilters/dlfilter-show-cycles.o
  MKDIR   /tmp/build/perf/pmu-events/
  CC      /tmp/build/perf/parse-filter.o
  CC      /tmp/build/perf/threadmap.o
  HOSTCC  /tmp/build/perf/pmu-events/jevents.o
  CC      /tmp/build/perf/plugin_hrtimer.o
  CC      /tmp/build/perf/parse-utils.o
  CC      /tmp/build/perf/evsel.o
  CC      /tmp/build/perf/plugin_kmem.o
  CC      /tmp/build/perf/kbuffer-parse.o
  LD      /tmp/build/perf/fd/libapi-in.o
  CC      /tmp/build/perf/evlist.o
  CC      /tmp/build/perf/plugin_kvm.o
  CC      /tmp/build/perf/tep_strerror.o
  CC      /tmp/build/perf/mmap.o
  CC      /tmp/build/perf/event-parse-api.o
  CC      /tmp/build/perf/plugin_mac80211.o
  LD      /tmp/build/perf/plugin_jbd2-in.o
  CC      /tmp/build/perf/zalloc.o
  LINK    /tmp/build/perf/dlfilters/dlfilter-test-api-v0.so
  CC      /tmp/build/perf/plugin_sched_switch.o
  LINK    /tmp/build/perf/dlfilters/dlfilter-show-cycles.so
  CC      /tmp/build/perf/plugin_function.o
  CC      /tmp/build/perf/xyarray.o
  LD      /tmp/build/perf/fs/libapi-in.o
  LD      /tmp/build/perf/plugin_hrtimer-in.o
  CC      /tmp/build/perf/plugin_futex.o
  CC      /tmp/build/perf/lib.o
  CC      /tmp/build/perf/plugin_xen.o
  LD      /tmp/build/perf/libapi-in.o
  LD      /tmp/build/perf/plugin_kmem-in.o
make[3]: *** No rule to make target `kernelversion'.  Stop.
  CC      /tmp/build/perf/plugin_scsi.o
  CC      /tmp/build/perf/plugin_cfg80211.o
  CC      /tmp/build/perf/plugin_tlb.o
  LINK    /tmp/build/perf/plugin_jbd2.so
  GEN     /tmp/build/perf/libbpf/bpf_helper_defs.h
  LINK    /tmp/build/perf/plugin_hrtimer.so
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf.h
  LINK    /tmp/build/perf/plugin_kmem.so
  AR      /tmp/build/perf/libapi.a
  INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf.h
  LD      /tmp/build/perf/plugin_kvm-in.o
  LD      /tmp/build/perf/plugin_sched_switch-in.o
  INSTALL /tmp/build/perf/libbpf/include/bpf/btf.h
  LD      /tmp/build/perf/plugin_mac80211-in.o
  INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf_common.h
  INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf_legacy.h
  INSTALL /tmp/build/perf/libbpf/include/bpf/xsk.h
  HOSTLD  /tmp/build/perf/pmu-events/jevents-in.o
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_helpers.h
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_tracing.h
  LD      /tmp/build/perf/plugin_function-in.o
  LINK    /tmp/build/perf/plugin_kvm.so
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_endian.h
  LINK    /tmp/build/perf/plugin_sched_switch.so
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_core_read.h
  INSTALL /tmp/build/perf/libbpf/include/bpf/skel_internal.h
  LINK    /tmp/build/perf/plugin_mac80211.so
  INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf_version.h
  LINK    /tmp/build/perf/pmu-events/jevents
  LD      /tmp/build/perf/plugin_futex-in.o
  LINK    /tmp/build/perf/plugin_function.so
  LD      /tmp/build/perf/plugin_xen-in.o
  LINK    /tmp/build/perf/plugin_futex.so
  LD      /tmp/build/perf/plugin_cfg80211-in.o
  LD      /tmp/build/perf/plugin_tlb-in.o
  LINK    /tmp/build/perf/plugin_xen.so
  LD      /tmp/build/perf/jvmti/jvmti-in.o
  LINK    /tmp/build/perf/plugin_cfg80211.so
  LINK    /tmp/build/perf/plugin_tlb.so
  LD      /tmp/build/perf/plugin_scsi-in.o
  INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  LINK    /tmp/build/perf/libperf-jvmti.so
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  LINK    /tmp/build/perf/plugin_scsi.so
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/libbpf/staticobjs/
  CC      /tmp/build/perf/libbpf/staticobjs/libbpf.o
  CC      /tmp/build/perf/libbpf/staticobjs/btf_dump.o
  CC      /tmp/build/perf/libbpf/staticobjs/bpf.o
  CC      /tmp/build/perf/libbpf/staticobjs/nlattr.o
  CC      /tmp/build/perf/libbpf/staticobjs/btf.o
  CC      /tmp/build/perf/libbpf/staticobjs/libbpf_errno.o
  CC      /tmp/build/perf/libbpf/staticobjs/str_error.o
  CC      /tmp/build/perf/libbpf/staticobjs/ringbuf.o
  CC      /tmp/build/perf/libbpf/staticobjs/netlink.o
  CC      /tmp/build/perf/libbpf/staticobjs/bpf_prog_linfo.o
  CC      /tmp/build/perf/libbpf/staticobjs/libbpf_probes.o
  CC      /tmp/build/perf/libbpf/staticobjs/xsk.o
  CC      /tmp/build/perf/libbpf/staticobjs/strset.o
  CC      /tmp/build/perf/libbpf/staticobjs/hashmap.o
  CC      /tmp/build/perf/libbpf/staticobjs/linker.o
  CC      /tmp/build/perf/libbpf/staticobjs/gen_loader.o
  CC      /tmp/build/perf/libbpf/staticobjs/relo_core.o
  LD      /tmp/build/perf/libsubcmd-in.o
  GEN     /tmp/build/perf/pmu-events/pmu-events.c
  GEN     /tmp/build/perf/libtraceevent-dynamic-list
  AR      /tmp/build/perf/libsubcmd.a
  LD      /tmp/build/perf/libtraceevent-in.o
  LINK    /tmp/build/perf/libtraceevent.a
  LD      /tmp/build/perf/libperf-in.o
  AR      /tmp/build/perf/libperf.a
  GEN     /tmp/build/perf/python/perf.so
  CC      /tmp/build/perf/pmu-events/pmu-events.o

Auto-detecting system features:

...                        libbfd: [ on  ]
...        disassembler-four-args: [ on  ]
...                          zlib: [ on  ]
...                        libcap: [ on  ]
...               clang-bpf-co-re: [ on  ]
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/hashmap.h
  GEN     /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/bpf_helper_defs.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/btf.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf_common.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf_legacy.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/xsk.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_helpers.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_tracing.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_endian.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_core_read.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/skel_internal.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf_version.h
  INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_helper_defs.h
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/xsk.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/nlattr.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/bpf.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/btf.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf_errno.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/str_error.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/hashmap.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/bpf_prog_linfo.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/netlink.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/btf_dump.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf_probes.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/ringbuf.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/strset.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/linker.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/gen_loader.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/relo_core.o
  LD      /tmp/build/perf/pmu-events/pmu-events-in.o
  LD      /tmp/build/perf/libbpf/staticobjs/libbpf-in.o
  LINK    /tmp/build/perf/libbpf/libbpf.a
  LD      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf-in.o
  LINK    /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/libbpf.a
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/main.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/json_writer.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/gen.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/common.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/btf.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/xlated_dumper.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/btf_dumper.o
  CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/disasm.o
  LINK    /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/bpftool
  GEN     /tmp/build/perf/util/bpf_skel/vmlinux.h
  CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_leader.bpf.o
  CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bpf_prog_profiler.bpf.o
  CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_cgroup.bpf.o
  CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_follower.bpf.o
util/bpf_skel/bpf_prog_profiler.bpf.c:18:21: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value'
        __uint(value_size, sizeof(struct bpf_perf_event_value));
                           ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/build/perf/libbpf/include/bpf/bpf_helpers.h:13:39: note: expanded from macro '__uint'
#define __uint(name, val) int (*name)[val]
                                      ^~~
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:26:21: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value'
        __uint(value_size, sizeof(struct bpf_perf_event_value));
                           ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/build/perf/libbpf/include/bpf/bpf_helpers.h:13:39: note: expanded from macro '__uint'
#define __uint(name, val) int (*name)[val]
                                      ^~~
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:45:59: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value'
        err = bpf_perf_event_read_value(&events, key, ptr, sizeof(*ptr));
                                                                 ^~~~~~
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:55:39: error: variable has incomplete type 'struct bpf_perf_event_value'
        struct bpf_perf_event_value *before, diff;
                                             ^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:60:22: error: incomplete definition of type 'struct bpf_perf_event_value'
        if (before && before->counter) {
                      ~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:63:23: error: incomplete definition of type 'struct bpf_perf_event_value'
                diff.counter = after->counter - before->counter;
                               ~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:63:41: error: incomplete definition of type 'struct bpf_perf_event_value'
                diff.counter = after->counter - before->counter;
                                                ~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:64:23: error: incomplete definition of type 'struct bpf_perf_event_value'
                diff.enabled = after->enabled - before->enabled;
                               ~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:64:41: error: incomplete definition of type 'struct bpf_perf_event_value'
                diff.enabled = after->enabled - before->enabled;
                                                ~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:65:23: error: incomplete definition of type 'struct bpf_perf_event_value'
                diff.running = after->running - before->running;
                               ~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:65:41: error: incomplete definition of type 'struct bpf_perf_event_value'
                diff.running = after->running - before->running;
                                                ~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:69:9: error: incomplete definition of type 'struct bpf_perf_event_value'
                        accum->counter += diff.counter;
                        ~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:70:9: error: incomplete definition of type 'struct bpf_perf_event_value'
                        accum->enabled += diff.enabled;
                        ~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:71:9: error: incomplete definition of type 'struct bpf_perf_event_value'
                        accum->running += diff.running;
                        ~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bpf_prog_profiler.bpf.c:79:30: error: variable has incomplete type 'struct bpf_perf_event_value'
        struct bpf_perf_event_value reading;
                                    ^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^

15 errors generated.
In file included from util/bpf_skel/bperf_follower.bpf.c:7:
util/bpf_skel/bperf.h:10:21: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value' In file included from util/bpf_skel/bperf_leader.bpf.c:7:
util/bpf_skel/bperf.h:10: __uint(value_size, sizeof(struct bpf_perf_event_value));21
: ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value'/tmp/build/perf/libbpf/include/bpf/bpf_helpers.h :13:39: note: expanded from macro '__uint'

#define __uint(name, val) int (*name)[val]
                                      ^~~
        __uint(value_size, sizeof(struct bpf_perf_event_value));
                           ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/build/perf/libbpf/include/bpf/bpf_helpers.h:13:39: note: expanded from macro '__uint'
#define __uint(name, val) int (*name)[val]
                                      ^~~
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_follower.bpf.c:62:11: error: incomplete definition of type 'struct bpf_perf_event_value'
        accum_val->counter += diff_val->counter;
        ~~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_follower.bpf.c:62:32: error: incomplete definition of type 'struct bpf_perf_event_value'
        accum_val->counter += diff_val->counter;
                              ~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_follower.bpf.c:63:11: error: incomplete definition of type 'struct bpf_perf_event_value'
        accum_val->enabled += diff_val->enabled;
        ~~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_follower.bpf.c:63:32: error: incomplete definition of type 'struct bpf_perf_event_value'
        accum_val->enabled += diff_val->enabled;
                              ~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_follower.bpf.c:64:11: error: incomplete definition of type 'struct bpf_perf_event_value'
        accum_val->running += diff_val->running;
        ~~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_follower.bpf.c:64:32: error: incomplete definition of type 'struct bpf_perf_event_value'
        accum_val->running += diff_val->running;
                              ~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'
        __uint(map_flags, BPF_F_PRESERVE_ELEMS);
                          ^
7 errors generated.
util/bpf_skel/bperf_leader.bpf.c:22:30: error: variable has incomplete type 'struct bpf_perf_event_value'
        struct bpf_perf_event_value val, *prev_val, *diff_val;
                                    ^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_leader.bpf.c:39:10: error: incomplete definition of type 'struct bpf_perf_event_value'
        diff_val->counter = val.counter - prev_val->counter;
        ~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_leader.bpf.c:39:44: error: incomplete definition of type 'struct bpf_perf_event_value'
        diff_val->counter = val.counter - prev_val->counter;
                                          ~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_leader.bpf.c:40:10: error: incomplete definition of type 'struct bpf_perf_event_value'
        diff_val->enabled = val.enabled - prev_val->enabled;
        ~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_leader.bpf.c:40:44: error: incomplete definition of type 'struct bpf_perf_event_value'
        diff_val->enabled = val.enabled - prev_val->enabled;
                                          ~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_leader.bpf.c:41:10: error: incomplete definition of type 'struct bpf_perf_event_value'
        diff_val->running = val.running - prev_val->running;
        ~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^
util/bpf_skel/bperf_leader.bpf.c:41:44: error: incomplete definition of type 'struct bpf_perf_event_value'
        diff_val->running = val.running - prev_val->running;
                                          ~~~~~~~~^
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
       ^

9 errors generated.
make[2]: *** [/tmp/build/perf/util/bpf_skel/.tmp/bpf_prog_profiler.bpf.o] Error 1 make[2]: *** Waiting for unfinished jobs....
make[2]: *** [/tmp/build/perf/util/bpf_skel/.tmp/bperf_follower.bpf.o] Error 1 make[2]: *** [/tmp/build/perf/util/bpf_skel/.tmp/bperf_leader.bpf.o] Error 1 make[1]: *** [sub-make] Error 2
make: *** [all] Error 2
make: Leaving directory `/git/perf-5.16.0-rc3/tools/perf'
+ exit 1

    29.38 amazonlinux:2                 : FAIL clang version 11.1.0 (Amazon Linux 2 11.1.0-1.amzn2.0.2)
    util/bpf_skel/bpf_prog_profiler.bpf.c:18:21: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value'
            __uint(value_size, sizeof(struct bpf_perf_event_value));
                               ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    #define __uint(name, val) int (*name)[val]
                                          ^~~
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:26:21: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value'
            __uint(value_size, sizeof(struct bpf_perf_event_value));
                               ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    #define __uint(name, val) int (*name)[val]
                                          ^~~
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:45:59: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value'
            err = bpf_perf_event_read_value(&events, key, ptr, sizeof(*ptr));
                                                                     ^~~~~~
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:55:39: error: variable has incomplete type 'struct bpf_perf_event_value'
            struct bpf_perf_event_value *before, diff;
                                                 ^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:60:22: error: incomplete definition of type 'struct bpf_perf_event_value'
            if (before && before->counter) {
                          ~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:63:23: error: incomplete definition of type 'struct bpf_perf_event_value'
                    diff.counter = after->counter - before->counter;
                                   ~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:63:41: error: incomplete definition of type 'struct bpf_perf_event_value'
                    diff.counter = after->counter - before->counter;
                                                    ~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:64:23: error: incomplete definition of type 'struct bpf_perf_event_value'
                    diff.enabled = after->enabled - before->enabled;
                                   ~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:64:41: error: incomplete definition of type 'struct bpf_perf_event_value'
                    diff.enabled = after->enabled - before->enabled;
                                                    ~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:65:23: error: incomplete definition of type 'struct bpf_perf_event_value'
                    diff.running = after->running - before->running;
                                   ~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:65:41: error: incomplete definition of type 'struct bpf_perf_event_value'
                    diff.running = after->running - before->running;
                                                    ~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:69:9: error: incomplete definition of type 'struct bpf_perf_event_value'
                            accum->counter += diff.counter;
                            ~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:70:9: error: incomplete definition of type 'struct bpf_perf_event_value'
                            accum->enabled += diff.enabled;
                            ~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:71:9: error: incomplete definition of type 'struct bpf_perf_event_value'
                            accum->running += diff.running;
                            ~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bpf_prog_profiler.bpf.c:79:30: error: variable has incomplete type 'struct bpf_perf_event_value'
            struct bpf_perf_event_value reading;
                                        ^
    struct bpf_perf_event_value;
           ^
    15 errors generated.
    In file included from util/bpf_skel/bperf_follower.bpf.c:7:
    util/bpf_skel/bperf.h:10:21: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value'
    In file included from util/bpf_skel/bperf_leader.bpf.c:7:
    util/bpf_skel/bperf.h:10:        __uint(value_size, sizeof(struct bpf_perf_event_value));21
    :                            ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    :13:39: note: expanded from macro '__uint'
    #define __uint(name, val) int (*name)[val]
                                          ^~~
            __uint(value_size, sizeof(struct bpf_perf_event_value));
                               ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    #define __uint(name, val) int (*name)[val]
                                          ^~~
    struct bpf_perf_event_value;
           ^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_follower.bpf.c:62:11: error: incomplete definition of type 'struct bpf_perf_event_value'
            accum_val->counter += diff_val->counter;
            ~~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_follower.bpf.c:62:32: error: incomplete definition of type 'struct bpf_perf_event_value'
            accum_val->counter += diff_val->counter;
                                  ~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_follower.bpf.c:63:11: error: incomplete definition of type 'struct bpf_perf_event_value'
            accum_val->enabled += diff_val->enabled;
            ~~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_follower.bpf.c:63:32: error: incomplete definition of type 'struct bpf_perf_event_value'
            accum_val->enabled += diff_val->enabled;
                                  ~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_follower.bpf.c:64:11: error: incomplete definition of type 'struct bpf_perf_event_value'
            accum_val->running += diff_val->running;
            ~~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_follower.bpf.c:64:32: error: incomplete definition of type 'struct bpf_perf_event_value'
            accum_val->running += diff_val->running;
                                  ~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'
            __uint(map_flags, BPF_F_PRESERVE_ELEMS);
                              ^
    7 errors generated.
    util/bpf_skel/bperf_leader.bpf.c:22:30: error: variable has incomplete type 'struct bpf_perf_event_value'
            struct bpf_perf_event_value val, *prev_val, *diff_val;
                                        ^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_leader.bpf.c:39:10: error: incomplete definition of type 'struct bpf_perf_event_value'
            diff_val->counter = val.counter - prev_val->counter;
            ~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_leader.bpf.c:39:44: error: incomplete definition of type 'struct bpf_perf_event_value'
            diff_val->counter = val.counter - prev_val->counter;
                                              ~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_leader.bpf.c:40:10: error: incomplete definition of type 'struct bpf_perf_event_value'
            diff_val->enabled = val.enabled - prev_val->enabled;
            ~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_leader.bpf.c:40:44: error: incomplete definition of type 'struct bpf_perf_event_value'
            diff_val->enabled = val.enabled - prev_val->enabled;
                                              ~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_leader.bpf.c:41:10: error: incomplete definition of type 'struct bpf_perf_event_value'
            diff_val->running = val.running - prev_val->running;
            ~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    util/bpf_skel/bperf_leader.bpf.c:41:44: error: incomplete definition of type 'struct bpf_perf_event_value'
            diff_val->running = val.running - prev_val->running;
                                              ~~~~~~~~^
    struct bpf_perf_event_value;
           ^
    9 errors generated.

2 29.3792

real 0m30.510s
user 0m0.535s
sys 0m0.485s
[perfbuilder@five 2]$
[perfbuilder@five 2]$ pwd /var/home/perfbuilder/git/linux-perf-tools-build/amazonlinux/2 [perfbuilder@five 2]$

#3 lore raw
Quoted 6 lines · Arnaldo Carvalho de Melo

Em Fri, Dec 03, 2021 at 10:28:00AM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 1 line

Em Fri, Dec 03, 2021 at 10:10:42AM -0300, Arnaldo Carvalho de Melo escreve

[...]

Quoted 14 lines

CLANG /tmp/build/perf/util/bpf_skel/.tmp/bperf_cgroup.bpf.o
CLANG /tmp/build/perf/util/bpf_skel/.tmp/bperf_follower.bpf.o
util/bpf_skel/bpf_prog_profiler.bpf.c:18:21: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value'

       __uint(value_size, sizeof(struct bpf_perf_event_value));
                          ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/build/perf/libbpf/include/bpf/bpf_helpers.h:13:39: note: expanded from macro '__uint'
#define __uint(name, val) int (*name)[val]
                                     ^~~
/tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h:7:8: note: forward declaration of 'struct bpf_perf_event_value'
struct bpf_perf_event_value;
      ^
util/bpf_skel/bpf_prog_profiler.bpf.c:26:21: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value'
       __uint(value_size, sizeof(struct bpf_perf_event_value));
                          ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I guess the vmlinux.h fix should also resolve this one?

Thanks,
Song

#4 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 318 lines · Arnaldo Carvalho de Melo

Em Fri, Dec 03, 2021 at 10:10:42AM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 29 lines

Hi Song,

So I'm changing all my containers to build with BUILD_BPF_SKEL=1
to then make this the default, so far older containers fail either because the clang available is too old, so I've added a NO_BUILD_BPF_SKEL=1 env var to disable that in those containers and then there is this other case where clang is recent enough but:

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'

           __uint(map_flags, BPF_F_PRESERVE_ELEMS);

Because the system's /usr/include/linux/bpf.h doesn't have that BPF_F_PRESERVE_ELEMS enum entry.

These are enums to make them available via BTF, but then I can't use the:

#ifdef BPF_F_PRESERVE_ELEMS
#define BPF_F_PRESERVE_ELEMS (1U << 11)
#endif

approach.

But then we _have_ it in the tools/include/uapi/linux/bpf.h we ship:

$ grep BPF_F_PRESERVE_ELEMS tools/include/uapi/linux/bpf.h
	BPF_F_PRESERVE_ELEMS	= (1U << 11),
$
Quoted 6 lines

so we need to switch to using it somehow, this way we can build in more systems and make bperf and other BPF enabled features.

From a quick look I couldn't find where to add $(sourcedir)/tools/include/uapi/ to the include path used to build util/bpf_skel/bperf_leader.bpf.c, should be easy, can you take a look?

The failure for fedora:32, which seems to be on the threshold for clang to be considered recent enough for building skels fails differently:

clang version 10.0.1 (Fedora 10.0.1-3.fc32)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/10 Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/10 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/10 Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
+ '[' '!' ']'
+ rm -rf /tmp/build/perf
+ mkdir /tmp/build/perf
+ make ARCH= CROSS_COMPILE= EXTRA_CFLAGS= BUILD_BPF_SKEL=1 -C tools/perf O=/tmp/build/perf
make: Entering directory '/git/perf-5.16.0-rc3/tools/perf'
BUILD: Doing 'make -j32' parallel build

 HOSTCC  /tmp/build/perf/fixdep.o
 HOSTLD  /tmp/build/perf/fixdep-in.o
 LINK    /tmp/build/perf/fixdep

/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `expr bison (GNU Bison) 3.5 \>\= 371'
Makefile.config:997: No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-dev[el]/libbabeltrace-ctf-dev

Auto-detecting system features:

...                         dwarf: [ on  ]
...            dwarf_getlocations: [ on  ]
...                         glibc: [ on  ]
...                        libbfd: [ on  ]
...                libbfd-buildid: [ on  ]
...                        libcap: [ on  ]
...                        libelf: [ on  ]
...                       libnuma: [ on  ]
...        numa_num_possible_cpus: [ on  ]
...                       libperl: [ on  ]
...                     libpython: [ on  ]
...                     libcrypto: [ on  ]
...                     libunwind: [ on  ]
...            libdw-dwarf-unwind: [ on  ]
...                          zlib: [ on  ]
...                          lzma: [ on  ]
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]
...                        libaio: [ on  ]
...                       libzstd: [ on  ]
...        disassembler-four-args: [ on  ]

GEN /tmp/build/perf/common-cmds.h
PERF_VERSION = 5.16.rc3.g0c338bcdfdb1

 CC      /tmp/build/perf/exec-cmd.o
 CC      /tmp/build/perf/help.o
 CC      /tmp/build/perf/cpu.o
 CC      /tmp/build/perf/pager.o
 CC      /tmp/build/perf/debug.o
 GEN     perf-archive
 CC      /tmp/build/perf/parse-options.o
 CC      /tmp/build/perf/str_error_r.o
 GEN     perf-with-kcore
 CC      /tmp/build/perf/run-command.o
 CC      /tmp/build/perf/sigchain.o
 GEN     perf-iostat
 MKDIR   /tmp/build/perf/fd/
 MKDIR   /tmp/build/perf/fs/
 CC      /tmp/build/perf/subcmd-config.o
 MKDIR   /tmp/build/perf/jvmti/
 MKDIR   /tmp/build/perf/fs/
 MKDIR   /tmp/build/perf/jvmti/
 CC      /tmp/build/perf/dlfilters/dlfilter-test-api-v0.o
 CC      /tmp/build/perf/dlfilters/dlfilter-show-cycles.o
 MKDIR   /tmp/build/perf/jvmti/
 CC      /tmp/build/perf/fd/array.o
 MKDIR   /tmp/build/perf/fs/
 CC      /tmp/build/perf/fs/fs.o
 MKDIR   /tmp/build/perf/jvmti/
 MKDIR   /tmp/build/perf/pmu-events/
 CC      /tmp/build/perf/event-parse.o
 MKDIR   /tmp/build/perf/pmu-events/
 CC      /tmp/build/perf/jvmti/libjvmti.o
 CC      /tmp/build/perf/jvmti/jvmti_agent.o
 CC      /tmp/build/perf/fs/tracing_path.o
 CC      /tmp/build/perf/jvmti/libstring.o
 CC      /tmp/build/perf/fs/cgroup.o
 HOSTCC  /tmp/build/perf/pmu-events/json.o
 CC      /tmp/build/perf/jvmti/libctype.o
 CC      /tmp/build/perf/core.o
 HOSTCC  /tmp/build/perf/pmu-events/jsmn.o
 MKDIR   /tmp/build/perf/pmu-events/
 HOSTCC  /tmp/build/perf/pmu-events/jevents.o
 CC      /tmp/build/perf/cpumap.o
 CC      /tmp/build/perf/plugin_jbd2.o
 CC      /tmp/build/perf/plugin_hrtimer.o
 CC      /tmp/build/perf/plugin_kmem.o
 CC      /tmp/build/perf/threadmap.o
 CC      /tmp/build/perf/plugin_kvm.o
 CC      /tmp/build/perf/evsel.o
 CC      /tmp/build/perf/plugin_mac80211.o
 LINK    /tmp/build/perf/dlfilters/dlfilter-show-cycles.so
 CC      /tmp/build/perf/plugin_sched_switch.o
make[3]: *** No rule to make target 'kernelversion'.  Stop.
 CC      /tmp/build/perf/plugin_function.o
 LD      /tmp/build/perf/fd/libapi-in.o
 CC      /tmp/build/perf/event-plugin.o
 GEN     /tmp/build/perf/libbpf/bpf_helper_defs.h
 CC      /tmp/build/perf/plugin_futex.o
 CC      /tmp/build/perf/plugin_xen.o
 CC      /tmp/build/perf/plugin_scsi.o
 CC      /tmp/build/perf/evlist.o
 INSTALL /tmp/build/perf/libbpf/include/bpf/bpf.h
 LD      /tmp/build/perf/plugin_jbd2-in.o
 CC      /tmp/build/perf/plugin_cfg80211.o
 INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf.h
 LD      /tmp/build/perf/plugin_hrtimer-in.o
 LD      /tmp/build/perf/fs/libapi-in.o
 LD      /tmp/build/perf/plugin_kmem-in.o
 CC      /tmp/build/perf/plugin_tlb.o
 LINK    /tmp/build/perf/dlfilters/dlfilter-test-api-v0.so
 CC      /tmp/build/perf/mmap.o
 CC      /tmp/build/perf/zalloc.o
 LINK    /tmp/build/perf/plugin_jbd2.so
 LINK    /tmp/build/perf/plugin_hrtimer.so
 INSTALL /tmp/build/perf/libbpf/include/bpf/btf.h
 CC      /tmp/build/perf/xyarray.o
 CC      /tmp/build/perf/trace-seq.o
 LD      /tmp/build/perf/plugin_mac80211-in.o
 LD      /tmp/build/perf/libapi-in.o
 LD      /tmp/build/perf/plugin_kvm-in.o
 CC      /tmp/build/perf/parse-filter.o
 CC      /tmp/build/perf/parse-utils.o
 LD      /tmp/build/perf/plugin_sched_switch-in.o
 CC      /tmp/build/perf/kbuffer-parse.o
 CC      /tmp/build/perf/tep_strerror.o
 HOSTLD  /tmp/build/perf/pmu-events/jevents-in.o
 INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf_common.h
 INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf_legacy.h
 CC      /tmp/build/perf/event-parse-api.o
 INSTALL /tmp/build/perf/libbpf/include/bpf/xsk.h
 INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_helpers.h
 AR      /tmp/build/perf/libapi.a
 LD      /tmp/build/perf/plugin_function-in.o
 INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_tracing.h
 LINK    /tmp/build/perf/plugin_kmem.so
 LINK    /tmp/build/perf/plugin_mac80211.so
 LINK    /tmp/build/perf/plugin_kvm.so
 LD      /tmp/build/perf/plugin_xen-in.o
 LINK    /tmp/build/perf/plugin_sched_switch.so
 LD      /tmp/build/perf/plugin_cfg80211-in.o
 CC      /tmp/build/perf/lib.o
 INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_endian.h
 LD      /tmp/build/perf/plugin_futex-in.o
 LD      /tmp/build/perf/plugin_scsi-in.o
 INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_core_read.h
 LINK    /tmp/build/perf/pmu-events/jevents
 LD      /tmp/build/perf/plugin_tlb-in.o
 LINK    /tmp/build/perf/plugin_function.so
 LINK    /tmp/build/perf/plugin_xen.so
 INSTALL /tmp/build/perf/libbpf/include/bpf/skel_internal.h
 LINK    /tmp/build/perf/plugin_cfg80211.so
 INSTALL /tmp/build/perf/libbpf/include/bpf/libbpf_version.h
 LINK    /tmp/build/perf/plugin_futex.so
 INSTALL /tmp/build/perf/libbpf/include/bpf/bpf_helper_defs.h
 LINK    /tmp/build/perf/plugin_scsi.so
 LD      /tmp/build/perf/jvmti/jvmti-in.o
 LINK    /tmp/build/perf/plugin_tlb.so
 MKDIR   /tmp/build/perf/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/libbpf/staticobjs/
 LD      /tmp/build/perf/libsubcmd-in.o
 MKDIR   /tmp/build/perf/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/libbpf/staticobjs/
 CC      /tmp/build/perf/libbpf/staticobjs/xsk.o
 CC      /tmp/build/perf/libbpf/staticobjs/libbpf.o
 LINK    /tmp/build/perf/libperf-jvmti.so
 CC      /tmp/build/perf/libbpf/staticobjs/bpf.o
 CC      /tmp/build/perf/libbpf/staticobjs/nlattr.o
 CC      /tmp/build/perf/libbpf/staticobjs/btf.o
 CC      /tmp/build/perf/libbpf/staticobjs/libbpf_errno.o
 CC      /tmp/build/perf/libbpf/staticobjs/hashmap.o
 CC      /tmp/build/perf/libbpf/staticobjs/str_error.o
 CC      /tmp/build/perf/libbpf/staticobjs/netlink.o
 CC      /tmp/build/perf/libbpf/staticobjs/btf_dump.o
 CC      /tmp/build/perf/libbpf/staticobjs/bpf_prog_linfo.o
 CC      /tmp/build/perf/libbpf/staticobjs/libbpf_probes.o
 CC      /tmp/build/perf/libbpf/staticobjs/ringbuf.o
 CC      /tmp/build/perf/libbpf/staticobjs/strset.o
 GEN     /tmp/build/perf/libtraceevent-dynamic-list
 CC      /tmp/build/perf/libbpf/staticobjs/linker.o
 CC      /tmp/build/perf/libbpf/staticobjs/gen_loader.o
 CC      /tmp/build/perf/libbpf/staticobjs/relo_core.o
 AR      /tmp/build/perf/libsubcmd.a
 LD      /tmp/build/perf/libtraceevent-in.o
 GEN     /tmp/build/perf/pmu-events/pmu-events.c
 LINK    /tmp/build/perf/libtraceevent.a
 LD      /tmp/build/perf/libperf-in.o
 AR      /tmp/build/perf/libperf.a
 GEN     /tmp/build/perf/python/perf.so
 CC      /tmp/build/perf/pmu-events/pmu-events.o

Auto-detecting system features:

...                        libbfd: [ on  ]
...        disassembler-four-args: [ on  ]
...                          zlib: [ on  ]
...                        libcap: [ on  ]
...               clang-bpf-co-re: [ on  ]
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/hashmap.h
 GEN     /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/bpf_helper_defs.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/btf.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf_common.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf_legacy.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/xsk.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_helpers.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_tracing.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_endian.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_core_read.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/skel_internal.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/libbpf_version.h
 INSTALL /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf//include/bpf/bpf_helper_defs.h
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
 MKDIR   /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/btf_dump.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/bpf.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/nlattr.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/btf.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/ringbuf.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf_errno.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/str_error.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/netlink.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/bpf_prog_linfo.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/strset.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf_probes.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/xsk.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/hashmap.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/linker.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/gen_loader.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/relo_core.o
 LD      /tmp/build/perf/pmu-events/pmu-events-in.o
 LD      /tmp/build/perf/libbpf/staticobjs/libbpf-in.o
 LINK    /tmp/build/perf/libbpf/libbpf.a
 LD      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/staticobjs/libbpf-in.o
 LINK    /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/libbpf/libbpf.a
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/common.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/main.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/json_writer.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/gen.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/xlated_dumper.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/btf.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/btf_dumper.o
 CC      /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/disasm.o
 LINK    /tmp/build/perf/util/bpf_skel/.tmp//bootstrap/bpftool
 GEN     /tmp/build/perf/util/bpf_skel/vmlinux.h
 CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bpf_prog_profiler.bpf.o
 CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_leader.bpf.o
 CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_follower.bpf.o
 CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_cgroup.bpf.o
Quoted 15 lines

GENSKEL /tmp/build/perf/util/bpf_skel/bpf_prog_profiler.skel.h
GENSKEL /tmp/build/perf/util/bpf_skel/bperf_leader.skel.h
libbpf: elf: skipping unrecognized data section(9) .eh_frame
libbpf: elf: skipping relo section(15) .rel.eh_frame for section(9) .eh_frame
libbpf: elf: skipping unrecognized data section(7) .eh_frame
libbpf: elf: skipping relo section(12) .rel.eh_frame for section(7) .eh_frame
libbpf: map 'prev_readings': unexpected def kind var.
Error: failed to open BPF object file: Invalid argument
make[2]: *** [Makefile.perf:1076: /tmp/build/perf/util/bpf_skel/bperf_leader.skel.h] Error 255 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [Makefile.perf:240: sub-make] Error 2
make: *** [Makefile:70: all] Error 2
make: Leaving directory '/git/perf-5.16.0-rc3/tools/perf'
+ exit 1
[perfbuilder@five 32]$

Cc Andrii.

Could you please try the fix below?

If it doesn't work, could you please dump btf for debugging?

bpftool btf dump file /tmp/build/perf/util/bpf_skel/.tmp/bperf_leader.bpf.o

Thanks,
Song

tools/perf/util/bpf_skel/bperf_follower.bpf.c +13 -2
index 4a6acfde14937..bd191c9bc49bc 100644
@@ -6,8 +6,19 @@
 #include "bperf.h"
 #include "bperf_u.h"
 
-reading_map diff_readings SEC(".maps");
-reading_map accum_readings SEC(".maps");
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} diff_readings SEC(".maps");
+
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} accum_readings SEC(".maps");
 
 struct {
        __uint(type, BPF_MAP_TYPE_HASH);
tools/perf/util/bpf_skel/bperf_leader.bpf.c +13 -2
index 40d962b058634..d52a335e63f0f 100644
@@ -12,8 +12,19 @@ struct {
        __uint(map_flags, BPF_F_PRESERVE_ELEMS);
 } events SEC(".maps");
 
-reading_map prev_readings SEC(".maps");
-reading_map diff_readings SEC(".maps");
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} prev_readings SEC(".maps");
+
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} diff_readings SEC(".maps");
 
 SEC("raw_tp/sched_switch")
 int BPF_PROG(on_switch)
author maintainer
#5 lore raw

Em Fri, Dec 03, 2021 at 08:05:59PM +0000, Song Liu escreveu:

Quoted 50 lines
Quoted 50 lines · Arnaldo Carvalho de Melo

The failure for fedora:32, which seems to be on the threshold for clang to be considered recent enough for building skels fails differently:

clang version 10.0.1 (Fedora 10.0.1-3.fc32)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/10 Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/10 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/10 Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
+ '[' '!' ']'
+ rm -rf /tmp/build/perf
+ mkdir /tmp/build/perf
+ make ARCH= CROSS_COMPILE= EXTRA_CFLAGS= BUILD_BPF_SKEL=1 -C tools/perf O=/tmp/build/perf
make: Entering directory '/git/perf-5.16.0-rc3/tools/perf'
BUILD: Doing 'make -j32' parallel build

 HOSTCC  /tmp/build/perf/fixdep.o
 HOSTLD  /tmp/build/perf/fixdep-in.o
 LINK    /tmp/build/perf/fixdep

/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `expr bison (GNU Bison) 3.5 \>\= 371'
Makefile.config:997: No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-dev[el]/libbabeltrace-ctf-dev

Auto-detecting system features:

...                         dwarf: [ on  ]
...            dwarf_getlocations: [ on  ]
...                         glibc: [ on  ]
...                        libbfd: [ on  ]
...                libbfd-buildid: [ on  ]
...                        libcap: [ on  ]
...                        libelf: [ on  ]
...                       libnuma: [ on  ]
...        numa_num_possible_cpus: [ on  ]
...                       libperl: [ on  ]
...                     libpython: [ on  ]
...                     libcrypto: [ on  ]
...                     libunwind: [ on  ]
...            libdw-dwarf-unwind: [ on  ]
...                          zlib: [ on  ]
...                          lzma: [ on  ]
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]
...                        libaio: [ on  ]
...                       libzstd: [ on  ]
...        disassembler-four-args: [ on  ]

<SNIP>

Quoted 21 lines
Quoted 5 lines
 GEN     /tmp/build/perf/util/bpf_skel/vmlinux.h
 CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bpf_prog_profiler.bpf.o
 CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_leader.bpf.o
 CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_follower.bpf.o
 CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_cgroup.bpf.o
Quoted 15 lines

GENSKEL /tmp/build/perf/util/bpf_skel/bpf_prog_profiler.skel.h
GENSKEL /tmp/build/perf/util/bpf_skel/bperf_leader.skel.h
libbpf: elf: skipping unrecognized data section(9) .eh_frame
libbpf: elf: skipping relo section(15) .rel.eh_frame for section(9) .eh_frame
libbpf: elf: skipping unrecognized data section(7) .eh_frame
libbpf: elf: skipping relo section(12) .rel.eh_frame for section(7) .eh_frame
libbpf: map 'prev_readings': unexpected def kind var.
Error: failed to open BPF object file: Invalid argument
make[2]: *** [Makefile.perf:1076: /tmp/build/perf/util/bpf_skel/bperf_leader.skel.h] Error 255 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [Makefile.perf:240: sub-make] Error 2
make: *** [Makefile:70: all] Error 2
make: Leaving directory '/git/perf-5.16.0-rc3/tools/perf'
+ exit 1
[perfbuilder@five 32]$

Quoted 7 lines

Cc Andrii.

Could you please try the fix below?

If it doesn't work, could you please dump btf for debugging?

bpftool btf dump file /tmp/build/perf/util/bpf_skel/.tmp/bperf_leader.bpf.o

It fixes the issue on Alpine Linux 3.12 and 3.13, Alt Linux p9 and fedora:32, probably others, still testing.

Can you please provide a cset commit log for me to stamp on it?

- Arnaldo

Quoted 58 lines

Thanks,
Song

tools/perf/util/bpf_skel/bperf_follower.bpf.c +13 -2
index 4a6acfde14937..bd191c9bc49bc 100644
@@ -6,8 +6,19 @@
 #include "bperf.h"
 #include "bperf_u.h"
 
-reading_map diff_readings SEC(".maps");
-reading_map accum_readings SEC(".maps");
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} diff_readings SEC(".maps");
+
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} accum_readings SEC(".maps");
 
 struct {
        __uint(type, BPF_MAP_TYPE_HASH);
tools/perf/util/bpf_skel/bperf_leader.bpf.c +13 -2
index 40d962b058634..d52a335e63f0f 100644
@@ -12,8 +12,19 @@ struct {
        __uint(map_flags, BPF_F_PRESERVE_ELEMS);
 } events SEC(".maps");
 
-reading_map prev_readings SEC(".maps");
-reading_map diff_readings SEC(".maps");
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} prev_readings SEC(".maps");
+
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} diff_readings SEC(".maps");
 
 SEC("raw_tp/sched_switch")
 int BPF_PROG(on_switch)
Tested-by Athira Rajeev
#6 lore raw
Quoted 83 lines · Arnaldo Carvalho de Melo

Em Fri, Dec 03, 2021 at 08:05:59PM +0000, Song Liu escreveu:

Quoted 50 lines
Quoted 50 lines · Arnaldo Carvalho de Melo

The failure for fedora:32, which seems to be on the threshold for clang to be considered recent enough for building skels fails differently:

clang version 10.0.1 (Fedora 10.0.1-3.fc32)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/10 Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/10 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/10 Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
+ '[' '!' ']'
+ rm -rf /tmp/build/perf
+ mkdir /tmp/build/perf
+ make ARCH= CROSS_COMPILE= EXTRA_CFLAGS= BUILD_BPF_SKEL=1 -C tools/perf O=/tmp/build/perf
make: Entering directory '/git/perf-5.16.0-rc3/tools/perf'
BUILD: Doing 'make -j32' parallel build

HOSTCC  /tmp/build/perf/fixdep.o
HOSTLD  /tmp/build/perf/fixdep-in.o
LINK    /tmp/build/perf/fixdep

/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `expr bison (GNU Bison) 3.5 \>\= 371'
Makefile.config:997: No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-dev[el]/libbabeltrace-ctf-dev

Auto-detecting system features:

...                         dwarf: [ on  ]
...            dwarf_getlocations: [ on  ]
...                         glibc: [ on  ]
...                        libbfd: [ on  ]
...                libbfd-buildid: [ on  ]
...                        libcap: [ on  ]
...                        libelf: [ on  ]
...                       libnuma: [ on  ]
...        numa_num_possible_cpus: [ on  ]
...                       libperl: [ on  ]
...                     libpython: [ on  ]
...                     libcrypto: [ on  ]
...                     libunwind: [ on  ]
...            libdw-dwarf-unwind: [ on  ]
...                          zlib: [ on  ]
...                          lzma: [ on  ]
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]
...                        libaio: [ on  ]
...                       libzstd: [ on  ]
...        disassembler-four-args: [ on  ]

<SNIP>

Quoted 25 lines
Quoted 5 lines
GEN     /tmp/build/perf/util/bpf_skel/vmlinux.h
CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bpf_prog_profiler.bpf.o
CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_leader.bpf.o
CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_follower.bpf.o
CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_cgroup.bpf.o
Quoted 15 lines

GENSKEL /tmp/build/perf/util/bpf_skel/bpf_prog_profiler.skel.h
GENSKEL /tmp/build/perf/util/bpf_skel/bperf_leader.skel.h
libbpf: elf: skipping unrecognized data section(9) .eh_frame
libbpf: elf: skipping relo section(15) .rel.eh_frame for section(9) .eh_frame
libbpf: elf: skipping unrecognized data section(7) .eh_frame
libbpf: elf: skipping relo section(12) .rel.eh_frame for section(7) .eh_frame
libbpf: map 'prev_readings': unexpected def kind var.
Error: failed to open BPF object file: Invalid argument
make[2]: *** [Makefile.perf:1076: /tmp/build/perf/util/bpf_skel/bperf_leader.skel.h] Error 255 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [Makefile.perf:240: sub-make] Error 2
make: *** [Makefile:70: all] Error 2
make: Leaving directory '/git/perf-5.16.0-rc3/tools/perf'
+ exit 1
[perfbuilder@five 32]$

Cc Andrii.

Could you please try the fix below?

Quoted 7 lines
Quoted 3 lines

If it doesn't work, could you please dump btf for debugging?

bpftool btf dump file /tmp/build/perf/util/bpf_skel/.tmp/bperf_leader.bpf.o

It fixes the issue on Alpine Linux 3.12 and 3.13, Alt Linux p9 and fedora:32, probably others, still testing.

Can you please provide a cset commit log for me to stamp on it?

Please see the following commit log.

Thanks,
Song

====================================== 8< ========================================

perf/bpf_skel: use vmlinux.h in skeletons

When building bpf_skel in perf on a system with older linux/bpf.h header, we got errors like:

   util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared
   identifier 'BPF_F_PRESERVE_ELEMS'
           __uint(map_flags, BPF_F_PRESERVE_ELEMS);

Fix this by using vmlinux.h instead. Also remove include of perf_event.h, as it redefines structs in vmlinux.h.

Fixes fa853c4b839e ("perf stat: Enable counting events for BPF programs")
Fixes 7fac83aaf2ee ("perf stat: Introduce 'bperf' to share hardware PMCs with BPF")
author maintainer
#7 lore raw

Em Mon, Dec 06, 2021 at 05:08:21PM +0000, Song Liu escreveu:

Quoted 85 lines
Quoted 85 lines · Arnaldo Carvalho de Melo

Em Fri, Dec 03, 2021 at 08:05:59PM +0000, Song Liu escreveu:

Quoted 50 lines
Quoted 50 lines · Arnaldo Carvalho de Melo

The failure for fedora:32, which seems to be on the threshold for clang to be considered recent enough for building skels fails differently:

clang version 10.0.1 (Fedora 10.0.1-3.fc32)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/10 Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/10 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/10 Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
+ '[' '!' ']'
+ rm -rf /tmp/build/perf
+ mkdir /tmp/build/perf
+ make ARCH= CROSS_COMPILE= EXTRA_CFLAGS= BUILD_BPF_SKEL=1 -C tools/perf O=/tmp/build/perf
make: Entering directory '/git/perf-5.16.0-rc3/tools/perf'
BUILD: Doing 'make -j32' parallel build

HOSTCC  /tmp/build/perf/fixdep.o
HOSTLD  /tmp/build/perf/fixdep-in.o
LINK    /tmp/build/perf/fixdep

/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `expr bison (GNU Bison) 3.5 \>\= 371'
Makefile.config:997: No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-dev[el]/libbabeltrace-ctf-dev

Auto-detecting system features:

...                         dwarf: [ on  ]
...            dwarf_getlocations: [ on  ]
...                         glibc: [ on  ]
...                        libbfd: [ on  ]
...                libbfd-buildid: [ on  ]
...                        libcap: [ on  ]
...                        libelf: [ on  ]
...                       libnuma: [ on  ]
...        numa_num_possible_cpus: [ on  ]
...                       libperl: [ on  ]
...                     libpython: [ on  ]
...                     libcrypto: [ on  ]
...                     libunwind: [ on  ]
...            libdw-dwarf-unwind: [ on  ]
...                          zlib: [ on  ]
...                          lzma: [ on  ]
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]
...                        libaio: [ on  ]
...                       libzstd: [ on  ]
...        disassembler-four-args: [ on  ]

<SNIP>

Quoted 27 lines
Quoted 5 lines
GEN     /tmp/build/perf/util/bpf_skel/vmlinux.h
CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bpf_prog_profiler.bpf.o
CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_leader.bpf.o
CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_follower.bpf.o
CLANG   /tmp/build/perf/util/bpf_skel/.tmp/bperf_cgroup.bpf.o
Quoted 15 lines

GENSKEL /tmp/build/perf/util/bpf_skel/bpf_prog_profiler.skel.h
GENSKEL /tmp/build/perf/util/bpf_skel/bperf_leader.skel.h
libbpf: elf: skipping unrecognized data section(9) .eh_frame
libbpf: elf: skipping relo section(15) .rel.eh_frame for section(9) .eh_frame
libbpf: elf: skipping unrecognized data section(7) .eh_frame
libbpf: elf: skipping relo section(12) .rel.eh_frame for section(7) .eh_frame
libbpf: map 'prev_readings': unexpected def kind var.
Error: failed to open BPF object file: Invalid argument
make[2]: *** [Makefile.perf:1076: /tmp/build/perf/util/bpf_skel/bperf_leader.skel.h] Error 255 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [Makefile.perf:240: sub-make] Error 2
make: *** [Makefile:70: all] Error 2
make: Leaving directory '/git/perf-5.16.0-rc3/tools/perf'
+ exit 1
[perfbuilder@five 32]$

Cc Andrii.

Could you please try the fix below?

If it doesn't work, could you please dump btf for debugging?

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

bpftool btf dump file /tmp/build/perf/util/bpf_skel/.tmp/bperf_leader.bpf.o

It fixes the issue on Alpine Linux 3.12 and 3.13, Alt Linux p9 and fedora:32, probably others, still testing.

Can you please provide a cset commit log for me to stamp on it?

Please see the following commit log.

This is for the other one, where I stamped this:

commit ce71038e673ee8291c64631359e56c48c8616dc7 (HEAD -> perf/urgent)

Author: Song Liu <[email protected]>
Date:   Fri Dec 3 19:32:34 2021 +0000

perf bpf: Fix building perf with BUILD_BPF_SKEL=1 by default in more distros

Arnaldo reported that building all his containers with BUILD_BPF_SKEL=1 to then make this the default he found problems in some distros where the system linux/bpf.h file was being used and lacked this:

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'
__uint(map_flags, BPF_F_PRESERVE_ELEMS);

So use instead the vmlinux.h file generated by bpftool from BTF info.

This fixed these as well, getting the build back working on debian:11, debian:experimental and ubuntu:21.10:

In file included from In file included from util/bpf_skel/bperf_leader.bpf.cutil/bpf_skel/bpf_prog_profiler.bpf.c::33:
:
In file included from In file included from /usr/include/linux/bpf.h/usr/include/linux/bpf.h::1111:
:
/usr/include/linux/types.h/usr/include/linux/types.h::55::1010:: In file included from util/bpf_skel/bperf_follower.bpf.c:3fatal errorfatal error:
: : In file included from /usr/include/linux/bpf.h:'asm/types.h' file not found11'asm/types.h' file not found:

/usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found #include <asm/types.h>#include <asm/types.h>

^~~~~~~~~~~~~ ^~~~~~~~~~~~~

#include <asm/types.h>
^~~~~~~~~~~~~
1 error generated.

For this patch:

tools/perf/util/bpf_skel/bperf_follower.bpf.c +1 -2
index b8fa3cb2da230803..4a6acfde14937704 100644
@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2021 Facebook
-#include <linux/bpf.h>
-#include <linux/perf_event.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 #include "bperf.h"
tools/perf/util/bpf_skel/bperf_leader.bpf.c +1 -2
index 4f70d1459e86cb99..40d962b05863421e 100644
@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2021 Facebook
-#include <linux/bpf.h>
-#include <linux/perf_event.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 #include "bperf.h"
tools/perf/util/bpf_skel/bpf_prog_profiler.bpf.c +1 -1
index ab12b4c4ece21a9a..97037d3b3d9fa4cd 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2020 Facebook
-#include <linux/bpf.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 

But what I asked now was a commit log message for this other one:

tools/perf/util/bpf_skel/bperf_follower.bpf.c +13 -2
index 4a6acfde14937704..ace00ebbd8e5701f 100644
@@ -6,8 +6,19 @@
 #include "bperf.h"
 #include "bperf_u.h"
 
-reading_map diff_readings SEC(".maps");
-reading_map accum_readings SEC(".maps");
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} diff_readings SEC(".maps");
+
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} accum_readings SEC(".maps");
 
 struct {
        __uint(type, BPF_MAP_TYPE_HASH);
tools/perf/util/bpf_skel/bperf_leader.bpf.c +13 -2
index 40d962b05863421e..22ef924f8396f8e0 100644
@@ -12,8 +12,19 @@ struct {
        __uint(map_flags, BPF_F_PRESERVE_ELEMS);
 } events SEC(".maps");
 
-reading_map prev_readings SEC(".maps");
-reading_map diff_readings SEC(".maps");
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} prev_readings SEC(".maps");
+
+struct {
+       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+       __uint(key_size, sizeof(__u32));
+       __uint(value_size, sizeof(struct bpf_perf_event_value));
+       __uint(max_entries, 1);
+} diff_readings SEC(".maps");
 
 SEC("raw_tp/sched_switch")
 int BPF_PROG(on_switch)
Quoted 23 lines

Thanks,
Song

====================================== 8< ========================================

perf/bpf_skel: use vmlinux.h in skeletons

When building bpf_skel in perf on a system with older linux/bpf.h header, we got errors like:

   util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared
   identifier 'BPF_F_PRESERVE_ELEMS'
           __uint(map_flags, BPF_F_PRESERVE_ELEMS);

Fix this by using vmlinux.h instead. Also remove include of perf_event.h, as it redefines structs in vmlinux.h.

Fixes fa853c4b839e ("perf stat: Enable counting events for BPF programs")
Fixes 7fac83aaf2ee ("perf stat: Introduce 'bperf' to share hardware PMCs with BPF")
Signature · 1 line
- Arnaldo
#8 lore raw
Quoted 9 lines · Arnaldo Carvalho de Melo

Em Mon, Dec 06, 2021 at 05:08:21PM +0000, Song Liu escreveu:

Quoted 4 lines
Quoted 4 lines · Arnaldo Carvalho de Melo

[...]

Quoted 87 lines
Quoted 2 lines

Please see the following commit log.

This is for the other one, where I stamped this:

commit ce71038e673ee8291c64631359e56c48c8616dc7 (HEAD -> perf/urgent)

Author: Song Liu <[email protected]>
Date:   Fri Dec 3 19:32:34 2021 +0000

perf bpf: Fix building perf with BUILD_BPF_SKEL=1 by default in more distros

Arnaldo reported that building all his containers with BUILD_BPF_SKEL=1 to then make this the default he found problems in some distros where the system linux/bpf.h file was being used and lacked this:

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'
__uint(map_flags, BPF_F_PRESERVE_ELEMS);

So use instead the vmlinux.h file generated by bpftool from BTF info.

This fixed these as well, getting the build back working on debian:11, debian:experimental and ubuntu:21.10:

In file included from In file included from util/bpf_skel/bperf_leader.bpf.cutil/bpf_skel/bpf_prog_profiler.bpf.c::33:
:
In file included from In file included from /usr/include/linux/bpf.h/usr/include/linux/bpf.h::1111:
:
/usr/include/linux/types.h/usr/include/linux/types.h::55::1010:: In file included from util/bpf_skel/bperf_follower.bpf.c:3fatal errorfatal error:
: : In file included from /usr/include/linux/bpf.h:'asm/types.h' file not found11'asm/types.h' file not found:

/usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found #include <asm/types.h>#include <asm/types.h>

^~~~~~~~~~~~~ ^~~~~~~~~~~~~

#include <asm/types.h>
^~~~~~~~~~~~~
1 error generated.

For this patch:

tools/perf/util/bpf_skel/bperf_follower.bpf.c +1 -2
index b8fa3cb2da230803..4a6acfde14937704 100644
@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2021 Facebook
-#include <linux/bpf.h>
-#include <linux/perf_event.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 #include "bperf.h"
tools/perf/util/bpf_skel/bperf_leader.bpf.c +1 -2
index 4f70d1459e86cb99..40d962b05863421e 100644
@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2021 Facebook
-#include <linux/bpf.h>
-#include <linux/perf_event.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 #include "bperf.h"
tools/perf/util/bpf_skel/bpf_prog_profiler.bpf.c +1 -1
index ab12b4c4ece21a9a..97037d3b3d9fa4cd 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2020 Facebook
-#include <linux/bpf.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 
Quoted 1 line

But what I asked now was a commit log message for this other one:

Sorry for the confusion. Please use this one, which also removes bperf.h (which holds the typedef).

Thanks,
Song

====================================== 8< ========================================

From e033fd36657288ece383ba528aaff7b56eebfee2 Mon Sep 17 00:00:00 2001
From: Song Liu <[email protected]>
Date: Fri, 3 Dec 2021 15:14:41 -0800
Subject: [PATCH] perf/bpf_skel: do not use typedef to avoid error on old clang

When building bpf_skel with clang-10, typedef causes confusions like:

libbpf: map 'prev_readings': unexpected def kind var.

Fix this by removing the typedef.

Fixes 7fac83aaf2ee ("perf stat: Introduce 'bperf' to share hardware PMCs with BPF")
---
 tools/perf/util/bpf_skel/bperf.h              | 14 --------------
 tools/perf/util/bpf_skel/bperf_follower.bpf.c | 16 +++++++++++++---
 tools/perf/util/bpf_skel/bperf_leader.bpf.c   | 16 +++++++++++++---
 3 files changed, 26 insertions(+), 20 deletions(-)
 delete mode 100644 tools/perf/util/bpf_skel/bperf.h
tools/perf/util/bpf_skel/bperf.h deleted +0 -14
deleted file mode 100644
index 186a5551ddb9d..0000000000000
@@ -1,14 +0,0 @@
-// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
-// Copyright (c) 2021 Facebook
-
-#ifndef __BPERF_STAT_H
-#define __BPERF_STAT_H
-
-typedef struct {
-	__uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
-	__uint(key_size, sizeof(__u32));
-	__uint(value_size, sizeof(struct bpf_perf_event_value));
-	__uint(max_entries, 1);
-} reading_map;
-
-#endif /* __BPERF_STAT_H */
tools/perf/util/bpf_skel/bperf_follower.bpf.c +13 -3
index 4a6acfde14937..f193998530d43 100644
@@ -3,11 +3,21 @@
 #include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
-#include "bperf.h"
 #include "bperf_u.h"
 
-reading_map diff_readings SEC(".maps");
-reading_map accum_readings SEC(".maps");
+struct {
+	__uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+	__uint(key_size, sizeof(__u32));
+	__uint(value_size, sizeof(struct bpf_perf_event_value));
+	__uint(max_entries, 1);
+} diff_readings SEC(".maps");
+
+struct {
+	__uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+	__uint(key_size, sizeof(__u32));
+	__uint(value_size, sizeof(struct bpf_perf_event_value));
+	__uint(max_entries, 1);
+} accum_readings SEC(".maps");
 
 struct {
 	__uint(type, BPF_MAP_TYPE_HASH);
tools/perf/util/bpf_skel/bperf_leader.bpf.c +13 -3
index 40d962b058634..e2a2d4cd7779c 100644
@@ -3,7 +3,6 @@
 #include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
-#include "bperf.h"
 
 struct {
 	__uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY);
@@ -12,8 +11,19 @@ struct {
 	__uint(map_flags, BPF_F_PRESERVE_ELEMS);
 } events SEC(".maps");
 
-reading_map prev_readings SEC(".maps");
-reading_map diff_readings SEC(".maps");
+struct {
+	__uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+	__uint(key_size, sizeof(__u32));
+	__uint(value_size, sizeof(struct bpf_perf_event_value));
+	__uint(max_entries, 1);
+} prev_readings SEC(".maps");
+
+struct {
+	__uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+	__uint(key_size, sizeof(__u32));
+	__uint(value_size, sizeof(struct bpf_perf_event_value));
+	__uint(max_entries, 1);
+} diff_readings SEC(".maps");
 
 SEC("raw_tp/sched_switch")
 int BPF_PROG(on_switch)
Signature · 1 line
2.30.2
author maintainer Applied
#9 lore raw

Em Mon, Dec 06, 2021 at 10:34:31PM +0000, Song Liu escreveu:

Quoted 5 lines
Quoted 1 line

From e033fd36657288ece383ba528aaff7b56eebfee2 Mon Sep 17 00:00:00 2001

From: Song Liu <[email protected]>
Date: Fri, 3 Dec 2021 15:14:41 -0800
Subject: [PATCH] perf/bpf_skel: do not use typedef to avoid error on old clang

Quoted 7 lines

When building bpf_skel with clang-10, typedef causes confusions like:

libbpf: map 'prev_readings': unexpected def kind var.

Fix this by removing the typedef.

Fixes 7fac83aaf2ee ("perf stat: Introduce 'bperf' to share hardware PMCs with BPF")

Please get a larger sha abbrev:

⬢[acme@toolbox perf]$ grep core -A1 ~/.gitconfig
[core]

	abbrev = 16

⬢[acme@toolbox perf]$

Thanks, applied.

- Arnaldo

#10 lore raw
Quoted 16 lines · Arnaldo Carvalho de Melo

Em Mon, Dec 06, 2021 at 10:34:31PM +0000, Song Liu escreveu:

Quoted 11 lines
Quoted 1 line

From e033fd36657288ece383ba528aaff7b56eebfee2 Mon Sep 17 00:00:00 2001

From: Song Liu <[email protected]>
Date: Fri, 3 Dec 2021 15:14:41 -0800
Subject: [PATCH] perf/bpf_skel: do not use typedef to avoid error on old clang

When building bpf_skel with clang-10, typedef causes confusions like:

libbpf: map 'prev_readings': unexpected def kind var.

Fix this by removing the typedef.

Quoted 8 lines
Quoted 1 line
Fixes 7fac83aaf2ee ("perf stat: Introduce 'bperf' to share hardware PMCs with BPF")

Please get a larger sha abbrev:

⬢[acme@toolbox perf]$ grep core -A1 ~/.gitconfig
[core]

	abbrev = 16

⬢[acme@toolbox perf]$

Hmm.. I think the rule is to have 12 letters sha in the Fixes tag. Did we change something recently?

Thanks,
Song

#11 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 36 lines · Arnaldo Carvalho de Melo

Hi Song,

So I'm changing all my containers to build with BUILD_BPF_SKEL=1
to then make this the default, so far older containers fail either because the clang available is too old, so I've added a NO_BUILD_BPF_SKEL=1 env var to disable that in those containers and then there is this other case where clang is recent enough but:

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'

           __uint(map_flags, BPF_F_PRESERVE_ELEMS);

Because the system's /usr/include/linux/bpf.h doesn't have that BPF_F_PRESERVE_ELEMS enum entry.

These are enums to make them available via BTF, but then I can't use the:

#ifdef BPF_F_PRESERVE_ELEMS
#define BPF_F_PRESERVE_ELEMS (1U << 11)
#endif

approach.

But then we _have_ it in the tools/include/uapi/linux/bpf.h we ship:

$ grep BPF_F_PRESERVE_ELEMS tools/include/uapi/linux/bpf.h
	BPF_F_PRESERVE_ELEMS	= (1U << 11),
$

so we need to switch to using it somehow, this way we can build in more systems and make bperf and other BPF enabled features.

Quoted 7 lines

From a quick look I couldn't find where to add $(sourcedir)/tools/include/uapi/ to the include path used to build util/bpf_skel/bperf_leader.bpf.c, should be easy, can you take a look?

Thanks,

- Arnaldo

I think the following should fix it

Thanks,
Song

tools/perf/util/bpf_skel/bperf_follower.bpf.c +1 -2
index b8fa3cb2da230..4a6acfde14937 100644
@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2021 Facebook
-#include <linux/bpf.h>
-#include <linux/perf_event.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 #include "bperf.h"
tools/perf/util/bpf_skel/bperf_leader.bpf.c +1 -2
index 4f70d1459e86c..40d962b058634 100644
@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2021 Facebook
-#include <linux/bpf.h>
-#include <linux/perf_event.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 #include "bperf.h"
tools/perf/util/bpf_skel/bpf_prog_profiler.bpf.c +1 -1
index ab12b4c4ece21..97037d3b3d9fa 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2020 Facebook
-#include <linux/bpf.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 
author maintainer
#12 lore raw

Em Fri, Dec 03, 2021 at 07:32:34PM +0000, Song Liu escreveu:

Quoted 40 lines
Quoted 40 lines · Arnaldo Carvalho de Melo

Hi Song,

So I'm changing all my containers to build with BUILD_BPF_SKEL=1
to then make this the default, so far older containers fail either because the clang available is too old, so I've added a NO_BUILD_BPF_SKEL=1 env var to disable that in those containers and then there is this other case where clang is recent enough but:

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'

           __uint(map_flags, BPF_F_PRESERVE_ELEMS);

Because the system's /usr/include/linux/bpf.h doesn't have that BPF_F_PRESERVE_ELEMS enum entry.

These are enums to make them available via BTF, but then I can't use the:

#ifdef BPF_F_PRESERVE_ELEMS
#define BPF_F_PRESERVE_ELEMS (1U << 11)
#endif

approach.

But then we _have_ it in the tools/include/uapi/linux/bpf.h we ship:

$ grep BPF_F_PRESERVE_ELEMS tools/include/uapi/linux/bpf.h
	BPF_F_PRESERVE_ELEMS	= (1U << 11),
$

so we need to switch to using it somehow, this way we can build in more systems and make bperf and other BPF enabled features.

From a quick look I couldn't find where to add $(sourcedir)/tools/include/uapi/ to the include path used to build util/bpf_skel/bperf_leader.bpf.c, should be easy, can you take a look?

Quoted 5 lines
Quoted 3 lines

Thanks,

- Arnaldo

I think the following should fix it

I'm trying this now and tentatively sticking a Signed-off-by: you to it, ok?

- Arnaldo

Quoted 42 lines

Thanks,
Song

tools/perf/util/bpf_skel/bperf_follower.bpf.c +1 -2
index b8fa3cb2da230..4a6acfde14937 100644
@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2021 Facebook
-#include <linux/bpf.h>
-#include <linux/perf_event.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 #include "bperf.h"
tools/perf/util/bpf_skel/bperf_leader.bpf.c +1 -2
index 4f70d1459e86c..40d962b058634 100644
@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2021 Facebook
-#include <linux/bpf.h>
-#include <linux/perf_event.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 #include "bperf.h"
tools/perf/util/bpf_skel/bpf_prog_profiler.bpf.c +1 -1
index ab12b4c4ece21..97037d3b3d9fa 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2020 Facebook
-#include <linux/bpf.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 
Signature · 1 line
- Arnaldo
author maintainer
#13 lore raw

Em Sun, Dec 05, 2021 at 10:06:42AM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 41 lines

Em Fri, Dec 03, 2021 at 07:32:34PM +0000, Song Liu escreveu:

Quoted 40 lines
Quoted 40 lines · Arnaldo Carvalho de Melo

Hi Song,

So I'm changing all my containers to build with BUILD_BPF_SKEL=1
to then make this the default, so far older containers fail either because the clang available is too old, so I've added a NO_BUILD_BPF_SKEL=1 env var to disable that in those containers and then there is this other case where clang is recent enough but:

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'

           __uint(map_flags, BPF_F_PRESERVE_ELEMS);

Because the system's /usr/include/linux/bpf.h doesn't have that BPF_F_PRESERVE_ELEMS enum entry.

These are enums to make them available via BTF, but then I can't use the:

#ifdef BPF_F_PRESERVE_ELEMS
#define BPF_F_PRESERVE_ELEMS (1U << 11)
#endif

approach.

But then we _have_ it in the tools/include/uapi/linux/bpf.h we ship:

$ grep BPF_F_PRESERVE_ELEMS tools/include/uapi/linux/bpf.h
	BPF_F_PRESERVE_ELEMS	= (1U << 11),
$

so we need to switch to using it somehow, this way we can build in more systems and make bperf and other BPF enabled features.

From a quick look I couldn't find where to add $(sourcedir)/tools/include/uapi/ to the include path used to build util/bpf_skel/bperf_leader.bpf.c, should be easy, can you take a look?

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Thanks,

- Arnaldo

I think the following should fix it

I'm trying this now and tentatively sticking a Signed-off-by: you to it, ok?

Fixed these as well (debian:11, debian:experimental, some recent ubuntus):

In file included from util/bpf_skel/bperf_follower.bpf.c:3:
In file included from /usr/include/linux/bpf.h:11:
In file included from /usr/include/linux/types.hutil/bpf_skel/bperf_leader.bpf.c::5In file included from :3util/bpf_skel/bpf_prog_profiler.bpf.c10:
::In file included from 3 /usr/include/linux/bpf.h:
:In file included from 11/usr/include/linux/bpf.h:
fatal error:: 11/usr/include/linux/types.h:
:'asm/types.h' file not found5:/usr/include/linux/types.h10
::5 :10: fatal errorfatal error: : 'asm/types.h' file not found'asm/types.h' file not found#include <asm/types.h>

         ^~~~~~~~~~~~~

#include <asm/types.h>#include <asm/types.h>

         ^~~~~~~~~~~~~         ^~~~~~~~~~~~~

1 error generated.
1 error generated.
1 error generated.
make[2]: *** [Makefile.perf:1072: /tmp/build/perf/util/bpf_skel/.tmp/bperf_follower.bpf.o] Error 1

#14 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 45 lines · Arnaldo Carvalho de Melo

Em Fri, Dec 03, 2021 at 07:32:34PM +0000, Song Liu escreveu:

Quoted 40 lines
Quoted 40 lines · Arnaldo Carvalho de Melo

Hi Song,

So I'm changing all my containers to build with BUILD_BPF_SKEL=1
to then make this the default, so far older containers fail either because the clang available is too old, so I've added a NO_BUILD_BPF_SKEL=1 env var to disable that in those containers and then there is this other case where clang is recent enough but:

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'

          __uint(map_flags, BPF_F_PRESERVE_ELEMS);

Because the system's /usr/include/linux/bpf.h doesn't have that BPF_F_PRESERVE_ELEMS enum entry.

These are enums to make them available via BTF, but then I can't use the:

#ifdef BPF_F_PRESERVE_ELEMS
#define BPF_F_PRESERVE_ELEMS (1U << 11)
#endif

approach.

But then we _have_ it in the tools/include/uapi/linux/bpf.h we ship:

$ grep BPF_F_PRESERVE_ELEMS tools/include/uapi/linux/bpf.h
	BPF_F_PRESERVE_ELEMS	= (1U << 11),
$

so we need to switch to using it somehow, this way we can build in more systems and make bperf and other BPF enabled features.

From a quick look I couldn't find where to add $(sourcedir)/tools/include/uapi/ to the include path used to build util/bpf_skel/bperf_leader.bpf.c, should be easy, can you take a look?

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Thanks,

- Arnaldo

I think the following should fix it

I'm trying this now and tentatively sticking a Signed-off-by: you to it, ok?

Thanks for verifying this! I should learn to do these container tests myself. Yes, you can include

(or my fb email, either one works. )

Thanks,
Song

Tested-by
#15 lore raw
In reply to Song Liu
Quoted 44 lines · Song Liu
Quoted 41 lines · Arnaldo Carvalho de Melo

Hi Song,

So I'm changing all my containers to build with BUILD_BPF_SKEL=1
to then make this the default, so far older containers fail either because the clang available is too old, so I've added a NO_BUILD_BPF_SKEL=1 env var to disable that in those containers and then there is this other case where clang is recent enough but:

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'

          __uint(map_flags, BPF_F_PRESERVE_ELEMS);

Because the system's /usr/include/linux/bpf.h doesn't have that BPF_F_PRESERVE_ELEMS enum entry.

These are enums to make them available via BTF, but then I can't use the:

#ifdef BPF_F_PRESERVE_ELEMS
#define BPF_F_PRESERVE_ELEMS (1U << 11)
#endif

approach.

But then we _have_ it in the tools/include/uapi/linux/bpf.h we ship:

$ grep BPF_F_PRESERVE_ELEMS tools/include/uapi/linux/bpf.h
	BPF_F_PRESERVE_ELEMS	= (1U << 11),
$

so we need to switch to using it somehow, this way we can build in more systems and make bperf and other BPF enabled features.

From a quick look I couldn't find where to add $(sourcedir)/tools/include/uapi/ to the include path used to build util/bpf_skel/bperf_leader.bpf.c, should be easy, can you take a look?

Quoted 9 lines
Quoted 3 lines

Thanks,

- Arnaldo

I think the following should fix it

Thanks,
Song

Hi,

Encountered this error while building perf with BUILD_BPF_SKEL.

util/bpf_skel/bperf_leader.bpf.c:13:20: error: use of undeclared identifier 'BPF_F_PRESERVE_ELEMS'

        __uint(map_flags, BPF_F_PRESERVE_ELEMS);

And verified this patch fixes this issue.

Thanks
Athira

Quoted 40 lines
tools/perf/util/bpf_skel/bperf_follower.bpf.c +1 -2
index b8fa3cb2da230..4a6acfde14937 100644
@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2021 Facebook
-#include <linux/bpf.h>
-#include <linux/perf_event.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 #include "bperf.h"
tools/perf/util/bpf_skel/bperf_leader.bpf.c +1 -2
index 4f70d1459e86c..40d962b058634 100644
@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2021 Facebook
-#include <linux/bpf.h>
-#include <linux/perf_event.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
 #include "bperf.h"
tools/perf/util/bpf_skel/bpf_prog_profiler.bpf.c +1 -1
index ab12b4c4ece21..97037d3b3d9fa 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 // Copyright (c) 2020 Facebook
-#include <linux/bpf.h>
+#include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>