perf build broken seemingly due to libbpf changes, checking...
After merging torvalds/master to perf/urgent I'm getting this:
util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
25 | struct btf * __weak btf__load_from_kernel_by_id(__u32 id)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes]
37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev)
| ^~~~~~~~~~~~~~~~~~~~~~~~
util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes]
46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev)
| ^~~~~~~~~~~~~~~~~~~~
util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes]
55 | btf__raw_data(const struct btf *btf_ro, __u32 *size)
| ^~~~~~~~~~~~~cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf-event.o] Error 1 make[4]: *** Waiting for unfinished jobs....util/bpf_counter.c: In function ‘bpf_target_prog_name’:util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]
82 | btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
82 | btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
| ^
cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf_counter.o] Error 1
I'm checking now...
BTW I test perf builds with:
make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C tools/perf install-bin && git status && perf test python
Signature · 1 line
- Arnaldo
Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 27 lines
After merging torvalds/master to perf/urgent I'm getting this:
util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
25 | struct btf *__weakbtf__load_from_kernel_by_id(__u32 id)| ^~~~~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes] 37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev) | ^~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes] 46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev) | ^~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes] 55 | btf__raw_data(const struct btf *btf_ro, __u32 *size) | ^~~~~~~~~~~~~cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf-event.o] Error 1 make[4]: *** Waiting for unfinished jobs....util/bpf_counter.c: In function ‘bpf_target_prog_name’:util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]82 | btf = btf__load_from_kernel_by_id(info_linear->info.btf_id); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
82 | btf =btf__load_from_kernel_by_id(info_linear->info.btf_id);
| ^
cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf_counter.o] Error 1
Quoted 5 lines
I'm checking now...
BTW I test perf builds with:
make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C
tools/perfinstall-bin && git status && perf test python
Nevermind, this was due to a patch by Ian Rogers I was testing, bisecting get up to the last patch, since I had merged torvalds/master today it got me to a wrong correlation, sorry for the disturbance.
For reference, this is the patch:
http://lore.kernel.org/lkml/[email protected]
- Arnaldo
Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 34 lines
Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 27 lines
After merging torvalds/master to perf/urgent I'm getting this:
util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
25 | struct btf *__weakbtf__load_from_kernel_by_id(__u32 id)| ^~~~~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes] 37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev) | ^~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes] 46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev) | ^~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes] 55 | btf__raw_data(const struct btf *btf_ro, __u32 *size) | ^~~~~~~~~~~~~cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf-event.o] Error 1 make[4]: *** Waiting for unfinished jobs....util/bpf_counter.c: In function ‘bpf_target_prog_name’:util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]82 | btf = btf__load_from_kernel_by_id(info_linear->info.btf_id); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
82 | btf =btf__load_from_kernel_by_id(info_linear->info.btf_id);
| ^
cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf_counter.o] Error 1Quoted 5 lines
I'm checking now...
BTW I test perf builds with:
make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C
tools/perfinstall-bin && git status && perf test python
Quoted 7 lines
Nevermind, this was due to a patch by Ian Rogers I was testing, bisecting get up to the last patch, since I had merged torvalds/master today it got me to a wrong correlation, sorry for the disturbance.
For reference, this is the patch:
Ian, I have libbpf-devel installed:
⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
libbpf-0.4.0-1.fc34.x86_64
libbpf-devel-0.4.0-1.fc34.x86_64
⬢[acme@toolbox perf]$
But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to system headers for all of the homies in tools/lib/.
I bet that if I remove the libbpf-devel package it works, yeah, just tested. So we need to make those overrides dependent on using LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer that is -Itools/lib/, using a more finegrained approach, right?
- Arnaldo
Quoted 51 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 34 lines
Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 27 lines
After merging torvalds/master to perf/urgent I'm getting this:
util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
25 | struct btf *__weakbtf__load_from_kernel_by_id(__u32 id)| ^~~~~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes] 37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev) | ^~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes] 46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev) | ^~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes] 55 | btf__raw_data(const struct btf *btf_ro, __u32 *size) | ^~~~~~~~~~~~~cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf-event.o] Error 1 make[4]: *** Waiting for unfinished jobs....util/bpf_counter.c: In function ‘bpf_target_prog_name’:util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]82 | btf = btf__load_from_kernel_by_id(info_linear->info.btf_id); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
82 | btf =btf__load_from_kernel_by_id(info_linear->info.btf_id);
| ^
cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf_counter.o] Error 1Quoted 5 lines
I'm checking now...
BTW I test perf builds with:
make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C
tools/perfinstall-bin && git status && perf test pythonQuoted 7 lines
Nevermind, this was due to a patch by Ian Rogers I was testing, bisecting get up to the last patch, since I had merged torvalds/master today it got me to a wrong correlation, sorry for the disturbance.
For reference, this is the patch:
Ian, I have libbpf-devel installed:
⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
libbpf-0.4.0-1.fc34.x86_64
libbpf-devel-0.4.0-1.fc34.x86_64
⬢[acme@toolbox perf]$
Quoted 7 lines
But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to system headers for all of the homies in
tools/lib/.I bet that if I remove the libbpf-devel package it works, yeah, just tested. So we need to make those overrides dependent on using LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer that is -Itools/lib/, using a more finegrained approach, right?
Ugh, this is messy. The -I for tools/lib is overloaded and being used in tools/perf/util/bpf-event.c so that bpf/bpf.h, bpf/btf.h and bpf/libbpf.h can be found. Likewise, for tools/perf/util/debug.c it is used to pick up traceevent/event-parse.h.
Assuming LIBBPF_DYNAMIC=1 and LIBTRACEEVENT_DYNAMIC=1 then we get different combinations of:
libtraceevent >= 1.3 && libbpf >= 0.6 - -I is broken for debug.c, -idirafter okay
libtraceevent >= 1.3 && libbpf < 0.6 - -I is broken for debug.c, -idirafter broken for bpf-event.c
libtraceevent < 1.3 && libbpf >= 0.6 - -I should build okay but headers mismatched, -idirafter okay
libtraceevent < 1.3 && libbpf < 0.6 - -I will fail to link bpf-event.c, -idirafter broken for bpf-event.c
As the choice of -I and -idirafter are binary then there will always be a way to break the build. We could modify the build so that the -I/-idirafter only applies to the affected C files. This postpones the problem to when libbpf and libtracevent are in the same file, which doesn't happen currently. I think if you want the dynamic behavior then you need to use idirafter.
Thanks,
Ian
Quoted 57 lines · Ian Rogers
Quoted 54 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 34 lines
Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 27 lines
After merging torvalds/master to perf/urgent I'm getting this:
util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
25 | struct btf *__weakbtf__load_from_kernel_by_id(__u32 id)| ^~~~~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes] 37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev) | ^~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes] 46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev) | ^~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes] 55 | btf__raw_data(const struct btf *btf_ro, __u32 *size) | ^~~~~~~~~~~~~cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf-event.o] Error 1 make[4]: *** Waiting for unfinished jobs....util/bpf_counter.c: In function ‘bpf_target_prog_name’:util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]82 | btf = btf__load_from_kernel_by_id(info_linear->info.btf_id); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
82 | btf =btf__load_from_kernel_by_id(info_linear->info.btf_id);
| ^
cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf_counter.o] Error 1Quoted 5 lines
I'm checking now...
BTW I test perf builds with:
make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C
tools/perfinstall-bin && git status && perf test pythonQuoted 7 lines
Nevermind, this was due to a patch by Ian Rogers I was testing, bisecting get up to the last patch, since I had merged torvalds/master today it got me to a wrong correlation, sorry for the disturbance.
For reference, this is the patch:
Ian, I have libbpf-devel installed:
⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
libbpf-0.4.0-1.fc34.x86_64
libbpf-devel-0.4.0-1.fc34.x86_64
⬢[acme@toolbox perf]$But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to system headers for all of the homies in
tools/lib/.
Quoted 7 lines
Quoted 4 lines
I bet that if I remove the libbpf-devel package it works, yeah, just tested. So we need to make those overrides dependent on using LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer that is -Itools/lib/, using a more finegrained approach, right?
Ugh, this is messy. The -I for
tools/libis overloaded and being used intools/perf/util/bpf-event.cso thatbpf/bpf.h,bpf/btf.hand
can you do make install for libbpf instead and have it install headers into a dedicated target directory which can be added into -I search path. Quentin did this for all the other libbpf users in kernel tree (bpftool, resolve_btfids, etc) and it works great.
Quoted 23 lines
bpf/libbpf.hcan be found. Likewise, fortools/perf/util/debug.cit is used to pick uptraceevent/event-parse.h.Assuming LIBBPF_DYNAMIC=1 and LIBTRACEEVENT_DYNAMIC=1 then we get different combinations of:
libtraceevent >= 1.3 && libbpf >= 0.6 - -I is broken fordebug.c, -idirafter okay
libtraceevent >= 1.3 && libbpf < 0.6 - -I is broken fordebug.c, -idirafter broken forbpf-event.c
libtraceevent < 1.3 && libbpf >= 0.6 - -I should build okay but headers mismatched, -idirafter okay
libtraceevent < 1.3 && libbpf < 0.6 - -I will fail to linkbpf-event.c, -idirafter broken forbpf-event.cAs the choice of -I and -idirafter are binary then there will always be a way to break the build. We could modify the build so that the -I/-idirafter only applies to the affected C files. This postpones the problem to when libbpf and libtracevent are in the same file, which doesn't happen currently. I think if you want the dynamic behavior then you need to use idirafter.
Thanks,
Ian
Quoted 64 lines · Andrii Nakryiko
Quoted 62 lines · Ian Rogers
Quoted 58 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 34 lines
Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 27 lines
After merging torvalds/master to perf/urgent I'm getting this:
util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
25 | struct btf *__weakbtf__load_from_kernel_by_id(__u32 id)| ^~~~~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes] 37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev) | ^~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes] 46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev) | ^~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes] 55 | btf__raw_data(const struct btf *btf_ro, __u32 *size) | ^~~~~~~~~~~~~cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf-event.o] Error 1 make[4]: *** Waiting for unfinished jobs....util/bpf_counter.c: In function ‘bpf_target_prog_name’:util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]82 | btf = btf__load_from_kernel_by_id(info_linear->info.btf_id); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
82 | btf =btf__load_from_kernel_by_id(info_linear->info.btf_id);
| ^
cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf_counter.o] Error 1Quoted 5 lines
I'm checking now...
BTW I test perf builds with:
make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C
tools/perfinstall-bin && git status && perf test pythonQuoted 7 lines
Nevermind, this was due to a patch by Ian Rogers I was testing, bisecting get up to the last patch, since I had merged torvalds/master today it got me to a wrong correlation, sorry for the disturbance.
For reference, this is the patch:
Ian, I have libbpf-devel installed:
⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
libbpf-0.4.0-1.fc34.x86_64
libbpf-devel-0.4.0-1.fc34.x86_64
⬢[acme@toolbox perf]$But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to system headers for all of the homies in
tools/lib/.I bet that if I remove the libbpf-devel package it works, yeah, just tested. So we need to make those overrides dependent on using LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer that is -Itools/lib/, using a more finegrained approach, right?
Quoted 7 lines
Quoted 2 lines
Ugh, this is messy. The -I for
tools/libis overloaded and being used intools/perf/util/bpf-event.cso thatbpf/bpf.h,bpf/btf.handcan you do
make installfor libbpf instead and have it install headers into a dedicated target directory which can be added into -I search path. Quentin did this for all the other libbpf users in kernel tree (bpftool,resolve_btfids, etc) and it works great.
This sounds good to me, and being able to borrow code from bpftool should make writing it is straightforward. I'll try to find time to do a patch, but I don't mind someone getting there before me :-)
Thanks,
Ian
Quoted 23 lines
Quoted 23 lines
bpf/libbpf.hcan be found. Likewise, fortools/perf/util/debug.cit is used to pick uptraceevent/event-parse.h.Assuming LIBBPF_DYNAMIC=1 and LIBTRACEEVENT_DYNAMIC=1 then we get different combinations of:
libtraceevent >= 1.3 && libbpf >= 0.6 - -I is broken fordebug.c, -idirafter okay
libtraceevent >= 1.3 && libbpf < 0.6 - -I is broken fordebug.c, -idirafter broken forbpf-event.c
libtraceevent < 1.3 && libbpf >= 0.6 - -I should build okay but headers mismatched, -idirafter okay
libtraceevent < 1.3 && libbpf < 0.6 - -I will fail to linkbpf-event.c, -idirafter broken forbpf-event.cAs the choice of -I and -idirafter are binary then there will always be a way to break the build. We could modify the build so that the -I/-idirafter only applies to the affected C files. This postpones the problem to when libbpf and libtracevent are in the same file, which doesn't happen currently. I think if you want the dynamic behavior then you need to use idirafter.
Thanks,
Ian
Quoted 70 lines · Ian Rogers
Quoted 67 lines · Andrii Nakryiko
Quoted 64 lines · Ian Rogers
Quoted 51 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 34 lines
Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 27 lines
After merging torvalds/master to perf/urgent I'm getting this:
util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
25 | struct btf *__weakbtf__load_from_kernel_by_id(__u32 id)| ^~~~~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes] 37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev) | ^~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes] 46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev) | ^~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes] 55 | btf__raw_data(const struct btf *btf_ro, __u32 *size) | ^~~~~~~~~~~~~cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf-event.o] Error 1 make[4]: *** Waiting for unfinished jobs....util/bpf_counter.c: In function ‘bpf_target_prog_name’:util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]82 | btf = btf__load_from_kernel_by_id(info_linear->info.btf_id); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
82 | btf =btf__load_from_kernel_by_id(info_linear->info.btf_id);
| ^
cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf_counter.o] Error 1Quoted 5 lines
I'm checking now...
BTW I test perf builds with:
make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C
tools/perfinstall-bin && git status && perf test pythonQuoted 7 lines
Nevermind, this was due to a patch by Ian Rogers I was testing, bisecting get up to the last patch, since I had merged torvalds/master today it got me to a wrong correlation, sorry for the disturbance.
For reference, this is the patch:
Ian, I have libbpf-devel installed:
⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
libbpf-0.4.0-1.fc34.x86_64
libbpf-devel-0.4.0-1.fc34.x86_64
⬢[acme@toolbox perf]$Quoted 7 lines
But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to system headers for all of the homies in
tools/lib/.I bet that if I remove the libbpf-devel package it works, yeah, just tested. So we need to make those overrides dependent on using LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer that is -Itools/lib/, using a more finegrained approach, right?
Ugh, this is messy. The -I for
tools/libis overloaded and being used intools/perf/util/bpf-event.cso thatbpf/bpf.h,bpf/btf.hand
Quoted 8 lines
Quoted 4 lines
can you do
make installfor libbpf instead and have it install headers into a dedicated target directory which can be added into -I search path. Quentin did this for all the other libbpf users in kernel tree (bpftool,resolve_btfids, etc) and it works great.This sounds good to me, and being able to borrow code from bpftool should make writing it is straightforward. I'll try to find time to do a patch, but I don't mind someone getting there before me :-)
So tools/lib also provides subcmd, symbol and api. These will need Makefiles to allow an install and likely the header file structure altering. This seems like too big a fix for the next 5.16rc, wdyt?
Thanks,
Ian
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
bpf/libbpf.hcan be found. Likewise, fortools/perf/util/debug.cit is used to pick uptraceevent/event-parse.h.Assuming LIBBPF_DYNAMIC=1 and LIBTRACEEVENT_DYNAMIC=1 then we get different combinations of:
libtraceevent >= 1.3 && libbpf >= 0.6 - -I is broken fordebug.c, -idirafter okay
libtraceevent >= 1.3 && libbpf < 0.6 - -I is broken fordebug.c, -idirafter broken forbpf-event.c
libtraceevent < 1.3 && libbpf >= 0.6 - -I should build okay but headers mismatched, -idirafter okay
libtraceevent < 1.3 && libbpf < 0.6 - -I will fail to linkbpf-event.c, -idirafter broken forbpf-event.cAs the choice of -I and -idirafter are binary then there will always be a way to break the build. We could modify the build so that the -I/-idirafter only applies to the affected C files. This postpones the problem to when libbpf and libtracevent are in the same file, which doesn't happen currently. I think if you want the dynamic behavior then you need to use idirafter.
Thanks,
Ian
Em Thu, Jan 06, 2022 at 07:30:34PM -0800, Ian Rogers escreveu:
Quoted 76 lines · Ian Rogers
Quoted 75 lines
Quoted 71 lines · Andrii Nakryiko
Quoted 57 lines · Ian Rogers
Quoted 54 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 34 lines
Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 27 lines
After merging torvalds/master to perf/urgent I'm getting this:
util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
25 | struct btf *__weakbtf__load_from_kernel_by_id(__u32 id)| ^~~~~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes] 37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev) | ^~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes] 46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev) | ^~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes] 55 | btf__raw_data(const struct btf *btf_ro, __u32 *size) | ^~~~~~~~~~~~~cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf-event.o] Error 1 make[4]: *** Waiting for unfinished jobs....util/bpf_counter.c: In function ‘bpf_target_prog_name’:util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]82 | btf = btf__load_from_kernel_by_id(info_linear->info.btf_id); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
82 | btf =btf__load_from_kernel_by_id(info_linear->info.btf_id);
| ^
cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf_counter.o] Error 1Quoted 5 lines
I'm checking now...
BTW I test perf builds with:
make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C
tools/perfinstall-bin && git status && perf test pythonQuoted 7 lines
Nevermind, this was due to a patch by Ian Rogers I was testing, bisecting get up to the last patch, since I had merged torvalds/master today it got me to a wrong correlation, sorry for the disturbance.
For reference, this is the patch:
Ian, I have libbpf-devel installed:
⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
libbpf-0.4.0-1.fc34.x86_64
libbpf-devel-0.4.0-1.fc34.x86_64
⬢[acme@toolbox perf]$But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to system headers for all of the homies in
tools/lib/.Quoted 7 lines
Quoted 4 lines
I bet that if I remove the libbpf-devel package it works, yeah, just tested. So we need to make those overrides dependent on using LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer that is -Itools/lib/, using a more finegrained approach, right?
Ugh, this is messy. The -I for
tools/libis overloaded and being used intools/perf/util/bpf-event.cso thatbpf/bpf.h,bpf/btf.handcan you do
make installfor libbpf instead and have it install headers into a dedicated target directory which can be added into -I search path. Quentin did this for all the other libbpf users in kernel tree (bpftool,resolve_btfids, etc) and it works great.
Quoted 7 lines
Quoted 3 lines
This sounds good to me, and being able to borrow code from bpftool should make writing it is straightforward. I'll try to find time to do a patch, but I don't mind someone getting there before me :-)
So
tools/libalso provides subcmd, symbol and api. These will need Makefiles to allow an install and likely the header file structure altering. This seems like too big a fix for the next 5.16rc, wdyt?
Right, I think the best thing is to revert the patch Jiri pointed out, right?
- Arnaldo
Quoted 82 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 06, 2022 at 07:30:34PM -0800, Ian Rogers escreveu:
Quoted 80 lines · Ian Rogers
Quoted 78 lines
Quoted 64 lines · Andrii Nakryiko
Quoted 62 lines · Ian Rogers
Quoted 58 lines · Arnaldo Carvalho de Melo
Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 34 lines
Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 27 lines
After merging torvalds/master to perf/urgent I'm getting this:
util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
25 | struct btf *__weakbtf__load_from_kernel_by_id(__u32 id)| ^~~~~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes] 37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev) | ^~~~~~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes] 46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev) | ^~~~~~~~~~~~~~~~~~~~ util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes] 55 | btf__raw_data(const struct btf *btf_ro, __u32 *size) | ^~~~~~~~~~~~~cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf-event.o] Error 1 make[4]: *** Waiting for unfinished jobs....util/bpf_counter.c: In function ‘bpf_target_prog_name’:util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]82 | btf = btf__load_from_kernel_by_id(info_linear->info.btf_id); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
82 | btf =btf__load_from_kernel_by_id(info_linear->info.btf_id);
| ^
cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96:/tmp/build/perf/util/bpf_counter.o] Error 1Quoted 5 lines
I'm checking now...
BTW I test perf builds with:
make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C
tools/perfinstall-bin && git status && perf test pythonQuoted 7 lines
Nevermind, this was due to a patch by Ian Rogers I was testing, bisecting get up to the last patch, since I had merged torvalds/master today it got me to a wrong correlation, sorry for the disturbance.
For reference, this is the patch:
Ian, I have libbpf-devel installed:
⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
libbpf-0.4.0-1.fc34.x86_64
libbpf-devel-0.4.0-1.fc34.x86_64
⬢[acme@toolbox perf]$But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to system headers for all of the homies in
tools/lib/.I bet that if I remove the libbpf-devel package it works, yeah, just tested. So we need to make those overrides dependent on using LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer that is -Itools/lib/, using a more finegrained approach, right?
Quoted 7 lines
Quoted 2 lines
Ugh, this is messy. The -I for
tools/libis overloaded and being used intools/perf/util/bpf-event.cso thatbpf/bpf.h,bpf/btf.handcan you do
make installfor libbpf instead and have it install headers into a dedicated target directory which can be added into -I search path. Quentin did this for all the other libbpf users in kernel tree (bpftool,resolve_btfids, etc) and it works great.This sounds good to me, and being able to borrow code from bpftool should make writing it is straightforward. I'll try to find time to do a patch, but I don't mind someone getting there before me :-)
Quoted 6 lines
Quoted 3 lines
So
tools/libalso provides subcmd, symbol and api. These will need Makefiles to allow an install and likely the header file structure altering. This seems like too big a fix for the next 5.16rc, wdyt?Right, I think the best thing is to revert the patch Jiri pointed out, right?
Your call. There is a latent bug that with LIBTRACEEVENT_DYNAMIC we are using tools/lib/traceevent header files. Reverting the change means we don't break because of this, but it means that people building with LIBTRACEEVENT_DYNAMIC and newer libtraceevent (at least my employer :-) ) lose logging. I can carry the change locally, so not a big loss :-) There are a few issues stemming from this:
1) we've identified the current build is wrong for xxx_DYNAMIC options as tools/lib versions headers always override
2) to address this we should make the tools/lib things proper libraries like libbpf, libtraceevent, etc.
3) once we have proper libraries, we need to update the perf build to build non-dynamic libraries then depend on the built/installed header files
I expect at least some of this is going to break when testing on many distributions as that just seems to be what always happens, and changing the build in this significant way is going to have implications. Doing this means that the code base is in better shape and logging works. To counter some of the many distribution pain, do you have a way to reproduce your testing? My OpenSuSE recipe is:
``` # Get the image docker pull opensuse/tumbleweed # Start it with an interactive bash shell and mounting the current directory as /kernel-src sudo docker run --privileged -it --net=host --env="DISPLAY" --mount type=bind,source="$(pwd)",target=/kernel-src opensuse/tumbleweed /bin/bash # Install missing rpms zypper install make gcc diffutils flex bison kernel-devel findutils libelf-devel python3 kernel-kvmsmall-devel glibc-devel # Go to /kernel-src and build into /tmp, etc. ```
But finding every distribution, every rpm, etc. is quite laborious.
Thanks,
Ian
Quoted 1 line
- Arnaldo
Em Fri, Jan 07, 2022 at 11:26:50AM -0800, Ian Rogers escreveu:
Quoted 12 lines · Arnaldo Carvalho de Melo
Quoted 10 lines
Em Thu, Jan 06, 2022 at 07:30:34PM -0800, Ian Rogers escreveu:
Quoted 6 lines · Ian Rogers
Quoted 2 lines · Andrii Nakryiko
Quoted 1 line · Ian Rogers
So
tools/libalso provides subcmd, symbol and api. These will need Makefiles to allow an install and likely the header file structure altering. This seems like too big a fix for the next 5.16rc, wdyt?Right, I think the best thing is to revert the patch Jiri pointed out, right?
Quoted 6 lines
Your call. There is a latent bug that with
LIBTRACEEVENT_DYNAMICwe are usingtools/lib/traceeventheader files. Reverting the change means we don't break because of this, but it means that people building withLIBTRACEEVENT_DYNAMICand newer libtraceevent (at least my employer :-) ) lose logging. I can carry the change locally, so not a big loss :-)
Just for a while, we should get this fixed for v5.17, for v5.16 a revert is required :-\
Quoted 3 lines
There are a few issues stemming from this:
1) we've identified the current build is wrong forxxx_DYNAMICoptions astools/libversions headers always override
yeap
Quoted 2 lines
2) to address this we should make the
tools/libthings proper libraries like libbpf, libtraceevent, etc.
yeap
Quoted 3 lines
3) once we have proper libraries, we need to update the perf build to build non-dynamic libraries then depend on the built/installed header files
yeap
Quoted 5 lines
I expect at least some of this is going to break when testing on many distributions as that just seems to be what always happens, and changing the build in this significant way is going to have implications. Doing this means that the code base is in better shape and logging works.
Right, its nice that I have a mechanism to test build in 80+ distro versions/cross-build environments. :-)
Quoted 2 lines
To counter some of the many distribution pain, do
you have a way to reproduce your testing? My OpenSuSE recipe is:
So I'm adding this:
[perfbuilder@five tumbleweed]$ git diff rx_and_build.sh
opensuse/tumbleweed/rx_and_build.sh +11 -0
@@ -11,11 +11,19 @@ build_perf_gcc() { make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" NO_LIBELF=1 -C tools/perf O=/tmp/build/perf || exit 1 rm -rf /tmp/build/perf ; mkdir /tmp/build/perf make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" NO_LIBBPF=1 -C tools/perf O=/tmp/build/perf || exit 1 + [ -d /usr/include/traceevent/ ] && \ + make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBTRACEEVENT_DYNAMIC=1 -C tools/perf O=/tmp/build/perf || exit 1 set +o xtrace } build_perf_clang() { set -o xtrace + + if [ ! $NO_BUILD_BPF_SKEL ] ; then + rm -rf /tmp/build/perf ; mkdir /tmp/build/perf + make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" BUILD_BPF_SKEL=1 -C tools/perf O=/tmp/build/perf || exit 1 + fi + rm -rf /tmp/build/perf ; mkdir /tmp/build/perf make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" -C tools/perf O=/tmp/build/perf CC=clang || exit 1 rm -rf /tmp/build/perf ; mkdir /tmp/build/perf
@@ -26,6 +34,8 @@ build_perf_clang() { make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBCLANGLLVM=1 -C tools/perf O=/tmp/build/perf CC=clang || exit 1 rm -rf /tmp/build/perf ; mkdir /tmp/build/perf make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBCLANGLLVM=1 -C tools/perf O=/tmp/build/perf || exit 1 + [ -d /usr/include/traceevent/ ] && \ + make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBTRACEEVENT_DYNAMIC=1 -C tools/perf O=/tmp/build/perf || exit 1 set +o xtrace }
@@ -48,6 +58,7 @@ TARBALL=`basename $TARBALL_URL` xzcat $TARBALL | tar xvf - SRCDIR=`echo $TARBALL | sed -r 's/(.*).tar\..*/\1/g'` cd /git/$SRCDIR +echo -n BUILD_TARBALL_HEAD= cat HEAD # print the version for dm to harvest and put in the status line
[perfbuilder@five tumbleweed]$
And I'll add this to 'make -C tools/perf build-test' too, what I have in rx_and_build.sh is a limited set of 'build-test', building with it in all containers take more time, it is possible if one does:
[perfbuilder@five ~]$ export BUILD_TARBALL=http://192.168.100.2/perf/perf-5.16.0-rc8.tar.xz [perfbuilder@five ~]$ export BUILD_CMD='make -C tools/perf build-cmd' [perfbuilder@five ~]$ dm
Then it will do it for gcc and for clang, takes a while...
My Dockerfile for tumbleweed, btw, is:
[perfbuilder@five tumbleweed]$ cat Dockerfile # acmel/linux-perf-tools-build-opensuse:tumbleweed FROM docker.io/opensuse/tumbleweed
MAINTAINER Arnaldo Carvalho de Melo <[email protected]>
# shadow for groupadd
RUN zypper -n update && \
zypper -n install shadow \
make gcc gcc-c++ flex bison cpio \
bc file findutils clang llvm curl tar xz \
libunwind-devel openssl-devel slang-devel python-devel \
systemtap-sdt-devel gtk2-devel xz-devel binutils-devel \
libelf-devel libdw-devel audit-devel libzstd-devel \
java-1_8_0-openjdk-headless libcap-devel \
clang-devel llvm-devel perl libnuma-devel \
libbpf-devel libtraceevent-devel \
babeltrace-devel OpenCSD-devel cmake xmlto asciidoc && \
zypper clean --all && \
mkdir -m 777 -p /git /tmp/build/perf /tmp/build/objtool /tmp/build/linux && \
groupadd -r perfbuilder && \
useradd -m -r -g perfbuilder perfbuilder && \
chown -R perfbuilder.perfbuilder /tmp/build/ /git/
USER perfbuilder
COPY rx_and_build.sh /
COPY .bash_profile /home/perfbuilder/
ENTRYPOINT ["/rx_and_build.sh"]
[perfbuilder@five tumbleweed]$Quoted 13 lines
``` # Get the image docker pull opensuse/tumbleweed # Start it with an interactive bash shell and mounting the current directory as /kernel-src sudo docker run --privileged -it --net=host --env="DISPLAY" --mount type=bind,source="$(pwd)",target=/kernel-src opensuse/tumbleweed /bin/bash # Install missing rpms zypper install make gcc diffutils flex bison kernel-devel findutils libelf-devel python3 kernel-kvmsmall-devel glibc-devel # Go to /kernel-src and build into /tmp, etc. ```
Quoted 1 line
But finding every distribution, every rpm, etc. is quite laborious.
Yeah, but I did it already for quite a few distros :-)
[perfbuilder@five linux-perf-tools-build]$ find . -name Dockerfile | nl
1 ./alpine/3.10/Dockerfile
2 ./alpine/3.11/Dockerfile
3 ./alpine/3.12/Dockerfile
4 ./alpine/3.13/Dockerfile
5 ./alpine/3.5/Dockerfile
6 ./alpine/3.6/Dockerfile
7 ./alpine/3.7/Dockerfile
8 ./alpine/3.8/Dockerfile
9 ./alpine/3.9/Dockerfile
10 ./alpine/edge/Dockerfile
11 ./alpine/3.14/Dockerfile
12 ./alpine/3.4/Dockerfile
13 ./alpine/3.15/Dockerfile
14 ./alt/p8/Dockerfile
15 ./alt/p9/Dockerfile
16 ./alt/sisyphus/Dockerfile
17 ./alt/p10/Dockerfile
18 ./amazonlinux/1/Dockerfile
19 ./amazonlinux/2/Dockerfile
20 ./android/end-of-life/r22b/arm/Dockerfile
21 ./android/end-of-life/r12b/arm/Dockerfile
22 ./android/end-of-life/r15c/arm/Dockerfile
23 ./archlinux/base/Dockerfile
24 ./centos/8/Dockerfile
25 ./centos/end-of-life/5/Dockerfile
26 ./centos/end-of-life/6/Dockerfile
27 ./centos/end-of-life/7/Dockerfile
28 ./centos/stream/Dockerfile
29 ./clearlinux/latest/Dockerfile
30 ./debian/10/Dockerfile
31 ./debian/9/Dockerfile
32 ./debian/end-of-life/7/Dockerfile
33 ./debian/end-of-life/8/Dockerfile
34 ./debian/experimental/x-arm64/Dockerfile
35 ./debian/experimental/x-mips/Dockerfile
36 ./debian/experimental/x-mips64/Dockerfile
37 ./debian/experimental/x-mipsel/Dockerfile
38 ./debian/experimental/Dockerfile
39 ./debian/11/Dockerfile
40 ./fedora/22/Dockerfile
41 ./fedora/23/Dockerfile
42 ./fedora/24/x-ARC-uClibc/Dockerfile
43 ./fedora/24/Dockerfile
44 ./fedora/25/Dockerfile
45 ./fedora/26/Dockerfile
46 ./fedora/27/Dockerfile
47 ./fedora/28/Dockerfile
48 ./fedora/29/Dockerfile
49 ./fedora/30/Dockerfile
50 ./fedora/31/Dockerfile
51 ./fedora/32/Dockerfile
52 ./fedora/33/Dockerfile
53 ./fedora/34/x-ARC-glibc/Dockerfile
54 ./fedora/34/x-ARC-uClibc/Dockerfile
55 ./fedora/34/Dockerfile
56 ./fedora/end-of-life/21/Dockerfile
57 ./fedora/end-of-life/20/Dockerfile
58 ./fedora/rawhide/Dockerfile
59 ./fedora/35/Dockerfile
60 ./gentoo/end-of-life/stage3-amd64/Dockerfile
61 ./gentoo/stage3/Dockerfile
62 ./mageia/5/Dockerfile
63 ./mageia/6/Dockerfile
64 ./mageia/7/Dockerfile
65 ./mageia/8/Dockerfile
66 ./manjaro/base/Dockerfile
67 ./openmandriva/cooker/Dockerfile
68 ./opensuse/15.0/Dockerfile
69 ./opensuse/15.1/Dockerfile
70 ./opensuse/15.2/Dockerfile
71 ./opensuse/15.3/Dockerfile
72 ./opensuse/end-of-life/13.2/Dockerfile
73 ./opensuse/end-of-life/42.1/Dockerfile
74 ./opensuse/end-of-life/42.2/Dockerfile
75 ./opensuse/end-of-life/42.3/Dockerfile
76 ./opensuse/tumbleweed/Dockerfile
77 ./opensuse/15.4/Dockerfile
78 ./oraclelinux/8/Dockerfile
79 ./oraclelinux/end-of-life/6/Dockerfile
80 ./oraclelinux/end-of-life/7/Dockerfile
81 ./rhel7/Dockerfile
82 ./ubuntu/16.04/x-arm/Dockerfile
83 ./ubuntu/16.04/x-arm64/Dockerfile
84 ./ubuntu/16.04/x-powerpc/Dockerfile
85 ./ubuntu/16.04/x-powerpc64/Dockerfile
86 ./ubuntu/16.04/x-powerpc64el/Dockerfile
87 ./ubuntu/16.04/x-s390/Dockerfile
88 ./ubuntu/16.04/Dockerfile
89 ./ubuntu/18.04/x-arm/Dockerfile
90 ./ubuntu/18.04/x-arm64/Dockerfile
91 ./ubuntu/18.04/x-m68k/Dockerfile
92 ./ubuntu/18.04/x-powerpc/Dockerfile
93 ./ubuntu/18.04/x-powerpc64/Dockerfile
94 ./ubuntu/18.04/x-powerpc64el/Dockerfile
95 ./ubuntu/18.04/x-riscv64/Dockerfile
96 ./ubuntu/18.04/x-s390/Dockerfile
97 ./ubuntu/18.04/x-sh4/Dockerfile
98 ./ubuntu/18.04/x-sparc64/Dockerfile
99 ./ubuntu/18.04/Dockerfile 100 ./ubuntu/20.04/x-powerpc64el/Dockerfile
101 ./ubuntu/20.04/x-s390/Dockerfile
102 ./ubuntu/20.04/Dockerfile
103 ./ubuntu/20.04/x-arm/Dockerfile
104 ./ubuntu/20.10/Dockerfile
105 ./ubuntu/21.04/Dockerfile
106 ./ubuntu/21.04/end-of-life/x-mips/Dockerfile
107 ./ubuntu/21.04/end-of-life/x-mips64/Dockerfile
108 ./ubuntu/end-of-life/15.04/Dockerfile
109 ./ubuntu/end-of-life/15.10/Dockerfile
110 ./ubuntu/end-of-life/16.10/Dockerfile
111 ./ubuntu/end-of-life/17.04/Dockerfile
112 ./ubuntu/end-of-life/17.10/Dockerfile
113 ./ubuntu/end-of-life/18.10/Dockerfile
114 ./ubuntu/end-of-life/19.04/Dockerfile
115 ./ubuntu/end-of-life/19.10/Dockerfile
116 ./ubuntu/end-of-life/19.10/x-alpha/Dockerfile
117 ./ubuntu/end-of-life/19.10/x-arm64/Dockerfile
118 ./ubuntu/end-of-life/19.10/x-hppa/Dockerfile
119 ./ubuntu/end-of-life/12.04/Dockerfile
120 ./ubuntu/end-of-life/14.04/end-of-life/x-linaro-arm64/Dockerfile
121 ./ubuntu/end-of-life/14.04/Dockerfile
122 ./ubuntu/21.10/Dockerfile
123 ./ubuntu/22.04/Dockerfile
124 ./almalinux/8/Dockerfile
125 ./rockylinux/8/Dockerfile
126 ./ubi8/Dockerfile
[perfbuilder@five linux-perf-tools-build]$
Quoted 282 lines · Arnaldo Carvalho de Melo
Em Fri, Jan 07, 2022 at 11:26:50AM -0800, Ian Rogers escreveu:
Quoted 12 lines · Arnaldo Carvalho de Melo
Quoted 10 lines
Em Thu, Jan 06, 2022 at 07:30:34PM -0800, Ian Rogers escreveu:
Quoted 6 lines · Ian Rogers
Quoted 2 lines · Andrii Nakryiko
Quoted 1 line · Ian Rogers
So
tools/libalso provides subcmd, symbol and api. These will need Makefiles to allow an install and likely the header file structure altering. This seems like too big a fix for the next 5.16rc, wdyt?Right, I think the best thing is to revert the patch Jiri pointed out, right?
Quoted 6 lines
Your call. There is a latent bug that with
LIBTRACEEVENT_DYNAMICwe are usingtools/lib/traceeventheader files. Reverting the change means we don't break because of this, but it means that people building withLIBTRACEEVENT_DYNAMICand newer libtraceevent (at least my employer :-) ) lose logging. I can carry the change locally, so not a big loss :-)Just for a while, we should get this fixed for v5.17, for v5.16 a revert is required :-\
Quoted 3 lines
There are a few issues stemming from this:
1) we've identified the current build is wrong forxxx_DYNAMICoptions astools/libversions headers always overrideyeap
Quoted 2 lines
2) to address this we should make the
tools/libthings proper libraries like libbpf, libtraceevent, etc.yeap
Quoted 3 lines
3) once we have proper libraries, we need to update the perf build to build non-dynamic libraries then depend on the built/installed header files
yeap
Quoted 5 lines
I expect at least some of this is going to break when testing on many distributions as that just seems to be what always happens, and changing the build in this significant way is going to have implications. Doing this means that the code base is in better shape and logging works.
Right, its nice that I have a mechanism to test build in 80+ distro versions/cross-build environments. :-)
Quoted 2 lines
To counter some of the many distribution pain, do
you have a way to reproduce your testing? My OpenSuSE recipe is:So I'm adding this:
[perfbuilder@five tumbleweed]$ git diff
rx_and_build.shopensuse/tumbleweed/rx_and_build.sh +11
-0@@ -11,11 +11,19 @@ build_perf_gcc() { make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" NO_LIBELF=1 -C tools/perf O=/tmp/build/perf || exit 1 rm -rf /tmp/build/perf ; mkdir /tmp/build/perf make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" NO_LIBBPF=1 -C tools/perf O=/tmp/build/perf || exit 1 + [ -d /usr/include/traceevent/ ] && \ + make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBTRACEEVENT_DYNAMIC=1 -C tools/perf O=/tmp/build/perf || exit 1 set +o xtrace } build_perf_clang() { set -o xtrace + + if [ ! $NO_BUILD_BPF_SKEL ] ; then + rm -rf /tmp/build/perf ; mkdir /tmp/build/perf + make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" BUILD_BPF_SKEL=1 -C tools/perf O=/tmp/build/perf || exit 1 + fi + rm -rf /tmp/build/perf ; mkdir /tmp/build/perf make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" -C tools/perf O=/tmp/build/perf CC=clang || exit 1 rm -rf /tmp/build/perf ; mkdir /tmp/build/perf@@ -26,6 +34,8 @@ build_perf_clang() { make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBCLANGLLVM=1 -C tools/perf O=/tmp/build/perf CC=clang || exit 1 rm -rf /tmp/build/perf ; mkdir /tmp/build/perf make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBCLANGLLVM=1 -C tools/perf O=/tmp/build/perf || exit 1 + [ -d /usr/include/traceevent/ ] && \ + make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBTRACEEVENT_DYNAMIC=1 -C tools/perf O=/tmp/build/perf || exit 1 set +o xtrace }@@ -48,6 +58,7 @@ TARBALL=`basename $TARBALL_URL` xzcat $TARBALL | tar xvf - SRCDIR=`echo $TARBALL | sed -r 's/(.*).tar\..*/\1/g'` cd /git/$SRCDIR +echo -n BUILD_TARBALL_HEAD= cat HEAD # print the version for dm to harvest and put in the status line[perfbuilder@five tumbleweed]$
And I'll add this to 'make -C
tools/perfbuild-test' too, what I have inrx_and_build.shis a limited set of 'build-test', building with it in all containers take more time, it is possible if one does:[perfbuilder@five ~]$ export BUILD_TARBALL=http://192.168.100.2/perf/perf-5.16.0-rc8.tar.xz [perfbuilder@five ~]$ export BUILD_CMD='make -C
tools/perfbuild-cmd' [perfbuilder@five ~]$ dmThen it will do it for gcc and for clang, takes a while...
My Dockerfile for tumbleweed, btw, is:
[perfbuilder@five tumbleweed]$ cat Dockerfile # acmel/linux-perf-tools-build-opensuse:tumbleweed FROM docker.io/opensuse/tumbleweed
MAINTAINER Arnaldo Carvalho de Melo <[email protected]># shadow for groupadd RUN zypper -n update && \ zypper -n install shadow \ make gcc gcc-c++ flex bison cpio \ bc file findutils clang llvm curl tar xz \ libunwind-devel openssl-devel slang-devel python-devel \ systemtap-sdt-devel gtk2-devel xz-devel binutils-devel \ libelf-devel libdw-devel audit-devel libzstd-devel \ java-1_8_0-openjdk-headless libcap-devel \ clang-devel llvm-devel perl libnuma-devel \ libbpf-devel libtraceevent-devel \ babeltrace-devel OpenCSD-devel cmake xmlto asciidoc && \ zypper clean --all && \ mkdir -m 777 -p /git /tmp/build/perf /tmp/build/objtool /tmp/build/linux && \ groupadd -r perfbuilder && \ useradd -m -r -g perfbuilder perfbuilder && \ chown -R perfbuilder.perfbuilder /tmp/build/ /git/ USER perfbuilder COPY rx_and_build.sh / COPY .bash_profile /home/perfbuilder/ ENTRYPOINT ["/rx_and_build.sh"] [perfbuilder@five tumbleweed]$Quoted 13 lines
``` # Get the image docker pull opensuse/tumbleweed # Start it with an interactive bash shell and mounting the current directory as /kernel-src sudo docker run --privileged -it --net=host --env="DISPLAY" --mount type=bind,source="$(pwd)",target=/kernel-src opensuse/tumbleweed /bin/bash # Install missing rpms zypper install make gcc diffutils flex bison kernel-devel findutils libelf-devel python3 kernel-kvmsmall-devel glibc-devel # Go to /kernel-src and build into /tmp, etc. ```Quoted 1 line
But finding every distribution, every rpm, etc. is quite laborious.
Yeah, but I did it already for quite a few distros :-)
[perfbuilder@five linux-perf-tools-build]$ find . -name Dockerfile | nl
1 ./alpine/3.10/Dockerfile 2 ./alpine/3.11/Dockerfile 3 ./alpine/3.12/Dockerfile 4 ./alpine/3.13/Dockerfile 5 ./alpine/3.5/Dockerfile 6 ./alpine/3.6/Dockerfile 7 ./alpine/3.7/Dockerfile 8 ./alpine/3.8/Dockerfile 9 ./alpine/3.9/Dockerfile 10 ./alpine/edge/Dockerfile 11 ./alpine/3.14/Dockerfile 12 ./alpine/3.4/Dockerfile 13 ./alpine/3.15/Dockerfile 14 ./alt/p8/Dockerfile 15 ./alt/p9/Dockerfile 16 ./alt/sisyphus/Dockerfile 17 ./alt/p10/Dockerfile 18 ./amazonlinux/1/Dockerfile 19 ./amazonlinux/2/Dockerfile 20 ./android/end-of-life/r22b/arm/Dockerfile 21 ./android/end-of-life/r12b/arm/Dockerfile 22 ./android/end-of-life/r15c/arm/Dockerfile 23 ./archlinux/base/Dockerfile 24 ./centos/8/Dockerfile 25 ./centos/end-of-life/5/Dockerfile 26 ./centos/end-of-life/6/Dockerfile 27 ./centos/end-of-life/7/Dockerfile 28 ./centos/stream/Dockerfile 29 ./clearlinux/latest/Dockerfile 30 ./debian/10/Dockerfile 31 ./debian/9/Dockerfile 32 ./debian/end-of-life/7/Dockerfile 33 ./debian/end-of-life/8/Dockerfile 34 ./debian/experimental/x-arm64/Dockerfile 35 ./debian/experimental/x-mips/Dockerfile 36 ./debian/experimental/x-mips64/Dockerfile 37 ./debian/experimental/x-mipsel/Dockerfile 38 ./debian/experimental/Dockerfile 39 ./debian/11/Dockerfile 40 ./fedora/22/Dockerfile 41 ./fedora/23/Dockerfile 42 ./fedora/24/x-ARC-uClibc/Dockerfile 43 ./fedora/24/Dockerfile 44 ./fedora/25/Dockerfile 45 ./fedora/26/Dockerfile 46 ./fedora/27/Dockerfile 47 ./fedora/28/Dockerfile 48 ./fedora/29/Dockerfile 49 ./fedora/30/Dockerfile 50 ./fedora/31/Dockerfile 51 ./fedora/32/Dockerfile 52 ./fedora/33/Dockerfile 53 ./fedora/34/x-ARC-glibc/Dockerfile 54 ./fedora/34/x-ARC-uClibc/Dockerfile 55 ./fedora/34/Dockerfile 56 ./fedora/end-of-life/21/Dockerfile 57 ./fedora/end-of-life/20/Dockerfile 58 ./fedora/rawhide/Dockerfile 59 ./fedora/35/Dockerfile 60 ./gentoo/end-of-life/stage3-amd64/Dockerfile 61 ./gentoo/stage3/Dockerfile 62 ./mageia/5/Dockerfile 63 ./mageia/6/Dockerfile 64 ./mageia/7/Dockerfile 65 ./mageia/8/Dockerfile 66 ./manjaro/base/Dockerfile 67 ./openmandriva/cooker/Dockerfile 68 ./opensuse/15.0/Dockerfile 69 ./opensuse/15.1/Dockerfile 70 ./opensuse/15.2/Dockerfile 71 ./opensuse/15.3/Dockerfile 72 ./opensuse/end-of-life/13.2/Dockerfile 73 ./opensuse/end-of-life/42.1/Dockerfile 74 ./opensuse/end-of-life/42.2/Dockerfile 75 ./opensuse/end-of-life/42.3/Dockerfile 76 ./opensuse/tumbleweed/Dockerfile 77 ./opensuse/15.4/Dockerfile 78 ./oraclelinux/8/Dockerfile 79 ./oraclelinux/end-of-life/6/Dockerfile 80 ./oraclelinux/end-of-life/7/Dockerfile 81 ./rhel7/Dockerfile 82 ./ubuntu/16.04/x-arm/Dockerfile 83 ./ubuntu/16.04/x-arm64/Dockerfile 84 ./ubuntu/16.04/x-powerpc/Dockerfile 85 ./ubuntu/16.04/x-powerpc64/Dockerfile 86 ./ubuntu/16.04/x-powerpc64el/Dockerfile 87 ./ubuntu/16.04/x-s390/Dockerfile 88 ./ubuntu/16.04/Dockerfile 89 ./ubuntu/18.04/x-arm/Dockerfile 90 ./ubuntu/18.04/x-arm64/Dockerfile 91 ./ubuntu/18.04/x-m68k/Dockerfile 92 ./ubuntu/18.04/x-powerpc/Dockerfile 93 ./ubuntu/18.04/x-powerpc64/Dockerfile 94 ./ubuntu/18.04/x-powerpc64el/Dockerfile 95 ./ubuntu/18.04/x-riscv64/Dockerfile 96 ./ubuntu/18.04/x-s390/Dockerfile 97 ./ubuntu/18.04/x-sh4/Dockerfile 98 ./ubuntu/18.04/x-sparc64/Dockerfile 99 ./ubuntu/18.04/Dockerfile 100 ./ubuntu/20.04/x-powerpc64el/Dockerfile 101 ./ubuntu/20.04/x-s390/Dockerfile 102 ./ubuntu/20.04/Dockerfile 103 ./ubuntu/20.04/x-arm/Dockerfile 104 ./ubuntu/20.10/Dockerfile 105 ./ubuntu/21.04/Dockerfile 106 ./ubuntu/21.04/end-of-life/x-mips/Dockerfile 107 ./ubuntu/21.04/end-of-life/x-mips64/Dockerfile 108 ./ubuntu/end-of-life/15.04/Dockerfile 109 ./ubuntu/end-of-life/15.10/Dockerfile 110 ./ubuntu/end-of-life/16.10/Dockerfile 111 ./ubuntu/end-of-life/17.04/Dockerfile 112 ./ubuntu/end-of-life/17.10/Dockerfile 113 ./ubuntu/end-of-life/18.10/Dockerfile 114 ./ubuntu/end-of-life/19.04/Dockerfile 115 ./ubuntu/end-of-life/19.10/Dockerfile 116 ./ubuntu/end-of-life/19.10/x-alpha/Dockerfile 117 ./ubuntu/end-of-life/19.10/x-arm64/Dockerfile 118 ./ubuntu/end-of-life/19.10/x-hppa/Dockerfile 119 ./ubuntu/end-of-life/12.04/Dockerfile 120 ./ubuntu/end-of-life/14.04/end-of-life/x-linaro-arm64/Dockerfile 121 ./ubuntu/end-of-life/14.04/Dockerfile 122 ./ubuntu/21.10/Dockerfile 123 ./ubuntu/22.04/Dockerfile 124 ./almalinux/8/Dockerfile 125 ./rockylinux/8/Dockerfile 126 ./ubi8/Dockerfile
Quoted 1 line
[perfbuilder@five linux-perf-tools-build]$
Cool. Do you have this available externally?
Thanks,
Ian