libbpf: Fix up generation of bpf_helper_defs.h
Subsystems (5)
PATCH 1/1 libbpf: Fix up generation of bpf_helper_defs.h
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,
Thanks,
- Arnaldo
commit 94b2e22463f592d2161eb491ddb0b4659e2a91b4
Author: Arnaldo Carvalho de Melo <[email protected]> Date: Tue Nov 26 11:46:08 2019 -0300
libbpf: Fix up generation of bpf_helper_defs.h
Building perf as a detached tarball, i.e. by using one of:
$ make help | grep perf
perf-tar-src-pkg - Build perf-5.4.0.tar source tarball
perf-targz-src-pkg - Build perf-5.4.0.tar.gz source tarball perf-tarbz2-src-pkg - Build perf-5.4.0.tar.bz2 source tarball perf-tarxz-src-pkg - Build perf-5.4.0.tar.xz source tarball
$
And then trying to build the resulting tarball, which is the first thing that running:
$ make -C tools/perf build-test
does, ends up with these two problems:
make[3]: *** No rule to make target '/tmp/tmp.zq13cHILGB/perf-5.3.0/include/uapi/linux/bpf.h', needed by 'bpf_helper_defs.h'. Stop. make[3]: *** Waiting for unfinished jobs....
make[2]: *** [Makefile.perf:757: /tmp/tmp.zq13cHILGB/perf-5.3.0/tools/lib/bpf/libbpf.a] Error 2
make[2]: *** Waiting for unfinished jobs....
Because $(srcdir) points to the /tmp/tmp.zq13cHILGB/perf-5.3.0 directory and we need '/tools/ after that variable, and after fixing this then we get to another problem:
/bin/sh: /home/acme/git/perf/tools/scripts/bpf_helpers_doc.py: No such file or directory make[3]: *** [Makefile:184: bpf_helper_defs.h] Error 127
make[3]: *** Deleting file 'bpf_helper_defs.h'
LD /tmp/build/perf/libapi-in.o
make[2]: *** [Makefile.perf:778: /tmp/build/perf/libbpf.a] Error 2 make[2]: *** Waiting for unfinished jobs....
Because this requires something outside the tools/ directories that gets collected into perf's detached tarballs, to fix it just add it to tools/perf/MANIFEST, which this patch does, now it works for that case and also for all these other cases after doing a:
$ make -C tools clean
On a kernel sources directory:
$ make -C tools/bpf/bpftool/
make: Entering directory '/home/acme/git/perf/tools/bpf/bpftool'
Auto-detecting system features:
... libbfd: [ on ]
... disassembler-four-args: [ on ]
... zlib: [ on ]
CC map_perf_ring.o
<SNIP>
CC disasm.o
make[1]: Entering directory '/home/acme/git/perf/tools/lib/bpf'
Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]
MKDIR staticobjs/
CC staticobjs/libbpf.o
<SNIP>
LD staticobjs/libbpf-in.o
LINK libbpf.a
make[1]: Leaving directory '/home/acme/git/perf/tools/lib/bpf'
LINK bpftool
make: Leaving directory '/home/acme/git/perf/tools/bpf/bpftool' $
$ make -C tools/perf
<SNIP>
Auto-detecting system features:
... dwarf: [ on ]
... dwarf_getlocations: [ on ]
... glibc: [ on ]
... gtk2: [ on ]
... libaudit: [ on ]
... libbfd: [ on ]
... libcap: [ on ]
... libelf: [ on ]
... libnuma: [ on ]
... numa_num_possible_cpus: [ on ]
... libperl: [ on ]
... libpython: [ on ]
... libcrypto: [ on ]
... libunwind: [ on ]
... libdw-dwarf-unwind: [ on ]
... zlib: [ on ]
... lzma: [ on ]
... get_cpuid: [ on ]
... bpf: [ on ]
... libaio: [ on ]
... libzstd: [ on ]
... disassembler-four-args: [ on ]
GEN common-cmds.h
CC exec-cmd.o
<SNIP>
CC util/pmu.o
CC util/pmu-flex.o
LD util/perf-in.o
LD perf-in.o
LINK perf
make: Leaving directory '/home/acme/git/perf/tools/perf' $
$ make -C tools/lib/bpf
make: Entering directory '/home/acme/git/perf/tools/lib/bpf'
Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]
HOSTCC fixdep.o
HOSTLD fixdep-in.o
LINK fixdep
Parsed description of 117 helper function(s)
MKDIR staticobjs/
CC staticobjs/libbpf.o
CC staticobjs/bpf.o
CC staticobjs/nlattr.o
CC staticobjs/btf.o
CC staticobjs/libbpf_errno.o
CC staticobjs/str_error.o
CC staticobjs/netlink.o
CC staticobjs/bpf_prog_linfo.o
CC staticobjs/libbpf_probes.o
CC staticobjs/xsk.o
CC staticobjs/hashmap.o
CC staticobjs/btf_dump.o
LD staticobjs/libbpf-in.o
LINK libbpf.a
MKDIR sharedobjs/
CC sharedobjs/libbpf.o
CC sharedobjs/bpf.o
CC sharedobjs/nlattr.o
CC sharedobjs/btf.o
CC sharedobjs/libbpf_errno.o
CC sharedobjs/str_error.o
CC sharedobjs/netlink.o
CC sharedobjs/bpf_prog_linfo.o
CC sharedobjs/libbpf_probes.o
CC sharedobjs/xsk.o
CC sharedobjs/hashmap.o
CC sharedobjs/btf_dump.o
LD sharedobjs/libbpf-in.o
LINK libbpf.so.0.0.6
GEN libbpf.pc
LINK test_libbpf
make: Leaving directory '/home/acme/git/perf/tools/lib/bpf' $
tools/lib/bpf/Makefile +2 -2
@@ -180,9 +180,9 @@ $(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h $(BPF_IN_STATIC): force elfdep bpfdep bpf_helper_defs.h $(Q)$(MAKE) $(build)=libbpf OUTPUT=$(STATIC_OBJDIR) -bpf_helper_defs.h: $(srctree)/include/uapi/linux/bpf.h +bpf_helper_defs.h: $(srctree)/tools/include/uapi/linux/bpf.h $(Q)$(srctree)/scripts/bpf_helpers_doc.py --header \ - --file $(srctree)/include/uapi/linux/bpf.h > bpf_helper_defs.h + --file $(srctree)/tools/include/uapi/linux/bpf.h > bpf_helper_defs.h $(OUTPUT)libbpf.so: $(OUTPUT)libbpf.so.$(LIBBPF_VERSION)
tools/perf/MANIFEST +1 -0
@@ -19,3 +19,4 @@ tools/lib/bitmap.c tools/lib/str_error_r.c tools/lib/vsprintf.c tools/lib/zalloc.c +scripts/bpf_helpers_doc.py
Em Tue, Nov 26, 2019 at 12:10:45PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 6 lines
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,
This is still not enough, fails building in a container where all we have is the tarball contents, will try to fix later.
- Arnaldo
Quoted 197 lines
Thanks,
- Arnaldo
commit
94b2e22463f592d2161eb491ddb0b4659e2a91b4Author: Arnaldo Carvalho de Melo <[email protected]> Date: Tue Nov 26 11:46:08 2019 -0300libbpf: Fix up generation of
bpf_helper_defs.hBuilding perf as a detached tarball, i.e. by using one of:
$ make help | grep perf
perf-tar-src-pkg - Build perf-5.4.0.tar source tarball
perf-targz-src-pkg - Build perf-5.4.0.tar.gz source tarball perf-tarbz2-src-pkg - Build perf-5.4.0.tar.bz2 source tarball perf-tarxz-src-pkg - Build perf-5.4.0.tar.xz source tarball
$And then trying to build the resulting tarball, which is the first thing that running:
$ make -C
tools/perfbuild-testdoes, ends up with these two problems:
make[3]: *** No rule to make target '
/tmp/tmp.zq13cHILGB/perf-5.3.0/include/uapi/linux/bpf.h', needed by 'bpf_helper_defs.h'. Stop. make[3]: *** Waiting for unfinished jobs....
make[2]: *** [Makefile.perf:757:/tmp/tmp.zq13cHILGB/perf-5.3.0/tools/lib/bpf/libbpf.a] Error 2
make[2]: *** Waiting for unfinished jobs....Because $(srcdir) points to the
/tmp/tmp.zq13cHILGB/perf-5.3.0directory and we need '/tools/ after that variable, and after fixing this then we get to another problem:
/bin/sh:/home/acme/git/perf/tools/scripts/bpf_helpers_doc.py: No such file or directory make[3]: *** [Makefile:184:bpf_helper_defs.h] Error 127
make[3]: *** Deleting file 'bpf_helper_defs.h'
LD/tmp/build/perf/libapi-in.o
make[2]: *** [Makefile.perf:778:/tmp/build/perf/libbpf.a] Error 2 make[2]: *** Waiting for unfinished jobs....Because this requires something outside the
tools/directories that gets collected into perf's detached tarballs, to fix it just add it totools/perf/MANIFEST, which this patch does, now it works for that case and also for all these other cases after doing a:$ make -C tools clean
On a kernel sources directory:
$ make -C
tools/bpf/bpftool/
make: Entering directory '/home/acme/git/perf/tools/bpf/bpftool'Auto-detecting system features:
... libbfd: [ on ]
... disassembler-four-args: [ on ]
... zlib: [ on ]CC map_perf_ring.o
<SNIP>
CC disasm.o
make[1]: Entering directory '/home/acme/git/perf/tools/lib/bpf'Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]MKDIR staticobjs/
CCstaticobjs/libbpf.o
<SNIP>
LDstaticobjs/libbpf-in.o
LINK libbpf.a
make[1]: Leaving directory '/home/acme/git/perf/tools/lib/bpf'
LINK bpftool
make: Leaving directory '/home/acme/git/perf/tools/bpf/bpftool' $$ make -C
tools/perf
<SNIP>
Auto-detecting system features:
... dwarf: [ on ]
...dwarf_getlocations: [ on ]
... glibc: [ on ]
... gtk2: [ on ]
... libaudit: [ on ]
... libbfd: [ on ]
... libcap: [ on ]
... libelf: [ on ]
... libnuma: [ on ]
...numa_num_possible_cpus: [ on ]
... libperl: [ on ]
... libpython: [ on ]
... libcrypto: [ on ]
... libunwind: [ on ]
... libdw-dwarf-unwind: [ on ]
... zlib: [ on ]
... lzma: [ on ]
...get_cpuid: [ on ]
... bpf: [ on ]
... libaio: [ on ]
... libzstd: [ on ]
... disassembler-four-args: [ on ]GEN
common-cmds.h
CC exec-cmd.o
<SNIP>
CCutil/pmu.o
CCutil/pmu-flex.o
LDutil/perf-in.o
LD perf-in.o
LINK perf
make: Leaving directory '/home/acme/git/perf/tools/perf' $$ make -C
tools/lib/bpf
make: Entering directory '/home/acme/git/perf/tools/lib/bpf'Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]HOSTCC fixdep.o
HOSTLD fixdep-in.o
LINK fixdep
Parsed description of 117 helper function(s)
MKDIR staticobjs/
CCstaticobjs/libbpf.o
CCstaticobjs/bpf.o
CCstaticobjs/nlattr.o
CCstaticobjs/btf.o
CCstaticobjs/libbpf_errno.o
CCstaticobjs/str_error.o
CCstaticobjs/netlink.o
CCstaticobjs/bpf_prog_linfo.o
CCstaticobjs/libbpf_probes.o
CCstaticobjs/xsk.o
CCstaticobjs/hashmap.o
CCstaticobjs/btf_dump.o
LDstaticobjs/libbpf-in.o
LINK libbpf.a
MKDIR sharedobjs/
CCsharedobjs/libbpf.o
CCsharedobjs/bpf.o
CCsharedobjs/nlattr.o
CCsharedobjs/btf.o
CCsharedobjs/libbpf_errno.o
CCsharedobjs/str_error.o
CCsharedobjs/netlink.o
CCsharedobjs/bpf_prog_linfo.o
CCsharedobjs/libbpf_probes.o
CCsharedobjs/xsk.o
CCsharedobjs/hashmap.o
CCsharedobjs/btf_dump.o
LDsharedobjs/libbpf-in.o
LINK libbpf.so.0.0.6
GEN libbpf.pc
LINKtest_libbpf
make: Leaving directory '/home/acme/git/perf/tools/lib/bpf' $Fixes e01a75c15969 ("libbpf: Move bpf_{helpers, helper_defs, endian, tracing}.h into libbpf")Cc Adrian Hunter <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Andrii Nakryiko <[email protected]>Cc Daniel Borkmann <[email protected]>Cc Jiri Olsa <[email protected]>Cc Martin KaFai Lau <[email protected]>Cc Namhyung Kim <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>tools/lib/bpf/Makefile +2
-2@@ -180,9 +180,9 @@ $(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h $(BPF_IN_STATIC): force elfdep bpfdep bpf_helper_defs.h $(Q)$(MAKE) $(build)=libbpf OUTPUT=$(STATIC_OBJDIR) -bpf_helper_defs.h: $(srctree)/include/uapi/linux/bpf.h +bpf_helper_defs.h: $(srctree)/tools/include/uapi/linux/bpf.h $(Q)$(srctree)/scripts/bpf_helpers_doc.py --header \ - --file $(srctree)/include/uapi/linux/bpf.h > bpf_helper_defs.h + --file $(srctree)/tools/include/uapi/linux/bpf.h > bpf_helper_defs.h $(OUTPUT)libbpf.so: $(OUTPUT)libbpf.so.$(LIBBPF_VERSION)tools/perf/MANIFEST +1
-0@@ -19,3 +19,4 @@ tools/lib/bitmap.c tools/lib/str_error_r.c tools/lib/vsprintf.c tools/lib/zalloc.c +scripts/bpf_helpers_doc.py
Signature · 1 line
- Arnaldo
Quoted 10 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 12:10:45PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 6 lines
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,This is still not enough, fails building in a container where all we have is the tarball contents, will try to fix later.
Wouldn't the right thing to do not be to just run the script, and then put the generated bpf_helper_defs.h into the tarball?
-Toke
Quoted 185 lines · Arnaldo Carvalho de Melo
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,Thanks,
- Arnaldo
commit
94b2e22463f592d2161eb491ddb0b4659e2a91b4Author: Arnaldo Carvalho de Melo <[email protected]> Date: Tue Nov 26 11:46:08 2019 -0300libbpf: Fix up generation of
bpf_helper_defs.hBuilding perf as a detached tarball, i.e. by using one of:
$ make help | grep perf
perf-tar-src-pkg - Build perf-5.4.0.tar source tarball
perf-targz-src-pkg - Build perf-5.4.0.tar.gz source tarball perf-tarbz2-src-pkg - Build perf-5.4.0.tar.bz2 source tarball perf-tarxz-src-pkg - Build perf-5.4.0.tar.xz source tarball
$And then trying to build the resulting tarball, which is the first thing that running:
$ make -C
tools/perfbuild-testdoes, ends up with these two problems:
make[3]: *** No rule to make target '
/tmp/tmp.zq13cHILGB/perf-5.3.0/include/uapi/linux/bpf.h', needed by 'bpf_helper_defs.h'. Stop. make[3]: *** Waiting for unfinished jobs....
make[2]: *** [Makefile.perf:757:/tmp/tmp.zq13cHILGB/perf-5.3.0/tools/lib/bpf/libbpf.a] Error 2
make[2]: *** Waiting for unfinished jobs....Because $(srcdir) points to the
/tmp/tmp.zq13cHILGB/perf-5.3.0directory and we need '/tools/ after that variable, and after fixing this then we get to another problem:
/bin/sh:/home/acme/git/perf/tools/scripts/bpf_helpers_doc.py: No such file or directory make[3]: *** [Makefile:184:bpf_helper_defs.h] Error 127
make[3]: *** Deleting file 'bpf_helper_defs.h'
LD/tmp/build/perf/libapi-in.o
make[2]: *** [Makefile.perf:778:/tmp/build/perf/libbpf.a] Error 2 make[2]: *** Waiting for unfinished jobs....Because this requires something outside the
tools/directories that gets collected into perf's detached tarballs, to fix it just add it totools/perf/MANIFEST, which this patch does, now it works for that case and also for all these other cases after doing a:$ make -C tools clean
On a kernel sources directory:
$ make -C
tools/bpf/bpftool/
make: Entering directory '/home/acme/git/perf/tools/bpf/bpftool'Auto-detecting system features:
... libbfd: [ on ]
... disassembler-four-args: [ on ]
... zlib: [ on ]CC map_perf_ring.o
<SNIP>
CC disasm.o
make[1]: Entering directory '/home/acme/git/perf/tools/lib/bpf'Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]MKDIR staticobjs/
CCstaticobjs/libbpf.o
<SNIP>
LDstaticobjs/libbpf-in.o
LINK libbpf.a
make[1]: Leaving directory '/home/acme/git/perf/tools/lib/bpf'
LINK bpftool
make: Leaving directory '/home/acme/git/perf/tools/bpf/bpftool' $$ make -C
tools/perf
<SNIP>
Auto-detecting system features:
... dwarf: [ on ]
...dwarf_getlocations: [ on ]
... glibc: [ on ]
... gtk2: [ on ]
... libaudit: [ on ]
... libbfd: [ on ]
... libcap: [ on ]
... libelf: [ on ]
... libnuma: [ on ]
...numa_num_possible_cpus: [ on ]
... libperl: [ on ]
... libpython: [ on ]
... libcrypto: [ on ]
... libunwind: [ on ]
... libdw-dwarf-unwind: [ on ]
... zlib: [ on ]
... lzma: [ on ]
...get_cpuid: [ on ]
... bpf: [ on ]
... libaio: [ on ]
... libzstd: [ on ]
... disassembler-four-args: [ on ]GEN
common-cmds.h
CC exec-cmd.o
<SNIP>
CCutil/pmu.o
CCutil/pmu-flex.o
LDutil/perf-in.o
LD perf-in.o
LINK perf
make: Leaving directory '/home/acme/git/perf/tools/perf' $$ make -C
tools/lib/bpf
make: Entering directory '/home/acme/git/perf/tools/lib/bpf'Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]HOSTCC fixdep.o
HOSTLD fixdep-in.o
LINK fixdep
Parsed description of 117 helper function(s)
MKDIR staticobjs/
CCstaticobjs/libbpf.o
CCstaticobjs/bpf.o
CCstaticobjs/nlattr.o
CCstaticobjs/btf.o
CCstaticobjs/libbpf_errno.o
CCstaticobjs/str_error.o
CCstaticobjs/netlink.o
CCstaticobjs/bpf_prog_linfo.o
CCstaticobjs/libbpf_probes.o
CCstaticobjs/xsk.o
CCstaticobjs/hashmap.o
CCstaticobjs/btf_dump.o
LDstaticobjs/libbpf-in.o
LINK libbpf.a
MKDIR sharedobjs/
CCsharedobjs/libbpf.o
CCsharedobjs/bpf.o
CCsharedobjs/nlattr.o
CCsharedobjs/btf.o
CCsharedobjs/libbpf_errno.o
CCsharedobjs/str_error.o
CCsharedobjs/netlink.o
CCsharedobjs/bpf_prog_linfo.o
CCsharedobjs/libbpf_probes.o
CCsharedobjs/xsk.o
CCsharedobjs/hashmap.o
CCsharedobjs/btf_dump.o
LDsharedobjs/libbpf-in.o
LINK libbpf.so.0.0.6
GEN libbpf.pc
LINKtest_libbpf
make: Leaving directory '/home/acme/git/perf/tools/lib/bpf' $Fixes e01a75c15969 ("libbpf: Move bpf_{helpers, helper_defs, endian, tracing}.h into libbpf")Cc Adrian Hunter <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Andrii Nakryiko <[email protected]>Cc Daniel Borkmann <[email protected]>Cc Jiri Olsa <[email protected]>Cc Martin KaFai Lau <[email protected]>Cc Namhyung Kim <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>tools/lib/bpf/Makefile +0
-0
tools/lib/bpf/Makefile $(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h · line 185
Quoted 8 lines
tools/lib/bpf/Makefile +2
-2@@ -180,9 +180,9 @@ $(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h $(BPF_IN_STATIC): force elfdep bpfdep bpf_helper_defs.h $(Q)$(MAKE) $(build)=libbpf OUTPUT=$(STATIC_OBJDIR) -bpf_helper_defs.h: $(srctree)/include/uapi/linux/bpf.h +bpf_helper_defs.h: $(srctree)/tools/include/uapi/linux/bpf.h $(Q)$(srctree)/scripts/bpf_helpers_doc.py --header \ - --file $(srctree)/include/uapi/linux/bpf.h > bpf_helper_defs.h + --file $(srctree)/tools/include/uapi/linux/bpf.h > bpf_helper_defs.h
fwiw. this bit looks good. Makes sense to do regardless.
tools/perf/MANIFEST tools/lib/bitmap.c · line 22
Quoted 11 lines
tools/lib/bpf/Makefile +0
-0$(OUTPUT)libbpf.so: $(OUTPUT)libbpf.so.$(LIBBPF_VERSION)tools/perf/MANIFEST +1
-0@@ -19,3 +19,4 @@ tools/lib/bitmap.c tools/lib/str_error_r.c tools/lib/vsprintf.c tools/lib/zalloc.c +scripts/bpf_helpers_doc.py
This one I don't understand. I couldn't find any piece that uses this file. Some out of tree usage?
Em Tue, Nov 26, 2019 at 08:53:53AM -0800, Alexei Starovoitov escreveu:
Quoted 190 lines
Quoted 188 lines · Arnaldo Carvalho de Melo
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,Thanks,
- Arnaldo
commit
94b2e22463f592d2161eb491ddb0b4659e2a91b4Author: Arnaldo Carvalho de Melo <[email protected]> Date: Tue Nov 26 11:46:08 2019 -0300libbpf: Fix up generation of
bpf_helper_defs.hBuilding perf as a detached tarball, i.e. by using one of:
$ make help | grep perf
perf-tar-src-pkg - Build perf-5.4.0.tar source tarball
perf-targz-src-pkg - Build perf-5.4.0.tar.gz source tarball perf-tarbz2-src-pkg - Build perf-5.4.0.tar.bz2 source tarball perf-tarxz-src-pkg - Build perf-5.4.0.tar.xz source tarball
$And then trying to build the resulting tarball, which is the first thing that running:
$ make -C
tools/perfbuild-testdoes, ends up with these two problems:
make[3]: *** No rule to make target '
/tmp/tmp.zq13cHILGB/perf-5.3.0/include/uapi/linux/bpf.h', needed by 'bpf_helper_defs.h'. Stop. make[3]: *** Waiting for unfinished jobs....
make[2]: *** [Makefile.perf:757:/tmp/tmp.zq13cHILGB/perf-5.3.0/tools/lib/bpf/libbpf.a] Error 2
make[2]: *** Waiting for unfinished jobs....Because $(srcdir) points to the
/tmp/tmp.zq13cHILGB/perf-5.3.0directory and we need '/tools/ after that variable, and after fixing this then we get to another problem:
/bin/sh:/home/acme/git/perf/tools/scripts/bpf_helpers_doc.py: No such file or directory make[3]: *** [Makefile:184:bpf_helper_defs.h] Error 127
make[3]: *** Deleting file 'bpf_helper_defs.h'
LD/tmp/build/perf/libapi-in.o
make[2]: *** [Makefile.perf:778:/tmp/build/perf/libbpf.a] Error 2 make[2]: *** Waiting for unfinished jobs....Because this requires something outside the
tools/directories that gets collected into perf's detached tarballs, to fix it just add it totools/perf/MANIFEST, which this patch does, now it works for that case and also for all these other cases after doing a:$ make -C tools clean
On a kernel sources directory:
$ make -C
tools/bpf/bpftool/
make: Entering directory '/home/acme/git/perf/tools/bpf/bpftool'Auto-detecting system features:
... libbfd: [ on ]
... disassembler-four-args: [ on ]
... zlib: [ on ]CC map_perf_ring.o
<SNIP>
CC disasm.o
make[1]: Entering directory '/home/acme/git/perf/tools/lib/bpf'Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]MKDIR staticobjs/
CCstaticobjs/libbpf.o
<SNIP>
LDstaticobjs/libbpf-in.o
LINK libbpf.a
make[1]: Leaving directory '/home/acme/git/perf/tools/lib/bpf'
LINK bpftool
make: Leaving directory '/home/acme/git/perf/tools/bpf/bpftool' $$ make -C
tools/perf
<SNIP>
Auto-detecting system features:
... dwarf: [ on ]
...dwarf_getlocations: [ on ]
... glibc: [ on ]
... gtk2: [ on ]
... libaudit: [ on ]
... libbfd: [ on ]
... libcap: [ on ]
... libelf: [ on ]
... libnuma: [ on ]
...numa_num_possible_cpus: [ on ]
... libperl: [ on ]
... libpython: [ on ]
... libcrypto: [ on ]
... libunwind: [ on ]
... libdw-dwarf-unwind: [ on ]
... zlib: [ on ]
... lzma: [ on ]
...get_cpuid: [ on ]
... bpf: [ on ]
... libaio: [ on ]
... libzstd: [ on ]
... disassembler-four-args: [ on ]GEN
common-cmds.h
CC exec-cmd.o
<SNIP>
CCutil/pmu.o
CCutil/pmu-flex.o
LDutil/perf-in.o
LD perf-in.o
LINK perf
make: Leaving directory '/home/acme/git/perf/tools/perf' $$ make -C
tools/lib/bpf
make: Entering directory '/home/acme/git/perf/tools/lib/bpf'Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]HOSTCC fixdep.o
HOSTLD fixdep-in.o
LINK fixdep
Parsed description of 117 helper function(s)
MKDIR staticobjs/
CCstaticobjs/libbpf.o
CCstaticobjs/bpf.o
CCstaticobjs/nlattr.o
CCstaticobjs/btf.o
CCstaticobjs/libbpf_errno.o
CCstaticobjs/str_error.o
CCstaticobjs/netlink.o
CCstaticobjs/bpf_prog_linfo.o
CCstaticobjs/libbpf_probes.o
CCstaticobjs/xsk.o
CCstaticobjs/hashmap.o
CCstaticobjs/btf_dump.o
LDstaticobjs/libbpf-in.o
LINK libbpf.a
MKDIR sharedobjs/
CCsharedobjs/libbpf.o
CCsharedobjs/bpf.o
CCsharedobjs/nlattr.o
CCsharedobjs/btf.o
CCsharedobjs/libbpf_errno.o
CCsharedobjs/str_error.o
CCsharedobjs/netlink.o
CCsharedobjs/bpf_prog_linfo.o
CCsharedobjs/libbpf_probes.o
CCsharedobjs/xsk.o
CCsharedobjs/hashmap.o
CCsharedobjs/btf_dump.o
LDsharedobjs/libbpf-in.o
LINK libbpf.so.0.0.6
GEN libbpf.pc
LINKtest_libbpf
make: Leaving directory '/home/acme/git/perf/tools/lib/bpf' $Fixes e01a75c15969 ("libbpf: Move bpf_{helpers, helper_defs, endian, tracing}.h into libbpf")Cc Adrian Hunter <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Andrii Nakryiko <[email protected]>Cc Daniel Borkmann <[email protected]>Cc Jiri Olsa <[email protected]>Cc Martin KaFai Lau <[email protected]>Cc Namhyung Kim <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>tools/lib/bpf/Makefile +0
-0@@ -180,9 +180,9 @@ $(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h $(BPF_IN_STATIC): force elfdep bpfdep bpf_helper_defs.h $(Q)$(MAKE) $(build)=libbpf OUTPUT=$(STATIC_OBJDIR)
Quoted 3 lines
tools/lib/bpf/Makefile $(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h · line 184
Quoted 3 lines
tools/lib/bpf/Makefile +1
-1-bpf_helper_defs.h: $(srctree)/include/uapi/linux/bpf.h +bpf_helper_defs.h: $(srctree)/tools/include/uapi/linux/bpf.h $(Q)$(srctree)/scripts/bpf_helpers_doc.py --header \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Quoted 7 lines
tools/lib/bpf/Makefile $(BPF_IN_SHARED): force elfdep bpfdep bpf_helper_defs.h · line 185
Quoted 2 lines
tools/lib/bpf/Makefile +1
-1- --file $(srctree)/include/uapi/linux/bpf.h > bpf_helper_defs.h + --file $(srctree)/tools/include/uapi/linux/bpf.h > bpf_helper_defs.hfwiw. this bit looks good. Makes sense to do regardless.
Quoted 2 lines
tools/lib/bpf/Makefile +0
-0$(OUTPUT)libbpf.so: $(OUTPUT)libbpf.so.$(LIBBPF_VERSION)
Quoted 12 lines
tools/perf/MANIFEST tools/lib/bitmap.c · line 22
Quoted 9 lines
tools/perf/MANIFEST +1
-0@@ -19,3 +19,4 @@ tools/lib/bitmap.c tools/lib/str_error_r.c tools/lib/vsprintf.c tools/lib/zalloc.c +scripts/bpf_helpers_doc.pyThis one I don't understand. I couldn't find any piece that uses this file. Some out of tree usage?
See above on the part that you considered good.
First it couldn't find $(srctree)/include/uapi/linux/bpf.h when it tried to handle that bpf_helper_defs.h target, I fixed that by adding the missing /tools/ bit and then it tried to run scripts/bpf_helpers_doc.py.
The perf tarball doesn't use anything from the kernel sources (outside tools/), but since libbpf now uses something that is in the kernel top level 'scripts' directory, I have to put that script in tools/perf/MANIFEST so that it can be used when building from the tarball, detached from the kernel sources.
Or am I still missing something?
- Arnaldo
Em Tue, Nov 26, 2019 at 05:38:18PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 10 lines
Quoted 8 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 12:10:45PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 6 lines
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,
Quoted 5 lines
Quoted 2 lines
This is still not enough, fails building in a container where all we have is the tarball contents, will try to fix later.
Wouldn't the right thing to do not be to just run the script, and then put the generated
bpf_helper_defs.hinto the tarball?
I would rather continue just running tar and have the build process in-tree or outside be the same.
- Arnaldo
Quoted 11 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 05:38:18PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 10 lines
Quoted 8 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 12:10:45PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 6 lines
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
This is still not enough, fails building in a container where all we have is the tarball contents, will try to fix later.
Wouldn't the right thing to do not be to just run the script, and then put the generated
bpf_helper_defs.hinto the tarball?I would rather continue just running tar and have the build process in-tree or outside be the same.
Hmm, right. Well that Python script basically just parses include/uapi/linux/bpf.h; and it can be given the path of that file with the --filename argument. So as long as that file is present, it should be possible to make it work, I guess?
However, isn't the point of the tarball to make a "stand-alone" source distribution? I'd argue that it makes more sense to just include the generated header, then: The point of the Python script is specifically to extract the latest version of the helper definitions from the kernel source tree. And if you're "freezing" a version into a tarball, doesn't it make more sense to also freeze the list of BPF helpers?
-Toke
Em Tue, Nov 26, 2019 at 07:50:44PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 22 lines
Quoted 11 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 05:38:18PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 10 lines
Quoted 8 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 12:10:45PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 6 lines
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
This is still not enough, fails building in a container where all we have is the tarball contents, will try to fix later.
Wouldn't the right thing to do not be to just run the script, and then put the generated
bpf_helper_defs.hinto the tarball?I would rather continue just running tar and have the build process in-tree or outside be the same.
Quoted 7 lines
Hmm, right. Well that Python script basically just parses
include/uapi/linux/bpf.h; and it can be given the path of that file with the --filename argument. So as long as that file is present, it should be possible to make it work, I guess?However, isn't the point of the tarball to make a "stand-alone" source distribution?
Yes, it is, and as far as possible without any prep, just include the in-source tree files needed to build it.
Quoted 5 lines
I'd argue that it makes more sense to just include the generated header, then: The point of the Python script is specifically to extract the latest version of the helper definitions from the kernel source tree. And if you're "freezing" a version into a tarball, doesn't it make more sense to also freeze the list of BPF helpers?
Your suggestion may well even be the only solution, as older systems don't have python3, and that script requires it :-\
Some containers were showing this:
/bin/sh: 1: /git/linux/scripts/bpf_helpers_doc.py: not found Makefile:184: recipe for target 'bpf_helper_defs.h' failed make[3]: *** [bpf_helper_defs.h] Error 127
make[3]: *** Deleting file 'bpf_helper_defs.h' Makefile.perf:778: recipe for target '/tmp/build/perf/libbpf.a' failed
That "not found" doesn't mean what it looks from staring at the above, its just that:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ head -1 /tmp/perf-5.4.0/scripts/bpf_helpers_doc.py #!/usr/bin/python3 nobody@1fb841e33ba3:/tmp/perf-5.4.0$ ls -la /usr/bin/python3 ls: cannot access /usr/bin/python3: No such file or directory nobody@1fb841e33ba3:/tmp/perf-5.4.0$
So, for now, I'll keep my fix and start modifying the containers where this fails and disable testing libbpf/perf integration with BPF on those containers :-\
I.e. doing:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ make NO_LIBBPF=1 -C /tmp/perf-5.4.0/tools/perf/ O=/tmp/build/perf
which ends up with a functional perf, just one without libbpf linked in:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ /tmp/build/perf/perf -vv
perf version 5.4.gf69779ce8f86
dwarf: [ on ] # HAVE_DWARF_SUPPORT
dwarf_getlocations: [ OFF ] # HAVE_DWARF_GETLOCATIONS_SUPPORT
glibc: [ on ] # HAVE_GLIBC_SUPPORT
gtk2: [ on ] # HAVE_GTK2_SUPPORT
syscall_table: [ on ] # HAVE_SYSCALL_TABLE_SUPPORT
libbfd: [ on ] # HAVE_LIBBFD_SUPPORT
libelf: [ on ] # HAVE_LIBELF_SUPPORT
libnuma: [ OFF ] # HAVE_LIBNUMA_SUPPORT
numa_num_possible_cpus: [ OFF ] # HAVE_LIBNUMA_SUPPORT
libperl: [ on ] # HAVE_LIBPERL_SUPPORT
libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT
libslang: [ on ] # HAVE_SLANG_SUPPORT
libcrypto: [ on ] # HAVE_LIBCRYPTO_SUPPORT
libunwind: [ on ] # HAVE_LIBUNWIND_SUPPORT
libdw-dwarf-unwind: [ on ] # HAVE_DWARF_SUPPORT
zlib: [ on ] # HAVE_ZLIB_SUPPORT
lzma: [ on ] # HAVE_LZMA_SUPPORT
get_cpuid: [ on ] # HAVE_AUXTRACE_SUPPORT
bpf: [ OFF ] # HAVE_LIBBPF_SUPPORT
aio: [ on ] # HAVE_AIO_SUPPORT
zstd: [ OFF ] # HAVE_ZSTD_SUPPORT
nobody@1fb841e33ba3:/tmp/perf-5.4.0$The the build tests for libbpf and the bpf support in perf will continue, but for a reduced set of containers, those with python3.
People wanting to build libbpf on such older systems will hopefully find this discussion in google, run the script, get the output and have it working.
- Arnaldo
Quoted 61 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 07:50:44PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 22 lines
Quoted 11 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 05:38:18PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 10 lines
Quoted 8 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 12:10:45PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 6 lines
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
This is still not enough, fails building in a container where all we have is the tarball contents, will try to fix later.
Wouldn't the right thing to do not be to just run the script, and then put the generated
bpf_helper_defs.hinto the tarball?I would rather continue just running tar and have the build process in-tree or outside be the same.
Quoted 7 lines
Hmm, right. Well that Python script basically just parses
include/uapi/linux/bpf.h; and it can be given the path of that file with the --filename argument. So as long as that file is present, it should be possible to make it work, I guess?However, isn't the point of the tarball to make a "stand-alone" source distribution?
Yes, it is, and as far as possible without any prep, just include the in-source tree files needed to build it.
Quoted 5 lines
I'd argue that it makes more sense to just include the generated header, then: The point of the Python script is specifically to extract the latest version of the helper definitions from the kernel source tree. And if you're "freezing" a version into a tarball, doesn't it make more sense to also freeze the list of BPF helpers?
Your suggestion may well even be the only solution, as older systems don't have python3, and that script requires it :-\
Some containers were showing this:
/bin/sh: 1: /git/linux/scripts/bpf_helpers_doc.py: not found Makefile:184: recipe for target 'bpf_helper_defs.h' failed make[3]: *** [bpf_helper_defs.h] Error 127
make[3]: *** Deleting file 'bpf_helper_defs.h' Makefile.perf:778: recipe for target '/tmp/build/perf/libbpf.a' failedThat "not found" doesn't mean what it looks from staring at the above, its just that:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ head -1 /tmp/perf-5.4.0/scripts/bpf_helpers_doc.py #!/usr/bin/python3 nobody@1fb841e33ba3:/tmp/perf-5.4.0$ ls -la /usr/bin/python3 ls: cannot access /usr/bin/python3: No such file or directory nobody@1fb841e33ba3:/tmp/perf-5.4.0$
Quoted 3 lines
So, for now, I'll keep my fix and start modifying the containers where this fails and disable testing libbpf/perf integration with BPF on those containers :-\
I don't think there is anything Python3-specific in that script. I changed first line to
#!/usr/bin/env python
and it worked just fine. Do you mind adding this fix and make those older containers happy(-ier?).
Quoted 40 lines
I.e. doing:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ make NO_LIBBPF=1 -C /tmp/perf-5.4.0/tools/perf/ O=/tmp/build/perfwhich ends up with a functional perf, just one without libbpf linked in:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ /tmp/build/perf/perf -vv perf version 5.4.gf69779ce8f86 dwarf: [ on ] # HAVE_DWARF_SUPPORT dwarf_getlocations: [ OFF ] # HAVE_DWARF_GETLOCATIONS_SUPPORT glibc: [ on ] # HAVE_GLIBC_SUPPORT gtk2: [ on ] # HAVE_GTK2_SUPPORT syscall_table: [ on ] # HAVE_SYSCALL_TABLE_SUPPORT libbfd: [ on ] # HAVE_LIBBFD_SUPPORT libelf: [ on ] # HAVE_LIBELF_SUPPORT libnuma: [ OFF ] # HAVE_LIBNUMA_SUPPORT numa_num_possible_cpus: [ OFF ] # HAVE_LIBNUMA_SUPPORT libperl: [ on ] # HAVE_LIBPERL_SUPPORT libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT libslang: [ on ] # HAVE_SLANG_SUPPORT libcrypto: [ on ] # HAVE_LIBCRYPTO_SUPPORT libunwind: [ on ] # HAVE_LIBUNWIND_SUPPORT libdw-dwarf-unwind: [ on ] # HAVE_DWARF_SUPPORT zlib: [ on ] # HAVE_ZLIB_SUPPORT lzma: [ on ] # HAVE_LZMA_SUPPORT get_cpuid: [ on ] # HAVE_AUXTRACE_SUPPORT bpf: [ OFF ] # HAVE_LIBBPF_SUPPORT aio: [ on ] # HAVE_AIO_SUPPORT zstd: [ OFF ] # HAVE_ZSTD_SUPPORT nobody@1fb841e33ba3:/tmp/perf-5.4.0$The the build tests for libbpf and the bpf support in perf will continue, but for a reduced set of containers, those with python3.
People wanting to build libbpf on such older systems will hopefully find this discussion in google, run the script, get the output and have it working.
- Arnaldo
Em Tue, Nov 26, 2019 at 02:05:41PM -0800, Andrii Nakryiko escreveu:
Quoted 67 lines
Quoted 61 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 07:50:44PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 22 lines
Quoted 11 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 05:38:18PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 10 lines
Quoted 8 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 12:10:45PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 6 lines
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
This is still not enough, fails building in a container where all we have is the tarball contents, will try to fix later.
Wouldn't the right thing to do not be to just run the script, and then put the generated
bpf_helper_defs.hinto the tarball?I would rather continue just running tar and have the build process in-tree or outside be the same.
Quoted 7 lines
Hmm, right. Well that Python script basically just parses
include/uapi/linux/bpf.h; and it can be given the path of that file with the --filename argument. So as long as that file is present, it should be possible to make it work, I guess?However, isn't the point of the tarball to make a "stand-alone" source distribution?
Yes, it is, and as far as possible without any prep, just include the in-source tree files needed to build it.
Quoted 5 lines
I'd argue that it makes more sense to just include the generated header, then: The point of the Python script is specifically to extract the latest version of the helper definitions from the kernel source tree. And if you're "freezing" a version into a tarball, doesn't it make more sense to also freeze the list of BPF helpers?
Your suggestion may well even be the only solution, as older systems don't have python3, and that script requires it :-\
Some containers were showing this:
/bin/sh: 1: /git/linux/scripts/bpf_helpers_doc.py: not found Makefile:184: recipe for target 'bpf_helper_defs.h' failed make[3]: *** [bpf_helper_defs.h] Error 127
make[3]: *** Deleting file 'bpf_helper_defs.h' Makefile.perf:778: recipe for target '/tmp/build/perf/libbpf.a' failedThat "not found" doesn't mean what it looks from staring at the above, its just that:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ head -1 /tmp/perf-5.4.0/scripts/bpf_helpers_doc.py #!/usr/bin/python3 nobody@1fb841e33ba3:/tmp/perf-5.4.0$ ls -la /usr/bin/python3 ls: cannot access /usr/bin/python3: No such file or directory nobody@1fb841e33ba3:/tmp/perf-5.4.0$Quoted 3 lines
So, for now, I'll keep my fix and start modifying the containers where this fails and disable testing libbpf/perf integration with BPF on those containers :-\
Quoted 7 lines
I don't think there is anything Python3-specific in that script. I changed first line to
#!/usr/bin/env python
and it worked just fine. Do you mind adding this fix and make those older containers happy(-ier?).
I'll try it, was trying the other way around, i.e. adding python3 to those containers and they got happier, but fatter, so I'll remove that and try your way, thanks!
I didn't try it that way due to what comes right after the interpreter line:
#!/usr/bin/python3
# SPDX-License-Identifier: GPL-2.0-only
#
# Copyright (C) 2018-2019 Netronome Systems, Inc.
# In case user attempts to run with Python 2.
from __future__ import print_function
- Arnaldo
Quoted 40 lines
Quoted 40 lines
I.e. doing:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ make NO_LIBBPF=1 -C /tmp/perf-5.4.0/tools/perf/ O=/tmp/build/perfwhich ends up with a functional perf, just one without libbpf linked in:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ /tmp/build/perf/perf -vv perf version 5.4.gf69779ce8f86 dwarf: [ on ] # HAVE_DWARF_SUPPORT dwarf_getlocations: [ OFF ] # HAVE_DWARF_GETLOCATIONS_SUPPORT glibc: [ on ] # HAVE_GLIBC_SUPPORT gtk2: [ on ] # HAVE_GTK2_SUPPORT syscall_table: [ on ] # HAVE_SYSCALL_TABLE_SUPPORT libbfd: [ on ] # HAVE_LIBBFD_SUPPORT libelf: [ on ] # HAVE_LIBELF_SUPPORT libnuma: [ OFF ] # HAVE_LIBNUMA_SUPPORT numa_num_possible_cpus: [ OFF ] # HAVE_LIBNUMA_SUPPORT libperl: [ on ] # HAVE_LIBPERL_SUPPORT libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT libslang: [ on ] # HAVE_SLANG_SUPPORT libcrypto: [ on ] # HAVE_LIBCRYPTO_SUPPORT libunwind: [ on ] # HAVE_LIBUNWIND_SUPPORT libdw-dwarf-unwind: [ on ] # HAVE_DWARF_SUPPORT zlib: [ on ] # HAVE_ZLIB_SUPPORT lzma: [ on ] # HAVE_LZMA_SUPPORT get_cpuid: [ on ] # HAVE_AUXTRACE_SUPPORT bpf: [ OFF ] # HAVE_LIBBPF_SUPPORT aio: [ on ] # HAVE_AIO_SUPPORT zstd: [ OFF ] # HAVE_ZSTD_SUPPORT nobody@1fb841e33ba3:/tmp/perf-5.4.0$The the build tests for libbpf and the bpf support in perf will continue, but for a reduced set of containers, those with python3.
People wanting to build libbpf on such older systems will hopefully find this discussion in google, run the script, get the output and have it working.
- Arnaldo
Signature · 1 line
- Arnaldo
Em Tue, Nov 26, 2019 at 07:10:18PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 83 lines
Em Tue, Nov 26, 2019 at 02:05:41PM -0800, Andrii Nakryiko escreveu:
Quoted 67 lines
Quoted 61 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 07:50:44PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 22 lines
Quoted 11 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 05:38:18PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 10 lines
Quoted 8 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 12:10:45PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 6 lines
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
This is still not enough, fails building in a container where all we have is the tarball contents, will try to fix later.
Wouldn't the right thing to do not be to just run the script, and then put the generated
bpf_helper_defs.hinto the tarball?I would rather continue just running tar and have the build process in-tree or outside be the same.
Quoted 7 lines
Hmm, right. Well that Python script basically just parses
include/uapi/linux/bpf.h; and it can be given the path of that file with the --filename argument. So as long as that file is present, it should be possible to make it work, I guess?However, isn't the point of the tarball to make a "stand-alone" source distribution?
Yes, it is, and as far as possible without any prep, just include the in-source tree files needed to build it.
Quoted 5 lines
I'd argue that it makes more sense to just include the generated header, then: The point of the Python script is specifically to extract the latest version of the helper definitions from the kernel source tree. And if you're "freezing" a version into a tarball, doesn't it make more sense to also freeze the list of BPF helpers?
Your suggestion may well even be the only solution, as older systems don't have python3, and that script requires it :-\
Some containers were showing this:
/bin/sh: 1: /git/linux/scripts/bpf_helpers_doc.py: not found Makefile:184: recipe for target 'bpf_helper_defs.h' failed make[3]: *** [bpf_helper_defs.h] Error 127
make[3]: *** Deleting file 'bpf_helper_defs.h' Makefile.perf:778: recipe for target '/tmp/build/perf/libbpf.a' failedThat "not found" doesn't mean what it looks from staring at the above, its just that:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ head -1 /tmp/perf-5.4.0/scripts/bpf_helpers_doc.py #!/usr/bin/python3 nobody@1fb841e33ba3:/tmp/perf-5.4.0$ ls -la /usr/bin/python3 ls: cannot access /usr/bin/python3: No such file or directory nobody@1fb841e33ba3:/tmp/perf-5.4.0$Quoted 3 lines
So, for now, I'll keep my fix and start modifying the containers where this fails and disable testing libbpf/perf integration with BPF on those containers :-\
Quoted 7 lines
I don't think there is anything Python3-specific in that script. I changed first line to
#!/usr/bin/env python
and it worked just fine. Do you mind adding this fix and make those older containers happy(-ier?).
I'll try it, was trying the other way around, i.e. adding python3 to those containers and they got happier, but fatter, so I'll remove that and try your way, thanks!
I didn't try it that way due to what comes right after the interpreter line:
Quoted 7 lines
#!/usr/bin/python3
# SPDX-License-Identifier: GPL-2.0-only
#
# Copyright (C) 2018-2019 Netronome Systems, Inc.# In case user attempts to run with Python 2.
from__future__importprint_function
And that is why I think you got it working, that script uses things like:
print('Parsed description of %d helper function(s)' % len(self.helpers),
file=sys.stderr)
That python2 thinks its science fiction, what tuple is that? Can't understand, print isn't a function back then.
https://sebastianraschka.com/Articles/2014_python_2_3_key_diff.html#the-print-function
I've been adding python3 to where it is available and not yet in the container images, most are working after that, some don't need because they need other packages for BPF to work and those are not available, so nevermind, lets have just the fix I provided, I'll add python3 and life goes on.
- Arnaldo
Quoted 93 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 07:10:18PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 83 lines
Em Tue, Nov 26, 2019 at 02:05:41PM -0800, Andrii Nakryiko escreveu:
Quoted 67 lines
Quoted 61 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 07:50:44PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 22 lines
Quoted 11 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 05:38:18PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 10 lines
Quoted 8 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 12:10:45PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 6 lines
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
This is still not enough, fails building in a container where all we have is the tarball contents, will try to fix later.
Wouldn't the right thing to do not be to just run the script, and then put the generated
bpf_helper_defs.hinto the tarball?I would rather continue just running tar and have the build process in-tree or outside be the same.
Quoted 7 lines
Hmm, right. Well that Python script basically just parses
include/uapi/linux/bpf.h; and it can be given the path of that file with the --filename argument. So as long as that file is present, it should be possible to make it work, I guess?However, isn't the point of the tarball to make a "stand-alone" source distribution?
Yes, it is, and as far as possible without any prep, just include the in-source tree files needed to build it.
Quoted 5 lines
I'd argue that it makes more sense to just include the generated header, then: The point of the Python script is specifically to extract the latest version of the helper definitions from the kernel source tree. And if you're "freezing" a version into a tarball, doesn't it make more sense to also freeze the list of BPF helpers?
Your suggestion may well even be the only solution, as older systems don't have python3, and that script requires it :-\
Some containers were showing this:
/bin/sh: 1: /git/linux/scripts/bpf_helpers_doc.py: not found Makefile:184: recipe for target 'bpf_helper_defs.h' failed make[3]: *** [bpf_helper_defs.h] Error 127
make[3]: *** Deleting file 'bpf_helper_defs.h' Makefile.perf:778: recipe for target '/tmp/build/perf/libbpf.a' failedThat "not found" doesn't mean what it looks from staring at the above, its just that:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ head -1 /tmp/perf-5.4.0/scripts/bpf_helpers_doc.py #!/usr/bin/python3 nobody@1fb841e33ba3:/tmp/perf-5.4.0$ ls -la /usr/bin/python3 ls: cannot access /usr/bin/python3: No such file or directory nobody@1fb841e33ba3:/tmp/perf-5.4.0$Quoted 3 lines
So, for now, I'll keep my fix and start modifying the containers where this fails and disable testing libbpf/perf integration with BPF on those containers :-\
Quoted 7 lines
I don't think there is anything Python3-specific in that script. I changed first line to
#!/usr/bin/env python
and it worked just fine. Do you mind adding this fix and make those older containers happy(-ier?).
I'll try it, was trying the other way around, i.e. adding python3 to those containers and they got happier, but fatter, so I'll remove that and try your way, thanks!
I didn't try it that way due to what comes right after the interpreter line:
Quoted 7 lines
#!/usr/bin/python3
# SPDX-License-Identifier: GPL-2.0-only
#
# Copyright (C) 2018-2019 Netronome Systems, Inc.# In case user attempts to run with Python 2.
from__future__importprint_function
Quoted 8 lines
And that is why I think you got it working, that script uses things like:
print('Parsed description of %d helper function(s)' % len(self.helpers),
file=sys.stderr)That python2 thinks its science fiction, what tuple is that? Can't understand, print isn't a function back then.
Not a Python expert (or even regular user), but quick googling showed that this import is the way to go to use Python3 semantics of print within Python2, so seems like that's fine. But maybe Quentin has anything to say about this.
Quoted 10 lines
https://sebastianraschka.com/Articles/2014_python_2_3_key_diff.html#the-print-function
I've been adding python3 to where it is available and not yet in the container images, most are working after that, some don't need because they need other packages for BPF to work and those are not available, so nevermind, lets have just the fix I provided, I'll add python3 and life goes on.
- Arnaldo
Quoted 101 lines · Andrii Nakryiko
Quoted 99 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 07:10:18PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 83 lines
Em Tue, Nov 26, 2019 at 02:05:41PM -0800, Andrii Nakryiko escreveu:
Quoted 67 lines
Quoted 61 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 07:50:44PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 22 lines
Quoted 11 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 05:38:18PM +0100, Toke Høiland-Jørgensen escreveu:
Quoted 10 lines
Quoted 8 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 12:10:45PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 6 lines
Hi guys,
While merging perf/core with mainline I found the problem below for
which I'm adding this patch to my perf/core branch, that soon will go Ingo's way, etc. Please let me know if you think this should be handled some other way,Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
This is still not enough, fails building in a container where all we have is the tarball contents, will try to fix later.
Wouldn't the right thing to do not be to just run the script, and then put the generated
bpf_helper_defs.hinto the tarball?I would rather continue just running tar and have the build process in-tree or outside be the same.
Quoted 7 lines
Hmm, right. Well that Python script basically just parses
include/uapi/linux/bpf.h; and it can be given the path of that file with the --filename argument. So as long as that file is present, it should be possible to make it work, I guess?However, isn't the point of the tarball to make a "stand-alone" source distribution?
Yes, it is, and as far as possible without any prep, just include the in-source tree files needed to build it.
Quoted 5 lines
I'd argue that it makes more sense to just include the generated header, then: The point of the Python script is specifically to extract the latest version of the helper definitions from the kernel source tree. And if you're "freezing" a version into a tarball, doesn't it make more sense to also freeze the list of BPF helpers?
Your suggestion may well even be the only solution, as older systems don't have python3, and that script requires it :-\
Some containers were showing this:
/bin/sh: 1: /git/linux/scripts/bpf_helpers_doc.py: not found Makefile:184: recipe for target 'bpf_helper_defs.h' failed make[3]: *** [bpf_helper_defs.h] Error 127
make[3]: *** Deleting file 'bpf_helper_defs.h' Makefile.perf:778: recipe for target '/tmp/build/perf/libbpf.a' failedThat "not found" doesn't mean what it looks from staring at the above, its just that:
nobody@1fb841e33ba3:/tmp/perf-5.4.0$ head -1 /tmp/perf-5.4.0/scripts/bpf_helpers_doc.py #!/usr/bin/python3 nobody@1fb841e33ba3:/tmp/perf-5.4.0$ ls -la /usr/bin/python3 ls: cannot access /usr/bin/python3: No such file or directory nobody@1fb841e33ba3:/tmp/perf-5.4.0$Quoted 3 lines
So, for now, I'll keep my fix and start modifying the containers where this fails and disable testing libbpf/perf integration with BPF on those containers :-\
Quoted 7 lines
I don't think there is anything Python3-specific in that script. I changed first line to
#!/usr/bin/env python
and it worked just fine. Do you mind adding this fix and make those older containers happy(-ier?).
I'll try it, was trying the other way around, i.e. adding python3 to those containers and they got happier, but fatter, so I'll remove that and try your way, thanks!
I didn't try it that way due to what comes right after the interpreter line:
Quoted 7 lines
#!/usr/bin/python3
# SPDX-License-Identifier: GPL-2.0-only
#
# Copyright (C) 2018-2019 Netronome Systems, Inc.# In case user attempts to run with Python 2.
from__future__importprint_functionAnd that is why I think you got it working, that script uses things like:
print('Parsed description of %d helper function(s)' % len(self.helpers),
file=sys.stderr)
Quoted 7 lines
Quoted 2 lines
That python2 thinks its science fiction, what tuple is that? Can't understand, print isn't a function back then.
Not a Python expert (or even regular user), but quick googling showed that this import is the way to go to use Python3 semantics of print within Python2, so seems like that's fine. But maybe Quentin has anything to say about this.
We are using this script with python2.7, works just fine :-) So maybe doing s/python3/python/ is the way to go, whatever default python is installed, it should work with that.
Quoted 9 lines
Quoted 9 lines
https://sebastianraschka.com/Articles/2014_python_2_3_key_diff.html#the-print-function
I've been adding python3 to where it is available and not yet in the container images, most are working after that, some don't need because they need other packages for BPF to work and those are not available, so nevermind, lets have just the fix I provided, I'll add python3 and life goes on.
- Arnaldo
Quoted 3 lines · Stanislav Fomichev
We are using this script with python2.7, works just fine :-) So maybe doing s/python3/python/ is the way to go, whatever default python is installed, it should work with that.
That increases the risk someone will make a python2-only change and break Python 3.
Python 2 is dead, I'm honestly surprised this needs to be said :)
Em Tue, Nov 26, 2019 at 03:52:28PM -0800, Jakub Kicinski escreveu:
Quoted 9 lines · Stanislav Fomichev
Quoted 3 lines
We are using this script with python2.7, works just fine :-) So maybe doing s/python3/python/ is the way to go, whatever default python is installed, it should work with that.
That increases the risk someone will make a python2-only change and break Python 3.
Python 2 is dead, I'm honestly surprised this needs to be said :)
It shouldn't have to be said, and probably it is old school to try and keep things portable when there is no need to use new stuff for simple tasks like this.
Anyway, it seems its just a matter of adding the python3 package to the old container images and then most of them will work with what is in that script, what doesn't work is really old and then NO_LIBBPF=1 is the way to go.
In the end, kinda nothing to see here, go back to adding cool new stuff, lets not hold eBPF from progressing ;-P
- Arnaldo
PATCH 1/1 libbpf: Use PRIu64 for sym->st_value to fix build on 32-bit arches
Another fix I'm carrying in my perf/core branch,
Regards,
- Arnaldo
commit 98bb09f90a0ae33125fabc8f41529345382f1498
Author: Arnaldo Carvalho de Melo <[email protected]> Date: Wed Nov 27 09:26:54 2019 -0300
libbpf: Use PRIu64 for sym->st_value to fix build on 32-bit arches
The st_value field is a 64-bit value, so use PRIu64 to fix this error on 32-bit arches:
In file included from libbpf.c:52:
libbpf.c: In function 'bpf_program__record_reloc':
libbpf_internal.h:59:22: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'Elf64_Addr' {aka 'const long long unsigned int'} [-Werror=format=]
libbpf_print(level, "libbpf: " fmt, ##__VA_ARGS__); \
^~~~~~~~~~
libbpf_internal.h:62:27: note: in expansion of macro '__pr'
#define pr_warn(fmt, ...) __pr(LIBBPF_WARN, fmt, ##__VA_ARGS__)
^~~~
libbpf.c:1822:4: note: in expansion of macro 'pr_warn'
pr_warn("bad call relo offset: %lu\n", sym->st_value);
^~~~~~~
libbpf.c:1822:37: note: format string is defined here
pr_warn("bad call relo offset: %lu\n", sym->st_value);
~~^
%llu
tools/lib/bpf/libbpf.c +1 -1
@@ -1819,7 +1819,7 @@ static int bpf_program__record_reloc(struct bpf_program *prog,return -LIBBPF_ERRNO__RELOC ; }if (sym->st_value %8 ) { -pr_warn ("bad call relo offset: %lu\n" , sym->st_value); +pr_warn ("bad call relo offset: %" PRIu64"\n" , sym->st_value);return -LIBBPF_ERRNO__RELOC ; } reloc_desc->type =RELO_CALL ;
Re: [PATCH] libbpf: Use PRIu64 for sym->st_value to fix build on 32-bit arches
Quoted 2 lines · Arnaldo Carvalho de Melo
Another fix I'm carrying in my perf/core branch,
Why in perf/core?
I very much prefer all libbpf patches to go via normal route via bpf/net trees. We had enough conflicts in this merge window. Let's avoid them.
Re: [PATCH] libbpf: Use PRIu64 for sym->st_value to fix build on 32-bit arches
Em Wed, Nov 27, 2019 at 08:39:28AM -0800, Alexei Starovoitov escreveu:
Quoted 8 lines
Quoted 2 lines · Arnaldo Carvalho de Melo
Another fix I'm carrying in my perf/core branch,
Why in perf/core?
I very much prefer all libbpf patches to go via normal route via bpf/net trees. We had enough conflicts in this merge window. Let's avoid them.
Humm, if we both carry the same patch the merge process can do its magic and nobody gets hurt? Besides these are really minor things, no?
- Arnaldo
Re: [PATCH] libbpf: Use PRIu64 for sym->st_value to fix build on 32-bit arches
Quoted 5 lines · Arnaldo Carvalho de Melo
Em Wed, Nov 27, 2019 at 08:39:28AM -0800, Alexei Starovoitov escreveu:
Quoted 3 lines
Quoted 1 line · Arnaldo Carvalho de Melo
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Another fix I'm carrying in my perf/core branch,
Why in perf/core?
I very much prefer all libbpf patches to go via normal route via bpf/net trees. We had enough conflicts in this merge window. Let's avoid them.Humm, if we both carry the same patch the merge process can do its magic and nobody gets hurt? Besides these are really minor things, no?
I thought so too, but learned the hard lesson recently.
We should try to avoid that as much as possible.
Andrii's is fixing stuff in the same lines:
https://patchwork.ozlabs.org/patch/1201344/
these two patches will likely conflict. I'd rather have them both in bpf tree. What is the value for this patch in perf tree?
To fix the build on 32-bit arches, right?
But how urgent is it? Can you wait few days until this one and other libbpf fixes
land via bpf/net trees?
Re: [PATCH] libbpf: Use PRIu64 for sym->st_value to fix build on 32-bit arches
In reply to Arnaldo Carvalho de MeloQuoted 43 lines · Arnaldo Carvalho de Melo
Another fix I'm carrying in my perf/core branch,
Regards,
- Arnaldo
commit
98bb09f90a0ae33125fabc8f41529345382f1498Author: Arnaldo Carvalho de Melo <[email protected]> Date: Wed Nov 27 09:26:54 2019 -0300libbpf: Use PRIu64 for
sym->st_valueto fix build on 32-bit archesThe
st_valuefield is a 64-bit value, so use PRIu64 to fix this error on 32-bit arches:In file included from libbpf.c:52:
libbpf.c: In function 'bpf_program__record_reloc':
libbpf_internal.h:59:22: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'Elf64_Addr' {aka 'const long long unsigned int'} [-Werror=format=]
libbpf_print(level, "libbpf: " fmt, ##__VA_ARGS__); \
^~~~~~~~~~
libbpf_internal.h:62:27: note: in expansion of macro '__pr'
#definepr_warn(fmt, ...)__pr(LIBBPF_WARN, fmt, ##__VA_ARGS__)
^~~~
libbpf.c:1822:4: note: in expansion of macro 'pr_warn'
pr_warn("bad call relo offset: %lu\n", sym->st_value);
^~~~~~~
libbpf.c:1822:37: note: format string is defined here
pr_warn("bad call relo offset: %lu\n", sym->st_value);
~~^
%lluFixes 1f8e2bcb2cd5 ("libbpf: Refactor relocation handling")Cc Alexei Starovoitov <[email protected]>Cc Andrii Nakryiko <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>tools/lib/bpf/libbpf.c +0
-0
tools/lib/bpf/libbpf.c static int bpf_program__record_reloc(struct bpf_program *prog, · line 1822
Quoted 5 lines
tools/lib/bpf/libbpf.c +1
-1@@ -1819,7 +1819,7 @@ static int bpf_program__record_reloc(struct bpf_program *prog,return -LIBBPF_ERRNO__RELOC ; }if (sym->st_value %8 ) { -pr_warn ("bad call relo offset: %lu\n" , sym->st_value); +pr_warn ("bad call relo offset: %" PRIu64"\n" , sym->st_value);
Looking at this more... I never liked this PRI stuff. It makes for such unreadable code.
How about just typecasting st_value to (long) ?
Re: [PATCH] libbpf: Use PRIu64 for sym->st_value to fix build on 32-bit arches
In reply to Alexei StarovoitovEm Wed, Nov 27, 2019 at 10:55:31AM -0800, Alexei Starovoitov escreveu:
Quoted 16 lines
Quoted 5 lines · Arnaldo Carvalho de Melo
Em Wed, Nov 27, 2019 at 08:39:28AM -0800, Alexei Starovoitov escreveu:
Quoted 3 lines
Quoted 1 line · Arnaldo Carvalho de Melo
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Another fix I'm carrying in my perf/core branch,
Why in perf/core?
I very much prefer all libbpf patches to go via normal route via bpf/net trees. We had enough conflicts in this merge window. Let's avoid them.Humm, if we both carry the same patch the merge process can do its magic and nobody gets hurt? Besides these are really minor things, no?
Quoted 10 lines
I thought so too, but learned the hard lesson recently.
We should try to avoid that as much as possible.
Andrii's is fixing stuff in the same lines:
https://patchwork.ozlabs.org/patch/1201344/
these two patches will likely conflict. I'd rather have them both in bpf tree. What is the value for this patch in perf tree?
To fix the build on 32-bit arches, right?
But how urgent is it? Can you wait few days until this one and other libbpf fixes
land via bpf/net trees?
Ok, I'll add a note to the pull request report about where the perf build is clean in all containers because I added these two patches, but that they'll go via the bpf tree, as soon as that gets merged, the problem will go away.
And I wasn't strictly defending that I should carry this in perf/core, just said I was, to fix something minor that I found while doing my usual testing, patch was posted, you got notified and got the patch, I'll remove it from perf/core now since you stated that it'll eventually land upstream.
Thanks,
- Arnaldo
Quoted 16 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 03:52:28PM -0800, Jakub Kicinski escreveu:
Quoted 9 lines · Stanislav Fomichev
Quoted 3 lines
We are using this script with python2.7, works just fine :-) So maybe doing s/python3/python/ is the way to go, whatever default python is installed, it should work with that.
That increases the risk someone will make a python2-only change and break Python 3.
Python 2 is dead, I'm honestly surprised this needs to be said :)
It shouldn't have to be said, and probably it is old school to try and keep things portable when there is no need to use new stuff for simple tasks like this.
Quoted 7 lines
Anyway, it seems its just a matter of adding the python3 package to the old container images and then most of them will work with what is in that script, what doesn't work is really old and then NO_LIBBPF=1 is the way to go.
In the end, kinda nothing to see here, go back to adding cool new stuff, lets not hold eBPF from progressing ;-P
Absolutely. I think if some distro is still using 32-bit userland it's likely so much behind anything modern that its kernel is equally old too and appeal of new features (bpf or anything else) is probably low. So if I were you I would keep 32-bit builds of perf supported, but with minimal effort.
Re: patch itself.
I can take it as-is into bpf tree and it will be in Linus's tree in few days. Or I can take only tools/lib/bpf/Makefile hunk and you can take tools/perf/MANIFEST via perf tree?
Whichever way is fine.
Quoted 29 lines · Alexei Starovoitov
Quoted 7 lines · Arnaldo Carvalho de Melo
Em Tue, Nov 26, 2019 at 03:52:28PM -0800, Jakub Kicinski escreveu:
Quoted 4 lines · Stanislav Fomichev
Quoted 3 lines
We are using this script with python2.7, works just fine :-) So maybe doing s/python3/python/ is the way to go, whatever default python is installed, it should work with that.
Quoted 6 lines
Quoted 4 lines
That increases the risk someone will make a python2-only change and break Python 3.
Python 2 is dead, I'm honestly surprised this needs to be said :)
It shouldn't have to be said, and probably it is old school to try
and
Quoted 1 line
keep things portable when there is no need to use new stuff for
simple
Quoted 3 lines
tasks like this.
Anyway, it seems its just a matter of adding the python3 package to
the
Quoted 2 lines
old container images and then most of them will work with what is in that script, what doesn't work is really old and then NO_LIBBPF=1 is
the
Quoted 3 lines
way to go.
In the end, kinda nothing to see here, go back to adding cool new
stuff,
Quoted 8 lines
Quoted 1 line
lets not hold eBPF from progressing ;-P
Absolutely. I think if some distro is still using 32-bit userland it's likely
so much behind anything modern that its kernel is equally old too and appeal of new features (bpf or anything else) is probably low. So if I were you I would keep 32-bit builds of perf supported, but with minimal effort.
I try not to assume too much, just try to keep what's being tested to continue to at least build.
Quoted 6 lines
Re: patch itself.
I can take it as-is into bpf tree and it will be in Linus's tree in few days.
Or I can take onlytools/lib/bpf/Makefilehunk and you can taketools/perf/MANIFESTvia perf tree?
Whichever way is fine.
Take it as one, I think it's what should have been in the cset it is fixing, that way no breakage would have happened.
- Arnaldo
Quoted 2 lines · Arnaldo Carvalho de Melo
Take it as one, I think it's what should have been in the cset it is fixing, that way no breakage would have happened.
Ok. I trimmed commit log and applied here:
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/commit/?id=1fd450f99272791df8ea8e1b0f5657678e118e90
What about your other fix and my suggestion there? (__u64) cast instead of PRI ?
We do this already in two places:
libbpf.c: shdr_idx, (__u64)sym->st_value); libbpf.c: (__u64)sym.st_value, GELF_ST_TYPE(sym.st_info),
Quoted 6 lines · Alexei Starovoitov
Quoted 2 lines · Arnaldo Carvalho de Melo
Take it as one, I think it's what should have been in the cset it is
fixing, that way no breakage would have happened.
Quoted 8 lines
Ok. I trimmed commit log and applied here:
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/commit/?id=1fd450f99272791df8ea8e1b0f5657678e118e90What about your other fix and my suggestion there? (
__u64) cast instead of PRI ?
We do this already in two places:libbpf.c: shdr_idx, (__u64)sym->st_value); libbpf.c: (__u64)sym.st_value, GELF_ST_TYPE(sym.st_info),
I'm using the smartphone now, but I thought you first suggested using a cast to long, if you mean using %llu + cast to __u64, then should be was ugly as using PRI, IOW, should work on both 64 bit and 32 bit. :-)
- Arnaldo
Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 9 lines · Alexei Starovoitov
Quoted 2 lines · Arnaldo Carvalho de Melo
Take it as one, I think it's what should have been in the cset it is
fixing, that way no breakage would have happened.
Ok. I trimmed commit log and applied here:
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/commit/?id=1fd450f99272791df8ea8e1b0f5657678e118e90
Quoted 8 lines
Quoted 5 lines
What about your other fix and my suggestion there? (
__u64) cast instead of PRI ?
We do this already in two places:libbpf.c: shdr_idx, (__u64)sym->st_value); libbpf.c: (__u64)sym.st_value, GELF_ST_TYPE(sym.st_info),I'm using the smartphone now, but I thought you first suggested using a cast to long, if you mean using %llu + cast to
__u64, then should be was ugly as using PRI, IOW, should work on both 64 bit and 32 bit. :-)
Yes. I suggested (long) first, but then found two cases in libbpf that solve this with (__u64),
so better to stick to that for consistency.
Quoted 24 lines · Alexei Starovoitov
Quoted 2 lines · Arnaldo Carvalho de Melo
On November 27, 2019 9:59:15 PM GMT-03:00, Alexei Starovoitov
Quoted 4 lines · <[email protected]> wrote:
Quoted 4 lines
Quoted 2 lines · Arnaldo Carvalho de Melo
Take it as one, I think it's what should have been in the cset it
is
Quoted 11 lines
Quoted 3 lines
fixing, that way no breakage would have happened.
Ok. I trimmed commit log and applied here:
Quoted 6 lines
What about your other fix and my suggestion there? (
__u64) cast instead of PRI ?
We do this already in two places:libbpf.c: shdr_idx, (__u64)sym->st_value); libbpf.c: (__u64)sym.st_value,
GELF_ST_TYPE(sym.st_info),Quoted 2 lines
Quoted 7 lines
Quoted 1 line
I'm using the smartphone now, but I thought you first suggested using
a cast to long, if you mean using %llu + cast to
__u64, then should be was ugly as using PRI, IOW, should work on both 64 bit and 32 bit. :-)Yes. I suggested (long) first, but then found two cases in libbpf that solve this with (
__u64),
so better to stick to that for consistency.
If it's already being used elsewhere in lubbpf, it was tested already with the build containers and since nobody complained, go with it :-)
- Arnaldo
Quoted 27 lines · Arnaldo Carvalho de Melo
Quoted 25 lines · Alexei Starovoitov
Quoted 2 lines · Arnaldo Carvalho de Melo
On November 27, 2019 9:59:15 PM GMT-03:00, Alexei Starovoitov
Quoted 4 lines · <[email protected]> wrote:
Quoted 4 lines
Quoted 2 lines · Arnaldo Carvalho de Melo
Take it as one, I think it's what should have been in the cset it
is
Quoted 11 lines
Quoted 3 lines
fixing, that way no breakage would have happened.
Ok. I trimmed commit log and applied here:
Quoted 6 lines
What about your other fix and my suggestion there? (
__u64) cast instead of PRI ?
We do this already in two places:libbpf.c: shdr_idx, (__u64)sym->st_value); libbpf.c: (__u64)sym.st_value,
GELF_ST_TYPE(sym.st_info),Quoted 3 lines
I'm using the smartphone now, but I thought you first suggested using
Quoted 8 lines
Quoted 6 lines
a cast to long, if you mean using %llu + cast to
__u64, then should be was ugly as using PRI, IOW, should work on both 64 bit and 32 bit. :-)Yes. I suggested (long) first, but then found two cases in libbpf that solve this with (
__u64),
so better to stick to that for consistency.If it's already being used elsewhere in lubbpf, it was tested already with the build containers and since nobody complained, go with it :-)
Ok.
Pushed this fix:
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/commit/?id=7c3977d1e80401b1a25efded698b05d60ee26e31
Likely will send PR for bpf tree to Dave tomorrow.
Re: [PATCH] libbpf: Use PRIu64 for sym->st_value to fix build on 32-bit arches
In reply to Arnaldo Carvalho de MeloHi Arnaldo,
FYI,
Quoted 23 lines · Arnaldo Carvalho de Melo
Another fix I'm carrying in my perf/core branch,
Regards,
- Arnaldo
commit
98bb09f90a0ae33125fabc8f41529345382f1498Author: Arnaldo Carvalho de Melo <[email protected]> Date: Wed Nov 27 09:26:54 2019 -0300libbpf: Use PRIu64 for sym->st_value to fix build on 32-bit arches The st_value field is a 64-bit value, so use PRIu64 to fix this error on 32-bit arches: In file included from libbpf.c:52: libbpf.c: In function 'bpf_program__record_reloc': libbpf_internal.h:59:22: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'Elf64_Addr' {aka 'const long long unsigned int'} [-Werror=format=] libbpf_print(level, "libbpf: " fmt, ##__VA_ARGS__); \ ^~~~~~~~~~ libbpf_internal.h:62:27: note: in expansion of macro '__pr' #define pr_warn(fmt, ...) __pr(LIBBPF_WARN, fmt, ##__VA_ARGS__)
Quoted 8 lines
^~~~
libbpf.c:1822:4: note: in expansion of macro 'pr_warn'
pr_warn("bad call relo offset: %lu\n", sym->st_value);
^~~~~~~
libbpf.c:1822:37: note: format string is defined here
pr_warn("bad call relo offset: %lu\n", sym->st_value);
~~^
%lluThis build error is been noticed on Linux mainline kernel for 32-bit architectures from Nov 26.
Full build log,
https://ci.linaro.org/job/openembedded-lkft-linux-mainline/DISTRO=lkft,MACHINE=intel-core2-32,label=docker-lkft/2297/consoleText
https://ci.linaro.org/job/openembedded-lkft-linux-mainline/
- Naresh
Re: [PATCH] libbpf: Use PRIu64 for sym->st_value to fix build on 32-bit arches
Em Tue, Dec 03, 2019 at 07:20:08PM +0530, Naresh Kamboju escreveu:
Quoted 38 lines
Hi Arnaldo,
FYI,
Quoted 23 lines · Arnaldo Carvalho de Melo
Another fix I'm carrying in my perf/core branch,
Regards,
- Arnaldo
commit
98bb09f90a0ae33125fabc8f41529345382f1498Author: Arnaldo Carvalho de Melo <[email protected]> Date: Wed Nov 27 09:26:54 2019 -0300libbpf: Use PRIu64 for sym->st_value to fix build on 32-bit arches The st_value field is a 64-bit value, so use PRIu64 to fix this error on 32-bit arches: In file included from libbpf.c:52: libbpf.c: In function 'bpf_program__record_reloc': libbpf_internal.h:59:22: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'Elf64_Addr' {aka 'const long long unsigned int'} [-Werror=format=] libbpf_print(level, "libbpf: " fmt, ##__VA_ARGS__); \ ^~~~~~~~~~ libbpf_internal.h:62:27: note: in expansion of macro '__pr' #define pr_warn(fmt, ...) __pr(LIBBPF_WARN, fmt, ##__VA_ARGS__)Quoted 8 lines
^~~~ libbpf.c:1822:4: note: in expansion of macro 'pr_warn' pr_warn("bad call relo offset: %lu\n", sym->st_value); ^~~~~~~ libbpf.c:1822:37: note: format string is defined here pr_warn("bad call relo offset: %lu\n", sym->st_value); ~~^ %llu
Quoted 2 lines
This build error is been noticed on Linux mainline kernel for 32-bit architectures from Nov 26.
Right, the fix is in the bpf tree:
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/commit/?id=7c3977d1e804
Should go upstream soon.
- Arnaldo
Quoted 5 lines
Full build log,
https://ci.linaro.org/job/openembedded-lkft-linux-mainline/DISTRO=lkft,MACHINE=intel-core2-32,label=docker-lkft/2297/consoleText
https://ci.linaro.org/job/openembedded-lkft-linux-mainline/- Naresh
Signature · 1 line
- Arnaldo