libbpf: Sync with latest libbpf repo
Subsystems (2)
PATCH v2 1/2 libbpf: Sync with latest libbpf repo
Sync up to commit 87dff0a2c775 (vmtest: allow building foreign debian rootfs).
--- lib/bpf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/bpf +1 -1
@@ -1 +1 @@ -Subproject commit 393a058d061d49d5c3055fa9eefafb4c0c31ccc3 +Subproject commit 87dff0a2c775c5943ca9233e69c81a25f2ed1a77
Signature · 1 line
2.30.2
PATCH v2 2/2 btf_encoder: Normalize array index type for parallel dwarf loading case
With latest llvm15 built kernel (make -j LLVM=1), I hit the following error when build selftests (make -C tools/testing/selftests/bpf -j LLVM=1):
In file included from skeleton/pid_iter.bpf.c:3:
.../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name
'__builtin_va_list___2'; did you mean '__builtin_va_list'?
typedef __builtin_va_list___2 va_list___2;
^~~~~~~~~~~~~~~~~~~~~
__builtin_va_list
note: '__builtin_va_list' declared here
In file included from skeleton/profiler.bpf.c:3:
.../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name
'__builtin_va_list__ _2'; did you mean '__builtin_va_list'?
typedef __builtin_va_list___2 va_list___2;
^~~~~~~~~~~~~~~~~~~~~
__builtin_va_list note: '__builtin_va_list' declared here
The error can be easily explained with after-dedup vmlinux btf:
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [2300] STRUCT '__va_list_tag' size=24 vlen=4
'gp_offset' type_id=2 bits_offset=0
'fp_offset' type_id=2 bits_offset=32
'overflow_arg_area' type_id=32 bits_offset=64
'reg_save_area' type_id=32 bits_offset=128 [2308] TYPEDEF 'va_list' type_id=2309
[2309] TYPEDEF '__builtin_va_list' type_id=2310 [2310] ARRAY '(anon)' type_id=2300 index_type_id=21 nr_elems=1
[5289] PTR '(anon)' type_id=2308
[158520] STRUCT 'warn_args' size=32 vlen=2
'fmt' type_id=14 bits_offset=0
'args' type_id=2308 bits_offset=64 [27299] INT '__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32 encoding=(none) [34590] TYPEDEF '__builtin_va_list' type_id=34591
[34591] ARRAY '(anon)' type_id=2300 index_type_id=27299 nr_elems=1
Note that two array index_type_id's are different so the va_list and __builtin_va_list will have two versions in the BTF. With this, vmlinux.h contains the following code,
typedef __builtin_va_list va_list;
typedef __builtin_va_list___2 va_list___2;
Since __builtin_va_list is a builtin type for the compiler,
libbpf does not generate
typedef <...> __builtin_va_list
and this caused __builtin_va_list___2 is not defined and hence compilation error. This happened when pahole is running with more than one jobs when parsing dwarf and generating btfs.
Function btf_encoder__encode_cu() is used to do btf encoding for each cu. The function will try to find an "int" type for the cu if it is available, otherwise, it will create a special type with name __ARRAY_SIZE_TYPE__. For example,
file1: yes 'int' type
file2: no 'int' type
In serial mode, file1 is processed first, followed by file2. both will have 'int' type as the array index type since file2 will inherit the index type from file1.
In parallel mode though, arrays in file1 will have index type 'int', and arrays in file2 wil have index type '__ARRAY_SIZE_TYPE__'. This will prevent some legitimate dedup and may have generated vmlinux.h having compilation error.
This patch fixed the issue by creating an 'int' type as the array index type, so all array index type should be the same for all cu's even in parallel mode.
--- btf_encoder.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Changelog:
v1 -> v2:
- change creation of array index type to be 'int' type, the same as the type encoder tries to search in the current types.
btf_encoder.c +2 -1
@@ -1460,7 +1460,8 @@ int btf_encoder__encode_cu(struct btf_encoder *encoder, struct cu *cu) bt.name =0 ; bt.bit_size =32 ; -btf_encoder__add_base_type (encoder, &bt,"__ARRAY_SIZE_TYPE__" ); + bt.is_signed = true; +btf_encoder__add_base_type (encoder, &bt,"int" ); encoder->has_index_type = true; }
Signature · 1 line
2.30.2
Re: [PATCH dwarves v2 2/2] btf_encoder: Normalize array index type for parallel dwarf loading case
Quoted 64 lines · Yonghong Song
With latest llvm15 built kernel (make -j LLVM=1), I hit the following error when build selftests (make -C
tools/testing/selftests/bpf-j LLVM=1):
In file included fromskeleton/pid_iter.bpf.c:3:
.../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name'__builtin_va_list___2'; did you mean '__builtin_va_list'? typedef __builtin_va_list___2 va_list___2; ^~~~~~~~~~~~~~~~~~~~~ __builtin_va_list note: '__builtin_va_list' declared here In file included from skeleton/profiler.bpf.c:3: .../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name '__builtin_va_list__ _2'; did you mean '__builtin_va_list'? typedef __builtin_va_list___2 va_list___2; ^~~~~~~~~~~~~~~~~~~~~ __builtin_va_listnote: '
__builtin_va_list' declared hereThe error can be easily explained with after-dedup vmlinux btf:
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [2300] STRUCT '__va_list_tag' size=24 vlen=4'gp_offset' type_id=2 bits_offset=0 'fp_offset' type_id=2 bits_offset=32 'overflow_arg_area' type_id=32 bits_offset=64 'reg_save_area' type_id=32 bits_offset=128[2308] TYPEDEF '
va_list' type_id=2309
[2309] TYPEDEF '__builtin_va_list' type_id=2310 [2310] ARRAY '(anon)' type_id=2300 index_type_id=21 nr_elems=1[5289] PTR '(anon)' type_id=2308
[158520] STRUCT 'warn_args' size=32 vlen=2'fmt' type_id=14 bits_offset=0 'args' type_id=2308 bits_offset=64[27299] INT '
__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32 encoding=(none) [34590] TYPEDEF '__builtin_va_list' type_id=34591
[34591] ARRAY '(anon)' type_id=2300 index_type_id=27299 nr_elems=1Note that two array index_type_id's are different so the
va_listand__builtin_va_listwill have two versions in the BTF. With this,vmlinux.hcontains the following code,
typedef__builtin_va_listva_list;
typedef__builtin_va_list___2va_list___2;
Since__builtin_va_listis a builtin type for the compiler,
libbpf does not generate
typedef <...>__builtin_va_list
and this caused__builtin_va_list___2is not defined and hence compilation error. This happened when pahole is running with more than one jobs when parsing dwarf and generating btfs.Function
btf_encoder__encode_cu()is used to do btf encoding for each cu. The function will try to find an "int" type for the cu if it is available, otherwise, it will create a special type with name__ARRAY_SIZE_TYPE__. For example,
file1: yes 'int' type
file2: no 'int' typeIn serial mode, file1 is processed first, followed by file2. both will have 'int' type as the array index type since file2 will inherit the index type from file1.
In parallel mode though, arrays in file1 will have index type 'int', and arrays in file2 wil have index type '
__ARRAY_SIZE_TYPE__'. This will prevent some legitimate dedup and may have generatedvmlinux.hhaving compilation error.
Quoted 9 lines
This patch fixed the issue by creating an 'int' type as the array index type, so all array index type should be the same for all cu's even in parallel mode.
Signed-off-by Yonghong Song <[email protected]>--- btf_encoder.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
LGTM, it should work reliably.
Quoted 23 lines
Changelog:
v1 -> v2:
- change creation of array index type to be 'int' type, the same as the type encoder tries to search in the current types.btf_encoder.c +2
-2@@ -1460,7 +1460,8 @@ int btf_encoder__encode_cu(struct btf_encoder *encoder, struct cu *cu) bt.name =0 ; bt.bit_size =32 ; -btf_encoder__add_base_type (encoder, &bt,"__ARRAY_SIZE_TYPE__" ); + bt.is_signed = true; +btf_encoder__add_base_type (encoder, &bt,"int" ); encoder->has_index_type = true; } --2.30.2
Re: [PATCH dwarves v2 2/2] btf_encoder: Normalize array index type for parallel dwarf loading case
Em Thu, May 12, 2022 at 03:55:14PM -0700, Andrii Nakryiko escreveu:
Quoted 75 lines · Yonghong Song
Quoted 64 lines
With latest llvm15 built kernel (make -j LLVM=1), I hit the following error when build selftests (make -C
tools/testing/selftests/bpf-j LLVM=1):
In file included fromskeleton/pid_iter.bpf.c:3:
.../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name'__builtin_va_list___2'; did you mean '__builtin_va_list'? typedef __builtin_va_list___2 va_list___2; ^~~~~~~~~~~~~~~~~~~~~ __builtin_va_list note: '__builtin_va_list' declared here In file included from skeleton/profiler.bpf.c:3: .../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name '__builtin_va_list__ _2'; did you mean '__builtin_va_list'? typedef __builtin_va_list___2 va_list___2; ^~~~~~~~~~~~~~~~~~~~~ __builtin_va_listnote: '
__builtin_va_list' declared hereThe error can be easily explained with after-dedup vmlinux btf:
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [2300] STRUCT '__va_list_tag' size=24 vlen=4'gp_offset' type_id=2 bits_offset=0 'fp_offset' type_id=2 bits_offset=32 'overflow_arg_area' type_id=32 bits_offset=64 'reg_save_area' type_id=32 bits_offset=128[2308] TYPEDEF '
va_list' type_id=2309
[2309] TYPEDEF '__builtin_va_list' type_id=2310 [2310] ARRAY '(anon)' type_id=2300 index_type_id=21 nr_elems=1[5289] PTR '(anon)' type_id=2308
[158520] STRUCT 'warn_args' size=32 vlen=2'fmt' type_id=14 bits_offset=0 'args' type_id=2308 bits_offset=64[27299] INT '
__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32 encoding=(none) [34590] TYPEDEF '__builtin_va_list' type_id=34591
[34591] ARRAY '(anon)' type_id=2300 index_type_id=27299 nr_elems=1Note that two array index_type_id's are different so the
va_listand__builtin_va_listwill have two versions in the BTF. With this,vmlinux.hcontains the following code,
typedef__builtin_va_listva_list;
typedef__builtin_va_list___2va_list___2;
Since__builtin_va_listis a builtin type for the compiler,
libbpf does not generate
typedef <...>__builtin_va_list
and this caused__builtin_va_list___2is not defined and hence compilation error. This happened when pahole is running with more than one jobs when parsing dwarf and generating btfs.Function
btf_encoder__encode_cu()is used to do btf encoding for each cu. The function will try to find an "int" type for the cu if it is available, otherwise, it will create a special type with name__ARRAY_SIZE_TYPE__. For example,
file1: yes 'int' type
file2: no 'int' typeIn serial mode, file1 is processed first, followed by file2. both will have 'int' type as the array index type since file2 will inherit the index type from file1.
In parallel mode though, arrays in file1 will have index type 'int', and arrays in file2 wil have index type '
__ARRAY_SIZE_TYPE__'. This will prevent some legitimate dedup and may have generatedvmlinux.hhaving compilation error.Quoted 9 lines
This patch fixed the issue by creating an 'int' type as the array index type, so all array index type should be the same for all cu's even in parallel mode.
Signed-off-by Yonghong Song <[email protected]>--- btf_encoder.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Quoted 3 lines
LGTM, it should work reliably.
Acked-by Andrii Nakryiko <[email protected]>
Applied and testing.
- Arnaldo
Quoted 23 lines
Quoted 23 lines
Changelog:
v1 -> v2:
- change creation of array index type to be 'int' type, the same as the type encoder tries to search in the current types.btf_encoder.c +2
-2@@ -1460,7 +1460,8 @@ int btf_encoder__encode_cu(struct btf_encoder *encoder, struct cu *cu) bt.name =0 ; bt.bit_size =32 ; -btf_encoder__add_base_type (encoder, &bt,"__ARRAY_SIZE_TYPE__" ); + bt.is_signed = true; +btf_encoder__add_base_type (encoder, &bt,"int" ); encoder->has_index_type = true; } --2.30.2
Signature · 1 line
- Arnaldo
Re: [PATCH dwarves v2 2/2] btf_encoder: Normalize array index type for parallel dwarf loading case
Quoted 76 lines · Arnaldo Carvalho de Melo
Em Thu, May 12, 2022 at 03:55:14PM -0700, Andrii Nakryiko escreveu:
Quoted 75 lines · Yonghong Song
Quoted 73 lines
With latest llvm15 built kernel (make -j LLVM=1), I hit the following error when build selftests (make -C
tools/testing/selftests/bpf-j LLVM=1):
In file included fromskeleton/pid_iter.bpf.c:3:.../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name '__builtin_va_list___2'; did you mean '__builtin_va_list'? typedef __builtin_va_list___2 va_list___2; ^~~~~~~~~~~~~~~~~~~~~ __builtin_va_list note: '__builtin_va_list' declared here In file included from skeleton/profiler.bpf.c:3: .../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name '__builtin_va_list__ _2'; did you mean '__builtin_va_list'? typedef __builtin_va_list___2 va_list___2; ^~~~~~~~~~~~~~~~~~~~~ __builtin_va_list note: '__builtin_va_list' declared hereThe error can be easily explained with after-dedup vmlinux btf:
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [2300] STRUCT '__va_list_tag' size=24 vlen=4 'gp_offset' type_id=2 bits_offset=0 'fp_offset' type_id=2 bits_offset=32 'overflow_arg_area' type_id=32 bits_offset=64 'reg_save_area' type_id=32 bits_offset=128 [2308] TYPEDEF 'va_list' type_id=2309 [2309] TYPEDEF '__builtin_va_list' type_id=2310 [2310] ARRAY '(anon)' type_id=2300 index_type_id=21 nr_elems=1 [5289] PTR '(anon)' type_id=2308 [158520] STRUCT 'warn_args' size=32 vlen=2 'fmt' type_id=14 bits_offset=0 'args' type_id=2308 bits_offset=64 [27299] INT '__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32 encoding=(none) [34590] TYPEDEF '__builtin_va_list' type_id=34591 [34591] ARRAY '(anon)' type_id=2300 index_type_id=27299 nr_elems=1Note that two array index_type_id's are different so the
va_listand__builtin_va_listwill have two versions in the BTF. With this,vmlinux.hcontains the following code,typedef __builtin_va_list va_list; typedef __builtin_va_list___2 va_list___2;Since
__builtin_va_listis a builtin type for the compiler,
libbpf does not generate
typedef <...>__builtin_va_list
and this caused__builtin_va_list___2is not defined and hence compilation error. This happened when pahole is running with more than one jobs when parsing dwarf and generating btfs.Function
btf_encoder__encode_cu()is used to do btf encoding for each cu. The function will try to find an "int" type for the cu if it is available, otherwise, it will create a special type with name__ARRAY_SIZE_TYPE__. For example,
file1: yes 'int' type
file2: no 'int' typeIn serial mode, file1 is processed first, followed by file2. both will have 'int' type as the array index type since file2 will inherit the index type from file1.
In parallel mode though, arrays in file1 will have index type 'int', and arrays in file2 wil have index type '
__ARRAY_SIZE_TYPE__'. This will prevent some legitimate dedup and may have generatedvmlinux.hhaving compilation error.This patch fixed the issue by creating an 'int' type as the array index type, so all array index type should be the same for all cu's even in parallel mode.
Signed-off-by Yonghong Song <[email protected]>--- btf_encoder.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Quoted 5 lines
Quoted 3 lines
LGTM, it should work reliably.
Acked-by Andrii Nakryiko <[email protected]>Applied and testing.
Thanks!
Quoted 27 lines
- Arnaldo
Quoted 23 lines
Quoted 23 lines
Changelog:
v1 -> v2:
- change creation of array index type to be 'int' type, the same as the type encoder tries to search in the current types.btf_encoder.c +2
-2@@ -1460,7 +1460,8 @@ int btf_encoder__encode_cu(struct btf_encoder *encoder, struct cu *cu) bt.name =0 ; bt.bit_size =32 ; -btf_encoder__add_base_type (encoder, &bt,"__ARRAY_SIZE_TYPE__" ); + bt.is_signed = true; +btf_encoder__add_base_type (encoder, &bt,"int" ); encoder->has_index_type = true; } --2.30.2