btf_encoder: Filter out __gendwarfksyms_ptr_
PATCH 1/1 btf_encoder: Filter out __gendwarfksyms_ptr_
With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel in EXPORT_SYMBOL() to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit 9810758003ce ("btf_encoder: Verify 0 address DWARF variables are in ELF section")).
Filter out symbols with the __gendwarfksyms_ptr_ name prefix in filter_variable_name() instead of calling variable_in_sec() for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:
Before: 35.775 +- 0.121 seconds time elapsed ( +- 0.34% ) After: 8.3516 +- 0.0407 seconds time elapsed ( +- 0.49% )
--- btf_encoder.c | 1 + 1 file changed, 1 insertion(+)
btf_encoder.c +1 -0
@@ -2239,6 +2239,7 @@ static bool filter_variable_name(const char *name)X ("__UNIQUE_ID" ),X ("__tpstrtab_" ),X ("__exitcall_" ), +X ("__gendwarfksyms_ptr_" ), X("__func_stack_frame_non_standard_")#undef X };
base-commit: a0be596ae76c720d21eef257dec1cf2462130da1
Signature · 1 line
2.49.0.rc1.451.g8f38331e32-goog
Quoted 10 lines · Sami Tolvanen
With
CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel inEXPORT_SYMBOL()to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit9810758003ce("btf_encoder: Verify 0 address DWARF variables are in ELF section")).
Quoted 7 lines
Filter out symbols with the
__gendwarfksyms_ptr_name prefix infilter_variable_name()instead of callingvariable_in_sec()for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:Before: 35.775 +- 0.121 seconds time elapsed ( +- 0.34% ) After: 8.3516 +- 0.0407 seconds time elapsed ( +- 0.49% )
makes sense to me, I just can't reproduce the speedup could you please share your .config?
thanks,
jirka
Quoted 23 lines
Signed-off-by Sami Tolvanen <[email protected]>--- btf_encoder.c | 1 + 1 file changed, 1 insertion(+)btf_encoder.c +1
-0@@ -2239,6 +2239,7 @@ static bool filter_variable_name(const char *name)X ("__UNIQUE_ID" ),X ("__tpstrtab_" ),X ("__exitcall_" ), +X ("__gendwarfksyms_ptr_" ), X("__func_stack_frame_non_standard_")#undef X };base-commit:
a0be596ae76c720d21eef257dec1cf2462130da1Signature · 1 line
2.49.0.rc1.451.g8f38331e32-goog
Hi Jiri,
Quoted 17 lines · Jiri Olsa
Quoted 15 lines · Sami Tolvanen
With
CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel inEXPORT_SYMBOL()to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit9810758003ce("btf_encoder: Verify 0 address DWARF variables are in ELF section")).Filter out symbols with the
__gendwarfksyms_ptr_name prefix infilter_variable_name()instead of callingvariable_in_sec()for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:
Quoted 5 lines
Quoted 2 lines
Before: 35.775 +- 0.121 seconds time elapsed ( +- 0.34% ) After: 8.3516 +- 0.0407 seconds time elapsed ( +- 0.49% )makes sense to me, I just can't reproduce the speedup could you please share your .config?
Sure, here's the config I used to repro this:
https://gist.github.com/samitolvanen/dca66a1a779861be27579f88c9b6ba5d
This is essentially x86_64 defconfig with GENDWARFKSYMS and DEBUG_INFO_BTF both enabled. When building this config with gcc, we end up with 0 address __gendwarfksyms_ptr variables in DWARF:
...0x0001b5c6: DW_TAG_variable
DW_AT_name ("__gendwarfksyms_ptr_system_state")
DW_AT_decl_file ("../init/main.c")
DW_AT_decl_line (129)
DW_AT_decl_column (1)
DW_AT_type (0x0001b5dc "system_states *")
DW_AT_location (DW_OP_addr 0x0)...
Note that this doesn't seem to happen when building with Clang.
Before commit 9810758003ce this resulted in pahole thinking all these variables are in the .data..percpu section, which resulted in btf_datasec_check_meta() failing with "Invalid offset" during boot. pahole/next doesn't have this issue, but validating the 0 address variables is unfortunately a bit slow when we have a lot of them.
Sami
Quoted 44 lines · Sami Tolvanen
Hi Jiri,
Quoted 17 lines · Jiri Olsa
Quoted 15 lines · Sami Tolvanen
With
CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel inEXPORT_SYMBOL()to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit9810758003ce("btf_encoder: Verify 0 address DWARF variables are in ELF section")).Filter out symbols with the
__gendwarfksyms_ptr_name prefix infilter_variable_name()instead of callingvariable_in_sec()for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:Quoted 5 lines
Quoted 2 lines
Before: 35.775 +- 0.121 seconds time elapsed ( +- 0.34% ) After: 8.3516 +- 0.0407 seconds time elapsed ( +- 0.49% )makes sense to me, I just can't reproduce the speedup could you please share your .config?
Sure, here's the config I used to repro this:
https://gist.github.com/samitolvanen/dca66a1a779861be27579f88c9b6ba5d
This is essentially
x86_64defconfig with GENDWARFKSYMS andDEBUG_INFO_BTFboth enabled. When building this config with gcc, we end up with 0 address__gendwarfksyms_ptrvariables in DWARF:...
0x0001b5c6:DW_TAG_variableDW_AT_name ("__gendwarfksyms_ptr_system_state") DW_AT_decl_file ("../init/main.c") DW_AT_decl_line (129) DW_AT_decl_column (1) DW_AT_type (0x0001b5dc "system_states *") DW_AT_location (DW_OP_addr 0x0)...
Quoted 8 lines
Note that this doesn't seem to happen when building with Clang.
Before commit
9810758003cethis resulted in pahole thinking all these variables are in the .data..percpu section, which resulted inbtf_datasec_check_meta()failing with "Invalid offset" during boot. pahole/next doesn't have this issue, but validating the 0 address variables is unfortunately a bit slow when we have a lot of them.
Thanks for the fix Sami! I've tested it at my end and can reproduce the longer time for BTF encoding on x86_64 prior to the fix and its resolution. Let's wait a bit longer before landing it to see if anyone else gets a chance to test/ack it, but I think we should probably also add a
are in ELF section")
(no need to resend for this; I can add it when committing it)
I'm thinking we should also try and incorporate some performance tests for vmlinux BTF encoding into the tests subdirectory to better catch issues like this; perhaps the CI can baseline encoding performance on the next branch versus the branch that has the changes..
Thanks again!
Alan
Quoted 1 line
Sami
Jiri Olsa
Acked-byQuoted 54 lines · Alan Maguire
Quoted 44 lines · Sami Tolvanen
Hi Jiri,
Quoted 17 lines · Jiri Olsa
Quoted 15 lines · Sami Tolvanen
With
CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel inEXPORT_SYMBOL()to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit9810758003ce("btf_encoder: Verify 0 address DWARF variables are in ELF section")).Filter out symbols with the
__gendwarfksyms_ptr_name prefix infilter_variable_name()instead of callingvariable_in_sec()for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:Quoted 5 lines
Quoted 2 lines
Before: 35.775 +- 0.121 seconds time elapsed ( +- 0.34% ) After: 8.3516 +- 0.0407 seconds time elapsed ( +- 0.49% )makes sense to me, I just can't reproduce the speedup could you please share your .config?
Sure, here's the config I used to repro this:
https://gist.github.com/samitolvanen/dca66a1a779861be27579f88c9b6ba5d
This is essentially
x86_64defconfig with GENDWARFKSYMS andDEBUG_INFO_BTFboth enabled. When building this config with gcc, we end up with 0 address__gendwarfksyms_ptrvariables in DWARF:...
0x0001b5c6:DW_TAG_variableDW_AT_name ("__gendwarfksyms_ptr_system_state") DW_AT_decl_file ("../init/main.c") DW_AT_decl_line (129) DW_AT_decl_column (1) DW_AT_type (0x0001b5dc "system_states *") DW_AT_location (DW_OP_addr 0x0)...
Quoted 8 lines
Note that this doesn't seem to happen when building with Clang.
Before commit
9810758003cethis resulted in pahole thinking all these variables are in the .data..percpu section, which resulted inbtf_datasec_check_meta()failing with "Invalid offset" during boot. pahole/next doesn't have this issue, but validating the 0 address variables is unfortunately a bit slow when we have a lot of them.
Quoted 7 lines
Thanks for the fix Sami! I've tested it at my end and can reproduce the longer time for BTF encoding on
x86_64prior to the fix and its resolution. Let's wait a bit longer before landing it to see if anyone else gets a chance to test/ack it, but I think we should probably also add aFixes 9810758003ce9f ("btf_encoder: Verify 0 address DWARF variablesare in ELF section")
+1
thanks,
jirka
Quoted 13 lines
(no need to resend for this; I can add it when committing it)
I'm thinking we should also try and incorporate some performance tests for vmlinux BTF encoding into the tests subdirectory to better catch issues like this; perhaps the CI can baseline encoding performance on the next branch versus the branch that has the changes..
Thanks again!
Alan
Quoted 1 line
Sami
Alan Maguire
AppliedQuoted 15 lines · Sami Tolvanen
With
CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel inEXPORT_SYMBOL()to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit9810758003ce("btf_encoder: Verify 0 address DWARF variables are in ELF section")).Filter out symbols with the
__gendwarfksyms_ptr_name prefix infilter_variable_name()instead of callingvariable_in_sec()for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:
Quoted 4 lines
Before: 35.775 +- 0.121 seconds time elapsed ( +- 0.34% ) After: 8.3516 +- 0.0407 seconds time elapsed ( +- 0.49% )Signed-off-by Sami Tolvanen <[email protected]>
applied to the next branch of
https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
thanks!
Quoted 18 lines
--- btf_encoder.c | 1 + 1 file changed, 1 insertion(+)btf_encoder.c +1
-0@@ -2239,6 +2239,7 @@ static bool filter_variable_name(const char *name)X ("__UNIQUE_ID" ),X ("__tpstrtab_" ),X ("__exitcall_" ), +X ("__gendwarfksyms_ptr_" ), X("__func_stack_frame_non_standard_")#undef X };base-commit:
a0be596ae76c720d21eef257dec1cf2462130da1