Skip to contentSkip to conversation

btf_encoder: Filter out __gendwarfksyms_ptr_

PATCH Applied Sami Tolvanen 3 participants 6 messages Mar 17, 2025 – Mar 24, 2025 BPF

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 btf_encoder: Filter out __gendwarfksyms_ptr_

With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel in EXPORT_SYMBOL() to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit 9810758003ce ("btf_encoder: Verify 0 address DWARF variables are in ELF section")).

Filter out symbols with the __gendwarfksyms_ptr_ name prefix in filter_variable_name() instead of calling variable_in_sec() for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:

Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
 After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )
---
 btf_encoder.c | 1 +
 1 file changed, 1 insertion(+)
btf_encoder.c +1 -0
index 1bde310..2bf7c59 100644
@@ -2239,6 +2239,7 @@ static bool filter_variable_name(const char *name)
 		X("__UNIQUE_ID"),
 		X("__tpstrtab_"),
 		X("__exitcall_"),
+		X("__gendwarfksyms_ptr_"),
 		X("__func_stack_frame_non_standard_")
 		#undef X
 	};

base-commit: a0be596ae76c720d21eef257dec1cf2462130da1

Signature · 1 line
2.49.0.rc1.451.g8f38331e32-goog
#1 lore raw
Quoted 10 lines · Sami Tolvanen

With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel in EXPORT_SYMBOL() to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit 9810758003ce ("btf_encoder: Verify 0 address DWARF variables are in ELF section")).

Quoted 7 lines

Filter out symbols with the __gendwarfksyms_ptr_ name prefix in filter_variable_name() instead of calling variable_in_sec() for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:

Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
 After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )

makes sense to me, I just can't reproduce the speedup could you please share your .config?

thanks,
jirka

Quoted 23 lines
---
 btf_encoder.c | 1 +
 1 file changed, 1 insertion(+)
btf_encoder.c +1 -0
index 1bde310..2bf7c59 100644
@@ -2239,6 +2239,7 @@ static bool filter_variable_name(const char *name)
 		X("__UNIQUE_ID"),
 		X("__tpstrtab_"),
 		X("__exitcall_"),
+		X("__gendwarfksyms_ptr_"),
 		X("__func_stack_frame_non_standard_")
 		#undef X
 	};

base-commit: a0be596ae76c720d21eef257dec1cf2462130da1

Signature · 1 line
2.49.0.rc1.451.g8f38331e32-goog
#2 lore raw

Hi Jiri,

Quoted 17 lines · Jiri Olsa
Quoted 15 lines · Sami Tolvanen

With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel in EXPORT_SYMBOL() to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit 9810758003ce ("btf_encoder: Verify 0 address DWARF variables are in ELF section")).

Filter out symbols with the __gendwarfksyms_ptr_ name prefix in filter_variable_name() instead of calling variable_in_sec() for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:

Quoted 5 lines
Quoted 2 lines
Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
 After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )

makes sense to me, I just can't reproduce the speedup could you please share your .config?

Sure, here's the config I used to repro this:

https://gist.github.com/samitolvanen/dca66a1a779861be27579f88c9b6ba5d

This is essentially x86_64 defconfig with GENDWARFKSYMS and DEBUG_INFO_BTF both enabled. When building this config with gcc, we end up with 0 address __gendwarfksyms_ptr variables in DWARF:

...
0x0001b5c6: DW_TAG_variable

                DW_AT_name      ("__gendwarfksyms_ptr_system_state")
                DW_AT_decl_file ("../init/main.c")
                DW_AT_decl_line (129)
                DW_AT_decl_column       (1)
                DW_AT_type      (0x0001b5dc "system_states *")
                DW_AT_location  (DW_OP_addr 0x0)

...

Note that this doesn't seem to happen when building with Clang.

Before commit 9810758003ce this resulted in pahole thinking all these variables are in the .data..percpu section, which resulted in btf_datasec_check_meta() failing with "Invalid offset" during boot. pahole/next doesn't have this issue, but validating the 0 address variables is unfortunately a bit slow when we have a lot of them.

Sami

#3 lore raw
Quoted 44 lines · Sami Tolvanen

Hi Jiri,

Quoted 17 lines · Jiri Olsa
Quoted 15 lines · Sami Tolvanen

With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel in EXPORT_SYMBOL() to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit 9810758003ce ("btf_encoder: Verify 0 address DWARF variables are in ELF section")).

Filter out symbols with the __gendwarfksyms_ptr_ name prefix in filter_variable_name() instead of calling variable_in_sec() for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:

Quoted 5 lines
Quoted 2 lines
Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
 After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )

makes sense to me, I just can't reproduce the speedup could you please share your .config?

Sure, here's the config I used to repro this:

https://gist.github.com/samitolvanen/dca66a1a779861be27579f88c9b6ba5d

This is essentially x86_64 defconfig with GENDWARFKSYMS and DEBUG_INFO_BTF both enabled. When building this config with gcc, we end up with 0 address __gendwarfksyms_ptr variables in DWARF:

...
0x0001b5c6: DW_TAG_variable

                DW_AT_name      ("__gendwarfksyms_ptr_system_state")
                DW_AT_decl_file ("../init/main.c")
                DW_AT_decl_line (129)
                DW_AT_decl_column       (1)
                DW_AT_type      (0x0001b5dc "system_states *")
                DW_AT_location  (DW_OP_addr 0x0)

...

Quoted 8 lines

Note that this doesn't seem to happen when building with Clang.

Before commit 9810758003ce this resulted in pahole thinking all these variables are in the .data..percpu section, which resulted in btf_datasec_check_meta() failing with "Invalid offset" during boot. pahole/next doesn't have this issue, but validating the 0 address variables is unfortunately a bit slow when we have a lot of them.

Thanks for the fix Sami! I've tested it at my end and can reproduce the longer time for BTF encoding on x86_64 prior to the fix and its resolution. Let's wait a bit longer before landing it to see if anyone else gets a chance to test/ack it, but I think we should probably also add a

Fixes 9810758003ce9f ("btf_encoder: Verify 0 address DWARF variables

are in ELF section")

(no need to resend for this; I can add it when committing it)

I'm thinking we should also try and incorporate some performance tests for vmlinux BTF encoding into the tests subdirectory to better catch issues like this; perhaps the CI can baseline encoding performance on the next branch versus the branch that has the changes..

Thanks again!

Alan

Quoted 1 line

Sami

Acked-by
#4 lore raw
Quoted 54 lines · Alan Maguire
Quoted 44 lines · Sami Tolvanen

Hi Jiri,

Quoted 17 lines · Jiri Olsa
Quoted 15 lines · Sami Tolvanen

With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel in EXPORT_SYMBOL() to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit 9810758003ce ("btf_encoder: Verify 0 address DWARF variables are in ELF section")).

Filter out symbols with the __gendwarfksyms_ptr_ name prefix in filter_variable_name() instead of calling variable_in_sec() for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:

Quoted 5 lines
Quoted 2 lines
Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
 After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )

makes sense to me, I just can't reproduce the speedup could you please share your .config?

Sure, here's the config I used to repro this:

https://gist.github.com/samitolvanen/dca66a1a779861be27579f88c9b6ba5d

This is essentially x86_64 defconfig with GENDWARFKSYMS and DEBUG_INFO_BTF both enabled. When building this config with gcc, we end up with 0 address __gendwarfksyms_ptr variables in DWARF:

...
0x0001b5c6: DW_TAG_variable

                DW_AT_name      ("__gendwarfksyms_ptr_system_state")
                DW_AT_decl_file ("../init/main.c")
                DW_AT_decl_line (129)
                DW_AT_decl_column       (1)
                DW_AT_type      (0x0001b5dc "system_states *")
                DW_AT_location  (DW_OP_addr 0x0)

...

Quoted 8 lines

Note that this doesn't seem to happen when building with Clang.

Before commit 9810758003ce this resulted in pahole thinking all these variables are in the .data..percpu section, which resulted in btf_datasec_check_meta() failing with "Invalid offset" during boot. pahole/next doesn't have this issue, but validating the 0 address variables is unfortunately a bit slow when we have a lot of them.

Quoted 7 lines

Thanks for the fix Sami! I've tested it at my end and can reproduce the longer time for BTF encoding on x86_64 prior to the fix and its resolution. Let's wait a bit longer before landing it to see if anyone else gets a chance to test/ack it, but I think we should probably also add a

Fixes 9810758003ce9f ("btf_encoder: Verify 0 address DWARF variables

are in ELF section")

+1

thanks,
jirka

Quoted 13 lines

(no need to resend for this; I can add it when committing it)

I'm thinking we should also try and incorporate some performance tests for vmlinux BTF encoding into the tests subdirectory to better catch issues like this; perhaps the CI can baseline encoding performance on the next branch versus the branch that has the changes..

Thanks again!

Alan

Quoted 1 line

Sami

Applied
#5 lore raw
In reply to Sami Tolvanen
Quoted 15 lines · Sami Tolvanen

With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol> variables are added to the kernel in EXPORT_SYMBOL() to ensure DWARF type information is available for exported symbols in the TUs where they're actually exported. These symbols are dropped when linking vmlinux, but dangling references to them remain in DWARF, which results in thousands of 0 address variables that pahole needs to validate (since commit 9810758003ce ("btf_encoder: Verify 0 address DWARF variables are in ELF section")).

Filter out symbols with the __gendwarfksyms_ptr_ name prefix in filter_variable_name() instead of calling variable_in_sec() for all of them. This reduces the time it takes to process .tmp_vmlinux1 by ~77% on my test system:

Quoted 4 lines
Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
 After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )

applied to the next branch of

https://git.kernel.org/pub/scm/devel/pahole/pahole.git/

thanks!

Quoted 18 lines
---
 btf_encoder.c | 1 +
 1 file changed, 1 insertion(+)
btf_encoder.c +1 -0
index 1bde310..2bf7c59 100644
@@ -2239,6 +2239,7 @@ static bool filter_variable_name(const char *name)
 		X("__UNIQUE_ID"),
 		X("__tpstrtab_"),
 		X("__exitcall_"),
+		X("__gendwarfksyms_ptr_"),
 		X("__func_stack_frame_non_standard_")
 		#undef X
 	};

base-commit: a0be596ae76c720d21eef257dec1cf2462130da1