Skip to contentSkip to conversation

kbuild: Merge module sections if and only if CONFIG_LTO_CLANG is enabled

PATCH Applied Sean Christopherson 3 participants 9 messages Mar 22, 2021 – Apr 1, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kbuild: Merge module sections if and only if CONFIG_LTO_CLANG is enabled

Merge module sections only when using Clang LTO. With gcc-10, merging sections does not appear to update the symbol tables for the module, e.g. 'readelf -s' shows the value that a symbol would have had, if sections were not merged.

The stale symbol table breaks gdb's function disassambler, and presumably other things, e.g.

gdb -batch -ex "file arch/x86/kvm/kvm.ko" -ex "disassemble kvm_init"

reads the wrong bytes and dumps garbage.

Fixes dd2776222abb ("kbuild: lto: merge module sections")
Cc Nick Desaulniers <[email protected]>
Cc Sami Tolvanen <[email protected]>
Cc Kees Cook <[email protected]>

---

This is obviously the quick and dirty approach to fixing the problem, presumably there is a way to actually update the symbols, but that is far, far outside my area of expertise.

IIUC how the disassemblers work, the section headers are correctly updated, e.g. objdump displays the correct info, and even gdb's disassembler shows the "correct" offset, it's just the symbol tables that are out of whack.

An earlier version of the LTO series did have exactly this #ifdef, but it was dropped when no one objected to Kees' suggestion to unconditionally merge sections. https://lkml.kernel.org/r/202010141548.47CB1BC@keescook

 scripts/module.lds.S | 2 ++
 1 file changed, 2 insertions(+)
scripts/module.lds.S +2 -0
index 168cd27e6122..3580c6d02957 100644
@@ -25,6 +25,7 @@ SECTIONS {
 	 * -ffunction-sections, which increases the size of the final module.
 	 * Merge the split sections in the final binary.
 	 */
+#ifdef CONFIG_LTO_CLANG
 	.bss : {
 		*(.bss .bss.[0-9a-zA-Z_]*)
 		*(.bss..L*)
@@ -41,6 +42,7 @@ SECTIONS {
 	}
 
 	.text : { *(.text .text.[0-9a-zA-Z_]*) }
+#endif
 }
 
 /* bring in arch-specific sections */
Signature · 1 line
2.31.0.rc2.261.g7f71774620-goog
maintainer
#1 lore raw
Quoted 5 lines · Sean Christopherson

Merge module sections only when using Clang LTO. With gcc-10, merging sections does not appear to update the symbol tables for the module, e.g. 'readelf -s' shows the value that a symbol would have had, if sections were not merged.

I'm fine with limiting this to LTO only, but it would be helpful to understand which sections are actually getting merged here. Are you compiling the kernel with -ffunction-sections and/or -fdata-sections? Does this issue only happen with gcc 10?

Sami

#2 lore raw
Quoted 9 lines · Sami Tolvanen
Quoted 5 lines · Sean Christopherson

Merge module sections only when using Clang LTO. With gcc-10, merging sections does not appear to update the symbol tables for the module, e.g. 'readelf -s' shows the value that a symbol would have had, if sections were not merged.

I'm fine with limiting this to LTO only, but it would be helpful to understand which sections are actually getting merged here.

It doesn't appear to matter which sections get merged, the tables only show the correct data if there is no merging whatsoever, e.g. allowing merging for any one of the four types (.bss, .data, .rodata and .text) results in breakage. AFAICT, merging any sections causes the layout to change and throw off the symbol tables.

Quoted 1 line

Are you compiling the kernel with -ffunction-sections and/or -fdata-sections?

I tried both. Default off, and forcing those flags by hacking the Makefile had no effect.

Quoted 1 line

Does this issue only happen with gcc 10?

gcc-7 shows the same behavior, I haven't checked anything older or anything in between.

maintainer Reviewed-by Tested-by
#3 lore raw
Quoted 9 lines · Sean Christopherson
Quoted 7 lines · Sami Tolvanen
Quoted 5 lines · Sean Christopherson

Merge module sections only when using Clang LTO. With gcc-10, merging sections does not appear to update the symbol tables for the module, e.g. 'readelf -s' shows the value that a symbol would have had, if sections were not merged.

Quoted 8 lines
Quoted 2 lines

I'm fine with limiting this to LTO only, but it would be helpful to understand which sections are actually getting merged here.

It doesn't appear to matter which sections get merged, the tables only show the correct data if there is no merging whatsoever, e.g. allowing merging for any one of the four types (.bss, .data, .rodata and .text) results in breakage. AFAICT, merging any sections causes the layout to change and throw off the symbol tables.

Thanks for the clarification. I can reproduce this issue with gcc + bfd if any of the sections are merged, but gcc + lld produces valid symbol tables.

Perhaps someone more familiar with bfd can comment on whether this is a bug or a feature, and if there's a flag we can pass to bfd that would fix the issue. In the meanwhile, this patch looks like a reasonable workaround to me.

Sami

#4 lore raw
Quoted 19 lines · Sami Tolvanen
Quoted 9 lines · Sean Christopherson
Quoted 7 lines · Sami Tolvanen
Quoted 5 lines · Sean Christopherson

Merge module sections only when using Clang LTO. With gcc-10, merging sections does not appear to update the symbol tables for the module, e.g. 'readelf -s' shows the value that a symbol would have had, if sections were not merged.

Quoted 8 lines
Quoted 2 lines

I'm fine with limiting this to LTO only, but it would be helpful to understand which sections are actually getting merged here.

It doesn't appear to matter which sections get merged, the tables only show the correct data if there is no merging whatsoever, e.g. allowing merging for any one of the four types (.bss, .data, .rodata and .text) results in breakage. AFAICT, merging any sections causes the layout to change and throw off the symbol tables.

Quoted 3 lines

Thanks for the clarification. I can reproduce this issue with gcc + bfd if any of the sections are merged, but gcc + lld produces valid symbol tables.

FWIW, clang + bfd also produces mangled tables, so it does appear to be bfd specific.

Quoted 9 lines

Perhaps someone more familiar with bfd can comment on whether this is a bug or a feature, and if there's a flag we can pass to bfd that would fix the issue. In the meanwhile, this patch looks like a reasonable workaround to me.

Sami

#5 lore raw
Quoted 20 lines · Sean Christopherson
Quoted 19 lines · Sami Tolvanen
Quoted 17 lines · Sean Christopherson
Quoted 9 lines · Sami Tolvanen
Quoted 5 lines · Sean Christopherson

Merge module sections only when using Clang LTO. With gcc-10, merging sections does not appear to update the symbol tables for the module, e.g. 'readelf -s' shows the value that a symbol would have had, if sections were not merged.

I'm fine with limiting this to LTO only, but it would be helpful to understand which sections are actually getting merged here.

It doesn't appear to matter which sections get merged, the tables only show the correct data if there is no merging whatsoever, e.g. allowing merging for any one of the four types (.bss, .data, .rodata and .text) results in breakage. AFAICT, merging any sections causes the layout to change and throw off the symbol tables.

Quoted 6 lines
Quoted 3 lines

Thanks for the clarification. I can reproduce this issue with gcc + bfd if any of the sections are merged, but gcc + lld produces valid symbol tables.

FWIW, clang + bfd also produces mangled tables, so it does appear to be bfd specific.

Are you able to open a bug against bfd for this?

Quoted 7 lines
Quoted 7 lines

Perhaps someone more familiar with bfd can comment on whether this is a bug or a feature, and if there's a flag we can pass to bfd that would fix the issue. In the meanwhile, this patch looks like a reasonable workaround to me.

Thanks, I'll get this sent to Linus.

Signature · 1 line
Kees Cook
#6 lore raw
Quoted 21 lines · Kees Cook
Quoted 20 lines · Sean Christopherson
Quoted 19 lines · Sami Tolvanen
Quoted 17 lines · Sean Christopherson
Quoted 9 lines · Sami Tolvanen
Quoted 5 lines · Sean Christopherson

Merge module sections only when using Clang LTO. With gcc-10, merging sections does not appear to update the symbol tables for the module, e.g. 'readelf -s' shows the value that a symbol would have had, if sections were not merged.

I'm fine with limiting this to LTO only, but it would be helpful to understand which sections are actually getting merged here.

It doesn't appear to matter which sections get merged, the tables only show the correct data if there is no merging whatsoever, e.g. allowing merging for any one of the four types (.bss, .data, .rodata and .text) results in breakage. AFAICT, merging any sections causes the layout to change and throw off the symbol tables.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Thanks for the clarification. I can reproduce this issue with gcc + bfd if any of the sections are merged, but gcc + lld produces valid symbol tables.

FWIW, clang + bfd also produces mangled tables, so it does appear to be bfd specific.

Are you able to open a bug against bfd for this?

Yes? I'll ping you when I run into trouble ;-)

#7 lore raw
Quoted 26 lines · Sean Christopherson
Quoted 25 lines · Kees Cook
Quoted 24 lines · Sean Christopherson
Quoted 22 lines · Sami Tolvanen
Quoted 9 lines · Sean Christopherson
Quoted 7 lines · Sami Tolvanen
Quoted 5 lines · Sean Christopherson

Merge module sections only when using Clang LTO. With gcc-10, merging sections does not appear to update the symbol tables for the module, e.g. 'readelf -s' shows the value that a symbol would have had, if sections were not merged.

Quoted 8 lines
Quoted 2 lines

I'm fine with limiting this to LTO only, but it would be helpful to understand which sections are actually getting merged here.

It doesn't appear to matter which sections get merged, the tables only show the correct data if there is no merging whatsoever, e.g. allowing merging for any one of the four types (.bss, .data, .rodata and .text) results in breakage. AFAICT, merging any sections causes the layout to change and throw off the symbol tables.

Thanks for the clarification. I can reproduce this issue with gcc + bfd if any of the sections are merged, but gcc + lld produces valid symbol tables.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

FWIW, clang + bfd also produces mangled tables, so it does appear to be bfd specific.

Are you able to open a bug against bfd for this?

Yes? I'll ping you when I run into trouble ;-)

Hm, I can't tell if any of these are duplicates:

https://sourceware.org/bugzilla/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=SUSPENDED&bug_status=WAITING&bug_status=REOPENED&component=ld&list_id=59962&product=binutils&query_format=advanced&short_desc=symbol&short_desc_type=allwordssubstr

Signature · 1 line
Kees Cook
Applied
#8 lore raw
In reply to Sean Christopherson
Quoted 9 lines · Sean Christopherson

Merge module sections only when using Clang LTO. With gcc-10, merging sections does not appear to update the symbol tables for the module, e.g. 'readelf -s' shows the value that a symbol would have had, if sections were not merged.

The stale symbol table breaks gdb's function disassambler, and presumably other things, e.g.

[...]

Applied to for-linus/lto, thanks!

[1/1] kbuild: lto: Merge module sections if and only if CONFIG_LTO_CLANG is enabled

      https://git.kernel.org/kees/c/8b382ebc86a8
Signature · 1 line
Kees Cook