kbuild: Avoid weak external linkage where possible
Subsystems (6)
cover kbuild: Avoid weak external linkage where possible
From: Ard Biesheuvel <[email protected]>
Weak external linkage is intended for cases where a symbol reference can remain unsatisfied in the final link. Taking the address of such a symbol should yield NULL if the reference was not satisfied.
Given that ordinary RIP or PC relative references cannot produce NULL, some kind of indirection is always needed in such cases, and in position independent code, this results in a GOT entry. In ordinary code, it is arch specific but amounts to the same thing.
While unavoidable in some cases, weak references are currently also used to declare symbols that are always defined in the final link, but not in the first linker pass. This means we end up with worse codegen for no good reason. So let's clean this up, by providing preliminary definitions that are only used as a fallback.
Ard Biesheuvel (3):
kallsyms: Avoid weak references for kallsyms symbols
vmlinux: Avoid weak reference to notes section
btf: Avoid weak external references
include/asm-generic/vmlinux.lds.h | 21 ++++++++++++++ kernel/bpf/btf.c | 4 +-- kernel/kallsyms.c | 6 ---- kernel/kallsyms_internal.h | 30 ++++++++------------ kernel/ksysfs.c | 4 +-- lib/buildid.c | 4 +-- 6 files changed, 39 insertions(+), 30 deletions(-)
Signature · 1 line
2.44.0.478.gd926399ef9-goog
PATCH 1/3 kallsyms: Avoid weak references for kallsyms symbols
From: Ard Biesheuvel <[email protected]>
kallsyms is a directory of all the symbols in the vmlinux binary, and so creating it is somewhat of a chicken-and-egg problem, as its non-zero size affects the layout of the binary, and therefore the values of the symbols.
For this reason, the kernel is linked more than once, and the first pass does not include any kallsyms data at all. For the linker to accept this, the symbol declarations describing the kallsyms metadata are emitted as having weak linkage, so they can remain unsatisfied. During the subsequent passes, the weak references are satisfied by the kallsyms metadata that was constructed based on information gathered from the preceding passes.
Weak references lead to somewhat worse codegen, because taking their address may need to produce NULL (if the reference was unsatisfied), and this is not usually supported by RIP or PC relative symbol references.
Given that these references are ultimately always satisfied in the final link, let's drop the weak annotation, and instead, provide fallback definitions in the linker script that are only emitted if an unsatisfied reference exists.
While at it, drop the FRV specific annotation that these symbols reside in .rodata - FRV is long gone.
--- include/asm-generic/vmlinux.lds.h | 19 +++++++++++++ kernel/kallsyms.c | 6 ---- kernel/kallsyms_internal.h | 30 ++++++++------------ 3 files changed, 31 insertions(+), 24 deletions(-)
include/asm-generic/vmlinux.lds.h +19 -0
@@ -448,11 +448,30 @@#endif #endif +/* +* Some symbol definitions will not exist yet during the first pass of the +* link, but are guaranteed to exist in the final link. Provide preliminary +* definitions that will be superseded in the final link to avoid having to +* rely on weak external linkage, which requires a GOT when used in position +* independent code. +*/ +#define PRELIMINARY_SYMBOL_DEFINITIONS \ + PROVIDE(kallsyms_addresses = .); \ + PROVIDE(kallsyms_offsets = .); \ + PROVIDE(kallsyms_names = .); \ + PROVIDE(kallsyms_num_syms = .); \ + PROVIDE(kallsyms_relative_base = .); \ + PROVIDE(kallsyms_token_table = .); \ + PROVIDE(kallsyms_token_index = .); \ + PROVIDE(kallsyms_markers = .); \ + PROVIDE(kallsyms_seqs_of_names = .); +/* * Read only Data */ #define RO_DATA (align) \ . = ALIGN((align)); \ + PRELIMINARY_SYMBOL_DEFINITIONS \ .rodata : AT(ADDR(.rodata) - LOAD_OFFSET) { \ __start_rodata = .; \ *(.rodata) *(.rodata.*) \
kernel/kallsyms.c +0 -6
@@ -325,12 +325,6 @@ static unsigned long get_symbol_pos(unsigned long addr,unsigned long symbol_start =0 , symbol_end =0 ;unsigned long i, low, high, mid; -/* This kernel should never had been booted. */ -if (!IS_ENABLED (CONFIG_KALLSYMS_BASE_RELATIVE )) -BUG_ON (!kallsyms_addresses); -else -BUG_ON (!kallsyms_offsets); -/* Do a binary search on the sorted kallsyms_addresses array. */ low =0 ; high = kallsyms_num_syms;
kernel/kallsyms_internal.h +12 -18
@@ -5,27 +5,21 @@#include <linux/types.h> /* -* These will be re-linked against their real values -* during the second link stage. +* These will be re-linked against their real values during the second link +* stage. Preliminary values must be provided in the linker script using the +* PROVIDE() directive so that the first link stage can complete successfully. */ -extern const unsigned long kallsyms_addresses[]__weak ; -extern const int kallsyms_offsets[]__weak ; -extern const u8 kallsyms_names[]__weak ; +extern const unsigned long kallsyms_addresses[]; +extern const int kallsyms_offsets[]; +extern const u8 kallsyms_names[]; -/* -* Tell the compiler that the count isn't in the small data section if the arch -* has one (eg: FRV). -*/ -extern const unsigned int kallsyms_num_syms -__section(".rodata") __attribute__((weak)); - -extern const unsigned long kallsyms_relative_base -__section(".rodata") __attribute__((weak)); +extern const unsigned int kallsyms_num_syms; +extern const unsigned long kallsyms_relative_base; -extern const char kallsyms_token_table[]__weak ; -extern const u16 kallsyms_token_index[]__weak ; +extern const char kallsyms_token_table[]; +extern const u16 kallsyms_token_index[]; -extern const unsigned int kallsyms_markers[]__weak ; -extern const u8 kallsyms_seqs_of_names[]__weak ; +extern const unsigned int kallsyms_markers[]; +extern const u8 kallsyms_seqs_of_names[];#endif // LINUX_KALLSYMS_INTERNAL_H_
Signature · 1 line
2.44.0.478.gd926399ef9-goog
PATCH 2/3 vmlinux: Avoid weak reference to notes section
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Weak references are references that are permitted to remain unsatisfied in the final link. This means they cannot be implemented using place relative relocations, resulting in GOT entries when using position independent code generation.
The notes section should always exist, so the weak annotations can be omitted.
--- kernel/ksysfs.c | 4 ++-- lib/buildid.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-)
kernel/ksysfs.c +2 -2
@@ -228,8 +228,8 @@ KERNEL_ATTR_RW(rcu_normal);/* * Make /sys/kernel/notes give the raw contents of our kernel .notes section. */ -extern const void __start_notes__weak ; -extern const void __stop_notes__weak ; +extern const void __start_notes; +extern const void __stop_notes;#define notes_size (&__stop_notes - &__start_notes) static ssize_t notes_read(struct file *filp, struct kobject *kobj,
lib/buildid.c +2 -2
@@ -182,8 +182,8 @@ unsigned char vmlinux_build_id[BUILD_ID_SIZE_MAX] __ro_after_init;*/ void __init init_vmlinux_build_id (void ) { -extern const void __start_notes__weak ; -extern const void __stop_notes__weak ; +extern const void __start_notes; +extern const void __stop_notes;unsigned int size = &__stop_notes - &__start_notes;build_id_parse_buf (&__start_notes, vmlinux_build_id, size);
Signature · 1 line
2.44.0.478.gd926399ef9-goog
PATCH 3/3 btf: Avoid weak external references
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
If the BTF code is enabled in the build configuration, the start/stop BTF markers are guaranteed to exist in the final link but not during the first linker pass.
Avoid GOT based relocations to these markers in the final executable by providing preliminary definitions that will be used by the first linker pass, and superseded by the actual definitions in the subsequent ones.
--- include/asm-generic/vmlinux.lds.h | 2 ++ kernel/bpf/btf.c | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-)
include/asm-generic/vmlinux.lds.h +2 -0
@@ -456,6 +456,8 @@* independent code. */ #define PRELIMINARY_SYMBOL_DEFINITIONS \ + PROVIDE(__start_BTF = .); \ + PROVIDE(__stop_BTF = .); \ PROVIDE(kallsyms_addresses = .); \ PROVIDE(kallsyms_offsets = .); \ PROVIDE(kallsyms_names = .); \
kernel/bpf/btf.c +2 -2
@@ -5642,8 +5642,8 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uatreturn ERR_PTR (err); } -extern char __weak __start_BTF[]; -extern char __weak __stop_BTF[]; +extern char __start_BTF[]; +extern char __stop_BTF[];extern struct btf *btf_vmlinux;#define BPF_MAP_TYPE (_id, _ops)
Signature · 1 line
2.44.0.478.gd926399ef9-goog
Re: [PATCH 3/3] btf: Avoid weak external references
Quoted 22 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
If the BTF code is enabled in the build configuration, the start/stop BTF markers are guaranteed to exist in the final link but not during the first linker pass.
Avoid GOT based relocations to these markers in the final executable by providing preliminary definitions that will be used by the first linker pass, and superseded by the actual definitions in the subsequent ones.
Signed-off-by Ard Biesheuvel <[email protected]>--- include/asm-generic/vmlinux.lds.h | 2 ++ kernel/bpf/btf.c | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-)include/asm-generic/vmlinux.lds.h +0
-0
include/asm-generic/vmlinux.lds.h line 460
Quoted 5 lines
include/asm-generic/vmlinux.lds.h +2
-0@@ -456,6 +456,8 @@* independent code. */ #define PRELIMINARY_SYMBOL_DEFINITIONS \ + PROVIDE(__start_BTF = .); \ + PROVIDE(__stop_BTF = .); \
should this be guarded by CONFIG_DEBUG_INFO_BTF condition?
Quoted 8 lines
include/asm-generic/vmlinux.lds.h +0
-0PROVIDE(kallsyms_addresses = .); \ PROVIDE(kallsyms_offsets = .); \ PROVIDE(kallsyms_names = .); \kernel/bpf/btf.c +0
-0
kernel/bpf/btf.c static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uat · line 5646
Quoted 7 lines
kernel/bpf/btf.c +2
-2@@ -5642,8 +5642,8 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uatreturn ERR_PTR (err); } -extern char __weak __start_BTF[]; -extern char __weak __stop_BTF[]; +extern char __start_BTF[]; +extern char __stop_BTF[];
kernel/bpf/sysfs_btf.c also defines __weak externs for these symbols, you probably need to adjust that as well?
Other than that looks good to me:
Quoted 7 lines
kernel/bpf/btf.c +0
-1extern struct btf *btf_vmlinux;#define BPF_MAP_TYPE (_id, _ops) --2.44.0.478.gd926399ef9-goog
Re: [PATCH 3/3] btf: Avoid weak external references
Quoted 19 lines · Andrii Nakryiko
Quoted 17 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
If the BTF code is enabled in the build configuration, the start/stop BTF markers are guaranteed to exist in the final link but not during the first linker pass.
Avoid GOT based relocations to these markers in the final executable by providing preliminary definitions that will be used by the first linker pass, and superseded by the actual definitions in the subsequent ones.
Signed-off-by Ard Biesheuvel <[email protected]>--- include/asm-generic/vmlinux.lds.h | 2 ++ kernel/bpf/btf.c | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-)
Quoted 13 lines
include/asm-generic/vmlinux.lds.h line 460
Quoted 10 lines
include/asm-generic/vmlinux.lds.h +2
-0@@ -456,6 +456,8 @@* independent code. */ #define PRELIMINARY_SYMBOL_DEFINITIONS \ + PROVIDE(__start_BTF = .); \ + PROVIDE(__stop_BTF = .); \should this be guarded by
CONFIG_DEBUG_INFO_BTFcondition?
That shouldn't matter - the linker will not create the symbol unless an unsatisfied reference to it exists anywhere in the input.
Quoted 11 lines
Quoted 11 lines
include/asm-generic/vmlinux.lds.h +0
-0PROVIDE(kallsyms_addresses = .); \ PROVIDE(kallsyms_offsets = .); \ PROVIDE(kallsyms_names = .); \kernel/bpf/btf.c +0
-0@@ -5642,8 +5642,8 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uatreturn ERR_PTR (err); }
Quoted 8 lines
kernel/bpf/btf.c static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uat · line 5646
Quoted 4 lines
kernel/bpf/btf.c +2
-2-extern char __weak __start_BTF[]; -extern char __weak __stop_BTF[]; +extern char __start_BTF[]; +extern char __stop_BTF[];
kernel/bpf/sysfs_btf.calso defines__weakexterns for these symbols, you probably need to adjust that as well?
Yes, thanks for pointing that out.
Quoted 4 lines
Other than that looks good to me:
Acked-by Andrii Nakryiko <[email protected]>
Thanks!
Quoted 8 lines
Quoted 8 lines
kernel/bpf/btf.c +0
-1extern struct btf *btf_vmlinux;#define BPF_MAP_TYPE (_id, _ops) --2.44.0.478.gd926399ef9-goog