kbuild patches for 4.10
Subsystems (2)
cover kbuild patches for 4.10
Hi,
I have a few various kbuild patches for review and consideration for 4.10 branch that did not make 4.9. All of these except the first one have been posted before without big changes. The first, I posted an RFC for moving kallsyms data section but this approach is simpler.
All patches are independent, so any can be taken in any order. I'm posting as a group just for convenience but if that is wrong then I will submit individual patches in future.
Thanks,
Nick
Nicholas Piggin (7):
kbuild: kallsyms allow 3-pass generation if symbols size has changed
kbuild: thin archives for multi-y targets kbuild/genksyms: handle va_list type
kbuild: improve linker compatibility with lib-ksyms.o build
kbuild: keep data tables through dead code elimination
kbuild: modpost warn if export version crc is missing
kbuild: minor improvement for thin archives build
include/asm-generic/vmlinux.lds.h | 68 +++++++++++++++++++-------------------- scripts/Makefile.build | 15 ++++++--- scripts/genksyms/keywords.gperf | 1 + scripts/genksyms/parse.y | 2 ++ scripts/link-vmlinux.sh | 37 ++++++++++++--------- scripts/mod/modpost.c | 6 ++++ 6 files changed, 76 insertions(+), 53 deletions(-)
Signature · 1 line
2.10.2
PATCH 1/7 kbuild: kallsyms allow 3-pass generation if symbols size has changed
kallsyms generation is not foolproof, due to some linkers adding symbols (e.g., branch trampolines) when a binary size changes. Have it attempt a 3rd pass automatically if the kallsyms size changes in the 2nd pass.
This allows powerpc64 allyesconfig to build without adding another pass when it's not required.
This can be solved other ways by directing the linker not to add labels on branch stubs, or to move kallsyms near the end of the image. The former is undesirable for debugging/tracing, and the latter is a more significant change that requires more testing and review.
--- scripts/link-vmlinux.sh | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-)
scripts/link-vmlinux.sh +13 -6
@@ -246,10 +246,14 @@ if [ -n "${CONFIG_KALLSYMS}" ]; then # the right size, but due to the added section, some # addresses have shifted. # From here, we generate a correct .tmp_kallsyms2.o - # 2a) We may use an extra pass as this has been necessary to - # woraround some alignment related bugs. - # KALLSYMS_EXTRA_PASS=1 is used to trigger this. - # 3) The correct ${kallsymso} is linked into the final vmlinux. + # 3) That link may have expanded the kernel image enough that + # more linker branch stubs / trampolines had to be added, which + # introduces new names, which further expands kallsyms. Do another + # pass if that is the case. In theory it's possible this results + # in even more stubs, but unlikely. + # KALLSYMS_EXTRA_PASS=1 may also used to debug or work around + # other bugs. + # 4) The correct ${kallsymso} is linked into the final vmlinux. # # a) Verify that the System.map from vmlinux matches the map from # ${kallsymso}.
@@ -265,8 +269,11 @@ if [ -n "${CONFIG_KALLSYMS}" ]; then vmlinux_link .tmp_kallsyms1.o .tmp_vmlinux2 kallsyms .tmp_vmlinux2 .tmp_kallsyms2.o - # step 2a - if [ -n "${KALLSYMS_EXTRA_PASS}" ]; then + # step 3 + size1=$(stat -c "%s" .tmp_kallsyms1.o) + size2=$(stat -c "%s" .tmp_kallsyms2.o) + + if [ $size1 -ne $size2 ] || [ -n "${KALLSYMS_EXTRA_PASS}" ]; then kallsymso=.tmp_kallsyms3.o kallsyms_vmlinux=.tmp_vmlinux3
Signature · 1 line
2.10.2
PATCH 2/7 kbuild: thin archives for multi-y targets
In reply to Nicholas PigginTHIN_ARCHIVES builds archives for built-in.o targets, have it build multi-y targets as archives as well.
This saves another ~15% of the size of intermediate artifacts in the build tree. After this patch, the linker is only used in final link, and special cases like vdsos.
--- scripts/Makefile.build | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
scripts/Makefile.build +10 -3
@@ -517,11 +517,18 @@ $($(subst $(obj)/,,$(@:.o=-objs))) \ $($(subst $(obj)/,,$(@:.o=-y))) \ $($(subst $(obj)/,,$(@:.o=-m)))), $^) -quiet_cmd_link_multi-y = LD $@ -cmd_link_multi-y = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps) $(cmd_secanalysis) +cmd_link_multi-link = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps) $(cmd_secanalysis) + +ifdef CONFIG_THIN_ARCHIVES + quiet_cmd_link_multi-y = AR $@ + cmd_link_multi-y = rm -f $@; $(AR) rcST$(KBUILD_ARFLAGS) $@ $(link_multi_deps) +else + quiet_cmd_link_multi-y = LD $@ + cmd_link_multi-y = $(cmd_link_multi-link) +endif quiet_cmd_link_multi-m = LD [M] $@ -cmd_link_multi-m = $(cmd_link_multi-y) +cmd_link_multi-m = $(cmd_link_multi-link) $(multi-used-y): FORCE $(call if_changed,link_multi-y)
Signature · 1 line
2.10.2
PATCH 3/7 kbuild/genksyms: handle va_list type
In reply to Nicholas Piggingenksyms currently does not handle va_list. Add the __builtin_va_list keyword as a type. This reduces the amount of syntax errors thrown, but so far no export symbol has a type with a va_list argument, so there is currently no bug in the end result.
Note: this patch does not regenerate shipped parser files.
--- scripts/genksyms/keywords.gperf | 1 + scripts/genksyms/parse.y | 2 ++ 2 files changed, 3 insertions(+)
scripts/genksyms/keywords.gperf +1 -0
@@ -27,6 +27,7 @@ __typeof, TYPEOF_KEYW __typeof__, TYPEOF_KEYW __volatile, VOLATILE_KEYW __volatile__, VOLATILE_KEYW +__builtin_va_list, VA_LIST_KEYW # According to rth, c99 defines _Bool, __restrict, __restrict__, restrict. KAO _Bool, BOOL_KEYW _restrict, RESTRICT_KEYW
scripts/genksyms/parse.y +2 -0
@@ -98,6 +98,7 @@ static void record_compound(struct string_list **keyw, %token VOID_KEYW %token VOLATILE_KEYW %token TYPEOF_KEYW +%token VA_LIST_KEYW %token EXPORT_SYMBOL_KEYW
@@ -261,6 +262,7 @@ simple_type_specifier: | DOUBLE_KEYW | VOID_KEYW | BOOL_KEYW + | VA_LIST_KEYW | TYPE { (*$1)->tag = SYM_TYPEDEF; $$ = $1; } ;
Signature · 1 line
2.10.2
PATCH 4/7 kbuild: improve linker compatibility with lib-ksyms.o build
In reply to Nicholas Pigginlib-ksyms.o is created by linking an empty input file with a linker script containing the interesting bits. Currently the empty input file is an archive containing nothing, however this causes the gold linker to segfault.
I have opened a bug against gold
https://sourceware.org/bugzilla/show_bug.cgi?id=20767
However this can be worked around by assembling an empty file to link with instead. The resulting lib-ksyms.o is slightly larger (seemingly due to empty .text, .data, .bss setions added), but final linked output should not be changed.
--- scripts/Makefile.build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/Makefile.build +1 -1
@@ -490,7 +490,7 @@ quiet_cmd_export_list = EXPORTS $@ cmd_export_list = $(OBJDUMP) -h $< | \ sed -ne '/___ksymtab/{s/.*+/$(ref_prefix)/;s/ .*/)/;p}' >$(ksyms-lds);\ rm -f $(dummy-object);\ - $(AR) rcs$(KBUILD_ARFLAGS) $(dummy-object);\ + echo | $(CC) $(a_flags) -c -o $(dummy-object) -x assembler -;\ $(LD) $(ld_flags) -r -o $@ -T $(ksyms-lds) $(dummy-object);\ rm $(dummy-object) $(ksyms-lds)
Signature · 1 line
2.10.2
PATCH 5/7 kbuild: keep data tables through dead code elimination
In reply to Nicholas PigginWhen CONFIG_LD_DEAD_CODE_DATA_ELIMINATION is enabled we must ensure that we still keep various programatically-accessed tables.
[npiggin: Fold Paul's patches into one, and add a few more tables.
diff symbol tables of allyesconfig with/without -gc-sections shows up lost tables quite easily.]
--- include/asm-generic/vmlinux.lds.h | 68 +++++++++++++++++++-------------------- 1 file changed, 34 insertions(+), 34 deletions(-)
include/asm-generic/vmlinux.lds.h +34 -34
@@ -114,7 +114,7 @@#ifdef CONFIG_KPROBES #define KPROBE_BLACKLIST () . = ALIGN(8); \ VMLINUX_SYMBOL(__start_kprobe_blacklist) = .; \ - *(_kprobe_blacklist) \ + KEEP(*(_kprobe_blacklist)) \ VMLINUX_SYMBOL(__stop_kprobe_blacklist) = .;#else #define KPROBE_BLACKLIST ()
@@ -123,10 +123,10 @@#ifdef CONFIG_EVENT_TRACING #define FTRACE_EVENTS () . = ALIGN(8); \ VMLINUX_SYMBOL(__start_ftrace_events) = .; \ - *(_ftrace_events) \ + KEEP(*(_ftrace_events)) \ VMLINUX_SYMBOL(__stop_ftrace_events) = .; \ VMLINUX_SYMBOL(__start_ftrace_enum_maps) = .; \ - *(_ftrace_enum_map) \ + KEEP(*(_ftrace_enum_map)) \ VMLINUX_SYMBOL(__stop_ftrace_enum_maps) = .;#else #define FTRACE_EVENTS ()
@@ -134,10 +134,10 @@#ifdef CONFIG_TRACING #define TRACE_PRINTKS () VMLINUX_SYMBOL(__start___trace_bprintk_fmt) = .; \ - *(__trace_printk_fmt)/* Trace_printk fmt' pointer */ \ + KEEP(*(__trace_printk_fmt))/* Trace_printk fmt' pointer */ \ VMLINUX_SYMBOL(__stop___trace_bprintk_fmt) = .;#define TRACEPOINT_STR () VMLINUX_SYMBOL(__start___tracepoint_str) = .; \ - *(__tracepoint_str)/* Trace_printk fmt' pointer */ \ + KEEP(*(__tracepoint_str))/* Trace_printk fmt' pointer */ \ VMLINUX_SYMBOL(__stop___tracepoint_str) = .;#else #define TRACE_PRINTKS ()
@@ -147,7 +147,7 @@#ifdef CONFIG_FTRACE_SYSCALLS #define TRACE_SYSCALLS () . = ALIGN(8); \ VMLINUX_SYMBOL(__start_syscalls_metadata) = .; \ - *(__syscalls_metadata) \ + KEEP(*(__syscalls_metadata)) \ VMLINUX_SYMBOL(__stop_syscalls_metadata) = .;#else #define TRACE_SYSCALLS ()
@@ -156,7 +156,7 @@#ifdef CONFIG_SERIAL_EARLYCON #define EARLYCON_TABLE () STRUCT_ALIGN(); \ VMLINUX_SYMBOL(__earlycon_table) = .; \ - *(__earlycon_table) \ + KEEP(*(__earlycon_table)) \ VMLINUX_SYMBOL(__earlycon_table_end) = .;#else #define EARLYCON_TABLE ()
@@ -169,8 +169,8 @@#define _OF_TABLE_1 (name) \ . = ALIGN(8); \ VMLINUX_SYMBOL(__##name##_of_table) = .; \ - *(__##name##_of_table) \ - *(__##name##_of_table_end) + KEEP(*(__##name##_of_table)) \ + KEEP(*(__##name##_of_table_end))#define CLKSRC_OF_TABLES () OF_TABLE(CONFIG_CLKSRC_OF, clksrc)#define IRQCHIP_OF_MATCH_TABLE () OF_TABLE(CONFIG_IRQCHIP, irqchip)
@@ -184,7 +184,7 @@#define ACPI_PROBE_TABLE (name) \ . = ALIGN(8); \ VMLINUX_SYMBOL(__##name##_acpi_probe_table) = .; \ - *(__##name##_acpi_probe_table) \ + KEEP(*(__##name##_acpi_probe_table)) \ VMLINUX_SYMBOL(__##name##_acpi_probe_table_end) = .;#else #define ACPI_PROBE_TABLE (name)
@@ -193,7 +193,7 @@#define KERNEL_DTB () \ STRUCT_ALIGN(); \ VMLINUX_SYMBOL(__dtb_start) = .; \ - *(.dtb.init.rodata) \ + KEEP(*(.dtb.init.rodata)) \ VMLINUX_SYMBOL(__dtb_end) = .;/*
@@ -214,11 +214,11 @@/* implement dynamic printk debug */ \ . = ALIGN(8); \ VMLINUX_SYMBOL(__start___jump_table) = .; \ - *(__jump_table ) \ + KEEP(*(__jump_table)) \ VMLINUX_SYMBOL(__stop___jump_table) = .; \ . = ALIGN(8); \ VMLINUX_SYMBOL(__start___verbose) = .; \ - *(__verbose ) \ + KEEP(*(__verbose)) \ VMLINUX_SYMBOL(__stop___verbose) = .; \ LIKELY_PROFILE() \BRANCH_PROFILE () \
@@ -274,10 +274,10 @@ VMLINUX_SYMBOL(__start_rodata) = .; \ *(.rodata) *(.rodata.*) \ RO_AFTER_INIT_DATA /* Read only after init */ \ - *(__vermagic) /* Kernel version magic */ \ +KEEP (*(__vermagic))/* Kernel version magic */ \ . = ALIGN(8); \ VMLINUX_SYMBOL(__start___tracepoints_ptrs) = .; \ - *(__tracepoints_ptrs) /* Tracepoints: pointer array */\ + KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \ VMLINUX_SYMBOL(__stop___tracepoints_ptrs) = .; \ *(__tracepoints_strings)/* Tracepoints: strings */ \ } \
@@ -291,35 +291,35 @@/* PCI quirks */ \ .pci_fixup : AT(ADDR(.pci_fixup) - LOAD_OFFSET) { \ VMLINUX_SYMBOL(__start_pci_fixups_early) = .; \ - *(.pci_fixup_early) \ + KEEP(*(.pci_fixup_early)) \ VMLINUX_SYMBOL(__end_pci_fixups_early) = .; \ VMLINUX_SYMBOL(__start_pci_fixups_header) = .; \ - *(.pci_fixup_header) \ + KEEP(*(.pci_fixup_header)) \ VMLINUX_SYMBOL(__end_pci_fixups_header) = .; \ VMLINUX_SYMBOL(__start_pci_fixups_final) = .; \ - *(.pci_fixup_final) \ + KEEP(*(.pci_fixup_final)) \ VMLINUX_SYMBOL(__end_pci_fixups_final) = .; \ VMLINUX_SYMBOL(__start_pci_fixups_enable) = .; \ - *(.pci_fixup_enable) \ + KEEP(*(.pci_fixup_enable)) \ VMLINUX_SYMBOL(__end_pci_fixups_enable) = .; \ VMLINUX_SYMBOL(__start_pci_fixups_resume) = .; \ - *(.pci_fixup_resume) \ + KEEP(*(.pci_fixup_resume)) \ VMLINUX_SYMBOL(__end_pci_fixups_resume) = .; \ VMLINUX_SYMBOL(__start_pci_fixups_resume_early) = .; \ - *(.pci_fixup_resume_early) \ + KEEP(*(.pci_fixup_resume_early)) \ VMLINUX_SYMBOL(__end_pci_fixups_resume_early) = .; \ VMLINUX_SYMBOL(__start_pci_fixups_suspend) = .; \ - *(.pci_fixup_suspend) \ + KEEP(*(.pci_fixup_suspend)) \ VMLINUX_SYMBOL(__end_pci_fixups_suspend) = .; \ VMLINUX_SYMBOL(__start_pci_fixups_suspend_late) = .; \ - *(.pci_fixup_suspend_late) \ + KEEP(*(.pci_fixup_suspend_late)) \ VMLINUX_SYMBOL(__end_pci_fixups_suspend_late) = .; \ } \ \/* Built-in firmware blobs */ \ .builtin_fw : AT(ADDR(.builtin_fw) - LOAD_OFFSET) { \ VMLINUX_SYMBOL(__start_builtin_fw) = .; \ - *(.builtin_fw) \ + KEEP(*(.builtin_fw)) \ VMLINUX_SYMBOL(__end_builtin_fw) = .; \ } \ \
@@ -397,7 +397,7 @@ \/* Kernel symbol table: strings */ \ __ksymtab_strings : AT(ADDR(__ksymtab_strings) - LOAD_OFFSET) { \ - KEEP(*(__ksymtab_strings)) \ + *(__ksymtab_strings) \ } \ \/* __*init sections */ \
@@ -410,14 +410,14 @@/* Built-in module parameters. */ \ __param : AT(ADDR(__param) - LOAD_OFFSET) { \ VMLINUX_SYMBOL(__start___param) = .; \ - *(__param ) \ + KEEP(*(__param)) \ VMLINUX_SYMBOL(__stop___param) = .; \ } \ \/* Built-in module versions. */ \ __modver : AT(ADDR(__modver) - LOAD_OFFSET) { \ VMLINUX_SYMBOL(__start___modver) = .; \ - *(__modver ) \ + KEEP(*(__modver)) \ VMLINUX_SYMBOL(__stop___modver) = .; \ . = ALIGN((align)); \ VMLINUX_SYMBOL(__end_rodata) = .; \
@@ -520,7 +520,7 @@ . = ALIGN(align); \ __ex_table : AT(ADDR(__ex_table) - LOAD_OFFSET) { \ VMLINUX_SYMBOL(__start___ex_table) = .; \ - *(__ex_table ) \ + KEEP(*(__ex_table)) \ VMLINUX_SYMBOL(__stop___ex_table) = .; \ }
@@ -536,9 +536,9 @@#ifdef CONFIG_CONSTRUCTORS #define KERNEL_CTORS () . = ALIGN(8); \ VMLINUX_SYMBOL(__ctors_start) = .; \ - *(.ctors) \ - *(SORT(.init_array.*)) \ - *(.init_array) \ + KEEP(*(.ctors)) \ + KEEP(*(SORT(.init_array.*))) \ + KEEP(*(.init_array)) \ VMLINUX_SYMBOL(__ctors_end) = .;#else #define KERNEL_CTORS ()
@@ -662,7 +662,7 @@ . = ALIGN(8); \ __bug_table : AT(ADDR(__bug_table) - LOAD_OFFSET) { \ VMLINUX_SYMBOL(__start___bug_table) = .; \ - *(__bug_table ) \ + KEEP(*(__bug_table)) \ VMLINUX_SYMBOL(__stop___bug_table) = .; \ }#else
@@ -674,7 +674,7 @@ . = ALIGN(4); \ .tracedata : AT(ADDR(.tracedata) - LOAD_OFFSET) { \ VMLINUX_SYMBOL(__tracedata_start) = .; \ - *(.tracedata) \ + KEEP(*(.tracedata)) \ VMLINUX_SYMBOL(__tracedata_end) = .; \ } #else
@@ -691,7 +691,7 @@#define INIT_SETUP (initsetup_align) \ . = ALIGN(initsetup_align); \ VMLINUX_SYMBOL(__setup_start) = .; \ - *(.init.setup) \ + KEEP(*(.init.setup)) \ VMLINUX_SYMBOL(__setup_end) = .;#define INIT_CALLS_LEVEL (level) \
Signature · 1 line
2.10.2
PATCH 6/7 kbuild: modpost warn if export version crc is missing
In reply to Nicholas PigginThis catches the failing ceph CRC on with:
LD vmlinux.o
MODPOST vmlinux.o
WARNING: EXPORT symbol "ceph_monc_do_statfs" [vmlinux] version generation failed, symbol will not be versioned.
When the modules referring to exported symbols are built, there is an existing warning for missing CRC, but it's not always the case such any such module will be built, and in any case it is useful to get a warning at the source.
This gets a little verbose with CONFIG_DEBUG_SECTION_MISMATCH, producing a warning with each object linked, but I didn't think that warranted extra complexity to avoid.
--- scripts/mod/modpost.c | 6 ++++++ 1 file changed, 6 insertions(+)
scripts/mod/modpost.c +6 -0
@@ -609,6 +609,7 @@ static void handle_modversions(struct module *mod, struct elf_info *info, {unsigned int crc;enum export export; +bool is_crc = false;if ((!is_vmlinux (mod->name) || mod->is_dot_o) &&strncmp (symname,"__ksymtab" ,9 ) ==0 )
@@ -618,6 +619,7 @@ static void handle_modversions(struct module *mod, struct elf_info *info,/* CRC'd symbol */ if (strncmp (symname,CRC_PFX ,strlen (CRC_PFX )) ==0 ) { + is_crc = true; crc = (unsigned int ) sym->st_value;sym_update_crc (symname +strlen (CRC_PFX ), mod, crc, export);
@@ -663,6 +665,10 @@ static void handle_modversions(struct module *mod, struct elf_info *info, else symname++;#endif +if (is_crc) { +const char *e =is_vmlinux (mod->name) ?"" :".ko" ; +warn ("EXPORT symbol \"%s\" [%s%s] version generation failed, symbol will not be versioned.\n" , symname +strlen (CRC_PFX ), mod->name, e); + } mod->unres =alloc_symbol (symname,ELF_ST_BIND (sym->st_info) ==STB_WEAK , mod->unres);
Signature · 1 line
2.10.2
PATCH 7/7 kbuild: minor improvement for thin archives build
In reply to Nicholas PigginThe root built-in.o archive is currently generated before all object files are built for the final link, due to final build of init/ after version update. In practice it seems like it doesn't matter because the archive symbol table does not change, but it is more logical to create the final archive as the last step.
--- scripts/link-vmlinux.sh | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
scripts/link-vmlinux.sh +9 -9
@@ -209,15 +209,6 @@ case "${KCONFIG_CONFIG}" in . "./${KCONFIG_CONFIG}" esac -archive_builtin - -#link vmlinux.o -info LD vmlinux.o -modpost_link vmlinux.o - -# modpost vmlinux.o to check for section mismatches -${MAKE} -f "${srctree}/scripts/Makefile.modpost" vmlinux.o - # Update version info GEN .version if [ ! -r .version ]; then
@@ -231,6 +222,15 @@ fi; # final build of init/ ${MAKE} -f "${srctree}/scripts/Makefile.build" obj=init GCC_PLUGINS_CFLAGS="${GCC_PLUGINS_CFLAGS}" +archive_builtin + +#link vmlinux.o +info LD vmlinux.o +modpost_link vmlinux.o + +# modpost vmlinux.o to check for section mismatches +${MAKE} -f "${srctree}/scripts/Makefile.modpost" vmlinux.o + kallsymso="" kallsyms_vmlinux="" if [ -n "${CONFIG_KALLSYMS}" ]; then
Signature · 1 line
2.10.2
Dne 23.11.2016 v 17:41 Nicholas Piggin napsal(a):
Quoted 10 lines
Hi,
I have a few various kbuild patches for review and consideration for 4.10 branch that did not make 4.9. All of these except the first one have been posted before without big changes. The first, I posted an RFC for moving kallsyms data section but this approach is simpler.
All patches are independent, so any can be taken in any order. I'm posting as a group just for convenience but if that is wrong then I will submit individual patches in future.
Thanks, I applied them to kbuild.git#kbuild now.
Michal