Skip to contentSkip to conversation

kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y

PATCH In review Masahiro Yamada 3 participants 4 messages Mar 9, 2020 – Mar 10, 2020 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y

Kbuild supports not only obj-y but also lib-y to list objects linked to vmlinux.

The difference between them is that all the objects from obj-y are forcibly linked to vmlinux by using --whole-archive, whereas the objects from lib-y are linked as needed; if there is no user of a lib-y object, it is not linked.

lib-y is intended to list utility functions that may be called from all over the place (and may be unused at all), but it is a problem for EXPORT_SYMBOL(). Even if there is no call-site in the vmlinux, we need to keep exported symbols for the use from loadable modules.

Commit 7f2084fa55e6 ("[kbuild] handle exports in lib-y objects reliably") worked around it by linking a dummy object, lib-ksyms.o, which contains references to all the symbols exported from lib.a in that directory. It uses the linker script command, EXTERN. Unfortunately, the meaning of EXTERN of ld.lld is different from that of ld.bfd. Therefore, this does not work with LD=ld.lld (CBL issue #515).

Anyway, the build rule of lib-ksyms.o is somewhat tricky. So, I want to get rid of it.

At first, I was thinking of accumulating lib-y objects into obj-y (or even replacing lib-y with obj-y entirely), but the lib-y syntax is used beyond the ordinary use in lib/ and arch/*/lib/.

Examples:

- drivers/firmware/efi/libstub/Makefile builds lib.a, which is linked into vmlinux in the own way (arm64), or linked to the decompressor (arm, x86).

- arch/alpha/lib/Makefile builds lib.a which is linked not only to vmlinux, but also to bootloaders in arch/alpha/boot/Makefile.

- arch/xtensa/boot/lib/Makefile builds lib.a for use from arch/xtensa/boot/boot-redboot/Makefile.

One more thing, adding everything to obj-y would increase the vmlinux size of allnoconfig (or tinyconfig).

For less impact, I tweaked the destination of lib.a at the top Makefile; when CONFIG_MODULES=y, lib.a goes to KBUILD_VMLINUX_OBJS, which is forcibly linked to vmlinux, otherwise lib.a goes to KBUILD_VMLINUX_LIBS as before.

The size impact for normal usecases is quite small since at lease one symbol in every lib-y object is eventually called by someone. In case you are intrested, here are the figures.

x86_64_defconfig:

text data bss dec hex filename
19566602 5422072 1589328 26578002 1958c52 vmlinux.before 19566932 5422104 1589328 26578364 1958dbc vmlinux.after

The case with the biggest impact is allnoconfig + CONFIG_MODULES=y.

ARCH=x86 allnoconfig + CONFIG_MODULES=y:

text data bss dec hex filename
1175162 254740 1220608 2650510 28718e vmlinux.before
1177974 254836 1220608 2653418 287cea vmlinux.after

Hopefully this is still not a big deal. The per-file trimming with the static library is not so effective after all.

If fine-grained optimization is desired, some architectures support CONFIG_LD_DEAD_CODE_DATA_ELIMINATION, which trims dead code per-symbol basis. When LTO is supported in mainline, even better optimization will be possible.

---

 Makefile               |  7 ++++++-
 scripts/Makefile.build | 17 -----------------
 2 files changed, 6 insertions(+), 18 deletions(-)
Makefile +6 -1
index 86035d866f2c..07f89d2a581a 100644
@@ -1031,8 +1031,13 @@ init-y		:= $(patsubst %/, %/built-in.a, $(init-y))
 core-y		:= $(patsubst %/, %/built-in.a, $(core-y))
 drivers-y	:= $(patsubst %/, %/built-in.a, $(drivers-y))
 net-y		:= $(patsubst %/, %/built-in.a, $(net-y))
+libs-y2		:= $(patsubst %/, %/built-in.a, $(filter %/, $(libs-y)))
+ifdef CONFIG_MODULES
+libs-y1		:= $(filter-out %/, $(libs-y))
+libs-y2		+= $(patsubst %/, %/lib.a, $(filter %/, $(libs-y)))
+else
 libs-y1		:= $(patsubst %/, %/lib.a, $(libs-y))
-libs-y2		:= $(patsubst %/, %/built-in.a, $(filter-out %.a, $(libs-y)))
+endif
 virt-y		:= $(patsubst %/, %/built-in.a, $(virt-y))
 
 # Externally visible symbols (used by link-vmlinux.sh)
scripts/Makefile.build +0 -17
index a1730d42e5f3..356601994f3a 100644
@@ -65,7 +65,6 @@ endif
 
 ifneq ($(strip $(lib-y) $(lib-m) $(lib-)),)
 lib-target := $(obj)/lib.a
-real-obj-y += $(obj)/lib-ksyms.o
 endif
 
 ifdef need-builtin
@@ -410,22 +409,6 @@ $(lib-target): $(lib-y) FORCE
 
 targets += $(lib-target)
 
-dummy-object = $(obj)/.lib_exports.o
-ksyms-lds = $(dot-target).lds
-
-quiet_cmd_export_list = EXPORTS $@
-cmd_export_list = $(OBJDUMP) -h $< | \
-	sed -ne '/___ksymtab/s/.*+\([^ ]*\).*/EXTERN(\1)/p' >$(ksyms-lds);\
-	rm -f $(dummy-object);\
-	echo | $(CC) $(a_flags) -c -o $(dummy-object) -x assembler -;\
-	$(LD) $(ld_flags) -r -o $@ -T $(ksyms-lds) $(dummy-object);\
-	rm $(dummy-object) $(ksyms-lds)
-
-$(obj)/lib-ksyms.o: $(lib-target) FORCE
-	$(call if_changed,export_list)
-
-targets += $(obj)/lib-ksyms.o
-
 endif
 
 # NOTE:
Signature · 1 line
2.17.1
#1 lore raw
I love your patch! Yet something to improve:

Hi Masahiro,

I love your patch! Yet something to improve:

[auto build test ERROR on kbuild/for-next]
[also build test ERROR on linux/master linus/master v5.6-rc5 next-20200306] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-link-lib-y-objects-to-vmlinux-forcibly-when-CONFIG_MODULES-y/20200309-115312
base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
config: sparc-defconfig (attached as .config)
compiler: sparc-linux-gcc (GCC) 7.5.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.5.0 make.cross ARCH=sparc

If you fix the issue, kindly add following tag

All errors (new ones prefixed by >>):

arch/sparc/lib/strlen.o: In function `strlen':

>> (.text+0x0): multiple definition of `strlen'

lib/string.o:string.c:(.text+0x2b0): first defined here

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#2 lore raw
Quoted 21 lines · kbuild test robot

Hi Masahiro,

I love your patch! Yet something to improve:

[auto build test ERROR on kbuild/for-next]
[also build test ERROR on linux/master linus/master v5.6-rc5 next-20200306] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-link-lib-y-objects-to-vmlinux-forcibly-when-CONFIG_MODULES-y/20200309-115312
base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
config: sparc-defconfig (attached as .config)
compiler: sparc-linux-gcc (GCC) 7.5.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.5.0 make.cross ARCH=sparc

Quoted 8 lines

If you fix the issue, kindly add following tag

All errors (new ones prefixed by >>):

arch/sparc/lib/strlen.o: In function `strlen':

Quoted 1 line
Quoted 1 line

(.text+0x0): multiple definition of `strlen'

lib/string.o:string.c:(.text+0x2b0): first defined here

So this looks like a bug in 32b Sparc, that we should fix first.

arch/sparc/lib/strlen.S provides a definition of strlen, but it does not define the preprocessor token __HAVE_ARCH_STRLEN to avoid multiple definitions from producing a link error.

Or rather, __HAVE_ARCH_STRLEN is provided in include/asm/string_64.h, but not for 32b sparc. arch/sparc/strlen.o is unconditionally required in lib-y in arch/sparc/lib/Makefile.

Either:
1. arch/sparc/lib/strlen.S supports 32b sparc, then __HAVE_ARCH_STRLEN and a forward declaration of strlen should be provided in arch/sparc/include/asm/string.h rather than arch/sparc/include/asm/string_64.h, or...
2. arch/sparc/lib/strlen.S does not support 32b sparc, then the inclusion of strlen.o in arch/sparc/lib/Makefile should be predicated on CONFIG_SPARC64.

+ Dave who maybe can provide guidance on how to proceed? The use of the BRANCH32 macro in arch/sparc/lib/strlen.S seems to have different definitions based on CONFIG_SPARC64 vs CONFIG_SPARC32, which makes me thing it's case 1 above, but I'm not familiar with Sparc assembly to be confident.

Signature · 2 lines
Thanks,
~Nick Desaulniers
#3 lore raw

Hi Nick, David,

Quoted 51 lines · Nick Desaulniers
Quoted 21 lines · kbuild test robot

Hi Masahiro,

I love your patch! Yet something to improve:

[auto build test ERROR on kbuild/for-next]
[also build test ERROR on linux/master linus/master v5.6-rc5 next-20200306] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-link-lib-y-objects-to-vmlinux-forcibly-when-CONFIG_MODULES-y/20200309-115312
base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
config: sparc-defconfig (attached as .config)
compiler: sparc-linux-gcc (GCC) 7.5.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.5.0 make.cross ARCH=sparc

Quoted 8 lines

If you fix the issue, kindly add following tag

All errors (new ones prefixed by >>):

arch/sparc/lib/strlen.o: In function `strlen':

Quoted 1 line
Quoted 1 line

(.text+0x0): multiple definition of `strlen'

lib/string.o:string.c:(.text+0x2b0): first defined here

So this looks like a bug in 32b Sparc, that we should fix first.

arch/sparc/lib/strlen.S provides a definition of strlen, but it does not define the preprocessor token __HAVE_ARCH_STRLEN to avoid multiple definitions from producing a link error.

Or rather, __HAVE_ARCH_STRLEN is provided in include/asm/string_64.h, but not for 32b sparc. arch/sparc/strlen.o is unconditionally required in lib-y in arch/sparc/lib/Makefile.

Either:
1. arch/sparc/lib/strlen.S supports 32b sparc, then __HAVE_ARCH_STRLEN and a forward declaration of strlen should be provided in arch/sparc/include/asm/string.h rather than arch/sparc/include/asm/string_64.h, or...
2. arch/sparc/lib/strlen.S does not support 32b sparc, then the inclusion of strlen.o in arch/sparc/lib/Makefile should be predicated on CONFIG_SPARC64.

Quoted 5 lines

+ Dave who maybe can provide guidance on how to proceed? The use of the BRANCH32 macro in arch/sparc/lib/strlen.S seems to have different definitions based on CONFIG_SPARC64 vs CONFIG_SPARC32, which makes me thing it's case 1 above, but I'm not familiar with Sparc assembly to be confident.

I agree.
The former is probably the intention
according to this commit:

commit ae984d72e0632782dd98c3fcf469b9045ad0d449

Author: David S. Miller <[email protected]>
Date:   Tue Dec 9 01:07:09 2008 -0800

sparc: Unify strlen assembler.

Use the new asm/asm.h header to help commonize the strlen assembler between 32-bit and 64-bit

While we're here, use proper linux/linkage.h macros instead of by-hand stuff.

Signature · 2 lines
Best Regards
Masahiro Yamada