Skip to contentSkip to conversation

kbuild: drop use of "ld -r" for intermediate links of vmlinux

RFC In review Sam Ravnborg 2 participants 5 messages Feb 16, 2014 – Feb 20, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC 1/1 kbuild: drop use of "ld -r" for intermediate links of vmlinux

The approach today is that all files that ends up in vmlinux are linked directory by directory using "ld -r".

The following patch moves away from all the intermediate links and do one huge link of all the .o files used for the final vmlinux.

The trick used is that for each directory a linker script is generated that name all the file to be linked - and for sub-directories this is just the linker script for that directory.

The file continue to be named "built-in.o" but it is really a linker script.

A limitation using linker scripts like this is that this introduces a maximum depth of 10 levels (ld limitation when using INPUT). And the final link will for a kernel likely involves > 1000 .o files.

For now this is an experimental path - and I have not even tried to boot the kernel (as this is not simple in my current setup).

Feedback from the binutils people would be great if this is actually a good way to get rid of the use of "ld -r" in the kernel. Copied a few binutils people to hopefully get some feedback.

A typical file looks like this:

$cat kernel/built-in.o
INPUT(kernel/fork.o kernel/exec_domain.o ... kernel/locking/built-in.o ...)

kernel/locking/built-in.o is another linker script that again reference some .o files.

The background for this patch was that the LTO patch-kit that Andi Kleen is working on referenced a thread in binutils mailing list that complained about the use of "ld -r" in the kernel build and that LTO support for "ld -r" builds would likely not be accepted in binutils.

I had woked on the following some years ago and recreated the patch. I recall the idea is from someone else - but have long lost memory of from where I saw it.

Sam

lib/Kconfig.debug +0 -8
index a48abea..b6ca48a 100644
@@ -249,14 +249,6 @@ config DEBUG_SECTION_MISMATCH
 	    the analysis would not catch the illegal reference.
 	    This option tells gcc to inline less (but it does result in
 	    a larger kernel).
-	  - Run the section mismatch analysis for each module/built-in.o file.
-	    When we run the section mismatch analysis on vmlinux.o, we
-	    lose valueble information about where the mismatch was
-	    introduced.
-	    Running the analysis for each module/built-in.o file
-	    tells where the mismatch happens much closer to the
-	    source. The drawback is that the same mismatch is
-	    reported at least twice.
 	  - Enable verbose reporting from modpost in order to help resolve
 	    the section mismatches that are reported.
 
scripts/Makefile.build +5 -11
index d5d859c..a568276 100644
@@ -158,11 +158,6 @@ ifneq ($(KBUILD_CHECKSRC),0)
   endif
 endif
 
-# Do section mismatch analysis for each module/built-in.o
-ifdef CONFIG_DEBUG_SECTION_MISMATCH
-  cmd_secanalysis = ; scripts/mod/modpost $@
-endif
-
 # Compile C sources (.c)
 # ---------------------------------------------------------------------------
 
@@ -378,10 +373,9 @@ $(sort $(subdir-obj-y)): $(subdir-ym) ;
 ifdef builtin-target
 quiet_cmd_link_o_target = LD      $@
 # If the list of objects to link is empty, just create an empty built-in.o
-cmd_link_o_target = $(if $(strip $(obj-y)),\
-		      $(LD) $(ld_flags) -r -o $@ $(filter $(obj-y), $^) \
-		      $(cmd_secanalysis),\
-		      rm -f $@; $(AR) rcs$(KBUILD_ARFLAGS) $@)
+cmd_link_o_target = $(if $(filter $(obj-y), $^),                   \
+			echo INPUT\($(filter $(obj-y), $^)\) > $@, \
+			echo "/* empty */" > $@)
 
 $(builtin-target): $(obj-y) FORCE
 	$(call if_changed,link_o_target)
@@ -428,10 +422,10 @@ $($(subst $(obj)/,,$(@:.o=-objs)))    \
 $($(subst $(obj)/,,$(@:.o=-y)))), $^)
  
 quiet_cmd_link_multi-y = LD      $@
-cmd_link_multi-y = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps) $(cmd_secanalysis)
+cmd_link_multi-y = echo INPUT\($(link_multi_deps)\) > $@
 
 quiet_cmd_link_multi-m = LD [M]  $@
-cmd_link_multi-m = $(cmd_link_multi-y)
+cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps)
 
 # We would rather have a list of rules like
 # 	foo.o: $(foo-objs)
#1 lore raw
Quoted 5 lines · Sam Ravnborg

The trick used is that for each directory a linker script is generated that name all the file to be linked - and for sub-directories this is just the linker script for that directory.

The file continue to be named "built-in.o" but it is really a linker script.

You'll need binutils newer than 2012-06-25
https://sourceware.org/ml/binutils/2012-06/msg00201.html

Quoted 2 lines

I recall the idea is from someone else - but have long lost memory of from where I saw it.

I think it might have been my idea originally, implemented by Stephen Rothwell.

Another possibility is to use thin archives to package up the objects. Thin archives are like normal archives except they just contain paths to the objects, not the object contents. Paths are flattened when adding one thin archive to another. I think Stephen may have tried that idea too.

Signature · 2 lines
Alan Modra
Australia Development Lab, IBM
#2 lore raw
Quoted 7 lines · Alan Modra
Quoted 5 lines · Sam Ravnborg

The trick used is that for each directory a linker script is generated that name all the file to be linked - and for sub-directories this is just the linker script for that directory.

The file continue to be named "built-in.o" but it is really a linker script.

Quoted 8 lines

You'll need binutils newer than 2012-06-25
https://sourceware.org/ml/binutils/2012-06/msg00201.html

Quoted 2 lines

I recall the idea is from someone else - but have long lost memory of from where I saw it.

I think it might have been my idea originally, implemented by Stephen Rothwell.

Sounds right - added in cc:

Quoted 6 lines

Another possibility is to use thin archives to package up the objects. Thin archives are like normal archives except they just contain paths to the objects, not the object contents. Paths are flattened when adding one thin archive to another. I think Stephen may have tried that idea too.

I just coded this in a similar simple fashion.
It broke building vdso - needs to look into that and will get back when I may understand why my link suddenly broke.

Thanks for the feedback!

Sam

#3 lore raw
Quoted 8 lines · Sam Ravnborg
Quoted 5 lines

Another possibility is to use thin archives to package up the objects. Thin archives are like normal archives except they just contain paths to the objects, not the object contents. Paths are flattened when adding one thin archive to another. I think Stephen may have tried that idea too.

I just coded this in a similar simple fashion.
It broke building vdso - needs to look into that and will get back when I may understand why my link suddenly broke.

Missing --whole-archive?

Signature · 2 lines
Alan Modra
Australia Development Lab, IBM
#4 lore raw
Quoted 11 lines · Alan Modra
Quoted 8 lines · Sam Ravnborg
Quoted 5 lines

Another possibility is to use thin archives to package up the objects. Thin archives are like normal archives except they just contain paths to the objects, not the object contents. Paths are flattened when adding one thin archive to another. I think Stephen may have tried that idea too.

I just coded this in a similar simple fashion.
It broke building vdso - needs to look into that and will get back when I may understand why my link suddenly broke.

Missing --whole-archive?

No - at least something else to begin with.
A special liker script is used when building the vdsp files for x86 - and the linker script is ignored as such when embedded in a .o file with ar it seems. This is not a big suprise.

The makefile magic used for the vdso part of the kernel needs a bit more coffee than what is good for me for now...
Will look at it again later.

Thanks for the feedback.

Sam