Skip to contentSkip to conversation

kbuild: revive parallel execution for .tmp_initcalls.lds rule

PATCH In review Masahiro Yamada 3 participants 4 messages Nov 14, 2022 – Nov 15, 2022 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kbuild: revive parallel execution for .tmp_initcalls.lds rule

Prior to commit 5d45950dfbb1 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o"), jobserver-exec was invoked from the shell script, link-vmlinux.sh. It can get access to the jobserver because Makefile adds '+' prefix, as in:

    +$(call if_changed_dep,link_vmlinux)

Since 5d45950dfbb1, jobserver-exec is invoked from Makefile, but the '+' prefix is missing, hence jobserver-exec has no access to the jobserver.

Fixes 5d45950dfbb1 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o")

---

 scripts/Makefile.vmlinux_o | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/Makefile.vmlinux_o +1 -1
index 0edfdb40364b..ae52d3b3f063 100644
@@ -19,7 +19,7 @@ quiet_cmd_gen_initcalls_lds = GEN     $@
 
 .tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \
 		vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
-	$(call if_changed,gen_initcalls_lds)
+	+$(call if_changed,gen_initcalls_lds)
 
 targets := .tmp_initcalls.lds
 
Signature · 1 line
2.34.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 7 lines · Masahiro Yamada

Prior to commit 5d45950dfbb1 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o"), jobserver-exec was invoked from the shell script, link-vmlinux.sh. It can get access to the jobserver because Makefile adds '+' prefix, as in:

    +$(call if_changed_dep,link_vmlinux)
Quoted 7 lines

Since 5d45950dfbb1, jobserver-exec is invoked from Makefile, but the '+' prefix is missing, hence jobserver-exec has no access to the jobserver.

Fixes 5d45950dfbb1 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o")

---

Quoted 19 lines
 scripts/Makefile.vmlinux_o | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/Makefile.vmlinux_o +1 -1
index 0edfdb40364b..ae52d3b3f063 100644
@@ -19,7 +19,7 @@ quiet_cmd_gen_initcalls_lds = GEN     $@
 
 .tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \
 		vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
-	$(call if_changed,gen_initcalls_lds)
+	+$(call if_changed,gen_initcalls_lds)
 
 targets := .tmp_initcalls.lds
 
Signature · 1 line
2.34.1
Signature · 3 lines
epost|xmpp: [email protected]          irc://oftc.net/nsc
↳ gpg: 18ed 52db e34f 860e e9fb  c82b 7d97 0932 55a0 ce7f
     -- frykten for herren er opphav til kunnskap --
maintainer Reviewed-by on v1 1/1
#2 lore raw
In reply to Masahiro Yamada
Quoted 5 lines · Masahiro Yamada

Prior to commit 5d45950dfbb1 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o"), jobserver-exec was invoked from the shell script, link-vmlinux.sh. It can get access to the jobserver because Makefile adds '+' prefix, as in:

Quoted 8 lines
    +$(call if_changed_dep,link_vmlinux)

Since 5d45950dfbb1, jobserver-exec is invoked from Makefile, but the '+' prefix is missing, hence jobserver-exec has no access to the jobserver.

Fixes 5d45950dfbb1 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o")

At least it doesn't seem like compile times were majorly affected. I benchmarked arm64 allmodconfig (worst case scenario with ThinLTO):

Benchmark 1: 094226ad94f4 ("Linux 6.1-rc5")
Time (mean ± σ): 899.036 s ± 1.133 s [User: 49314.495 s, System: 3840.796 s]
Range (min … max): 898.118 s … 900.302 s 3 runs

Benchmark 2: 0f45cbb5399b ("kbuild: revive parallel execution for .tmp_initcalls.lds rule")
Time (mean ± σ): 898.482 s ± 0.152 s [User: 49329.703 s, System: 3836.408 s]
Range (min … max): 898.306 s … 898.584 s 3 runs

Summary
'0f45cbb5399b ("kbuild: revive parallel execution for .tmp_initcalls.lds rule")' ran

    1.00 ± 0.00 times faster than '094226ad94f4 ("Linux 6.1-rc5")'
Quoted 21 lines

---

 scripts/Makefile.vmlinux_o | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/Makefile.vmlinux_o +1 -1
index 0edfdb40364b..ae52d3b3f063 100644
@@ -19,7 +19,7 @@ quiet_cmd_gen_initcalls_lds = GEN     $@
 
 .tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \
 		vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
-	$(call if_changed,gen_initcalls_lds)
+	+$(call if_changed,gen_initcalls_lds)
 
 targets := .tmp_initcalls.lds
 
Signature · 1 line
2.34.1
#3 lore raw
Quoted 13 lines · Nathan Chancellor
Quoted 11 lines · Masahiro Yamada

Prior to commit 5d45950dfbb1 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o"), jobserver-exec was invoked from the shell script, link-vmlinux.sh. It can get access to the jobserver because Makefile adds '+' prefix, as in:

    +$(call if_changed_dep,link_vmlinux)

Since 5d45950dfbb1, jobserver-exec is invoked from Makefile, but the '+' prefix is missing, hence jobserver-exec has no access to the jobserver.

Quoted 7 lines
Quoted 2 lines
Fixes 5d45950dfbb1 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o")

At least it doesn't seem like compile times were majorly affected. I benchmarked arm64 allmodconfig (worst case scenario with ThinLTO):

Ah, sorry.

./scripts/generate_initcall_order.pl always runs in two threads (the parent and a single child) because it is given only one argument (vmlinux.a).

So, using jobserver-exec is meaningless. Also, the code for parallel execution control in ./scripts/generate_initcall_order.pl is meaningless.

BTW, allmodconfig is not the worst case because most of the objects go into modules instead of vmlinux.

allyesconfig would add more symbols to vmlinux although I did not test it.

Quoted 29 lines

Benchmark 1: 094226ad94f4 ("Linux 6.1-rc5")
Time (mean ± σ): 899.036 s ± 1.133 s [User: 49314.495 s, System: 3840.796 s]
Range (min … max): 898.118 s … 900.302 s 3 runs

Benchmark 2: 0f45cbb5399b ("kbuild: revive parallel execution for .tmp_initcalls.lds rule")
Time (mean ± σ): 898.482 s ± 0.152 s [User: 49329.703 s, System: 3836.408 s]
Range (min … max): 898.306 s … 898.584 s 3 runs

Summary
'0f45cbb5399b ("kbuild: revive parallel execution for .tmp_initcalls.lds rule")' ran

    1.00 ± 0.00 times faster than '094226ad94f4 ("Linux 6.1-rc5")'
Quoted 16 lines

---

 scripts/Makefile.vmlinux_o | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/Makefile.vmlinux_o +1 -1
index 0edfdb40364b..ae52d3b3f063 100644
@@ -19,7 +19,7 @@ quiet_cmd_gen_initcalls_lds = GEN     $@
 
 .tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \
              vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
-     $(call if_changed,gen_initcalls_lds)
+     +$(call if_changed,gen_initcalls_lds)
 
Quoted 5 lines
Quoted 5 lines
scripts/Makefile.vmlinux_o +0 -1
 targets := .tmp_initcalls.lds
 
--

2.34.1

--
Best Regards

Masahiro Yamada