Skip to contentSkip to conversation

modpost: Fix secondary errors seen if a single module build fails

PATCH Superseded Guenter Roeck 3 participants 8 messages Sep 15, 2013 – Sep 25, 2013 Kbuild

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 modpost: Fix secondary errors seen if a single module build fails

Commit ea4054a23 (modpost: handle huge numbers of modules) added support for building a large number of modules.

Unfortunately, the commit changed the semantics of the makefile: Instead of passing only existing object files to modpost, make now passes all expected object files. If make was started with option -i, this results in a modpost error if a single file failed to build.

Example with the current btrfs build falure on m68k:

fs/btrfs/btrfs.o: No such file or directory
make[1]: [__modpost] Error 1 (ignored)

This error is followed by lots of errors such as:

m68k-linux-gcc: error: arch/m68k/emu/nfcon.mod.c: No such file or directory
m68k-linux-gcc: fatal error: no input files
compilation terminated.
make[1]: [arch/m68k/emu/nfcon.mod.o] Error 1 (ignored)

This doesn't matter much for normal builds, but it is annoying for builds started with "make -i" due to the large number of secondary errors. Those errors unnececessarily clog any error log and make it difficult to find the real errors in the build.

Fix the problem by only passing existing object files to modpost.

---
 scripts/Makefile.modpost |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
scripts/Makefile.modpost +2 -1
index 8dcdca2..387c806 100644
@@ -81,7 +81,8 @@ modpost = scripts/mod/modpost                    \
 
 # We can go over command line length here, so be careful.
 quiet_cmd_modpost = MODPOST $(words $(filter-out vmlinux FORCE, $^)) modules
-      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$$/.o/' | $(modpost) -s -T -
+      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$$/.o/' | \
+	while read a; do [ -f $$a ] && echo $$a; done | $(modpost) -s -T -
 
 PHONY += __modpost
 __modpost: $(modules:.ko=.o) FORCE
Signature · 1 line
1.7.9.7
#1 lore raw

Review

Quoted 3 lines · Guenter Roeck
(quoted diff) +2 -1
-      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$/.o/' | $(modpost) -s -T -
+      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$/.o/' | \
+       while read a; do [ -f $a ] && echo $a; done | $(modpost) -s -T -

I'm wondering whether this can be filtered without using a shell while loop? E.g. using "ls"?

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#2 lore raw
Quoted 8 lines · Geert Uytterhoeven

Review

Quoted 3 lines · Guenter Roeck
(quoted diff) +2 -1
-      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$/.o/' | $(modpost) -s -T -
+      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$/.o/' | \
+       while read a; do [ -f $a ] && echo $a; done | $(modpost) -s -T -

I'm wondering whether this can be filtered without using a shell while loop? E.g. using "ls"?

An alternative would be "$(MODLISTCMD) | sed 's/\.ko$/.o/' | xargs -r ls 2>/dev/null | ..."

I was a bit concerned about side effects of unexpected ls output. But I'll be happy to change it along that line if people think that it is better (and/or less ugly).

Guenter

#3 lore raw
In reply to Geert Uytterhoeven
Quoted 8 lines · Geert Uytterhoeven

Review

Quoted 3 lines · Guenter Roeck
(quoted diff) +2 -1
-      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$/.o/' | $(modpost) -s -T -
+      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$/.o/' | \
+       while read a; do [ -f $a ] && echo $a; done | $(modpost) -s -T -

I'm wondering whether this can be filtered without using a shell while loop? E.g. using "ls"?

I sent another version using 'ls'. Using 'find' would be possible as well, though both need an error redirect. Hope one of the versions is acceptable.

If anyone has a better idea, please let me know.

Thanks,
Guenter

#4 lore raw
In reply to Guenter Roeck

Dne 15.9.2013 06:39, Guenter Roeck napsal(a):

Quoted 38 lines

Commit ea4054a23 (modpost: handle huge numbers of modules) added support for building a large number of modules.

Unfortunately, the commit changed the semantics of the makefile: Instead of passing only existing object files to modpost, make now passes all expected object files. If make was started with option -i, this results in a modpost error if a single file failed to build.

Example with the current btrfs build falure on m68k:

fs/btrfs/btrfs.o: No such file or directory
make[1]: [__modpost] Error 1 (ignored)

This error is followed by lots of errors such as:

m68k-linux-gcc: error: arch/m68k/emu/nfcon.mod.c: No such file or directory
m68k-linux-gcc: fatal error: no input files
compilation terminated.
make[1]: [arch/m68k/emu/nfcon.mod.o] Error 1 (ignored)

This doesn't matter much for normal builds, but it is annoying for builds started with "make -i" due to the large number of secondary errors. Those errors unnececessarily clog any error log and make it difficult to find the real errors in the build.

Fix the problem by only passing existing object files to modpost.

---
 scripts/Makefile.modpost |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
scripts/Makefile.modpost +0 -0
index 8dcdca2..387c806 100644

scripts/Makefile.modpost modpost = scripts/mod/modpost \ · line 85

Quoted 6 lines
scripts/Makefile.modpost +2 -1
@@ -81,7 +81,8 @@ modpost = scripts/mod/modpost                    \
 
 # We can go over command line length here, so be careful.
 quiet_cmd_modpost = MODPOST $(words $(filter-out vmlinux FORCE, $^)) modules
-      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$$/.o/' | $(modpost) -s -T -
+      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$$/.o/' | \
+	while read a; do [ -f $$a ] && echo $$a; done | $(modpost) -s -T -

Can you do this filtering only if make -i is used ('i' is present in $(MAKEFLAGS)), to not hide potential buildsystem bugs? Regarding shell loop vs. ls, maybe the cleanest way would be to add an option to modpost to ignore missing files.

Thanks,
Michal

#5 lore raw
Quoted 40 lines · Michal Marek

Dne 15.9.2013 06:39, Guenter Roeck napsal(a):

Quoted 39 lines

Commit ea4054a23 (modpost: handle huge numbers of modules) added support for building a large number of modules.

Unfortunately, the commit changed the semantics of the makefile: Instead of passing only existing object files to modpost, make now passes all expected object files. If make was started with option -i, this results in a modpost error if a single file failed to build.

Example with the current btrfs build falure on m68k:

fs/btrfs/btrfs.o: No such file or directory
make[1]: [__modpost] Error 1 (ignored)

This error is followed by lots of errors such as:

m68k-linux-gcc: error: arch/m68k/emu/nfcon.mod.c: No such file or directory
m68k-linux-gcc: fatal error: no input files
compilation terminated.
make[1]: [arch/m68k/emu/nfcon.mod.o] Error 1 (ignored)

This doesn't matter much for normal builds, but it is annoying for builds started with "make -i" due to the large number of secondary errors. Those errors unnececessarily clog any error log and make it difficult to find the real errors in the build.

Fix the problem by only passing existing object files to modpost.

---
  scripts/Makefile.modpost |    3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
scripts/Makefile.modpost +0 -0
index 8dcdca2..387c806 100644
@@ -81,7 +81,8 @@ modpost = scripts/mod/modpost                    \
 
Quoted 11 lines

scripts/Makefile.modpost modpost = scripts/mod/modpost \ · line 85

Quoted 5 lines
scripts/Makefile.modpost +2 -1
  # We can go over command line length here, so be careful.
  quiet_cmd_modpost = MODPOST $(words $(filter-out vmlinux FORCE, $^)) modules
-      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$$/.o/' | $(modpost) -s -T -
+      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$$/.o/' | \
+	while read a; do [ -f $$a ] && echo $$a; done | $(modpost) -s -T -

Can you do this filtering only if make -i is used ('i' is present in $(MAKEFLAGS)), to not hide potential buildsystem bugs? Regarding shell loop vs. ls, maybe the cleanest way would be to add an option to modpost to ignore missing files.

Possibly, but I don't really see the point, as the rest of the makefile does and always did the same filtering already (using $(wildcard ...) and it doesn't make sense to even try to run modpost on a file that does not exist.

Sure, it might be a possibility to drop all the wildcard and other filtering conditionally and only keep it if -i was specified, but I think that should be a separate patch as it would add its own risks and complexities.

Guenter

#6 lore raw
In reply to Michal Marek
Quoted 40 lines · Michal Marek

Dne 15.9.2013 06:39, Guenter Roeck napsal(a):

Quoted 39 lines

Commit ea4054a23 (modpost: handle huge numbers of modules) added support for building a large number of modules.

Unfortunately, the commit changed the semantics of the makefile: Instead of passing only existing object files to modpost, make now passes all expected object files. If make was started with option -i, this results in a modpost error if a single file failed to build.

Example with the current btrfs build falure on m68k:

fs/btrfs/btrfs.o: No such file or directory
make[1]: [__modpost] Error 1 (ignored)

This error is followed by lots of errors such as:

m68k-linux-gcc: error: arch/m68k/emu/nfcon.mod.c: No such file or directory
m68k-linux-gcc: fatal error: no input files
compilation terminated.
make[1]: [arch/m68k/emu/nfcon.mod.o] Error 1 (ignored)

This doesn't matter much for normal builds, but it is annoying for builds started with "make -i" due to the large number of secondary errors. Those errors unnececessarily clog any error log and make it difficult to find the real errors in the build.

Fix the problem by only passing existing object files to modpost.

---
 scripts/Makefile.modpost |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
scripts/Makefile.modpost +0 -0
index 8dcdca2..387c806 100644
@@ -81,7 +81,8 @@ modpost = scripts/mod/modpost                    \
 
Quoted 11 lines

scripts/Makefile.modpost modpost = scripts/mod/modpost \ · line 85

Quoted 5 lines
scripts/Makefile.modpost +2 -1
 # We can go over command line length here, so be careful.
 quiet_cmd_modpost = MODPOST $(words $(filter-out vmlinux FORCE, $^)) modules
-      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$$/.o/' | $(modpost) -s -T -
+      cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$$/.o/' | \
+	while read a; do [ -f $$a ] && echo $$a; done | $(modpost) -s -T -

Can you do this filtering only if make -i is used ('i' is present in $(MAKEFLAGS)), to not hide potential buildsystem bugs? Regarding shell loop vs. ls, maybe the cleanest way would be to add an option to modpost to ignore missing files.

To follow up on this - are you at the Linux Plumbers conference ? If yes maybe we can meet briefly and discuss how to proceed. I'd volunteer to create a patch to add the above functionality if people think it is valuable, but I would like to keep it as separate patch.

Thanks,
Guenter

#7 lore raw
Quoted 5 lines · Guenter Roeck

To follow up on this - are you at the Linux Plumbers conference ? If yes maybe we can meet briefly and discuss how to proceed. I'd volunteer to create a patch to add the above functionality if people think it is valuable, but I would like to keep it as separate patch.

Sorry, I was not at the LPC and had vacation at that time. I see Rusty already merged your patch, that's great.

Michal