kbuild: possible regression?
Hi all,
I'm not quite sure if this might be a regression, but I recognized a change to kbuild's behaviour, which causes some of my build scripts to fail.
The build scripts do:
return system(('make -C %s O=%s ARCH=%s CROSS_COMPILE=%s '+
'oldconfig %s %s < /dev/null || exit %i') %
(srcdir,
builddir,
arch,
crosscompile,
target,
modules_target,
FAILED_RETCODE))which results in something like:
make -C /here/workdir/linux-2.6/common/src O=/here/workdir/linux-2.6/common/build/build \
ARCH=i386 CROSS_COMPILE=/opt/i686-linux/bin/i686-linux- \
oldconfig bzImage < /dev/null || exit 1
In the past, oldconfig was the first target, which has been handled.
Now, bzImage seems to get handled at first. That causes my build scripts to fail, because the bzImage target does a silentoldconfig, which doesn't allow console redirection.
[...]
Kernel log buffer size (16 => 64KB, 17 => 128KB) (LOG_BUF_SHIFT) [14] 14 Control Group support (CGROUPS) [N/y/?] (NEW) aborted!
Console input/output is redirected. Run 'make oldconfig' to update configuration.
make[5]: *** [silentoldconfig] Error 1
make[4]: *** [silentoldconfig] Error 2
make[3]: *** [include/config/auto.conf] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [/here/workdir/linux-2.6/common/src/scripts/Kbuild.include] Error 2
make: *** [sub-make] Error 2
make: Leaving directory `/here/workdir/linux-2.6/common/src'
[...]
After silentoldconfig has failed, oldconfig seems to get executed and after that the bzImage build is starting. So far so good, BUT: My script checks the return value and fails after the execution of silentoldconfig. That's why I recognized the different behaviour.
I did a git bisect to identify the commit, which caused the change to kbuild's behaviour. The offending commit is:
commit 0b35786d77ba4037f181982cc8ca20a7a3bf0fd2
Author: Milton Miller <[email protected]> Date: Fri Sep 21 18:09:02 2007 -0500
kbuild: call make once for all targets when O=.. is used
Change the invocations of make in the output directory Makefile and the main Makefile for separate object trees to pass all goals to one $(MAKE) via a new phony target "sub-make" and the existing target _all.
When compiling with separate object directories, a separate make is called in the context of another directory (from the output directory the main Makefile is called, the Makefile is then restarted with current directory set to the object tree). Before this patch, when multiple make command goals are specified, each target results in a separate make invocation. With make -j, these invocations may run in parallel, resulting in multiple commands running in the same directory clobbering each others results.
I did not try to address make -j for mixed dot-config and no-dot-config targets. Because the order does matter, a solution was not obvious. Perhaps a simple check for MAKEFLAGS having -j and refusing to run would be appropriate.
So, am I facing a kbuild regression?
Cheers,
Jan
Quoted 78 lines · Jan Altenberg
Hi all,
I'm not quite sure if this might be a regression, but I recognized a change to kbuild's behaviour, which causes some of my build scripts to fail.
The build scripts do:
return system(('make -C %s O=%s ARCH=%s CROSS_COMPILE=%s '+
'oldconfig %s %s < /dev/null || exit %i') % (srcdir, builddir, arch, crosscompile, target, modules_target, FAILED_RETCODE))which results in something like:
make -C /here/workdir/linux-2.6/common/src O=/here/workdir/linux-2.6/common/build/build \
ARCH=i386 CROSS_COMPILE=/opt/i686-linux/bin/i686-linux- \
oldconfig bzImage </dev/null|| exit 1In the past, oldconfig was the first target, which has been handled.
Now, bzImage seems to get handled at first. That causes my build scripts to fail, because the bzImage target does a silentoldconfig, which doesn't allow console redirection.
[...]
Kernel log buffer size (16 => 64KB, 17 => 128KB) (
LOG_BUF_SHIFT) [14] 14 Control Group support (CGROUPS) [N/y/?] (NEW) aborted!Console input/output is redirected. Run 'make oldconfig' to update configuration.
make[5]: *** [silentoldconfig] Error 1
make[4]: *** [silentoldconfig] Error 2
make[3]: *** [include/config/auto.conf] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [/here/workdir/linux-2.6/common/src/scripts/Kbuild.include] Error 2
make: *** [sub-make] Error 2
make: Leaving directory `/here/workdir/linux-2.6/common/src'[...]
After silentoldconfig has failed, oldconfig seems to get executed and after that the bzImage build is starting. So far so good, BUT: My script checks the return value and fails after the execution of silentoldconfig. That's why I recognized the different behaviour.
I did a git bisect to identify the commit, which caused the change to kbuild's behaviour. The offending commit is:
commit
0b35786d77ba4037f181982cc8ca20a7a3bf0fd2Author: Milton Miller <[email protected]> Date: Fri Sep 21 18:09:02 2007 -0500kbuild: call make once for all targets when O=.. is used
Change the invocations of make in the output directory
Makefileand the mainMakefilefor separate object trees to pass all goals to one $(MAKE) via a new phony target "sub-make" and the existing target_all.When compiling with separate object directories, a separate make is called in the context of another directory (from the output directory the main
Makefileis called, theMakefileis then restarted with current directory set to the object tree). Before this patch, when multiple make command goals are specified, each target results in a separate make invocation. With make -j, these invocations may run in parallel, resulting in multiple commands running in the same directory clobbering each others results.I did not try to address make -j for mixed dot-config and no-dot-config targets. Because the order does matter, a solution was not obvious. Perhaps a simple check for MAKEFLAGS having -j and refusing to run would be appropriate.
Quoted 4 lines
Signed-off-by Milton Miller <[email protected]>Signed-off-by Sam Ravnborg <[email protected]>So, am I facing a kbuild regression?
Yes - I will try to fix it during the weekend (if Milton does not beat me). Thanks for reporting and bisecting!
Sam
Hi Sam,
Quoted 23 lines
Quoted 23 lines
commit
0b35786d77ba4037f181982cc8ca20a7a3bf0fd2Author: Milton Miller <[email protected]> Date: Fri Sep 21 18:09:02 2007 -0500kbuild: call make once for all targets when O=.. is used
Change the invocations of make in the output directory
Makefileand the mainMakefilefor separate object trees to pass all goals to one $(MAKE) via a new phony target "sub-make" and the existing target_all.When compiling with separate object directories, a separate make is called in the context of another directory (from the output directory the main
Makefileis called, theMakefileis then restarted with current directory set to the object tree). Before this patch, when multiple make command goals are specified, each target results in a separate make invocation. With make -j, these invocations may run in parallel, resulting in multiple commands running in the same directory clobbering each others results.I did not try to address make -j for mixed dot-config and no-dot-config targets. Because the order does matter, a solution was not obvious. Perhaps a simple check for MAKEFLAGS having -j and refusing to run would be appropriate.
Quoted 7 lines
Quoted 4 lines
Signed-off-by Milton Miller <[email protected]>Signed-off-by Sam Ravnborg <[email protected]>So, am I facing a kbuild regression?
Yes - I will try to fix it during the weekend (if Milton does not beat me). Thanks for reporting and bisecting!
Have you made any progress on this? Let me know, if I can assist with testing.
Cheers,
Jan
Quoted 28 lines · Jan Altenberg
Hi Sam,
Quoted 26 lines
Quoted 26 lines
commit
0b35786d77ba4037f181982cc8ca20a7a3bf0fd2Author: Milton Miller <[email protected]> Date: Fri Sep 21 18:09:02 2007 -0500kbuild: call make once for all targets when O=.. is used
Change the invocations of make in the output directory
Makefileand the mainMakefilefor separate object trees to pass all goals to one $(MAKE) via a new phony target "sub-make" and the existing target_all.When compiling with separate object directories, a separate make is called in the context of another directory (from the output directory the main
Makefileis called, theMakefileis then restarted with current directory set to the object tree). Before this patch, when multiple make command goals are specified, each target results in a separate make invocation. With make -j, these invocations may run in parallel, resulting in multiple commands running in the same directory clobbering each others results.I did not try to address make -j for mixed dot-config and no-dot-config targets. Because the order does matter, a solution was not obvious. Perhaps a simple check for MAKEFLAGS having -j and refusing to run would be appropriate.
Signed-off-by Milton Miller <[email protected]>Signed-off-by Sam Ravnborg <[email protected]>
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
So, am I facing a kbuild regression?
Yes - I will try to fix it during the weekend (if Milton does not beat me). Thanks for reporting and bisecting!
Have you made any progress on this? Let me know, if I can assist with testing.
Hi Jan.
Not at all. My limited linux time goes into some x86 unification that I have given higher priority.
But your report is saved and I will return to it.
Sam
Quoted 28 lines · Jan Altenberg
Hi Sam,
Quoted 26 lines
Quoted 26 lines
commit
0b35786d77ba4037f181982cc8ca20a7a3bf0fd2Author: Milton Miller <[email protected]> Date: Fri Sep 21 18:09:02 2007 -0500kbuild: call make once for all targets when O=.. is used
Change the invocations of make in the output directory
Makefileand the mainMakefilefor separate object trees to pass all goals to one $(MAKE) via a new phony target "sub-make" and the existing target_all.When compiling with separate object directories, a separate make is called in the context of another directory (from the output directory the main
Makefileis called, theMakefileis then restarted with current directory set to the object tree). Before this patch, when multiple make command goals are specified, each target results in a separate make invocation. With make -j, these invocations may run in parallel, resulting in multiple commands running in the same directory clobbering each others results.I did not try to address make -j for mixed dot-config and no-dot-config targets. Because the order does matter, a solution was not obvious. Perhaps a simple check for MAKEFLAGS having -j and refusing to run would be appropriate.
Signed-off-by Milton Miller <[email protected]>Signed-off-by Sam Ravnborg <[email protected]>
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
So, am I facing a kbuild regression?
Yes - I will try to fix it during the weekend (if Milton does not beat me). Thanks for reporting and bisecting!
Have you made any progress on this? Let me know, if I can assist with testing.
Following seems to fix it with my limited testing.
I did an allnoconfig and then deleted CONFIG_LOGSHIFT from .config. It failed before and succeeded after.
Please test and report back.
Sam
Makefile +6 -5
@@ -108,6 +108,9 @@ endif PHONY := _all _all: +# Cancel implicit rules on top Makefile. +$(CURDIR)/Makefile Makefile: ; + ifneq ($(KBUILD_OUTPUT),) # Invoke a second make in the output directory, passing relevant variables # check that the output directory actually exists
@@ -121,7 +124,7 @@ $(if $(filter-out $(KBUILD_OUTPUT),$(shell /bin/pwd)),, \ PHONY += $(MAKECMDGOALS) sub-make -$(filter-out _all sub-make,$(MAKECMDGOALS)) _all: sub-make +$(filter-out _all sub-make $(CURDIR)/Makefile, $(MAKECMDGOALS)) _all: sub-make $(Q)@: sub-make: FORCE
@@ -293,7 +296,8 @@ export quiet Q KBUILD_VERBOSE # Look for make include files relative to root of kernel src MAKEFLAGS += --include-dir=$(srctree) -# We need some generic definitions. +# We need some generic definitions (do not try to remake the file). +$(srctree)/scripts/Kbuild.include: ; include $(srctree)/scripts/Kbuild.include # Make variables (CC, etc...)
@@ -1567,9 +1571,6 @@ endif # skip-makefile PHONY += FORCE FORCE: -# Cancel implicit rules on top Makefile, `-rR' will apply to sub-makes. -Makefile: ; - # Declare the contents of the .PHONY variable as phony. We keep that # information in a variable se we can use it in if_changed and friends. .PHONY: $(PHONY)
Hi Sam,
sorry for the delayed feedback. Still suffering a bad stomach flu...
Quoted 1 line
Please test and report back.
Thanks - Your Patch seems to fix things for me!
Cheers,
Jan
Quoted 7 lines · Jan Altenberg
Hi Sam,
sorry for the delayed feedback. Still suffering a bad stomach flu...
Quoted 1 line
Please test and report back.
Thanks - Your Patch seems to fix things for me!
Thanks Jan, hope you recover soon.
Sam