Skip to contentSkip to conversation

make -j12 all fails in uImage target

PATCH In review Olaf Hering 4 participants 12 messages Feb 6, 2005 – Jul 12, 2005 Kbuild

lore
Share & tools
lore

Olaf Hering

author
#0 lore raw

this is what I got with 2.6.11rc3:

make ARCH=ppc O=../O-2.6.11-rc3-b50-SMP -j12 all
...

  LD      vmlinux
  SYSMAP  System.map
  SYSMAP  .tmp_System.map
  OBJCOPY arch/ppc/boot/images/vmlinux.bin
  HOSTCC  arch/ppc/boot/utils/addnote
  HOSTCC  arch/ppc/boot/utils/mknote
  HOSTCC  arch/ppc/boot/utils/mkprep
  LD      arch/ppc/boot/lib/built-in.o
  HOSTCC  arch/ppc/boot/utils/hack-coff
  HOSTCC  arch/ppc/boot/utils/mkbugboot

OBJCOPY arch/ppc/boot/images/vmlinux.bin
GZIP arch/ppc/boot/images/vmlinux.gz
/bin/sh: line 1: arch/ppc/boot/images/vmlinux.bin: No such file or directory make[2]: *** [arch/ppc/boot/images/vmlinux.gz] Error 1
make[1]: *** [uImage] Error 2
make[1]: *** Waiting for unfinished jobs....

also, one time that zImage.chrp was only 570K instead of the expected 1.4M, so netboot failed. But I got no build error.
Any idea what dependency is missing?

#1 lore raw
Quoted 22 lines · Olaf Hering

this is what I got with 2.6.11rc3:

make ARCH=ppc O=../O-2.6.11-rc3-b50-SMP -j12 all
...

  LD      vmlinux
  SYSMAP  System.map
  SYSMAP  .tmp_System.map
  OBJCOPY arch/ppc/boot/images/vmlinux.bin
  HOSTCC  arch/ppc/boot/utils/addnote
  HOSTCC  arch/ppc/boot/utils/mknote
  HOSTCC  arch/ppc/boot/utils/mkprep
  LD      arch/ppc/boot/lib/built-in.o
  HOSTCC  arch/ppc/boot/utils/hack-coff
  HOSTCC  arch/ppc/boot/utils/mkbugboot

OBJCOPY arch/ppc/boot/images/vmlinux.bin
GZIP arch/ppc/boot/images/vmlinux.gz
/bin/sh: line 1: arch/ppc/boot/images/vmlinux.bin: No such file or directory make[2]: *** [arch/ppc/boot/images/vmlinux.gz] Error 1
make[1]: *** [uImage] Error 2
make[1]: *** Waiting for unfinished jobs....

Quoted 3 lines

also, one time that zImage.chrp was only 570K instead of the expected 1.4M, so netboot failed. But I got no build error.
Any idea what dependency is missing?

That is kinda odd. My thought is that arch/ppc/boot/ just isn't fully safe for -j'ing. Sam, any ideas on how to debug this kinda problem? Thanks.

Signature · 2 lines
Tom Rini
http://gate.crashing.org/~trini/
author
#2 lore raw
Quoted 3 lines · Tom Rini
Quoted 3 lines

GZIP arch/ppc/boot/images/vmlinux.gz

Any idea what dependency is missing?

Sometimes I see that GZIP twice.

#3 lore raw
In reply to Tom Rini
Quoted 23 lines · Tom Rini
Quoted 22 lines · Olaf Hering

this is what I got with 2.6.11rc3:

make ARCH=ppc O=../O-2.6.11-rc3-b50-SMP -j12 all
...

  LD      vmlinux
  SYSMAP  System.map
  SYSMAP  .tmp_System.map
  OBJCOPY arch/ppc/boot/images/vmlinux.bin
  HOSTCC  arch/ppc/boot/utils/addnote
  HOSTCC  arch/ppc/boot/utils/mknote
  HOSTCC  arch/ppc/boot/utils/mkprep
  LD      arch/ppc/boot/lib/built-in.o
  HOSTCC  arch/ppc/boot/utils/hack-coff
  HOSTCC  arch/ppc/boot/utils/mkbugboot

OBJCOPY arch/ppc/boot/images/vmlinux.bin
GZIP arch/ppc/boot/images/vmlinux.gz
/bin/sh: line 1: arch/ppc/boot/images/vmlinux.bin: No such file or directory make[2]: *** [arch/ppc/boot/images/vmlinux.gz] Error 1
make[1]: *** [uImage] Error 2
make[1]: *** Waiting for unfinished jobs....

Quoted 6 lines
Quoted 3 lines

also, one time that zImage.chrp was only 570K instead of the expected 1.4M, so netboot failed. But I got no build error.
Any idea what dependency is missing?

That is kinda odd. My thought is that arch/ppc/boot/ just isn't fully safe for -j'ing. Sam, any ideas on how to debug this kinda problem?

The problem is that the images/ sub directory is visited before the prerequisites are finished.
I have no good way to debug this - I can see it based on the Makefile and the output Olaf included.

Also the problem Olaf describe is just a cp of a half finished file.

So the fix is to let the images/ directory depends on the rest of the directories.

Something like this may do the trick:

$(obj)/images: $(addprefix $(obj)/,$(subdir-y) $(bootdir-y))

	$(Q)$(MAKE) $(build)=$@

And then delete the assignment of images to subdir-y

Another solution would be to get rid on the images/ drectory and place outputfiles where they are being built.

Sam

#4 lore raw
Quoted 6 lines · Sam Ravnborg

The problem is that the images/ sub directory is visited before the prerequisites are finished.
I have no good way to debug this - I can see it based on the Makefile and the output Olaf included.

Also the problem Olaf describe is just a cp of a half finished file.

We had a similar problem with paul recently where the build worked but the resulting zImage contained a corrupted file as the objcopy/gzip/whatever wasn't fully finished when the zImage file got linked.

Quoted 5 lines

So the fix is to let the images/ directory depends on the rest of the directories.

Something like this may do the trick:

We should maybe also have an explicit dependency on vmlinux no ? Or is this handled already for the boot dir as a whole ?

Quoted 14 lines

$(obj)/images: $(addprefix $(obj)/,$(subdir-y) $(bootdir-y))

	$(Q)$(MAKE) $(build)=$@

And then delete the assignment of images to subdir-y

Another solution would be to get rid on the images/ drectory and place outputfiles where they are being built.

Sam
_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-dev

Signature · 1 line
Benjamin Herrenschmidt <[email protected]>
#5 lore raw
Quoted 3 lines

We should maybe also have an explicit dependency on vmlinux no ? Or is this handled already for the boot dir as a whole ?

This is handled in arch/ppc/MAkefile - so already dealt with.

Sam

#6 lore raw
In reply to Sam Ravnborg
Quoted 44 lines · Sam Ravnborg
Quoted 23 lines · Tom Rini
Quoted 22 lines · Olaf Hering

this is what I got with 2.6.11rc3:

make ARCH=ppc O=../O-2.6.11-rc3-b50-SMP -j12 all
...

  LD      vmlinux
  SYSMAP  System.map
  SYSMAP  .tmp_System.map
  OBJCOPY arch/ppc/boot/images/vmlinux.bin
  HOSTCC  arch/ppc/boot/utils/addnote
  HOSTCC  arch/ppc/boot/utils/mknote
  HOSTCC  arch/ppc/boot/utils/mkprep
  LD      arch/ppc/boot/lib/built-in.o
  HOSTCC  arch/ppc/boot/utils/hack-coff
  HOSTCC  arch/ppc/boot/utils/mkbugboot

OBJCOPY arch/ppc/boot/images/vmlinux.bin
GZIP arch/ppc/boot/images/vmlinux.gz
/bin/sh: line 1: arch/ppc/boot/images/vmlinux.bin: No such file or directory make[2]: *** [arch/ppc/boot/images/vmlinux.gz] Error 1
make[1]: *** [uImage] Error 2
make[1]: *** Waiting for unfinished jobs....

Quoted 6 lines
Quoted 3 lines

also, one time that zImage.chrp was only 570K instead of the expected 1.4M, so netboot failed. But I got no build error.
Any idea what dependency is missing?

That is kinda odd. My thought is that arch/ppc/boot/ just isn't fully safe for -j'ing. Sam, any ideas on how to debug this kinda problem?

The problem is that the images/ sub directory is visited before the prerequisites are finished.
I have no good way to debug this - I can see it based on the Makefile and the output Olaf included.

Also the problem Olaf describe is just a cp of a half finished file.

So the fix is to let the images/ directory depends on the rest of the directories.

Something like this may do the trick:

Quoted 8 lines

$(obj)/images: $(addprefix $(obj)/,$(subdir-y) $(bootdir-y))

	$(Q)$(MAKE) $(build)=$@

And then delete the assignment of images to subdir-y

Another solution would be to get rid on the images/ drectory and place outputfiles where they are being built.

Hopefully the first works, since I really do like having all the various images we create end up in one spot. :)

Signature · 2 lines
Tom Rini
http://gate.crashing.org/~trini/
author
#7 lore raw
In reply to Sam Ravnborg
Quoted 6 lines · Sam Ravnborg

Something like this may do the trick:

$(obj)/images: $(addprefix $(obj)/,$(subdir-y) $(bootdir-y))

	$(Q)$(MAKE) $(build)=$@

And then delete the assignment of images to subdir-y

That still doesnt work right, I touched one .c file which lead to a new vmlinux, now zImage.rs6k is too small:

arch/ppc/boot/Makefile 2005-02-07 09:24:38.713179331 +0100 renamed from linux-2.6.11-rc3.orig/arch/ppc/boot/Makefile 2005-02-03 02:56:10.000000000 +0100 +3 -1
diff -purNx tags ../linux-2.6.11-rc3.orig/arch/ppc/boot/Makefile ./arch/ppc/boot/Makefile
@@ -17,8 +17,10 @@ BOOT_TARGETS = zImage zImage.initrd znet
 
 bootdir-y                      := simple
 bootdir-$(CONFIG_PPC_OF)       += openfirmware
-subdir-y                       := lib common images
+subdir-y                       := lib common
 subdir-$(CONFIG_PPC_OF)                += of1275
+$(obj)/images: $(addprefix $(obj)/,$(subdir-y) $(bootdir-y))
+       $(Q)$(MAKE) $(build)=$@
 
 # for cleaning
 subdir-                                += simple openfirmware

CHK include/linux/version.h
make[2]: `arch/ppc/kernel/asm-offsets.s' is up to date.

  CHK     include/asm-ppc/offsets.h
  CHK     usr/initramfs_list
  CHK     include/linux/compile.h
  CC      arch/ppc/platforms/chrp_nvram.o
  LD      arch/ppc/platforms/built-in.o
  GEN     .version
  CHK     include/linux/compile.h
  UPD     include/linux/compile.h
  CC      init/version.o
  LD      init/built-in.o
  LD      .tmp_vmlinux1
  KSYM    .tmp_kallsyms1.S
  AS      .tmp_kallsyms1.o
  LD      .tmp_vmlinux2
  KSYM    .tmp_kallsyms2.S
  AS      .tmp_kallsyms2.o
  LD      vmlinux
  SYSMAP  System.map
  SYSMAP  .tmp_System.map
  OBJCOPY arch/ppc/boot/images/vmlinux.bin
  GZIP    arch/ppc/boot/images/vmlinux.gz
  AS      arch/ppc/boot/simple/head.o
  AS      arch/ppc/boot/simple/relocate.o
  CC      arch/ppc/boot/simple/prepmap.o
  GEN     arch/ppc/boot/openfirmware/image.o
  CC      arch/ppc/boot/simple/misc.o
  GEN     arch/ppc/boot/images/miboot.image
  COFF    arch/ppc/boot/openfirmware/coffboot
  ELF     arch/ppc/boot/images/vmlinux.elf-pmac
  CHRP    arch/ppc/boot/images/zImage.chrp
  CC      arch/ppc/boot/simple/misc-prep.o
  CC      arch/ppc/boot/simple/mpc10x_memory.o
  COFF    arch/ppc/boot/images/vmlinux.coff

ADDNOTE arch/ppc/boot/images/zImage.chrp-rs6k
kernel: zImage is ready (arch/ppc/boot/images/vmlinux.coff)
...
../O-2.6.11-rc3-b50/arch/ppc/boot/images/:
total 6.2M

drwxr-xr-x  2 olaf users  496 2005-02-07 09:26 ./   
drwxr-xr-x  9 olaf users  224 2005-02-07 09:25 ../  
-rw-r--r--  1 olaf users  256 2005-02-07 09:26 .uImage.cmd
-rw-r--r--  1 olaf users  100 2005-02-07 09:26 .vmlinux.bin.cmd
-rw-r--r--  1 olaf users  119 2005-02-07 09:26 .vmlinux.gz.cmd
-rw-r--r--  1 olaf users  49K 2005-02-07 09:26 miboot.image
-rwxr-xr-x  1 olaf users 2.8M 2005-02-07 09:26 vmlinux.bin*
-rwxr-xr-x  1 olaf users  79K 2005-02-07 09:26 vmlinux.coff*
-rwxr-xr-x  1 olaf users 146K 2005-02-07 09:26 vmlinux.elf-pmac*
-rw-r--r--  1 olaf users 1.3M 2005-02-07 09:26 vmlinux.gz
-rwxr-xr-x  1 olaf users 575K 2005-02-07 09:26 zImage.bugboot*
-rwxr-xr-x  1 olaf users 146K 2005-02-07 09:26 zImage.chrp*
-rw-r--r--  1 olaf users 146K 2005-02-07 09:26 zImage.chrp-rs6k
-rwxr-xr-x  1 olaf users 639K 2005-02-07 09:26 zImage.elf*
lrwxrwxrwx  1 olaf users   12 2005-02-07 09:26 zImage.pmac -> vmlinux.coff*
-rwxr-xr-x  1 olaf users 576K 2005-02-07 09:26 zImage.prep*
#8 lore raw
Quoted 11 lines · Olaf Hering
Quoted 6 lines · Sam Ravnborg

Something like this may do the trick:

$(obj)/images: $(addprefix $(obj)/,$(subdir-y) $(bootdir-y))

	$(Q)$(MAKE) $(build)=$@

And then delete the assignment of images to subdir-y

That still doesnt work right, I touched one .c file which lead to a new vmlinux, now zImage.rs6k is too small:

Is it possible that the dependency is being met by the file being there, but not done? If so, would perhaps depending on System.map as well for vmlinux.bin fix the problem?

This doesn't make sense to me, but taking a quick look over things, zImage.chrp (/etc) do have dependncies on vmlinux.gz which has deps on vmlinux.bin, which deps on vmlinux.

Signature · 2 lines
Tom Rini
http://gate.crashing.org/~trini/
author
#9 lore raw
In reply to Sam Ravnborg
Quoted 29 lines · Sam Ravnborg
Quoted 8 lines · Tom Rini
Quoted 6 lines · Olaf Hering

GZIP arch/ppc/boot/images/vmlinux.gz
/bin/sh: line 1: arch/ppc/boot/images/vmlinux.bin: No such file or directory make[2]: *** [arch/ppc/boot/images/vmlinux.gz] Error 1
make[1]: *** [uImage] Error 2
make[1]: *** Waiting for unfinished jobs....

Quoted 6 lines
Quoted 3 lines

also, one time that zImage.chrp was only 570K instead of the expected 1.4M, so netboot failed. But I got no build error.
Any idea what dependency is missing?

That is kinda odd. My thought is that arch/ppc/boot/ just isn't fully safe for -j'ing. Sam, any ideas on how to debug this kinda problem?

The problem is that the images/ sub directory is visited before the prerequisites are finished.
I have no good way to debug this - I can see it based on the Makefile and the output Olaf included.

Also the problem Olaf describe is just a cp of a half finished file.

So the fix is to let the images/ directory depends on the rest of the directories.

Something like this may do the trick:

Quoted 8 lines

$(obj)/images: $(addprefix $(obj)/,$(subdir-y) $(bootdir-y))

	$(Q)$(MAKE) $(build)=$@

And then delete the assignment of images to subdir-y

Another solution would be to get rid on the images/ drectory and place outputfiles where they are being built.

I did it this way:

...

  AS      arch/ppc/boot/common/util.o
  CC      arch/ppc/boot/of1275/write.o
  AR      arch/ppc/boot/common/lib.a
  AR      arch/ppc/boot/of1275/lib.a

arch/ppc/boot/images/vmlinux.gz.27631' -> arch/ppc/boot/images/vmlinux.gz'
AR arch/ppc/boot/lib/lib.a
UIMAGE arch/ppc/boot/images/uImage
"mkimage" command not found - U-Boot images will not be built
Image: arch/ppc/boot/images/uImage not made
arch/ppc/boot/images/vmlinux.gz.27676' -> arch/ppc/boot/images/vmlinux.gz'

  CC      arch/ppc/boot/openfirmware/dummy.o
  AS      arch/ppc/boot/simple/head.o
  AS      arch/ppc/boot/simple/relocate.o

...

scripts/Makefile.lib +2 -1
Index: linux-2.6.12-rc5-olh/scripts/Makefile.lib
===================================================================
@@ -1,3 +1,4 @@
+# vim: syntax=sh
 # ===========================================================================
 # kbuild: Generic definitions
 # ===========================================================================

@@ -177,7 +178,7 @@ cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS)
 # ---------------------------------------------------------------------------
 
 quiet_cmd_gzip = GZIP    $@
-cmd_gzip = gzip -f -9 < $< > $@
+cmd_gzip = gzip -f -9 < $< > $@.$$$$ ; mv -v $@.$$$$ $@
 
 # ===========================================================================
 # Generic stuff
author
#10 lore raw

PATCH 1/1 make -j12 all fails in uImage target

In reply to Olaf Hering
Quoted 22 lines · Olaf Hering

this is what I got with 2.6.11rc3:

make ARCH=ppc O=../O-2.6.11-rc3-b50-SMP -j12 all
...

  LD      vmlinux
  SYSMAP  System.map
  SYSMAP  .tmp_System.map
  OBJCOPY arch/ppc/boot/images/vmlinux.bin
  HOSTCC  arch/ppc/boot/utils/addnote
  HOSTCC  arch/ppc/boot/utils/mknote
  HOSTCC  arch/ppc/boot/utils/mkprep
  LD      arch/ppc/boot/lib/built-in.o
  HOSTCC  arch/ppc/boot/utils/hack-coff
  HOSTCC  arch/ppc/boot/utils/mkbugboot

OBJCOPY arch/ppc/boot/images/vmlinux.bin
GZIP arch/ppc/boot/images/vmlinux.gz
/bin/sh: line 1: arch/ppc/boot/images/vmlinux.bin: No such file or directory make[2]: *** [arch/ppc/boot/images/vmlinux.gz] Error 1
make[1]: *** [uImage] Error 2
make[1]: *** Waiting for unfinished jobs....

Quoted 2 lines

also, one time that zImage.chrp was only 570K instead of the expected 1.4M, so netboot failed. But I got no build error.

make -j zImage may call if_changed twice at the same time, the result is a corrupted vmlinux.gz
write to a temporary file for the time being until someone with make skills fix the serialization properly.

Signed-off-by Olaf Hering <[email protected]>
 arch/ppc/boot/images/Makefile |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletion(-)
arch/ppc/boot/images/Makefile +6 -1
Index: linux-2.6.13-rc2-olh/arch/ppc/boot/images/Makefile
===================================================================
@@ -6,12 +6,17 @@ MKIMAGE		:= $(srctree)/scripts/mkuboot.s
 
 extra-y		:= vmlinux.bin vmlinux.gz
 
+# two make processes may write to vmlinux.gz at the same time with make -j
+quiet_cmd_mygzip = GZIP    $@
+cmd_mygzip = gzip -f -9 < $< > $@.$$$$ && mv $@.$$$$ $@
+
+
 OBJCOPYFLAGS_vmlinux.bin := -O binary
 $(obj)/vmlinux.bin: vmlinux FORCE
 	$(call if_changed,objcopy)
 
 $(obj)/vmlinux.gz: $(obj)/vmlinux.bin FORCE
-	$(call if_changed,gzip)
+	$(call if_changed,mygzip)
 
 quiet_cmd_uimage = UIMAGE  $@
       cmd_uimage = $(CONFIG_SHELL) $(MKIMAGE) -A ppc -O linux -T kernel \
#11 lore raw
Quoted 28 lines · Olaf Hering
Quoted 22 lines · Olaf Hering

this is what I got with 2.6.11rc3:

make ARCH=ppc O=../O-2.6.11-rc3-b50-SMP -j12 all
...

  LD      vmlinux
  SYSMAP  System.map
  SYSMAP  .tmp_System.map
  OBJCOPY arch/ppc/boot/images/vmlinux.bin
  HOSTCC  arch/ppc/boot/utils/addnote
  HOSTCC  arch/ppc/boot/utils/mknote
  HOSTCC  arch/ppc/boot/utils/mkprep
  LD      arch/ppc/boot/lib/built-in.o
  HOSTCC  arch/ppc/boot/utils/hack-coff
  HOSTCC  arch/ppc/boot/utils/mkbugboot

OBJCOPY arch/ppc/boot/images/vmlinux.bin
GZIP arch/ppc/boot/images/vmlinux.gz
/bin/sh: line 1: arch/ppc/boot/images/vmlinux.bin: No such file or directory make[2]: *** [arch/ppc/boot/images/vmlinux.gz] Error 1
make[1]: *** [uImage] Error 2
make[1]: *** Waiting for unfinished jobs....

Quoted 2 lines

also, one time that zImage.chrp was only 570K instead of the expected 1.4M, so netboot failed. But I got no build error.

Quoted 6 lines

make -j zImage may call if_changed twice at the same time, the result is a corrupted vmlinux.gz
write to a temporary file for the time being until someone with make skills fix the serialization properly.

Signed-off-by Olaf Hering <[email protected]>

This is at least functional, so

Signed-off-by Tom Rini <[email protected]>
Signature · 2 lines
Tom Rini
http://gate.crashing.org/~trini/