Makefile: fix parallel build
PATCH 1/1 Makefile: fix parallel build
During parallel build, the top Makefile spawns multiple sub-makes for targets in cpu/$(CPU). If cpu/$(CPU)/.depend is not present, the sub-makes may end up generating this file simultaneously which leads to corrupted content.
A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of cpu/$(CPU)/.depend by adding cpu/$(CPU) to the depend target in the top Makefile.
--- Makefile | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
Makefile +1 -1
@@ -400,7 +400,7 @@ env: $(MAKE) -C tools/env all MTD_VERSION=${MTD_VERSION} || exit 1 depend dep: $(TIMESTAMP_FILE) $(VERSION_FILE) $(obj)include/autoconf.mk - for dir in $(SUBDIRS) ; do $(MAKE) -C $$dir _depend ; done + for dir in $(SUBDIRS) cpu/$(CPU); do $(MAKE) -C $$dir _depend ; done TAG_SUBDIRS += include TAG_SUBDIRS += lib_generic board/$(BOARDDIR)
Signature · 1 line
1.6.5.4
PATCH 1/1 Makefile: fix parallel build
Quoted 11 lines · On Thursday 10 December 2009 08:41:07 Daniel Hobi wrote:
During parallel build, the top
Makefilespawns multiple sub-makes for targets in cpu/$(CPU). If cpu/$(CPU)/.depend is not present, the sub-makes may end up generating this file simultaneously which leads to corrupted content.A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of cpu/$(CPU)/.depend by adding cpu/$(CPU) to the depend target in the top
Makefile.
seems to happen in lib_$(ARCH)/ too, but in reviewing my logs from the last few months, most parallel .depend failures have indeed been in cpu/$(CPU)/. maybe this is just coincidence though ... perhaps the depend target should walk all subdirs instead of a just random few ones (use LIBS).
-mike
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part.
Url : http://lists.denx.de/pipermail/u-boot/attachments/20091211/38ce3d46/attachment.pgp
PATCH 1/1 Makefile: fix parallel build
Quoted 10 lines · Mike Frysinger
Quoted 9 lines · On Thursday 10 December 2009 08:41:07 Daniel Hobi wrote:
During parallel build, the top
Makefilespawns multiple sub-makes for targets in cpu/$(CPU). If cpu/$(CPU)/.depend is not present, the sub-makes may end up generating this file simultaneously which leads to corrupted content.A typical error message is:
.depend:39: *** multiple target patterns. Stop.
Quoted 7 lines
Quoted 2 lines
This patch serializes the creation of cpu/$(CPU)/.depend by adding cpu/$(CPU) to the depend target in the top
Makefile.seems to happen in lib_$(ARCH)/ too, but in reviewing my logs from the last few months, most parallel .depend failures have indeed been in cpu/$(CPU)/. maybe this is just coincidence though ... perhaps the depend target should walk all subdirs instead of a just random few ones (use LIBS).
The problem with cpu/$(CPU)/ is that the top Makefile builds multiple targets within this directory in parallel - at least start.o and lib$(CPU).a. For all other directories, parallel build should work, since there is only one target per directory.
Can you provide any commit ID where building lib_$(ARCH)/ failed?
Thus, adding all LIBS to the depend target should not make any difference. In order to properly track such dependencies we should switch to non-recursive Makefiles (as seen in Linux) since Recursive Make [is] Considered Harmful[1].
[1] http://miller.emu.id.au/pmiller/books/rmch/
Best regards,
Daniel
PATCH 1/1 Makefile: fix parallel build
Quoted 20 lines · On Tuesday 15 December 2009 04:21:02 Daniel Hobi wrote:
Quoted 10 lines · Mike Frysinger
Quoted 9 lines · On Thursday 10 December 2009 08:41:07 Daniel Hobi wrote:
During parallel build, the top
Makefilespawns multiple sub-makes for targets in cpu/$(CPU). If cpu/$(CPU)/.depend is not present, the sub-makes may end up generating this file simultaneously which leads to corrupted content.A typical error message is:
.depend:39: *** multiple target patterns. Stop.
Quoted 8 lines
Quoted 2 lines
This patch serializes the creation of cpu/$(CPU)/.depend by adding cpu/$(CPU) to the depend target in the top
Makefile.seems to happen in lib_$(ARCH)/ too, but in reviewing my logs from the last few months, most parallel .depend failures have indeed been in cpu/$(CPU)/. maybe this is just coincidence though ... perhaps the depend target should walk all subdirs instead of a just random few ones (use LIBS).
Quoted 4 lines
The problem with cpu/$(CPU)/ is that the top
Makefilebuilds multiple targets within this directory in parallel - at least start.o and lib$(CPU).a. For all other directories, parallel build should work, since there is only one target per directory.
OK, that answers the next part
Quoted 1 line
Can you provide any commit ID where building lib_$(ARCH)/ failed?
it isnt specific to lib_$(ARCH). wherever the LDSCRiPT is located will cause a problem. in the Blackfin case, it's in lib_blackfin/.
so maybe to extend this workaround, add like $(dir $(LDSCRIPT)) to the list. -mike
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part.
Url : http://lists.denx.de/pipermail/u-boot/attachments/20091215/de33dcbb/attachment.pgp
PATCH v2 1/1 Makefile: fix parallel build
During parallel build, the top Makefile spawns multiple sub-makes for targets in cpu/$(CPU) and $(dir $(LDSCRIPT)). If the .depend files are not present in these directories, the sub-makes may end up generating these files simultaneously which leads to corrupted content.
A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of .depend in cpu/$(CPU) and $(dir $(LDSCRIPT)) by adding these directories to the depend target in the top Makefile.
Other directories in $(LIBS) are not affected since they contain only one Make target and thus only one sub-make is spawned per directory.
---
v2:
- Break overlong line
- Add comment about other directories to commit message
- Also build target _depend in $(dir $(LDSCRIPT)) (as suggested by Mike) This requires the Makefile in $(dir $(LDSCRIPT)) to provide a _depend target, which is the case for all LDSCRIPT values in the current tree
Makefile +2 -1
@@ -400,7 +400,8 @@ env: $(MAKE) -C tools/env all MTD_VERSION=${MTD_VERSION} || exit 1 depend dep: $(TIMESTAMP_FILE) $(VERSION_FILE) $(obj)include/autoconf.mk - for dir in $(SUBDIRS) ; do $(MAKE) -C $$dir _depend ; done + for dir in $(SUBDIRS) cpu/$(CPU) $(dir $(LDSCRIPT)) ; do \ + $(MAKE) -C $$dir _depend ; done TAG_SUBDIRS += include TAG_SUBDIRS += lib_generic board/$(BOARDDIR)
Signature · 1 line
1.6.5.4
PATCH v2 1/1 Makefile: fix parallel build
Makefile env: · line 404
Quoted 4 lines · On Tuesday 15 December 2009 08:51:46 Daniel Hobi wrote:
depend dep: $(TIMESTAMP_FILE) $(VERSION_FILE) $(obj)include/autoconf.mk
Makefile +2
-1- for dir in $(SUBDIRS) ; do $(MAKE) -C $$dir _depend ; done + for dir in $(SUBDIRS) cpu/$(CPU) $(dir $(LDSCRIPT)) ; do \ + $(MAKE) -C $$dir _depend ; done
i'd stick a short comment here explaining the reason for the specific subdirs. otherwise, the new additions all look good to me. this is something that has annoyed me for quite a long time, but has been hard to reproduce to figure out what exactly was failing. thanks !
-mike
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part.
Url : http://lists.denx.de/pipermail/u-boot/attachments/20091215/14c3d5b2/attachment.pgp
PATCH v2 1/1 Makefile: fix parallel build
In reply to Daniel HobiDear Daniel Hobi,
Quoted 4 lines · In message <[email protected]>…
During parallel build, the top
Makefilespawns multiple sub-makes for targets in cpu/$(CPU) and $(dir $(LDSCRIPT)). If the .depend files are not present in these directories, the sub-makes may end up generating these files simultaneously which leads to corrupted content.
...
Makefile env: · line 404
Quoted 11 lines
Makefile +2
-1@@ -400,7 +400,8 @@ env: $(MAKE) -C tools/env all MTD_VERSION=${MTD_VERSION} || exit 1 depend dep: $(TIMESTAMP_FILE) $(VERSION_FILE) $(obj)include/autoconf.mk - for dir in $(SUBDIRS) ; do $(MAKE) -C $$dir _depend ; done + for dir in $(SUBDIRS) cpu/$(CPU) $(dir $(LDSCRIPT)) ; do \ + $(MAKE) -C $$dir _depend ; done
I ageree with Mike: please add a comment here.
Best regards,
Wolfgang Denk
Signature · 5 lines
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de The important thing about being a leader is not being right or wrong, but being *certain*. - Terry Pratchett, _Truckers_
PATCH v3 1/1 Makefile: fix parallel build
During parallel build, the top Makefile spawns multiple sub-makes for targets in cpu/$(CPU) and $(dir $(LDSCRIPT)). If the .depend files are not present in these directories, the sub-makes may end up generating these files simultaneously which leads to corrupted content.
A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of .depend in cpu/$(CPU) and $(dir $(LDSCRIPT)) by adding these directories to the depend target in the top Makefile.
Other directories in $(LIBS) are not affected since they contain only one Make target and thus only one sub-make is spawned per directory.
---
v3:
- Add a comment why make _depend is required for these two subdirs
v2:
- Also build target depend in $(dir $(LDSCRIPT)) (suggested by Mike)
- Break overlong line
Makefile | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)
Makefile +4 -1
@@ -398,8 +398,11 @@ updater: env: $(MAKE) -C tools/env all MTD_VERSION=${MTD_VERSION} || exit 1 +# Explicitly make _depend in subdirs containing multiple targets to prevent +# parallel sub-makes creating .depend files simultaneously. depend dep: $(TIMESTAMP_FILE) $(VERSION_FILE) $(obj)include/autoconf.mk - for dir in $(SUBDIRS) ; do $(MAKE) -C $$dir _depend ; done + for dir in $(SUBDIRS) cpu/$(CPU) $(dir $(LDSCRIPT)) ; do \ + $(MAKE) -C $$dir _depend ; done TAG_SUBDIRS = $(SUBDIRS) TAG_SUBDIRS += $(dir $(__LIBS))
Signature · 1 line
1.6.6
Wolfgang Denk
AppliedPATCH v3 1/1 Makefile: fix parallel build
Dear Daniel Hobi,
Quoted 24 lines · In message <[email protected]>…
During parallel build, the top
Makefilespawns multiple sub-makes for targets in cpu/$(CPU) and $(dir $(LDSCRIPT)). If the .depend files are not present in these directories, the sub-makes may end up generating these files simultaneously which leads to corrupted content.A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of .depend in cpu/$(CPU) and $(dir $(LDSCRIPT)) by adding these directories to the depend target in the top
Makefile.Other directories in $(LIBS) are not affected since they contain only one Make target and thus only one sub-make is spawned per directory.
Signed-off-by Daniel Hobi <[email protected]>---
v3:
- Add a comment why make_dependis required for these two subdirs
v2:
- Also build target depend in $(dir $(LDSCRIPT)) (suggested by Mike)
- Break overlong line
Quoted 2 lines
Makefile | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)
Applied, thanks.
Best regards,
Wolfgang Denk
Signature · 5 lines
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de "To take a significant step forward, you must make a series of finite improvements." - Donald J. Atwood, General Motors