tools/shim: Bodge things harder
cover tools/shim: Bodge things harder
This logic is all terrible. This series should resolve the reported build failure, but definitely isn't a "proper" fix.
Andrew Cooper (2):
tools/shim: Fix race condition creating linkfarm.stamp tools/shim: Apply more duct tape to the linkfarm logic
tools/firmware/xen-dir/Makefile | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-)
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 1/2 tools/shim: Fix race condition creating linkfarm.stamp
In the case the while loop gets interrupted, the target musn't appear as up-to-date. The mov $X.tmp $X must be the last action of the rule.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: George Dunlap <[email protected]>
CC: Sander Eikelenboom <[email protected]>
--- tools/firmware/xen-dir/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/firmware/xen-dir/Makefile +2 -2
@@ -32,9 +32,9 @@ linkfarm.stamp: $(DEP_DIRS) $(DEP_FILES) FORCE echo $(f) >> linkfarm.stamp.tmp ;) cmp -s linkfarm.stamp.tmp linkfarm.stamp && \ rm linkfarm.stamp.tmp || { \ + cat linkfarm.stamp.tmp | while read f; \ + do rm -f "$(D)/$$f"; ln -s "$(XEN_ROOT)/$$f" "$(D)/$$f"; done; \ mv linkfarm.stamp.tmp linkfarm.stamp; \ - cat linkfarm.stamp | while read f; \ - do rm -f "$(D)/$$f"; ln -s "$(XEN_ROOT)/$$f" "$(D)/$$f"; done \ } # Copy enough of the tree to build the shim hypervisor
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 1/2] tools/shim: Fix race condition creating linkfarm.stamp
Andrew Cooper writes ("[PATCH 1/2] tools/shim: Fix race condition creating linkfarm.stamp"):
Quoted 4 lines
In the case the while loop gets interrupted, the target musn't appear as up-to-date. The mov $X.tmp $X must be the last action of the rule.
Signed-off-by Andrew Cooper <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 2/2 tools/shim: Apply more duct tape to the linkfarm logic
In reply to Andrew CooperSander reported a build failure which manifests as make; make install failing with:
<snip>/cross-install -m0644 -p xen-dir/xen-shim //usr/local/lib/xen/boot/xen-shim
install: cannot stat 'xen-dir/xen-shim': No such file or directory make[4]: *** [Makefile:52: install] Error 1
make[4]: Leaving directory '/usr/src/new/xen-unstable/tools/firmware'
It has subsequently been seen intermittently by OSSTest. This was caused by c/s 32b1d628 triggering a preexisting linkfarm bug for partial rebuilds.
Between the first make and the subsequent make install, the linkfarm logic observes new final build products and regenerates the linkfarm. This includes a distclean, which throws away everything from the first make.
As the xen-shim rule use a symlink, the link itself remains still up-to-date but is broken due to the distclean, which causes install to fail.
Update the linkfarm logic to not regenerate itself when build artefacts appear. This isn't a comprehensive fix but is the best which can be done easily. Any further effort would be better spent making out-of-tree builds work for Xen.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: George Dunlap <[email protected]>
CC: Sander Eikelenboom <[email protected]>
--- tools/firmware/xen-dir/Makefile | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-)
tools/firmware/xen-dir/Makefile +21 -2
@@ -14,6 +14,26 @@ LINK_FILES=Config.mk DEP_DIRS=$(foreach i, $(LINK_DIRS), $(XEN_ROOT)/$(i)) DEP_FILES=$(foreach i, $(LINK_FILES), $(XEN_ROOT)/$(i)) +# Exclude some intermediate files and final build products +LINK_EXCLUDES := '*.[isoa]' '.*.d' '.*.d2' '.config' +LINK_EXCLUDES += '*.map' 'xen' 'xen.gz' 'xen.efi' 'xen-syms' + +# This is all a giant mess and doesn't really work. +# +# The correct solution is to fix Xen to be able to do out-of-tree builds. +# +# Until that happens, we set up a linkfarm by iterating over the xen/ tree, +# linking source files. This is repeated each time we enter this directory, +# which poses a problem for a two-step "make; make install" build process. +# +# Any time the list of files to link changes, we relink all files, then +# distclean to take out not-easy-to-classify intermediate files. This is to +# support easy development of the shim, but has a side effect of clobbering +# the already-built shim. +# +# $(LINK_EXCLUDES) should be set such that a parallel build of shim and xen/ +# doesn't cause a subsequent `make install` to decide to regenerate the +# linkfarm. This means that all final build artefacts must be excluded. linkfarm.stamp: $(DEP_DIRS) $(DEP_FILES) FORCE mkdir -p $(D) rm -f linkfarm.stamp.tmp
@@ -25,8 +45,7 @@ linkfarm.stamp: $(DEP_DIRS) $(DEP_FILES) FORCE sed 's,^$(XEN_ROOT)/$(d)/,,g' | xargs mkdir -p .);) \ $(foreach d, $(LINK_DIRS), \ (cd $(XEN_ROOT); \ - find $(d) ! -type l -type f \ - $(addprefix ! -name , '*.[isoa]' '.*.d' '.*.d2')) \ + find $(d) ! -type l -type f $(addprefix ! -name ,$(LINK_EXCLUDES))) \ >> linkfarm.stamp.tmp ; ) \ $(foreach f, $(LINK_FILES), \ echo $(f) >> linkfarm.stamp.tmp ;)
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 2/2] tools/shim: Apply more duct tape to the linkfarm logic
Andrew Cooper writes ("[PATCH 2/2] tools/shim: Apply more duct tape to the linkfarm logic"):
Quoted 2 lines
Sander reported a build failure which manifests as
make; make installfailing with:
Quoted 4 lines
Update the linkfarm logic to not regenerate itself when build artefacts appear. This isn't a comprehensive fix but is the best which can be done easily. Any further effort would be better spent making out-of-tree builds work for Xen.
Fair enough.
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 10 lines · Andrew Cooper
This logic is all terrible. This series should resolve the reported build failure, but definitely isn't a "proper" fix.
Andrew Cooper (2):
tools/shim: Fix race condition creating linkfarm.stamptools/shim: Apply more duct tape to the linkfarm logictools/firmware/xen-dir/Makefile | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-)
Thanks Andrew, Just tested and it works for me!
--
Sander
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel