kbuild: don't delete in-flight filechk temporaries in asm-headers
Subsystems (2)
PATCH 1/1 kbuild: don't delete in-flight filechk temporaries in asm-headers
Commit 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories") switched the stale-wrapper sweep in scripts/Makefile.asm-headers from $(wildcard $(obj)/*.h) to a find(1) invocation, so that generated headers in subdirectories are considered.
The two do not match the same set of files. Make's $(wildcard) uses glob semantics, where a leading '.' has to be matched explicitly, whereas find's -name uses fnmatch() without FNM_PERIOD, so '*.h' matches dotfiles as well. filechk writes its output to $(dir $@).tmp_$(notdir $@) before renaming it into place, so such a scratch file, if it happens to exist in $(obj) when the sub-make is parsed, is now picked up in old-headers. It appears in neither generic-y, generated-y nor syscall-y, is therefore classified as unwanted, and cmd_remove deletes it.
On x86 this races with archprepare, which lists both asm-generic and arch/x86/include/generated/asm/cpufeaturemasks.h as prerequisites. Under -j they run concurrently against the same directory, and the build fails intermittently:
mv: cannot stat 'arch/x86/include/generated/asm/.tmp_cpufeaturemasks.h': No such file or directory make[1]: *** [arch/x86/Makefile:269: arch/x86/include/generated/asm/cpufeaturemasks.h] Error 1
The same commit also converted the generic wrapper rule to filechk, so those wrappers now create .tmp_*.h in $(obj) too and can race among themselves.
Restore the previous behaviour by excluding dotfiles from the sweep. Subdirectories, which is what the find(1) conversion was for, keep being descended into. While at it, quote the -name argument: it is currently expanded by the shell against the build directory before find sees it.
---
#regzbot introduced: 2d69b891e646
scripts/Makefile.asm-headers | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/Makefile.asm-headers +1 -1
@@ -48,7 +48,7 @@ syscall-y := $(addprefix $(obj)/, $(syscall-y)) generated-y := $(addprefix $(obj)/, $(generated-y)) # Remove stale wrappers when the corresponding files are removed from generic-y -old-headers := $(shell test -d $(obj) && find $(obj) -name *.h) +old-headers := $(shell test -d $(obj) && find $(obj) -name '*.h' ! -name '.*') unwanted := $(filter-out $(generic-y) $(generated-y) $(syscall-y),$(old-headers)) filechk_wrap = echo "\#include <asm-generic/$*.h>"
Signature · 1 line
2.53.0-Meta
Quoted 26 lines · Vlad Poenaru
Commit
2d69b891e646("kbuild: Support generated asm-headers in subdirectories") switched the stale-wrapper sweep inscripts/Makefile.asm-headersfrom $(wildcard $(obj)/*.h) to a find(1) invocation, so that generated headers in subdirectories are considered.The two do not match the same set of files. Make's $(wildcard) uses glob semantics, where a leading '.' has to be matched explicitly, whereas find's -name uses
fnmatch()withoutFNM_PERIOD, so '*.h' matches dotfiles as well. filechk writes its output to $(dir $@).tmp_$(notdir $@) before renaming it into place, so such a scratch file, if it happens to exist in $(obj) when the sub-make is parsed, is now picked up in old-headers. It appears in neither generic-y, generated-y nor syscall-y, is therefore classified as unwanted, andcmd_removedeletes it.On x86 this races with archprepare, which lists both asm-generic and
arch/x86/include/generated/asm/cpufeaturemasks.has prerequisites. Under -j they run concurrently against the same directory, and the build fails intermittently:mv: cannot stat '
arch/x86/include/generated/asm/.tmp_cpufeaturemasks.h': No such file or directory make[1]: *** [arch/x86/Makefile:269:arch/x86/include/generated/asm/cpufeaturemasks.h] Error 1The same commit also converted the generic wrapper rule to filechk, so those wrappers now create .tmp_*.h in $(obj) too and can race among themselves.
Quoted 7 lines
Restore the previous behaviour by excluding dotfiles from the sweep. Subdirectories, which is what the find(1) conversion was for, keep being descended into. While at it, quote the -name argument: it is currently expanded by the shell against the build directory before find sees it.
Fixes 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories")Signed-off-by Vlad Poenaru <[email protected]>
Oof :( that's unfortunate, thanks a lot for tracking this down!
Nicolas, I assume you will pick this up for 7.3.
Signature · 2 lines
Cheers, Nathan
Nicolas Schier
Reviewed-byQuoted 32 lines · Nathan Chancellor
Quoted 31 lines · Vlad Poenaru
Commit
2d69b891e646("kbuild: Support generated asm-headers in subdirectories") switched the stale-wrapper sweep inscripts/Makefile.asm-headersfrom $(wildcard $(obj)/*.h) to a find(1) invocation, so that generated headers in subdirectories are considered.The two do not match the same set of files. Make's $(wildcard) uses glob semantics, where a leading '.' has to be matched explicitly, whereas find's -name uses
fnmatch()withoutFNM_PERIOD, so '*.h' matches dotfiles as well. filechk writes its output to $(dir $@).tmp_$(notdir $@) before renaming it into place, so such a scratch file, if it happens to exist in $(obj) when the sub-make is parsed, is now picked up in old-headers. It appears in neither generic-y, generated-y nor syscall-y, is therefore classified as unwanted, andcmd_removedeletes it.On x86 this races with archprepare, which lists both asm-generic and
arch/x86/include/generated/asm/cpufeaturemasks.has prerequisites. Under -j they run concurrently against the same directory, and the build fails intermittently:mv: cannot stat '
arch/x86/include/generated/asm/.tmp_cpufeaturemasks.h': No such file or directory make[1]: *** [arch/x86/Makefile:269:arch/x86/include/generated/asm/cpufeaturemasks.h] Error 1The same commit also converted the generic wrapper rule to filechk, so those wrappers now create .tmp_*.h in $(obj) too and can race among themselves.
Restore the previous behaviour by excluding dotfiles from the sweep. Subdirectories, which is what the find(1) conversion was for, keep being descended into. While at it, quote the -name argument: it is currently expanded by the shell against the build directory before find sees it.
Quoted 8 lines
Quoted 2 lines
Fixes 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories")Signed-off-by Vlad Poenaru <[email protected]>Oof :( that's unfortunate, thanks a lot for tracking this down!
Reviewed-by Nathan Chancellor <[email protected]>Nicolas, I assume you will pick this up for 7.3.
Sure, sorry for the delay.
Signature · 1 line
Nicolas
Quoted 26 lines · Vlad Poenaru
Commit
2d69b891e646("kbuild: Support generated asm-headers in subdirectories") switched the stale-wrapper sweep inscripts/Makefile.asm-headersfrom $(wildcard $(obj)/*.h) to a find(1) invocation, so that generated headers in subdirectories are considered.The two do not match the same set of files. Make's $(wildcard) uses glob semantics, where a leading '.' has to be matched explicitly, whereas find's -name uses
fnmatch()withoutFNM_PERIOD, so '*.h' matches dotfiles as well. filechk writes its output to $(dir $@).tmp_$(notdir $@) before renaming it into place, so such a scratch file, if it happens to exist in $(obj) when the sub-make is parsed, is now picked up in old-headers. It appears in neither generic-y, generated-y nor syscall-y, is therefore classified as unwanted, andcmd_removedeletes it.On x86 this races with archprepare, which lists both asm-generic and
arch/x86/include/generated/asm/cpufeaturemasks.has prerequisites. Under -j they run concurrently against the same directory, and the build fails intermittently:mv: cannot stat '
arch/x86/include/generated/asm/.tmp_cpufeaturemasks.h': No such file or directory make[1]: *** [arch/x86/Makefile:269:arch/x86/include/generated/asm/cpufeaturemasks.h] Error 1The same commit also converted the generic wrapper rule to filechk, so those wrappers now create .tmp_*.h in $(obj) too and can race among themselves.
Quoted 7 lines
Restore the previous behaviour by excluding dotfiles from the sweep. Subdirectories, which is what the find(1) conversion was for, keep being descended into. While at it, quote the -name argument: it is currently expanded by the shell against the build directory before find sees it.
Fixes 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories")Signed-off-by Vlad Poenaru <[email protected]>
Thanks, and sorry for the breakage!
(...)
Nicolas Schier
maintainerQuoted 1 line · Vlad Poenaru
kbuild: don't delete in-flight filechk temporaries in asm-headers
Applied to kbuild/linux.git (kbuild-fixes-unstable), thanks!
[1/1] kbuild: don't delete in-flight filechk temporaries in asm-headers
https://git.kernel.org/kbuild/c/94f7bba7
Please look out for regression or issue reports or other follow up comments, as they may result in the patch/series getting dropped, reverted or modified (e.g. trailers).
Patches applied to the kbuild-fixes-unstable branch are accepted pending wider testing in linux-next and any post-commit review; they will generally be moved to the kbuild-fixes branch within about a week if no issues are found.
Best regards,
Signature · 1 line
Nicolas