Skip to contentSkip to conversation

kbuild: don't delete in-flight filechk temporaries in asm-headers

PATCH In review Vlad Poenaru 5 participants 5 messages Sep 2 – Sep 4

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kbuild: don't delete in-flight filechk temporaries in asm-headers

Commit 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories") switched the stale-wrapper sweep in scripts/Makefile.asm-headers from $(wildcard $(obj)/*.h) to a find(1) invocation, so that generated headers in subdirectories are considered.

The two do not match the same set of files. Make's $(wildcard) uses glob semantics, where a leading '.' has to be matched explicitly, whereas find's -name uses fnmatch() without FNM_PERIOD, so '*.h' matches dotfiles as well. filechk writes its output to $(dir $@).tmp_$(notdir $@) before renaming it into place, so such a scratch file, if it happens to exist in $(obj) when the sub-make is parsed, is now picked up in old-headers. It appears in neither generic-y, generated-y nor syscall-y, is therefore classified as unwanted, and cmd_remove deletes it.

On x86 this races with archprepare, which lists both asm-generic and arch/x86/include/generated/asm/cpufeaturemasks.h as prerequisites. Under -j they run concurrently against the same directory, and the build fails intermittently:

mv: cannot stat 'arch/x86/include/generated/asm/.tmp_cpufeaturemasks.h': No such file or directory make[1]: *** [arch/x86/Makefile:269: arch/x86/include/generated/asm/cpufeaturemasks.h] Error 1

The same commit also converted the generic wrapper rule to filechk, so those wrappers now create .tmp_*.h in $(obj) too and can race among themselves.

Restore the previous behaviour by excluding dotfiles from the sweep. Subdirectories, which is what the find(1) conversion was for, keep being descended into. While at it, quote the -name argument: it is currently expanded by the shell against the build directory before find sees it.

Fixes 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories")

---
#regzbot introduced: 2d69b891e646

 scripts/Makefile.asm-headers | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/Makefile.asm-headers +1 -1
index b38931314ad7c..f1c3d287c14b0 100644
@@ -48,7 +48,7 @@ syscall-y   := $(addprefix $(obj)/, $(syscall-y))
 generated-y := $(addprefix $(obj)/, $(generated-y))
 
 # Remove stale wrappers when the corresponding files are removed from generic-y
-old-headers := $(shell test -d $(obj) && find $(obj) -name *.h)
+old-headers := $(shell test -d $(obj) && find $(obj) -name '*.h' ! -name '.*')
 unwanted    := $(filter-out $(generic-y) $(generated-y) $(syscall-y),$(old-headers))
 
 filechk_wrap = echo "\#include <asm-generic/$*.h>"
Signature · 1 line
2.53.0-Meta
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 26 lines · Vlad Poenaru

Commit 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories") switched the stale-wrapper sweep in scripts/Makefile.asm-headers from $(wildcard $(obj)/*.h) to a find(1) invocation, so that generated headers in subdirectories are considered.

The two do not match the same set of files. Make's $(wildcard) uses glob semantics, where a leading '.' has to be matched explicitly, whereas find's -name uses fnmatch() without FNM_PERIOD, so '*.h' matches dotfiles as well. filechk writes its output to $(dir $@).tmp_$(notdir $@) before renaming it into place, so such a scratch file, if it happens to exist in $(obj) when the sub-make is parsed, is now picked up in old-headers. It appears in neither generic-y, generated-y nor syscall-y, is therefore classified as unwanted, and cmd_remove deletes it.

On x86 this races with archprepare, which lists both asm-generic and arch/x86/include/generated/asm/cpufeaturemasks.h as prerequisites. Under -j they run concurrently against the same directory, and the build fails intermittently:

mv: cannot stat 'arch/x86/include/generated/asm/.tmp_cpufeaturemasks.h': No such file or directory make[1]: *** [arch/x86/Makefile:269: arch/x86/include/generated/asm/cpufeaturemasks.h] Error 1

The same commit also converted the generic wrapper rule to filechk, so those wrappers now create .tmp_*.h in $(obj) too and can race among themselves.

Quoted 7 lines

Restore the previous behaviour by excluding dotfiles from the sweep. Subdirectories, which is what the find(1) conversion was for, keep being descended into. While at it, quote the -name argument: it is currently expanded by the shell against the build directory before find sees it.

Fixes 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories")

Oof :( that's unfortunate, thanks a lot for tracking this down!

Nicolas, I assume you will pick this up for 7.3.

Signature · 2 lines
Cheers,
Nathan
Reviewed-by
#2 lore raw
Quoted 32 lines · Nathan Chancellor
Quoted 31 lines · Vlad Poenaru

Commit 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories") switched the stale-wrapper sweep in scripts/Makefile.asm-headers from $(wildcard $(obj)/*.h) to a find(1) invocation, so that generated headers in subdirectories are considered.

The two do not match the same set of files. Make's $(wildcard) uses glob semantics, where a leading '.' has to be matched explicitly, whereas find's -name uses fnmatch() without FNM_PERIOD, so '*.h' matches dotfiles as well. filechk writes its output to $(dir $@).tmp_$(notdir $@) before renaming it into place, so such a scratch file, if it happens to exist in $(obj) when the sub-make is parsed, is now picked up in old-headers. It appears in neither generic-y, generated-y nor syscall-y, is therefore classified as unwanted, and cmd_remove deletes it.

On x86 this races with archprepare, which lists both asm-generic and arch/x86/include/generated/asm/cpufeaturemasks.h as prerequisites. Under -j they run concurrently against the same directory, and the build fails intermittently:

mv: cannot stat 'arch/x86/include/generated/asm/.tmp_cpufeaturemasks.h': No such file or directory make[1]: *** [arch/x86/Makefile:269: arch/x86/include/generated/asm/cpufeaturemasks.h] Error 1

The same commit also converted the generic wrapper rule to filechk, so those wrappers now create .tmp_*.h in $(obj) too and can race among themselves.

Restore the previous behaviour by excluding dotfiles from the sweep. Subdirectories, which is what the find(1) conversion was for, keep being descended into. While at it, quote the -name argument: it is currently expanded by the shell against the build directory before find sees it.

Quoted 8 lines
Quoted 2 lines
Fixes 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories")

Oof :( that's unfortunate, thanks a lot for tracking this down!

Nicolas, I assume you will pick this up for 7.3.

Sure, sorry for the delay.

Signature · 1 line
Nicolas
Reviewed-by on v1 1/1
#3 lore raw
In reply to Vlad Poenaru
Quoted 26 lines · Vlad Poenaru

Commit 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories") switched the stale-wrapper sweep in scripts/Makefile.asm-headers from $(wildcard $(obj)/*.h) to a find(1) invocation, so that generated headers in subdirectories are considered.

The two do not match the same set of files. Make's $(wildcard) uses glob semantics, where a leading '.' has to be matched explicitly, whereas find's -name uses fnmatch() without FNM_PERIOD, so '*.h' matches dotfiles as well. filechk writes its output to $(dir $@).tmp_$(notdir $@) before renaming it into place, so such a scratch file, if it happens to exist in $(obj) when the sub-make is parsed, is now picked up in old-headers. It appears in neither generic-y, generated-y nor syscall-y, is therefore classified as unwanted, and cmd_remove deletes it.

On x86 this races with archprepare, which lists both asm-generic and arch/x86/include/generated/asm/cpufeaturemasks.h as prerequisites. Under -j they run concurrently against the same directory, and the build fails intermittently:

mv: cannot stat 'arch/x86/include/generated/asm/.tmp_cpufeaturemasks.h': No such file or directory make[1]: *** [arch/x86/Makefile:269: arch/x86/include/generated/asm/cpufeaturemasks.h] Error 1

The same commit also converted the generic wrapper rule to filechk, so those wrappers now create .tmp_*.h in $(obj) too and can race among themselves.

Quoted 7 lines

Restore the previous behaviour by excluding dotfiles from the sweep. Subdirectories, which is what the find(1) conversion was for, keep being descended into. While at it, quote the -name argument: it is currently expanded by the shell against the build directory before find sees it.

Fixes 2d69b891e646 ("kbuild: Support generated asm-headers in subdirectories")

Thanks, and sorry for the breakage!

(...)

maintainer
#4 lore raw
In reply to Vlad Poenaru
Quoted 1 line · Vlad Poenaru

kbuild: don't delete in-flight filechk temporaries in asm-headers

Applied to kbuild/linux.git (kbuild-fixes-unstable), thanks!

[1/1] kbuild: don't delete in-flight filechk temporaries in asm-headers

      https://git.kernel.org/kbuild/c/94f7bba7

Please look out for regression or issue reports or other follow up comments, as they may result in the patch/series getting dropped, reverted or modified (e.g. trailers).

Patches applied to the kbuild-fixes-unstable branch are accepted pending wider testing in linux-next and any post-commit review; they will generally be moved to the kbuild-fixes branch within about a week if no issues are found.

Best regards,

Signature · 1 line
Nicolas