kbuild: Remove stale asm-generic wrappers
Subsystems (5)
v2 cover kbuild: Remove stale asm-generic wrappers
This patchset attempts to fix kbuild to automatically remove stale asm-generic wrappers, i.e. when files are removed from generic-y and added directly into arch/*/include/uapi/asm/, but where the existing wrapper in arch/*/include/generated/asm/ continues to be used.
MIPS was recently burned by this in v4.3 (see patch 2), with continuing reports of build failures when people upgrade their trees, which go away after arch/mips/include/generated is removed (or reportedly make mrproper/distclean). It is particularly irritating during bisection.
Changes in v2:
- New patch 1 to add tracking of generated headers that aren't generic-y wrappers, via generated-y, particularly for x86 (thanks to kbuild test robot).
- Rewrite a bit, drawing inspiration from Makefile.headersinst.
- Exclude genhdr-y and generated-y (thanks to kbuild test robot).
James Hogan (2):
kbuild, x86: Track generated headers with generated-y
kbuild: Remove stale asm-generic wrappers
Documentation/kbuild/makefiles.txt | 14 ++++++++++++++ arch/x86/include/asm/Kbuild | 6 ++++++ scripts/Makefile.asm-generic | 17 ++++++++++++++++- 3 files changed, 36 insertions(+), 1 deletion(-)
Signature · 1 line
2.4.10
PATCH v2 1/2 kbuild, x86: Track generated headers with generated-y
Track generated header files which aren't already in genhdr-y, alongside generic-y wrappers in the */include/generated/[uapi/]asm/ directories. Currently only x86 generates extra headers in these directories, for the purposes of enumerating system calls for different ABIs, and xen hypercalls.
This will allow the asm-generic wrapper handling code to remove stale wrappers when files are removed from generic-y, without also removing these headers which are generated separately.
---
Changes in v2:
- New patch (thanks to kbuild test robot).
--- Documentation/kbuild/makefiles.txt | 14 ++++++++++++++ arch/x86/include/asm/Kbuild | 6 ++++++ 2 files changed, 20 insertions(+)
Documentation/kbuild/makefiles.txt +14 -0
@@ -47,6 +47,7 @@ This document describes the Linux kernel Makefiles. --- 7.2 genhdr-y --- 7.3 destination-y --- 7.4 generic-y + --- 7.5 generated-y === 8 Kbuild Variables === 9 Makefile language
@@ -1319,6 +1320,19 @@ See subsequent chapter for the syntax of the Kbuild file. Example: termios.h #include <asm-generic/termios.h> + --- 7.5 generated-y + + If an architecture generates other header files alongside generic-y + wrappers, and not included in genhdr-y, then generated-y specifies + them. + + This prevents them being treated as stale asm-generic wrappers and + removed. + + Example: + #arch/x86/include/asm/Kbuild + generated-y += syscalls_32.h + === 8 Kbuild Variables The top Makefile exports the following variables:
arch/x86/include/asm/Kbuild +6 -0
@@ -1,5 +1,11 @@ +generated-y += syscalls_32.h +generated-y += syscalls_64.h +generated-y += unistd_32_ia32.h +generated-y += unistd_64_x32.h +generated-y += xen-hypercalls.h + genhdr-y += unistd_32.h genhdr-y += unistd_64.h genhdr-y += unistd_x32.h
Signature · 1 line
2.4.10
PATCH v2 2/2 kbuild: Remove stale asm-generic wrappers
In reply to James HoganWhen a header file is removed from generic-y (often accompanied by the addition of an arch specific header), the generated wrapper file will persist, and in some cases may still take precedence over the new arch header.
For example commit f1fe2d21f4e1 ("MIPS: Add definitions for extended context") removed ucontext.h from generic-y in arch/mips/include/asm/, and added an arch/mips/include/uapi/asm/ucontext.h. The continued use of the wrapper when reusing a dirty build tree resulted in build failures in arch/mips/kernel/signal.c:
arch/mips/kernel/signal.c: In function ‘sc_to_extcontext’:arch/mips/kernel/signal.c:142:12: error: ‘struct ucontext’ has no member named ‘uc_extcontext’
return &uc->uc_extcontext;
^Fix by detecting and removing wrapper headers in generated header directories that do not correspond to a filename in generic-y, genhdr-y, or the newly introduced generated-y.
---
Changes in v2:
- Rewrite a bit, drawing inspiration from Makefile.headersinst.
- Exclude genhdr-y and generated-y (thanks to kbuild test robot).
--- scripts/Makefile.asm-generic | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)
scripts/Makefile.asm-generic +16 -1
@@ -13,11 +13,26 @@ include scripts/Kbuild.include # Create output directory if not already present _dummy := $(shell [ -d $(obj) ] || mkdir -p $(obj)) +# Stale wrappers when the corresponding files are removed from generic-y +# need removing. +generated-y := $(generic-y) $(genhdr-y) $(generated-y) +all-files := $(patsubst %, $(obj)/%, $(generated-y)) +old-headers := $(wildcard $(obj)/*.h) +unwanted := $(filter-out $(all-files),$(old-headers)) + quiet_cmd_wrap = WRAP $@ cmd_wrap = echo "\#include <asm-generic/$*.h>" >$@ -all: $(patsubst %, $(obj)/%, $(generic-y)) +quiet_cmd_remove = REMOVE $(unwanted) +cmd_remove = rm -f $(unwanted) + +all: $(patsubst %, $(obj)/%, $(generic-y)) FORCE + $(if $(unwanted),$(call cmd,remove),) @: $(obj)/%.h: $(call cmd,wrap) + +.PHONY: $(PHONY) +PHONY += FORCE +FORCE: ;
Signature · 6 lines
2.4.10 -- To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 11 lines · On Tuesday 19 January 2016 13:37:50 James Hogan wrote:
When a header file is removed from generic-y (often accompanied by the addition of an arch specific header), the generated wrapper file will persist, and in some cases may still take precedence over the new arch header.
For example commit
f1fe2d21f4e1("MIPS: Add definitions for extended context") removeducontext.hfrom generic-y inarch/mips/include/asm/, and added anarch/mips/include/uapi/asm/ucontext.h. The continued use of the wrapper when reusing a dirty build tree resulted in build failures inarch/mips/kernel/signal.c:
Quoted 8 lines
arch/mips/kernel/signal.c: In function ‘sc_to_extcontext’:arch/mips/kernel/signal.c:142:12: error: ‘struct ucontext’ has no member named ‘uc_extcontext’return &uc->uc_extcontext; ^Fix by detecting and removing wrapper headers in generated header directories that do not correspond to a filename in generic-y, genhdr-y, or the newly introduced generated-y.
Good idea.
Can you merge this through the mips tree, or do you need me to pick it up through asm-generic?
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 17 lines · Arnd Bergmann
Quoted 16 lines · On Tuesday 19 January 2016 13:37:50 James Hogan wrote:
When a header file is removed from generic-y (often accompanied by the addition of an arch specific header), the generated wrapper file will persist, and in some cases may still take precedence over the new arch header.
For example commit
f1fe2d21f4e1("MIPS: Add definitions for extended context") removeducontext.hfrom generic-y inarch/mips/include/asm/, and added anarch/mips/include/uapi/asm/ucontext.h. The continued use of the wrapper when reusing a dirty build tree resulted in build failures inarch/mips/kernel/signal.c:
arch/mips/kernel/signal.c: In function ‘sc_to_extcontext’:arch/mips/kernel/signal.c:142:12: error: ‘struct ucontext’ has no member named ‘uc_extcontext’return &uc->uc_extcontext; ^
Quoted 7 lines
Quoted 3 lines
Fix by detecting and removing wrapper headers in generated header directories that do not correspond to a filename in generic-y, genhdr-y, or the newly introduced generated-y.
Good idea.
Acked-by Arnd Bergmann <[email protected]>
Thanks Arnd
Quoted 2 lines
Can you merge this through the mips tree, or do you need me to pick it up through asm-generic?
I was envisaging the kbuild tree tbh, but I don't really mind how it gets merged. This patch depends on patch 1, which adds generated-y to x86 so we don't delete their other generated headers, but other than that it doesn't really have any dependencies.
Cheers
James
Arnd Bergmann
maintainerQuoted 28 lines · On Tuesday 19 January 2016 14:22:13 James Hogan wrote:
Quoted 17 lines · Arnd Bergmann
Quoted 16 lines · On Tuesday 19 January 2016 13:37:50 James Hogan wrote:
When a header file is removed from generic-y (often accompanied by the addition of an arch specific header), the generated wrapper file will persist, and in some cases may still take precedence over the new arch header.
For example commit
f1fe2d21f4e1("MIPS: Add definitions for extended context") removeducontext.hfrom generic-y inarch/mips/include/asm/, and added anarch/mips/include/uapi/asm/ucontext.h. The continued use of the wrapper when reusing a dirty build tree resulted in build failures inarch/mips/kernel/signal.c:
arch/mips/kernel/signal.c: In function ‘sc_to_extcontext’:arch/mips/kernel/signal.c:142:12: error: ‘struct ucontext’ has no member named ‘uc_extcontext’return &uc->uc_extcontext; ^Quoted 7 lines
Quoted 3 lines
Fix by detecting and removing wrapper headers in generated header directories that do not correspond to a filename in generic-y, genhdr-y, or the newly introduced generated-y.
Good idea.
Acked-by Arnd Bergmann <[email protected]>Thanks Arnd
Quoted 7 lines
Quoted 2 lines
Can you merge this through the mips tree, or do you need me to pick it up through asm-generic?
I was envisaging the kbuild tree tbh, but I don't really mind how it gets merged. This patch depends on patch 1, which adds generated-y to x86 so we don't delete their other generated headers, but other than that it doesn't really have any dependencies.
Ok, the kbuild tree works fine too, and I guess the x86 tree would also be fine if that helps avoid the dependency.
Arnd
Hi Michal,
Quoted 32 lines · Arnd Bergmann
Quoted 31 lines · On Tuesday 19 January 2016 14:22:13 James Hogan wrote:
Quoted 17 lines · Arnd Bergmann
Quoted 16 lines · On Tuesday 19 January 2016 13:37:50 James Hogan wrote:
When a header file is removed from generic-y (often accompanied by the addition of an arch specific header), the generated wrapper file will persist, and in some cases may still take precedence over the new arch header.
For example commit
f1fe2d21f4e1("MIPS: Add definitions for extended context") removeducontext.hfrom generic-y inarch/mips/include/asm/, and added anarch/mips/include/uapi/asm/ucontext.h. The continued use of the wrapper when reusing a dirty build tree resulted in build failures inarch/mips/kernel/signal.c:
arch/mips/kernel/signal.c: In function ‘sc_to_extcontext’:arch/mips/kernel/signal.c:142:12: error: ‘struct ucontext’ has no member named ‘uc_extcontext’return &uc->uc_extcontext; ^Quoted 7 lines
Quoted 3 lines
Fix by detecting and removing wrapper headers in generated header directories that do not correspond to a filename in generic-y, genhdr-y, or the newly introduced generated-y.
Good idea.
Acked-by Arnd Bergmann <[email protected]>Thanks Arnd
Quoted 2 lines
Can you merge this through the mips tree, or do you need me to pick it up through asm-generic?
Quoted 7 lines
Quoted 4 lines
I was envisaging the kbuild tree tbh, but I don't really mind how it gets merged. This patch depends on patch 1, which adds generated-y to x86 so we don't delete their other generated headers, but other than that it doesn't really have any dependencies.
Ok, the kbuild tree works fine too, and I guess the x86 tree would also be fine if that helps avoid the dependency.
Were you okay to take these patches, or would you prefer they go via the MIPS tree?
Thanks
James
Quoted 40 lines · James Hogan
Hi Michal,
Quoted 37 lines · Arnd Bergmann
Quoted 35 lines · On Tuesday 19 January 2016 14:22:13 James Hogan wrote:
Quoted 17 lines · Arnd Bergmann
Quoted 16 lines · On Tuesday 19 January 2016 13:37:50 James Hogan wrote:
When a header file is removed from generic-y (often accompanied by the addition of an arch specific header), the generated wrapper file will persist, and in some cases may still take precedence over the new arch header.
For example commit
f1fe2d21f4e1("MIPS: Add definitions for extended context") removeducontext.hfrom generic-y inarch/mips/include/asm/, and added anarch/mips/include/uapi/asm/ucontext.h. The continued use of the wrapper when reusing a dirty build tree resulted in build failures inarch/mips/kernel/signal.c:
arch/mips/kernel/signal.c: In function ‘sc_to_extcontext’:arch/mips/kernel/signal.c:142:12: error: ‘struct ucontext’ has no member named ‘uc_extcontext’return &uc->uc_extcontext; ^Quoted 7 lines
Quoted 3 lines
Fix by detecting and removing wrapper headers in generated header directories that do not correspond to a filename in generic-y, genhdr-y, or the newly introduced generated-y.
Good idea.
Acked-by Arnd Bergmann <[email protected]>Thanks Arnd
Quoted 2 lines
Can you merge this through the mips tree, or do you need me to pick it up through asm-generic?
I was envisaging the kbuild tree tbh, but I don't really mind how it gets merged. This patch depends on patch 1, which adds generated-y to x86 so we don't delete their other generated headers, but other than that it doesn't really have any dependencies.
Quoted 5 lines
Quoted 2 lines
Ok, the kbuild tree works fine too, and I guess the x86 tree would also be fine if that helps avoid the dependency.
Were you okay to take these patches, or would you prefer they go via the MIPS tree?
I'm keen for these two patches to make their way upstream one way or another.
Ralf: Since it affects MIPS, would you be able to take them?
Cheers
James
Quoted 16 lines · James Hogan
When a header file is removed from generic-y (often accompanied by the addition of an arch specific header), the generated wrapper file will persist, and in some cases may still take precedence over the new arch header.
For example commit
f1fe2d21f4e1("MIPS: Add definitions for extended context") removeducontext.hfrom generic-y inarch/mips/include/asm/, and added anarch/mips/include/uapi/asm/ucontext.h. The continued use of the wrapper when reusing a dirty build tree resulted in build failures inarch/mips/kernel/signal.c:
arch/mips/kernel/signal.c: In function ‘sc_to_extcontext’:arch/mips/kernel/signal.c:142:12: error: ‘struct ucontext’ has no member named ‘uc_extcontext’return &uc->uc_extcontext; ^
Quoted 8 lines
Fix by detecting and removing wrapper headers in generated header directories that do not correspond to a filename in generic-y, genhdr-y, or the newly introduced generated-y.
Reported-by Jacek Anaszewski <[email protected]>Reported-by Hauke Mehrtens <[email protected]>Reported-by Heinrich Schuchardt <[email protected]>Signed-off-by James Hogan <[email protected]>
Thanks for looking into this James.
Quoted 49 lines
Cc Michal Marek <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Ralf Baechle <[email protected]>Cc Paul Burton <[email protected]>Cc Florian Fainelli <[email protected]>---
Changes in v2:
- Rewrite a bit, drawing inspiration from Makefile.headersinst.
- Exclude genhdr-y and generated-y (thanks to kbuild test robot).--- scripts/Makefile.asm-generic | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)scripts/Makefile.asm-generic +16
-1@@ -13,11 +13,26 @@ include scripts/Kbuild.include # Create output directory if not already present _dummy := $(shell [ -d $(obj) ] || mkdir -p $(obj)) +# Stale wrappers when the corresponding files are removed from generic-y +# need removing. +generated-y := $(generic-y) $(genhdr-y) $(generated-y) +all-files := $(patsubst %, $(obj)/%, $(generated-y)) +old-headers := $(wildcard $(obj)/*.h) +unwanted := $(filter-out $(all-files),$(old-headers)) + quiet_cmd_wrap = WRAP $@ cmd_wrap = echo "\#include <asm-generic/$*.h>" >$@ -all: $(patsubst %, $(obj)/%, $(generic-y)) +quiet_cmd_remove = REMOVE $(unwanted) +cmd_remove = rm -f $(unwanted) + +all: $(patsubst %, $(obj)/%, $(generic-y)) FORCE + $(if $(unwanted),$(call cmd,remove),) @: $(obj)/%.h: $(call cmd,wrap) + +.PHONY: $(PHONY) +PHONY += FORCE +FORCE: ;
Signature · 1 line
Florian
Quoted 45 lines · James Hogan
When a header file is removed from generic-y (often accompanied by the addition of an arch specific header), the generated wrapper file will persist, and in some cases may still take precedence over the new arch header.
For example commit
f1fe2d21f4e1("MIPS: Add definitions for extended context") removeducontext.hfrom generic-y inarch/mips/include/asm/, and added anarch/mips/include/uapi/asm/ucontext.h. The continued use of the wrapper when reusing a dirty build tree resulted in build failures inarch/mips/kernel/signal.c:
arch/mips/kernel/signal.c: In function ‘sc_to_extcontext’:arch/mips/kernel/signal.c:142:12: error: ‘struct ucontext’ has no member named ‘uc_extcontext’return &uc->uc_extcontext; ^Fix by detecting and removing wrapper headers in generated header directories that do not correspond to a filename in generic-y, genhdr-y, or the newly introduced generated-y.
Reported-by Jacek Anaszewski <[email protected]>Reported-by Hauke Mehrtens <[email protected]>Reported-by Heinrich Schuchardt <[email protected]>Signed-off-by James Hogan <[email protected]>Cc Michal Marek <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Ralf Baechle <[email protected]>Cc Paul Burton <[email protected]>Cc Florian Fainelli <[email protected]>---
Changes in v2:
- Rewrite a bit, drawing inspiration from Makefile.headersinst.
- Exclude genhdr-y and generated-y (thanks to kbuild test robot).--- scripts/Makefile.asm-generic | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)scripts/Makefile.asm-generic +0
-0
scripts/Makefile.asm-generic include scripts/Kbuild.include · line 21
Quoted 9 lines
scripts/Makefile.asm-generic +6
-0@@ -13,11 +13,26 @@ include scripts/Kbuild.include # Create output directory if not already present _dummy := $(shell [ -d $(obj) ] || mkdir -p $(obj)) +# Stale wrappers when the corresponding files are removed from generic-y +# need removing. +generated-y := $(generic-y) $(genhdr-y) $(generated-y) +all-files := $(patsubst %, $(obj)/%, $(generated-y)) +old-headers := $(wildcard $(obj)/*.h) +unwanted := $(filter-out $(all-files),$(old-headers))
Hi James,
Thanks a bunch for fixing this!
Though is it my sleepy self or are all-files & old-headers misnomers? That is, isn't all-files actually a list of headers to be kept, and old-headers actually the list of all (header) files?
Thanks,
Paul
Quoted 21 lines
scripts/Makefile.asm-generic +10
-1+ quiet_cmd_wrap = WRAP $@ cmd_wrap = echo "\#include <asm-generic/$*.h>" >$@ -all: $(patsubst %, $(obj)/%, $(generic-y)) +quiet_cmd_remove = REMOVE $(unwanted) +cmd_remove = rm -f $(unwanted) + +all: $(patsubst %, $(obj)/%, $(generic-y)) FORCE + $(if $(unwanted),$(call cmd,remove),) @: $(obj)/%.h: $(call cmd,wrap) + +.PHONY: $(PHONY) +PHONY += FORCE +FORCE: ;Signature · 1 line
2.4.10
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 58 lines · Paul Burton
Quoted 45 lines · James Hogan
When a header file is removed from generic-y (often accompanied by the addition of an arch specific header), the generated wrapper file will persist, and in some cases may still take precedence over the new arch header.
For example commit
f1fe2d21f4e1("MIPS: Add definitions for extended context") removeducontext.hfrom generic-y inarch/mips/include/asm/, and added anarch/mips/include/uapi/asm/ucontext.h. The continued use of the wrapper when reusing a dirty build tree resulted in build failures inarch/mips/kernel/signal.c:
arch/mips/kernel/signal.c: In function ‘sc_to_extcontext’:arch/mips/kernel/signal.c:142:12: error: ‘struct ucontext’ has no member named ‘uc_extcontext’return &uc->uc_extcontext; ^Fix by detecting and removing wrapper headers in generated header directories that do not correspond to a filename in generic-y, genhdr-y, or the newly introduced generated-y.
Reported-by Jacek Anaszewski <[email protected]>Reported-by Hauke Mehrtens <[email protected]>Reported-by Heinrich Schuchardt <[email protected]>Signed-off-by James Hogan <[email protected]>Cc Michal Marek <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Ralf Baechle <[email protected]>Cc Paul Burton <[email protected]>Cc Florian Fainelli <[email protected]>---
Changes in v2:
- Rewrite a bit, drawing inspiration from Makefile.headersinst.
- Exclude genhdr-y and generated-y (thanks to kbuild test robot).--- scripts/Makefile.asm-generic | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)scripts/Makefile.asm-generic +0
-0scripts/Makefile.asm-generic include scripts/Kbuild.include · line 21
Quoted 9 lines
scripts/Makefile.asm-generic +6
-0@@ -13,11 +13,26 @@ include scripts/Kbuild.include # Create output directory if not already present _dummy := $(shell [ -d $(obj) ] || mkdir -p $(obj)) +# Stale wrappers when the corresponding files are removed from generic-y +# need removing. +generated-y := $(generic-y) $(genhdr-y) $(generated-y) +all-files := $(patsubst %, $(obj)/%, $(generated-y)) +old-headers := $(wildcard $(obj)/*.h) +unwanted := $(filter-out $(all-files),$(old-headers))Hi James,
Quoted 1 line
Thanks a bunch for fixing this!
FTR, I noticed yesterday it fixes a similar case when switching v4.3 to v4.4 too:
arch/mips/kernel/../../../fs/binfmt_elf.c In function ‘create_elf_tables’:./arch/mips/include/asm/elf.h +425 :14: error: ‘AT_SYSINFO_EHDR’ undeclared (first use in this function)
NEW_AUX_ENT(AT_SYSINFO_EHDR, \
^Due to commit ebb5e78cc634 ("MIPS: Initial implementation of a VDSO") adding uapi/asm/auxvec.h and changing generic-y to header-y. Should ucontext.h be exported via header-y too?
With these patches, it removes the stale file:
REMOVE arch/mips/include/generated/uapi/asm/auxvec.h
Quoted 4 lines
Though is it my sleepy self or are all-files & old-headers misnomers? That is, isn't all-files actually a list of headers to be kept, and old-headers actually the list of all (header) files?
I've followed the naming in Makefile.headersinst. I read all-files as "all the files we care about" (i.e. its a combination of several sets of generated files, hence "all") and old-headers as in "existing headers" (since it won't include files which haven't been generated yet).
all-files could perhaps be renamed new-headers, but that could be misleading too.
Cheers
James
Quoted 25 lines
Thanks,
PaulQuoted 21 lines
scripts/Makefile.asm-generic +10
-1+ quiet_cmd_wrap = WRAP $@ cmd_wrap = echo "\#include <asm-generic/$*.h>" >$@ -all: $(patsubst %, $(obj)/%, $(generic-y)) +quiet_cmd_remove = REMOVE $(unwanted) +cmd_remove = rm -f $(unwanted) + +all: $(patsubst %, $(obj)/%, $(generic-y)) FORCE + $(if $(unwanted),$(call cmd,remove),) @: $(obj)/%.h: $(call cmd,wrap) + +.PHONY: $(PHONY) +PHONY += FORCE +FORCE: ;Signature · 1 line
2.4.10