Skip to contentSkip to conversation

kconfig: Prepend interpeters when calling scripts

PATCH In review Ujjwal Kumar 3 participants 12 messages Sep 25, 2020 – Sep 30, 2020 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kconfig: Prepend interpeters when calling scripts

---
 init/Kconfig | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
init/Kconfig +8 -8
index 91456ac0ef20..9f6ec575e3d1 100644
@@ -30,12 +30,12 @@ config CC_IS_GCC
 
 config GCC_VERSION
 	int
-	default $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC
+	default $(shell,sh $(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC
 	default 0
 
 config LD_VERSION
 	int
-	default $(shell,$(LD) --version | $(srctree)/scripts/ld-version.sh)
+	default $(shell,$(LD) --version | awk -f $(srctree)/scripts/ld-version.sh)
 
 config CC_IS_CLANG
 	def_bool $(success,echo "$(CC_VERSION_TEXT)" | grep -q clang)
@@ -45,20 +45,20 @@ config LD_IS_LLD
 
 config CLANG_VERSION
 	int
-	default $(shell,$(srctree)/scripts/clang-version.sh $(CC))
+	default $(shell,sh $(srctree)/scripts/clang-version.sh $(CC))
 
 config CC_CAN_LINK
 	bool
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag)) if 64BIT
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag))
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag)) if 64BIT
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag))
 
 config CC_CAN_LINK_STATIC
 	bool
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag) -static) if 64BIT
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag) -static)
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag) -static) if 64BIT
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag) -static)
 
 config CC_HAS_ASM_GOTO
-	def_bool $(success,$(srctree)/scripts/gcc-goto.sh $(CC))
+	def_bool $(success,sh $(srctree)/scripts/gcc-goto.sh $(CC))
 
 config CC_HAS_ASM_GOTO_OUTPUT
 	depends on CC_HAS_ASM_GOTO
Signature · 6 lines
2.26.2

_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees
#1 lore raw
Quoted 42 lines · Ujjwal Kumar
---
 init/Kconfig | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
init/Kconfig +5 -6
index 91456ac0ef20..9f6ec575e3d1 100644
@@ -30,12 +30,12 @@ config CC_IS_GCC
 
 config GCC_VERSION
 	int
-	default $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC
+	default $(shell,sh $(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC
 	default 0
 
 config LD_VERSION
 	int
-	default $(shell,$(LD) --version | $(srctree)/scripts/ld-version.sh)
+	default $(shell,$(LD) --version | awk -f $(srctree)/scripts/ld-version.sh)
 
 config CC_IS_CLANG
 	def_bool $(success,echo "$(CC_VERSION_TEXT)" | grep -q clang)
@@ -45,20 +45,20 @@ config LD_IS_LLD
 
 config CLANG_VERSION
 	int
-	default $(shell,$(srctree)/scripts/clang-version.sh $(CC))
+	default $(shell,sh $(srctree)/scripts/clang-version.sh $(CC))
 
 config CC_CAN_LINK
 	bool
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag)) if 64BIT
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag))
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag)) if 64BIT
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag))
 
 config CC_CAN_LINK_STATIC
 	bool
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag) -static) if 64BIT

init/Kconfig config LD_IS_LLD · line 64

Quoted 10 lines
init/Kconfig +3 -2
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag) -static)
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag) -static) if 64BIT
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag) -static)
 
 config CC_HAS_ASM_GOTO
-	def_bool $(success,$(srctree)/scripts/gcc-goto.sh $(CC))
+	def_bool $(success,sh $(srctree)/scripts/gcc-goto.sh $(CC))
 
 config CC_HAS_ASM_GOTO_OUTPUT
 	depends on CC_HAS_ASM_GOTO

To unset the executable bit I ran following command:
for i in $(find -executable -type f); do chmod -x $i; done

One thing that I noticed is, once we initiate a kernel build some programs are compiled and removing their executable bit is not desirable. Even after performing "make clean" some of those programs still persist and removing their executable bit fails any further attempt to build the kernel.

For instance, tools/objtool/objtool file. In my case I had a total of 19 files remaining after performing make clean.

Ujjwal Kumar
_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#2 lore raw
Quoted 4 lines · Ujjwal Kumar

To unset the executable bit I ran following command:
for i in $(find -executable -type f); do chmod -x $i; done

Quoted 8 lines

One thing that I noticed is, once we initiate a kernel build some programs are compiled and removing their executable bit is not desirable. Even after performing "make clean" some of those programs still persist and removing their executable bit fails any further attempt to build the kernel.

For instance, tools/objtool/objtool file. In my case I had a total of 19 files remaining after performing make clean.

Good observation.

You can properly clean your build with make mrproper, but maybe that is not even enough... all generated files might not mean _all generated tools_, such as objtool, etc.

$ make help

Cleaning targets:
clean - Remove most generated files but keep the config and
enough build support to build external modules
mrproper - Remove all generated files + config + various backup files
distclean - mrproper + remove editor backup and patch files

[...]

After for i in $(find -executable -type f); do chmod -x $i; done:
I saw those failures (probably as you did as well):

*** Default configuration is based on 'x86_64_defconfig'
sh: 1: ./scripts/gcc-version.sh: Permission denied init/Kconfig:34: syntax error
init/Kconfig:33: invalid statement
init/Kconfig:34: invalid statement
sh: 1: ./scripts/ld-version.sh: Permission denied init/Kconfig:39: syntax error
init/Kconfig:38: invalid statement
sh: 1: ./scripts/clang-version.sh: Permission denied init/Kconfig:49: syntax error
init/Kconfig:48: invalid statement scripts/kconfig/Makefile:80: recipe for target 'defconfig' failed make[1]: *** [defconfig] Error 1
Makefile:606: recipe for target 'defconfig' failed
make: *** [defconfig] Error 2

After your patch was applied, they disappear. So, it does the job. More comments on the patch itself later.

make defconfig works.

make then fails:

HOSTCC /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o
/bin/sh: 1: /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool//fixdep: Permission denied /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/build/Makefile.build:96: recipe for target '/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o' failed
make[4]: *** [/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o] Error 126 Makefile:43: recipe for target '/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep-in.o' failed make[3]: *** [/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep-in.o] Error 2 /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/build/Makefile.include:5: recipe for target 'fixdep' failed make[2]: *** [fixdep] Error 2
Makefile:68: recipe for target 'objtool' failed
make[1]: *** [objtool] Error 2
Makefile:1885: recipe for target 'tools/objtool' failed
make: *** [tools/objtool] Error 2
make: *** Waiting for unfinished jobs....

The make target also goes into tools/build/ and requires some targets there and then uses those tools during the further build.

So, we need some more patches here as well.

Can you share the list of 19 files that remained even with make clean?

Then, we go through them and check if they disappear with make mrproper or with a different make target to clean them. Potentially, we also hook those identified targets up into make mrproper then.

Lukas
_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#3 lore raw
Quoted 30 lines · Lukas Bulwahn
Quoted 4 lines · Ujjwal Kumar

To unset the executable bit I ran following command:
for i in $(find -executable -type f); do chmod -x $i; done

Quoted 8 lines

One thing that I noticed is, once we initiate a kernel build some programs are compiled and removing their executable bit is not desirable. Even after performing "make clean" some of those programs still persist and removing their executable bit fails any further attempt to build the kernel.

For instance, tools/objtool/objtool file. In my case I had a total of 19 files remaining after performing make clean.

Good observation.

You can properly clean your build with make mrproper, but maybe that is not even enough... all generated files might not mean _all generated tools_, such as objtool, etc.

$ make help

Cleaning targets:
clean - Remove most generated files but keep the config and
enough build support to build external modules
mrproper - Remove all generated files + config + various backup files
distclean - mrproper + remove editor backup and patch files

Quoted 4 lines

[...]

After for i in $(find -executable -type f); do chmod -x $i; done:
I saw those failures (probably as you did as well):

Yes these are the same errors that I received.

Quoted 18 lines

*** Default configuration is based on 'x86_64_defconfig'
sh: 1: ./scripts/gcc-version.sh: Permission denied init/Kconfig:34: syntax error
init/Kconfig:33: invalid statement
init/Kconfig:34: invalid statement
sh: 1: ./scripts/ld-version.sh: Permission denied init/Kconfig:39: syntax error
init/Kconfig:38: invalid statement
sh: 1: ./scripts/clang-version.sh: Permission denied init/Kconfig:49: syntax error
init/Kconfig:48: invalid statement scripts/kconfig/Makefile:80: recipe for target 'defconfig' failed make[1]: *** [defconfig] Error 1
Makefile:606: recipe for target 'defconfig' failed
make: *** [defconfig] Error 2

Quoted 5 lines

After your patch was applied, they disappear. So, it does the job. More comments on the patch itself later.

make defconfig works.

Yes, these errors appeared when we unset executable bits on files after performing 'make clean' and then perform a kernel build.

Quoted 17 lines

make then fails:

HOSTCC /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o
/bin/sh: 1: /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool//fixdep: Permission denied /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/build/Makefile.build:96: recipe for target '/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o' failed
make[4]: *** [/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o] Error 126 Makefile:43: recipe for target '/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep-in.o' failed make[3]: *** [/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep-in.o] Error 2 /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/build/Makefile.include:5: recipe for target 'fixdep' failed make[2]: *** [fixdep] Error 2
Makefile:68: recipe for target 'objtool' failed
make[1]: *** [objtool] Error 2
Makefile:1885: recipe for target 'tools/objtool' failed
make: *** [tools/objtool] Error 2
make: *** Waiting for unfinished jobs....

Quoted 7 lines

The make target also goes into tools/build/ and requires some targets there and then uses those tools during the further build.

So, we need some more patches here as well.

Can you share the list of 19 files that remained even with make clean?

After performing a make clean, the files that persist are:
./tools/objtool/objtool
./tools/objtool/fixdep
./scripts/sorttable
./scripts/mod/mk_elfconfig
./scripts/mod/modpost
./scripts/unifdef
./scripts/basic/fixdep
./scripts/bin2c
./scripts/extract-cert
./scripts/kconfig/conf
./scripts/selinux/mdp/mdp ./scripts/selinux/genheaders/genheaders ./scripts/asn1_compiler
./scripts/kallsyms
./scripts/sign-file
./scripts/genksyms/genksyms ./scripts/recordmcount
./scripts/insert-sys-cert
./scripts/dtc/dtc

After performing 'make mrproper'/'make distclean' the files that persist are:
./tools/objtool/objtool
./tools/objtool/fixdep

Quoted 3 lines

Then, we go through them and check if they disappear with make mrproper or with a different make target to clean them. Potentially, we also hook those identified targets up into make mrproper then.

Is it appropriate to include the files here:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/Makefile?h=next-20200925#n1466
?
Or should it be included inside tools/objtool/Makefile here:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/tools/objtool/Makefile?h=next-20200925#n78
?
In the second link, it appears to me that the files are already included for clean task but still they are not cleaned.

Quoted 2 lines
Quoted 1 line

Lukas

Thanks
Ujjwal Kumar
_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#4 lore raw
Quoted 111 lines · Ujjwal Kumar
Quoted 30 lines · Lukas Bulwahn
Quoted 4 lines · Ujjwal Kumar

To unset the executable bit I ran following command:
for i in $(find -executable -type f); do chmod -x $i; done

Quoted 8 lines

One thing that I noticed is, once we initiate a kernel build some programs are compiled and removing their executable bit is not desirable. Even after performing "make clean" some of those programs still persist and removing their executable bit fails any further attempt to build the kernel.

For instance, tools/objtool/objtool file. In my case I had a total of 19 files remaining after performing make clean.

Good observation.

You can properly clean your build with make mrproper, but maybe that is not even enough... all generated files might not mean _all generated tools_, such as objtool, etc.

$ make help

Cleaning targets:
clean - Remove most generated files but keep the config and
enough build support to build external modules
mrproper - Remove all generated files + config + various backup files
distclean - mrproper + remove editor backup and patch files

Quoted 4 lines

[...]

After for i in $(find -executable -type f); do chmod -x $i; done:
I saw those failures (probably as you did as well):

Yes these are the same errors that I received.

Quoted 18 lines

*** Default configuration is based on 'x86_64_defconfig'
sh: 1: ./scripts/gcc-version.sh: Permission denied init/Kconfig:34: syntax error
init/Kconfig:33: invalid statement
init/Kconfig:34: invalid statement
sh: 1: ./scripts/ld-version.sh: Permission denied init/Kconfig:39: syntax error
init/Kconfig:38: invalid statement
sh: 1: ./scripts/clang-version.sh: Permission denied init/Kconfig:49: syntax error
init/Kconfig:48: invalid statement scripts/kconfig/Makefile:80: recipe for target 'defconfig' failed make[1]: *** [defconfig] Error 1
Makefile:606: recipe for target 'defconfig' failed
make: *** [defconfig] Error 2

Quoted 5 lines

After your patch was applied, they disappear. So, it does the job. More comments on the patch itself later.

make defconfig works.

Yes, these errors appeared when we unset executable bits on files after performing 'make clean' and then perform a kernel build.

Quoted 17 lines

make then fails:

HOSTCC /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o
/bin/sh: 1: /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool//fixdep: Permission denied /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/build/Makefile.build:96: recipe for target '/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o' failed
make[4]: *** [/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o] Error 126 Makefile:43: recipe for target '/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep-in.o' failed make[3]: *** [/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep-in.o] Error 2 /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/build/Makefile.include:5: recipe for target 'fixdep' failed make[2]: *** [fixdep] Error 2
Makefile:68: recipe for target 'objtool' failed
make[1]: *** [objtool] Error 2
Makefile:1885: recipe for target 'tools/objtool' failed
make: *** [tools/objtool] Error 2
make: *** Waiting for unfinished jobs....

Quoted 7 lines

The make target also goes into tools/build/ and requires some targets there and then uses those tools during the further build.

So, we need some more patches here as well.

Can you share the list of 19 files that remained even with make clean?

After performing a make clean, the files that persist are:
./tools/objtool/objtool
./tools/objtool/fixdep
./scripts/sorttable
./scripts/mod/mk_elfconfig
./scripts/mod/modpost
./scripts/unifdef
./scripts/basic/fixdep
./scripts/bin2c
./scripts/extract-cert
./scripts/kconfig/conf
./scripts/selinux/mdp/mdp ./scripts/selinux/genheaders/genheaders ./scripts/asn1_compiler
./scripts/kallsyms
./scripts/sign-file
./scripts/genksyms/genksyms ./scripts/recordmcount
./scripts/insert-sys-cert
./scripts/dtc/dtc

Quoted 4 lines

After performing 'make mrproper'/'make distclean' the files that persist are:
./tools/objtool/objtool
./tools/objtool/fixdep

These files can be removed using following command from srctree:
make tools/objtool_clean

So after performing 'make mrproper' and 'make tools/objtool_clean', we get rid of all the compiled programs.

Instead of 'make tools/objtool_clean' we can perform following commands:
From the srctree directory, make tools/clean
From the tools directory, make clean
From the tools directory, make objtool_clean
to remove the 2 files.

Quoted 4 lines
Quoted 4 lines

Then, we go through them and check if they disappear with make mrproper or with a different make target to clean them. Potentially, we also hook those identified targets up into make mrproper then.

I tried hooking the clean task with 'make mrproper' in top-level Makefile as follows:

$ git diff Makefile
Makefile +1 -0
index 5f1399a576d4..43ab6ce98c64 100644
@@ -1498,6 +1498,7 @@ $(mrproper-dirs):
        $(Q)$(MAKE) $(clean)=$(patsubst _mrproper_%,%,$@)
 
 mrproper: clean $(mrproper-dirs)
+       $(Q)$(MAKE) tools/objtool_clean
        $(call cmd,rmfiles)
 
 # distclean

---

Instead of '$(Q)$(MAKE) tools/objtool_clean' we can perform following command:
'$(Q)$(MAKE) tools/clean' to clean the entire tools/ directory.

Quoted 10 lines

Is it appropriate to include the files here:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/Makefile?h=next-20200925#n1466
?
Or should it be included inside tools/objtool/Makefile here:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/tools/objtool/Makefile?h=next-20200925#n78
?
In the second link, it appears to me that the files are already included for clean task but still they are not cleaned.

Quoted 6 lines
Quoted 2 lines
Quoted 1 line

Lukas

Thanks
Ujjwal Kumar

Ujjwal Kumar
_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#5 lore raw
Quoted 152 lines · Ujjwal Kumar
Quoted 111 lines · Ujjwal Kumar
Quoted 30 lines · Lukas Bulwahn
Quoted 4 lines · Ujjwal Kumar

To unset the executable bit I ran following command:
for i in $(find -executable -type f); do chmod -x $i; done

Quoted 8 lines

One thing that I noticed is, once we initiate a kernel build some programs are compiled and removing their executable bit is not desirable. Even after performing "make clean" some of those programs still persist and removing their executable bit fails any further attempt to build the kernel.

For instance, tools/objtool/objtool file. In my case I had a total of 19 files remaining after performing make clean.

Good observation.

You can properly clean your build with make mrproper, but maybe that is not even enough... all generated files might not mean _all generated tools_, such as objtool, etc.

$ make help

Cleaning targets:
clean - Remove most generated files but keep the config and
enough build support to build external modules
mrproper - Remove all generated files + config + various backup files
distclean - mrproper + remove editor backup and patch files

Quoted 4 lines

[...]

After for i in $(find -executable -type f); do chmod -x $i; done:
I saw those failures (probably as you did as well):

Yes these are the same errors that I received.

Quoted 18 lines

*** Default configuration is based on 'x86_64_defconfig'
sh: 1: ./scripts/gcc-version.sh: Permission denied init/Kconfig:34: syntax error
init/Kconfig:33: invalid statement
init/Kconfig:34: invalid statement
sh: 1: ./scripts/ld-version.sh: Permission denied init/Kconfig:39: syntax error
init/Kconfig:38: invalid statement
sh: 1: ./scripts/clang-version.sh: Permission denied init/Kconfig:49: syntax error
init/Kconfig:48: invalid statement scripts/kconfig/Makefile:80: recipe for target 'defconfig' failed make[1]: *** [defconfig] Error 1
Makefile:606: recipe for target 'defconfig' failed
make: *** [defconfig] Error 2

Quoted 5 lines

After your patch was applied, they disappear. So, it does the job. More comments on the patch itself later.

make defconfig works.

Yes, these errors appeared when we unset executable bits on files after performing 'make clean' and then perform a kernel build.

Quoted 17 lines

make then fails:

HOSTCC /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o
/bin/sh: 1: /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool//fixdep: Permission denied /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/build/Makefile.build:96: recipe for target '/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o' failed
make[4]: *** [/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep.o] Error 126 Makefile:43: recipe for target '/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep-in.o' failed make[3]: *** [/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/objtool/fixdep-in.o] Error 2 /home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/torvalds/linux/tools/build/Makefile.include:5: recipe for target 'fixdep' failed make[2]: *** [fixdep] Error 2
Makefile:68: recipe for target 'objtool' failed
make[1]: *** [objtool] Error 2
Makefile:1885: recipe for target 'tools/objtool' failed
make: *** [tools/objtool] Error 2
make: *** Waiting for unfinished jobs....

Quoted 7 lines

The make target also goes into tools/build/ and requires some targets there and then uses those tools during the further build.

So, we need some more patches here as well.

Can you share the list of 19 files that remained even with make clean?

After performing a make clean, the files that persist are:
./tools/objtool/objtool
./tools/objtool/fixdep
./scripts/sorttable
./scripts/mod/mk_elfconfig
./scripts/mod/modpost
./scripts/unifdef
./scripts/basic/fixdep
./scripts/bin2c
./scripts/extract-cert
./scripts/kconfig/conf
./scripts/selinux/mdp/mdp ./scripts/selinux/genheaders/genheaders ./scripts/asn1_compiler
./scripts/kallsyms
./scripts/sign-file
./scripts/genksyms/genksyms ./scripts/recordmcount
./scripts/insert-sys-cert
./scripts/dtc/dtc

Quoted 4 lines

After performing 'make mrproper'/'make distclean' the files that persist are:
./tools/objtool/objtool
./tools/objtool/fixdep

These files can be removed using following command from srctree:
make tools/objtool_clean

So after performing 'make mrproper' and 'make tools/objtool_clean', we get rid of all the compiled programs.

Instead of 'make tools/objtool_clean' we can perform following commands:
From the srctree directory, make tools/clean
From the tools directory, make clean
From the tools directory, make objtool_clean
to remove the 2 files.

Quoted 4 lines
Quoted 4 lines

Then, we go through them and check if they disappear with make mrproper or with a different make target to clean them. Potentially, we also hook those identified targets up into make mrproper then.

I tried hooking the clean task with 'make mrproper' in top-level Makefile as follows:

$ git diff Makefile
Makefile +1 -1
index 5f1399a576d4..43ab6ce98c64 100644
@@ -1498,6 +1498,7 @@ $(mrproper-dirs):
        $(Q)$(MAKE) $(clean)=$(patsubst _mrproper_%,%,$@)
 
 mrproper: clean $(mrproper-dirs)
+       $(Q)$(MAKE) tools/objtool_clean
        $(call cmd,rmfiles)
 
 # distclean
---
Quoted 4 lines

Instead of '$(Q)$(MAKE) tools/objtool_clean' we can perform following command:
'$(Q)$(MAKE) tools/clean' to clean the entire tools/ directory.

I guess that change above might need further discussion later. Let us focus on the clear patches you already identified using appropriate interpreters in all build scripts.

What is the state there? Could you already identify all places that should be touched? If so, feel free to send out your patch series as RFC patch series on this mailing list here.

Lukas
_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#7 lore raw
Quoted 6 lines · Greg KH
Quoted 1 line · Ujjwal Kumar

I know I can't take pathces with no changelog text, maybe other maintainers are more lax :(

I would like find that maintainer as well... ;)

But *serious mode* now:

This patch was only on the linux-kernel-mentees list so far.

Ujjwal and me are just using the linux-kernel-mentees to develop the patch together. Before we send it to lkml and linux-kbuild, we will have done a first review and will have a proper commit message in place.

Greg, your feedback on these patches is of course always welcome when you spot something smelly.

Lukas
_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#8 lore raw
Quoted 15 lines · Lukas Bulwahn
Quoted 6 lines · Greg KH
Quoted 1 line · Ujjwal Kumar

I know I can't take pathces with no changelog text, maybe other maintainers are more lax :(

I would like find that maintainer as well... ;)

But *serious mode* now:

Quoted 8 lines

This patch was only on the linux-kernel-mentees list so far.

Ujjwal and me are just using the linux-kernel-mentees to develop the patch together. Before we send it to lkml and linux-kbuild, we will have done a first review and will have a proper commit message in place.

Greg, your feedback on these patches is of course always welcome when you spot something smelly.

That was my "smelly" spotting :)

greg k-h
_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#9 lore raw
In reply to Ujjwal Kumar

Basically the diff is okay; already start writing the commit message.

Quoted 10 lines · Ujjwal Kumar
---
 init/Kconfig | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
init/Kconfig +0 -0
index 91456ac0ef20..9f6ec575e3d1 100644

init/Kconfig config CC_IS_GCC · line 33

Quoted 5 lines
init/Kconfig +1 -1
@@ -30,12 +30,12 @@ config CC_IS_GCC
 
 config GCC_VERSION
 	int
-	default $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC
+	default $(shell,sh $(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC

in the top-level Makefile, (./Makefile), you will find a section:

# SHELL used by kbuild
CONFIG_SHELL := sh

I think that is what is supposed to be used here.

init/Kconfig config CC_IS_GCC · line 39

Quoted 7 lines
init/Kconfig +1 -1
 	default 0
 
 config LD_VERSION
 	int
-	default $(shell,$(LD) --version | $(srctree)/scripts/ld-version.sh)
+	default $(shell,$(LD) --version | awk -f $(srctree)/scripts/ld-version.sh)
 

Further you find:

# Make variables (CC, etc...)

This section defines variables for each interpreter, in case someone wants to modify/override the default interpreters.

You can find there:

AWK = awk

Use that.

init/Kconfig config LD_IS_LLD · line 49

Quoted 9 lines
init/Kconfig +1 -1
 config CC_IS_CLANG
 	def_bool $(success,echo "$(CC_VERSION_TEXT)" | grep -q clang)
@@ -45,20 +45,20 @@ config LD_IS_LLD
 
 config CLANG_VERSION
 	int
-	default $(shell,$(srctree)/scripts/clang-version.sh $(CC))
+	default $(shell,sh $(srctree)/scripts/clang-version.sh $(CC))
 

I think CONFIG_SHELL is right here.

init/Kconfig config LD_IS_LLD · line 54

Quoted 7 lines
init/Kconfig +2 -2
 config CC_CAN_LINK
 	bool
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag)) if 64BIT
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag))
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag)) if 64BIT
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag))
 

Same here.

init/Kconfig config LD_IS_LLD · line 59

Quoted 7 lines
init/Kconfig +2 -2
 config CC_CAN_LINK_STATIC
 	bool
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag) -static) if 64BIT
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag) -static)
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag) -static) if 64BIT
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag) -static)
 

Same here.

init/Kconfig config LD_IS_LLD · line 62

Quoted 4 lines
init/Kconfig +1 -1
 config CC_HAS_ASM_GOTO
-	def_bool $(success,$(srctree)/scripts/gcc-goto.sh $(CC))
+	def_bool $(success,sh $(srctree)/scripts/gcc-goto.sh $(CC))
 

Same here.

I think this is already a good size for the patch, it solves one problem to have the basic Configuration targets in kbuild work without the need for any executable flag on repository files.

Fix it up, write an appropriate commit message, we will review here on the linux-kernel-mentees list and then it can go out into the wild wild west of the kernel mailing list.

Lukas

Quoted 6 lines
init/Kconfig +0 -0
 config CC_HAS_ASM_GOTO_OUTPUT
 	depends on CC_HAS_ASM_GOTO
Signature · 1 line
2.26.2

_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#10 lore raw
Quoted 88 lines · Lukas Bulwahn

Basically the diff is okay; already start writing the commit message.

Quoted 10 lines · Ujjwal Kumar
---
 init/Kconfig | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
init/Kconfig +0 -0
index 91456ac0ef20..9f6ec575e3d1 100644

init/Kconfig config CC_IS_GCC · line 33

Quoted 5 lines
init/Kconfig +1 -1
@@ -30,12 +30,12 @@ config CC_IS_GCC
 
 config GCC_VERSION
 	int
-	default $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC
+	default $(shell,sh $(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC

in the top-level Makefile, (./Makefile), you will find a section:

# SHELL used by kbuild
CONFIG_SHELL := sh

I think that is what is supposed to be used here.

init/Kconfig config CC_IS_GCC · line 39

Quoted 7 lines
init/Kconfig +1 -1
 	default 0
 
 config LD_VERSION
 	int
-	default $(shell,$(LD) --version | $(srctree)/scripts/ld-version.sh)
+	default $(shell,$(LD) --version | awk -f $(srctree)/scripts/ld-version.sh)
 

Further you find:

# Make variables (CC, etc...)

This section defines variables for each interpreter, in case someone wants to modify/override the default interpreters.

You can find there:

AWK = awk

Use that.

init/Kconfig config LD_IS_LLD · line 49

Quoted 9 lines
init/Kconfig +1 -1
 config CC_IS_CLANG
 	def_bool $(success,echo "$(CC_VERSION_TEXT)" | grep -q clang)
@@ -45,20 +45,20 @@ config LD_IS_LLD
 
 config CLANG_VERSION
 	int
-	default $(shell,$(srctree)/scripts/clang-version.sh $(CC))
+	default $(shell,sh $(srctree)/scripts/clang-version.sh $(CC))
 

I think CONFIG_SHELL is right here.

init/Kconfig config LD_IS_LLD · line 54

Quoted 7 lines
init/Kconfig +2 -2
 config CC_CAN_LINK
 	bool
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag)) if 64BIT
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag))
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag)) if 64BIT
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag))
 

Same here.

init/Kconfig config LD_IS_LLD · line 59

Quoted 7 lines
init/Kconfig +2 -2
 config CC_CAN_LINK_STATIC
 	bool
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag) -static) if 64BIT
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag) -static)
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag) -static) if 64BIT
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag) -static)
 

Same here.

init/Kconfig config LD_IS_LLD · line 62

Quoted 4 lines
init/Kconfig +1 -1
 config CC_HAS_ASM_GOTO
-	def_bool $(success,$(srctree)/scripts/gcc-goto.sh $(CC))
+	def_bool $(success,sh $(srctree)/scripts/gcc-goto.sh $(CC))
 

Same here.

Quoted 5 lines

I think this is already a good size for the patch, it solves one problem to have the basic Configuration targets in kbuild work without the need for any executable flag on repository files.

Fix it up, write an appropriate commit message, we will review here on the

How is the following for a commit message?
kconfig: use interpreters and not rely on x-bit to invoke scripts

Having the execute bits set wasn't something that we could depend on. Any portion of the kernel that invokes such files provides the interpreter explicitly.

Additionally, relying on x-bit is not recommended for following reasons:

- An official guidance is currently unavailable to decide whether it is mandatory to have the x-bit set.

- It is possible that the x-bit will be erased/ignored when handling patches (for example, using patch(1)).

I am wondering who should I include for 'Suggested-by:' tag?

Quoted 3 lines

linux-kernel-mentees list and then it can go out into the wild wild west of the kernel mailing list.

Yes I will make the changes.

Also, some Makefile(s) do not use '$(CONFIG_SHELL)' and instead use 'sh'. For instance,
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/scripts/Makefile.lib?h=next-20200925#n422
AND
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/arch/s390/boot/Makefile?h=next-20200925#n74

I wonder if they should be improved to maintain the consistency or left at their current state.

Quoted 9 lines

Lukas

Quoted 6 lines
init/Kconfig +0 -0
 config CC_HAS_ASM_GOTO_OUTPUT
 	depends on CC_HAS_ASM_GOTO
Signature · 1 line
2.26.2

On looking at Makefile(s), I found some places where we miss specifying the interpreter and so we are relying on executable bit again. here:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/arch/nds32/kernel/vdso/Makefile?h=next-20200925#n42

I wonder if it should go as a separate patch, simply because its Makefile now.

Thanks
Ujjwal Kumar
_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#11 lore raw
Quoted 89 lines · Ujjwal Kumar
Quoted 88 lines · Lukas Bulwahn

Basically the diff is okay; already start writing the commit message.

Quoted 10 lines · Ujjwal Kumar
---
 init/Kconfig | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
init/Kconfig +0 -0
index 91456ac0ef20..9f6ec575e3d1 100644

init/Kconfig config CC_IS_GCC · line 33

Quoted 5 lines
init/Kconfig +1 -1
@@ -30,12 +30,12 @@ config CC_IS_GCC
 
 config GCC_VERSION
 	int
-	default $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC
+	default $(shell,sh $(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC

in the top-level Makefile, (./Makefile), you will find a section:

# SHELL used by kbuild
CONFIG_SHELL := sh

I think that is what is supposed to be used here.

init/Kconfig config CC_IS_GCC · line 39

Quoted 7 lines
init/Kconfig +1 -1
 	default 0
 
 config LD_VERSION
 	int
-	default $(shell,$(LD) --version | $(srctree)/scripts/ld-version.sh)
+	default $(shell,$(LD) --version | awk -f $(srctree)/scripts/ld-version.sh)
 

Further you find:

# Make variables (CC, etc...)

This section defines variables for each interpreter, in case someone wants to modify/override the default interpreters.

You can find there:

AWK = awk

Use that.

init/Kconfig config LD_IS_LLD · line 49

Quoted 9 lines
init/Kconfig +1 -1
 config CC_IS_CLANG
 	def_bool $(success,echo "$(CC_VERSION_TEXT)" | grep -q clang)
@@ -45,20 +45,20 @@ config LD_IS_LLD
 
 config CLANG_VERSION
 	int
-	default $(shell,$(srctree)/scripts/clang-version.sh $(CC))
+	default $(shell,sh $(srctree)/scripts/clang-version.sh $(CC))
 

I think CONFIG_SHELL is right here.

init/Kconfig config LD_IS_LLD · line 54

Quoted 7 lines
init/Kconfig +2 -2
 config CC_CAN_LINK
 	bool
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag)) if 64BIT
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag))
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag)) if 64BIT
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag))
 

Same here.

init/Kconfig config LD_IS_LLD · line 59

Quoted 7 lines
init/Kconfig +2 -2
 config CC_CAN_LINK_STATIC
 	bool
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag) -static) if 64BIT
-	default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag) -static)
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag) -static) if 64BIT
+	default $(success,sh $(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m32-flag) -static)
 

Same here.

init/Kconfig config LD_IS_LLD · line 62

Quoted 4 lines
init/Kconfig +1 -1
 config CC_HAS_ASM_GOTO
-	def_bool $(success,$(srctree)/scripts/gcc-goto.sh $(CC))
+	def_bool $(success,sh $(srctree)/scripts/gcc-goto.sh $(CC))
 

Same here.

Quoted 9 lines
Quoted 5 lines

I think this is already a good size for the patch, it solves one problem to have the basic Configuration targets in kbuild work without the need for any executable flag on repository files.

Fix it up, write an appropriate commit message, we will review here on the

How is the following for a commit message?
kconfig: use interpreters and not rely on x-bit to invoke scripts

kconfig: use interpreters to invoke scripts

Quoted 1 line

Having the execute bits set wasn't something that we could depend on.

We cannot rely on execute bits to be set on files in the repository.

Quoted 3 lines

Any portion of the kernel that invokes such files provides the interpreter explicitly.

The build script should use the explicit interpreter when invoking any script from the repository.

Quoted 5 lines

Additionally, relying on x-bit is not recommended for following reasons:

- An official guidance is currently unavailable to decide whether it is mandatory to have the x-bit set.

Well, I think it is better to prepare an "official guidance" as commit in the kernel documentation and then simply refer to that. That will be a separate patch in this series.

Quoted 5 lines

- It is possible that the x-bit will be erased/ignored when handling patches (for example, using patch(1)).

I am wondering who should I include for 'Suggested-by:' tag?

I think there is nothing wrong with having everyone that suggested the work mentioned, so Andrew, Kees and me.

Also add a Link: to the discussion on lore.kernel.org as reference.

Quoted 6 lines
Quoted 3 lines

linux-kernel-mentees list and then it can go out into the wild wild west of the kernel mailing list.

Yes I will make the changes.

Quoted 9 lines

Also, some Makefile(s) do not use '$(CONFIG_SHELL)' and instead use 'sh'. For instance,
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/scripts/Makefile.lib?h=next-20200925#n422
AND
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/arch/s390/boot/Makefile?h=next-20200925#n74

I wonder if they should be improved to maintain the consistency or left at their current state.

Again, another good catch. Make that yet another patch.

Quoted 10 lines
Quoted 9 lines

Lukas

Quoted 6 lines
init/Kconfig +0 -0
 config CC_HAS_ASM_GOTO_OUTPUT
 	depends on CC_HAS_ASM_GOTO
Signature · 1 line
2.26.2
Quoted 6 lines

On looking at Makefile(s), I found some places where we miss specifying the interpreter and so we are relying on executable bit again. here:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/arch/nds32/kernel/vdso/Makefile?h=next-20200925#n42

I wonder if it should go as a separate patch, simply because its Makefile now.

Yes, another good catch and patch.

Collect all those patches and then send them out as patch series.

So something like:

- Documentation patch: describe in the kbuild documentation not to rely on the x-bit.
- patch on kconfig (this one)
- patch on Makefile.lib
- patch on s390
- ... and more if we found some more issues

The kernel community can handle many small patches, don't worry. Also, there is no need to rush with some first patch now, we can collect all patches and then send them out as one series. These are not showstopper problems, but some real nice small cleanups nobody noticed so far.

If I find some time, I can write a quick patch for the Documentation and we can review that together.

Lukas
_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees