Skip to contentSkip to conversation

Kbuild: lto: add make version checking for MODVERSIONS

PATCH NAK Lecopzer Chen 4 participants 11 messages Jul 2, 2021 – Jul 9, 2021 Kbuild

lore
Share & tools
lore
#0 lore raw

v3 cover Kbuild: lto: add make version checking for MODVERSIONS

LTO with MODVERSIONS will fail in generating correct CRC because the makefile rule doesn't work for make with version 3.8X.

Refer to [1]:

Quoted 13 lines

When building modules(CONFIG_...=m), I found some of module versions are incorrect and set to 0.
This can be found in build log for first clean build which shows

WARNING: EXPORT symbol "XXXX" [drivers/XXX/XXX.ko] version generation failed, symbol will not be versioned.

But in second build(incremental build), the WARNING disappeared and the module version becomes valid CRC and make someone who want to change modules without updating kernel image can't insert their modules.

The problematic code is
+ $(foreach n, $(filter-out FORCE,$^), \

Review

Quoted 2 lines
(quoted diff) +2 -0
+   $(if $(wildcard $(n).symversions),      \
+     ; cat $(n).symversions >> [email protected]))

The issue is fixed when make version upgrading to 4.2.

Thus we need to check make version during selecting on LTO Kconfig. Add CONFIG_MAKE_VERSION which means MAKE_VERSION in canonical digits for arithmetic comparisons.

[1] https://lore.kernel.org/lkml/[email protected]/

----
v3:
- fix comment in make-version.sh

v2:
https://lore.kernel.org/lkml/[email protected]/
- change MAKE_VERSION_INT to MAKE_VERSION
- remove $(make-version)
- tweak commit message

v1:
https://lore.kernel.org/lkml/[email protected]/
----

Lecopzer Chen (2):
Kbuild: lto: add CONFIG_MAKE_VERSION
Kbuild: lto: add make version checking

 Makefile                |  2 +-
 arch/Kconfig            |  1 +
 init/Kconfig            |  4 ++++
 scripts/make-version.sh | 13 +++++++++++++
 4 files changed, 19 insertions(+), 1 deletion(-)
 create mode 100755 scripts/make-version.sh
Signature · 1 line
2.18.0
#1 lore raw

PATCH v3 1/2 Kbuild: lto: add CONFIG_MAKE_VERSION

To check the GNU make version. Used by the LTO Kconfig.

LTO with MODVERSIONS will fail in generating correct CRC because the makefile rule doesn't work for make with version 3.8X.[1]

Thus we need to check make version during selecting on LTO Kconfig. Add CONFIG_MAKE_VERSION which means MAKE_VERSION in canonical digits for arithmetic comparisons.

[1] https://lore.kernel.org/lkml/[email protected]/

---
 Makefile                |  2 +-
 init/Kconfig            |  4 ++++
 scripts/make-version.sh | 13 +++++++++++++
 3 files changed, 18 insertions(+), 1 deletion(-)
 create mode 100755 scripts/make-version.sh
Makefile +1 -1
index 88888fff4c62..2402745b2ba9 100644
@@ -516,7 +516,7 @@ CLANG_FLAGS :=
 
 export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE LD CC
 export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
-export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
+export PERL PYTHON3 CHECK CHECKFLAGS MAKE MAKE_VERSION UTS_MACHINE HOSTCXX
 export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD
 export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE
 
init/Kconfig +4 -0
index 55f9f7738ebb..ecc110504f87 100644
@@ -86,6 +86,10 @@ config CC_HAS_ASM_INLINE
 config CC_HAS_NO_PROFILE_FN_ATTR
 	def_bool $(success,echo '__attribute__((no_profile_instrument_function)) int x();' | $(CC) -x c - -c -o /dev/null -Werror)
 
+config MAKE_VERSION
+	int
+	default $(shell,$(srctree)/scripts/make-version.sh $(MAKE_VERSION))
+
 config CONSTRUCTORS
 	bool
 
scripts/make-version.sh new file +13 -0
new file mode 100755
index 000000000000..3a451db3c067
@@ -0,0 +1,13 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Print the GNU make version in a 5 or 6-digit form.
+
+set -e
+
+# Convert the version string x.y.z to a canonical 5 or 6-digit form.
+IFS=.
+set -- $1
+
+# If the 2nd or 3rd field is missing, fill it with a zero.
+echo $((10000 * $1 + 100 * ${2:-0} + ${3:-0}))
Signature · 1 line
2.18.0
maintainer Reviewed-by on v3 1/2
#2 lore raw

Re: [PATCH v3 1/2] Kbuild: lto: add CONFIG_MAKE_VERSION

Quoted 11 lines · Lecopzer Chen

To check the GNU make version. Used by the LTO Kconfig.

LTO with MODVERSIONS will fail in generating correct CRC because the makefile rule doesn't work for make with version 3.8X.[1]

Thus we need to check make version during selecting on LTO Kconfig. Add CONFIG_MAKE_VERSION which means MAKE_VERSION in canonical digits for arithmetic comparisons.

[1] https://lore.kernel.org/lkml/[email protected]/

Quoted 56 lines
---
 Makefile                |  2 +-
 init/Kconfig            |  4 ++++
 scripts/make-version.sh | 13 +++++++++++++
 3 files changed, 18 insertions(+), 1 deletion(-)
 create mode 100755 scripts/make-version.sh
Makefile +1 -1
index 88888fff4c62..2402745b2ba9 100644
@@ -516,7 +516,7 @@ CLANG_FLAGS :=
 
 export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE LD CC
 export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
-export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
+export PERL PYTHON3 CHECK CHECKFLAGS MAKE MAKE_VERSION UTS_MACHINE HOSTCXX
 export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD
 export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE
 
init/Kconfig +4 -0
index 55f9f7738ebb..ecc110504f87 100644
@@ -86,6 +86,10 @@ config CC_HAS_ASM_INLINE
 config CC_HAS_NO_PROFILE_FN_ATTR
 	def_bool $(success,echo '__attribute__((no_profile_instrument_function)) int x();' | $(CC) -x c - -c -o /dev/null -Werror)
 
+config MAKE_VERSION
+	int
+	default $(shell,$(srctree)/scripts/make-version.sh $(MAKE_VERSION))
+
 config CONSTRUCTORS
 	bool
 
scripts/make-version.sh new file +13 -0
new file mode 100755
index 000000000000..3a451db3c067
@@ -0,0 +1,13 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Print the GNU make version in a 5 or 6-digit form.
+
+set -e
+
+# Convert the version string x.y.z to a canonical 5 or 6-digit form.
+IFS=.
+set -- $1
+
+# If the 2nd or 3rd field is missing, fill it with a zero.
+echo $((10000 * $1 + 100 * ${2:-0} + ${3:-0}))
Signature · 1 line
2.18.0
#3 lore raw

Re: [PATCH v3 1/2] Kbuild: lto: add CONFIG_MAKE_VERSION

In reply to Lecopzer Chen
Quoted 6 lines · Lecopzer Chen

To check the GNU make version. Used by the LTO Kconfig.

LTO with MODVERSIONS will fail in generating correct CRC because the makefile rule doesn't work for make with version 3.8X.[1]

Quoted 7 lines

Thus we need to check make version during selecting on LTO Kconfig. Add CONFIG_MAKE_VERSION which means MAKE_VERSION in canonical digits for arithmetic comparisons.

[1] https://lore.kernel.org/lkml/[email protected]/

---

NACK.

"Let's add MAKE_VERSION >= 40200 restriction just because I cannot write correct code that works for older Make" is a horrible idea.

Also, Kconfig is supposed to check the compiler (or toolchains) capability, not host tool versions.

Quoted 54 lines
 Makefile                |  2 +-
 init/Kconfig            |  4 ++++
 scripts/make-version.sh | 13 +++++++++++++
 3 files changed, 18 insertions(+), 1 deletion(-)
 create mode 100755 scripts/make-version.sh
Makefile +1 -1
index 88888fff4c62..2402745b2ba9 100644
@@ -516,7 +516,7 @@ CLANG_FLAGS :=
 
 export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE LD CC
 export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
-export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
+export PERL PYTHON3 CHECK CHECKFLAGS MAKE MAKE_VERSION UTS_MACHINE HOSTCXX
 export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD
 export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE
 
init/Kconfig +4 -0
index 55f9f7738ebb..ecc110504f87 100644
@@ -86,6 +86,10 @@ config CC_HAS_ASM_INLINE
 config CC_HAS_NO_PROFILE_FN_ATTR
        def_bool $(success,echo '__attribute__((no_profile_instrument_function)) int x();' | $(CC) -x c - -c -o /dev/null -Werror)
 
+config MAKE_VERSION
+       int
+       default $(shell,$(srctree)/scripts/make-version.sh $(MAKE_VERSION))
+
 config CONSTRUCTORS
        bool
 
scripts/make-version.sh new file +13 -1
new file mode 100755
index 000000000000..3a451db3c067
@@ -0,0 +1,13 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Print the GNU make version in a 5 or 6-digit form.
+
+set -e
+
+# Convert the version string x.y.z to a canonical 5 or 6-digit form.
+IFS=.
+set -- $1
+
+# If the 2nd or 3rd field is missing, fill it with a zero.
+echo $((10000 * $1 + 100 * ${2:-0} + ${3:-0}))
--
Quoted 2 lines

2.18.0

--
Best Regards
Masahiro Yamada

#4 lore raw

Re: [PATCH v3 1/2] Kbuild: lto: add CONFIG_MAKE_VERSION

Quoted 18 lines · Masahiro Yamada
Quoted 6 lines · Lecopzer Chen

To check the GNU make version. Used by the LTO Kconfig.

LTO with MODVERSIONS will fail in generating correct CRC because the makefile rule doesn't work for make with version 3.8X.[1]

Quoted 7 lines

Thus we need to check make version during selecting on LTO Kconfig. Add CONFIG_MAKE_VERSION which means MAKE_VERSION in canonical digits for arithmetic comparisons.

[1] https://lore.kernel.org/lkml/[email protected]/

---

Quoted 8 lines

NACK.

"Let's add MAKE_VERSION >= 40200 restriction just because I cannot write correct code that works for older Make" is a horrible idea.

Also, Kconfig is supposed to check the compiler (or toolchains) capability, not host tool versions.

I feel like requiring a Make that's half a decade old for a feature that also requires a toolchain released last October ago isn't entirely unreasonable.

That being said, if Masahiro prefers not to rely on the wildcard function's behavior here, which is a reasonable request, we could simply use the shell to test for the file's existence:

scripts/Makefile.build +1 -1
index 34d257653fb4..c6bd62f518ff 100644
@@ -388,7 +388,7 @@ ifeq ($(CONFIG_LTO_CLANG) $(CONFIG_MODVERSIONS),y y)
       cmd_update_lto_symversions =                                     \
        rm -f [email protected]                                            \
        $(foreach n, $(filter-out FORCE,$^),                            \
-               $(if $(wildcard $(n).symversions),                      \
+               $(if $(shell test -s $(n).symversions && echo y),       \
                        ; cat $(n).symversions >> [email protected]))
 else
       cmd_update_lto_symversions = echo >/dev/null

This is not quite as efficient as using wildcard, but should work with older Make versions too. Thoughts?

Sami

author Tested-by
#5 lore raw

Re: [PATCH v3 1/2] Kbuild: lto: add CONFIG_MAKE_VERSION

Quoted 50 lines · Masahiro Yamada
Quoted 18 lines
Quoted 6 lines · Lecopzer Chen

To check the GNU make version. Used by the LTO Kconfig.

LTO with MODVERSIONS will fail in generating correct CRC because the makefile rule doesn't work for make with version 3.8X.[1]

Quoted 7 lines

Thus we need to check make version during selecting on LTO Kconfig. Add CONFIG_MAKE_VERSION which means MAKE_VERSION in canonical digits for arithmetic comparisons.

[1] https://lore.kernel.org/lkml/[email protected]/

---

Quoted 8 lines

NACK.

"Let's add MAKE_VERSION >= 40200 restriction just because I cannot write correct code that works for older Make" is a horrible idea.

Also, Kconfig is supposed to check the compiler (or toolchains) capability, not host tool versions.

I feel like requiring a Make that's half a decade old for a feature that also requires a toolchain released last October ago isn't entirely unreasonable.

That being said, if Masahiro prefers not to rely on the wildcard function's behavior here, which is a reasonable request, we could simply use the shell to test for the file's existence:

scripts/Makefile.build +1 -1
index 34d257653fb4..c6bd62f518ff 100644
@@ -388,7 +388,7 @@ ifeq ($(CONFIG_LTO_CLANG) $(CONFIG_MODVERSIONS),y y)
       cmd_update_lto_symversions =                                     \
        rm -f [email protected]                                            \
        $(foreach n, $(filter-out FORCE,$^),                            \
-               $(if $(wildcard $(n).symversions),                      \
+               $(if $(shell test -s $(n).symversions && echo y),       \
                        ; cat $(n).symversions >> [email protected]))
 else
       cmd_update_lto_symversions = echo >/dev/null
Quoted 3 lines

This is not quite as efficient as using wildcard, but should work with older Make versions too. Thoughts?

I've tested this in both make-4.3 and 3.81, and the CRC is correct. But I'm not sure if anyone would have the "arg list too long" issue.

#6 lore raw

Re: [PATCH v3 1/2] Kbuild: lto: add CONFIG_MAKE_VERSION

Quoted 59 lines · Lecopzer Chen
Quoted 51 lines · Masahiro Yamada
Quoted 18 lines
Quoted 6 lines · Lecopzer Chen

To check the GNU make version. Used by the LTO Kconfig.

LTO with MODVERSIONS will fail in generating correct CRC because the makefile rule doesn't work for make with version 3.8X.[1]

Quoted 7 lines

Thus we need to check make version during selecting on LTO Kconfig. Add CONFIG_MAKE_VERSION which means MAKE_VERSION in canonical digits for arithmetic comparisons.

[1] https://lore.kernel.org/lkml/[email protected]/

---

Quoted 8 lines

NACK.

"Let's add MAKE_VERSION >= 40200 restriction just because I cannot write correct code that works for older Make" is a horrible idea.

Also, Kconfig is supposed to check the compiler (or toolchains) capability, not host tool versions.

I feel like requiring a Make that's half a decade old for a feature that also requires a toolchain released last October ago isn't entirely unreasonable.

That being said, if Masahiro prefers not to rely on the wildcard function's behavior here, which is a reasonable request, we could simply use the shell to test for the file's existence:

scripts/Makefile.build +1 -1
index 34d257653fb4..c6bd62f518ff 100644
@@ -388,7 +388,7 @@ ifeq ($(CONFIG_LTO_CLANG) $(CONFIG_MODVERSIONS),y y)
       cmd_update_lto_symversions =                                     \
        rm -f [email protected]                                            \
        $(foreach n, $(filter-out FORCE,$^),                            \
-               $(if $(wildcard $(n).symversions),                      \
+               $(if $(shell test -s $(n).symversions && echo y),       \
                        ; cat $(n).symversions >> [email protected]))
 else
       cmd_update_lto_symversions = echo >/dev/null
Quoted 3 lines

This is not quite as efficient as using wildcard, but should work with older Make versions too. Thoughts?

I've tested this in both make-4.3 and 3.81, and the CRC is correct. But I'm not sure if anyone would have the "arg list too long" issue.

Quoted 1 line

Thank you for testing. This should produce a command identical to the wildcard version (with newer Make versions), so that shouldn't be an issue. If nobody objects to this approach, would you mind putting this into a proper patch and sending it as v4?

Sami

#7 lore raw

Re: [PATCH v3 1/2] Kbuild: lto: add CONFIG_MAKE_VERSION

Quoted 60 lines · Lecopzer Chen
Quoted 59 lines
Quoted 51 lines · Masahiro Yamada
Quoted 18 lines
Quoted 6 lines · Lecopzer Chen

To check the GNU make version. Used by the LTO Kconfig.

LTO with MODVERSIONS will fail in generating correct CRC because the makefile rule doesn't work for make with version 3.8X.[1]

Quoted 7 lines

Thus we need to check make version during selecting on LTO Kconfig. Add CONFIG_MAKE_VERSION which means MAKE_VERSION in canonical digits for arithmetic comparisons.

[1] https://lore.kernel.org/lkml/[email protected]/

---

Quoted 8 lines

NACK.

"Let's add MAKE_VERSION >= 40200 restriction just because I cannot write correct code that works for older Make" is a horrible idea.

Also, Kconfig is supposed to check the compiler (or toolchains) capability, not host tool versions.

I feel like requiring a Make that's half a decade old for a feature that also requires a toolchain released last October ago isn't entirely unreasonable.

That being said, if Masahiro prefers not to rely on the wildcard function's behavior here, which is a reasonable request, we could simply use the shell to test for the file's existence:

scripts/Makefile.build +1 -1
index 34d257653fb4..c6bd62f518ff 100644
@@ -388,7 +388,7 @@ ifeq ($(CONFIG_LTO_CLANG) $(CONFIG_MODVERSIONS),y y)
       cmd_update_lto_symversions =                                     \
        rm -f [email protected]                                            \
        $(foreach n, $(filter-out FORCE,$^),                            \
-               $(if $(wildcard $(n).symversions),                      \
+               $(if $(shell test -s $(n).symversions && echo y),       \
                        ; cat $(n).symversions >> [email protected]))
 else
       cmd_update_lto_symversions = echo >/dev/null
Quoted 3 lines

This is not quite as efficient as using wildcard, but should work with older Make versions too. Thoughts?

I've tested this in both make-4.3 and 3.81, and the CRC is correct. But I'm not sure if anyone would have the "arg list too long" issue.

Quoted 6 lines
Quoted 1 line

Thank you for testing. This should produce a command identical to the wildcard version (with newer Make versions), so that shouldn't be an issue. If nobody objects to this approach, would you mind putting this into a proper patch and sending it as v4?

Sure, I'll rebase the whole commit and send as v4 soon.

#8 lore raw

PATCH v3 2/2 Kbuild: lto: add make version checking

In reply to Lecopzer Chen

LTO with MODVERSIONS will fail in generating correct CRC because the makefile rule doesn't work for make with version 3.8X.[1]

Thus we need to check make version during selecting on LTO Kconfig. and the suitable version should be 4.2(40200) which released in 2016[2].

[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://ftp.gnu.org/gnu/make/

---
 arch/Kconfig | 1 +
 1 file changed, 1 insertion(+)
arch/Kconfig +1 -0
index 129df498a8e1..beee68149da8 100644
@@ -639,6 +639,7 @@ config HAS_LTO_CLANG
 	def_bool y
 	# Clang >= 11: https://github.com/ClangBuiltLinux/linux/issues/510
 	depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD && AS_IS_LLVM
+	depends on !MODVERSIONS || MAKE_VERSION >= 40200
 	depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm)
 	depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm)
 	depends on ARCH_SUPPORTS_LTO_CLANG
Signature · 1 line
2.18.0
maintainer Reviewed-by on v3 2/2
#9 lore raw

Re: [PATCH v3 2/2] Kbuild: lto: add make version checking

Quoted 9 lines · Lecopzer Chen

LTO with MODVERSIONS will fail in generating correct CRC because the makefile rule doesn't work for make with version 3.8X.[1]

Thus we need to check make version during selecting on LTO Kconfig. and the suitable version should be 4.2(40200) which released in 2016[2].

[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://ftp.gnu.org/gnu/make/

Quoted 18 lines
---
 arch/Kconfig | 1 +
 1 file changed, 1 insertion(+)
arch/Kconfig +1 -0
index 129df498a8e1..beee68149da8 100644
@@ -639,6 +639,7 @@ config HAS_LTO_CLANG
 	def_bool y
 	# Clang >= 11: https://github.com/ClangBuiltLinux/linux/issues/510
 	depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD && AS_IS_LLVM
+	depends on !MODVERSIONS || MAKE_VERSION >= 40200
 	depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm)
 	depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm)
 	depends on ARCH_SUPPORTS_LTO_CLANG
Signature · 1 line
2.18.0
#10 lore raw

Re: [PATCH v3 2/2] Kbuild: lto: add make version checking

In reply to Lecopzer Chen
Quoted 11 lines · Lecopzer Chen

LTO with MODVERSIONS will fail in generating correct CRC because the makefile rule doesn't work for make with version 3.8X.[1]

Thus we need to check make version during selecting on LTO Kconfig. and the suitable version should be 4.2(40200) which released in 2016[2].

[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://ftp.gnu.org/gnu/make/

---

NACK for the same reason as 1/2.

We need to write correct code.
If we cannot, we need to add 'depends on BROKEN'.

Quoted 18 lines
 arch/Kconfig | 1 +
 1 file changed, 1 insertion(+)
arch/Kconfig +1 -1
index 129df498a8e1..beee68149da8 100644
@@ -639,6 +639,7 @@ config HAS_LTO_CLANG
        def_bool y
        # Clang >= 11: https://github.com/ClangBuiltLinux/linux/issues/510
        depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD && AS_IS_LLVM
+       depends on !MODVERSIONS || MAKE_VERSION >= 40200
        depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm)
        depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm)
        depends on ARCH_SUPPORTS_LTO_CLANG
--

2.18.0

Signature · 2 lines
Best Regards
Masahiro Yamada