Fix CONFIG_X86_KERNEL_IBT for clang and ld.lld < 14.0.0
cover Fix CONFIG_X86_KERNEL_IBT for clang and ld.lld < 14.0.0
While testing -tip after the IBT bits were merged, our continuous integration noticed a crash with clang-11:
https://builds.tuxbuild.com/26ZRIuAPHx1L802ExiYCuLuvMhK/build.log
This crash is a separate issue from the one that is already being checked for with '-fcf-protection=branch -mfentry -pg'. This new crash was fixed in clang-12:
https://github.com/llvm/llvm-project/commit/e0b89df2e0f0130881bf6c39bf31d7f6aac00e0f
Unfortunately, I have not been able to tease out a simplified reproducer for this crash that always triggers. The reproducer in that commit message and the reproducer commented in the LLVM test do not trigger all the time (at least with the clang-11 that is in Debian/Ubuntu, assertions might make the crash always happen but we cannot count on that); even the full preprocessed files straight from the kernel source do not always trigger a crash.
As a result, this series proposes just having a hard version check for clang 14.0.0 and newer, which will cover this new clang-11 crash and the '-fcf-protection=branch -mfentry -pg' crash.
Adding a check for a version of clang that has not been released yet is a little riskier than dynamically testing the clang binary for an issue, as it is possible that someone is using/testing a version of clang from before a particular issue was fixed, while still having the same version number. However, our policy for version checks in the kernel has always been to use the final version of clang that will have the issue fixed, as people should be upgrading their compilers often if they are using prereleased versions.
In this particular instance, the '-fcf-protection=branch -mfentry -pg' crash has been fixed for over three months, which is plenty of time to upgrade:
https://github.com/llvm/llvm-project/commit/dfcf69770bc522b9e411c66454934a37c1f35332
At the moment, I only know of one version of clang that is fairly widely used that will show this crash, which is Android's clang 14.0.2. We are in the process of getting a newer version (14.0.3) deployed to minimize the impact this change will have on various testing groups:
https://android-review.googlesource.com/c/kernel/common/+/2032664
https://android-review.googlesource.com/c/kernel/common/+/2032665
https://android-review.googlesource.com/c/kernel/common/+/2032666
https://gitlab.com/Linaro/tuxmake/-/merge_requests/244
If for some reason there are any reports of issues after this change, feel free to direct them to this cover letter and tell them to upgrade their compiler :)
The second patch just codifies the result of this discussion and issue:
https://lore.kernel.org/r/202202241627.EEF3D5D2@keescook/
https://github.com/ClangBuiltLinux/linux/issues/1606
This is based on -tip x86/core and it has survived an allmodconfig build with clang-11 through clang-15 (CC=clang and LLVM=1).
Nathan Chancellor (2):
x86/Kconfig: Only enable CONFIG_CC_HAS_IBT for clang >= 14.0.0 x86/Kconfig: Only allow CONFIG_X86_KERNEL_IBT with ld.lld >= 14.0.0
arch/x86/Kconfig | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
base-commit: 2f35e67f621fffc636cb802a4f93fd168cf38274
Signature · 1 line
2.35.1
PATCH 1/2 x86/Kconfig: Only enable CONFIG_CC_HAS_IBT for clang >= 14.0.0
Commit 156ff4a544ae ("x86/ibt: Base IBT bits") added a check for a crash with 'clang -fcf-protection=branch -mfentry -pg', which intended to exclude Clang versions older than 14.0.0 from selecting CONFIG_X86_KERNEL_IBT.
clang-11 does not have the issue that the check is testing for, so CONFIG_X86_KERNEL_IBT is selectable. Unfortunately, there is a different crash in clang-11 that was fixed in clang-12. To make matters worse, that crash does not appear to be entirely deterministic, as the same input to the compiler will sometimes crash and other times not, which makes dynamically checking for the crash like the '-pg' one unreliable.
To make everything work properly for all common versions of clang, use a hard version check of 14.0.0, as that will be the first release upstream that has both bugs properly fixed.
--- arch/x86/Kconfig | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/Kconfig +3 -2
@@ -1865,9 +1865,10 @@ config CC_HAS_IBT # GCC >= 9 and binutils >= 2.29 # Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654 # Clang/LLVM >= 14 - # fentry check to work around https://reviews.llvm.org/D111108 + # https://github.com/llvm/llvm-project/commit/e0b89df2e0f0130881bf6c39bf31d7f6aac00e0f + # https://github.com/llvm/llvm-project/commit/dfcf69770bc522b9e411c66454934a37c1f35332 def_bool ((CC_IS_GCC && $(cc-option, -fcf-protection=branch -mindirect-branch-register)) || \ - (CC_IS_CLANG && $(success,echo "void a(void) {}" | $(CC) -Werror $(CLANG_FLAGS) -fcf-protection=branch -mfentry -pg -x c - -c -o /dev/null))) && \ + (CC_IS_CLANG && CLANG_VERSION >= 140000)) && \ $(as-instr,endbr64) config X86_KERNEL_IBT
Signature · 1 line
2.35.1
[tip: x86/core] x86/Kconfig: Only enable CONFIG_CC_HAS_IBT for clang >= 14.0.0
The following commit has been merged into the x86/core branch of tip:
The following commit has been merged into the x86/core branch of tip:
Commit-ID: 71a7580b17e9510e5f74c0687567f9d8b5d48f5d Gitweb: https://git.kernel.org/tip/71a7580b17e9510e5f74c0687567f9d8b5d48f5d Author: Nathan Chancellor <[email protected]> AuthorDate: Fri, 18 Mar 2022 16:07:46 -07:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Sat, 19 Mar 2022 00:47:12 +01:00
x86/Kconfig: Only enable CONFIG_CC_HAS_IBT for clang >= 14.0.0
Commit 156ff4a544ae ("x86/ibt: Base IBT bits") added a check for a crash with 'clang -fcf-protection=branch -mfentry -pg', which intended to exclude Clang versions older than 14.0.0 from selecting CONFIG_X86_KERNEL_IBT.
clang-11 does not have the issue that the check is testing for, so CONFIG_X86_KERNEL_IBT is selectable. Unfortunately, there is a different crash in clang-11 that was fixed in clang-12. To make matters worse, that crash does not appear to be entirely deterministic, as the same input to the compiler will sometimes crash and other times not, which makes dynamically checking for the crash like the '-pg' one unreliable.
To make everything work properly for all common versions of clang, use a hard version check of 14.0.0, as that will be the first release upstream that has both bugs properly fixed.
--- arch/x86/Kconfig | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/Kconfig +3 -2
@@ -1865,9 +1865,10 @@ config CC_HAS_IBT # GCC >= 9 and binutils >= 2.29 # Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654 # Clang/LLVM >= 14 - # fentry check to work around https://reviews.llvm.org/D111108 + # https://github.com/llvm/llvm-project/commit/e0b89df2e0f0130881bf6c39bf31d7f6aac00e0f + # https://github.com/llvm/llvm-project/commit/dfcf69770bc522b9e411c66454934a37c1f35332 def_bool ((CC_IS_GCC && $(cc-option, -fcf-protection=branch -mindirect-branch-register)) || \ - (CC_IS_CLANG && $(success,echo "void a(void) {}" | $(CC) -Werror $(CLANG_FLAGS) -fcf-protection=branch -mfentry -pg -x c - -c -o /dev/null))) && \ + (CC_IS_CLANG && CLANG_VERSION >= 140000)) && \ $(as-instr,endbr64) config X86_KERNEL_IBT
[tip: x86/core] x86/Kconfig: Only enable CONFIG_CC_HAS_IBT for clang >= 14.0.0
In reply to Nathan ChancellorThe following commit has been merged into the x86/core branch of tip:
The following commit has been merged into the x86/core branch of tip:
Commit-ID: 262448f3d18959d175b10e28a3b65f41d1d7313f Gitweb: https://git.kernel.org/tip/262448f3d18959d175b10e28a3b65f41d1d7313f Author: Nathan Chancellor <[email protected]> AuthorDate: Fri, 18 Mar 2022 16:07:46 -07:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Tue, 22 Mar 2022 21:12:13 +01:00
x86/Kconfig: Only enable CONFIG_CC_HAS_IBT for clang >= 14.0.0
Commit 156ff4a544ae ("x86/ibt: Base IBT bits") added a check for a crash with 'clang -fcf-protection=branch -mfentry -pg', which intended to exclude Clang versions older than 14.0.0 from selecting CONFIG_X86_KERNEL_IBT.
clang-11 does not have the issue that the check is testing for, so CONFIG_X86_KERNEL_IBT is selectable. Unfortunately, there is a different crash in clang-11 that was fixed in clang-12. To make matters worse, that crash does not appear to be entirely deterministic, as the same input to the compiler will sometimes crash and other times not, which makes dynamically checking for the crash like the '-pg' one unreliable.
To make everything work properly for all common versions of clang, use a hard version check of 14.0.0, as that will be the first release upstream that has both bugs properly fixed.
--- arch/x86/Kconfig | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/Kconfig +3 -2
@@ -1865,9 +1865,10 @@ config CC_HAS_IBT # GCC >= 9 and binutils >= 2.29 # Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654 # Clang/LLVM >= 14 - # fentry check to work around https://reviews.llvm.org/D111108 + # https://github.com/llvm/llvm-project/commit/e0b89df2e0f0130881bf6c39bf31d7f6aac00e0f + # https://github.com/llvm/llvm-project/commit/dfcf69770bc522b9e411c66454934a37c1f35332 def_bool ((CC_IS_GCC && $(cc-option, -fcf-protection=branch -mindirect-branch-register)) || \ - (CC_IS_CLANG && $(success,echo "void a(void) {}" | $(CC) -Werror $(CLANG_FLAGS) -fcf-protection=branch -mfentry -pg -x c - -c -o /dev/null))) && \ + (CC_IS_CLANG && CLANG_VERSION >= 140000)) && \ $(as-instr,endbr64) config X86_KERNEL_IBT
PATCH 2/2 x86/Kconfig: Only allow CONFIG_X86_KERNEL_IBT with ld.lld >= 14.0.0
In reply to Nathan ChancellorWith CONFIG_X86_KERNEL_IBT=y and a version of ld.lld prior to 14.0.0, there are numerous objtool warnings along the lines of:
warning: objtool: .plt+0x6: indirect jump found in RETPOLINE build
This is a known issue that has been resolved in ld.lld 14.0.0. Prevent CONFIG_X86_KERNEL_IBT from being selectable when using one of these problematic ld.lld versions.
--- arch/x86/Kconfig | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/Kconfig +2 -0
@@ -1875,6 +1875,8 @@ config X86_KERNEL_IBT prompt "Indirect Branch Tracking" bool depends on X86_64 && CC_HAS_IBT && STACK_VALIDATION + # https://github.com/llvm/llvm-project/commit/9d7001eba9c4cb311e03cd8cdc231f9e579f2d0f + depends on !LD_IS_LLD || LLD_VERSION >= 140000 help Build the kernel with support for Indirect Branch Tracking, a hardware support course-grain forward-edge Control Flow Integrity
Signature · 1 line
2.35.1
[tip: x86/core] x86/Kconfig: Only allow CONFIG_X86_KERNEL_IBT with ld.lld >= 14.0.0
The following commit has been merged into the x86/core branch of tip:
The following commit has been merged into the x86/core branch of tip:
Commit-ID: 46a5cf598a940e674a574664c32ad3518dc9ed6f Gitweb: https://git.kernel.org/tip/46a5cf598a940e674a574664c32ad3518dc9ed6f Author: Nathan Chancellor <[email protected]> AuthorDate: Fri, 18 Mar 2022 16:07:47 -07:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Sat, 19 Mar 2022 00:47:13 +01:00
x86/Kconfig: Only allow CONFIG_X86_KERNEL_IBT with ld.lld >= 14.0.0
With CONFIG_X86_KERNEL_IBT=y and a version of ld.lld prior to 14.0.0, there are numerous objtool warnings along the lines of:
warning: objtool: .plt+0x6: indirect jump found in RETPOLINE build
This is a known issue that has been resolved in ld.lld 14.0.0. Prevent CONFIG_X86_KERNEL_IBT from being selectable when using one of these problematic ld.lld versions.
--- arch/x86/Kconfig | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/Kconfig +2 -0
@@ -1875,6 +1875,8 @@ config X86_KERNEL_IBT prompt "Indirect Branch Tracking" bool depends on X86_64 && CC_HAS_IBT && STACK_VALIDATION + # https://github.com/llvm/llvm-project/commit/9d7001eba9c4cb311e03cd8cdc231f9e579f2d0f + depends on !LD_IS_LLD || LLD_VERSION >= 140000 help Build the kernel with support for Indirect Branch Tracking, a hardware support course-grain forward-edge Control Flow Integrity
[tip: x86/core] x86/Kconfig: Only allow CONFIG_X86_KERNEL_IBT with ld.lld >= 14.0.0
In reply to Nathan ChancellorThe following commit has been merged into the x86/core branch of tip:
The following commit has been merged into the x86/core branch of tip:
Commit-ID: f6a2c2b2de817078ac5a7e58c10e746165e7825d Gitweb: https://git.kernel.org/tip/f6a2c2b2de817078ac5a7e58c10e746165e7825d Author: Nathan Chancellor <[email protected]> AuthorDate: Fri, 18 Mar 2022 16:07:47 -07:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Tue, 22 Mar 2022 21:12:13 +01:00
x86/Kconfig: Only allow CONFIG_X86_KERNEL_IBT with ld.lld >= 14.0.0
With CONFIG_X86_KERNEL_IBT=y and a version of ld.lld prior to 14.0.0, there are numerous objtool warnings along the lines of:
warning: objtool: .plt+0x6: indirect jump found in RETPOLINE build
This is a known issue that has been resolved in ld.lld 14.0.0. Prevent CONFIG_X86_KERNEL_IBT from being selectable when using one of these problematic ld.lld versions.
--- arch/x86/Kconfig | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/Kconfig +2 -0
@@ -1875,6 +1875,8 @@ config X86_KERNEL_IBT prompt "Indirect Branch Tracking" bool depends on X86_64 && CC_HAS_IBT && STACK_VALIDATION + # https://github.com/llvm/llvm-project/commit/9d7001eba9c4cb311e03cd8cdc231f9e579f2d0f + depends on !LD_IS_LLD || LLD_VERSION >= 140000 help Build the kernel with support for Indirect Branch Tracking, a hardware support course-grain forward-edge Control Flow Integrity