Skip to contentSkip to conversation

Revisiting c0a454b9044f

DISCUSSION Nathan Chancellor 3 participants 5 messages Jul 14, 2025 – Dec 30, 2025 arm64

lore
Share & tools
lore

Nathan Chancellor

author
#0 lore raw

Hi all,

I am looking to potentially bump the minimum version of LLVM for building the kernel to 15.0.0 after the next merge window. In my quest to look for workarounds that can be dropped, I noticed that CONFIG_ARM64_BTI_KERNEL was disabled unconditionally for GCC in commit c0a454b9044f ("arm64/bti: Disable in kernel BTI when cross section thunks are broken") as a result of [1]. Looking at that GCC report, it seems like the AArch64 ABI now documents [2] the GNU toolchain's behavior as expected and LLVM has been adjusted [3][4][5] to match. Do I need to block CONFIG_ARM64_BTI_KERNEL from being selected with LLVM 21.0.0? Or should the kernel adjust its expectations now that the ABI and toolchains all agree?

Cheers,
Nathan

[1]: https://gcc.gnu.org/pr106671
[2]: https://github.com/ARM-software/abi-aa/commit/606ce44fe4d3419c15cd9ed598f18fb5d520fcfc
[3]: https://github.com/llvm/llvm-project/commit/7af2b51e761f49974a64c3009882239cea618f2a
[4]: https://github.com/llvm/llvm-project/commit/edf21314c98a4fe05d48f83dfab2b201ed8bfe9c
[5]: https://github.com/llvm/llvm-project/commit/098b0d18add97dea94e16006486b2fded65e228d

reviewer
#1 lore raw
Quoted 1 line · Nathan Chancellor

Hi all,

Hi Nethan,

Quoted 8 lines

I am looking to potentially bump the minimum version of LLVM for building the kernel to 15.0.0 after the next merge window. In my quest to look for workarounds that can be dropped, I noticed that CONFIG_ARM64_BTI_KERNEL was disabled unconditionally for GCC in commit c0a454b9044f ("arm64/bti: Disable in kernel BTI when cross section thunks are broken") as a result of [1]. Looking at that GCC report, it seems like the AArch64 ABI now documents [2] the GNU toolchain's behavior as expected

For context, at the time of commit c0a454b9044f, GNU LD did not handle this appropriately, leading to runtime BTI failures where two sections were too far apart.

GNU LD was subsequently fixed, and the ABI documentation was updated, but I'm not sure which specific versions of GNU LD have the fix, and we hadn't chased that up to re-enable BTI with GCC.

Quoted 2 lines

and LLVM has been adjusted [3][4][5] to match. Do I need to block CONFIG_ARM64_BTI_KERNEL from being selected with LLVM 21.0.0?

I'm missing something; why would we need to dsiable BTI in that case?

The concern from the kernel side is simply whether we get unexpected BTI failures. IIUC so long as compiler and linker agree we should be good, and we simply need to forbid broken combinations.

Quoted 2 lines

Or should the kernel adjust its expectations now that the ABI and toolchains all agree?

Yes, we can probably rework this.

IIUC we'd need to forbid BTI with:

* GCC + old GNU LD
* GCC + old LLD
* new clang + old GNU LD
* new clang + old LLD

... and can enable BTI otherwise.

Does that make sense to you?

Mark.

Quoted 8 lines

Cheers,
Nathan

[1]: https://gcc.gnu.org/pr106671
[2]: https://github.com/ARM-software/abi-aa/commit/606ce44fe4d3419c15cd9ed598f18fb5d520fcfc
[3]: https://github.com/llvm/llvm-project/commit/7af2b51e761f49974a64c3009882239cea618f2a
[4]: https://github.com/llvm/llvm-project/commit/edf21314c98a4fe05d48f83dfab2b201ed8bfe9c
[5]: https://github.com/llvm/llvm-project/commit/098b0d18add97dea94e16006486b2fded65e228d

author
#2 lore raw
Quoted 10 lines · Mark Rutland
Quoted 8 lines · Nathan Chancellor

I am looking to potentially bump the minimum version of LLVM for building the kernel to 15.0.0 after the next merge window. In my quest to look for workarounds that can be dropped, I noticed that CONFIG_ARM64_BTI_KERNEL was disabled unconditionally for GCC in commit c0a454b9044f ("arm64/bti: Disable in kernel BTI when cross section thunks are broken") as a result of [1]. Looking at that GCC report, it seems like the AArch64 ABI now documents [2] the GNU toolchain's behavior as expected

Quoted 7 lines

For context, at the time of commit c0a454b9044f, GNU LD did not handle this appropriately, leading to runtime BTI failures where two sections were too far apart.

GNU LD was subsequently fixed, and the ABI documentation was updated, but I'm not sure which specific versions of GNU LD have the fix, and we hadn't chased that up to re-enable BTI with GCC.

Based on https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671#c7 and
https://sourceware.org/bugzilla/show_bug.cgi?id=30076, it sounds like binutils 2.41 would be the first fixed version.

Quoted 4 lines
Quoted 2 lines

and LLVM has been adjusted [3][4][5] to match. Do I need to block CONFIG_ARM64_BTI_KERNEL from being selected with LLVM 21.0.0?

I'm missing something; why would we need to dsiable BTI in that case?

Nope, you are not missing anything, I missed the correlation between the compiler and linker.

Quoted 3 lines

The concern from the kernel side is simply whether we get unexpected BTI failures. IIUC so long as compiler and linker agree we should be good, and we simply need to forbid broken combinations.

Mark Brown did mention something about the module loader as well so I was not sure if that was relevant here.

Quoted 7 lines
Quoted 2 lines

Or should the kernel adjust its expectations now that the ABI and toolchains all agree?

Yes, we can probably rework this.

IIUC we'd need to forbid BTI with:

Quoted 8 lines

* GCC + old GNU LD
* GCC + old LLD
* new clang + old GNU LD
* new clang + old LLD

... and can enable BTI otherwise.

Does that make sense to you?

So something like this if I understand correctly?

Cheers,
Nathan

arch/arm64/Kconfig +5 -1
index 393d71124f5d..fe523f9f2d61 100644
@@ -2097,7 +2097,11 @@ config ARM64_BTI_KERNEL
 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697
 	depends on !CC_IS_GCC || GCC_VERSION >= 100100
 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671
-	depends on !CC_IS_GCC
+	# https://sourceware.org/bugzilla/show_bug.cgi?id=30076
+	depends on !CC_IS_GCC || LD_VERSION >= 24100 || LLD_VERSION >= 210000
+	# https://github.com/llvm/llvm-project/commit/7af2b51e761f49974a64c3009882239cea618f2a
+	# https://github.com/llvm/llvm-project/commit/098b0d18add97dea94e16006486b2fded65e228d
+	depends on !CC_IS_CLANG || CLANG_VERSION < 210000 || (CLANG_VERSION >= 210000 && (LD_VERSION >= 24100 || LLD_VERSION >= 210000))
 	depends on (!FUNCTION_GRAPH_TRACER || DYNAMIC_FTRACE_WITH_ARGS)
 	help
 	  Build the kernel with Branch Target Identification annotations
reviewer
#3 lore raw
Quoted 7 lines · Nathan Chancellor
Quoted 3 lines · Mark Rutland

The concern from the kernel side is simply whether we get unexpected BTI failures. IIUC so long as compiler and linker agree we should be good, and we simply need to forbid broken combinations.

Mark Brown did mention something about the module loader as well so I was not sure if that was relevant here.

Sorry, I had forgotten anout that, and that is a concern.

If a single module has executable sections placed more than 128MiB apart we'd potentially have the same problem with any cross-section branch. Truly handling that would be quite tricky and require a fair amount of rework, so the best bet is probably to reject loading modules that are too large (or where we specifically find such cross-section branches needing veneers).

Note that exported symbols and address-taken functions should have a BTI, so this only really matters for cross-section calls within a single module.

I suspect it should be relatively simple but I'm not sure exactly where to plumb that in. I can put that on my TODO list if no-one gets around to it.

Quoted 26 lines
Quoted 7 lines
Quoted 2 lines

Or should the kernel adjust its expectations now that the ABI and toolchains all agree?

Yes, we can probably rework this.

IIUC we'd need to forbid BTI with:

Quoted 8 lines

* GCC + old GNU LD
* GCC + old LLD
* new clang + old GNU LD
* new clang + old LLD

... and can enable BTI otherwise.

Does that make sense to you?

So something like this if I understand correctly?

Cheers,
Nathan

arch/arm64/Kconfig +0 -0
index 393d71124f5d..fe523f9f2d61 100644

arch/arm64/Kconfig config ARM64_BTI_KERNEL · line 2104

Quoted 9 lines
arch/arm64/Kconfig +5 -1
@@ -2097,7 +2097,11 @@ config ARM64_BTI_KERNEL
 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697
 	depends on !CC_IS_GCC || GCC_VERSION >= 100100
 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671
-	depends on !CC_IS_GCC
+	# https://sourceware.org/bugzilla/show_bug.cgi?id=30076
+	depends on !CC_IS_GCC || LD_VERSION >= 24100 || LLD_VERSION >= 210000
+	# https://github.com/llvm/llvm-project/commit/7af2b51e761f49974a64c3009882239cea618f2a
+	# https://github.com/llvm/llvm-project/commit/098b0d18add97dea94e16006486b2fded65e228d
+	depends on !CC_IS_CLANG || CLANG_VERSION < 210000 || (CLANG_VERSION >= 210000 && (LD_VERSION >= 24100 || LLD_VERSION >= 210000))

Yep, something like that.

I was thinking that we could factor this out into a separate config, like we have for BUILTIN_RETURN_ADDRESS_STRIPS_PAC, as that'll make it easier to avoid duplication, e.g.

| config TOOLCHAIN_HAS_WORKING_BTI
| 	bool
| 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697
| 	default n if CC_IS_GCC && GCC_VERSION < 100100
|	# Newer LD/LLD handle BTI in veneers automatically
| 	default y if LD_IS_LLD && LLD_VERSION >= 210000
| 	default y if LD_IS_GNU && LD_VERSION >= 24100
| 	# Newer clang requires newer LD/LLD above
| 	default y if CC_IS_CLANG && CLANG_VERSION < 210000
| 	default n

... and we can easily extend that to handle fixed stable versions, like with BUILTIN_RETURN_ADDRESS_STRIPS_PAC.

Mark.

#4 lore raw

ARM64_BTI_KERNEL and gcc? (was Re: Revisiting c0a454b9044f )

Hi,

Quoted 79 lines · Mark Rutland
Quoted 7 lines · Nathan Chancellor
Quoted 3 lines · Mark Rutland

The concern from the kernel side is simply whether we get unexpected BTI failures. IIUC so long as compiler and linker agree we should be good, and we simply need to forbid broken combinations.

Mark Brown did mention something about the module loader as well so I was not sure if that was relevant here.

Sorry, I had forgotten anout that, and that is a concern.

If a single module has executable sections placed more than 128MiB apart we'd potentially have the same problem with any cross-section branch. Truly handling that would be quite tricky and require a fair amount of rework, so the best bet is probably to reject loading modules that are too large (or where we specifically find such cross-section branches needing veneers).

Note that exported symbols and address-taken functions should have a BTI, so this only really matters for cross-section calls within a single module.

I suspect it should be relatively simple but I'm not sure exactly where to plumb that in. I can put that on my TODO list if no-one gets around to it.

Quoted 26 lines
Quoted 7 lines
Quoted 2 lines

Or should the kernel adjust its expectations now that the ABI and toolchains all agree?

Yes, we can probably rework this.

IIUC we'd need to forbid BTI with:

Quoted 8 lines

* GCC + old GNU LD
* GCC + old LLD
* new clang + old GNU LD
* new clang + old LLD

... and can enable BTI otherwise.

Does that make sense to you?

So something like this if I understand correctly?

Cheers,
Nathan

arch/arm64/Kconfig +0 -0
index 393d71124f5d..fe523f9f2d61 100644

arch/arm64/Kconfig config ARM64_BTI_KERNEL · line 2104

Quoted 9 lines
arch/arm64/Kconfig +5 -1
@@ -2097,7 +2097,11 @@ config ARM64_BTI_KERNEL
 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697
 	depends on !CC_IS_GCC || GCC_VERSION >= 100100
 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671
-	depends on !CC_IS_GCC
+	# https://sourceware.org/bugzilla/show_bug.cgi?id=30076
+	depends on !CC_IS_GCC || LD_VERSION >= 24100 || LLD_VERSION >= 210000
+	# https://github.com/llvm/llvm-project/commit/7af2b51e761f49974a64c3009882239cea618f2a
+	# https://github.com/llvm/llvm-project/commit/098b0d18add97dea94e16006486b2fded65e228d
+	depends on !CC_IS_CLANG || CLANG_VERSION < 210000 || (CLANG_VERSION >= 210000 && (LD_VERSION >= 24100 || LLD_VERSION >= 210000))

Yep, something like that.

I was thinking that we could factor this out into a separate config, like we have for BUILTIN_RETURN_ADDRESS_STRIPS_PAC, as that'll make it easier to avoid duplication, e.g.

| config TOOLCHAIN_HAS_WORKING_BTI
| 	bool
| 	# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697
| 	default n if CC_IS_GCC && GCC_VERSION < 100100
|	# Newer LD/LLD handle BTI in veneers automatically
| 	default y if LD_IS_LLD && LLD_VERSION >= 210000
| 	default y if LD_IS_GNU && LD_VERSION >= 24100
| 	# Newer clang requires newer LD/LLD above
| 	default y if CC_IS_CLANG && CLANG_VERSION < 210000
| 	default n
Quoted 2 lines

... and we can easily extend that to handle fixed stable versions, like with BUILTIN_RETURN_ADDRESS_STRIPS_PAC.

Has there been patches to implement this somewhere?

I'm checking yocto genericarm64 kernel config and noticed that ARM64_BTI_KERNEL was effectively disabled with gcc in generated .config. Maybe some additional tagging or config could be used to mark it as not supported with gcc since the disappearing "CONFIG_ARM64_BTI_KERNEL=y" was not expected, and enabling this is recomended in a number of places including arch/arm64/configs/hardening.config and
https://kspp.github.io/Recommended_Settings#arm64

Cheers,

-Mikko