arm64/bti: Fix kernel BTI issues with livepatch, large kernels, toolchains
Subsystems (3)
Josh Poimboeuf
maintainercover arm64/bti: Fix kernel BTI issues with livepatch, large kernels, toolchains
These patches are a continuation of the following discussions about some bugs related to kernel BTI:
https://lore.kernel.org/ed4fe1f95071897859ec7fbe9176246cbd4962bf.1786138806.git.jpoimboe@kernel.org
https://lore.kernel.org/[email protected]
This started as a livepatch crash investigation but quickly ballooned into a rabbit hole of latent bugs and toolchain quirks/crashes.
The most notable patch is probably the last one, which enables BTI on GCC.
I also hit a SIGSEGV in the GNU linker, for which this series has a workaround. The binutils bug is here:
https://sourceware.org/bugzilla/show_bug.cgi?id=34525
Several of these patches probably wouldn't be needed if we had a flag like -fno-omit-bti-pads or so for both compilers, but that currently doesn't exist, and this at least fixes things for the existing toolchains out there.
There is some toolchain discussion about that here:
https://github.com/llvm/llvm-project/issues/215547
Josh Poimboeuf (12):
arm64/bti: Add BTI landing pad to __sdei_asm_handler() arm64/module: Fix BTI exceptions caused by omitted landing pads in
Clang 21
arm64/bti: Fix BTI linker failures with long branches into .idmap.text arm64/bti: Work around ld crash caused by linker script aliases arm64/bti: Add link error for large kernels with BTI and unsupported
toolchains
arm64/bti: Add link error for large kernels with BTI and livepatch arm64/bti: Advertise BTI in assembly objects
arm64/bti: Enable BTI in the pi/ startup code
efi/libstub: Preserve the GNU property note
efi/libstub: Remove obsolete .note.gnu.property workaround arm64/bti: Force-enable BTI linker veneers
arm64/bti: Enable kernel BTI for GCC
arch/arm64/Kconfig | 9 +- arch/arm64/Makefile | 5 + arch/arm64/include/asm/bti-note.h | 32 +++++ arch/arm64/include/asm/module.h | 5 + arch/arm64/kernel/cpu-reset.S | 2 +- arch/arm64/kernel/entry.S | 1 + arch/arm64/kernel/head.S | 8 +- arch/arm64/kernel/hyp-stub.S | 1 + arch/arm64/kernel/image-vars.h | 13 +- arch/arm64/kernel/module-plts.c | 186 ++++++++++++++++++++++++++ arch/arm64/kernel/pi/Makefile | 11 +- arch/arm64/kernel/sleep.S | 2 +- arch/arm64/kernel/vmlinux.lds.S | 12 ++ arch/arm64/lib/memcpy.S | 3 + arch/arm64/lib/memset.S | 2 + arch/arm64/mm/cache.S | 1 + arch/arm64/mm/proc.S | 8 +- drivers/firmware/efi/libstub/Makefile | 15 ++- 18 files changed, 282 insertions(+), 34 deletions(-) create mode 100644 arch/arm64/include/asm/bti-note.h
Signature · 1 line
2.55.0
Josh Poimboeuf
maintainerPATCH 1/12 arm64/bti: Add BTI landing pad to __sdei_asm_handler()
With CONFIG_UNMAP_KERNEL_AT_EL0, __sdei_asm_entry_trampoline() indirect jumps to __sdei_asm_handler(). Add "bti j" to prevent a Branch Target exception.
Discovered by code inspection.
--- arch/arm64/kernel/entry.S | 1 + 1 file changed, 1 insertion(+)
arch/arm64/kernel/entry.S +1 -0
@@ -976,6 +976,7 @@ NOKPROBE(__sdei_asm_exit_trampoline) * want them. */ SYM_CODE_START(__sdei_asm_handler) + bti j stp x2, x3, [x1, #SDEI_EVENT_INTREGS + S_PC] stp x4, x5, [x1, #SDEI_EVENT_INTREGS + 16 * 2] stp x6, x7, [x1, #SDEI_EVENT_INTREGS + 16 * 3]
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH 01/12] arm64/bti: Add BTI landing pad to __sdei_asm_handler()
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Missing BTI landing pads (bti j) on indirect branch targets in the KPTI trampoline (__sdei_asm_exit_trampoline and tramp_exit).
- [High] The KPTI entry trampoline mapping at TRAMP_VALIAS lacks the BTI Guarded Page (PTE_GP) attribute, disabling hardware CFI checks. --
--- Patch [1]: [PATCH 01/12] arm64/bti: Add BTI landing pad to __sdei_asm_handler() --- commit 59fadd16326c687a5994c4bec81d9484edcc08f9
Author: Josh Poimboeuf <[email protected]>
arm64/bti: Add BTI landing pad to __sdei_asm_handler()
With CONFIG_UNMAP_KERNEL_AT_EL0, __sdei_asm_entry_trampoline() indirect jumps to __sdei_asm_handler(). Add "bti j" to prevent a Branch Target exception.
arch/arm64/kernel/entry.S NOKPROBE(__sdei_asm_exit_trampoline)
Quoted 5 lines
arch/arm64/kernel/entry.S +0
-0@@ -976,6 +976,7 @@ NOKPROBE(__sdei_asm_exit_trampoline)
[Severity: High]
This is a pre-existing issue, but do we also need bti j landing pads on the indirect branch targets for the exit paths?
For example, __sdei_asm_exit_trampoline() is called via an indirect branch (br x5):
arch/arm64/kernel/entry.S:__sdei_asm_exit_trampoline() {
SYM_CODE_START(__sdei_asm_exit_trampoline)
ldr x4, [x4, #(SDEI_EVENT_INTREGS + S_SDEI_TTBR1)]
...
}Similarly, tramp_exit() in the same file lacks a landing pad despite being called via an indirect branch (br x29) on the EL0 exit path:
arch/arm64/kernel/entry.S:tramp_exit() {
SYM_CODE_START_LOCAL(tramp_exit)
tramp_unmap_kernel x29
...
}Would it be better to add bti j annotations to these functions as well to prevent Branch Target exceptions?
[Severity: High]
This isn't a bug introduced by this patch, but does the KPTI trampoline mapping need the BTI Guarded Page (PTE_GP) attribute?
Looking at map_entry_trampoline() in arch/arm64/mm/mmu.c, it uses kernel_exec_prot() to map the trampoline text:
arch/arm64/mm/mmu.c:map_entry_trampoline() {
pgprot_t prot = kernel_exec_prot();
...
early_create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS,
entry_tramp_text_size(), prot,
pgd_pgtable_alloc_init_mm, NO_BLOCK_MAPPINGS);
}The prot variable is derived from PAGE_KERNEL_ROX, which does not appear to contain the PTE_GP bit. Does this inadvertently disable hardware BTI enforcement for the trampoline pages? If so, could this architectural gap be masking the missing bti j annotations on the trampoline exit paths mentioned above?
Quoted 7 lines
arch/arm64/kernel/entry.S +1
-0* want them. */ SYM_CODE_START(__sdei_asm_handler) + bti j stp x2, x3, [x1, #SDEI_EVENT_INTREGS + S_PC] stp x4, x5, [x1, #SDEI_EVENT_INTREGS + 16 * 2] stp x6, x7, [x1, #SDEI_EVENT_INTREGS + 16 * 3]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Josh Poimboeuf
maintainerPATCH 2/12 arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21
In reply to Josh PoimboeufThe following BTI exception was seen when loading a livepatch module:
Internal error: Oops - BTI: 0000000036000001 [#1] SMP pstate: 634004c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=jc) pc : kill_orphaned_pgrp+0x0/0x150 lr : do_exit+0x498/0xaf0 [livepatch_combined]
The problem is that the patch module's do_exit() is branching to a static function in vmlinux using a module PLT veneer (indirect branch), but the target function doesn't have a BTI landing pad.
Clang 21+ omits the landing pad for static functions which can only be reached by a direct branch. But livepatch modules use klp relocations to reference arbitrary kernel symbols, and with CONFIG_RANDOMIZE_MODULE_REGION_FULL the module is far enough away that every call to vmlinux needs a PLT.
Note this problem is actually not specific to livepatch. It's possible for any module's .init section to be allocated > 128MB away from its .text section. So calls from .init to .text via a PLT can trigger a BTI exception when the target function doesn't have a landing pad.
GCC has always omitted the landing pad when possible, so kernel BTI is already considered incompatible with GCC since commit c0a454b9044f ("arm64/bti: Disable in kernel BTI when cross section thunks are broken").
When missing landing pads are detected, allocate a page close to the target which can be used to hold BTI veneers which receive PLT veneer indirect branches and direct branch to the final target:
bti c b <target>
--- arch/arm64/include/asm/module.h | 5 + arch/arm64/kernel/module-plts.c | 186 ++++++++++++++++++++++++++++++++ 2 files changed, 191 insertions(+)
arch/arm64/include/asm/module.h +5 -0
@@ -13,10 +13,15 @@ struct mod_plt_sec {int plt_max_entries; }; +struct bti_veneer_page ; +struct mod_arch_specific {struct mod_plt_sec core;struct mod_plt_sec init; +/* for CONFIG_ARM64_BTI_KERNEL */ +struct bti_veneer_page *bti_veneers; +/* for CONFIG_DYNAMIC_FTRACE */ struct plt_entry *ftrace_trampolines;struct plt_entry *init_ftrace_trampolines;
arch/arm64/kernel/module-plts.c +186 -0
210 lines not on this page · raw message
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Direct memory assignment to the PLT section bypasses text patching, causing a Data Abort when a livepatch module applies relocations to a newly loaded target module.
- [High] The aarch64_insn_set() function incorrectly writes A64 instructions in host byte order, causing BTI veneer pages to be filled with corrupt instruction encodings on Big Endian systems. --
--- Patch [2]: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21 --- commit 74ebe0a6ff8b7ada8b741ca9247269a7b1b55fb6
Author: Josh Poimboeuf <[email protected]>
arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21
This commit fixes BTI exceptions caused by Clang 21+ omitting landing pads for static functions. It allocates a page close to the target to hold BTI veneers which receive PLT indirect branches and direct branch to the final target.
Quoted 4 lines
arch/arm64/kernel/module-plts.c +0
-0
[ ... ]
Quoted 1 line
arch/arm64/kernel/module-plts.c +0
-0@@ -66,6 +72,180 @@ static bool plt_entries_equal(const struct plt_entry *a,
[ ... ]
Quoted 3 lines
arch/arm64/kernel/module-plts.c +3
-0+static struct bti_veneer_page *bti_veneer_page_alloc (struct module *mod, +u64 target) +{
[ ... ]
arch/arm64/kernel/module-plts.c static bool plt_entries_equal(const struct plt_entry *a, · line 165
Quoted 8 lines
arch/arm64/kernel/module-plts.c +8
-0+if (!p) { +pr_err ("%s: no address space within branch range of %pS for a BTI veneer\n" , + mod->name, (void *)target); +return NULL ; + } + +/* Don't leave unused slots executable */ +aarch64_insn_set (p,AARCH64_BREAK_FAULT ,PAGE_SIZE );
[Severity: High]
This is a pre-existing issue with aarch64_insn_set(), but does this new usage write the AARCH64_BREAK_FAULT instruction in the wrong byte order on Big Endian systems?
Looking at aarch64_insn_set(), it passes the host-endian integer to memset32(), which performs native 32-bit stores without cpu_to_le32() conversion:
arch/arm64/kernel/patching.c:aarch64_insn_set() {
...
return __text_poke(text_poke_memset, dst, &insn, len);
}Since ARMv8 instruction fetches are architecturally always Little Endian, wouldn't the CPU fetch a byte-reversed, corrupt instruction (reading 0x000020d4) on Big Endian systems, resulting in a fatal Unknown instruction exception rather than a BRK exception if an unused slot is executed?
Quoted 3 lines
arch/arm64/kernel/module-plts.c +3
-0+ + page =kzalloc_obj (*page,GFP_KERNEL ); +if (!page) {
[ ... ]
Quoted 4 lines
arch/arm64/kernel/module-plts.c +0
-0@@ -77,6 +257,12 @@ u64 module_emit_plt_entry(struct module *mod, Elf64_Shdr *sechdrs, void *loc, const Elf64_Rela *rela, Elf64_Sym *sym) {
[ ... ]
arch/arm64/kernel/module-plts.c u64 module_emit_plt_entry(struct module *mod, Elf64_Shdr *sechdrs,
Quoted 12 lines
arch/arm64/kernel/module-plts.c +6
-0u64 val = sym->st_value + rela->r_addend; +if (!plt_target_has_landing_pad (val)) { + val =module_emit_bti_veneer (mod, val); +if (!val) +return 0 ; + } +if (is_forbidden_offset_for_adrp (&plt[i].adrp)) i++; plt[i] =get_plt_entry (val, &plt[i]);
[Severity: High]
This isn't a bug introduced by this patch, but can this direct memory assignment cause a Data Abort kernel panic when a livepatch module applies relocations to a newly loaded target module?
If a livepatch module is already loaded and active, its sections (including the PLT) are marked ROX. When a new target module loads, klp_module_coming() calls apply_relocate_add() to apply the patch module's relocations for this new target.
If an out-of-range jump requires a new PLT entry, it seems this direct assignment to the ROX plt memory array bypasses text patching mechanisms like patch_map() or aarch64_insn_copy().
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ard Biesheuvel
maintainerRe: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21
In reply to Josh PoimboeufHi Josh,
Quoted 22 lines · Josh Poimboeuf
The following BTI exception was seen when loading a livepatch module:
Internal error: Oops - BTI: 0000000036000001 [#1] SMP pstate: 634004c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=jc) pc : kill_orphaned_pgrp+0x0/0x150 lr : do_exit+0x498/0xaf0 [livepatch_combined]The problem is that the patch module's
do_exit()is branching to a static function in vmlinux using a module PLT veneer (indirect branch), but the target function doesn't have a BTI landing pad.Clang 21+ omits the landing pad for static functions which can only be reached by a direct branch. But livepatch modules use klp relocations to reference arbitrary kernel symbols, and with
CONFIG_RANDOMIZE_MODULE_REGION_FULLthe module is far enough away that every call to vmlinux needs a PLT.Note this problem is actually not specific to livepatch. It's possible for any module's .init section to be allocated > 128MB away from its .text section. So calls from .init to .text via a PLT can trigger a BTI exception when the target function doesn't have a landing pad.
Quoted 9 lines
GCC has always omitted the landing pad when possible, so kernel BTI is already considered incompatible with GCC since commit
c0a454b9044f("arm64/bti: Disable in kernel BTI when cross section thunks are broken").When missing landing pads are detected, allocate a page close to the target which can be used to hold BTI veneers which receive PLT veneer indirect branches and direct branch to the final target:
This does not work for cross-section calls from .init.text to .text.
If .init.text is far away from .text, it is likely because .text ended up in the 128M 'near' module region, and .init.text did not. (They tend to end up in direct branching range of each otherwise.)
Given that the module init code is typically small, I don't think it is safe to assume that allocating a single page close enough to .text is going to be possible if allocating the space for .init.* was not.
IOW, the fix I proposed for cross-section calls is still needed with this approach.
Josh Poimboeuf
maintainerRe: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21
Quoted 41 lines · Ard Biesheuvel
Hi Josh,
Quoted 22 lines · Josh Poimboeuf
The following BTI exception was seen when loading a livepatch module:
Internal error: Oops - BTI: 0000000036000001 [#1] SMP pstate: 634004c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=jc) pc : kill_orphaned_pgrp+0x0/0x150 lr : do_exit+0x498/0xaf0 [livepatch_combined]The problem is that the patch module's
do_exit()is branching to a static function in vmlinux using a module PLT veneer (indirect branch), but the target function doesn't have a BTI landing pad.Clang 21+ omits the landing pad for static functions which can only be reached by a direct branch. But livepatch modules use klp relocations to reference arbitrary kernel symbols, and with
CONFIG_RANDOMIZE_MODULE_REGION_FULLthe module is far enough away that every call to vmlinux needs a PLT.Note this problem is actually not specific to livepatch. It's possible for any module's .init section to be allocated > 128MB away from its .text section. So calls from .init to .text via a PLT can trigger a BTI exception when the target function doesn't have a landing pad.
Quoted 9 lines
GCC has always omitted the landing pad when possible, so kernel BTI is already considered incompatible with GCC since commit
c0a454b9044f("arm64/bti: Disable in kernel BTI when cross section thunks are broken").When missing landing pads are detected, allocate a page close to the target which can be used to hold BTI veneers which receive PLT veneer indirect branches and direct branch to the final target:
This does not work for cross-section calls from .init.text to .text.
If .init.text is far away from .text, it is likely because .text ended up in the 128M 'near' module region, and .init.text did not. (They tend to end up in direct branching range of each otherwise.)
Quoted 7 lines
Given that the module init code is typically small, I don't think it is safe to assume that allocating a single page close enough to .text is going to be possible if allocating the space for .init.* was not.
IOW, the fix I proposed for cross-section calls is still needed with this approach.
But the BTI veneer page is allocated from a *256MB* window, of which the near region is only a 128MB subset.
There is a theoretical case where the 256MB window around the target is completely full without any fragmentation, but I would think that there would almost always be some fragmentation. If that window is modules stacked together, most modules have at least .init.plt and .init.text, and many have .init.data.
Right now it needs two pages (because of the default guard page) but we could maybe fall back to VM_NO_GUARD in case of emergency.
Signature · 1 line
Josh
Ard Biesheuvel
maintainerRe: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21
Quoted 47 lines · Josh Poimboeuf
Quoted 46 lines · Ard Biesheuvel
Hi Josh,
Quoted 22 lines · Josh Poimboeuf
The following BTI exception was seen when loading a livepatch module:
Internal error: Oops - BTI: 0000000036000001 [#1] SMP pstate: 634004c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=jc) pc : kill_orphaned_pgrp+0x0/0x150 lr : do_exit+0x498/0xaf0 [livepatch_combined]The problem is that the patch module's
do_exit()is branching to a static function in vmlinux using a module PLT veneer (indirect branch), but the target function doesn't have a BTI landing pad.Clang 21+ omits the landing pad for static functions which can only be reached by a direct branch. But livepatch modules use klp relocations to reference arbitrary kernel symbols, and with
CONFIG_RANDOMIZE_MODULE_REGION_FULLthe module is far enough away that every call to vmlinux needs a PLT.Note this problem is actually not specific to livepatch. It's possible for any module's .init section to be allocated > 128MB away from its .text section. So calls from .init to .text via a PLT can trigger a BTI exception when the target function doesn't have a landing pad.
Quoted 9 lines
GCC has always omitted the landing pad when possible, so kernel BTI is already considered incompatible with GCC since commit
c0a454b9044f("arm64/bti: Disable in kernel BTI when cross section thunks are broken").When missing landing pads are detected, allocate a page close to the target which can be used to hold BTI veneers which receive PLT veneer indirect branches and direct branch to the final target:
This does not work for cross-section calls from .init.text to .text.
If .init.text is far away from .text, it is likely because .text ended up in the 128M 'near' module region, and .init.text did not. (They tend to end up in direct branching range of each otherwise.)
Given that the module init code is typically small, I don't think it is safe to assume that allocating a single page close enough to .text is going to be possible if allocating the space for .init.* was not.
Quoted 6 lines
Quoted 2 lines
IOW, the fix I proposed for cross-section calls is still needed with this approach.
But the BTI veneer page is allocated from a *256MB* window, of which the near region is only a 128MB subset.
OK, so this window would be [_etext - 128M, _text + 128M), right? Or whichever the equivalent is for a kernel module?
I agree that this considerably reduces the likelihood of running out of veneer space within direct branching range.
Quoted 6 lines
There is a theoretical case where the 256MB window around the target is completely full without any fragmentation, but I would think that there would almost always be some fragmentation. If that window is modules stacked together, most modules have at least .init.plt and .init.text, and many have .init.data.
Are you saying these are freed again once the module is initialized, and therefore there is fragmentation? Not sure if it works like that if modules are loaded one after the other.
Quoted 3 lines
Right now it needs two pages (because of the default guard page) but we could maybe fall back to
VM_NO_GUARDin case of emergency.
I like the advantage of only allocating veneers if they are really needed, as opposed to allocating the space unconditionally, which is what my solution does. But I need more convincing that this will actually cover all these cases, given that the cross-section call issue only occurs when running out of direct branching space to begin with.
Josh Poimboeuf
maintainerRe: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21
Quoted 53 lines · Ard Biesheuvel
Quoted 45 lines · Josh Poimboeuf
Quoted 44 lines · Ard Biesheuvel
Quoted 22 lines · Josh Poimboeuf
The following BTI exception was seen when loading a livepatch module:
Internal error: Oops - BTI: 0000000036000001 [#1] SMP pstate: 634004c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=jc) pc : kill_orphaned_pgrp+0x0/0x150 lr : do_exit+0x498/0xaf0 [livepatch_combined]The problem is that the patch module's
do_exit()is branching to a static function in vmlinux using a module PLT veneer (indirect branch), but the target function doesn't have a BTI landing pad.Clang 21+ omits the landing pad for static functions which can only be reached by a direct branch. But livepatch modules use klp relocations to reference arbitrary kernel symbols, and with
CONFIG_RANDOMIZE_MODULE_REGION_FULLthe module is far enough away that every call to vmlinux needs a PLT.Note this problem is actually not specific to livepatch. It's possible for any module's .init section to be allocated > 128MB away from its .text section. So calls from .init to .text via a PLT can trigger a BTI exception when the target function doesn't have a landing pad.
Quoted 9 lines
GCC has always omitted the landing pad when possible, so kernel BTI is already considered incompatible with GCC since commit
c0a454b9044f("arm64/bti: Disable in kernel BTI when cross section thunks are broken").When missing landing pads are detected, allocate a page close to the target which can be used to hold BTI veneers which receive PLT veneer indirect branches and direct branch to the final target:
This does not work for cross-section calls from .init.text to .text.
If .init.text is far away from .text, it is likely because .text ended up in the 128M 'near' module region, and .init.text did not. (They tend to end up in direct branching range of each otherwise.)
Given that the module init code is typically small, I don't think it is safe to assume that allocating a single page close enough to .text is going to be possible if allocating the space for .init.* was not.
Quoted 6 lines
Quoted 2 lines
IOW, the fix I proposed for cross-section calls is still needed with this approach.
But the BTI veneer page is allocated from a *256MB* window, of which the near region is only a 128MB subset.
Quoted 5 lines
OK, so this window would be [
_etext- 128M,_text+ 128M), right? Or whichever the equivalent is for a kernel module?I agree that this considerably reduces the likelihood of running out of veneer space within direct branching range.
It's a *moving* 256M window centered around a given branch target:
[target - 128M, target + 128M). The window always fully contains the near region, *plus* another 128MB: 128M - (target - base) below the region and (target - base) above, which always adds up to an extra 128MB. So it has a much bigger pool to choose from than module text does.
It does try to allocate inside the region, or right after it, so the page can be shared by other branches in the same module if possible. But it can allocate more above or below as needed.
Quoted 10 lines
Quoted 6 lines
There is a theoretical case where the 256MB window around the target is completely full without any fragmentation, but I would think that there would almost always be some fragmentation. If that window is modules stacked together, most modules have at least .init.plt and .init.text, and many have .init.data.
Are you saying these are freed again once the module is initialized, and therefore there is fragmentation? Not sure if it works like that if modules are loaded one after the other.
Not sure I understand your question, but yes, the module init mem is allocated from the same pool as other module memory, and freed once module init is done, able to be reused, causing fragmentation.
If multiple modules get loaded all at the same time before any module init memory has a chance to get freed, presumably fragmentation wouldn't be much of an issue yet and the given module's .init.text would get allocated close to .text in the first place.
Quoted 9 lines
Quoted 3 lines
Right now it needs two pages (because of the default guard page) but we could maybe fall back to
VM_NO_GUARDin case of emergency.I like the advantage of only allocating veneers if they are really needed, as opposed to allocating the space unconditionally, which is what my solution does. But I need more convincing that this will actually cover all these cases, given that the cross-section call issue only occurs when running out of direct branching space to begin with.
Right, so if .text fits in the near region, and a one-page .init.text doesn't, meaning the 128MB near region is completely full, the BTI veneer allocation still likely succeeds because it still has another 128MB pool to allocate from above/below the region.
So for the allocation to fail, it would need to be a perfect storm:
1) a *lot* of modules loaded and/or huge vmlinux
2) the branch in init code is placed >128MB away from its .text
3) the target is a static .text function with no landing pad
4) there are 0 free pages in the 256MB region around the target
Seems unrealistic to me, unless I'm missing something.
Signature · 1 line
Josh
Will Deacon
maintainerRe: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21
In reply to Ard BiesheuvelHi Josh, Ard,
Quoted 39 lines · Ard Biesheuvel
Quoted 22 lines · Josh Poimboeuf
The following BTI exception was seen when loading a livepatch module:
Internal error: Oops - BTI: 0000000036000001 [#1] SMP pstate: 634004c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=jc) pc : kill_orphaned_pgrp+0x0/0x150 lr : do_exit+0x498/0xaf0 [livepatch_combined]The problem is that the patch module's
do_exit()is branching to a static function in vmlinux using a module PLT veneer (indirect branch), but the target function doesn't have a BTI landing pad.Clang 21+ omits the landing pad for static functions which can only be reached by a direct branch. But livepatch modules use klp relocations to reference arbitrary kernel symbols, and with
CONFIG_RANDOMIZE_MODULE_REGION_FULLthe module is far enough away that every call to vmlinux needs a PLT.Note this problem is actually not specific to livepatch. It's possible for any module's .init section to be allocated > 128MB away from its .text section. So calls from .init to .text via a PLT can trigger a BTI exception when the target function doesn't have a landing pad.
Quoted 9 lines
GCC has always omitted the landing pad when possible, so kernel BTI is already considered incompatible with GCC since commit
c0a454b9044f("arm64/bti: Disable in kernel BTI when cross section thunks are broken").When missing landing pads are detected, allocate a page close to the target which can be used to hold BTI veneers which receive PLT veneer indirect branches and direct branch to the final target:
This does not work for cross-section calls from .init.text to .text.
If .init.text is far away from .text, it is likely because .text ended up in the 128M 'near' module region, and .init.text did not. (They tend to end up in direct branching range of each otherwise.)
Quoted 7 lines
Given that the module init code is typically small, I don't think it is safe to assume that allocating a single page close enough to .text is going to be possible if allocating the space for .init.* was not.
IOW, the fix I proposed for cross-section calls is still needed with this approach.
Sorry to jump in here, but I couldn't figure out a better place to get involved as we have a few threads on this now.
Overall, it seems to me like there are three cases we need to consider for re-enabling BTI in the kernel:
1. A cross-section call that spans beyond the 128M range and therefore needs a veneer. I think the static linker should resolve this, probably by emitting a second veneer with the landing pad. Do we know if LLD gets this right?
2. Calls from modules to exported symbols that end up going via a PLT due to module placement. Exported symbols shouldn't be static, so we should have the correct BTI landing pad in this case (and if we don't, we should fix EXPORT_SYMBOL() to add it).
3. The livepatch case where a module appears to branch directly to static functions in the kernel. For this, I frankly think we should either make in-kernel BTI depend on !LIVEPATCH _or_ pass some compiler option (tbd) when livepatch is enabled so that we get landing pads for static functions (which is what I believe Clang/GCC used to do?).
What am I missing?
Will
Ard Biesheuvel
maintainerRe: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21
Quoted 53 lines · Will Deacon
Hi Josh, Ard,
Quoted 39 lines · Ard Biesheuvel
Quoted 22 lines · Josh Poimboeuf
The following BTI exception was seen when loading a livepatch module:
Internal error: Oops - BTI: 0000000036000001 [#1] SMP pstate: 634004c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=jc) pc : kill_orphaned_pgrp+0x0/0x150 lr : do_exit+0x498/0xaf0 [livepatch_combined]The problem is that the patch module's
do_exit()is branching to a static function in vmlinux using a module PLT veneer (indirect branch), but the target function doesn't have a BTI landing pad.Clang 21+ omits the landing pad for static functions which can only be reached by a direct branch. But livepatch modules use klp relocations to reference arbitrary kernel symbols, and with
CONFIG_RANDOMIZE_MODULE_REGION_FULLthe module is far enough away that every call to vmlinux needs a PLT.Note this problem is actually not specific to livepatch. It's possible for any module's .init section to be allocated > 128MB away from its .text section. So calls from .init to .text via a PLT can trigger a BTI exception when the target function doesn't have a landing pad.
Quoted 9 lines
GCC has always omitted the landing pad when possible, so kernel BTI is already considered incompatible with GCC since commit
c0a454b9044f("arm64/bti: Disable in kernel BTI when cross section thunks are broken").When missing landing pads are detected, allocate a page close to the target which can be used to hold BTI veneers which receive PLT veneer indirect branches and direct branch to the final target:
This does not work for cross-section calls from .init.text to .text.
If .init.text is far away from .text, it is likely because .text ended up in the 128M 'near' module region, and .init.text did not. (They tend to end up in direct branching range of each otherwise.)
Quoted 7 lines
Given that the module init code is typically small, I don't think it is safe to assume that allocating a single page close enough to .text is going to be possible if allocating the space for .init.* was not.
IOW, the fix I proposed for cross-section calls is still needed with this approach.
Sorry to jump in here, but I couldn't figure out a better place to get involved as we have a few threads on this now.
Quoted 8 lines
Overall, it seems to me like there are three cases we need to consider for re-enabling BTI in the kernel:
1. A cross-section call that spans beyond the 128M range and therefore needs a veneer. I think the static linker should resolve this, probably by emitting a second veneer with the landing pad. Do we know if LLD gets this right?
There are two variants here:
A cross-section call .init.text to .text that spans beyond the 128M range:
1a. inside vmlinux, which should be dealt with by the linker, but which might be unreliable in practice due to the lack of BTI annotations in asm files, missing exec permissions on ELF sections etc. This is addressed by this series, but is only an issue for unusually large kernel images (e.g., allyesconfig).
1b. inside a module, which the toolchains's static linker cannot be expected to deal with, given that modules are not fully linked executables. Therefore, the module loader should deal with this, which is what my patch [0] implements. Note that this only occurs when .init.text ends up far away from the .text section of the same module, which occurs rarely in practice.
Quoted 5 lines
2. Calls from modules to exported symbols that end up going via a PLT due to module placement. Exported symbols shouldn't be static, so we should have the correct BTI landing pad in this case (and if we don't, we should fix
EXPORT_SYMBOL()to add it).
This is not an issue today as far as I am aware.
Quoted 6 lines
3. The livepatch case where a module appears to branch directly to static functions in the kernel. For this, I frankly think we should either make in-kernel BTI depend on !LIVEPATCH _or_ pass some compiler option (tbd) when livepatch is enabled so that we get landing pads for static functions (which is what I believe Clang/GCC used to do?).
The livepatch practice of lifting code out of vmlinux at function granularity and copying it into KLP modules in order to patch a running kernel is so far removed from the typical usage expected by the toolchain that it is justified to treat it as a separate case. Working with the compiler folks to allow this optimization to be disabled would be one way to deal with this (and disabling kernel mode BTI for compilers that do not implement it)
That said, my patch [0] only fixes case 1b., which is quite rare in practice, and could perhaps be eliminated entirely if we manage to tweak the module loader so that it always allocates .text and .init.text from the same region. (I haven't checked whether this is feasible at all tbh)
If we decide that the KLP case is worth fixing too, and tweaking the module loader is impractical, I think we should rely on the same logic for case 1b (I mistakenly thought that it doesn't work for that case but Josh pointed out that the 256M allocation window around the branch target should be large enough to ensure that a veneer can be allocated within direct branching range)
Whatever we end up doing, we might still consider my approach as a temp solution that can be backported if desired.
[0] https://lore.kernel.org/linux-arm-kernel/[email protected]/
Josh Poimboeuf
maintainerRe: [PATCH 02/12] arm64/module: Fix BTI exceptions caused by omitted landing pads in Clang 21
Quoted 10 lines · Ard Biesheuvel
Quoted 8 lines · Will Deacon
Overall, it seems to me like there are three cases we need to consider for re-enabling BTI in the kernel:
1. A cross-section call that spans beyond the 128M range and therefore needs a veneer. I think the static linker should resolve this, probably by emitting a second veneer with the landing pad. Do we know if LLD gets this right?
Quoted 8 lines
There are two variants here:
A cross-section call .init.text to .text that spans beyond the 128M range:
1a. inside vmlinux, which should be dealt with by the linker, but which might be unreliable in practice due to the lack of BTI annotations in asm files, missing exec permissions on ELF sections etc. This is addressed by this series, but is only an issue for unusually large kernel images (e.g., allyesconfig).
Right, and if we don't care about the >128MB kernel case (I have no idea if anybody actually uses that big of a kernel?), then we can basically just drop patches 3-11 in favor of a simple linker assertion (similar to patches 5 and 6) which just *always* triggers a build error on a large kernel + BTI + !COMPILE_TEST.
Signature · 1 line
Josh
Josh Poimboeuf
maintainerPATCH 3/12 arm64/bti: Fix BTI linker failures with long branches into .idmap.text
In reply to Josh PoimboeufOn a kernel whose text exceeds the +/128MB direct branch range, the linker inserts veneers. With BTI enabled, the veneers' indirect branch targets need a BTI landing pad, which not all functions have starting with Clang 21 (and for all versions of GCC).
In such cases the linker can emit a second veneer close to the target which has the landing pad along with a direct branch to the target. But a long branch to .idmap.text never gets one because it's missing the executable section flag.
With the LLVM linker, it's a silent failure, presumably only discovered by a BTI exception at runtime. With the GNU linker it's even worse, as it dereferences the missing stub/veneer group entry and seg faults (this was how I discovered it).
Make sure the section is executable by adding the "x" flag to all the creators of the input section.
Also manually add "bti c" to primary_entry() and enter_vhe(), otherwise the linker-generated veneer page pushes the .idmap.text past its asserted 4KB size:
ld.bfd: ID map text too big or misaligned
--- arch/arm64/kernel/cpu-reset.S | 2 +- arch/arm64/kernel/head.S | 7 ++++--- arch/arm64/kernel/hyp-stub.S | 1 + arch/arm64/kernel/sleep.S | 2 +- arch/arm64/mm/proc.S | 8 ++++---- 5 files changed, 11 insertions(+), 9 deletions(-)
arch/arm64/kernel/cpu-reset.S +1 -1
@@ -14,7 +14,7 @@ #include <asm/virt.h> .text -.pushsection .idmap.text, "a" +.pushsection .idmap.text, "ax" /* * cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2)
arch/arm64/kernel/head.S +4 -3
@@ -71,7 +71,7 @@ __EFI_PE_HEADER - .section ".idmap.text","a" + .section ".idmap.text","ax" /* * The following callee saved general purpose registers are used on the
@@ -83,6 +83,7 @@ * x21 primary_entry() .. start_kernel() FDT pointer passed at boot in x0 */ SYM_CODE_START(primary_entry) + bti c bl record_mmu_state bl preserve_boot_args
@@ -251,7 +252,7 @@ SYM_FUNC_END(__primary_switched) * end early head section, begin head code that is also used for * hotplug and needs to have the same protections as the text region */ - .section ".idmap.text","a" + .section ".idmap.text","ax" /* * Starting from EL2 or EL1, configure the CPU to execute at the highest
@@ -455,7 +456,7 @@ SYM_FUNC_END(set_cpu_boot_mode_flag) * Checks if the selected granule size is supported by the CPU. * If it isn't, park the CPU */ - .section ".idmap.text","a" + .section ".idmap.text","ax" SYM_FUNC_START(__enable_mmu) mrs x3, ID_AA64MMFR0_EL1 ubfx x3, x3, #ID_AA64MMFR0_EL1_TGRAN_SHIFT, 4
arch/arm64/kernel/hyp-stub.S +1 -0
@@ -167,6 +167,7 @@ SYM_CODE_END(__finalise_el2) .pushsection .idmap.text, "ax" SYM_CODE_START_LOCAL(enter_vhe) + bti c // Invalidate TLBs before enabling the MMU tlbi vmalle1 dsb nsh
arch/arm64/kernel/sleep.S +1 -1
@@ -97,7 +97,7 @@ SYM_FUNC_START(__cpu_suspend_enter) ret SYM_FUNC_END(__cpu_suspend_enter) - .pushsection ".idmap.text", "a" + .pushsection ".idmap.text", "ax" SYM_CODE_START(cpu_resume) mov x0, xzr bl init_kernel_el
arch/arm64/mm/proc.S +4 -4
@@ -175,7 +175,7 @@ alternative_else_nop_endif SYM_FUNC_END(cpu_do_resume) #endif - .pushsection ".idmap.text", "a" + .pushsection ".idmap.text", "ax" .macro __idmap_cpu_set_reserved_ttbr1, tmp1, tmp2 adrp \tmp1, reserved_pg_dir
@@ -211,7 +211,7 @@ SYM_FUNC_ALIAS(__pi_idmap_cpu_replace_ttbr1, idmap_cpu_replace_ttbr1) #define KPTI_NG_PTE_FLAGS (PTE_ATTRINDX(MT_NORMAL) | PTE_TYPE_PAGE | \ PTE_AF | PTE_SHARED | PTE_UXN | PTE_WRITE) - .pushsection ".idmap.text", "a" + .pushsection ".idmap.text", "ax" .macro pte_to_phys, phys, pte and \phys, \pte, #PTE_ADDR_LOW
@@ -438,7 +438,7 @@ SYM_FUNC_END(idmap_kpti_install_ng_mappings) .popsection #endif - .pushsection ".idmap.text", "a" + .pushsection ".idmap.text", "ax" SYM_TYPED_FUNC_START(wait_linear_map_split_to_ptes) /* Must be same registers as in idmap_kpti_install_ng_mappings */ swapper_ttb .req x3
@@ -479,7 +479,7 @@ SYM_FUNC_END(wait_linear_map_split_to_ptes) * Output: * Return in x0 the value of the SCTLR_EL1 register. */ - .pushsection ".idmap.text", "a" + .pushsection ".idmap.text", "ax" SYM_FUNC_START(__cpu_setup) tlbi vmalle1 // Invalidate local TLB dsb nsh
Signature · 1 line
2.55.0
Ard Biesheuvel
maintainerRe: [PATCH 03/12] arm64/bti: Fix BTI linker failures with long branches into .idmap.text
Hi Josh,
Quoted 39 lines · Josh Poimboeuf
On a kernel whose text exceeds the +/128MB direct branch range, the linker inserts veneers. With BTI enabled, the veneers' indirect branch targets need a BTI landing pad, which not all functions have starting with Clang 21 (and for all versions of GCC).
In such cases the linker can emit a second veneer close to the target which has the landing pad along with a direct branch to the target. But a long branch to .idmap.text never gets one because it's missing the executable section flag.
With the LLVM linker, it's a silent failure, presumably only discovered by a BTI exception at runtime. With the GNU linker it's even worse, as it dereferences the missing stub/veneer group entry and seg faults (this was how I discovered it).
Make sure the section is executable by adding the "x" flag to all the creators of the input section.
Also manually add "bti c" to
primary_entry()andenter_vhe(), otherwise the linker-generated veneer page pushes the .idmap.text past its asserted 4KB size:ld.bfd: ID map text too big or misaligned
Signed-off-by Josh Poimboeuf <[email protected]>--- arch/arm64/kernel/cpu-reset.S | 2 +- arch/arm64/kernel/head.S | 7 ++++--- arch/arm64/kernel/hyp-stub.S | 1 + arch/arm64/kernel/sleep.S | 2 +- arch/arm64/mm/proc.S | 8 ++++---- 5 files changed, 11 insertions(+), 9 deletions(-)arch/arm64/kernel/cpu-reset.S +0
-0
arch/arm64/kernel/cpu-reset.S line 18
Quoted 6 lines
arch/arm64/kernel/cpu-reset.S +1
-1@@ -14,7 +14,7 @@ #include <asm/virt.h> .text -.pushsection .idmap.text, "a" +.pushsection .idmap.text, "ax"
We've had issues in the past with this change.
The problem is that the ID map is restricted to a single page, and [some versions of] the GNU linker concatenate generated veneers onto whichever executable input section came last in the input, which was this one at that point.
That resulted in the ID map start/end boundaries being placed further apart than the linker asserts would tolerate, even though the generated code in question would never be called via the 1:1 mapping.
Not sure whether that problem has simply disappeared by now, or older versions of the linker may still trigger it. But it is something to be aware of.
Josh Poimboeuf
maintainerPATCH 4/12 arm64/bti: Work around ld crash caused by linker script aliases
In reply to Josh PoimboeufLinking an allyesconfig kernel (~700MB of text) with the GNU linker and CONFIG_ARM64_BTI_KERNEL triggers a seg fault in the linker:
#1 elf64_aarch64_stub_name (input_section=0x100000040, sym_sec=..., hash=..., rel=...) at ../../bfd/elfnn-aarch64.c:3042 #2 _bfd_aarch64_add_call_stub_entries (...) at ../../bfd/elfnn-aarch64.c:4674 #3 elf64_aarch64_size_stubs (...) at ../../bfd/elfnn-aarch64.c:4839
On a kernel whose text exceeds the +/128MB direct branch range, the linker inserts veneers. With BTI enabled, the veneers' indirect branch targets need a BTI landing pad, which not all functions have starting with Clang 21 (and for all versions of GCC).
In such cases the linker can emit a second veneer close to the target which has the landing pad along with a direct branch to the target. The GNU linker assumes each target has an input section. But linker-defined symbols don't have input sections, and ld crashes trying to access their veneer data.
vmlinux.lds.S (via image-vars.h) uses PROVIDE() to alias some function symbols into the __efistub_ and __pi_ namespaces, which are used by the EFI stub and the position-independent startup code, respectively.
Work around the crash by defining those aliases in code. Note this might also end up being a permanent fix, depending on whether the ld fix ends up reporting an error for such cases, or whether it will infer the alias's section from the aliasee.
Define the five affected aliases in the objects which define their aliasees so they live in a real input section.
--- arch/arm64/kernel/head.S | 1 + arch/arm64/kernel/image-vars.h | 13 +------------ arch/arm64/lib/memcpy.S | 3 +++ arch/arm64/lib/memset.S | 2 ++ arch/arm64/mm/cache.S | 1 + 5 files changed, 8 insertions(+), 12 deletions(-)
arch/arm64/kernel/head.S +1 -0
@@ -130,6 +130,7 @@ SYM_CODE_START(primary_entry) bl __cpu_setup // initialise processor b __primary_switch SYM_CODE_END(primary_entry) +SYM_FUNC_ALIAS(__efistub_primary_entry, primary_entry) __INIT SYM_CODE_START_LOCAL(record_mmu_state)
arch/arm64/kernel/image-vars.h +1 -12
@@ -20,19 +20,12 @@PROVIDE (pisym = sym); \ ASSERT((sym - KIMAGE_VADDR) < (__bss_start - KIMAGE_VADDR), #msg) -PROVIDE(__efistub_primary_entry = primary_entry); -/* * The EFI stub has its own symbol namespace prefixed by __efistub_, to * isolate it from the kernel proper. The following symbols are legally * accessed by the stub, so provide some aliases to make them accessible. -* Only include data symbols here, or text symbols of functions that are -* guaranteed to be safe when executed at another offset than they were -* linked at. The routines below are all implemented in assembler in a -* position independent manner +* Only include data symbols here. */ -PROVIDE (__efistub_caches_clean_inval_pou = __pi_caches_clean_inval_pou); -PROVIDE (__efistub__text = _text);PROVIDE (__efistub__end = _end);PROVIDE (__efistub___inittext_end = __inittext_end);
@@ -42,10 +35,6 @@ PROVIDE(__efistub_sysfb_primary_display = sysfb_primary_display);#endif PROVIDE (__efistub__ctype = _ctype); -PROVIDE (__pi___memcpy = __pi_memcpy); -PROVIDE (__pi___memmove = __pi_memmove); -PROVIDE (__pi___memset = __pi_memset); -PI_EXPORT_SYM (id_aa64isar1_override);PI_EXPORT_SYM (id_aa64isar2_override);PI_EXPORT_SYM (id_aa64mmfr0_override);
arch/arm64/lib/memcpy.S +3 -0
@@ -268,3 +268,6 @@ SYM_FUNC_ALIAS(__memmove, __pi_memmove) EXPORT_SYMBOL(__memmove) SYM_FUNC_ALIAS_WEAK(memmove, __memmove) EXPORT_SYMBOL(memmove) + +SYM_FUNC_ALIAS(__pi___memcpy, __pi_memcpy) +SYM_FUNC_ALIAS(__pi___memmove, __pi_memmove)
arch/arm64/lib/memset.S +2 -0
@@ -226,3 +226,5 @@ EXPORT_SYMBOL(__memset) SYM_FUNC_ALIAS_WEAK(memset, __pi_memset) EXPORT_SYMBOL(memset) + +SYM_FUNC_ALIAS(__pi___memset, __pi_memset)
arch/arm64/mm/cache.S +1 -0
@@ -57,6 +57,7 @@ SYM_FUNC_START(caches_clean_inval_pou) ret SYM_FUNC_END(caches_clean_inval_pou) SYM_FUNC_ALIAS(__pi_caches_clean_inval_pou, caches_clean_inval_pou) +SYM_FUNC_ALIAS(__efistub_caches_clean_inval_pou, caches_clean_inval_pou) /* * caches_clean_inval_user_pou(start,end)
Signature · 1 line
2.55.0
Josh Poimboeuf
maintainerPATCH 5/12 arm64/bti: Add link error for large kernels with BTI and unsupported toolchains
In reply to Josh PoimboeufOn a kernel whose text exceeds the +/128MB direct branch range, the linker inserts veneers. With BTI enabled, the veneers' indirect branch targets need a BTI landing pad, which not all functions have starting with Clang 21 (and for all versions of GCC).
In such cases the linker can emit a second veneer close to the target which has the landing pad along with a direct branch to the target. However, that's currently not being done, so GCC and Clang 21+ are broken with BTI on large kernels.
Only newer linkers have proper support for adding the second veneer, starting with binutils 2.42 and LLVM 20.
Only allow a large kernel with the right combination of compiler and linker.
--- arch/arm64/Kconfig | 7 +++++++ arch/arm64/kernel/vmlinux.lds.S | 5 +++++ 2 files changed, 12 insertions(+)
arch/arm64/Kconfig +7 -0
@@ -2123,6 +2123,13 @@ config ARM64_BTI_KERNEL is enabled and the system supports BTI all kernel code including modular code must have BTI enabled. +config CC_OMITS_BTI_LANDING_PADS + def_bool !CC_IS_CLANG || CLANG_VERSION >= 210000 + +config LD_HAS_BTI_STUBS + def_bool (LD_IS_BFD && LD_VERSION >= 24200) || \ + (LD_IS_LLD && LLD_VERSION >= 200000) + config CC_HAS_BRANCH_PROT_PAC_RET_BTI # GCC 9 or later, clang 8 or later def_bool $(cc-option,-mbranch-protection=pac-ret+leaf+bti)
arch/arm64/kernel/vmlinux.lds.S +5 -0
@@ -414,6 +414,11 @@ ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) <= 3*PAGE_SIZE, #ifdef CONFIG_KVM ASSERT(__hyp_bss_start == __bss_start, "HYP and Host BSS are misaligned") #endif + +#if defined(CONFIG_ARM64_BTI_KERNEL) && !defined(CONFIG_COMPILE_TEST) && \ + defined(CONFIG_CC_OMITS_BTI_LANDING_PADS) && !defined(CONFIG_LD_HAS_BTI_STUBS) +ASSERT(__exittext_end - _text <= SZ_128M, "Kernel text too big for BTI") +#endif /* * If padding is applied before .head.text, virt<->phys conversions will fail. */
Signature · 1 line
2.55.0
Josh Poimboeuf
maintainerPATCH 6/12 arm64/bti: Add link error for large kernels with BTI and livepatch
In reply to Josh PoimboeufA livepatch module uses klp relocations to reference static functions in vmlinux. If the livepatch module is placed at an address far away from vmlinux, it needs PLT veneers to call those functions.
With BTI enabled, the veneers' indirect branch targets need a BTI landing pad, which not all functions have starting with Clang 21 (and for all versions of GCC).
In such cases the module loader attempts to allocate a page close to the target for emitting a second veneer which has the landing pad along with a direct branch to the target.
That page has to come from either the region below _text or the region above _end, as there's no way to allocate memory in the middle of the vmlinux image. For a target near the end of a >128MB text region there's no address space below _text within branch range at all, leaving only whatever room happens to remain above _end after rodata, data and bss.
If the branch target is out of range from any available free pages, the livepatch module load fails with -ENOEXEC and a "no address space within branch range" error.
Add an assertion so it fails the build instead. Only count the module region below _text; counting the space above _end would make a text size limit depend on the size of rodata, data and bss, and doesn't help unless text is bigger than the rest of the image. The span is _text to _etext rather than __exittext_end, as init and exit text are freed before any module loads and can't be klp targets.
Distro kernels don't seem to come close to hitting the 128MB text mark, so this should hopefully be more of a safety backstop than something people are actually hitting.
--- arch/arm64/kernel/vmlinux.lds.S | 7 +++++++ 1 file changed, 7 insertions(+)
arch/arm64/kernel/vmlinux.lds.S +7 -0
@@ -419,6 +419,13 @@ ASSERT(__hyp_bss_start == __bss_start, "HYP and Host BSS are misaligned") defined(CONFIG_CC_OMITS_BTI_LANDING_PADS) && !defined(CONFIG_LD_HAS_BTI_STUBS) ASSERT(__exittext_end - _text <= SZ_128M, "Kernel text too big for BTI") #endif + +#if defined(CONFIG_ARM64_BTI_KERNEL) && !defined(CONFIG_COMPILE_TEST) && \ + defined(CONFIG_CC_OMITS_BTI_LANDING_PADS) && defined(CONFIG_LIVEPATCH) +ASSERT(_etext - _text <= SZ_128M - PAGE_SIZE, + "Kernel text too big for BTI live patching") +#endif + /* * If padding is applied before .head.text, virt<->phys conversions will fail. */
Signature · 1 line
2.55.0
Josh Poimboeuf
maintainerPATCH 7/12 arm64/bti: Advertise BTI in assembly objects
In reply to Josh PoimboeufOn a kernel whose text exceeds the +/-128MB direct branch range, the linker inserts veneers. With BTI enabled, the veneers' indirect branch targets need a BTI landing pad, which not all functions have starting with Clang 21 (and for all versions of GCC).
In such cases the linker can emit a second veneer close to the target which has the landing pad along with a direct branch to the target. However, that's currently not being done, so GCC and Clang 21+ are broken with BTI on large kernels.
The linker only emits the BTI veneer if *all* input objects have GNU_PROPERTY_AARCH64_FEATURE_1_BTI, which is not being done for hand-written asm.
Force-include a property note with the BTI bit into every assembly translation unit, which is accurate as SYM_FUNC_START*() already emits a "bti c" landing pad for callable assembly functions.
--- arch/arm64/Makefile | 4 ++++ arch/arm64/include/asm/bti-note.h | 32 +++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+) create mode 100644 arch/arm64/include/asm/bti-note.h
arch/arm64/Makefile +4 -0
@@ -23,6 +23,10 @@ ifeq ($(CONFIG_ARM64_ERRATUM_843419),y) LDFLAGS_vmlinux += --fix-cortex-a53-843419 endif +ifeq ($(CONFIG_ARM64_BTI_KERNEL),y) +KBUILD_AFLAGS += -include $(srctree)/arch/arm64/include/asm/bti-note.h +endif + cc_has_k_constraint := $(call try-run,echo \ 'int main(void) { \ asm volatile("and w0, w0, %w0" :: "K" (4294967295)); \
arch/arm64/include/asm/bti-note.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Emit a GNU_PROPERTY_AARCH64_FEATURE_1_BTI note. This is force-included in +* every assembly file so the linker emits BTI veneers for >128MB kernels. +* +* Clang has -mmark-bti-property, but there's no equivalent for GCC/GAS. +* +* Binutils 2.44+ and LLVM 22+ support a much more compact version: +* +* .aeabi_subsection aeabi_feature_and_bits, optional, ULEB128 +* .aeabi_attribute Tag_Feature_BTI, 1 +*/ +#ifndef __ASM_BTI_NOTE_H +#define __ASM_BTI_NOTE_H + + .pushsection .note.gnu.property, "a" + .align 3 + .long 2f - 1f + .long 6f - 3f + .long 5 /* NT_GNU_PROPERTY_TYPE_0 */ +1: .string "GNU" +2: + .align 3 +3: .long 0xc0000000 /* GNU_PROPERTY_AARCH64_FEATURE_1_AND */ + .long 5f - 4f +4: .long 1 /* GNU_PROPERTY_AARCH64_FEATURE_1_BTI */ +5: + .align 3 +6: + .popsection + +#endif /* __ASM_BTI_NOTE_H */
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH 07/12] arm64/bti: Advertise BTI in assembly objects
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] The patch fails to enable BTI veneers for the kernel because EFI stub and PI object files explicitly strip the property note. --
--- Patch [7]: [PATCH 07/12] arm64/bti: Advertise BTI in assembly objects --- commit f368db6f176a29dc1fd3803f6ac93282119ab6aa
Author: Josh Poimboeuf <[email protected]>
arm64/bti: Advertise BTI in assembly objects
This commit forces the inclusion of a GNU property note with the BTI bit into every assembly file. This is intended to ensure the linker emits BTI veneers for >128MB kernels by ensuring all input objects have the property.
arch/arm64/Makefile ifeq ($(CONFIG_ARM64_ERRATUM_843419),y) · line 28
Quoted 11 lines
arch/arm64/Makefile +3
-0@@ -23,6 +23,10 @@ ifeq ($(CONFIG_ARM64_ERRATUM_843419),y) LDFLAGS_vmlinux += --fix-cortex-a53-843419 endif +ifeq ($(CONFIG_ARM64_BTI_KERNEL),y) +KBUILD_AFLAGS += -include $(srctree)/arch/arm64/include/asm/bti-note.h +endif
[Severity: High]
This isn't a bug introduced by this patch, but will this approach still fail to emit BTI veneers for the kernel?
The GNU linker requires all input objects to have the BTI property, but the EFI stub and PI object files explicitly strip the property note before they are linked into vmlinux.
In drivers/firmware/efi/libstub/Makefile:
STUBCOPY_FLAGS-y += --remove-section=.note.gnu.property
And in arch/arm64/kernel/pi/Makefile:
$(obj)/%.pi.o: OBJCOPYFLAGS := --prefix-symbols=__pi_ \
--remove-section=.note.gnu.property
Since these objects are stripped of the note and linked into vmlinux, will the linker continue to evaluate the global link with objects lacking the BTI property, leaving the original large-kernel crash unresolved?
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Re: [PATCH 07/12] arm64/bti: Advertise BTI in assembly objects
In reply to Josh PoimboeufQuoted 4 lines · Josh Poimboeuf
On a kernel whose text exceeds the +/-128MB direct branch range, the linker inserts veneers. With BTI enabled, the veneers' indirect branch targets need a BTI landing pad, which not all functions have starting with Clang 21 (and for all versions of GCC).
We already have explicit note generation support in asm/assembler.h which is used in the vDSO code, it probably makes sense to replace that too. I didn't check if there's potential for the two to collide with this as is.
Josh Poimboeuf
maintainerPATCH 8/12 arm64/bti: Enable BTI in the pi/ startup code
In reply to Josh PoimboeufThe early startup code in kernel/pi/ is built with -mbranch-protection=none and has its property note stripped, which prevents the linker from generating BTI veneers in >128MB kernels.
Unlike the assembly objects, which only lacked the note because SYM_FUNC_START*() already emits the landing pads, pi/ needs the real thing. Some of it runs with the MMU on, where the kernel text is mapped PTE_GP and BTI is enforced: __pi_scs_patch() is called when loading a module, and __pi_map_range() is called from create_idmap().
Add -mbranch-protection=bti. The added "bti c" landing pads execute as NOPs in the startup code wherever BTI isn't implemented or isn't enforced.
Also stop stripping the objects' property notes, as the linker only marks the output BTI-compatible if *all* objects advertise it. Keeping the note as-is isn't enough either: --prefix-alloc-sections=.init renames it to .init.note.gnu.property, which GNU ld still parses but LLD ignores. Rename it back afterwards so both linkers see it.
--- arch/arm64/kernel/pi/Makefile | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
arch/arm64/kernel/pi/Makefile +7 -4
@@ -4,7 +4,7 @@ KBUILD_CFLAGS := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) -fpie \ -Os -DDISABLE_BRANCH_PROFILING $(DISABLE_KSTACK_ERASE) \ $(DISABLE_LATENT_ENTROPY_PLUGIN) \ - $(call cc-option,-mbranch-protection=none) \ + $(call cc-option,-mbranch-protection=bti) \ -I$(srctree)/scripts/dtc/libfdt -fno-stack-protector \ -include $(srctree)/include/linux/hidden.h \ -D__DISABLE_EXPORTS -ffreestanding -D__NO_FORTIFY \
@@ -21,11 +21,14 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS)) hostprogs := relacheck +# --prefix-alloc-sections=.init also renames .note.gnu.property, which LLD then +# ignores, dropping the BTI property. Rename it back. quiet_cmd_piobjcopy = $(quiet_cmd_objcopy) - cmd_piobjcopy = $(cmd_objcopy) && $(obj)/relacheck $(@) $(<) + cmd_piobjcopy = $(cmd_objcopy) && \ + $(OBJCOPY) --rename-section .init.note.gnu.property=.note.gnu.property $(@) && \ + $(obj)/relacheck $(@) $(<) -$(obj)/%.pi.o: OBJCOPYFLAGS := --prefix-symbols=__pi_ \ - --remove-section=.note.gnu.property +$(obj)/%.pi.o: OBJCOPYFLAGS := --prefix-symbols=__pi_ $(obj)/%.pi.o: $(obj)/%.o $(obj)/relacheck FORCE $(call if_changed,piobjcopy)
Signature · 1 line
2.55.0
Josh Poimboeuf
maintainerPATCH 9/12 efi/libstub: Preserve the GNU property note
In reply to Josh PoimboeufWith "-z force-bti" the linker warns (or errors with CONFIG_WERROR) for every libstub object:
warning: BTI is required by -z force-bti, but this input object file lacks the necessary property note.
On arm64 the EFI stub is part of vmlinux, so its objects are annotated as __init at the section level by running objcopy with --prefix-alloc-sections=.init. That has the side effect of renaming .note.gnu.property to .init.note.gnu.property, which LLD ignores, resulting in the BTI bit getting removed.
That's only a problem for toolchains which advertise the branch protection features solely through .note.gnu.property. GCC 16 and Clang 20 also record them in .ARM.attributes, which objcopy leaves alone, but older compilers emit only the note.
Rename the section back to its original name so the linker sees it, and so the generic NOTES macro can discard it. objcopy applies --rename-section before --prefix-alloc-sections regardless of their order on the command line, so this needs a second objcopy invocation rather than another flag on the existing one.
RISC-V and LoongArch apply the same .init prefix, so key the rename off that rather than on arm64 alone. Neither emits .note.gnu.property today, and renaming a section which isn't present is a no-op, so this changes nothing for them for now, but it prevents a future silent failure mode.
The note is still stripped by --remove-section=.note.gnu.property, so this change is inert until that section removal goes away in a subsequent patch.
--- drivers/firmware/efi/libstub/Makefile | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/firmware/efi/libstub/Makefile +8 -1
@@ -155,6 +155,12 @@ STUBCOPY_FLAGS-$(CONFIG_LOONGARCH) += --prefix-alloc-sections=.init \ --prefix-symbols=__efistub_ STUBCOPY_RELOC-$(CONFIG_LOONGARCH) := R_LARCH_MARK_LA +# --prefix-alloc-sections=.init also renames .note.gnu.property, which the +# linker then ignores, dropping the branch protection properties. Rename it +# back on any architecture which applies the prefix. +STUBCOPY_RENAME-y = $(if $(findstring --prefix-alloc-sections,$(STUBCOPY_FLAGS-y)), \ + --rename-section .init.note.gnu.property=.note.gnu.property) + $(obj)/%.stub.o: $(obj)/%.o FORCE $(call if_changed,stubcopy)
@@ -171,4 +177,5 @@ quiet_cmd_stubcopy = STUBCPY $@ echo "$@: absolute symbol references not allowed in the EFI stub" >&2; \ /bin/false; \ fi; \ - $(OBJCOPY) $(STUBCOPY_FLAGS-y) $< $@ + $(OBJCOPY) $(STUBCOPY_FLAGS-y) $< $@ \ + $(if $(STUBCOPY_RENAME-y),; $(OBJCOPY) $(STUBCOPY_RENAME-y) $@)
Signature · 1 line
2.55.0
Josh Poimboeuf
maintainerPATCH 10/12 efi/libstub: Remove obsolete .note.gnu.property workaround
In reply to Josh PoimboeufCommit e2179a09ab08 ("efi/libstub: Disable -mbranch-protection") added --remove-section=.note.gnu.property to the stub objcopy invocation to work around a Clang bug where the note was emitted for code-less object files even with -mbranch-protection=none.
That was fixed by LLVM commit a48f6079f288 ("[AArch64] Generate .note.gnu.property based on module flags") which was released with Clang 12. The minimum Clang version is now Clang 17, so this workaround is no longer needed.
On arm64, the stub also no longer builds with -mbranch-protection=none, as it has inherited the kernel's flags since commit 8358098b9787 ("arm64: efi: Enable BTI codegen and add PE/COFF annotation").
Remove the workaround. This fixes BTI on arm64, and is a no-op on RISC-V and LoongArch where the vmlinux generic NOTES macro discards it, and the x86 and ARM decompressors discard .note.* explicitly.
--- drivers/firmware/efi/libstub/Makefile | 6 ------ 1 file changed, 6 deletions(-)
drivers/firmware/efi/libstub/Makefile +0 -6
@@ -106,12 +106,6 @@ lib-$(CONFIG_UNACCEPTED_MEMORY) += unaccepted_memory.o bitmap.o find.o targets := $(lib-y) lib-y := $(patsubst %.o,%.stub.o,$(lib-y)) -# Even when -mbranch-protection=none is set, Clang will generate a -# .note.gnu.property for code-less object files (like lib/ctype.c), -# so work around this by explicitly removing the unwanted section. -# https://llvm.org/pr46480 -STUBCOPY_FLAGS-y += --remove-section=.note.gnu.property - STUBCOPY_RELOC-$(CONFIG_X86_32) := R_386_32 STUBCOPY_RELOC-$(CONFIG_X86_64) := R_X86_64_64
Signature · 1 line
2.55.0
Re: [PATCH 10/12] efi/libstub: Remove obsolete .note.gnu.property workaround
Quoted 6 lines · Josh Poimboeuf
Commit
e2179a09ab08("efi/libstub: Disable -mbranch-protection") added --remove-section=.note.gnu.property to the stub objcopy invocation to work around a Clang bug where the note was emitted for code-less object files even with -mbranch-protection=none.
Quoted 8 lines
That was fixed by LLVM commit
a48f6079f288("[AArch64] Generate .note.gnu.property based on module flags") which was released with Clang 12. The minimum Clang version is now Clang 17, so this workaround is no longer needed.On arm64, the stub also no longer builds with -mbranch-protection=none, as it has inherited the kernel's flags since commit
8358098b9787("arm64: efi: Enable BTI codegen and add PE/COFF annotation").
I verified the above all looks correct. Thanks for the cleanup.
Quoted 30 lines
Remove the workaround. This fixes BTI on arm64, and is a no-op on RISC-V and LoongArch where the vmlinux generic NOTES macro discards it, and the x86 and ARM decompressors discard .note.* explicitly.
Signed-off-by Josh Poimboeuf <[email protected]>--- drivers/firmware/efi/libstub/Makefile | 6 ------ 1 file changed, 6 deletions(-)drivers/firmware/efi/libstub/Makefile +0
-7@@ -106,12 +106,6 @@ lib-$(CONFIG_UNACCEPTED_MEMORY) += unaccepted_memory.o bitmap.o find.o targets := $(lib-y) lib-y := $(patsubst %.o,%.stub.o,$(lib-y)) -# Even when -mbranch-protection=none is set, Clang will generate a -# .note.gnu.property for code-less object files (like lib/ctype.c), -# so work around this by explicitly removing the unwanted section. -# https://llvm.org/pr46480 -STUBCOPY_FLAGS-y += --remove-section=.note.gnu.property - STUBCOPY_RELOC-$(CONFIG_X86_32) := R_386_32 STUBCOPY_RELOC-$(CONFIG_X86_64) := R_X86_64_64 --2.55.0
Signature · 2 lines
Thanks, ~Nick Desaulniers
Ard Biesheuvel
maintainerRe: [PATCH 10/12] efi/libstub: Remove obsolete .note.gnu.property workaround
Quoted 20 lines · Nick Desaulniers
Quoted 19 lines · Josh Poimboeuf
Commit
e2179a09ab08("efi/libstub: Disable -mbranch-protection") added --remove-section=.note.gnu.property to the stub objcopy invocation to work around a Clang bug where the note was emitted for code-less object files even with -mbranch-protection=none.That was fixed by LLVM commit
a48f6079f288("[AArch64] Generate .note.gnu.property based on module flags") which was released with Clang 12. The minimum Clang version is now Clang 17, so this workaround is no longer needed.On arm64, the stub also no longer builds with -mbranch-protection=none, as it has inherited the kernel's flags since commit
8358098b9787("arm64: efi: Enable BTI codegen and add PE/COFF annotation").Remove the workaround. This fixes BTI on arm64, and is a no-op on RISC-V and LoongArch where the vmlinux generic NOTES macro discards it, and the x86 and ARM decompressors discard .note.* explicitly.
Quoted 9 lines
Quoted 5 lines
Signed-off-by Josh Poimboeuf <[email protected]>--- drivers/firmware/efi/libstub/Makefile | 6 ------ 1 file changed, 6 deletions(-)I verified the above all looks correct. Thanks for the cleanup.
Reviewed-by Nick Desaulniers <[email protected]>
Thanks. I've cherry-picked just this patch for v7.3
Josh Poimboeuf
maintainerPATCH 11/12 arm64/bti: Force-enable BTI linker veneers
In reply to Josh PoimboeufThe linker only emits the BTI veneers if *all* input objects have GNU_PROPERTY_AARCH64_FEATURE_1_BTI. All the kernel's objects now advertise it, but if some object stops doing so, BTI veneers silently stop getting created on >128MB kernels, resulting in BTI exceptions at runtime.
Force BTI veneers enabled with "-z force-bti" to prevent a single bad object from ruining things. This also emits a warning (or error with CONFIG_WERROR) if any objects are missing the bit.
--- arch/arm64/Makefile | 1 + 1 file changed, 1 insertion(+)
arch/arm64/Makefile +1 -0
@@ -25,6 +25,7 @@ endif ifeq ($(CONFIG_ARM64_BTI_KERNEL),y) KBUILD_AFLAGS += -include $(srctree)/arch/arm64/include/asm/bti-note.h +LDFLAGS_vmlinux += $(call ld-option,-z force-bti) endif cc_has_k_constraint := $(call try-run,echo \
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH 11/12] arm64/bti: Force-enable BTI linker veneers
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] ld-option incorrectly evaluates to true for -z force-bti on older GNU ld versions, causing build…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] ld-option incorrectly evaluates to true for -z force-bti on older GNU ld versions, causing build failures. --
--- Patch [11]: [PATCH 11/12] arm64/bti: Force-enable BTI linker veneers --- commit dc92fd3fe21010a76154b8b30a8e4ff7c0d55416
Author: Josh Poimboeuf <[email protected]>
arm64/bti: Force-enable BTI linker veneers
This commit force-enables BTI veneers using the "-z force-bti" linker option to ensure they are created even if some input objects are missing the BTI feature bit. This aims to prevent silent runtime BTI exceptions on larger kernels.
arch/arm64/Makefile endif · line 28
Quoted 9 lines
arch/arm64/Makefile +1
-0@@ -25,6 +25,7 @@ endif ifeq ($(CONFIG_ARM64_BTI_KERNEL),y) KBUILD_AFLAGS += -include $(srctree)/arch/arm64/include/asm/bti-note.h +LDFLAGS_vmlinux += $(call ld-option,-z force-bti)
[Severity: Medium]
Does this ld-option check reliably detect if the linker supports "-z force-bti"?
When Kbuild's ld-option tests linker support, it executes the linker with "-v". However, when GNU ld is invoked with "-v" and no input files, it prints its version information and immediately exits with a 0 status code, completely bypassing any validation of keyword arguments like "-z".
Because of this, ld-option will falsely report that "-z force-bti" is supported on all versions of GNU ld, even older versions (prior to 2.34) where the option was not yet introduced.
During the actual kernel link, ld parses the "-z" list, fails to recognize "force-bti", and emits an ignored warning. If CONFIG_WERROR=y is set, the linker escalates this warning to a fatal error and breaks the build. On builds without CONFIG_WERROR, the warning is ignored and the patch silently fails to enforce the intended BTI veneers.
Quoted 3 lines
arch/arm64/Makefile +0
-0endif cc_has_k_constraint := $(call try-run,echo \
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Josh Poimboeuf
maintainerPATCH 12/12 arm64/bti: Enable kernel BTI for GCC
In reply to Josh PoimboeufCommit c0a454b9044f ("arm64/bti: Disable in kernel BTI when cross section thunks are broken") disabled in-kernel BTI for GCC because it omits a function's BTI landing pad when it determines that a function can only ever reached by a direct branch. The observed failure was a module whose init text landed far enough from its core text to need a PLT, whose "br x16" then hit a function with no "bti c".
Note that behavior isn't GCC-specific: Clang 21 has started ommitting landing pads as well.
Now that all the known BTI bugs have been sorted out, re-enable it for GCC.
--- arch/arm64/Kconfig | 2 -- 1 file changed, 2 deletions(-)
arch/arm64/Kconfig +0 -2
@@ -2114,8 +2114,6 @@ config ARM64_BTI_KERNEL depends on CC_HAS_BRANCH_PROT_PAC_RET_BTI # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697 depends on !CC_IS_GCC || GCC_VERSION >= 100100 - # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671 - depends on !CC_IS_GCC depends on (!FUNCTION_GRAPH_TRACER || DYNAMIC_FTRACE_WITH_ARGS) help Build the kernel with Branch Target Identification annotations
Signature · 1 line
2.55.0