arm64: Initial BTI kernel support
PATCH 1/5 arm64: bti: Support building kernel C code using BTI
In reply to Mark BrownWhen running with BTI enabled we need to ask the compiler to enable generation of BTI landing pads beyond those generated as a result of pointer authentication instructions being landing pads. Since the two features are practically speaking unlikely to be used separately we will make kernel mode BTI depend on pointer authentication in order to simplify the Makefile.
--- arch/arm64/Makefile | 4 ++++ 1 file changed, 4 insertions(+)
arch/arm64/Makefile +4 -0
@@ -67,7 +67,11 @@ endif ifeq ($(CONFIG_ARM64_PTR_AUTH),y) branch-prot-flags-$(CONFIG_CC_HAS_SIGN_RETURN_ADDRESS) := -msign-return-address=all +ifeq ($(CONFIG_ARM64_BTI_KERNEL),y) +branch-prot-flags-$(CONFIG_CC_HAS_BRANCH_PROT_PAC_RET_BTI) := -mbranch-protection=pac-ret+leaf+bti +else branch-prot-flags-$(CONFIG_CC_HAS_BRANCH_PROT_PAC_RET) := -mbranch-protection=pac-ret+leaf +endif # -march=armv8.3-a enables the non-nops instructions for PAC, to avoid the # compiler to generate them and consequently to break the single image contract # we pass it only to the assembler. This option is utilized only in case of non
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH 2/5 arm64: asm: Override SYM_FUNC_START when building the kernel with BTI
In reply to Mark BrownWhen the kernel is built for BTI override SYM_FUNC_START and related macros to add a BTI landing pad to the start of all global functions, ensuring that they are BTI safe. The ; at the end of the BTI_C macro is for the benefit of the macro-generated functions in xen-hypercall.S.
--- arch/arm64/include/asm/linkage.h | 46 ++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+)
arch/arm64/include/asm/linkage.h +46 -0
@@ -4,6 +4,52 @@#define __ALIGN .align 2#define __ALIGN_STR ".align 2" +#ifdef CONFIG_ARM64_BTI_KERNEL + +/* +* Since current versions of gas reject the BTI instruction unless we +* set the architecture version to v8.5 we use the hint instruction +* instead. +*/ +#define BTI_C hint 34 ; +#define BTI_J hint 36 ; + +/* +* When using in-kernel BTI we need to ensure that assembly functions +* have suitable annotations. Override SYM_FUNC_START to insert a BTI +* landing pad at the start of everything. +*/ +#define SYM_FUNC_START (name) \ + SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN) \ + BTI_C + +#define SYM_FUNC_START_NOALIGN (name) \ + SYM_START(name, SYM_L_GLOBAL, SYM_A_NONE) \ + BTI_C + +#define SYM_FUNC_START_LOCAL (name) \ + SYM_START(name, SYM_L_LOCAL, SYM_A_ALIGN) \ + BTI_C + +#define SYM_FUNC_START_LOCAL_NOALIGN (name) \ + SYM_START(name, SYM_L_LOCAL, SYM_A_NONE) \ + BTI_C + +#define SYM_FUNC_START_WEAK (name) \ + SYM_START(name, SYM_L_WEAK, SYM_A_ALIGN) \ + BTI_C + +#define SYM_FUNC_START_WEAK_NOALIGN (name) \ + SYM_START(name, SYM_L_WEAK, SYM_A_NONE) \ + BTI_C + +#define SYM_INNER_LABEL (name, linkage) \ + .type name SYM_T_NONE ASM_NL \ + SYM_ENTRY(name, linkage, SYM_A_NONE) \ + BTI_J + +#endif +/* * Annotate a function as position independent, i.e., safe to be called before * the kernel virtual mapping is activated.
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
cover arm64: Initial BTI kernel support
This patch series extends the use of BTI to most kernel executable code, it does not currently handle BPF or the VDSO - those will be handled by future updates. It builds the kernel with annotations for BTI and then maps the kernel with GP based on the support on the boot CPU, rejecting secondaries that don't have BTI support. If there is a need to handle big.LITTLE systems with mismatched BTI support we will have to revisit this but no implementations currently exist.
This is just for initial review, it obviously can't be applied until after the merge window. It depends on the userspace BTI patches and the assembler annotation patches.
Since at least some versions of GCC appear to have an issue with generating incorrect BTI annotations when BTI is used in conjunction with pointer authentication for leaf functions this version of the series disables building with GCC entirely. The issue is still under investigation, no issues have been seen with clang.
Mark Brown (5):
arm64: bti: Support building kernel C code using BTI
arm64: asm: Override SYM_FUNC_START when building the kernel with BTI
arm64: Set GP bit in kernel page tables to enable BTI for the kernel
arm64: mm: Mark module text as guarded pages
arm64: bti: Provide Kconfig for kernel mode BTI
arch/arm64/Kconfig | 18 +++++++++++ arch/arm64/Makefile | 4 +++ arch/arm64/include/asm/linkage.h | 46 +++++++++++++++++++++++++++ arch/arm64/include/asm/pgtable-prot.h | 5 +++ arch/arm64/kernel/cpufeature.c | 4 +++ arch/arm64/mm/mmu.c | 24 ++++++++++++++ arch/arm64/mm/pageattr.c | 4 +-- 7 files changed, 103 insertions(+), 2 deletions(-)
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH 3/5 arm64: Set GP bit in kernel page tables to enable BTI for the kernel
Now that the kernel is built with BTI annotations enable the feature by setting the GP bit in the stage 1 translation tables. This is done based on the features supported by the boot CPU so that we do not need to rewrite the translation tables.
In order to avoid potential issues on big.LITTLE systems when there are a mix of BTI and non-BTI capable CPUs in the system when we have enabled kernel mode BTI we change BTI to be a _STRICT_BOOT_CPU_FEATURE when we have kernel BTI. This will prevent any CPUs that don't support BTI being started if the boot CPU supports BTI rather than simply not using BTI as we do when supporting BTI only in userspace. The main concern is the possibility of BTYPE being preserved by a CPU that does not implement BTI when a thread is migrated to it resulting in an incorrect state which could generate an exception when the thread migrates back to a CPU that does support BTI. If we encounter practical systems which mix BTI and non-BTI CPUs we will need to revisit this implementation.
Since we currently do not generate landing pads in the BPF JIT we only map the base kernel text in this way.
--- arch/arm64/include/asm/pgtable-prot.h | 5 +++++ arch/arm64/kernel/cpufeature.c | 4 ++++ arch/arm64/mm/mmu.c | 24 ++++++++++++++++++++++++ 3 files changed, 33 insertions(+)
arch/arm64/include/asm/pgtable-prot.h +5 -0
@@ -21,6 +21,7 @@#ifndef __ASSEMBLY__ +#include <asm/cpufeature.h> #include <asm/pgtable-types.h> extern bool arm64_use_ng_mappings;
@@ -31,6 +32,8 @@ extern bool arm64_use_ng_mappings;#define PTE_MAYBE_NG (arm64_use_ng_mappings ? PTE_NG : 0)#define PMD_MAYBE_NG (arm64_use_ng_mappings ? PMD_SECT_NG : 0) +#define PTE_MAYBE_GP (system_supports_bti() ? PTE_GP : 0) +#define PROT_DEFAULT (_PROT_DEFAULT | PTE_MAYBE_NG)#define PROT_SECT_DEFAULT (_PROT_SECT_DEFAULT | PMD_MAYBE_NG)
@@ -50,7 +53,9 @@ extern bool arm64_use_ng_mappings;#define PAGE_KERNEL __pgprot(PROT_NORMAL)#define PAGE_KERNEL_RO __pgprot((PROT_NORMAL & ~PTE_WRITE) | PTE_RDONLY)#define PAGE_KERNEL_ROX __pgprot((PROT_NORMAL & ~(PTE_WRITE | PTE_PXN)) | PTE_RDONLY) +#define PAGE_KERNEL_ROX_GP __pgprot((PROT_NORMAL & ~(PTE_WRITE | PTE_PXN)) | PTE_RDONLY | PTE_GP)#define PAGE_KERNEL_EXEC __pgprot(PROT_NORMAL & ~PTE_PXN) +#define PAGE_KERNEL_EXEC_GP __pgprot((PROT_NORMAL & ~PTE_PXN) | PTE_GP)#define PAGE_KERNEL_EXEC_CONT __pgprot((PROT_NORMAL & ~PTE_PXN) | PTE_CONT)#define PAGE_HYP __pgprot(_HYP_PAGE_DEFAULT | PTE_HYP | PTE_HYP_XN)
arch/arm64/kernel/cpufeature.c +4 -0
@@ -1802,7 +1802,11 @@ static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "Branch Target Identification", .capability =ARM64_BTI , +#ifdef CONFIG_ARM64_BTI_KERNEL + .type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE, +#else .type = ARM64_CPUCAP_SYSTEM_FEATURE, +#endif .matches = has_cpuid_feature, .cpu_enable = bti_enable, .sys_reg = SYS_ID_AA64PFR1_EL1,
arch/arm64/mm/mmu.c +24 -0
@@ -609,6 +609,22 @@ static int __init map_entry_trampoline(void)core_initcall (map_entry_trampoline);#endif +/* +* Open coded check for BTI, only for use to determine configuration +* for early mappings for before the cpufeature code has run. +*/ +static bool arm64_early_this_cpu_has_bti (void ) +{ +u64 pfr1; + +if (!IS_ENABLED (CONFIG_ARM64_BTI_KERNEL )) +return false; + + pfr1 =read_sysreg_s (SYS_ID_AA64PFR1_EL1 ); +return cpuid_feature_extract_unsigned_field (pfr1, +ID_AA64PFR1_BT_SHIFT ); +} +/* * Create fine-grained mappings for the kernel. */
@@ -624,6 +640,14 @@ static void __init map_kernel(pgd_t *pgdp)*/ pgprot_t text_prot = rodata_enabled ?PAGE_KERNEL_ROX :PAGE_KERNEL_EXEC ; +/* +* If we have a CPU that supports BTI and a kernel built for +* BTI then mark the kernel executable text as guarded pages +* now so we don't have to rewrite the page tables later. +*/ +if (arm64_early_this_cpu_has_bti ()) + text_prot =__pgprot_modify (text_prot,PTE_GP ,PTE_GP ); +/* * Only rodata will be remapped with different permissions later on, * all other segments are allowed to use contiguous mappings.
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH 5/5 arm64: bti: Provide Kconfig for kernel mode BTI
In reply to Mark BrownNow that all the code is in place provide a Kconfig option allowing users to enable BTI for the kernel if their toolchain supports it, defaulting it on since this has security benefits. This is a separate configuration option since we currently don't support secondary CPUs that lack BTI if the boot CPU supports it.
Current testing appears to show an issue with GCC which causes it to emit unsuitable landing pads for function entries in some cases, until this issue is understood and either our usage fixed or a suitable version of GCC is identified the feature is marked as incompatible with GCC. No issues have been identified with clang.
--- arch/arm64/Kconfig | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
arch/arm64/Kconfig +18 -0
@@ -1607,6 +1607,24 @@ config ARM64_BTI BTI, such binaries can still run, but you get no additional enforcement of branch destinations. +config ARM64_BTI_KERNEL + bool "Use Branch Target Identification for kernel" + default y + depends on ARM64_BTI + depends on ARM64_PTR_AUTH + depends on CC_HAS_BRANCH_PROT_PAC_RET_BTI + depends on !CC_IS_GCC + depends on (!FUNCTION_GRAPH_TRACER || DYNAMIC_FTRACE_WITH_REGS) + help + Build the kernel with Branch Target Identification annotations + and enable enforcement of this for kernel code. When this option + is enabled and the system supports BTI all kernel code including + modular code must have BTI enabled. + +config CC_HAS_BRANCH_PROT_PAC_RET_BTI + # GCC 9 or later, clang 8 or later + def_bool $(cc-option,-mbranch-protection=pac-ret+leaf+bti) + config ARM64_E0PD bool "Enable support for E0PD" default y
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH 4/5 arm64: mm: Mark module text as guarded pages
In reply to Mark BrownWhen the kernel is built for BTI and running on a system which supports it extend guard page coverage to include executable code from loadable modules.
--- arch/arm64/mm/pageattr.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/mm/pageattr.c +2 -2
@@ -126,13 +126,13 @@ int set_memory_nx(unsigned long addr, int numpages) {return change_memory_common (addr, numpages,__pgprot (PTE_PXN ), -__pgprot (0 )); +__pgprot (PTE_MAYBE_GP )); }int set_memory_x (unsigned long addr,int numpages) {return change_memory_common (addr, numpages, -__pgprot (0 ), +__pgprot (PTE_MAYBE_GP ),__pgprot (PTE_PXN )); }
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 1/5] arm64: bti: Support building kernel C code using BTI
In reply to Mark BrownQuoted 6 lines · Mark Brown
When running with BTI enabled we need to ask the compiler to enable generation of BTI landing pads beyond those generated as a result of pointer authentication instructions being landing pads. Since the two features are practically speaking unlikely to be used separately we will make kernel mode BTI depend on pointer authentication in order to simplify the
Makefile.
Some vendors have been making chips with weird combinations of features. Is there a better justification to use beyond "unlikely"?
Quoted 11 lines
Signed-off-by Mark Brown <[email protected]>--- arch/arm64/Makefile | 4 ++++ 1 file changed, 4 insertions(+)arch/arm64/Makefile +0
-0
arch/arm64/Makefile endif · line 74
Quoted 8 lines
arch/arm64/Makefile +4
-0@@ -67,7 +67,11 @@ endif ifeq ($(CONFIG_ARM64_PTR_AUTH),y) branch-prot-flags-$(CONFIG_CC_HAS_SIGN_RETURN_ADDRESS) := -msign-return-address=all +ifeq ($(CONFIG_ARM64_BTI_KERNEL),y) +branch-prot-flags-$(CONFIG_CC_HAS_BRANCH_PROT_PAC_RET_BTI) := -mbranch-protection=pac-ret+leaf+bti +else branch-prot-flags-$(CONFIG_CC_HAS_BRANCH_PROT_PAC_RET) := -mbranch-protection=pac-ret+leaf +endif
The compiler appears to accept a leading +, so how about:
__branch-prot-opts-y =
ifeq ($(CONFIG_ARM64_PTR_AUTH),y) branch-prot-flags-$(CONFIG_CC_HAS_SIGN_RETURN_ADDRESS) := -msign-return-address=all __branch-prot-opts-$(CONFIG_CC_HAS_BRANCH_PROT_PAC_RET) += +pac-ret+leaf # -march=armv8.3-a enables the non-nops instructions for PAC, to avoid the
# compiler to generate them and consequently to break the single image contract # we pass it only to the assembler. This option is utilized only in case of non # integrated assemblers. branch-prot-flags-$(CONFIG_AS_HAS_PAC) += -Wa,-march=armv8.3-a endif
if ($(CONFIG_ARM64_BTI_KERNEL),y) __branch-prot-opts-$(CONFIG_CC_HAS_BRANCH_PROT_PAC_RET_BTI) += +bti endif
ifneq ($(___branch-prot-opts-y),)KBUILD_CFLAGS += -mbranch-protection=$(__branch-prot-opts-y) $(branch-prot-flags-y) endif
Or, this is all overkill. :)
-Kees
Quoted 6 lines
arch/arm64/Makefile +0
-0# -march=armv8.3-a enables the non-nops instructions for PAC, to avoid the # compiler to generate them and consequently to break the single image contract # we pass it only to the assembler. This option is utilized only in case of nonSignature · 1 line
2.20.1
Signature · 6 lines
Kees Cook _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 5/5] arm64: bti: Provide Kconfig for kernel mode BTI
In reply to Mark BrownQuoted 22 lines · Mark Brown
Now that all the code is in place provide a
Kconfigoption allowing users to enable BTI for the kernel if their toolchain supports it, defaulting it on since this has security benefits. This is a separate configuration option since we currently don't support secondary CPUs that lack BTI if the boot CPU supports it.Current testing appears to show an issue with GCC which causes it to emit unsuitable landing pads for function entries in some cases, until this issue is understood and either our usage fixed or a suitable version of GCC is identified the feature is marked as incompatible with GCC. No issues have been identified with clang.
Signed-off-by Mark Brown <[email protected]>--- arch/arm64/Kconfig | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)arch/arm64/Kconfig +0
-0
arch/arm64/Kconfig config ARM64_BTI · line 1615
Quoted 9 lines
arch/arm64/Kconfig +6
-0@@ -1607,6 +1607,24 @@ config ARM64_BTI BTI, such binaries can still run, but you get no additional enforcement of branch destinations. +config ARM64_BTI_KERNEL + bool "Use Branch Target Identification for kernel" + default y + depends on ARM64_BTI + depends on ARM64_PTR_AUTH + depends on CC_HAS_BRANCH_PROT_PAC_RET_BTI
(this depend can be dropped with my crazy Makefile suggestion)
arch/arm64/Kconfig config ARM64_BTI · line 1626
Quoted 11 lines
arch/arm64/Kconfig +11
-0+ depends on !CC_IS_GCC + depends on (!FUNCTION_GRAPH_TRACER || DYNAMIC_FTRACE_WITH_REGS) + help + Build the kernel with Branch Target Identification annotations + and enable enforcement of this for kernel code. When this option + is enabled and the system supports BTI all kernel code including + modular code must have BTI enabled. + +config CC_HAS_BRANCH_PROT_PAC_RET_BTI + # GCC 9 or later, clang 8 or later + def_bool $(cc-option,-mbranch-protection=pac-ret+leaf+bti)
Exciting! :)
Quoted 7 lines
arch/arm64/Kconfig +1
-0+ config ARM64_E0PD bool "Enable support for E0PD" default ySignature · 1 line
2.20.1
Signature · 6 lines
Kees Cook _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 1/5] arm64: bti: Support building kernel C code using BTI
In reply to Kees CookQuoted 11 lines · Kees Cook
Quoted 6 lines · Mark Brown
When running with BTI enabled we need to ask the compiler to enable generation of BTI landing pads beyond those generated as a result of pointer authentication instructions being landing pads. Since the two features are practically speaking unlikely to be used separately we will make kernel mode BTI depend on pointer authentication in order to simplify the
Makefile.Some vendors have been making chips with weird combinations of features. Is there a better justification to use beyond "unlikely"?
The design intent is that BTI is complementary to PAC so it would be a peculiar implementation choice to do BTI without also doing PAC but yes, implementors and system integrators have the freedom to innovate in ways that we cannot always forsee. The other bit of it is that there's fairly limited overhead from running PAC enabled code on hardware without the support.
Quoted 1 line
The compiler appears to accept a leading +, so how about:
...
Quoted 1 line
Or, this is all overkill. :)
I feel better about adding the extra dependency than feeding an option to the compiler that looks wrong like -mbranch-protection=+bti (more BTI!) but ultimately I don't have strong feelings either way so whatever Catalin and Will prefer.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 1/5] arm64: bti: Support building kernel C code using BTI
Quoted 4 lines · Mark Brown
I feel better about adding the extra dependency than feeding an option to the compiler that looks wrong like -mbranch-protection=+bti (more BTI!) but ultimately I don't have strong feelings either way so whatever Catalin and Will prefer.
Cool cool. If I end up with a use-case for stand-alone BTI I'll take the responsibility to sort out the future patches to support it. :)
Signature · 6 lines
Kees Cook _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 1/5] arm64: bti: Support building kernel C code using BTI
Quoted 9 lines · Kees Cook
Quoted 4 lines · Mark Brown
I feel better about adding the extra dependency than feeding an option to the compiler that looks wrong like -mbranch-protection=+bti (more BTI!) but ultimately I don't have strong feelings either way so whatever Catalin and Will prefer.
Cool cool. If I end up with a use-case for stand-alone BTI I'll take the responsibility to sort out the future patches to support it. :)
It turns out that we also need to explicitly force -mbranch-protection=none for cases where we have PAC disabled for the kernel but userspace support (configuring the pointer auth for userspace can end up causing pointer auth instructions in the kernel to start doing things we're not ready for) and at least GCC doesn't like none+anything combinations anyway.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel