arm64: BTI kernel and vDSO support
Subsystems (3)
v3 cover arm64: BTI kernel and vDSO support
This patch series adds support for protecting the kernel and vDSO with BTI including code compiled with the BPF JIT at runtime.
We build the kernel with annotations for BTI and then map the kernel with GP based on the support on the boot CPU, rejecting secondaries that don't have BTI support. If there is a need to handle big.LITTLE systems with mismatched BTI support we will have to revisit this, currently no such implementations exist.
This series depends on several branches in the arm64 tree:
- for-next/bti-user
- for-next/insn
- for-next/asm
v3:
- Add a patch adding a comment about why we enable leaf support for PAC.
- Fix build of the 32 bit vDSO.
- Refactor the macro for emitting the ELF note for BTI code so that the flags are defined separately in order to make it easier to add handling for any future users.
v2:
- Enable support for building with GCC version 10 and later, a fix for BTI code generation is being backported to GCC 9 but is not yet available.
- Add BPF support.
- Remove some unused page attribute defines.
- One assembler modernisation patch has been removed and sent separately.
Mark Brown (11):
arm64: Document why we enable PAC support for leaf functions
arm64: bti: Support building kernel C code using BTI
arm64: asm: Override SYM_FUNC_START when building the kernel with BTI
arm64: Set GP bit in kernel page tables to enable BTI for the kernel
arm64: bpf: Annotate JITed code for BTI
arm64: mm: Mark executable text as guarded pages
arm64: bti: Provide Kconfig for kernel mode BTI
arm64: asm: Provide a mechanism for generating ELF note for BTI
arm64: vdso: Annotate for BTI
arm64: vdso: Force the vDSO to be linked as BTI when built for BTI
arm64: vdso: Map the vDSO text with guarded pages when built for BTI
arch/arm64/Kconfig | 18 ++++++++++ arch/arm64/Makefile | 7 ++++ arch/arm64/include/asm/assembler.h | 50 +++++++++++++++++++++++++++ arch/arm64/include/asm/linkage.h | 46 ++++++++++++++++++++++++ arch/arm64/include/asm/pgtable-prot.h | 3 ++ arch/arm64/kernel/cpufeature.c | 4 +++ arch/arm64/kernel/vdso.c | 6 +++- arch/arm64/kernel/vdso/Makefile | 4 ++- arch/arm64/kernel/vdso/note.S | 3 ++ arch/arm64/kernel/vdso/sigreturn.S | 3 ++ arch/arm64/kernel/vdso/vdso.S | 3 ++ arch/arm64/mm/mmu.c | 24 +++++++++++++ arch/arm64/mm/pageattr.c | 4 +-- arch/arm64/net/bpf_jit.h | 8 +++++ arch/arm64/net/bpf_jit_comp.c | 12 +++++++ 15 files changed, 191 insertions(+), 4 deletions(-)
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 1/11 arm64: Document why we enable PAC support for leaf functions
Document the fact that we enable pointer authentication protection for leaf functions since there is some narrow potential for ROP protection benefits and little overhead has been observed.
--- arch/arm64/Makefile | 3 +++ 1 file changed, 3 insertions(+)
arch/arm64/Makefile +3 -0
@@ -71,6 +71,9 @@ branch-prot-flags-y += $(call cc-option,-mbranch-protection=none) ifeq ($(CONFIG_ARM64_PTR_AUTH),y) branch-prot-flags-$(CONFIG_CC_HAS_SIGN_RETURN_ADDRESS) := -msign-return-address=all +# We enable additional protection for leaf functions as there is some +# narrow potential for ROP protection benefits and no substantial +# performance impact has been observed. branch-prot-flags-$(CONFIG_CC_HAS_BRANCH_PROT_PAC_RET) := -mbranch-protection=pac-ret+leaf # -march=armv8.3-a enables the non-nops instructions for PAC, to avoid the # compiler to generate them and consequently to break the single image contract
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 2/11 arm64: bti: Support building kernel C code using BTI
In reply to Mark BrownWhen running with BTI enabled we need to ask the compiler to enable generation of BTI landing pads beyond those generated as a result of pointer authentication instructions being landing pads. Since the two features are practically speaking unlikely to be used separately we will make kernel mode BTI depend on pointer authentication in order to simplify the Makefile.
--- arch/arm64/Makefile | 4 ++++ 1 file changed, 4 insertions(+)
arch/arm64/Makefile +4 -0
@@ -74,7 +74,11 @@ branch-prot-flags-$(CONFIG_CC_HAS_SIGN_RETURN_ADDRESS) := -msign-return-address= # We enable additional protection for leaf functions as there is some # narrow potential for ROP protection benefits and no substantial # performance impact has been observed. +ifeq ($(CONFIG_ARM64_BTI_KERNEL),y) +branch-prot-flags-$(CONFIG_CC_HAS_BRANCH_PROT_PAC_RET_BTI) := -mbranch-protection=pac-ret+leaf+bti +else branch-prot-flags-$(CONFIG_CC_HAS_BRANCH_PROT_PAC_RET) := -mbranch-protection=pac-ret+leaf +endif # -march=armv8.3-a enables the non-nops instructions for PAC, to avoid the # compiler to generate them and consequently to break the single image contract # we pass it only to the assembler. This option is utilized only in case of non
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 3/11 arm64: asm: Override SYM_FUNC_START when building the kernel with BTI
In reply to Mark BrownWhen the kernel is built for BTI override SYM_FUNC_START and related macros to add a BTI landing pad to the start of all global functions, ensuring that they are BTI safe. The ; at the end of the BTI_x macros is for the benefit of the macro-generated functions in xen-hypercall.S.
--- arch/arm64/include/asm/linkage.h | 46 ++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+)
arch/arm64/include/asm/linkage.h +46 -0
@@ -4,6 +4,52 @@#define __ALIGN .align 2#define __ALIGN_STR ".align 2" +#if defined(CONFIG_ARM64_BTI_KERNEL ) && defined(__aarch64__) + +/* +* Since current versions of gas reject the BTI instruction unless we +* set the architecture version to v8.5 we use the hint instruction +* instead. +*/ +#define BTI_C hint 34 ; +#define BTI_J hint 36 ; + +/* +* When using in-kernel BTI we need to ensure that assembly functions +* have suitable annotations. Override SYM_FUNC_START to insert a BTI +* landing pad at the start of everything. +*/ +#define SYM_FUNC_START (name) \ + SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN) \ + BTI_C + +#define SYM_FUNC_START_NOALIGN (name) \ + SYM_START(name, SYM_L_GLOBAL, SYM_A_NONE) \ + BTI_C + +#define SYM_FUNC_START_LOCAL (name) \ + SYM_START(name, SYM_L_LOCAL, SYM_A_ALIGN) \ + BTI_C + +#define SYM_FUNC_START_LOCAL_NOALIGN (name) \ + SYM_START(name, SYM_L_LOCAL, SYM_A_NONE) \ + BTI_C + +#define SYM_FUNC_START_WEAK (name) \ + SYM_START(name, SYM_L_WEAK, SYM_A_ALIGN) \ + BTI_C + +#define SYM_FUNC_START_WEAK_NOALIGN (name) \ + SYM_START(name, SYM_L_WEAK, SYM_A_NONE) \ + BTI_C + +#define SYM_INNER_LABEL (name, linkage) \ + .type name SYM_T_NONE ASM_NL \ + SYM_ENTRY(name, linkage, SYM_A_NONE) \ + BTI_J + +#endif +/* * Annotate a function as position independent, i.e., safe to be called before * the kernel virtual mapping is activated.
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 4/11 arm64: Set GP bit in kernel page tables to enable BTI for the kernel
In reply to Mark BrownNow that the kernel is built with BTI annotations enable the feature by setting the GP bit in the stage 1 translation tables. This is done based on the features supported by the boot CPU so that we do not need to rewrite the translation tables.
In order to avoid potential issues on big.LITTLE systems when there are a mix of BTI and non-BTI capable CPUs in the system when we have enabled kernel mode BTI we change BTI to be a _STRICT_BOOT_CPU_FEATURE when we have kernel BTI. This will prevent any CPUs that don't support BTI being started if the boot CPU supports BTI rather than simply not using BTI as we do when supporting BTI only in userspace. The main concern is the possibility of BTYPE being preserved by a CPU that does not implement BTI when a thread is migrated to it resulting in an incorrect state which could generate an exception when the thread migrates back to a CPU that does support BTI. If we encounter practical systems which mix BTI and non-BTI CPUs we will need to revisit this implementation.
Since we currently do not generate landing pads in the BPF JIT we only map the base kernel text in this way.
--- arch/arm64/include/asm/pgtable-prot.h | 3 +++ arch/arm64/kernel/cpufeature.c | 4 ++++ arch/arm64/mm/mmu.c | 24 ++++++++++++++++++++++++ 3 files changed, 31 insertions(+)
arch/arm64/include/asm/pgtable-prot.h +3 -0
@@ -21,6 +21,7 @@#ifndef __ASSEMBLY__ +#include <asm/cpufeature.h> #include <asm/pgtable-types.h> extern bool arm64_use_ng_mappings;
@@ -31,6 +32,8 @@ extern bool arm64_use_ng_mappings;#define PTE_MAYBE_NG (arm64_use_ng_mappings ? PTE_NG : 0)#define PMD_MAYBE_NG (arm64_use_ng_mappings ? PMD_SECT_NG : 0) +#define PTE_MAYBE_GP (system_supports_bti() ? PTE_GP : 0) +#define PROT_DEFAULT (_PROT_DEFAULT | PTE_MAYBE_NG)#define PROT_SECT_DEFAULT (_PROT_SECT_DEFAULT | PMD_MAYBE_NG)
arch/arm64/kernel/cpufeature.c +4 -0
@@ -1800,7 +1800,11 @@ static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "Branch Target Identification", .capability =ARM64_BTI , +#ifdef CONFIG_ARM64_BTI_KERNEL + .type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE, +#else .type = ARM64_CPUCAP_SYSTEM_FEATURE, +#endif .matches = has_cpuid_feature, .cpu_enable = bti_enable, .sys_reg = SYS_ID_AA64PFR1_EL1,
arch/arm64/mm/mmu.c +24 -0
@@ -609,6 +609,22 @@ static int __init map_entry_trampoline(void)core_initcall (map_entry_trampoline);#endif +/* +* Open coded check for BTI, only for use to determine configuration +* for early mappings for before the cpufeature code has run. +*/ +static bool arm64_early_this_cpu_has_bti (void ) +{ +u64 pfr1; + +if (!IS_ENABLED (CONFIG_ARM64_BTI_KERNEL )) +return false; + + pfr1 =read_sysreg_s (SYS_ID_AA64PFR1_EL1 ); +return cpuid_feature_extract_unsigned_field (pfr1, +ID_AA64PFR1_BT_SHIFT ); +} +/* * Create fine-grained mappings for the kernel. */
@@ -624,6 +640,14 @@ static void __init map_kernel(pgd_t *pgdp)*/ pgprot_t text_prot = rodata_enabled ?PAGE_KERNEL_ROX :PAGE_KERNEL_EXEC ; +/* +* If we have a CPU that supports BTI and a kernel built for +* BTI then mark the kernel executable text as guarded pages +* now so we don't have to rewrite the page tables later. +*/ +if (arm64_early_this_cpu_has_bti ()) + text_prot =__pgprot_modify (text_prot,PTE_GP ,PTE_GP ); +/* * Only rodata will be remapped with different permissions later on, * all other segments are allowed to use contiguous mappings.
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 5/11 arm64: bpf: Annotate JITed code for BTI
In reply to Mark BrownIn order to extend the protection offered by BTI to all code executing in kernel mode we need to annotate JITed BPF code appropriately for BTI. To do this we need to add a landing pad to the start of each BPF function and also immediately after the function prologue if we are emitting a function which can be tail called. Jumps within BPF functions are all to immediate offsets and therefore do not require landing pads.
--- arch/arm64/net/bpf_jit.h | 8 ++++++++ arch/arm64/net/bpf_jit_comp.c | 12 ++++++++++++ 2 files changed, 20 insertions(+)
arch/arm64/net/bpf_jit.h +8 -0
@@ -189,4 +189,12 @@/* Rn & Rm; set condition flags */ #define A64_TST (sf, Rn, Rm) A64_ANDS(sf, A64_ZR, Rn, Rm) +/* HINTs */ +#define A64_HINT (x) aarch64_insn_gen_hint(x) + +/* BTI */ +#define A64_BTI_C A64_HINT(AARCH64_INSN_HINT_BTIC) +#define A64_BTI_J A64_HINT(AARCH64_INSN_HINT_BTIJ) +#define A64_BTI_JC A64_HINT(AARCH64_INSN_HINT_BTIJC) +#endif /* _BPF_JIT_H */
arch/arm64/net/bpf_jit_comp.c +12 -0
@@ -171,7 +171,11 @@ static inline int epilogue_offset(const struct jit_ctx *ctx)#define STACK_ALIGN (sz) (((sz) + 15) & ~15)/* Tail call offset to jump into */ +#if IS_ENABLED (CONFIG_ARM64_BTI_KERNEL ) +#define PROLOGUE_OFFSET 8 +#else #define PROLOGUE_OFFSET 7 +#endif static int build_prologue (struct jit_ctx *ctx,bool ebpf_from_cbpf) {
@@ -208,6 +212,10 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf)* */ +/* BTI landing pad */ +if (IS_ENABLED (CONFIG_ARM64_BTI_KERNEL )) +emit (A64_BTI_C , ctx); +/* Save FP and LR registers to stay align with ARM64 AAPCS */ emit (A64_PUSH (A64_FP ,A64_LR ,A64_SP ), ctx);emit (A64_MOV (1 ,A64_FP ,A64_SP ), ctx);
@@ -230,6 +238,10 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf) cur_offset, PROLOGUE_OFFSET);return -1 ; } + +/* BTI landing pad for the tail call, done with a BR */ +if (IS_ENABLED (CONFIG_ARM64_BTI_KERNEL )) +emit (A64_BTI_J , ctx); } ctx->stack_size =STACK_ALIGN (prog->aux->stack_depth);
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH v3 05/11] arm64: bpf: Annotate JITed code for BTI
[ Cc +bpf ]
Quoted 9 lines · Mark Brown
In order to extend the protection offered by BTI to all code executing in kernel mode we need to annotate JITed BPF code appropriately for BTI. To do this we need to add a landing pad to the start of each BPF function and also immediately after the function prologue if we are emitting a function which can be tail called. Jumps within BPF functions are all to immediate offsets and therefore do not require landing pads.
Signed-off-by Mark Brown <[email protected]>Reviewed-by Catalin Marinas <[email protected]>
Quoted 61 lines
--- arch/arm64/net/bpf_jit.h | 8 ++++++++ arch/arm64/net/bpf_jit_comp.c | 12 ++++++++++++ 2 files changed, 20 insertions(+)arch/arm64/net/bpf_jit.h +8
-0@@ -189,4 +189,12 @@/* Rn & Rm; set condition flags */ #define A64_TST (sf, Rn, Rm) A64_ANDS(sf, A64_ZR, Rn, Rm) +/* HINTs */ +#define A64_HINT (x) aarch64_insn_gen_hint(x) + +/* BTI */ +#define A64_BTI_C A64_HINT(AARCH64_INSN_HINT_BTIC) +#define A64_BTI_J A64_HINT(AARCH64_INSN_HINT_BTIJ) +#define A64_BTI_JC A64_HINT(AARCH64_INSN_HINT_BTIJC) +#endif /* _BPF_JIT_H */ arch/arm64/net/bpf_jit_comp.c +12
-0@@ -171,7 +171,11 @@ static inline int epilogue_offset(const struct jit_ctx *ctx)#define STACK_ALIGN (sz) (((sz) + 15) & ~15)/* Tail call offset to jump into */ +#if IS_ENABLED (CONFIG_ARM64_BTI_KERNEL ) +#define PROLOGUE_OFFSET 8 +#else #define PROLOGUE_OFFSET 7 +#endif static int build_prologue (struct jit_ctx *ctx,bool ebpf_from_cbpf) {@@ -208,6 +212,10 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf)* */ +/* BTI landing pad */ +if (IS_ENABLED (CONFIG_ARM64_BTI_KERNEL )) +emit (A64_BTI_C , ctx); +/* Save FP and LR registers to stay align with ARM64 AAPCS */ emit (A64_PUSH (A64_FP ,A64_LR ,A64_SP ), ctx);emit (A64_MOV (1 ,A64_FP ,A64_SP ), ctx);@@ -230,6 +238,10 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf) cur_offset, PROLOGUE_OFFSET);return -1 ; } + +/* BTI landing pad for the tail call, done with a BR */ +if (IS_ENABLED (CONFIG_ARM64_BTI_KERNEL )) +emit (A64_BTI_J , ctx); } ctx->stack_size =STACK_ALIGN (prog->aux->stack_depth);
PATCH v3 6/11 arm64: mm: Mark executable text as guarded pages
In reply to Mark BrownWhen the kernel is built for BTI and running on a system which supports make all executable text guarded pages to ensure that loadable module and JITed BPF code is protected by BTI.
--- arch/arm64/mm/pageattr.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/mm/pageattr.c +2 -2
@@ -126,13 +126,13 @@ int set_memory_nx(unsigned long addr, int numpages) {return change_memory_common (addr, numpages,__pgprot (PTE_PXN ), -__pgprot (0 )); +__pgprot (PTE_MAYBE_GP )); }int set_memory_x (unsigned long addr,int numpages) {return change_memory_common (addr, numpages, -__pgprot (0 ), +__pgprot (PTE_MAYBE_GP ),__pgprot (PTE_PXN )); }
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 7/11 arm64: bti: Provide Kconfig for kernel mode BTI
In reply to Mark BrownNow that all the code is in place provide a Kconfig option allowing users to enable BTI for the kernel if their toolchain supports it, defaulting it on since this has security benefits. This is a separate configuration option since we currently don't support secondary CPUs that lack BTI if the boot CPU supports it.
Code generation issues mean that current GCC 9 versions are not able to produce usable BTI binaries so we disable support for building with GCC versions prior to 10, once a fix is backported to GCC 9 the dependencies will be updated.
--- arch/arm64/Kconfig | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
arch/arm64/Kconfig +18 -0
@@ -1610,6 +1610,24 @@ config ARM64_BTI BTI, such binaries can still run, but you get no additional enforcement of branch destinations. +config ARM64_BTI_KERNEL + bool "Use Branch Target Identification for kernel" + default y + depends on ARM64_BTI + depends on ARM64_PTR_AUTH + depends on CC_HAS_BRANCH_PROT_PAC_RET_BTI + depends on !CC_IS_GCC || GCC_VERSION >= 100000 + depends on (!FUNCTION_GRAPH_TRACER || DYNAMIC_FTRACE_WITH_REGS) + help + Build the kernel with Branch Target Identification annotations + and enable enforcement of this for kernel code. When this option + is enabled and the system supports BTI all kernel code including + modular code must have BTI enabled. + +config CC_HAS_BRANCH_PROT_PAC_RET_BTI + # GCC 9 or later, clang 8 or later + def_bool $(cc-option,-mbranch-protection=pac-ret+leaf+bti) + config ARM64_E0PD bool "Enable support for E0PD" default y
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 8/11 arm64: asm: Provide a mechanism for generating ELF note for BTI
In reply to Mark BrownELF files built for BTI should have a program property note section which identifies them as such. The linker expects to find this note in all object files it is linking into a BTI annotated output, the compiler will ensure that this happens for C files but for assembler files we need to do this in the source so provide a macro which can be used for this purpose. To support likely future requirements for additional notes we split the defininition of the flags to set for BTI code from the macro that creates the note itself.
This is mainly for use in the vDSO which should be a normal ELF shared library and should therefore include BTI annotations when built for BTI.
--- arch/arm64/include/asm/assembler.h | 50 ++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+)
arch/arm64/include/asm/assembler.h +50 -0
@@ -736,4 +736,54 @@ USER(\label, ic ivau, \tmp2) // invalidate I line PoU .Lyield_out_\@ : .endm +/* +* This macro emits a program property note section identifying +* architecture features which require special handling, mainly for +* use in assembly files included in the VDSO. + */ + +#define NT_GNU_PROPERTY_TYPE_0 5 +#define GNU_PROPERTY_AARCH64_FEATURE_1_AND 0xc0000000 + +#define GNU_PROPERTY_AARCH64_FEATURE_1_BTI (1U << 0) +#define GNU_PROPERTY_AARCH64_FEATURE_1_PAC (1U << 1) + +#ifdef CONFIG_ARM64_BTI_KERNEL +#define GNU_PROPERTY_AARCH64_FEATURE_1_DEFAULT \ + ((GNU_PROPERTY_AARCH64_FEATURE_1_BTI | \ + GNU_PROPERTY_AARCH64_FEATURE_1_PAC)) +#endif + +#ifdef GNU_PROPERTY_AARCH64_FEATURE_1_DEFAULT +.macro emit_aarch64_feature_1_and, feat=GNU_PROPERTY_AARCH64_FEATURE_1_DEFAULT + .pushsection .note.gnu.property, "a" + .align 3 + .long 2f - 1f + .long 6f - 3f + .long NT_GNU_PROPERTY_TYPE_0 +1: .string "GNU" +2: + .align 3 +3: .long GNU_PROPERTY_AARCH64_FEATURE_1_AND + .long 5f - 4f +4: +/* +* This is described with an array of char in the Linux API +* spec but the text and all other usage (including binutils, +* clang and GCC) treat this as a 32 bit value so no swizzling +* is required for big endian. +*/ + .long \feat +5: + .align 3 +6: + .popsection +.endm + +#else +.macro emit_aarch64_feature_1_and, feat=0 +.endm + +#endif /* GNU_PROPERTY_AARCH64_FEATURE_1_DEFAULT */ +#endif /* __ASM_ASSEMBLER_H */
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 9/11 arm64: vdso: Annotate for BTI
In reply to Mark BrownGenerate BTI annotations for all assembly files included in the 64 bit vDSO.
--- arch/arm64/kernel/vdso/note.S | 3 +++ arch/arm64/kernel/vdso/sigreturn.S | 3 +++ arch/arm64/kernel/vdso/vdso.S | 3 +++ 3 files changed, 9 insertions(+)
arch/arm64/kernel/vdso/note.S +3 -0
@@ -12,9 +12,12 @@ #include <linux/version.h> #include <linux/elfnote.h> #include <linux/build-salt.h> +#include <asm/assembler.h> ELFNOTE_START(Linux, 0, "a") .long LINUX_VERSION_CODE ELFNOTE_END BUILD_SALT + +emit_aarch64_feature_1_and
arch/arm64/kernel/vdso/sigreturn.S +3 -0
@@ -9,6 +9,7 @@ */ #include <linux/linkage.h> +#include <asm/assembler.h> #include <asm/unistd.h> .text
@@ -24,3 +25,5 @@ SYM_FUNC_START(__kernel_rt_sigreturn) svc #0 .cfi_endproc SYM_FUNC_END(__kernel_rt_sigreturn) + +emit_aarch64_feature_1_and
arch/arm64/kernel/vdso/vdso.S +3 -0
@@ -8,6 +8,7 @@ #include <linux/init.h> #include <linux/linkage.h> #include <linux/const.h> +#include <asm/assembler.h> #include <asm/page.h> .globl vdso_start, vdso_end
@@ -19,3 +20,5 @@ vdso_start: vdso_end: .previous + +emit_aarch64_feature_1_and
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 10/11 arm64: vdso: Force the vDSO to be linked as BTI when built for BTI
In reply to Mark BrownWhen the kernel and hence vDSO are built with BTI enabled force the linker to link the vDSO as BTI. This will cause the linker to warn if any of the input files do not have the BTI annotation, ensuring that we don't silently fail to provide a vDSO that is built and annotated for BTI when the kernel is being built with BTI.
--- arch/arm64/kernel/vdso/Makefile | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/arm64/kernel/vdso/Makefile +3 -1
@@ -17,8 +17,10 @@ obj-vdso := vgettimeofday.o note.o sigreturn.o targets := $(obj-vdso) vdso.so vdso.so.dbg obj-vdso := $(addprefix $(obj)/, $(obj-vdso)) +btildflags-$(CONFIG_ARM64_BTI_KERNEL) += -z force-bti + ldflags-y := -shared -nostdlib -soname=linux-vdso.so.1 --hash-style=sysv \ - --build-id -n -T + --build-id -n $(btildflags-y) -T ccflags-y := -fno-common -fno-builtin -fno-stack-protector -ffixed-x18 ccflags-y += -DDISABLE_BRANCH_PROFILING
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 11/11 arm64: vdso: Map the vDSO text with guarded pages when built for BTI
In reply to Mark BrownThe kernel is responsible for mapping the vDSO into userspace processes, including mapping the text section as executable. Handle the mapping of the vDSO for BTI similarly, mapping the text section as guarded pages so the BTI annotations in the vDSO become effective when they are present.
This will mean that we can have BTI active for the vDSO in processes that do not otherwise support BTI. This should not be an issue for any expected use of the vDSO.
--- arch/arm64/kernel/vdso.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/arm64/kernel/vdso.c +5 -1
@@ -142,6 +142,7 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index, int uses_interp) {unsigned long vdso_base, vdso_text_len, vdso_mapping_len; +unsigned long gp_flags =0 ;void *ret; vdso_text_len = vdso_lookup[arch_index].vdso_pages <<PAGE_SHIFT ;
@@ -160,10 +161,13 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index,if (IS_ERR (ret)) goto up_fail; +if (IS_ENABLED (CONFIG_ARM64_BTI_KERNEL ) &&system_supports_bti ()) + gp_flags =VM_ARM64_BTI ; + vdso_base +=PAGE_SIZE ; mm->context.vdso = (void *)vdso_base; ret =_install_special_mapping (mm, vdso_base, vdso_text_len, -VM_READ |VM_EXEC | +VM_READ |VM_EXEC |gp_flags|VM_MAYREAD |VM_MAYWRITE |VM_MAYEXEC , vdso_lookup[arch_index].cm);if (IS_ERR (ret))
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Will Deacon
maintainerQuoted 15 lines · Mark Brown
This patch series adds support for protecting the kernel and vDSO with BTI including code compiled with the BPF JIT at runtime.
We build the kernel with annotations for BTI and then map the kernel with GP based on the support on the boot CPU, rejecting secondaries that don't have BTI support. If there is a need to handle big.LITTLE systems with mismatched BTI support we will have to revisit this, currently no such implementations exist.
This series depends on several branches in the arm64 tree:
- for-next/bti-user
- for-next/insn
- for-next/asm
Quoted 7 lines
v3:
- Add a patch adding a comment about why we enable leaf support for PAC.
- Fix build of the 32 bit vDSO.
- Refactor the macro for emitting the ELF note for BTI code so that the flags are defined separately in order to make it easier to add handling for any future users.
Bugger, I'm still getting warnings (clang 11.0.1), but from an allmodconfig build now:
warning: some functions compiled with BTI and some compiled without BTI
warning: not setting BTI in feature flags
(repeated many, many times).
I'll try to get you some more info.
Will
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Will Deacon
maintainerQuoted 27 lines · Will Deacon
Quoted 15 lines · Mark Brown
This patch series adds support for protecting the kernel and vDSO with BTI including code compiled with the BPF JIT at runtime.
We build the kernel with annotations for BTI and then map the kernel with GP based on the support on the boot CPU, rejecting secondaries that don't have BTI support. If there is a need to handle big.LITTLE systems with mismatched BTI support we will have to revisit this, currently no such implementations exist.
This series depends on several branches in the arm64 tree:
- for-next/bti-user
- for-next/insn
- for-next/asmQuoted 7 lines
v3:
- Add a patch adding a comment about why we enable leaf support for PAC.
- Fix build of the 32 bit vDSO.
- Refactor the macro for emitting the ELF note for BTI code so that the flags are defined separately in order to make it easier to add handling for any future users.Bugger, I'm still getting warnings (clang 11.0.1), but from an allmodconfig build now:
Quoted 6 lines
warning: some functions compiled with BTI and some compiled without BTI
warning: not setting BTI in feature flags(repeated many, many times).
I'll try to get you some more info.
Quick look at the log suggests that these are caused by HDRTEST, whatever that is.
Will
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Will Deacon
maintainerQuoted 31 lines · Will Deacon
Quoted 30 lines · Will Deacon
Quoted 15 lines · Mark Brown
This patch series adds support for protecting the kernel and vDSO with BTI including code compiled with the BPF JIT at runtime.
We build the kernel with annotations for BTI and then map the kernel with GP based on the support on the boot CPU, rejecting secondaries that don't have BTI support. If there is a need to handle big.LITTLE systems with mismatched BTI support we will have to revisit this, currently no such implementations exist.
This series depends on several branches in the arm64 tree:
- for-next/bti-user
- for-next/insn
- for-next/asmQuoted 7 lines
v3:
- Add a patch adding a comment about why we enable leaf support for PAC.
- Fix build of the 32 bit vDSO.
- Refactor the macro for emitting the ELF note for BTI code so that the flags are defined separately in order to make it easier to add handling for any future users.Bugger, I'm still getting warnings (clang 11.0.1), but from an allmodconfig build now:
warning: some functions compiled with BTI and some compiled without BTI
warning: not setting BTI in feature flags
Quoted 6 lines
Quoted 3 lines
(repeated many, many times).
I'll try to get you some more info.
Quick look at the log suggests that these are caused by HDRTEST, whatever that is.
Sorry, my parallel build threw me off, so I don't think it's HDRTEST after all. One of the problems appears to be tracing:
CC kernel/trace/trace_clock.o
warning: some functions compiled with BTI and some compiled without BTI
warning: not setting BTI in feature flags
Looking at objdump, there are funny gcov functions with PACISAP prologues:
0000000000023e40 <__llvm_gcov_writeout>:
23e40: a9be57fe stp x30, x21, [sp, #-32]! 23e44: a9014ff4 stp x20, x19, [sp, #16] 23e48: 94000000 bl 0 <__sanitizer_cov_trace_pc> 23e4c: 90000000 adrp x0, 0 <__register_ftrace_function> 23e50: 90000001 adrp x1, 0 <__register_ftrace_function> 23e54: 5293ac02 mov w2, #0x9d60 // #40288 23e58: 91000000 add x0, x0, #0x0
[...]
:/
Will
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 6 lines · Will Deacon
Sorry, my parallel build threw me off, so I don't think it's HDRTEST after all. One of the problems appears to be tracing:
CC
kernel/trace/trace_clock.o
warning: some functions compiled with BTI and some compiled without BTI
warning: not setting BTI in feature flags
Can you share a .config?
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Will Deacon
maintainerQuoted 10 lines · Mark Brown
Quoted 6 lines · Will Deacon
Sorry, my parallel build threw me off, so I don't think it's HDRTEST after all. One of the problems appears to be tracing:
CC
kernel/trace/trace_clock.o
warning: some functions compiled with BTI and some compiled without BTI
warning: not setting BTI in feature flagsCan you share a .config?
allmodconfig
Will
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 5 lines · Will Deacon
Quoted 1 line · Mark Brown
Can you share a .config?
allmodconfig
Right, I'm seeing it here now - it's when CONFIG_GCOV_KERNEL is enabled and happens for clang-10 as well but not a GCC 10 prerelease build. Adding
depends !(CC_IS_CLANG && GCOV_KERNEL)
avoids the warning but obviously we should actually fix the issue.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Will Deacon
maintainerQuoted 9 lines · Mark Brown
Quoted 5 lines · Will Deacon
Quoted 1 line · Mark Brown
Can you share a .config?
allmodconfig
Right, I'm seeing it here now - it's when
CONFIG_GCOV_KERNELis enabled and happens for clang-10 as well but not a GCC 10 prerelease build.
Interesting. Is that because GCC doesn't emit out-of-line GCOV functions, or does it emit PAC/BTI instructions for them instead? (you can disassemble one of the problematic opjects to have a look).
Quoted 5 lines
Adding
depends !(CC_IS_CLANG && GCOV_KERNEL)avoids the warning but obviously we should actually fix the issue.
I can't immediately see how to fix it, so your hack above might be the best bet for now. I'm just a little wary that it might not be limited to GCOV, but rather anything where the compiler provides a form of runtime.
Anyway, I'll try your hack and see if I run into any more issues.
Cheers,
Will
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 8 lines · Will Deacon
Quoted 2 lines · Mark Brown
Right, I'm seeing it here now - it's when
CONFIG_GCOV_KERNELis enabled and happens for clang-10 as well but not a GCC 10 prerelease build.Interesting. Is that because GCC doesn't emit out-of-line GCOV functions, or does it emit PAC/BTI instructions for them instead? (you can disassemble one of the problematic opjects to have a look).
GCC does emit some helper functions wrapping GCOV stuff but they have appropriate annotations, eg:
00000000000000ac <_sub_D_00100_1>:
ac: d503245f bti c
b0: a9bf7bfd stp x29, x30, [sp, #-16]!
b4: 910003fd mov x29, sp
b8: 94000000 bl 0 <__gcov_exit>
bc: a8c17bfd ldp x29, x30, [sp], #16
c0: d65f03c0 ret
I can also reproduce this for clang with a trivial standalone source file and -fprofile-arcs -mbranch-protection=bti so it's nothing funky the kernel is doing as far as I can see.
Quoted 3 lines
I can't immediately see how to fix it, so your hack above might be the best bet for now. I'm just a little wary that it might not be limited to GCOV, but rather anything where the compiler provides a form of runtime.
Indeed. I guess the nice thing with BTI is that if something goes wrong it will do so rather visibly so unless there are situations where the toolchain emits rarely called functions the problems will tend to be very obvious, and it seems that clang is detecting the problem itself and complaining loudly which makes it even more likely that if something else is affected it'll be noticed and we can at least add similar bodges.
It does seem it's a straight compiler issue, if the compiler is emitting runtime then the compiler ought to be ensuring that it agrees with the build options the compiler was given and I can't think how this would be fixable or avoidable outside of the compiler other than "don't do that" which is what my Kconfig bodge did. I'm talking to the toolchain people internally about this.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Will Deacon
maintainerQuoted 13 lines · Mark Brown
Quoted 8 lines · Will Deacon
Quoted 2 lines · Mark Brown
Right, I'm seeing it here now - it's when
CONFIG_GCOV_KERNELis enabled and happens for clang-10 as well but not a GCC 10 prerelease build.Interesting. Is that because GCC doesn't emit out-of-line GCOV functions, or does it emit PAC/BTI instructions for them instead? (you can disassemble one of the problematic opjects to have a look).
GCC does emit some helper functions wrapping GCOV stuff but they have appropriate annotations, eg:
Quoted 7 lines
00000000000000ac<_sub_D_00100_1>:
ac:d503245fbti c
b0:a9bf7bfdstp x29, x30, [sp, #-16]!
b4:910003fdmov x29, sp
b8: 94000000 bl 0 <__gcov_exit>
bc:a8c17bfdldp x29, x30, [sp], #16
c0:d65f03c0ret
Hmm, where have the PAC/AUT instructions gone?
Quoted 3 lines
I can also reproduce this for clang with a trivial standalone source file and -fprofile-arcs -mbranch-protection=bti so it's nothing funky the kernel is doing as far as I can see.
Good.
Quoted 12 lines
Quoted 3 lines
I can't immediately see how to fix it, so your hack above might be the best bet for now. I'm just a little wary that it might not be limited to GCOV, but rather anything where the compiler provides a form of runtime.
Indeed. I guess the nice thing with BTI is that if something goes wrong it will do so rather visibly so unless there are situations where the toolchain emits rarely called functions the problems will tend to be very obvious, and it seems that clang is detecting the problem itself and complaining loudly which makes it even more likely that if something else is affected it'll be noticed and we can at least add similar bodges.
Quoted 6 lines
It does seem it's a straight compiler issue, if the compiler is emitting runtime then the compiler ought to be ensuring that it agrees with the build options the compiler was given and I can't think how this would be fixable or avoidable outside of the compiler other than "don't do that" which is what my
Kconfigbodge did. I'm talking to the toolchain people internally about this.
Thanks. I'll apply your 'depends on ...' line locally and push that out if I don't run into any more issues.
Will
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 9 lines · Will Deacon
Quoted 5 lines · Mark Brown
GCC does emit some helper functions wrapping GCOV stuff but they have appropriate annotations, eg:
00000000000000ac<_sub_D_00100_1>:
ac:d503245fbti cHmm, where have the PAC/AUT instructions gone?
I was testing with -mbranch-protection=bti while trying to narrow down the issue when I pasted that example in, if PAC is enabled then you get the PAC/AUT instructions instead.
Quoted 9 lines
Quoted 6 lines
It does seem it's a straight compiler issue, if the compiler is emitting runtime then the compiler ought to be ensuring that it agrees with the build options the compiler was given and I can't think how this would be fixable or avoidable outside of the compiler other than "don't do that" which is what my
Kconfigbodge did. I'm talking to the toolchain people internally about this.Thanks. I'll apply your 'depends on ...' line locally and push that out if I don't run into any more issues.
Thanks, hopefully it'll be fine.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 11 lines · Will Deacon
Quoted 6 lines · Mark Brown
It does seem it's a straight compiler issue, if the compiler is emitting runtime then the compiler ought to be ensuring that it agrees with the build options the compiler was given and I can't think how this would be fixable or avoidable outside of the compiler other than "don't do that" which is what my
Kconfigbodge did. I'm talking to the toolchain people internally about this.Thanks. I'll apply your 'depends on ...' line locally and push that out if I don't run into any more issues.
Thanks. The issue should be fixed by clang upstream with:
https://reviews.llvm.org/D75181
Once that is sorted out and lands I'll send followup patches opening up the dependencies to match.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 11 lines · Will Deacon
Quoted 9 lines · Will Deacon
Bugger, I'm still getting warnings (clang 11.0.1), but from an allmodconfig build now:
warning: some functions compiled with BTI and some compiled without BTI
warning: not setting BTI in feature flags(repeated many, many times).
I'll try to get you some more info.
Where are you getting this clang from? When I test using clang 11 from the LLVM apt repos right now it seems fine, and clang 11 doesn't seem to have been released yet so I'm guessing this is some kind of snapshot.
Quoted 2 lines
Quick look at the log suggests that these are caused by HDRTEST, whatever that is.
AFAICT that's something that's supposed to be checking for include guards and not doing anything fancy... I can't think what could cause this on a per-function basis.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Will Deacon
maintainerQuoted 9 lines · Mark Brown
Quoted 7 lines · Will Deacon
Quoted 5 lines · Will Deacon
Bugger, I'm still getting warnings (clang 11.0.1), but from an allmodconfig build now:
warning: some functions compiled with BTI and some compiled without BTI
warning: not setting BTI in feature flags
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
(repeated many, many times).
I'll try to get you some more info.
Where are you getting this clang from? When I test using clang 11 from the LLVM apt repos right now it seems fine, and clang 11 doesn't seem to have been released yet so I'm guessing this is some kind of snapshot.
I'm using a build of:
https://android.googlesource.com/toolchain/llvm-project/+/b397f81060ce6d701042b782172ed13bee898b79
Quoted 6 lines
Quoted 2 lines
Quick look at the log suggests that these are caused by HDRTEST, whatever that is.
AFAICT that's something that's supposed to be checking for include guards and not doing anything fancy... I can't think what could cause this on a per-function basis.
Indeed, sorry for jumping the gun on that diagnosis.
Will
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 10 lines · Mark Brown
This patch series adds support for protecting the kernel and vDSO with BTI including code compiled with the BPF JIT at runtime.
We build the kernel with annotations for BTI and then map the kernel with GP based on the support on the boot CPU, rejecting secondaries that don't have BTI support. If there is a need to handle big.LITTLE systems with mismatched BTI support we will have to revisit this, currently no such implementations exist.
[...]
Applied to arm64 (for-next/bti), thanks!
[01/11] arm64: Document why we enable PAC support for leaf functions
https://git.kernel.org/arm64/c/717b938e22f8 [02/11] arm64: bti: Support building kernel C code using BTI https://git.kernel.org/arm64/c/92e2294d870b [03/11] arm64: asm: Override SYM_FUNC_START when building the kernel with BTI https://git.kernel.org/arm64/c/714a8d02ca4d [04/11] arm64: Set GP bit in kernel page tables to enable BTI for the kernel https://git.kernel.org/arm64/c/c8027285e366 [05/11] arm64: bpf: Annotate JITed code for BTI https://git.kernel.org/arm64/c/fa76cfe65c1d [06/11] arm64: mm: Mark executable text as guarded pages https://git.kernel.org/arm64/c/67d4a1cd0976 [07/11] arm64: bti: Provide Kconfig for kernel mode BTI https://git.kernel.org/arm64/c/97fed779f2a6 [08/11] arm64: asm: Provide a mechanism for generating ELF note for BTI https://git.kernel.org/arm64/c/3a9b136c998f [09/11] arm64: vdso: Annotate for BTI https://git.kernel.org/arm64/c/a6aadc28278a [10/11] arm64: vdso: Force the vDSO to be linked as BTI when built for BTI https://git.kernel.org/arm64/c/5e02a1887fce [11/11] arm64: vdso: Map the vDSO text with guarded pages when built for BTI https://git.kernel.org/arm64/c/bf740a905ffe
Cheers,
Signature · 10 lines
Will https://fixes.arm64.dev https://next.arm64.dev https://will.arm64.dev _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel