32bit ARM branch predictor hardening
v3 cover 32bit ARM branch predictor hardening
This small series implements some basic BP hardening by invalidating the BTB on 32bit ARM CPUs that are known to be susceptible to aliasing attacks (Spectre variant 2). It doesn't help non-ARM 32bit CPUs, nor 32bit kernels that run on 64bit capable CPUs. This series doesn't mitigate Spectre variant 1 either.
These patches are closely modelled against what we do on arm64, although simpler as we can rely on an architected instruction to perform the invalidation. The notable exception is Cortex-A15, where BTB invalidation behaves like a NOP, and the only way to shoot the predictor down is to invalidate the icache *and* to have ACTLR[0] set to 1 (which is a secure-only operation).
The first patch reuses the Cortex-A8 BTB invalidation in switch_mm and generalises it to be used on all affected CPUs. The second perform the same invalidation on prefetch abort outside of the userspace range. The third one nukes it on guest exit, and results in some major surgery as we cannot take a branch from the vectors (that, and Thumb2 being a massive pain).
Patches 4 to 6 are doing a similar thing for Cortex-A15, which the aforementioned ICIALLU.
To sum up the requirements:
- Both Cortex-A8 and Cortex-A15 need to have ACTLR.IBE (bit 0) set to 1 from secure mode. For Cortex-A8, this overlaps with ARM_ERRATA_430973 which also requires it.
- Cortex-A9, A12 and A17 do not require any extra configuration.
Note 1: Contrary to the initial version, this new series relies on the arm64/kpti branch (I reuse the per-CPU vector hook for KVM).
Note 2: M-class CPUs are not affected and for R-class cores, the mitigation doesn't make much sense since we do not enforce user/kernel isolation.
* From v2:
- Fixed !MMU build
- Small KVM optimisation (suggested by Robin)
- Fixed register zeroing in cpu_v7_btbinv_switch_mm (noticed by Andre)
* From v1:
- Fixed broken hyp_fiq vector (noticed by Ard)
- Fixed broken BTB invalidation in LPAE switch_mm (reported by Andre)
- Revamped invalidation on PABT (noticed by James on arm64, suggested by Will)
- Rewrote the whole HYP sequence, as Thumb2 was pretty unhappy about arithmetic with the stack pointer
Marc Zyngier (6):
arm: Add BTB invalidation on switch_mm for Cortex-A9, A12 and A17
arm: Invalidate BTB on prefetch abort outside of user mapping on
Cortex A8, A9, A12 and A17
arm: KVM: Invalidate BTB on guest exit for Cortex-A12/A17
arm: Add icache invalidation on switch_mm for Cortex-A15
arm: Invalidate icache on prefetch abort outside of user mapping on
Cortex-A15arm: KVM: Invalidate icache on guest exit for Cortex-A15
arch/arm/include/asm/cp15.h | 3 ++ arch/arm/include/asm/kvm_asm.h | 2 - arch/arm/include/asm/kvm_mmu.h | 17 ++++++++- arch/arm/kvm/hyp/hyp-entry.S | 85 +++++++++++++++++++++++++++++++++++++++++- arch/arm/mm/fault.c | 29 ++++++++++++++ arch/arm/mm/fsr-2level.c | 4 +- arch/arm/mm/fsr-3level.c | 67 ++++++++++++++++++++++++++++++++- arch/arm/mm/proc-v7-2level.S | 14 ++++++- arch/arm/mm/proc-v7-3level.S | 22 +++++++++++ arch/arm/mm/proc-v7.S | 48 ++++++++++++++++-------- 10 files changed, 265 insertions(+), 26 deletions(-)
Signature · 1 line
2.14.2
PATCH v3 1/6 arm: Add BTB invalidation on switch_mm for Cortex-A9, A12 and A17
In order to avoid aliasing attacks against the branch predictor, some implementations require to invalidate the BTB when switching from one user context to another.
For this, we reuse the existing implementation for Cortex-A8, and apply it to A9, A12 and A17.
--- arch/arm/mm/proc-v7-2level.S | 4 ++-- arch/arm/mm/proc-v7-3level.S | 6 ++++++ arch/arm/mm/proc-v7.S | 30 +++++++++++++++--------------- 3 files changed, 23 insertions(+), 17 deletions(-)
arch/arm/mm/proc-v7-2level.S +2 -2
@@ -41,7 +41,7 @@ * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. */ -ENTRY(cpu_ca8_switch_mm) +ENTRY(cpu_v7_btbinv_switch_mm) #ifdef CONFIG_MMU mov r2, #0 mcr p15, 0, r2, c7, c5, 6 @ flush BTAC/BTB
@@ -66,7 +66,7 @@ ENTRY(cpu_v7_switch_mm) #endif bx lr ENDPROC(cpu_v7_switch_mm) -ENDPROC(cpu_ca8_switch_mm) +ENDPROC(cpu_v7_btbinv_switch_mm) /* * cpu_v7_set_pte_ext(ptep, pte)
arch/arm/mm/proc-v7-3level.S +6 -0
@@ -54,6 +54,11 @@ * Set the translation table base pointer to be pgd_phys (physical address of * the new TTB). */ +ENTRY(cpu_v7_btbinv_switch_mm) +#ifdef CONFIG_MMU + mov r2, #0 + mcr p15, 0, r2, c7, c5, 6 @ flush BTAC/BTB +#endif ENTRY(cpu_v7_switch_mm) #ifdef CONFIG_MMU mmid r2, r2
@@ -64,6 +69,7 @@ ENTRY(cpu_v7_switch_mm) #endif ret lr ENDPROC(cpu_v7_switch_mm) +ENDPROC(cpu_v7_btbinv_switch_mm) #ifdef __ARMEB__ #define rl r3
arch/arm/mm/proc-v7.S +15 -15
@@ -159,18 +159,18 @@ ENDPROC(cpu_v7_do_resume) #endif /* - * Cortex-A8 + * Cortex-A8/A12/A17 that require a BTB invalidation on switch_mm */ - globl_equ cpu_ca8_proc_init, cpu_v7_proc_init - globl_equ cpu_ca8_proc_fin, cpu_v7_proc_fin - globl_equ cpu_ca8_reset, cpu_v7_reset - globl_equ cpu_ca8_do_idle, cpu_v7_do_idle - globl_equ cpu_ca8_dcache_clean_area, cpu_v7_dcache_clean_area - globl_equ cpu_ca8_set_pte_ext, cpu_v7_set_pte_ext - globl_equ cpu_ca8_suspend_size, cpu_v7_suspend_size + globl_equ cpu_v7_btbinv_proc_init, cpu_v7_proc_init + globl_equ cpu_v7_btbinv_proc_fin, cpu_v7_proc_fin + globl_equ cpu_v7_btbinv_reset, cpu_v7_reset + globl_equ cpu_v7_btbinv_do_idle, cpu_v7_do_idle + globl_equ cpu_v7_btbinv_dcache_clean_area, cpu_v7_dcache_clean_area + globl_equ cpu_v7_btbinv_set_pte_ext, cpu_v7_set_pte_ext + globl_equ cpu_v7_btbinv_suspend_size, cpu_v7_suspend_size #ifdef CONFIG_ARM_CPU_SUSPEND - globl_equ cpu_ca8_do_suspend, cpu_v7_do_suspend - globl_equ cpu_ca8_do_resume, cpu_v7_do_resume + globl_equ cpu_v7_btbinv_do_suspend, cpu_v7_do_suspend + globl_equ cpu_v7_btbinv_do_resume, cpu_v7_do_resume #endif /*
@@ -181,7 +181,7 @@ ENDPROC(cpu_v7_do_resume) globl_equ cpu_ca9mp_reset, cpu_v7_reset globl_equ cpu_ca9mp_do_idle, cpu_v7_do_idle globl_equ cpu_ca9mp_dcache_clean_area, cpu_v7_dcache_clean_area - globl_equ cpu_ca9mp_switch_mm, cpu_v7_switch_mm + globl_equ cpu_ca9mp_switch_mm, cpu_v7_btbinv_switch_mm globl_equ cpu_ca9mp_set_pte_ext, cpu_v7_set_pte_ext .globl cpu_ca9mp_suspend_size .equ cpu_ca9mp_suspend_size, cpu_v7_suspend_size + 4 * 2
@@ -548,8 +548,8 @@ __v7_setup_stack: @ define struct processor (see <asm/proc-fns.h> and proc-macros.S) define_processor_functions v7, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 + define_processor_functions v7_btbinv, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 #ifndef CONFIG_ARM_LPAE - define_processor_functions ca8, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 define_processor_functions ca9mp, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 #endif #ifdef CONFIG_CPU_PJ4B
@@ -614,7 +614,7 @@ __v7_ca9mp_proc_info: __v7_ca8_proc_info: .long 0x410fc080 .long 0xff0ffff0 - __v7_proc __v7_ca8_proc_info, __v7_setup, proc_fns = ca8_processor_functions + __v7_proc __v7_ca8_proc_info, __v7_setup, proc_fns = v7_btbinv_processor_functions .size __v7_ca8_proc_info, . - __v7_ca8_proc_info #endif /* CONFIG_ARM_LPAE */
@@ -658,7 +658,7 @@ __v7_ca7mp_proc_info: __v7_ca12mp_proc_info: .long 0x410fc0d0 .long 0xff0ffff0 - __v7_proc __v7_ca12mp_proc_info, __v7_ca12mp_setup + __v7_proc __v7_ca12mp_proc_info, __v7_ca12mp_setup, proc_fns = v7_btbinv_processor_functions .size __v7_ca12mp_proc_info, . - __v7_ca12mp_proc_info /*
@@ -688,7 +688,7 @@ __v7_b15mp_proc_info: __v7_ca17mp_proc_info: .long 0x410fc0e0 .long 0xff0ffff0 - __v7_proc __v7_ca17mp_proc_info, __v7_ca17mp_setup + __v7_proc __v7_ca17mp_proc_info, __v7_ca17mp_setup, proc_fns = v7_btbinv_processor_functions .size __v7_ca17mp_proc_info, . - __v7_ca17mp_proc_info /*
Signature · 1 line
2.14.2
PATCH v3 1/6 arm: Add BTB invalidation on switch_mm for Cortex-A9, A12 and A17
Quoted 6 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, some implementations require to invalidate the BTB when switching from one user context to another.
For this, we reuse the existing implementation for Cortex-A8, and apply it to A9, A12 and A17.
Should this read: and apply it to A8, A9, A12 and A17
Quoted 59 lines
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm/mm/proc-v7-2level.S | 4 ++-- arch/arm/mm/proc-v7-3level.S | 6 ++++++ arch/arm/mm/proc-v7.S | 30 +++++++++++++++--------------- 3 files changed, 23 insertions(+), 17 deletions(-)arch/arm/mm/proc-v7-2level.S +2
-2@@ -41,7 +41,7 @@ * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. */ -ENTRY(cpu_ca8_switch_mm) +ENTRY(cpu_v7_btbinv_switch_mm) #ifdef CONFIG_MMU mov r2, #0 mcr p15, 0, r2, c7, c5, 6 @ flush BTAC/BTB@@ -66,7 +66,7 @@ ENTRY(cpu_v7_switch_mm) #endif bx lr ENDPROC(cpu_v7_switch_mm) -ENDPROC(cpu_ca8_switch_mm) +ENDPROC(cpu_v7_btbinv_switch_mm) /* * cpu_v7_set_pte_ext(ptep, pte)arch/arm/mm/proc-v7-3level.S +6
-0@@ -54,6 +54,11 @@ * Set the translation table base pointer to be pgd_phys (physical address of * the new TTB). */ +ENTRY(cpu_v7_btbinv_switch_mm) +#ifdef CONFIG_MMU + mov r2, #0 + mcr p15, 0, r2, c7, c5, 6 @ flush BTAC/BTB +#endif ENTRY(cpu_v7_switch_mm) #ifdef CONFIG_MMU mmid r2, r2@@ -64,6 +69,7 @@ ENTRY(cpu_v7_switch_mm) #endif ret lr ENDPROC(cpu_v7_switch_mm) +ENDPROC(cpu_v7_btbinv_switch_mm) #ifdef __ARMEB__ #define rl r3arch/arm/mm/proc-v7.S +0
-0
arch/arm/mm/proc-v7.S ENDPROC(cpu_v7_do_resume) · line 162
Quoted 5 lines
arch/arm/mm/proc-v7.S +1
-1@@ -159,18 +159,18 @@ ENDPROC(cpu_v7_do_resume) #endif /* - * Cortex-A8 + * Cortex-A8/A12/A17 that require a BTB invalidation on switch_mm
Should this read Cortex-A8/A9/A12/A17?
Signature · 1 line
Florian
PATCH v3 1/6 arm: Add BTB invalidation on switch_mm for Cortex-A9, A12 and A17
Quoted 9 lines · Florian Fainelli
Quoted 6 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, some implementations require to invalidate the BTB when switching from one user context to another.
For this, we reuse the existing implementation for Cortex-A8, and apply it to A9, A12 and A17.
Should this read: and apply it to A8, A9, A12 and A17
A8 already has a BTB invalidation there. We are expanding the use of that function to other v7 cores. Or am I missing something obvious?
Quoted 62 lines
Quoted 62 lines
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm/mm/proc-v7-2level.S | 4 ++-- arch/arm/mm/proc-v7-3level.S | 6 ++++++ arch/arm/mm/proc-v7.S | 30 +++++++++++++++--------------- 3 files changed, 23 insertions(+), 17 deletions(-)arch/arm/mm/proc-v7-2level.S +2
-2@@ -41,7 +41,7 @@ * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. */ -ENTRY(cpu_ca8_switch_mm) +ENTRY(cpu_v7_btbinv_switch_mm) #ifdef CONFIG_MMU mov r2, #0 mcr p15, 0, r2, c7, c5, 6 @ flush BTAC/BTB@@ -66,7 +66,7 @@ ENTRY(cpu_v7_switch_mm) #endif bx lr ENDPROC(cpu_v7_switch_mm) -ENDPROC(cpu_ca8_switch_mm) +ENDPROC(cpu_v7_btbinv_switch_mm) /* * cpu_v7_set_pte_ext(ptep, pte)arch/arm/mm/proc-v7-3level.S +6
-0@@ -54,6 +54,11 @@ * Set the translation table base pointer to be pgd_phys (physical address of * the new TTB). */ +ENTRY(cpu_v7_btbinv_switch_mm) +#ifdef CONFIG_MMU + mov r2, #0 + mcr p15, 0, r2, c7, c5, 6 @ flush BTAC/BTB +#endif ENTRY(cpu_v7_switch_mm) #ifdef CONFIG_MMU mmid r2, r2@@ -64,6 +69,7 @@ ENTRY(cpu_v7_switch_mm) #endif ret lr ENDPROC(cpu_v7_switch_mm) +ENDPROC(cpu_v7_btbinv_switch_mm) #ifdef __ARMEB__ #define rl r3arch/arm/mm/proc-v7.S +0
-0@@ -159,18 +159,18 @@ ENDPROC(cpu_v7_do_resume) #endif
Quoted 5 lines
arch/arm/mm/proc-v7.S ENDPROC(cpu_v7_do_resume) · line 162
Quoted 3 lines
arch/arm/mm/proc-v7.S +1
-1/* - * Cortex-A8 + * Cortex-A8/A12/A17 that require a BTB invalidation on switch_mmShould this read Cortex-A8/A9/A12/A17?
Indeed, I missed our dear A9 friend.
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
PATCH v3 2/6 arm: Invalidate BTB on prefetch abort outside of user mapping on Cortex A8, A9, A12 and A17
In reply to Marc ZyngierIn order to prevent aliasing attacks on the branch predictor, invalidate the BTB on CPUs that are known to be affected when taking a prefetch abort on a address that is outside of a user task limit.
--- arch/arm/include/asm/cp15.h | 2 ++ arch/arm/mm/fault.c | 25 +++++++++++++++++ arch/arm/mm/fsr-2level.c | 4 +-- arch/arm/mm/fsr-3level.c | 67 ++++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 95 insertions(+), 3 deletions(-)
arch/arm/include/asm/cp15.h +2 -0
@@ -65,6 +65,8 @@#define __write_sysreg (v, r, w, c, t) asm volatile(w " " c : : "r" ((t)(v)))#define write_sysreg (v, ...) __write_sysreg(v, __VA_ARGS__) +#define BPIALL __ACCESS_CP15(c7, 0, c5, 6) +extern unsigned long cr_alignment;/* defined in entry-armv.S */ static inline unsigned long get_cr (void )
arch/arm/mm/fault.c +25 -0
@@ -21,6 +21,7 @@#include <linux/highmem.h> #include <linux/perf_event.h> +#include <asm/cp15.h> #include <asm/exception.h> #include <asm/pgtable.h> #include <asm/system_misc.h>
@@ -181,6 +182,7 @@ __do_user_fault(struct task_struct *tsk, unsigned long addr, si.si_errno =0 ; si.si_code = code; si.si_addr = (void __user *)addr; +force_sig_info (sig, &si, tsk); }
@@ -396,12 +398,35 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs)__do_kernel_fault (mm, addr, fsr, regs); return 0; } + +static int +do_pabt_page_fault (unsigned long addr,unsigned int fsr,struct pt_regs *regs) +{ +if (addr >TASK_SIZE ) { +switch (read_cpuid_part ()) { +case ARM_CPU_PART_CORTEX_A8 : +case ARM_CPU_PART_CORTEX_A9 : +case ARM_CPU_PART_CORTEX_A12 : +case ARM_CPU_PART_CORTEX_A17 : +write_sysreg (0 ,BPIALL ); +break ; + } + } + +return do_page_fault (addr, fsr, regs); +}#else /* CONFIG_MMU */ static int do_page_fault (unsigned long addr,unsigned int fsr,struct pt_regs *regs) {return 0 ; } + +static int +do_pabt_page_fault (unsigned long addr,unsigned int fsr,struct pt_regs *regs) +{ +return 0 ; +}#endif /* CONFIG_MMU */ /*
arch/arm/mm/fsr-2level.c +2 -2
@@ -51,7 +51,7 @@ static struct fsr_info ifsr_info[] = { { do_bad, SIGBUS, 0, "unknown 4" }, { do_translation_fault, SIGSEGV, SEGV_MAPERR, "section translation fault" }, { do_bad, SIGSEGV, SEGV_ACCERR, "page access flag fault" }, - { do_page_fault, SIGSEGV, SEGV_MAPERR, "page translation fault" }, + { do_pabt_page_fault, SIGSEGV, SEGV_MAPERR, "page translation fault" }, { do_bad, SIGBUS, 0, "external abort on non-linefetch" }, { do_bad, SIGSEGV, SEGV_ACCERR, "section domain fault" }, { do_bad, SIGBUS, 0, "unknown 10" },
@@ -59,7 +59,7 @@ static struct fsr_info ifsr_info[] = { { do_bad, SIGBUS, 0, "external abort on translation" }, { do_sect_fault, SIGSEGV, SEGV_ACCERR, "section permission fault" }, { do_bad, SIGBUS, 0, "external abort on translation" }, - { do_page_fault, SIGSEGV, SEGV_ACCERR, "page permission fault" }, + { do_pabt_page_fault, SIGSEGV, SEGV_ACCERR, "page permission fault" }, { do_bad, SIGBUS, 0, "unknown 16" }, { do_bad, SIGBUS, 0, "unknown 17" }, { do_bad, SIGBUS, 0, "unknown 18" },
arch/arm/mm/fsr-3level.c +66 -1
70 lines not on this page · raw message
Signature · 1 line
2.14.2
PATCH v3 2/6 arm: Invalidate BTB on prefetch abort outside of user mapping on Cortex A8, A9, A12 and A17
Hi Marc,
Quoted 11 lines · Marc Zyngier
arch/arm/mm/fault.c +1
-0@@ -21,6 +21,7 @@#include <linux/highmem.h> #include <linux/perf_event.h> +#include <asm/cp15.h> #include <asm/exception.h> #include <asm/pgtable.h> #include <asm/system_misc.h>
arch/arm/mm/fault.c __do_user_fault(struct task_struct *tsk, unsigned long addr, · line 185
Quoted 4 lines
arch/arm/mm/fault.c +1
-0@@ -181,6 +182,7 @@ __do_user_fault(struct task_struct *tsk, unsigned long addr, si.si_errno =0 ; si.si_code = code; si.si_addr = (void __user *)addr; +
Unneeded blank line.
arch/arm/mm/fault.c do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) · line 406
Quoted 13 lines
arch/arm/mm/fault.c +6
-0force_sig_info (sig, &si, tsk); }@@ -396,12 +398,35 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs)__do_kernel_fault (mm, addr, fsr, regs);return 0 ; } + +static int +do_pabt_page_fault (unsigned long addr,unsigned int fsr,struct pt_regs *regs) +{ +if (addr >TASK_SIZE ) { +switch (read_cpuid_part ()) {
checkpatch complains here:
ERROR: space required before the open parenthesis '(' #62: FILE: arch/arm/mm/fault.c:406:
+ switch(read_cpuid_part()) {
PATCH v3 3/6 arm: KVM: Invalidate BTB on guest exit for Cortex-A12/A17
In reply to Marc ZyngierIn order to avoid aliasing attacks against the branch predictor, let's invalidate the BTB on guest exit. This is made complicated by the fact that we cannot take a branch before invalidating the BTB.
We only apply this to A12 and A17, which are the only two ARM cores on which this useful.
--- arch/arm/include/asm/kvm_asm.h | 2 -- arch/arm/include/asm/kvm_mmu.h | 13 ++++++++- arch/arm/kvm/hyp/hyp-entry.S | 62 ++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 72 insertions(+), 5 deletions(-)
arch/arm/include/asm/kvm_asm.h +0 -2
@@ -61,8 +61,6 @@ struct kvm_vcpu;extern char __kvm_hyp_init[];extern char __kvm_hyp_init_end[]; -extern char __kvm_hyp_vector[]; -extern void __kvm_flush_vm_context (void );extern void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa);extern void __kvm_tlb_flush_vmid (struct kvm *kvm);
arch/arm/include/asm/kvm_mmu.h +12 -1
@@ -37,6 +37,7 @@#include <linux/highmem.h> #include <asm/cacheflush.h> +#include <asm/cputype.h> #include <asm/pgalloc.h> #include <asm/stage2_pgtable.h>
@@ -223,7 +224,17 @@ static inline unsigned int kvm_get_vmid_bits(void)static inline void *kvm_get_hyp_vector (void ) { -return kvm_ksym_ref (__kvm_hyp_vector); +extern char __kvm_hyp_vector[]; +extern char __kvm_hyp_vector_bp_inv[]; + +switch (read_cpuid_part ()) { +case ARM_CPU_PART_CORTEX_A12 : +case ARM_CPU_PART_CORTEX_A17 : +return kvm_ksym_ref (__kvm_hyp_vector_bp_inv); + +default : +return kvm_ksym_ref (__kvm_hyp_vector); + } }static inline int kvm_map_vectors (void )
arch/arm/kvm/hyp/hyp-entry.S +60 -2
80 lines not on this page · raw message
Signature · 1 line
2.14.2
PATCH v3 3/6 arm: KVM: Invalidate BTB on guest exit for Cortex-A12/A17
Quoted 64 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, let's invalidate the BTB on guest exit. This is made complicated by the fact that we cannot take a branch before invalidating the BTB.
We only apply this to A12 and A17, which are the only two ARM cores on which this useful.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm/include/asm/kvm_asm.h | 2 -- arch/arm/include/asm/kvm_mmu.h | 13 ++++++++- arch/arm/kvm/hyp/hyp-entry.S | 62 ++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 72 insertions(+), 5 deletions(-)arch/arm/include/asm/kvm_asm.h +0
-2@@ -61,8 +61,6 @@ struct kvm_vcpu;extern char __kvm_hyp_init[];extern char __kvm_hyp_init_end[]; -extern char __kvm_hyp_vector[]; -extern void __kvm_flush_vm_context (void );extern void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa);extern void __kvm_tlb_flush_vmid (struct kvm *kvm);arch/arm/include/asm/kvm_mmu.h +12
-1@@ -37,6 +37,7 @@#include <linux/highmem.h> #include <asm/cacheflush.h> +#include <asm/cputype.h> #include <asm/pgalloc.h> #include <asm/stage2_pgtable.h> @@ -223,7 +224,17 @@ static inline unsigned int kvm_get_vmid_bits(void)static inline void *kvm_get_hyp_vector (void ) { -return kvm_ksym_ref (__kvm_hyp_vector); +extern char __kvm_hyp_vector[]; +extern char __kvm_hyp_vector_bp_inv[]; + +switch (read_cpuid_part ()) { +case ARM_CPU_PART_CORTEX_A12 : +case ARM_CPU_PART_CORTEX_A17 : +return kvm_ksym_ref (__kvm_hyp_vector_bp_inv); + +default : +return kvm_ksym_ref (__kvm_hyp_vector); + } }static inline int kvm_map_vectors (void )arch/arm/kvm/hyp/hyp-entry.S +0
-0
arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 73
Quoted 4 lines
arch/arm/kvm/hyp/hyp-entry.S +1
-0@@ -70,6 +70,57 @@ __kvm_hyp_vector: W(b) hyp_hvc W(b) hyp_irq W(b) hyp_fiq +
nit: trailing white space
Quoted 81 lines
arch/arm/kvm/hyp/hyp-entry.S +59
-2+ .align 5 +__kvm_hyp_vector_bp_inv: + .global __kvm_hyp_vector_bp_inv + + /* + * We encode the exception entry in the bottom 3 bits of + * SP, and we have to guarantee to be 8 bytes aligned. + */ + W(add) sp, sp, #1 /* Reset 7 */ + W(add) sp, sp, #1 /* Undef 6 */ + W(add) sp, sp, #1 /* Syscall 5 */ + W(add) sp, sp, #1 /* Prefetch abort 4 */ + W(add) sp, sp, #1 /* Data abort 3 */ + W(add) sp, sp, #1 /* HVC 2 */ + W(add) sp, sp, #1 /* IRQ 1 */ + W(nop) /* FIQ 0 */ + + mcr p15, 0, r0, c7, c5, 6 /* BPIALL */ + isb + + /* + * Yet another silly hack: Use VPIDR as a temp register. + * Thumb2 is really a pain, as SP cannot be used with most + * of the bitwise instructions. The vect_br macro ensures + * things gets cleaned-up. + */ + mcr p15, 4, r0, c0, c0, 0 /* VPIDR */ + mov r0, sp + and r0, r0, #7 + sub sp, sp, r0 + push {r1, r2} + mov r1, r0 + mrc p15, 4, r0, c0, c0, 0 /* VPIDR */ + mrc p15, 0, r2, c0, c0, 0 /* MIDR */ + mcr p15, 4, r2, c0, c0, 0 /* VPIDR */ + +.macro vect_br val, targ + cmp r1, #\val + popeq {r1, r2} + beq \targ +.endm + + vect_br 0, hyp_fiq + vect_br 1, hyp_irq + vect_br 2, hyp_hvc + vect_br 3, hyp_dabt + vect_br 4, hyp_pabt + vect_br 5, hyp_svc + vect_br 6, hyp_undef + vect_br 7, hyp_reset .macro invalid_vector label, cause .align@@ -149,7 +200,14 @@ hyp_hvc: bx ip 1: - push {lr} + /* + * Pushing r2 here is just a way of keeping the stack aligned to + * 8 bytes on any path that can trigger a HYP exception. Here, + * we may well be about to jump into the guest, and the guest + * exit would otherwise be badly decoded by our fancy + * "decode-exception-without-a-branch" code... + */ + push {r2, lr} mov lr, r0 mov r0, r1@@ -159,7 +217,7 @@ hyp_hvc: THUMB( orr lr, #1) blx lr @ Call the HYP function - pop {lr} + pop {r2, lr} eret guest_trap:Signature · 1 line
2.14.2
PATCH v3 3/6 arm: KVM: Invalidate BTB on guest exit for Cortex-A12/A17
In reply to Marc ZyngierQuoted 78 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, let's invalidate the BTB on guest exit. This is made complicated by the fact that we cannot take a branch before invalidating the BTB.
We only apply this to A12 and A17, which are the only two ARM cores on which this useful.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm/include/asm/kvm_asm.h | 2 -- arch/arm/include/asm/kvm_mmu.h | 13 ++++++++- arch/arm/kvm/hyp/hyp-entry.S | 62 ++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 72 insertions(+), 5 deletions(-)arch/arm/include/asm/kvm_asm.h +0
-2@@ -61,8 +61,6 @@ struct kvm_vcpu;extern char __kvm_hyp_init[];extern char __kvm_hyp_init_end[]; -extern char __kvm_hyp_vector[]; -extern void __kvm_flush_vm_context (void );extern void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa);extern void __kvm_tlb_flush_vmid (struct kvm *kvm);arch/arm/include/asm/kvm_mmu.h +12
-1@@ -37,6 +37,7 @@#include <linux/highmem.h> #include <asm/cacheflush.h> +#include <asm/cputype.h> #include <asm/pgalloc.h> #include <asm/stage2_pgtable.h> @@ -223,7 +224,17 @@ static inline unsigned int kvm_get_vmid_bits(void)static inline void *kvm_get_hyp_vector (void ) { -return kvm_ksym_ref (__kvm_hyp_vector); +extern char __kvm_hyp_vector[]; +extern char __kvm_hyp_vector_bp_inv[]; + +switch (read_cpuid_part ()) { +case ARM_CPU_PART_CORTEX_A12 : +case ARM_CPU_PART_CORTEX_A17 : +return kvm_ksym_ref (__kvm_hyp_vector_bp_inv); + +default : +return kvm_ksym_ref (__kvm_hyp_vector); + } }static inline int kvm_map_vectors (void )arch/arm/kvm/hyp/hyp-entry.S +11
-0@@ -70,6 +70,57 @@ __kvm_hyp_vector: W(b) hyp_hvc W(b) hyp_irq W(b) hyp_fiq + + .align 5 +__kvm_hyp_vector_bp_inv: + .global __kvm_hyp_vector_bp_inv + + /* + * We encode the exception entry in the bottom 3 bits of + * SP, and we have to guarantee to be 8 bytes aligned. + */ + W(add) sp, sp, #1 /* Reset 7 */ + W(add) sp, sp, #1 /* Undef 6 */
arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 93
Quoted 10 lines
arch/arm/kvm/hyp/hyp-entry.S +10
-0+ W(add) sp, sp, #1 /* Syscall 5 */ + W(add) sp, sp, #1 /* Prefetch abort 4 */ + W(add) sp, sp, #1 /* Data abort 3 */ + W(add) sp, sp, #1 /* HVC 2 */ + W(add) sp, sp, #1 /* IRQ 1 */ + W(nop) /* FIQ 0 */ + + mcr p15, 0, r0, c7, c5, 6 /* BPIALL */ + isb +
The below is quite a bit of faff; might it be worth an
#ifdef CONFIG_THUMB2_KERNEL
Quoted 5 lines
arch/arm/kvm/hyp/hyp-entry.S +5
-0+ /* + * Yet another silly hack: Use VPIDR as a temp register. + * Thumb2 is really a pain, as SP cannot be used with most + * of the bitwise instructions. The vect_br macro ensures + * things gets cleaned-up.
arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 108
Quoted 10 lines
arch/arm/kvm/hyp/hyp-entry.S +10
-0+ */ + mcr p15, 4, r0, c0, c0, 0 /* VPIDR */ + mov r0, sp + and r0, r0, #7 + sub sp, sp, r0 + push {r1, r2} + mov r1, r0 + mrc p15, 4, r0, c0, c0, 0 /* VPIDR */ + mrc p15, 0, r2, c0, c0, 0 /* MIDR */ + mcr p15, 4, r2, c0, c0, 0 /* VPIDR */
#endif
arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 110
Quoted 2 lines
arch/arm/kvm/hyp/hyp-entry.S +2
-0+ +.macro vect_br val, targ
ARM(cmp sp, #val)THUMB(cmp r1, #\val)
THUMB(popeq {r1, r2}
arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 114
Quoted 2 lines
arch/arm/kvm/hyp/hyp-entry.S +2
-0+ beq \targ +.endm
...to keep the "normal" path relatively streamlined?
Robin.
Quoted 38 lines
arch/arm/kvm/hyp/hyp-entry.S +18
-2+ + vect_br 0, hyp_fiq + vect_br 1, hyp_irq + vect_br 2, hyp_hvc + vect_br 3, hyp_dabt + vect_br 4, hyp_pabt + vect_br 5, hyp_svc + vect_br 6, hyp_undef + vect_br 7, hyp_reset .macro invalid_vector label, cause .align@@ -149,7 +200,14 @@ hyp_hvc: bx ip 1: - push {lr} + /* + * Pushing r2 here is just a way of keeping the stack aligned to + * 8 bytes on any path that can trigger a HYP exception. Here, + * we may well be about to jump into the guest, and the guest + * exit would otherwise be badly decoded by our fancy + * "decode-exception-without-a-branch" code... + */ + push {r2, lr} mov lr, r0 mov r0, r1@@ -159,7 +217,7 @@ hyp_hvc: THUMB( orr lr, #1) blx lr @ Call the HYP function - pop {lr} + pop {r2, lr} eret guest_trap:
PATCH v3 3/6 arm: KVM: Invalidate BTB on guest exit for Cortex-A12/A17
Hi Robin,
Quoted 112 lines · Robin Murphy
Quoted 78 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, let's invalidate the BTB on guest exit. This is made complicated by the fact that we cannot take a branch before invalidating the BTB.
We only apply this to A12 and A17, which are the only two ARM cores on which this useful.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm/include/asm/kvm_asm.h | 2 -- arch/arm/include/asm/kvm_mmu.h | 13 ++++++++- arch/arm/kvm/hyp/hyp-entry.S | 62 ++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 72 insertions(+), 5 deletions(-)arch/arm/include/asm/kvm_asm.h +0
-2@@ -61,8 +61,6 @@ struct kvm_vcpu;extern char __kvm_hyp_init[];extern char __kvm_hyp_init_end[]; -extern char __kvm_hyp_vector[]; -extern void __kvm_flush_vm_context (void );extern void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa);extern void __kvm_tlb_flush_vmid (struct kvm *kvm);arch/arm/include/asm/kvm_mmu.h +12
-1@@ -37,6 +37,7 @@#include <linux/highmem.h> #include <asm/cacheflush.h> +#include <asm/cputype.h> #include <asm/pgalloc.h> #include <asm/stage2_pgtable.h> @@ -223,7 +224,17 @@ static inline unsigned int kvm_get_vmid_bits(void)static inline void *kvm_get_hyp_vector (void ) { -return kvm_ksym_ref (__kvm_hyp_vector); +extern char __kvm_hyp_vector[]; +extern char __kvm_hyp_vector_bp_inv[]; + +switch (read_cpuid_part ()) { +case ARM_CPU_PART_CORTEX_A12 : +case ARM_CPU_PART_CORTEX_A17 : +return kvm_ksym_ref (__kvm_hyp_vector_bp_inv); + +default : +return kvm_ksym_ref (__kvm_hyp_vector); + } }static inline int kvm_map_vectors (void )arch/arm/kvm/hyp/hyp-entry.S +11
-0@@ -70,6 +70,57 @@ __kvm_hyp_vector: W(b) hyp_hvc W(b) hyp_irq W(b) hyp_fiq + + .align 5 +__kvm_hyp_vector_bp_inv: + .global __kvm_hyp_vector_bp_inv + + /* + * We encode the exception entry in the bottom 3 bits of + * SP, and we have to guarantee to be 8 bytes aligned. + */ + W(add) sp, sp, #1 /* Reset 7 */ + W(add) sp, sp, #1 /* Undef 6 */arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 93
Quoted 10 lines
arch/arm/kvm/hyp/hyp-entry.S +10
-0+ W(add) sp, sp, #1 /* Syscall 5 */ + W(add) sp, sp, #1 /* Prefetch abort 4 */ + W(add) sp, sp, #1 /* Data abort 3 */ + W(add) sp, sp, #1 /* HVC 2 */ + W(add) sp, sp, #1 /* IRQ 1 */ + W(nop) /* FIQ 0 */ + + mcr p15, 0, r0, c7, c5, 6 /* BPIALL */ + isb +The below is quite a bit of faff; might it be worth an
#ifdef
CONFIG_THUMB2_KERNELQuoted 5 lines
arch/arm/kvm/hyp/hyp-entry.S +5
-0+ /* + * Yet another silly hack: Use VPIDR as a temp register. + * Thumb2 is really a pain, as SP cannot be used with most + * of the bitwise instructions. The vect_br macro ensures + * things gets cleaned-up.arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 108
Quoted 10 lines
arch/arm/kvm/hyp/hyp-entry.S +10
-0+ */ + mcr p15, 4, r0, c0, c0, 0 /* VPIDR */ + mov r0, sp + and r0, r0, #7 + sub sp, sp, r0 + push {r1, r2} + mov r1, r0 + mrc p15, 4, r0, c0, c0, 0 /* VPIDR */ + mrc p15, 0, r2, c0, c0, 0 /* MIDR */ + mcr p15, 4, r2, c0, c0, 0 /* VPIDR */#endif
Quoted 4 lines
arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 110
Quoted 2 lines
arch/arm/kvm/hyp/hyp-entry.S +2
-0+ +.macro vect_br val, targ
ARM(cmp sp, #val)
Doesn't quite work, as we still have all the top bits that contain the stack address. But I like the idea of making it baster for non-T2. How about this instead?
arch/arm/kvm/hyp/hyp-entry.S +10 -2
@@ -114,6 +114,8 @@ __kvm_hyp_vector_bp_inv: isb decode_vectors: + +#ifdef CONFIG_THUMB2_KERNEL /* * Yet another silly hack: Use VPIDR as a temp register. * Thumb2 is really a pain, as SP cannot be used with most
@@ -129,10 +131,16 @@ decode_vectors: mrc p15, 4, r0, c0, c0, 0 /* VPIDR */ mrc p15, 0, r2, c0, c0, 0 /* MIDR */ mcr p15, 4, r2, c0, c0, 0 /* VPIDR */ +#endif .macro vect_br val, targ - cmp r1, #\val - popeq {r1, r2} +ARM( eor sp, sp, #\val ) +ARM( tst sp, #7 ) +ARM( eorne sp, sp, #\val ) + +THUMB( cmp r1, #\val ) +THUMB( popeq {r1, r2} ) + beq \targ .endm
Quoted 7 lines
THUMB(cmp r1, #\val)
THUMB(popeq {r1, r2}arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 112
Quoted 2 lines
arch/arm/kvm/hyp/hyp-entry.S +2
-0+ beq \targ +.endm...to keep the "normal" path relatively streamlined?
I think the above achieves it... Thoughts?
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
PATCH v3 3/6 arm: KVM: Invalidate BTB on guest exit for Cortex-A12/A17
Quoted 115 lines · Marc Zyngier
Hi Robin,
Quoted 112 lines · Robin Murphy
Quoted 78 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, let's invalidate the BTB on guest exit. This is made complicated by the fact that we cannot take a branch before invalidating the BTB.
We only apply this to A12 and A17, which are the only two ARM cores on which this useful.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm/include/asm/kvm_asm.h | 2 -- arch/arm/include/asm/kvm_mmu.h | 13 ++++++++- arch/arm/kvm/hyp/hyp-entry.S | 62 ++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 72 insertions(+), 5 deletions(-)arch/arm/include/asm/kvm_asm.h +0
-2@@ -61,8 +61,6 @@ struct kvm_vcpu;extern char __kvm_hyp_init[];extern char __kvm_hyp_init_end[]; -extern char __kvm_hyp_vector[]; -extern void __kvm_flush_vm_context (void );extern void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa);extern void __kvm_tlb_flush_vmid (struct kvm *kvm);arch/arm/include/asm/kvm_mmu.h +12
-1@@ -37,6 +37,7 @@#include <linux/highmem.h> #include <asm/cacheflush.h> +#include <asm/cputype.h> #include <asm/pgalloc.h> #include <asm/stage2_pgtable.h> @@ -223,7 +224,17 @@ static inline unsigned int kvm_get_vmid_bits(void)static inline void *kvm_get_hyp_vector (void ) { -return kvm_ksym_ref (__kvm_hyp_vector); +extern char __kvm_hyp_vector[]; +extern char __kvm_hyp_vector_bp_inv[]; + +switch (read_cpuid_part ()) { +case ARM_CPU_PART_CORTEX_A12 : +case ARM_CPU_PART_CORTEX_A17 : +return kvm_ksym_ref (__kvm_hyp_vector_bp_inv); + +default : +return kvm_ksym_ref (__kvm_hyp_vector); + } }static inline int kvm_map_vectors (void )arch/arm/kvm/hyp/hyp-entry.S +11
-0@@ -70,6 +70,57 @@ __kvm_hyp_vector: W(b) hyp_hvc W(b) hyp_irq W(b) hyp_fiq + + .align 5 +__kvm_hyp_vector_bp_inv: + .global __kvm_hyp_vector_bp_inv + + /* + * We encode the exception entry in the bottom 3 bits of + * SP, and we have to guarantee to be 8 bytes aligned. + */ + W(add) sp, sp, #1 /* Reset 7 */ + W(add) sp, sp, #1 /* Undef 6 */arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 93
Quoted 10 lines
arch/arm/kvm/hyp/hyp-entry.S +10
-0+ W(add) sp, sp, #1 /* Syscall 5 */ + W(add) sp, sp, #1 /* Prefetch abort 4 */ + W(add) sp, sp, #1 /* Data abort 3 */ + W(add) sp, sp, #1 /* HVC 2 */ + W(add) sp, sp, #1 /* IRQ 1 */ + W(nop) /* FIQ 0 */ + + mcr p15, 0, r0, c7, c5, 6 /* BPIALL */ + isb +The below is quite a bit of faff; might it be worth an
#ifdef
CONFIG_THUMB2_KERNELQuoted 5 lines
arch/arm/kvm/hyp/hyp-entry.S +5
-0+ /* + * Yet another silly hack: Use VPIDR as a temp register. + * Thumb2 is really a pain, as SP cannot be used with most + * of the bitwise instructions. The vect_br macro ensures + * things gets cleaned-up.arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 108
Quoted 10 lines
arch/arm/kvm/hyp/hyp-entry.S +10
-0+ */ + mcr p15, 4, r0, c0, c0, 0 /* VPIDR */ + mov r0, sp + and r0, r0, #7 + sub sp, sp, r0 + push {r1, r2} + mov r1, r0 + mrc p15, 4, r0, c0, c0, 0 /* VPIDR */ + mrc p15, 0, r2, c0, c0, 0 /* MIDR */ + mcr p15, 4, r2, c0, c0, 0 /* VPIDR */#endif
Quoted 8 lines
Quoted 4 lines
arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 110
Quoted 2 lines
arch/arm/kvm/hyp/hyp-entry.S +2
-0+ +.macro vect_br val, targ
ARM(cmp sp, #val)Doesn't quite work, as we still have all the top bits that contain the stack address. But I like the idea of making it baster for non-T2. How about this instead?
Right, the CMP is indeed totally bogus - I hadn't exactly reasoned this through in detail ;)
Quoted 37 lines
arch/arm/kvm/hyp/hyp-entry.S +10
-2@@ -114,6 +114,8 @@ __kvm_hyp_vector_bp_inv: isb decode_vectors: + +#ifdef CONFIG_THUMB2_KERNEL /* * Yet another silly hack: Use VPIDR as a temp register. * Thumb2 is really a pain, as SP cannot be used with most@@ -129,10 +131,16 @@ decode_vectors: mrc p15, 4, r0, c0, c0, 0 /* VPIDR */ mrc p15, 0, r2, c0, c0, 0 /* MIDR */ mcr p15, 4, r2, c0, c0, 0 /* VPIDR */ +#endif .macro vect_br val, targ - cmp r1, #\val - popeq {r1, r2} +ARM( eor sp, sp, #\val ) +ARM( tst sp, #7 ) +ARM( eorne sp, sp, #\val ) + +THUMB( cmp r1, #\val ) +THUMB( popeq {r1, r2} ) + beq \targ .endmQuoted 5 lines
THUMB(cmp r1, #\val)
THUMB(popeq {r1, r2}
Quoted 6 lines
Quoted 4 lines
arch/arm/kvm/hyp/hyp-entry.S __kvm_hyp_vector: · line 112
Quoted 2 lines
arch/arm/kvm/hyp/hyp-entry.S +2
-0+ beq \targ +.endm...to keep the "normal" path relatively streamlined?
I think the above achieves it... Thoughts?
Yeah, that looks like it should do the trick; very cunning!
Robin.
PATCH v3 4/6 arm: Add icache invalidation on switch_mm for Cortex-A15
In reply to Marc ZyngierIn order to avoid aliasing attacks against the branch predictor, Cortex-A15 require to invalidate the BTB when switching from one user context to another. The only way to do so on this CPU is to perform an ICIALLU, having set ACTLR[0] to 1 from secure mode.
--- arch/arm/mm/proc-v7-2level.S | 10 ++++++++++ arch/arm/mm/proc-v7-3level.S | 16 ++++++++++++++++ arch/arm/mm/proc-v7.S | 18 +++++++++++++++++- 3 files changed, 43 insertions(+), 1 deletion(-)
arch/arm/mm/proc-v7-2level.S +10 -0
@@ -40,7 +40,17 @@ * Note that we always need to flush BTAC/BTB if IBE is set * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. + * + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. */ +ENTRY(cpu_ca15_switch_mm) +#ifdef CONFIG_MMU + mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + isb + b cpu_v7_switch_mm +#endif +ENDPROC(cpu_ca15_switch_mm) ENTRY(cpu_v7_btbinv_switch_mm) #ifdef CONFIG_MMU mov r2, #0
arch/arm/mm/proc-v7-3level.S +16 -0
@@ -71,6 +71,22 @@ ENTRY(cpu_v7_switch_mm) ENDPROC(cpu_v7_switch_mm) ENDPROC(cpu_v7_btbinv_switch_mm) +/* + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. + */ +ENTRY(cpu_ca15_switch_mm) +#ifdef CONFIG_MMU + mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + mmid r2, r2 + asid r2, r2 + orr rpgdh, rpgdh, r2, lsl #(48 - 32) @ upper 32-bits of pgd + mcrr p15, 0, rpgdl, rpgdh, c2 @ set TTB 0 + isb +#endif + ret lr +ENDPROC(cpu_ca15_switch_mm) + #ifdef __ARMEB__ #define rl r3 #define rh r2
arch/arm/mm/proc-v7.S +17 -1
@@ -173,6 +173,21 @@ ENDPROC(cpu_v7_do_resume) globl_equ cpu_v7_btbinv_do_resume, cpu_v7_do_resume #endif +/* + * Cortex-A15 that require an icache invalidation on switch_mm + */ + globl_equ cpu_ca15_proc_init, cpu_v7_proc_init + globl_equ cpu_ca15_proc_fin, cpu_v7_proc_fin + globl_equ cpu_ca15_reset, cpu_v7_reset + globl_equ cpu_ca15_do_idle, cpu_v7_do_idle + globl_equ cpu_ca15_dcache_clean_area, cpu_v7_dcache_clean_area + globl_equ cpu_ca15_set_pte_ext, cpu_v7_set_pte_ext + globl_equ cpu_ca15_suspend_size, cpu_v7_suspend_size +#ifdef CONFIG_ARM_CPU_SUSPEND + globl_equ cpu_ca15_do_suspend, cpu_v7_do_suspend + globl_equ cpu_ca15_do_resume, cpu_v7_do_resume +#endif + /* * Cortex-A9 processor functions */
@@ -549,6 +564,7 @@ __v7_setup_stack: @ define struct processor (see <asm/proc-fns.h> and proc-macros.S) define_processor_functions v7, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 define_processor_functions v7_btbinv, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 + define_processor_functions ca15, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 #ifndef CONFIG_ARM_LPAE define_processor_functions ca9mp, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 #endif
@@ -668,7 +684,7 @@ __v7_ca12mp_proc_info: __v7_ca15mp_proc_info: .long 0x410fc0f0 .long 0xff0ffff0 - __v7_proc __v7_ca15mp_proc_info, __v7_ca15mp_setup + __v7_proc __v7_ca15mp_proc_info, __v7_ca15mp_setup, proc_fns = ca15_processor_functions .size __v7_ca15mp_proc_info, . - __v7_ca15mp_proc_info /*
Signature · 1 line
2.14.2
PATCH v3 4/6 arm: Add icache invalidation on switch_mm for Cortex-A15
Quoted 18 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, Cortex-A15 require to invalidate the BTB when switching from one user context to another. The only way to do so on this CPU is to perform an ICIALLU, having set ACTLR[0] to 1 from secure mode.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm/mm/proc-v7-2level.S | 10 ++++++++++ arch/arm/mm/proc-v7-3level.S | 16 ++++++++++++++++ arch/arm/mm/proc-v7.S | 18 +++++++++++++++++- 3 files changed, 43 insertions(+), 1 deletion(-)arch/arm/mm/proc-v7-2level.S +0
-0
arch/arm/mm/proc-v7-2level.S line 46
Quoted 7 lines
arch/arm/mm/proc-v7-2level.S +3
-0@@ -40,7 +40,17 @@ * Note that we always need to flush BTAC/BTB if IBE is set * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. + * + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. */
Seems like we can read (but not write) this bit from non-secure. Should we test if it's set somewhere during boot and warn the user if not?
Quoted 24 lines
arch/arm/mm/proc-v7-2level.S +7
-0+ENTRY(cpu_ca15_switch_mm) +#ifdef CONFIG_MMU + mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + isb + b cpu_v7_switch_mm +#endif +ENDPROC(cpu_ca15_switch_mm) ENTRY(cpu_v7_btbinv_switch_mm) #ifdef CONFIG_MMU mov r2, #0arch/arm/mm/proc-v7-3level.S +6
-0@@ -71,6 +71,22 @@ ENTRY(cpu_v7_switch_mm) ENDPROC(cpu_v7_switch_mm) ENDPROC(cpu_v7_btbinv_switch_mm) +/* + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. + */ +ENTRY(cpu_ca15_switch_mm) +#ifdef CONFIG_MMU
arch/arm/mm/proc-v7-3level.S ENTRY(cpu_v7_switch_mm) · line 89
Quoted 10 lines
arch/arm/mm/proc-v7-3level.S +10
-0+ mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + mmid r2, r2 + asid r2, r2 + orr rpgdh, rpgdh, r2, lsl #(48 - 32) @ upper 32-bits of pgd + mcrr p15, 0, rpgdl, rpgdh, c2 @ set TTB 0 + isb +#endif + ret lr +ENDPROC(cpu_ca15_switch_mm) +
There's some potential for code shaing with cpu_v7_switch_mm here, either via a macro or by simply calling cpu_v7_switch_mm from cpu_ca15_switch_mm, but I'm not sure if we care?
arch/arm/mm/proc-v7.S ENDPROC(cpu_v7_do_resume) · line 177
Quoted 13 lines
arch/arm/mm/proc-v7-3level.S +0
-0#ifdef __ARMEB__ #define rl r3 #define rh r2arch/arm/mm/proc-v7.S +2
-0@@ -173,6 +173,21 @@ ENDPROC(cpu_v7_do_resume) globl_equ cpu_v7_btbinv_do_resume, cpu_v7_do_resume #endif +/* + * Cortex-A15 that require an icache invalidation on switch_mm
uber nit: The wording is weird here, how about "Cortex-A15 requires an..." ?
Quoted 36 lines
arch/arm/mm/proc-v7.S +15
-1+ */ + globl_equ cpu_ca15_proc_init, cpu_v7_proc_init + globl_equ cpu_ca15_proc_fin, cpu_v7_proc_fin + globl_equ cpu_ca15_reset, cpu_v7_reset + globl_equ cpu_ca15_do_idle, cpu_v7_do_idle + globl_equ cpu_ca15_dcache_clean_area, cpu_v7_dcache_clean_area + globl_equ cpu_ca15_set_pte_ext, cpu_v7_set_pte_ext + globl_equ cpu_ca15_suspend_size, cpu_v7_suspend_size +#ifdef CONFIG_ARM_CPU_SUSPEND + globl_equ cpu_ca15_do_suspend, cpu_v7_do_suspend + globl_equ cpu_ca15_do_resume, cpu_v7_do_resume +#endif + /* * Cortex-A9 processor functions */@@ -549,6 +564,7 @@ __v7_setup_stack: @ define struct processor (see <asm/proc-fns.h> and proc-macros.S) define_processor_functions v7, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 define_processor_functions v7_btbinv, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 + define_processor_functions ca15, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 #ifndef CONFIG_ARM_LPAE define_processor_functions ca9mp, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 #endif@@ -668,7 +684,7 @@ __v7_ca12mp_proc_info: __v7_ca15mp_proc_info: .long 0x410fc0f0 .long 0xff0ffff0 - __v7_proc __v7_ca15mp_proc_info, __v7_ca15mp_setup + __v7_proc __v7_ca15mp_proc_info, __v7_ca15mp_setup, proc_fns = ca15_processor_functions .size __v7_ca15mp_proc_info, . - __v7_ca15mp_proc_info /*Signature · 1 line
2.14.2
Thanks,
-Christoffer
PATCH v3 4/6 arm: Add icache invalidation on switch_mm for Cortex-A15
Quoted 14 lines · Christoffer Dall
Quoted 13 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, Cortex-A15 require to invalidate the BTB when switching from one user context to another. The only way to do so on this CPU is to perform an ICIALLU, having set ACTLR[0] to 1 from secure mode.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm/mm/proc-v7-2level.S | 10 ++++++++++ arch/arm/mm/proc-v7-3level.S | 16 ++++++++++++++++ arch/arm/mm/proc-v7.S | 18 +++++++++++++++++- 3 files changed, 43 insertions(+), 1 deletion(-)
Quoted 15 lines
arch/arm/mm/proc-v7-2level.S line 46
Quoted 12 lines
arch/arm/mm/proc-v7-2level.S +3
-0@@ -40,7 +40,17 @@ * Note that we always need to flush BTAC/BTB if IBE is set * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. + * + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. */Seems like we can read (but not write) this bit from non-secure. Should we test if it's set somewhere during boot and warn the user if not?
That would have to be something that happens much later in the boot process, as the proc structure is picked up very early, before we run any C code.
I could add some detection code later, but we'll be stuck with a icache invalidation for nothing. Not a pretty situation.
Quoted 31 lines
Quoted 31 lines
arch/arm/mm/proc-v7-2level.S +7
-0+ENTRY(cpu_ca15_switch_mm) +#ifdef CONFIG_MMU + mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + isb + b cpu_v7_switch_mm +#endif +ENDPROC(cpu_ca15_switch_mm) ENTRY(cpu_v7_btbinv_switch_mm) #ifdef CONFIG_MMU mov r2, #0arch/arm/mm/proc-v7-3level.S +12
-0@@ -71,6 +71,22 @@ ENTRY(cpu_v7_switch_mm) ENDPROC(cpu_v7_switch_mm) ENDPROC(cpu_v7_btbinv_switch_mm) +/* + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. + */ +ENTRY(cpu_ca15_switch_mm) +#ifdef CONFIG_MMU + mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + mmid r2, r2 + asid r2, r2 + orr rpgdh, rpgdh, r2, lsl #(48 - 32) @ upper 32-bits of pgd + mcrr p15, 0, rpgdl, rpgdh, c2 @ set TTB 0 + isb
Quoted 8 lines
arch/arm/mm/proc-v7-3level.S ENTRY(cpu_v7_switch_mm) · line 89
Quoted 4 lines
arch/arm/mm/proc-v7-3level.S +4
-0+#endif + ret lr +ENDPROC(cpu_ca15_switch_mm) +There's some potential for code shaing with
cpu_v7_switch_mmhere, either via a macro or by simply callingcpu_v7_switch_mmfromcpu_ca15_switch_mm, but I'm not sure if we care?
We could, but most things seems to been duplicated between the two page-table implementations. I can look in to it if this is going anywhere.
Quoted 12 lines
Quoted 12 lines
arch/arm/mm/proc-v7-3level.S +0
-0#ifdef __ARMEB__ #define rl r3 #define rh r2arch/arm/mm/proc-v7.S +0
-0@@ -173,6 +173,21 @@ ENDPROC(cpu_v7_do_resume) globl_equ cpu_v7_btbinv_do_resume, cpu_v7_do_resume #endif
Quoted 5 lines
arch/arm/mm/proc-v7.S ENDPROC(cpu_v7_do_resume) · line 177
Quoted 2 lines
arch/arm/mm/proc-v7.S +2
-0+/* + * Cortex-A15 that require an icache invalidation on switch_mmuber nit: The wording is weird here, how about "Cortex-A15 requires an..." ?
Sure.
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
PATCH v3 4/6 arm: Add icache invalidation on switch_mm for Cortex-A15
Quoted 31 lines · Marc Zyngier
Quoted 14 lines · Christoffer Dall
Quoted 13 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, Cortex-A15 require to invalidate the BTB when switching from one user context to another. The only way to do so on this CPU is to perform an ICIALLU, having set ACTLR[0] to 1 from secure mode.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm/mm/proc-v7-2level.S | 10 ++++++++++ arch/arm/mm/proc-v7-3level.S | 16 ++++++++++++++++ arch/arm/mm/proc-v7.S | 18 +++++++++++++++++- 3 files changed, 43 insertions(+), 1 deletion(-)Quoted 15 lines
arch/arm/mm/proc-v7-2level.S line 46
Quoted 12 lines
arch/arm/mm/proc-v7-2level.S +3
-0@@ -40,7 +40,17 @@ * Note that we always need to flush BTAC/BTB if IBE is set * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. + * + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. */Seems like we can read (but not write) this bit from non-secure. Should we test if it's set somewhere during boot and warn the user if not?
Quoted 6 lines
That would have to be something that happens much later in the boot process, as the proc structure is picked up very early, before we run any C code.
I could add some detection code later, but we'll be stuck with a icache invalidation for nothing. Not a pretty situation.
Would it make sense to put some code in the decompressor (maybe some people don't use the decompressor...) similar to how we check for an ARMv7 CPU w/o LPAE attempting to boot a LPAE enabled kernel for instance.
This is not technically a hard failure if ACTLR[0] is not set, everything will still work fine, it just is not ideal from a security perspective.
Should we have a Kconfig option, similar to what Nishanth proposed for u-boot, with which we could guard an optional check against ACTLR[0] being set?
Quoted 68 lines
Quoted 32 lines
Quoted 32 lines
arch/arm/mm/proc-v7-2level.S +7
-0+ENTRY(cpu_ca15_switch_mm) +#ifdef CONFIG_MMU + mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + isb + b cpu_v7_switch_mm +#endif +ENDPROC(cpu_ca15_switch_mm) ENTRY(cpu_v7_btbinv_switch_mm) #ifdef CONFIG_MMU mov r2, #0arch/arm/mm/proc-v7-3level.S +12
-0@@ -71,6 +71,22 @@ ENTRY(cpu_v7_switch_mm) ENDPROC(cpu_v7_switch_mm) ENDPROC(cpu_v7_btbinv_switch_mm) +/* + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. + */ +ENTRY(cpu_ca15_switch_mm) +#ifdef CONFIG_MMU + mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + mmid r2, r2 + asid r2, r2 + orr rpgdh, rpgdh, r2, lsl #(48 - 32) @ upper 32-bits of pgd + mcrr p15, 0, rpgdl, rpgdh, c2 @ set TTB 0 + isbQuoted 8 lines
arch/arm/mm/proc-v7-3level.S ENTRY(cpu_v7_switch_mm) · line 89
Quoted 4 lines
arch/arm/mm/proc-v7-3level.S +4
-0+#endif + ret lr +ENDPROC(cpu_ca15_switch_mm) +There's some potential for code shaing with
cpu_v7_switch_mmhere, either via a macro or by simply callingcpu_v7_switch_mmfromcpu_ca15_switch_mm, but I'm not sure if we care?We could, but most things seems to been duplicated between the two page-table implementations. I can look in to it if this is going anywhere.
Quoted 12 lines
Quoted 12 lines
arch/arm/mm/proc-v7-3level.S +0
-0#ifdef __ARMEB__ #define rl r3 #define rh r2arch/arm/mm/proc-v7.S +0
-0@@ -173,6 +173,21 @@ ENDPROC(cpu_v7_do_resume) globl_equ cpu_v7_btbinv_do_resume, cpu_v7_do_resume #endifQuoted 5 lines
arch/arm/mm/proc-v7.S ENDPROC(cpu_v7_do_resume) · line 177
Quoted 2 lines
arch/arm/mm/proc-v7.S +2
-0+/* + * Cortex-A15 that require an icache invalidation on switch_mmuber nit: The wording is weird here, how about "Cortex-A15 requires an..." ?
Sure.
Thanks,
M.
Signature · 1 line
Florian
PATCH v3 4/6 arm: Add icache invalidation on switch_mm for Cortex-A15
Quoted 38 lines · Florian Fainelli
Quoted 35 lines · Marc Zyngier
Quoted 14 lines · Christoffer Dall
Quoted 13 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, Cortex-A15 require to invalidate the BTB when switching from one user context to another. The only way to do so on this CPU is to perform an ICIALLU, having set ACTLR[0] to 1 from secure mode.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm/mm/proc-v7-2level.S | 10 ++++++++++ arch/arm/mm/proc-v7-3level.S | 16 ++++++++++++++++ arch/arm/mm/proc-v7.S | 18 +++++++++++++++++- 3 files changed, 43 insertions(+), 1 deletion(-)Quoted 15 lines
arch/arm/mm/proc-v7-2level.S line 46
Quoted 12 lines
arch/arm/mm/proc-v7-2level.S +3
-0@@ -40,7 +40,17 @@ * Note that we always need to flush BTAC/BTB if IBE is set * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. + * + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. */Seems like we can read (but not write) this bit from non-secure. Should we test if it's set somewhere during boot and warn the user if not?
That would have to be something that happens much later in the boot process, as the proc structure is picked up very early, before we run any C code.
Quoted 6 lines
Quoted 2 lines
I could add some detection code later, but we'll be stuck with a icache invalidation for nothing. Not a pretty situation.
Would it make sense to put some code in the decompressor (maybe some people don't use the decompressor...) similar to how we check for an ARMv7 CPU w/o LPAE attempting to boot a LPAE enabled kernel for instance.
Which only works if you have CONFIG_DEBUG_LL, and thus not generally usable. Also, the LPAE test is not in the decompressor. but in the kernel itself (head-common.S). Not that it changes anything...
Also, testing that at early boot is not that useful, as you only have the boot CPU enabled at this point. You want to check that you have whatever configuration is required on a per-CPU basis, which means you have to hook this into the SMP bring-up code. And play nice with things like big-little, in both SMP and MCPM configurations.
Quoted 7 lines
This is not technically a hard failure if ACTLR[0] is not set, everything will still work fine, it just is not ideal from a security perspective.
Should we have a
Kconfigoption, similar to what Nishanth proposed for u-boot, with which we could guard an optional check against ACTLR[0] being set?
Sure, why not... If you have something in mind, I can review it.
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
PATCH v3 4/6 arm: Add icache invalidation on switch_mm for Cortex-A15
In reply to Marc ZyngierQuoted 78 lines · Marc Zyngier
Quoted 14 lines · Christoffer Dall
Quoted 13 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, Cortex-A15 require to invalidate the BTB when switching from one user context to another. The only way to do so on this CPU is to perform an ICIALLU, having set ACTLR[0] to 1 from secure mode.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm/mm/proc-v7-2level.S | 10 ++++++++++ arch/arm/mm/proc-v7-3level.S | 16 ++++++++++++++++ arch/arm/mm/proc-v7.S | 18 +++++++++++++++++- 3 files changed, 43 insertions(+), 1 deletion(-)Quoted 15 lines
arch/arm/mm/proc-v7-2level.S line 46
Quoted 12 lines
arch/arm/mm/proc-v7-2level.S +3
-0@@ -40,7 +40,17 @@ * Note that we always need to flush BTAC/BTB if IBE is set * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. + * + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. */Seems like we can read (but not write) this bit from non-secure. Should we test if it's set somewhere during boot and warn the user if not?
That would have to be something that happens much later in the boot process, as the proc structure is picked up very early, before we run any C code.
I could add some detection code later, but we'll be stuck with a icache invalidation for nothing. Not a pretty situation.
Quoted 31 lines
Quoted 31 lines
arch/arm/mm/proc-v7-2level.S +7
-0+ENTRY(cpu_ca15_switch_mm) +#ifdef CONFIG_MMU + mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + isb + b cpu_v7_switch_mm +#endif +ENDPROC(cpu_ca15_switch_mm) ENTRY(cpu_v7_btbinv_switch_mm) #ifdef CONFIG_MMU mov r2, #0arch/arm/mm/proc-v7-3level.S +12
-0@@ -71,6 +71,22 @@ ENTRY(cpu_v7_switch_mm) ENDPROC(cpu_v7_switch_mm) ENDPROC(cpu_v7_btbinv_switch_mm) +/* + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. + */ +ENTRY(cpu_ca15_switch_mm) +#ifdef CONFIG_MMU + mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + mmid r2, r2 + asid r2, r2 + orr rpgdh, rpgdh, r2, lsl #(48 - 32) @ upper 32-bits of pgd + mcrr p15, 0, rpgdl, rpgdh, c2 @ set TTB 0 + isbQuoted 8 lines
arch/arm/mm/proc-v7-3level.S ENTRY(cpu_v7_switch_mm) · line 89
Quoted 4 lines
arch/arm/mm/proc-v7-3level.S +4
-0+#endif + ret lr +ENDPROC(cpu_ca15_switch_mm) +There's some potential for code shaing with
cpu_v7_switch_mmhere, either via a macro or by simply callingcpu_v7_switch_mmfromcpu_ca15_switch_mm, but I'm not sure if we care?
Quoted 2 lines
We could, but most things seems to been duplicated between the two page-table implementations. I can look in to it if this is going anywhere.
Could this not just do the equivalent of the cpu_ca8 version for 2-level paging, i.e. execute the ICIALLU then fall through - everything else is the same, right?
Robin.
Quoted 25 lines
Quoted 13 lines
Quoted 13 lines
arch/arm/mm/proc-v7-3level.S +0
-0#ifdef __ARMEB__ #define rl r3 #define rh r2arch/arm/mm/proc-v7.S +0
-0@@ -173,6 +173,21 @@ ENDPROC(cpu_v7_do_resume) globl_equ cpu_v7_btbinv_do_resume, cpu_v7_do_resume #endifQuoted 5 lines
arch/arm/mm/proc-v7.S ENDPROC(cpu_v7_do_resume) · line 177
Quoted 2 lines
arch/arm/mm/proc-v7.S +2
-0+/* + * Cortex-A15 that require an icache invalidation on switch_mmuber nit: The wording is weird here, how about "Cortex-A15 requires an..." ?
Sure.
Thanks,
M.
PATCH v3 4/6 arm: Add icache invalidation on switch_mm for Cortex-A15
In reply to Marc ZyngierQuoted 5 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, Cortex-A15 require to invalidate the BTB when switching from one user context to another. The only way to do so on this CPU is to perform an ICIALLU, having set ACTLR[0] to 1 from secure mode.
Even though this is a platform design mistake, let's say your Linux kernel boots in secure supervisor mode, we could have code that tries to set ACTLR[0] as early as possible, since the writes are ignored if executing from non-secure mode. If Linux is booted normally either PL1 or PL2 non-secure we could still check ACTLR[0].
My concern is that without doing this, we may have a hard time catching improper firmware as well as having bogus bug reports. This is completely RFC though since:
- I could not quite figure out yet why update ca15_actlr_status from assembly is not reflected in the C code despite using PC relative loads
- this is done in __ca15mp_setup and __b15mp_setup because we know by then that we have either of these two CPUs but we could presumably move this check under a Kconfig option and earlier where all erratas are checked
Signature · 9 lines
Florian -------------- next part -------------- A non-text attachment was scrubbed... Name: actlr-kernel-check.patch Type: text/x-patch Size: 4133 bytes Desc: not available URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180127/da1e1735/attachment-0001.bin>
PATCH v3 4/6 arm: Add icache invalidation on switch_mm for Cortex-A15
Quoted 13 lines · On Sat, 27 Jan 2018 14:23:57 -0800 Florian Fainelli <[email protected]>…
Quoted 5 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, Cortex-A15 require to invalidate the BTB when switching from one user context to another. The only way to do so on this CPU is to perform an ICIALLU, having set ACTLR[0] to 1 from secure mode.
Even though this is a platform design mistake, let's say your Linux kernel boots in secure supervisor mode, we could have code that tries to set ACTLR[0] as early as possible, since the writes are ignored if executing from non-secure mode. If Linux is booted normally either PL1 or PL2 non-secure we could still check ACTLR[0].
Quoted 6 lines
My concern is that without doing this, we may have a hard time catching improper firmware as well as having bogus bug reports. This is completely RFC though since:
- I could not quite figure out yet why update
ca15_actlr_statusfrom assembly is not reflected in the C code despite using PC relative loads
One potential problem is that you're writing this with the MMU off, and reading it with the MMU on. Unless you've added CMOs to make this visible, it will not be reliable.
Also, I'm not sure how this works with MCPM.
Quoted 3 lines
- this is done in
__ca15mp_setupand__b15mp_setupbecause we know by then that we have either of these two CPUs but we could presumably move this check under aKconfigoption and earlier where all erratas are checked
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
PATCH v3 4/6 arm: Add icache invalidation on switch_mm for Cortex-A15
Quoted 20 lines · Marc Zyngier
Quoted 17 lines · On Sat, 27 Jan 2018 14:23:57 -0800 Florian Fainelli <[email protected]>…
Quoted 5 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, Cortex-A15 require to invalidate the BTB when switching from one user context to another. The only way to do so on this CPU is to perform an ICIALLU, having set ACTLR[0] to 1 from secure mode.
Even though this is a platform design mistake, let's say your Linux kernel boots in secure supervisor mode, we could have code that tries to set ACTLR[0] as early as possible, since the writes are ignored if executing from non-secure mode. If Linux is booted normally either PL1 or PL2 non-secure we could still check ACTLR[0].
My concern is that without doing this, we may have a hard time catching improper firmware as well as having bogus bug reports. This is completely RFC though since:
Quoted 6 lines
Quoted 2 lines
- I could not quite figure out yet why update
ca15_actlr_statusfrom assembly is not reflected in the C code despite using PC relative loadsOne potential problem is that you're writing this with the MMU off, and reading it with the MMU on. Unless you've added CMOs to make this visible, it will not be reliable.
I did not, let's change that.
Quoted 2 lines
Also, I'm not sure how this works with MCPM.
I just started looking at MCPM, do you know roughly at what point do non-MCPM and MCPM entry points converge? If we moved the check for ACTLR[0] there, would that be too late.
Quoted 9 lines
Quoted 4 lines
- this is done in
__ca15mp_setupand__b15mp_setupbecause we know by then that we have either of these two CPUs but we could presumably move this check under aKconfigoption and earlier where all erratas are checkedThanks,
M.
Signature · 1 line
Florian
PATCH v3 4/6 arm: Add icache invalidation on switch_mm for Cortex-A15
Quoted 28 lines · Florian Fainelli
Quoted 20 lines · Marc Zyngier
Quoted 17 lines · On Sat, 27 Jan 2018 14:23:57 -0800 Florian Fainelli <[email protected]>…
Quoted 5 lines · Marc Zyngier
In order to avoid aliasing attacks against the branch predictor, Cortex-A15 require to invalidate the BTB when switching from one user context to another. The only way to do so on this CPU is to perform an ICIALLU, having set ACTLR[0] to 1 from secure mode.
Even though this is a platform design mistake, let's say your Linux kernel boots in secure supervisor mode, we could have code that tries to set ACTLR[0] as early as possible, since the writes are ignored if executing from non-secure mode. If Linux is booted normally either PL1 or PL2 non-secure we could still check ACTLR[0].
My concern is that without doing this, we may have a hard time catching improper firmware as well as having bogus bug reports. This is completely RFC though since:
Quoted 6 lines
Quoted 2 lines
- I could not quite figure out yet why update
ca15_actlr_statusfrom assembly is not reflected in the C code despite using PC relative loadsOne potential problem is that you're writing this with the MMU off, and reading it with the MMU on. Unless you've added CMOs to make this visible, it will not be reliable.
Quoted 8 lines
I did not, let's change that.
Quoted 2 lines
Also, I'm not sure how this works with MCPM.
I just started looking at MCPM, do you know roughly at what point do non-MCPM and MCPM entry points converge? If we moved the check for ACTLR[0] there, would that be too late.
I have no idea. You'll have to reverse-engineer it (MCPM makes me feel slightly sick).
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
PATCH v3 5/6 arm: Invalidate icache on prefetch abort outside of user mapping on Cortex-A15
In reply to Marc ZyngierIn order to prevent aliasing attacks on the branch predictor, invalidate the icache on Cortex-A15, which has the side effect of invalidating the BTB. This requires ACTLR[0] to be set to 1 (secure operation).
--- arch/arm/include/asm/cp15.h | 1 + arch/arm/mm/fault.c | 4 ++++ 2 files changed, 5 insertions(+)
arch/arm/include/asm/cp15.h +1 -0
@@ -66,6 +66,7 @@#define write_sysreg (v, ...) __write_sysreg(v, __VA_ARGS__)#define BPIALL __ACCESS_CP15(c7, 0, c5, 6) +#define ICIALLU __ACCESS_CP15(c7, 0, c5, 0)extern unsigned long cr_alignment;/* defined in entry-armv.S */
arch/arm/mm/fault.c +4 -0
@@ -410,6 +410,10 @@ do_pabt_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs)case ARM_CPU_PART_CORTEX_A17 :write_sysreg (0 ,BPIALL );break ; + +case ARM_CPU_PART_CORTEX_A15 : +write_sysreg (0 ,ICIALLU ); +break ; } }
Signature · 1 line
2.14.2
PATCH v3 6/6 arm: KVM: Invalidate icache on guest exit for Cortex-A15
In reply to Marc ZyngierIn order to avoid aliasing attacks against the branch predictor on Cortex-A15, let's invalidate the BTB on guest exit, which can only be done by invalidating the icache (with ACTLR[0] being set).
We use the same hack as for A12/A17 to perform the vector decoding.
--- arch/arm/include/asm/kvm_mmu.h | 4 ++++ arch/arm/kvm/hyp/hyp-entry.S | 25 ++++++++++++++++++++++++- 2 files changed, 28 insertions(+), 1 deletion(-)
arch/arm/include/asm/kvm_mmu.h +4 -0
@@ -226,12 +226,16 @@ static inline void *kvm_get_hyp_vector(void) {extern char __kvm_hyp_vector[];extern char __kvm_hyp_vector_bp_inv[]; +extern char __kvm_hyp_vector_ic_inv[];switch (read_cpuid_part ()) {case ARM_CPU_PART_CORTEX_A12 :case ARM_CPU_PART_CORTEX_A17 :return kvm_ksym_ref (__kvm_hyp_vector_bp_inv); +case ARM_CPU_PART_CORTEX_A15 : +return kvm_ksym_ref (__kvm_hyp_vector_ic_inv); +default :return kvm_ksym_ref (__kvm_hyp_vector); }
arch/arm/kvm/hyp/hyp-entry.S +24 -1
@@ -70,7 +70,29 @@ __kvm_hyp_vector: W(b) hyp_hvc W(b) hyp_irq W(b) hyp_fiq - + + .align 5 +__kvm_hyp_vector_ic_inv: + .global __kvm_hyp_vector_ic_inv + + /* + * We encode the exception entry in the bottom 3 bits of + * SP, and we have to guarantee to be 8 bytes aligned. + */ + W(add) sp, sp, #1 /* Reset 7 */ + W(add) sp, sp, #1 /* Undef 6 */ + W(add) sp, sp, #1 /* Syscall 5 */ + W(add) sp, sp, #1 /* Prefetch abort 4 */ + W(add) sp, sp, #1 /* Data abort 3 */ + W(add) sp, sp, #1 /* HVC 2 */ + W(add) sp, sp, #1 /* IRQ 1 */ + W(nop) /* FIQ 0 */ + + mcr p15, 0, r0, c7, c5, 0 /* ICIALLU */ + isb + + b decode_vectors + .align 5 __kvm_hyp_vector_bp_inv: .global __kvm_hyp_vector_bp_inv
@@ -91,6 +113,7 @@ __kvm_hyp_vector_bp_inv: mcr p15, 0, r0, c7, c5, 6 /* BPIALL */ isb +decode_vectors: /* * Yet another silly hack: Use VPIDR as a temp register. * Thumb2 is really a pain, as SP cannot be used with most
Signature · 1 line
2.14.2
v3 cover 32bit ARM branch predictor hardening
In reply to Marc ZyngierQuoted 29 lines · Marc Zyngier
This small series implements some basic BP hardening by invalidating the BTB on 32bit ARM CPUs that are known to be susceptible to aliasing attacks (Spectre variant 2). It doesn't help non-ARM 32bit CPUs, nor 32bit kernels that run on 64bit capable CPUs. This series doesn't mitigate Spectre variant 1 either.
These patches are closely modelled against what we do on arm64, although simpler as we can rely on an architected instruction to perform the invalidation. The notable exception is Cortex-A15, where BTB invalidation behaves like a NOP, and the only way to shoot the predictor down is to invalidate the icache *and* to have ACTLR[0] set to 1 (which is a secure-only operation).
The first patch reuses the Cortex-A8 BTB invalidation in
switch_mmand generalises it to be used on all affected CPUs. The second perform the same invalidation on prefetch abort outside of the userspace range. The third one nukes it on guest exit, and results in some major surgery as we cannot take a branch from the vectors (that, and Thumb2 being a massive pain).Patches 4 to 6 are doing a similar thing for Cortex-A15, which the aforementioned ICIALLU.
To sum up the requirements:
- Both Cortex-A8 and Cortex-A15 need to have ACTLR.IBE (bit 0) set to 1 from secure mode. For Cortex-A8, this overlaps withARM_ERRATA_430973which also requires it.
- Cortex-A9, A12 and A17 do not require any extra configuration.
Quoted 6 lines
Note 1: Contrary to the initial version, this new series relies on the arm64/kpti branch (I reuse the per-CPU vector hook for KVM).
Note 2: M-class CPUs are not affected and for R-class cores, the mitigation doesn't make much sense since we do not enforce user/kernel isolation.
Besides the minor nits in some of the patches, for the series:
v3 cover 32bit ARM branch predictor hardening
In reply to Marc ZyngierHi,
Quoted 6 lines · Marc Zyngier
This small series implements some basic BP hardening by invalidating the BTB on 32bit ARM CPUs that are known to be susceptible to aliasing attacks (Spectre variant 2). It doesn't help non-ARM 32bit CPUs, nor 32bit kernels that run on 64bit capable CPUs. This series doesn't mitigate Spectre variant 1 either.
Quoted 6 lines
These patches are closely modelled against what we do on arm64, although simpler as we can rely on an architected instruction to perform the invalidation. The notable exception is Cortex-A15, where BTB invalidation behaves like a NOP, and the only way to shoot the predictor down is to invalidate the icache *and* to have ACTLR[0] set to 1 (which is a secure-only operation).
FWIW, I tested this series briefly on a Calxeda Midway (Cortex-A15). Both without and with ACTLR[0] (set in the secure firmware code), I could boot it fine and ran two KVM guests under some stress.
So at least no regression here.
Thanks for putting this together!
Cheers,
Andre.
Quoted 59 lines
The first patch reuses the Cortex-A8 BTB invalidation in
switch_mmand generalises it to be used on all affected CPUs. The second perform the same invalidation on prefetch abort outside of the userspace range. The third one nukes it on guest exit, and results in some major surgery as we cannot take a branch from the vectors (that, and Thumb2 being a massive pain).Patches 4 to 6 are doing a similar thing for Cortex-A15, which the aforementioned ICIALLU.
To sum up the requirements:
- Both Cortex-A8 and Cortex-A15 need to have ACTLR.IBE (bit 0) set to 1 from secure mode. For Cortex-A8, this overlaps withARM_ERRATA_430973which also requires it.
- Cortex-A9, A12 and A17 do not require any extra configuration.Note 1: Contrary to the initial version, this new series relies on the arm64/kpti branch (I reuse the per-CPU vector hook for KVM).
Note 2: M-class CPUs are not affected and for R-class cores, the mitigation doesn't make much sense since we do not enforce user/kernel isolation.
* From v2:
- Fixed !MMU build
- Small KVM optimisation (suggested by Robin)
- Fixed register zeroing incpu_v7_btbinv_switch_mm(noticed by Andre)* From v1:
- Fixed brokenhyp_fiqvector (noticed by Ard)
- Fixed broken BTB invalidation in LPAEswitch_mm(reported by Andre)
- Revamped invalidation on PABT (noticed by James on arm64, suggested by Will)
- Rewrote the whole HYP sequence, as Thumb2 was pretty unhappy about arithmetic with the stack pointerMarc Zyngier (6):
arm: Add BTB invalidation onswitch_mmfor Cortex-A9, A12 and A17
arm: Invalidate BTB on prefetch abort outside of user mapping onCortex A8, A9, A12 and A17 arm: KVM: Invalidate BTB on guest exit for Cortex-A12/A17 arm: Add icache invalidation on switch_mm for Cortex-A15 arm: Invalidate icache on prefetch abort outside of user mapping on Cortex-A15arm: KVM: Invalidate icache on guest exit for Cortex-A15
arch/arm/include/asm/cp15.h | 3 ++ arch/arm/include/asm/kvm_asm.h | 2 - arch/arm/include/asm/kvm_mmu.h | 17 ++++++++- arch/arm/kvm/hyp/hyp-entry.S | 85 +++++++++++++++++++++++++++++++++++++++++- arch/arm/mm/fault.c | 29 ++++++++++++++ arch/arm/mm/fsr-2level.c | 4 +- arch/arm/mm/fsr-3level.c | 67 ++++++++++++++++++++++++++++++++- arch/arm/mm/proc-v7-2level.S | 14 ++++++- arch/arm/mm/proc-v7-3level.S | 22 +++++++++++ arch/arm/mm/proc-v7.S | 48 ++++++++++++++++-------- 10 files changed, 265 insertions(+), 26 deletions(-)
v3 cover 32bit ARM branch predictor hardening
In reply to Marc ZyngierHi Marc,
Quoted 20 lines · Marc Zyngier
This small series implements some basic BP hardening by invalidating the BTB on 32bit ARM CPUs that are known to be susceptible to aliasing attacks (Spectre variant 2). It doesn't help non-ARM 32bit CPUs, nor 32bit kernels that run on 64bit capable CPUs. This series doesn't mitigate Spectre variant 1 either.
These patches are closely modelled against what we do on arm64, although simpler as we can rely on an architected instruction to perform the invalidation. The notable exception is Cortex-A15, where BTB invalidation behaves like a NOP, and the only way to shoot the predictor down is to invalidate the icache *and* to have ACTLR[0] set to 1 (which is a secure-only operation).
The first patch reuses the Cortex-A8 BTB invalidation in
switch_mmand generalises it to be used on all affected CPUs. The second perform the same invalidation on prefetch abort outside of the userspace range. The third one nukes it on guest exit, and results in some major surgery as we cannot take a branch from the vectors (that, and Thumb2 being a massive pain).
Quoted 2 lines
Patches 4 to 6 are doing a similar thing for Cortex-A15, which the aforementioned ICIALLU.
I have tested this patch set on A9 and A15 based machines, but with no kvm support, it booted ok, and passed our test framework which has thousands of test cases (not related to performance :) ), so with patch 1,2,4,5 in this patch set,
By the way, this patch set just enable branch predictor hardening on arm32 unconditionally, but some of machines (such as wireless network base station) will not be exposed to user to take advantage of variant 2, and those machines will be pretty sensitive for performance, so can we introduce Kconfig or boot option to disable branch predictor hardening as an option?
Thanks
Hanjun
v3 cover 32bit ARM branch predictor hardening
On Mon, Jan 29, 2018 at 5:36 AM, Hanjun Guo <[email protected]> wrote:
[...]
Quoted 6 lines
By the way, this patch set just enable branch predictor hardening on arm32 unconditionally, but some of machines (such as wireless network base station) will not be exposed to user to take advantage of variant 2, and those machines will be pretty sensitive for performance, so can we introduce
Kconfigor boot option to disable branch predictor hardening as an option?
I am curious: Have you seen performance degradation with this series? If yes, is it possible to share the information?
---
Regards,
Nishanth Menon
v3 cover 32bit ARM branch predictor hardening
Quoted 10 lines · Nishanth Menon
On Mon, Jan 29, 2018 at 5:36 AM, Hanjun Guo <[email protected]> wrote:
[...]Quoted 6 lines
By the way, this patch set just enable branch predictor hardening on arm32 unconditionally, but some of machines (such as wireless network base station) will not be exposed to user to take advantage of variant 2, and those machines will be pretty sensitive for performance, so can we introduce
Kconfigor boot option to disable branch predictor hardening as an option?
Quoted 2 lines
I am curious: Have you seen performance degradation with this series? If yes, is it possible to share the information?
Sorry for the late reply, the performance data for context switch (CFS) is about 6%~12% drop (A9 based machine) for the first around test, but the data is not stable, I need to retest then I will update here.
Thanks
Hanjun
v3 cover 32bit ARM branch predictor hardening
Quoted 11 lines · Hanjun Guo
Quoted 10 lines · Nishanth Menon
On Mon, Jan 29, 2018 at 5:36 AM, Hanjun Guo <[email protected]> wrote:
[...]Quoted 6 lines
By the way, this patch set just enable branch predictor hardening on arm32 unconditionally, but some of machines (such as wireless network base station) will not be exposed to user to take advantage of variant 2, and those machines will be pretty sensitive for performance, so can we introduce
Kconfigor boot option to disable branch predictor hardening as an option?
Quoted 6 lines
Quoted 2 lines
I am curious: Have you seen performance degradation with this series? If yes, is it possible to share the information?
Sorry for the late reply, the performance data for context switch (CFS) is about 6%~12% drop (A9 based machine) for the first around test, but the data is not stable, I need to retest then I will update here.
What tool did you use to measure this? On a Brahma-B15 platform clocked at 1.5Ghz, across kernels 4.1, 4.9 (4.15 in progress as we speak), I measured the following, with two memory configurations, one giving 256MB of usable memory, another giving 3GB of usable memory, results below are only the most extreme 256MB case. This is running 13 groups because the ASID space is 256bits so this should force at least two full ASID generation rollovers (assuming the logic is correct here).
for i in $(seq 0 9)
do
hackbench 13 process 10000
done
Average values, in seconds:
1) 4.1.45, ACTLR[0] = 0, no spectre variant 2 patches: 114,2666
2) 4.1.45, ACTLR[0] = 1, no spectre variant 2 patches: 114,2952
3) 4.1.45, ACTLR[0] =1 , spectre variant 2 patches: 115,5853
=> 3) is a 1.15% degradation against 1)
4.9.51, ACTLR[0] = 0, no spectre variant 2 patches: 130,7676 4.9.51, ACTLR[0] = 1, no spectre variant 2 patches: 130,6848 4.9.51, ACTLR[0] =1 , spectre variant 2 patches: 132,4274
=> 3) is a 1.26% degradation against 1)
The relative differences between 4.1 and 4.9 appear consistent (with 4.9 being slower for a reason I ignore).
Marc, are there any performance tests/results that you ran that you could share?
Signature · 1 line
Florian
v3 cover 32bit ARM branch predictor hardening
Quoted 44 lines · Florian Fainelli
Quoted 11 lines · Hanjun Guo
Quoted 10 lines · Nishanth Menon
On Mon, Jan 29, 2018 at 5:36 AM, Hanjun Guo <[email protected]> wrote:
[...]Quoted 6 lines
By the way, this patch set just enable branch predictor hardening on arm32 unconditionally, but some of machines (such as wireless network base station) will not be exposed to user to take advantage of variant 2, and those machines will be pretty sensitive for performance, so can we introduce
Kconfigor boot option to disable branch predictor hardening as an option?Quoted 6 lines
Quoted 2 lines
I am curious: Have you seen performance degradation with this series? If yes, is it possible to share the information?
Sorry for the late reply, the performance data for context switch (CFS) is about 6%~12% drop (A9 based machine) for the first around test, but the data is not stable, I need to retest then I will update here.
What tool did you use to measure this? On a Brahma-B15 platform clocked at 1.5Ghz, across kernels 4.1, 4.9 (4.15 in progress as we speak), I measured the following, with two memory configurations, one giving 256MB of usable memory, another giving 3GB of usable memory, results below are only the most extreme 256MB case. This is running 13 groups because the ASID space is 256bits so this should force at least two full ASID generation rollovers (assuming the logic is correct here).
for i in $(seq 0 9)
dohackbench 13 process 10000done
Average values, in seconds:
1) 4.1.45, ACTLR[0] = 0, no spectre variant 2 patches: 114,2666
2) 4.1.45, ACTLR[0] = 1, no spectre variant 2 patches: 114,2952
3) 4.1.45, ACTLR[0] =1 , spectre variant 2 patches: 115,5853=> 3) is a 1.15% degradation against 1)
4.9.51, ACTLR[0] = 0, no spectre variant 2 patches: 130,7676 4.9.51, ACTLR[0] = 1, no spectre variant 2 patches: 130,6848 4.9.51, ACTLR[0] =1 , spectre variant 2 patches: 132,4274
Quoted 7 lines
=> 3) is a 1.26% degradation against 1)
The relative differences between 4.1 and 4.9 appear consistent (with 4.9 being slower for a reason I ignore).
Marc, are there any performance tests/results that you ran that you could share?
None. I usually don't run benchmarks, because they are not representative of a real workload. I urge people to run their own real workload, as it is very unlikely to have hackbench's profile...
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
v3 cover 32bit ARM branch predictor hardening
Quoted 50 lines · Marc Zyngier
Quoted 49 lines · Florian Fainelli
Quoted 11 lines · Hanjun Guo
Quoted 10 lines · Nishanth Menon
On Mon, Jan 29, 2018 at 5:36 AM, Hanjun Guo <[email protected]> wrote:
[...]Quoted 6 lines
By the way, this patch set just enable branch predictor hardening on arm32 unconditionally, but some of machines (such as wireless network base station) will not be exposed to user to take advantage of variant 2, and those machines will be pretty sensitive for performance, so can we introduce
Kconfigor boot option to disable branch predictor hardening as an option?Quoted 6 lines
Quoted 2 lines
I am curious: Have you seen performance degradation with this series? If yes, is it possible to share the information?
Sorry for the late reply, the performance data for context switch (CFS) is about 6%~12% drop (A9 based machine) for the first around test, but the data is not stable, I need to retest then I will update here.
What tool did you use to measure this? On a Brahma-B15 platform clocked at 1.5Ghz, across kernels 4.1, 4.9 (4.15 in progress as we speak), I measured the following, with two memory configurations, one giving 256MB of usable memory, another giving 3GB of usable memory, results below are only the most extreme 256MB case. This is running 13 groups because the ASID space is 256bits so this should force at least two full ASID generation rollovers (assuming the logic is correct here).
for i in $(seq 0 9)
dohackbench 13 process 10000done
Average values, in seconds:
1) 4.1.45, ACTLR[0] = 0, no spectre variant 2 patches: 114,2666
2) 4.1.45, ACTLR[0] = 1, no spectre variant 2 patches: 114,2952
3) 4.1.45, ACTLR[0] =1 , spectre variant 2 patches: 115,5853=> 3) is a 1.15% degradation against 1)
4.9.51, ACTLR[0] = 0, no spectre variant 2 patches: 130,7676 4.9.51, ACTLR[0] = 1, no spectre variant 2 patches: 130,6848 4.9.51, ACTLR[0] =1 , spectre variant 2 patches: 132,4274
=> 3) is a 1.26% degradation against 1)
The relative differences between 4.1 and 4.9 appear consistent (with 4.9 being slower for a reason I ignore).
Quoted 6 lines
Quoted 2 lines
Marc, are there any performance tests/results that you ran that you could share?
None. I usually don't run benchmarks, because they are not representative of a real workload. I urge people to run their own real workload, as it is very unlikely to have hackbench's profile...
Very true.
Out of curiosity (and to prove that the patches and my home-baked firmware fix actually had an effect), I also ran hackbench (of course!) on a Calxeda Midway (4*Cortex-A15, 8GB RAM). Native runs showed only very little degradation, not unlike Florian's numbers. Running hackbench in a KVM guest however showed a bigger impact, which is of course somewhat expected.
Cheers,
Andre.
v3 cover 32bit ARM branch predictor hardening
Quoted 51 lines · Andr? Przywara
Quoted 50 lines · Marc Zyngier
Quoted 49 lines · Florian Fainelli
Quoted 11 lines · Hanjun Guo
Quoted 10 lines · Nishanth Menon
On Mon, Jan 29, 2018 at 5:36 AM, Hanjun Guo <[email protected]> wrote:
[...]Quoted 6 lines
By the way, this patch set just enable branch predictor hardening on arm32 unconditionally, but some of machines (such as wireless network base station) will not be exposed to user to take advantage of variant 2, and those machines will be pretty sensitive for performance, so can we introduce
Kconfigor boot option to disable branch predictor hardening as an option?Quoted 6 lines
Quoted 2 lines
I am curious: Have you seen performance degradation with this series? If yes, is it possible to share the information?
Sorry for the late reply, the performance data for context switch (CFS) is about 6%~12% drop (A9 based machine) for the first around test, but the data is not stable, I need to retest then I will update here.
What tool did you use to measure this? On a Brahma-B15 platform clocked at 1.5Ghz, across kernels 4.1, 4.9 (4.15 in progress as we speak), I measured the following, with two memory configurations, one giving 256MB of usable memory, another giving 3GB of usable memory, results below are only the most extreme 256MB case. This is running 13 groups because the ASID space is 256bits so this should force at least two full ASID generation rollovers (assuming the logic is correct here).
for i in $(seq 0 9)
dohackbench 13 process 10000done
Average values, in seconds:
1) 4.1.45, ACTLR[0] = 0, no spectre variant 2 patches: 114,2666
2) 4.1.45, ACTLR[0] = 1, no spectre variant 2 patches: 114,2952
3) 4.1.45, ACTLR[0] =1 , spectre variant 2 patches: 115,5853=> 3) is a 1.15% degradation against 1)
4.9.51, ACTLR[0] = 0, no spectre variant 2 patches: 130,7676 4.9.51, ACTLR[0] = 1, no spectre variant 2 patches: 130,6848 4.9.51, ACTLR[0] =1 , spectre variant 2 patches: 132,4274
=> 3) is a 1.26% degradation against 1)
The relative differences between 4.1 and 4.9 appear consistent (with 4.9 being slower for a reason I ignore).
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Marc, are there any performance tests/results that you ran that you could share?
None. I usually don't run benchmarks, because they are not representative of a real workload. I urge people to run their own real workload, as it is very unlikely to have hackbench's profile...
Very true.
Of course, but that does not mean you don't want to characterize some sort of worst case scenario ;)
Quoted 5 lines
Out of curiosity (and to prove that the patches and my home-baked firmware fix actually had an effect), I also ran hackbench (of course!) on a Calxeda Midway (4*Cortex-A15, 8GB RAM). Native runs showed only very little degradation, not unlike Florian's numbers.
How did you run hackbench, out of curiosity?
Quoted 2 lines
Running hackbench in a KVM guest however showed a bigger impact, which is of course somewhat expected.
That I have not done, not too interested by KVM, but this is indeed expected.
Signature · 1 line
Florian