Linux 5.10.105
Greg Kroah-Hartman
maintainerI'm announcing the release of the 5.10.105 kernel.
All users of the 5.10 kernel series must upgrade.
The updated 5.10.y git tree can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-5.10.y
and can be browsed at the normal kernel.org git web browser:
https://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary
thanks,
greg k-h
------------
Documentation/admin-guide/hw-vuln/spectre.rst | 48 ++- Documentation/admin-guide/kernel-parameters.txt | 8 Documentation/arm64/cpu-feature-registers.rst | 29 + Documentation/arm64/elf_hwcaps.rst | 12 Makefile | 2 arch/arm/include/asm/assembler.h | 10 arch/arm/include/asm/spectre.h | 32 ++ arch/arm/include/asm/vmlinux.lds.h | 43 ++ arch/arm/kernel/Makefile | 2 arch/arm/kernel/entry-armv.S | 79 ++++- arch/arm/kernel/entry-common.S | 24 + arch/arm/kernel/spectre.c | 71 ++++ arch/arm/kernel/traps.c | 65 +++- arch/arm/mm/Kconfig | 11 arch/arm/mm/proc-v7-bugs.c | 208 +++++++++++-- arch/arm64/Kconfig | 9 arch/arm64/include/asm/assembler.h | 33 ++ arch/arm64/include/asm/cpu.h | 1 arch/arm64/include/asm/cpucaps.h | 3 arch/arm64/include/asm/cpufeature.h | 28 + arch/arm64/include/asm/cputype.h | 22 + arch/arm64/include/asm/fixmap.h | 6 arch/arm64/include/asm/hwcap.h | 3 arch/arm64/include/asm/insn.h | 1 arch/arm64/include/asm/kvm_asm.h | 8 arch/arm64/include/asm/kvm_mmu.h | 3 arch/arm64/include/asm/mmu.h | 6 arch/arm64/include/asm/sections.h | 5 arch/arm64/include/asm/spectre.h | 4 arch/arm64/include/asm/sysreg.h | 18 + arch/arm64/include/asm/vectors.h | 73 ++++ arch/arm64/include/uapi/asm/hwcap.h | 3 arch/arm64/include/uapi/asm/kvm.h | 5 arch/arm64/kernel/cpu_errata.c | 7 arch/arm64/kernel/cpufeature.c | 28 + arch/arm64/kernel/cpuinfo.c | 4 arch/arm64/kernel/entry.S | 213 ++++++++++---- arch/arm64/kernel/proton-pack.c | 359 +++++++++++++++++++++++- arch/arm64/kernel/vmlinux.lds.S | 2 arch/arm64/kvm/arm.c | 3 arch/arm64/kvm/hyp/hyp-entry.S | 4 arch/arm64/kvm/hyp/smccc_wa.S | 75 +++++ arch/arm64/kvm/hyp/vhe/switch.c | 9 arch/arm64/kvm/hypercalls.c | 12 arch/arm64/kvm/psci.c | 18 + arch/arm64/kvm/sys_regs.c | 2 arch/arm64/mm/mmu.c | 12 arch/x86/include/asm/cpufeatures.h | 2 arch/x86/include/asm/nospec-branch.h | 16 - arch/x86/kernel/cpu/bugs.c | 205 ++++++++++--- drivers/acpi/ec.c | 10 drivers/acpi/sleep.c | 14 drivers/block/xen-blkfront.c | 63 ++-- drivers/net/xen-netfront.c | 54 ++- drivers/scsi/xen-scsifront.c | 3 drivers/xen/gntalloc.c | 25 - drivers/xen/grant-table.c | 71 ++-- drivers/xen/pvcalls-front.c | 8 drivers/xen/xenbus/xenbus_client.c | 24 - include/linux/arm-smccc.h | 5 include/linux/bpf.h | 12 include/xen/grant_table.h | 19 + kernel/sysctl.c | 7 net/9p/trans_xen.c | 14 tools/arch/x86/include/asm/cpufeatures.h | 2 65 files changed, 1824 insertions(+), 353 deletions(-)
Anshuman Khandual (2):
arm64: Add Cortex-X2 CPU part definition
arm64: Add Cortex-A510 CPU part definition
Emmanuel Gil Peyrot (1):
ARM: fix build error when BPF_SYSCALL is disabled
Greg Kroah-Hartman (2):
Revert "ACPI: PM: s2idle: Cancel wakeup before dispatching EC GPE" Linux 5.10.105
Hector Martin (1):
arm64: cputype: Add CPU implementor & types for the Apple M1 cores
James Morse (20):
arm64: entry.S: Add ventry overflow sanity checks
arm64: spectre: Rename spectre_v4_patch_fw_mitigation_conduit
arm64: entry: Make the trampoline cleanup optional
arm64: entry: Free up another register on kpti's tramp_exit path
arm64: entry: Move the trampoline data page before the text page
arm64: entry: Allow tramp_alias to access symbols after the 4K boundary
arm64: entry: Don't assume tramp_vectors is the start of the vectors
arm64: entry: Move trampoline macros out of ifdef'd section
arm64: entry: Make the kpti trampoline's kpti sequence optional
arm64: entry: Allow the trampoline text to occupy multiple pages
arm64: entry: Add non-kpti __bp_harden_el1_vectors for mitigations
arm64: entry: Add vectors that have the bhb mitigation sequences
arm64: entry: Add macro for reading symbol addresses from the trampoline
arm64: Add percpu vectors for EL1
arm64: proton-pack: Report Spectre-BHB vulnerabilities as part of Spectre-v2
KVM: arm64: Allow indirect vectors to be used without SPECTRE_V3A
arm64: Mitigate spectre style branch history side channels
KVM: arm64: Allow SMCCC_ARCH_WORKAROUND_3 to be discovered and migrated
arm64: Use the clearbhb instruction in mitigations
arm64: proton-pack: Include unprivileged eBPF status in Spectre v2 mitigation reporting
Joey Gouly (3):
arm64: add ID_AA64ISAR2_EL1 sys register
arm64: cpufeature: add HWCAP for FEAT_AFP
arm64: cpufeature: add HWCAP for FEAT_RPRES
Josh Poimboeuf (3):
x86/speculation: Include unprivileged eBPF status in Spectre v2 mitigation reporting x86/speculation: Warn about Spectre v2 LFENCE mitigation x86/speculation: Warn about eIBRS + LFENCE + Unprivileged eBPF + SMT
Juergen Gross (11):
xen/xenbus: don't let xenbus_grant_ring() remove grants in error case xen/grant-table: add gnttab_try_end_foreign_access() xen/blkfront: don't use gnttab_query_foreign_access() for mapped status xen/netfront: don't use gnttab_query_foreign_access() for mapped status xen/scsifront: don't use gnttab_query_foreign_access() for mapped status xen/gntalloc: don't use gnttab_query_foreign_access()
xen: remove gnttab_query_foreign_access()
xen/9p: use alloc/free_pages_exact()
xen/pvcalls: use alloc/free_pages_exact()
xen/gnttab: fix gnttab_end_foreign_access() without page specified xen/netfront: react properly to failing gnttab_end_foreign_access_ref()
Kim Phillips (2):
x86/speculation: Use generic retpoline by default on AMD x86/speculation: Update link to AMD speculation whitepaper
Marc Zyngier (1):
arm64: Add HWCAP for self-synchronising virtual counter
Nathan Chancellor (1):
ARM: Do not use NOCROSSREFS directive with ld.lld
Peter Zijlstra (3):
x86,bugs: Unconditionally allow spectre_v2=retpoline,amd x86/speculation: Add eIBRS + Retpoline options Documentation/hw-vuln: Update spectre doc
Peter Zijlstra (Intel) (1):
x86/speculation: Rename RETPOLINE_AMD to RETPOLINE_LFENCE
Russell King (Oracle) (7):
ARM: report Spectre v2 status through sysfs
ARM: early traps initialisation
ARM: use LOADADDR() to get load address of sections
ARM: Spectre-BHB workaround
ARM: include unprivileged BPF status in Spectre V2 reporting
ARM: fix co-processor register typo
ARM: fix build warning in proc-v7-bugs.c
Suzuki K Poulose (1):
arm64: Add Neoverse-N2, Cortex-A710 CPU part definition
Greg Kroah-Hartman
maintainerDocumentation/admin-guide/hw-vuln/spectre.rst +32 -16
@@ -60,8 +60,8 @@ privileged data touched during the speculative execution. Spectre variant 1 attacks take advantage of speculative execution of conditional branches, while Spectre variant 2 attacks use speculative execution of indirect branches to leak privileged memory. -See :ref:`[1] <spec_ref1>` :ref:`[5] <spec_ref5>` :ref:`[7] <spec_ref7>` -:ref:`[10] <spec_ref10>` :ref:`[11] <spec_ref11>`. +See :ref:`[1] <spec_ref1>` :ref:`[5] <spec_ref5>` :ref:`[6] <spec_ref6>` +:ref:`[7] <spec_ref7>` :ref:`[10] <spec_ref10>` :ref:`[11] <spec_ref11>`. Spectre variant 1 (Bounds Check Bypass) ---------------------------------------
@@ -131,6 +131,19 @@ steer its indirect branch speculations to gadget code, and measure the speculative execution's side effects left in level 1 cache to infer the victim's data. +Yet another variant 2 attack vector is for the attacker to poison the +Branch History Buffer (BHB) to speculatively steer an indirect branch +to a specific Branch Target Buffer (BTB) entry, even if the entry isn't +associated with the source address of the indirect branch. Specifically, +the BHB might be shared across privilege levels even in the presence of +Enhanced IBRS. + +Currently the only known real-world BHB attack vector is via +unprivileged eBPF. Therefore, it's highly recommended to not enable +unprivileged eBPF, especially when eIBRS is used (without retpolines). +For a full mitigation against BHB attacks, it's recommended to use +retpolines (or eIBRS combined with retpolines). + Attack scenarios ----------------
@@ -364,13 +377,15 @@ The possible values in this file are: - Kernel status: - ==================================== ================================= - 'Not affected' The processor is not vulnerable - 'Vulnerable' Vulnerable, no mitigation - 'Mitigation: Full generic retpoline' Software-focused mitigation - 'Mitigation: Full AMD retpoline' AMD-specific software mitigation - 'Mitigation: Enhanced IBRS' Hardware-focused mitigation - ==================================== ================================= + ======================================== ================================= + 'Not affected' The processor is not vulnerable + 'Mitigation: None' Vulnerable, no mitigation + 'Mitigation: Retpolines' Use Retpoline thunks + 'Mitigation: LFENCE' Use LFENCE instructions + 'Mitigation: Enhanced IBRS' Hardware-focused mitigation + 'Mitigation: Enhanced IBRS + Retpolines' Hardware-focused + Retpolines + 'Mitigation: Enhanced IBRS + LFENCE' Hardware-focused + LFENCE + ======================================== ================================= - Firmware status: Show if Indirect Branch Restricted Speculation (IBRS) is used to protect against Spectre variant 2 attacks when calling firmware (x86 only).
@@ -584,12 +599,13 @@ kernel command line. Specific mitigations can also be selected manually: - retpoline - replace indirect branches - retpoline,generic - google's original retpoline - retpoline,amd - AMD-specific minimal thunk + retpoline auto pick between generic,lfence + retpoline,generic Retpolines + retpoline,lfence LFENCE; indirect branch + retpoline,amd alias for retpoline,lfence + eibrs enhanced IBRS + eibrs,retpoline enhanced IBRS + Retpolines + eibrs,lfence enhanced IBRS + LFENCE Not specifying this option is equivalent to spectre_v2=auto.
@@ -730,7 +746,7 @@ AMD white papers: .. _spec_ref6: -[6] `Software techniques for managing speculation on AMD processors <https://developer.amd.com/wp-content/resources/90343-B_SoftwareTechniquesforManagingSpeculation_WP_7-18Update_FNL.pdf>`_. +[6] `Software techniques for managing speculation on AMD processors <https://developer.amd.com/wp-content/resources/Managing-Speculation-on-AMD-Processors.pdf>`_. ARM white papers:
Documentation/admin-guide/kernel-parameters.txt +6 -2
@@ -4957,8 +4957,12 @@ Specific mitigations can also be selected manually: retpoline - replace indirect branches - retpoline,generic - google's original retpoline - retpoline,amd - AMD-specific minimal thunk + retpoline,generic - Retpolines + retpoline,lfence - LFENCE; indirect branch + retpoline,amd - alias for retpoline,lfence + eibrs - enhanced IBRS + eibrs,retpoline - enhanced IBRS + Retpolines + eibrs,lfence - enhanced IBRS + LFENCE Not specifying this option is equivalent to spectre_v2=auto.
Documentation/arm64/cpu-feature-registers.rst +27 -2
@@ -235,7 +235,15 @@ infrastructure: | DPB | [3-0] | y | +------------------------------+---------+---------+ - 6) ID_AA64MMFR2_EL1 - Memory model feature register 2 + 6) ID_AA64MMFR0_EL1 - Memory model feature register 0 + + +------------------------------+---------+---------+ + | Name | bits | visible | + +------------------------------+---------+---------+ + | ECV | [63-60] | y | + +------------------------------+---------+---------+ + + 7) ID_AA64MMFR2_EL1 - Memory model feature register 2 +------------------------------+---------+---------+ | Name | bits | visible |
@@ -243,7 +251,7 @@ infrastructure: | AT | [35-32] | y | +------------------------------+---------+---------+ - 7) ID_AA64ZFR0_EL1 - SVE feature ID register 0 + 8) ID_AA64ZFR0_EL1 - SVE feature ID register 0 +------------------------------+---------+---------+ | Name | bits | visible |
@@ -267,6 +275,23 @@ infrastructure: | SVEVer | [3-0] | y | +------------------------------+---------+---------+ + 8) ID_AA64MMFR1_EL1 - Memory model feature register 1 + + +------------------------------+---------+---------+ + | Name | bits | visible | + +------------------------------+---------+---------+ + | AFP | [47-44] | y | + +------------------------------+---------+---------+ + + 9) ID_AA64ISAR2_EL1 - Instruction set attribute register 2 + + +------------------------------+---------+---------+ + | Name | bits | visible | + +------------------------------+---------+---------+ + | RPRES | [7-4] | y | + +------------------------------+---------+---------+ + + Appendix I: Example -------------------
Documentation/arm64/elf_hwcaps.rst +12 -0
@@ -245,6 +245,18 @@ HWCAP2_MTE Functionality implied by ID_AA64PFR1_EL1.MTE == 0b0010, as described by Documentation/arm64/memory-tagging-extension.rst. +HWCAP2_ECV + + Functionality implied by ID_AA64MMFR0_EL1.ECV == 0b0001. + +HWCAP2_AFP + + Functionality implied by ID_AA64MFR1_EL1.AFP == 0b0001. + +HWCAP2_RPRES + + Functionality implied by ID_AA64ISAR2_EL1.RPRES == 0b0001. + 4. Unused AT_HWCAP bits -----------------------
Makefile +1 -1
@@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 VERSION = 5 PATCHLEVEL = 10 -SUBLEVEL = 104 +SUBLEVEL = 105 EXTRAVERSION = NAME = Dare mighty things
arch/arm/include/asm/assembler.h +10 -0
@@ -107,6 +107,16 @@ .endm#endif +#if __LINUX_ARM_ARCH__ <7 + .macro dsb, args + mcr p15, 0, r0, c7, c10, 4 + .endm + + .macro isb, args + mcr p15, 0, r0, c7, c5, 4 + .endm +#endif + .macro asm_trace_hardirqs_off, save=1#if defined(CONFIG_TRACE_IRQFLAGS ) .if \save
arch/arm/include/asm/spectre.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __ASM_SPECTRE_H +#define __ASM_SPECTRE_H + +enum { +SPECTRE_UNAFFECTED , +SPECTRE_MITIGATED , +SPECTRE_VULNERABLE , +}; + +enum { + __SPECTRE_V2_METHOD_BPIALL, + __SPECTRE_V2_METHOD_ICIALLU, + __SPECTRE_V2_METHOD_SMC, + __SPECTRE_V2_METHOD_HVC, + __SPECTRE_V2_METHOD_LOOP8, +}; + +enum { +SPECTRE_V2_METHOD_BPIALL =BIT (__SPECTRE_V2_METHOD_BPIALL), +SPECTRE_V2_METHOD_ICIALLU =BIT (__SPECTRE_V2_METHOD_ICIALLU), +SPECTRE_V2_METHOD_SMC =BIT (__SPECTRE_V2_METHOD_SMC), +SPECTRE_V2_METHOD_HVC =BIT (__SPECTRE_V2_METHOD_HVC), +SPECTRE_V2_METHOD_LOOP8 =BIT (__SPECTRE_V2_METHOD_LOOP8), +}; + +void spectre_v2_update_state (unsigned int state,unsigned int methods); + +int spectre_bhb_update_vectors (unsigned int method); + +#endif
arch/arm/include/asm/vmlinux.lds.h +34 -9
59 lines not on this page · raw message
arch/arm/kernel/Makefile +2 -0
@@ -106,4 +106,6 @@ endif obj-$(CONFIG_HAVE_ARM_SMCCC) += smccc-call.o +obj-$(CONFIG_GENERIC_CPU_VULNERABILITIES) += spectre.o + extra-y := $(head-y) vmlinux.lds
arch/arm/kernel/entry-armv.S +73 -6
111 lines not on this page · raw message
arch/arm/kernel/entry-common.S +24 -0
@@ -162,6 +162,29 @@ ENDPROC(ret_from_fork) *----------------------------------------------------------------------------- */ + .align 5 +#ifdef CONFIG_HARDEN_BRANCH_HISTORY +ENTRY(vector_bhb_loop8_swi) + sub sp, sp, #PT_REGS_SIZE + stmia sp, {r0 - r12} + mov r8, #8 +1: b 2f +2: subs r8, r8, #1 + bne 1b + dsb + isb + b 3f +ENDPROC(vector_bhb_loop8_swi) + + .align 5 +ENTRY(vector_bhb_bpiall_swi) + sub sp, sp, #PT_REGS_SIZE + stmia sp, {r0 - r12} + mcr p15, 0, r8, c7, c5, 6 @ BPIALL + isb + b 3f +ENDPROC(vector_bhb_bpiall_swi) +#endif .align 5 ENTRY(vector_swi) #ifdef CONFIG_CPU_V7M
@@ -169,6 +192,7 @@ ENTRY(vector_swi) #else sub sp, sp, #PT_REGS_SIZE stmia sp, {r0 - r12} @ Calling r0 - r12 +3: ARM( add r8, sp, #S_PC ) ARM( stmdb r8, {sp, lr}^ ) @ Calling sp, lr THUMB( mov r8, sp )
arch/arm/kernel/spectre.c new file +71 -0
71 lines not on this page · raw message
arch/arm/kernel/traps.c +59 -6
92 lines not on this page · raw message
arch/arm/mm/Kconfig +11 -0
@@ -833,6 +833,7 @@ config CPU_BPREDICT_DISABLE config CPU_SPECTRE bool + select GENERIC_CPU_VULNERABILITIES config HARDEN_BRANCH_PREDICTOR bool "Harden the branch predictor against aliasing attacks" if EXPERT
@@ -853,6 +854,16 @@ config HARDEN_BRANCH_PREDICTOR If unsure, say Y. +config HARDEN_BRANCH_HISTORY + bool "Harden Spectre style attacks against branch history" if EXPERT + depends on CPU_SPECTRE + default y + help + Speculation attacks against some high-performance processors can + make use of branch history to influence future speculation. When + taking an exception, a sequence of branches overwrites the branch + history, or branch history is invalidated. + config TLS_REG_EMUL bool select NEED_KUSER_HELPERS
arch/arm/mm/proc-v7-bugs.c +174 -34
280 lines not on this page · raw message
arch/arm64/Kconfig +9 -0
@@ -1184,6 +1184,15 @@ config UNMAP_KERNEL_AT_EL0 If unsure, say Y. +config MITIGATE_SPECTRE_BRANCH_HISTORY + bool "Mitigate Spectre style attacks against branch history" if EXPERT + default y + help + Speculation attacks against some high-performance processors can + make use of branch history to influence future speculation. + When taking an exception from user-space, a sequence of branches + or a firmware call overwrites the branch history. + config RODATA_FULL_DEFAULT_ENABLED bool "Apply r/o permissions of VM areas also to their linear aliases" default y
arch/arm64/include/asm/assembler.h +33 -0
@@ -97,6 +97,13 @@ hint #20 .endm +/* +* Clear Branch History instruction +*/ + .macro clearbhb + hint #22 + .endm +/* * Speculation barrier */
@@ -795,4 +802,30 @@ USER(\label, ic ivau, \tmp2) // invalidate I line PoU #endif /* GNU_PROPERTY_AARCH64_FEATURE_1_DEFAULT */ + .macro __mitigate_spectre_bhb_loop tmp +#ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY +alternative_cb spectre_bhb_patch_loop_iter + mov \tmp, #32 // Patched to correct the immediate +alternative_cb_end +.Lspectre_bhb_loop\@: + b . + 4 + subs \tmp, \tmp, #1 + b.ne .Lspectre_bhb_loop\@ + sb +#endif /* CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ + .endm + + /* Save/restores x0-x3 to the stack */ + .macro __mitigate_spectre_bhb_fw +#ifdef CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY + stp x0, x1, [sp, #-16]! + stp x2, x3, [sp, #-16]! + mov w0, #ARM_SMCCC_ARCH_WORKAROUND_3 +alternative_cb smccc_patch_fw_mitigation_conduit + nop // Patched to SMC/HVC #0 +alternative_cb_end + ldp x2, x3, [sp], #16 + ldp x0, x1, [sp], #16 +#endif /* CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ + .endm #endif /* __ASM_ASSEMBLER_H */
arch/arm64/include/asm/cpu.h +1 -0
@@ -25,6 +25,7 @@ struct cpuinfo_arm64 {u64 reg_id_aa64dfr1;u64 reg_id_aa64isar0;u64 reg_id_aa64isar1; +u64 reg_id_aa64isar2;u64 reg_id_aa64mmfr0;u64 reg_id_aa64mmfr1;u64 reg_id_aa64mmfr2;
arch/arm64/include/asm/cpucaps.h +2 -1
@@ -66,7 +66,8 @@#define ARM64_HAS_TLB_RANGE 56#define ARM64_MTE 57#define ARM64_WORKAROUND_1508412 58 +#define ARM64_SPECTRE_BHB 59 -#define ARM64_NCAPS 59 +#define ARM64_NCAPS 60#endif /* __ASM_CPUCAPS_H */
arch/arm64/include/asm/cpufeature.h +28 -0
@@ -606,6 +606,34 @@ static inline bool cpu_supports_mixed_endian_el0(void)return id_aa64mmfr0_mixed_endian_el0 (read_cpuid (ID_AA64MMFR0_EL1 )); } +static inline bool supports_csv2p3 (int scope) +{ +u64 pfr0; +u8 csv2_val; + +if (scope ==SCOPE_LOCAL_CPU ) + pfr0 =read_sysreg_s (SYS_ID_AA64PFR0_EL1 ); +else + pfr0 =read_sanitised_ftr_reg (SYS_ID_AA64PFR0_EL1 ); + + csv2_val =cpuid_feature_extract_unsigned_field (pfr0, +ID_AA64PFR0_CSV2_SHIFT ); +return csv2_val ==3 ; +} + +static inline bool supports_clearbhb (int scope) +{ +u64 isar2; + +if (scope ==SCOPE_LOCAL_CPU ) + isar2 =read_sysreg_s (SYS_ID_AA64ISAR2_EL1 ); +else + isar2 =read_sanitised_ftr_reg (SYS_ID_AA64ISAR2_EL1 ); + +return cpuid_feature_extract_unsigned_field (isar2, +ID_AA64ISAR2_CLEARBHB_SHIFT ); +} +static inline bool system_supports_32bit_el0 (void ) {return cpus_have_const_cap (ARM64_HAS_32BIT_EL0 );
arch/arm64/include/asm/cputype.h +22 -0
@@ -59,6 +59,7 @@#define ARM_CPU_IMP_NVIDIA 0x4E#define ARM_CPU_IMP_FUJITSU 0x46#define ARM_CPU_IMP_HISI 0x48 +#define ARM_CPU_IMP_APPLE 0x61#define ARM_CPU_PART_AEM_V8 0xD0F#define ARM_CPU_PART_FOUNDATION 0xD00
@@ -72,6 +73,14 @@#define ARM_CPU_PART_CORTEX_A76 0xD0B#define ARM_CPU_PART_NEOVERSE_N1 0xD0C#define ARM_CPU_PART_CORTEX_A77 0xD0D +#define ARM_CPU_PART_NEOVERSE_V1 0xD40 +#define ARM_CPU_PART_CORTEX_A78 0xD41 +#define ARM_CPU_PART_CORTEX_X1 0xD44 +#define ARM_CPU_PART_CORTEX_A510 0xD46 +#define ARM_CPU_PART_CORTEX_A710 0xD47 +#define ARM_CPU_PART_CORTEX_X2 0xD48 +#define ARM_CPU_PART_NEOVERSE_N2 0xD49 +#define ARM_CPU_PART_CORTEX_A78C 0xD4B#define APM_CPU_PART_POTENZA 0x000
@@ -99,6 +108,9 @@#define HISI_CPU_PART_TSV110 0xD01 +#define APPLE_CPU_PART_M1_ICESTORM 0x022 +#define APPLE_CPU_PART_M1_FIRESTORM 0x023 +#define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)#define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
@@ -109,6 +121,14 @@#define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A76)#define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N1)#define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A77) +#define MIDR_NEOVERSE_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V1) +#define MIDR_CORTEX_A78 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78) +#define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1) +#define MIDR_CORTEX_A510 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A510) +#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710) +#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2) +#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2) +#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
@@ -127,6 +147,8 @@#define MIDR_NVIDIA_CARMEL MIDR_CPU_MODEL(ARM_CPU_IMP_NVIDIA, NVIDIA_CPU_PART_CARMEL)#define MIDR_FUJITSU_A64FX MIDR_CPU_MODEL(ARM_CPU_IMP_FUJITSU, FUJITSU_CPU_PART_A64FX)#define MIDR_HISI_TSV110 MIDR_CPU_MODEL(ARM_CPU_IMP_HISI, HISI_CPU_PART_TSV110) +#define MIDR_APPLE_M1_ICESTORM MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M1_ICESTORM) +#define MIDR_APPLE_M1_FIRESTORM MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M1_FIRESTORM)/* Fujitsu Erratum 010001 affects A64FX 1.0 and 1.1, (v0r0 and v1r0) */ #define MIDR_FUJITSU_ERRATUM_010001 MIDR_FUJITSU_A64FX
arch/arm64/include/asm/fixmap.h +4 -2
@@ -62,9 +62,11 @@ enum fixed_addresses {#endif /* CONFIG_ACPI_APEI_GHES */ #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +FIX_ENTRY_TRAMP_TEXT3 , +FIX_ENTRY_TRAMP_TEXT2 , +FIX_ENTRY_TRAMP_TEXT1 ,FIX_ENTRY_TRAMP_DATA , -FIX_ENTRY_TRAMP_TEXT , -#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT)) +#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT1))#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ __end_of_permanent_fixed_addresses,
arch/arm64/include/asm/hwcap.h +3 -0
@@ -105,6 +105,9 @@#define KERNEL_HWCAP_RNG __khwcap2_feature(RNG)#define KERNEL_HWCAP_BTI __khwcap2_feature(BTI)#define KERNEL_HWCAP_MTE __khwcap2_feature(MTE) +#define KERNEL_HWCAP_ECV __khwcap2_feature(ECV) +#define KERNEL_HWCAP_AFP __khwcap2_feature(AFP) +#define KERNEL_HWCAP_RPRES __khwcap2_feature(RPRES)/* * This yields a mask that user programs can use to figure out what
arch/arm64/include/asm/insn.h +1 -0
@@ -65,6 +65,7 @@ enum aarch64_insn_hint_cr_op {AARCH64_INSN_HINT_PSB =0x11 <<5 ,AARCH64_INSN_HINT_TSB =0x12 <<5 ,AARCH64_INSN_HINT_CSDB =0x14 <<5 , +AARCH64_INSN_HINT_CLEARBHB =0x16 <<5 ,AARCH64_INSN_HINT_BTI =0x20 <<5 , AARCH64_INSN_HINT_BTIC = 0x22 << 5,
arch/arm64/include/asm/kvm_asm.h +8 -0
@@ -35,6 +35,9 @@#define KVM_VECTOR_PREAMBLE (2 * AARCH64_INSN_SIZE)#define __SMCCC_WORKAROUND_1_SMC_SZ 36 +#define __SMCCC_WORKAROUND_3_SMC_SZ 36 +#define __SPECTRE_BHB_LOOP_SZ 44 +#define __SPECTRE_BHB_CLEARBHB_SZ 12#define KVM_HOST_SMCCC_ID (id) \ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
@@ -199,6 +202,11 @@ extern void __vgic_v3_init_lrs(void);extern u32 __kvm_get_mdcr_el2 (void );extern char __smccc_workaround_1_smc[__SMCCC_WORKAROUND_1_SMC_SZ]; +extern char __smccc_workaround_3_smc[__SMCCC_WORKAROUND_3_SMC_SZ]; +extern char __spectre_bhb_loop_k8[__SPECTRE_BHB_LOOP_SZ]; +extern char __spectre_bhb_loop_k24[__SPECTRE_BHB_LOOP_SZ]; +extern char __spectre_bhb_loop_k32[__SPECTRE_BHB_LOOP_SZ]; +extern char __spectre_bhb_clearbhb[__SPECTRE_BHB_LOOP_SZ];/* * Obtain the PC-relative address of a kernel symbol
arch/arm64/include/asm/kvm_mmu.h +2 -1
@@ -237,7 +237,8 @@ static inline void *kvm_get_hyp_vector(void)void *vect =kern_hyp_va (kvm_ksym_ref (__kvm_hyp_vector));int slot =-1 ; -if (cpus_have_const_cap (ARM64_SPECTRE_V2 ) && data->fn) { +if ((cpus_have_const_cap (ARM64_SPECTRE_V2 ) || +cpus_have_const_cap (ARM64_SPECTRE_BHB )) && data->template_start) { vect =kern_hyp_va (kvm_ksym_ref (__bp_harden_hyp_vecs)); slot = data->hyp_vectors_slot; }
arch/arm64/include/asm/mmu.h +6 -0
@@ -67,6 +67,12 @@ typedef void (*bp_hardening_cb_t)(void);struct bp_hardening_data {int hyp_vectors_slot;bp_hardening_cb_t fn; + +/* +* template_start is only used by the BHB mitigation to identify the +* hyp_vectors_slot sequence. +*/ +const char *template_start; }; DECLARE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data);
arch/arm64/include/asm/sections.h +5 -0
@@ -19,4 +19,9 @@ extern char __irqentry_text_start[], __irqentry_text_end[];extern char __mmuoff_data_start[], __mmuoff_data_end[];extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +static inline size_t entry_tramp_text_size (void ) +{ +return __entry_tramp_text_end - __entry_tramp_text_start; +} +#endif /* __ASM_SECTIONS_H */
arch/arm64/include/asm/spectre.h +4 -0
@@ -29,4 +29,8 @@ bool has_spectre_v4(const struct arm64_cpu_capabilities *cap, int scope);void spectre_v4_enable_mitigation (const struct arm64_cpu_capabilities *__unused);void spectre_v4_enable_task_mitigation (struct task_struct *tsk); +enum mitigation_state arm64_get_spectre_bhb_state (void ); +bool is_spectre_bhb_affected (const struct arm64_cpu_capabilities *entry,int scope); +u8 spectre_bhb_loop_affected (int scope); +void spectre_bhb_enable_mitigation (const struct arm64_cpu_capabilities *__unused);#endif /* __ASM_SPECTRE_H */
arch/arm64/include/asm/sysreg.h +18 -0
@@ -175,6 +175,7 @@#define SYS_ID_AA64ISAR0_EL1 sys_reg(3, 0, 0, 6, 0)#define SYS_ID_AA64ISAR1_EL1 sys_reg(3, 0, 0, 6, 1) +#define SYS_ID_AA64ISAR2_EL1 sys_reg(3, 0, 0, 6, 2)#define SYS_ID_AA64MMFR0_EL1 sys_reg(3, 0, 0, 7, 0)#define SYS_ID_AA64MMFR1_EL1 sys_reg(3, 0, 0, 7, 1)
@@ -687,6 +688,21 @@#define ID_AA64ISAR1_GPI_NI 0x0#define ID_AA64ISAR1_GPI_IMP_DEF 0x1 +/* id_aa64isar2 */ +#define ID_AA64ISAR2_CLEARBHB_SHIFT 28 +#define ID_AA64ISAR2_RPRES_SHIFT 4 +#define ID_AA64ISAR2_WFXT_SHIFT 0 + +#define ID_AA64ISAR2_RPRES_8BIT 0x0 +#define ID_AA64ISAR2_RPRES_12BIT 0x1 +/* +* Value 0x1 has been removed from the architecture, and is +* reserved, but has not yet been removed from the ARM ARM +* as of ARM DDI 0487G.b. +*/ +#define ID_AA64ISAR2_WFXT_NI 0x0 +#define ID_AA64ISAR2_WFXT_SUPPORTED 0x2 +/* id_aa64pfr0 */ #define ID_AA64PFR0_CSV3_SHIFT 60#define ID_AA64PFR0_CSV2_SHIFT 56
@@ -786,6 +802,8 @@#endif /* id_aa64mmfr1 */ +#define ID_AA64MMFR1_ECBHB_SHIFT 60 +#define ID_AA64MMFR1_AFP_SHIFT 44#define ID_AA64MMFR1_ETS_SHIFT 36#define ID_AA64MMFR1_TWED_SHIFT 32#define ID_AA64MMFR1_XNX_SHIFT 28
arch/arm64/include/asm/vectors.h new file +73 -0
73 lines not on this page · raw message
arch/arm64/include/uapi/asm/hwcap.h +3 -0
@@ -75,5 +75,8 @@#define HWCAP2_RNG (1 << 16)#define HWCAP2_BTI (1 << 17)#define HWCAP2_MTE (1 << 18) +#define HWCAP2_ECV (1 << 19) +#define HWCAP2_AFP (1 << 20) +#define HWCAP2_RPRES (1 << 21)#endif /* _UAPI__ASM_HWCAP_H */
arch/arm64/include/uapi/asm/kvm.h +5 -0
@@ -273,6 +273,11 @@ struct kvm_vcpu_events {#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_NOT_REQUIRED 3#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_ENABLED (1U << 4) +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 KVM_REG_ARM_FW_REG(3) +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_AVAIL 0 +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_AVAIL 1 +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_REQUIRED 2 +/* SVE registers */ #define KVM_REG_ARM64_SVE (0x15 << KVM_REG_ARM_COPROC_SHIFT)
arch/arm64/kernel/cpu_errata.c +7 -0
@@ -473,6 +473,13 @@ const struct arm64_cpu_capabilities arm64_errata[] = { .matches = has_spectre_v4, .cpu_enable = spectre_v4_enable_mitigation, }, + { + .desc = "Spectre-BHB", + .capability = ARM64_SPECTRE_BHB, + .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, + .matches = is_spectre_bhb_affected, + .cpu_enable = spectre_bhb_enable_mitigation, + }, #ifdef CONFIG_ARM64_ERRATUM_1418040 { .desc = "ARM erratum 1418040",
arch/arm64/kernel/cpufeature.c +27 -1
@@ -65,11 +65,13 @@#include <linux/bsearch.h> #include <linux/cpumask.h> #include <linux/crash_dump.h> +#include <linux/percpu.h> #include <linux/sort.h> #include <linux/stop_machine.h> #include <linux/types.h> #include <linux/mm.h> #include <linux/cpu.h> +#include <asm/cpu.h> #include <asm/cpufeature.h> #include <asm/cpu_ops.h>
@@ -79,6 +81,7 @@#include <asm/processor.h> #include <asm/sysreg.h> #include <asm/traps.h> +#include <asm/vectors.h> #include <asm/virt.h> /* Kernel representation of AT_HWCAP and AT_HWCAP2 */
@@ -104,6 +107,8 @@ DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);bool arm64_use_ng_mappings = false;EXPORT_SYMBOL (arm64_use_ng_mappings); +DEFINE_PER_CPU_READ_MOSTLY(const char *, this_cpu_vector) = vectors; +/* * Flag to indicate if we have computed the system wide * capabilities based on the boot time active CPUs. This
@@ -205,6 +210,12 @@ static const struct arm64_ftr_bits ftr_id_aa64isar1[] = { ARM64_FTR_END, }; +static const struct arm64_ftr_bits ftr_id_aa64isar2[] = { +ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_HIGHER_SAFE ,ID_AA64ISAR2_CLEARBHB_SHIFT ,4 ,0 ), +ARM64_FTR_BITS (FTR_VISIBLE ,FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64ISAR2_RPRES_SHIFT ,4 ,0 ), +ARM64_FTR_END , +}; +static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {ARM64_FTR_BITS (FTR_HIDDEN ,FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64PFR0_CSV3_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64PFR0_CSV2_SHIFT ,4 ,0 ),
@@ -259,7 +270,7 @@ static const struct arm64_ftr_bits ftr_id_aa64zfr0[] = { };static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = { -ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR0_ECV_SHIFT ,4 ,0 ), +ARM64_FTR_BITS (FTR_VISIBLE ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR0_ECV_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR0_FGT_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR0_EXS_SHIFT ,4 ,0 ),/*
@@ -305,6 +316,7 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = { };static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = { +ARM64_FTR_BITS (FTR_VISIBLE ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR1_AFP_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR1_ETS_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR1_TWED_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR1_XNX_SHIFT ,4 ,0 ),
@@ -596,6 +608,7 @@ static const struct __ftr_reg_entry {/* Op1 = 0, CRn = 0, CRm = 6 */ ARM64_FTR_REG (SYS_ID_AA64ISAR0_EL1 , ftr_id_aa64isar0),ARM64_FTR_REG (SYS_ID_AA64ISAR1_EL1 , ftr_id_aa64isar1), +ARM64_FTR_REG (SYS_ID_AA64ISAR2_EL1 , ftr_id_aa64isar2),/* Op1 = 0, CRn = 0, CRm = 7 */ ARM64_FTR_REG(SYS_ID_AA64MMFR0_EL1, ftr_id_aa64mmfr0),
@@ -830,6 +843,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)init_cpu_ftr_reg (SYS_ID_AA64DFR1_EL1 , info->reg_id_aa64dfr1);init_cpu_ftr_reg (SYS_ID_AA64ISAR0_EL1 , info->reg_id_aa64isar0);init_cpu_ftr_reg (SYS_ID_AA64ISAR1_EL1 , info->reg_id_aa64isar1); +init_cpu_ftr_reg (SYS_ID_AA64ISAR2_EL1 , info->reg_id_aa64isar2);init_cpu_ftr_reg (SYS_ID_AA64MMFR0_EL1 , info->reg_id_aa64mmfr0);init_cpu_ftr_reg (SYS_ID_AA64MMFR1_EL1 , info->reg_id_aa64mmfr1);init_cpu_ftr_reg (SYS_ID_AA64MMFR2_EL1 , info->reg_id_aa64mmfr2);
@@ -1058,6 +1072,8 @@ void update_cpu_features(int cpu, info->reg_id_aa64isar0, boot->reg_id_aa64isar0); taint |=check_update_ftr_reg (SYS_ID_AA64ISAR1_EL1 , cpu, info->reg_id_aa64isar1, boot->reg_id_aa64isar1); + taint |=check_update_ftr_reg (SYS_ID_AA64ISAR2_EL1 , cpu, + info->reg_id_aa64isar2, boot->reg_id_aa64isar2);/* * Differing PARange support is fine as long as all peripherals and
@@ -1157,6 +1173,7 @@ static u64 __read_sysreg_by_encoding(u32 sys_id)read_sysreg_case (SYS_ID_AA64MMFR2_EL1 );read_sysreg_case (SYS_ID_AA64ISAR0_EL1 );read_sysreg_case (SYS_ID_AA64ISAR1_EL1 ); +read_sysreg_case (SYS_ID_AA64ISAR2_EL1 );read_sysreg_case (SYS_CNTFRQ_EL0 );read_sysreg_case (SYS_CTR_EL0 );
@@ -1402,6 +1419,12 @@ kpti_install_ng_mappings(const struct arm64_cpu_capabilities *__unused)int cpu =smp_processor_id (); +if (__this_cpu_read (this_cpu_vector) == vectors) { +const char *v =arm64_get_bp_hardening_vector (EL1_VECTOR_KPTI ); + +__this_cpu_write (this_cpu_vector, v); + } +/* * We don't need to rewrite the page-tables if either we've done * it already or we have KASLR enabled and therefore have not
@@ -2252,6 +2275,9 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = {#ifdef CONFIG_ARM64_MTE HWCAP_CAP (SYS_ID_AA64PFR1_EL1 ,ID_AA64PFR1_MTE_SHIFT ,FTR_UNSIGNED ,ID_AA64PFR1_MTE ,CAP_HWCAP ,KERNEL_HWCAP_MTE ), #endif /* CONFIG_ARM64_MTE */ +HWCAP_CAP (SYS_ID_AA64MMFR0_EL1 ,ID_AA64MMFR0_ECV_SHIFT ,FTR_UNSIGNED ,1 ,CAP_HWCAP ,KERNEL_HWCAP_ECV ), +HWCAP_CAP (SYS_ID_AA64MMFR1_EL1 ,ID_AA64MMFR1_AFP_SHIFT ,FTR_UNSIGNED ,1 ,CAP_HWCAP ,KERNEL_HWCAP_AFP ), + HWCAP_CAP(SYS_ID_AA64ISAR2_EL1, ID_AA64ISAR2_RPRES_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_RPRES), {}, };
arch/arm64/kernel/cpuinfo.c +4 -0
@@ -94,6 +94,9 @@ static const char *const hwcap_str[] = { [KERNEL_HWCAP_RNG] = "rng", [KERNEL_HWCAP_BTI] = "bti", [KERNEL_HWCAP_MTE] = "mte", + [KERNEL_HWCAP_ECV] = "ecv", + [KERNEL_HWCAP_AFP] = "afp", + [KERNEL_HWCAP_RPRES] = "rpres", };#ifdef CONFIG_COMPAT
@@ -364,6 +367,7 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info) info->reg_id_aa64dfr1 =read_cpuid (ID_AA64DFR1_EL1 ); info->reg_id_aa64isar0 =read_cpuid (ID_AA64ISAR0_EL1 ); info->reg_id_aa64isar1 =read_cpuid (ID_AA64ISAR1_EL1 ); + info->reg_id_aa64isar2 =read_cpuid (ID_AA64ISAR2_EL1 ); info->reg_id_aa64mmfr0 =read_cpuid (ID_AA64MMFR0_EL1 ); info->reg_id_aa64mmfr1 =read_cpuid (ID_AA64MMFR1_EL1 ); info->reg_id_aa64mmfr2 =read_cpuid (ID_AA64MMFR2_EL1 );
arch/arm64/kernel/entry.S +157 -56
329 lines not on this page · raw message
arch/arm64/kernel/proton-pack.c +353 -6
408 lines not on this page · raw message
arch/arm64/kernel/vmlinux.lds.S +1 -1
@@ -299,7 +299,7 @@ ASSERT(__hibernate_exit_text_end - (__hibernate_exit_text_start & ~(SZ_4K - 1)) <= SZ_4K, "Hibernate exit text too big or misaligned") #endif #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 -ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) == PAGE_SIZE, +ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) <= 3*PAGE_SIZE, "Entry trampoline text too big") #endif /*
arch/arm64/kvm/arm.c +2 -1
@@ -1337,7 +1337,8 @@ static int kvm_map_vectors(void)* !SV2 + HEL2 -> allocate one vector slot and use exec mapping * SV2 + HEL2 -> use hardened vectors and use exec mapping */ -if (cpus_have_const_cap (ARM64_SPECTRE_V2 )) { +if (cpus_have_const_cap (ARM64_SPECTRE_V2 ) || +cpus_have_const_cap (ARM64_SPECTRE_BHB )) { __kvm_bp_vect_base =kvm_ksym_ref (__bp_harden_hyp_vecs); __kvm_bp_vect_base =kern_hyp_va (__kvm_bp_vect_base); }
arch/arm64/kvm/hyp/hyp-entry.S +4 -0
@@ -61,6 +61,10 @@ el1_sync: // Guest trapped into EL2 /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */ eor w1, w1, #(ARM_SMCCC_ARCH_WORKAROUND_1 ^ \ ARM_SMCCC_ARCH_WORKAROUND_2) + cbz w1, wa_epilogue + + eor w1, w1, #(ARM_SMCCC_ARCH_WORKAROUND_2 ^ \ + ARM_SMCCC_ARCH_WORKAROUND_3) cbnz w1, el1_trap wa_epilogue:
arch/arm64/kvm/hyp/smccc_wa.S +75 -0
78 lines not on this page · raw message
arch/arm64/kvm/hyp/vhe/switch.c +7 -2
@@ -10,6 +10,7 @@#include <linux/kvm_host.h> #include <linux/types.h> #include <linux/jump_label.h> +#include <linux/percpu.h> #include <uapi/linux/psci.h> #include <kvm/arm_psci.h>
@@ -25,6 +26,7 @@#include <asm/debug-monitors.h> #include <asm/processor.h> #include <asm/thread_info.h> +#include <asm/vectors.h> const char __hyp_panic_string[] ="HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n" ;
@@ -70,7 +72,7 @@ NOKPROBE_SYMBOL(__activate_traps);static void __deactivate_traps (struct kvm_vcpu *vcpu) { -extern char vectors[];/* kernel exception vectors */ +const char *host_vectors = vectors;___deactivate_traps (vcpu);
@@ -84,7 +86,10 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu) asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT));write_sysreg (CPACR_EL1_DEFAULT , cpacr_el1); -write_sysreg (vectors, vbar_el1); + +if (!arm64_kernel_unmapped_at_el0 ()) + host_vectors =__this_cpu_read (this_cpu_vector); +write_sysreg (host_vectors, vbar_el1); }NOKPROBE_SYMBOL (__deactivate_traps);
arch/arm64/kvm/hypercalls.c +12 -0
@@ -58,6 +58,18 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)break ; }break ; +case ARM_SMCCC_ARCH_WORKAROUND_3 : +switch (arm64_get_spectre_bhb_state ()) { +case SPECTRE_VULNERABLE : +break ; +case SPECTRE_MITIGATED : + val =SMCCC_RET_SUCCESS ; +break ; +case SPECTRE_UNAFFECTED : + val =SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED ; +break ; + } +break ;case ARM_SMCCC_HV_PV_TIME_FEATURES : val =SMCCC_RET_SUCCESS ;break ;
arch/arm64/kvm/psci.c +17 -1
@@ -397,7 +397,7 @@ int kvm_psci_call(struct kvm_vcpu *vcpu)int kvm_arm_get_fw_num_regs (struct kvm_vcpu *vcpu) { -return 3 ;/* PSCI version and two workaround registers */ +return 4 ;/* PSCI version and three workaround registers */ }int kvm_arm_copy_fw_reg_indices (struct kvm_vcpu *vcpu,u64 __user *uindices)
@@ -411,6 +411,9 @@ int kvm_arm_copy_fw_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)if (put_user (KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2 , uindices++))return -EFAULT ; +if (put_user (KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 , uindices++)) +return -EFAULT ; +return 0 ; }
@@ -450,6 +453,17 @@ static int get_kernel_wa_level(u64 regid)case SPECTRE_VULNERABLE :return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_NOT_AVAIL ; } +break ; +case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 : +switch (arm64_get_spectre_bhb_state ()) { +case SPECTRE_VULNERABLE : +return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_AVAIL ; +case SPECTRE_MITIGATED : +return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_AVAIL ; +case SPECTRE_UNAFFECTED : +return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_REQUIRED ; + } +return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_AVAIL ; }return -EINVAL ;
@@ -466,6 +480,7 @@ int kvm_arm_get_fw_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)break ;case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1 :case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2 : +case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 : val =get_kernel_wa_level (reg->id) &KVM_REG_FEATURE_LEVEL_MASK ;break ;default :
@@ -511,6 +526,7 @@ int kvm_arm_set_fw_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) }case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1 : +case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 :if (val & ~KVM_REG_FEATURE_LEVEL_MASK )return -EINVAL ;
arch/arm64/kvm/sys_regs.c +1 -1
@@ -1517,7 +1517,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {/* CRm=6 */ ID_SANITISED (ID_AA64ISAR0_EL1 ),ID_SANITISED (ID_AA64ISAR1_EL1 ), -ID_UNALLOCATED (6 ,2 ), +ID_SANITISED (ID_AA64ISAR2_EL1 ),ID_UNALLOCATED (6 ,3 ),ID_UNALLOCATED (6 ,4 ), ID_UNALLOCATED(6,5),
arch/arm64/mm/mmu.c +9 -3
@@ -592,6 +592,8 @@ early_param("rodata", parse_rodata);#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 static int __init map_entry_trampoline (void ) { +int i; +pgprot_t prot = rodata_enabled ?PAGE_KERNEL_ROX :PAGE_KERNEL_EXEC ;phys_addr_t pa_start =__pa_symbol (__entry_tramp_text_start);
@@ -600,11 +602,15 @@ static int __init map_entry_trampoline(void)/* Map only the text into the trampoline page table */ memset (tramp_pg_dir,0 ,PGD_SIZE ); -__create_pgd_mapping (tramp_pg_dir, pa_start,TRAMP_VALIAS ,PAGE_SIZE , - prot, __pgd_pgtable_alloc,0 ); +__create_pgd_mapping (tramp_pg_dir, pa_start,TRAMP_VALIAS , +entry_tramp_text_size (), prot, + __pgd_pgtable_alloc,NO_BLOCK_MAPPINGS );/* Map both the text and data into the kernel page table */ -__set_fixmap (FIX_ENTRY_TRAMP_TEXT , pa_start, prot); +for (i =0 ; i <DIV_ROUND_UP (entry_tramp_text_size (),PAGE_SIZE ); i++) +__set_fixmap (FIX_ENTRY_TRAMP_TEXT1 - i, + pa_start + i *PAGE_SIZE , prot); +if (IS_ENABLED (CONFIG_RANDOMIZE_BASE )) {extern char __entry_tramp_data_start[];
arch/x86/include/asm/cpufeatures.h +1 -1
@@ -204,7 +204,7 @@#define X86_FEATURE_SME ( 7*32+10)/* AMD Secure Memory Encryption */ #define X86_FEATURE_PTI ( 7*32+11)/* Kernel Page Table Isolation enabled */ #define X86_FEATURE_RETPOLINE ( 7*32+12)/* "" Generic Retpoline mitigation for Spectre variant 2 */ -#define X86_FEATURE_RETPOLINE_AMD ( 7*32+13)/* "" AMD Retpoline mitigation for Spectre variant 2 */ +#define X86_FEATURE_RETPOLINE_LFENCE ( 7*32+13)/* "" Use LFENCE for Spectre variant 2 */ #define X86_FEATURE_INTEL_PPIN ( 7*32+14)/* Intel Processor Inventory Number */ #define X86_FEATURE_CDP_L2 ( 7*32+15)/* Code and Data Prioritization L2 */ #define X86_FEATURE_MSR_SPEC_CTRL ( 7*32+16)/* "" MSR SPEC_CTRL is implemented */
arch/x86/include/asm/nospec-branch.h +9 -7
@@ -82,7 +82,7 @@#ifdef CONFIG_RETPOLINE ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \ __stringify(jmp __x86_retpoline_\reg), X86_FEATURE_RETPOLINE, \ - __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_AMD + __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_LFENCE#else jmp *%\reg#endif
@@ -92,7 +92,7 @@#ifdef CONFIG_RETPOLINE ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; call *%\reg), \ __stringify(call __x86_retpoline_\reg), X86_FEATURE_RETPOLINE, \ - __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; call *%\reg), X86_FEATURE_RETPOLINE_AMD + __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; call *%\reg), X86_FEATURE_RETPOLINE_LFENCE#else call *%\reg#endif
@@ -134,7 +134,7 @@ "lfence;\n" \ ANNOTATE_RETPOLINE_SAFE \ "call *%[thunk_target]\n", \ - X86_FEATURE_RETPOLINE_AMD) + X86_FEATURE_RETPOLINE_LFENCE)# define THUNK_TARGET (addr) [thunk_target] "r" (addr)
@@ -164,7 +164,7 @@ "lfence;\n" \ ANNOTATE_RETPOLINE_SAFE \ "call *%[thunk_target]\n", \ - X86_FEATURE_RETPOLINE_AMD) + X86_FEATURE_RETPOLINE_LFENCE)# define THUNK_TARGET (addr) [thunk_target] "rm" (addr)#endif
@@ -176,9 +176,11 @@/* The Spectre V2 mitigation variants */ enum spectre_v2_mitigation {SPECTRE_V2_NONE , -SPECTRE_V2_RETPOLINE_GENERIC , -SPECTRE_V2_RETPOLINE_AMD , -SPECTRE_V2_IBRS_ENHANCED , +SPECTRE_V2_RETPOLINE , +SPECTRE_V2_LFENCE , +SPECTRE_V2_EIBRS , +SPECTRE_V2_EIBRS_RETPOLINE , +SPECTRE_V2_EIBRS_LFENCE , };/* The indirect branch speculation control variants */
arch/x86/kernel/cpu/bugs.c +153 -52
312 lines not on this page · raw message
drivers/acpi/ec.c +0 -10
@@ -2064,16 +2064,6 @@ bool acpi_ec_dispatch_gpe(void)if (acpi_any_gpe_status_set (first_ec->gpe))return true; -/* -* Cancel the SCI wakeup and process all pending events in case there -* are any wakeup ones in there. -* -* Note that if any non-EC GPEs are active at this point, the SCI will -* retrigger after the rearming in acpi_s2idle_wake(), so no events -* should be missed by canceling the wakeup here. -*/ -pm_system_cancel_wakeup (); -/* * Dispatch the EC GPE in-band, but do not report wakeup in any case * to allow the caller to process events properly after that.
drivers/acpi/sleep.c +10 -4
@@ -1012,15 +1012,21 @@ static bool acpi_s2idle_wake(void)return true; } -/* -* Check non-EC GPE wakeups and if there are none, cancel the -* SCI-related wakeup and dispatch the EC GPE. -*/ +/* Check non-EC GPE wakeups and dispatch the EC GPE. */ if (acpi_ec_dispatch_gpe ()) {pm_pr_dbg ("ACPI non-EC GPE wakeup\n" );return true; } +/* +* Cancel the SCI wakeup and process all pending events in case +* there are any wakeup ones in there. +* +* Note that if any non-EC GPEs are active at this point, the +* SCI will retrigger after the rearming below, so no events +* should be missed by canceling the wakeup here. +*/ +pm_system_cancel_wakeup ();acpi_os_wait_events_complete ();/*
drivers/block/xen-blkfront.c +37 -26
147 lines not on this page · raw message
drivers/net/xen-netfront.c +33 -21
71 lines not on this page · raw message
drivers/xen/grant-table.c +39 -32
71 lines not on this page · raw message
tools/arch/x86/include/asm/cpufeatures.h +1 -1
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.