Linux 5.16.14
Greg Kroah-Hartman
maintainerI'm announcing the release of the 5.16.14 kernel.
All users of the 5.16 kernel series must upgrade.
The updated 5.16.y git tree can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-5.16.y
and can be browsed at the normal kernel.org git web browser:
https://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary
thanks,
greg k-h
------------
Documentation/admin-guide/hw-vuln/spectre.rst | 50 ++- Documentation/admin-guide/kernel-parameters.txt | 8 Documentation/arm64/cpu-feature-registers.rst | 17 + Documentation/arm64/elf_hwcaps.rst | 8 Makefile | 2 arch/arm/include/asm/assembler.h | 10 arch/arm/include/asm/spectre.h | 32 + arch/arm/include/asm/vmlinux.lds.h | 43 ++ arch/arm/kernel/Makefile | 2 arch/arm/kernel/entry-armv.S | 79 ++++ arch/arm/kernel/entry-common.S | 24 + arch/arm/kernel/spectre.c | 71 ++++ arch/arm/kernel/traps.c | 65 +++ arch/arm/mm/Kconfig | 11 arch/arm/mm/proc-v7-bugs.c | 208 ++++++++++-- arch/arm64/Kconfig | 9 arch/arm64/include/asm/assembler.h | 53 +++ arch/arm64/include/asm/cpu.h | 1 arch/arm64/include/asm/cpufeature.h | 29 + arch/arm64/include/asm/cputype.h | 8 arch/arm64/include/asm/fixmap.h | 6 arch/arm64/include/asm/hwcap.h | 2 arch/arm64/include/asm/insn.h | 1 arch/arm64/include/asm/kvm_host.h | 5 arch/arm64/include/asm/rwonce.h | 4 arch/arm64/include/asm/sections.h | 5 arch/arm64/include/asm/spectre.h | 4 arch/arm64/include/asm/sysreg.h | 18 + arch/arm64/include/asm/vectors.h | 73 ++++ arch/arm64/include/uapi/asm/hwcap.h | 2 arch/arm64/include/uapi/asm/kvm.h | 5 arch/arm64/kernel/cpu_errata.c | 7 arch/arm64/kernel/cpufeature.c | 25 + arch/arm64/kernel/cpuinfo.c | 3 arch/arm64/kernel/entry.S | 214 +++++++++---- arch/arm64/kernel/image-vars.h | 4 arch/arm64/kernel/proton-pack.c | 391 +++++++++++++++++++++++- arch/arm64/kernel/vmlinux.lds.S | 2 arch/arm64/kvm/arm.c | 5 arch/arm64/kvm/hyp/hyp-entry.S | 9 arch/arm64/kvm/hyp/nvhe/mm.c | 4 arch/arm64/kvm/hyp/vhe/switch.c | 9 arch/arm64/kvm/hypercalls.c | 12 arch/arm64/kvm/psci.c | 18 + arch/arm64/kvm/sys_regs.c | 2 arch/arm64/mm/mmu.c | 12 arch/arm64/tools/cpucaps | 1 arch/x86/include/asm/cpufeatures.h | 2 arch/x86/include/asm/nospec-branch.h | 16 arch/x86/kernel/alternative.c | 8 arch/x86/kernel/cpu/bugs.c | 204 +++++++++--- arch/x86/lib/retpoline.S | 2 arch/x86/net/bpf_jit_comp.c | 2 drivers/acpi/ec.c | 10 drivers/acpi/sleep.c | 14 drivers/block/xen-blkfront.c | 63 ++- drivers/net/xen-netfront.c | 54 ++- drivers/scsi/xen-scsifront.c | 3 drivers/xen/gntalloc.c | 25 - drivers/xen/grant-table.c | 71 ++-- drivers/xen/pvcalls-front.c | 8 drivers/xen/xenbus/xenbus_client.c | 24 - include/linux/arm-smccc.h | 5 include/linux/bpf.h | 12 include/xen/grant_table.h | 19 + kernel/sysctl.c | 7 net/9p/trans_xen.c | 14 tools/arch/x86/include/asm/cpufeatures.h | 2 68 files changed, 1781 insertions(+), 357 deletions(-)
Emmanuel Gil Peyrot (1):
ARM: fix build error when BPF_SYSCALL is disabled
Greg Kroah-Hartman (2):
Revert "ACPI: PM: s2idle: Cancel wakeup before dispatching EC GPE" Linux 5.16.14
James Morse (20):
arm64: entry.S: Add ventry overflow sanity checks
arm64: spectre: Rename spectre_v4_patch_fw_mitigation_conduit
KVM: arm64: Allow indirect vectors to be used without SPECTRE_V3A
arm64: entry: Make the trampoline cleanup optional
arm64: entry: Free up another register on kpti's tramp_exit path
arm64: entry: Move the trampoline data page before the text page
arm64: entry: Allow tramp_alias to access symbols after the 4K boundary
arm64: entry: Don't assume tramp_vectors is the start of the vectors
arm64: entry: Move trampoline macros out of ifdef'd section
arm64: entry: Make the kpti trampoline's kpti sequence optional
arm64: entry: Allow the trampoline text to occupy multiple pages
arm64: entry: Add non-kpti __bp_harden_el1_vectors for mitigations
arm64: entry: Add vectors that have the bhb mitigation sequences
arm64: entry: Add macro for reading symbol addresses from the trampoline
arm64: Add percpu vectors for EL1
arm64: proton-pack: Report Spectre-BHB vulnerabilities as part of Spectre-v2
arm64: Mitigate spectre style branch history side channels
KVM: arm64: Allow SMCCC_ARCH_WORKAROUND_3 to be discovered and migrated
arm64: Use the clearbhb instruction in mitigations
arm64: proton-pack: Include unprivileged eBPF status in Spectre v2 mitigation reporting
Joey Gouly (3):
arm64: add ID_AA64ISAR2_EL1 sys register
arm64: cpufeature: add HWCAP for FEAT_AFP
arm64: cpufeature: add HWCAP for FEAT_RPRES
Josh Poimboeuf (3):
x86/speculation: Include unprivileged eBPF status in Spectre v2 mitigation reporting x86/speculation: Warn about Spectre v2 LFENCE mitigation x86/speculation: Warn about eIBRS + LFENCE + Unprivileged eBPF + SMT
Juergen Gross (11):
xen/xenbus: don't let xenbus_grant_ring() remove grants in error case xen/grant-table: add gnttab_try_end_foreign_access() xen/blkfront: don't use gnttab_query_foreign_access() for mapped status xen/netfront: don't use gnttab_query_foreign_access() for mapped status xen/scsifront: don't use gnttab_query_foreign_access() for mapped status xen/gntalloc: don't use gnttab_query_foreign_access()
xen: remove gnttab_query_foreign_access()
xen/9p: use alloc/free_pages_exact()
xen/pvcalls: use alloc/free_pages_exact()
xen/gnttab: fix gnttab_end_foreign_access() without page specified xen/netfront: react properly to failing gnttab_end_foreign_access_ref()
Kim Phillips (2):
x86/speculation: Use generic retpoline by default on AMD x86/speculation: Update link to AMD speculation whitepaper
Nathan Chancellor (2):
ARM: Do not use NOCROSSREFS directive with ld.lld
arm64: Do not include __READ_ONCE() block in assembly files
Peter Zijlstra (2):
x86/speculation: Add eIBRS + Retpoline options Documentation/hw-vuln: Update spectre doc
Peter Zijlstra (Intel) (1):
x86/speculation: Rename RETPOLINE_AMD to RETPOLINE_LFENCE
Russell King (Oracle) (7):
ARM: report Spectre v2 status through sysfs
ARM: early traps initialisation
ARM: use LOADADDR() to get load address of sections
ARM: Spectre-BHB workaround
ARM: include unprivileged BPF status in Spectre V2 reporting
ARM: fix co-processor register typo
ARM: fix build warning in proc-v7-bugs.c
Greg Kroah-Hartman
maintainerDocumentation/admin-guide/hw-vuln/spectre.rst +33 -17
@@ -60,8 +60,8 @@ privileged data touched during the speculative execution. Spectre variant 1 attacks take advantage of speculative execution of conditional branches, while Spectre variant 2 attacks use speculative execution of indirect branches to leak privileged memory. -See :ref:`[1] <spec_ref1>` :ref:`[5] <spec_ref5>` :ref:`[7] <spec_ref7>` -:ref:`[10] <spec_ref10>` :ref:`[11] <spec_ref11>`. +See :ref:`[1] <spec_ref1>` :ref:`[5] <spec_ref5>` :ref:`[6] <spec_ref6>` +:ref:`[7] <spec_ref7>` :ref:`[10] <spec_ref10>` :ref:`[11] <spec_ref11>`. Spectre variant 1 (Bounds Check Bypass) ---------------------------------------
@@ -131,6 +131,19 @@ steer its indirect branch speculations to gadget code, and measure the speculative execution's side effects left in level 1 cache to infer the victim's data. +Yet another variant 2 attack vector is for the attacker to poison the +Branch History Buffer (BHB) to speculatively steer an indirect branch +to a specific Branch Target Buffer (BTB) entry, even if the entry isn't +associated with the source address of the indirect branch. Specifically, +the BHB might be shared across privilege levels even in the presence of +Enhanced IBRS. + +Currently the only known real-world BHB attack vector is via +unprivileged eBPF. Therefore, it's highly recommended to not enable +unprivileged eBPF, especially when eIBRS is used (without retpolines). +For a full mitigation against BHB attacks, it's recommended to use +retpolines (or eIBRS combined with retpolines). + Attack scenarios ----------------
@@ -364,13 +377,15 @@ The possible values in this file are: - Kernel status: - ==================================== ================================= - 'Not affected' The processor is not vulnerable - 'Vulnerable' Vulnerable, no mitigation - 'Mitigation: Full generic retpoline' Software-focused mitigation - 'Mitigation: Full AMD retpoline' AMD-specific software mitigation - 'Mitigation: Enhanced IBRS' Hardware-focused mitigation - ==================================== ================================= + ======================================== ================================= + 'Not affected' The processor is not vulnerable + 'Mitigation: None' Vulnerable, no mitigation + 'Mitigation: Retpolines' Use Retpoline thunks + 'Mitigation: LFENCE' Use LFENCE instructions + 'Mitigation: Enhanced IBRS' Hardware-focused mitigation + 'Mitigation: Enhanced IBRS + Retpolines' Hardware-focused + Retpolines + 'Mitigation: Enhanced IBRS + LFENCE' Hardware-focused + LFENCE + ======================================== ================================= - Firmware status: Show if Indirect Branch Restricted Speculation (IBRS) is used to protect against Spectre variant 2 attacks when calling firmware (x86 only).
@@ -583,12 +598,13 @@ kernel command line. Specific mitigations can also be selected manually: - retpoline - replace indirect branches - retpoline,generic - google's original retpoline - retpoline,amd - AMD-specific minimal thunk + retpoline auto pick between generic,lfence + retpoline,generic Retpolines + retpoline,lfence LFENCE; indirect branch + retpoline,amd alias for retpoline,lfence + eibrs enhanced IBRS + eibrs,retpoline enhanced IBRS + Retpolines + eibrs,lfence enhanced IBRS + LFENCE Not specifying this option is equivalent to spectre_v2=auto.
@@ -599,7 +615,7 @@ kernel command line. spectre_v2=off. Spectre variant 1 mitigations cannot be disabled. -For spectre_v2_user see :doc:`/admin-guide/kernel-parameters`. +For spectre_v2_user see Documentation/admin-guide/kernel-parameters.txt Mitigation selection guide --------------------------
@@ -681,7 +697,7 @@ AMD white papers: .. _spec_ref6: -[6] `Software techniques for managing speculation on AMD processors <https://developer.amd.com/wp-content/resources/90343-B_SoftwareTechniquesforManagingSpeculation_WP_7-18Update_FNL.pdf>`_. +[6] `Software techniques for managing speculation on AMD processors <https://developer.amd.com/wp-content/resources/Managing-Speculation-on-AMD-Processors.pdf>`_. ARM white papers:
Documentation/admin-guide/kernel-parameters.txt +6 -2
@@ -5303,8 +5303,12 @@ Specific mitigations can also be selected manually: retpoline - replace indirect branches - retpoline,generic - google's original retpoline - retpoline,amd - AMD-specific minimal thunk + retpoline,generic - Retpolines + retpoline,lfence - LFENCE; indirect branch + retpoline,amd - alias for retpoline,lfence + eibrs - enhanced IBRS + eibrs,retpoline - enhanced IBRS + Retpolines + eibrs,lfence - enhanced IBRS + LFENCE Not specifying this option is equivalent to spectre_v2=auto.
Documentation/arm64/cpu-feature-registers.rst +17 -0
@@ -275,6 +275,23 @@ infrastructure: | SVEVer | [3-0] | y | +------------------------------+---------+---------+ + 8) ID_AA64MMFR1_EL1 - Memory model feature register 1 + + +------------------------------+---------+---------+ + | Name | bits | visible | + +------------------------------+---------+---------+ + | AFP | [47-44] | y | + +------------------------------+---------+---------+ + + 9) ID_AA64ISAR2_EL1 - Instruction set attribute register 2 + + +------------------------------+---------+---------+ + | Name | bits | visible | + +------------------------------+---------+---------+ + | RPRES | [7-4] | y | + +------------------------------+---------+---------+ + + Appendix I: Example -------------------
Documentation/arm64/elf_hwcaps.rst +8 -0
@@ -251,6 +251,14 @@ HWCAP2_ECV Functionality implied by ID_AA64MMFR0_EL1.ECV == 0b0001. +HWCAP2_AFP + + Functionality implied by ID_AA64MFR1_EL1.AFP == 0b0001. + +HWCAP2_RPRES + + Functionality implied by ID_AA64ISAR2_EL1.RPRES == 0b0001. + 4. Unused AT_HWCAP bits -----------------------
Makefile +1 -1
@@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 VERSION = 5 PATCHLEVEL = 16 -SUBLEVEL = 13 +SUBLEVEL = 14 EXTRAVERSION = NAME = Gobble Gobble
arch/arm/include/asm/assembler.h +10 -0
@@ -107,6 +107,16 @@ .endm#endif +#if __LINUX_ARM_ARCH__ <7 + .macro dsb, args + mcr p15, 0, r0, c7, c10, 4 + .endm + + .macro isb, args + mcr p15, 0, r0, c7, c5, 4 + .endm +#endif + .macro asm_trace_hardirqs_off, save=1#if defined(CONFIG_TRACE_IRQFLAGS ) .if \save
arch/arm/include/asm/spectre.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __ASM_SPECTRE_H +#define __ASM_SPECTRE_H + +enum { +SPECTRE_UNAFFECTED , +SPECTRE_MITIGATED , +SPECTRE_VULNERABLE , +}; + +enum { + __SPECTRE_V2_METHOD_BPIALL, + __SPECTRE_V2_METHOD_ICIALLU, + __SPECTRE_V2_METHOD_SMC, + __SPECTRE_V2_METHOD_HVC, + __SPECTRE_V2_METHOD_LOOP8, +}; + +enum { +SPECTRE_V2_METHOD_BPIALL =BIT (__SPECTRE_V2_METHOD_BPIALL), +SPECTRE_V2_METHOD_ICIALLU =BIT (__SPECTRE_V2_METHOD_ICIALLU), +SPECTRE_V2_METHOD_SMC =BIT (__SPECTRE_V2_METHOD_SMC), +SPECTRE_V2_METHOD_HVC =BIT (__SPECTRE_V2_METHOD_HVC), +SPECTRE_V2_METHOD_LOOP8 =BIT (__SPECTRE_V2_METHOD_LOOP8), +}; + +void spectre_v2_update_state (unsigned int state,unsigned int methods); + +int spectre_bhb_update_vectors (unsigned int method); + +#endif
arch/arm/include/asm/vmlinux.lds.h +34 -9
59 lines not on this page · raw message
arch/arm/kernel/Makefile +2 -0
@@ -106,4 +106,6 @@ endif obj-$(CONFIG_HAVE_ARM_SMCCC) += smccc-call.o +obj-$(CONFIG_GENERIC_CPU_VULNERABILITIES) += spectre.o + extra-y := $(head-y) vmlinux.lds
arch/arm/kernel/entry-armv.S +73 -6
111 lines not on this page · raw message
arch/arm/kernel/entry-common.S +24 -0
@@ -153,6 +153,29 @@ ENDPROC(ret_from_fork) *----------------------------------------------------------------------------- */ + .align 5 +#ifdef CONFIG_HARDEN_BRANCH_HISTORY +ENTRY(vector_bhb_loop8_swi) + sub sp, sp, #PT_REGS_SIZE + stmia sp, {r0 - r12} + mov r8, #8 +1: b 2f +2: subs r8, r8, #1 + bne 1b + dsb + isb + b 3f +ENDPROC(vector_bhb_loop8_swi) + + .align 5 +ENTRY(vector_bhb_bpiall_swi) + sub sp, sp, #PT_REGS_SIZE + stmia sp, {r0 - r12} + mcr p15, 0, r8, c7, c5, 6 @ BPIALL + isb + b 3f +ENDPROC(vector_bhb_bpiall_swi) +#endif .align 5 ENTRY(vector_swi) #ifdef CONFIG_CPU_V7M
@@ -160,6 +183,7 @@ ENTRY(vector_swi) #else sub sp, sp, #PT_REGS_SIZE stmia sp, {r0 - r12} @ Calling r0 - r12 +3: ARM( add r8, sp, #S_PC ) ARM( stmdb r8, {sp, lr}^ ) @ Calling sp, lr THUMB( mov r8, sp )
arch/arm/kernel/spectre.c new file +71 -0
71 lines not on this page · raw message
arch/arm/kernel/traps.c +59 -6
92 lines not on this page · raw message
arch/arm/mm/Kconfig +11 -0
@@ -830,6 +830,7 @@ config CPU_BPREDICT_DISABLE config CPU_SPECTRE bool + select GENERIC_CPU_VULNERABILITIES config HARDEN_BRANCH_PREDICTOR bool "Harden the branch predictor against aliasing attacks" if EXPERT
@@ -850,6 +851,16 @@ config HARDEN_BRANCH_PREDICTOR If unsure, say Y. +config HARDEN_BRANCH_HISTORY + bool "Harden Spectre style attacks against branch history" if EXPERT + depends on CPU_SPECTRE + default y + help + Speculation attacks against some high-performance processors can + make use of branch history to influence future speculation. When + taking an exception, a sequence of branches overwrites the branch + history, or branch history is invalidated. + config TLS_REG_EMUL bool select NEED_KUSER_HELPERS
arch/arm/mm/proc-v7-bugs.c +174 -34
280 lines not on this page · raw message
arch/arm64/Kconfig +9 -0
@@ -1395,6 +1395,15 @@ config UNMAP_KERNEL_AT_EL0 If unsure, say Y. +config MITIGATE_SPECTRE_BRANCH_HISTORY + bool "Mitigate Spectre style attacks against branch history" if EXPERT + default y + help + Speculation attacks against some high-performance processors can + make use of branch history to influence future speculation. + When taking an exception from user-space, a sequence of branches + or a firmware call overwrites the branch history. + config RODATA_FULL_DEFAULT_ENABLED bool "Apply r/o permissions of VM areas also to their linear aliases" default y
arch/arm64/include/asm/assembler.h +53 -0
63 lines not on this page · raw message
arch/arm64/include/asm/cpu.h +1 -0
@@ -51,6 +51,7 @@ struct cpuinfo_arm64 {u64 reg_id_aa64dfr1;u64 reg_id_aa64isar0;u64 reg_id_aa64isar1; +u64 reg_id_aa64isar2;u64 reg_id_aa64mmfr0;u64 reg_id_aa64mmfr1;u64 reg_id_aa64mmfr2;
arch/arm64/include/asm/cpufeature.h +29 -0
@@ -637,6 +637,35 @@ static inline bool cpu_supports_mixed_endian_el0(void)return id_aa64mmfr0_mixed_endian_el0 (read_cpuid (ID_AA64MMFR0_EL1 )); } + +static inline bool supports_csv2p3 (int scope) +{ +u64 pfr0; +u8 csv2_val; + +if (scope ==SCOPE_LOCAL_CPU ) + pfr0 =read_sysreg_s (SYS_ID_AA64PFR0_EL1 ); +else + pfr0 =read_sanitised_ftr_reg (SYS_ID_AA64PFR0_EL1 ); + + csv2_val =cpuid_feature_extract_unsigned_field (pfr0, +ID_AA64PFR0_CSV2_SHIFT ); +return csv2_val ==3 ; +} + +static inline bool supports_clearbhb (int scope) +{ +u64 isar2; + +if (scope ==SCOPE_LOCAL_CPU ) + isar2 =read_sysreg_s (SYS_ID_AA64ISAR2_EL1 ); +else + isar2 =read_sanitised_ftr_reg (SYS_ID_AA64ISAR2_EL1 ); + +return cpuid_feature_extract_unsigned_field (isar2, +ID_AA64ISAR2_CLEARBHB_SHIFT ); +} +const struct cpumask *system_32bit_el0_cpumask (void );DECLARE_STATIC_KEY_FALSE (arm64_mismatched_32bit_el0);
arch/arm64/include/asm/cputype.h +8 -0
@@ -73,10 +73,14 @@#define ARM_CPU_PART_CORTEX_A76 0xD0B#define ARM_CPU_PART_NEOVERSE_N1 0xD0C#define ARM_CPU_PART_CORTEX_A77 0xD0D +#define ARM_CPU_PART_NEOVERSE_V1 0xD40 +#define ARM_CPU_PART_CORTEX_A78 0xD41 +#define ARM_CPU_PART_CORTEX_X1 0xD44#define ARM_CPU_PART_CORTEX_A510 0xD46#define ARM_CPU_PART_CORTEX_A710 0xD47#define ARM_CPU_PART_CORTEX_X2 0xD48#define ARM_CPU_PART_NEOVERSE_N2 0xD49 +#define ARM_CPU_PART_CORTEX_A78C 0xD4B#define APM_CPU_PART_POTENZA 0x000
@@ -117,10 +121,14 @@#define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A76)#define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N1)#define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A77) +#define MIDR_NEOVERSE_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V1) +#define MIDR_CORTEX_A78 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78) +#define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1)#define MIDR_CORTEX_A510 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A510)#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710)#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2)#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2) +#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
arch/arm64/include/asm/fixmap.h +4 -2
@@ -62,9 +62,11 @@ enum fixed_addresses {#endif /* CONFIG_ACPI_APEI_GHES */ #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +FIX_ENTRY_TRAMP_TEXT3 , +FIX_ENTRY_TRAMP_TEXT2 , +FIX_ENTRY_TRAMP_TEXT1 ,FIX_ENTRY_TRAMP_DATA , -FIX_ENTRY_TRAMP_TEXT , -#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT)) +#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT1))#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ __end_of_permanent_fixed_addresses,
arch/arm64/include/asm/hwcap.h +2 -0
@@ -106,6 +106,8 @@#define KERNEL_HWCAP_BTI __khwcap2_feature(BTI)#define KERNEL_HWCAP_MTE __khwcap2_feature(MTE)#define KERNEL_HWCAP_ECV __khwcap2_feature(ECV) +#define KERNEL_HWCAP_AFP __khwcap2_feature(AFP) +#define KERNEL_HWCAP_RPRES __khwcap2_feature(RPRES)/* * This yields a mask that user programs can use to figure out what
arch/arm64/include/asm/insn.h +1 -0
@@ -65,6 +65,7 @@ enum aarch64_insn_hint_cr_op {AARCH64_INSN_HINT_PSB =0x11 <<5 ,AARCH64_INSN_HINT_TSB =0x12 <<5 ,AARCH64_INSN_HINT_CSDB =0x14 <<5 , +AARCH64_INSN_HINT_CLEARBHB =0x16 <<5 ,AARCH64_INSN_HINT_BTI =0x20 <<5 , AARCH64_INSN_HINT_BTIC = 0x22 << 5,
arch/arm64/include/asm/kvm_host.h +5 -0
@@ -712,6 +712,11 @@ static inline void kvm_init_host_cpu_context(struct kvm_cpu_context *cpu_ctxt)ctxt_sys_reg (cpu_ctxt,MPIDR_EL1 ) =read_cpuid_mpidr (); } +static inline bool kvm_system_needs_idmapped_vectors (void ) +{ +return cpus_have_const_cap (ARM64_SPECTRE_V3A ); +} +void kvm_arm_vcpu_ptrauth_trap (struct kvm_vcpu *vcpu);static inline void kvm_arch_hardware_unsetup (void ) {}
arch/arm64/include/asm/rwonce.h +2 -2
@@ -5,7 +5,7 @@#ifndef __ASM_RWONCE_H#define __ASM_RWONCE_H -#ifdef CONFIG_LTO +#if defined(CONFIG_LTO ) && !defined(__ASSEMBLY__)#include <linux/compiler_types.h> #include <asm/alternative-macros.h>
@@ -66,7 +66,7 @@ })#endif /* !BUILD_VDSO */ -#endif /* CONFIG_LTO */ +#endif /* CONFIG_LTO && !__ASSEMBLY__ */ #include <asm-generic/rwonce.h>
arch/arm64/include/asm/sections.h +5 -0
@@ -23,4 +23,9 @@ extern char __mmuoff_data_start[], __mmuoff_data_end[];extern char __entry_tramp_text_start[], __entry_tramp_text_end[];extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; +static inline size_t entry_tramp_text_size (void ) +{ +return __entry_tramp_text_end - __entry_tramp_text_start; +} +#endif /* __ASM_SECTIONS_H */
arch/arm64/include/asm/spectre.h +4 -0
@@ -93,5 +93,9 @@ void spectre_v4_enable_task_mitigation(struct task_struct *tsk);enum mitigation_state arm64_get_meltdown_state (void ); +enum mitigation_state arm64_get_spectre_bhb_state (void ); +bool is_spectre_bhb_affected (const struct arm64_cpu_capabilities *entry,int scope); +u8 spectre_bhb_loop_affected (int scope); +void spectre_bhb_enable_mitigation (const struct arm64_cpu_capabilities *__unused);#endif /* __ASSEMBLY__ */ #endif /* __ASM_SPECTRE_H */
arch/arm64/include/asm/sysreg.h +18 -0
@@ -182,6 +182,7 @@#define SYS_ID_AA64ISAR0_EL1 sys_reg(3, 0, 0, 6, 0)#define SYS_ID_AA64ISAR1_EL1 sys_reg(3, 0, 0, 6, 1) +#define SYS_ID_AA64ISAR2_EL1 sys_reg(3, 0, 0, 6, 2)#define SYS_ID_AA64MMFR0_EL1 sys_reg(3, 0, 0, 7, 0)#define SYS_ID_AA64MMFR1_EL1 sys_reg(3, 0, 0, 7, 1)
@@ -771,6 +772,21 @@#define ID_AA64ISAR1_GPI_NI 0x0#define ID_AA64ISAR1_GPI_IMP_DEF 0x1 +/* id_aa64isar2 */ +#define ID_AA64ISAR2_CLEARBHB_SHIFT 28 +#define ID_AA64ISAR2_RPRES_SHIFT 4 +#define ID_AA64ISAR2_WFXT_SHIFT 0 + +#define ID_AA64ISAR2_RPRES_8BIT 0x0 +#define ID_AA64ISAR2_RPRES_12BIT 0x1 +/* +* Value 0x1 has been removed from the architecture, and is +* reserved, but has not yet been removed from the ARM ARM +* as of ARM DDI 0487G.b. +*/ +#define ID_AA64ISAR2_WFXT_NI 0x0 +#define ID_AA64ISAR2_WFXT_SUPPORTED 0x2 +/* id_aa64pfr0 */ #define ID_AA64PFR0_CSV3_SHIFT 60#define ID_AA64PFR0_CSV2_SHIFT 56
@@ -889,6 +905,8 @@#endif /* id_aa64mmfr1 */ +#define ID_AA64MMFR1_ECBHB_SHIFT 60 +#define ID_AA64MMFR1_AFP_SHIFT 44#define ID_AA64MMFR1_ETS_SHIFT 36#define ID_AA64MMFR1_TWED_SHIFT 32#define ID_AA64MMFR1_XNX_SHIFT 28
arch/arm64/include/asm/vectors.h new file +73 -0
73 lines not on this page · raw message
arch/arm64/include/uapi/asm/hwcap.h +2 -0
@@ -76,5 +76,7 @@#define HWCAP2_BTI (1 << 17)#define HWCAP2_MTE (1 << 18)#define HWCAP2_ECV (1 << 19) +#define HWCAP2_AFP (1 << 20) +#define HWCAP2_RPRES (1 << 21)#endif /* _UAPI__ASM_HWCAP_H */
arch/arm64/include/uapi/asm/kvm.h +5 -0
@@ -281,6 +281,11 @@ struct kvm_arm_copy_mte_tags {#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_NOT_REQUIRED 3#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_ENABLED (1U << 4) +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 KVM_REG_ARM_FW_REG(3) +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_AVAIL 0 +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_AVAIL 1 +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_REQUIRED 2 +/* SVE registers */ #define KVM_REG_ARM64_SVE (0x15 << KVM_REG_ARM_COPROC_SHIFT)
arch/arm64/kernel/cpu_errata.c +7 -0
@@ -502,6 +502,13 @@ const struct arm64_cpu_capabilities arm64_errata[] = { .matches = has_spectre_v4, .cpu_enable = spectre_v4_enable_mitigation, }, + { + .desc = "Spectre-BHB", + .capability = ARM64_SPECTRE_BHB, + .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, + .matches = is_spectre_bhb_affected, + .cpu_enable = spectre_bhb_enable_mitigation, + }, #ifdef CONFIG_ARM64_ERRATUM_1418040 { .desc = "ARM erratum 1418040",
arch/arm64/kernel/cpufeature.c +25 -0
@@ -73,6 +73,8 @@#include <linux/mm.h> #include <linux/cpu.h> #include <linux/kasan.h> +#include <linux/percpu.h> +#include <asm/cpu.h> #include <asm/cpufeature.h> #include <asm/cpu_ops.h>
@@ -85,6 +87,7 @@#include <asm/smp.h> #include <asm/sysreg.h> #include <asm/traps.h> +#include <asm/vectors.h> #include <asm/virt.h> /* Kernel representation of AT_HWCAP and AT_HWCAP2 */
@@ -110,6 +113,8 @@ DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);bool arm64_use_ng_mappings = false;EXPORT_SYMBOL (arm64_use_ng_mappings); +DEFINE_PER_CPU_READ_MOSTLY(const char *, this_cpu_vector) = vectors; +/* * Permit PER_LINUX32 and execve() of 32-bit binaries even if not all CPUs * support it?
@@ -225,6 +230,12 @@ static const struct arm64_ftr_bits ftr_id_aa64isar1[] = { ARM64_FTR_END, }; +static const struct arm64_ftr_bits ftr_id_aa64isar2[] = { +ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_HIGHER_SAFE ,ID_AA64ISAR2_CLEARBHB_SHIFT ,4 ,0 ), +ARM64_FTR_BITS (FTR_VISIBLE ,FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64ISAR2_RPRES_SHIFT ,4 ,0 ), +ARM64_FTR_END , +}; +static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {ARM64_FTR_BITS (FTR_HIDDEN ,FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64PFR0_CSV3_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64PFR0_CSV2_SHIFT ,4 ,0 ),
@@ -325,6 +336,7 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = { };static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = { +ARM64_FTR_BITS (FTR_VISIBLE ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR1_AFP_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR1_ETS_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR1_TWED_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR1_XNX_SHIFT ,4 ,0 ),
@@ -637,6 +649,7 @@ static const struct __ftr_reg_entry {ARM64_FTR_REG (SYS_ID_AA64ISAR0_EL1 , ftr_id_aa64isar0),ARM64_FTR_REG_OVERRIDE (SYS_ID_AA64ISAR1_EL1 , ftr_id_aa64isar1, &id_aa64isar1_override), +ARM64_FTR_REG (SYS_ID_AA64ISAR2_EL1 , ftr_id_aa64isar2),/* Op1 = 0, CRn = 0, CRm = 7 */ ARM64_FTR_REG(SYS_ID_AA64MMFR0_EL1, ftr_id_aa64mmfr0),
@@ -933,6 +946,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)init_cpu_ftr_reg (SYS_ID_AA64DFR1_EL1 , info->reg_id_aa64dfr1);init_cpu_ftr_reg (SYS_ID_AA64ISAR0_EL1 , info->reg_id_aa64isar0);init_cpu_ftr_reg (SYS_ID_AA64ISAR1_EL1 , info->reg_id_aa64isar1); +init_cpu_ftr_reg (SYS_ID_AA64ISAR2_EL1 , info->reg_id_aa64isar2);init_cpu_ftr_reg (SYS_ID_AA64MMFR0_EL1 , info->reg_id_aa64mmfr0);init_cpu_ftr_reg (SYS_ID_AA64MMFR1_EL1 , info->reg_id_aa64mmfr1);init_cpu_ftr_reg (SYS_ID_AA64MMFR2_EL1 , info->reg_id_aa64mmfr2);
@@ -1151,6 +1165,8 @@ void update_cpu_features(int cpu, info->reg_id_aa64isar0, boot->reg_id_aa64isar0); taint |=check_update_ftr_reg (SYS_ID_AA64ISAR1_EL1 , cpu, info->reg_id_aa64isar1, boot->reg_id_aa64isar1); + taint |=check_update_ftr_reg (SYS_ID_AA64ISAR2_EL1 , cpu, + info->reg_id_aa64isar2, boot->reg_id_aa64isar2);/* * Differing PARange support is fine as long as all peripherals and
@@ -1272,6 +1288,7 @@ u64 __read_sysreg_by_encoding(u32 sys_id)read_sysreg_case (SYS_ID_AA64MMFR2_EL1 );read_sysreg_case (SYS_ID_AA64ISAR0_EL1 );read_sysreg_case (SYS_ID_AA64ISAR1_EL1 ); +read_sysreg_case (SYS_ID_AA64ISAR2_EL1 );read_sysreg_case (SYS_CNTFRQ_EL0 );read_sysreg_case (SYS_CTR_EL0 );
@@ -1579,6 +1596,12 @@ kpti_install_ng_mappings(const struct arm64_cpu_capabilities *__unused)int cpu =smp_processor_id (); +if (__this_cpu_read (this_cpu_vector) == vectors) { +const char *v =arm64_get_bp_hardening_vector (EL1_VECTOR_KPTI ); + +__this_cpu_write (this_cpu_vector, v); + } +/* * We don't need to rewrite the page-tables if either we've done * it already or we have KASLR enabled and therefore have not
@@ -2479,6 +2502,8 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = {HWCAP_CAP (SYS_ID_AA64PFR1_EL1 ,ID_AA64PFR1_MTE_SHIFT ,FTR_UNSIGNED ,ID_AA64PFR1_MTE ,CAP_HWCAP ,KERNEL_HWCAP_MTE ), #endif /* CONFIG_ARM64_MTE */HWCAP_CAP (SYS_ID_AA64MMFR0_EL1 ,ID_AA64MMFR0_ECV_SHIFT ,FTR_UNSIGNED ,1 ,CAP_HWCAP ,KERNEL_HWCAP_ECV ), +HWCAP_CAP (SYS_ID_AA64MMFR1_EL1 ,ID_AA64MMFR1_AFP_SHIFT ,FTR_UNSIGNED ,1 ,CAP_HWCAP ,KERNEL_HWCAP_AFP ), + HWCAP_CAP(SYS_ID_AA64ISAR2_EL1, ID_AA64ISAR2_RPRES_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_RPRES), {}, };
arch/arm64/kernel/cpuinfo.c +3 -0
@@ -95,6 +95,8 @@ static const char *const hwcap_str[] = { [KERNEL_HWCAP_BTI] = "bti", [KERNEL_HWCAP_MTE] = "mte", [KERNEL_HWCAP_ECV] = "ecv", + [KERNEL_HWCAP_AFP] = "afp", + [KERNEL_HWCAP_RPRES] = "rpres", };#ifdef CONFIG_COMPAT
@@ -391,6 +393,7 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info) info->reg_id_aa64dfr1 =read_cpuid (ID_AA64DFR1_EL1 ); info->reg_id_aa64isar0 =read_cpuid (ID_AA64ISAR0_EL1 ); info->reg_id_aa64isar1 =read_cpuid (ID_AA64ISAR1_EL1 ); + info->reg_id_aa64isar2 =read_cpuid (ID_AA64ISAR2_EL1 ); info->reg_id_aa64mmfr0 =read_cpuid (ID_AA64MMFR0_EL1 ); info->reg_id_aa64mmfr1 =read_cpuid (ID_AA64MMFR1_EL1 ); info->reg_id_aa64mmfr2 =read_cpuid (ID_AA64MMFR2_EL1 );
arch/arm64/kernel/entry.S +157 -57
330 lines not on this page · raw message
arch/arm64/kernel/image-vars.h +4 -0
@@ -66,6 +66,10 @@ KVM_NVHE_ALIAS(kvm_patch_vector_branch);KVM_NVHE_ALIAS (kvm_update_va_mask);KVM_NVHE_ALIAS (kvm_get_kimage_voffset);KVM_NVHE_ALIAS (kvm_compute_final_ctr_el0); +KVM_NVHE_ALIAS (spectre_bhb_patch_loop_iter); +KVM_NVHE_ALIAS (spectre_bhb_patch_loop_mitigation_enable); +KVM_NVHE_ALIAS (spectre_bhb_patch_wa3); +KVM_NVHE_ALIAS (spectre_bhb_patch_clearbhb);/* Global kernel state accessed by nVHE hyp code. */ KVM_NVHE_ALIAS (kvm_vgic_global_state);
arch/arm64/kernel/proton-pack.c +386 -5
427 lines not on this page · raw message
arch/arm64/kernel/vmlinux.lds.S +1 -1
@@ -341,7 +341,7 @@ ASSERT(__hibernate_exit_text_end - (__hibernate_exit_text_start & ~(SZ_4K - 1)) <= SZ_4K, "Hibernate exit text too big or misaligned") #endif #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 -ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) == PAGE_SIZE, +ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) <= 3*PAGE_SIZE, "Entry trampoline text too big") #endif #ifdef CONFIG_KVM
arch/arm64/kvm/arm.c +1 -4
@@ -1464,10 +1464,7 @@ static int kvm_init_vector_slots(void) base =kern_hyp_va (kvm_ksym_ref (__bp_harden_hyp_vecs));kvm_init_vector_slot (base,HYP_VECTOR_SPECTRE_DIRECT ); -if (!cpus_have_const_cap (ARM64_SPECTRE_V3A )) -return 0 ; - -if (!has_vhe ()) { +if (kvm_system_needs_idmapped_vectors () && !has_vhe ()) { err =create_hyp_exec_mappings (__pa_symbol (__bp_harden_hyp_vecs), __BP_HARDEN_HYP_VECS_SZ, &base);if (err)
arch/arm64/kvm/hyp/hyp-entry.S +9 -0
@@ -62,6 +62,10 @@ el1_sync: // Guest trapped into EL2 /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */ eor w1, w1, #(ARM_SMCCC_ARCH_WORKAROUND_1 ^ \ ARM_SMCCC_ARCH_WORKAROUND_2) + cbz w1, wa_epilogue + + eor w1, w1, #(ARM_SMCCC_ARCH_WORKAROUND_2 ^ \ + ARM_SMCCC_ARCH_WORKAROUND_3) cbnz w1, el1_trap wa_epilogue:
@@ -192,7 +196,10 @@ SYM_CODE_END(__kvm_hyp_vector) sub sp, sp, #(8 * 4) stp x2, x3, [sp, #(8 * 0)] stp x0, x1, [sp, #(8 * 2)] + alternative_cb spectre_bhb_patch_wa3 + /* Patched to mov WA3 when supported */ mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1 + alternative_cb_end smc #0 ldp x2, x3, [sp, #(8 * 0)] add sp, sp, #(8 * 2)
@@ -205,6 +212,8 @@ SYM_CODE_END(__kvm_hyp_vector) spectrev2_smccc_wa1_smc .else stp x0, x1, [sp, #-16]! + mitigate_spectre_bhb_loop x0 + mitigate_spectre_bhb_clear_insn .endif .if \indirect != 0 alternative_cb kvm_patch_vector_branch
arch/arm64/kvm/hyp/nvhe/mm.c +3 -1
@@ -146,8 +146,10 @@ int hyp_map_vectors(void)phys_addr_t phys;void *bp_base; -if (!cpus_have_const_cap (ARM64_SPECTRE_V3A )) +if (!kvm_system_needs_idmapped_vectors ()) { + __hyp_bp_vect_base = __bp_harden_hyp_vecs;return 0 ; + } phys =__hyp_pa (__bp_harden_hyp_vecs); bp_base = (void *)__pkvm_create_private_mapping(phys,
arch/arm64/kvm/hyp/vhe/switch.c +7 -2
@@ -10,6 +10,7 @@#include <linux/kvm_host.h> #include <linux/types.h> #include <linux/jump_label.h> +#include <linux/percpu.h> #include <uapi/linux/psci.h> #include <kvm/arm_psci.h>
@@ -25,6 +26,7 @@#include <asm/debug-monitors.h> #include <asm/processor.h> #include <asm/thread_info.h> +#include <asm/vectors.h> /* VHE specific context */ DEFINE_PER_CPU(struct kvm_host_data, kvm_host_data);
@@ -68,7 +70,7 @@ NOKPROBE_SYMBOL(__activate_traps);static void __deactivate_traps (struct kvm_vcpu *vcpu) { -extern char vectors[];/* kernel exception vectors */ +const char *host_vectors = vectors;___deactivate_traps (vcpu);
@@ -82,7 +84,10 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu) asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT));write_sysreg (CPACR_EL1_DEFAULT , cpacr_el1); -write_sysreg (vectors, vbar_el1); + +if (!arm64_kernel_unmapped_at_el0 ()) + host_vectors =__this_cpu_read (this_cpu_vector); +write_sysreg (host_vectors, vbar_el1); }NOKPROBE_SYMBOL (__deactivate_traps);
arch/arm64/kvm/hypercalls.c +12 -0
@@ -107,6 +107,18 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)break ; }break ; +case ARM_SMCCC_ARCH_WORKAROUND_3 : +switch (arm64_get_spectre_bhb_state ()) { +case SPECTRE_VULNERABLE : +break ; +case SPECTRE_MITIGATED : + val[0 ] =SMCCC_RET_SUCCESS ; +break ; +case SPECTRE_UNAFFECTED : + val[0 ] =SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED ; +break ; + } +break ;case ARM_SMCCC_HV_PV_TIME_FEATURES : val[0 ] =SMCCC_RET_SUCCESS ;break ;
arch/arm64/kvm/psci.c +17 -1
@@ -406,7 +406,7 @@ int kvm_psci_call(struct kvm_vcpu *vcpu)int kvm_arm_get_fw_num_regs (struct kvm_vcpu *vcpu) { -return 3 ;/* PSCI version and two workaround registers */ +return 4 ;/* PSCI version and three workaround registers */ }int kvm_arm_copy_fw_reg_indices (struct kvm_vcpu *vcpu,u64 __user *uindices)
@@ -420,6 +420,9 @@ int kvm_arm_copy_fw_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)if (put_user (KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2 , uindices++))return -EFAULT ; +if (put_user (KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 , uindices++)) +return -EFAULT ; +return 0 ; }
@@ -459,6 +462,17 @@ static int get_kernel_wa_level(u64 regid)case SPECTRE_VULNERABLE :return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_NOT_AVAIL ; } +break ; +case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 : +switch (arm64_get_spectre_bhb_state ()) { +case SPECTRE_VULNERABLE : +return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_AVAIL ; +case SPECTRE_MITIGATED : +return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_AVAIL ; +case SPECTRE_UNAFFECTED : +return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_REQUIRED ; + } +return KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_AVAIL ; }return -EINVAL ;
@@ -475,6 +489,7 @@ int kvm_arm_get_fw_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)break ;case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1 :case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2 : +case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 : val =get_kernel_wa_level (reg->id) &KVM_REG_FEATURE_LEVEL_MASK ;break ;default :
@@ -520,6 +535,7 @@ int kvm_arm_set_fw_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) }case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1 : +case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 :if (val & ~KVM_REG_FEATURE_LEVEL_MASK )return -EINVAL ;
arch/arm64/kvm/sys_regs.c +1 -1
@@ -1525,7 +1525,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {/* CRm=6 */ ID_SANITISED (ID_AA64ISAR0_EL1 ),ID_SANITISED (ID_AA64ISAR1_EL1 ), -ID_UNALLOCATED (6 ,2 ), +ID_SANITISED (ID_AA64ISAR2_EL1 ),ID_UNALLOCATED (6 ,3 ),ID_UNALLOCATED (6 ,4 ), ID_UNALLOCATED(6,5),
arch/arm64/mm/mmu.c +9 -3
@@ -617,6 +617,8 @@ early_param("rodata", parse_rodata);#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 static int __init map_entry_trampoline (void ) { +int i; +pgprot_t prot = rodata_enabled ?PAGE_KERNEL_ROX :PAGE_KERNEL_EXEC ;phys_addr_t pa_start =__pa_symbol (__entry_tramp_text_start);
@@ -625,11 +627,15 @@ static int __init map_entry_trampoline(void)/* Map only the text into the trampoline page table */ memset (tramp_pg_dir,0 ,PGD_SIZE ); -__create_pgd_mapping (tramp_pg_dir, pa_start,TRAMP_VALIAS ,PAGE_SIZE , - prot, __pgd_pgtable_alloc,0 ); +__create_pgd_mapping (tramp_pg_dir, pa_start,TRAMP_VALIAS , +entry_tramp_text_size (), prot, + __pgd_pgtable_alloc,NO_BLOCK_MAPPINGS );/* Map both the text and data into the kernel page table */ -__set_fixmap (FIX_ENTRY_TRAMP_TEXT , pa_start, prot); +for (i =0 ; i <DIV_ROUND_UP (entry_tramp_text_size (),PAGE_SIZE ); i++) +__set_fixmap (FIX_ENTRY_TRAMP_TEXT1 - i, + pa_start + i *PAGE_SIZE , prot); +if (IS_ENABLED (CONFIG_RANDOMIZE_BASE )) {extern char __entry_tramp_data_start[];
arch/arm64/tools/cpucaps +1 -0
@@ -44,6 +44,7 @@ MTE_ASYMM SPECTRE_V2 SPECTRE_V3A SPECTRE_V4 +SPECTRE_BHB SSBS SVE UNMAP_KERNEL_AT_EL0
arch/x86/include/asm/cpufeatures.h +1 -1
@@ -204,7 +204,7 @@/* FREE! ( 7*32+10) */ #define X86_FEATURE_PTI ( 7*32+11)/* Kernel Page Table Isolation enabled */ #define X86_FEATURE_RETPOLINE ( 7*32+12)/* "" Generic Retpoline mitigation for Spectre variant 2 */ -#define X86_FEATURE_RETPOLINE_AMD ( 7*32+13)/* "" AMD Retpoline mitigation for Spectre variant 2 */ +#define X86_FEATURE_RETPOLINE_LFENCE ( 7*32+13)/* "" Use LFENCE for Spectre variant 2 */ #define X86_FEATURE_INTEL_PPIN ( 7*32+14)/* Intel Processor Inventory Number */ #define X86_FEATURE_CDP_L2 ( 7*32+15)/* Code and Data Prioritization L2 */ #define X86_FEATURE_MSR_SPEC_CTRL ( 7*32+16)/* "" MSR SPEC_CTRL is implemented */
arch/x86/include/asm/nospec-branch.h +9 -7
@@ -84,7 +84,7 @@#ifdef CONFIG_RETPOLINE ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \ __stringify(jmp __x86_indirect_thunk_\reg), X86_FEATURE_RETPOLINE, \ - __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_AMD + __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_LFENCE#else jmp *%\reg#endif
@@ -94,7 +94,7 @@#ifdef CONFIG_RETPOLINE ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; call *%\reg), \ __stringify(call __x86_indirect_thunk_\reg), X86_FEATURE_RETPOLINE, \ - __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; call *%\reg), X86_FEATURE_RETPOLINE_AMD + __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; call *%\reg), X86_FEATURE_RETPOLINE_LFENCE#else call *%\reg#endif
@@ -146,7 +146,7 @@ extern retpoline_thunk_t __x86_indirect_thunk_array[]; "lfence;\n" \ ANNOTATE_RETPOLINE_SAFE \ "call *%[thunk_target]\n", \ - X86_FEATURE_RETPOLINE_AMD) + X86_FEATURE_RETPOLINE_LFENCE)# define THUNK_TARGET (addr) [thunk_target] "r" (addr)
@@ -176,7 +176,7 @@ extern retpoline_thunk_t __x86_indirect_thunk_array[]; "lfence;\n" \ ANNOTATE_RETPOLINE_SAFE \ "call *%[thunk_target]\n", \ - X86_FEATURE_RETPOLINE_AMD) + X86_FEATURE_RETPOLINE_LFENCE)# define THUNK_TARGET (addr) [thunk_target] "rm" (addr)#endif
@@ -188,9 +188,11 @@ extern retpoline_thunk_t __x86_indirect_thunk_array[];/* The Spectre V2 mitigation variants */ enum spectre_v2_mitigation {SPECTRE_V2_NONE , -SPECTRE_V2_RETPOLINE_GENERIC , -SPECTRE_V2_RETPOLINE_AMD , -SPECTRE_V2_IBRS_ENHANCED , +SPECTRE_V2_RETPOLINE , +SPECTRE_V2_LFENCE , +SPECTRE_V2_EIBRS , +SPECTRE_V2_EIBRS_RETPOLINE , +SPECTRE_V2_EIBRS_LFENCE , };/* The indirect branch speculation control variants */
arch/x86/kernel/alternative.c +4 -4
@@ -389,7 +389,7 @@ static int emit_indirect(int op, int reg, u8 *bytes)* * CALL *%\reg * -* It also tries to inline spectre_v2=retpoline,amd when size permits. +* It also tries to inline spectre_v2=retpoline,lfence when size permits. */ static int patch_retpoline (void *addr,struct insn *insn,u8 *bytes) {
@@ -407,7 +407,7 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)BUG_ON (reg ==4 );if (cpu_feature_enabled (X86_FEATURE_RETPOLINE ) && - !cpu_feature_enabled (X86_FEATURE_RETPOLINE_AMD )) + !cpu_feature_enabled (X86_FEATURE_RETPOLINE_LFENCE ))return -1 ; op = insn->opcode.bytes[0 ];
@@ -438,9 +438,9 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes) }/* -* For RETPOLINE_AMD: prepend the indirect CALL/JMP with an LFENCE. +* For RETPOLINE_LFENCE: prepend the indirect CALL/JMP with an LFENCE. */ -if (cpu_feature_enabled (X86_FEATURE_RETPOLINE_AMD )) { +if (cpu_feature_enabled (X86_FEATURE_RETPOLINE_LFENCE )) { bytes[i++] =0x0f ; bytes[i++] =0xae ; bytes[i++] =0xe8 ;/* LFENCE */
arch/x86/kernel/cpu/bugs.c +156 -48
308 lines not on this page · raw message
arch/x86/lib/retpoline.S +1 -1
@@ -34,7 +34,7 @@ SYM_INNER_LABEL(__x86_indirect_thunk_\reg, SYM_L_GLOBAL) ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \ __stringify(RETPOLINE \reg), X86_FEATURE_RETPOLINE, \ - __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_AMD + __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_LFENCE .endm
arch/x86/net/bpf_jit_comp.c +1 -1
@@ -394,7 +394,7 @@ static void emit_indirect_jump(u8 **pprog, int reg, u8 *ip)u8 *prog = *pprog;#ifdef CONFIG_RETPOLINE -if (cpu_feature_enabled (X86_FEATURE_RETPOLINE_AMD )) { +if (cpu_feature_enabled (X86_FEATURE_RETPOLINE_LFENCE )) {EMIT_LFENCE ();EMIT2 (0xFF ,0xE0 + reg); }else if (cpu_feature_enabled (X86_FEATURE_RETPOLINE )) {
drivers/acpi/ec.c +0 -10
@@ -2051,16 +2051,6 @@ bool acpi_ec_dispatch_gpe(void)if (acpi_any_gpe_status_set (first_ec->gpe))return true; -/* -* Cancel the SCI wakeup and process all pending events in case there -* are any wakeup ones in there. -* -* Note that if any non-EC GPEs are active at this point, the SCI will -* retrigger after the rearming in acpi_s2idle_wake(), so no events -* should be missed by canceling the wakeup here. -*/ -pm_system_cancel_wakeup (); -/* * Dispatch the EC GPE in-band, but do not report wakeup in any case * to allow the caller to process events properly after that.
drivers/acpi/sleep.c +10 -4
@@ -739,15 +739,21 @@ bool acpi_s2idle_wake(void)return true; } -/* -* Check non-EC GPE wakeups and if there are none, cancel the -* SCI-related wakeup and dispatch the EC GPE. -*/ +/* Check non-EC GPE wakeups and dispatch the EC GPE. */ if (acpi_ec_dispatch_gpe ()) {pm_pr_dbg ("ACPI non-EC GPE wakeup\n" );return true; } +/* +* Cancel the SCI wakeup and process all pending events in case +* there are any wakeup ones in there. +* +* Note that if any non-EC GPEs are active at this point, the +* SCI will retrigger after the rearming below, so no events +* should be missed by canceling the wakeup here. +*/ +pm_system_cancel_wakeup ();acpi_os_wait_events_complete ();/*
drivers/block/xen-blkfront.c +37 -26
147 lines not on this page · raw message
drivers/net/xen-netfront.c +33 -21
104 lines not on this page · raw message
drivers/xen/grant-table.c +39 -32
71 lines not on this page · raw message
tools/arch/x86/include/asm/cpufeatures.h +1 -1
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.