Linux 4.17.15
Greg KH
maintainerI'm announcing the release of the 4.17.15 kernel.
All users of the 4.17 kernel series must upgrade.
The updated 4.17.y git tree can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-4.17.y
and can be browsed at the normal kernel.org git web browser:
http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary
thanks,
greg k-h
------------
Documentation/ABI/testing/sysfs-devices-system-cpu | 24 Documentation/admin-guide/index.rst | 9 Documentation/admin-guide/kernel-parameters.txt | 78 + Documentation/admin-guide/l1tf.rst | 610 +++++++++++ Makefile | 2 arch/Kconfig | 3 arch/arm/boot/dts/imx6sx.dtsi | 2 arch/parisc/Kconfig | 2 arch/parisc/include/asm/barrier.h | 32 arch/parisc/kernel/entry.S | 2 arch/parisc/kernel/pacache.S | 1 arch/parisc/kernel/syscall.S | 4 arch/x86/Kconfig | 1 arch/x86/include/asm/apic.h | 9 arch/x86/include/asm/cpufeatures.h | 3 arch/x86/include/asm/dmi.h | 2 arch/x86/include/asm/hardirq.h | 26 arch/x86/include/asm/irqflags.h | 2 arch/x86/include/asm/kvm_host.h | 6 arch/x86/include/asm/msr-index.h | 7 arch/x86/include/asm/page_32_types.h | 9 arch/x86/include/asm/pgtable-2level.h | 17 arch/x86/include/asm/pgtable-3level.h | 37 arch/x86/include/asm/pgtable-invert.h | 32 arch/x86/include/asm/pgtable.h | 74 - arch/x86/include/asm/pgtable_64.h | 38 arch/x86/include/asm/processor.h | 17 arch/x86/include/asm/smp.h | 1 arch/x86/include/asm/topology.h | 6 arch/x86/include/asm/vmx.h | 11 arch/x86/kernel/apic/apic.c | 18 arch/x86/kernel/apic/io_apic.c | 1 arch/x86/kernel/apic/msi.c | 1 arch/x86/kernel/apic/vector.c | 1 arch/x86/kernel/cpu/amd.c | 59 - arch/x86/kernel/cpu/bugs.c | 171 ++- arch/x86/kernel/cpu/common.c | 63 - arch/x86/kernel/cpu/cpu.h | 2 arch/x86/kernel/cpu/intel.c | 7 arch/x86/kernel/cpu/microcode/core.c | 16 arch/x86/kernel/cpu/topology.c | 41 arch/x86/kernel/fpu/core.c | 1 arch/x86/kernel/hpet.c | 1 arch/x86/kernel/i8259.c | 1 arch/x86/kernel/idt.c | 1 arch/x86/kernel/irq.c | 1 arch/x86/kernel/irq_32.c | 1 arch/x86/kernel/irq_64.c | 1 arch/x86/kernel/irqinit.c | 1 arch/x86/kernel/kprobes/core.c | 5 arch/x86/kernel/paravirt.c | 14 arch/x86/kernel/setup.c | 6 arch/x86/kernel/smp.c | 1 arch/x86/kernel/smpboot.c | 25 arch/x86/kernel/time.c | 1 arch/x86/kvm/mmu.c | 1 arch/x86/kvm/vmx.c | 455 ++++++-- arch/x86/kvm/x86.c | 34 arch/x86/mm/init.c | 25 arch/x86/mm/kmmio.c | 25 arch/x86/mm/mmap.c | 21 arch/x86/mm/pageattr.c | 8 arch/x86/mm/pti.c | 1 arch/x86/platform/intel-mid/device_libs/platform_mrfld_wdt.c | 1 arch/x86/platform/uv/tlb_uv.c | 1 arch/x86/xen/enlighten.c | 1 drivers/base/cpu.c | 8 drivers/block/zram/zram_drv.c | 15 drivers/gpu/drm/i915/i915_pmu.c | 1 drivers/gpu/drm/i915/intel_lpe_audio.c | 1 drivers/net/xen-netfront.c | 8 drivers/pci/host/pci-hyperv.c | 2 drivers/scsi/qla2xxx/qla_iocb.c | 53 drivers/scsi/sr.c | 29 fs/dcache.c | 13 fs/namespace.c | 28 include/asm-generic/pgtable.h | 12 include/linux/cpu.h | 23 include/linux/swapfile.h | 2 init/main.c | 2 kernel/bpf/sockmap.c | 9 kernel/cpu.c | 284 ++++- kernel/sched/core.c | 30 kernel/sched/deadline.c | 8 kernel/sched/fair.c | 1 kernel/smp.c | 2 kernel/softirq.c | 12 kernel/stop_machine.c | 10 mm/memory.c | 37 mm/mprotect.c | 49 mm/swapfile.c | 46 tools/arch/x86/include/asm/cpufeatures.h | 23 tools/include/uapi/linux/prctl.h | 12 93 files changed, 2419 insertions(+), 381 deletions(-)
Abel Vesa (1):
cpu/hotplug: Non-SMP machines do not make use of booted_once
Al Viro (4):
root dentries need RCU-delayed freeing
make sure that __dentry_kill() always invalidates d_seq, unhashed or not fix mntput/mntput race
fix __legitimize_mnt()/mntput() race
Andi Kleen (10):
x86/speculation/l1tf: Increase 32bit PAE __PHYSICAL_PAGE_SHIFT x86/speculation/l1tf: Protect PROT_NONE PTEs against speculation x86/speculation/l1tf: Make sure the first page is always reserved x86/speculation/l1tf: Add sysfs reporting for l1tf x86/speculation/l1tf: Disallow non privileged high MMIO PROT_NONE mappings x86/speculation/l1tf: Limit swap file size to MAX_PA/2 x86/speculation/l1tf: Invert all not present mappings x86/speculation/l1tf: Make pmd/pud_mknotpresent() invert x86/mm/pat: Make set_memory_np() L1TF safe
x86/mm/kmmio: Make the tracer robust against L1TF
Arnaldo Carvalho de Melo (1):
tools headers: Synchronize prctl.h ABI header
Bart Van Assche (1):
scsi: sr: Avoid that opening a CD-ROM hangs with runtime power management enabled
Borislav Petkov (3):
x86/CPU/AMD: Do not check CPUID max ext level before parsing SMP info x86/CPU/AMD: Move TOPOEXT reenablement before reading smp_num_siblings x86/CPU/AMD: Have smp_num_siblings and cpu_llc_id always be present
Daniel Borkmann (2):
bpf, sockmap: fix leak in bpf_tcp_sendmsg wait for mem path bpf, sockmap: fix bpf_tcp_sendmsg sock error handling
Daniel Bristot de Oliveira (1):
sched/deadline: Update rq_clock of later_rq when pushing a task
David Woodhouse (1):
tools headers: Synchronise x86 cpufeatures.h for L1TF additions
Greg Kroah-Hartman (1):
Linux 4.17.15
Helge Deller (1):
parisc: Enable CONFIG_MLONGCALLS by default
Isaac J. Manjarres (1):
stop_machine: Disable preemption after queueing stopper threads
Jiri Kosina (4):
x86/speculation: Protect against userspace-userspace spectreRSB cpu/hotplug: Expose SMT control init function
x86/bugs, kvm: Introduce boot-time control of L1TF mitigations x86/speculation/l1tf: Unbreak !__HAVE_ARCH_PFN_MODIFY_ALLOWED architectures
John David Anglin (1):
parisc: Define mb() and add memory barriers to assembler unlock sequences
Josh Poimboeuf (2):
cpu/hotplug: detect SMT disabled by BIOS x86/microcode: Allow late microcode loading with SMT disabled
Juergen Gross (1):
xen/netfront: don't cache skb_shinfo()
Konrad Rzeszutek Wilk (9):
x86/bugs: Move the l1tf function and define pr_fmt properly x86/cpufeatures: Add detection of L1D cache flush support. x86/KVM: Warn user if KVM is loaded SMT and L1TF CPU bug being present x86/KVM/VMX: Add module argument for L1TF mitigation x86/KVM/VMX: Split the VMX MSR LOAD structures to have an host/guest numbers x86/KVM/VMX: Add find_msr() helper function
x86/KVM/VMX: Separate the VMX AUTOLOAD guest/host number accounting x86/KVM/VMX: Extend add_atomic_switch_msr() to allow VMENTER only MSRs x86/KVM/VMX: Use MSR save list for IA32_FLUSH_CMD if required
Linus Torvalds (4):
Mark HI and TASKLET softirq synchronous
init: rename and re-order boot_cpu_state_init() x86/speculation/l1tf: Change order of offset/type in swap entry x86/speculation/l1tf: Protect swap entries against L1TF
Masami Hiramatsu (1):
kprobes/x86: Fix %p uses in error messages
Michal Hocko (1):
x86/speculation/l1tf: Fix up pte->pfn conversion for PAE
Minchan Kim (1):
zram: remove BD_CAP_SYNCHRONOUS_IO with writeback feature
Nick Desaulniers (1):
x86/irqflags: Provide a declaration for native_save_fl
Nicolai Stange (9):
x86/KVM/VMX: Initialize the vmx_l1d_flush_pages' content x86/KVM/VMX: Don't set l1tf_flush_l1d to true from vmx_l1d_flush() x86/KVM/VMX: Replace 'vmx_l1d_flush_always' with 'vmx_l1d_flush_cond' x86/KVM/VMX: Move the l1tf_flush_l1d test to vmx_l1d_flush() x86/irq: Demote irq_cpustat_t::__softirq_pending to u16 x86/KVM/VMX: Introduce per-host-cpu analogue of l1tf_flush_l1d
x86: Don't include linux/irq.h from asm/hardirq.h x86/irq: Let interrupt handlers set kvm_cpu_l1tf_flush_l1d x86/KVM/VMX: Don't set l1tf_flush_l1d from vmx_handle_external_intr()
Oleksij Rempel (1):
ARM: dts: imx6sx: fix irq for pcie bridge
Paolo Bonzini (6):
x86/KVM/VMX: Add L1D flush algorithm
x86/KVM/VMX: Add L1D MSR based flush
x86/KVM/VMX: Add L1D flush logic
x86/speculation: Simplify sysfs report of VMX L1TF vulnerability x86/speculation: Use ARCH_CAPABILITIES to skip L1D flush on vmentry
KVM: VMX: Tell the nested hypervisor to skip L1D flush on vmentry
Peter Zijlstra (2):
x86/paravirt: Fix spectre-v2 mitigations for paravirt guests sched/smt: Update sched_smt_present at runtime
Quinn Tran (1):
scsi: qla2xxx: Fix memory leak for allocating abort IOCB
Thomas Gleixner (26):
x86/smp: Provide topology_is_primary_thread() x86/topology: Provide topology_smt_supported() cpu/hotplug: Make bringup/teardown of smp threads symmetric cpu/hotplug: Split do_cpu_down()
cpu/hotplug: Provide knobs to control SMT
x86/cpu: Remove the pointless CPU printout
x86/cpu/AMD: Remove the pointless detect_ht() call x86/cpu/common: Provide detect_ht_early() x86/cpu/topology: Provide detect_extended_topology_early() x86/cpu/intel: Evaluate smp_num_siblings early x86/cpu/AMD: Evaluate smp_num_siblings early
x86/apic: Ignore secondary threads if nosmt=force
Revert "x86/apic: Ignore secondary threads if nosmt=force" cpu/hotplug: Boot HT siblings at least once
cpu/hotplug: Online siblings when SMT control is turned on x86/litf: Introduce vmx status variable
x86/kvm: Drop L1TF MSR list approach
x86/l1tf: Handle EPT disabled state proper
x86/kvm: Move l1tf setup function
x86/kvm: Add static key for flush always
x86/kvm: Serialize L1D flush parameter setter
x86/kvm: Allow runtime control of L1D flush
cpu/hotplug: Set CPU_SMT_NOT_SUPPORTED early
Documentation: Add section about CPU vulnerabilities Documentation/l1tf: Remove Yonah processors from not vulnerable list cpu/hotplug: Fix SMT supported evaluation
Tony Luck (1):
Documentation/l1tf: Fix typos
Vlastimil Babka (4):
x86/speculation/l1tf: Extend 64bit swap file size limit x86/speculation/l1tf: Protect PAE swap entries against L1TF
x86/smp: fix non-SMP broken build due to redefinition of apic_id_is_primary_thread x86/init: fix build with CONFIG_SWAP=n
Greg KH
maintainerDocumentation/ABI/testing/sysfs-devices-system-cpu +24 -0
@@ -479,6 +479,7 @@ What: /sys/devices/system/cpu/vulnerabilities /sys/devices/system/cpu/vulnerabilities/spectre_v1 /sys/devices/system/cpu/vulnerabilities/spectre_v2 /sys/devices/system/cpu/vulnerabilities/spec_store_bypass + /sys/devices/system/cpu/vulnerabilities/l1tf Date: January 2018 Contact: Linux kernel mailing list <[email protected]> Description: Information about CPU vulnerabilities
@@ -490,3 +491,26 @@ Description: Information about CPU vulnerabilities "Not affected" CPU is not affected by the vulnerability "Vulnerable" CPU is affected and no mitigation in effect "Mitigation: $M" CPU is affected and mitigation $M is in effect + + Details about the l1tf file can be found in + Documentation/admin-guide/l1tf.rst + +What: /sys/devices/system/cpu/smt + /sys/devices/system/cpu/smt/active + /sys/devices/system/cpu/smt/control +Date: June 2018 +Contact: Linux kernel mailing list <[email protected]> +Description: Control Symetric Multi Threading (SMT) + + active: Tells whether SMT is active (enabled and siblings online) + + control: Read/write interface to control SMT. Possible + values: + + "on" SMT is enabled + "off" SMT is disabled + "forceoff" SMT is force disabled. Cannot be changed. + "notsupported" SMT is not supported by the CPU + + If control status is "forceoff" or "notsupported" writes + are rejected.
Documentation/admin-guide/index.rst +9 -0
@@ -17,6 +17,15 @@ etc. kernel-parameters devices +This section describes CPU vulnerabilities and provides an overview of the +possible mitigations along with guidance for selecting mitigations if they +are configurable at compile, boot or run time. + +.. toctree:: + :maxdepth: 1 + + l1tf + Here is a set of documents aimed at users who are trying to track down problems and bugs in particular.
Documentation/admin-guide/kernel-parameters.txt +78 -0
94 lines not on this page · raw message
Documentation/admin-guide/l1tf.rst new file +610 -0
610 lines not on this page · raw message
Makefile +1 -1
@@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 VERSION = 4 PATCHLEVEL = 17 -SUBLEVEL = 14 +SUBLEVEL = 15 EXTRAVERSION = NAME = Merciless Moray
arch/Kconfig +3 -0
@@ -13,6 +13,9 @@ config KEXEC_CORE config HAVE_IMA_KEXEC bool +config HOTPLUG_SMT + bool + config OPROFILE tristate "OProfile system profiling" depends on PROFILING
arch/arm/boot/dts/imx6sx.dtsi +1 -1
@@ -1351,7 +1351,7 @@ ranges = <0x81000000 0 0 0x08f80000 0 0x00010000 /* downstream I/O */ 0x82000000 0 0x08000000 0x08000000 0 0x00f00000>; /* non-prefetchable memory */ num-lanes = <1>; - interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH>; + interrupts = <GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>; interrupt-names = "msi"; #interrupt-cells = <1>; interrupt-map-mask = <0 0 0 0x7>;
arch/parisc/Kconfig +1 -1
@@ -199,7 +199,7 @@ config PREFETCH config MLONGCALLS bool "Enable the -mlong-calls compiler option for big kernels" - def_bool y if (!MODULES) + default y depends on PA8X00 help If you configure the kernel to include many drivers built-in instead
arch/parisc/include/asm/barrier.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_BARRIER_H +#define __ASM_BARRIER_H + +#ifndef __ASSEMBLY__ + +/* The synchronize caches instruction executes as a nop on systems in +which all memory references are performed in order. */ +#define synchronize_caches () __asm__ __volatile__ ("sync" : : : "memory") + +#if defined(CONFIG_SMP ) +#define mb () do { synchronize_caches(); } while (0) +#define rmb () mb() +#define wmb () mb() +#define dma_rmb () mb() +#define dma_wmb () mb() +#else +#define mb () barrier() +#define rmb () barrier() +#define wmb () barrier() +#define dma_rmb () barrier() +#define dma_wmb () barrier() +#endif + +#define __smp_mb () mb() +#define __smp_rmb () mb() +#define __smp_wmb () mb() + +#include <asm-generic/barrier.h> + +#endif /* !__ASSEMBLY__ */ +#endif /* __ASM_BARRIER_H */
arch/parisc/kernel/entry.S +2 -0
@@ -481,6 +481,8 @@ /* Release pa_tlb_lock lock without reloading lock address. */ .macro tlb_unlock0 spc,tmp #ifdef CONFIG_SMP + or,COND(=) %r0,\spc,%r0 + sync or,COND(=) %r0,\spc,%r0 stw \spc,0(\tmp) #endif
arch/parisc/kernel/pacache.S +1 -0
@@ -353,6 +353,7 @@ ENDPROC_CFI(flush_data_cache_local) .macro tlb_unlock la,flags,tmp #ifdef CONFIG_SMP ldi 1,\tmp + sync stw \tmp,0(\la) mtsm \flags #endif
arch/parisc/kernel/syscall.S +4 -0
@@ -633,6 +633,7 @@ cas_action: sub,<> %r28, %r25, %r0 2: stw,ma %r24, 0(%r26) /* Free lock */ + sync stw,ma %r20, 0(%sr2,%r20) #if ENABLE_LWS_DEBUG /* Clear thread register indicator */
@@ -647,6 +648,7 @@ cas_action: 3: /* Error occurred on load or store */ /* Free lock */ + sync stw %r20, 0(%sr2,%r20) #if ENABLE_LWS_DEBUG stw %r0, 4(%sr2,%r20)
@@ -848,6 +850,7 @@ cas2_action: cas2_end: /* Free lock */ + sync stw,ma %r20, 0(%sr2,%r20) /* Enable interrupts */ ssm PSW_SM_I, %r0
@@ -858,6 +861,7 @@ cas2_end: 22: /* Error occurred on load or store */ /* Free lock */ + sync stw %r20, 0(%sr2,%r20) ssm PSW_SM_I, %r0 ldo 1(%r0),%r28
arch/x86/Kconfig +1 -0
@@ -183,6 +183,7 @@ config X86 select HAVE_SYSCALL_TRACEPOINTS select HAVE_UNSTABLE_SCHED_CLOCK select HAVE_USER_RETURN_NOTIFIER + select HOTPLUG_SMT if SMP select IRQ_FORCED_THREADING select PCI_LOCKLESS_CONFIG select PERF_EVENTS
arch/x86/include/asm/apic.h +9 -0
@@ -10,6 +10,7 @@#include <asm/fixmap.h> #include <asm/mpspec.h> #include <asm/msr.h> +#include <asm/hardirq.h> #define ARCH_APICTIMER_STOPS_ON_C3 1
@@ -502,12 +503,19 @@ extern int default_check_phys_apicid_present(int phys_apicid);#endif /* CONFIG_X86_LOCAL_APIC */ +#ifdef CONFIG_SMP +bool apic_id_is_primary_thread (unsigned int id); +#else +static inline bool apic_id_is_primary_thread (unsigned int id) {return false; } +#endif +extern void irq_enter (void );extern void irq_exit (void );static inline void entering_irq (void ) {irq_enter (); +kvm_set_cpu_l1tf_flush_l1d (); }static inline void entering_ack_irq (void )
@@ -520,6 +528,7 @@ static inline void ipi_entering_ack_irq(void) {irq_enter ();ack_APIC_irq (); +kvm_set_cpu_l1tf_flush_l1d (); }static inline void exiting_irq (void )
arch/x86/include/asm/cpufeatures.h +3 -0
@@ -219,6 +219,7 @@#define X86_FEATURE_IBPB ( 7*32+26)/* Indirect Branch Prediction Barrier */ #define X86_FEATURE_STIBP ( 7*32+27)/* Single Thread Indirect Branch Predictors */ #define X86_FEATURE_ZEN ( 7*32+28)/* "" CPU is AMD family 0x17 (Zen) */ +#define X86_FEATURE_L1TF_PTEINV ( 7*32+29)/* "" L1TF workaround PTE inversion */ /* Virtualization flags: Linux defined, word 8 */ #define X86_FEATURE_TPR_SHADOW ( 8*32+ 0)/* Intel TPR Shadow */
@@ -339,6 +340,7 @@#define X86_FEATURE_PCONFIG (18*32+18)/* Intel PCONFIG */ #define X86_FEATURE_SPEC_CTRL (18*32+26)/* "" Speculation Control (IBRS + IBPB) */ #define X86_FEATURE_INTEL_STIBP (18*32+27)/* "" Single Thread Indirect Branch Predictors */ +#define X86_FEATURE_FLUSH_L1D (18*32+28)/* Flush L1D cache */ #define X86_FEATURE_ARCH_CAPABILITIES (18*32+29)/* IA32_ARCH_CAPABILITIES MSR (Intel) */ #define X86_FEATURE_SPEC_CTRL_SSBD (18*32+31)/* "" Speculative Store Bypass Disable */
@@ -371,5 +373,6 @@#define X86_BUG_SPECTRE_V1 X86_BUG(15)/* CPU is affected by Spectre variant 1 attack with conditional branches */ #define X86_BUG_SPECTRE_V2 X86_BUG(16)/* CPU is affected by Spectre variant 2 attack with indirect branches */ #define X86_BUG_SPEC_STORE_BYPASS X86_BUG(17)/* CPU is affected by speculative store bypass attack */ +#define X86_BUG_L1TF X86_BUG(18)/* CPU is affected by L1 Terminal Fault */ #endif /* _ASM_X86_CPUFEATURES_H */
arch/x86/include/asm/dmi.h +1 -1
@@ -4,8 +4,8 @@#include <linux/compiler.h> #include <linux/init.h> +#include <linux/io.h> -#include <asm/io.h> #include <asm/setup.h> static __always_inline __init void *dmi_alloc(unsigned len)
arch/x86/include/asm/hardirq.h +24 -2
@@ -3,10 +3,12 @@#define _ASM_X86_HARDIRQ_H#include <linux/threads.h> -#include <linux/irq.h> typedef struct { - unsigned int __softirq_pending; + u16 __softirq_pending; +#if IS_ENABLED(CONFIG_KVM_INTEL) + u8 kvm_cpu_l1tf_flush_l1d; +#endif unsigned int __nmi_count; /* arch dependent */ #ifdef CONFIG_X86_LOCAL_APIC unsigned int apic_timer_irqs; /* arch dependent */
@@ -66,4 +68,24 @@ extern u64 arch_irq_stat_cpu(unsigned int cpu);extern u64 arch_irq_stat (void );#define arch_irq_stat arch_irq_stat + +#if IS_ENABLED (CONFIG_KVM_INTEL ) +static inline void kvm_set_cpu_l1tf_flush_l1d (void ) +{ +__this_cpu_write (irq_stat.kvm_cpu_l1tf_flush_l1d,1 ); +} + +static inline void kvm_clear_cpu_l1tf_flush_l1d (void ) +{ +__this_cpu_write (irq_stat.kvm_cpu_l1tf_flush_l1d,0 ); +} + +static inline bool kvm_get_cpu_l1tf_flush_l1d (void ) +{ +return __this_cpu_read (irq_stat.kvm_cpu_l1tf_flush_l1d); +} +#else /* !IS_ENABLED(CONFIG_KVM_INTEL) */ +static inline void kvm_set_cpu_l1tf_flush_l1d (void ) { } +#endif /* IS_ENABLED(CONFIG_KVM_INTEL) */ +#endif /* _ASM_X86_HARDIRQ_H */
arch/x86/include/asm/irqflags.h +2 -0
@@ -13,6 +13,8 @@* Interrupt control: */ +/* Declaration required for gcc < 4.9 to prevent -Werror=missing-prototypes */ +extern inline unsigned long native_save_fl (void );extern inline unsigned long native_save_fl (void ) {unsigned long flags;
arch/x86/include/asm/kvm_host.h +6 -0
@@ -17,6 +17,7 @@#include <linux/tracepoint.h> #include <linux/cpumask.h> #include <linux/irq_work.h> +#include <linux/irq.h> #include <linux/kvm.h> #include <linux/kvm_para.h>
@@ -711,6 +712,9 @@ struct kvm_vcpu_arch {/* be preempted when it's in kernel-mode(cpl=0) */ bool preempted_in_kernel; + +/* Flush the L1 Data cache for L1TF mitigation on VMENTER */ +bool l1tf_flush_l1d; };struct kvm_lpage_info {
@@ -879,6 +883,7 @@ struct kvm_vcpu_stat {u64 signal_exits;u64 irq_window_exits;u64 nmi_window_exits; +u64 l1d_flush;u64 halt_exits;u64 halt_successful_poll;u64 halt_attempted_poll;
@@ -1410,6 +1415,7 @@ int kvm_cpu_get_interrupt(struct kvm_vcpu *v);void kvm_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event);void kvm_vcpu_reload_apic_access_page (struct kvm_vcpu *vcpu); +u64 kvm_get_arch_capabilities (void );void kvm_define_shared_msr (unsigned index,u32 msr);int kvm_set_shared_msr (unsigned index,u64 val,u64 mask);
arch/x86/include/asm/msr-index.h +7 -0
@@ -70,12 +70,19 @@#define MSR_IA32_ARCH_CAPABILITIES 0x0000010a#define ARCH_CAP_RDCL_NO (1 << 0)/* Not susceptible to Meltdown */ #define ARCH_CAP_IBRS_ALL (1 << 1)/* Enhanced IBRS support */ +#define ARCH_CAP_SKIP_VMENTRY_L1DFLUSH (1 << 3)/* Skip L1D flush on vmentry */ #define ARCH_CAP_SSB_NO (1 << 4)/* * Not susceptible to Speculative Store Bypass * attack, so no Speculative Store Bypass * control required. */ +#define MSR_IA32_FLUSH_CMD 0x0000010b +#define L1D_FLUSH (1 << 0)/* +* Writeback and invalidate the +* L1 data cache. +*/ +#define MSR_IA32_BBL_CR_CTL 0x00000119#define MSR_IA32_BBL_CR_CTL3 0x0000011e
arch/x86/include/asm/page_32_types.h +7 -2
@@ -29,8 +29,13 @@#define N_EXCEPTION_STACKS 1#ifdef CONFIG_X86_PAE -/* 44=32+12, the limit we can fit into an unsigned long pfn */ -#define __PHYSICAL_MASK_SHIFT 44 +/* +* This is beyond the 44 bit limit imposed by the 32bit long pfns, +* but we need the full mask to make sure inverted PROT_NONE +* entries have all the host bits set in a guest. +* The real limit is still 44 bits. +*/ +#define __PHYSICAL_MASK_SHIFT 52#define __VIRTUAL_MASK_SHIFT 32#else /* !CONFIG_X86_PAE */
arch/x86/include/asm/pgtable-2level.h +17 -0
@@ -95,4 +95,21 @@ static inline unsigned long pte_bitop(unsigned long value, unsigned int rightshi#define __pte_to_swp_entry (pte) ((swp_entry_t) { (pte).pte_low })#define __swp_entry_to_pte (x) ((pte_t) { .pte = (x).val }) +/* No inverted PFNs on 2 level page tables */ + +static inline u64 protnone_mask (u64 val) +{ +return 0 ; +} + +static inline u64 flip_protnone_guard (u64 oldval,u64 val,u64 mask) +{ +return val; +} + +static inline bool __pte_needs_invert (u64 val) +{ +return false; +} +#endif /* _ASM_X86_PGTABLE_2LEVEL_H */
arch/x86/include/asm/pgtable-3level.h +35 -2
@@ -241,12 +241,43 @@ static inline pud_t native_pudp_get_and_clear(pud_t *pudp)#endif /* Encode and de-code a swap entry */ +#define SWP_TYPE_BITS 5 + +#define SWP_OFFSET_FIRST_BIT (_PAGE_BIT_PROTNONE + 1) + +/* We always extract/encode the offset by shifting it all the way up, and then down again */ +#define SWP_OFFSET_SHIFT (SWP_OFFSET_FIRST_BIT + SWP_TYPE_BITS) +#define MAX_SWAPFILES_CHECK () BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > 5)#define __swp_type (x) (((x).val) & 0x1f)#define __swp_offset (x) ((x).val >> 5)#define __swp_entry (type, offset) ((swp_entry_t){(type) | (offset) << 5}) -#define __pte_to_swp_entry (pte) ((swp_entry_t){ (pte).pte_high }) -#define __swp_entry_to_pte (x) ((pte_t){ { .pte_high = (x).val } }) + +/* +* Normally, __swp_entry() converts from arch-independent swp_entry_t to +* arch-dependent swp_entry_t, and __swp_entry_to_pte() just stores the result +* to pte. But here we have 32bit swp_entry_t and 64bit pte, and need to use the +* whole 64 bits. Thus, we shift the "real" arch-dependent conversion to +* __swp_entry_to_pte() through the following helper macro based on 64bit +* __swp_entry(). +*/ +#define __swp_pteval_entry (type, offset) ((pteval_t) { \ + (~(pteval_t)(offset) << SWP_OFFSET_SHIFT >> SWP_TYPE_BITS) \ + | ((pteval_t)(type) << (64 - SWP_TYPE_BITS)) }) + +#define __swp_entry_to_pte (x) ((pte_t){ .pte = \ + __swp_pteval_entry(__swp_type(x), __swp_offset(x)) }) +/* +* Analogically, __pte_to_swp_entry() doesn't just extract the arch-dependent +* swp_entry_t, but also has to convert it from 64bit to the 32bit +* intermediate representation, using the following macros based on 64bit +* __swp_type() and __swp_offset(). +*/ +#define __pteval_swp_type (x) ((unsigned long)((x).pte >> (64 - SWP_TYPE_BITS))) +#define __pteval_swp_offset (x) ((unsigned long)(~((x).pte) << SWP_TYPE_BITS >> SWP_OFFSET_SHIFT)) + +#define __pte_to_swp_entry (pte) (__swp_entry(__pteval_swp_type(pte), \ + __pteval_swp_offset(pte)))#define gup_get_pte gup_get_pte/*
@@ -295,4 +326,6 @@ static inline pte_t gup_get_pte(pte_t *ptep) return pte; } +#include <asm/pgtable-invert.h> +#endif /* _ASM_X86_PGTABLE_3LEVEL_H */
arch/x86/include/asm/pgtable-invert.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_PGTABLE_INVERT_H +#define _ASM_PGTABLE_INVERT_H 1 + +#ifndef __ASSEMBLY__ + +static inline bool __pte_needs_invert (u64 val) +{ +return !(val & _PAGE_PRESENT); +} + +/* Get a mask to xor with the page table entry to get the correct pfn. */ +static inline u64 protnone_mask (u64 val) +{ +return __pte_needs_invert (val) ? ~0ull :0 ; +} + +static inline u64 flip_protnone_guard (u64 oldval,u64 val,u64 mask) +{ +/* +* When a PTE transitions from NONE to !NONE or vice-versa +* invert the PFN part to stop speculation. +* pte_pfn undoes this when needed. +*/ +if (__pte_needs_invert (oldval) !=__pte_needs_invert (val)) + val = (val & ~mask) | (~val & mask); +return val; +} + +#endif /* __ASSEMBLY__ */ + +#endif
arch/x86/include/asm/pgtable.h +51 -23
140 lines not on this page · raw message
arch/x86/include/asm/pgtable_64.h +27 -11
@@ -273,7 +273,7 @@ static inline int pgd_large(pgd_t pgd) { return 0; }* * | ... | 11| 10| 9|8|7|6|5| 4| 3|2| 1|0| <- bit number * | ... |SW3|SW2|SW1|G|L|D|A|CD|WT|U| W|P| <- bit names -* | OFFSET (14->63) | TYPE (9-13) |0|0|X|X| X| X|X|SD|0| <- swp entry +* | TYPE (59-63) | ~OFFSET (9-58) |0|0|X|X| X| X|X|SD|0| <- swp entry * * G (8) is aliased and used as a PROT_NONE indicator for * !present ptes. We need to start storing swap entries above
@@ -286,20 +286,34 @@ static inline int pgd_large(pgd_t pgd) { return 0; }* * Bit 7 in swp entry should be 0 because pmd_present checks not only P, * but also L and G. +* +* The offset is inverted by a binary not operation to make the high +* physical bits set. */ -#define SWP_TYPE_FIRST_BIT (_PAGE_BIT_PROTNONE + 1) -#define SWP_TYPE_BITS 5 -/* Place the offset above the type: */ -#define SWP_OFFSET_FIRST_BIT (SWP_TYPE_FIRST_BIT + SWP_TYPE_BITS) +#define SWP_TYPE_BITS 5 + +#define SWP_OFFSET_FIRST_BIT (_PAGE_BIT_PROTNONE + 1) + +/* We always extract/encode the offset by shifting it all the way up, and then down again */ +#define SWP_OFFSET_SHIFT (SWP_OFFSET_FIRST_BIT+SWP_TYPE_BITS)#define MAX_SWAPFILES_CHECK () BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > SWP_TYPE_BITS) -#define __swp_type (x) (((x).val >> (SWP_TYPE_FIRST_BIT)) \ - & ((1U << SWP_TYPE_BITS) - 1)) -#define __swp_offset (x) ((x).val >> SWP_OFFSET_FIRST_BIT) -#define __swp_entry (type, offset) ((swp_entry_t) { \ - ((type) << (SWP_TYPE_FIRST_BIT)) \ - | ((offset) << SWP_OFFSET_FIRST_BIT) }) +/* Extract the high bits for type */ +#define __swp_type (x) ((x).val >> (64 - SWP_TYPE_BITS)) + +/* Shift up (to get rid of type), then down to get value */ +#define __swp_offset (x) (~(x).val << SWP_TYPE_BITS >> SWP_OFFSET_SHIFT) + +/* +* Shift the offset up "too far" by TYPE bits, then down again +* The offset is inverted by a binary not operation to make the high +* physical bits set. +*/ +#define __swp_entry (type, offset) ((swp_entry_t) { \ + (~(unsigned long)(offset) << SWP_OFFSET_SHIFT >> SWP_TYPE_BITS) \ + | ((unsigned long)(type) << (64-SWP_TYPE_BITS)) }) +#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val((pte)) })#define __pmd_to_swp_entry (pmd) ((swp_entry_t) { pmd_val((pmd)) })#define __swp_entry_to_pte (x) ((pte_t) { .pte = (x).val })
@@ -343,5 +357,7 @@ static inline bool gup_fast_permitted(unsigned long start, int nr_pages,return true; } +#include <asm/pgtable-invert.h> +#endif /* !__ASSEMBLY__ */ #endif /* _ASM_X86_PGTABLE_64_H */
arch/x86/include/asm/processor.h +17 -0
@@ -181,6 +181,11 @@ extern const struct seq_operations cpuinfo_op;extern void cpu_detect (struct cpuinfo_x86 *c); +static inline unsigned long l1tf_pfn_limit (void ) +{ +return BIT (boot_cpu_data.x86_phys_bits -1 -PAGE_SHIFT ) -1 ; +} +extern void early_cpu_init (void );extern void identify_boot_cpu (void );extern void identify_secondary_cpu (struct cpuinfo_x86 *);
@@ -986,4 +991,16 @@ bool xen_set_default_idle(void);void stop_this_cpu (void *dummy);void df_debug (struct pt_regs *regs,long error_code);void microcode_check (void ); + +enum l1tf_mitigations { +L1TF_MITIGATION_OFF , +L1TF_MITIGATION_FLUSH_NOWARN , +L1TF_MITIGATION_FLUSH , +L1TF_MITIGATION_FLUSH_NOSMT , +L1TF_MITIGATION_FULL , +L1TF_MITIGATION_FULL_FORCE +}; + +extern enum l1tf_mitigations l1tf_mitigation; +#endif /* _ASM_X86_PROCESSOR_H */
arch/x86/include/asm/smp.h +0 -1
@@ -171,7 +171,6 @@ static inline int wbinvd_on_all_cpus(void)wbinvd (); return 0; } -#define smp_num_siblings 1#endif /* CONFIG_SMP */ extern unsigned disabled_cpus;
arch/x86/include/asm/topology.h +5 -1
@@ -123,13 +123,17 @@ static inline int topology_max_smt_threads(void) }int topology_update_package_map (unsigned int apicid,unsigned int cpu); -extern int topology_phys_to_logical_pkg (unsigned int pkg); +int topology_phys_to_logical_pkg (unsigned int pkg); +bool topology_is_primary_thread (unsigned int cpu); +bool topology_smt_supported (void );#else #define topology_max_packages () (1)static inline int topology_update_package_map (unsigned int apicid,unsigned int cpu) {return 0 ; }static inline int topology_phys_to_logical_pkg (unsigned int pkg) {return 0 ; }static inline int topology_max_smt_threads (void ) {return 1 ; } +static inline bool topology_is_primary_thread (unsigned int cpu) {return true; } +static inline bool topology_smt_supported (void ) {return false; }#endif static inline void arch_fix_phys_package_id (int num,u32 slot)
arch/x86/include/asm/vmx.h +11 -0
@@ -574,4 +574,15 @@ enum vm_instruction_error_number { VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID = 28, }; +enum vmx_l1d_flush_state { +VMENTER_L1D_FLUSH_AUTO , +VMENTER_L1D_FLUSH_NEVER , +VMENTER_L1D_FLUSH_COND , +VMENTER_L1D_FLUSH_ALWAYS , +VMENTER_L1D_FLUSH_EPT_DISABLED , +VMENTER_L1D_FLUSH_NOT_REQUIRED , +}; + +extern enum vmx_l1d_flush_state l1tf_vmx_mitigation; +#endif
arch/x86/kernel/apic/apic.c +18 -0
@@ -56,6 +56,7 @@#include <asm/hypervisor.h> #include <asm/cpu_device_id.h> #include <asm/intel-family.h> +#include <asm/irq_regs.h> unsigned int num_processors;
@@ -2192,6 +2193,23 @@ static int cpuid_to_apicid[] = { [0 ... NR_CPUS - 1] = -1, }; +#ifdef CONFIG_SMP +/** +* apic_id_is_primary_thread - Check whether APIC ID belongs to a primary thread +* @id: APIC ID to check +*/ +bool apic_id_is_primary_thread (unsigned int apicid) +{ +u32 mask; + +if (smp_num_siblings ==1 ) +return true; +/* Isolate the SMT bit(s) in the APICID and check for 0 */ + mask = (1U << (fls (smp_num_siblings) -1 )) -1 ; +return !(apicid & mask); +} +#endif +/* * Should use this API to allocate logical CPU IDs to keep nr_logical_cpuids * and cpuid_to_apicid[] synchronized.
arch/x86/kernel/apic/io_apic.c +1 -0
@@ -33,6 +33,7 @@#include <linux/mm.h> #include <linux/interrupt.h> +#include <linux/irq.h> #include <linux/init.h> #include <linux/delay.h> #include <linux/sched.h>
arch/x86/kernel/apic/msi.c +1 -0
@@ -12,6 +12,7 @@*/ #include <linux/mm.h> #include <linux/interrupt.h> +#include <linux/irq.h> #include <linux/pci.h> #include <linux/dmar.h> #include <linux/hpet.h>
arch/x86/kernel/apic/vector.c +1 -0
@@ -11,6 +11,7 @@* published by the Free Software Foundation. */ #include <linux/interrupt.h> +#include <linux/irq.h> #include <linux/seq_file.h> #include <linux/init.h> #include <linux/compiler.h>
arch/x86/kernel/cpu/amd.c +27 -32
123 lines not on this page · raw message
arch/x86/kernel/cpu/bugs.c +140 -31
229 lines not on this page · raw message
arch/x86/kernel/cpu/common.c +42 -21
108 lines not on this page · raw message
arch/x86/kernel/cpu/cpu.h +2 -0
@@ -47,6 +47,8 @@ extern const struct cpu_dev *const __x86_cpu_dev_start[],extern void get_cpu_cap (struct cpuinfo_x86 *c);extern void cpu_detect_cache_sizes (struct cpuinfo_x86 *c); +extern int detect_extended_topology_early (struct cpuinfo_x86 *c); +extern int detect_ht_early (struct cpuinfo_x86 *c);unsigned int aperfmperf_get_khz (int cpu);
arch/x86/kernel/cpu/intel.c +7 -0
@@ -301,6 +301,13 @@ static void early_init_intel(struct cpuinfo_x86 *c) }check_mpx_erratum (c); + +/* +* Get the number of SMT siblings early from the extended topology +* leaf, if available. Otherwise try the legacy SMT detection. +*/ +if (detect_extended_topology_early (c) <0 ) +detect_ht_early (c); }#ifdef CONFIG_X86_32
arch/x86/kernel/cpu/microcode/core.c +12 -4
@@ -509,12 +509,20 @@ static struct platform_device *microcode_pdev;static int check_online_cpus (void ) { -if (num_online_cpus () ==num_present_cpus ()) -return 0 ; +unsigned int cpu; -pr_err ("Not all CPUs online, aborting microcode update.\n" ); +/* +* Make sure all CPUs are online. It's fine for SMT to be disabled if +* all the primary threads are still online. +*/ +for_each_present_cpu (cpu) { +if (topology_is_primary_thread (cpu) && !cpu_online (cpu)) { +pr_err ("Not all CPUs online, aborting microcode update.\n" ); +return -EINVAL ; + } + } -return -EINVAL ; +return 0 ; }static atomic_t late_cpus_in;
arch/x86/kernel/cpu/topology.c +24 -17
72 lines not on this page · raw message
arch/x86/kernel/fpu/core.c +1 -0
@@ -10,6 +10,7 @@#include <asm/fpu/signal.h> #include <asm/fpu/types.h> #include <asm/traps.h> +#include <asm/irq_regs.h> #include <linux/hardirq.h> #include <linux/pkeys.h>
arch/x86/kernel/hpet.c +1 -0
@@ -1,6 +1,7 @@#include <linux/clocksource.h> #include <linux/clockchips.h> #include <linux/interrupt.h> +#include <linux/irq.h> #include <linux/export.h> #include <linux/delay.h> #include <linux/errno.h>
arch/x86/kernel/i8259.c +1 -0
@@ -5,6 +5,7 @@#include <linux/sched.h> #include <linux/ioport.h> #include <linux/interrupt.h> +#include <linux/irq.h> #include <linux/timex.h> #include <linux/random.h> #include <linux/init.h>
arch/x86/kernel/idt.c +1 -0
@@ -8,6 +8,7 @@#include <asm/traps.h> #include <asm/proto.h> #include <asm/desc.h> +#include <asm/hw_irq.h> struct idt_data {unsigned int vector;
arch/x86/kernel/irq.c +1 -0
@@ -10,6 +10,7 @@#include <linux/ftrace.h> #include <linux/delay.h> #include <linux/export.h> +#include <linux/irq.h> #include <asm/apic.h> #include <asm/io_apic.h>
arch/x86/kernel/irq_32.c +1 -0
@@ -11,6 +11,7 @@#include <linux/seq_file.h> #include <linux/interrupt.h> +#include <linux/irq.h> #include <linux/kernel_stat.h> #include <linux/notifier.h> #include <linux/cpu.h>
arch/x86/kernel/irq_64.c +1 -0
@@ -11,6 +11,7 @@#include <linux/kernel_stat.h> #include <linux/interrupt.h> +#include <linux/irq.h> #include <linux/seq_file.h> #include <linux/delay.h> #include <linux/ftrace.h>
arch/x86/kernel/irqinit.c +1 -0
@@ -5,6 +5,7 @@#include <linux/sched.h> #include <linux/ioport.h> #include <linux/interrupt.h> +#include <linux/irq.h> #include <linux/timex.h> #include <linux/random.h> #include <linux/kprobes.h>
arch/x86/kernel/kprobes/core.c +1 -4
@@ -395,8 +395,6 @@ int __copy_instruction(u8 *dest, u8 *src, u8 *real, struct insn *insn) - (u8 *) real;if ((s64 ) (s32 ) newdisp != newdisp) {pr_err ("Kprobes error: new displacement does not fit into s32 (%llx)\n" , newdisp); -pr_err ("\tSrc: %p, Dest: %p, old disp: %x\n" , - src, real, insn->displacement.value);return 0 ; } disp = (u8 *) dest +insn_offset_displacement (insn);
@@ -640,8 +638,7 @@ static int reenter_kprobe(struct kprobe *p, struct pt_regs *regs,* Raise a BUG or we'll continue in an endless reentering loop * and eventually a stack overflow. */ -printk (KERN_WARNING "Unrecoverable kprobe detected at %p.\n" , - p->addr); +pr_err ("Unrecoverable kprobe detected.\n" );dump_kprobe (p);BUG ();default :
arch/x86/kernel/paravirt.c +10 -4
@@ -88,10 +88,12 @@ unsigned paravirt_patch_call(void *insnbuf,struct branch *b = insnbuf;unsigned long delta = (unsigned long )target - (addr+5 ); -if (tgt_clobbers & ~site_clobbers) -return len;/* target would clobber too much for this site */ -if (len <5 ) +if (len <5 ) { +#ifdef CONFIG_RETPOLINE +WARN_ONCE ("Failing to patch indirect CALL in %ps\n" , (void *)addr); +#endif return len;/* call too long for patch site */ + } b->opcode =0xe8 ;/* call */ b->delta = delta;
@@ -106,8 +108,12 @@ unsigned paravirt_patch_jmp(void *insnbuf, const void *target,struct branch *b = insnbuf;unsigned long delta = (unsigned long )target - (addr+5 ); -if (len <5 ) +if (len <5 ) { +#ifdef CONFIG_RETPOLINE +WARN_ONCE ("Failing to patch indirect JMP in %ps\n" , (void *)addr); +#endif return len;/* call too long for patch site */ + } b->opcode =0xe9 ;/* jmp */ b->delta = delta;
arch/x86/kernel/setup.c +6 -0
@@ -823,6 +823,12 @@ void __init setup_arch(char **cmdline_p)memblock_reserve (__pa_symbol (_text), (unsigned long )__bss_stop - (unsigned long )_text); +/* +* Make sure page 0 is always reserved because on systems with +* L1TF its contents can be leaked to user processes. +*/ +memblock_reserve (0 ,PAGE_SIZE ); +early_reserve_initrd ();/*
arch/x86/kernel/smp.c +1 -0
@@ -261,6 +261,7 @@ __visible void __irq_entry smp_reschedule_interrupt(struct pt_regs *regs) {ack_APIC_irq ();inc_irq_stat (irq_resched_count); +kvm_set_cpu_l1tf_flush_l1d ();if (trace_resched_ipi_enabled ()) {/*
arch/x86/kernel/smpboot.c +18 -7
@@ -80,13 +80,7 @@#include <asm/intel-family.h> #include <asm/cpu_device_id.h> #include <asm/spec-ctrl.h> - -/* Number of siblings per CPU package */ -int smp_num_siblings =1 ; -EXPORT_SYMBOL (smp_num_siblings); - -/* Last level cache ID of each logical CPU */ -DEFINE_PER_CPU_READ_MOSTLY (u16, cpu_llc_id) =BAD_APICID ; +#include <asm/hw_irq.h> /* representing HT siblings of each logical CPU */ DEFINE_PER_CPU_READ_MOSTLY (cpumask_var_t, cpu_sibling_map);
@@ -272,6 +266,23 @@ static void notrace start_secondary(void *unused)cpu_startup_entry (CPUHP_AP_ONLINE_IDLE ); } +/** +* topology_is_primary_thread - Check whether CPU is the primary SMT thread +* @cpu: CPU to check +*/ +bool topology_is_primary_thread (unsigned int cpu) +{ +return apic_id_is_primary_thread (per_cpu (x86_cpu_to_apicid, cpu)); +} + +/** +* topology_smt_supported - Check whether SMT is supported by the CPUs +*/ +bool topology_smt_supported (void ) +{ +return smp_num_siblings >1 ; +} +/** * topology_phys_to_logical_pkg - Map a physical package id to a logical *
arch/x86/kernel/time.c +1 -0
@@ -12,6 +12,7 @@#include <linux/clockchips.h> #include <linux/interrupt.h> +#include <linux/irq.h> #include <linux/i8253.h> #include <linux/time.h> #include <linux/export.h>
arch/x86/kvm/mmu.c +1 -0
@@ -3836,6 +3836,7 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code, {int r =1 ; + vcpu->arch.l1tf_flush_l1d = true;switch (vcpu->arch.apf.host_apf_reason) {default :trace_kvm_page_fault (fault_address, error_code);
arch/x86/kvm/vmx.c +377 -78
597 lines not on this page · raw message
arch/x86/kvm/x86.c +33 -1
@@ -194,6 +194,7 @@ struct kvm_stats_debugfs_item debugfs_entries[] = { { "irq_injections", VCPU_STAT(irq_injections) }, { "nmi_injections", VCPU_STAT(nmi_injections) }, { "req_event", VCPU_STAT(req_event) }, + { "l1d_flush", VCPU_STAT(l1d_flush) }, { "mmu_shadow_zapped", VM_STAT(mmu_shadow_zapped) }, { "mmu_pte_write", VM_STAT(mmu_pte_write) }, { "mmu_pte_updated", VM_STAT(mmu_pte_updated) },
@@ -1097,11 +1098,35 @@ static u32 msr_based_features[] = {static unsigned int num_msr_based_features; +u64 kvm_get_arch_capabilities (void ) +{ +u64 data; + +rdmsrl_safe (MSR_IA32_ARCH_CAPABILITIES , &data); + +/* +* If we're doing cache flushes (either "always" or "cond") +* we will do one whenever the guest does a vmlaunch/vmresume. +* If an outer hypervisor is doing the cache flush for us +* (VMENTER_L1D_FLUSH_NESTED_VM), we can safely pass that +* capability to the guest too, and if EPT is disabled we're not +* vulnerable. Overall, only VMENTER_L1D_FLUSH_NEVER will +* require a nested hypervisor to do a flush of its own. +*/ +if (l1tf_vmx_mitigation !=VMENTER_L1D_FLUSH_NEVER ) + data |=ARCH_CAP_SKIP_VMENTRY_L1DFLUSH ; + +return data; +} +EXPORT_SYMBOL_GPL (kvm_get_arch_capabilities); +static int kvm_get_msr_feature (struct kvm_msr_entry *msr) {switch (msr->index) { -case MSR_IA32_UCODE_REV :case MSR_IA32_ARCH_CAPABILITIES : + msr->data =kvm_get_arch_capabilities (); +break ; +case MSR_IA32_UCODE_REV :rdmsrl_safe (msr->index, &msr->data);break ;default :
@@ -4870,6 +4895,9 @@ static int emulator_write_std(struct x86_emulate_ctxt *ctxt, gva_t addr, void *vint kvm_write_guest_virt_system (struct kvm_vcpu *vcpu,gva_t addr,void *val,unsigned int bytes,struct x86_exception *exception) { +/* kvm_write_guest_virt_system can pull in tons of pages. */ + vcpu->arch.l1tf_flush_l1d = true; +return kvm_write_guest_virt_helper (addr, val, bytes, vcpu,PFERR_WRITE_MASK , exception); }
@@ -6046,6 +6074,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu,bool writeback = true;bool write_fault_to_spt = vcpu->arch.write_fault_to_shadow_pgtable; + vcpu->arch.l1tf_flush_l1d = true; +/* * Clear write_fault_to_shadow_pgtable here to ensure it is * never reused.
@@ -7575,6 +7605,7 @@ static int vcpu_run(struct kvm_vcpu *vcpu)struct kvm *kvm = vcpu->kvm; vcpu->srcu_idx =srcu_read_lock (&kvm->srcu); + vcpu->arch.l1tf_flush_l1d = true;for (;;) {if (kvm_vcpu_running (vcpu)) {
@@ -8694,6 +8725,7 @@ void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu)void kvm_arch_sched_in (struct kvm_vcpu *vcpu,int cpu) { + vcpu->arch.l1tf_flush_l1d = true; kvm_x86_ops->sched_in (vcpu, cpu); }
arch/x86/mm/init.c +25 -0
@@ -4,6 +4,8 @@#include <linux/swap.h> #include <linux/memblock.h> #include <linux/bootmem.h> /* for max_low_pfn */ +#include <linux/swapfile.h> +#include <linux/swapops.h> #include <asm/set_memory.h> #include <asm/e820/api.h>
@@ -880,3 +882,26 @@ void update_cache_mode_entry(unsigned entry, enum page_cache_mode cache) __cachemode2pte_tbl[cache] =__cm_idx2pte (entry); __pte2cachemode_tbl[entry] = cache; } + +#ifdef CONFIG_SWAP +unsigned long max_swapfile_size (void ) +{ +unsigned long pages; + + pages =generic_max_swapfile_size (); + +if (boot_cpu_has_bug (X86_BUG_L1TF )) { +/* Limit the swap file size to MAX_PA/2 for L1TF workaround */ +unsigned long l1tf_limit =l1tf_pfn_limit () +1 ; +/* +* We encode swap offsets also with 3 bits below those for pfn +* which makes the usable limit higher. +*/ +#if CONFIG_PGTABLE_LEVELS >2 + l1tf_limit <<=PAGE_SHIFT -SWP_OFFSET_FIRST_BIT ; +#endif + pages = min_t(unsigned long, l1tf_limit, pages); + } +return pages; +} +#endif
arch/x86/mm/kmmio.c +15 -10
@@ -126,24 +126,29 @@ static struct kmmio_fault_page *get_kmmio_fault_page(unsigned long addr)static void clear_pmd_presence (pmd_t *pmd,bool clear,pmdval_t *old) { +pmd_t new_pmd;pmdval_t v =pmd_val (*pmd);if (clear) { - *old = v & _PAGE_PRESENT; - v &= ~_PAGE_PRESENT; - }else /* presume this has been called with clear==true previously */ - v |= *old; -set_pmd (pmd,__pmd (v)); + *old = v; + new_pmd =pmd_mknotpresent (*pmd); + }else { +/* Presume this has been called with clear==true previously */ + new_pmd =__pmd (*old); + } +set_pmd (pmd, new_pmd); }static void clear_pte_presence (pte_t *pte,bool clear,pteval_t *old) {pteval_t v =pte_val (*pte);if (clear) { - *old = v & _PAGE_PRESENT; - v &= ~_PAGE_PRESENT; - }else /* presume this has been called with clear==true previously */ - v |= *old; -set_pte_atomic (pte,__pte (v)); + *old = v; +/* Nothing should care about address */ +pte_clear (&init_mm,0 , pte); + }else { +/* Presume this has been called with clear==true previously */ +set_pte_atomic (pte,__pte (*old)); + } }static int clear_page_presence (struct kmmio_fault_page *f,bool clear)
arch/x86/mm/mmap.c +21 -0
@@ -240,3 +240,24 @@ int valid_mmap_phys_addr_range(unsigned long pfn, size_t count)return phys_addr_valid (addr + count -1 ); } + +/* +* Only allow root to set high MMIO mappings to PROT_NONE. +* This prevents an unpriv. user to set them to PROT_NONE and invert +* them, then pointing to valid memory for L1TF speculation. +* +* Note: for locked down kernels may want to disable the root override. +*/ +bool pfn_modify_allowed (unsigned long pfn,pgprot_t prot) +{ +if (!boot_cpu_has_bug (X86_BUG_L1TF )) +return true; +if (!__pte_needs_invert (pgprot_val (prot))) +return true; +/* If it's real memory always allow */ +if (pfn_valid (pfn)) +return true; +if (pfn >l1tf_pfn_limit () && !capable (CAP_SYS_ADMIN )) +return false; +return true; +}
arch/x86/mm/pageattr.c +4 -4
@@ -1014,8 +1014,8 @@ static long populate_pmd(struct cpa_data *cpa, pmd =pmd_offset (pud, start); -set_pmd (pmd,__pmd (cpa->pfn <<PAGE_SHIFT | _PAGE_PSE | -massage_pgprot (pmd_pgprot))); +set_pmd (pmd,pmd_mkhuge (pfn_pmd (cpa->pfn, +canon_pgprot (pmd_pgprot)))); start +=PMD_SIZE ; cpa->pfn +=PMD_SIZE >>PAGE_SHIFT ;
@@ -1087,8 +1087,8 @@ static int populate_pud(struct cpa_data *cpa, unsigned long start, p4d_t *p4d,* Map everything starting from the Gb boundary, possibly with 1G pages */ while (boot_cpu_has (X86_FEATURE_GBPAGES ) && end - start >=PUD_SIZE ) { -set_pud (pud,__pud (cpa->pfn <<PAGE_SHIFT | _PAGE_PSE | -massage_pgprot (pud_pgprot))); +set_pud (pud,pud_mkhuge (pfn_pud (cpa->pfn, +canon_pgprot (pud_pgprot)))); start +=PUD_SIZE ; cpa->pfn +=PUD_SIZE >>PAGE_SHIFT ;
arch/x86/mm/pti.c +1 -0
@@ -45,6 +45,7 @@#include <asm/pgalloc.h> #include <asm/tlbflush.h> #include <asm/desc.h> +#include <asm/sections.h>
Lines of this file are not shown. View the raw message.
arch/x86/platform/intel-mid/device_libs/platform_mrfld_wdt.c +1 -0
Lines of this file are not shown. View the raw message.
drivers/scsi/qla2xxx/qla_iocb.c +27 -26
53 lines not on this page · raw message
kernel/cpu.c +271 -13
284 lines not on this page · raw message
mm/mprotect.c +49 -0
49 lines not on this page · raw message
mm/swapfile.c +30 -16
46 lines not on this page · raw message
tools/arch/x86/include/asm/cpufeatures.h +17 -6
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.