Skip to contentSkip to conversation

Linux 4.17.15

DISCUSSION Greg KH 1 participant 2 messages Aug 16, 2018 Stable branch

lore
Share & tools
lore

Greg KH

author maintainer
#0 lore raw

I'm announcing the release of the 4.17.15 kernel.

All users of the 4.17 kernel series must upgrade.

The updated 4.17.y git tree can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-4.17.y
and can be browsed at the normal kernel.org git web browser:
http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary

thanks,

greg k-h

------------

 Documentation/ABI/testing/sysfs-devices-system-cpu           |   24 
 Documentation/admin-guide/index.rst                          |    9 
 Documentation/admin-guide/kernel-parameters.txt              |   78 +
 Documentation/admin-guide/l1tf.rst                           |  610 +++++++++++
 Makefile                                                     |    2 
 arch/Kconfig                                                 |    3 
 arch/arm/boot/dts/imx6sx.dtsi                                |    2 
 arch/parisc/Kconfig                                          |    2 
 arch/parisc/include/asm/barrier.h                            |   32 
 arch/parisc/kernel/entry.S                                   |    2 
 arch/parisc/kernel/pacache.S                                 |    1 
 arch/parisc/kernel/syscall.S                                 |    4 
 arch/x86/Kconfig                                             |    1 
 arch/x86/include/asm/apic.h                                  |    9 
 arch/x86/include/asm/cpufeatures.h                           |    3 
 arch/x86/include/asm/dmi.h                                   |    2 
 arch/x86/include/asm/hardirq.h                               |   26 
 arch/x86/include/asm/irqflags.h                              |    2 
 arch/x86/include/asm/kvm_host.h                              |    6 
 arch/x86/include/asm/msr-index.h                             |    7 
 arch/x86/include/asm/page_32_types.h                         |    9 
 arch/x86/include/asm/pgtable-2level.h                        |   17 
 arch/x86/include/asm/pgtable-3level.h                        |   37 
 arch/x86/include/asm/pgtable-invert.h                        |   32 
 arch/x86/include/asm/pgtable.h                               |   74 -
 arch/x86/include/asm/pgtable_64.h                            |   38 
 arch/x86/include/asm/processor.h                             |   17 
 arch/x86/include/asm/smp.h                                   |    1 
 arch/x86/include/asm/topology.h                              |    6 
 arch/x86/include/asm/vmx.h                                   |   11 
 arch/x86/kernel/apic/apic.c                                  |   18 
 arch/x86/kernel/apic/io_apic.c                               |    1 
 arch/x86/kernel/apic/msi.c                                   |    1 
 arch/x86/kernel/apic/vector.c                                |    1 
 arch/x86/kernel/cpu/amd.c                                    |   59 -
 arch/x86/kernel/cpu/bugs.c                                   |  171 ++-
 arch/x86/kernel/cpu/common.c                                 |   63 -
 arch/x86/kernel/cpu/cpu.h                                    |    2 
 arch/x86/kernel/cpu/intel.c                                  |    7 
 arch/x86/kernel/cpu/microcode/core.c                         |   16 
 arch/x86/kernel/cpu/topology.c                               |   41 
 arch/x86/kernel/fpu/core.c                                   |    1 
 arch/x86/kernel/hpet.c                                       |    1 
 arch/x86/kernel/i8259.c                                      |    1 
 arch/x86/kernel/idt.c                                        |    1 
 arch/x86/kernel/irq.c                                        |    1 
 arch/x86/kernel/irq_32.c                                     |    1 
 arch/x86/kernel/irq_64.c                                     |    1 
 arch/x86/kernel/irqinit.c                                    |    1 
 arch/x86/kernel/kprobes/core.c                               |    5 
 arch/x86/kernel/paravirt.c                                   |   14 
 arch/x86/kernel/setup.c                                      |    6 
 arch/x86/kernel/smp.c                                        |    1 
 arch/x86/kernel/smpboot.c                                    |   25 
 arch/x86/kernel/time.c                                       |    1 
 arch/x86/kvm/mmu.c                                           |    1 
 arch/x86/kvm/vmx.c                                           |  455 ++++++--
 arch/x86/kvm/x86.c                                           |   34 
 arch/x86/mm/init.c                                           |   25 
 arch/x86/mm/kmmio.c                                          |   25 
 arch/x86/mm/mmap.c                                           |   21 
 arch/x86/mm/pageattr.c                                       |    8 
 arch/x86/mm/pti.c                                            |    1 
 arch/x86/platform/intel-mid/device_libs/platform_mrfld_wdt.c |    1 
 arch/x86/platform/uv/tlb_uv.c                                |    1 
 arch/x86/xen/enlighten.c                                     |    1 
 drivers/base/cpu.c                                           |    8 
 drivers/block/zram/zram_drv.c                                |   15 
 drivers/gpu/drm/i915/i915_pmu.c                              |    1 
 drivers/gpu/drm/i915/intel_lpe_audio.c                       |    1 
 drivers/net/xen-netfront.c                                   |    8 
 drivers/pci/host/pci-hyperv.c                                |    2 
 drivers/scsi/qla2xxx/qla_iocb.c                              |   53 
 drivers/scsi/sr.c                                            |   29 
 fs/dcache.c                                                  |   13 
 fs/namespace.c                                               |   28 
 include/asm-generic/pgtable.h                                |   12 
 include/linux/cpu.h                                          |   23 
 include/linux/swapfile.h                                     |    2 
 init/main.c                                                  |    2 
 kernel/bpf/sockmap.c                                         |    9 
 kernel/cpu.c                                                 |  284 ++++-
 kernel/sched/core.c                                          |   30 
 kernel/sched/deadline.c                                      |    8 
 kernel/sched/fair.c                                          |    1 
 kernel/smp.c                                                 |    2 
 kernel/softirq.c                                             |   12 
 kernel/stop_machine.c                                        |   10 
 mm/memory.c                                                  |   37 
 mm/mprotect.c                                                |   49 
 mm/swapfile.c                                                |   46 
 tools/arch/x86/include/asm/cpufeatures.h                     |   23 
 tools/include/uapi/linux/prctl.h                             |   12 
 93 files changed, 2419 insertions(+), 381 deletions(-)

Abel Vesa (1):
cpu/hotplug: Non-SMP machines do not make use of booted_once

Al Viro (4):
root dentries need RCU-delayed freeing
make sure that __dentry_kill() always invalidates d_seq, unhashed or not fix mntput/mntput race
fix __legitimize_mnt()/mntput() race

Andi Kleen (10):
x86/speculation/l1tf: Increase 32bit PAE __PHYSICAL_PAGE_SHIFT x86/speculation/l1tf: Protect PROT_NONE PTEs against speculation x86/speculation/l1tf: Make sure the first page is always reserved x86/speculation/l1tf: Add sysfs reporting for l1tf x86/speculation/l1tf: Disallow non privileged high MMIO PROT_NONE mappings x86/speculation/l1tf: Limit swap file size to MAX_PA/2 x86/speculation/l1tf: Invert all not present mappings x86/speculation/l1tf: Make pmd/pud_mknotpresent() invert x86/mm/pat: Make set_memory_np() L1TF safe
x86/mm/kmmio: Make the tracer robust against L1TF

Arnaldo Carvalho de Melo (1):
tools headers: Synchronize prctl.h ABI header

Bart Van Assche (1):
scsi: sr: Avoid that opening a CD-ROM hangs with runtime power management enabled

Borislav Petkov (3):
x86/CPU/AMD: Do not check CPUID max ext level before parsing SMP info x86/CPU/AMD: Move TOPOEXT reenablement before reading smp_num_siblings x86/CPU/AMD: Have smp_num_siblings and cpu_llc_id always be present

Daniel Borkmann (2):
bpf, sockmap: fix leak in bpf_tcp_sendmsg wait for mem path bpf, sockmap: fix bpf_tcp_sendmsg sock error handling

Daniel Bristot de Oliveira (1):
sched/deadline: Update rq_clock of later_rq when pushing a task

David Woodhouse (1):
tools headers: Synchronise x86 cpufeatures.h for L1TF additions

Greg Kroah-Hartman (1):

      Linux 4.17.15

Helge Deller (1):
parisc: Enable CONFIG_MLONGCALLS by default

Isaac J. Manjarres (1):
stop_machine: Disable preemption after queueing stopper threads

Jiri Kosina (4):
x86/speculation: Protect against userspace-userspace spectreRSB cpu/hotplug: Expose SMT control init function
x86/bugs, kvm: Introduce boot-time control of L1TF mitigations x86/speculation/l1tf: Unbreak !__HAVE_ARCH_PFN_MODIFY_ALLOWED architectures

John David Anglin (1):
parisc: Define mb() and add memory barriers to assembler unlock sequences

Josh Poimboeuf (2):
cpu/hotplug: detect SMT disabled by BIOS x86/microcode: Allow late microcode loading with SMT disabled

Juergen Gross (1):
xen/netfront: don't cache skb_shinfo()

Konrad Rzeszutek Wilk (9):
x86/bugs: Move the l1tf function and define pr_fmt properly x86/cpufeatures: Add detection of L1D cache flush support. x86/KVM: Warn user if KVM is loaded SMT and L1TF CPU bug being present x86/KVM/VMX: Add module argument for L1TF mitigation x86/KVM/VMX: Split the VMX MSR LOAD structures to have an host/guest numbers x86/KVM/VMX: Add find_msr() helper function
x86/KVM/VMX: Separate the VMX AUTOLOAD guest/host number accounting x86/KVM/VMX: Extend add_atomic_switch_msr() to allow VMENTER only MSRs x86/KVM/VMX: Use MSR save list for IA32_FLUSH_CMD if required

Linus Torvalds (4):
Mark HI and TASKLET softirq synchronous
init: rename and re-order boot_cpu_state_init() x86/speculation/l1tf: Change order of offset/type in swap entry x86/speculation/l1tf: Protect swap entries against L1TF

Masami Hiramatsu (1):
kprobes/x86: Fix %p uses in error messages

Michal Hocko (1):

      x86/speculation/l1tf: Fix up pte->pfn conversion for PAE

Minchan Kim (1):
zram: remove BD_CAP_SYNCHRONOUS_IO with writeback feature

Nick Desaulniers (1):
x86/irqflags: Provide a declaration for native_save_fl

Nicolai Stange (9):
x86/KVM/VMX: Initialize the vmx_l1d_flush_pages' content x86/KVM/VMX: Don't set l1tf_flush_l1d to true from vmx_l1d_flush() x86/KVM/VMX: Replace 'vmx_l1d_flush_always' with 'vmx_l1d_flush_cond' x86/KVM/VMX: Move the l1tf_flush_l1d test to vmx_l1d_flush() x86/irq: Demote irq_cpustat_t::__softirq_pending to u16 x86/KVM/VMX: Introduce per-host-cpu analogue of l1tf_flush_l1d
x86: Don't include linux/irq.h from asm/hardirq.h x86/irq: Let interrupt handlers set kvm_cpu_l1tf_flush_l1d x86/KVM/VMX: Don't set l1tf_flush_l1d from vmx_handle_external_intr()

Oleksij Rempel (1):
ARM: dts: imx6sx: fix irq for pcie bridge

Paolo Bonzini (6):
x86/KVM/VMX: Add L1D flush algorithm
x86/KVM/VMX: Add L1D MSR based flush
x86/KVM/VMX: Add L1D flush logic
x86/speculation: Simplify sysfs report of VMX L1TF vulnerability x86/speculation: Use ARCH_CAPABILITIES to skip L1D flush on vmentry
KVM: VMX: Tell the nested hypervisor to skip L1D flush on vmentry

Peter Zijlstra (2):
x86/paravirt: Fix spectre-v2 mitigations for paravirt guests sched/smt: Update sched_smt_present at runtime

Quinn Tran (1):
scsi: qla2xxx: Fix memory leak for allocating abort IOCB

Thomas Gleixner (26):
x86/smp: Provide topology_is_primary_thread() x86/topology: Provide topology_smt_supported() cpu/hotplug: Make bringup/teardown of smp threads symmetric cpu/hotplug: Split do_cpu_down()
cpu/hotplug: Provide knobs to control SMT
x86/cpu: Remove the pointless CPU printout
x86/cpu/AMD: Remove the pointless detect_ht() call x86/cpu/common: Provide detect_ht_early() x86/cpu/topology: Provide detect_extended_topology_early() x86/cpu/intel: Evaluate smp_num_siblings early x86/cpu/AMD: Evaluate smp_num_siblings early
x86/apic: Ignore secondary threads if nosmt=force
Revert "x86/apic: Ignore secondary threads if nosmt=force" cpu/hotplug: Boot HT siblings at least once
cpu/hotplug: Online siblings when SMT control is turned on x86/litf: Introduce vmx status variable
x86/kvm: Drop L1TF MSR list approach
x86/l1tf: Handle EPT disabled state proper
x86/kvm: Move l1tf setup function
x86/kvm: Add static key for flush always
x86/kvm: Serialize L1D flush parameter setter
x86/kvm: Allow runtime control of L1D flush
cpu/hotplug: Set CPU_SMT_NOT_SUPPORTED early
Documentation: Add section about CPU vulnerabilities Documentation/l1tf: Remove Yonah processors from not vulnerable list cpu/hotplug: Fix SMT supported evaluation

Tony Luck (1):
Documentation/l1tf: Fix typos

Vlastimil Babka (4):
x86/speculation/l1tf: Extend 64bit swap file size limit x86/speculation/l1tf: Protect PAE swap entries against L1TF
x86/smp: fix non-SMP broken build due to redefinition of apic_id_is_primary_thread x86/init: fix build with CONFIG_SWAP=n

author maintainer
#1 lore raw
Documentation/ABI/testing/sysfs-devices-system-cpu +24 -0
index bd4975e132d3..6048a81fa744 100644
@@ -479,6 +479,7 @@ What:		/sys/devices/system/cpu/vulnerabilities
 		/sys/devices/system/cpu/vulnerabilities/spectre_v1
 		/sys/devices/system/cpu/vulnerabilities/spectre_v2
 		/sys/devices/system/cpu/vulnerabilities/spec_store_bypass
+		/sys/devices/system/cpu/vulnerabilities/l1tf
 Date:		January 2018
 Contact:	Linux kernel mailing list <[email protected]>
 Description:	Information about CPU vulnerabilities
@@ -490,3 +491,26 @@ Description:	Information about CPU vulnerabilities
 		"Not affected"	  CPU is not affected by the vulnerability
 		"Vulnerable"	  CPU is affected and no mitigation in effect
 		"Mitigation: $M"  CPU is affected and mitigation $M is in effect
+
+		Details about the l1tf file can be found in
+		Documentation/admin-guide/l1tf.rst
+
+What:		/sys/devices/system/cpu/smt
+		/sys/devices/system/cpu/smt/active
+		/sys/devices/system/cpu/smt/control
+Date:		June 2018
+Contact:	Linux kernel mailing list <[email protected]>
+Description:	Control Symetric Multi Threading (SMT)
+
+		active:  Tells whether SMT is active (enabled and siblings online)
+
+		control: Read/write interface to control SMT. Possible
+			 values:
+
+			 "on"		SMT is enabled
+			 "off"		SMT is disabled
+			 "forceoff"	SMT is force disabled. Cannot be changed.
+			 "notsupported" SMT is not supported by the CPU
+
+			 If control status is "forceoff" or "notsupported" writes
+			 are rejected.
Documentation/admin-guide/index.rst +9 -0
index 5bb9161dbe6a..78f8f00c369f 100644
@@ -17,6 +17,15 @@ etc.
    kernel-parameters
    devices
 
+This section describes CPU vulnerabilities and provides an overview of the
+possible mitigations along with guidance for selecting mitigations if they
+are configurable at compile, boot or run time.
+
+.. toctree::
+   :maxdepth: 1
+
+   l1tf
+
 Here is a set of documents aimed at users who are trying to track down
 problems and bugs in particular.
 
Documentation/admin-guide/kernel-parameters.txt +78 -0

94 lines not on this page · raw message

Documentation/admin-guide/l1tf.rst new file +610 -0

610 lines not on this page · raw message

Makefile +1 -1
index ce4248f558d1..e8cbf2dd3069 100644
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 VERSION = 4
 PATCHLEVEL = 17
-SUBLEVEL = 14
+SUBLEVEL = 15
 EXTRAVERSION =
 NAME = Merciless Moray
 
arch/Kconfig +3 -0
index 75dd23acf133..95ee27f372ed 100644
@@ -13,6 +13,9 @@ config KEXEC_CORE
 config HAVE_IMA_KEXEC
 	bool
 
+config HOTPLUG_SMT
+	bool
+
 config OPROFILE
 	tristate "OProfile system profiling"
 	depends on PROFILING
arch/arm/boot/dts/imx6sx.dtsi +1 -1
index 49c7205b8db8..77fdad65e2bb 100644
@@ -1351,7 +1351,7 @@
 			ranges = <0x81000000 0 0          0x08f80000 0 0x00010000 /* downstream I/O */
 				  0x82000000 0 0x08000000 0x08000000 0 0x00f00000>; /* non-prefetchable memory */
 			num-lanes = <1>;
-			interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH>;
+			interrupts = <GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>;
 			interrupt-names = "msi";
 			#interrupt-cells = <1>;
 			interrupt-map-mask = <0 0 0 0x7>;
arch/parisc/Kconfig +1 -1
index fc5a574c3482..f02087656528 100644
@@ -199,7 +199,7 @@ config PREFETCH
 
 config MLONGCALLS
 	bool "Enable the -mlong-calls compiler option for big kernels"
-	def_bool y if (!MODULES)
+	default y
 	depends on PA8X00
 	help
 	  If you configure the kernel to include many drivers built-in instead
arch/parisc/include/asm/barrier.h new file +32 -0
new file mode 100644
index 000000000000..dbaaca84f27f
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ASM_BARRIER_H
+#define __ASM_BARRIER_H
+
+#ifndef __ASSEMBLY__
+
+/* The synchronize caches instruction executes as a nop on systems in
+   which all memory references are performed in order. */
+#define synchronize_caches() __asm__ __volatile__ ("sync" : : : "memory")
+
+#if defined(CONFIG_SMP)
+#define mb()		do { synchronize_caches(); } while (0)
+#define rmb()		mb()
+#define wmb()		mb()
+#define dma_rmb()	mb()
+#define dma_wmb()	mb()
+#else
+#define mb()		barrier()
+#define rmb()		barrier()
+#define wmb()		barrier()
+#define dma_rmb()	barrier()
+#define dma_wmb()	barrier()
+#endif
+
+#define __smp_mb()	mb()
+#define __smp_rmb()	mb()
+#define __smp_wmb()	mb()
+
+#include <asm-generic/barrier.h>
+
+#endif /* !__ASSEMBLY__ */
+#endif /* __ASM_BARRIER_H */
arch/parisc/kernel/entry.S +2 -0
index e95207c0565e..1b4732e20137 100644
@@ -481,6 +481,8 @@
 	/* Release pa_tlb_lock lock without reloading lock address. */
 	.macro		tlb_unlock0	spc,tmp
 #ifdef CONFIG_SMP
+	or,COND(=)	%r0,\spc,%r0
+	sync
 	or,COND(=)	%r0,\spc,%r0
 	stw             \spc,0(\tmp)
 #endif
arch/parisc/kernel/pacache.S +1 -0
index 22e6374ece44..97451e67d35b 100644
@@ -353,6 +353,7 @@ ENDPROC_CFI(flush_data_cache_local)
 	.macro	tlb_unlock	la,flags,tmp
 #ifdef CONFIG_SMP
 	ldi		1,\tmp
+	sync
 	stw		\tmp,0(\la)
 	mtsm		\flags
 #endif
arch/parisc/kernel/syscall.S +4 -0
index e775f80ae28c..4886a6db42e9 100644
@@ -633,6 +633,7 @@ cas_action:
 	sub,<>	%r28, %r25, %r0
 2:	stw,ma	%r24, 0(%r26)
 	/* Free lock */
+	sync
 	stw,ma	%r20, 0(%sr2,%r20)
 #if ENABLE_LWS_DEBUG
 	/* Clear thread register indicator */
@@ -647,6 +648,7 @@ cas_action:
 3:		
 	/* Error occurred on load or store */
 	/* Free lock */
+	sync
 	stw	%r20, 0(%sr2,%r20)
 #if ENABLE_LWS_DEBUG
 	stw	%r0, 4(%sr2,%r20)
@@ -848,6 +850,7 @@ cas2_action:
 
 cas2_end:
 	/* Free lock */
+	sync
 	stw,ma	%r20, 0(%sr2,%r20)
 	/* Enable interrupts */
 	ssm	PSW_SM_I, %r0
@@ -858,6 +861,7 @@ cas2_end:
 22:
 	/* Error occurred on load or store */
 	/* Free lock */
+	sync
 	stw	%r20, 0(%sr2,%r20)
 	ssm	PSW_SM_I, %r0
 	ldo	1(%r0),%r28
arch/x86/Kconfig +1 -0
index c07f492b871a..960539ae701c 100644
@@ -183,6 +183,7 @@ config X86
 	select HAVE_SYSCALL_TRACEPOINTS
 	select HAVE_UNSTABLE_SCHED_CLOCK
 	select HAVE_USER_RETURN_NOTIFIER
+	select HOTPLUG_SMT			if SMP
 	select IRQ_FORCED_THREADING
 	select PCI_LOCKLESS_CONFIG
 	select PERF_EVENTS
arch/x86/include/asm/apic.h +9 -0
index 74a9e06b6cfd..130e81e10fc7 100644
@@ -10,6 +10,7 @@
 #include <asm/fixmap.h>
 #include <asm/mpspec.h>
 #include <asm/msr.h>
+#include <asm/hardirq.h>
 
 #define ARCH_APICTIMER_STOPS_ON_C3	1
 
@@ -502,12 +503,19 @@ extern int default_check_phys_apicid_present(int phys_apicid);
 
 #endif /* CONFIG_X86_LOCAL_APIC */
 
+#ifdef CONFIG_SMP
+bool apic_id_is_primary_thread(unsigned int id);
+#else
+static inline bool apic_id_is_primary_thread(unsigned int id) { return false; }
+#endif
+
 extern void irq_enter(void);
 extern void irq_exit(void);
 
 static inline void entering_irq(void)
 {
 	irq_enter();
+	kvm_set_cpu_l1tf_flush_l1d();
 }
 
 static inline void entering_ack_irq(void)
@@ -520,6 +528,7 @@ static inline void ipi_entering_ack_irq(void)
 {
 	irq_enter();
 	ack_APIC_irq();
+	kvm_set_cpu_l1tf_flush_l1d();
 }
 
 static inline void exiting_irq(void)
arch/x86/include/asm/cpufeatures.h +3 -0
index fb00a2fca990..f8659f070fc6 100644
@@ -219,6 +219,7 @@
 #define X86_FEATURE_IBPB		( 7*32+26) /* Indirect Branch Prediction Barrier */
 #define X86_FEATURE_STIBP		( 7*32+27) /* Single Thread Indirect Branch Predictors */
 #define X86_FEATURE_ZEN			( 7*32+28) /* "" CPU is AMD family 0x17 (Zen) */
+#define X86_FEATURE_L1TF_PTEINV		( 7*32+29) /* "" L1TF workaround PTE inversion */
 
 /* Virtualization flags: Linux defined, word 8 */
 #define X86_FEATURE_TPR_SHADOW		( 8*32+ 0) /* Intel TPR Shadow */
@@ -339,6 +340,7 @@
 #define X86_FEATURE_PCONFIG		(18*32+18) /* Intel PCONFIG */
 #define X86_FEATURE_SPEC_CTRL		(18*32+26) /* "" Speculation Control (IBRS + IBPB) */
 #define X86_FEATURE_INTEL_STIBP		(18*32+27) /* "" Single Thread Indirect Branch Predictors */
+#define X86_FEATURE_FLUSH_L1D		(18*32+28) /* Flush L1D cache */
 #define X86_FEATURE_ARCH_CAPABILITIES	(18*32+29) /* IA32_ARCH_CAPABILITIES MSR (Intel) */
 #define X86_FEATURE_SPEC_CTRL_SSBD	(18*32+31) /* "" Speculative Store Bypass Disable */
 
@@ -371,5 +373,6 @@
 #define X86_BUG_SPECTRE_V1		X86_BUG(15) /* CPU is affected by Spectre variant 1 attack with conditional branches */
 #define X86_BUG_SPECTRE_V2		X86_BUG(16) /* CPU is affected by Spectre variant 2 attack with indirect branches */
 #define X86_BUG_SPEC_STORE_BYPASS	X86_BUG(17) /* CPU is affected by speculative store bypass attack */
+#define X86_BUG_L1TF			X86_BUG(18) /* CPU is affected by L1 Terminal Fault */
 
 #endif /* _ASM_X86_CPUFEATURES_H */
arch/x86/include/asm/dmi.h +1 -1
index 0ab2ab27ad1f..b825cb201251 100644
@@ -4,8 +4,8 @@
 
 #include <linux/compiler.h>
 #include <linux/init.h>
+#include <linux/io.h>
 
-#include <asm/io.h>
 #include <asm/setup.h>
 
 static __always_inline __init void *dmi_alloc(unsigned len)
arch/x86/include/asm/hardirq.h +24 -2
index 5ea2afd4c871..0459169ab589 100644
@@ -3,10 +3,12 @@
 #define _ASM_X86_HARDIRQ_H
 
 #include <linux/threads.h>
-#include <linux/irq.h>
 
 typedef struct {
-	unsigned int __softirq_pending;
+	u16	     __softirq_pending;
+#if IS_ENABLED(CONFIG_KVM_INTEL)
+	u8	     kvm_cpu_l1tf_flush_l1d;
+#endif
 	unsigned int __nmi_count;	/* arch dependent */
 #ifdef CONFIG_X86_LOCAL_APIC
 	unsigned int apic_timer_irqs;	/* arch dependent */
@@ -66,4 +68,24 @@ extern u64 arch_irq_stat_cpu(unsigned int cpu);
 extern u64 arch_irq_stat(void);
 #define arch_irq_stat		arch_irq_stat
 
+
+#if IS_ENABLED(CONFIG_KVM_INTEL)
+static inline void kvm_set_cpu_l1tf_flush_l1d(void)
+{
+	__this_cpu_write(irq_stat.kvm_cpu_l1tf_flush_l1d, 1);
+}
+
+static inline void kvm_clear_cpu_l1tf_flush_l1d(void)
+{
+	__this_cpu_write(irq_stat.kvm_cpu_l1tf_flush_l1d, 0);
+}
+
+static inline bool kvm_get_cpu_l1tf_flush_l1d(void)
+{
+	return __this_cpu_read(irq_stat.kvm_cpu_l1tf_flush_l1d);
+}
+#else /* !IS_ENABLED(CONFIG_KVM_INTEL) */
+static inline void kvm_set_cpu_l1tf_flush_l1d(void) { }
+#endif /* IS_ENABLED(CONFIG_KVM_INTEL) */
+
 #endif /* _ASM_X86_HARDIRQ_H */
arch/x86/include/asm/irqflags.h +2 -0
index c4fc17220df9..c14f2a74b2be 100644
@@ -13,6 +13,8 @@
  * Interrupt control:
  */
 
+/* Declaration required for gcc < 4.9 to prevent -Werror=missing-prototypes */
+extern inline unsigned long native_save_fl(void);
 extern inline unsigned long native_save_fl(void)
 {
 	unsigned long flags;
arch/x86/include/asm/kvm_host.h +6 -0
index f4b2588865e9..5d216d1f40a2 100644
@@ -17,6 +17,7 @@
 #include <linux/tracepoint.h>
 #include <linux/cpumask.h>
 #include <linux/irq_work.h>
+#include <linux/irq.h>
 
 #include <linux/kvm.h>
 #include <linux/kvm_para.h>
@@ -711,6 +712,9 @@ struct kvm_vcpu_arch {
 
 	/* be preempted when it's in kernel-mode(cpl=0) */
 	bool preempted_in_kernel;
+
+	/* Flush the L1 Data cache for L1TF mitigation on VMENTER */
+	bool l1tf_flush_l1d;
 };
 
 struct kvm_lpage_info {
@@ -879,6 +883,7 @@ struct kvm_vcpu_stat {
 	u64 signal_exits;
 	u64 irq_window_exits;
 	u64 nmi_window_exits;
+	u64 l1d_flush;
 	u64 halt_exits;
 	u64 halt_successful_poll;
 	u64 halt_attempted_poll;
@@ -1410,6 +1415,7 @@ int kvm_cpu_get_interrupt(struct kvm_vcpu *v);
 void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event);
 void kvm_vcpu_reload_apic_access_page(struct kvm_vcpu *vcpu);
 
+u64 kvm_get_arch_capabilities(void);
 void kvm_define_shared_msr(unsigned index, u32 msr);
 int kvm_set_shared_msr(unsigned index, u64 val, u64 mask);
 
arch/x86/include/asm/msr-index.h +7 -0
index fda2114197b3..a7df14793e1d 100644
@@ -70,12 +70,19 @@
 #define MSR_IA32_ARCH_CAPABILITIES	0x0000010a
 #define ARCH_CAP_RDCL_NO		(1 << 0)   /* Not susceptible to Meltdown */
 #define ARCH_CAP_IBRS_ALL		(1 << 1)   /* Enhanced IBRS support */
+#define ARCH_CAP_SKIP_VMENTRY_L1DFLUSH	(1 << 3)   /* Skip L1D flush on vmentry */
 #define ARCH_CAP_SSB_NO			(1 << 4)   /*
 						    * Not susceptible to Speculative Store Bypass
 						    * attack, so no Speculative Store Bypass
 						    * control required.
 						    */
 
+#define MSR_IA32_FLUSH_CMD		0x0000010b
+#define L1D_FLUSH			(1 << 0)   /*
+						    * Writeback and invalidate the
+						    * L1 data cache.
+						    */
+
 #define MSR_IA32_BBL_CR_CTL		0x00000119
 #define MSR_IA32_BBL_CR_CTL3		0x0000011e
 
arch/x86/include/asm/page_32_types.h +7 -2
index aa30c3241ea7..0d5c739eebd7 100644
@@ -29,8 +29,13 @@
 #define N_EXCEPTION_STACKS 1
 
 #ifdef CONFIG_X86_PAE
-/* 44=32+12, the limit we can fit into an unsigned long pfn */
-#define __PHYSICAL_MASK_SHIFT	44
+/*
+ * This is beyond the 44 bit limit imposed by the 32bit long pfns,
+ * but we need the full mask to make sure inverted PROT_NONE
+ * entries have all the host bits set in a guest.
+ * The real limit is still 44 bits.
+ */
+#define __PHYSICAL_MASK_SHIFT	52
 #define __VIRTUAL_MASK_SHIFT	32
 
 #else  /* !CONFIG_X86_PAE */
arch/x86/include/asm/pgtable-2level.h +17 -0
index 685ffe8a0eaf..60d0f9015317 100644
@@ -95,4 +95,21 @@ static inline unsigned long pte_bitop(unsigned long value, unsigned int rightshi
 #define __pte_to_swp_entry(pte)		((swp_entry_t) { (pte).pte_low })
 #define __swp_entry_to_pte(x)		((pte_t) { .pte = (x).val })
 
+/* No inverted PFNs on 2 level page tables */
+
+static inline u64 protnone_mask(u64 val)
+{
+	return 0;
+}
+
+static inline u64 flip_protnone_guard(u64 oldval, u64 val, u64 mask)
+{
+	return val;
+}
+
+static inline bool __pte_needs_invert(u64 val)
+{
+	return false;
+}
+
 #endif /* _ASM_X86_PGTABLE_2LEVEL_H */
arch/x86/include/asm/pgtable-3level.h +35 -2
index f24df59c40b2..bb035a4cbc8c 100644
@@ -241,12 +241,43 @@ static inline pud_t native_pudp_get_and_clear(pud_t *pudp)
 #endif
 
 /* Encode and de-code a swap entry */
+#define SWP_TYPE_BITS		5
+
+#define SWP_OFFSET_FIRST_BIT	(_PAGE_BIT_PROTNONE + 1)
+
+/* We always extract/encode the offset by shifting it all the way up, and then down again */
+#define SWP_OFFSET_SHIFT	(SWP_OFFSET_FIRST_BIT + SWP_TYPE_BITS)
+
 #define MAX_SWAPFILES_CHECK() BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > 5)
 #define __swp_type(x)			(((x).val) & 0x1f)
 #define __swp_offset(x)			((x).val >> 5)
 #define __swp_entry(type, offset)	((swp_entry_t){(type) | (offset) << 5})
-#define __pte_to_swp_entry(pte)		((swp_entry_t){ (pte).pte_high })
-#define __swp_entry_to_pte(x)		((pte_t){ { .pte_high = (x).val } })
+
+/*
+ * Normally, __swp_entry() converts from arch-independent swp_entry_t to
+ * arch-dependent swp_entry_t, and __swp_entry_to_pte() just stores the result
+ * to pte. But here we have 32bit swp_entry_t and 64bit pte, and need to use the
+ * whole 64 bits. Thus, we shift the "real" arch-dependent conversion to
+ * __swp_entry_to_pte() through the following helper macro based on 64bit
+ * __swp_entry().
+ */
+#define __swp_pteval_entry(type, offset) ((pteval_t) { \
+	(~(pteval_t)(offset) << SWP_OFFSET_SHIFT >> SWP_TYPE_BITS) \
+	| ((pteval_t)(type) << (64 - SWP_TYPE_BITS)) })
+
+#define __swp_entry_to_pte(x)	((pte_t){ .pte = \
+		__swp_pteval_entry(__swp_type(x), __swp_offset(x)) })
+/*
+ * Analogically, __pte_to_swp_entry() doesn't just extract the arch-dependent
+ * swp_entry_t, but also has to convert it from 64bit to the 32bit
+ * intermediate representation, using the following macros based on 64bit
+ * __swp_type() and __swp_offset().
+ */
+#define __pteval_swp_type(x) ((unsigned long)((x).pte >> (64 - SWP_TYPE_BITS)))
+#define __pteval_swp_offset(x) ((unsigned long)(~((x).pte) << SWP_TYPE_BITS >> SWP_OFFSET_SHIFT))
+
+#define __pte_to_swp_entry(pte)	(__swp_entry(__pteval_swp_type(pte), \
+					     __pteval_swp_offset(pte)))
 
 #define gup_get_pte gup_get_pte
 /*
@@ -295,4 +326,6 @@ static inline pte_t gup_get_pte(pte_t *ptep)
 	return pte;
 }
 
+#include <asm/pgtable-invert.h>
+
 #endif /* _ASM_X86_PGTABLE_3LEVEL_H */
arch/x86/include/asm/pgtable-invert.h new file +32 -0
new file mode 100644
index 000000000000..44b1203ece12
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_PGTABLE_INVERT_H
+#define _ASM_PGTABLE_INVERT_H 1
+
+#ifndef __ASSEMBLY__
+
+static inline bool __pte_needs_invert(u64 val)
+{
+	return !(val & _PAGE_PRESENT);
+}
+
+/* Get a mask to xor with the page table entry to get the correct pfn. */
+static inline u64 protnone_mask(u64 val)
+{
+	return __pte_needs_invert(val) ?  ~0ull : 0;
+}
+
+static inline u64 flip_protnone_guard(u64 oldval, u64 val, u64 mask)
+{
+	/*
+	 * When a PTE transitions from NONE to !NONE or vice-versa
+	 * invert the PFN part to stop speculation.
+	 * pte_pfn undoes this when needed.
+	 */
+	if (__pte_needs_invert(oldval) != __pte_needs_invert(val))
+		val = (val & ~mask) | (~val & mask);
+	return val;
+}
+
+#endif /* __ASSEMBLY__ */
+
+#endif
arch/x86/include/asm/pgtable.h +51 -23

140 lines not on this page · raw message

arch/x86/include/asm/pgtable_64.h +27 -11
index 877bc27718ae..ea99272ab63e 100644
@@ -273,7 +273,7 @@ static inline int pgd_large(pgd_t pgd) { return 0; }
  *
  * |     ...            | 11| 10|  9|8|7|6|5| 4| 3|2| 1|0| <- bit number
  * |     ...            |SW3|SW2|SW1|G|L|D|A|CD|WT|U| W|P| <- bit names
- * | OFFSET (14->63) | TYPE (9-13)  |0|0|X|X| X| X|X|SD|0| <- swp entry
+ * | TYPE (59-63) | ~OFFSET (9-58)  |0|0|X|X| X| X|X|SD|0| <- swp entry
  *
  * G (8) is aliased and used as a PROT_NONE indicator for
  * !present ptes.  We need to start storing swap entries above
@@ -286,20 +286,34 @@ static inline int pgd_large(pgd_t pgd) { return 0; }
  *
  * Bit 7 in swp entry should be 0 because pmd_present checks not only P,
  * but also L and G.
+ *
+ * The offset is inverted by a binary not operation to make the high
+ * physical bits set.
  */
-#define SWP_TYPE_FIRST_BIT (_PAGE_BIT_PROTNONE + 1)
-#define SWP_TYPE_BITS 5
-/* Place the offset above the type: */
-#define SWP_OFFSET_FIRST_BIT (SWP_TYPE_FIRST_BIT + SWP_TYPE_BITS)
+#define SWP_TYPE_BITS		5
+
+#define SWP_OFFSET_FIRST_BIT	(_PAGE_BIT_PROTNONE + 1)
+
+/* We always extract/encode the offset by shifting it all the way up, and then down again */
+#define SWP_OFFSET_SHIFT	(SWP_OFFSET_FIRST_BIT+SWP_TYPE_BITS)
 
 #define MAX_SWAPFILES_CHECK() BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > SWP_TYPE_BITS)
 
-#define __swp_type(x)			(((x).val >> (SWP_TYPE_FIRST_BIT)) \
-					 & ((1U << SWP_TYPE_BITS) - 1))
-#define __swp_offset(x)			((x).val >> SWP_OFFSET_FIRST_BIT)
-#define __swp_entry(type, offset)	((swp_entry_t) { \
-					 ((type) << (SWP_TYPE_FIRST_BIT)) \
-					 | ((offset) << SWP_OFFSET_FIRST_BIT) })
+/* Extract the high bits for type */
+#define __swp_type(x) ((x).val >> (64 - SWP_TYPE_BITS))
+
+/* Shift up (to get rid of type), then down to get value */
+#define __swp_offset(x) (~(x).val << SWP_TYPE_BITS >> SWP_OFFSET_SHIFT)
+
+/*
+ * Shift the offset up "too far" by TYPE bits, then down again
+ * The offset is inverted by a binary not operation to make the high
+ * physical bits set.
+ */
+#define __swp_entry(type, offset) ((swp_entry_t) { \
+	(~(unsigned long)(offset) << SWP_OFFSET_SHIFT >> SWP_TYPE_BITS) \
+	| ((unsigned long)(type) << (64-SWP_TYPE_BITS)) })
+
 #define __pte_to_swp_entry(pte)		((swp_entry_t) { pte_val((pte)) })
 #define __pmd_to_swp_entry(pmd)		((swp_entry_t) { pmd_val((pmd)) })
 #define __swp_entry_to_pte(x)		((pte_t) { .pte = (x).val })
@@ -343,5 +357,7 @@ static inline bool gup_fast_permitted(unsigned long start, int nr_pages,
 	return true;
 }
 
+#include <asm/pgtable-invert.h>
+
 #endif /* !__ASSEMBLY__ */
 #endif /* _ASM_X86_PGTABLE_64_H */
arch/x86/include/asm/processor.h +17 -0
index 21a114914ba4..d7a9dea8563d 100644
@@ -181,6 +181,11 @@ extern const struct seq_operations cpuinfo_op;
 
 extern void cpu_detect(struct cpuinfo_x86 *c);
 
+static inline unsigned long l1tf_pfn_limit(void)
+{
+	return BIT(boot_cpu_data.x86_phys_bits - 1 - PAGE_SHIFT) - 1;
+}
+
 extern void early_cpu_init(void);
 extern void identify_boot_cpu(void);
 extern void identify_secondary_cpu(struct cpuinfo_x86 *);
@@ -986,4 +991,16 @@ bool xen_set_default_idle(void);
 void stop_this_cpu(void *dummy);
 void df_debug(struct pt_regs *regs, long error_code);
 void microcode_check(void);
+
+enum l1tf_mitigations {
+	L1TF_MITIGATION_OFF,
+	L1TF_MITIGATION_FLUSH_NOWARN,
+	L1TF_MITIGATION_FLUSH,
+	L1TF_MITIGATION_FLUSH_NOSMT,
+	L1TF_MITIGATION_FULL,
+	L1TF_MITIGATION_FULL_FORCE
+};
+
+extern enum l1tf_mitigations l1tf_mitigation;
+
 #endif /* _ASM_X86_PROCESSOR_H */
arch/x86/include/asm/smp.h +0 -1
index f75bff8f9d82..547c4fe50711 100644
@@ -171,7 +171,6 @@ static inline int wbinvd_on_all_cpus(void)
 	wbinvd();
 	return 0;
 }
-#define smp_num_siblings	1
 #endif /* CONFIG_SMP */
 
 extern unsigned disabled_cpus;
arch/x86/include/asm/topology.h +5 -1
index c1d2a9892352..453cf38a1c33 100644
@@ -123,13 +123,17 @@ static inline int topology_max_smt_threads(void)
 }
 
 int topology_update_package_map(unsigned int apicid, unsigned int cpu);
-extern int topology_phys_to_logical_pkg(unsigned int pkg);
+int topology_phys_to_logical_pkg(unsigned int pkg);
+bool topology_is_primary_thread(unsigned int cpu);
+bool topology_smt_supported(void);
 #else
 #define topology_max_packages()			(1)
 static inline int
 topology_update_package_map(unsigned int apicid, unsigned int cpu) { return 0; }
 static inline int topology_phys_to_logical_pkg(unsigned int pkg) { return 0; }
 static inline int topology_max_smt_threads(void) { return 1; }
+static inline bool topology_is_primary_thread(unsigned int cpu) { return true; }
+static inline bool topology_smt_supported(void) { return false; }
 #endif
 
 static inline void arch_fix_phys_package_id(int num, u32 slot)
arch/x86/include/asm/vmx.h +11 -0
index 7cc81c586d71..9b71d0d24db1 100644
@@ -574,4 +574,15 @@ enum vm_instruction_error_number {
 	VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID = 28,
 };
 
+enum vmx_l1d_flush_state {
+	VMENTER_L1D_FLUSH_AUTO,
+	VMENTER_L1D_FLUSH_NEVER,
+	VMENTER_L1D_FLUSH_COND,
+	VMENTER_L1D_FLUSH_ALWAYS,
+	VMENTER_L1D_FLUSH_EPT_DISABLED,
+	VMENTER_L1D_FLUSH_NOT_REQUIRED,
+};
+
+extern enum vmx_l1d_flush_state l1tf_vmx_mitigation;
+
 #endif
arch/x86/kernel/apic/apic.c +18 -0
index adbda5847b14..3b3a2d0af78d 100644
@@ -56,6 +56,7 @@
 #include <asm/hypervisor.h>
 #include <asm/cpu_device_id.h>
 #include <asm/intel-family.h>
+#include <asm/irq_regs.h>
 
 unsigned int num_processors;
 
@@ -2192,6 +2193,23 @@ static int cpuid_to_apicid[] = {
 	[0 ... NR_CPUS - 1] = -1,
 };
 
+#ifdef CONFIG_SMP
+/**
+ * apic_id_is_primary_thread - Check whether APIC ID belongs to a primary thread
+ * @id:	APIC ID to check
+ */
+bool apic_id_is_primary_thread(unsigned int apicid)
+{
+	u32 mask;
+
+	if (smp_num_siblings == 1)
+		return true;
+	/* Isolate the SMT bit(s) in the APICID and check for 0 */
+	mask = (1U << (fls(smp_num_siblings) - 1)) - 1;
+	return !(apicid & mask);
+}
+#endif
+
 /*
  * Should use this API to allocate logical CPU IDs to keep nr_logical_cpuids
  * and cpuid_to_apicid[] synchronized.
arch/x86/kernel/apic/io_apic.c +1 -0
index 3982f79d2377..ff0d14cd9e82 100644
@@ -33,6 +33,7 @@
 
 #include <linux/mm.h>
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/init.h>
 #include <linux/delay.h>
 #include <linux/sched.h>
arch/x86/kernel/apic/msi.c +1 -0
index ce503c99f5c4..72a94401f9e0 100644
@@ -12,6 +12,7 @@
  */
 #include <linux/mm.h>
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/pci.h>
 #include <linux/dmar.h>
 #include <linux/hpet.h>
arch/x86/kernel/apic/vector.c +1 -0
index b708f597eee3..9f38b4140c27 100644
@@ -11,6 +11,7 @@
  * published by the Free Software Foundation.
  */
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/seq_file.h>
 #include <linux/init.h>
 #include <linux/compiler.h>
arch/x86/kernel/cpu/amd.c +27 -32

123 lines not on this page · raw message

arch/x86/kernel/cpu/bugs.c +140 -31

229 lines not on this page · raw message

arch/x86/kernel/cpu/common.c +42 -21

108 lines not on this page · raw message

arch/x86/kernel/cpu/cpu.h +2 -0
index 37672d299e35..cca588407dca 100644
@@ -47,6 +47,8 @@ extern const struct cpu_dev *const __x86_cpu_dev_start[],
 
 extern void get_cpu_cap(struct cpuinfo_x86 *c);
 extern void cpu_detect_cache_sizes(struct cpuinfo_x86 *c);
+extern int detect_extended_topology_early(struct cpuinfo_x86 *c);
+extern int detect_ht_early(struct cpuinfo_x86 *c);
 
 unsigned int aperfmperf_get_khz(int cpu);
 
arch/x86/kernel/cpu/intel.c +7 -0
index 577e7f7ae273..bfaaa92ef6de 100644
@@ -301,6 +301,13 @@ static void early_init_intel(struct cpuinfo_x86 *c)
 	}
 
 	check_mpx_erratum(c);
+
+	/*
+	 * Get the number of SMT siblings early from the extended topology
+	 * leaf, if available. Otherwise try the legacy SMT detection.
+	 */
+	if (detect_extended_topology_early(c) < 0)
+		detect_ht_early(c);
 }
 
 #ifdef CONFIG_X86_32
arch/x86/kernel/cpu/microcode/core.c +12 -4
index 08286269fd24..b9bc8a1a584e 100644
@@ -509,12 +509,20 @@ static struct platform_device	*microcode_pdev;
 
 static int check_online_cpus(void)
 {
-	if (num_online_cpus() == num_present_cpus())
-		return 0;
+	unsigned int cpu;
 
-	pr_err("Not all CPUs online, aborting microcode update.\n");
+	/*
+	 * Make sure all CPUs are online.  It's fine for SMT to be disabled if
+	 * all the primary threads are still online.
+	 */
+	for_each_present_cpu(cpu) {
+		if (topology_is_primary_thread(cpu) && !cpu_online(cpu)) {
+			pr_err("Not all CPUs online, aborting microcode update.\n");
+			return -EINVAL;
+		}
+	}
 
-	return -EINVAL;
+	return 0;
 }
 
 static atomic_t late_cpus_in;
arch/x86/kernel/cpu/topology.c +24 -17

72 lines not on this page · raw message

arch/x86/kernel/fpu/core.c +1 -0
index f92a6593de1e..2ea85b32421a 100644
@@ -10,6 +10,7 @@
 #include <asm/fpu/signal.h>
 #include <asm/fpu/types.h>
 #include <asm/traps.h>
+#include <asm/irq_regs.h>
 
 #include <linux/hardirq.h>
 #include <linux/pkeys.h>
arch/x86/kernel/hpet.c +1 -0
index 8ce4212e2b8d..afa1a204bc6d 100644
@@ -1,6 +1,7 @@
 #include <linux/clocksource.h>
 #include <linux/clockchips.h>
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/export.h>
 #include <linux/delay.h>
 #include <linux/errno.h>
arch/x86/kernel/i8259.c +1 -0
index 86c4439f9d74..519649ddf100 100644
@@ -5,6 +5,7 @@
 #include <linux/sched.h>
 #include <linux/ioport.h>
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/timex.h>
 #include <linux/random.h>
 #include <linux/init.h>
arch/x86/kernel/idt.c +1 -0
index 2c3a1b4294eb..7f6cffaa5322 100644
@@ -8,6 +8,7 @@
 #include <asm/traps.h>
 #include <asm/proto.h>
 #include <asm/desc.h>
+#include <asm/hw_irq.h>
 
 struct idt_data {
 	unsigned int	vector;
arch/x86/kernel/irq.c +1 -0
index 328d027d829d..59b5f2ea7c2f 100644
@@ -10,6 +10,7 @@
 #include <linux/ftrace.h>
 #include <linux/delay.h>
 #include <linux/export.h>
+#include <linux/irq.h>
 
 #include <asm/apic.h>
 #include <asm/io_apic.h>
arch/x86/kernel/irq_32.c +1 -0
index c1bdbd3d3232..95600a99ae93 100644
@@ -11,6 +11,7 @@
 
 #include <linux/seq_file.h>
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/kernel_stat.h>
 #include <linux/notifier.h>
 #include <linux/cpu.h>
arch/x86/kernel/irq_64.c +1 -0
index d86e344f5b3d..0469cd078db1 100644
@@ -11,6 +11,7 @@
 
 #include <linux/kernel_stat.h>
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/seq_file.h>
 #include <linux/delay.h>
 #include <linux/ftrace.h>
arch/x86/kernel/irqinit.c +1 -0
index 772196c1b8c4..a0693b71cfc1 100644
@@ -5,6 +5,7 @@
 #include <linux/sched.h>
 #include <linux/ioport.h>
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/timex.h>
 #include <linux/random.h>
 #include <linux/kprobes.h>
arch/x86/kernel/kprobes/core.c +1 -4
index 6f4d42377fe5..44e26dc326d5 100644
@@ -395,8 +395,6 @@ int __copy_instruction(u8 *dest, u8 *src, u8 *real, struct insn *insn)
 			  - (u8 *) real;
 		if ((s64) (s32) newdisp != newdisp) {
 			pr_err("Kprobes error: new displacement does not fit into s32 (%llx)\n", newdisp);
-			pr_err("\tSrc: %p, Dest: %p, old disp: %x\n",
-				src, real, insn->displacement.value);
 			return 0;
 		}
 		disp = (u8 *) dest + insn_offset_displacement(insn);
@@ -640,8 +638,7 @@ static int reenter_kprobe(struct kprobe *p, struct pt_regs *regs,
 		 * Raise a BUG or we'll continue in an endless reentering loop
 		 * and eventually a stack overflow.
 		 */
-		printk(KERN_WARNING "Unrecoverable kprobe detected at %p.\n",
-		       p->addr);
+		pr_err("Unrecoverable kprobe detected.\n");
 		dump_kprobe(p);
 		BUG();
 	default:
arch/x86/kernel/paravirt.c +10 -4
index 99dc79e76bdc..930c88341e4e 100644
@@ -88,10 +88,12 @@ unsigned paravirt_patch_call(void *insnbuf,
 	struct branch *b = insnbuf;
 	unsigned long delta = (unsigned long)target - (addr+5);
 
-	if (tgt_clobbers & ~site_clobbers)
-		return len;	/* target would clobber too much for this site */
-	if (len < 5)
+	if (len < 5) {
+#ifdef CONFIG_RETPOLINE
+		WARN_ONCE("Failing to patch indirect CALL in %ps\n", (void *)addr);
+#endif
 		return len;	/* call too long for patch site */
+	}
 
 	b->opcode = 0xe8; /* call */
 	b->delta = delta;
@@ -106,8 +108,12 @@ unsigned paravirt_patch_jmp(void *insnbuf, const void *target,
 	struct branch *b = insnbuf;
 	unsigned long delta = (unsigned long)target - (addr+5);
 
-	if (len < 5)
+	if (len < 5) {
+#ifdef CONFIG_RETPOLINE
+		WARN_ONCE("Failing to patch indirect JMP in %ps\n", (void *)addr);
+#endif
 		return len;	/* call too long for patch site */
+	}
 
 	b->opcode = 0xe9;	/* jmp */
 	b->delta = delta;
arch/x86/kernel/setup.c +6 -0
index 5c623dfe39d1..89fd35349412 100644
@@ -823,6 +823,12 @@ void __init setup_arch(char **cmdline_p)
 	memblock_reserve(__pa_symbol(_text),
 			 (unsigned long)__bss_stop - (unsigned long)_text);
 
+	/*
+	 * Make sure page 0 is always reserved because on systems with
+	 * L1TF its contents can be leaked to user processes.
+	 */
+	memblock_reserve(0, PAGE_SIZE);
+
 	early_reserve_initrd();
 
 	/*
arch/x86/kernel/smp.c +1 -0
index 5c574dff4c1a..04adc8d60aed 100644
@@ -261,6 +261,7 @@ __visible void __irq_entry smp_reschedule_interrupt(struct pt_regs *regs)
 {
 	ack_APIC_irq();
 	inc_irq_stat(irq_resched_count);
+	kvm_set_cpu_l1tf_flush_l1d();
 
 	if (trace_resched_ipi_enabled()) {
 		/*
arch/x86/kernel/smpboot.c +18 -7
index 9dd324ae4832..f5d30c68fd09 100644
@@ -80,13 +80,7 @@
 #include <asm/intel-family.h>
 #include <asm/cpu_device_id.h>
 #include <asm/spec-ctrl.h>
-
-/* Number of siblings per CPU package */
-int smp_num_siblings = 1;
-EXPORT_SYMBOL(smp_num_siblings);
-
-/* Last level cache ID of each logical CPU */
-DEFINE_PER_CPU_READ_MOSTLY(u16, cpu_llc_id) = BAD_APICID;
+#include <asm/hw_irq.h>
 
 /* representing HT siblings of each logical CPU */
 DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_map);
@@ -272,6 +266,23 @@ static void notrace start_secondary(void *unused)
 	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
+/**
+ * topology_is_primary_thread - Check whether CPU is the primary SMT thread
+ * @cpu:	CPU to check
+ */
+bool topology_is_primary_thread(unsigned int cpu)
+{
+	return apic_id_is_primary_thread(per_cpu(x86_cpu_to_apicid, cpu));
+}
+
+/**
+ * topology_smt_supported - Check whether SMT is supported by the CPUs
+ */
+bool topology_smt_supported(void)
+{
+	return smp_num_siblings > 1;
+}
+
 /**
  * topology_phys_to_logical_pkg - Map a physical package id to a logical
  *
arch/x86/kernel/time.c +1 -0
index 774ebafa97c4..be01328eb755 100644
@@ -12,6 +12,7 @@
 
 #include <linux/clockchips.h>
 #include <linux/interrupt.h>
+#include <linux/irq.h>
 #include <linux/i8253.h>
 #include <linux/time.h>
 #include <linux/export.h>
arch/x86/kvm/mmu.c +1 -0
index 030c6bb240d9..2b974d4e1489 100644
@@ -3836,6 +3836,7 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code,
 {
 	int r = 1;
 
+	vcpu->arch.l1tf_flush_l1d = true;
 	switch (vcpu->arch.apf.host_apf_reason) {
 	default:
 		trace_kvm_page_fault(fault_address, error_code);
arch/x86/kvm/vmx.c +377 -78

597 lines not on this page · raw message

arch/x86/kvm/x86.c +33 -1
index ac01341f2d1f..0125698b9b70 100644
@@ -194,6 +194,7 @@ struct kvm_stats_debugfs_item debugfs_entries[] = {
 	{ "irq_injections", VCPU_STAT(irq_injections) },
 	{ "nmi_injections", VCPU_STAT(nmi_injections) },
 	{ "req_event", VCPU_STAT(req_event) },
+	{ "l1d_flush", VCPU_STAT(l1d_flush) },
 	{ "mmu_shadow_zapped", VM_STAT(mmu_shadow_zapped) },
 	{ "mmu_pte_write", VM_STAT(mmu_pte_write) },
 	{ "mmu_pte_updated", VM_STAT(mmu_pte_updated) },
@@ -1097,11 +1098,35 @@ static u32 msr_based_features[] = {
 
 static unsigned int num_msr_based_features;
 
+u64 kvm_get_arch_capabilities(void)
+{
+	u64 data;
+
+	rdmsrl_safe(MSR_IA32_ARCH_CAPABILITIES, &data);
+
+	/*
+	 * If we're doing cache flushes (either "always" or "cond")
+	 * we will do one whenever the guest does a vmlaunch/vmresume.
+	 * If an outer hypervisor is doing the cache flush for us
+	 * (VMENTER_L1D_FLUSH_NESTED_VM), we can safely pass that
+	 * capability to the guest too, and if EPT is disabled we're not
+	 * vulnerable.  Overall, only VMENTER_L1D_FLUSH_NEVER will
+	 * require a nested hypervisor to do a flush of its own.
+	 */
+	if (l1tf_vmx_mitigation != VMENTER_L1D_FLUSH_NEVER)
+		data |= ARCH_CAP_SKIP_VMENTRY_L1DFLUSH;
+
+	return data;
+}
+EXPORT_SYMBOL_GPL(kvm_get_arch_capabilities);
+
 static int kvm_get_msr_feature(struct kvm_msr_entry *msr)
 {
 	switch (msr->index) {
-	case MSR_IA32_UCODE_REV:
 	case MSR_IA32_ARCH_CAPABILITIES:
+		msr->data = kvm_get_arch_capabilities();
+		break;
+	case MSR_IA32_UCODE_REV:
 		rdmsrl_safe(msr->index, &msr->data);
 		break;
 	default:
@@ -4870,6 +4895,9 @@ static int emulator_write_std(struct x86_emulate_ctxt *ctxt, gva_t addr, void *v
 int kvm_write_guest_virt_system(struct kvm_vcpu *vcpu, gva_t addr, void *val,
 				unsigned int bytes, struct x86_exception *exception)
 {
+	/* kvm_write_guest_virt_system can pull in tons of pages. */
+	vcpu->arch.l1tf_flush_l1d = true;
+
 	return kvm_write_guest_virt_helper(addr, val, bytes, vcpu,
 					   PFERR_WRITE_MASK, exception);
 }
@@ -6046,6 +6074,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu,
 	bool writeback = true;
 	bool write_fault_to_spt = vcpu->arch.write_fault_to_shadow_pgtable;
 
+	vcpu->arch.l1tf_flush_l1d = true;
+
 	/*
 	 * Clear write_fault_to_shadow_pgtable here to ensure it is
 	 * never reused.
@@ -7575,6 +7605,7 @@ static int vcpu_run(struct kvm_vcpu *vcpu)
 	struct kvm *kvm = vcpu->kvm;
 
 	vcpu->srcu_idx = srcu_read_lock(&kvm->srcu);
+	vcpu->arch.l1tf_flush_l1d = true;
 
 	for (;;) {
 		if (kvm_vcpu_running(vcpu)) {
@@ -8694,6 +8725,7 @@ void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu)
 
 void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu)
 {
+	vcpu->arch.l1tf_flush_l1d = true;
 	kvm_x86_ops->sched_in(vcpu, cpu);
 }
 
arch/x86/mm/init.c +25 -0
index cee58a972cb2..83241eb71cd4 100644
@@ -4,6 +4,8 @@
 #include <linux/swap.h>
 #include <linux/memblock.h>
 #include <linux/bootmem.h>	/* for max_low_pfn */
+#include <linux/swapfile.h>
+#include <linux/swapops.h>
 
 #include <asm/set_memory.h>
 #include <asm/e820/api.h>
@@ -880,3 +882,26 @@ void update_cache_mode_entry(unsigned entry, enum page_cache_mode cache)
 	__cachemode2pte_tbl[cache] = __cm_idx2pte(entry);
 	__pte2cachemode_tbl[entry] = cache;
 }
+
+#ifdef CONFIG_SWAP
+unsigned long max_swapfile_size(void)
+{
+	unsigned long pages;
+
+	pages = generic_max_swapfile_size();
+
+	if (boot_cpu_has_bug(X86_BUG_L1TF)) {
+		/* Limit the swap file size to MAX_PA/2 for L1TF workaround */
+		unsigned long l1tf_limit = l1tf_pfn_limit() + 1;
+		/*
+		 * We encode swap offsets also with 3 bits below those for pfn
+		 * which makes the usable limit higher.
+		 */
+#if CONFIG_PGTABLE_LEVELS > 2
+		l1tf_limit <<= PAGE_SHIFT - SWP_OFFSET_FIRST_BIT;
+#endif
+		pages = min_t(unsigned long, l1tf_limit, pages);
+	}
+	return pages;
+}
+#endif
arch/x86/mm/kmmio.c +15 -10
index 7c8686709636..79eb55ce69a9 100644
@@ -126,24 +126,29 @@ static struct kmmio_fault_page *get_kmmio_fault_page(unsigned long addr)
 
 static void clear_pmd_presence(pmd_t *pmd, bool clear, pmdval_t *old)
 {
+	pmd_t new_pmd;
 	pmdval_t v = pmd_val(*pmd);
 	if (clear) {
-		*old = v & _PAGE_PRESENT;
-		v &= ~_PAGE_PRESENT;
-	} else	/* presume this has been called with clear==true previously */
-		v |= *old;
-	set_pmd(pmd, __pmd(v));
+		*old = v;
+		new_pmd = pmd_mknotpresent(*pmd);
+	} else {
+		/* Presume this has been called with clear==true previously */
+		new_pmd = __pmd(*old);
+	}
+	set_pmd(pmd, new_pmd);
 }
 
 static void clear_pte_presence(pte_t *pte, bool clear, pteval_t *old)
 {
 	pteval_t v = pte_val(*pte);
 	if (clear) {
-		*old = v & _PAGE_PRESENT;
-		v &= ~_PAGE_PRESENT;
-	} else	/* presume this has been called with clear==true previously */
-		v |= *old;
-	set_pte_atomic(pte, __pte(v));
+		*old = v;
+		/* Nothing should care about address */
+		pte_clear(&init_mm, 0, pte);
+	} else {
+		/* Presume this has been called with clear==true previously */
+		set_pte_atomic(pte, __pte(*old));
+	}
 }
 
 static int clear_page_presence(struct kmmio_fault_page *f, bool clear)
arch/x86/mm/mmap.c +21 -0
index 48c591251600..f40ab8185d94 100644
@@ -240,3 +240,24 @@ int valid_mmap_phys_addr_range(unsigned long pfn, size_t count)
 
 	return phys_addr_valid(addr + count - 1);
 }
+
+/*
+ * Only allow root to set high MMIO mappings to PROT_NONE.
+ * This prevents an unpriv. user to set them to PROT_NONE and invert
+ * them, then pointing to valid memory for L1TF speculation.
+ *
+ * Note: for locked down kernels may want to disable the root override.
+ */
+bool pfn_modify_allowed(unsigned long pfn, pgprot_t prot)
+{
+	if (!boot_cpu_has_bug(X86_BUG_L1TF))
+		return true;
+	if (!__pte_needs_invert(pgprot_val(prot)))
+		return true;
+	/* If it's real memory always allow */
+	if (pfn_valid(pfn))
+		return true;
+	if (pfn > l1tf_pfn_limit() && !capable(CAP_SYS_ADMIN))
+		return false;
+	return true;
+}
arch/x86/mm/pageattr.c +4 -4
index 3bded76e8d5c..7bb6f65c79de 100644
@@ -1014,8 +1014,8 @@ static long populate_pmd(struct cpa_data *cpa,
 
 		pmd = pmd_offset(pud, start);
 
-		set_pmd(pmd, __pmd(cpa->pfn << PAGE_SHIFT | _PAGE_PSE |
-				   massage_pgprot(pmd_pgprot)));
+		set_pmd(pmd, pmd_mkhuge(pfn_pmd(cpa->pfn,
+					canon_pgprot(pmd_pgprot))));
 
 		start	  += PMD_SIZE;
 		cpa->pfn  += PMD_SIZE >> PAGE_SHIFT;
@@ -1087,8 +1087,8 @@ static int populate_pud(struct cpa_data *cpa, unsigned long start, p4d_t *p4d,
 	 * Map everything starting from the Gb boundary, possibly with 1G pages
 	 */
 	while (boot_cpu_has(X86_FEATURE_GBPAGES) && end - start >= PUD_SIZE) {
-		set_pud(pud, __pud(cpa->pfn << PAGE_SHIFT | _PAGE_PSE |
-				   massage_pgprot(pud_pgprot)));
+		set_pud(pud, pud_mkhuge(pfn_pud(cpa->pfn,
+				   canon_pgprot(pud_pgprot))));
 
 		start	  += PUD_SIZE;
 		cpa->pfn  += PUD_SIZE >> PAGE_SHIFT;
arch/x86/mm/pti.c +1 -0
index 4d418e705878..fb752d9a3ce9 100644
@@ -45,6 +45,7 @@
 #include <asm/pgalloc.h>
 #include <asm/tlbflush.h>
 #include <asm/desc.h>
+#include <asm/sections.h>
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

arch/x86/platform/intel-mid/device_libs/platform_mrfld_wdt.c +1 -0
index 4f5fa65a1011..2acd6be13375 100644

Lines of this file are not shown. View the raw message.

arch/x86/platform/uv/tlb_uv.c +1 -0
index b36caae0fb2f..c29d3152f5a4 100644

Lines of this file are not shown. View the raw message.

arch/x86/xen/enlighten.c +1 -0
index c9081c6671f0..df208af3cd74 100644

Lines of this file are not shown. View the raw message.

drivers/base/cpu.c +8 -0
index 30cc9c877ebb..eb9443d5bae1 100644

Lines of this file are not shown. View the raw message.

drivers/block/zram/zram_drv.c +14 -1
index 0f3fadd71230..9b0b69598e23 100644

Lines of this file are not shown. View the raw message.

drivers/gpu/drm/i915/i915_pmu.c +1 -0
index f0519e31543a..fbe2a9bee07f 100644

Lines of this file are not shown. View the raw message.

drivers/gpu/drm/i915/intel_lpe_audio.c +1 -0
index 6269750e2b54..b4941101f21a 100644

Lines of this file are not shown. View the raw message.

drivers/net/xen-netfront.c +4 -4
index 42e93cb4eca7..ea2da91a96ea 100644

Lines of this file are not shown. View the raw message.

drivers/pci/host/pci-hyperv.c +2 -0
index 4690814cfc51..c07e952b94ee 100644

Lines of this file are not shown. View the raw message.

drivers/scsi/qla2xxx/qla_iocb.c +27 -26

53 lines not on this page · raw message

drivers/scsi/sr.c +21 -8
index 3f3cb72e0c0c..d0389b20574d 100644

Lines of this file are not shown. View the raw message.

fs/dcache.c +6 -7
index 2acfc69878f5..811114b229b1 100644

Lines of this file are not shown. View the raw message.

fs/namespace.c +26 -2
index 5f75969adff1..51a1935060a9 100644

Lines of this file are not shown. View the raw message.

include/asm-generic/pgtable.h +12 -0
index f59639afaa39..26ca0276b503 100644

Lines of this file are not shown. View the raw message.

include/linux/cpu.h +22 -1
index a97a63eef59f..45789a892c41 100644

Lines of this file are not shown. View the raw message.

include/linux/swapfile.h +2 -0
index 06bd7b096167..e06febf62978 100644

Lines of this file are not shown. View the raw message.

init/main.c +1 -1
index 3b4ada11ed52..5e13c544bbf4 100644

Lines of this file are not shown. View the raw message.

kernel/bpf/sockmap.c +6 -3
index fc7ee4357381..70edc41a88d5 100644

Lines of this file are not shown. View the raw message.

kernel/cpu.c +271 -13

284 lines not on this page · raw message

kernel/sched/core.c +12 -18
index 211890edf37e..ec945451b9ef 100644

Lines of this file are not shown. View the raw message.

kernel/sched/deadline.c +7 -1
index fbfc3f1d368a..8b50eea4b607 100644

Lines of this file are not shown. View the raw message.

kernel/sched/fair.c +1 -0
index 79f574dba096..183068d22849 100644

Lines of this file are not shown. View the raw message.

kernel/smp.c +2 -0
index 084c8b3a2681..d86eec5f51c1 100644

Lines of this file are not shown. View the raw message.

kernel/softirq.c +8 -4
index ce4fb0e12504..7a076b6c537a 100644

Lines of this file are not shown. View the raw message.

kernel/stop_machine.c +9 -1
index 1ff523dae6e2..e190d1ef3a23 100644

Lines of this file are not shown. View the raw message.

mm/memory.c +27 -10
index 01f5464e0fd2..fe497cecd2ab 100644

Lines of this file are not shown. View the raw message.

mm/mprotect.c +49 -0

49 lines not on this page · raw message

mm/swapfile.c +30 -16

46 lines not on this page · raw message

tools/arch/x86/include/asm/cpufeatures.h +17 -6
index 578793e97431..f8659f070fc6 100644

Lines of this file are not shown. View the raw message.

tools/include/uapi/linux/prctl.h +12 -0
index af5f8c2df87a..db9f15f5db04 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Part of the diff is not shown here. View the raw message.