MDS basics 0
Subsystems (6)
cover MDS basics 0
Subject: [patch 0/8] MDS basics
From: Thomas Gleixner <[email protected]>
Hi!
I got the following information yesterday night:
"All - FYI. There has been some chatter/ discussion on the subject.
Hopefully this note will help clarify. We received a report from a researcher who independently identified what we formerly referred to as PSF (aka Microarchitectural Store Buffer Data Sampling). There were some initial indications (this week) this researcher would elect to release a paper publicly PRIOR to the May 14 embargo was lifted.
We have been working closely with them, and it appears for now that will NOT be the case. Were that to happen however, we DID begin prepping materials to disclose PSF ONLY. I.e. we would disclose only that particular issue after having consulted with this team. This includes a modified/ reduced section of the existing whitepaper, press statement and standard security advisory language. We are finalizing this material and will then hold it in reserve.
As we have done in the past, we would convene a meeting of reps from this group before activating those assets. I will keep you apprised of any change in the situation, and can provide those assets for your use/ adaptation once finalized."
This was posted on that keybase io chat on friday night and of course not made available to those who are not part of that. Even people who are subscribed there missed the message because it scrolled away due to other chit/chat.
Now we maybe got lucky this time, but I wouldn't hold my breath as the propability that other people will figure that out as well is surely way larger than 0.
If that happens, then it makes exactly ZERO sense to expose only the MSBDS part as everything else is lumped together with this. But why am I still trying to make sense of all this?
So while being grumpy about this communication fail, I'm even more grumpy about the fact, that we don't have even the minimal full/off mitigation in place in a workable form. I asked specifically for this weeks ago just for the case that the embargo breaks early so we don't stand there with pants down.
So being grumpy as hell made me sit down and write the basic mitigation implementation myself (again).
It reuses a single patch from that Intel pile which is defining the bug and MSR bits. Guess what, it took me less than 4 hours to do so and another 2 hours in the morning to write at least the basic admin documentation. The latter surely needs some work still, but I wanted to get the patches out. There is also another TODO mentioned further down.
The series comes with:
- A consistent command line interface
- A consistent sysfs interface
- Static key based control for the exit to user and idle invocations
- Dynamic update of the idle invocation key according to the actual SMT state similar to the STIPB update.
- Idle invocations are inside the halt/mwait inlines and not randomly sprinkled all over the kernel tree.
It builds and boots and while I was able to emit the VERW instruction by hacking the mitigation selection to omit the MD_CLEAR supported check, I have no access to real hardware with updated micro code.
This is how it should have looked from the very beginning and the extra bits and pieces (cond mode) can be built on top of it. Please review and give it a testride when you have a machine with updated microcode available.
The lot is also available from the speck git tree in the WIP.mds branch.
Note, that I moved the L!TF document to a separate folder so the hw vulnerabilities are not showing up at the top level index of the admin guide as separate items. Should have thought about that back then already...
TODO:
For CPUs which are not affected by L1TF but are affected by MDS there needs to be CPU buffer clear mitigation at VMENTER. That applies at least to XEON PHI, SILVERMONT and AIRMONT and probably to some of the newer models which have RDCL_NO set.
Thanks,
tglx
8<-----------------------
Documentation/ABI/testing/sysfs-devices-system-cpu | 1 Documentation/admin-guide/hw-vuln/index.rst | 13 + Documentation/admin-guide/hw-vuln/l1tf.rst | 1 Documentation/admin-guide/hw-vuln/mds.rst | 230 +++++++++++++++++++++ Documentation/admin-guide/index.rst | 6 Documentation/admin-guide/kernel-parameters.txt | 27 ++ arch/x86/entry/common.c | 3 arch/x86/include/asm/cpufeatures.h | 2 arch/x86/include/asm/irqflags.h | 4 arch/x86/include/asm/msr-index.h | 5 arch/x86/include/asm/mwait.h | 7 arch/x86/include/asm/nospec-branch.h | 22 ++ arch/x86/include/asm/processor.h | 6 arch/x86/kernel/cpu/bugs.c | 102 +++++++++ arch/x86/kernel/cpu/common.c | 13 + drivers/base/cpu.c | 6 include/linux/cpu.h | 2 17 files changed, 443 insertions(+), 7 deletions(-)
PATCH 1/8 MDS basics 1
Subject: [patch 1/8] x86/speculation/mds: Add basic bug infrastructure for MDS
From: Andi Kleen <[email protected]>
Microarchitectural Data Sampling (MDS), is a class of side channel attacks on internal buffers in Intel CPUs. The variants are:
- Microarchitectural Store Buffer Data Sampling (MSBDS) (CVE-2018-12126)
- Microarchitectural Fill Buffer Data Sampling (MFBDS) (CVE-2018-12130)
- Microarchitectural Load Port Data (MLPDS) (CVE-2018-12127)
MSBDS leaks Store Buffer Entries which can be speculatively forwarded to a dependent load (store-load forwarding) as an optimization. The forward can also happen to a faulting or assisting load operation for a different memory address, which can be exploited under certain conditions. Store buffers are partitionened between Hyper-Threads so cross thread forwarding is not possible. But if a thread enters or exits a sleep state the store buffer is repartioned which can expose data from one thread to the other.
MFBDS leaks Fill Buffer Entries. Fill buffers are used internally to manage L1 miss situations and to hold data which is returned or sent in response to a memory or I/O operation. Fill buffers can forward data to a load operation and also write data to the cache. When the fill buffer is deallocated it can retain the stale data of the preceeding operations which can then be forwarded to a faulting or assisting load operation, which can be exploited under certain conditions. Fill buffers are shared between Hyper-Threads so cross thread leakage is possible.
MLDPS leaks Load Port Data. Load ports are used to perform load operations from memory or I/O. The received data is then forwarded to the register file or a subsequent operation. In some implementations the Load Port can contain stale data from a previous operation which can be forwarded to faulting or assisting loads under certain conditions, which again can be exploited eventually. Load poorts are shared between Hyper-Threads so cross thread leakage is possible.
All variants have the same mitigation for single CPU thread case (SMT off), so the kernel can treat them as one MDS issue.
Add the basic infrastructure to detect if the current CPU is affected by MDS.
[ tglx: Rewrote changelog ]
--- arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/include/asm/msr-index.h | 5 +++++ arch/x86/kernel/cpu/common.c | 13 +++++++++++++ 3 files changed, 20 insertions(+)
arch/x86/include/asm/cpufeatures.h +2 -0
@@ -344,6 +344,7 @@/* Intel-defined CPU features, CPUID level 0x00000007:0 (EDX), word 18 */ #define X86_FEATURE_AVX512_4VNNIW (18*32+ 2)/* AVX-512 Neural Network Instructions */ #define X86_FEATURE_AVX512_4FMAPS (18*32+ 3)/* AVX-512 Multiply Accumulation Single precision */ +#define X86_FEATURE_MD_CLEAR (18*32+10)/* VERW flushs CPU state */ #define X86_FEATURE_PCONFIG (18*32+18)/* Intel PCONFIG */ #define X86_FEATURE_SPEC_CTRL (18*32+26)/* "" Speculation Control (IBRS + IBPB) */ #define X86_FEATURE_INTEL_STIBP (18*32+27)/* "" Single Thread Indirect Branch Predictors */
@@ -381,5 +382,6 @@#define X86_BUG_SPECTRE_V2 X86_BUG(16)/* CPU is affected by Spectre variant 2 attack with indirect branches */ #define X86_BUG_SPEC_STORE_BYPASS X86_BUG(17)/* CPU is affected by speculative store bypass attack */ #define X86_BUG_L1TF X86_BUG(18)/* CPU is affected by L1 Terminal Fault */ +#define X86_BUG_MDS X86_BUG(19)/* CPU is affected by Microarchitectural data sampling */ #endif /* _ASM_X86_CPUFEATURES_H */
arch/x86/include/asm/msr-index.h +5 -0
@@ -77,6 +77,11 @@* attack, so no Speculative Store Bypass * control required. */ +#define ARCH_CAP_MDS_NO (1 << 5)/* +* Not susceptible to +* Microarchitectural Data +* Sampling (MDS) vulnerabilities. +*/ #define MSR_IA32_FLUSH_CMD 0x0000010b#define L1D_FLUSH (1 << 0)/*
arch/x86/kernel/cpu/common.c +13 -0
@@ -998,6 +998,14 @@ static const __initconst struct x86_cpu_ {} }; +static const __initconst struct x86_cpu_id cpu_no_mds[] = { +/* in addition to cpu_no_speculation */ + {X86_VENDOR_INTEL ,6 ,INTEL_FAM6_ATOM_GOLDMONT }, + {X86_VENDOR_INTEL ,6 ,INTEL_FAM6_ATOM_GOLDMONT_X }, + {X86_VENDOR_INTEL ,6 ,INTEL_FAM6_ATOM_GOLDMONT_PLUS }, + {} +}; +static void __init cpu_set_bug_bits (struct cpuinfo_x86 *c) {u64 ia32_cap =0 ;
@@ -1019,6 +1027,11 @@ static void __init cpu_set_bug_bits(struif (ia32_cap &ARCH_CAP_IBRS_ALL )setup_force_cpu_cap (X86_FEATURE_IBRS_ENHANCED ); +if ((boot_cpu_data.x86_vendor ==X86_VENDOR_INTEL && + !x86_match_cpu (cpu_no_mds)) && + !(ia32_cap &ARCH_CAP_MDS_NO )) +setup_force_cpu_bug (X86_BUG_MDS ); +if (x86_match_cpu (cpu_no_meltdown))return ;
PATCH 2/8 MDS basics 2
In reply to Thomas GleixnerSubject: [patch 2/8] x86/speculation/mds: Add mds_clear_cpu_buffer()
From: Thomas Gleixner <[email protected]>
The Microarchitectural Data Sampling (MDS) vulernabilities are mitigated by clearing the affected CPU buffers. The mechanism for clearing the buffers uses the unused and obsolete VERW instruction in combination with a microcode update which triggers a CPU buffer clear when VERW is executed.
Provide a inline function with the assembly magic. The argument of the VERW instruction must be a memory operand.
The function takes a pointer to a static key, so different call sites can depend on different static keys for controlling the invocation. This avoids the conditionals at the call sites and allows for fine grained control, e.g. the SMT only CPU buffer clearing on idle entry can be enabled independent of the exit to user space clear.
--- arch/x86/include/asm/nospec-branch.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
arch/x86/include/asm/nospec-branch.h +19 -0
@@ -314,6 +314,25 @@ do { \preempt_enable (); \ } while (0) +#include <asm/segment.h> + +/** +* mds_clear_cpu_buffers - Mitigation for MDS vulnerability +* +* This uses the otherwise unused and obsolete VERW instruction in +* combination with microcode which triggers a CPU buffer flush when the +* instruction is executed. +*/ +static inline void mds_clear_cpu_buffers (struct static_key_false *key) +{ +if (static_branch_likely (key)) { +static const u16 ds = __KERNEL_DS; + +/* Has to be memory form, don't modify to use a register */ + asmvolatile ("verw %[ds]" : :"i" (0 ), [ds]"m" (ds)); + } +} +DECLARE_STATIC_KEY_FALSE (switch_to_cond_stibp);DECLARE_STATIC_KEY_FALSE (switch_mm_cond_ibpb);DECLARE_STATIC_KEY_FALSE (switch_mm_always_ibpb);
PATCH 3/8 MDS basics 3
In reply to Thomas GleixnerSubject: [patch 3/8] x86/speculation/mds: Clear CPU buffers on exit to user
From: Thomas Gleixner <[email protected]>
Add a static key which controls the invocation of the CPU buffer clear mechanism on exit to user space and add the call into prepare_exit_to_usermode() right before actually returning.
--- arch/x86/entry/common.c | 3 +++ arch/x86/include/asm/nospec-branch.h | 2 ++ arch/x86/kernel/cpu/bugs.c | 4 +++- 3 files changed, 8 insertions(+), 1 deletion(-)
arch/x86/entry/common.c +3 -0
@@ -31,6 +31,7 @@#include <asm/vdso.h> #include <linux/uaccess.h> #include <asm/cpufeature.h> +#include <asm/nospec-branch.h> #define CREATE_TRACE_POINTS #include <trace/events/syscalls.h>
@@ -211,6 +212,8 @@ static void exit_to_usermode_loop(struct ti->status &= ~(TS_COMPAT |TS_I386_REGS_POKED );#endif +mds_clear_cpu_buffers (&user_mds_clear_cpu_buffers); +user_enter_irqoff (); }
arch/x86/include/asm/nospec-branch.h +2 -0
@@ -337,6 +337,8 @@ DECLARE_STATIC_KEY_FALSE(switch_to_cond_DECLARE_STATIC_KEY_FALSE (switch_mm_cond_ibpb);DECLARE_STATIC_KEY_FALSE (switch_mm_always_ibpb); +DECLARE_STATIC_KEY_FALSE (user_mds_clear_cpu_buffers); +#endif /* __ASSEMBLY__ */ /*
arch/x86/kernel/cpu/bugs.c +3 -1
@@ -63,10 +63,12 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_cond_i/* Control unconditional IBPB in switch_mm() */ DEFINE_STATIC_KEY_FALSE (switch_mm_always_ibpb); +/* Control MDS CPU buffer clear before returning to user space */ +DEFINE_STATIC_KEY_FALSE (user_mds_clear_cpu_buffers); +void __init check_bugs (void ) {identify_boot_cpu (); -/* * identify_boot_cpu() initialized SMT support information, let the * core code know.
PATCH 4/8 MDS basics 4
In reply to Thomas GleixnerSubject: [patch 4/8] x86/speculation/mds: Conditionaly clear CPU buffers on idle entry
From: Thomas Gleixner <[email protected]>
Add a static key which controls the invocation of the CPU buffer clear mechanism on idle entry. This is independent of other MDS mitigations because the idle entry invocation to mitigate the potential leakage due to store buffer repartitioning is only necessary on SMT systems.
Add the actual invocations to the different halt/mwait variants which covers all usage sites. mwaitx is not patched as it's not available on Intel CPUs.
--- arch/x86/include/asm/irqflags.h | 4 ++++ arch/x86/include/asm/mwait.h | 7 +++++++ arch/x86/include/asm/nospec-branch.h | 1 + arch/x86/kernel/cpu/bugs.c | 2 ++ 4 files changed, 14 insertions(+)
arch/x86/include/asm/irqflags.h +4 -0
@@ -6,6 +6,8 @@#ifndef __ASSEMBLY__ +#include <asm/nospec-branch.h> +/* Provide __cpuidle; we can't safely include <linux/cpu.h> */ #define __cpuidle __attribute__((__section__(".cpuidle.text")))
@@ -54,11 +56,13 @@ static inline void native_irq_enable(voi static inline __cpuidle void native_safe_halt(void) { +mds_clear_cpu_buffers (&idle_mds_clear_cpu_buffers); asmvolatile ("sti; hlt" : : :"memory" ); } static inline __cpuidle void native_halt(void) { +mds_clear_cpu_buffers (&idle_mds_clear_cpu_buffers); asmvolatile ("hlt" : : :"memory" ); }
arch/x86/include/asm/mwait.h +7 -0
@@ -6,6 +6,7 @@#include <linux/sched/idle.h> #include <asm/cpufeature.h> +#include <asm/nospec-branch.h> #define MWAIT_SUBSTATE_MASK 0xf#define MWAIT_CSTATE_MASK 0xf
@@ -40,6 +41,8 @@ static inline void __monitorx(const voidstatic inline void __mwait (unsigned long eax,unsigned long ecx) { +mds_clear_cpu_buffers (&idle_mds_clear_cpu_buffers); +/* "mwait %eax, %ecx;" */ asmvolatile (".byte 0x0f, 0x01, 0xc9;" ::"a" (eax),"c" (ecx));
@@ -74,6 +77,8 @@ static inline void __mwait(unsigned longstatic inline void __mwaitx (unsigned long eax,unsigned long ebx,unsigned long ecx) { +/* No MDS buffer clear as this is AMD/HYGON only */ +/* "mwaitx %eax, %ebx, %ecx;" */ asmvolatile (".byte 0x0f, 0x01, 0xfb;" ::"a" (eax),"b" (ebx),"c" (ecx));
@@ -81,6 +86,8 @@ static inline void __mwaitx(unsigned lonstatic inline void __sti_mwait (unsigned long eax,unsigned long ecx) { +mds_clear_cpu_buffers (&idle_mds_clear_cpu_buffers); +trace_hardirqs_on ();/* "mwait %eax, %ecx;" */ asm volatile("sti; .byte 0x0f, 0x01, 0xc9;"
arch/x86/include/asm/nospec-branch.h +1 -0
@@ -338,6 +338,7 @@ DECLARE_STATIC_KEY_FALSE(switch_mm_cond_DECLARE_STATIC_KEY_FALSE (switch_mm_always_ibpb);DECLARE_STATIC_KEY_FALSE (user_mds_clear_cpu_buffers); +DECLARE_STATIC_KEY_FALSE (idle_mds_clear_cpu_buffers);#endif /* __ASSEMBLY__ */
arch/x86/kernel/cpu/bugs.c +2 -0
@@ -65,6 +65,8 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_always/* Control MDS CPU buffer clear before returning to user space */ DEFINE_STATIC_KEY_FALSE (user_mds_clear_cpu_buffers); +/* Control MDS CPU buffer clear before idling (halt, mwait) */ +DEFINE_STATIC_KEY_FALSE (idle_mds_clear_cpu_buffers);void __init check_bugs (void ) {
PATCH 5/8 MDS basics 5
In reply to Thomas GleixnerSubject: [patch 5/8] x86/speculation/mds: Add mitigation control for MDS
From: Thomas Gleixner <[email protected]>
Now that the mitigations are in place, add a command line parameter to control the mitigation, a mitigation selector function and a SMT update mechanism.
This is the minimal straight forward initial implementation which just provides an always on/off mode. The command line parameter is:
mds=[full|off|auto]
This is consistent with the existing mitigations for other speculative hardware vulnerabilities.
The idle invocation is dynamically updated according to the SMT state of the system similar to the dynamic update of the STIBP mitigation.
--- Documentation/admin-guide/kernel-parameters.txt | 27 ++++++++ arch/x86/include/asm/processor.h | 6 + arch/x86/kernel/cpu/bugs.c | 76 ++++++++++++++++++++++++ 3 files changed, 109 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +27 -0
@@ -2362,6 +2362,33 @@ Format: <first>,<last> Specifies range of consoles to be captured by the MDA. + mds= [X86,INTEL] + Control mitigation for the Micro-architectural Data + Sampling (MDS) vulnerability. + + Certain CPUs are vulnerable to an exploit against CPU + internal buffers which can forward information to a + disclosure gadget under certain conditions. + + In vulnerable processors, the speculatively + forwarded data can be used in a cache side channel + attack, to access data to which the attacker does + not have direct access. + + This parameter controls the MDS mitigation. The the + options are: + + full - Unconditionally enable MDS mitigation + off - Unconditionally disable MDS mitigation + auto - Kernel detects whether the CPU model is + vulnerable to MDS and picks the most + appropriate mitigation. If the CPU is not + vulnerable, "off" is selected. If the CPU + is vulnerable "full" is selected. + + Not specifying this option is equivalent to + mds=auto. + mem=nn[KMG] [KNL,BOOT] Force usage of a specific amount of memory Amount of memory to be used when the kernel is not able to see the whole system memory or for test.
arch/x86/include/asm/processor.h +6 -0
@@ -991,4 +991,10 @@ enum l1tf_mitigations {extern enum l1tf_mitigations l1tf_mitigation; +enum mds_mitigations { +MDS_MITIGATION_OFF , +MDS_MITIGATION_AUTO , +MDS_MITIGATION_FULL , +}; +#endif /* _ASM_X86_PROCESSOR_H */
arch/x86/kernel/cpu/bugs.c +76 -0
106 lines not on this page · raw message
PATCH 6/8 MDS basics 6
In reply to Thomas GleixnerSubject: [patch 6/8] x86/speculation/mds: Add sysfs reporting for MDS
From: Thomas Gleixner <[email protected]>
Add the sysfs reporting file for MDS. It exposes the vulnerability and mitigation state similar to the existing files for the other speculative hardware vulnerabilities.
--- Documentation/ABI/testing/sysfs-devices-system-cpu | 1 + arch/x86/kernel/cpu/bugs.c | 20 ++++++++++++++++++++ drivers/base/cpu.c | 6 ++++-- include/linux/cpu.h | 2 ++ 4 files changed, 27 insertions(+), 2 deletions(-)
Documentation/ABI/testing/sysfs-devices-system-cpu +1 -0
@@ -484,6 +484,7 @@ What: /sys/devices/system/cpu/vulnerabi /sys/devices/system/cpu/vulnerabilities/spectre_v2 /sys/devices/system/cpu/vulnerabilities/spec_store_bypass /sys/devices/system/cpu/vulnerabilities/l1tf + /sys/devices/system/cpu/vulnerabilities/mds Date: January 2018 Contact: Linux kernel mailing list <[email protected]> Description: Information about CPU vulnerabilities
arch/x86/kernel/cpu/bugs.c +20 -0
@@ -1187,6 +1187,17 @@ static ssize_t l1tf_show_state(char *buf }#endif +static ssize_t mds_show_state (char *buf) +{ +if (!hypervisor_is_type (X86_HYPER_NATIVE )) { +return sprintf (buf,"%s; SMT Host state unknown\n" , + mds_strings[mds_mitigation]); + } + +return sprintf (buf,"%s; SMT %s\n" , mds_strings[mds_mitigation], +sched_smt_active () ?"vulnerable" :"disabled" ); +} +static char *stibp_state (void ) {if (spectre_v2_enabled ==SPECTRE_V2_IBRS_ENHANCED )
@@ -1253,6 +1264,10 @@ static ssize_t cpu_show_common(struct deif (boot_cpu_has (X86_FEATURE_L1TF_PTEINV ))return l1tf_show_state (buf);break ; + +case X86_BUG_MDS : +return mds_show_state (buf); +default :break ; }
@@ -1284,4 +1299,9 @@ ssize_t cpu_show_l1tf(struct device *dev {return cpu_show_common (dev, attr, buf,X86_BUG_L1TF ); } + +ssize_t cpu_show_mds (struct device *dev,struct device_attribute *attr,char *buf) +{ +return cpu_show_common (dev, attr, buf,X86_BUG_MDS ); +}#endif
drivers/base/cpu.c +4 -2
@@ -540,8 +540,8 @@ ssize_t __weak cpu_show_spec_store_bypasreturn sprintf (buf,"Not affected\n" ); } -ssize_t __weak cpu_show_l1tf (struct device *dev, -struct device_attribute *attr,char *buf) +ssize_t __weak cpu_show_mds (struct device *dev, +struct device_attribute *attr,char *buf) {return sprintf (buf,"Not affected\n" ); }
@@ -551,6 +551,7 @@ static DEVICE_ATTR(spectre_v1, 0444, cpu static DEVICE_ATTR(spectre_v2, 0444, cpu_show_spectre_v2, NULL); static DEVICE_ATTR(spec_store_bypass, 0444, cpu_show_spec_store_bypass, NULL); static DEVICE_ATTR(l1tf, 0444, cpu_show_l1tf, NULL); +static DEVICE_ATTR(mds, 0444, cpu_show_mds, NULL);static struct attribute *cpu_root_vulnerabilities_attrs[] = { &dev_attr_meltdown.attr,
@@ -558,6 +559,7 @@ static struct attribute *cpu_root_vulner &dev_attr_spectre_v2.attr, &dev_attr_spec_store_bypass.attr, &dev_attr_l1tf.attr, + &dev_attr_mds.attr, NULL };
include/linux/cpu.h +2 -0
@@ -57,6 +57,8 @@ extern ssize_t cpu_show_spec_store_bypas struct device_attribute *attr, char *buf);extern ssize_t cpu_show_l1tf (struct device *dev,struct device_attribute *attr,char *buf); +extern ssize_t cpu_show_mds (struct device *dev, +struct device_attribute *attr,char *buf); extern __printf(4, 5) struct device *cpu_device_create(struct device *parent, void *drvdata,
PATCH 7/8 MDS basics 7
In reply to Thomas GleixnerSubject: [patch 7/8] Documentation: Move L1TF to separate directory
From: Thomas Gleixner <[email protected]>
Move L!TF to a separate directory so the MDS stuff can be added at the side. Otherwise the all hardware vulnerabilites have their own top level entry. Should have done that right away.
--- Documentation/admin-guide/hw-vuln/index.rst | 12 Documentation/admin-guide/hw-vuln/l1tf.rst | 614 ++++++++++++++++++++++++++++ Documentation/admin-guide/index.rst | 6 Documentation/admin-guide/l1tf.rst | 614 ---------------------------- 4 files changed, 628 insertions(+), 618 deletions(-)
Documentation/admin-guide/hw-vuln/index.rst new file +12 -0
@@ -0,0 +1,12 @@ +======================== +Hardware vulnerabilities +======================== + +This section describes CPU vulnerabilities and provides an overview of the +possible mitigations along with guidance for selecting mitigations if they +are configurable at compile, boot or run time. + +.. toctree:: + :maxdepth: 1 + + l1tf
Documentation/admin-guide/hw-vuln/l1tf.rst new file +614 -0
614 lines not on this page · raw message
Documentation/admin-guide/index.rst +2 -4
@@ -17,14 +17,12 @@ etc. kernel-parameters devices -This section describes CPU vulnerabilities and provides an overview of the -possible mitigations along with guidance for selecting mitigations if they -are configurable at compile, boot or run time. +This section describes CPU vulnerabilities and their mitigations. .. toctree:: :maxdepth: 1 - l1tf + hw-vuln/index Here is a set of documents aimed at users who are trying to track down problems and bugs in particular.
Documentation/admin-guide/l1tf.rst deleted +0 -614
614 lines not on this page · raw message
PATCH 8/8 MDS basics 8
In reply to Thomas GleixnerSubject: [patch 8/8] Documentation: Add MDS vulnerability documentation
From: Thomas Gleixner <[email protected]>
Add the initial MDS vulnerability documentation.
Still needs a lot of work....
--- Documentation/admin-guide/hw-vuln/index.rst | 1 Documentation/admin-guide/hw-vuln/l1tf.rst | 1 Documentation/admin-guide/hw-vuln/mds.rst | 230 ++++++++++++++++++++++++++++ 3 files changed, 232 insertions(+)
Documentation/admin-guide/hw-vuln/index.rst +1 -0
@@ -10,3 +10,4 @@ are configurable at compile, boot or run :maxdepth: 1 l1tf + mds
Documentation/admin-guide/hw-vuln/l1tf.rst +1 -0
@@ -445,6 +445,7 @@ The default is 'cond'. If 'l1tf=full,for line, then 'always' is enforced and the kvm-intel.vmentry_l1d_flush module parameter is ignored and writes to the sysfs file are rejected. +.. _mitigation_selection: Mitigation selection guide --------------------------
Documentation/admin-guide/hw-vuln/mds.rst new file +230 -0
230 lines not on this page · raw message
[MODERATED] Re: [patch 4/8] MDS basics 4
In reply to Thomas GleixnerQuoted 8 lines · speck for Thomas Gleixner
Subject: [patch 4/8] x86/speculation/mds: Conditionaly clear CPU buffers on idle entry
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on idle entry. This is independent of other MDS mitigations because the idle entry invocation to mitigate the potential leakage due to store buffer repartitioning is only necessary on SMT systems.
Quoted 5 lines
Add the actual invocations to the different halt/mwait variants which covers all usage sites. mwaitx is not patched as it's not available on Intel CPUs.
Signed-off-by Thomas Gleixner <[email protected]>
Unfortunately, clearing is needed on the exit from idle as well as the entry.
This only impacts the store buffer (MSBDS, previously PSF) because it gets statically re-partitioned when a thread comes in and out of idle.
From the point of view of the thread going idle, when going idle my half of the store buffers get given to the other thread and potentially leak my secrets, whereas when coming out of idle, the other threads store buffers get split with me, potentially leaking their secrets.
~Andrew
[MODERATED] Re: [patch 1/8] MDS basics 1
In reply to Thomas Gleixnerarch/x86/include/asm/cpufeatures.h line 347
Quoted 7 lines · speck for Thomas Gleixner
arch/x86/include/asm/cpufeatures.h +1
-0@@ -344,6 +344,7 @@/* Intel-defined CPU features, CPUID level 0x00000007:0 (EDX), word 18 */ #define X86_FEATURE_AVX512_4VNNIW (18*32+ 2)/* AVX-512 Neural Network Instructions */ #define X86_FEATURE_AVX512_4FMAPS (18*32+ 3)/* AVX-512 Multiply Accumulation Single precision */ +#define X86_FEATURE_MD_CLEAR (18*32+10)/* VERW flushs CPU state */
"flushes"
Otherwise:
Signature · 5 lines
Regards/Gruss,
Boris.
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- Re: [patch 4/8] MDS basics 4
In reply to Andrew CooperQuoted 18 lines · speck for Andrew Cooper
Quoted 8 lines · speck for Thomas Gleixner
Subject: [patch 4/8] x86/speculation/mds: Conditionaly clear CPU buffers on idle entry
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on idle entry. This is independent of other MDS mitigations because the idle entry invocation to mitigate the potential leakage due to store buffer repartitioning is only necessary on SMT systems.
Quoted 5 lines
Add the actual invocations to the different halt/mwait variants which covers all usage sites. mwaitx is not patched as it's not available on Intel CPUs.
Signed-off-by Thomas Gleixner <[email protected]>Unfortunately, clearing is needed on the exit from idle as well as the entry.
Quoted 7 lines
This only impacts the store buffer (MSBDS, previously PSF) because it gets statically re-partitioned when a thread comes in and out of idle.
Quoted 1 line
From the point of view of the thread going idle, when going idle my half
of the store buffers get given to the other thread and potentially leak my secrets, whereas when coming out of idle, the other threads store buffers get split with me, potentially leaking their secrets.
Duh, indeed. Easy enough to fix.
Thanks,
tglx
Quoted 5 lines · speck for Thomas Gleixner
Subject: [patch 0/8] MDS basics
From: Thomas Gleixner <[email protected]>Hi!
Quoted 8 lines
I got the following information yesterday night:
"All - FYI. There has been some chatter/ discussion on the subject.
Hopefully this note will help clarify. We received a report from a researcher who independently identified what we formerly referred to as PSF (aka Microarchitectural Store Buffer Data Sampling). There were some initial indications (this week) this researcher would elect to release a paper publicly PRIOR to the May 14 embargo was lifted.
I do apologize. I did post the information that I had the specxen list, but it didn't occur to me that it was liable to have been missed from here.
In future, I'll cross post any updates which don't appear to have already made their way here.
~Andrew
Re: [patch 4/8] MDS basics 4
In reply to Thomas GleixnerQuoted 23 lines · speck for Thomas Gleixner
Quoted 21 lines · speck for Andrew Cooper
Quoted 8 lines · speck for Thomas Gleixner
Subject: [patch 4/8] x86/speculation/mds: Conditionaly clear CPU buffers on idle entry
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on idle entry. This is independent of other MDS mitigations because the idle entry invocation to mitigate the potential leakage due to store buffer repartitioning is only necessary on SMT systems.
Quoted 5 lines
Add the actual invocations to the different halt/mwait variants which covers all usage sites. mwaitx is not patched as it's not available on Intel CPUs.
Signed-off-by Thomas Gleixner <[email protected]>Unfortunately, clearing is needed on the exit from idle as well as the entry.
This only impacts the store buffer (MSBDS, previously PSF) because it gets statically re-partitioned when a thread comes in and out of idle.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
From the point of view of the thread going idle, when going idle my half
of the store buffers get given to the other thread and potentially leak my secrets, whereas when coming out of idle, the other threads store buffers get split with me, potentially leaking their secrets.
Duh, indeed. Easy enough to fix.
Delta patch below. Stupid me even mentioned the repartioning on both sides in the changelog.
Thanks,
tglx
8<------------------
arch/x86/include/asm/irqflags.h +2 -0
@@ -58,12 +58,14 @@ static inline __cpuidle void native_safe {mds_clear_cpu_buffers (&idle_mds_clear_cpu_buffers); asmvolatile ("sti; hlt" : : :"memory" ); +mds_clear_cpu_buffers (&idle_mds_clear_cpu_buffers); } static inline __cpuidle void native_halt(void) {mds_clear_cpu_buffers (&idle_mds_clear_cpu_buffers); asmvolatile ("hlt" : : :"memory" ); +mds_clear_cpu_buffers (&idle_mds_clear_cpu_buffers); }#endif
arch/x86/include/asm/mwait.h +2 -0
@@ -46,6 +46,8 @@ static inline void __mwait(unsigned long/* "mwait %eax, %ecx;" */ asmvolatile (".byte 0x0f, 0x01, 0xc9;" ::"a" (eax),"c" (ecx)); + +mds_clear_cpu_buffers (&idle_mds_clear_cpu_buffers); }/*
Andrew,
Quoted 15 lines · speck for Andrew Cooper
Quoted 5 lines · speck for Thomas Gleixner
Subject: [patch 0/8] MDS basics
From: Thomas Gleixner <[email protected]>Hi!
Quoted 8 lines
I got the following information yesterday night:
"All - FYI. There has been some chatter/ discussion on the subject.
Hopefully this note will help clarify. We received a report from a researcher who independently identified what we formerly referred to as PSF (aka Microarchitectural Store Buffer Data Sampling). There were some initial indications (this week) this researcher would elect to release a paper publicly PRIOR to the May 14 embargo was lifted.
Quoted 5 lines
I do apologize. I did post the information that I had the specxen list, but it didn't occur to me that it was liable to have been missed from here.
In future, I'll cross post any updates which don't appear to have already made their way here.
It's surely not your fault and your problem, but I appreciate the offer.
Thanks,
tglx
Quoted 15 lines · speck for Andrew Cooper
Quoted 5 lines · speck for Thomas Gleixner
Subject: [patch 0/8] MDS basics
From: Thomas Gleixner <[email protected]>Hi!
Quoted 8 lines
I got the following information yesterday night:
"All - FYI. There has been some chatter/ discussion on the subject.
Hopefully this note will help clarify. We received a report from a researcher who independently identified what we formerly referred to as PSF (aka Microarchitectural Store Buffer Data Sampling). There were some initial indications (this week) this researcher would elect to release a paper publicly PRIOR to the May 14 embargo was lifted.
Quoted 5 lines
I do apologize. I did post the information that I had the specxen list, but it didn't occur to me that it was liable to have been missed from here.
In future, I'll cross post any updates which don't appear to have already made their way here.
Good idea. I'll also try to make sure nothing important slips through the cracks.
Tyler
Greg KH
maintainer[MODERATED] Re: [patch 8/8] MDS basics 8
In reply to Thomas GleixnerQuoted 8 lines · speck for Thomas Gleixner
Documentation/admin-guide/hw-vuln/mds.rst new file +8
-0+Mitigation control on the kernel command line +--------------------------------------------- + +The kernel command line allows to control the MDS mitigations at boot +time with the option "mds=". The valid arguments for this option are: + + ============ ============================================================= + full Provides all available mitigations for the MDS vulnerability
Documentation/admin-guide/hw-vuln/mds.rst line 179
Quoted 10 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +10
-0+ vulnerability, unconditional CPU buffer clearing on exit to + userspace and when entering a VM. + + It does not automatically disable SMT. + + off Disables MDS mitigations completely. + + ============ ============================================================= + +The default is 'full'. For details see :ref:`cpu_buffer_clear_full`.
I think default is 'auto' according to the patch that enabled it :)
thanks,
greg k-h
Re: [patch 8/8] MDS basics 8
Quoted 13 lines · speck for Greg KH
Quoted 12 lines · speck for Thomas Gleixner
Documentation/admin-guide/hw-vuln/mds.rst new file +12
-0+Mitigation control on the kernel command line +--------------------------------------------- + +The kernel command line allows to control the MDS mitigations at boot +time with the option "mds=". The valid arguments for this option are: + + ============ ============================================================= + full Provides all available mitigations for the MDS vulnerability + vulnerability, unconditional CPU buffer clearing on exit to + userspace and when entering a VM. + + It does not automatically disable SMT.
Quoted 8 lines
Documentation/admin-guide/hw-vuln/mds.rst line 179
Quoted 6 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +6
-0+ + off Disables MDS mitigations completely. + + ============ ============================================================= + +The default is 'full'. For details see :ref:`cpu_buffer_clear_full`.I think default is 'auto' according to the patch that enabled it :)
Yes. Forgot to document auto as well.
Re: [patch 5/8] MDS basics 5
In reply to Thomas Gleixnerarch/x86/kernel/cpu/bugs.c static void update_indir_branch_cond(voi · line 679
Quoted 7 lines · speck for Thomas Gleixner
arch/x86/kernel/cpu/bugs.c +7
-0+/* Update the static key controlling the MDS CPU buffer clear in idle */ +static void update_mds_branch_idle (void ) +{ +if (sched_smt_active ()) +static_branch_enable (&user_mds_clear_cpu_buffers); +else +static_branch_disable (&user_mds_clear_cpu_buffers);
Those obviously need s/user/idle/
Quoted 5 lines
So while being grumpy about this communication fail, I'm even more grumpy about the fact, that we don't have even the minimal full/off mitigation in place in a workable form. I asked specifically for this
There seems to be certainly a communication fail here.
The minimal version has been posted on this mailing list in late December
(although it only showed up in early January because ...)
-Andi
[MODERATED] Re: [patch 5/8] MDS basics 5
In reply to Thomas Gleixnerarch/x86/kernel/cpu/bugs.c static void x86_amd_ssb_disable(void) · line 241
Quoted 5 lines
arch/x86/kernel/cpu/bugs.c +5
-0+case MDS_MITIGATION_AUTO : +case MDS_MITIGATION_FULL : +if (boot_cpu_has (X86_FEATURE_MD_CLEAR )) { + mds_mitigation =MDS_MITIGATION_FULL ; +static_branch_enable (&user_mds_clear_cpu_buffers);
This violates Linus' feedback of unconditionally enabling for one year to work around VMWare breakage.
-Andi
[MODERATED] Re: [patch 3/8] MDS basics 3
In reply to Thomas GleixnerQuoted 6 lines · speck for Thomas Gleixner
Subject: [patch 3/8] x86/speculation/mds: Clear CPU buffers on exit to user
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on exit to user space and add the call into
prepare_exit_to_usermode()right before actually returning.
This is missing the NMI exit case.
-Andi
[MODERATED] Re: [patch 4/8] MDS basics 4
In reply to Thomas GleixnerQuoted 11 lines · speck for Thomas Gleixner
Subject: [patch 4/8] x86/speculation/mds: Conditionaly clear CPU buffers on idle entry
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on idle entry. This is independent of other MDS mitigations because the idle entry invocation to mitigate the potential leakage due to store buffer repartitioning is only necessary on SMT systems.
Add the actual invocations to the different halt/mwait variants which covers all usage sites. mwaitx is not patched as it's not available on Intel CPUs.
This doesn't handle ACPI IO port idling correctly.
-Andi
[MODERATED] Re: [patch 4/8] MDS basics 4
In reply to Andrew CooperQuoted 9 lines · speck for Andrew Cooper
Quoted 8 lines · speck for Thomas Gleixner
Subject: [patch 4/8] x86/speculation/mds: Conditionaly clear CPU buffers on idle entry
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on idle entry. This is independent of other MDS mitigations because the idle entry invocation to mitigate the potential leakage due to store buffer repartitioning is only necessary on SMT systems.
Quoted 8 lines
Quoted 5 lines
Add the actual invocations to the different halt/mwait variants which covers all usage sites. mwaitx is not patched as it's not available on Intel CPUs.
Signed-off-by Thomas Gleixner <[email protected]>Unfortunately, clearing is needed on the exit from idle as well as the entry.
Not with the full flush. There it would always clear on next kernel exit.
-Andi
[MODERATED] Re: [patch 4/8] MDS basics 4
In reply to Thomas GleixnerQuoted 25 lines · speck for Thomas Gleixner
Quoted 24 lines · speck for Thomas Gleixner
Quoted 22 lines · speck for Andrew Cooper
Quoted 8 lines · speck for Thomas Gleixner
Subject: [patch 4/8] x86/speculation/mds: Conditionaly clear CPU buffers on idle entry
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on idle entry. This is independent of other MDS mitigations because the idle entry invocation to mitigate the potential leakage due to store buffer repartitioning is only necessary on SMT systems.
Quoted 5 lines
Add the actual invocations to the different halt/mwait variants which covers all usage sites. mwaitx is not patched as it's not available on Intel CPUs.
Signed-off-by Thomas Gleixner <[email protected]>Unfortunately, clearing is needed on the exit from idle as well as the entry.
This only impacts the store buffer (MSBDS, previously PSF) because it gets statically re-partitioned when a thread comes in and out of idle.
Quoted 1 line
From the point of view of the thread going idle, when going idle my half
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
of the store buffers get given to the other thread and potentially leak my secrets, whereas when coming out of idle, the other threads store buffers get split with me, potentially leaking their secrets.
Duh, indeed. Easy enough to fix.
Delta patch below. Stupid me even mentioned the repartioning on both sides in the changelog.
It's not needed.
You can drop it.
-Andi
[MODERATED] Re: [patch 5/8] MDS basics 5
In reply to Thomas GleixnerQuoted 10 lines · speck for Thomas Gleixner
arch/x86/kernel/cpu/bugs.c static void update_indir_branch_cond(voi · line 679
Quoted 7 lines · speck for Thomas Gleixner
arch/x86/kernel/cpu/bugs.c +7
-0+/* Update the static key controlling the MDS CPU buffer clear in idle */ +static void update_mds_branch_idle (void ) +{ +if (sched_smt_active ()) +static_branch_enable (&user_mds_clear_cpu_buffers); +else +static_branch_disable (&user_mds_clear_cpu_buffers);Those obviously need s/user/idle/
Please don't post untested crap like this. It would be really bad if someone applied it.
This can be easily verified just by looking at a few PT traces.
Or just use MDSv2 which actually worked.
BTW even for the minimum version virtualization is definitely needed.
-Andi
Peter Zijlstra
maintainer[MODERATED] Re: [patch 4/8] MDS basics 4
In reply to Andi KleenQuoted 30 lines · speck for Andi Kleen
Quoted 29 lines · speck for Thomas Gleixner
Quoted 28 lines · speck for Thomas Gleixner
Quoted 25 lines · speck for Andrew Cooper
Quoted 8 lines · speck for Thomas Gleixner
Subject: [patch 4/8] x86/speculation/mds: Conditionaly clear CPU buffers on idle entry
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on idle entry. This is independent of other MDS mitigations because the idle entry invocation to mitigate the potential leakage due to store buffer repartitioning is only necessary on SMT systems.
Quoted 5 lines
Add the actual invocations to the different halt/mwait variants which covers all usage sites. mwaitx is not patched as it's not available on Intel CPUs.
Signed-off-by Thomas Gleixner <[email protected]>Unfortunately, clearing is needed on the exit from idle as well as the entry.
This only impacts the store buffer (MSBDS, previously PSF) because it gets statically re-partitioned when a thread comes in and out of idle.
Quoted 1 line
From the point of view of the thread going idle, when going idle my half
of the store buffers get given to the other thread and potentially leak my secrets, whereas when coming out of idle, the other threads store buffers get split with me, potentially leaking their secrets.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Duh, indeed. Easy enough to fix.
Delta patch below. Stupid me even mentioned the repartioning on both sides in the changelog.
It's not needed.
But will the part of the store buffer that gets repartitioned not contain state of the other sibling?
If it can see our data after we go idle; why can't we see its data when we come out of idle?
[MODERATED] Re: [patch 4/8] MDS basics 4
In reply to Andi KleenQuoted 10 lines · speck for Andi Kleen
Quoted 9 lines · speck for Andrew Cooper
Quoted 8 lines · speck for Thomas Gleixner
Subject: [patch 4/8] x86/speculation/mds: Conditionaly clear CPU buffers on idle entry
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on idle entry. This is independent of other MDS mitigations because the idle entry invocation to mitigate the potential leakage due to store buffer repartitioning is only necessary on SMT systems.
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Add the actual invocations to the different halt/mwait variants which covers all usage sites. mwaitx is not patched as it's not available on Intel CPUs.
Signed-off-by Thomas Gleixner <[email protected]>Unfortunately, clearing is needed on the exit from idle as well as the entry.
Not with the full flush. There it would always clear on next kernel exit.
What about a PSF gadget between idle and the return to userspace? That can end up leaking data before the next flush happens.
What is the latency of a microcoded VERW? Its surely unmeasurable in the noise compared to everything else involved in going idle.
~Andrew
Re: [patch 4/8] MDS basics 4
In reply to Thomas GleixnerQuoted 23 lines · speck for Thomas Gleixner
Quoted 21 lines · speck for Andrew Cooper
Quoted 8 lines · speck for Thomas Gleixner
Subject: [patch 4/8] x86/speculation/mds: Conditionaly clear CPU buffers on idle entry
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on idle entry. This is independent of other MDS mitigations because the idle entry invocation to mitigate the potential leakage due to store buffer repartitioning is only necessary on SMT systems.
Quoted 5 lines
Add the actual invocations to the different halt/mwait variants which covers all usage sites. mwaitx is not patched as it's not available on Intel CPUs.
Signed-off-by Thomas Gleixner <[email protected]>Unfortunately, clearing is needed on the exit from idle as well as the entry.
This only impacts the store buffer (MSBDS, previously PSF) because it gets statically re-partitioned when a thread comes in and out of idle.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
From the point of view of the thread going idle, when going idle my half
of the store buffers get given to the other thread and potentially leak my secrets, whereas when coming out of idle, the other threads store buffers get split with me, potentially leaking their secrets.
Duh, indeed. Easy enough to fix.
I take that back. It's a pointless excercise. Either the flush happens on exit to user or on vmenter. Up to there nothing to worry about. If we have to worry about the kernel being the malicious entity, we have other problems.
Thanks,
tglx
[MODERATED] Re: [patch 8/8] MDS basics 8
In reply to Thomas GleixnerDocumentation/admin-guide/hw-vuln/mds.rst line 6
Quoted 9 lines · speck for Thomas Gleixner
Documentation/admin-guide/hw-vuln/mds.rst new file +6
-0@@ -0,0 +1,230 @@ +MDS - Microarchitectural Data Sampling +====================================== + +Microarchitectural Data Sampling is a hardware vulnerability which allows +unprivileged speculative access to data which is available in various CPU +internal buffers.
Strictly speaking, it is a group of related vulnerabilities.
The distinction is further complicated because some processors are only affected by a subset of the group.
Quoted 19 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +19
-0+ +Affected processors +------------------- + +This vulnerability affects a wide range of Intel processors. The +vulnerability is not present on: + + - Processors from AMD, Centaur and other non Intel vendors + + - Older processor models, where the CPU family is < 6 + + - Some Atoms (Bonnell, Saltwell, Goldmont, GoldmontPlus) + + - Intel processors which have the ARCH_CAP_MDS_NO bit set in the + IA32_ARCH_CAPABILITIES MSR. + +Whether a processor is affected or not can be read out from the MDS +vulnerability file in sysfs. See :ref:`mds_sys_info`. +
Documentation/admin-guide/hw-vuln/mds.rst line 35
Quoted 10 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +10
-0+Related CVEs +------------ + +The following CVE entries are related to the MDS vulnerability: + + ============== ===== ============================================== + CVE-2018-12126 MSBDS Microarchitectural Store Buffer Data Sampling + CVE-2018-12130 MFBDS Microarchitectural Fill Buffer Data Sampling + CVE-2018-12127 MLPDS Microarchitectural Load Port Data Sampling + ============== ===== ==============================================
Any chance of listing in CVE order? Something feels weird having it like this.
Documentation/admin-guide/hw-vuln/mds.rst line 40
Quoted 5 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +5
-0+ +Problem +------- + +When performing store, load, L1 refill operations, processors write data
Possibly just limit it to loads and stores? From what I've been told, the fill buffers get all data exiting the pipeline (including WC and UC) and the L1 refill bit is just one aspect of their functionality.
Documentation/admin-guide/hw-vuln/mds.rst line 48
Quoted 8 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +8
-0+into temporary microarchitectural structures (buffers). The data in the +buffer can be forwarded to load operations as an optimization. + +Under certain conditions, usually a fault/assist caused by a load +operation, data unrelated to the load memory address can be speculatively +forwarded from the buffers. Because the load operation causes a fault or +assist and its result will be discarded, the forwarded data will not cause +incorrect programm execution or state changes. But a malicious operation
Hmm - Today I learnt that there is a difference between the English and German spelling of programm.
Documentation/admin-guide/hw-vuln/mds.rst line 50
Quoted 2 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +2
-0+may be able to forward this speculative data to a disclosure gadget which +allows in turn to infer the value via a cache side channel attack.
Its not restricted to cache side channels. There are other options available for a sufficiently crafty attacker.
Documentation/admin-guide/hw-vuln/mds.rst line 53
Quoted 3 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +3
-0+ +Because the buffers are potentially shared between Hyper-Threads cross +Hyper-Thread attacks may be possible.
The cooperating-hyperthread attack has been demonstrated in practice.
Another piece of information which may not have filtered through from Keybase a while ago is (an indirect report) of a researcher's description of their PoC against Linux.
One thread sits and makes setrlimit() syscalls with a userspace pointer to train the branch predictor to strongly take the copy_from_user() path. It then switches the pointer to a kernel address of interest for the next syscall.
Speculation takes the strongly taken path, reads from kernel space and writes into the local buffer (also in kernel space), but this uses a fill buffer in the pipeline. The other thread can snatch the data by repeatedly sampling the fill buffers.
The data rate wasn't very high (6 bytes per second) when trying to leak a 4k page containing known ASCII text, but it clearly demonstrates that the attack is possible.
This particular PoC also depends on SMAP being disabled, as STAC/CLAC instructions have lfence properties, but this is an implementation detail rather than an architectural guarantee. There are also plenty of opportunities to pull this attack off outside of a STAC/CLAC pair.
Quoted 33 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +33
-0+ +As the buffer sizes are smaller than the L1 cache, which was target of +previous vulnerabilities, e.g. Meltdown, L1TF, the vulnerability is harder +to exploit than with those attack vectors. + + +Attack scenarios +---------------- + + TBD + +.. _mds_sys_info: + +MDS system information +----------------------- + +The Linux kernel provides a sysfs interface to enumerate the current MDS +status of the system: whether the system is vulnerable, and which +mitigations are active. The relevant sysfs file is: + +/sys/devices/system/cpu/vulnerabilities/mds + +The possible values in this file are: + + ============================== ==================================== + 'Not affected' The processor is not vulnerable + 'Vulnerable' The processor is vulnerable, but no + mitigation enabled + 'Mitigation: CPU buffer clear' The processor is vulnerable and the + CPU buffer clearing mitigation is + enabled. + ============================== ==================================== +
Documentation/admin-guide/hw-vuln/mds.rst line 96
Quoted 10 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +10
-0+If the processor is vulnerable then the following information is appended +to the above information: + + - SMT status: + + ======================== ============================================ + 'SMT vulnerable' SMT is enabled + 'SMT disabled' SMT is disabled + 'SMT Host state unknown' Kernel runs in a VM, Host SMT state unknown + ======================== ============================================
To the virtualisation side of things, in Xen we are working to ensure that the reported topology is accurate and is scheduled in a safe way.
There is a proposed extension to the HyperV Viridian specification, which will be a "topology can be trusted for safety" bit, and Xen will be gaining a similar mechanism as soon as we've made it work.
Quoted 25 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +25
-0+ + +Mitigation mechanism +------------------------- + +The kernel detects the affected CPUs and the presence of the microcode +which is required. + +If a CPU is affected and the microcode is available, then the kernel +enables the mitigation by default. The mitigation can be controlled at boot +time via a kernel command line option. See +:ref:`mds_mitigation_control_command_line`. + +.. _cpu_buffer_clear_full: + +Unconditional CPU buffer clearing +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + + The mitigation for MDS clears the affected CPU buffers unconditionally + on return to user space and when entering a guest. + + If SMT is enabled it also clears the buffers on idle entry, but that's + not a sufficient SMT protection for all MDS variants; it covers solely + MSBDS. +
Documentation/admin-guide/hw-vuln/mds.rst line 131
Quoted 10 lines
Documentation/admin-guide/hw-vuln/mds.rst new file +10
-0+.. _virt_mechanism: + +Virtualization mitigation +^^^^^^^^^^^^^^^^^^^^^^^^^ + + If the CPU is also affected by L1TF and the L1D flush mitigation is + enabled and up to date microcode is available, the L1D flush mitigation + is automatically protecting the guest transition. For details on L1TF + and virtualization see: + :ref:`Documentation/admin-guide/hw-vuln//l1tf.rst <mitigation_control_kvm>`.
Spurious double slash ?
~Andrew
Re: [patch 5/8] MDS basics 5
In reply to Andi KleenQuoted 12 lines · speck for Andi Kleen
Quoted 10 lines · speck for Thomas Gleixner
arch/x86/kernel/cpu/bugs.c static void update_indir_branch_cond(voi · line 679
Quoted 7 lines · speck for Thomas Gleixner
arch/x86/kernel/cpu/bugs.c +7
-0+/* Update the static key controlling the MDS CPU buffer clear in idle */ +static void update_mds_branch_idle (void ) +{ +if (sched_smt_active ()) +static_branch_enable (&user_mds_clear_cpu_buffers); +else +static_branch_disable (&user_mds_clear_cpu_buffers);Those obviously need s/user/idle/
Quoted 2 lines
Please don't post untested crap like this. It would be really bad if someone applied it.
Your last series contained serious bugs and did not even compile...
Quoted 5 lines
This can be easily verified just by looking at a few PT traces.
Or just use MDSv2 which actually worked.
BTW even for the minimum version virtualization is definitely needed.
Did you read the cover letter? It's a todo and I did nowhere claim that it is complete and perfect.
Thanks,
tglx
Re: [patch 5/8] MDS basics 5
In reply to Andi KleenQuoted 9 lines · speck for Andi Kleen
arch/x86/kernel/cpu/bugs.c static void x86_amd_ssb_disable(void) · line 241
Quoted 5 lines
arch/x86/kernel/cpu/bugs.c +5
-0+case MDS_MITIGATION_AUTO : +case MDS_MITIGATION_FULL : +if (boot_cpu_has (X86_FEATURE_MD_CLEAR )) { + mds_mitigation =MDS_MITIGATION_FULL ; +static_branch_enable (&user_mds_clear_cpu_buffers);This violates Linus' feedback of unconditionally enabling for one year to work around VMWare breakage.
That's easy enough to turn the other way round. That's what review is for, isn't it?
What's the VMware breakage? I can't remember that I've read that mail.
Thanks,
tglx
Quoted 5 lines · speck for Andi Kleen
Quoted 3 lines
So while being grumpy about this communication fail, I'm even more grumpy about the fact, that we don't have even the minimal full/off mitigation in place in a workable form. I asked specifically for this
There seems to be certainly a communication fail here.
Definitely not. I asked you to provide one based on review feedback.
Quoted 1 line
The minimal version has been posted on this mailing list in late December
You just forgot to mention that it's in unmergeable state as is any later version.
Quoted 1 line
(although it only showed up in early January because ...)
Because I went on vacation and did not serve as Intel's butler for a while.
Thanks,
tglx
[MODERATED] Re: [patch 5/8] MDS basics 5
In reply to Thomas GleixnerQuoted 10 lines · speck for Thomas Gleixner
Quoted 8 lines · speck for Andi Kleen
arch/x86/kernel/cpu/bugs.c static void x86_amd_ssb_disable(void) · line 241
Quoted 5 lines
arch/x86/kernel/cpu/bugs.c +5
-0+case MDS_MITIGATION_AUTO : +case MDS_MITIGATION_FULL : +if (boot_cpu_has (X86_FEATURE_MD_CLEAR )) { + mds_mitigation =MDS_MITIGATION_FULL ; +static_branch_enable (&user_mds_clear_cpu_buffers);This violates Linus' feedback of unconditionally enabling for one year to work around VMWare breakage.
Quoted 4 lines
That's easy enough to turn the other way round. That's what review is for, isn't it?
What's the VMware breakage? I can't remember that I've read that mail.
VMWare reckon they will have updated microcode long before guest VM's get to the the MD_CLEAR CPUID bit, and wanted guests to issue the VERW instruction in the hope that it might do something.
~Andrew
Re: [patch 5/8] MDS basics 5
Quoted 11 lines · speck for Andrew Cooper
Quoted 10 lines · speck for Thomas Gleixner
Quoted 8 lines · speck for Andi Kleen
arch/x86/kernel/cpu/bugs.c static void x86_amd_ssb_disable(void) · line 241
Quoted 5 lines
arch/x86/kernel/cpu/bugs.c +5
-0+case MDS_MITIGATION_AUTO : +case MDS_MITIGATION_FULL : +if (boot_cpu_has (X86_FEATURE_MD_CLEAR )) { + mds_mitigation =MDS_MITIGATION_FULL ; +static_branch_enable (&user_mds_clear_cpu_buffers);This violates Linus' feedback of unconditionally enabling for one year to work around VMWare breakage.
Quoted 8 lines
Quoted 4 lines
That's easy enough to turn the other way round. That's what review is for, isn't it?
What's the VMware breakage? I can't remember that I've read that mail.
VMWare reckon they will have updated microcode long before guest VM's get to the the
MD_CLEARCPUID bit, and wanted guests to issue the VERW instruction in the hope that it might do something.
Thanks for the reminder. I was in the room and chuckled when this was discussed, but it slipped from memory.
Oh well...
Thanks,
tglx
Re: [patch 4/8] MDS basics 4
In reply to Andi KleenQuoted 9 lines · speck for Andi Kleen
Quoted 8 lines · speck for Thomas Gleixner
Subject: [patch 4/8] x86/speculation/mds: Conditionaly clear CPU buffers on idle entry
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on idle entry. This is independent of other MDS mitigations because the idle entry invocation to mitigate the potential leakage due to store buffer repartitioning is only necessary on SMT systems.
Quoted 5 lines
Quoted 3 lines
Add the actual invocations to the different halt/mwait variants which covers all usage sites. mwaitx is not patched as it's not available on Intel CPUs.
This doesn't handle ACPI IO port idling correctly.
But it neither breaks ARM64 nor IA64 due to cpu_clear_idle() not being defined in acpi/processor_idle.c. Talking about untested crap....
That aside, it's easy enough to fix that.
OTOH, the question is whether it matters. Everything what matters moved towards intel_idle almost a decade ago and even my nehalem box does not use the old acpi processor idle stuff.
If you can come up with a compelling reason to address that, I'm happy to fix it. If not, it'd be just wasted time and disk space.
Thanks,
tglx
Re: [patch 3/8] MDS basics 3
In reply to Andi KleenQuoted 9 lines · speck for Andi Kleen
Quoted 6 lines · speck for Thomas Gleixner
Subject: [patch 3/8] x86/speculation/mds: Clear CPU buffers on exit to user
From: Thomas Gleixner <[email protected]>Add a static key which controls the invocation of the CPU buffer clear mechanism on exit to user space and add the call into
prepare_exit_to_usermode()right before actually returning.This is missing the NMI exit case.
And that exit case is what? Security voodoo just in case?
Your NMI covering patch 'changelog':
"NMIs don't go through the normal exit code when exiting to user
space. Normally we consider NMIs not sensitive anyways, but they need special handling with mds=full.
So add an explicit check to do_nmi to clear the CPU with mds=full"
is useful in that regard as always. It neither explains why NMIs are not considered sensitive nor does it provide any useful argument why mds=full would make any difference.
Can you please answer the following questions:
1) Who is 'we'?
Andi Kleen using pluralis majestatis or are you referring to the kernel community as a whole?
I don't care either way because none of the options provides a rationale.
2) What is 'normally' ?
I'm not aware of any normative statement regarding this matter.
3) Why are NMIs considered not sensitive?
This is missing any explanation why this is the case
4) What makes mds=full different?
Just because it's named 'full'? That surely does not qualify as a technical argument which provides a useful rationale for adding this flush.
Please provide a conclusive technical explanation why a NMI entered from user space and returning to that would expose sensible information to an attacker in an exploitable way.
Thanks,
tglx
Re: [patch 3/8] MDS basics 3
Quoted 4 lines · speck for Thomas Gleixner
Please provide a conclusive technical explanation why a NMI entered from user space and returning to that would expose sensible information to an attacker in an exploitable way.
And while at it, please provide explanations why this does not apply for exceptions which hit the paranoid section, i.e. in the middle of returning to user space where the regular exit path cannot be invoked.
Thanks,
tglx
Re: [patch 5/8] MDS basics 5
In reply to Thomas GleixnerQuoted 5 lines · speck for Thomas Gleixner
Quoted 1 line · speck for Andi Kleen
BTW even for the minimum version virtualization is definitely needed.
Did you read the cover letter? It's a todo and I did nowhere claim that it is complete and perfect.
And for completeness sake:
Do you refer to that so complicated exposure of the MD_CLEAR feature bit to the guest?
I assume so, because in your patches nothing else is done regarding virtualization.
But if you've read my cover letter carefully you might have noticed that there is no support for the case where the CPU is not affected by L1TF, but is affected by MDS. That's not a theoretical case at all.
I fail to see that case addressed in your so complete and working patchset version whatever. That's a way more interesting problem than adding a CPU feature flag to some existing initializer, which is is an obvious and purely mechanical operation.
Thanks,
tglx
[MODERATED] Re: [patch 3/8] MDS basics 3
In reply to Thomas GleixnerThomas,
Quoted 3 lines
Please provide a conclusive technical explanation why a NMI entered from user space and returning to that would expose sensible information to an attacker in an exploitable way.
Okay so you're not actually implementing full, like you stated earlier, but a strange undocumented minimal version of lazy.
Lazy is always a trade off. Since we still run with a single address space the CPU can always do some prefetching or speculative execution and fetch something nearby that the code doesn't actually access intentionally, which may end up leaking through some buffer.
In such a case even a NMI could leak.
Of course such a case is unlikely, but in theory it could happen.
The intention of the full option was to allow an option for people who cannot accept any risk at all. Of course that's not the right approach for most users, but for a few it might be.
If you implement full you should implement it properly and not leave holes.
If you're not interested in code review please state so and I will not bother anymore.
-Andi
Re: [patch 3/8] MDS basics 3
Andi,
Quoted 12 lines · speck for Andi Kleen
Quoted 3 lines
Please provide a conclusive technical explanation why a NMI entered from user space and returning to that would expose sensible information to an attacker in an exploitable way.
Okay so you're not actually implementing full, like you stated earlier, but a strange undocumented minimal version of lazy.
Lazy is always a trade off. Since we still run with a single address space the CPU can always do some prefetching or speculative execution and fetch something nearby that the code doesn't actually access intentionally, which may end up leaking through some buffer.
Quoted 8 lines
In such a case even a NMI could leak.
Of course such a case is unlikely, but in theory it could happen.
The intention of the full option was to allow an option for people who cannot accept any risk at all. Of course that's not the right approach for most users, but for a few it might be.
That makes sense if you map full to paranoid, which I tried to avoid, but fair enough.
Thanks for the explanation.
tglx
[MODERATED] Encrypted Message
In reply to Thomas GleixnerFrom: Jon Masters <[email protected]>
To: speck for Thomas Gleixner <[email protected]>
Subject: Re: [patch 0/8] MDS basics 0
Hi Thomas,
Just a note on testing. I built a few Coffelake client systems for Red Hat using the 8086K anniversary processor for which we have test ucode. I will build and test these patches and ask the RH perf team to test.
Jon.
Signature · 1 line
Computer Architect | Sent with my Fedora powered laptop