x86: work around MPX Erratum
RFC PATCH 1/1 x86: work around MPX Erratum
From: Dave Hansen <[email protected]>
Big core processors (Xeon/Core) are affected by an MPX erratum. This erratum can only be triggered when a system is not using Supervisor Mode Execution Prevention (SMEP). To work around this, we ensure that MPX can only be used in cases where SMEP is present in the processor and enabled.
MPX and SMEP are present together in the *vast* majority of cases, and the kernel does not generally execute code that is readable by userspace, so the real-world impact of this issue is expected to be very limited.
Note that we don't have a good way to tell if we are on an unaffected Atom processor or an affected Core/Xeon. The solution for the moment is to just be conservative and assume that everything is affected unless explicitly known to be unaffected.
More details on erratum:
SKD046 Branch Instructions May Initialize MPX Bound Registers Incorrectly
Problem:
Depending on the current Intel MPX (Memory Protection Extensions) configuration, execution of certain branch instructions (near CALL, near RET, near JMP, and Jcc instructions) without a BND prefix (F2H) initialize the MPX bound registers. Due to this erratum, such a branch instruction that is executed both with CPL = 3 and with CPL < 3 may not use the correct MPX configuration register (BNDCFGU or BNDCFGS, respectively) for determining whether to initialize the bound registers; it may thus initialize the bound registers when it should not, or fail to initialize them when it should.
Implication:
A branch instruction that has executed both in user mode and in supervisor mode (from the same linear address) may cause a #BR (bound range fault) when it should not have or may not cause a #BR when it should have. Workaround An operating system can avoid this erratum by setting CR4.SMEP[bit 20] to enable supervisor-mode execution prevention (SMEP). When SMEP is enabled, no code can be executed both with CPL = 3 and with CPL < 3.
---
b/arch/x86/include/asm/bugs.h | 9 ++++-- b/arch/x86/kernel/cpu/common.c | 3 ++ b/arch/x86/kernel/cpu/intel.c | 58 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 68 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/intel.c renamed from arch/x86/kernel/cpu/intel.c~skl-mpx-errata-exclude-atom +58 -0
@@ -25,6 +25,62 @@#include <asm/apic.h> #endif +/* +* Just in case our CPU detection goes bad, allow a way to +* override the disabling of MPX. +*/ +static int forcempx; +static int __init forcempx_setup (char *__unused) +{ + forcempx =1 ; +return 1 ; +} +__setup ("intel-skd-046-workaround=disable" , forcempx_setup); + +/* +* x86_model values come from: SDM Vol 3. Chapter 35 +*/ +static int is_mpx_affected_microarch (struct cpuinfo_x86 *c) +{ +/* Only family 6 is affected */ +if (c->x86 !=0x6 ) +return 0 ; + +/* We know these Atom models are unaffected, for sure */ +switch (c->x86_model) { +case 0x5F :/* "Future Intel Atom ... Goldmont */ +case 0x5C :/* "Future Intel Atom ... Goldmont */ +return 0 ; + } +/* +* We will get here on future unknown processors and all +* Core/Xeons. They might be unaffected Atoms or +* affected Core/Xeons. Be conservative and assume +* processor is affected. +* +* Once the complete list of Core/Xeon models is known +* it can be added here, and the Atom list removed. +*/ +return 1 ; +} + +void check_mpx_erratum (struct cpuinfo_x86 *c) +{ +if (forcempx) +return ; +/* +* Turn off MPX feature on affected CPUs where SMEP is not +* available or disabled. +* +* Works around Intel Erratum: "SKD046 Branch Instructions +* May Initialize MPX Bound Registers Incorrectly." +*/ +if (is_mpx_affected_microarch (c) && +cpu_has (c,X86_FEATURE_MPX ) && + !cpu_has (c,X86_FEATURE_SMEP )) +setup_clear_cpu_cap (X86_FEATURE_MPX ); +} +static void early_init_intel (struct cpuinfo_x86 *c) {u64 misc_enable;
@@ -173,6 +229,8 @@ static void early_init_intel(struct cpuiif (edx & (1U <<28 )) c->x86_coreid_bits =get_count_order ((ebx >>16 ) &0xff ); } + +check_mpx_erratum (c); }#ifdef CONFIG_X86_32
arch/x86/kernel/cpu/common.c renamed from arch/x86/kernel/cpu/common.c~skl-mpx-errata-exclude-atom +3 -0
@@ -37,6 +37,7 @@#include <asm/mtrr.h> #include <linux/numa.h> #include <asm/asm.h> +#include <asm/bugs.h> #include <asm/cpu.h> #include <asm/mce.h> #include <asm/msr.h>
@@ -270,6 +271,8 @@ static inline void squash_the_stupid_serstatic __init int setup_disable_smep (char *arg) {setup_clear_cpu_cap (X86_FEATURE_SMEP ); +/* also check for things that depend on SMEP being enabled */ +check_mpx_erratum (&boot_cpu_data);return 1 ; }__setup ("nosmep" , setup_disable_smep);
arch/x86/include/asm/bugs.h renamed from arch/x86/include/asm/bugs.h~skl-mpx-errata-exclude-atom +7 -2
@@ -3,10 +3,15 @@extern void check_bugs (void ); -#if defined(CONFIG_CPU_SUP_INTEL ) && defined(CONFIG_X86_32 ) +#if defined(CONFIG_CPU_SUP_INTEL ) +void check_mpx_erratum (struct cpuinfo_x86 *c); +#else +static inline void check_mpx_erratum (struct cpuinfo_x86 *c) {} +#if defined(CONFIG_X86_32 )int ppro_with_ram_bug (void );#else static inline int ppro_with_ram_bug (void ) {return 0 ; } -#endif +#endif /* CONFIG_X86_32 */ +#endif /* CONFIG_CPU_SUP_INTEL */ #endif /* _ASM_X86_BUGS_H */
_
Quoted 8 lines · Dave Hansen
From: Dave Hansen <[email protected]>
Big core processors (Xeon/Core) are affected by an MPX erratum. This erratum can only be triggered when a system is not using Supervisor Mode Execution Prevention (SMEP). To work around this, we ensure that MPX can only be used in cases where SMEP is present in the processor and enabled.
That's fair enough IMHO.
Quoted 23 lines
MPX and SMEP are present together in the *vast* majority of cases, and the kernel does not generally execute code that is readable by userspace, so the real-world impact of this issue is expected to be very limited.
Note that we don't have a good way to tell if we are on an unaffected Atom processor or an affected Core/Xeon. The solution for the moment is to just be conservative and assume that everything is affected unless explicitly known to be unaffected.
arch/x86/kernel/cpu/intel.c renamed from arch/x86/kernel/cpu/intel.c~skl-mpx-errata-exclude-atom +12
-0+static int is_mpx_affected_microarch (struct cpuinfo_x86 *c) +{ +/* Only family 6 is affected */ +if (c->x86 !=0x6 ) +return 0 ; + +/* We know these Atom models are unaffected, for sure */ +switch (c->x86_model) { +case 0x5F :/* "Future Intel Atom ... Goldmont */ +case 0x5C :/* "Future Intel Atom ... Goldmont */ +return 0 ; + }
arch/x86/kernel/cpu/intel.c line 64
Quoted 10 lines
arch/x86/kernel/cpu/intel.c renamed from arch/x86/kernel/cpu/intel.c~skl-mpx-errata-exclude-atom +10
-0+/* +* We will get here on future unknown processors and all +* Core/Xeons. They might be unaffected Atoms or +* affected Core/Xeons. Be conservative and assume +* processor is affected. +* +* Once the complete list of Core/Xeon models is known +* it can be added here, and the Atom list removed. +*/ +return 1 ;
So instead of trying to sort out the erratum, could we not just generally make MPX dependent on SMEP and be done with it? MPX is a sophisticated security feature, and it makes little sense to not do SMEP if you have it available.
Anyone who is absolutely desperate to disable SMEP while enabling MPX is free to step in and make his case.
I.e. I think the simplest workaround would be to disable MPX if SMEP is disabled.
Thanks,
Ingo
Quoted 26 lines · Ingo Molnar
Quoted 14 lines
Quoted 14 lines
arch/x86/kernel/cpu/intel.c renamed from arch/x86/kernel/cpu/intel.c~skl-mpx-errata-exclude-atom +14
-0+static int is_mpx_affected_microarch (struct cpuinfo_x86 *c) +{ +/* Only family 6 is affected */ +if (c->x86 !=0x6 ) +return 0 ; + +/* We know these Atom models are unaffected, for sure */ +switch (c->x86_model) { +case 0x5F :/* "Future Intel Atom ... Goldmont */ +case 0x5C :/* "Future Intel Atom ... Goldmont */ +return 0 ; + } +/* +* We will get here on future unknown processors and all Quoted 8 lines
arch/x86/kernel/cpu/intel.c line 64
Quoted 8 lines
arch/x86/kernel/cpu/intel.c renamed from arch/x86/kernel/cpu/intel.c~skl-mpx-errata-exclude-atom +8
-0+* Core/Xeons. They might be unaffected Atoms or +* affected Core/Xeons. Be conservative and assume +* processor is affected. +* +* Once the complete list of Core/Xeon models is known +* it can be added here, and the Atom list removed. +*/ +return 1 ;So instead of trying to sort out the erratum, could we not just generally make MPX dependent on SMEP and be done with it? MPX is a sophisticated security feature, and it makes little sense to not do SMEP if you have it available.
Quoted 2 lines
Anyone who is absolutely desperate to disable SMEP while enabling MPX is free to step in and make his case.
My concern was not necessarily with folks booting with 'nosmep', but with processors that have MPX present and SMEP fused off (or made unavailable by a hypervisor) and which are unaffected by this issue.
People would have to be very careful to never create a processor which did not have SMEP but did have MPX, since MPX would effectively be unusable on such a processor.
Borislav Petkov
maintainerQuoted 1 line · Dave Hansen
My concern was not necessarily with folks booting with 'nosmep', but
Btw, does anything speak for even keeping that 'nosmep' thing?
Quoted 2 lines
with processors that have MPX present and SMEP fused off (or made unavailable by a hypervisor) and which are unaffected by this issue.
So we won't init MPX on those...
Quoted 3 lines
People would have to be very careful to never create a processor which did not have SMEP but did have MPX, since MPX would effectively be unusable on such a processor.
We can disable that combination in qemu too, right?
Signature · 4 lines
Regards/Gruss,
Boris.
ECO tip #101: Trim your mails when you reply.Quoted 4 lines · Borislav Petkov
Quoted 1 line · Dave Hansen
My concern was not necessarily with folks booting with 'nosmep', but
Btw, does anything speak for even keeping that 'nosmep' thing?
Generally, I'm not sure we need the no$foo options at all. There's always "clearcpuid=" which does the same thing. It just requires you to go look up the X86_FEATURE_* bit first.
Quoted 4 lines
Quoted 2 lines
with processors that have MPX present and SMEP fused off (or made unavailable by a hypervisor) and which are unaffected by this issue.
So we won't init MPX on those...
Yes, and as long as such a processor doesn't exist today and never exists in the future or the folks that buy such a processor truly don't care about MPX, that's fine to do. I'm just a bit nervous about the whole "never exists in the future" part.
Quoted 5 lines
Quoted 3 lines
People would have to be very careful to never create a processor which did not have SMEP but did have MPX, since MPX would effectively be unusable on such a processor.
We can disable that combination in qemu too, right?
What do you mean by disable? Have qemu error out if MPX and SMEP aren't disabled in concert with each other?
Quoted 7 lines · Dave Hansen
Quoted 2 lines
So we won't init MPX on those...
Yes, and as long as such a processor doesn't exist today and never exists in the future or the folks that buy such a processor truly don't care about MPX, that's fine to do. I'm just a bit nervous about the whole "never exists in the future" part.
I don't think we need to care.
If a CPU doesn't support SMEP, there really is no reason for us to ever support MPX either.
It's not like there is a clamoring for MPX support in the first place (have you ever heard of anybody actually asking for or using it?), and quite frankly, it's a _lot_ more complicated from a CPU core side than SMEP (which is trivial).
So if Intel ever releases a CPU with MPX but without SMEP, nobody will ever care about the MPX part being useless.
Linus
Borislav Petkov
maintainerQuoted 3 lines · Dave Hansen
Generally, I'm not sure we need the no$foo options at all. There's always "clearcpuid=" which does the same thing. It just requires you to go look up the X86_FEATURE_* bit first.
Yeah, the "no-" things are all chicken bits which we should get rid of at some point anyway.
Quoted 2 lines
What do you mean by disable? Have qemu error out if MPX and SMEP aren't disabled in concert with each other?
Yap.
Signature · 4 lines
Regards/Gruss,
Boris.
ECO tip #101: Trim your mails when you reply.Quoted 31 lines · Dave Hansen
Quoted 26 lines · Ingo Molnar
Quoted 14 lines
Quoted 14 lines
arch/x86/kernel/cpu/intel.c renamed from arch/x86/kernel/cpu/intel.c~skl-mpx-errata-exclude-atom +14
-0+static int is_mpx_affected_microarch (struct cpuinfo_x86 *c) +{ +/* Only family 6 is affected */ +if (c->x86 !=0x6 ) +return 0 ; + +/* We know these Atom models are unaffected, for sure */ +switch (c->x86_model) { +case 0x5F :/* "Future Intel Atom ... Goldmont */ +case 0x5C :/* "Future Intel Atom ... Goldmont */ +return 0 ; + } +/* +* We will get here on future unknown processors and all Quoted 8 lines
arch/x86/kernel/cpu/intel.c line 64
Quoted 8 lines
arch/x86/kernel/cpu/intel.c renamed from arch/x86/kernel/cpu/intel.c~skl-mpx-errata-exclude-atom +8
-0+* Core/Xeons. They might be unaffected Atoms or +* affected Core/Xeons. Be conservative and assume +* processor is affected. +* +* Once the complete list of Core/Xeon models is known +* it can be added here, and the Atom list removed. +*/ +return 1 ;So instead of trying to sort out the erratum, could we not just generally make MPX dependent on SMEP and be done with it? MPX is a sophisticated security feature, and it makes little sense to not do SMEP if you have it available.
Quoted 2 lines
Anyone who is absolutely desperate to disable SMEP while enabling MPX is free to step in and make his case.
Quoted 9 lines
My concern was not necessarily with folks booting with 'nosmep', but with processors that have MPX present and SMEP fused off (or made unavailable by a hypervisor) and which are unaffected by this issue.
People would have to be very careful to never create a processor which did not have SMEP but did have MPX, since MPX would effectively be unusable on such a processor.
Having actually read the erratum: how can this affect Linux at all under any scenario where user code hasn't already completely compromised the kernel?
I.e. why do we care about this erratum?
Quoted 4 lines · Andy Lutomirski
Having actually read the erratum: how can this affect Linux at all under any scenario where user code hasn't already completely compromised the kernel?
If it matches purely on linear address, you will potentially have interesting situations with people running in virtualized environments and crashing programs in other virtual containers or the host.
I do agree that we likely don't care all that much (especially with the lack of users in the first place). But considering that the trivial workaround is to make sure SMEP is enabled - which we want to heavily encourage anyway - I also don't think there's any real downside to just enforcing that workaround.
Linus
Quoted 9 lines · Linus Torvalds
Quoted 4 lines · Andy Lutomirski
Having actually read the erratum: how can this affect Linux at all under any scenario where user code hasn't already completely compromised the kernel?
If it matches purely on linear address, you will potentially have interesting situations with people running in virtualized environments and crashing programs in other virtual containers or the host.
If so, how does SMEP help? There's no guarantee that the same linear address in two different VMs has the same U/S bit in the page tables.
IOW, I'm having trouble seeing a situation under which SMEP matters and Linux would be affected in the absence of SMEP.
--Andy
Quoted 5 lines · Andy Lutomirski
Having actually read the erratum: how can this affect Linux at all under any scenario where user code hasn't already completely compromised the kernel?
I.e. why do we care about this erratum?
First of all, with SMEP, it doesn't affect us. At all.
Without SMEP, there would have to be a page accessible to userspace that the kernel executes instructions from. The only thing that I can think of that's normally user-accessible and not _controlled_ by userspace is the VDSO. But the kernel never actually executes from it, so it doesn't matter here.
I've heard reports of (but no actual cases in the wild of) folks remapping kernel text to be user-accessible so that userspace can execute it, or of having the kernel jump into user-provided libraries. Those are both obviously bonkers and would only be done with out-of-tree gunk, but even if somebody did that, they would be safe from the erratum, with this workaround.
Quoted 15 lines · Dave Hansen
Quoted 5 lines · Andy Lutomirski
Having actually read the erratum: how can this affect Linux at all under any scenario where user code hasn't already completely compromised the kernel?
I.e. why do we care about this erratum?
First of all, with SMEP, it doesn't affect us. At all.
Without SMEP, there would have to be a page accessible to userspace that the kernel executes instructions from. The only thing that I can think of that's normally user-accessible and not _controlled_ by userspace is the VDSO. But the kernel never actually executes from it, so it doesn't matter here.
Quoted 8 lines
I've heard reports of (but no actual cases in the wild of) folks remapping kernel text to be user-accessible so that userspace can execute it, or of having the kernel jump into user-provided libraries. Those are both obviously bonkers and would only be done with out-of-tree gunk, but even if somebody did that, they would be safe from the erratum, with this workaround.
I'm not convinced this is worth adding any code for, though. If someone adds out of tree crap that does this and manually turns off SMEP, I think they should get to keep both pieces. Frankly, I think I'd *prefer* if the kernel crashed when calling user addresses like that just to discourage it.
Signature · 2 lines
Andy Lutomirski AMA Capital Management, LLC
Quoted 21 lines · Andy Lutomirski
Quoted 14 lines · Dave Hansen
Quoted 5 lines · Andy Lutomirski
Having actually read the erratum: how can this affect Linux at all under any scenario where user code hasn't already completely compromised the kernel?
I.e. why do we care about this erratum?
First of all, with SMEP, it doesn't affect us. At all.
Without SMEP, there would have to be a page accessible to userspace that the kernel executes instructions from. The only thing that I can think of that's normally user-accessible and not _controlled_ by userspace is the VDSO. But the kernel never actually executes from it, so it doesn't matter here.
Quoted 5 lines
I've heard reports of (but no actual cases in the wild of) folks remapping kernel text to be user-accessible so that userspace can execute it, or of having the kernel jump into user-provided libraries. Those are both obviously bonkers and would only be done with out-of-tree gunk, but even if somebody did that, they would be safe from the erratum, with this workaround.
Quoted 4 lines
I'm not convinced this is worth adding any code for, though. If someone adds out of tree crap that does this and manually turns off SMEP, I think they should get to keep both pieces. Frankly, I think I'd *prefer* if the kernel crashed when calling user addresses like that just to discourage it.
So the thing is, this doesn't have to be any (or much) code per se: my suggestion was to make MPX depend on SMEP on the Kconfig level, so that it's not possible to build MPX without having SMEP.
Secondly, even if you were right and if this erratum didn't affect us, I'm still happy to use pretty much any excuse to further simplify the x86 security state space. 'This erratum suggests that the hardware might be borken without SMEP' is excuse enough in my book to couple MPX with SMEP.
All these zillions of flags and variants really only ever add new failure modes, they don't truly improve security, nor performance.
Thanks,
Ingo
Quoted 24 lines · Ingo Molnar
Quoted 21 lines · Andy Lutomirski
Quoted 19 lines · Dave Hansen
Quoted 5 lines · Andy Lutomirski
Having actually read the erratum: how can this affect Linux at all under any scenario where user code hasn't already completely compromised the kernel?
I.e. why do we care about this erratum?
First of all, with SMEP, it doesn't affect us. At all.
Without SMEP, there would have to be a page accessible to userspace that the kernel executes instructions from. The only thing that I can think of that's normally user-accessible and not _controlled_ by userspace is the VDSO. But the kernel never actually executes from it, so it doesn't matter here.
I've heard reports of (but no actual cases in the wild of) folks remapping kernel text to be user-accessible so that userspace can execute it, or of having the kernel jump into user-provided libraries. Those are both obviously bonkers and would only be done with out-of-tree gunk, but even if somebody did that, they would be safe from the erratum, with this workaround.
Quoted 9 lines
Quoted 4 lines
I'm not convinced this is worth adding any code for, though. If someone adds out of tree crap that does this and manually turns off SMEP, I think they should get to keep both pieces. Frankly, I think I'd *prefer* if the kernel crashed when calling user addresses like that just to discourage it.
So the thing is, this doesn't have to be any (or much) code per se: my suggestion was to make MPX depend on SMEP on the
Kconfiglevel, so that it's not possible to build MPX without having SMEP.
I don't think I understand that suggestion. How can Kconfig protect against:
qemu -cpu host,-smep
?
Quoted 5 lines
Secondly, even if you were right and if this erratum didn't affect us, I'm still happy to use pretty much any excuse to further simplify the x86 security state space. 'This erratum suggests that the hardware might be borken without SMEP' is excuse enough in my book to couple MPX with SMEP.
I'm fine with any variant of this patch. I just can't see any scenario in which it matters, so I think it would also be okay to add a comment somewhere and otherwise ignore it.
--Andy
Quoted 34 lines · Andy Lutomirski
Quoted 24 lines · Ingo Molnar
Quoted 21 lines · Andy Lutomirski
Quoted 19 lines · Dave Hansen
Quoted 5 lines · Andy Lutomirski
Having actually read the erratum: how can this affect Linux at all under any scenario where user code hasn't already completely compromised the kernel?
I.e. why do we care about this erratum?
First of all, with SMEP, it doesn't affect us. At all.
Without SMEP, there would have to be a page accessible to userspace that the kernel executes instructions from. The only thing that I can think of that's normally user-accessible and not _controlled_ by userspace is the VDSO. But the kernel never actually executes from it, so it doesn't matter here.
I've heard reports of (but no actual cases in the wild of) folks remapping kernel text to be user-accessible so that userspace can execute it, or of having the kernel jump into user-provided libraries. Those are both obviously bonkers and would only be done with out-of-tree gunk, but even if somebody did that, they would be safe from the erratum, with this workaround.
Quoted 8 lines
Quoted 4 lines
I'm not convinced this is worth adding any code for, though. If someone adds out of tree crap that does this and manually turns off SMEP, I think they should get to keep both pieces. Frankly, I think I'd *prefer* if the kernel crashed when calling user addresses like that just to discourage it.
So the thing is, this doesn't have to be any (or much) code per se: my suggestion was to make MPX depend on SMEP on the
Kconfiglevel, so that it's not possible to build MPX without having SMEP.
Quoted 5 lines
I don't think I understand that suggestion. How can
Kconfigprotect against:qemu -cpu host,-smep
?
Right, it cannot - but I think the latest patch was pretty close and pretty simple.
Thanks,
Ingo
Quoted 37 lines · Ingo Molnar
Quoted 34 lines · Andy Lutomirski
Quoted 32 lines · Ingo Molnar
Quoted 21 lines · Andy Lutomirski
Quoted 14 lines · Dave Hansen
Quoted 5 lines · Andy Lutomirski
Having actually read the erratum: how can this affect Linux at all under any scenario where user code hasn't already completely compromised the kernel?
I.e. why do we care about this erratum?
First of all, with SMEP, it doesn't affect us. At all.
Without SMEP, there would have to be a page accessible to userspace that the kernel executes instructions from. The only thing that I can think of that's normally user-accessible and not _controlled_ by userspace is the VDSO. But the kernel never actually executes from it, so it doesn't matter here.
Quoted 5 lines
I've heard reports of (but no actual cases in the wild of) folks remapping kernel text to be user-accessible so that userspace can execute it, or of having the kernel jump into user-provided libraries. Those are both obviously bonkers and would only be done with out-of-tree gunk, but even if somebody did that, they would be safe from the erratum, with this workaround.
Quoted 4 lines
I'm not convinced this is worth adding any code for, though. If someone adds out of tree crap that does this and manually turns off SMEP, I think they should get to keep both pieces. Frankly, I think I'd *prefer* if the kernel crashed when calling user addresses like that just to discourage it.
So the thing is, this doesn't have to be any (or much) code per se: my suggestion was to make MPX depend on SMEP on the
Kconfiglevel, so that it's not possible to build MPX without having SMEP.
Quoted 8 lines
Quoted 5 lines
I don't think I understand that suggestion. How can
Kconfigprotect against:qemu -cpu host,-smep
?
Right, it cannot - but I think the latest patch was pretty close and pretty simple.
No objections from me for that patch.
Quoted 4 lines
Thanks,
Ingo
Signature · 2 lines
Andy Lutomirski AMA Capital Management, LLC