FAILED: patch "[PATCH] x86/retpoline: Fill RSB on context switch for affected CPUs" failed to apply to 4.4-stable tree
Subsystems (8)
[email protected]
maintainerThe patch below does not apply to the 4.4-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.
thanks,
greg k-h
------------------ original commit in Linus's tree ------------------
Quoted 1 line
From
c995efd5a740d9cbafbf58bde4973e8b50b4d761Mon Sep 17 00:00:00 2001
From: David Woodhouse <[email protected]>
Date: Fri, 12 Jan 2018 17:49:25 +0000
Subject: [PATCH] x86/retpoline: Fill RSB on context switch for affected CPUs
On context switch from a shallow call stack to a deeper one, as the CPU does 'ret' up the deeper side it may encounter RSB entries (predictions for where the 'ret' goes to) which were populated in userspace.
This is problematic if neither SMEP nor KPTI (the latter of which marks userspace pages as NX for the kernel) are active, as malicious code in userspace may then be executed speculatively.
Overwrite the CPU's return prediction stack with calls which are predicted to return to an infinite loop, to "capture" speculation if this happens. This is required both for retpoline, and also in conjunction with IBRS for !SMEP && !KPTI.
On Skylake+ the problem is slightly different, and an *underflow* of the RSB may cause errant branch predictions to occur. So there it's not so much overwrite, as *filling* the RSB to attempt to prevent it getting empty. This is only a partial solution for Skylake+ since there are many other conditions which may result in the RSB becoming empty. The full solution on Skylake+ is to use IBRS, which will prevent the problem even when the RSB becomes empty. With IBRS, the RSB-stuffing will not be required on context switch.
[ tglx: Added missing vendor check and slighty massaged comments and
changelog ]
arch/x86/entry/entry_32.S +11 -0
@@ -244,6 +244,17 @@ ENTRY(__switch_to_asm) movl %ebx, PER_CPU_VAR(stack_canary)+stack_canary_offset #endif +#ifdef CONFIG_RETPOLINE + /* + * When switching from a shallower to a deeper call stack + * the RSB may either underflow or use entries populated + * with userspace addresses. On CPUs where those concerns + * exist, overwrite the RSB with entries which capture + * speculative execution to prevent attack. + */ + FILL_RETURN_BUFFER %ebx, RSB_CLEAR_LOOPS, X86_FEATURE_RSB_CTXSW +#endif + /* restore callee-saved registers */ popl %esi popl %edi
arch/x86/entry/entry_64.S +11 -0
@@ -487,6 +487,17 @@ ENTRY(__switch_to_asm) movq %rbx, PER_CPU_VAR(irq_stack_union)+stack_canary_offset #endif +#ifdef CONFIG_RETPOLINE + /* + * When switching from a shallower to a deeper call stack + * the RSB may either underflow or use entries populated + * with userspace addresses. On CPUs where those concerns + * exist, overwrite the RSB with entries which capture + * speculative execution to prevent attack. + */ + FILL_RETURN_BUFFER %r12, RSB_CLEAR_LOOPS, X86_FEATURE_RSB_CTXSW +#endif + /* restore callee-saved registers */ popq %r15 popq %r14
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -211,6 +211,7 @@#define X86_FEATURE_AVX512_4FMAPS ( 7*32+17)/* AVX-512 Multiply Accumulation Single precision */ #define X86_FEATURE_MBA ( 7*32+18)/* Memory Bandwidth Allocation */ +#define X86_FEATURE_RSB_CTXSW ( 7*32+19)/* Fill RSB on context switches */ /* Virtualization flags: Linux defined, word 8 */ #define X86_FEATURE_TPR_SHADOW ( 8*32+ 0)/* Intel TPR Shadow */
arch/x86/kernel/cpu/bugs.c +36 -0
@@ -23,6 +23,7 @@#include <asm/alternative.h> #include <asm/pgtable.h> #include <asm/set_memory.h> +#include <asm/intel-family.h> static void __init spectre_v2_select_mitigation (void );
@@ -155,6 +156,23 @@ static enum spectre_v2_mitigation_cmd __init spectre_v2_parse_cmdline(void)return SPECTRE_V2_CMD_NONE ; } +/* Check for Skylake-like CPUs (for RSB handling) */ +static bool __init is_skylake_era (void ) +{ +if (boot_cpu_data.x86_vendor ==X86_VENDOR_INTEL && + boot_cpu_data.x86 ==6 ) { +switch (boot_cpu_data.x86_model) { +case INTEL_FAM6_SKYLAKE_MOBILE : +case INTEL_FAM6_SKYLAKE_DESKTOP : +case INTEL_FAM6_SKYLAKE_X : +case INTEL_FAM6_KABYLAKE_MOBILE : +case INTEL_FAM6_KABYLAKE_DESKTOP : +return true; + } + } +return false; +} +static void __init spectre_v2_select_mitigation (void ) {enum spectre_v2_mitigation_cmd cmd =spectre_v2_parse_cmdline ();
@@ -213,6 +231,24 @@ static void __init spectre_v2_select_mitigation(void) spectre_v2_enabled = mode;pr_info ("%s\n" , spectre_v2_strings[mode]); + +/* +* If neither SMEP or KPTI are available, there is a risk of +* hitting userspace addresses in the RSB after a context switch +* from a shallow call stack to a deeper one. To prevent this fill +* the entire RSB, even when using IBRS. +* +* Skylake era CPUs have a separate issue with *underflow* of the +* RSB, when they will predict 'ret' targets from the generic BTB. +* The proper mitigation for this is IBRS. If IBRS is not supported +* or deactivated in favour of retpolines the RSB fill on context +* switch is required. +*/ +if ((!boot_cpu_has (X86_FEATURE_PTI ) && + !boot_cpu_has (X86_FEATURE_SMEP )) ||is_skylake_era ()) { +setup_force_cpu_cap (X86_FEATURE_RSB_CTXSW ); +pr_info ("Filling RSB on context switch\n" ); + } }#undef pr_fmt
PATCH 1/2 x86/cpu/intel: Introduce macros for Intel family numbers
From: Dave Hansen <[email protected]>
commit 970442c599b22ccd644ebfe94d1d303bf6f87c05 upstream.
Problem:
We have a boatload of open-coded family-6 model numbers. Half of them have these model numbers in hex and the other half in decimal. This makes grepping for them tons of fun, if you were to try.
Solution:
Consolidate all the magic numbers. Put all the definitions in one header.
The names here are closely derived from the comments describing the models from arch/x86/events/intel/core.c. We could easily make them shorter by doing things like s/SANDYBRIDGE/SNB/, but they seemed fine even with the longer versions to me.
Do not take any of these names too literally, like "DESKTOP" or "MOBILE". These are all colloquial names and not precise descriptions of everywhere a given model will show up.
--- arch/x86/include/asm/intel-family.h | 68 +++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 arch/x86/include/asm/intel-family.h
arch/x86/include/asm/intel-family.h new file +68 -0
@@ -0,0 +1,68 @@ +#ifndef _ASM_X86_INTEL_FAMILY_H +#define _ASM_X86_INTEL_FAMILY_H + +/* +* "Big Core" Processors (Branded as Core, Xeon, etc...) +* +* The "_X" parts are generally the EP and EX Xeons, or the +* "Extreme" ones, like Broadwell-E. +* +* Things ending in "2" are usually because we have no better +* name for them. There's no processor called "WESTMERE2". +*/ + +#define INTEL_FAM6_CORE_YONAH 0x0E +#define INTEL_FAM6_CORE2_MEROM 0x0F +#define INTEL_FAM6_CORE2_MEROM_L 0x16 +#define INTEL_FAM6_CORE2_PENRYN 0x17 +#define INTEL_FAM6_CORE2_DUNNINGTON 0x1D + +#define INTEL_FAM6_NEHALEM 0x1E +#define INTEL_FAM6_NEHALEM_EP 0x1A +#define INTEL_FAM6_NEHALEM_EX 0x2E +#define INTEL_FAM6_WESTMERE 0x25 +#define INTEL_FAM6_WESTMERE2 0x1F +#define INTEL_FAM6_WESTMERE_EP 0x2C +#define INTEL_FAM6_WESTMERE_EX 0x2F + +#define INTEL_FAM6_SANDYBRIDGE 0x2A +#define INTEL_FAM6_SANDYBRIDGE_X 0x2D +#define INTEL_FAM6_IVYBRIDGE 0x3A +#define INTEL_FAM6_IVYBRIDGE_X 0x3E + +#define INTEL_FAM6_HASWELL_CORE 0x3C +#define INTEL_FAM6_HASWELL_X 0x3F +#define INTEL_FAM6_HASWELL_ULT 0x45 +#define INTEL_FAM6_HASWELL_GT3E 0x46 + +#define INTEL_FAM6_BROADWELL_CORE 0x3D +#define INTEL_FAM6_BROADWELL_XEON_D 0x56 +#define INTEL_FAM6_BROADWELL_GT3E 0x47 +#define INTEL_FAM6_BROADWELL_X 0x4F + +#define INTEL_FAM6_SKYLAKE_MOBILE 0x4E +#define INTEL_FAM6_SKYLAKE_DESKTOP 0x5E +#define INTEL_FAM6_SKYLAKE_X 0x55 +#define INTEL_FAM6_KABYLAKE_MOBILE 0x8E +#define INTEL_FAM6_KABYLAKE_DESKTOP 0x9E + +/* "Small Core" Processors (Atom) */ + +#define INTEL_FAM6_ATOM_PINEVIEW 0x1C +#define INTEL_FAM6_ATOM_LINCROFT 0x26 +#define INTEL_FAM6_ATOM_PENWELL 0x27 +#define INTEL_FAM6_ATOM_CLOVERVIEW 0x35 +#define INTEL_FAM6_ATOM_CEDARVIEW 0x36 +#define INTEL_FAM6_ATOM_SILVERMONT1 0x37/* BayTrail/BYT / Valleyview */ +#define INTEL_FAM6_ATOM_SILVERMONT2 0x4D/* Avaton/Rangely */ +#define INTEL_FAM6_ATOM_AIRMONT 0x4C/* CherryTrail / Braswell */ +#define INTEL_FAM6_ATOM_MERRIFIELD1 0x4A/* Tangier */ +#define INTEL_FAM6_ATOM_MERRIFIELD2 0x5A/* Annidale */ +#define INTEL_FAM6_ATOM_GOLDMONT 0x5C +#define INTEL_FAM6_ATOM_DENVERTON 0x5F/* Goldmont Microserver */ + +/* Xeon Phi */ + +#define INTEL_FAM6_XEON_PHI_KNL 0x57/* Knights Landing */ + +#endif /* _ASM_X86_INTEL_FAMILY_H */
Signature · 1 line
2.16.0
PATCH 2/2 x86/retpoline: Fill RSB on context switch for affected CPUs
From: David Woodhouse <[email protected]>
commit c995efd5a740d9cbafbf58bde4973e8b50b4d761 upstream.
On context switch from a shallow call stack to a deeper one, as the CPU does 'ret' up the deeper side it may encounter RSB entries (predictions for where the 'ret' goes to) which were populated in userspace.
This is problematic if neither SMEP nor KPTI (the latter of which marks userspace pages as NX for the kernel) are active, as malicious code in userspace may then be executed speculatively.
Overwrite the CPU's return prediction stack with calls which are predicted to return to an infinite loop, to "capture" speculation if this happens. This is required both for retpoline, and also in conjunction with IBRS for !SMEP && !KPTI.
On Skylake+ the problem is slightly different, and an *underflow* of the RSB may cause errant branch predictions to occur. So there it's not so much overwrite, as *filling* the RSB to attempt to prevent it getting empty. This is only a partial solution for Skylake+ since there are many other conditions which may result in the RSB becoming empty. The full solution on Skylake+ is to use IBRS, which will prevent the problem even when the RSB becomes empty. With IBRS, the RSB-stuffing will not be required on context switch.
[ tglx: Added missing vendor check and slighty massaged comments and
changelog ]
[js] backport to 4.4 -- __switch_to_asm does not exist there, we
have to patch the switch_to macros for both x86_32 and x86_64.
--- arch/x86/include/asm/cpufeature.h | 1 + arch/x86/include/asm/switch_to.h | 38 ++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/cpu/bugs.c | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 75 insertions(+)
arch/x86/include/asm/cpufeature.h +1 -0
@@ -199,6 +199,7 @@#define X86_FEATURE_HWP_EPP ( 7*32+13)/* Intel HWP_EPP */ #define X86_FEATURE_HWP_PKG_REQ ( 7*32+14)/* Intel HWP_PKG_REQ */ #define X86_FEATURE_INTEL_PT ( 7*32+15)/* Intel Processor Trace */ +#define X86_FEATURE_RSB_CTXSW ( 7*32+19)/* Fill RSB on context switches */ #define X86_FEATURE_RETPOLINE ( 7*32+29)/* Generic Retpoline mitigation for Spectre variant 2 */ #define X86_FEATURE_RETPOLINE_AMD ( 7*32+30)/* AMD Retpoline mitigation for Spectre variant 2 */
arch/x86/include/asm/switch_to.h +38 -0
@@ -1,6 +1,8 @@#ifndef _ASM_X86_SWITCH_TO_H#define _ASM_X86_SWITCH_TO_H +#include <asm/nospec-branch.h> +struct task_struct ;/* one of the stranger aspects of C forward declarations */ __visible struct task_struct *__switch_to(struct task_struct *prev,struct task_struct *next);
@@ -24,6 +26,23 @@ void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p,#define __switch_canary_iparam#endif /* CC_STACKPROTECTOR */ +#ifdef CONFIG_RETPOLINE +/* +* When switching from a shallower to a deeper call stack +* the RSB may either underflow or use entries populated +* with userspace addresses. On CPUs where those concerns +* exist, overwrite the RSB with entries which capture +* speculative execution to prevent attack. +*/ +#define __retpoline_fill_return_buffer \ + ALTERNATIVE("jmp 910f", \ + __stringify(__FILL_RETURN_BUFFER(%%ebx, RSB_CLEAR_LOOPS, %%esp)),\ + X86_FEATURE_RSB_CTXSW) \ + "910:\n\t" +#else +#define __retpoline_fill_return_buffer +#endif +/* * Saving eflags is important. It switches not only IOPL between tasks, * it also protects other tasks from NT leaking through sysenter etc.
@@ -46,6 +65,7 @@ do { \"movl $1f,%[prev_ip]\n\t" /* save EIP */ \"pushl %[next_ip]\n\t" /* restore EIP */ \ __switch_canary \ + __retpoline_fill_return_buffer \"jmp __switch_to\n" /* regparm call */ \"1:\t" \"popl %%ebp\n\t" /* restore EBP */ \
@@ -100,6 +120,23 @@ do { \#define __switch_canary_iparam#endif /* CC_STACKPROTECTOR */ +#ifdef CONFIG_RETPOLINE +/* +* When switching from a shallower to a deeper call stack +* the RSB may either underflow or use entries populated +* with userspace addresses. On CPUs where those concerns +* exist, overwrite the RSB with entries which capture +* speculative execution to prevent attack. +*/ +#define __retpoline_fill_return_buffer \ + ALTERNATIVE("jmp 910f", \ + __stringify(__FILL_RETURN_BUFFER(%%r12, RSB_CLEAR_LOOPS, %%rsp)),\ + X86_FEATURE_RSB_CTXSW) \ + "910:\n\t" +#else +#define __retpoline_fill_return_buffer +#endif +/* * There is no need to save or restore flags, because flags are always * clean in kernel mode, with the possible exception of IOPL. Kernel IOPL
@@ -112,6 +149,7 @@ do { \"call __switch_to\n\t" \ "movq "__percpu_arg([current_task])",%%rsi\n\t" \ __switch_canary \ + __retpoline_fill_return_buffer \"movq %P[thread_info](%%rsi),%%r8\n\t" \"movq %%rax,%%rdi\n\t" \"testl %[_tif_fork],%P[ti_flags](%%r8)\n\t" \
arch/x86/kernel/cpu/bugs.c +36 -0
@@ -22,6 +22,7 @@#include <asm/alternative.h> #include <asm/pgtable.h> #include <asm/cacheflush.h> +#include <asm/intel-family.h> static void __init spectre_v2_select_mitigation (void );
@@ -154,6 +155,23 @@ disable:return SPECTRE_V2_CMD_NONE ; } +/* Check for Skylake-like CPUs (for RSB handling) */ +static bool __init is_skylake_era (void ) +{ +if (boot_cpu_data.x86_vendor ==X86_VENDOR_INTEL && + boot_cpu_data.x86 ==6 ) { +switch (boot_cpu_data.x86_model) { +case INTEL_FAM6_SKYLAKE_MOBILE : +case INTEL_FAM6_SKYLAKE_DESKTOP : +case INTEL_FAM6_SKYLAKE_X : +case INTEL_FAM6_KABYLAKE_MOBILE : +case INTEL_FAM6_KABYLAKE_DESKTOP : +return true; + } + } +return false; +} +static void __init spectre_v2_select_mitigation (void ) {enum spectre_v2_mitigation_cmd cmd =spectre_v2_parse_cmdline ();
@@ -212,6 +230,24 @@ retpoline_auto: spectre_v2_enabled = mode;pr_info ("%s\n" , spectre_v2_strings[mode]); + +/* +* If neither SMEP or KPTI are available, there is a risk of +* hitting userspace addresses in the RSB after a context switch +* from a shallow call stack to a deeper one. To prevent this fill +* the entire RSB, even when using IBRS. +* +* Skylake era CPUs have a separate issue with *underflow* of the +* RSB, when they will predict 'ret' targets from the generic BTB. +* The proper mitigation for this is IBRS. If IBRS is not supported +* or deactivated in favour of retpolines the RSB fill on context +* switch is required. +*/ +if ((!boot_cpu_has (X86_FEATURE_KAISER ) && + !boot_cpu_has (X86_FEATURE_SMEP )) ||is_skylake_era ()) { +setup_force_cpu_cap (X86_FEATURE_RSB_CTXSW ); +pr_info ("Filling RSB on context switch\n" ); + } }#undef pr_fmt
Signature · 1 line
2.16.0
Greg KH
maintainerRe: [PATCH 4.4-stable 2/2] x86/retpoline: Fill RSB on context switch for affected CPUs
Quoted 26 lines · Jiri Slaby
From: David Woodhouse <[email protected]>
commit
c995efd5a740d9cbafbf58bde4973e8b50b4d761upstream.On context switch from a shallow call stack to a deeper one, as the CPU does 'ret' up the deeper side it may encounter RSB entries (predictions for where the 'ret' goes to) which were populated in userspace.
This is problematic if neither SMEP nor KPTI (the latter of which marks userspace pages as NX for the kernel) are active, as malicious code in userspace may then be executed speculatively.
Overwrite the CPU's return prediction stack with calls which are predicted to return to an infinite loop, to "capture" speculation if this happens. This is required both for retpoline, and also in conjunction with IBRS for !SMEP && !KPTI.
On Skylake+ the problem is slightly different, and an *underflow* of the RSB may cause errant branch predictions to occur. So there it's not so much overwrite, as *filling* the RSB to attempt to prevent it getting empty. This is only a partial solution for Skylake+ since there are many other conditions which may result in the RSB becoming empty. The full solution on Skylake+ is to use IBRS, which will prevent the problem even when the RSB becomes empty. With IBRS, the RSB-stuffing will not be required on context switch.
Quoted 5 lines
[ tglx: Added missing vendor check and slighty massaged comments and
changelog ][js] backport to 4.4 --
__switch_to_asmdoes not exist there, we
have to patch theswitch_tomacros for bothx86_32andx86_64.
Many thanks for the backport, both patches in this series now queued up.
greg k-h