x86: RSBA and RRSBA handling
v3 cover x86: RSBA and RRSBA handling
This series deals with the hanlding of the RSBA and RRSBA bits across all parts and all mistakes encountered in various microcode versions.
There are only minor changes from v2. See patches for details.
Andrew Cooper (4):
x86/spec-ctrl: Use a taint for CET without MSR_SPEC_CTRL x86/spec-ctrl: Rename retpoline_safe() to retpoline_calculations() x86/spec-ctrl: Fix up the RSBA/RRSBA bits as appropriate x86/cpu-policy: Derive RSBA/RRSBA for guest policies
xen/arch/x86/cpu-policy.c | 39 ++++++ xen/arch/x86/include/asm/cpufeature.h | 1 + xen/arch/x86/spec_ctrl.c | 142 +++++++++++++++++--- xen/common/kernel.c | 2 +- xen/include/public/arch-x86/cpufeatureset.h | 4 +- xen/tools/gen-cpuid.py | 5 +- 6 files changed, 170 insertions(+), 23 deletions(-)
Signature · 1 line
2.30.2
PATCH v3 1/4 x86/spec-ctrl: Use a taint for CET without MSR_SPEC_CTRL
Reword the comment for 'S' to include an incompatbile set of features on the same core.
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>
v3:
* New
--- xen/arch/x86/spec_ctrl.c | 3 +++ xen/common/kernel.c | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-)
xen/arch/x86/spec_ctrl.c +3 -0
@@ -1132,7 +1132,10 @@ void __init init_speculation_mitigations(void)if (read_cr4 () &X86_CR4_CET ) {if ( !has_spec_ctrl ) + {printk (XENLOG_WARNING "?!? CET active, but no MSR_SPEC_CTRL?\n" ); +add_taint (TAINT_CPU_OUT_OF_SPEC ); + }else if ( opt_ibrs ==-1 ) opt_ibrs = ibrs = true;
xen/common/kernel.c +1 -1
@@ -373,7 +373,7 @@ unsigned int tainted;* 'H' - HVM forced emulation prefix is permitted. * 'I' - Platform is insecure (usually due to an errata on the platform). * 'M' - Machine had a machine check experience. -* 'S' - Out of spec CPU (One core has a feature incompatible with others). +* 'S' - Out of spec CPU (Incompatible features on one or more cores). * * The string is overwritten by the next call to print_taint(). */
Signature · 1 line
2.30.2
Re: [PATCH v3 1/4] x86/spec-ctrl: Use a taint for CET without MSR_SPEC_CTRL
Quoted 4 lines · Andrew Cooper
Reword the comment for 'S' to include an incompatbile set of features on the same core.
Signed-off-by Andrew Cooper <[email protected]>
PATCH v3 2/4 x86/spec-ctrl: Rename retpoline_safe() to retpoline_calculations()
In reply to Andrew CooperThis is prep work, split out to simply the diff on the following change.
* Rename to retpoline_calculations(), and call unconditionally. It is shortly going to synthesise missing enumerations required for guest safety.
* For the model check switch statement, store the result in a variable and break rather than returning directly.
No functional change.
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>
v2:
* Extend the 'safe' variable to the entire switch statement.
--- xen/arch/x86/spec_ctrl.c | 41 +++++++++++++++++++++++++--------------- 1 file changed, 26 insertions(+), 15 deletions(-)
xen/arch/x86/spec_ctrl.c +26 -15
@@ -580,9 +580,10 @@ static bool __init check_smt_enabled(void) }/* Calculate whether Retpoline is known-safe on this CPU. */ -static bool __init retpoline_safe (void ) +static bool __init retpoline_calculations (void ) {unsigned int ucode_rev =this_cpu (cpu_sig).rev; +bool safe = false;if ( boot_cpu_data.x86_vendor & (X86_VENDOR_AMD |X86_VENDOR_HYGON ) )return true;
@@ -620,29 +621,31 @@ static bool __init retpoline_safe(void)case 0x3f :/* Haswell EX/EP */ case 0x45 :/* Haswell D */ case 0x46 :/* Haswell H */ -return true; + safe = true; +break ;/* * Broadwell processors are retpoline-safe after specific microcode * versions. */ case 0x3d :/* Broadwell */ -return ucode_rev >=0x2a ; + safe = ucode_rev >=0x2a ;break ;case 0x47 :/* Broadwell H */ -return ucode_rev >=0x1d ; + safe = ucode_rev >=0x1d ;break ;case 0x4f :/* Broadwell EP/EX */ -return ucode_rev >=0xb000021 ; + safe = ucode_rev >=0xb000021 ;break ;case 0x56 :/* Broadwell D */ switch ( boot_cpu_data.x86_mask ) { -case 2 :return ucode_rev >=0x15 ; -case 3 :return ucode_rev >=0x7000012 ; -case 4 :return ucode_rev >=0xf000011 ; -case 5 :return ucode_rev >=0xe000009 ; +case 2 : safe = ucode_rev >=0x15 ;break ; +case 3 : safe = ucode_rev >=0x7000012 ;break ; +case 4 : safe = ucode_rev >=0xf000011 ;break ; +case 5 : safe = ucode_rev >=0xe000009 ;break ;default :printk ("Unrecognised CPU stepping %#x - assuming not reptpoline safe\n" , boot_cpu_data.x86_mask); -return false; + safe = false; +break ; }break ;
@@ -656,7 +659,8 @@ static bool __init retpoline_safe(void)case 0x67 :/* Cannonlake? */ case 0x8e :/* Kabylake M */ case 0x9e :/* Kabylake D */ -return false; + safe = false; +break ;/* * Atom processors before Goldmont Plus/Gemini Lake are retpoline-safe.
@@ -675,13 +679,17 @@ static bool __init retpoline_safe(void)case 0x5c :/* Goldmont */ case 0x5f :/* Denverton */ case 0x85 :/* Knights Mill */ -return true; + safe = true; +break ;default :printk ("Unrecognised CPU model %#x - assuming not reptpoline safe\n" , boot_cpu_data.x86_model); -return false; + safe = false; +break ; } + +return safe; }/*
@@ -1114,7 +1122,7 @@ void __init init_speculation_mitigations(void) {enum ind_thunk thunk =THUNK_DEFAULT ;bool has_spec_ctrl, ibrs = false, hw_smt_enabled; -bool cpu_has_bug_taa; +bool cpu_has_bug_taa, retpoline_safe; hw_smt_enabled =check_smt_enabled ();
@@ -1143,6 +1151,9 @@ void __init init_speculation_mitigations(void) thunk =THUNK_JMP ; } +/* Determine if retpoline is safe on this CPU. */ + retpoline_safe =retpoline_calculations (); +/* * Has the user specified any custom BTI mitigations? If so, follow their * instructions exactly and disable all heuristics.
@@ -1164,7 +1175,7 @@ void __init init_speculation_mitigations(void)* On all hardware, we'd like to use retpoline in preference to * IBRS, but only if it is safe on this hardware. */ -if (retpoline_safe () ) +if ( retpoline_safe ) thunk =THUNK_RETPOLINE ;else if ( has_spec_ctrl ) ibrs = true;
Signature · 1 line
2.30.2
PATCH v3 3/4 x86/spec-ctrl: Fix up the RSBA/RRSBA bits as appropriate
In reply to Andrew CooperIn order to level a VM safely for migration, the toolstack needs to know the RSBA/RRSBA properties of the CPU, whether or not they happen to be enumerated.
See the code comment for details.
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>
v3:
* Add a taint for bad EIBRS vs RSBA/RRSBA.
* Minor comment improvements.
v2:
* Rewrite almost from scratch.
--- xen/arch/x86/include/asm/cpufeature.h | 1 + xen/arch/x86/spec_ctrl.c | 100 ++++++++++++++++++++++++-- 2 files changed, 96 insertions(+), 5 deletions(-)
xen/arch/x86/include/asm/cpufeature.h +1 -0
@@ -193,6 +193,7 @@ static inline bool boot_cpu_has(unsigned int feat)#define cpu_has_tsx_ctrl boot_cpu_has(X86_FEATURE_TSX_CTRL)#define cpu_has_taa_no boot_cpu_has(X86_FEATURE_TAA_NO)#define cpu_has_fb_clear boot_cpu_has(X86_FEATURE_FB_CLEAR) +#define cpu_has_rrsba boot_cpu_has(X86_FEATURE_RRSBA)/* Synthesized. */ #define cpu_has_arch_perfmon boot_cpu_has(X86_FEATURE_ARCH_PERFMON)
xen/arch/x86/spec_ctrl.c +95 -5
130 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [PATCH v3 3/4] x86/spec-ctrl: Fix up the RSBA/RRSBA bits as appropriate
Quoted 27 lines · Andrew Cooper
xen/arch/x86/spec_ctrl.c +21
-2@@ -593,15 +596,93 @@ static bool __init retpoline_calculations(void)return false;/* -* RSBA may be set by a hypervisor to indicate that we may move to a -* processor which isn't retpoline-safe. +* The meaning of the RSBA and RRSBA bits have evolved over time. The +* agreed upon meaning at the time of writing (May 2023) is thus: +* +* - RSBA (RSB Alternative) means that an RSB may fall back to an +* alternative predictor on underflow. Skylake uarch and later all have +* this property. Broadwell too, when running microcode versions prior +* to Jan 2018. +* +* - All eIBRS-capable processors suffer RSBA, but eIBRS also introduces +* tagging of predictions with the mode in which they were learned. So +* when eIBRS is active, RSBA becomes RRSBA (Restricted RSBA). +* +* - CPUs are not expected to enumerate both RSBA and RRSBA. +* +* Some parts (Broadwell) are not expected to ever enumerate this +* behaviour directly. Other parts have differing enumeration with +* microcode version. Fix up Xen's idea, so we can advertise them safely +* to guests, and so toolstacks can level a VM safety for migration. +* +* The following states exist: +*
xen/arch/x86/spec_ctrl.c static bool __init retpoline_calculations(void) · line 629
Quoted 10 lines
xen/arch/x86/spec_ctrl.c +10
-0+* | | RSBA | EIBRS | RRSBA | Notes | Action | +* |---+------+-------+-------+--------------------+---------------| +* | 1 | 0 | 0 | 0 | OK (older parts) | Maybe +RSBA | +* | 2 | 0 | 0 | 1 | Broken | +RSBA, -RRSBA | +* | 3 | 0 | 1 | 0 | OK (pre-Aug ucode) | +RRSBA | +* | 4 | 0 | 1 | 1 | OK | | +* | 5 | 1 | 0 | 0 | OK | | +* | 6 | 1 | 0 | 1 | Broken | -RRSBA | +* | 7 | 1 | 1 | 0 | Broken | -RSBA, +RRSBA | +* | 8 | 1 | 1 | 1 | Broken | -RSBA |
You've kept the Action column as you had it originally, despite no longer applying all the fixups. Wouldn't it make sense to mark those we don't do, e.g. by enclosing in parentheses?
xen/arch/x86/spec_ctrl.c static bool __init retpoline_calculations(void) · line 638
Quoted 8 lines
xen/arch/x86/spec_ctrl.c +7
-0+* However, we don't need perfect adherence to the spec. We only need +* RSBA || RRSBA to indicate "alternative predictors potentially in use". +* Rows 1 & 3 are fixed up by later logic, as they're known configurations +* which exist in the world. * +* Complain loudly at the broken cases. They're safe for Xen to use (so we +* don't attempt to correct), and may or may not exist in reality, but if +* we ever encoutner them in practice, something is wrong and needs
Nit: "encounter"
xen/arch/x86/spec_ctrl.c static bool __init retpoline_calculations(void) · line 648
Quoted 10 lines
xen/arch/x86/spec_ctrl.c +10
-0+* further investigation. +*/ +if ( cpu_has_eibrs ? cpu_has_rsba/* Rows 7, 8 */ + : cpu_has_rrsba/* Rows 2, 6 */ ) + { +printk (XENLOG_ERR +"FIRMWARE BUG: CPU %02x-%02x-%02x, ucode 0x%08x: RSBA %u, EIBRS %u, RRSBA %u\n" , + boot_cpu_data.x86, boot_cpu_data.x86_model, + boot_cpu_data.x86_mask, ucode_rev, + cpu_has_rsba, cpu_has_eibrs, cpu_has_rrsba);
Perhaps with adjustments (as you deem them sensible)
Jan
Re: [PATCH v3 3/4] x86/spec-ctrl: Fix up the RSBA/RRSBA bits as appropriate
Quoted 33 lines · Jan Beulich
Quoted 32 lines · Andrew Cooper
xen/arch/x86/spec_ctrl.c +26
-2@@ -593,15 +596,93 @@ static bool __init retpoline_calculations(void)return false;/* -* RSBA may be set by a hypervisor to indicate that we may move to a -* processor which isn't retpoline-safe. +* The meaning of the RSBA and RRSBA bits have evolved over time. The +* agreed upon meaning at the time of writing (May 2023) is thus: +* +* - RSBA (RSB Alternative) means that an RSB may fall back to an +* alternative predictor on underflow. Skylake uarch and later all have +* this property. Broadwell too, when running microcode versions prior +* to Jan 2018. +* +* - All eIBRS-capable processors suffer RSBA, but eIBRS also introduces +* tagging of predictions with the mode in which they were learned. So +* when eIBRS is active, RSBA becomes RRSBA (Restricted RSBA). +* +* - CPUs are not expected to enumerate both RSBA and RRSBA. +* +* Some parts (Broadwell) are not expected to ever enumerate this +* behaviour directly. Other parts have differing enumeration with +* microcode version. Fix up Xen's idea, so we can advertise them safely +* to guests, and so toolstacks can level a VM safety for migration. +* +* The following states exist: +* +* | | RSBA | EIBRS | RRSBA | Notes | Action | +* |---+------+-------+-------+--------------------+---------------| +* | 1 | 0 | 0 | 0 | OK (older parts) | Maybe +RSBA | +* | 2 | 0 | 0 | 1 | Broken | +RSBA, -RRSBA | +* | 3 | 0 | 1 | 0 | OK (pre-Aug ucode) | +RRSBA |
Quoted 8 lines
xen/arch/x86/spec_ctrl.c static bool __init retpoline_calculations(void) · line 629
Quoted 5 lines
xen/arch/x86/spec_ctrl.c +5
-0+* | 4 | 0 | 1 | 1 | OK | | +* | 5 | 1 | 0 | 0 | OK | | +* | 6 | 1 | 0 | 1 | Broken | -RRSBA | +* | 7 | 1 | 1 | 0 | Broken | -RSBA, +RRSBA | +* | 8 | 1 | 1 | 1 | Broken | -RSBA | You've kept the Action column as you had it originally, despite no longer applying all the fixups. Wouldn't it make sense to mark those we don't do, e.g. by enclosing in parentheses?
Hmm, yes. How does this look?
| | RSBA | EIBRS | RRSBA | Notes | Action (in principle) | |---+------+-------+-------+--------------------+-----------------------| | 1 | 0 | 0 | 0 | OK (older parts) | Maybe +RSBA | | 2 | 0 | 0 | 1 | Broken | (+RSBA, -RRSBA) | | 3 | 0 | 1 | 0 | OK (pre-Aug ucode) | +RRSBA | | 4 | 0 | 1 | 1 | OK | | | 5 | 1 | 0 | 0 | OK | | | 6 | 1 | 0 | 1 | Broken | (-RRSBA) | | 7 | 1 | 1 | 0 | Broken | (-RSBA, +RRSBA) | | 8 | 1 | 1 | 1 | Broken | (-RSBA) |
Quoted 10 lines
xen/arch/x86/spec_ctrl.c static bool __init retpoline_calculations(void) · line 648
Quoted 10 lines
xen/arch/x86/spec_ctrl.c +10
-0+* further investigation. +*/ +if ( cpu_has_eibrs ? cpu_has_rsba/* Rows 7, 8 */ + : cpu_has_rrsba/* Rows 2, 6 */ ) + { +printk (XENLOG_ERR +"FIRMWARE BUG: CPU %02x-%02x-%02x, ucode 0x%08x: RSBA %u, EIBRS %u, RRSBA %u\n" , + boot_cpu_data.x86, boot_cpu_data.x86_model, + boot_cpu_data.x86_mask, ucode_rev, + cpu_has_rsba, cpu_has_eibrs, cpu_has_rrsba);
Quoted 2 lines
Perhaps with adjustments (as you deem them sensible)
Reviewed-by Jan Beulich <[email protected]>
Thanks.
~Andrew
Re: [PATCH v3 3/4] x86/spec-ctrl: Fix up the RSBA/RRSBA bits as appropriate
Quoted 45 lines · Andrew Cooper
Quoted 33 lines · Jan Beulich
Quoted 32 lines · Andrew Cooper
xen/arch/x86/spec_ctrl.c +26
-2@@ -593,15 +596,93 @@ static bool __init retpoline_calculations(void)return false;/* -* RSBA may be set by a hypervisor to indicate that we may move to a -* processor which isn't retpoline-safe. +* The meaning of the RSBA and RRSBA bits have evolved over time. The +* agreed upon meaning at the time of writing (May 2023) is thus: +* +* - RSBA (RSB Alternative) means that an RSB may fall back to an +* alternative predictor on underflow. Skylake uarch and later all have +* this property. Broadwell too, when running microcode versions prior +* to Jan 2018. +* +* - All eIBRS-capable processors suffer RSBA, but eIBRS also introduces +* tagging of predictions with the mode in which they were learned. So +* when eIBRS is active, RSBA becomes RRSBA (Restricted RSBA). +* +* - CPUs are not expected to enumerate both RSBA and RRSBA. +* +* Some parts (Broadwell) are not expected to ever enumerate this +* behaviour directly. Other parts have differing enumeration with +* microcode version. Fix up Xen's idea, so we can advertise them safely +* to guests, and so toolstacks can level a VM safety for migration. +* +* The following states exist: +* +* | | RSBA | EIBRS | RRSBA | Notes | Action | +* |---+------+-------+-------+--------------------+---------------| +* | 1 | 0 | 0 | 0 | OK (older parts) | Maybe +RSBA | +* | 2 | 0 | 0 | 1 | Broken | +RSBA, -RRSBA | +* | 3 | 0 | 1 | 0 | OK (pre-Aug ucode) | +RRSBA | Quoted 8 lines
xen/arch/x86/spec_ctrl.c static bool __init retpoline_calculations(void) · line 629
Quoted 5 lines
xen/arch/x86/spec_ctrl.c +5
-0+* | 4 | 0 | 1 | 1 | OK | | +* | 5 | 1 | 0 | 0 | OK | | +* | 6 | 1 | 0 | 1 | Broken | -RRSBA | +* | 7 | 1 | 1 | 0 | Broken | -RSBA, +RRSBA | +* | 8 | 1 | 1 | 1 | Broken | -RSBA | You've kept the Action column as you had it originally, despite no longer applying all the fixups. Wouldn't it make sense to mark those we don't do, e.g. by enclosing in parentheses?
Hmm, yes. How does this look?
Quoted 10 lines
| | RSBA | EIBRS | RRSBA | Notes | Action (in principle) | |---+------+-------+-------+--------------------+-----------------------| | 1 | 0 | 0 | 0 | OK (older parts) | Maybe +RSBA | | 2 | 0 | 0 | 1 | Broken | (+RSBA, -RRSBA) | | 3 | 0 | 1 | 0 | OK (pre-Aug ucode) | +RRSBA | | 4 | 0 | 1 | 1 | OK | | | 5 | 1 | 0 | 0 | OK | | | 6 | 1 | 0 | 1 | Broken | (-RRSBA) | | 7 | 1 | 1 | 0 | Broken | (-RSBA, +RRSBA) | | 8 | 1 | 1 | 1 | Broken | (-RSBA) |
Yes, I think it's better to have it this way, thanks.
Jan
PATCH v3 4/4 x86/cpu-policy: Derive RSBA/RRSBA for guest policies
In reply to Andrew CooperThe RSBA bit, "RSB Alternative", means that the RSB may use alternative predictors when empty. From a practical point of view, this mean "Retpoline not safe".
Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously IBRS_ATT) is a statement that IBRS is implemented in hardware (as opposed to the form retrofitted to existing CPUs in microcode).
The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS property that predictions are tagged with the mode in which they were learnt. Therefore, it means "when eIBRS is active, the RSB may fall back to alternative predictors but restricted to the current prediction mode". As such, it's stronger statement than RSBA, but still means "Retpoline not safe".
CPUs are not expected to enumerate both RSBA and RRSBA.
Add feature dependencies for EIBRS and RRSBA. While technically they're not linked, absolutely nothing good can come of letting the guest see RRSBA without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use this dependency to simplify the max derivation logic.
The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS dependency maybe hiding RRSBA). We can run any VM, even if it has been told "somewhere you might run, Retpoline isn't safe".
The default policies are more complicated. A guest shouldn't see both bits, but it needs to see one if the current host suffers from any form of RSBA, and which bit it needs to see depends on whether eIBRS is visible or not. Therefore, the calculation must be performed after sanitise_featureset().
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>
v3:
* Minor commit message adjustment.
* Drop changes to recalculate_cpuid_policy(). Deferred to a later series.
v2:
* Expand/adjust the comment for the max features.
* Rewrite the default feature derivation in light of new information.
* Fix up in recalculate_cpuid_policy() too.
--- xen/arch/x86/cpu-policy.c | 39 +++++++++++++++++++++ xen/include/public/arch-x86/cpufeatureset.h | 4 +-- xen/tools/gen-cpuid.py | 5 ++- 3 files changed, 45 insertions(+), 3 deletions(-)
xen/arch/x86/cpu-policy.c +39 -0
@@ -423,8 +423,17 @@ static void __init guest_common_max_feature_adjustments(uint32_t *fs)* Retpoline not safe)", so these need to be visible to a guest in all * cases, even when it's only some other server in the pool which * suffers the identified behaviour. +* +* We can always run any VM which has previously (or will +* subsequently) run on hardware where Retpoline is not safe. +* Note: +* - The dependency logic may hide RRSBA for other reasons. +* - The max policy does not contitute a sensible configuration to +* run a guest in. */ __set_bit (X86_FEATURE_ARCH_CAPS , fs); +__set_bit (X86_FEATURE_RSBA , fs); +__set_bit (X86_FEATURE_RRSBA , fs); } }
@@ -532,6 +541,21 @@ static void __init calculate_pv_def_policy(void)guest_common_default_feature_adjustments (fs);sanitise_featureset (fs); + +/* +* If the host suffers from RSBA of any form, and the guest can see +* MSR_ARCH_CAPS, reflect the appropriate RSBA/RRSBA property to the guest +* depending on the visibility of eIBRS. +*/ +if (test_bit (X86_FEATURE_ARCH_CAPS , fs) && + (cpu_has_rsba || cpu_has_rrsba) ) + { +bool eibrs =test_bit (X86_FEATURE_EIBRS , fs); + +__set_bit (eibrs ?X86_FEATURE_RRSBA + :X86_FEATURE_RSBA , fs); + } +x86_cpu_featureset_to_policy (fs, p);recalculate_xstate (p); }
@@ -664,6 +688,21 @@ static void __init calculate_hvm_def_policy(void)__set_bit (X86_FEATURE_VIRT_SSBD , fs);sanitise_featureset (fs); + +/* +* If the host suffers from RSBA of any form, and the guest can see +* MSR_ARCH_CAPS, reflect the appropriate RSBA/RRSBA property to the guest +* depending on the visibility of eIBRS. +*/ +if (test_bit (X86_FEATURE_ARCH_CAPS , fs) && + (cpu_has_rsba || cpu_has_rrsba) ) + { +bool eibrs =test_bit (X86_FEATURE_EIBRS , fs); + +__set_bit (eibrs ?X86_FEATURE_RRSBA + :X86_FEATURE_RSBA , fs); + } +x86_cpu_featureset_to_policy (fs, p);recalculate_xstate (p); }
xen/include/public/arch-x86/cpufeatureset.h +2 -2
@@ -311,7 +311,7 @@ XEN_CPUFEATURE(CET_SSS, 15*32+18) /* CET Supervisor Shadow Stacks s/* Intel-defined CPU features, MSR_ARCH_CAPS 0x10a.eax, word 16 */ XEN_CPUFEATURE(RDCL_NO, 16*32+ 0) /*A No Rogue Data Cache Load (Meltdown) */ XEN_CPUFEATURE(EIBRS, 16*32+ 1) /*A Enhanced IBRS */ -XEN_CPUFEATURE(RSBA, 16*32+ 2) /*!A RSB Alternative (Retpoline not safe) */ +XEN_CPUFEATURE(RSBA, 16*32+ 2) /*! RSB Alternative (Retpoline not safe) */ XEN_CPUFEATURE(SKIP_L1DFL, 16*32+ 3) /* Don't need to flush L1D on VMEntry */ XEN_CPUFEATURE(INTEL_SSB_NO, 16*32+ 4) /*A No Speculative Store Bypass */XEN_CPUFEATURE (MDS_NO ,16 *32 +5 )/*A No Microarchitectural Data Sampling */
@@ -327,7 +327,7 @@ XEN_CPUFEATURE(FBSDP_NO, 16*32+14) /*A No Fill Buffer Stale Data Prop XEN_CPUFEATURE(PSDP_NO, 16*32+15) /*A No Primary Stale Data Propagation */ XEN_CPUFEATURE(FB_CLEAR, 16*32+17) /*A Fill Buffers cleared by VERW */ XEN_CPUFEATURE(FB_CLEAR_CTRL, 16*32+18) /* MSR_OPT_CPU_CTRL.FB_CLEAR_DIS */ -XEN_CPUFEATURE(RRSBA, 16*32+19) /*!A Restricted RSB Alternative */ +XEN_CPUFEATURE(RRSBA, 16*32+19) /*! Restricted RSB Alternative */ XEN_CPUFEATURE(BHI_NO, 16*32+20) /*A No Branch History Injection */ XEN_CPUFEATURE(XAPIC_STATUS, 16*32+21) /* MSR_XAPIC_DISABLE_STATUS */XEN_CPUFEATURE (OVRCLK_STATUS ,16 *32 +23 )/* MSR_OVERCLOCKING_STATUS */
xen/tools/gen-cpuid.py +4 -1
@@ -318,7 +318,7 @@ def crunch_numbers(state): # IBRSB/IBRS, and we pass this MSR directly to guests. Treating them # as dependent features simplifies Xen's logic, and prevents the guest # from seeing implausible configurations. - IBRSB: [STIBP, SSBD, INTEL_PSFD], + IBRSB: [STIBP, SSBD, INTEL_PSFD, EIBRS], IBRS: [AMD_STIBP, AMD_SSBD, PSFD, AUTO_IBRS, IBRS_ALWAYS, IBRS_FAST, IBRS_SAME_MODE], AMD_STIBP: [STIBP_ALWAYS],
@@ -328,6 +328,9 @@ def crunch_numbers(state): # The ARCH_CAPS CPUID bit enumerates the availability of the whole register. ARCH_CAPS: list(range(RDCL_NO, RDCL_NO + 64)), + + # The behaviour described by RRSBA depend on eIBRS being active. + EIBRS: [RRSBA], } deep_features = tuple(sorted(deps.keys()))
Signature · 1 line
2.30.2
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
Quoted 31 lines · Andrew Cooper
The RSBA bit, "RSB Alternative", means that the RSB may use alternative predictors when empty. From a practical point of view, this mean "Retpoline not safe".
Enhanced IBRS (officially
IBRS_ALLin Intel's docs, previouslyIBRS_ATT) is a statement that IBRS is implemented in hardware (as opposed to the form retrofitted to existing CPUs in microcode).The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS property that predictions are tagged with the mode in which they were learnt. Therefore, it means "when eIBRS is active, the RSB may fall back to alternative predictors but restricted to the current prediction mode". As such, it's stronger statement than RSBA, but still means "Retpoline not safe".
CPUs are not expected to enumerate both RSBA and RRSBA.
Add feature dependencies for EIBRS and RRSBA. While technically they're not linked, absolutely nothing good can come of letting the guest see RRSBA without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use this dependency to simplify the max derivation logic.
The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS dependency maybe hiding RRSBA). We can run any VM, even if it has been told "somewhere you might run, Retpoline isn't safe".
The default policies are more complicated. A guest shouldn't see both bits, but it needs to see one if the current host suffers from any form of RSBA, and which bit it needs to see depends on whether eIBRS is visible or not. Therefore, the calculation must be performed after
sanitise_featureset().Signed-off-by Andrew Cooper <[email protected]>
Quoted 8 lines
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>v3:
* Minor commit message adjustment.
* Drop changes torecalculate_cpuid_policy(). Deferred to a later series.
With this dropped, with the title not saying "max/default", and with the description also not mentioning "live" policies at all, I don't think this patch is self-consistent (meaning in particular: leaving aside the fact that there's no way right now to requests e.g. both RSBA and RRSBA for a guest; aiui it is possible for Dom0).
As you may imagine I'm also curious why you decided to drop this.
Jan
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
Quoted 40 lines · Jan Beulich
Quoted 31 lines · Andrew Cooper
The RSBA bit, "RSB Alternative", means that the RSB may use alternative predictors when empty. From a practical point of view, this mean "Retpoline not safe".
Enhanced IBRS (officially
IBRS_ALLin Intel's docs, previouslyIBRS_ATT) is a statement that IBRS is implemented in hardware (as opposed to the form retrofitted to existing CPUs in microcode).The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS property that predictions are tagged with the mode in which they were learnt. Therefore, it means "when eIBRS is active, the RSB may fall back to alternative predictors but restricted to the current prediction mode". As such, it's stronger statement than RSBA, but still means "Retpoline not safe".
CPUs are not expected to enumerate both RSBA and RRSBA.
Add feature dependencies for EIBRS and RRSBA. While technically they're not linked, absolutely nothing good can come of letting the guest see RRSBA without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use this dependency to simplify the max derivation logic.
The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS dependency maybe hiding RRSBA). We can run any VM, even if it has been told "somewhere you might run, Retpoline isn't safe".
The default policies are more complicated. A guest shouldn't see both bits, but it needs to see one if the current host suffers from any form of RSBA, and which bit it needs to see depends on whether eIBRS is visible or not. Therefore, the calculation must be performed after
sanitise_featureset().Signed-off-by Andrew Cooper <[email protected]>Quoted 8 lines
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>v3:
* Minor commit message adjustment.
* Drop changes torecalculate_cpuid_policy(). Deferred to a later series.
Quoted 7 lines
With this dropped, with the title not saying "max/default", and with the description also not mentioning "live" policies at all, I don't think this patch is self-consistent (meaning in particular: leaving aside the fact that there's no way right now to requests e.g. both RSBA and RRSBA for a guest; aiui it is possible for Dom0).
As you may imagine I'm also curious why you decided to drop this.
Because when I tried doing levelling in Xapi, I remembered why I did it the way I did in v1, and why the v2 way was wrong.
Xen cannot safely edit what the toolstack provides, so must not. Instead, failing the set_policy() call is an option, and is what we want to do longterm, but also happens to be wrong too in this case. An admin may know that a VM isn't using retpoline, and may need to migrate it anyway for a number of reasons, so any safety checks need to be in the toolstack, and need to be overrideable with something like --force.
I don't really associate "derive policies" with anything other than the system policies. Domain construction isn't any kind of derivation - it's simply doing what the toolstack asks.
~Andrew
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
Quoted 47 lines · Andrew Cooper
Quoted 46 lines · Jan Beulich
Quoted 31 lines · Andrew Cooper
The RSBA bit, "RSB Alternative", means that the RSB may use alternative predictors when empty. From a practical point of view, this mean "Retpoline not safe".
Enhanced IBRS (officially
IBRS_ALLin Intel's docs, previouslyIBRS_ATT) is a statement that IBRS is implemented in hardware (as opposed to the form retrofitted to existing CPUs in microcode).The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS property that predictions are tagged with the mode in which they were learnt. Therefore, it means "when eIBRS is active, the RSB may fall back to alternative predictors but restricted to the current prediction mode". As such, it's stronger statement than RSBA, but still means "Retpoline not safe".
CPUs are not expected to enumerate both RSBA and RRSBA.
Add feature dependencies for EIBRS and RRSBA. While technically they're not linked, absolutely nothing good can come of letting the guest see RRSBA without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use this dependency to simplify the max derivation logic.
The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS dependency maybe hiding RRSBA). We can run any VM, even if it has been told "somewhere you might run, Retpoline isn't safe".
The default policies are more complicated. A guest shouldn't see both bits, but it needs to see one if the current host suffers from any form of RSBA, and which bit it needs to see depends on whether eIBRS is visible or not. Therefore, the calculation must be performed after
sanitise_featureset().Signed-off-by Andrew Cooper <[email protected]>Quoted 8 lines
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>v3:
* Minor commit message adjustment.
* Drop changes torecalculate_cpuid_policy(). Deferred to a later series.With this dropped, with the title not saying "max/default", and with the description also not mentioning "live" policies at all, I don't think this patch is self-consistent (meaning in particular: leaving aside the fact that there's no way right now to requests e.g. both RSBA and RRSBA for a guest; aiui it is possible for Dom0).
Quoted 6 lines
Quoted 1 line
As you may imagine I'm also curious why you decided to drop this.
Because when I tried doing levelling in Xapi, I remembered why I did it the way I did in v1, and why the v2 way was wrong.
Xen cannot safely edit what the toolstack provides, so must not.
And this is the part I don't understand: Why can't we correct the (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least as long as ...
Quoted 2 lines
Instead, failing the
set_policy()call is an option, and is what we want to do longterm,
... we aren't there.
Quoted 4 lines
but also happens to be wrong too in this case. An admin
may know that a VM isn't using retpoline, and may need to migrate it anyway for a number of reasons, so any safety checks need to be in the toolstack, and need to be overrideable with something like --force.
Possibly leading to an inconsistent policy exposed to a guest? I guess this may be the only option when we can't really resolve an ambiguity, but that isn't the case here, is it?
Quoted 3 lines
I don't really associate "derive policies" with anything other than the system policies. Domain construction isn't any kind of derivation - it's simply doing what the toolstack asks.
Hmm, I see. To me, since we do certain adjustments, "derive" still fits there as well. But I'm not going to insist on a subject adjustment then, given that imo both ways of looking at things make some sense.
Jan
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
Quoted 61 lines · Jan Beulich
Quoted 47 lines · Andrew Cooper
Quoted 46 lines · Jan Beulich
Quoted 31 lines · Andrew Cooper
The RSBA bit, "RSB Alternative", means that the RSB may use alternative predictors when empty. From a practical point of view, this mean "Retpoline not safe".
Enhanced IBRS (officially
IBRS_ALLin Intel's docs, previouslyIBRS_ATT) is a statement that IBRS is implemented in hardware (as opposed to the form retrofitted to existing CPUs in microcode).The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS property that predictions are tagged with the mode in which they were learnt. Therefore, it means "when eIBRS is active, the RSB may fall back to alternative predictors but restricted to the current prediction mode". As such, it's stronger statement than RSBA, but still means "Retpoline not safe".
CPUs are not expected to enumerate both RSBA and RRSBA.
Add feature dependencies for EIBRS and RRSBA. While technically they're not linked, absolutely nothing good can come of letting the guest see RRSBA without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use this dependency to simplify the max derivation logic.
The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS dependency maybe hiding RRSBA). We can run any VM, even if it has been told "somewhere you might run, Retpoline isn't safe".
The default policies are more complicated. A guest shouldn't see both bits, but it needs to see one if the current host suffers from any form of RSBA, and which bit it needs to see depends on whether eIBRS is visible or not. Therefore, the calculation must be performed after
sanitise_featureset().Signed-off-by Andrew Cooper <[email protected]>Quoted 8 lines
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>v3:
* Minor commit message adjustment.
* Drop changes torecalculate_cpuid_policy(). Deferred to a later series.With this dropped, with the title not saying "max/default", and with the description also not mentioning "live" policies at all, I don't think this patch is self-consistent (meaning in particular: leaving aside the fact that there's no way right now to requests e.g. both RSBA and RRSBA for a guest; aiui it is possible for Dom0).
Quoted 5 lines
Quoted 1 line
As you may imagine I'm also curious why you decided to drop this.
Because when I tried doing levelling in Xapi, I remembered why I did it the way I did in v1, and why the v2 way was wrong.
Xen cannot safely edit what the toolstack provides, so must not.
And this is the part I don't understand: Why can't we correct the (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least as long as ...
Quoted 2 lines
Instead, failing the
set_policy()call is an option, and is what we want to do longterm,... we aren't there.
Quoted 7 lines
Quoted 4 lines
but also happens to be wrong too in this case. An admin
may know that a VM isn't using retpoline, and may need to migrate it anyway for a number of reasons, so any safety checks need to be in the toolstack, and need to be overrideable with something like --force.Possibly leading to an inconsistent policy exposed to a guest? I guess this may be the only option when we can't really resolve an ambiguity, but that isn't the case here, is it?
Wrong. Xen does not have any knowledge of other hosts the VM might migrate to.
So while Xen can spot problem combinations *on this host*, which way to correct the problem combination depends on where the VM might migrate to.
Xen cannot safely correct a problem combination even if you don't wish to allow the admin the ability to override the safety check.
Quoted 8 lines
Quoted 4 lines
I don't really associate "derive policies" with anything other than the system policies. Domain construction isn't any kind of derivation - it's simply doing what the toolstack asks.
Hmm, I see. To me, since we do certain adjustments, "derive" still fits there as well. But I'm not going to insist on a subject adjustment then, given that imo both ways of looking at things make some sense.
It's a problem that Xen ever made adjustments behind the toolstack's back, and this decade of technical debt has been extremely difficult to address. I guess I still view it in terms of the end properties, not the intermediate mess.
~Andrew
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
Quoted 70 lines · Andrew Cooper
Quoted 61 lines · Jan Beulich
Quoted 47 lines · Andrew Cooper
Quoted 46 lines · Jan Beulich
Quoted 31 lines · Andrew Cooper
The RSBA bit, "RSB Alternative", means that the RSB may use alternative predictors when empty. From a practical point of view, this mean "Retpoline not safe".
Enhanced IBRS (officially
IBRS_ALLin Intel's docs, previouslyIBRS_ATT) is a statement that IBRS is implemented in hardware (as opposed to the form retrofitted to existing CPUs in microcode).The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS property that predictions are tagged with the mode in which they were learnt. Therefore, it means "when eIBRS is active, the RSB may fall back to alternative predictors but restricted to the current prediction mode". As such, it's stronger statement than RSBA, but still means "Retpoline not safe".
CPUs are not expected to enumerate both RSBA and RRSBA.
Add feature dependencies for EIBRS and RRSBA. While technically they're not linked, absolutely nothing good can come of letting the guest see RRSBA without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use this dependency to simplify the max derivation logic.
The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS dependency maybe hiding RRSBA). We can run any VM, even if it has been told "somewhere you might run, Retpoline isn't safe".
The default policies are more complicated. A guest shouldn't see both bits, but it needs to see one if the current host suffers from any form of RSBA, and which bit it needs to see depends on whether eIBRS is visible or not. Therefore, the calculation must be performed after
sanitise_featureset().Signed-off-by Andrew Cooper <[email protected]>Quoted 8 lines
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>v3:
* Minor commit message adjustment.
* Drop changes torecalculate_cpuid_policy(). Deferred to a later series.With this dropped, with the title not saying "max/default", and with the description also not mentioning "live" policies at all, I don't think this patch is self-consistent (meaning in particular: leaving aside the fact that there's no way right now to requests e.g. both RSBA and RRSBA for a guest; aiui it is possible for Dom0).
Quoted 5 lines
Quoted 1 line
As you may imagine I'm also curious why you decided to drop this.
Because when I tried doing levelling in Xapi, I remembered why I did it the way I did in v1, and why the v2 way was wrong.
Xen cannot safely edit what the toolstack provides, so must not.
And this is the part I don't understand: Why can't we correct the (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least as long as ...
Quoted 2 lines
Instead, failing the
set_policy()call is an option, and is what we want to do longterm,... we aren't there.
Quoted 7 lines
Quoted 4 lines
but also happens to be wrong too in this case. An admin
may know that a VM isn't using retpoline, and may need to migrate it anyway for a number of reasons, so any safety checks need to be in the toolstack, and need to be overrideable with something like --force.Possibly leading to an inconsistent policy exposed to a guest? I guess this may be the only option when we can't really resolve an ambiguity, but that isn't the case here, is it?
Quoted 5 lines
Wrong. Xen does not have any knowledge of other hosts the VM might migrate to.
So while Xen can spot problem combinations *on this host*, which way to correct the problem combination depends on where the VM might migrate to.
I actually view this as two different levels: With a flawed policy, the guest is liable to not work correctly at all. No point thinking about it being able to migrate. With a fixed up policy it may fail to migrate, but it'll at least work otherwise.
Jan
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
Quoted 73 lines · Jan Beulich
Quoted 72 lines · Andrew Cooper
Quoted 61 lines · Jan Beulich
Quoted 47 lines · Andrew Cooper
Quoted 46 lines · Jan Beulich
Quoted 31 lines · Andrew Cooper
The RSBA bit, "RSB Alternative", means that the RSB may use alternative predictors when empty. From a practical point of view, this mean "Retpoline not safe".
Enhanced IBRS (officially
IBRS_ALLin Intel's docs, previouslyIBRS_ATT) is a statement that IBRS is implemented in hardware (as opposed to the form retrofitted to existing CPUs in microcode).The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS property that predictions are tagged with the mode in which they were learnt. Therefore, it means "when eIBRS is active, the RSB may fall back to alternative predictors but restricted to the current prediction mode". As such, it's stronger statement than RSBA, but still means "Retpoline not safe".
CPUs are not expected to enumerate both RSBA and RRSBA.
Add feature dependencies for EIBRS and RRSBA. While technically they're not linked, absolutely nothing good can come of letting the guest see RRSBA without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use this dependency to simplify the max derivation logic.
The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS dependency maybe hiding RRSBA). We can run any VM, even if it has been told "somewhere you might run, Retpoline isn't safe".
The default policies are more complicated. A guest shouldn't see both bits, but it needs to see one if the current host suffers from any form of RSBA, and which bit it needs to see depends on whether eIBRS is visible or not. Therefore, the calculation must be performed after
sanitise_featureset().Signed-off-by Andrew Cooper <[email protected]>Quoted 8 lines
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>v3:
* Minor commit message adjustment.
* Drop changes torecalculate_cpuid_policy(). Deferred to a later series.With this dropped, with the title not saying "max/default", and with the description also not mentioning "live" policies at all, I don't think this patch is self-consistent (meaning in particular: leaving aside the fact that there's no way right now to requests e.g. both RSBA and RRSBA for a guest; aiui it is possible for Dom0).
Quoted 5 lines
Quoted 1 line
As you may imagine I'm also curious why you decided to drop this.
Because when I tried doing levelling in Xapi, I remembered why I did it the way I did in v1, and why the v2 way was wrong.
Xen cannot safely edit what the toolstack provides, so must not.
And this is the part I don't understand: Why can't we correct the (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least as long as ...
Quoted 2 lines
Instead, failing the
set_policy()call is an option, and is what we want to do longterm,... we aren't there.
Quoted 7 lines
Quoted 4 lines
but also happens to be wrong too in this case. An admin
may know that a VM isn't using retpoline, and may need to migrate it anyway for a number of reasons, so any safety checks need to be in the toolstack, and need to be overrideable with something like --force.Possibly leading to an inconsistent policy exposed to a guest? I guess this may be the only option when we can't really resolve an ambiguity, but that isn't the case here, is it?
Wrong. Xen does not have any knowledge of other hosts the VM might migrate to.
Quoted 6 lines
Quoted 2 lines
So while Xen can spot problem combinations *on this host*, which way to correct the problem combination depends on where the VM might migrate to.
I actually view this as two different levels: With a flawed policy, the guest is liable to not work correctly at all. No point thinking about it being able to migrate. With a fixed up policy it may fail to migrate, but it'll at least work otherwise.
If you get RSBA and/or RRSBA wrong, nothing is going to malfunction in the guest, even if you migrate it.
The consequence of getting RSBA and/or RRSBA wrong is the guest *might* think retpoline is safe to use, and *might* end up vulnerable to speculative attacks on this or other hardware.
And the admin might know that they overrode the default settings and forced the use of some other protection mechanism, so the guest is in fact safe despite having wrong RSBA/RRSBA settings.
I don't know how to put it any more plainly. Xen *does not* have the information necessary to make a safety judgement in this matter. Only the toolstack (as a proxy for the admin) has the necessary information.
~Andrew
Jan Beulich
Acked-byRe: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
Quoted 81 lines · Andrew Cooper
Quoted 73 lines · Jan Beulich
Quoted 72 lines · Andrew Cooper
Quoted 61 lines · Jan Beulich
Quoted 47 lines · Andrew Cooper
Quoted 46 lines · Jan Beulich
Quoted 31 lines · Andrew Cooper
The RSBA bit, "RSB Alternative", means that the RSB may use alternative predictors when empty. From a practical point of view, this mean "Retpoline not safe".
Enhanced IBRS (officially
IBRS_ALLin Intel's docs, previouslyIBRS_ATT) is a statement that IBRS is implemented in hardware (as opposed to the form retrofitted to existing CPUs in microcode).The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS property that predictions are tagged with the mode in which they were learnt. Therefore, it means "when eIBRS is active, the RSB may fall back to alternative predictors but restricted to the current prediction mode". As such, it's stronger statement than RSBA, but still means "Retpoline not safe".
CPUs are not expected to enumerate both RSBA and RRSBA.
Add feature dependencies for EIBRS and RRSBA. While technically they're not linked, absolutely nothing good can come of letting the guest see RRSBA without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use this dependency to simplify the max derivation logic.
The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS dependency maybe hiding RRSBA). We can run any VM, even if it has been told "somewhere you might run, Retpoline isn't safe".
The default policies are more complicated. A guest shouldn't see both bits, but it needs to see one if the current host suffers from any form of RSBA, and which bit it needs to see depends on whether eIBRS is visible or not. Therefore, the calculation must be performed after
sanitise_featureset().Signed-off-by Andrew Cooper <[email protected]>Quoted 8 lines
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>v3:
* Minor commit message adjustment.
* Drop changes torecalculate_cpuid_policy(). Deferred to a later series.With this dropped, with the title not saying "max/default", and with the description also not mentioning "live" policies at all, I don't think this patch is self-consistent (meaning in particular: leaving aside the fact that there's no way right now to requests e.g. both RSBA and RRSBA for a guest; aiui it is possible for Dom0).
Quoted 5 lines
Quoted 1 line
As you may imagine I'm also curious why you decided to drop this.
Because when I tried doing levelling in Xapi, I remembered why I did it the way I did in v1, and why the v2 way was wrong.
Xen cannot safely edit what the toolstack provides, so must not.
And this is the part I don't understand: Why can't we correct the (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least as long as ...
Quoted 2 lines
Instead, failing the
set_policy()call is an option, and is what we want to do longterm,... we aren't there.
Quoted 7 lines
Quoted 4 lines
but also happens to be wrong too in this case. An admin
may know that a VM isn't using retpoline, and may need to migrate it anyway for a number of reasons, so any safety checks need to be in the toolstack, and need to be overrideable with something like --force.Possibly leading to an inconsistent policy exposed to a guest? I guess this may be the only option when we can't really resolve an ambiguity, but that isn't the case here, is it?
Wrong. Xen does not have any knowledge of other hosts the VM might migrate to.
Quoted 6 lines
Quoted 2 lines
So while Xen can spot problem combinations *on this host*, which way to correct the problem combination depends on where the VM might migrate to.
I actually view this as two different levels: With a flawed policy, the guest is liable to not work correctly at all. No point thinking about it being able to migrate. With a fixed up policy it may fail to migrate, but it'll at least work otherwise.
Quoted 6 lines
If you get RSBA and/or RRSBA wrong, nothing is going to malfunction in the guest, even if you migrate it.
The consequence of getting RSBA and/or RRSBA wrong is the guest *might* think retpoline is safe to use, and *might* end up vulnerable to speculative attacks on this or other hardware.
Isn't that some sort of "malfunction"?
Quoted 3 lines
And the admin might know that they overrode the default settings and forced the use of some other protection mechanism, so the guest is in fact safe despite having wrong RSBA/RRSBA settings.
But then the guest would also be safe with adjusted settings, wouldn't it?
Quoted 3 lines
I don't know how to put it any more plainly. Xen *does not* have the information necessary to make a safety judgement in this matter. Only the toolstack (as a proxy for the admin) has the necessary information.
I'm not looking at it as Xen making things safe by adjusting bogus settings. I'm merely looking at it as not letting a guest run that way. For the safety aspect I agree it needs a wider view than Xen has.
Anyway, I don't think either of us is going to convince the other of there being only one way of looking at things vs there being at least two possible ways, so in order to allow things to progress
Jan
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
Quoted 81 lines · Jan Beulich
Quoted 80 lines · Andrew Cooper
Quoted 79 lines · Jan Beulich
Quoted 66 lines · Andrew Cooper
Quoted 65 lines · Jan Beulich
Quoted 47 lines · Andrew Cooper
Quoted 46 lines · Jan Beulich
Quoted 31 lines · Andrew Cooper
The RSBA bit, "RSB Alternative", means that the RSB may use alternative predictors when empty. From a practical point of view, this mean "Retpoline not safe".
Enhanced IBRS (officially
IBRS_ALLin Intel's docs, previouslyIBRS_ATT) is a statement that IBRS is implemented in hardware (as opposed to the form retrofitted to existing CPUs in microcode).The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS property that predictions are tagged with the mode in which they were learnt. Therefore, it means "when eIBRS is active, the RSB may fall back to alternative predictors but restricted to the current prediction mode". As such, it's stronger statement than RSBA, but still means "Retpoline not safe".
CPUs are not expected to enumerate both RSBA and RRSBA.
Add feature dependencies for EIBRS and RRSBA. While technically they're not linked, absolutely nothing good can come of letting the guest see RRSBA without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use this dependency to simplify the max derivation logic.
The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS dependency maybe hiding RRSBA). We can run any VM, even if it has been told "somewhere you might run, Retpoline isn't safe".
The default policies are more complicated. A guest shouldn't see both bits, but it needs to see one if the current host suffers from any form of RSBA, and which bit it needs to see depends on whether eIBRS is visible or not. Therefore, the calculation must be performed after
sanitise_featureset().Signed-off-by Andrew Cooper <[email protected]>Quoted 8 lines
---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>v3:
* Minor commit message adjustment.
* Drop changes torecalculate_cpuid_policy(). Deferred to a later series.With this dropped, with the title not saying "max/default", and with the description also not mentioning "live" policies at all, I don't think this patch is self-consistent (meaning in particular: leaving aside the fact that there's no way right now to requests e.g. both RSBA and RRSBA for a guest; aiui it is possible for Dom0).
Quoted 5 lines
Quoted 1 line
As you may imagine I'm also curious why you decided to drop this.
Because when I tried doing levelling in Xapi, I remembered why I did it the way I did in v1, and why the v2 way was wrong.
Xen cannot safely edit what the toolstack provides, so must not.
And this is the part I don't understand: Why can't we correct the (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least as long as ...
Quoted 2 lines
Instead, failing the
set_policy()call is an option, and is what we want to do longterm,... we aren't there.
Quoted 4 lines
but also happens to be wrong too in this case. An admin
may know that a VM isn't using retpoline, and may need to migrate it anyway for a number of reasons, so any safety checks need to be in the toolstack, and need to be overrideable with something like --force.Quoted 8 lines
Quoted 3 lines
Possibly leading to an inconsistent policy exposed to a guest? I guess this may be the only option when we can't really resolve an ambiguity, but that isn't the case here, is it?
Wrong. Xen does not have any knowledge of other hosts the VM might migrate to.
So while Xen can spot problem combinations *on this host*, which way to correct the problem combination depends on where the VM might migrate to.
I actually view this as two different levels: With a flawed policy, the guest is liable to not work correctly at all. No point thinking about it being able to migrate. With a fixed up policy it may fail to migrate, but it'll at least work otherwise.
Quoted 7 lines
Quoted 6 lines
If you get RSBA and/or RRSBA wrong, nothing is going to malfunction in the guest, even if you migrate it.
The consequence of getting RSBA and/or RRSBA wrong is the guest *might* think retpoline is safe to use, and *might* end up vulnerable to speculative attacks on this or other hardware.
Isn't that some sort of "malfunction"?
Perhaps, there's a difference between "it will likely crash hard" and "you won't notice the difference".
Quoted 5 lines
Quoted 4 lines
And the admin might know that they overrode the default settings and forced the use of some other protection mechanism, so the guest is in fact safe despite having wrong RSBA/RRSBA settings.
But then the guest would also be safe with adjusted settings, wouldn't it?
It doesn't mean the guest is going to tolerate having features change underfoot.
Quoted 4 lines
Quoted 4 lines
I don't know how to put it any more plainly. Xen *does not* have the information necessary to make a safety judgement in this matter. Only the toolstack (as a proxy for the admin) has the necessary information.
Quoted 8 lines
I'm not looking at it as Xen making things safe by adjusting bogus settings. I'm merely looking at it as not letting a guest run that way. For the safety aspect I agree it needs a wider view than Xen has.
Anyway, I don't think either of us is going to convince the other of there being only one way of looking at things vs there being at least two possible ways, so in order to allow things to progress
Acked-by Jan Beulich <[email protected]>
Thankyou.
To be clear, we are planning to put checks in place. We definitely don't want the admin to end up in this corner case accidentally.
~Andrew