x86/speculation: Allow enabling STIBP with legacy IBRS
Subsystems (4)
PATCH v2 1/2 x86/speculation: Allow enabling STIBP with legacy IBRS
Setting the IBRS bit implicitly enables STIBP to protect against cross-thread branch target injection. With enhanced IBRS, the bit it set once and is not cleared again. However, on CPUs with just legacy IBRS, IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a KERNEL_IBRS). Clearing this bit also disables the implicitly enabled STIBP, thus requiring some form of cross-thread protection in userspace.
Enable STIBP, either opt-in via prctl or seccomp, or always on depending on the choice of mitigation selected via spectre_v2_user.
--- arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-)
arch/x86/kernel/cpu/bugs.c +22 -11
@@ -1124,14 +1124,30 @@ spectre_v2_parse_user_cmdline(void)return SPECTRE_V2_USER_CMD_AUTO ; } -static inline bool spectre_v2_in_ibrs_mode (enum spectre_v2_mitigation mode) +static inline bool spectre_v2_in_eibrs_mode (enum spectre_v2_mitigation mode) { -return mode ==SPECTRE_V2_IBRS || - mode ==SPECTRE_V2_EIBRS || +return mode ==SPECTRE_V2_EIBRS || mode ==SPECTRE_V2_EIBRS_RETPOLINE || mode ==SPECTRE_V2_EIBRS_LFENCE ; } +static inline bool spectre_v2_in_ibrs_mode (enum spectre_v2_mitigation mode) +{ +return spectre_v2_in_eibrs_mode (mode) || mode ==SPECTRE_V2_IBRS ; +} + +static inline bool spectre_v2_user_needs_stibp (enum spectre_v2_mitigation mode) +{ +/* +* enhanced IBRS also protects against user-mode attacks as the IBRS bit +* remains always set which implicitly enables cross-thread protections. +* However, In legacy IBRS mode, the IBRS bit is set only in kernel +* and cleared on return to userspace. This disables the implicit +* cross-thread protections and STIBP is needed. +*/ +return !spectre_v2_in_eibrs_mode (mode); +} +static void __init spectre_v2_user_select_mitigation (void ) {
@@ -1193,13 +1209,8 @@ spectre_v2_user_select_mitigation(void) "always-on" : "conditional"); } -/* -* If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible, -* STIBP is not required. -*/ -if (!boot_cpu_has (X86_FEATURE_STIBP ) || - !smt_possible || -spectre_v2_in_ibrs_mode (spectre_v2_enabled)) +if (!boot_cpu_has (X86_FEATURE_STIBP ) || !smt_possible || + !spectre_v2_user_needs_stibp (spectre_v2_enabled))return ;/*
@@ -2327,7 +2338,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)static char *stibp_state (void ) { -if (spectre_v2_in_ibrs_mode (spectre_v2_enabled)) +if (!spectre_v2_user_needs_stibp (spectre_v2_enabled))return "" ;switch (spectre_v2_user_stibp) {
Signature · 1 line
2.39.2.637.g21b0678d19-goog
PATCH v2 2/2 Documentation/hw-vuln: Document the interaction between IBRS and STIBP
Explain why STIBP is needed with legacy IBRS as currently implemented (KERNEL_IBRS) and why STIBP is not needed when enhanced IBRS is enabled.
--- Documentation/admin-guide/hw-vuln/spectre.rst | 22 ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
Documentation/admin-guide/hw-vuln/spectre.rst +17 -5
@@ -479,8 +479,17 @@ Spectre variant 2 On Intel Skylake-era systems the mitigation covers most, but not all, cases. See :ref:`[3] <spec_ref3>` for more details. - On CPUs with hardware mitigation for Spectre variant 2 (e.g. Enhanced - IBRS on x86), retpoline is automatically disabled at run time. + On CPUs with hardware mitigation for Spectre variant 2 (e.g. IBRS + or enhanced IBRS on x86), retpoline is automatically disabled at run time. + + Setting the IBRS bit implicitly enables STIBP which guards against + cross-thread branch target injection on SMT systems. On systems with enhanced + IBRS, the kernel sets the bit once, which keeps cross-thread protections + always enabled, obviating the need for an explicit STIBP. On CPUs with legacy + IBRS, the kernel clears the IBRS bit on returning to user-space, thus also + disabling the implicit STIBP. Consequently, STIBP needs to be explicitly + enabled to guard against cross-thread attacks in userspace. + The retpoline mitigation is turned on by default on vulnerable CPUs. It can be forced on or off by the administrator
@@ -504,9 +513,12 @@ Spectre variant 2 For Spectre variant 2 mitigation, individual user programs can be compiled with return trampolines for indirect branches. This protects them from consuming poisoned entries in the branch - target buffer left by malicious software. Alternatively, the - programs can disable their indirect branch speculation via prctl() - (See :ref:`Documentation/userspace-api/spec_ctrl.rst <set_spec_ctrl>`). + target buffer left by malicious software. + + On legacy IBRS systems, at return to userspace, implicit STIBP is disabled + because the kernel clears the IBRS bit. In this case, the userspace programs + can disable indirect branch speculation via prctl() (See + :ref:`Documentation/userspace-api/spec_ctrl.rst <set_spec_ctrl>`). On x86, this will turn on STIBP to guard against attacks from the sibling thread when the user program is running, and use IBPB to flush the branch target buffer when switching to/from the program.
Signature · 1 line
2.39.2.637.g21b0678d19-goog
Borislav Petkov
maintainerRe: [PATCH v2 2/2] Documentation/hw-vuln: Document the interaction between IBRS and STIBP
Quoted 2 lines · KP Singh
... Consequently, STIBP needs to be explicitly
+ enabled to guard against cross-thread attacks in userspace.
needs?
That sounds like something the user needs to do. But we do it by default. Let's rephrase:
"Systems which support enhanced IBRS (eIBRS) enable IBRS protections once at boot and they're automatically protected against Spectre v2 variant attacks, including cross-thread branch target injections on SMT systems (STIBP). IOW, eIBRS enables STIBP too.
Legacy IBRS systems clear the IBRS bit on exit to userspace and therefore explicitly enable STIBP for that."
Simple.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH v2 2/2] Documentation/hw-vuln: Document the interaction between IBRS and STIBP
Quoted 10 lines · Borislav Petkov
Quoted 2 lines · KP Singh
... Consequently, STIBP needs to be explicitly
+ enabled to guard against cross-thread attacks in userspace.needs?
That sounds like something the user needs to do. But we do it by default. Let's rephrase:
Quoted 7 lines
"Systems which support enhanced IBRS (eIBRS) enable IBRS protections once at boot and they're automatically protected against Spectre v2 variant attacks, including cross-thread branch target injections on SMT systems (STIBP). IOW, eIBRS enables STIBP too.
Legacy IBRS systems clear the IBRS bit on exit to userspace and therefore explicitly enable STIBP for that."
ack, I will respin both patches with your suggestions.
- KP
Quoted 8 lines
Simple.
--
Regards/Gruss,
Boris.
Re: [PATCH v2 2/2] Documentation/hw-vuln: Document the interaction between IBRS and STIBP
In reply to Borislav PetkovQuoted 10 lines · Borislav Petkov
Quoted 2 lines · KP Singh
... Consequently, STIBP needs to be explicitly
+ enabled to guard against cross-thread attacks in userspace.needs?
That sounds like something the user needs to do. But we do it by default. Let's rephrase:
Quoted 7 lines
"Systems which support enhanced IBRS (eIBRS) enable IBRS protections once at boot and they're automatically protected against Spectre v2 variant attacks, including cross-thread branch target injections on SMT systems (STIBP). IOW, eIBRS enables STIBP too.
Legacy IBRS systems clear the IBRS bit on exit to userspace and therefore explicitly enable STIBP for that."
+ Systems which support enhanced IBRS (eIBRS) enable IBRS protection once at + boot, by setting the IBRS bit, and they're automatically protected against + Spectre v2 variant attacks, including cross-thread branch target injections + on SMT systems (STIBP). In other words, eIBRS enables STIBP too.
+
+ Legacy IBRS systems clear the IBRS bit on exit to userspace and
+ therefore explicitly enable STIBP for that
I did add one phrase, we really need to stress on the IBRS bit here. Had we been enabling KERNEL_IBRS accidentally with eIBRS, it would still mess things up as the bit being set is important.
This is why my original patch felt "obtuse" as it focused on KERNEL_IBRS instead of IBRS or eIBRS :).
Quoted 8 lines
Simple.
--
Regards/Gruss,
Boris.
Greg KH
maintainerQuoted 16 lines · KP Singh
Setting the IBRS bit implicitly enables STIBP to protect against cross-thread branch target injection. With enhanced IBRS, the bit it set once and is not cleared again. However, on CPUs with just legacy IBRS, IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
KERNEL_IBRS). Clearing this bit also disables the implicitly enabled STIBP, thus requiring some form of cross-thread protection in userspace.Enable STIBP, either opt-in via prctl or seccomp, or always on depending on the choice of mitigation selected via
spectre_v2_user.Reported-by José Oliveira <[email protected]>Reported-by Rodrigo Branco <[email protected]>Reviewed-by Alexandra Sandulescu <[email protected]>Fixes 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")Signed-off-by KP Singh <[email protected]>
Quoted 4 lines
--- arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-)
Hi,
This is the friendly patch-bot of Greg Kroah-Hartman. You have sent him a patch that has triggered this response. He used to manually respond to these common problems, but in order to save his sanity (he kept writing the same thing over and over, yet to different people), I was created. Hopefully you will not take offence and will fix the problem in your patch and resubmit it so that it can be accepted into the Linux kernel tree.
You are receiving this message because of the following common error(s) as indicated below:
- This looks like a new version of a previously submitted patch, but you did not list below the --- line any changes from the previous version. Please read the section entitled "The canonical patch format" in the kernel file, Documentation/process/submitting-patches.rst for what needs to be done here to properly describe this.
If you wish to discuss this problem further, or you have questions about how to resolve this issue, please feel free to respond to this email and Greg will reply once he has dug out from the pending patches received from other developers.
thanks,
greg k-h's patch email bot
Greg KH
maintainerQuoted 16 lines · KP Singh
Setting the IBRS bit implicitly enables STIBP to protect against cross-thread branch target injection. With enhanced IBRS, the bit it set once and is not cleared again. However, on CPUs with just legacy IBRS, IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
KERNEL_IBRS). Clearing this bit also disables the implicitly enabled STIBP, thus requiring some form of cross-thread protection in userspace.Enable STIBP, either opt-in via prctl or seccomp, or always on depending on the choice of mitigation selected via
spectre_v2_user.Reported-by José Oliveira <[email protected]>Reported-by Rodrigo Branco <[email protected]>Reviewed-by Alexandra Sandulescu <[email protected]>Fixes 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")Signed-off-by KP Singh <[email protected]>
Quoted 3 lines
--- arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-)
Why isn't patch 2/2 for stable as well?
thanks,
greg k-h
Quoted 22 lines · Greg KH
Quoted 16 lines · KP Singh
Setting the IBRS bit implicitly enables STIBP to protect against cross-thread branch target injection. With enhanced IBRS, the bit it set once and is not cleared again. However, on CPUs with just legacy IBRS, IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
KERNEL_IBRS). Clearing this bit also disables the implicitly enabled STIBP, thus requiring some form of cross-thread protection in userspace.Enable STIBP, either opt-in via prctl or seccomp, or always on depending on the choice of mitigation selected via
spectre_v2_user.Reported-by José Oliveira <[email protected]>Reported-by Rodrigo Branco <[email protected]>Reviewed-by Alexandra Sandulescu <[email protected]>Fixes 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")Signed-off-by KP Singh <[email protected]>Quoted 3 lines
--- arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-)
Quoted 1 line
Why isn't patch 2/2 for stable as well?
It should be. I actually forgot to remove stable from the first one as there are still ongoing discussions and people kept having to "drop stable". I can send a v3 with stable Cc'ed. Should it have a fixes tag too?
Quoted 4 lines
thanks,
greg k-h
Greg KH
maintainerQuoted 23 lines · KP Singh
Quoted 22 lines · Greg KH
Quoted 19 lines · KP Singh
Setting the IBRS bit implicitly enables STIBP to protect against cross-thread branch target injection. With enhanced IBRS, the bit it set once and is not cleared again. However, on CPUs with just legacy IBRS, IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
KERNEL_IBRS). Clearing this bit also disables the implicitly enabled STIBP, thus requiring some form of cross-thread protection in userspace.Enable STIBP, either opt-in via prctl or seccomp, or always on depending on the choice of mitigation selected via
spectre_v2_user.Reported-by José Oliveira <[email protected]>Reported-by Rodrigo Branco <[email protected]>Reviewed-by Alexandra Sandulescu <[email protected]>Fixes 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")Signed-off-by KP Singh <[email protected]>--- arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-)
Quoted 6 lines
Quoted 1 line
Why isn't patch 2/2 for stable as well?
It should be. I actually forgot to remove stable from the first one as there are still ongoing discussions and people kept having to "drop stable". I can send a v3 with stable Cc'ed. Should it have a fixes tag too?
Why does anyone need to "drop stable" from a patch discussion? That's not a problem, we _WANT_ to see the patch review and discussion also copied there to be aware of what is coming down the pipeline. So whomever said that is not correct, sorry.
And yes, a fixes: tag would be nice.
thanks,
greg k-h
Borislav Petkov
maintainerQuoted 4 lines · Greg KH
Why does anyone need to "drop stable" from a patch discussion? That's not a problem, we _WANT_ to see the patch review and discussion also copied there to be aware of what is coming down the pipeline. So whomever said that is not correct, sorry.
Someone dropped stable because you used to send automated formletter mails that this is not how one should submit a patch to stable. I guess that is not needed anymore so I'll stop dropping stable.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteGreg KH
maintainerQuoted 9 lines · Borislav Petkov
Quoted 4 lines · Greg KH
Why does anyone need to "drop stable" from a patch discussion? That's not a problem, we _WANT_ to see the patch review and discussion also copied there to be aware of what is coming down the pipeline. So whomever said that is not correct, sorry.
Someone dropped stable because you used to send automated formletter mails that this is not how one should submit a patch to stable. I guess that is not needed anymore so I'll stop dropping stable.
I still send them, and so does 0-day, IF you send the emails incorrectly. So far, that's not been the case for this series at all (hint, there needs to be a cc: stable in the signed-off-by area of the patch, that's all.)
thanks,
greg k-h
Borislav Petkov
maintainerQuoted 4 lines · Greg KH
I still send them, and so does 0-day, IF you send the emails incorrectly. So far, that's not been the case for this series at all (hint, there needs to be a cc: stable in the signed-off-by area of the patch, that's all.)
Aha, you send it when there's no Cc: stable in the SOB area but stable is still CCed.
Bah, forget what I said.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquettePawan Gupta
reviewerQuoted 37 lines · KP Singh
Setting the IBRS bit implicitly enables STIBP to protect against cross-thread branch target injection. With enhanced IBRS, the bit it set once and is not cleared again. However, on CPUs with just legacy IBRS, IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
KERNEL_IBRS). Clearing this bit also disables the implicitly enabled STIBP, thus requiring some form of cross-thread protection in userspace.Enable STIBP, either opt-in via prctl or seccomp, or always on depending on the choice of mitigation selected via
spectre_v2_user.Reported-by José Oliveira <[email protected]>Reported-by Rodrigo Branco <[email protected]>Reviewed-by Alexandra Sandulescu <[email protected]>Fixes 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")Signed-off-by KP Singh <[email protected]>--- arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-)arch/x86/kernel/cpu/bugs.c +2
-3@@ -1124,14 +1124,30 @@ spectre_v2_parse_user_cmdline(void)return SPECTRE_V2_USER_CMD_AUTO ; } -static inline bool spectre_v2_in_ibrs_mode (enum spectre_v2_mitigation mode) +static inline bool spectre_v2_in_eibrs_mode (enum spectre_v2_mitigation mode) { -return mode ==SPECTRE_V2_IBRS || - mode ==SPECTRE_V2_EIBRS || +return mode ==SPECTRE_V2_EIBRS || mode ==SPECTRE_V2_EIBRS_RETPOLINE || mode ==SPECTRE_V2_EIBRS_LFENCE ; }
arch/x86/kernel/cpu/bugs.c spectre_v2_parse_user_cmdline(void) · line 1142
Quoted 10 lines
arch/x86/kernel/cpu/bugs.c +9
-0+static inline bool spectre_v2_in_ibrs_mode (enum spectre_v2_mitigation mode) +{ +return spectre_v2_in_eibrs_mode (mode) || mode ==SPECTRE_V2_IBRS ; +} + +static inline bool spectre_v2_user_needs_stibp (enum spectre_v2_mitigation mode) +{ +/* +* enhanced IBRS also protects against user-mode attacks as the IBRS bit
Maybe:
* Enhanced IBRS mode also protects against cross-thread user-to-user
* attacks as the IBRS bit
Quoted 14 lines
arch/x86/kernel/cpu/bugs.c +8
-0+* remains always set which implicitly enables cross-thread protections. +* However, In legacy IBRS mode, the IBRS bit is set only in kernel +* and cleared on return to userspace. This disables the implicit +* cross-thread protections and STIBP is needed. +*/ +return !spectre_v2_in_eibrs_mode (mode); +} +static void __init spectre_v2_user_select_mitigation (void ) {@@ -1193,13 +1209,8 @@ spectre_v2_user_select_mitigation(void) "always-on" : "conditional"); }
arch/x86/kernel/cpu/bugs.c spectre_v2_user_select_mitigation(void) · line 1213
Quoted 10 lines
arch/x86/kernel/cpu/bugs.c +2
-7-/* -* If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible, -* STIBP is not required. -*/ -if (!boot_cpu_has (X86_FEATURE_STIBP ) || - !smt_possible || -spectre_v2_in_ibrs_mode (spectre_v2_enabled)) +if (!boot_cpu_has (X86_FEATURE_STIBP ) || !smt_possible || + !spectre_v2_user_needs_stibp (spectre_v2_enabled))
As pointed out in other discussions, it will be great if can get rid of eIBRS check, and do what the user asked for; or atleast print a warning about not setting STIBP bit explicitly.
arch/x86/kernel/cpu/bugs.c static ssize_t mmio_stale_data_show_state(char *buf) · line 2341
Quoted 9 lines
arch/x86/kernel/cpu/bugs.c +1
-1return ;/* @@ -2327,7 +2338,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)static char *stibp_state (void ) { -if (spectre_v2_in_ibrs_mode (spectre_v2_enabled)) +if (!spectre_v2_user_needs_stibp (spectre_v2_enabled))
Decoupling STIBP and eIBRS will also get rid of this check.
Quoted 1 line
arch/x86/kernel/cpu/bugs.c +0
-0return "" ;
Quoted 40 lines · Pawan Gupta
Quoted 38 lines · KP Singh
Setting the IBRS bit implicitly enables STIBP to protect against cross-thread branch target injection. With enhanced IBRS, the bit it set once and is not cleared again. However, on CPUs with just legacy IBRS, IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
KERNEL_IBRS). Clearing this bit also disables the implicitly enabled STIBP, thus requiring some form of cross-thread protection in userspace.Enable STIBP, either opt-in via prctl or seccomp, or always on depending on the choice of mitigation selected via
spectre_v2_user.Reported-by José Oliveira <[email protected]>Reported-by Rodrigo Branco <[email protected]>Reviewed-by Alexandra Sandulescu <[email protected]>Fixes 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")Signed-off-by KP Singh <[email protected]>--- arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-)arch/x86/kernel/cpu/bugs.c +2
-3@@ -1124,14 +1124,30 @@ spectre_v2_parse_user_cmdline(void)return SPECTRE_V2_USER_CMD_AUTO ; } -static inline bool spectre_v2_in_ibrs_mode (enum spectre_v2_mitigation mode) +static inline bool spectre_v2_in_eibrs_mode (enum spectre_v2_mitigation mode) { -return mode ==SPECTRE_V2_IBRS || - mode ==SPECTRE_V2_EIBRS || +return mode ==SPECTRE_V2_EIBRS || mode ==SPECTRE_V2_EIBRS_RETPOLINE || mode ==SPECTRE_V2_EIBRS_LFENCE ; }
Quoted 13 lines
arch/x86/kernel/cpu/bugs.c spectre_v2_parse_user_cmdline(void) · line 1142
Quoted 9 lines
arch/x86/kernel/cpu/bugs.c +9
-0+static inline bool spectre_v2_in_ibrs_mode (enum spectre_v2_mitigation mode) +{ +return spectre_v2_in_eibrs_mode (mode) || mode ==SPECTRE_V2_IBRS ; +} + +static inline bool spectre_v2_user_needs_stibp (enum spectre_v2_mitigation mode) +{ +/* +* enhanced IBRS also protects against user-mode attacks as the IBRS bit Maybe:
* Enhanced IBRS mode also protects against cross-thread user-to-user
* attacks as the IBRS bit
updated, thanks!
Quoted 16 lines
Quoted 16 lines
arch/x86/kernel/cpu/bugs.c +8
-0+* remains always set which implicitly enables cross-thread protections. +* However, In legacy IBRS mode, the IBRS bit is set only in kernel +* and cleared on return to userspace. This disables the implicit +* cross-thread protections and STIBP is needed. +*/ +return !spectre_v2_in_eibrs_mode (mode); +} +static void __init spectre_v2_user_select_mitigation (void ) {@@ -1193,13 +1209,8 @@ spectre_v2_user_select_mitigation(void) "always-on" : "conditional"); }
Quoted 12 lines
arch/x86/kernel/cpu/bugs.c spectre_v2_user_select_mitigation(void) · line 1213
Quoted 9 lines
arch/x86/kernel/cpu/bugs.c +2
-7-/* -* If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible, -* STIBP is not required. -*/ -if (!boot_cpu_has (X86_FEATURE_STIBP ) || - !smt_possible || -spectre_v2_in_ibrs_mode (spectre_v2_enabled)) +if (!boot_cpu_has (X86_FEATURE_STIBP ) || !smt_possible || + !spectre_v2_user_needs_stibp (spectre_v2_enabled))As pointed out in other discussions, it will be great if can get rid of eIBRS check, and do what the user asked for; or atleast print a warning
I think I will keep it as pr_info as, with eIBRS, the user does not really need STIBP and the mitigation is still effective.
Quoted 1 line
about not setting STIBP bit explicitly.
That is a bit more complicated as, for now, the user is not really exposed to STIBP explicitly yet.
{ "auto", SPECTRE_V2_USER_CMD_AUTO, false },
{ "off", SPECTRE_V2_USER_CMD_NONE, false },
{ "on", SPECTRE_V2_USER_CMD_FORCE, true },
{ "prctl", SPECTRE_V2_USER_CMD_PRCTL, false },
{ "prctl,ibpb", SPECTRE_V2_USER_CMD_PRCTL_IBPB, false }, { "seccomp", SPECTRE_V2_USER_CMD_SECCOMP, false }, { "seccomp,ibpb", SPECTRE_V2_USER_CMD_SECCOMP_IBPB, false },
I would prefer to do it as a follow up and fix this bug first.
It's a bit gnarly and I think we really need to think about the options that are exposed to the user [especially in light of Intel / AMD subtelties].
With the current patch the userspace is still getting working V2 mitigations on both dimensions time (Process A followed by Process B where A does BTI on the subsequent B that are flushed via an IBPB) and space (i.e. cross-thread branch target injection) whenever necessary.
Quoted 14 lines
arch/x86/kernel/cpu/bugs.c static ssize_t mmio_stale_data_show_state(char *buf) · line 2341
Quoted 10 lines
arch/x86/kernel/cpu/bugs.c +1
-1return ;/* @@ -2327,7 +2338,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)static char *stibp_state (void ) { -if (spectre_v2_in_ibrs_mode (spectre_v2_enabled)) +if (!spectre_v2_user_needs_stibp (spectre_v2_enabled))Decoupling STIBP and eIBRS will also get rid of this check.
Quoted 1 line
arch/x86/kernel/cpu/bugs.c +0
-0return "" ;
Pawan Gupta
reviewerQuoted 17 lines · KP Singh
Quoted 4 lines
Quoted 4 lines
arch/x86/kernel/cpu/bugs.c +0
-0@@ -1193,13 +1209,8 @@ spectre_v2_user_select_mitigation(void) "always-on" : "conditional"); }Quoted 12 lines
arch/x86/kernel/cpu/bugs.c spectre_v2_user_select_mitigation(void) · line 1213
Quoted 9 lines
arch/x86/kernel/cpu/bugs.c +2
-7-/* -* If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible, -* STIBP is not required. -*/ -if (!boot_cpu_has (X86_FEATURE_STIBP ) || - !smt_possible || -spectre_v2_in_ibrs_mode (spectre_v2_enabled)) +if (!boot_cpu_has (X86_FEATURE_STIBP ) || !smt_possible || + !spectre_v2_user_needs_stibp (spectre_v2_enabled))As pointed out in other discussions, it will be great if can get rid of eIBRS check, and do what the user asked for; or atleast print a warning
Quoted 2 lines
I think I will keep it as
pr_infoas, with eIBRS, the user does not really need STIBP and the mitigation is still effective.
Thats fair.
Quoted 6 lines
Quoted 1 line
about not setting STIBP bit explicitly.
That is a bit more complicated as, for now, the user is not really exposed to STIBP explicitly yet.
I would prefer to do it as a follow up and fix this bug first.
On a second thought, STIBP bit being explicitly set or not shouldn't matter as long as user is getting the STIBP protection that it asked for.
A print may just help catch some bugs sooner than later.
Quoted 3 lines
It's a bit gnarly and I think we really need to think about the options that are exposed to the user [especially in light of Intel / AMD subtelties].
With AMD's AutoIBRS support landing in mainline, and both (AutoIBRS and eIBRS) sharing the same =eibrs mitigation mode, those subtelties becomes more important.
Following up on Andrew's comment in the other thread, I hope AutoIBRS does not require setting STIBP explicitly?:
/sigh so we're still talking about 3 different things then.
1) Intel's legacy IBRS
2) AMD's regular IBRS
3) AMD's AutoIBRS
which all have different relevant behaviours for userspace. Just so it's written out coherently in at least one place... [...]
For any AMD configuration setting STIBP, there must be an IBPB after having set STIBP. Setting STIBP alone does not evict previously created shared predictions. This one can go subtly wrong for anyone who assumes that Intel STIBP and AMD STIBP have the same behaviour.
Borislav Petkov
maintainerQuoted 2 lines · KP Singh
That is a bit more complicated as, for now, the user is not really exposed to STIBP explicitly yet.
Remember that we're exposing the normal user to a gazillion switches already. And not every user has done a PhD in hw vulns like we were forced to in the last, at least 5, years.
So whatever you do, you should try to make it work automatic - if possible - and DTRT - i.e., sane defaults.
Every new functionality added to that madness needs a proper justification.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteFrom: Borislav Petkov
Quoted 9 lines
Sent: 22 February 2023 12:33
Quoted 2 lines · KP Singh
That is a bit more complicated as, for now, the user is not really exposed to STIBP explicitly yet.
Remember that we're exposing the normal user to a gazillion switches already. And not every user has done a PhD in hw vulns like we were forced to in the last, at least 5, years.
It is also worth explaining the acronyms and what the mitigations actually do.
There are also the big family where disabling hyperthreading is probably a better way to avoid the vulnerability.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Borislav Petkov
maintainerQuoted 16 lines · KP Singh
Setting the IBRS bit implicitly enables STIBP to protect against cross-thread branch target injection. With enhanced IBRS, the bit it set once and is not cleared again. However, on CPUs with just legacy IBRS, IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
KERNEL_IBRS). Clearing this bit also disables the implicitly enabled STIBP, thus requiring some form of cross-thread protection in userspace.Enable STIBP, either opt-in via prctl or seccomp, or always on depending on the choice of mitigation selected via
spectre_v2_user.Reported-by José Oliveira <[email protected]>Reported-by Rodrigo Branco <[email protected]>Reviewed-by Alexandra Sandulescu <[email protected]>Fixes 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")Signed-off-by KP Singh <[email protected]>
Quoted 3 lines
--- arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-)
Below is what I'd like to see. Holler if something's wrong.
It is totally untested ofc.
---
From: KP Singh <[email protected]>
Date: Tue, 21 Feb 2023 19:49:07 +0100
Subject: [PATCH] x86/speculation: Allow enabling STIBP with legacy IBRS
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
When plain IBRS is enabled (not enhanced IBRS), the logic in spectre_v2_user_select_mitigation() determines that STIBP is not needed.
However, on return to userspace, the IBRS bit is cleared for performance reasons. That leaves userspace threads vulnerable to cross-thread predictions influence against which STIBP protects.
Exclude IBRS from the spectre_v2_in_ibrs_mode() check to allow for enabling STIBP through seccomp/prctl().
[ bp: Rewrite commit message and massage. ]
--- arch/x86/kernel/cpu/bugs.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/bugs.c +18 -7
@@ -1133,14 +1133,18 @@ spectre_v2_parse_user_cmdline(void)return SPECTRE_V2_USER_CMD_AUTO ; } -static inline bool spectre_v2_in_ibrs_mode (enum spectre_v2_mitigation mode) +static inline bool spectre_v2_in_eibrs_mode (enum spectre_v2_mitigation mode) { -return mode ==SPECTRE_V2_IBRS || - mode ==SPECTRE_V2_EIBRS || +return mode ==SPECTRE_V2_EIBRS || mode ==SPECTRE_V2_EIBRS_RETPOLINE || mode ==SPECTRE_V2_EIBRS_LFENCE ; } +static inline bool spectre_v2_in_ibrs_mode (enum spectre_v2_mitigation mode) +{ +return spectre_v2_in_eibrs_mode (mode) || mode ==SPECTRE_V2_IBRS ; +} +static void __init spectre_v2_user_select_mitigation (void ) {
@@ -1203,12 +1207,19 @@ spectre_v2_user_select_mitigation(void) }/* -* If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible, -* STIBP is not required. +* If no STIBP, enhanced IBRS is enabled, or SMT impossible, STIBP +* is not required. +* +* Enhanced IBRS protects also against user-mode attacks as the IBRS bit +* remains always set which implicitly enables cross-thread protections. +* However, in legacy IBRS mode, the IBRS bit is set only on kernel +* entry and cleared on return to userspace. This disables the implicit +* cross-thread protections so allow for STIBP to be selected in that +* case. */ if (!boot_cpu_has (X86_FEATURE_STIBP ) || !smt_possible || -spectre_v2_in_ibrs_mode (spectre_v2_enabled)) +spectre_v2_in_eibrs_mode (spectre_v2_enabled))return ;/*
@@ -2340,7 +2351,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)static char *stibp_state (void ) { -if (spectre_v2_in_ibrs_mode (spectre_v2_enabled)) +if (spectre_v2_in_eibrs_mode (spectre_v2_enabled))return "" ;switch (spectre_v2_user_stibp) {
Signature · 7 lines
2.35.1
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteQuoted 22 lines · Borislav Petkov
Quoted 16 lines · KP Singh
Setting the IBRS bit implicitly enables STIBP to protect against cross-thread branch target injection. With enhanced IBRS, the bit it set once and is not cleared again. However, on CPUs with just legacy IBRS, IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
KERNEL_IBRS). Clearing this bit also disables the implicitly enabled STIBP, thus requiring some form of cross-thread protection in userspace.Enable STIBP, either opt-in via prctl or seccomp, or always on depending on the choice of mitigation selected via
spectre_v2_user.Reported-by José Oliveira <[email protected]>Reported-by Rodrigo Branco <[email protected]>Reviewed-by Alexandra Sandulescu <[email protected]>Fixes 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")Signed-off-by KP Singh <[email protected]>Quoted 3 lines
--- arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-)
Quoted 1 line
Below is what I'd like to see. Holler if something's wrong.
Thanks for iterating. I think your commit description and rewrite omits a few key subtleties which I have tried to reinforce in both the commit log and the comments.
Q: What does STIBP have to do with IBRS?
A: Setting the IBRS bit implicitly enables STIBP / some form of cross thread protection.
Q: Why does it work with eIBRS?
A: Because we set the IBRS bit once and leave it set when using eIBRS
I think this subtlety should be reinforced in the commit description and code comments so that we don't get it wrong again. Your commit does answer this one (thanks!)
Q: Why does it not work with the way the kernel currently implements legacy IBRS?
A: Because the kernel clears the bit on returning to user space.
Quoted 84 lines
It is totally untested ofc.
---
From: KP Singh <[email protected]>
Date: Tue, 21 Feb 2023 19:49:07 +0100
Subject: [PATCH] x86/speculation: Allow enabling STIBP with legacy IBRS
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bitWhen plain IBRS is enabled (not enhanced IBRS), the logic in
spectre_v2_user_select_mitigation()determines that STIBP is not needed.However, on return to userspace, the IBRS bit is cleared for performance reasons. That leaves userspace threads vulnerable to cross-thread predictions influence against which STIBP protects.
Exclude IBRS from the
spectre_v2_in_ibrs_mode()check to allow for enabling STIBP through seccomp/prctl().[ bp: Rewrite commit message and massage. ]
Fixes 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")Reported-by José Oliveira <[email protected]>Reported-by Rodrigo Branco <[email protected]>Signed-off-by KP Singh <[email protected]>Signed-off-by Borislav Petkov (AMD) <[email protected]>--- arch/x86/kernel/cpu/bugs.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-)arch/x86/kernel/cpu/bugs.c +17
-6@@ -1133,14 +1133,18 @@ spectre_v2_parse_user_cmdline(void)return SPECTRE_V2_USER_CMD_AUTO ; } -static inline bool spectre_v2_in_ibrs_mode (enum spectre_v2_mitigation mode) +static inline bool spectre_v2_in_eibrs_mode (enum spectre_v2_mitigation mode) { -return mode ==SPECTRE_V2_IBRS || - mode ==SPECTRE_V2_EIBRS || +return mode ==SPECTRE_V2_EIBRS || mode ==SPECTRE_V2_EIBRS_RETPOLINE || mode ==SPECTRE_V2_EIBRS_LFENCE ; } +static inline bool spectre_v2_in_ibrs_mode (enum spectre_v2_mitigation mode) +{ +return spectre_v2_in_eibrs_mode (mode) || mode ==SPECTRE_V2_IBRS ; +} +static void __init spectre_v2_user_select_mitigation (void ) {@@ -1203,12 +1207,19 @@ spectre_v2_user_select_mitigation(void) }/* -* If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible, -* STIBP is not required. +* If no STIBP, enhanced IBRS is enabled, or SMT impossible, STIBP +* is not required. +* +* Enhanced IBRS protects also against user-mode attacks as the IBRS bit +* remains always set which implicitly enables cross-thread protections. +* However, in legacy IBRS mode, the IBRS bit is set only on kernel +* entry and cleared on return to userspace. This disables the implicit +* cross-thread protections so allow for STIBP to be selected in that +* case. */ if (!boot_cpu_has (X86_FEATURE_STIBP ) || !smt_possible || -spectre_v2_in_ibrs_mode (spectre_v2_enabled)) +spectre_v2_in_eibrs_mode (spectre_v2_enabled))return ;/*
arch/x86/kernel/cpu/bugs.c static ssize_t mmio_stale_data_show_state(char *buf) · line 2354
Quoted 5 lines
arch/x86/kernel/cpu/bugs.c +1
-1@@ -2340,7 +2351,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)static char *stibp_state (void ) { -if (spectre_v2_in_ibrs_mode (spectre_v2_enabled)) +if (spectre_v2_in_eibrs_mode (spectre_v2_enabled))
The reason why I refactored this into a separate helper was to document the subtleties I mentioned above and anchor them to one place as the function is used in 2 places. But this is a maintainer's choice, so it's your call :)
I do agree with Pawan that it's worth adding a pr_info about what the kernel is doing about STIBP.
- KP
Quoted 11 lines
arch/x86/kernel/cpu/bugs.c +0
-1return "" ;switch (spectre_v2_user_stibp) { --2.35.1
--
Regards/Gruss,
Boris.
Borislav Petkov
maintainerQuoted 7 lines · KP Singh
Thanks for iterating. I think your commit description and rewrite omits a few key subtleties which I have tried to reinforce in both the commit log and the comments.
Q: What does STIBP have to do with IBRS?
A: Setting the IBRS bit implicitly enables STIBP / some form of cross thread protection.
That belongs in the docs, if you want to explain this properly.
Quoted 2 lines
Q: Why does it work with eIBRS?
A: Because we set the IBRS bit once and leave it set when using eIBRS
Also docs.
Quoted 3 lines
I think this subtlety should be reinforced in the commit description and code comments so that we don't get it wrong again. Your commit does answer this one (thanks!)
Commit messages are fine when explaining *why* a change is being done. What is even finer is when you put a lenghtier explanation in our documentation so that people can actually find it. Finding text in commit messages is harder...
Quoted 3 lines
Q: Why does it not work with the way the kernel currently implements legacy IBRS?
A: Because the kernel clears the bit on returning to user space.
From the commit message:
However, on return to userspace, the IBRS bit is cleared for performance reasons. That leaves userspace threads vulnerable to cross-thread predictions influence against which STIBP protects.
Quoted 4 lines
The reason why I refactored this into a separate helper was to document the subtleties I mentioned above and anchor them to one place as the function is used in 2 places. But this is a maintainer's choice, so it's your call :)
The less code gets added in that thing, the better. Not yet another helper pls.
Quoted 2 lines
I do agree with Pawan that it's worth adding a
pr_infoabout what the kernel is doing about STIBP.
STIBP status gets dumped through stibp_state().
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteQuoted 17 lines · Borislav Petkov
Quoted 7 lines · KP Singh
Thanks for iterating. I think your commit description and rewrite omits a few key subtleties which I have tried to reinforce in both the commit log and the comments.
Q: What does STIBP have to do with IBRS?
A: Setting the IBRS bit implicitly enables STIBP / some form of cross thread protection.That belongs in the docs, if you want to explain this properly.
Quoted 2 lines
Q: Why does it work with eIBRS?
A: Because we set the IBRS bit once and leave it set when using eIBRSAlso docs.
Quoted 8 lines
Quoted 3 lines
I think this subtlety should be reinforced in the commit description and code comments so that we don't get it wrong again. Your commit does answer this one (thanks!)
Commit messages are fine when explaining *why* a change is being done. What is even finer is when you put a lenghtier explanation in our documentation so that people can actually find it. Finding text in commit messages is harder...
Sure, I think the docs do already cover it, but I sort of disagree with your statement around the commit message. I feel the more context you can add in the commit message, the better it is. When I am looking at the change log, it would be helpful to have the information that I mentioned in the Q&A. Small things like, "eIBRS needs the IBRS bit set which also enables cross-thread protections" is a very important context for this patch IMHO. Without this one is just left head scratching and scrambling to read lengthy docs and processor manuals.
But again, totally your call. Others, feel free to chime in.
Quoted 10 lines
Quoted 4 lines
Q: Why does it not work with the way the kernel currently implements legacy IBRS?
A: Because the kernel clears the bit on returning to user space.From the commit message:
However, on return to userspace, the IBRS bit is cleared for performance reasons. That leaves userspace threads vulnerable to cross-thread predictions influence against which STIBP protects.
This sort of loosely implies that the IBRS bit also enables cross-thread protections. Can you atleast add this one explicitly?
"Setting the IBRS bit also enables cross thread protections"
Quoted 8 lines
Quoted 5 lines
The reason why I refactored this into a separate helper was to document the subtleties I mentioned above and anchor them to one place as the function is used in 2 places. But this is a maintainer's choice, so it's your call :)
The less code gets added in that thing, the better. Not yet another helper pls.
Sure, your call.
Quoted 5 lines
Quoted 3 lines
I do agree with Pawan that it's worth adding a
pr_infoabout what the kernel is doing about STIBP.STIBP status gets dumped through
stibp_state().
Not at the stage when the kernel decides to drop the STIBP protection when eIBRS is enabled. If we had this information when we had a positive POC, it would have been much easier to figure out what's going on here.
- KP
Quoted 6 lines
--
Regards/Gruss,
Boris.
Borislav Petkov
maintainerQuoted 1 line · KP Singh
Sure, I think the docs do already cover it,
I mean *our docs*. The stuff you're adding in your patch 2.
Quoted 3 lines
but I sort of disagree with your statement around the commit message. I feel the more context you can add in the commit message, the better it is.
That's ofc wrong. And you'll find that out when you do git archeology and you come across a huuuge wall of text explaining the world and some more.
No, commit messages should be to the point with a structure similar to something like this:
1. Prepare the context for the explanation briefly.
2. Explain the problem at hand.
3. "It happens because of <...>"
4. "Fix it by doing X"
5. "(Potentially do Y)."
concentrating on *why* the fix is being done.
Quoted 6 lines
When I am looking at the change log, it would be helpful to have the information that I mentioned in the Q&A. Small things like, "eIBRS needs the IBRS bit set which also enables cross-thread protections" is a very important context for this patch IMHO. Without this one is just left head scratching and scrambling to read lengthy docs and processor manuals.
Yes, that's why you say in the commit message: "For more details, see Documentation/admin-guide/hw-vuln/spectre.rst." where:
1. you can explain in a lot more detail
2. put it in place where people can find it *easily*
Quoted 4 lines
This sort of loosely implies that the IBRS bit also enables cross-thread protections. Can you atleast add this one explicitly?
"Setting the IBRS bit also enables cross thread protections"
Ok.
Quoted 2 lines
Not at the stage when the kernel decides to drop the STIBP protection when eIBRS is enabled.
We can't dump every possible interaction between the mitigations. It is a huge mess already. But I'm open to looking at improvements of the situation *and* documenting stuff as we go.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteQuoted 48 lines · Borislav Petkov
Quoted 1 line · KP Singh
Sure, I think the docs do already cover it,
I mean *our docs*. The stuff you're adding in your patch 2.
Quoted 3 lines
but I sort of disagree with your statement around the commit message. I feel the more context you can add in the commit message, the better it is.
That's ofc wrong. And you'll find that out when you do git archeology and you come across a huuuge wall of text explaining the world and some more.
No, commit messages should be to the point with a structure similar to something like this:
1. Prepare the context for the explanation briefly.
2. Explain the problem at hand.
3. "It happens because of <...>"
4. "Fix it by doing X"
5. "(Potentially do Y)."
concentrating on *why* the fix is being done.
Quoted 6 lines
When I am looking at the change log, it would be helpful to have the information that I mentioned in the Q&A. Small things like, "eIBRS needs the IBRS bit set which also enables cross-thread protections" is a very important context for this patch IMHO. Without this one is just left head scratching and scrambling to read lengthy docs and processor manuals.
Yes, that's why you say in the commit message: "For more details, see
Documentation/admin-guide/hw-vuln/spectre.rst." where:1. you can explain in a lot more detail
2. put it in place where people can find it *easily*
Quoted 4 lines
This sort of loosely implies that the IBRS bit also enables cross-thread protections. Can you atleast add this one explicitly?
"Setting the IBRS bit also enables cross thread protections"
Quoted 9 lines
Ok.
Quoted 2 lines
Not at the stage when the kernel decides to drop the STIBP protection when eIBRS is enabled.
We can't dump every possible interaction between the mitigations. It is a huge mess already. But I'm open to looking at improvements of the situation *and* documenting stuff as we go.
Well, we can try our best to print a message when we take a decision on behalf of the user. As I mentioned, had I got any message that the kernel was doing this it would have been much easier for me to figure out why the POC was working. It was tricky without this to figure out and I am capturing my broad chain of thought here:
1. cross thread training seems to work, this seems to be a bug
2. let me enable cross thread protections with a prctl
3. it still seems to work, did I get a printnk that my prctl did not succeed or the kernel did something special? - no - why?
I will still go ahead with the patch that was "re-written" for me and we can add debug information subsequently / as a follow up.
Quoted 7 lines
Thx.
--
Regards/Gruss,
Boris.