x86/cpufeatures: Define X86_FEATURE_AMD_IBPB_RET
PATCH v2 1/2 x86/cpufeatures: Define X86_FEATURE_AMD_IBPB_RET
AMD's initial implementation of IBPB did not clear the return address predictor. Beginning with Zen4, AMD's IBPB *does* clear the return address predictor. This behavior is enumerated by CPUID.80000008H:EBX.IBPB_RET[bit 30].
Define X86_FEATURE_AMD_IBPB_RET for use in KVM_GET_SUPPORTED_CPUID, when determining cross-vendor capabilities.
--- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -348,6 +348,7 @@#define X86_FEATURE_CPPC (13*32+27)/* "cppc" Collaborative Processor Performance Control */ #define X86_FEATURE_AMD_PSFD (13*32+28)/* Predictive Store Forwarding Disable */ #define X86_FEATURE_BTC_NO (13*32+29)/* Not vulnerable to Branch Type Confusion */ +#define X86_FEATURE_AMD_IBPB_RET (13*32+30)/* IBPB clears return address predictor */ #define X86_FEATURE_BRS (13*32+31)/* "brs" Branch Sampling available */ /* Thermal and Power Management Leaf, CPUID level 0x00000006 (EAX), word 14 */
Signature · 1 line
2.46.0.184.g6999bdac58-goog
PATCH v2 2/2 KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB
From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26] enumerates support for indirect branch restricted speculation (IBRS) and the indirect branch predictor barrier (IBPB)." Further, from [2], "Software that executed before the IBPB command cannot control the predicted targets of indirect branches (4) executed after the command on the same logical processor," where footnote 4 reads, "Note that indirect branches include near call indirect, near jump indirect and near return instructions. Because it includes near returns, it follows that **RSB entries created before an IBPB command cannot control the predicted targets of returns executed after the command on the same logical processor.**" [emphasis mine]
On the other hand, AMD's IBPB "may not prevent return branch predictions from being specified by pre-IBPB branch targets" [3].
However, some AMD processors have an "enhanced IBPB" [terminology mine] which does clear the return address predictor. This feature is enumerated by CPUID.80000008:EDX.IBPB_RET[bit 30] [4].
Adjust the cross-vendor features enumerated by KVM_GET_SUPPORTED_CPUID accordingly.
[1] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
[2] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html#Footnotes
[3] https://www.amd.com/en/resources/product-security/bulletin/amd-sb-1040.html
[4] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24594.pdf
---
v2: Use IBPB_RET to identify semantic equality (Venkatesh)
arch/x86/kvm/cpuid.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/cpuid.c +6 -2
@@ -690,7 +690,9 @@ void kvm_set_cpu_caps(void)kvm_cpu_cap_set (X86_FEATURE_TSC_ADJUST );kvm_cpu_cap_set (X86_FEATURE_ARCH_CAPABILITIES ); -if (boot_cpu_has (X86_FEATURE_IBPB ) &&boot_cpu_has (X86_FEATURE_IBRS )) +if (boot_cpu_has (X86_FEATURE_AMD_IBPB_RET ) && +boot_cpu_has (X86_FEATURE_AMD_IBPB ) && +boot_cpu_has (X86_FEATURE_AMD_IBRS ))kvm_cpu_cap_set (X86_FEATURE_SPEC_CTRL );if (boot_cpu_has (X86_FEATURE_STIBP ))kvm_cpu_cap_set (X86_FEATURE_INTEL_STIBP );
@@ -759,8 +761,10 @@ void kvm_set_cpu_caps(void)* arch/x86/kernel/cpu/bugs.c is kind enough to * record that in cpufeatures so use them. */ -if (boot_cpu_has (X86_FEATURE_IBPB )) +if (boot_cpu_has (X86_FEATURE_IBPB )) {kvm_cpu_cap_set (X86_FEATURE_AMD_IBPB ); +kvm_cpu_cap_set (X86_FEATURE_AMD_IBPB_RET ); + }if (boot_cpu_has (X86_FEATURE_IBRS ))kvm_cpu_cap_set (X86_FEATURE_AMD_IBRS );if (boot_cpu_has (X86_FEATURE_STIBP ))
Signature · 1 line
2.46.0.184.g6999bdac58-goog
Sean Christopherson
maintainerRe: [PATCH v2 2/2] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB
Quoted 22 lines · Jim Mattson
Quoted 1 line
From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]
enumerates support for indirect branch restricted speculation (IBRS) and the indirect branch predictor barrier (IBPB)." Further, from [2], "Software that executed before the IBPB command cannot control the predicted targets of indirect branches (4) executed after the command on the same logical processor," where footnote 4 reads, "Note that indirect branches include near call indirect, near jump indirect and near return instructions. Because it includes near returns, it follows that **RSB entries created before an IBPB command cannot control the predicted targets of returns executed after the command on the same logical processor.**" [emphasis mine]
On the other hand, AMD's IBPB "may not prevent return branch predictions from being specified by pre-IBPB branch targets" [3].
However, some AMD processors have an "enhanced IBPB" [terminology mine] which does clear the return address predictor. This feature is enumerated by CPUID.80000008:EDX.IBPB_RET[bit 30] [4].
Adjust the cross-vendor features enumerated by
KVM_GET_SUPPORTED_CPUIDaccordingly.
Quoted 8 lines
[1] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
[2] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html#Footnotes
[3] https://www.amd.com/en/resources/product-security/bulletin/amd-sb-1040.html
[4] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24594.pdfFixes 0c54914d0c52 ("KVM: x86: use Intel speculation bugs and features as derived in generic x86 code")Suggested-by Venkatesh Srinivas <[email protected]>Signed-off-by Jim Mattson <[email protected]>
Venkatesh, can I grab a review from you on this? You know this way better than
I do, and I honestly don't feel like reading mitigation disclosures right now :-)
Quoted 36 lines
---
v2: UseIBPB_RETto identify semantic equality (Venkatesh)arch/x86/kvm/cpuid.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)arch/x86/kvm/cpuid.c +6
-2@@ -690,7 +690,9 @@ void kvm_set_cpu_caps(void)kvm_cpu_cap_set (X86_FEATURE_TSC_ADJUST );kvm_cpu_cap_set (X86_FEATURE_ARCH_CAPABILITIES ); -if (boot_cpu_has (X86_FEATURE_IBPB ) &&boot_cpu_has (X86_FEATURE_IBRS )) +if (boot_cpu_has (X86_FEATURE_AMD_IBPB_RET ) && +boot_cpu_has (X86_FEATURE_AMD_IBPB ) && +boot_cpu_has (X86_FEATURE_AMD_IBRS ))kvm_cpu_cap_set (X86_FEATURE_SPEC_CTRL );if (boot_cpu_has (X86_FEATURE_STIBP ))kvm_cpu_cap_set (X86_FEATURE_INTEL_STIBP );@@ -759,8 +761,10 @@ void kvm_set_cpu_caps(void)* arch/x86/kernel/cpu/bugs.c is kind enough to * record that in cpufeatures so use them. */ -if (boot_cpu_has (X86_FEATURE_IBPB )) +if (boot_cpu_has (X86_FEATURE_IBPB )) {kvm_cpu_cap_set (X86_FEATURE_AMD_IBPB ); +kvm_cpu_cap_set (X86_FEATURE_AMD_IBPB_RET ); + }if (boot_cpu_has (X86_FEATURE_IBRS ))kvm_cpu_cap_set (X86_FEATURE_AMD_IBRS );if (boot_cpu_has (X86_FEATURE_STIBP ))Signature · 1 line
2.46.0.184.g6999bdac58-goog
Venkatesh Srinivas
Reviewed-byRe: [PATCH v2 2/2] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB
Quoted 28 lines · Sean Christopherson
Quoted 27 lines · Jim Mattson
Quoted 1 line
From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]
enumerates support for indirect branch restricted speculation (IBRS) and the indirect branch predictor barrier (IBPB)." Further, from [2], "Software that executed before the IBPB command cannot control the predicted targets of indirect branches (4) executed after the command on the same logical processor," where footnote 4 reads, "Note that indirect branches include near call indirect, near jump indirect and near return instructions. Because it includes near returns, it follows that **RSB entries created before an IBPB command cannot control the predicted targets of returns executed after the command on the same logical processor.**" [emphasis mine]
On the other hand, AMD's IBPB "may not prevent return branch predictions from being specified by pre-IBPB branch targets" [3].
However, some AMD processors have an "enhanced IBPB" [terminology mine] which does clear the return address predictor. This feature is enumerated by CPUID.80000008:EDX.IBPB_RET[bit 30] [4].
Adjust the cross-vendor features enumerated by
KVM_GET_SUPPORTED_CPUIDaccordingly.[1] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
[2] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html#Footnotes
[3] https://www.amd.com/en/resources/product-security/bulletin/amd-sb-1040.html
[4] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24594.pdf
Quoted 6 lines
Quoted 3 lines
Fixes 0c54914d0c52 ("KVM: x86: use Intel speculation bugs and features as derived in generic x86 code")Suggested-by Venkatesh Srinivas <[email protected]>Signed-off-by Jim Mattson <[email protected]>Venkatesh, can I grab a review from you on this? You know this way better than
I do, and I honestly don't feel like reading mitigation disclosures right now :-)
Got lost in my mailbox!
Quoted 37 lines
Quoted 37 lines
---
v2: UseIBPB_RETto identify semantic equality (Venkatesh)arch/x86/kvm/cpuid.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)arch/x86/kvm/cpuid.c +6
-2@@ -690,7 +690,9 @@ void kvm_set_cpu_caps(void)kvm_cpu_cap_set (X86_FEATURE_TSC_ADJUST );kvm_cpu_cap_set (X86_FEATURE_ARCH_CAPABILITIES ); -if (boot_cpu_has (X86_FEATURE_IBPB ) &&boot_cpu_has (X86_FEATURE_IBRS )) +if (boot_cpu_has (X86_FEATURE_AMD_IBPB_RET ) && +boot_cpu_has (X86_FEATURE_AMD_IBPB ) && +boot_cpu_has (X86_FEATURE_AMD_IBRS ))kvm_cpu_cap_set (X86_FEATURE_SPEC_CTRL );if (boot_cpu_has (X86_FEATURE_STIBP ))kvm_cpu_cap_set (X86_FEATURE_INTEL_STIBP );@@ -759,8 +761,10 @@ void kvm_set_cpu_caps(void)* arch/x86/kernel/cpu/bugs.c is kind enough to * record that in cpufeatures so use them. */ -if (boot_cpu_has (X86_FEATURE_IBPB )) +if (boot_cpu_has (X86_FEATURE_IBPB )) {kvm_cpu_cap_set (X86_FEATURE_AMD_IBPB ); +kvm_cpu_cap_set (X86_FEATURE_AMD_IBPB_RET ); + }if (boot_cpu_has (X86_FEATURE_IBRS ))kvm_cpu_cap_set (X86_FEATURE_AMD_IBRS );if (boot_cpu_has (X86_FEATURE_STIBP ))Signature · 1 line
2.46.0.184.g6999bdac58-goog
Re: [PATCH v2 2/2] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB
In reply to Jim MattsonQuoted 52 lines · Jim Mattson
From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26] enumerates support for indirect branch restricted speculation (IBRS) and the indirect branch predictor barrier (IBPB)." Further, from [2], "Software that executed before the IBPB command cannot control the predicted targets of indirect branches (4) executed after the command on the same logical processor," where footnote 4 reads, "Note that indirect branches include near call indirect, near jump indirect and near return instructions. Because it includes near returns, it follows that **RSB entries created before an IBPB command cannot control the predicted targets of returns executed after the command on the same logical processor.**" [emphasis mine]
On the other hand, AMD's IBPB "may not prevent return branch predictions from being specified by pre-IBPB branch targets" [3].
However, some AMD processors have an "enhanced IBPB" [terminology mine] which does clear the return address predictor. This feature is enumerated by CPUID.80000008:EDX.IBPB_RET[bit 30] [4].
Adjust the cross-vendor features enumerated by
KVM_GET_SUPPORTED_CPUIDaccordingly.[1] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
[2] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html#Footnotes
[3] https://www.amd.com/en/resources/product-security/bulletin/amd-sb-1040.html
[4] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24594.pdfFixes 0c54914d0c52 ("KVM: x86: use Intel speculation bugs and features as derived in generic x86 code")Suggested-by Venkatesh Srinivas <[email protected]>Signed-off-by Jim Mattson <[email protected]>---
v2: UseIBPB_RETto identify semantic equality (Venkatesh)arch/x86/kvm/cpuid.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)arch/x86/kvm/cpuid.c +3
-1@@ -690,7 +690,9 @@ void kvm_set_cpu_caps(void)kvm_cpu_cap_set (X86_FEATURE_TSC_ADJUST );kvm_cpu_cap_set (X86_FEATURE_ARCH_CAPABILITIES ); -if (boot_cpu_has (X86_FEATURE_IBPB ) &&boot_cpu_has (X86_FEATURE_IBRS )) +if (boot_cpu_has (X86_FEATURE_AMD_IBPB_RET ) && +boot_cpu_has (X86_FEATURE_AMD_IBPB ) && +boot_cpu_has (X86_FEATURE_AMD_IBRS ))kvm_cpu_cap_set (X86_FEATURE_SPEC_CTRL );if (boot_cpu_has (X86_FEATURE_STIBP ))kvm_cpu_cap_set (X86_FEATURE_INTEL_STIBP );
arch/x86/kvm/cpuid.c void kvm_set_cpu_caps(void) · line 766
Quoted 7 lines
arch/x86/kvm/cpuid.c +2
-1@@ -759,8 +761,10 @@ void kvm_set_cpu_caps(void)* arch/x86/kernel/cpu/bugs.c is kind enough to * record that in cpufeatures so use them. */ -if (boot_cpu_has (X86_FEATURE_IBPB )) +if (boot_cpu_has (X86_FEATURE_IBPB )) {kvm_cpu_cap_set (X86_FEATURE_AMD_IBPB ); +kvm_cpu_cap_set (X86_FEATURE_AMD_IBPB_RET );
Should IBPB_RET be conditionally set? I would think that you would only want to set IBPB_RET if either IBPB_RET or SPEC_CTRL is set on the hypervisor.
if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) || boot_cpu_has(X86_FEATURE_SPEC_CTRL) kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);
Right?
Thanks,
Tom
Quoted 4 lines
arch/x86/kvm/cpuid.c +1
-0+ }if (boot_cpu_has (X86_FEATURE_IBRS ))kvm_cpu_cap_set (X86_FEATURE_AMD_IBRS );if (boot_cpu_has (X86_FEATURE_STIBP ))
Re: [PATCH v2 2/2] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB
Quoted 65 lines · Tom Lendacky
Quoted 52 lines · Jim Mattson
From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26] enumerates support for indirect branch restricted speculation (IBRS) and the indirect branch predictor barrier (IBPB)." Further, from [2], "Software that executed before the IBPB command cannot control the predicted targets of indirect branches (4) executed after the command on the same logical processor," where footnote 4 reads, "Note that indirect branches include near call indirect, near jump indirect and near return instructions. Because it includes near returns, it follows that **RSB entries created before an IBPB command cannot control the predicted targets of returns executed after the command on the same logical processor.**" [emphasis mine]
On the other hand, AMD's IBPB "may not prevent return branch predictions from being specified by pre-IBPB branch targets" [3].
However, some AMD processors have an "enhanced IBPB" [terminology mine] which does clear the return address predictor. This feature is enumerated by CPUID.80000008:EDX.IBPB_RET[bit 30] [4].
Adjust the cross-vendor features enumerated by
KVM_GET_SUPPORTED_CPUIDaccordingly.[1] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
[2] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html#Footnotes
[3] https://www.amd.com/en/resources/product-security/bulletin/amd-sb-1040.html
[4] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24594.pdfFixes 0c54914d0c52 ("KVM: x86: use Intel speculation bugs and features as derived in generic x86 code")Suggested-by Venkatesh Srinivas <[email protected]>Signed-off-by Jim Mattson <[email protected]>---
v2: UseIBPB_RETto identify semantic equality (Venkatesh)arch/x86/kvm/cpuid.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)arch/x86/kvm/cpuid.c +3
-1@@ -690,7 +690,9 @@ void kvm_set_cpu_caps(void)kvm_cpu_cap_set (X86_FEATURE_TSC_ADJUST );kvm_cpu_cap_set (X86_FEATURE_ARCH_CAPABILITIES ); -if (boot_cpu_has (X86_FEATURE_IBPB ) &&boot_cpu_has (X86_FEATURE_IBRS )) +if (boot_cpu_has (X86_FEATURE_AMD_IBPB_RET ) && +boot_cpu_has (X86_FEATURE_AMD_IBPB ) && +boot_cpu_has (X86_FEATURE_AMD_IBRS ))kvm_cpu_cap_set (X86_FEATURE_SPEC_CTRL );if (boot_cpu_has (X86_FEATURE_STIBP ))kvm_cpu_cap_set (X86_FEATURE_INTEL_STIBP );arch/x86/kvm/cpuid.c void kvm_set_cpu_caps(void) · line 766
Quoted 7 lines
arch/x86/kvm/cpuid.c +2
-1@@ -759,8 +761,10 @@ void kvm_set_cpu_caps(void)* arch/x86/kernel/cpu/bugs.c is kind enough to * record that in cpufeatures so use them. */ -if (boot_cpu_has (X86_FEATURE_IBPB )) +if (boot_cpu_has (X86_FEATURE_IBPB )) {kvm_cpu_cap_set (X86_FEATURE_AMD_IBPB ); +kvm_cpu_cap_set (X86_FEATURE_AMD_IBPB_RET );Should
IBPB_RETbe conditionally set? I would think that you would only want to setIBPB_RETif eitherIBPB_RETorSPEC_CTRLis set on the hypervisor.
Quoted 5 lines
if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) || boot_cpu_has(X86_FEATURE_SPEC_CTRL) kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);Right?
Right. This clause is intended to set cross-vendor capabilities, so it should be:
if (boot_cpu_has(X86_FEATURE_SPEC_CTRL))
kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);Passing through AMD_IBPB_RET from the hardware should be done by adding the bit to the mask for CPUID_8000_0008_EBX.
I'll send out a v3.
Thanks!
Quoted 7 lines
Thanks,
TomQuoted 4 lines
arch/x86/kvm/cpuid.c +1
-0+ }if (boot_cpu_has (X86_FEATURE_IBRS ))kvm_cpu_cap_set (X86_FEATURE_AMD_IBRS );if (boot_cpu_has (X86_FEATURE_STIBP ))