Skip to contentSkip to conversation

x86/cpufeatures: Define X86_FEATURE_AMD_IBPB_RET

PATCH In review Jim Mattson 4 participants 6 messages Aug 16, 2024 – Aug 23, 2024

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 x86/cpufeatures: Define X86_FEATURE_AMD_IBPB_RET

AMD's initial implementation of IBPB did not clear the return address predictor. Beginning with Zen4, AMD's IBPB *does* clear the return address predictor. This behavior is enumerated by CPUID.80000008H:EBX.IBPB_RET[bit 30].

Define X86_FEATURE_AMD_IBPB_RET for use in KVM_GET_SUPPORTED_CPUID, when determining cross-vendor capabilities.

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
index dd4682857c12..e5bcb428dcf9 100644
@@ -348,6 +348,7 @@
 #define X86_FEATURE_CPPC		(13*32+27) /* "cppc" Collaborative Processor Performance Control */
 #define X86_FEATURE_AMD_PSFD            (13*32+28) /* Predictive Store Forwarding Disable */
 #define X86_FEATURE_BTC_NO		(13*32+29) /* Not vulnerable to Branch Type Confusion */
+#define X86_FEATURE_AMD_IBPB_RET	(13*32+30) /* IBPB clears return address predictor */
 #define X86_FEATURE_BRS			(13*32+31) /* "brs" Branch Sampling available */
 
 /* Thermal and Power Management Leaf, CPUID level 0x00000006 (EAX), word 14 */
Signature · 1 line
2.46.0.184.g6999bdac58-goog
#1 lore raw

PATCH v2 2/2 KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26] enumerates support for indirect branch restricted speculation (IBRS) and the indirect branch predictor barrier (IBPB)." Further, from [2], "Software that executed before the IBPB command cannot control the predicted targets of indirect branches (4) executed after the command on the same logical processor," where footnote 4 reads, "Note that indirect branches include near call indirect, near jump indirect and near return instructions. Because it includes near returns, it follows that **RSB entries created before an IBPB command cannot control the predicted targets of returns executed after the command on the same logical processor.**" [emphasis mine]

On the other hand, AMD's IBPB "may not prevent return branch predictions from being specified by pre-IBPB branch targets" [3].

However, some AMD processors have an "enhanced IBPB" [terminology mine] which does clear the return address predictor. This feature is enumerated by CPUID.80000008:EDX.IBPB_RET[bit 30] [4].

Adjust the cross-vendor features enumerated by KVM_GET_SUPPORTED_CPUID accordingly.

[1] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
[2] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html#Footnotes
[3] https://www.amd.com/en/resources/product-security/bulletin/amd-sb-1040.html
[4] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24594.pdf

Fixes 0c54914d0c52 ("KVM: x86: use Intel speculation bugs and features as derived in generic x86 code")

---
v2: Use IBPB_RET to identify semantic equality (Venkatesh)

 arch/x86/kvm/cpuid.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/cpuid.c +6 -2
index 2617be544480..044bdc9e938b 100644
@@ -690,7 +690,9 @@ void kvm_set_cpu_caps(void)
 	kvm_cpu_cap_set(X86_FEATURE_TSC_ADJUST);
 	kvm_cpu_cap_set(X86_FEATURE_ARCH_CAPABILITIES);
 
-	if (boot_cpu_has(X86_FEATURE_IBPB) && boot_cpu_has(X86_FEATURE_IBRS))
+	if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) &&
+	    boot_cpu_has(X86_FEATURE_AMD_IBPB) &&
+	    boot_cpu_has(X86_FEATURE_AMD_IBRS))
 		kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);
 	if (boot_cpu_has(X86_FEATURE_STIBP))
 		kvm_cpu_cap_set(X86_FEATURE_INTEL_STIBP);
@@ -759,8 +761,10 @@ void kvm_set_cpu_caps(void)
 	 * arch/x86/kernel/cpu/bugs.c is kind enough to
 	 * record that in cpufeatures so use them.
 	 */
-	if (boot_cpu_has(X86_FEATURE_IBPB))
+	if (boot_cpu_has(X86_FEATURE_IBPB)) {
 		kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB);
+		kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);
+	}
 	if (boot_cpu_has(X86_FEATURE_IBRS))
 		kvm_cpu_cap_set(X86_FEATURE_AMD_IBRS);
 	if (boot_cpu_has(X86_FEATURE_STIBP))
Signature · 1 line
2.46.0.184.g6999bdac58-goog
#2 lore raw

Re: [PATCH v2 2/2] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

Quoted 22 lines · Jim Mattson
Quoted 1 line

From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]

enumerates support for indirect branch restricted speculation (IBRS) and the indirect branch predictor barrier (IBPB)." Further, from [2], "Software that executed before the IBPB command cannot control the predicted targets of indirect branches (4) executed after the command on the same logical processor," where footnote 4 reads, "Note that indirect branches include near call indirect, near jump indirect and near return instructions. Because it includes near returns, it follows that **RSB entries created before an IBPB command cannot control the predicted targets of returns executed after the command on the same logical processor.**" [emphasis mine]

On the other hand, AMD's IBPB "may not prevent return branch predictions from being specified by pre-IBPB branch targets" [3].

However, some AMD processors have an "enhanced IBPB" [terminology mine] which does clear the return address predictor. This feature is enumerated by CPUID.80000008:EDX.IBPB_RET[bit 30] [4].

Adjust the cross-vendor features enumerated by KVM_GET_SUPPORTED_CPUID accordingly.

Quoted 8 lines

[1] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
[2] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html#Footnotes
[3] https://www.amd.com/en/resources/product-security/bulletin/amd-sb-1040.html
[4] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24594.pdf

Fixes 0c54914d0c52 ("KVM: x86: use Intel speculation bugs and features as derived in generic x86 code")

Venkatesh, can I grab a review from you on this? You know this way better than
I do, and I honestly don't feel like reading mitigation disclosures right now :-)

Quoted 36 lines

---
v2: Use IBPB_RET to identify semantic equality (Venkatesh)

 arch/x86/kvm/cpuid.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/cpuid.c +6 -2
index 2617be544480..044bdc9e938b 100644
@@ -690,7 +690,9 @@ void kvm_set_cpu_caps(void)
 	kvm_cpu_cap_set(X86_FEATURE_TSC_ADJUST);
 	kvm_cpu_cap_set(X86_FEATURE_ARCH_CAPABILITIES);
 
-	if (boot_cpu_has(X86_FEATURE_IBPB) && boot_cpu_has(X86_FEATURE_IBRS))
+	if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) &&
+	    boot_cpu_has(X86_FEATURE_AMD_IBPB) &&
+	    boot_cpu_has(X86_FEATURE_AMD_IBRS))
 		kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);
 	if (boot_cpu_has(X86_FEATURE_STIBP))
 		kvm_cpu_cap_set(X86_FEATURE_INTEL_STIBP);
@@ -759,8 +761,10 @@ void kvm_set_cpu_caps(void)
 	 * arch/x86/kernel/cpu/bugs.c is kind enough to
 	 * record that in cpufeatures so use them.
 	 */
-	if (boot_cpu_has(X86_FEATURE_IBPB))
+	if (boot_cpu_has(X86_FEATURE_IBPB)) {
 		kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB);
+		kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);
+	}
 	if (boot_cpu_has(X86_FEATURE_IBRS))
 		kvm_cpu_cap_set(X86_FEATURE_AMD_IBRS);
 	if (boot_cpu_has(X86_FEATURE_STIBP))
Signature · 1 line
2.46.0.184.g6999bdac58-goog
Reviewed-by
#3 lore raw

Re: [PATCH v2 2/2] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

Quoted 28 lines · Sean Christopherson
Quoted 27 lines · Jim Mattson
Quoted 1 line

From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]

enumerates support for indirect branch restricted speculation (IBRS) and the indirect branch predictor barrier (IBPB)." Further, from [2], "Software that executed before the IBPB command cannot control the predicted targets of indirect branches (4) executed after the command on the same logical processor," where footnote 4 reads, "Note that indirect branches include near call indirect, near jump indirect and near return instructions. Because it includes near returns, it follows that **RSB entries created before an IBPB command cannot control the predicted targets of returns executed after the command on the same logical processor.**" [emphasis mine]

On the other hand, AMD's IBPB "may not prevent return branch predictions from being specified by pre-IBPB branch targets" [3].

However, some AMD processors have an "enhanced IBPB" [terminology mine] which does clear the return address predictor. This feature is enumerated by CPUID.80000008:EDX.IBPB_RET[bit 30] [4].

Adjust the cross-vendor features enumerated by KVM_GET_SUPPORTED_CPUID accordingly.

[1] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
[2] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html#Footnotes
[3] https://www.amd.com/en/resources/product-security/bulletin/amd-sb-1040.html
[4] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24594.pdf

Quoted 6 lines
Quoted 3 lines
Fixes 0c54914d0c52 ("KVM: x86: use Intel speculation bugs and features as derived in generic x86 code")

Venkatesh, can I grab a review from you on this? You know this way better than
I do, and I honestly don't feel like reading mitigation disclosures right now :-)

Got lost in my mailbox!

Quoted 37 lines
Quoted 37 lines

---
v2: Use IBPB_RET to identify semantic equality (Venkatesh)

  arch/x86/kvm/cpuid.c | 8 ++++++--
  1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/cpuid.c +6 -2
index 2617be544480..044bdc9e938b 100644
@@ -690,7 +690,9 @@ void kvm_set_cpu_caps(void)
  	kvm_cpu_cap_set(X86_FEATURE_TSC_ADJUST);
  	kvm_cpu_cap_set(X86_FEATURE_ARCH_CAPABILITIES);
 
-	if (boot_cpu_has(X86_FEATURE_IBPB) && boot_cpu_has(X86_FEATURE_IBRS))
+	if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) &&
+	    boot_cpu_has(X86_FEATURE_AMD_IBPB) &&
+	    boot_cpu_has(X86_FEATURE_AMD_IBRS))
  		kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);
  	if (boot_cpu_has(X86_FEATURE_STIBP))
  		kvm_cpu_cap_set(X86_FEATURE_INTEL_STIBP);
@@ -759,8 +761,10 @@ void kvm_set_cpu_caps(void)
  	 * arch/x86/kernel/cpu/bugs.c is kind enough to
  	 * record that in cpufeatures so use them.
  	 */
-	if (boot_cpu_has(X86_FEATURE_IBPB))
+	if (boot_cpu_has(X86_FEATURE_IBPB)) {
  		kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB);
+		kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);
+	}
  	if (boot_cpu_has(X86_FEATURE_IBRS))
  		kvm_cpu_cap_set(X86_FEATURE_AMD_IBRS);
  	if (boot_cpu_has(X86_FEATURE_STIBP))
Signature · 1 line
2.46.0.184.g6999bdac58-goog
#4 lore raw

Re: [PATCH v2 2/2] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

In reply to Jim Mattson
Quoted 52 lines · Jim Mattson

From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26] enumerates support for indirect branch restricted speculation (IBRS) and the indirect branch predictor barrier (IBPB)." Further, from [2], "Software that executed before the IBPB command cannot control the predicted targets of indirect branches (4) executed after the command on the same logical processor," where footnote 4 reads, "Note that indirect branches include near call indirect, near jump indirect and near return instructions. Because it includes near returns, it follows that **RSB entries created before an IBPB command cannot control the predicted targets of returns executed after the command on the same logical processor.**" [emphasis mine]

On the other hand, AMD's IBPB "may not prevent return branch predictions from being specified by pre-IBPB branch targets" [3].

However, some AMD processors have an "enhanced IBPB" [terminology mine] which does clear the return address predictor. This feature is enumerated by CPUID.80000008:EDX.IBPB_RET[bit 30] [4].

Adjust the cross-vendor features enumerated by KVM_GET_SUPPORTED_CPUID accordingly.

[1] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
[2] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html#Footnotes
[3] https://www.amd.com/en/resources/product-security/bulletin/amd-sb-1040.html
[4] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24594.pdf

Fixes 0c54914d0c52 ("KVM: x86: use Intel speculation bugs and features as derived in generic x86 code")

---
v2: Use IBPB_RET to identify semantic equality (Venkatesh)

 arch/x86/kvm/cpuid.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/cpuid.c +3 -1
index 2617be544480..044bdc9e938b 100644
@@ -690,7 +690,9 @@ void kvm_set_cpu_caps(void)
 	kvm_cpu_cap_set(X86_FEATURE_TSC_ADJUST);
 	kvm_cpu_cap_set(X86_FEATURE_ARCH_CAPABILITIES);
 
-	if (boot_cpu_has(X86_FEATURE_IBPB) && boot_cpu_has(X86_FEATURE_IBRS))
+	if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) &&
+	    boot_cpu_has(X86_FEATURE_AMD_IBPB) &&
+	    boot_cpu_has(X86_FEATURE_AMD_IBRS))
 		kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);
 	if (boot_cpu_has(X86_FEATURE_STIBP))
 		kvm_cpu_cap_set(X86_FEATURE_INTEL_STIBP);

arch/x86/kvm/cpuid.c void kvm_set_cpu_caps(void) · line 766

Quoted 7 lines
arch/x86/kvm/cpuid.c +2 -1
@@ -759,8 +761,10 @@ void kvm_set_cpu_caps(void)
 	 * arch/x86/kernel/cpu/bugs.c is kind enough to
 	 * record that in cpufeatures so use them.
 	 */
-	if (boot_cpu_has(X86_FEATURE_IBPB))
+	if (boot_cpu_has(X86_FEATURE_IBPB)) {
 		kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB);
+		kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);

Should IBPB_RET be conditionally set? I would think that you would only want to set IBPB_RET if either IBPB_RET or SPEC_CTRL is set on the hypervisor.

		if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) ||
		    boot_cpu_has(X86_FEATURE_SPEC_CTRL)
			kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);

Right?

Thanks,
Tom

Quoted 4 lines
arch/x86/kvm/cpuid.c +1 -0
+	}
 	if (boot_cpu_has(X86_FEATURE_IBRS))
 		kvm_cpu_cap_set(X86_FEATURE_AMD_IBRS);
 	if (boot_cpu_has(X86_FEATURE_STIBP))
#5 lore raw

Re: [PATCH v2 2/2] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

Quoted 65 lines · Tom Lendacky
Quoted 52 lines · Jim Mattson

From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26] enumerates support for indirect branch restricted speculation (IBRS) and the indirect branch predictor barrier (IBPB)." Further, from [2], "Software that executed before the IBPB command cannot control the predicted targets of indirect branches (4) executed after the command on the same logical processor," where footnote 4 reads, "Note that indirect branches include near call indirect, near jump indirect and near return instructions. Because it includes near returns, it follows that **RSB entries created before an IBPB command cannot control the predicted targets of returns executed after the command on the same logical processor.**" [emphasis mine]

On the other hand, AMD's IBPB "may not prevent return branch predictions from being specified by pre-IBPB branch targets" [3].

However, some AMD processors have an "enhanced IBPB" [terminology mine] which does clear the return address predictor. This feature is enumerated by CPUID.80000008:EDX.IBPB_RET[bit 30] [4].

Adjust the cross-vendor features enumerated by KVM_GET_SUPPORTED_CPUID accordingly.

[1] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
[2] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html#Footnotes
[3] https://www.amd.com/en/resources/product-security/bulletin/amd-sb-1040.html
[4] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24594.pdf

Fixes 0c54914d0c52 ("KVM: x86: use Intel speculation bugs and features as derived in generic x86 code")

---
v2: Use IBPB_RET to identify semantic equality (Venkatesh)

 arch/x86/kvm/cpuid.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/cpuid.c +3 -1
index 2617be544480..044bdc9e938b 100644
@@ -690,7 +690,9 @@ void kvm_set_cpu_caps(void)
      kvm_cpu_cap_set(X86_FEATURE_TSC_ADJUST);
      kvm_cpu_cap_set(X86_FEATURE_ARCH_CAPABILITIES);
 
-     if (boot_cpu_has(X86_FEATURE_IBPB) && boot_cpu_has(X86_FEATURE_IBRS))
+     if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) &&
+         boot_cpu_has(X86_FEATURE_AMD_IBPB) &&
+         boot_cpu_has(X86_FEATURE_AMD_IBRS))
              kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);
      if (boot_cpu_has(X86_FEATURE_STIBP))
              kvm_cpu_cap_set(X86_FEATURE_INTEL_STIBP);

arch/x86/kvm/cpuid.c void kvm_set_cpu_caps(void) · line 766

Quoted 7 lines
arch/x86/kvm/cpuid.c +2 -1
@@ -759,8 +761,10 @@ void kvm_set_cpu_caps(void)
       * arch/x86/kernel/cpu/bugs.c is kind enough to
       * record that in cpufeatures so use them.
       */
-     if (boot_cpu_has(X86_FEATURE_IBPB))
+     if (boot_cpu_has(X86_FEATURE_IBPB)) {
              kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB);
+             kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);

Should IBPB_RET be conditionally set? I would think that you would only want to set IBPB_RET if either IBPB_RET or SPEC_CTRL is set on the hypervisor.

Quoted 5 lines
                if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) ||
                    boot_cpu_has(X86_FEATURE_SPEC_CTRL)
                        kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);

Right?

Right. This clause is intended to set cross-vendor capabilities, so it should be:

    if (boot_cpu_has(X86_FEATURE_SPEC_CTRL))
        kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);

Passing through AMD_IBPB_RET from the hardware should be done by adding the bit to the mask for CPUID_8000_0008_EBX.

I'll send out a v3.

Thanks!

Quoted 7 lines

Thanks,
Tom

Quoted 4 lines
arch/x86/kvm/cpuid.c +1 -0
+     }
      if (boot_cpu_has(X86_FEATURE_IBRS))
              kvm_cpu_cap_set(X86_FEATURE_AMD_IBRS);
      if (boot_cpu_has(X86_FEATURE_STIBP))