Skip to contentSkip to conversation

x86/cpuid: Fix handling of the CPUID.7[0].eax levelling MSR

PATCH In review Andrew Cooper 3 participants 7 messages Sep 9, 2019 – Sep 10, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/cpuid: Fix handling of the CPUID.7[0].eax levelling MSR

7a0 is an integer field, not a mask - taking the logical and of the hardware and policy values results in nonsense. Instead, take the policy value directly.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>

Even Rome hardware has 7[0].eax still as 0, and there is no sensible reason to set max_subleaf higher at this point, so this is only a latent bug for now.

---
 xen/arch/x86/domctl.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
xen/arch/x86/domctl.c +9 -4
index 1e98fc8009..35ad8cb51c 100644
@@ -218,11 +218,16 @@ static int update_domain_cpuid_info(struct domain *d,
         if ( is_pv_domain(d) && ((levelling_caps & LCAP_7ab0) == LCAP_7ab0) )
         {
             uint64_t mask = cpuidmask_defaults._7ab0;
-            uint32_t eax = ctl->eax;
-            uint32_t ebx = p->feat._7b0;
 
-            if ( boot_cpu_data.x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
-                mask &= ((uint64_t)eax << 32) | ebx;
+            /*
+             * Leaf 7[0].eax is max_subleaf, not a feature mask.  Take it
+             * wholesale from the policy, but clamp the features in 7[0].ebx
+             * per usual.
+             */
+            if ( boot_cpu_data.x86_vendor &
+                 (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
+                mask = (((uint64_t)p->feat.max_subleaf << 32) |
+                        ((uint32_t)mask | p->feat._7b0));
 
             d->arch.pv.cpuidmasks->_7ab0 = mask;
         }
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw
Quoted 28 lines · Andrew Cooper

7a0 is an integer field, not a mask - taking the logical and of the hardware and policy values results in nonsense. Instead, take the policy value directly.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>

Even Rome hardware has 7[0].eax still as 0, and there is no sensible reason to set max_subleaf higher at this point, so this is only a latent bug for now.

---
 xen/arch/x86/domctl.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
xen/arch/x86/domctl.c +0 -3
index 1e98fc8009..35ad8cb51c 100644
@@ -218,11 +218,16 @@ static int update_domain_cpuid_info(struct domain *d,
         if ( is_pv_domain(d) && ((levelling_caps & LCAP_7ab0) == LCAP_7ab0) )
         {
             uint64_t mask = cpuidmask_defaults._7ab0;
-            uint32_t eax = ctl->eax;
-            uint32_t ebx = p->feat._7b0;
 
-            if ( boot_cpu_data.x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON) )

xen/arch/x86/domctl.c static int update_domain_cpuid_info(struct domain *d, · line 230

Quoted 10 lines
xen/arch/x86/domctl.c +9 -1
-                mask &= ((uint64_t)eax << 32) | ebx;
+            /*
+             * Leaf 7[0].eax is max_subleaf, not a feature mask.  Take it
+             * wholesale from the policy, but clamp the features in 7[0].ebx
+             * per usual.
+             */
+            if ( boot_cpu_data.x86_vendor &
+                 (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
+                mask = (((uint64_t)p->feat.max_subleaf << 32) |
+                        ((uint32_t)mask | p->feat._7b0));

Why do you set the high bits of the mask (63:30) with the max subleaf? According to the document I have bits 63:30 are reserved, and that seems to match the expected CPUID return value, that lists CPUID Fn0000_0007_EAX_x0 content as reserved.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#2 lore raw
Quoted 28 lines · Roger Pau Monné
Quoted 27 lines · Andrew Cooper

7a0 is an integer field, not a mask - taking the logical and of the hardware and policy values results in nonsense. Instead, take the policy value directly.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>

Even Rome hardware has 7[0].eax still as 0, and there is no sensible reason to set max_subleaf higher at this point, so this is only a latent bug for now.

---
 xen/arch/x86/domctl.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
xen/arch/x86/domctl.c +0 -2
index 1e98fc8009..35ad8cb51c 100644
@@ -218,11 +218,16 @@ static int update_domain_cpuid_info(struct domain *d,
         if ( is_pv_domain(d) && ((levelling_caps & LCAP_7ab0) == LCAP_7ab0) )
         {
             uint64_t mask = cpuidmask_defaults._7ab0;
-            uint32_t eax = ctl->eax;
-            uint32_t ebx = p->feat._7b0;
 
Quoted 12 lines

xen/arch/x86/domctl.c static int update_domain_cpuid_info(struct domain *d, · line 230

Quoted 11 lines
xen/arch/x86/domctl.c +9 -2
-            if ( boot_cpu_data.x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
-                mask &= ((uint64_t)eax << 32) | ebx;
+            /*
+             * Leaf 7[0].eax is max_subleaf, not a feature mask.  Take it
+             * wholesale from the policy, but clamp the features in 7[0].ebx
+             * per usual.
+             */
+            if ( boot_cpu_data.x86_vendor &
+                 (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
+                mask = (((uint64_t)p->feat.max_subleaf << 32) |
+                        ((uint32_t)mask | p->feat._7b0));

Why do you set the high bits of the mask (63:30) with the max subleaf?

63:32

Quoted 3 lines

According to the document I have bits 63:30 are reserved, and that seems to match the expected CPUID return value, that lists CPUID Fn0000_0007_EAX_x0 content as reserved.

Yes, but reserved doesn't mean "will #GP".  As I said on IRC, this MSR *is* the value which gets forwarded into a CPUID invocation, and setting max_subleaf to 0xdead does work fine.

The point here is that in the future, on hardware capable of max_subleaf=2 and being levelled to max_subleaf=1, the value observed in CPUID should be 1, not 0.

The latter is what the current logic does, and is broken.  The CPUID derivation logic will ensure that policy max_subleaf <= hardware max_subleaf.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Reviewed-by
#3 lore raw
Quoted 48 lines · Andrew Cooper
Quoted 28 lines · Roger Pau Monné
Quoted 27 lines · Andrew Cooper

7a0 is an integer field, not a mask - taking the logical and of the hardware and policy values results in nonsense. Instead, take the policy value directly.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>

Even Rome hardware has 7[0].eax still as 0, and there is no sensible reason to set max_subleaf higher at this point, so this is only a latent bug for now.

---
 xen/arch/x86/domctl.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
xen/arch/x86/domctl.c +0 -2
index 1e98fc8009..35ad8cb51c 100644
@@ -218,11 +218,16 @@ static int update_domain_cpuid_info(struct domain *d,
         if ( is_pv_domain(d) && ((levelling_caps & LCAP_7ab0) == LCAP_7ab0) )
         {
             uint64_t mask = cpuidmask_defaults._7ab0;
-            uint32_t eax = ctl->eax;
-            uint32_t ebx = p->feat._7b0;
 
Quoted 12 lines

xen/arch/x86/domctl.c static int update_domain_cpuid_info(struct domain *d, · line 230

Quoted 11 lines
xen/arch/x86/domctl.c +9 -2
-            if ( boot_cpu_data.x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
-                mask &= ((uint64_t)eax << 32) | ebx;
+            /*
+             * Leaf 7[0].eax is max_subleaf, not a feature mask.  Take it
+             * wholesale from the policy, but clamp the features in 7[0].ebx
+             * per usual.
+             */
+            if ( boot_cpu_data.x86_vendor &
+                 (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
+                mask = (((uint64_t)p->feat.max_subleaf << 32) |
+                        ((uint32_t)mask | p->feat._7b0));

Why do you set the high bits of the mask (63:30) with the max subleaf?

63:32

Quoted 3 lines

According to the document I have bits 63:30 are reserved, and that seems to match the expected CPUID return value, that lists CPUID Fn0000_0007_EAX_x0 content as reserved.

Quoted 7 lines

Yes, but reserved doesn't mean "will #GP".  As I said on IRC, this MSR *is* the value which gets forwarded into a CPUID invocation, and setting max_subleaf to 0xdead does work fine.

The point here is that in the future, on hardware capable of max_subleaf=2 and being levelled to max_subleaf=1, the value observed in CPUID should be 1, not 0.

Oh, and that's from Intel specification which does indeed list Fn0000_0007_EAX_x0 as returning the max sub-leaf, and the expectation is that AMD will follow suit.

Quoted 3 lines

The latter is what the current logic does, and is broken.  The CPUID derivation logic will ensure that policy max_subleaf <= hardware max_subleaf.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw
Quoted 51 lines · Roger Pau Monné
Quoted 50 lines · Andrew Cooper
Quoted 28 lines · Roger Pau Monné
Quoted 27 lines · Andrew Cooper

7a0 is an integer field, not a mask - taking the logical and of the hardware and policy values results in nonsense. Instead, take the policy value directly.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>

Even Rome hardware has 7[0].eax still as 0, and there is no sensible reason to set max_subleaf higher at this point, so this is only a latent bug for now.

---
 xen/arch/x86/domctl.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
xen/arch/x86/domctl.c +0 -2
index 1e98fc8009..35ad8cb51c 100644
@@ -218,11 +218,16 @@ static int update_domain_cpuid_info(struct domain *d,
         if ( is_pv_domain(d) && ((levelling_caps & LCAP_7ab0) == LCAP_7ab0) )
         {
             uint64_t mask = cpuidmask_defaults._7ab0;
-            uint32_t eax = ctl->eax;
-            uint32_t ebx = p->feat._7b0;
 
Quoted 12 lines

xen/arch/x86/domctl.c static int update_domain_cpuid_info(struct domain *d, · line 230

Quoted 11 lines
xen/arch/x86/domctl.c +9 -2
-            if ( boot_cpu_data.x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
-                mask &= ((uint64_t)eax << 32) | ebx;
+            /*
+             * Leaf 7[0].eax is max_subleaf, not a feature mask.  Take it
+             * wholesale from the policy, but clamp the features in 7[0].ebx
+             * per usual.
+             */
+            if ( boot_cpu_data.x86_vendor &
+                 (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
+                mask = (((uint64_t)p->feat.max_subleaf << 32) |
+                        ((uint32_t)mask | p->feat._7b0));

Why do you set the high bits of the mask (63:30) with the max subleaf?

63:32

Quoted 3 lines

According to the document I have bits 63:30 are reserved, and that seems to match the expected CPUID return value, that lists CPUID Fn0000_0007_EAX_x0 content as reserved.

Yes, but reserved doesn't mean "will #GP".  As I said on IRC, this MSR *is* the value which gets forwarded into a CPUID invocation, and setting max_subleaf to 0xdead does work fine.

Quoted 6 lines
Quoted 3 lines

The point here is that in the future, on hardware capable of max_subleaf=2 and being levelled to max_subleaf=1, the value observed in CPUID should be 1, not 0.

Oh, and that's from Intel specification which does indeed list Fn0000_0007_EAX_x0 as returning the max sub-leaf, and the expectation is that AMD will follow suit.

The AMD CPUID documentation does identify CPUID 7[0].eax as max_subleaf.  It just isn't cross referenced very well with the MSR.

Quoted 5 lines
Quoted 4 lines

The latter is what the current logic does, and is broken.  The CPUID derivation logic will ensure that policy max_subleaf <= hardware max_subleaf.

Thanks,

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Reviewed-by on v1 1/1
#5 lore raw
In reply to Andrew Cooper
Quoted 10 lines · Andrew Cooper
xen/arch/x86/domctl.c +0 -3
@@ -218,11 +218,16 @@ static int update_domain_cpuid_info(struct domain *d,
         if ( is_pv_domain(d) && ((levelling_caps & LCAP_7ab0) == LCAP_7ab0) )
         {
             uint64_t mask = cpuidmask_defaults._7ab0;
-            uint32_t eax = ctl->eax;
-            uint32_t ebx = p->feat._7b0;
 
-            if ( boot_cpu_data.x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON) )

xen/arch/x86/domctl.c static int update_domain_cpuid_info(struct domain *d, · line 230

Quoted 10 lines
xen/arch/x86/domctl.c +9 -1
-                mask &= ((uint64_t)eax << 32) | ebx;
+            /*
+             * Leaf 7[0].eax is max_subleaf, not a feature mask.  Take it
+             * wholesale from the policy, but clamp the features in 7[0].ebx
+             * per usual.
+             */
+            if ( boot_cpu_data.x86_vendor &
+                 (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
+                mask = (((uint64_t)p->feat.max_subleaf << 32) |
+                        ((uint32_t)mask | p->feat._7b0));

Assuming you mean & instead of | here, with it fixed

Otherwise I'm afraid I don't understand, and need further explanation.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#6 lore raw
Quoted 10 lines · Jan Beulich
Quoted 9 lines · Andrew Cooper
xen/arch/x86/domctl.c +0 -2
@@ -218,11 +218,16 @@ static int update_domain_cpuid_info(struct domain *d,
         if ( is_pv_domain(d) && ((levelling_caps & LCAP_7ab0) == LCAP_7ab0) )
         {
             uint64_t mask = cpuidmask_defaults._7ab0;
-            uint32_t eax = ctl->eax;
-            uint32_t ebx = p->feat._7b0;
 
Quoted 12 lines

xen/arch/x86/domctl.c static int update_domain_cpuid_info(struct domain *d, · line 230

Quoted 11 lines
xen/arch/x86/domctl.c +9 -2
-            if ( boot_cpu_data.x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
-                mask &= ((uint64_t)eax << 32) | ebx;
+            /*
+             * Leaf 7[0].eax is max_subleaf, not a feature mask.  Take it
+             * wholesale from the policy, but clamp the features in 7[0].ebx
+             * per usual.
+             */
+            if ( boot_cpu_data.x86_vendor &
+                 (X86_VENDOR_AMD | X86_VENDOR_HYGON) )
+                mask = (((uint64_t)p->feat.max_subleaf << 32) |
+                        ((uint32_t)mask | p->feat._7b0));

Assuming you mean & instead of | here, with it fixed

Oops - I do.

Quoted 1 line

Thanks,

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel