Skip to contentSkip to conversation

x86/hvm: always expose x2APIC feature in max HVM cpuid policy

PATCH In review Roger Pau Monne 2 participants 6 messages Dec 24, 2019 – Jan 9, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/hvm: always expose x2APIC feature in max HVM cpuid policy

On hardware without x2APIC support Xen emulated local APIC will provide such mode, and hence the feature should be set in the maximum HVM cpuid policy.

Not exposing it in the maximum policy results in HVM domains not getting such feature exposed unless it's also supported by the underlying hardware.

---
This is a regression, but I'm not able to identify the commit that introduced the bogus behavior.

---
 xen/arch/x86/cpuid.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
xen/arch/x86/cpuid.c +3 -2
index 7055509ed6..b1ed33d524 100644
@@ -396,10 +396,11 @@ static void __init calculate_hvm_max_policy(void)
         hvm_featureset[i] &= hvm_featuremask[i];
 
     /*
-     * Xen can provide an APIC emulation to HVM guests even if the host's APIC
-     * isn't enabled.
+     * Xen can provide an (x2)APIC emulation to HVM guests even if the host's
+     * (x2)APIC isn't enabled.
      */
     __set_bit(X86_FEATURE_APIC, hvm_featureset);
+    __set_bit(X86_FEATURE_X2APIC, hvm_featureset);
 
     /*
      * On AMD, PV guests are entirely unable to use SYSENTER as Xen runs in
Signature · 7 lines
2.24.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw
Quoted 9 lines · Roger Pau Monne

On hardware without x2APIC support Xen emulated local APIC will provide such mode, and hence the feature should be set in the maximum HVM cpuid policy.

Not exposing it in the maximum policy results in HVM domains not getting such feature exposed unless it's also supported by the underlying hardware.

x2apic has never been exposed via this mechanism, due to its effects on topology calculations.

It has however always been down to the toolstack to opt in, and Xen permits this via recalculate_cpuid_policy(), on the expectation that the toolstack knew what it was doing WRT topology (all evidence actually to the contrary).

If we're seeing a recent change in behaviour, then it will be from libxc.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#2 lore raw
Quoted 5 lines · Andrew Cooper
Quoted 4 lines · Roger Pau Monne

On hardware without x2APIC support Xen emulated local APIC will provide such mode, and hence the feature should be set in the maximum HVM cpuid policy.

Quoted 8 lines
Quoted 5 lines

Not exposing it in the maximum policy results in HVM domains not getting such feature exposed unless it's also supported by the underlying hardware.

x2apic has never been exposed via this mechanism, due to its effects on topology calculations.

Well, it's exposed in hvm_max_cpuid_policy if it's present in the hardware. IMO it's kind of weird that the presence of the x2APIC feature on the max policy depends on the underlying hardware, when it's always supported by the emulated vlapic.

I think x2APIC must either always be part of the max policy, or never, or else it's very easy to cause regressions because it's not so easy to find a box without x2APIC.

Quoted 4 lines

It has however always been down to the toolstack to opt in, and Xen permits this via recalculate_cpuid_policy(), on the expectation that the toolstack knew what it was doing WRT topology (all evidence actually to the contrary).

Hm, I can try to force the setting in libxc.

Quoted 1 line

If we're seeing a recent change in behaviour, then it will be from libxc.

IIRC x2APIC was always exposed to HVM guests regardless of the underlying hardware support.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw
Quoted 13 lines · Roger Pau Monné
Quoted 5 lines · Andrew Cooper
Quoted 4 lines · Roger Pau Monne

On hardware without x2APIC support Xen emulated local APIC will provide such mode, and hence the feature should be set in the maximum HVM cpuid policy.

Quoted 7 lines
Quoted 5 lines

Not exposing it in the maximum policy results in HVM domains not getting such feature exposed unless it's also supported by the underlying hardware.

x2apic has never been exposed via this mechanism, due to its effects on topology calculations.

Quoted 8 lines

Well, it's exposed in hvm_max_cpuid_policy if it's present in the hardware. IMO it's kind of weird that the presence of the x2APIC feature on the max policy depends on the underlying hardware, when it's always supported by the emulated vlapic.

I think x2APIC must either always be part of the max policy, or never, or else it's very easy to cause regressions because it's not so easy to find a box without x2APIC.

Hmm - this does highlight the inconsistency in the existing logic.  I'm not overly surprised - this is a remnant of the old model which hasn't been rewritten yet.

Quoted 10 lines
Quoted 5 lines

It has however always been down to the toolstack to opt in, and Xen permits this via recalculate_cpuid_policy(), on the expectation that the toolstack knew what it was doing WRT topology (all evidence actually to the contrary).

Hm, I can try to force the setting in libxc.

Quoted 1 line

If we're seeing a recent change in behaviour, then it will be from libxc.

IIRC x2APIC was always exposed to HVM guests regardless of the underlying hardware support.

I have a suspicion that this may have been broken by c/s 3e0c8272f in 2015...

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw
Quoted 19 lines · Andrew Cooper
Quoted 18 lines · Roger Pau Monné
Quoted 5 lines · Andrew Cooper
Quoted 4 lines · Roger Pau Monne

On hardware without x2APIC support Xen emulated local APIC will provide such mode, and hence the feature should be set in the maximum HVM cpuid policy.

Quoted 7 lines
Quoted 5 lines

Not exposing it in the maximum policy results in HVM domains not getting such feature exposed unless it's also supported by the underlying hardware.

x2apic has never been exposed via this mechanism, due to its effects on topology calculations.

Well, it's exposed in hvm_max_cpuid_policy if it's present in the hardware. IMO it's kind of weird that the presence of the x2APIC feature on the max policy depends on the underlying hardware, when it's always supported by the emulated vlapic.

Quoted 7 lines
Quoted 3 lines

I think x2APIC must either always be part of the max policy, or never, or else it's very easy to cause regressions because it's not so easy to find a box without x2APIC.

Hmm - this does highlight the inconsistency in the existing logic.  I'm not overly surprised - this is a remnant of the old model which hasn't been rewritten yet.

I could do something like:

tools/libxc/xc_cpuid_x86.c +1 -0
index 519d6d8bd0..a7adc41854 100644
@@ -641,6 +641,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid,
         p->extd.itsc = true;
         p->basic.vmx = true;
         p->extd.svm = true;
+        p->basic.x2apic = true;
     }
 
     rc = x86_cpuid_copy_to_buffer(p, leaves, &nr_leaves);

But it seems kind of bogus to me that such feature is not part of the hvm_max_cpuid_policy, at the end of day the toolstack has no knowledge of whether the hypervisor provides a x2APIC interface or not (apart from us hardcoding it in the tools like the above patch).

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

It has however always been down to the toolstack to opt in, and Xen permits this via recalculate_cpuid_policy(), on the expectation that the toolstack knew what it was doing WRT topology (all evidence actually to the contrary).

Quoted 8 lines
Quoted 5 lines

Hm, I can try to force the setting in libxc.

Quoted 1 line

If we're seeing a recent change in behaviour, then it will be from libxc.

IIRC x2APIC was always exposed to HVM guests regardless of the underlying hardware support.

I have a suspicion that this may have been broken by c/s 3e0c8272f in 2015...

I could swear Xen was exposing the x2APIC CPUID feature on the box I'm currently testing during the 4.13 development cycle, or else I did hack it myself for development purposes and completely forgot to send a patch afterwards.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw
Quoted 43 lines · Roger Pau Monné
Quoted 19 lines · Andrew Cooper
Quoted 18 lines · Roger Pau Monné
Quoted 5 lines · Andrew Cooper
Quoted 4 lines · Roger Pau Monne

On hardware without x2APIC support Xen emulated local APIC will provide such mode, and hence the feature should be set in the maximum HVM cpuid policy.

Quoted 7 lines
Quoted 5 lines

Not exposing it in the maximum policy results in HVM domains not getting such feature exposed unless it's also supported by the underlying hardware.

x2apic has never been exposed via this mechanism, due to its effects on topology calculations.

Well, it's exposed in hvm_max_cpuid_policy if it's present in the hardware. IMO it's kind of weird that the presence of the x2APIC feature on the max policy depends on the underlying hardware, when it's always supported by the emulated vlapic.

Quoted 7 lines
Quoted 3 lines

I think x2APIC must either always be part of the max policy, or never, or else it's very easy to cause regressions because it's not so easy to find a box without x2APIC.

Hmm - this does highlight the inconsistency in the existing logic.  I'm not overly surprised - this is a remnant of the old model which hasn't been rewritten yet.

I could do something like:

tools/libxc/xc_cpuid_x86.c +1 -0
index 519d6d8bd0..a7adc41854 100644
@@ -641,6 +641,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid,
         p->extd.itsc = true;
         p->basic.vmx = true;
         p->extd.svm = true;
+        p->basic.x2apic = true;
     }
 
     rc = x86_cpuid_copy_to_buffer(p, leaves, &nr_leaves);
Quoted 4 lines

But it seems kind of bogus to me that such feature is not part of the hvm_max_cpuid_policy, at the end of day the toolstack has no knowledge of whether the hypervisor provides a x2APIC interface or not (apart from us hardcoding it in the tools like the above patch).

Ping?

I don't think we reached a conclusion as to whether x2APIC should be forced from the toolstack side or part of the HVM max policy.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel