x86: CPUID leaf 6 consolidation
cover x86: CPUID leaf 6 consolidation
Integrate this leaf with CPU policy handling, to leverage the host policy when feature bits need evaluating.
1: cpu-policy: move invocation of recalculate_misc() 2: cpu-policy: define bits of leaf 6
3: include: de-assemblify cpuid.h
4: replace APERRMPERF synthetic feature bit 5: rename ARAT feature flag
6: Intel: use host CPU policy for ARAT checking 7: cpufreq: use host CPU policy for Turbo checking 8: cpufreq: use host CPU policy in HWP driver
Jan
PATCH 1/8 x86/cpu-policy: move invocation of recalculate_misc()
The function is about guest exposure of features / leaves. There's no need for it to be applied on the host policy. In fact doing so gets in the way of using the host policy in certain places.
xen/arch/x86/cpu-policy.c +2 -1
@@ -383,7 +383,6 @@ static void __init calculate_host_policyx86_cpu_featureset_to_policy (boot_cpu_data.x86_capability, p);recalculate_xstate (p); -recalculate_misc (p);/* When vPMU is disabled, drop it from the host policy. */ if ( vpmu_mode ==XENPMU_MODE_OFF )
@@ -634,6 +633,7 @@ static void __init calculate_pv_max_poliunsigned int i; *p = host_cpu_policy; +recalculate_misc (p);guest_common_max_leaves (p);
@@ -736,6 +736,7 @@ static void __init calculate_hvm_max_polconst uint32_t *mask; *p = host_cpu_policy; +recalculate_misc (p);guest_common_max_leaves (p);
Re: [PATCH 1/8] x86/cpu-policy: move invocation of recalculate_misc()
Quoted 5 lines · Jan Beulich
The function is about guest exposure of features / leaves. There's no need for it to be applied on the host policy. In fact doing so gets in the way of using the host policy in certain places.
Signed-off-by Jan Beulich <[email protected]>
I've been meaning to do this for a while. It has interfered with some debugging I've done in the past.
PATCH 2/8 x86/cpu-policy: define bits of leaf 6
In reply to Jan Beulich... as far as we presently use them in the codebase.
---
Or should we make both parts proper featureset elements? At least APERFMPERF could likely be made visible to guests (in principle).
xen/include/xen/lib/x86/cpu-policy.h +25 -1
@@ -121,7 +121,31 @@ struct cpu_policy uint64_t :64, :64; /* Leaf 0x3 - PSN. */ uint64_t :64, :64; /* Leaf 0x4 - Structured Cache. */ uint64_t :64, :64; /* Leaf 0x5 - MONITOR. */ - uint64_t :64, :64; /* Leaf 0x6 - Therm/Perf. */ + +/* Leaf 0x6 - Therm/Perf. */ +struct { + uint32_t /* a */:1, + turbo:1 , + arat:1, + :4 , + hwp:1 , + hwp_notification:1 , + hwp_activity_window:1 , + hwp_epp:1 , + hwp_plr:1, + :1 , + hdc:1, + :2 , + hwp_peci:1, + :2 , + hw_feedback:1, + :12 ; + uint32_t /* b */:32; + uint32_t /* c */ aperfmperf:1, + :31 ; + uint32_t /* d */:32; + } pm; + uint64_t :64, :64; /* Leaf 0x7 - Structured Features. */ uint64_t :64, :64; /* Leaf 0x8 - rsvd */ uint64_t :64, :64; /* Leaf 0x9 - DCA */
Re: [PATCH 2/8] x86/cpu-policy: define bits of leaf 6
Quoted 6 lines · Jan Beulich
... as far as we presently use them in the codebase.
Signed-off-by Jan Beulich <[email protected]>---
Or should we make both parts proper featureset elements? At least APERFMPERF could likely be made visible to guests (in principle).
As before, A/MPERF can't be used safely by a VM.
In order to be persuaded to offer this to VMs, someone is going to have to present a mechanism for how a VM could even figure out that it read junk from the regsiters...
Quoted 21 lines
xen/include/xen/lib/x86/cpu-policy.h +13
-1@@ -121,7 +121,31 @@ struct cpu_policy uint64_t :64, :64; /* Leaf 0x3 - PSN. */ uint64_t :64, :64; /* Leaf 0x4 - Structured Cache. */ uint64_t :64, :64; /* Leaf 0x5 - MONITOR. */ - uint64_t :64, :64; /* Leaf 0x6 - Therm/Perf. */ + +/* Leaf 0x6 - Therm/Perf. */ +struct { + uint32_t /* a */:1, + turbo:1 , + arat:1, + :4 , + hwp:1 , + hwp_notification:1 , + hwp_activity_window:1 , + hwp_epp:1 , + hwp_plr:1, + :1,
xen/include/xen/lib/x86/cpu-policy.h struct cpu_policy · line 146
Quoted 10 lines
xen/include/xen/lib/x86/cpu-policy.h +10
-0+ hdc:1, + :2, + hwp_peci:1, + :2, + hw_feedback:1, + :12 ; + uint32_t /* b */:32; + uint32_t /* c */ aperfmperf:1, + :31; + uint32_t /* d */:32;
Elsewhere, single bit fields are bool foo:1, and these want to match for consistency. In particular using uint32_t:1 creates a latent bug in patch 8.
One problem with bool bitfields is that your :4 needs to become 4x :1. Right now his hidden in the macros that gen-cpuid.py makes.
Given that b is of type uint32_t, you can omit the :12 from the end of a and leave a comment. Similarly, the trailing :31 on c can be dropped.
xen/include/xen/lib/x86/cpu-policy.h struct cpu_policy · line 147
Quoted 1 line
xen/include/xen/lib/x86/cpu-policy.h +1
-0+ } pm;
Nothing else is sub-scoped. I'd prefer that you drop the 'pm'.
~Andrew
Re: [PATCH 2/8] x86/cpu-policy: define bits of leaf 6
Quoted 26 lines · Andrew Cooper
Quoted 25 lines · Jan Beulich
xen/include/xen/lib/x86/cpu-policy.h +18
-1@@ -121,7 +121,31 @@ struct cpu_policy uint64_t :64, :64; /* Leaf 0x3 - PSN. */ uint64_t :64, :64; /* Leaf 0x4 - Structured Cache. */ uint64_t :64, :64; /* Leaf 0x5 - MONITOR. */ - uint64_t :64, :64; /* Leaf 0x6 - Therm/Perf. */ + +/* Leaf 0x6 - Therm/Perf. */ +struct { + uint32_t /* a */:1, + turbo:1 , + arat:1, + :4 , + hwp:1 , + hwp_notification:1 , + hwp_activity_window:1 , + hwp_epp:1 , + hwp_plr:1, + :1 , + hdc:1, + :2 , + hwp_peci:1, + :2 , + hw_feedback:1,
Quoted 8 lines
xen/include/xen/lib/x86/cpu-policy.h struct cpu_policy · line 146
Quoted 5 lines
xen/include/xen/lib/x86/cpu-policy.h +5
-0+ :12; + uint32_t /* b */:32; + uint32_t /* c */ aperfmperf:1, + :31; + uint32_t /* d */:32;Elsewhere, single bit fields are bool foo:1, and these want to match for consistency.
Oh, yes, will change.
Quoted 2 lines
In particular using uint32_t:1 creates a latent bug in patch 8.
I don't see where that would be.
Quoted 5 lines
One problem with bool bitfields is that your :4 needs to become 4x :1. Right now his hidden in the macros that
gen-cpuid.pymakes.Given that b is of type
uint32_t, you can omit the :12 from the end of a and leave a comment. Similarly, the trailing :31 on c can be dropped.
We have these in many other places, and omitting in particular the :31 would also feel somewhat fragile / misleading. It'll need to be
bool /* c */ aperfmperf:1;
uint32_t :31;or something along these lines.
Quoted 3 lines
xen/include/xen/lib/x86/cpu-policy.h struct cpu_policy · line 147
Quoted 1 line
xen/include/xen/lib/x86/cpu-policy.h +1
-0+ } pm;Nothing else is sub-scoped. I'd prefer that you drop the 'pm'.
Wouldn't that require the use of the very extension you just talked about at the committer's call?
Jan
Re: [PATCH 2/8] x86/cpu-policy: define bits of leaf 6
Quoted 34 lines · Jan Beulich
Quoted 25 lines · Andrew Cooper
Quoted 24 lines · Jan Beulich
xen/include/xen/lib/x86/cpu-policy.h +17
-1@@ -121,7 +121,31 @@ struct cpu_policy uint64_t :64, :64; /* Leaf 0x3 - PSN. */ uint64_t :64, :64; /* Leaf 0x4 - Structured Cache. */ uint64_t :64, :64; /* Leaf 0x5 - MONITOR. */ - uint64_t :64, :64; /* Leaf 0x6 - Therm/Perf. */ + +/* Leaf 0x6 - Therm/Perf. */ +struct { + uint32_t /* a */:1, + turbo:1 , + arat:1, + :4 , + hwp:1 , + hwp_notification:1 , + hwp_activity_window:1 , + hwp_epp:1 , + hwp_plr:1, + :1 , + hdc:1, + :2 , + hwp_peci:1, + :2,Quoted 8 lines
xen/include/xen/lib/x86/cpu-policy.h struct cpu_policy · line 146
Quoted 6 lines
xen/include/xen/lib/x86/cpu-policy.h +6
-0+ hw_feedback:1 , + :12; + uint32_t /* b */:32; + uint32_t /* c */ aperfmperf:1, + :31; + uint32_t /* d */:32;Elsewhere, single bit fields are bool foo:1, and these want to match for consistency.
Quoted 5 lines
Oh, yes, will change.
Quoted 2 lines
In particular using uint32_t:1 creates a latent bug in patch 8.
I don't see where that would be.
In the printf. %d vs %u. Latent because it's ok until bit 31 gets used, and then it's not ok.
Quoted 6 lines
Quoted 6 lines
One problem with bool bitfields is that your :4 needs to become 4x :1. Right now his hidden in the macros that
gen-cpuid.pymakes.Given that b is of type
uint32_t, you can omit the :12 from the end of a and leave a comment. Similarly, the trailing :31 on c can be dropped.
Quoted 7 lines
We have these in many other places, and omitting in particular the :31 would also feel somewhat fragile / misleading. It'll need to be
bool /* c */ aperfmperf:1; uint32_t :31;or something along these lines.
This doesn't work. A gap of 31 bits gets inserted because of uint32_t's alignment, which is why the suggestion to ignore it does work (even if fragile).
I suggest a /* 31 spare bits */ comment, because the only other option is 31x :1's.
Quoted 5 lines
Quoted 3 lines
xen/include/xen/lib/x86/cpu-policy.h struct cpu_policy · line 147
Quoted 2 lines
xen/include/xen/lib/x86/cpu-policy.h +1
-0+ } pm;Nothing else is sub-scoped. I'd prefer that you drop the 'pm'.
Wouldn't that require the use of the very extension you just talked about at the committer's call?
No. It would just be a plain anonymous struct in this case, but it doesn't even need to be a struct.
leaf 0,2,10 are all "top level" insofar as they're all inside .basic. leaf 1 only has anonymous unions to join the the bitfield names and the field-wide name.
~Andrew
Re: [PATCH 2/8] x86/cpu-policy: define bits of leaf 6
Quoted 35 lines · Andrew Cooper
Quoted 34 lines · Jan Beulich
Quoted 33 lines · Andrew Cooper
Quoted 20 lines · Jan Beulich
xen/include/xen/lib/x86/cpu-policy.h +13
-1@@ -121,7 +121,31 @@ struct cpu_policy uint64_t :64, :64; /* Leaf 0x3 - PSN. */ uint64_t :64, :64; /* Leaf 0x4 - Structured Cache. */ uint64_t :64, :64; /* Leaf 0x5 - MONITOR. */ - uint64_t :64, :64; /* Leaf 0x6 - Therm/Perf. */ + +/* Leaf 0x6 - Therm/Perf. */ +struct { + uint32_t /* a */:1, + turbo:1 , + arat:1, + :4 , + hwp:1 , + hwp_notification:1 , + hwp_activity_window:1 , + hwp_epp:1 , + hwp_plr:1, + :1,xen/include/xen/lib/x86/cpu-policy.h struct cpu_policy · line 146
Quoted 10 lines
xen/include/xen/lib/x86/cpu-policy.h +10
-0+ hdc:1, + :2, + hwp_peci:1, + :2, + hw_feedback:1, + :12 ; + uint32_t /* b */:32; + uint32_t /* c */ aperfmperf:1, + :31; + uint32_t /* d */:32;Elsewhere, single bit fields are bool foo:1, and these want to match for consistency.
Quoted 8 lines
Quoted 5 lines
Oh, yes, will change.
Quoted 2 lines
In particular using uint32_t:1 creates a latent bug in patch 8.
I don't see where that would be.
In the printf. %d vs %u. Latent because it's ok until bit 31 gets used, and then it's not ok.
How would there be a difference? Every bit is individually unsigned when the field type is uint32_t, so even bit 31 will cleanly produce 0 / 1 when read.
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
One problem with bool bitfields is that your :4 needs to become 4x :1. Right now his hidden in the macros that
gen-cpuid.pymakes.Given that b is of type
uint32_t, you can omit the :12 from the end of a and leave a comment. Similarly, the trailing :31 on c can be dropped.We have these in many other places, and omitting in particular the :31 would also feel somewhat fragile / misleading. It'll need to be
Quoted 8 lines
Quoted 4 lines
bool /* c */ aperfmperf:1; uint32_t :31;or something along these lines.
This doesn't work. A gap of 31 bits gets inserted because of uint32_t's alignment, which is why the suggestion to ignore it does work (even if fragile).
Interesting. When (iirc) we converted the AMD IOMMU machinery to use struct-s, it was me to be concerned of this, and you telling me the opposite of what you say now. See how there's no problem in e.g. struct amd_iommu_dte.
Quoted 2 lines
I suggest a /* 31 spare bits */ comment, because the only other option is 31x :1's.
I'll do this differently anyway, ..
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
xen/include/xen/lib/x86/cpu-policy.h struct cpu_policy · line 147
Quoted 1 line
xen/include/xen/lib/x86/cpu-policy.h +1
-0+ } pm;Nothing else is sub-scoped. I'd prefer that you drop the 'pm'.
Wouldn't that require the use of the very extension you just talked about at the committer's call?
No. It would just be a plain anonymous struct in this case, but it doesn't even need to be a struct.
... having realized over night that this would be what you ask for, and ...
Quoted 3 lines
leaf 0,2,10 are all "top level" insofar as they're all inside .basic. leaf 1 only has anonymous unions to join the the bitfield names and the field-wide name.
... getting things closer to what we use there:
union {
uint32_t _6c;
struct { ... };
};Then I can safely and obviously omit any unused tail bits in the bitfield set. I hope that's going to be okay with you.
Jan
PATCH 3/8 x86/include: de-assemblify cpuid.h
In reply to Jan BeulichThere's nothing in that header which assembly files would need. Arrange for cpufeatureset.h to be included directly by cpufeature.h in that case, allowing respective guarding to be dropped
xen/arch/x86/include/asm/cpufeature.h +4 -0
@@ -10,7 +10,11 @@#include <xen/const.h> #include <xen/macros.h> +#ifndef __ASSEMBLY__#include <asm/cpuid.h> +#else +#include <asm/cpufeatureset.h> +#endif #define cpufeat_word (idx) ((idx) / 32)#define cpufeat_bit (idx) ((idx) % 32)
xen/arch/x86/include/asm/cpuid.h +0 -2
@@ -3,7 +3,6 @@#include <asm/cpufeatureset.h> -#ifndef __ASSEMBLY__#include <xen/types.h> #include <xen/kernel.h> #include <xen/percpu.h>
@@ -50,7 +49,6 @@ struct cpuid_leaf;void guest_cpuid (const struct vcpu *v,uint32_t leaf,uint32_t subleaf,struct cpuid_leaf *res); -#endif /* __ASSEMBLY__ */ #endif /* !__X86_CPUID_H__ */ /*
Re: [PATCH 3/8] x86/include: de-assemblify cpuid.h
Quoted 3 lines · Jan Beulich
There's nothing in that header which assembly files would need. Arrange for
cpufeatureset.hto be included directly bycpufeature.hin that case, allowing respective guarding to be dropped
Missing full stop.
Quoted 2 lines
Signed-off-by Jan Beulich <[email protected]>
PATCH 4/8 x86: replace APERFMPERF synthetic feature bit
In reply to Jan BeulichUse the respective host CPU policy bit instead.
xen/arch/x86/cpu/common.c +0 -4
@@ -523,10 +523,6 @@ static void generic_identify(struct cpuiif (cpu_has (c,X86_FEATURE_CLFLUSH ) ) c->x86_clflush_size = ((ebx >>8 ) &0xff ) *8 ; -if ( (c->cpuid_level >=CPUID_PM_LEAF ) && - (cpuid_ecx (CPUID_PM_LEAF ) &CPUID6_ECX_APERFMPERF_CAPABILITY ) ) -__set_bit (X86_FEATURE_APERFMPERF , c->x86_capability); -/* AMD-defined flags: level 0x80000001 */ if (c->extended_cpuid_level >=0x80000001 ) cpuid(0x80000001, &tmp, &tmp,
xen/arch/x86/include/asm/cpufeature.h +5 -2
@@ -11,7 +11,9 @@#include <xen/macros.h> #ifndef __ASSEMBLY__ +#include <asm/cpu-policy.h> #include <asm/cpuid.h> +#include <xen/lib/x86/cpu-policy.h> #else #include <asm/cpufeatureset.h> #endif
@@ -121,7 +123,6 @@ static inline bool boot_cpu_has(unsigned#define CPUID6_EAX_HDC BIT(13, U)#define CPUID6_EAX_HWP_PECI BIT(16, U)#define CPUID6_EAX_HW_FEEDBACK BIT(19, U) -#define CPUID6_ECX_APERFMPERF_CAPABILITY BIT(0, U)/* CPUID level 0x00000001.edx */ #define cpu_has_fpu 1
@@ -175,6 +176,9 @@ static inline bool boot_cpu_has(unsigned#define cpu_has_fma4 boot_cpu_has(X86_FEATURE_FMA4)#define cpu_has_tbm boot_cpu_has(X86_FEATURE_TBM) +/* CPUID level 0x00000006.ecx */ +#define cpu_has_aperfmperf host_cpu_policy.basic.pm.aperfmperf +/* CPUID level 0x0000000D:1.eax */ #define cpu_has_xsaveopt boot_cpu_has(X86_FEATURE_XSAVEOPT)#define cpu_has_xsavec boot_cpu_has(X86_FEATURE_XSAVEC)
@@ -292,7 +296,6 @@ static inline bool boot_cpu_has(unsigned/* Synthesized. */ #define cpu_has_arch_perfmon boot_cpu_has(X86_FEATURE_ARCH_PERFMON)#define cpu_has_cpuid_faulting boot_cpu_has(X86_FEATURE_CPUID_FAULTING) -#define cpu_has_aperfmperf boot_cpu_has(X86_FEATURE_APERFMPERF)#define cpu_has_xen_lbr boot_cpu_has(X86_FEATURE_XEN_LBR)#define cpu_has_xen_shstk (IS_ENABLED(CONFIG_XEN_SHSTK) && \ boot_cpu_has(X86_FEATURE_XEN_SHSTK))
xen/arch/x86/include/asm/cpufeatures.h +1 -1
@@ -19,7 +19,7 @@ XEN_CPUFEATURE(TSC_RELIABLE, X86_SY XEN_CPUFEATURE(XTOPOLOGY, X86_SYNTH( 5)) /* cpu topology enum extensions */ XEN_CPUFEATURE(CPUID_FAULTING, X86_SYNTH( 6)) /* cpuid faulting */XEN_CPUFEATURE (XEN_FRED ,X86_SYNTH (7 ))/* Xen uses FRED */ -XEN_CPUFEATURE(APERFMPERF, X86_SYNTH( 8)) /* APERFMPERF */ +/* Bit 8 unused */ XEN_CPUFEATURE(MFENCE_RDTSC, X86_SYNTH( 9)) /* MFENCE synchronizes RDTSC */ XEN_CPUFEATURE(XEN_SMEP, X86_SYNTH(10)) /* SMEP gets used by Xen itself */XEN_CPUFEATURE (XEN_SMAP ,X86_SYNTH (11 ))/* SMAP gets used by Xen itself */
Re: [PATCH 4/8] x86: replace APERFMPERF synthetic feature bit
Quoted 3 lines · Jan Beulich
Use the respective host CPU policy bit instead.
Signed-off-by Jan Beulich <[email protected]>
Right now, the synthetic features get levelled across the system. Now, we take the BSP's copy.
This change is broadly fine, but it does need mentioning in the commit message.
One thing we may want to do is take greater care to get the masking/levelling MSRs properly level. Right now, if they're asymmetric for any reason, we would previously end up using the common subset.
Quoted 16 lines
xen/arch/x86/cpu/common.c +0
-4@@ -523,10 +523,6 @@ static void generic_identify(struct cpuiif (cpu_has (c,X86_FEATURE_CLFLUSH ) ) c->x86_clflush_size = ((ebx >>8 ) &0xff ) *8 ; -if ( (c->cpuid_level >=CPUID_PM_LEAF ) && - (cpuid_ecx (CPUID_PM_LEAF ) &CPUID6_ECX_APERFMPERF_CAPABILITY ) ) -__set_bit (X86_FEATURE_APERFMPERF , c->x86_capability); -/* AMD-defined flags: level 0x80000001 */ if (c->extended_cpuid_level >=0x80000001 ) cpuid(0x80000001, &tmp, &tmp,
xen/arch/x86/include/asm/cpufeature.h line 16
Quoted 6 lines
xen/arch/x86/include/asm/cpufeature.h +2
-0@@ -11,7 +11,9 @@#include <xen/macros.h> #ifndef __ASSEMBLY__ +#include <asm/cpu-policy.h> #include <asm/cpuid.h> +#include <xen/lib/x86/cpu-policy.h>
Why both?
~Andrew
Re: [PATCH 4/8] x86: replace APERFMPERF synthetic feature bit
Quoted 10 lines · Andrew Cooper
Quoted 3 lines · Jan Beulich
Use the respective host CPU policy bit instead.
Signed-off-by Jan Beulich <[email protected]>Right now, the synthetic features get levelled across the system. Now, we take the BSP's copy.
This change is broadly fine, but it does need mentioning in the commit message.
"Use the respective host CPU policy bit instead. This has the (tolerable,
as we generally assume symmetry anyway) effect of using the BSP's (unleveled) setting, rather than the result of leveling (as is done by identify_cpu() on boot_cpu_data, rendering the variable name somewhat misleading)."
?
Quoted 3 lines
One thing we may want to do is take greater care to get the masking/levelling MSRs properly level. Right now, if they're asymmetric for any reason, we would previously end up using the common subset.
Possibly; I'd prefer to leave that to you, though.
Quoted 11 lines
xen/arch/x86/include/asm/cpufeature.h line 16
Quoted 9 lines
xen/arch/x86/include/asm/cpufeature.h +2
-0@@ -11,7 +11,9 @@#include <xen/macros.h> #ifndef __ASSEMBLY__ +#include <asm/cpu-policy.h> #include <asm/cpuid.h> +#include <xen/lib/x86/cpu-policy.h> Why both?
The former declares host_cpu_policy, while the latter defines struct cpu_policy. And neither of the two #include-s the other, afaics.
Jan
PATCH 5/8 x86: rename ARAT feature flag
In reply to Jan BeulichAdd a XEN infix, to properly distinguish it from the CPUID feature flag (leaf 6 EAX bit 2).
---
Question is whether we still need opt_arat (and the command line option), or whether we could go directly from the CPUID bit (overriding it to on for older AMD [and Hygon?] CPUs). Or whether to have opt_arat affect the (host) CPU policy directly.
xen/arch/x86/acpi/cpu_idle.c +2 -2
@@ -109,7 +109,7 @@ void (*__read_mostly lapic_timer_on)(voibool lapic_timer_init (void ) { -if (boot_cpu_has (X86_FEATURE_ARAT ) ) +if (boot_cpu_has (X86_FEATURE_XEN_ARAT ) ) { lapic_timer_off = lapic_timer_nop; lapic_timer_on = lapic_timer_nop;
@@ -1463,7 +1463,7 @@ static void amd_cpuidle_init(struct acpiif ( !vendor_override ) { -if ( !boot_cpu_has (X86_FEATURE_ARAT ) ) +if ( !boot_cpu_has (X86_FEATURE_XEN_ARAT ) )hpet_broadcast_init ();if ( !lapic_timer_init () )
xen/arch/x86/cpu/amd.c +1 -1
@@ -1355,7 +1355,7 @@ static void cf_check init_amd(struct cpu* running in deep C states. */ if ( opt_arat && c->x86 >0x11 ) -__set_bit (X86_FEATURE_ARAT , c->x86_capability); +__set_bit (X86_FEATURE_XEN_ARAT , c->x86_capability);/* * Prior to Family 0x14, perf counters are not reset during warm reboot.
xen/arch/x86/cpu/hygon.c +1 -1
@@ -76,7 +76,7 @@ static void cf_check init_hygon(struct c/* Hygon processors have APIC timer running in deep C states. */ if (opt_arat) -__set_bit (X86_FEATURE_ARAT , c->x86_capability); +__set_bit (X86_FEATURE_XEN_ARAT , c->x86_capability);if (cpu_has (c,X86_FEATURE_EFRO )) {rdmsrl (MSR_K8_HWCR , value);
xen/arch/x86/cpu/intel.c +1 -1
@@ -668,7 +668,7 @@ static void cf_check init_intel(struct cif ( opt_arat && ( c->cpuid_level >=0x00000006 ) && (cpuid_eax (0x00000006 ) & (1u <<2 ) ) ) -__set_bit (X86_FEATURE_ARAT , c->x86_capability); +__set_bit (X86_FEATURE_XEN_ARAT , c->x86_capability);if ((opt_cpu_info && !(c->apicid & (c->x86_num_siblings -1 ))) || c == &boot_cpu_data )
xen/arch/x86/cpu/mwait-idle.c +2 -2
@@ -1633,7 +1633,7 @@ static int __init mwait_idle_probe(void) icpu = id->driver_data; cpuidle_state_table = icpu->state_table; -if (boot_cpu_has (X86_FEATURE_ARAT )) +if (boot_cpu_has (X86_FEATURE_XEN_ARAT )) lapic_timer_reliable_states =LAPIC_TIMER_ALWAYS_RELIABLE ; pr_debug(PREFIX "v" MWAIT_IDLE_VERSION " model %#x\n",
@@ -1792,7 +1792,7 @@ int __init mwait_idle_init(struct notifireturn -ENODEV ; err =mwait_idle_probe (); -if (!err && !boot_cpu_has (X86_FEATURE_ARAT )) { +if (!err && !boot_cpu_has (X86_FEATURE_XEN_ARAT )) {hpet_broadcast_init ();if (xen_cpuidle <0 && !hpet_broadcast_is_available ()) err = -ENODEV ;
xen/arch/x86/include/asm/cpufeatures.h +1 -1
@@ -13,7 +13,7 @@/* Synthetic features */ XEN_CPUFEATURE(CONSTANT_TSC, X86_SYNTH( 0)) /* TSC ticks at a constant rate */ XEN_CPUFEATURE(NONSTOP_TSC, X86_SYNTH( 1)) /* TSC does not stop in C states */ -XEN_CPUFEATURE(ARAT, X86_SYNTH( 2)) /* Always running APIC timer */ +XEN_CPUFEATURE(XEN_ARAT, X86_SYNTH( 2)) /* Xen may utilize always running APIC timer */ XEN_CPUFEATURE(ARCH_PERFMON, X86_SYNTH( 3)) /* Intel Architectural PerfMon */ XEN_CPUFEATURE(TSC_RELIABLE, X86_SYNTH( 4)) /* TSC is known to be reliable */XEN_CPUFEATURE (XTOPOLOGY ,X86_SYNTH (5 ))/* cpu topology enum extensions */
xen/arch/x86/time.c +1 -1
@@ -2642,7 +2642,7 @@ static int _disable_pit_irq(bool init)* XXX dom0 may rely on RTC interrupt delivery, so only enable * hpet_broadcast if FSB mode available or if force_hpet_broadcast. */ -if (cpuidle_using_deep_cstate () && !boot_cpu_has (X86_FEATURE_ARAT ) ) +if (cpuidle_using_deep_cstate () && !boot_cpu_has (X86_FEATURE_XEN_ARAT ) ) { init ?hpet_broadcast_init () :hpet_broadcast_resume ();if ( !hpet_broadcast_is_available () )
Re: [PATCH 5/8] x86: rename ARAT feature flag
Quoted 9 lines · Jan Beulich
Add a XEN infix, to properly distinguish it from the CPUID feature flag (leaf 6 EAX bit 2).
Signed-off-by Jan Beulich <[email protected]>---
Question is whether we still needopt_arat(and the command line option), or whether we could go directly from the CPUID bit (overriding it to on for older AMD [and Hygon?] CPUs). Or whether to haveopt_arataffect the (host) CPU policy directly.
I have no particular love for opt_arat. I added it when there was no way to force a system to fall back to an older timer.
As long as we've got some way of configuring the exact timer to, I'm not fussed about keeping the top level boolean.
~Andrew
Re: [PATCH 5/8] x86: rename ARAT feature flag
Quoted 11 lines · Andrew Cooper
Quoted 9 lines · Jan Beulich
Add a XEN infix, to properly distinguish it from the CPUID feature flag (leaf 6 EAX bit 2).
Signed-off-by Jan Beulich <[email protected]>---
Question is whether we still needopt_arat(and the command line option), or whether we could go directly from the CPUID bit (overriding it to on for older AMD [and Hygon?] CPUs). Or whether to haveopt_arataffect the (host) CPU policy directly.
Quoted 5 lines
I have no particular love for opt_arat. I added it when there was no way to force a system to fall back to an older timer.
As long as we've got some way of configuring the exact timer to, I'm not fussed about keeping the top level boolean.
But the question is what the replacement would look like. Is it okay to override bits to true in the host policy? (It may be as long as we don't expose this in the guest policies, but at some point we may want or even need to.) Would the command line option also go away (in favor of integrating with cpuid=, which would require extra logic as long as we don't put leaf 6 bits in public/arch-x86/cpufeatureset.h)?
IOW for the patch at hand I wouldn't mean to make adjustments. The remark was more towards further, future adjustments.
Jan
PATCH 6/8 x86/Intel: use host CPU policy for ARAT checking
In reply to Jan BeulichThere's no need to invoke CPUID yet another time. However, as the host CPU policy is set up only shortly after init_intel() ran on the BSP, defer the logic to a pre-SMP initcall. This can't be (a new) one in cpu/intel.c though, as that's linked after acpi/cpu_idle.c (which is where we already need the feature set). Since opt_arat is local to the cpu/ subtree, introduce a new Intel-specific helper to hold the code needed.
Further, as we assume symmetry anyway, use setup_force_cpu_cap() and hence limit the checking to the boot CPU.
---
The need to move where cpu_has_arat is checked would go away if we did away with opt_arat (as mentioned in the previous patch), and hence could use cpu_has_arat directly where right now XEN_ARAT is checked.
xen/arch/x86/acpi/cpu_idle.c +3 -0
@@ -1666,6 +1666,9 @@ static int __init cf_check cpuidle_presm {void *cpu = (void *)(long )smp_processor_id (); +if ( boot_cpu_data.vendor ==X86_VENDOR_INTEL ) +intel_init_arat (); +if ( !xen_cpuidle )return 0 ;
xen/arch/x86/cpu/intel.c +6 -4
@@ -665,10 +665,6 @@ static void cf_check init_intel(struct c__set_bit (X86_FEATURE_NONSTOP_TSC , c->x86_capability);__set_bit (X86_FEATURE_TSC_RELIABLE , c->x86_capability); } -if ( opt_arat && - ( c->cpuid_level >=0x00000006 ) && - (cpuid_eax (0x00000006 ) & (1u <<2 ) ) ) -__set_bit (X86_FEATURE_XEN_ARAT , c->x86_capability);if ((opt_cpu_info && !(c->apicid & (c->x86_num_siblings -1 ))) || c == &boot_cpu_data )
@@ -690,3 +686,9 @@ const struct cpu_dev __initconst_cf_clob .c_early_init = early_init_intel, .c_init = init_intel, }; + +void __init intel_init_arat (void ) +{ +if ( opt_arat && cpu_has_arat ) +setup_force_cpu_cap (X86_FEATURE_XEN_ARAT ); +}
xen/arch/x86/include/asm/cpufeature.h +3 -0
@@ -176,6 +176,9 @@ static inline bool boot_cpu_has(unsigned#define cpu_has_fma4 boot_cpu_has(X86_FEATURE_FMA4)#define cpu_has_tbm boot_cpu_has(X86_FEATURE_TBM) +/* CPUID level 0x00000006.eax */ +#define cpu_has_arat host_cpu_policy.basic.pm.arat +/* CPUID level 0x00000006.ecx */ #define cpu_has_aperfmperf host_cpu_policy.basic.pm.aperfmperf
xen/arch/x86/include/asm/processor.h +1 -0
@@ -102,6 +102,7 @@ extern void setup_force_cpu_cap(unsignedextern bool is_forced_cpu_cap (unsigned int cap);extern void print_cpu_info (unsigned int cpu);extern void init_intel_cacheinfo (struct cpuinfo_x86 *c); +extern void intel_init_arat (void );#define cpu_to_core (_cpu) (cpu_data[_cpu].cpu_core_id)#define cpu_to_socket (_cpu) (cpu_data[_cpu].phys_proc_id)
Re: [PATCH 6/8] x86/Intel: use host CPU policy for ARAT checking
Quoted 19 lines · Jan Beulich
There's no need to invoke CPUID yet another time. However, as the host CPU policy is set up only shortly after
init_intel()ran on the BSP, defer the logic to a pre-SMP initcall. This can't be (a new) one incpu/intel.cthough, as that's linked afteracpi/cpu_idle.c(which is where we already need the feature set). Sinceopt_aratis local to the cpu/ subtree, introduce a new Intel-specific helper to hold the code needed.Further, as we assume symmetry anyway, use
setup_force_cpu_cap()and hence limit the checking to the boot CPU.Signed-off-by Jan Beulich <[email protected]>---
The need to move wherecpu_has_aratis checked would go away if we did away withopt_arat(as mentioned in the previous patch), and hence could usecpu_has_aratdirectly where right nowXEN_ARATis checked.
xen/arch/x86/acpi/cpu_idle.c static int __init cf_check cpuidle_presm · line 1670
Quoted 5 lines
xen/arch/x86/acpi/cpu_idle.c +2
-0@@ -1666,6 +1666,9 @@ static int __init cf_check cpuidle_presm {void *cpu = (void *)(long )smp_processor_id (); +if ( boot_cpu_data.vendor ==X86_VENDOR_INTEL ) +intel_init_arat ();
I really would prefer to avoid the need for this.
Now that microcode loading has moved to the start of day, we can drop most of the order-of-init complexity for CPUID/etc, and I expect that problems like this will cease to exist as a result.
Notably, we've now got no relevant difference between early_init() and regular init(). That was a complexity we inherited from Linux.
~Andrew
Re: [PATCH 6/8] x86/Intel: use host CPU policy for ARAT checking
Quoted 23 lines · Andrew Cooper
Quoted 22 lines · Jan Beulich
There's no need to invoke CPUID yet another time. However, as the host CPU policy is set up only shortly after
init_intel()ran on the BSP, defer the logic to a pre-SMP initcall. This can't be (a new) one incpu/intel.cthough, as that's linked afteracpi/cpu_idle.c(which is where we already need the feature set). Sinceopt_aratis local to the cpu/ subtree, introduce a new Intel-specific helper to hold the code needed.Further, as we assume symmetry anyway, use
setup_force_cpu_cap()and hence limit the checking to the boot CPU.Signed-off-by Jan Beulich <[email protected]>---
The need to move wherecpu_has_aratis checked would go away if we did away withopt_arat(as mentioned in the previous patch), and hence could usecpu_has_aratdirectly where right nowXEN_ARATis checked.xen/arch/x86/acpi/cpu_idle.c +0
-0@@ -1666,6 +1666,9 @@ static int __init cf_check cpuidle_presm {void *cpu = (void *)(long )smp_processor_id ();
Quoted 4 lines
xen/arch/x86/acpi/cpu_idle.c static int __init cf_check cpuidle_presm · line 1670
Quoted 2 lines
xen/arch/x86/acpi/cpu_idle.c +2
-0+if ( boot_cpu_data.vendor ==X86_VENDOR_INTEL ) +intel_init_arat ();I really would prefer to avoid the need for this.
So would I, but ...
Quoted 6 lines
Now that microcode loading has moved to the start of day, we can drop most of the order-of-init complexity for CPUID/etc, and I expect that problems like this will cease to exist as a result.
Notably, we've now got no relevant difference between
early_init()and regular init(). That was a complexity we inherited from Linux.
... I don't see how this leads to any concrete suggestion as to better arrangements. With cpu_has_arat using the host policy now, setup_force_cpu_cap(X86_FEATURE_XEN_ARAT) being conditional upon that has to run _after_ calculate_host_policy(), which in turn runs after intel_init(). We could add a .c_late_init hook to struct cpu_dev, but doing so felt like overkill for the purpose here. Plus it feels like that wouldn't really address your concern either.
Jan
PATCH 7/8 x86/cpufreq: use host CPU policy for Turbo checking
In reply to Jan BeulichThere's no need to invoke CPUID yet another time - we assume symmetry anyway. With symmetry assumed, logging per-CPU also isn't very useful.
xen/arch/x86/acpi/cpufreq/acpi.c +4 -7
@@ -220,14 +220,11 @@ static unsigned int cf_check get_cur_frevoid intel_feature_detect (struct cpufreq_policy *policy) { -unsigned int eax; - - eax =cpuid_eax (6 ); -if (eax &0x2 ) { +if ( cpu_has_turbo ) + { policy->turbo =CPUFREQ_TURBO_ENABLED ; -if (cpufreq_verbose) -printk (XENLOG_INFO "CPU%u: Turbo Mode detected and enabled\n" , -smp_processor_id ()); +if ( cpufreq_verbose ) +printk_once (XENLOG_INFO "Turbo Mode detected and enabled\n" ); } }
xen/arch/x86/include/asm/cpufeature.h +1 -0
@@ -177,6 +177,7 @@ static inline bool boot_cpu_has(unsigned#define cpu_has_tbm boot_cpu_has(X86_FEATURE_TBM)/* CPUID level 0x00000006.eax */ +#define cpu_has_turbo host_cpu_policy.basic.pm.turbo#define cpu_has_arat host_cpu_policy.basic.pm.arat/* CPUID level 0x00000006.ecx */
Re: [PATCH 7/8] x86/cpufreq: use host CPU policy for Turbo checking
Quoted 4 lines · Jan Beulich
There's no need to invoke CPUID yet another time - we assume symmetry anyway. With symmetry assumed, logging per-CPU also isn't very useful.
Signed-off-by Jan Beulich <[email protected]>
Subject to the feedback from earlier in the series, Reviewed-by: Andrew Cooper <[email protected]>
PATCH 8/8 x86/cpufreq: use host CPU policy in HWP driver
In reply to Jan BeulichThere's no need to invoke CPUID yet another time. This way two of the static booleans can also go away.
xen/arch/x86/acpi/cpufreq/hwp.c +17 -24
@@ -18,9 +18,6 @@static bool __ro_after_init hwp_in_use; -static bool __ro_after_init feature_hwp_notification; -static bool __ro_after_init feature_hwp_activity_window; -static bool __read_mostly feature_hdc;static bool __ro_after_init opt_cpufreq_hdc = true;
@@ -165,8 +162,6 @@ bool hwp_active(void)static bool __init hwp_available (void ) { -unsigned int eax; -if ( boot_cpu_data.cpuid_level <CPUID_PM_LEAF ) { hwp_verbose("cpuid_level (%#x) lacks HWP support\n",
@@ -183,29 +178,25 @@ static bool __init hwp_available(void)return false; } - eax =cpuid_eax (CPUID_PM_LEAF ); -hwp_verbose ("%d notify: %d act-window: %d energy-perf: %d pkg-level: %d peci: %d\n" , - !!(eax &CPUID6_EAX_HWP ), - !!(eax &CPUID6_EAX_HWP_NOTIFICATION ), - !!(eax &CPUID6_EAX_HWP_ACTIVITY_WINDOW ), - !!(eax &CPUID6_EAX_HWP_ENERGY_PERFORMANCE_PREFERENCE ), - !!(eax &CPUID6_EAX_HWP_PACKAGE_LEVEL_REQUEST ), - !!(eax &CPUID6_EAX_HWP_PECI )); + host_cpu_policy.basic.pm.hwp, + host_cpu_policy.basic.pm.hwp_notification, + host_cpu_policy.basic.pm.hwp_activity_window, + host_cpu_policy.basic.pm.hwp_epp, + host_cpu_policy.basic.pm.hwp_plr, + host_cpu_policy.basic.pm.hwp_peci); -if ( !(eax &CPUID6_EAX_HWP ) ) +if ( !host_cpu_policy.basic.pm.hwp )return false; -if ( !(eax &CPUID6_EAX_HWP_ENERGY_PERFORMANCE_PREFERENCE ) ) +if ( !host_cpu_policy.basic.pm.hwp_epp ) {hwp_verbose ("disabled: No energy/performance preference available" );return false; } - feature_hwp_notification = eax &CPUID6_EAX_HWP_NOTIFICATION ; - feature_hwp_activity_window = eax &CPUID6_EAX_HWP_ACTIVITY_WINDOW ; - feature_hdc = eax &CPUID6_EAX_HDC ; + feature_hdc = host_cpu_policy.basic.pm.hdc; hwp_verbose("Hardware Duty Cycling (HDC) %ssupported%s\n", feature_hdc ? "" : "not ",
@@ -213,7 +204,7 @@ static bool __init hwp_available(void) : "");hwp_verbose ("HW_FEEDBACK %ssupported\n" , - (eax &CPUID6_EAX_HW_FEEDBACK ) ?"" :"not " ); + host_cpu_policy.basic.pm.hw_feedback ?"" :"not " ); hwp_in_use = true;
@@ -226,7 +217,8 @@ static int cf_check hwp_cpufreq_verify(s {struct hwp_drv_data *data =per_cpu (hwp_drv_data, policy->cpu); -if ( !feature_hwp_activity_window && data->activity_window ) +if ( !host_cpu_policy.basic.pm.hwp_activity_window && + data->activity_window ) {hwp_verbose ("HWP activity window not supported\n" );
@@ -268,7 +260,7 @@ static int cf_check hwp_cpufreq_target(s hwp_req.max_perf = data->maximum; hwp_req.desired = data->desired; hwp_req.energy_perf = data->energy_perf; -if ( feature_hwp_activity_window ) +if ( host_cpu_policy.basic.pm.hwp_activity_window ) hwp_req.activity_window = data->activity_window;if ( hwp_req.raw == data->curr_req.raw )
@@ -365,7 +357,7 @@ static void cf_check hwp_init_msrs(void }/* Ensure we don't generate interrupts */ -if ( feature_hwp_notification ) +if ( host_cpu_policy.basic.pm.hwp_notification )wrmsr_safe (MSR_HWP_INTERRUPT ,0 );if ( !(val &PM_ENABLE_HWP_ENABLE ) )
@@ -537,7 +529,8 @@ int get_hwp_para(unsigned int cpu,return -ENODATA ; cppc_para->features = - (feature_hwp_activity_window ?XEN_SYSCTL_CPPC_FEAT_ACT_WINDOW :0 ); + (host_cpu_policy.basic.pm.hwp_activity_window + ?XEN_SYSCTL_CPPC_FEAT_ACT_WINDOW :0 ); cppc_para->lowest = data->hw.lowest; cppc_para->lowest_nonlinear = data->hw.most_efficient; cppc_para->nominal = data->hw.guaranteed;
@@ -585,7 +578,7 @@ int set_hwp_para(struct cpufreq_policy */* Clear out activity window if lacking HW supported. */ if ( (set_cppc->set_params &XEN_SYSCTL_CPPC_SET_ACT_WINDOW ) && - !feature_hwp_activity_window ) + !host_cpu_policy.basic.pm.hwp_activity_window ) { set_cppc->set_params &= ~XEN_SYSCTL_CPPC_SET_ACT_WINDOW ; cleared_act_window = true;
xen/arch/x86/include/asm/cpufeature.h +0 -8
@@ -115,14 +115,6 @@ static inline bool boot_cpu_has(unsigned }#define CPUID_PM_LEAF 6 -#define CPUID6_EAX_HWP BIT(7, U) -#define CPUID6_EAX_HWP_NOTIFICATION BIT(8, U) -#define CPUID6_EAX_HWP_ACTIVITY_WINDOW BIT(9, U) -#define CPUID6_EAX_HWP_ENERGY_PERFORMANCE_PREFERENCE BIT(10, U) -#define CPUID6_EAX_HWP_PACKAGE_LEVEL_REQUEST BIT(11, U) -#define CPUID6_EAX_HDC BIT(13, U) -#define CPUID6_EAX_HWP_PECI BIT(16, U) -#define CPUID6_EAX_HW_FEEDBACK BIT(19, U)/* CPUID level 0x00000001.edx */ #define cpu_has_fpu 1
Re: [PATCH 8/8] x86/cpufreq: use host CPU policy in HWP driver
Quoted 4 lines · Jan Beulich
There's no need to invoke CPUID yet another time. This way two of the static booleans can also go away.
Signed-off-by Jan Beulich <[email protected]>
Thanks,
Jason
Re: [PATCH 8/8] x86/cpufreq: use host CPU policy in HWP driver
In reply to Jan BeulichQuoted 14 lines · Jan Beulich
xen/arch/x86/acpi/cpufreq/hwp.c +0
-7@@ -183,29 +178,25 @@ static bool __init hwp_available(void)return false; } - eax =cpuid_eax (CPUID_PM_LEAF ); - hwp_verbose("%d notify: %d act-window: %d energy-perf: %d pkg-level: %d peci: %d\n", - !!(eax & CPUID6_EAX_HWP), - !!(eax & CPUID6_EAX_HWP_NOTIFICATION), - !!(eax & CPUID6_EAX_HWP_ACTIVITY_WINDOW), - !!(eax & CPUID6_EAX_HWP_ENERGY_PERFORMANCE_PREFERENCE), - !!(eax & CPUID6_EAX_HWP_PACKAGE_LEVEL_REQUEST),
xen/arch/x86/acpi/cpufreq/hwp.c static bool __init hwp_available(void) · line 189
Quoted 10 lines
xen/arch/x86/acpi/cpufreq/hwp.c +7
-2- !!(eax & CPUID6_EAX_HWP_PECI)); + host_cpu_policy.basic.pm.hwp, + host_cpu_policy.basic.pm.hwp_notification, + host_cpu_policy.basic.pm.hwp_activity_window, + host_cpu_policy.basic.pm.hwp_epp, + host_cpu_policy.basic.pm.hwp_plr, + host_cpu_policy.basic.pm.hwp_peci); -if ( !(eax &CPUID6_EAX_HWP ) ) +if ( !host_cpu_policy.basic.pm.hwp )
I think this justifies a cpu_has_hwp like we have for turbo/arat/etc. Similarly for the other features.
Quoted 4 lines
xen/arch/x86/acpi/cpufreq/hwp.c +1
-1return false; -if ( !(eax &CPUID6_EAX_HWP_ENERGY_PERFORMANCE_PREFERENCE ) ) +if ( !host_cpu_policy.basic.pm.hwp_epp )
xen/arch/x86/acpi/cpufreq/hwp.c static bool __init hwp_available(void) · line 199
Quoted 10 lines
xen/arch/x86/acpi/cpufreq/hwp.c +1
-3{hwp_verbose ("disabled: No energy/performance preference available" );return false; } - feature_hwp_notification = eax &CPUID6_EAX_HWP_NOTIFICATION ; - feature_hwp_activity_window = eax &CPUID6_EAX_HWP_ACTIVITY_WINDOW ; - feature_hdc = eax &CPUID6_EAX_HDC ; + feature_hdc = host_cpu_policy.basic.pm.hdc;
Looking at how feature_hdc is used, I think it should be the bit within the host policy, rather than a separate boolean.
The host policy "is" what Xen is using, so if the HWP code decides it doesn't like HDC, then that does want reflecting.
Unrelated to this patch, but as I've been looking at the code. What happens when hwp_init_msrs() fails on an AP? I can't see anything which unwinds the initialised HDC state on the prior CPUs...
xen/arch/x86/acpi/cpufreq/hwp.c static void cf_check hwp_init_msrs(void · line 361
Quoted 7 lines
xen/arch/x86/acpi/cpufreq/hwp.c +1
-1@@ -365,7 +357,7 @@ static void cf_check hwp_init_msrs(void }/* Ensure we don't generate interrupts */ -if ( feature_hwp_notification ) +if ( host_cpu_policy.basic.pm.hwp_notification )wrmsr_safe (MSR_HWP_INTERRUPT ,0 );
Again, unrelated to the patch, but why is this a wrmsr_safe() ?
If the feature is enumerated, the MSR had better work.
Things like this start to matter more when we consider the code generation for wrmsr_safe() using MSR_IMM.
xen/arch/x86/include/asm/cpufeature.h static inline bool boot_cpu_has(unsigned · line 117
Quoted 6 lines
xen/arch/x86/include/asm/cpufeature.h +0
-0@@ -115,14 +115,6 @@ static inline bool boot_cpu_has(unsigned }#define CPUID_PM_LEAF 6
Doesn't this patch drop the final user?
~Andrew
Re: [PATCH 8/8] x86/cpufreq: use host CPU policy in HWP driver
Quoted 23 lines · Andrew Cooper
Quoted 8 lines · Jan Beulich
xen/arch/x86/acpi/cpufreq/hwp.c +1
-1@@ -183,29 +178,25 @@ static bool __init hwp_available(void)return false; -if ( !(eax &CPUID6_EAX_HWP_ENERGY_PERFORMANCE_PREFERENCE ) ) +if ( !host_cpu_policy.basic.pm.hwp_epp )xen/arch/x86/acpi/cpufreq/hwp.c static bool __init hwp_available(void) · line 188
Quoted 10 lines
xen/arch/x86/acpi/cpufreq/hwp.c +1
-3{hwp_verbose ("disabled: No energy/performance preference available" );return false; } - feature_hwp_notification = eax &CPUID6_EAX_HWP_NOTIFICATION ; - feature_hwp_activity_window = eax &CPUID6_EAX_HWP_ACTIVITY_WINDOW ; - feature_hdc = eax &CPUID6_EAX_HDC ; + feature_hdc = host_cpu_policy.basic.pm.hdc;Looking at how
feature_hdcis used, I think it should be the bit within the host policy, rather than a separate boolean.
Quoted 6 lines
The host policy "is" what Xen is using, so if the HWP code decides it doesn't like HDC, then that does want reflecting.
Unrelated to this patch, but as I've been looking at the code. What happens when
hwp_init_msrs()fails on an AP? I can't see anything which unwinds the initialised HDC state on the prior CPUs...
Assuming symmetry, it'd fail on the BSP and never get to an AP. Yes, I didn't consider unwinding.
Quoted 9 lines
xen/arch/x86/acpi/cpufreq/hwp.c static void cf_check hwp_init_msrs(void · line 361
Quoted 7 lines
xen/arch/x86/acpi/cpufreq/hwp.c +1
-1@@ -365,7 +357,7 @@ static void cf_check hwp_init_msrs(void }/* Ensure we don't generate interrupts */ -if ( feature_hwp_notification ) +if ( host_cpu_policy.basic.pm.hwp_notification )wrmsr_safe (MSR_HWP_INTERRUPT ,0 );Again, unrelated to the patch, but why is this a
wrmsr_safe()?
It seemed safer ;)
Quoted 1 line
If the feature is enumerated, the MSR had better work.
Yes, but I wasn't sure how much to trust it. I only tested with 2 processor models, so I wanted to be safer. IIRC, I saw a #GP or two during development. Those might have been from writing a reserved bit and fixed during development.
Given my small testing size, I wanted to keep it safe and not take down Xen.
Regards,
Jason
Re: [PATCH 8/8] x86/cpufreq: use host CPU policy in HWP driver
In reply to Andrew CooperQuoted 23 lines · Andrew Cooper
Quoted 22 lines · Jan Beulich
xen/arch/x86/acpi/cpufreq/hwp.c +6
-8@@ -183,29 +178,25 @@ static bool __init hwp_available(void)return false; } - eax =cpuid_eax (CPUID_PM_LEAF ); -hwp_verbose ("%d notify: %d act-window: %d energy-perf: %d pkg-level: %d peci: %d\n" , - !!(eax &CPUID6_EAX_HWP ), - !!(eax &CPUID6_EAX_HWP_NOTIFICATION ), - !!(eax &CPUID6_EAX_HWP_ACTIVITY_WINDOW ), - !!(eax &CPUID6_EAX_HWP_ENERGY_PERFORMANCE_PREFERENCE ), - !!(eax &CPUID6_EAX_HWP_PACKAGE_LEVEL_REQUEST ), - !!(eax &CPUID6_EAX_HWP_PECI )); + host_cpu_policy.basic.pm.hwp, + host_cpu_policy.basic.pm.hwp_notification, + host_cpu_policy.basic.pm.hwp_activity_window, + host_cpu_policy.basic.pm.hwp_epp, + host_cpu_policy.basic.pm.hwp_plr, + host_cpu_policy.basic.pm.hwp_peci);
Quoted 5 lines
xen/arch/x86/acpi/cpufreq/hwp.c static bool __init hwp_available(void) · line 189
Quoted 2 lines
xen/arch/x86/acpi/cpufreq/hwp.c +1
-1-if ( !(eax &CPUID6_EAX_HWP ) ) +if ( !host_cpu_policy.basic.pm.hwp )I think this justifies a
cpu_has_hwplike we have for turbo/arat/etc. Similarly for the other features.
Hmm, okay, I can do that. The difference between using boot_cpu_data vs host_cpu_policy in cpu_has_* is completely that way, which in fact made me uncertain even for the introduction of the APERFMPERF, ARAT, and TURBO shorthands.
Quoted 15 lines
Quoted 4 lines
xen/arch/x86/acpi/cpufreq/hwp.c +1
-1return false; -if ( !(eax &CPUID6_EAX_HWP_ENERGY_PERFORMANCE_PREFERENCE ) ) +if ( !host_cpu_policy.basic.pm.hwp_epp )xen/arch/x86/acpi/cpufreq/hwp.c static bool __init hwp_available(void) · line 199
Quoted 10 lines
xen/arch/x86/acpi/cpufreq/hwp.c +1
-3{hwp_verbose ("disabled: No energy/performance preference available" );return false; } - feature_hwp_notification = eax &CPUID6_EAX_HWP_NOTIFICATION ; - feature_hwp_activity_window = eax &CPUID6_EAX_HWP_ACTIVITY_WINDOW ; - feature_hdc = eax &CPUID6_EAX_HDC ; + feature_hdc = host_cpu_policy.basic.pm.hdc;
Quoted 5 lines
Looking at how
feature_hdcis used, I think it should be the bit within the host policy, rather than a separate boolean.The host policy "is" what Xen is using, so if the HWP code decides it doesn't like HDC, then that does want reflecting.
I'm not sure about this. Yes as long as the host policy bits don't propagate to guest policies. But if they did (and as said earlier, sooner or later we may want / need to do that for some of the leaf 6 ones), why would the driver's choice affect what guests get to see? (That's applicable to a fair degree for the host policy as well: What the driver chooses doesn't need to match Xen's global view of the world.
Quoted 11 lines
xen/arch/x86/acpi/cpufreq/hwp.c static void cf_check hwp_init_msrs(void · line 361
Quoted 7 lines
xen/arch/x86/acpi/cpufreq/hwp.c +1
-1@@ -365,7 +357,7 @@ static void cf_check hwp_init_msrs(void }/* Ensure we don't generate interrupts */ -if ( feature_hwp_notification ) +if ( host_cpu_policy.basic.pm.hwp_notification )wrmsr_safe (MSR_HWP_INTERRUPT ,0 );Again, unrelated to the patch, but why is this a
wrmsr_safe()?If the feature is enumerated, the MSR had better work.
Yet as we know when running virtualized ourselves, we can't always rely on CPUID bits and MSR accessibility to be fully in sync. Yes, using in fact any cpufreq driver is pretty meaningless when running virtualized, but we don't prevent that scenario afaict.
Quoted 2 lines
Things like this start to matter more when we consider the code generation for
wrmsr_safe()usingMSR_IMM.
Your patch from August only altered wrmsrns() iirc, hence even if we switched to wrmsr() here there would be no difference (yet). If wrmsrns() was provably usable in this case, wouldn't wrmsrns_safe() (yet to be invented) ultimately be not significantly different in terms of code gen, wrt MSR_IMM? The difference would continue to be whether there's recovery code; the recovery code, however, isn't affected by MSR_IMM. Finally, for a purpose like the one here (infrequently executed code), would the code size increase from trying to use the immediate form really be justified by the to be expected perf gain?
Quoted 8 lines
xen/arch/x86/include/asm/cpufeature.h static inline bool boot_cpu_has(unsigned · line 117
Quoted 6 lines
xen/arch/x86/include/asm/cpufeature.h +0
-0@@ -115,14 +115,6 @@ static inline bool boot_cpu_has(unsigned }#define CPUID_PM_LEAF 6Doesn't this patch drop the final user?
No, there's one use left in the HWP driver.
Jan