x86: KVM: Advertise CPUID of new Intel platform instructions to user space
v5 cover x86: KVM: Advertise CPUID of new Intel platform instructions to user space
Latest Intel platform Granite Rapids/Sierra Forest has introduced below new instructions and CPUIDs:
- CMPccXADD CPUID.(EAX=7,ECX=1):EAX[bit 7]
- AMX-FP16 CPUID.(EAX=7,ECX=1):EAX[bit 21]
- AVX-IFMA CPUID.(EAX=7,ECX=1):EAX[bit 23]
- AVX-VNNI-INT8 CPUID.(EAX=7,ECX=1):EDX[bit 4]
- AVX-NE-CONVERT CPUID.(EAX=7,ECX=1):EDX[bit 5]
- PREFETCHITI CPUID.(EAX=7,ECX=1):EDX[bit 14]
Details can be found in recent Intel ISE (Instruction Set Extensions)[1].
These features bits are on two CPUID leafs: CPUID_7_1_EAX and CPUID_7_1_EDX. CPUID_7_1_EAX is an expected-dense leaf and some of its bits have kernel usages, therefore give bits on this leaf an X86_FEATURE definition in kernel. However, CPUID_7_1_EDX is dense and none of its bits have truly kernel usages for the moment. Given that, move CPUID_7_1_EDX to be a KVM-only leaf and plus an x86_FEATURE definition for bits on this leaf to direct them to the KVM entry.
This patch series advertises KVM support of these CPUIDs to host userspace. For all of these features, there are no new VMX controls or additional host enabling required for guests to use them.
[1] Intel ISE: https://cdrdv2.intel.com/v1/dl/getContent/671368
v5:
- Modify some inaccurate descriptions in the changelogs.
- Address some naming-confusing problems for adding KVM-only leaves, including renaming function and adding comments.
v4:
https://lore.kernel.org/kvm/[email protected]/
- Put CPUID_7_1_EAX back to cpuid_leaf[], considering more bits will be defined in the future for this leaf.
v3:
https://lore.kernel.org/kvm/[email protected]/
- Remain CPUID_8000_001F_EAX in the last leaf of cpuid_leaf[]
- Replace CPUID_7_1_EAX with CPUID_LNX_5, waiting for future new CPUIDs
v2:
https://lore.kernel.org/kvm/[email protected]/
- Remove vague descriptions in the changelogs, including pronouns and "this patch" kind of things.
- Move the two CPUIDs of cpuid_leaf[12] CPUID_7_1_EAX to KVM-only subleaves.
- Replace cpuid_leaf[12] CPUID_7_1_EAX with the last leaf CPUID_8000_001F_EAX to shorten array length.
- Change the newly-added CPUID leaf [CPUID_7_1_EDX] in v1 into KVM-only subleaves.
v1:
https://lore.kernel.org/kvm/[email protected]/
Chang S. Bae (1):
x86: KVM: Advertise AMX-FP16 CPUID to user space
Jiaxi Chen (5):
x86: KVM: Advertise CMPccXADD CPUID to user space
x86: KVM: Advertise AVX-IFMA CPUID to user space
KVM: x86: Advertise AVX-VNNI-INT8 CPUID to user space
KVM: x86: Advertise AVX-NE-CONVERT CPUID to user space
KVM: x86: Advertise PREFETCHIT0/1 CPUID to user space
Sean Christopherson (2):
KVM: x86: Add BUILD_BUG_ON() to detect bad usage of "scattered" flags
KVM: x86: Update KVM-only leaf handling to allow for 100% KVM-only
leafs
arch/x86/include/asm/cpufeatures.h | 3 +++ arch/x86/kvm/cpuid.c | 25 ++++++++++++++++++------- arch/x86/kvm/reverse_cpuid.h | 25 ++++++++++++++++++++++--- 3 files changed, 43 insertions(+), 10 deletions(-)
base-commit: 094226ad94f471a9f19e8f8e7140a09c2625abaa
Signature · 1 line
2.27.0
PATCH v5 1/8 KVM: x86: Add BUILD_BUG_ON() to detect bad usage of "scattered" flags
From: Sean Christopherson <[email protected]>
Add a compile-time assert in the SF() macro to detect improper usage, i.e. to detect passing in an X86_FEATURE_* flag that isn't actually scattered by the kernel. Upcoming feature flags will be 100% KVM-only and will have X86_FEATURE_* macros that point at a kvm_only_cpuid_leafs word, not a kernel-defined word. Using SF() and thus boot_cpu_has() for such feature flags would access memory beyond x86_capability[NCAPINTS] and at best incorrectly hide a feature, and at worst leak kernel state to userspace.
--- arch/x86/kvm/cpuid.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
arch/x86/kvm/cpuid.c +7 -1
@@ -65,7 +65,13 @@ u32 xstate_required_size(u64 xstate_bv, bool compacted)#define KVM_X86_FEATURE_PSFD (13*32+28)/* Predictive Store Forwarding Disable */ #define F feature_bit -#define SF (name) (boot_cpu_has(X86_FEATURE_##name) ? F(name) : 0) + +/* Scattered Flag - For features that are scattered by cpufeatures.h. */ +#define SF (name) \ +({ \ + BUILD_BUG_ON(X86_FEATURE_##name >= MAX_CPU_FEATURES); \ + (boot_cpu_has(X86_FEATURE_##name) ? F(name) : 0); \ +})/* * Magic value used by KVM when querying userspace-provided CPUID entries and
Signature · 1 line
2.27.0
PATCH v5 2/8 KVM: x86: Update KVM-only leaf handling to allow for 100% KVM-only leafs
In reply to Jiaxi ChenFrom: Sean Christopherson <[email protected]>
Rename kvm_cpu_cap_init_scattered() to kvm_cpu_cap_init_kvm_defined() in anticipation of adding KVM-only CPUID leafs that aren't recognized by the kernel and thus not scattered, i.e. for leafs that are 100% KVM-defined.
Adjust/add comments to kvm_only_cpuid_leafs and KVM_X86_FEATURE to document how to create new kvm_only_cpuid_leafs entries for scattered features as well as features that are entirely unknown to the kernel.
No functional change intended.
--- arch/x86/kvm/cpuid.c | 8 ++++---- arch/x86/kvm/reverse_cpuid.h | 18 +++++++++++++++--- 2 files changed, 19 insertions(+), 7 deletions(-)
arch/x86/kvm/cpuid.c +4 -4
@@ -549,9 +549,9 @@ static __always_inline void __kvm_cpu_cap_mask(unsigned int leaf) }static __always_inline -void kvm_cpu_cap_init_scattered(enum kvm_only_cpuid_leafs leaf, u32 mask) +void kvm_cpu_cap_init_kvm_defined(enum kvm_only_cpuid_leafs leaf, u32 mask) { -/* Use kvm_cpu_cap_mask for non-scattered leafs. */ +/* Use kvm_cpu_cap_mask for leafs that aren't KVM-only. */ BUILD_BUG_ON (leaf <NCAPINTS ); kvm_cpu_caps[leaf] = mask;
@@ -561,7 +561,7 @@ void kvm_cpu_cap_init_scattered(enum kvm_only_cpuid_leafs leaf, u32 mask) static __always_inline void kvm_cpu_cap_mask(enum cpuid_leafs leaf, u32 mask) { -/* Use kvm_cpu_cap_init_scattered for scattered leafs. */ +/* Use kvm_cpu_cap_init_kvm_defined for KVM-only leafs. */ BUILD_BUG_ON (leaf >=NCAPINTS ); kvm_cpu_caps[leaf] &= mask;
@@ -670,7 +670,7 @@ void kvm_set_cpu_caps(void)F (XSAVEOPT ) |F (XSAVEC ) |F (XGETBV1 ) |F (XSAVES ) | f_xfd ); -kvm_cpu_cap_init_scattered (CPUID_12_EAX , +kvm_cpu_cap_init_kvm_defined (CPUID_12_EAX ,SF (SGX1 ) |SF (SGX2 ) );
arch/x86/kvm/reverse_cpuid.h +15 -3
@@ -7,9 +7,9 @@#include <asm/cpufeatures.h> /* -* Hardware-defined CPUID leafs that are scattered in the kernel, but need to -* be directly used by KVM. Note, these word values conflict with the kernel's -* "bug" caps, but KVM doesn't use those. +* Hardware-defined CPUID leafs that are either scattered by the kernel or are +* unknown to the kernel, but need to be directly used by KVM. Note, these +* word values conflict with the kernel's "bug" caps, but KVM doesn't use those. */ enum kvm_only_cpuid_leafs {CPUID_12_EAX =NCAPINTS ,
@@ -18,6 +18,18 @@ enum kvm_only_cpuid_leafs { NKVMCAPINTS = NR_KVM_CPU_CAPS - NCAPINTS, }; +/* +* Define a KVM-only feature flag. +* +* For features that are scattered by cpufeatures.h, __feature_translate() also +* needs to be updated to translate the kernel-defined feature into the +* KVM-defined feature. +* +* For features that are 100% KVM-only, i.e. not defined by cpufeatures.h, +* forego the intermediate KVM_X86_FEATURE and directly define X86_FEATURE_* so +* that X86_FEATURE_* can be used in KVM. No __feature_translate() handling is +* needed in this case. +*/ #define KVM_X86_FEATURE (w, f) ((w)*32 + (f))/* Intel-defined SGX sub-features, CPUID level 0x12 (EAX). */
Signature · 1 line
2.27.0
PATCH v5 3/8 x86: KVM: Advertise CMPccXADD CPUID to user space
In reply to Jiaxi ChenCMPccXADD is a new set of instructions in the latest Intel platform Sierra Forest. This new instruction set includes a semaphore operation that can compare and add the operands if condition is met, which can improve database performance.
The bit definition:
CPUID.(EAX=7,ECX=1):EAX[bit 7]
CMPccXADD is on an expected-dense CPUID leaf and some other bits on this leaf have kernel usages. Given that, define this feature bit like X86_FEATURE_<name> in kernel. Considering CMPccXADD itself has no truly kernel usages and /proc/cpuinfo has too much unreadable flags, hide this one in /proc/cpuinfo.
Advertise CMPCCXADD to KVM userspace. This is safe because there are no new VMX controls or additional host enabling required for guests to use this feature.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -308,6 +308,7 @@/* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4)/* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5)/* AVX512 BFLOAT16 instructions */ +#define X86_FEATURE_CMPCCXADD (12*32+ 7)/* "" CMPccXADD instructions */ /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */
arch/x86/kvm/cpuid.c +1 -1
@@ -663,7 +663,7 @@ void kvm_set_cpu_caps(void)kvm_cpu_cap_set (X86_FEATURE_SPEC_CTRL_SSBD );kvm_cpu_cap_mask (CPUID_7_1_EAX , -F (AVX_VNNI ) |F (AVX512_BF16 ) +F (AVX_VNNI ) |F (AVX512_BF16 ) |F (CMPCCXADD ) ); kvm_cpu_cap_mask(CPUID_D_1_EAX,
Signature · 1 line
2.27.0
PATCH v5 4/8 x86: KVM: Advertise AMX-FP16 CPUID to user space
In reply to Jiaxi ChenFrom: "Chang S. Bae" <[email protected]>
Latest Intel platform Granite Rapids has introduced a new instruction - AMX-FP16, which performs dot-products of two FP16 tiles and accumulates the results into a packed single precision tile. AMX-FP16 adds FP16 capability and also allows a FP16 GPU trained model to run faster without loss of accuracy or added SW overhead.
The bit definition:
CPUID.(EAX=7,ECX=1):EAX[bit 21]
AMX-FP16 is on an expected-dense CPUID leaf and some other bits on this leaf have kernel usages. Given that, define this feature bit like X86_FEATURE_<name> in kernel. Considering AMX-FP16 itself has no truly kernel usages and /proc/cpuinfo has too much unreadable flags, hide this one in /proc/cpuinfo.
Advertise AMX-FP16 to KVM userspace. This is safe because there are no new VMX controls or additional host enabling required for guests to use this feature.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -309,6 +309,7 @@#define X86_FEATURE_AVX_VNNI (12*32+ 4)/* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5)/* AVX512 BFLOAT16 instructions */ #define X86_FEATURE_CMPCCXADD (12*32+ 7)/* "" CMPccXADD instructions */ +#define X86_FEATURE_AMX_FP16 (12*32+21)/* "" AMX fp16 Support */ /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */
arch/x86/kvm/cpuid.c +1 -1
@@ -663,7 +663,7 @@ void kvm_set_cpu_caps(void)kvm_cpu_cap_set (X86_FEATURE_SPEC_CTRL_SSBD );kvm_cpu_cap_mask (CPUID_7_1_EAX , -F (AVX_VNNI ) |F (AVX512_BF16 ) |F (CMPCCXADD ) +F (AVX_VNNI ) |F (AVX512_BF16 ) |F (CMPCCXADD ) |F (AMX_FP16 ) ); kvm_cpu_cap_mask(CPUID_D_1_EAX,
Signature · 1 line
2.27.0
PATCH v5 5/8 x86: KVM: Advertise AVX-IFMA CPUID to user space
In reply to Jiaxi ChenAVX-IFMA is a new instruction in the latest Intel platform Sierra Forest. This instruction packed multiplies unsigned 52-bit integers and adds the low/high 52-bit products to Qword Accumulators.
The bit definition:
CPUID.(EAX=7,ECX=1):EAX[bit 23]
AVX-IFMA is on an expected-dense CPUID leaf and some other bits on this leaf have kernel usages. Given that, define this feature bit like X86_FEATURE_<name> in kernel. Considering AVX-IFMA itself has no truly kernel usages and /proc/cpuinfo has too much unreadable flags, hide this one in /proc/cpuinfo.
Advertise AVX-IFMA to KVM userspace. This is safe because there are no new VMX controls or additional host enabling required for guests to use this feature.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -310,6 +310,7 @@#define X86_FEATURE_AVX512_BF16 (12*32+ 5)/* AVX512 BFLOAT16 instructions */ #define X86_FEATURE_CMPCCXADD (12*32+ 7)/* "" CMPccXADD instructions */ #define X86_FEATURE_AMX_FP16 (12*32+21)/* "" AMX fp16 Support */ +#define X86_FEATURE_AVX_IFMA (12*32+23)/* "" Support for VPMADD52[H,L]UQ */ /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */
arch/x86/kvm/cpuid.c +2 -1
@@ -663,7 +663,8 @@ void kvm_set_cpu_caps(void)kvm_cpu_cap_set (X86_FEATURE_SPEC_CTRL_SSBD );kvm_cpu_cap_mask (CPUID_7_1_EAX , -F (AVX_VNNI ) |F (AVX512_BF16 ) |F (CMPCCXADD ) |F (AMX_FP16 ) +F (AVX_VNNI ) |F (AVX512_BF16 ) |F (CMPCCXADD ) |F (AMX_FP16 ) | +F (AVX_IFMA ) ); kvm_cpu_cap_mask(CPUID_D_1_EAX,
Signature · 1 line
2.27.0
PATCH v5 6/8 KVM: x86: Advertise AVX-VNNI-INT8 CPUID to user space
In reply to Jiaxi ChenAVX-VNNI-INT8 is a new set of instructions in the latest Intel platform Sierra Forest, aims for the platform to have superior AI capabilities. This instruction multiplies the individual bytes of two unsigned or unsigned source operands, then adds and accumulates the results into the destination dword element size operand.
The bit definition:
CPUID.(EAX=7,ECX=1):EDX[bit 4]
AVX-VNNI-INT8 is on a new and sparse CPUID leaf and all bits on this leaf have no truly kernel use case for now. Given that and to save space for kernel feature bits, move this new leaf to KVM-only subleaf and plus an x86_FEATURE definition for AVX-VNNI-INT8 to direct it to the KVM entry.
Advertise AVX-VNNI-INT8 to KVM userspace. This is safe because there are no new VMX controls or additional host enabling required for guests to use this feature.
--- arch/x86/kvm/cpuid.c | 6 +++++- arch/x86/kvm/reverse_cpuid.h | 5 +++++ 2 files changed, 10 insertions(+), 1 deletion(-)
arch/x86/kvm/cpuid.c +5 -1
@@ -667,6 +667,10 @@ void kvm_set_cpu_caps(void)F (AVX_IFMA ) ); +kvm_cpu_cap_init_kvm_defined (CPUID_7_1_EDX , +F (AVX_VNNI_INT8 ) + ); +kvm_cpu_cap_mask (CPUID_D_1_EAX ,F (XSAVEOPT ) |F (XSAVEC ) |F (XGETBV1 ) |F (XSAVES ) | f_xfd );
@@ -920,9 +924,9 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) goto out;cpuid_entry_override (entry,CPUID_7_1_EAX ); +cpuid_entry_override (entry,CPUID_7_1_EDX ); entry->ebx =0 ; entry->ecx =0 ; - entry->edx =0 ; }break ;case 0xa : {/* Architectural Performance Monitoring */
arch/x86/kvm/reverse_cpuid.h +5 -0
@@ -13,6 +13,7 @@*/ enum kvm_only_cpuid_leafs {CPUID_12_EAX =NCAPINTS , +CPUID_7_1_EDX ,NR_KVM_CPU_CAPS ,NKVMCAPINTS =NR_KVM_CPU_CAPS -NCAPINTS ,
@@ -36,6 +37,9 @@ enum kvm_only_cpuid_leafs {#define KVM_X86_FEATURE_SGX1 KVM_X86_FEATURE(CPUID_12_EAX, 0)#define KVM_X86_FEATURE_SGX2 KVM_X86_FEATURE(CPUID_12_EAX, 1) +/* Intel-defined sub-features, CPUID level 0x00000007:1 (EDX) */ +#define X86_FEATURE_AVX_VNNI_INT8 KVM_X86_FEATURE(CPUID_7_1_EDX, 4) +struct cpuid_reg {u32 function;u32 index;
@@ -60,6 +64,7 @@ static const struct cpuid_reg reverse_cpuid[] = { [CPUID_7_1_EAX] = { 7, 1, CPUID_EAX}, [CPUID_12_EAX] = {0x00000012, 0, CPUID_EAX}, [CPUID_8000_001F_EAX] = {0x8000001f, 0, CPUID_EAX}, + [CPUID_7_1_EDX] = { 7, 1, CPUID_EDX}, };/*
Signature · 1 line
2.27.0
PATCH v5 7/8 KVM: x86: Advertise AVX-NE-CONVERT CPUID to user space
In reply to Jiaxi ChenAVX-NE-CONVERT is a new set of instructions which can convert low precision floating point like BF16/FP16 to high precision floating point FP32, and can also convert FP32 elements to BF16. This instruction allows the platform to have improved AI capabilities and better compatibility.
The bit definition:
CPUID.(EAX=7,ECX=1):EDX[bit 5]
AVX-NE-CONVERT is on a KVM-only subleaf. Plus an x86_FEATURE definition for this feature bit to direct it to the KVM entry.
Advertise AVX-NE-CONVERT to KVM userspace. This is safe because there are no new VMX controls or additional host enabling required for guests to use this feature.
--- arch/x86/kvm/cpuid.c | 2 +- arch/x86/kvm/reverse_cpuid.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/cpuid.c +1 -1
@@ -668,7 +668,7 @@ void kvm_set_cpu_caps(void) );kvm_cpu_cap_init_kvm_defined (CPUID_7_1_EDX , -F (AVX_VNNI_INT8 ) +F (AVX_VNNI_INT8 ) |F (AVX_NE_CONVERT ) ); kvm_cpu_cap_mask(CPUID_D_1_EAX,
arch/x86/kvm/reverse_cpuid.h +1 -0
@@ -39,6 +39,7 @@ enum kvm_only_cpuid_leafs {/* Intel-defined sub-features, CPUID level 0x00000007:1 (EDX) */ #define X86_FEATURE_AVX_VNNI_INT8 KVM_X86_FEATURE(CPUID_7_1_EDX, 4) +#define X86_FEATURE_AVX_NE_CONVERT KVM_X86_FEATURE(CPUID_7_1_EDX, 5)struct cpuid_reg {u32 function;
Signature · 1 line
2.27.0
PATCH v5 8/8 KVM: x86: Advertise PREFETCHIT0/1 CPUID to user space
In reply to Jiaxi ChenLatest Intel platform Granite Rapids has introduced a new instruction - PREFETCHIT0/1, which moves code to memory (cache) closer to the processor depending on specific hints.
The bit definition:
CPUID.(EAX=7,ECX=1):EDX[bit 14]
PREFETCHIT0/1 is on a KVM-only subleaf. Plus an x86_FEATURE definition for this feature bit to direct it to the KVM entry.
Advertise PREFETCHIT0/1 to KVM userspace. This is safe because there are no new VMX controls or additional host enabling required for guests to use this feature.
--- arch/x86/kvm/cpuid.c | 2 +- arch/x86/kvm/reverse_cpuid.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/cpuid.c +1 -1
@@ -668,7 +668,7 @@ void kvm_set_cpu_caps(void) );kvm_cpu_cap_init_kvm_defined (CPUID_7_1_EDX , -F (AVX_VNNI_INT8 ) |F (AVX_NE_CONVERT ) +F (AVX_VNNI_INT8 ) |F (AVX_NE_CONVERT ) |F (PREFETCHITI ) ); kvm_cpu_cap_mask(CPUID_D_1_EAX,
arch/x86/kvm/reverse_cpuid.h +1 -0
@@ -40,6 +40,7 @@ enum kvm_only_cpuid_leafs {/* Intel-defined sub-features, CPUID level 0x00000007:1 (EDX) */ #define X86_FEATURE_AVX_VNNI_INT8 KVM_X86_FEATURE(CPUID_7_1_EDX, 4)#define X86_FEATURE_AVX_NE_CONVERT KVM_X86_FEATURE(CPUID_7_1_EDX, 5) +#define X86_FEATURE_PREFETCHITI KVM_X86_FEATURE(CPUID_7_1_EDX, 14)struct cpuid_reg {u32 function;
Signature · 1 line
2.27.0
Queued, thanks.
Paolo