AMX Support in KVM
Subsystems (5)
v4 cover AMX Support in KVM
Highly appreciate for your review. This version mostly addressed the comments from Sean. Most comments are adopted except three which are not closed and need more discussions:
- Move the entire xfd write emulation code to x86.c. Doing so requires introducing a new kvm_x86_ops callback to disable msr write bitmap. According to Paolo's earlier comment he prefers to handle it in vmx.c.
- Directly check msr_bitmap in update_exception_bitmap() (for trapping #NM) and vcpu_enter_guest() (for syncing guest xfd after vm-exit) instead of introducing an extra flag in the last patch. However, doing so requires another new kvm_x86_ops callback for checking msr_bitmap since vcpu_enter_guest() is x86 common code. Having an extra flag sounds simpler here (at least for the initial AMX support). It does penalize nested guest with one xfd sync per exit, but it's not worse than a normal guest which initializes xfd but doesn't run AMX applications at all. Those could be improved afterwards.
- Disable #NM trap for nested guest. This version still chooses to always trap #NM (regardless in L1 or L2) as long as xfd write interception is disabled. In reality #NM is rare if nested guest doesn't intend to run AMX applications and always-trap is safer than dynamic trap for the basic support in case of any oversight here.
(Jing is temporarily leave for family reason, Yang helped work out this version)
----v3->v4:
- Verify kvm selftest for AMX (Paolo)
- Move fpstate buffer expansion from kvm_vcpu_after_set_cpuid () to kvm_check_cpuid() and improve patch description (Sean)
- Drop 'preemption' word in #NM interception patch (Sean)
- Remove 'trap_nm' flag. Replace it by: (Sean)
* Trapping #NM according to guest_fpu::xfd when write to xfd is intercepted.
* Always trapping #NM when xfd write interception is disabled
- Use better name for #NM related functions (Sean)
- Drop '#ifdef CONFIG_X86_64' in __kvm_set_xcr (Sean)
- Update description for KVM_CAP_XSAVE2 and prevent the guest from using the wrong ioctl (Sean)
- Replace 'xfd_out_of_sync' with a better name (Sean)
v2->v3:
- Trap #NM until write IA32_XFD with a non-zero value (Thomas)
- Revise return value in __xstate_request_perm() (Thomas)
- Revise doc for KVM_GET_SUPPORTED_CPUID (Paolo)
- Add Thomas's reviewed-by on one patch
- Reorder disabling read interception of XFD_ERR patch (Paolo)
- Move disabling r/w interception of XFD from x86.c to vmx.c (Paolo)
- Provide the API doc together with the new KVM_GET_XSAVE2 ioctl (Paolo)
- Make KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2) return minimum size of struct
kvm_xsave (4K) (Paolo)
- Request permission at the start of vm_create_with_vcpus() in selftest
- Request permission conditionally when XFD is supported (Paolo)
v1->v2:
- Live migration supported and verified with a selftest
- Rebase to Thomas's new series for guest fpstate reallocation [1]
- Expand fpstate at KVM_SET_CPUID2 instead of when emulating XCR0 and IA32_XFD (Thomas/Paolo)
- Accordingly remove all exit-to-userspace stuff
- Intercept #NM to save guest XFD_ERR and restore host/guest value at preemption on/off boundary (Thomas)
- Accordingly remove all xfd_err logic in preemption callback and fpu_swap_kvm_fpstate()
- Reuse KVM_SET_XSAVE to handle both legacy and expanded buffer (Paolo)
- Don't return dynamic bits w/o prctl() in KVM_GET_SUPPORTED_CPUID (Paolo)
- Check guest permissions for dynamic features in CPUID[0xD] instead of only for AMX at KVM_SET_CPUID (Paolo)
- Remove dynamic bit check for 32-bit guest in __kvm_set_xcr() (Paolo)
- Fix CPUID emulation for 0x1d and 0x1e (Paolo)
- Move "disable interception" to the end of the series (Paolo)
This series brings AMX (Advanced Matrix eXtensions) virtualization support to KVM. The preparatory series from Thomas [1] is also included.
A large portion of the changes in this series is to deal with eXtended Feature Disable (XFD) which allows resizing of the fpstate buffer to support dynamically-enabled XSTATE features with large state component (e.g. 8K for AMX).
There are a lot of simplications when comparing v2/v3 to the original proposal [2] and the first version [3]. Thanks to Thomas and Paolo for many good suggestions.
The support is based on following key changes:
- Guest permissions for dynamically-enabled XSAVE features
Native tasks have to request permission via prctl() before touching a dynamic-resized XSTATE compoenent. Introduce guest permissions for the similar purpose. Userspace VMM is expected to request guest permission only once when the first vCPU is created.
KVM checks guest permission in KVM_SET_CPUID2. Setting XFD in guest cpuid w/o proper permissions fails this operation. In the meantime, unpermitted features are also excluded in KVM_GET_SUPPORTED_CPUID.
- Extend fpstate reallocation mechanism to cover guest fpu
Unlike native tasks which have reallocation triggered from #NM handler, guest fpstate reallocation is requested by KVM when it identifies the intention on using dynamically-enabled XSAVE features inside guest.
Extend fpu core to allow KVM request fpstate buffer expansion for a guest fpu containter.
- Trigger fpstate reallocation in KVM
This could be done either statically (before guest runs) or dynamically (in the emulation path). According to discussion [1] we decide to statically enable all xfeatures allowed by guest perm in KVM_SET_CPUID2, with fpstate buffer sized accordingly. This spares a lot of code and also avoid imposing an ordered restore sequence (XCR0, XFD and XSTATE) to userspace VMM.
- RDMSR/WRMSR emulation for IA32_XFD
Because fpstate expansion is completed in KVM_SET_CPUID2, emulating r/w access to IA32_XFD simply involves the xfd field in the guest fpu container. If write and guest fpu is currently active, the software state (guest_fpstate::xfd and per-cpu xfd cache) is also updated.
- RDMSR/WRMSR emulation for XFD_ERR
When XFD causes an instruction to generate #NM, XFD_ERR contains information about which disabled state components are being accessed. It'd be problematic if the XFD_ERR value generated in guest is consumed/clobbered by the host before the guest itself doing so.
Intercept #NM exception to save the guest XFD_ERR value when write IA32_XFD with a non-zero value for 1st time. There is at most one interception per guest task given a dynamic feature.
RDMSR/WRMSR emulation uses the saved value. The host value (always ZERO outside of the host #NM handler) is restored before enabling preemption. The saved guest value is restored right before entering the guest (with preemption disabled).
- Get/set dynamic xfeature state for migration
Introduce new capability (KVM_CAP_XSAVE2) to deal with >4KB fpstate buffer. Reading this capability returns the size of the current guest fpstate (e.g. after expansion). Userspace VMM uses a new ioctl (KVM_GET_XSAVE2) to read guest fpstate from the kernel and reuses the existing ioctl (KVM_SET_XSAVE) to update guest fpsate to the kernel. KVM_SET_XSAVE is extended to do properly_sized memdup_user() based on the guest fpstate.
- Expose related cpuid bits to guest
The last step is to allow exposing XFD, AMX_TILE, AMX_INT8 and AMX_BF16 in guest cpuid. Adding those bits into kvm_cpu_caps finally activates all previous logics in this series
- Optimization: disable interception for IA32_XFD
IA32_XFD can be frequently updated by the guest, as it is part of the task state and swapped in context switch when prev and next have different XFD setting. Always intercepting WRMSR can easily cause non-negligible overhead.
Disable r/w emulation for IA32_XFD after intercepting the first WRMSR(IA32_XFD) with a non-zero value. However MSR passthrough implies the software state (guest_fpstate::xfd and per-cpu xfd cache) might be out of sync with MSR. This suggests KVM needs to re-sync them at VM-exit before preemption is enabled.
Thanks Jun Nakajima and Kevin Tian for the design suggestions when this version is being internally worked on.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://www.spinics.net/lists/kvm/msg259015.html
[3] https://lore.kernel.org/lkml/[email protected]/
Thanks,
Yang
---
Guang Zeng (1):
kvm: x86: Add support for getting/setting expanded xstate buffer
Jing Liu (11):
kvm: x86: Fix xstate_required_size() to follow XSTATE alignment rule
kvm: x86: Exclude unpermitted xfeatures at KVM_GET_SUPPORTED_CPUID x86/fpu: Make XFD initialization in __fpstate_reset() a function
argument
kvm: x86: Check and enable permitted dynamic xfeatures at
KVM_SET_CPUID2
kvm: x86: Add emulation for IA32_XFD
x86/fpu: Prepare xfd_err in struct fpu_guest
kvm: x86: Intercept #NM for saving IA32_XFD_ERR
kvm: x86: Emulate IA32_XFD_ERR for guest
kvm: x86: Disable RDMSR interception of IA32_XFD_ERR
kvm: x86: Add XCR0 support for Intel AMX
kvm: x86: Add CPUID support for Intel AMX
Kevin Tian (3):
x86/fpu: Provide fpu_update_guest_perm_features() for guest x86/fpu: Provide fpu_update_guest_xfd() for IA32_XFD emulation
kvm: x86: Disable interception for IA32_XFD on demand
Thomas Gleixner (5):
x86/fpu: Extend fpu_xstate_prctl() with guest permissions x86/fpu: Prepare guest FPU for dynamically enabled FPU features x86/fpu: Add guest support to xfd_enable_feature() x86/fpu: Add uabi_size to guest_fpu
x86/fpu: Provide fpu_sync_guest_vmexit_xfd_state()
Wei Wang (1):
kvm: selftests: Add support for KVM_CAP_XSAVE2
Documentation/virt/kvm/api.rst | 46 +++++- arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/fpu/api.h | 11 ++ arch/x86/include/asm/fpu/types.h | 32 ++++ arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 16 +- arch/x86/include/uapi/asm/prctl.h | 26 ++-- arch/x86/kernel/fpu/core.c | 104 ++++++++++++- arch/x86/kernel/fpu/xstate.c | 147 +++++++++++------- arch/x86/kernel/fpu/xstate.h | 15 +- arch/x86/kernel/process.c | 2 + arch/x86/kvm/cpuid.c | 99 +++++++++--- arch/x86/kvm/vmx/vmcs.h | 5 + arch/x86/kvm/vmx/vmx.c | 45 +++++- arch/x86/kvm/vmx/vmx.h | 2 +- arch/x86/kvm/x86.c | 105 ++++++++++++- include/uapi/linux/kvm.h | 4 + tools/arch/x86/include/uapi/asm/kvm.h | 16 +- tools/include/uapi/linux/kvm.h | 3 + .../testing/selftests/kvm/include/kvm_util.h | 2 + .../selftests/kvm/include/x86_64/processor.h | 10 ++ tools/testing/selftests/kvm/lib/kvm_util.c | 32 ++++ .../selftests/kvm/lib/x86_64/processor.c | 67 +++++++- .../testing/selftests/kvm/x86_64/evmcs_test.c | 2 +- tools/testing/selftests/kvm/x86_64/smm_test.c | 2 +- .../testing/selftests/kvm/x86_64/state_test.c | 2 +- .../kvm/x86_64/vmx_preemption_timer_test.c | 2 +- 27 files changed, 691 insertions(+), 109 deletions(-)
PATCH v4 1/21 x86/fpu: Extend fpu_xstate_prctl() with guest permissions
From: Thomas Gleixner <[email protected]>
KVM requires a clear separation of host user space and guest permissions for dynamic XSTATE components.
Add a guest permissions member to struct fpu and a separate set of prctl()
arguments: ARCH_GET_XCOMP_GUEST_PERM and ARCH_REQ_XCOMP_GUEST_PERM.
The semantics are equivalent to the host user space permission control except for the following constraints:
1) Permissions have to be requested before the first vCPU is created
2) Permissions are frozen when the first vCPU is created to ensure consistency. Any attempt to expand permissions via the prctl() after that point is rejected.
--- arch/x86/include/asm/fpu/api.h | 2 ++ arch/x86/include/asm/fpu/types.h | 9 ++++++ arch/x86/include/uapi/asm/prctl.h | 26 ++++++++------- arch/x86/kernel/fpu/core.c | 3 ++ arch/x86/kernel/fpu/xstate.c | 53 +++++++++++++++++++++++-------- arch/x86/kernel/fpu/xstate.h | 13 ++++++-- arch/x86/kernel/process.c | 2 ++ 7 files changed, 80 insertions(+), 28 deletions(-)
arch/x86/include/asm/fpu/api.h +2 -0
@@ -132,6 +132,8 @@ static inline void fpstate_free(struct fpu *fpu) { }/* fpstate-related functions which are exported to KVM */ extern void fpstate_clear_xstate_component (struct fpstate *fps,unsigned int xfeature); +extern inline u64 xstate_get_guest_group_perm (void ); +/* KVM specific functions */ extern bool fpu_alloc_guest_fpstate (struct fpu_guest *gfpu);extern void fpu_free_guest_fpstate (struct fpu_guest *gfpu);
arch/x86/include/asm/fpu/types.h +9 -0
@@ -387,6 +387,8 @@ struct fpstate {/* @regs is dynamically sized! Don't add anything after @regs! */ }__aligned(64) ; +#define FPU_GUEST_PERM_LOCKED BIT_ULL(63) +struct fpu_state_perm {/* * @__state_perm:
@@ -476,6 +478,13 @@ struct fpu {*/ struct fpu_state_perm perm; +/* +* @guest_perm: +* +* Permission related information for guest pseudo FPUs +*/ +struct fpu_state_perm guest_perm; +/* * @__fpstate: *
arch/x86/include/uapi/asm/prctl.h +14 -12
@@ -2,20 +2,22 @@#ifndef _ASM_X86_PRCTL_H#define _ASM_X86_PRCTL_H -#define ARCH_SET_GS 0x1001 -#define ARCH_SET_FS 0x1002 -#define ARCH_GET_FS 0x1003 -#define ARCH_GET_GS 0x1004 +#define ARCH_SET_GS 0x1001 +#define ARCH_SET_FS 0x1002 +#define ARCH_GET_FS 0x1003 +#define ARCH_GET_GS 0x1004 -#define ARCH_GET_CPUID 0x1011 -#define ARCH_SET_CPUID 0x1012 +#define ARCH_GET_CPUID 0x1011 +#define ARCH_SET_CPUID 0x1012 -#define ARCH_GET_XCOMP_SUPP 0x1021 -#define ARCH_GET_XCOMP_PERM 0x1022 -#define ARCH_REQ_XCOMP_PERM 0x1023 +#define ARCH_GET_XCOMP_SUPP 0x1021 +#define ARCH_GET_XCOMP_PERM 0x1022 +#define ARCH_REQ_XCOMP_PERM 0x1023 +#define ARCH_GET_XCOMP_GUEST_PERM 0x1024 +#define ARCH_REQ_XCOMP_GUEST_PERM 0x1025 -#define ARCH_MAP_VDSO_X32 0x2001 -#define ARCH_MAP_VDSO_32 0x2002 -#define ARCH_MAP_VDSO_64 0x2003 +#define ARCH_MAP_VDSO_X32 0x2001 +#define ARCH_MAP_VDSO_32 0x2002 +#define ARCH_MAP_VDSO_64 0x2003#endif /* _ASM_X86_PRCTL_H */
arch/x86/kernel/fpu/core.c +3 -0
@@ -450,6 +450,8 @@ void fpstate_reset(struct fpu *fpu) fpu->perm.__state_perm = fpu_kernel_cfg.default_features; fpu->perm.__state_size = fpu_kernel_cfg.default_size; fpu->perm.__user_state_size = fpu_user_cfg.default_size; +/* Same defaults for guests */ + fpu->guest_perm = fpu->perm; }static inline void fpu_inherit_perms (struct fpu *dst_fpu)
@@ -460,6 +462,7 @@ static inline void fpu_inherit_perms(struct fpu *dst_fpu)spin_lock_irq (¤t->sighand->siglock);/* Fork also inherits the permissions of the parent */ dst_fpu->perm = src_fpu->perm; + dst_fpu->guest_perm = src_fpu->guest_perm;spin_unlock_irq (¤t->sighand->siglock); } }
arch/x86/kernel/fpu/xstate.c +39 -14
120 lines not on this page · raw message
arch/x86/kernel/fpu/xstate.h +11 -2
@@ -20,10 +20,19 @@ static inline void xstate_init_xcomp_bv(struct xregs_state *xsave, u64 mask) xsave->header.xcomp_bv = mask |XCOMP_BV_COMPACTED_FORMAT ; } -static inline u64 xstate_get_host_group_perm (void ) +static inline u64 xstate_get_group_perm (bool guest) { +struct fpu *fpu = ¤t->group_leader->thread.fpu; +struct fpu_state_perm *perm; +/* Pairs with WRITE_ONCE() in xstate_request_perm() */ -return READ_ONCE (current->group_leader->thread.fpu.perm.__state_perm); + perm = guest ? &fpu->guest_perm : &fpu->perm; +return READ_ONCE (perm->__state_perm); +} + +static inline u64 xstate_get_host_group_perm (void ) +{ +return xstate_get_group_perm (false); }enum xstate_copy_mode {
arch/x86/kernel/process.c +2 -0
@@ -993,6 +993,8 @@ long do_arch_prctl_common(struct task_struct *task, int option,case ARCH_GET_XCOMP_SUPP :case ARCH_GET_XCOMP_PERM :case ARCH_REQ_XCOMP_PERM : +case ARCH_GET_XCOMP_GUEST_PERM : +case ARCH_REQ_XCOMP_GUEST_PERM :return fpu_xstate_prctl (task, option, arg2); }
PATCH v4 2/21 x86/fpu: Prepare guest FPU for dynamically enabled FPU features
In reply to Yang ZhongFrom: Thomas Gleixner <[email protected]>
To support dynamically enabled FPU features for guests prepare the guest pseudo FPU container to keep track of the currently enabled xfeatures and the guest permissions.
--- arch/x86/include/asm/fpu/types.h | 13 +++++++++++++ arch/x86/kernel/fpu/core.c | 26 +++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-)
arch/x86/include/asm/fpu/types.h +13 -0
@@ -504,6 +504,19 @@ struct fpu {* Guest pseudo FPU container */ struct fpu_guest { +/* +* @xfeatures: xfeature bitmap of features which are +* currently enabled for the guest vCPU. +*/ +u64 xfeatures; + +/* +* @perm: xfeature bitmap of features which are +* permitted to be enabled for the guest +* vCPU. +*/ +u64 perm; +/* * @fpstate: Pointer to the allocated guest fpstate */
arch/x86/kernel/fpu/core.c +25 -1
@@ -201,6 +201,26 @@ void fpu_reset_from_exception_fixup(void)#if IS_ENABLED (CONFIG_KVM )static void __fpstate_reset (struct fpstate *fpstate); +static void fpu_init_guest_permissions (struct fpu_guest *gfpu) +{ +struct fpu_state_perm *fpuperm; +u64 perm; + +if (!IS_ENABLED (CONFIG_X86_64 )) +return ; + +spin_lock_irq (¤t->sighand->siglock); + fpuperm = ¤t->group_leader->thread.fpu.guest_perm; + perm = fpuperm->__state_perm; + +/* First fpstate allocation locks down permissions. */ +WRITE_ONCE (fpuperm->__state_perm, perm |FPU_GUEST_PERM_LOCKED ); + +spin_unlock_irq (¤t->sighand->siglock); + + gfpu->perm = perm & ~FPU_GUEST_PERM_LOCKED ; +} +bool fpu_alloc_guest_fpstate (struct fpu_guest *gfpu) {struct fpstate *fpstate;
@@ -216,7 +236,11 @@ bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) fpstate->is_valloc = true; fpstate->is_guest = true; - gfpu->fpstate = fpstate; + gfpu->fpstate = fpstate; + gfpu->xfeatures = fpu_user_cfg.default_features; + gfpu->perm = fpu_user_cfg.default_features; +fpu_init_guest_permissions (gfpu); +return true; }EXPORT_SYMBOL_GPL (fpu_alloc_guest_fpstate);
PATCH v4 3/21 kvm: x86: Fix xstate_required_size() to follow XSTATE alignment rule
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
CPUID.0xD.1.EBX enumerates the size of the XSAVE area (in compacted format) required by XSAVES. If CPUID.0xD.i.ECX[1] is set for a state component (i), this state component should be located on the next 64-bytes boundary following the preceding state component in the compacted layout.
Fix xstate_required_size() to follow the alignment rule. AMX is the first state component with 64-bytes alignment to catch this bug.
--- arch/x86/kvm/cpuid.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/cpuid.c +2 -1
@@ -42,7 +42,8 @@ static u32 xstate_required_size(u64 xstate_bv, bool compacted)if (xstate_bv &0x1 ) {u32 eax, ebx, ecx, edx, offset;cpuid_count (0xD , feature_bit, &eax, &ebx, &ecx, &edx); - offset = compacted ? ret : ebx; +/* ECX[1]: 64B alignment in compacted form */ + offset = compacted ? ((ecx &0x2 ) ?ALIGN (ret,64 ) : ret) : ebx; ret =max (ret, offset + eax); }
Sean Christopherson
maintainerRe: [PATCH v4 03/21] kvm: x86: Fix xstate_required_size() to follow XSTATE alignment rule
Quoted 22 lines · Yang Zhong
From: Jing Liu <[email protected]>
CPUID.0xD.1.EBX enumerates the size of the XSAVE area (in compacted format) required by XSAVES. If CPUID.0xD.i.ECX[1] is set for a state component (i), this state component should be located on the next 64-bytes boundary following the preceding state component in the compacted layout.
Fix
xstate_required_size()to follow the alignment rule. AMX is the first state component with 64-bytes alignment to catch this bug.Signed-off-by Jing Liu <[email protected]>Signed-off-by Yang Zhong <[email protected]>--- arch/x86/kvm/cpuid.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/x86/kvm/cpuid.c +0
-0
arch/x86/kvm/cpuid.c static u32 xstate_required_size(u64 xstate_bv, bool compacted) · line 46
Quoted 6 lines
arch/x86/kvm/cpuid.c +2
-1@@ -42,7 +42,8 @@ static u32 xstate_required_size(u64 xstate_bv, bool compacted)if (xstate_bv &0x1 ) {u32 eax, ebx, ecx, edx, offset;cpuid_count (0xD , feature_bit, &eax, &ebx, &ecx, &edx); - offset = compacted ? ret : ebx; +/* ECX[1]: 64B alignment in compacted form */ + offset = compacted ? ((ecx &0x2 ) ?ALIGN (ret,64 ) : ret) : ebx;
That is impressively difficult to read.
if (compacted) offset = (ecx & 0x2) ? ALIGN(ret, 64) : ret; else offset = ebx;
Quoted 3 lines
arch/x86/kvm/cpuid.c +0
-0ret =max (ret, offset + eax); }
PATCH v4 4/21 kvm: x86: Exclude unpermitted xfeatures at KVM_GET_SUPPORTED_CPUID
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
KVM_GET_SUPPORTED_CPUID should not include any dynamic xstates in CPUID[0xD] if they have not been requested with prctl. Otherwise a process which directly passes KVM_GET_SUPPORTED_CPUID to KVM_SET_CPUID2 would now fail even if it doesn't intend to use a dynamically enabled feature. Userspace must know that prctl is required and allocate >4K xstate buffer before setting any dynamic bit.
--- Documentation/virt/kvm/api.rst | 4 ++++ arch/x86/kvm/cpuid.c | 9 ++++++--- 2 files changed, 10 insertions(+), 3 deletions(-)
Documentation/virt/kvm/api.rst +4 -0
@@ -1684,6 +1684,10 @@ userspace capabilities, and with user requirements (for example, the user may wish to constrain cpuid to emulate older hardware, or for feature consistency across a cluster). +Dynamically-enabled feature bits need to be requested with +``arch_prctl()`` before calling this ioctl. Feature bits that have not +been requested are excluded from the result. + Note that certain capabilities, such as KVM_CAP_X86_DISABLE_EXITS, may expose cpuid features (e.g. MONITOR) which are not supported by kvm in its default configuration. If userspace enables such capabilities, it
arch/x86/kvm/cpuid.c +6 -3
@@ -812,11 +812,13 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) goto out; }break ; -case 0xd : - entry->eax &= supported_xcr0; +case 0xd : { +u64 guest_perm =xstate_get_guest_group_perm (); + + entry->eax &= supported_xcr0 & guest_perm; entry->ebx =xstate_required_size (supported_xcr0, false); entry->ecx = entry->ebx; - entry->edx &= supported_xcr0 >>32 ; + entry->edx &= (supported_xcr0 & guest_perm) >>32 ;if (!supported_xcr0)break ;
@@ -863,6 +865,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) entry->edx =0 ; }break ; + }case 0x12 :/* Intel SGX */ if (!kvm_cpu_cap_has (X86_FEATURE_SGX )) {
PATCH v4 5/21 x86/fpu: Make XFD initialization in __fpstate_reset() a function argument
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
vCPU threads are different from native tasks regarding to the initial XFD value. While all native tasks follow a fixed value (init_fpstate::xfd) established by the FPU core at boot, vCPU threads need to obey the reset value (i.e. ZERO) defined by the specification, to meet the expectation of the guest.
Let the caller supply an argument and adjust the host and guest related invocations accordingly.
--- arch/x86/kernel/fpu/core.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
arch/x86/kernel/fpu/core.c +6 -5
@@ -199,7 +199,7 @@ void fpu_reset_from_exception_fixup(void) }#if IS_ENABLED (CONFIG_KVM ) -static void __fpstate_reset (struct fpstate *fpstate); +static void __fpstate_reset (struct fpstate *fpstate,u64 xfd);static void fpu_init_guest_permissions (struct fpu_guest *gfpu) {
@@ -231,7 +231,8 @@ bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu)if (!fpstate)return false; -__fpstate_reset (fpstate); +/* Leave xfd to 0 (the reset value defined by spec) */ +__fpstate_reset (fpstate,0 );fpstate_init_user (fpstate); fpstate->is_valloc = true; fpstate->is_guest = true;
@@ -454,21 +455,21 @@ void fpstate_init_user(struct fpstate *fpstate)fpstate_init_fstate (fpstate); } -static void __fpstate_reset (struct fpstate *fpstate) +static void __fpstate_reset (struct fpstate *fpstate,u64 xfd) {/* Initialize sizes and feature masks */ fpstate->size = fpu_kernel_cfg.default_size; fpstate->user_size = fpu_user_cfg.default_size; fpstate->xfeatures = fpu_kernel_cfg.default_features; fpstate->user_xfeatures = fpu_user_cfg.default_features; - fpstate->xfd = init_fpstate.xfd; + fpstate->xfd = xfd; }void fpstate_reset (struct fpu *fpu) {/* Set the fpstate pointer to the default fpstate */ fpu->fpstate = &fpu->__fpstate; -__fpstate_reset (fpu->fpstate); +__fpstate_reset (fpu->fpstate, init_fpstate.xfd);/* Initialize the permission related info in fpu */ fpu->perm.__state_perm = fpu_kernel_cfg.default_features;
PATCH v4 6/21 x86/fpu: Add guest support to xfd_enable_feature()
In reply to Yang ZhongFrom: Thomas Gleixner <[email protected]>
Guest support for dynamically enabled FPU features requires a few modifications to the enablement function which is currently invoked from the #NM handler:
1) Use guest permissions and sizes for the update
2) Update fpu_guest state accordingly
3) Take into account that the enabling can be triggered either from a running guest via XSETBV and MSR_IA32_XFD write emulation or from a guest restore. In the latter case the guests fpstate is not the current tasks active fpstate.
Split the function and implement the guest mechanics throughout the callchain.
--- arch/x86/kernel/fpu/xstate.c | 93 +++++++++++++++++++++--------------- arch/x86/kernel/fpu/xstate.h | 2 + 2 files changed, 56 insertions(+), 39 deletions(-)
arch/x86/kernel/fpu/xstate.c +54 -39
@@ -1499,29 +1499,6 @@ void fpstate_free(struct fpu *fpu)vfree (fpu->fpstate); } -/** -* fpu_install_fpstate - Update the active fpstate in the FPU -* -* @fpu: A struct fpu * pointer -* @newfps: A struct fpstate * pointer -* -* Returns: A null pointer if the last active fpstate is the embedded -* one or the new fpstate is already installed; -* otherwise, a pointer to the old fpstate which has to -* be freed by the caller. -*/ -static struct fpstate *fpu_install_fpstate (struct fpu *fpu, -struct fpstate *newfps) -{ -struct fpstate *oldfps = fpu->fpstate; - -if (fpu->fpstate == newfps) -return NULL ; - - fpu->fpstate = newfps; -return oldfps != &fpu->__fpstate ? oldfps :NULL ; -} -/** * fpstate_realloc - Reallocate struct fpstate for the requested new features *
@@ -1529,6 +1506,7 @@ static struct fpstate *fpu_install_fpstate(struct fpu *fpu,* of that task * @ksize: The required size for the kernel buffer * @usize: The required size for user space buffers +* @guest_fpu: Pointer to a guest FPU container. NULL for host allocations * * Note vs. vmalloc(): If the task with a vzalloc()-allocated buffer * terminates quickly, vfree()-induced IPIs may be a concern, but tasks
@@ -1537,13 +1515,13 @@ static struct fpstate *fpu_install_fpstate(struct fpu *fpu,* Returns: 0 on success, -ENOMEM on allocation error. */ static int fpstate_realloc (u64 xfeatures,unsigned int ksize, -unsigned int usize) +unsigned int usize,struct fpu_guest *guest_fpu) {struct fpu *fpu = ¤t->thread.fpu;struct fpstate *curfps, *newfps =NULL ;unsigned int fpsize; +bool in_use; - curfps = fpu->fpstate; fpsize = ksize +ALIGN (offsetof(struct fpstate , regs),64 ); newfps =vzalloc (fpsize);
@@ -1553,28 +1531,55 @@ static int fpstate_realloc(u64 xfeatures, unsigned int ksize, newfps->user_size = usize; newfps->is_valloc = true; +/* +* When a guest FPU is supplied, use @guest_fpu->fpstate +* as reference independent whether it is in use or not. +*/ + curfps = guest_fpu ? guest_fpu->fpstate : fpu->fpstate; + +/* Determine whether @curfps is the active fpstate */ + in_use = fpu->fpstate == curfps; + +if (guest_fpu) { + newfps->is_guest = true; + newfps->is_confidential = curfps->is_confidential; + newfps->in_use = curfps->in_use; + guest_fpu->xfeatures |= xfeatures; + } +fpregs_lock ();/* -* Ensure that the current state is in the registers before -* swapping fpstate as that might invalidate it due to layout -* changes. +* If @curfps is in use, ensure that the current state is in the +* registers before swapping fpstate as that might invalidate it +* due to layout changes. */ -if (test_thread_flag (TIF_NEED_FPU_LOAD )) +if (in_use &&test_thread_flag (TIF_NEED_FPU_LOAD ))fpregs_restore_userregs (); newfps->xfeatures = curfps->xfeatures | xfeatures; newfps->user_xfeatures = curfps->user_xfeatures | xfeatures; newfps->xfd = curfps->xfd & ~xfeatures; - curfps =fpu_install_fpstate (fpu, newfps); -/* Do the final updates within the locked region */ xstate_init_xcomp_bv (&newfps->regs.xsave, newfps->xfeatures); -xfd_update_state (newfps); +if (guest_fpu) { + guest_fpu->fpstate = newfps; +/* If curfps is active, update the FPU fpstate pointer */ +if (in_use) + fpu->fpstate = newfps; + }else { + fpu->fpstate = newfps; + } + +if (in_use) +xfd_update_state (fpu->fpstate);fpregs_unlock (); -vfree (curfps); +/* Only free valloc'ed state */ +if (curfps && curfps->is_valloc) +vfree (curfps); +return 0 ; }
@@ -1682,14 +1687,16 @@ static int xstate_request_perm(unsigned long idx, bool guest)return ret; } -int xfd_enable_feature (u64 xfd_err) +int __xfd_enable_feature (u64 xfd_err,struct fpu_guest *guest_fpu) {u64 xfd_event = xfd_err &XFEATURE_MASK_USER_DYNAMIC ; +struct fpu_state_perm *perm;unsigned int ksize, usize;struct fpu *fpu;if (!xfd_event) { -pr_err_once ("XFD: Invalid xfd error: %016llx\n" , xfd_err); +if (!guest_fpu) +pr_err_once ("XFD: Invalid xfd error: %016llx\n" , xfd_err);return 0 ; }
@@ -1697,14 +1704,16 @@ int xfd_enable_feature(u64 xfd_err)spin_lock_irq (¤t->sighand->siglock);/* If not permitted let it die */ -if ((xstate_get_host_group_perm () & xfd_event) != xfd_event) { +if ((xstate_get_group_perm (!!guest_fpu) & xfd_event) != xfd_event) {spin_unlock_irq (¤t->sighand->siglock);return -EPERM ; } fpu = ¤t->group_leader->thread.fpu; - ksize = fpu->perm.__state_size; - usize = fpu->perm.__user_state_size; + perm = guest_fpu ? &fpu->guest_perm : &fpu->perm; + ksize = perm->__state_size; + usize = perm->__user_state_size; +/* * The feature is permitted. State size is sufficient. Dropping * the lock is safe here even if more features are added from
@@ -1717,10 +1726,16 @@ int xfd_enable_feature(u64 xfd_err)* Try to allocate a new fpstate. If that fails there is no way * out. */ -if (fpstate_realloc (xfd_event, ksize, usize)) +if (fpstate_realloc (xfd_event, ksize, usize, guest_fpu))return -EFAULT ; return 0; } + +int xfd_enable_feature (u64 xfd_err) +{ +return __xfd_enable_feature (xfd_err,NULL ); +} +#else /* CONFIG_X86_64 */ static inline int xstate_request_perm (unsigned long idx,bool guest) {
arch/x86/kernel/fpu/xstate.h +2 -0
@@ -55,6 +55,8 @@ extern void fpu__init_system_xstate(unsigned int legacy_size);extern void *get_xsave_addr (struct xregs_state *xsave,int xfeature_nr); +extern int __xfd_enable_feature (u64 which,struct fpu_guest *guest_fpu); +static inline u64 xfeatures_mask_supervisor (void ) {return fpu_kernel_cfg.max_features &XFEATURE_MASK_SUPERVISOR_SUPPORTED ;
PATCH v4 7/21 x86/fpu: Provide fpu_update_guest_perm_features() for guest
In reply to Yang ZhongFrom: Kevin Tian <[email protected]>
KVM can require fpstate expansion in two approaches:
1) Dynamic expansion when intercepting guest updates to XCR0 and
XFD MSR;
2) Static expansion e.g. at KVM_SET_CPUID2;
The first option doesn't waste memory for legacy guest if it doesn't support XFD. However doing so introduces more complexity and also imposes an order requirement in the restoring path, i.e. XCR0/XFD must be restored before XSTATE.
Given that the agreement is to do the static approach. This is considered a better tradeoff though it does waste 8K memory for legacy guest if its CPUID includes XFD features.
Provide a wrapper to allow expanding the fpstate buffer to what guest perm allows. Once completed, both emulation and restore path don't need to worry about the buffer size.
--- arch/x86/include/asm/fpu/api.h | 1 + arch/x86/kernel/fpu/core.c | 27 +++++++++++++++++++++++++++ 2 files changed, 28 insertions(+)
arch/x86/include/asm/fpu/api.h +1 -0
@@ -138,6 +138,7 @@ extern inline u64 xstate_get_guest_group_perm(void);extern bool fpu_alloc_guest_fpstate (struct fpu_guest *gfpu);extern void fpu_free_guest_fpstate (struct fpu_guest *gfpu);extern int fpu_swap_kvm_fpstate (struct fpu_guest *gfpu,bool enter_guest); +extern int fpu_update_guest_perm_features (struct fpu_guest *guest_fpu);extern void fpu_copy_guest_fpstate_to_uabi (struct fpu_guest *gfpu,void *buf,unsigned int size,u32 pkru);extern int fpu_copy_uabi_to_guest_fpstate (struct fpu_guest *gfpu,const void *buf,u64 xcr0,u32 *vpkru);
arch/x86/kernel/fpu/core.c +27 -0
@@ -261,6 +261,33 @@ void fpu_free_guest_fpstate(struct fpu_guest *gfpu) }EXPORT_SYMBOL_GPL (fpu_free_guest_fpstate); +/* +* fpu_update_guest_perm_features - Enable xfeatures according to guest perm +* @guest_fpu: Pointer to the guest FPU container +* +* Enable all dynamic xfeatures according to guest perm. Invoked if the +* caller wants to conservatively expand fpstate buffer instead of waiting +* until XCR0 or XFD MSR is written. +* +* Return: 0 on success, error code otherwise +*/ +int fpu_update_guest_perm_features (struct fpu_guest *guest_fpu) +{ +u64 expand; + +lockdep_assert_preemption_enabled (); + +if (!IS_ENABLED (CONFIG_X86_64 )) +return 0 ; + + expand = guest_fpu->perm & ~guest_fpu->xfeatures; +if (!expand) +return 0 ; + +return __xfd_enable_feature (expand, guest_fpu); +} +EXPORT_SYMBOL_GPL (fpu_update_guest_perm_features); +int fpu_swap_kvm_fpstate (struct fpu_guest *guest_fpu,bool enter_guest) {struct fpstate *guest_fps = guest_fpu->fpstate;
PATCH v4 8/21 kvm: x86: Check and enable permitted dynamic xfeatures at KVM_SET_CPUID2
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Guest xstate permissions should be set by userspace VMM before vcpu creation. Extend KVM_SET_CPUID2 to verify that every feature reported in CPUID[0xD] has proper permission set. If permission allows, enable all xfeatures in guest cpuid with fpstate buffer sized accordingly.
This avoids introducing new KVM exit reason for reporting permission violation to userspace VMM at run-time and also removes the need of tricky fpstate buffer expansion in the emulation and restore path of XCR0 and IA32_XFD MSR.
--- arch/x86/kvm/cpuid.c | 62 +++++++++++++++++++++++++++++++++----------- 1 file changed, 47 insertions(+), 15 deletions(-)
arch/x86/kvm/cpuid.c +47 -15
@@ -81,9 +81,12 @@ static inline struct kvm_cpuid_entry2 *cpuid_entry2_find(return NULL ; } -static int kvm_check_cpuid (struct kvm_cpuid_entry2 *entries,int nent) +static int kvm_check_cpuid (struct kvm_vcpu *vcpu, +struct kvm_cpuid_entry2 *entries, +int nent) {struct kvm_cpuid_entry2 *best; +int r =0 ;/* * The existing code assumes virtual address is 48-bit or 57-bit in the
@@ -93,11 +96,40 @@ static int kvm_check_cpuid(struct kvm_cpuid_entry2 *entries, int nent)if (best) {int vaddr_bits = (best->eax &0xff00 ) >>8 ; -if (vaddr_bits !=48 && vaddr_bits !=57 && vaddr_bits !=0 ) -return -EINVAL ; +if (vaddr_bits !=48 && vaddr_bits !=57 && vaddr_bits !=0 ) { + r = -EINVAL ; + goto out; + } } -return 0 ; +/* +* Check guest permissions for dynamically-enabled xfeatures. +* Userspace VMM is expected to acquire permission before vCPU +* creation. If permission allows, enable all xfeatures with +* fpstate buffer sized accordingly. This avoids complexity of +* run-time expansion in the emulation and restore path of XCR0 +* and IA32_XFD MSR. +*/ + best =cpuid_entry2_find (entries, nent,0xd ,0 ); +if (best) { +u64 xfeatures; + + xfeatures = best->eax | ((u64 )best->edx <<32 ); +if (xfeatures & ~vcpu->arch.guest_fpu.perm) { + r = -ENXIO ; + goto out; + } + +if (xfeatures != vcpu->arch.guest_fpu.xfeatures) { + r =fpu_update_guest_perm_features ( + &vcpu->arch.guest_fpu); +if (r) + goto out; + } + } + +out: +return r; }static void kvm_update_kvm_cpuid_base (struct kvm_vcpu *vcpu)
@@ -277,21 +309,21 @@ u64 kvm_vcpu_reserved_gpa_bits_raw(struct kvm_vcpu *vcpu)static int kvm_set_cpuid (struct kvm_vcpu *vcpu,struct kvm_cpuid_entry2 *e2,int nent) { -int r; +int r; - r =kvm_check_cpuid (e2, nent); -if (r) -return r; + r =kvm_check_cpuid (vcpu, e2, nent); +if (r) +return r; -kvfree (vcpu->arch.cpuid_entries); - vcpu->arch.cpuid_entries = e2; - vcpu->arch.cpuid_nent = nent; +kvfree (vcpu->arch.cpuid_entries); + vcpu->arch.cpuid_entries = e2; + vcpu->arch.cpuid_nent = nent; -kvm_update_kvm_cpuid_base (vcpu); -kvm_update_cpuid_runtime (vcpu); -kvm_vcpu_after_set_cpuid (vcpu); +kvm_update_kvm_cpuid_base (vcpu); +kvm_update_cpuid_runtime (vcpu); +kvm_vcpu_after_set_cpuid (vcpu); -return 0 ; +return 0 ; }/* when an old userspace process fills a new kernel module */
Sean Christopherson
maintainerRe: [PATCH v4 08/21] kvm: x86: Check and enable permitted dynamic xfeatures at KVM_SET_CPUID2
Quoted 11 lines · Yang Zhong
From: Jing Liu <[email protected]>
Guest xstate permissions should be set by userspace VMM before vcpu creation. Extend
KVM_SET_CPUID2to verify that every feature reported in CPUID[0xD] has proper permission set. If permission allows, enable all xfeatures in guest cpuid with fpstate buffer sized accordingly.This avoids introducing new KVM exit reason for reporting permission violation to userspace VMM at run-time and also removes the need of tricky fpstate buffer expansion in the emulation and restore path of XCR0 and
IA32_XFDMSR.
How so? __do_cpuid_func() restricts what is advertised to userspace based on xstate_get_guest_group_perm(), so it's not like KVM is advertising something it can't provide? There should never be any danger to KVM that's mitigated by restricing guest CPUID because KVM can and should check vcpu->arch.guest_fpu.perm instead of guest CPUID.
In other words, I believe you're conflating the overall approach of requiring userspace to pre-acquire the necessary permissions with enforcing what userspace advertises to the guest.
Quoted 26 lines
Signed-off-by Jing Liu <[email protected]>Signed-off-by Kevin Tian <[email protected]>Signed-off-by Yang Zhong <[email protected]>--- arch/x86/kvm/cpuid.c | 62 +++++++++++++++++++++++++++++++++----------- 1 file changed, 47 insertions(+), 15 deletions(-)arch/x86/kvm/cpuid.c +4
-1@@ -81,9 +81,12 @@ static inline struct kvm_cpuid_entry2 *cpuid_entry2_find(return NULL ; } -static int kvm_check_cpuid (struct kvm_cpuid_entry2 *entries,int nent) +static int kvm_check_cpuid (struct kvm_vcpu *vcpu, +struct kvm_cpuid_entry2 *entries, +int nent) {struct kvm_cpuid_entry2 *best; +int r =0 ;/* * The existing code assumes virtual address is 48-bit or 57-bit in the
arch/x86/kvm/cpuid.c static int kvm_check_cpuid(struct kvm_cpuid_entry2 *entries, int nent) · line 101
Quoted 8 lines
arch/x86/kvm/cpuid.c +3
-2@@ -93,11 +96,40 @@ static int kvm_check_cpuid(struct kvm_cpuid_entry2 *entries, int nent)if (best) {int vaddr_bits = (best->eax &0xff00 ) >>8 ; -if (vaddr_bits !=48 && vaddr_bits !=57 && vaddr_bits !=0 ) -return -EINVAL ; +if (vaddr_bits !=48 && vaddr_bits !=57 && vaddr_bits !=0 ) { + r = -EINVAL ; + goto out;
Please don't change this to a goto, a return is perfectly ok and more readable as it doesn't imply there's some functional change that needs to be unwound at the end.
Quoted 9 lines
arch/x86/kvm/cpuid.c +6
-1+ } } -return 0 ; +/* +* Check guest permissions for dynamically-enabled xfeatures. +* Userspace VMM is expected to acquire permission before vCPU +* creation. If permission allows, enable all xfeatures with +* fpstate buffer sized accordingly. This avoids complexity of
arch/x86/kvm/cpuid.c static int kvm_check_cpuid(struct kvm_cpuid_entry2 *entries, int nent) · line 119
Quoted 10 lines
arch/x86/kvm/cpuid.c +10
-0+* run-time expansion in the emulation and restore path of XCR0 +* and IA32_XFD MSR. +*/ + best =cpuid_entry2_find (entries, nent,0xd ,0 ); +if (best) { +u64 xfeatures; + + xfeatures = best->eax | ((u64 )best->edx <<32 ); +if (xfeatures & ~vcpu->arch.guest_fpu.perm) { + r = -ENXIO ;
ENXIO is a rather odd error code for insufficient permissions, especially since the FPU returns -EPERM for what is effectively the same check.
arch/x86/kvm/cpuid.c static int kvm_check_cpuid(struct kvm_cpuid_entry2 *entries, int nent) · line 123
Quoted 4 lines
arch/x86/kvm/cpuid.c +4
-0+ goto out; + } + +if (xfeatures != vcpu->arch.guest_fpu.xfeatures) {
xfeatures is obviously not consumed anywhere, which is super confusing and arguably wrong, e.g. if userspace advertises xfeatures that are a subset of vcpu->arch.guest_fpu.perm, this will expand XSAVE state beyond what userspace actually wants to advertise to the guest. The really confusing case would be if userspace reduced xfeatures relative to vcpu->arch.guest_fpu.xfeatures and got an -ENOMEM due to the FPU failing to expand the XSAVE size.
I don't care about the waste of memory, and IIUC userspace would have to intentionally request permissions for the guest that it then ignores, but that doesn't make the code any less confusing. And as written, this check also prevents advertising non-XFD features that are not supported in hardware. I doubt there's a production use case for that (though MPX deprecation comes close), but I've certainly exposed unsupported features to a guest for testing purposes.
Rather than bleed details from the FPU into KVM, why not have the FPU do any and all checks? That also gives the FPU access to requested xfeatures so that it can opportunistically avoid unnecessary expansion. We can also tweak the kernel APIs to be more particular about input values.
At that point, I would be ok with fpu_update_guest_perm_features() rejecting attempts to advertise features that are not permitted, because then it's an FPU policy, not a KVM policy, and there's a valid reason for said policy. It's a bit of a pedantic distinction, but to me it matters because having KVM explicitly restrict guest CPUID implies that doing so is necessary for KVM correctness, which AFAICT is not the case.
E.g. in KVM
/* * Exposing dynamic xfeatures to the guest requires additional enabling * in the FPU, e.g. to expand the guest XSAVE state size. */ best = cpuid_entry2_find(entries, nent, 0xd, 0); if (!best) return 0; xfeatures = best->eax | ((u64)best->edx << 32); xfeatures &= XFEATURE_MASK_USER_DYNAMIC; if (!xfeatures) return 0; return fpu_enable_guest_xfd_features(&vcpu->arch.guest_fpu, xfeatures);
and then
int fpu_enable_guest_xfd_features(struct fpu_guest *guest_fpu, u64 xfeatures)
{
lockdep_assert_preemption_enabled();
/* Nothing to do if all requested features are already enabled. */
xfeatures &= ~guest_fpu->xfeatures;
if (!xfeatures)
return 0;
/* Dynamic xfeatures are not supported with 32-bit kernels. */
if (!IS_ENABLED(CONFIG_X86_64))
return -EPERM;
return __xfd_enable_feature(xfeatures, guest_fpu);}
with
int __xfd_enable_feature(u64 xfd_err, struct fpu_guest *guest_fpu)
{
struct fpu_state_perm *perm;
unsigned int ksize, usize;
struct fpu *fpu;
if (WARN_ON_ONCE(!xfd_err || (xfd_err & ~XFEATURE_MASK_USER_DYNAMIC)))
return 0;
...}
which addresses several things:
a) avoids explicitly restricing guest CPUID in KVM, and in particular doesn't prevent userspace from advertising non-XFD features that aren't supported in hardware, which for better or worse is allowed today.
b) returns -EPERM instead of '0' when userspace attempts to enable dynamic xfeatures with 32-bit kernels, which isn't a bug as posted only because KVM pre-checks vcpu->arch.guest_fpu.perm.
b) avoids reading guest_perm outside of siglock, which was technically a TOCTOU "bug", though it didn't put the kernel at risk because __xstate_request_perm() doesn't allow reducing permissions.
c) allows __xfd_enable_feature() to require the caller to provide just XFD features
Quoted 9 lines
arch/x86/kvm/cpuid.c +9
-0+ r =fpu_update_guest_perm_features ( + &vcpu->arch.guest_fpu); +if (r) + goto out; + } + } + +out: +return r
RE: [PATCH v4 08/21] kvm: x86: Check and enable permitted dynamic xfeatures at KVM_SET_CPUID2
Quoted 16 lines
From: Sean Christopherson <[email protected]>
Sent: Thursday, December 30, 2021 12:55 AMQuoted 11 lines · Yang Zhong
From: Jing Liu <[email protected]>
Guest xstate permissions should be set by userspace VMM before vcpu creation. Extend
KVM_SET_CPUID2to verify that every feature reported in CPUID[0xD] has proper permission set. If permission allows, enable all xfeatures in guest cpuid with fpstate buffer sized accordingly.This avoids introducing new KVM exit reason for reporting permission violation to userspace VMM at run-time and also removes the need of tricky fpstate buffer expansion in the emulation and restore path of XCR0 and
IA32_XFDMSR.
Quoted 8 lines
How so?
__do_cpuid_func()restricts what is advertised to userspace based onxstate_get_guest_group_perm(), so it's not like KVM is advertising something it
can't provide? There should never be any danger to KVM that's mitigated by restricing guest CPUID because KVM can and should check vcpu-Quoted 1 line
arch.guest_fpu.perm
instead of guest CPUID.
Well, above explains why we choose to expand fpstate buffer at KVM_SET_CPUID2 instead of in the emulation path when required permissions have been set, as discussed here:
https://lore.kernel.org/all/[email protected]/
Quoted 38 lines
In other words, I believe you're conflating the overall approach of requiring userspace to pre-acquire the necessary permissions with enforcing what userspace
advertises to the guest.Quoted 5 lines
Signed-off-by Jing Liu <[email protected]>Signed-off-by Kevin Tian <[email protected]>Signed-off-by Yang Zhong <[email protected]>--- arch/x86/kvm/cpuid.c | 62 +++++++++++++++++++++++++++++++++-----------
arch/x86/kvm/cpuid.c static inline struct kvm_cpuid_entry2
Quoted 7 lines
1 file changed, 47 insertions(+), 15 deletions(-)arch/x86/kvm/cpuid.c +0
-0@@ -81,9 +81,12 @@ static inline struct kvm_cpuid_entry2*cpuid_entry2_find(
arch/x86/kvm/cpuid.c static int kvm_check_cpuid(struct
Quoted 14 lines
arch/x86/kvm/cpuid.c +4
-1return NULL ; } -static int kvm_check_cpuid (struct kvm_cpuid_entry2 *entries,int nent) +static int kvm_check_cpuid (struct kvm_vcpu *vcpu, +struct kvm_cpuid_entry2 *entries, +int nent) {struct kvm_cpuid_entry2 *best; +int r =0 ;/* * The existing code assumes virtual address is 48-bit or 57-bit in the @@ -93,11 +96,40 @@ static int kvm_check_cpuid(struct
kvm_cpuid_entry2*entries, int nent)Quoted 3 lines
arch/x86/kvm/cpuid.c +0
-0if (best) {int vaddr_bits = (best->eax &0xff00 ) >>8 ;
Quoted 10 lines
arch/x86/kvm/cpuid.c static int kvm_check_cpuid(struct kvm_cpuid_entry2 *entries, int nent) · line 101
Quoted 5 lines
arch/x86/kvm/cpuid.c +3
-2-if (vaddr_bits !=48 && vaddr_bits !=57 && vaddr_bits !=0 ) -return -EINVAL ; +if (vaddr_bits !=48 && vaddr_bits !=57 && vaddr_bits !=0 ) { + r = -EINVAL ; + goto out;Please don't change this to a goto, a return is perfectly ok and more readable as it doesn't imply there's some functional change that needs to be unwound at
the end.
will fix
Quoted 24 lines
Quoted 10 lines
arch/x86/kvm/cpuid.c +6
-1+ } } -return 0 ; +/* +* Check guest permissions for dynamically-enabled xfeatures. +* Userspace VMM is expected to acquire permission before vCPU +* creation. If permission allows, enable all xfeatures with +* fpstate buffer sized accordingly. This avoids complexity of arch/x86/kvm/cpuid.c static int kvm_check_cpuid(struct kvm_cpuid_entry2 *entries, int nent) · line 119
Quoted 10 lines
arch/x86/kvm/cpuid.c +10
-0+* run-time expansion in the emulation and restore path of XCR0 +* and IA32_XFD MSR. +*/ + best =cpuid_entry2_find (entries, nent,0xd ,0 ); +if (best) { +u64 xfeatures; + + xfeatures = best->eax | ((u64 )best->edx <<32 ); +if (xfeatures & ~vcpu->arch.guest_fpu.perm) { + r = -ENXIO ;ENXIO is a rather odd error code for insufficient permissions, especially since the FPU returns
-EPERMfor what is effectively the same check.
Quoted 14 lines
arch/x86/kvm/cpuid.c static int kvm_check_cpuid(struct kvm_cpuid_entry2 *entries, int nent) · line 123
Quoted 4 lines
arch/x86/kvm/cpuid.c +4
-0+ goto out; + } + +if (xfeatures != vcpu->arch.guest_fpu.xfeatures) {xfeatures is obviously not consumed anywhere, which is super confusing and arguably wrong, e.g. if userspace advertises xfeatures that are a subset of
vcpu->arch.guest_fpu.perm, this will expand XSAVE state beyond what userspace
actually wants to advertise to the guest. The really confusing case would be if
userspace reduced xfeatures relative tovcpu->arch.guest_fpu.xfeaturesand got
an-ENOMEMdue to the FPU failing to expand the XSAVE size.
You are right.
Quoted 11 lines
I don't care about the waste of memory, and IIUC userspace would have to intentionally request permissions for the guest that it then ignores, but that doesn't make the code any less confusing. And as written, this check also prevents
advertising non-XFD features that are not supported in hardware. I doubt there's
a production use case for that (though MPX deprecation comes close), but I've
certainly exposed unsupported features to a guest for testing purposes.
Quoted 6 lines
Rather than bleed details from the FPU into KVM, why not have the FPU do any and
all checks? That also gives the FPU access to requested xfeatures so that it can opportunistically avoid unnecessary expansion. We can also tweak the kernel
APIs to be more particular about input values.
All above makes sense, especially when we combine permission check and buffer expansion in one step now.
Quoted 52 lines
At that point, I would be ok with
fpu_update_guest_perm_features()rejecting
attempts to advertise features that are not permitted, because then it's an FPU
policy, not a KVM policy, and there's a valid reason for said policy. It's a bit of a pedantic distinction, but to me it matters because having KVM explicitly restrict guest CPUID implies that doing so is necessary for KVM correctness, which
AFAICT is not the case.E.g. in KVM
/* * Exposing dynamic xfeatures to the guest requires additional enabling * in the FPU, e.g. to expand the guest XSAVE state size. */ best = cpuid_entry2_find(entries, nent, 0xd, 0); if (!best) return 0; xfeatures = best->eax | ((u64)best->edx << 32); xfeatures &= XFEATURE_MASK_USER_DYNAMIC; if (!xfeatures) return 0; return fpu_enable_guest_xfd_features(&vcpu->arch.guest_fpu,xfeatures);
and then
int fpu_enable_guest_xfd_features(struct
fpu_guest*guest_fpu, u64
xfeatures){ lockdep_assert_preemption_enabled(); /* Nothing to do if all requested features are already enabled. */ xfeatures &= ~guest_fpu->xfeatures; if (!xfeatures) return 0; /* Dynamic xfeatures are not supported with 32-bit kernels. */ if (!IS_ENABLED(CONFIG_X86_64)) return -EPERM; return __xfd_enable_feature(xfeatures, guest_fpu);}
with
int
__xfd_enable_feature(u64 xfd_err, struct fpu_guest *guest_fpu)
Quoted 8 lines
{
struct fpu_state_perm *perm;
unsigned int ksize, usize;
struct fpu *fpu;
if (WARN_ON_ONCE(!xfd_err || (xfd_err &
~XFEATURE_MASK_USER_DYNAMIC)))
return 0;Currently this is done as:
int __xfd_enable_feature(u64 xfd_err, struct fpu_guest *guest_fpu)
{
u64 xfd_event = xfd_err & XFEATURE_MASK_USER_DYNAMIC;
...
if (!xfd_event) {
if (!guest_fpu)
pr_err_once("XFD: Invalid xfd error: %016llx\n", xfd_err);
return 0;
}
...
}is it necessary to convert the error print to WARN_ON() (and also apply to guest_fpu)?
Quoted 16 lines
...}
which addresses several things:
a) avoids explicitly restricing guest CPUID in KVM, and in particular doesn't prevent userspace from advertising non-XFD features that aren't
supported in
hardware, which for better or worse is allowed today.b) returns
-EPERMinstead of '0' when userspace attempts to enable
dynamic
xfeatures with 32-bit kernels, which isn't a bug as posted only because KVM pre-checksvcpu->arch.guest_fpu.perm.
Quoted 9 lines
b) avoids reading
guest_permoutside of siglock, which was technically a
TOCTOU
"bug", though it didn't put the kernel at risk because__xstate_request_perm()
doesn't allow reducing permissions.c) allows
__xfd_enable_feature()to require the caller to provide just XFD features
All the sample code looks good to me, except WARN_ON() introduced in the last bullet.
If no objection from other maintainers, we can incorporate it in next version.
Thanks
Kevin
PATCH v4 9/21 x86/fpu: Provide fpu_update_guest_xfd() for IA32_XFD emulation
In reply to Yang ZhongFrom: Kevin Tian <[email protected]>
Guest XFD can be updated either in the emulation path or in the restore path.
Provide a wrapper to update guest_fpu::fpstate::xfd. If the guest fpstate is currently in-use, also update the per-cpu xfd cache and the actual MSR.
--- arch/x86/include/asm/fpu/api.h | 6 ++++++ arch/x86/kernel/fpu/core.c | 12 ++++++++++++ 2 files changed, 18 insertions(+)
arch/x86/include/asm/fpu/api.h +6 -0
@@ -140,6 +140,12 @@ extern void fpu_free_guest_fpstate(struct fpu_guest *gfpu);extern int fpu_swap_kvm_fpstate (struct fpu_guest *gfpu,bool enter_guest);extern int fpu_update_guest_perm_features (struct fpu_guest *guest_fpu); +#ifdef CONFIG_X86_64 +extern void fpu_update_guest_xfd (struct fpu_guest *guest_fpu,u64 xfd); +#else +static inline void fpu_update_guest_xfd (struct fpu_guest *guest_fpu,u64 xfd) { } +#endif +extern void fpu_copy_guest_fpstate_to_uabi (struct fpu_guest *gfpu,void *buf,unsigned int size,u32 pkru);extern int fpu_copy_uabi_to_guest_fpstate (struct fpu_guest *gfpu,const void *buf,u64 xcr0,u32 *vpkru);
arch/x86/kernel/fpu/core.c +12 -0
@@ -288,6 +288,18 @@ int fpu_update_guest_perm_features(struct fpu_guest *guest_fpu) }EXPORT_SYMBOL_GPL (fpu_update_guest_perm_features); +#ifdef CONFIG_X86_64 +void fpu_update_guest_xfd (struct fpu_guest *guest_fpu,u64 xfd) +{ +fpregs_lock (); + guest_fpu->fpstate->xfd = xfd; +if (guest_fpu->fpstate->in_use) +xfd_update_state (guest_fpu->fpstate); +fpregs_unlock (); +} +EXPORT_SYMBOL_GPL (fpu_update_guest_xfd); +#endif /* CONFIG_X86_64 */ +int fpu_swap_kvm_fpstate (struct fpu_guest *guest_fpu,bool enter_guest) {struct fpstate *guest_fps = guest_fpu->fpstate;
PATCH v4 10/21 kvm: x86: Add emulation for IA32_XFD
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Intel's eXtended Feature Disable (XFD) feature allows the software to dynamically adjust fpstate buffer size for XSAVE features which have large state.
Because fpstate has been expanded for all possible dynamic xstates at KVM_SET_CPUID2, emulation of the IA32_XFD MSR is straightforward. For write just call fpu_update_guest_xfd() to update the guest fpu container once all the sanity checks are passed. For read then return the cached value in the container.
--- arch/x86/kvm/x86.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
arch/x86/kvm/x86.c +27 -0
@@ -1359,6 +1359,7 @@ static const u32 msrs_to_save_all[] = {MSR_F15H_PERF_CTL3 ,MSR_F15H_PERF_CTL4 ,MSR_F15H_PERF_CTL5 ,MSR_F15H_PERF_CTR0 ,MSR_F15H_PERF_CTR1 ,MSR_F15H_PERF_CTR2 ,MSR_F15H_PERF_CTR3 ,MSR_F15H_PERF_CTR4 ,MSR_F15H_PERF_CTR5 , + MSR_IA32_XFD, };static u32 msrs_to_save[ARRAY_SIZE (msrs_to_save_all)];
@@ -3669,6 +3670,19 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)return 1 ; vcpu->arch.msr_misc_features_enables = data;break ; +#ifdef CONFIG_X86_64 +case MSR_IA32_XFD : +if (!msr_info->host_initiated && + !guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ; + +if (data & ~(XFEATURE_MASK_USER_DYNAMIC & + vcpu->arch.guest_supported_xcr0)) +return 1 ; + +fpu_update_guest_xfd (&vcpu->arch.guest_fpu, data); +break ; +#endif default :if (kvm_pmu_is_valid_msr (vcpu, msr))return kvm_pmu_set_msr (vcpu, msr_info);
@@ -3989,6 +4003,15 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)case MSR_K7_HWCR : msr_info->data = vcpu->arch.msr_hwcr;break ; +#ifdef CONFIG_X86_64 +case MSR_IA32_XFD : +if (!msr_info->host_initiated && + !guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ; + + msr_info->data = vcpu->arch.guest_fpu.fpstate->xfd; +break ; +#endif default :if (kvm_pmu_is_valid_msr (vcpu, msr_info->index))return kvm_pmu_get_msr (vcpu, msr_info);
@@ -6422,6 +6445,10 @@ static void kvm_init_msr_list(void) min(INTEL_PMC_MAX_GENERIC, x86_pmu.num_counters_gp)) continue;break ; +case MSR_IA32_XFD : +if (!kvm_cpu_cap_has (X86_FEATURE_XFD )) +continue ; +break ;default :break ; }
Sean Christopherson
maintainerRe: [PATCH v4 10/21] kvm: x86: Add emulation for IA32_XFD
Quoted 69 lines · Yang Zhong
From: Jing Liu <[email protected]>
Intel's eXtended Feature Disable (XFD) feature allows the software to dynamically adjust fpstate buffer size for XSAVE features which have large state.
Because fpstate has been expanded for all possible dynamic xstates at
KVM_SET_CPUID2, emulation of theIA32_XFDMSR is straightforward. For write just callfpu_update_guest_xfd()to update the guest fpu container once all the sanity checks are passed. For read then return the cached value in the container.Signed-off-by Zeng Guang <[email protected]>Signed-off-by Wei Wang <[email protected]>Signed-off-by Jing Liu <[email protected]>Signed-off-by Yang Zhong <[email protected]>--- arch/x86/kvm/x86.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)arch/x86/kvm/x86.c +23
-0@@ -1359,6 +1359,7 @@ static const u32 msrs_to_save_all[] = {MSR_F15H_PERF_CTL3 ,MSR_F15H_PERF_CTL4 ,MSR_F15H_PERF_CTL5 ,MSR_F15H_PERF_CTR0 ,MSR_F15H_PERF_CTR1 ,MSR_F15H_PERF_CTR2 ,MSR_F15H_PERF_CTR3 ,MSR_F15H_PERF_CTR4 ,MSR_F15H_PERF_CTR5 , + MSR_IA32_XFD, };static u32 msrs_to_save[ARRAY_SIZE (msrs_to_save_all)];@@ -3669,6 +3670,19 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)return 1 ; vcpu->arch.msr_misc_features_enables = data;break ; +#ifdef CONFIG_X86_64 +case MSR_IA32_XFD : +if (!msr_info->host_initiated && + !guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ; + +if (data & ~(XFEATURE_MASK_USER_DYNAMIC & + vcpu->arch.guest_supported_xcr0)) +return 1 ; + +fpu_update_guest_xfd (&vcpu->arch.guest_fpu, data); +break ; +#endif default :if (kvm_pmu_is_valid_msr (vcpu, msr))return kvm_pmu_set_msr (vcpu, msr_info);@@ -3989,6 +4003,15 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)case MSR_K7_HWCR : msr_info->data = vcpu->arch.msr_hwcr;break ; +#ifdef CONFIG_X86_64 +case MSR_IA32_XFD : +if (!msr_info->host_initiated && + !guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ; + + msr_info->data = vcpu->arch.guest_fpu.fpstate->xfd; +break ; +#endif default :if (kvm_pmu_is_valid_msr (vcpu, msr_info->index))return kvm_pmu_get_msr (vcpu, msr_info);
arch/x86/kvm/x86.c static void kvm_init_msr_list(void) · line 6450
Quoted 6 lines
arch/x86/kvm/x86.c +3
-0@@ -6422,6 +6445,10 @@ static void kvm_init_msr_list(void) min(INTEL_PMC_MAX_GENERIC, x86_pmu.num_counters_gp)) continue;break ; +case MSR_IA32_XFD : +if (!kvm_cpu_cap_has (X86_FEATURE_XFD )) +continue ;
I suspect the 32-bit host support is wrong. The kernel's handle_xfd_event() checks for 64-bit support in addition to the CPU feature itself, which implies that the feature can be reported in boot_cpu_data for 32-bit kernels.
static bool handle_xfd_event(struct pt_regs *regs)
{
u64 xfd_err;
int err;
if (!IS_ENABLED(CONFIG_X86_64) || !cpu_feature_enabled(X86_FEATURE_XFD))
return false;
...}
In this specific case, that means KVM will tell userspace it needs to mgirate MSR_IA32_XFD, and then reject attempts to read/write the MSR.
If 32-bit host kernels do not explicitly suppress X86_FEATURE_XFD, then KVM needs to do:
arch/x86/kvm/cpuid.c +3 -1
@@ -455,9 +455,11 @@ void kvm_set_cpu_caps(void)#ifdef CONFIG_X86_64 unsigned int f_gbpages =F (GBPAGES );unsigned int f_lm =F (LM ); +unsigned int f_xfd =F (XFD );#else unsigned int f_gbpages =0 ;unsigned int f_lm =0 ; +unsigned int f_xfd =0 ;#endif memset (kvm_cpu_caps,0 ,sizeof (kvm_cpu_caps));
@@ -545,7 +547,7 @@ void kvm_set_cpu_caps(void) );kvm_cpu_cap_mask (CPUID_D_1_EAX , -F (XSAVEOPT ) |F (XSAVEC ) |F (XGETBV1 ) |F (XSAVES ) |F (XFD ) +F (XSAVEOPT ) |F (XSAVEC ) |F (XGETBV1 ) |F (XSAVES ) | f_xfd ); kvm_cpu_cap_init_scattered(CPUID_12_EAX,
Quoted 4 lines
arch/x86/kvm/x86.c +1
-0+break ;default :break ; }
RE: [PATCH v4 10/21] kvm: x86: Add emulation for IA32_XFD
Quoted 100 lines
From: Sean Christopherson <[email protected]>
Sent: Wednesday, January 5, 2022 3:33 AMQuoted 25 lines · Yang Zhong
From: Jing Liu <[email protected]>
Intel's eXtended Feature Disable (XFD) feature allows the software to dynamically adjust fpstate buffer size for XSAVE features which have large state.
Because fpstate has been expanded for all possible dynamic xstates at
KVM_SET_CPUID2, emulation of theIA32_XFDMSR is straightforward. For write just callfpu_update_guest_xfd()to update the guest fpu container once all the sanity checks are passed. For read then return the cached value in the container.Signed-off-by Zeng Guang <[email protected]>Signed-off-by Wei Wang <[email protected]>Signed-off-by Jing Liu <[email protected]>Signed-off-by Yang Zhong <[email protected]>--- arch/x86/kvm/x86.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)arch/x86/kvm/x86.c +0
-0arch/x86/kvm/x86.c static const u32 msrs_to_save_all[] = { · line 1359
Quoted 1 line
arch/x86/kvm/x86.c +0
-0@@ -1359,6 +1359,7 @@ static const u32 msrs_to_save_all[] = { MSR_F15H_PERF_CTL3, MSR_F15H_PERF_CTL4,
MSR_F15H_PERF_CTL5,arch/x86/kvm/x86.c static const u32 msrs_to_save_all[] = { · line 1360
Quoted 1 line
arch/x86/kvm/x86.c +0
-0MSR_F15H_PERF_CTR0, MSR_F15H_PERF_CTR1,
MSR_F15H_PERF_CTR2,arch/x86/kvm/x86.c static const u32 msrs_to_save_all[] = { · line 1361
Quoted 1 line
arch/x86/kvm/x86.c +0
-0MSR_F15H_PERF_CTR3, MSR_F15H_PERF_CTR4,
MSR_F15H_PERF_CTR5,arch/x86/kvm/x86.c int kvm_set_msr_common(struct kvm_vcpu
Quoted 5 lines
arch/x86/kvm/x86.c +1
-0+ MSR_IA32_XFD, };static u32 msrs_to_save[ARRAY_SIZE (msrs_to_save_all)];@@ -3669,6 +3670,19 @@ int kvm_set_msr_common(struct kvm_vcpu*vcpu, struct
msr_data*msr_info)arch/x86/kvm/x86.c int kvm_get_msr_common(struct kvm_vcpu
Quoted 20 lines
arch/x86/kvm/x86.c +13
-0return 1 ; vcpu->arch.msr_misc_features_enables = data;break ; +#ifdef CONFIG_X86_64 +case MSR_IA32_XFD : +if (!msr_info->host_initiated && + !guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ; + +if (data & ~(XFEATURE_MASK_USER_DYNAMIC & + vcpu->arch.guest_supported_xcr0)) +return 1 ; + +fpu_update_guest_xfd (&vcpu->arch.guest_fpu, data); +break ; +#endif default :if (kvm_pmu_is_valid_msr (vcpu, msr))return kvm_pmu_set_msr (vcpu, msr_info);@@ -3989,6 +4003,15 @@ int kvm_get_msr_common(struct kvm_vcpu*vcpu, struct
msr_data*msr_info)Quoted 16 lines
arch/x86/kvm/x86.c +9
-0case MSR_K7_HWCR : msr_info->data = vcpu->arch.msr_hwcr;break ; +#ifdef CONFIG_X86_64 +case MSR_IA32_XFD : +if (!msr_info->host_initiated && + !guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ; + + msr_info->data = vcpu->arch.guest_fpu.fpstate->xfd; +break ; +#endif default :if (kvm_pmu_is_valid_msr (vcpu, msr_info->index))return kvm_pmu_get_msr (vcpu, msr_info);arch/x86/kvm/x86.c static void kvm_init_msr_list(void) · line 6445
Quoted 1 line
arch/x86/kvm/x86.c +0
-0@@ -6422,6 +6445,10 @@ static void kvm_init_msr_list(void) min(INTEL_PMC_MAX_GENERIC,x86_pmu.num_counters_gp))
arch/x86/kvm/x86.c static void kvm_init_msr_list(void) · line 6450
Quoted 5 lines
arch/x86/kvm/x86.c +3
-0continue ;break ; +case MSR_IA32_XFD : +if (!kvm_cpu_cap_has (X86_FEATURE_XFD )) +continue ;I suspect the 32-bit host support is wrong. The kernel's
handle_xfd_event()checks for 64-bit support in addition to the CPU feature itself, which implies that the feature can be reported inboot_cpu_datafor 32-bit kernels.static bool
handle_xfd_event(struct pt_regs *regs){ u64 xfd_err; int err; if (!IS_ENABLED(CONFIG_X86_64) || !cpu_feature_enabled(X86_FEATURE_XFD)) return false; ...
Quoted 6 lines
}
In this specific case, that means KVM will tell userspace it needs to mgirate
MSR_IA32_XFD, and then reject attempts to read/write the MSR.If 32-bit host kernels do not explicitly suppress
X86_FEATURE_XFD, then KVM
I didn't find explicit suppress
Quoted 21 lines
needs
to do:arch/x86/kvm/cpuid.c +2
-0@@ -455,9 +455,11 @@ void kvm_set_cpu_caps(void)#ifdef CONFIG_X86_64 unsigned int f_gbpages =F (GBPAGES );unsigned int f_lm =F (LM ); +unsigned int f_xfd =F (XFD );#else unsigned int f_gbpages =0 ;unsigned int f_lm =0 ; +unsigned int f_xfd =0 ;#endif memset (kvm_cpu_caps,0 ,sizeof (kvm_cpu_caps));
arch/x86/kvm/cpuid.c void kvm_set_cpu_caps(void) · line 553
Quoted 8 lines
arch/x86/kvm/cpuid.c +1
-1@@ -545,7 +547,7 @@ void kvm_set_cpu_caps(void) );kvm_cpu_cap_mask (CPUID_D_1_EAX , -F (XSAVEOPT ) |F (XSAVEC ) |F (XGETBV1 ) |F (XSAVES ) |F (XFD ) +F (XSAVEOPT ) |F (XSAVEC ) |F (XGETBV1 ) |F (XSAVES ) | f_xfd ); kvm_cpu_cap_init_scattered(CPUID_12_EAX,
so this change makes sense. will incorporate it in next version.
Thanks
Kevin
PATCH v4 11/21 x86/fpu: Prepare xfd_err in struct fpu_guest
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
When XFD causes an instruction to generate #NM, IA32_XFD_ERR contains information about which disabled state components are being accessed. The #NM handler is expected to check this information and then enable the state components by clearing IA32_XFD for the faulting task (if having permission).
If the XFD_ERR value generated in guest is consumed/clobbered by the host before the guest itself doing so. This may lead to non-XFD-related #NM treated as XFD #NM in host (due to non-zero value in XFD_ERR), or XFD-related #NM treated as non-XFD #NM in guest (XFD_ERR cleared by the host #NM handler).
Introduce a new field in fpu_guest to save the guest xfd_err value. KVM is expected to save guest xfd_err before preemption is enabled and restore it right before entering the guest (with preemption disabled).
--- arch/x86/include/asm/fpu/types.h | 5 +++++ 1 file changed, 5 insertions(+)
arch/x86/include/asm/fpu/types.h +5 -0
@@ -517,6 +517,11 @@ struct fpu_guest {*/ u64 perm; +/* +* @xfd_err: Save the guest value. +*/ +u64 xfd_err; +/* * @fpstate: Pointer to the allocated guest fpstate */
PATCH v4 12/21 kvm: x86: Intercept #NM for saving IA32_XFD_ERR
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Guest IA32_XFD_ERR is generally modified in two places:
- Set by CPU when #NM is triggered;
- Cleared by guest in its #NM handler;
Intercept #NM for the first case when a nonzero value is written to IA32_XFD. Nonzero indicates that the guest is willing to do dynamic fpstate expansion for certain xfeatures, thus KVM needs to manage and virtualize guest XFD_ERR properly. The vcpu exception bitmap is updated according to guest_fpu::xfd in XFD write emulation.
Save the current XFD_ERR value to the guest_fpu container in the #NM VM-exit handler. This must be done with interrupt disabled, otherwise the unsaved MSR value may be clobbered by host operations.
Inject a virtual #NM to the guest after saving the MSR value.
Restore the host value (always ZERO outside of the host #NM handler) before enabling interrupt.
Restore the guest value from the guest_fpu container right before entering the guest (with interrupt disabled).
--- arch/x86/kvm/vmx/vmcs.h | 5 +++++ arch/x86/kvm/vmx/vmx.c | 26 +++++++++++++++++++++++++- arch/x86/kvm/x86.c | 6 ++++++ 3 files changed, 36 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmcs.h +5 -0
@@ -129,6 +129,11 @@ static inline bool is_machine_check(u32 intr_info)return is_exception_n (intr_info,MC_VECTOR ); } +static inline bool is_nm_fault (u32 intr_info) +{ +return is_exception_n (intr_info,NM_VECTOR ); +} +/* Undocumented: icebp/int1 */ static inline bool is_icebp (u32 intr_info) {
arch/x86/kvm/vmx/vmx.c +25 -1
@@ -36,6 +36,7 @@#include <asm/debugreg.h> #include <asm/desc.h> #include <asm/fpu/api.h> +#include <asm/fpu/xstate.h> #include <asm/idtentry.h> #include <asm/io.h> #include <asm/irq_remapping.h>
@@ -763,6 +764,13 @@ void vmx_update_exception_bitmap(struct kvm_vcpu *vcpu)vmcs_write32 (PAGE_FAULT_ERROR_CODE_MATCH , match); } +/* +* Trap #NM if guest xfd contains a non-zero value so guest XFD_ERR +* can be saved timely. +*/ +if (vcpu->arch.guest_fpu.fpstate->xfd) + eb |= (1u <<NM_VECTOR ); +vmcs_write32 (EXCEPTION_BITMAP , eb); }
@@ -1960,6 +1968,12 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)case MSR_KERNEL_GS_BASE :vmx_write_guest_kernel_gs_base (vmx, data);break ; +case MSR_IA32_XFD : + ret =kvm_set_msr_common (vcpu, msr_info); +/* Update #NM interception according to guest xfd */ +if (!ret) +vmx_update_exception_bitmap (vcpu); +break ;#endif case MSR_IA32_SYSENTER_CS :if (is_guest_mode (vcpu))
@@ -4745,7 +4759,8 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu) vect_info = vmx->idt_vectoring_info; intr_info =vmx_get_intr_info (vcpu); -if (is_machine_check (intr_info) ||is_nmi (intr_info)) +if (is_machine_check (intr_info) ||is_nmi (intr_info) || +is_nm_fault (intr_info))return 1 ;/* handled by handle_exception_nmi_irqoff() */ if (is_invalid_opcode (intr_info))
@@ -6349,6 +6364,12 @@ static void handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu,kvm_after_interrupt (vcpu); } +static void handle_nm_fault_irqoff (struct kvm_vcpu *vcpu) +{ +rdmsrl (MSR_IA32_XFD_ERR , vcpu->arch.guest_fpu.xfd_err); +kvm_queue_exception (vcpu,NM_VECTOR ); +} +static void handle_exception_nmi_irqoff (struct vcpu_vmx *vmx) {const unsigned long nmi_entry = (unsigned long )asm_exc_nmi_noist;
@@ -6357,6 +6378,9 @@ static void handle_exception_nmi_irqoff(struct vcpu_vmx *vmx)/* if exit due to PF check for async PF */ if (is_page_fault (intr_info)) vmx->vcpu.arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags (); +/* if exit due to NM, handle before preemptions are enabled */ +else if (is_nm_fault (intr_info)) +handle_nm_fault_irqoff (&vmx->vcpu);/* Handle machine checks before interrupts are enabled */ else if (is_machine_check (intr_info))kvm_machine_check ();
arch/x86/kvm/x86.c +6 -0
@@ -9893,6 +9893,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (test_thread_flag (TIF_NEED_FPU_LOAD ))switch_fpu_return (); +if (vcpu->arch.guest_fpu.xfd_err) +wrmsrl (MSR_IA32_XFD_ERR , vcpu->arch.guest_fpu.xfd_err); +if (unlikely (vcpu->arch.switch_db_regs)) {set_debugreg (0 ,7 );set_debugreg (vcpu->arch.eff_db[0 ],0 );
@@ -9956,6 +9959,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)static_call (kvm_x86_handle_exit_irqoff)(vcpu); +if (vcpu->arch.guest_fpu.xfd_err) +wrmsrl (MSR_IA32_XFD_ERR ,0 ); +/* * Consume any pending interrupts, including the possible source of * VM-Exit on SVM and any ticks that occur between VM-Exit and now.
Sean Christopherson
maintainerRe: [PATCH v4 12/21] kvm: x86: Intercept #NM for saving IA32_XFD_ERR
arch/x86/kvm/vmx/vmx.c static void handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu, · line 6370
Quoted 4 lines · Yang Zhong
arch/x86/kvm/vmx/vmx.c +4
-0+static void handle_nm_fault_irqoff (struct kvm_vcpu *vcpu) +{ +rdmsrl (MSR_IA32_XFD_ERR , vcpu->arch.guest_fpu.xfd_err); +kvm_queue_exception (vcpu,NM_VECTOR );
This is still wrong, even though no additional supported is needed to support nested XFD. If L1 wants to intercept #NM, then KVM must not inject the #NM and must not read XFD_ERR.
That this was posted multiple times is disturbing, because kvm-unit-tests has a test for exactly this, and running it against this series on a host without XFD yields:
unchecked MSR access error: RDMSR from 0x1c5 at rIP: 0xffffffffa02478ee (vmx_handle_exit_irqoff+0x1de/0x240 [kvm_intel])
Call Trace: <TASK> kvm_arch_vcpu_ioctl_run+0x11a0/0x1fb0 [kvm] kvm_vcpu_ioctl+0x279/0x690 [kvm] __x64_sys_ioctl+0x83/0xb0 do_syscall_64+0x3b/0xc0 entry_SYSCALL_64_after_hwframe+0x44/0xae </TASK>
RE: [PATCH v4 12/21] kvm: x86: Intercept #NM for saving IA32_XFD_ERR
Quoted 4 lines
From: Sean Christopherson <[email protected]>
Sent: Wednesday, January 5, 2022 4:01 AMOn Wed, Dec 29, 2021, Yang Zhong wrote:
Quoted 9 lines
arch/x86/kvm/vmx/vmx.c static void handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu, · line 6370
Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +4
-0+static void handle_nm_fault_irqoff (struct kvm_vcpu *vcpu) +{ +rdmsrl (MSR_IA32_XFD_ERR , vcpu->arch.guest_fpu.xfd_err); +kvm_queue_exception (vcpu,NM_VECTOR );This is still wrong, even though no additional supported is needed to support nested XFD. If L1 wants to intercept #NM, then KVM must not inject the #NM and
must not readXFD_ERR.
Sure. This v4 was sent out before we settled down this open in v3 discussion. We did fix it later and posted the sample logic here:
Will include this fix in v5.
Thanks
Kevin
PATCH v4 13/21 kvm: x86: Emulate IA32_XFD_ERR for guest
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Emulate read/write to IA32_XFD_ERR MSR.
Only the saved value in the guest_fpu container is touched in the emulation handler. Actual MSR update is handled right before entering the guest (with preemption disabled)
--- arch/x86/kvm/x86.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +20 -1
@@ -1359,7 +1359,7 @@ static const u32 msrs_to_save_all[] = {MSR_F15H_PERF_CTL3 ,MSR_F15H_PERF_CTL4 ,MSR_F15H_PERF_CTL5 ,MSR_F15H_PERF_CTR0 ,MSR_F15H_PERF_CTR1 ,MSR_F15H_PERF_CTR2 ,MSR_F15H_PERF_CTR3 ,MSR_F15H_PERF_CTR4 ,MSR_F15H_PERF_CTR5 , - MSR_IA32_XFD, + MSR_IA32_XFD, MSR_IA32_XFD_ERR, };static u32 msrs_to_save[ARRAY_SIZE (msrs_to_save_all)];
@@ -3682,6 +3682,17 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)fpu_update_guest_xfd (&vcpu->arch.guest_fpu, data);break ; +case MSR_IA32_XFD_ERR : +if (!msr_info->host_initiated && + !guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ; + +if (data & ~(XFEATURE_MASK_USER_DYNAMIC & + vcpu->arch.guest_supported_xcr0)) +return 1 ; + + vcpu->arch.guest_fpu.xfd_err = data; +break ;#endif default :if (kvm_pmu_is_valid_msr (vcpu, msr))
@@ -4011,6 +4022,13 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) msr_info->data = vcpu->arch.guest_fpu.fpstate->xfd;break ; +case MSR_IA32_XFD_ERR : +if (!msr_info->host_initiated && + !guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ; + + msr_info->data = vcpu->arch.guest_fpu.xfd_err; +break ;#endif default :if (kvm_pmu_is_valid_msr (vcpu, msr_info->index))
@@ -6446,6 +6464,7 @@ static void kvm_init_msr_list(void)continue ;break ;case MSR_IA32_XFD : +case MSR_IA32_XFD_ERR :if (!kvm_cpu_cap_has (X86_FEATURE_XFD ))continue ;break ;
PATCH v4 14/21 kvm: x86: Disable RDMSR interception of IA32_XFD_ERR
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Disable read emulation of IA32_XFD_ERR MSR if guest cpuid includes XFD. This saves one unnecessary VM-exit in guest #NM handler, given that the MSR is already restored with the guest value before the guest is resumed.
--- arch/x86/kvm/vmx/vmx.c | 6 ++++++ arch/x86/kvm/vmx/vmx.h | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +6 -0
@@ -162,6 +162,7 @@ static u32 vmx_possible_passthrough_msrs[MAX_POSSIBLE_PASSTHROUGH_MSRS] = {MSR_FS_BASE ,MSR_GS_BASE ,MSR_KERNEL_GS_BASE , +MSR_IA32_XFD_ERR , #endifMSR_IA32_SYSENTER_CS , MSR_IA32_SYSENTER_ESP,
@@ -7228,6 +7229,11 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) } } +if (boot_cpu_has (X86_FEATURE_XFD )) +vmx_set_intercept_for_msr (vcpu,MSR_IA32_XFD_ERR ,MSR_TYPE_R , + !guest_cpuid_has (vcpu,X86_FEATURE_XFD )); + +set_cr4_guest_host_mask (vmx);vmx_write_encls_bitmap (vcpu,NULL );
arch/x86/kvm/vmx/vmx.h +1 -1
@@ -340,7 +340,7 @@ struct vcpu_vmx {struct lbr_desc lbr_desc;/* Save desired MSR intercept (read: pass-through) state */ -#define MAX_POSSIBLE_PASSTHROUGH_MSRS 13 +#define MAX_POSSIBLE_PASSTHROUGH_MSRS 14struct { DECLARE_BITMAP(read, MAX_POSSIBLE_PASSTHROUGH_MSRS); DECLARE_BITMAP(write, MAX_POSSIBLE_PASSTHROUGH_MSRS);
Sean Christopherson
maintainerRe: [PATCH v4 14/21] kvm: x86: Disable RDMSR interception of IA32_XFD_ERR
Quoted 27 lines · Yang Zhong
From: Jing Liu <[email protected]>
Disable read emulation of
IA32_XFD_ERRMSR if guest cpuid includes XFD. This saves one unnecessary VM-exit in guest #NM handler, given that the MSR is already restored with the guest value before the guest is resumed.Suggested-by Paolo Bonzini <[email protected]>Signed-off-by Jing Liu <[email protected]>Signed-off-by Yang Zhong <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 6 ++++++ arch/x86/kvm/vmx/vmx.h | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-)arch/x86/kvm/vmx/vmx.c +1
-0@@ -162,6 +162,7 @@ static u32 vmx_possible_passthrough_msrs[MAX_POSSIBLE_PASSTHROUGH_MSRS] = {MSR_FS_BASE ,MSR_GS_BASE ,MSR_KERNEL_GS_BASE , +MSR_IA32_XFD_ERR , #endifMSR_IA32_SYSENTER_CS , MSR_IA32_SYSENTER_ESP,
arch/x86/kvm/vmx/vmx.c static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 7232
Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +1
-0@@ -7228,6 +7229,11 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) } } +if (boot_cpu_has (X86_FEATURE_XFD ))
This should be kvm_cpu_cap_has(), not boot_cpu_has(). If 32-bit kernels don't suppress XFD in boot_cpu_data, then using boot_cpus_has() is wrong. And even if XFD is suppressed, using kvm_cpu_cap_has() is still preferable.
Quoted 20 lines
arch/x86/kvm/vmx/vmx.c +4
-0+vmx_set_intercept_for_msr (vcpu,MSR_IA32_XFD_ERR ,MSR_TYPE_R , + !guest_cpuid_has (vcpu,X86_FEATURE_XFD )); + +set_cr4_guest_host_mask (vmx);vmx_write_encls_bitmap (vcpu,NULL );arch/x86/kvm/vmx/vmx.h +1
-1@@ -340,7 +340,7 @@ struct vcpu_vmx {struct lbr_desc lbr_desc;/* Save desired MSR intercept (read: pass-through) state */ -#define MAX_POSSIBLE_PASSTHROUGH_MSRS 13 +#define MAX_POSSIBLE_PASSTHROUGH_MSRS 14struct { DECLARE_BITMAP(read, MAX_POSSIBLE_PASSTHROUGH_MSRS); DECLARE_BITMAP(write, MAX_POSSIBLE_PASSTHROUGH_MSRS);
RE: [PATCH v4 14/21] kvm: x86: Disable RDMSR interception of IA32_XFD_ERR
Quoted 36 lines
From: Sean Christopherson <[email protected]>
Sent: Wednesday, January 5, 2022 3:35 AMarch/x86/kvm/vmx/vmx.c static u32
Quoted 19 lines · Yang Zhong
From: Jing Liu <[email protected]>
Disable read emulation of
IA32_XFD_ERRMSR if guest cpuid includes XFD. This saves one unnecessary VM-exit in guest #NM handler, given that the MSR is already restored with the guest value before the guest is resumed.Suggested-by Paolo Bonzini <[email protected]>Signed-off-by Jing Liu <[email protected]>Signed-off-by Yang Zhong <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 6 ++++++ arch/x86/kvm/vmx/vmx.h | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-)arch/x86/kvm/vmx/vmx.c +0
-0@@ -162,6 +162,7 @@ static u32vmx_possible_passthrough_msrs[MAX_POSSIBLE_PASSTHROUGH_MSRS] = {
arch/x86/kvm/vmx/vmx.c static void vmx_vcpu_after_set_cpuid(struct
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +1
-0MSR_FS_BASE ,MSR_GS_BASE ,MSR_KERNEL_GS_BASE , +MSR_IA32_XFD_ERR , #endifMSR_IA32_SYSENTER_CS , MSR_IA32_SYSENTER_ESP,@@ -7228,6 +7229,11 @@ static void vmx_vcpu_after_set_cpuid(struct
kvm_vcpu*vcpu)Quoted 3 lines
arch/x86/kvm/vmx/vmx.c +0
-0} }
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 7232
Quoted 1 line
arch/x86/kvm/vmx/vmx.c +1
-0+if (boot_cpu_has (X86_FEATURE_XFD ))This should be
kvm_cpu_cap_has(), notboot_cpu_has(). If 32-bit kernels don't
suppress XFD inboot_cpu_data, then usingboot_cpus_has()is wrong. And even if
XFD is suppressed, usingkvm_cpu_cap_has()is still preferable.
Make sense.
PATCH v4 15/21 kvm: x86: Add XCR0 support for Intel AMX
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Two XCR0 bits are defined for AMX to support XSAVE mechanism. Bit 17 is for tilecfg and bit 18 is for tiledata.
The value of XCR0[17:18] is always either 00b or 11b. Also, SDM recommends that only 64-bit operating systems enable Intel AMX by setting XCR0[18:17]. 32-bit host kernel never sets the tile bits in vcpu->arch.guest_supported_xcr0.
--- arch/x86/kvm/x86.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +6 -1
@@ -210,7 +210,7 @@ static struct kvm_user_return_msrs __percpu *user_return_msrs;#define KVM_SUPPORTED_XCR0 (XFEATURE_MASK_FP | XFEATURE_MASK_SSE \ | XFEATURE_MASK_YMM | XFEATURE_MASK_BNDREGS \ | XFEATURE_MASK_BNDCSR | XFEATURE_MASK_AVX512 \ - | XFEATURE_MASK_PKRU) + | XFEATURE_MASK_PKRU | XFEATURE_MASK_XTILE)u64 __read_mostly host_efer;EXPORT_SYMBOL_GPL (host_efer);
@@ -990,6 +990,11 @@ static int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr)if ((xcr0 &XFEATURE_MASK_AVX512 ) !=XFEATURE_MASK_AVX512 )return 1 ; } + +if ((xcr0 &XFEATURE_MASK_XTILE ) && + ((xcr0 &XFEATURE_MASK_XTILE ) !=XFEATURE_MASK_XTILE )) +return 1 ; + vcpu->arch.xcr0 = xcr0;if ((xcr0 ^ old_xcr0) &XFEATURE_MASK_EXTEND )
PATCH v4 16/21 kvm: x86: Add CPUID support for Intel AMX
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Extend CPUID emulation to support XFD, AMX_TILE, AMX_INT8 and AMX_BF16. Adding those bits into kvm_cpu_caps finally activates all previous logics in this series.
--- arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/kvm/cpuid.c | 25 +++++++++++++++++++++++-- 2 files changed, 25 insertions(+), 2 deletions(-)
arch/x86/include/asm/cpufeatures.h +2 -0
@@ -299,7 +299,9 @@/* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4)/* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5)/* AVX512 BFLOAT16 instructions */ +#define X86_FEATURE_AMX_BF16 (18*32+22)/* AMX bf16 Support */ #define X86_FEATURE_AMX_TILE (18*32+24)/* AMX tile Support */ +#define X86_FEATURE_AMX_INT8 (18*32+25)/* AMX int8 Support */ /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */
arch/x86/kvm/cpuid.c +23 -2
@@ -525,7 +525,8 @@ void kvm_set_cpu_caps(void)F (AVX512_4VNNIW ) |F (AVX512_4FMAPS ) |F (SPEC_CTRL ) |F (SPEC_CTRL_SSBD ) |F (ARCH_CAPABILITIES ) |F (INTEL_STIBP ) |F (MD_CLEAR ) |F (AVX512_VP2INTERSECT ) |F (FSRM ) | -F (SERIALIZE ) |F (TSXLDTRK ) |F (AVX512_FP16 ) +F (SERIALIZE ) |F (TSXLDTRK ) |F (AVX512_FP16 ) | +F (AMX_TILE ) |F (AMX_INT8 ) |F (AMX_BF16 ) );/* TSC_ADJUST and ARCH_CAPABILITIES are emulated in software. */
@@ -544,7 +545,7 @@ void kvm_set_cpu_caps(void) );kvm_cpu_cap_mask (CPUID_D_1_EAX , -F (XSAVEOPT ) |F (XSAVEC ) |F (XGETBV1 ) |F (XSAVES ) +F (XSAVEOPT ) |F (XSAVEC ) |F (XGETBV1 ) |F (XSAVES ) |F (XFD ) ); kvm_cpu_cap_init_scattered(CPUID_12_EAX,
@@ -670,6 +671,8 @@ static struct kvm_cpuid_entry2 *do_host_cpuid(struct kvm_cpuid_array *array,case 0x14 :case 0x17 :case 0x18 : +case 0x1d : +case 0x1e :case 0x1f :case 0x8000001d : entry->flags |=KVM_CPUID_FLAG_SIGNIFCANT_INDEX ;
@@ -942,6 +945,24 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) goto out; }break ; +/* Intel AMX TILE */ +case 0x1d : +if (!kvm_cpu_cap_has (X86_FEATURE_AMX_TILE )) { + entry->eax = entry->ebx = entry->ecx = entry->edx =0 ; +break ; + } + +for (i =1 , max_idx = entry->eax; i <= max_idx; ++i) { +if (!do_host_cpuid (array, function, i)) + goto out; + } +break ; +case 0x1e :/* TMUL information */ +if (!kvm_cpu_cap_has (X86_FEATURE_AMX_TILE )) { + entry->eax = entry->ebx = entry->ecx = entry->edx =0 ; +break ; + } +break ;case KVM_CPUID_SIGNATURE : {const u32 *sigptr = (const u32 *)KVM_SIGNATURE ; entry->eax =KVM_CPUID_FEATURES ;
PATCH v4 17/21 x86/fpu: Add uabi_size to guest_fpu
In reply to Yang ZhongFrom: Thomas Gleixner <[email protected]>
Userspace needs to inquire KVM about the buffer size to work with the new KVM_SET_XSAVE and KVM_GET_XSAVE2. Add the size info to guest_fpu for KVM to access.
--- arch/x86/include/asm/fpu/types.h | 5 +++++ arch/x86/kernel/fpu/core.c | 1 + arch/x86/kernel/fpu/xstate.c | 1 + 3 files changed, 7 insertions(+)
arch/x86/include/asm/fpu/types.h +5 -0
@@ -522,6 +522,11 @@ struct fpu_guest {*/ u64 xfd_err; +/* +* @uabi_size: Size required for save/restore +*/ +unsigned int uabi_size; +/* * @fpstate: Pointer to the allocated guest fpstate */
arch/x86/kernel/fpu/core.c +1 -0
@@ -240,6 +240,7 @@ bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) gfpu->fpstate = fpstate; gfpu->xfeatures = fpu_user_cfg.default_features; gfpu->perm = fpu_user_cfg.default_features; + gfpu->uabi_size = fpu_user_cfg.default_size;fpu_init_guest_permissions (gfpu);return true;
arch/x86/kernel/fpu/xstate.c +1 -0
@@ -1545,6 +1545,7 @@ static int fpstate_realloc(u64 xfeatures, unsigned int ksize, newfps->is_confidential = curfps->is_confidential; newfps->in_use = curfps->in_use; guest_fpu->xfeatures |= xfeatures; + guest_fpu->uabi_size = usize; }fpregs_lock ();
PATCH v4 18/21 kvm: x86: Add support for getting/setting expanded xstate buffer
In reply to Yang ZhongFrom: Guang Zeng <[email protected]>
With KVM_CAP_XSAVE, userspace uses a hardcoded 4KB buffer to get/set xstate data from/to KVM. This doesn't work when dynamic xfeatures (e.g. AMX) are exposed to the guest as they require a larger buffer size.
Introduce a new capability (KVM_CAP_XSAVE2). Userspace VMM gets the required xstate buffer size via KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2). KVM_SET_XSAVE is extended to work with both legacy and new capabilities by doing properly-sized memdup_user() based on the guest fpu container. KVM_GET_XSAVE is kept for backward-compatible reason. Instead, KVM_GET_XSAVE2 is introduced under KVM_CAP_XSAVE2 as the preferred interface for getting xstate buffer (4KB or larger size) from KVM (Link: https://lkml.org/lkml/2021/12/15/510)
Also, update the api doc with the new KVM_GET_XSAVE2 ioctl.
--- Documentation/virt/kvm/api.rst | 42 ++++++++++++++++++++++++++++++-- arch/x86/include/uapi/asm/kvm.h | 16 +++++++++++- arch/x86/kvm/x86.c | 43 ++++++++++++++++++++++++++++++++- include/uapi/linux/kvm.h | 4 +++ 4 files changed, 101 insertions(+), 4 deletions(-)
Documentation/virt/kvm/api.rst +40 -2
@@ -1566,6 +1566,7 @@ otherwise it will return EBUSY error. struct kvm_xsave { __u32 region[1024]; + __u32 extra[0]; }; This ioctl would copy current vcpu's xsave struct to the userspace.
@@ -1574,7 +1575,7 @@ This ioctl would copy current vcpu's xsave struct to the userspace. 4.43 KVM_SET_XSAVE ------------------ -:Capability: KVM_CAP_XSAVE +:Capability: KVM_CAP_XSAVE and KVM_CAP_XSAVE2 :Architectures: x86 :Type: vcpu ioctl :Parameters: struct kvm_xsave (in)
@@ -1585,9 +1586,18 @@ This ioctl would copy current vcpu's xsave struct to the userspace. struct kvm_xsave { __u32 region[1024]; + __u32 extra[0]; }; -This ioctl would copy userspace's xsave struct to the kernel. +This ioctl would copy userspace's xsave struct to the kernel. It copies +as many bytes as are returned by KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2), +when invoked on the vm file descriptor. The size value returned by +KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2) will always be at least 4096. +Currently, it is only greater than 4096 if a dynamic feature has been +enabled with ``arch_prctl()``, but this may change in the future. + +The offsets of the state save areas in struct kvm_xsave follow the +contents of CPUID leaf 0xD on the host. 4.44 KVM_GET_XCRS
@@ -5507,6 +5517,34 @@ the trailing ``'\0'``, is indicated by ``name_size`` in the header. The Stats Data block contains an array of 64-bit values in the same order as the descriptors in Descriptors block. +4.42 KVM_GET_XSAVE2 +------------------ + +:Capability: KVM_CAP_XSAVE2 +:Architectures: x86 +:Type: vcpu ioctl +:Parameters: struct kvm_xsave (out) +:Returns: 0 on success, -1 on error + + +:: + + struct kvm_xsave { + __u32 region[1024]; + __u32 extra[0]; + }; + +This ioctl would copy current vcpu's xsave struct to the userspace. It +copies as many bytes as are returned by KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2) +when invoked on the vm file descriptor. The size value returned by +KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2) will always be at least 4096. +Currently, it is only greater than 4096 if a dynamic feature has been +enabled with ``arch_prctl()``, but this may change in the future. + +The offsets of the state save areas in struct kvm_xsave follow the contents +of CPUID leaf 0xD on the host. + + 5. The kvm_run structure ========================
arch/x86/include/uapi/asm/kvm.h +15 -1
@@ -373,9 +373,23 @@ struct kvm_debugregs {__u64 reserved[9 ]; }; -/* for KVM_CAP_XSAVE */ +/* for KVM_CAP_XSAVE and KVM_CAP_XSAVE2 */ struct kvm_xsave { +/* +* KVM_GET_XSAVE2 and KVM_SET_XSAVE write and read as many bytes +* as are returned by KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2) +* respectively, when invoked on the vm file descriptor. +* +* The size value returned by KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2) +* will always be at least 4096. Currently, it is only greater +* than 4096 if a dynamic feature has been enabled with +* ``arch_prctl()``, but this may change in the future. +* +* The offsets of the state save areas in struct kvm_xsave follow +* the contents of CPUID leaf 0xD on the host. +*/ __u32 region[1024 ]; +__u32 extra[0 ]; };#define KVM_MAX_XCRS 16
arch/x86/kvm/x86.c +42 -1
73 lines not on this page · raw message
include/uapi/linux/kvm.h +4 -0
@@ -1131,6 +1131,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204#define KVM_CAP_ARM_MTE 205#define KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM 206 +#define KVM_CAP_XSAVE2 207#ifdef KVM_CAP_IRQ_ROUTING
@@ -1610,6 +1611,9 @@ struct kvm_enc_region {#define KVM_S390_NORMAL_RESET _IO(KVMIO, 0xc3)#define KVM_S390_CLEAR_RESET _IO(KVMIO, 0xc4) +/* Available with KVM_CAP_XSAVE2 */ +#define KVM_GET_XSAVE2 _IOR(KVMIO, 0xcf, struct kvm_xsave) +struct kvm_s390_pv_sec_parm {__u64 origin;__u64 length;
Sean Christopherson
maintainerRe: [PATCH v4 18/21] kvm: x86: Add support for getting/setting expanded xstate buffer
arch/x86/kvm/x86.c int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · line 4300
Quoted 10 lines · Yang Zhong
arch/x86/kvm/x86.c +2
-0@@ -4296,6 +4296,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)else r =0 ;break ; +case KVM_CAP_XSAVE2 : + r = kvm->vcpus[0 ]->arch.guest_fpu.uabi_size;
a) This does not compile against kvm/queue.
arch/x86/kvm/x86.c: In function ‘kvm_vm_ioctl_check_extension’:
arch/x86/kvm/x86.c:4317:24: error: ‘struct kvm’ has no member named ‘vcpus’
4317 | r = kvm->vcpus[0]->arch.guest_fpu.uabi_size;
b) vcpu0 is not guaranteed to be non-NULL at this point.
Quoted 6 lines
arch/x86/kvm/x86.c +3
-0+if (r <sizeof (struct kvm_xsave )) + r =sizeof (struct kvm_xsave ); +break ;default :break ; }
Paolo Bonzini
maintainerRe: [PATCH v4 18/21] kvm: x86: Add support for getting/setting expanded xstate buffer
Quoted 14 lines · Sean Christopherson
arch/x86/kvm/x86.c int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · line 4300
Quoted 10 lines · Yang Zhong
arch/x86/kvm/x86.c +2
-0@@ -4296,6 +4296,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)else r =0 ;break ; +case KVM_CAP_XSAVE2 : + r = kvm->vcpus[0 ]->arch.guest_fpu.uabi_size;a) This does not compile against kvm/queue.
Quoted 5 lines
arch/x86/kvm/x86.c: In function ‘kvm_vm_ioctl_check_extension’:
arch/x86/kvm/x86.c:4317:24: error: ‘struct kvm’ has no member named ‘vcpus’
4317 | r = kvm->vcpus[0]->arch.guest_fpu.uabi_size;b) vcpu0 is not guaranteed to be non-NULL at this point.
Yang, you can post an incremental patch for this. You can use the highest bit of the guest-permitted xcr0 (i.e. the OR of KVM's supported XCR0 an the guest-permitted dynamic features) and pass it to cpuid(0xD).
Paolo
RE: [PATCH v4 18/21] kvm: x86: Add support for getting/setting expanded xstate buffer
Quoted 21 lines
From: Paolo Bonzini <[email protected]>
Sent: Wednesday, January 5, 2022 4:46 AMQuoted 6 lines · Sean Christopherson
arch/x86/kvm/x86.c int kvm_vm_ioctl_check_extension(struct kvm
Quoted 5 lines · Yang Zhong
arch/x86/kvm/x86.c +0
-0@@ -4296,6 +4296,11 @@ int kvm_vm_ioctl_check_extension(struct kvm*kvm, long ext)
Quoted 10 lines
arch/x86/kvm/x86.c int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · line 4300
Quoted 5 lines
arch/x86/kvm/x86.c +2
-0else r =0 ;break ; +case KVM_CAP_XSAVE2 : + r = kvm->vcpus[0 ]->arch.guest_fpu.uabi_size;a) This does not compile against kvm/queue.
arch/x86/kvm/x86.c: In function ‘kvm_vm_ioctl_check_extension’:
arch/x86/kvm/x86.c:4317:24: error: ‘struct kvm’ has no member named
Quoted 9 lines
‘vcpus’
Quoted 3 lines
4317 | r = kvm->vcpus[0]->arch.guest_fpu.uabi_size;b) vcpu0 is not guaranteed to be non-NULL at this point.
Yang, you can post an incremental patch for this. You can use the highest bit of the guest-permitted xcr0 (i.e. the OR of KVM's supported XCR0 an the guest-permitted dynamic features) and pass it to cpuid(0xD).
Will do, except one correction that 'OR' should be 'AND' here. 😊
Thanks
Kevin
PATCH v4 19/21 kvm: selftests: Add support for KVM_CAP_XSAVE2
In reply to Yang ZhongFrom: Wei Wang <[email protected]>
When KVM_CAP_XSAVE2 is supported, userspace is expected to allocate buffer for KVM_GET_XSAVE2 and KVM_SET_XSAVE using the size returned by KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2).
--- tools/arch/x86/include/uapi/asm/kvm.h | 16 ++++- tools/include/uapi/linux/kvm.h | 3 + .../testing/selftests/kvm/include/kvm_util.h | 2 + .../selftests/kvm/include/x86_64/processor.h | 10 +++ tools/testing/selftests/kvm/lib/kvm_util.c | 32 +++++++++ .../selftests/kvm/lib/x86_64/processor.c | 67 ++++++++++++++++++- .../testing/selftests/kvm/x86_64/evmcs_test.c | 2 +- tools/testing/selftests/kvm/x86_64/smm_test.c | 2 +- .../testing/selftests/kvm/x86_64/state_test.c | 2 +- .../kvm/x86_64/vmx_preemption_timer_test.c | 2 +- 10 files changed, 130 insertions(+), 8 deletions(-)
tools/arch/x86/include/uapi/asm/kvm.h +15 -1
@@ -373,9 +373,23 @@ struct kvm_debugregs {__u64 reserved[9 ]; }; -/* for KVM_CAP_XSAVE */ +/* for KVM_CAP_XSAVE and KVM_CAP_XSAVE2 */ struct kvm_xsave { +/* +* KVM_GET_XSAVE2 and KVM_SET_XSAVE write and read as many bytes +* as are returned by KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2) +* respectively, when invoked on the vm file descriptor. +* +* The size value returned by KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2) +* will always be at least 4096. Currently, it is only greater +* than 4096 if a dynamic feature has been enabled with +* ``arch_prctl()``, but this may change in the future. +* +* The offsets of the state save areas in struct kvm_xsave follow +* the contents of CPUID leaf 0xD on the host. +*/ __u32 region[1024 ]; +__u32 extra[0 ]; };#define KVM_MAX_XCRS 16
tools/include/uapi/linux/kvm.h +3 -0
@@ -1131,6 +1131,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204#define KVM_CAP_ARM_MTE 205#define KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM 206 +#define KVM_CAP_XSAVE2 207#ifdef KVM_CAP_IRQ_ROUTING
@@ -1551,6 +1552,8 @@ struct kvm_s390_ucas_mapping {/* Available with KVM_CAP_XSAVE */ #define KVM_GET_XSAVE _IOR(KVMIO, 0xa4, struct kvm_xsave)#define KVM_SET_XSAVE _IOW(KVMIO, 0xa5, struct kvm_xsave) +/* Available with KVM_CAP_XSAVE2 */ +#define KVM_GET_XSAVE2 _IOR(KVMIO, 0xcf, struct kvm_xsave)/* Available with KVM_CAP_XCRS */ #define KVM_GET_XCRS _IOR(KVMIO, 0xa6, struct kvm_xcrs)#define KVM_SET_XCRS _IOW(KVMIO, 0xa7, struct kvm_xcrs)
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
@@ -85,6 +85,7 @@ extern const struct vm_guest_mode_params vm_guest_mode_params[];int open_path_or_exit (const char *path,int flags);int open_kvm_dev_path_or_exit (void );int kvm_check_cap (long cap); +int vm_check_cap (struct kvm_vm *vm,long cap);int vm_enable_cap (struct kvm_vm *vm,struct kvm_enable_cap *cap);int vcpu_enable_cap (struct kvm_vm *vm,uint32_t vcpu_id,struct kvm_enable_cap *cap);
@@ -316,6 +317,7 @@ struct kvm_vm *vm_create_with_vcpus(enum vm_guest_mode mode, uint32_t nr_vcpus,* guest_code - The vCPU's entry point */ void vm_vcpu_add_default (struct kvm_vm *vm,uint32_t vcpuid,void *guest_code); +void vm_xsave_req_perm (void );bool vm_is_unrestricted_guest (struct kvm_vm *vm);
tools/testing/selftests/kvm/include/x86_64/processor.h +10 -0
@@ -10,8 +10,10 @@#include <assert.h> #include <stdint.h> +#include <syscall.h> #include <asm/msr-index.h> +#include <asm/prctl.h> #include "../kvm_util.h"
@@ -352,6 +354,7 @@ struct kvm_x86_state;struct kvm_x86_state *vcpu_save_state (struct kvm_vm *vm,uint32_t vcpuid);void vcpu_load_state (struct kvm_vm *vm,uint32_t vcpuid,struct kvm_x86_state *state); +void kvm_x86_state_cleanup (struct kvm_x86_state *state);struct kvm_msr_list *kvm_get_msr_index_list (void );uint64_t kvm_get_feature_msr (uint64_t msr_index);
@@ -443,4 +446,11 @@ void __virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,/* VMX_EPT_VPID_CAP bits */ #define VMX_EPT_VPID_CAP_AD_BITS (1ULL << 21) +#define XSTATE_XTILE_CFG_BIT 17 +#define XSTATE_XTILE_DATA_BIT 18 + +#define XSTATE_XTILE_CFG_MASK (1ULL << XSTATE_XTILE_CFG_BIT) +#define XSTATE_XTILE_DATA_MASK (1ULL << XSTATE_XTILE_DATA_BIT) +#define XFEATURE_XTILE_MASK (XSTATE_XTILE_CFG_MASK | \ + XSTATE_XTILE_DATA_MASK)#endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/kvm_util.c +32 -0
@@ -85,6 +85,33 @@ int kvm_check_cap(long cap)return ret; } +/* VM Check Capability +* +* Input Args: +* vm - Virtual Machine +* cap - Capability +* +* Output Args: None +* +* Return: +* On success, the Value corresponding to the capability (KVM_CAP_*) +* specified by the value of cap. On failure a TEST_ASSERT failure +* is produced. +* +* Looks up and returns the value corresponding to the capability +* (KVM_CAP_*) given by cap. +*/ +int vm_check_cap (struct kvm_vm *vm,long cap) +{ +int ret; + + ret =ioctl (vm->fd,KVM_CHECK_EXTENSION , cap); +TEST_ASSERT (ret >=0 ,"KVM_CHECK_EXTENSION VM IOCTL failed,\n" +" rc: %i errno: %i" , ret, errno); + +return ret; +} +/* VM Enable Capability * * Input Args:
@@ -344,6 +371,11 @@ struct kvm_vm *vm_create_with_vcpus(enum vm_guest_mode mode, uint32_t nr_vcpus,struct kvm_vm *vm;int i; +/* +* Permission needs to be requested before KVM_SET_CPUID2. +*/ +vm_xsave_req_perm (); +/* Force slot0 memory size not small than DEFAULT_GUEST_PHY_PAGES */ if (slot0_mem_pages <DEFAULT_GUEST_PHY_PAGES ) slot0_mem_pages =DEFAULT_GUEST_PHY_PAGES ;
tools/testing/selftests/kvm/lib/x86_64/processor.c +64 -3
106 lines not on this page · raw message
tools/testing/selftests/kvm/x86_64/evmcs_test.c +1 -1
@@ -129,7 +129,7 @@ static void save_restore_vm(struct kvm_vm *vm)vcpu_set_hv_cpuid (vm,VCPU_ID );vcpu_enable_evmcs (vm,VCPU_ID );vcpu_load_state (vm,VCPU_ID , state); -free (state); +kvm_x86_state_cleanup (state);memset (®s2,0 ,sizeof (regs2));vcpu_regs_get (vm,VCPU_ID , ®s2);
tools/testing/selftests/kvm/x86_64/smm_test.c +1 -1
@@ -212,7 +212,7 @@ int main(int argc, char *argv[])vcpu_set_cpuid (vm,VCPU_ID ,kvm_get_supported_cpuid ());vcpu_load_state (vm,VCPU_ID , state); run =vcpu_state (vm,VCPU_ID ); -free (state); +kvm_x86_state_cleanup (state); } done:
tools/testing/selftests/kvm/x86_64/state_test.c +1 -1
@@ -218,7 +218,7 @@ int main(int argc, char *argv[])vcpu_set_cpuid (vm,VCPU_ID ,kvm_get_supported_cpuid ());vcpu_load_state (vm,VCPU_ID , state); run =vcpu_state (vm,VCPU_ID ); -free (state); +kvm_x86_state_cleanup (state);memset (®s2,0 ,sizeof (regs2));vcpu_regs_get (vm,VCPU_ID , ®s2);
tools/testing/selftests/kvm/x86_64/vmx_preemption_timer_test.c +1 -1
@@ -244,7 +244,7 @@ int main(int argc, char *argv[])vcpu_set_cpuid (vm,VCPU_ID ,kvm_get_supported_cpuid ());vcpu_load_state (vm,VCPU_ID , state); run =vcpu_state (vm,VCPU_ID ); -free (state); +kvm_x86_state_cleanup (state);memset (®s2,0 ,sizeof (regs2));vcpu_regs_get (vm,VCPU_ID , ®s2);
PATCH v4 20/21 x86/fpu: Provide fpu_sync_guest_vmexit_xfd_state()
In reply to Yang ZhongFrom: Thomas Gleixner <[email protected]>
KVM can disable the write emulation for the XFD MSR when the vCPU's fpstate is already correctly sized to reduce the overhead.
When write emulation is disabled the XFD MSR state after a VMEXIT is unknown and therefore not in sync with the software states in fpstate and the per CPU XFD cache.
Provide fpu_sync_guest_vmexit_xfd_state() which has to be invoked after a VMEXIT before enabling interrupts when write emulation is disabled for the XFD MSR.
It could be invoked unconditionally even when write emulation is enabled for the price of a pointless MSR read.
--- arch/x86/include/asm/fpu/api.h | 2 ++ arch/x86/kernel/fpu/core.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+)
arch/x86/include/asm/fpu/api.h +2 -0
@@ -142,8 +142,10 @@ extern int fpu_update_guest_perm_features(struct fpu_guest *guest_fpu);#ifdef CONFIG_X86_64 extern void fpu_update_guest_xfd (struct fpu_guest *guest_fpu,u64 xfd); +extern void fpu_sync_guest_vmexit_xfd_state (void );#else static inline void fpu_update_guest_xfd (struct fpu_guest *guest_fpu,u64 xfd) { } +static inline void fpu_sync_guest_vmexit_xfd_state (void ) { }#endif extern void fpu_copy_guest_fpstate_to_uabi (struct fpu_guest *gfpu,void *buf,unsigned int size,u32 pkru);
arch/x86/kernel/fpu/core.c +24 -0
@@ -299,6 +299,30 @@ void fpu_update_guest_xfd(struct fpu_guest *guest_fpu, u64 xfd) fpregs_unlock(); }EXPORT_SYMBOL_GPL (fpu_update_guest_xfd); + +/** +* fpu_sync_guest_vmexit_xfd_state - Synchronize XFD MSR and software state +* +* Must be invoked from KVM after a VMEXIT before enabling interrupts when +* XFD write emulation is disabled. This is required because the guest can +* freely modify XFD and the state at VMEXIT is not guaranteed to be the +* same as the state on VMENTER. So software state has to be udpated before +* any operation which depends on it can take place. +* +* Note: It can be invoked unconditionally even when write emulation is +* enabled for the price of a then pointless MSR read. +*/ +void fpu_sync_guest_vmexit_xfd_state (void ) +{ +struct fpstate *fps = current->thread.fpu.fpstate; + +lockdep_assert_irqs_disabled (); +if (fpu_state_size_dynamic ()) { +rdmsrl (MSR_IA32_XFD , fps->xfd); +__this_cpu_write (xfd_state, fps->xfd); + } +} +EXPORT_SYMBOL_GPL (fpu_sync_guest_vmexit_xfd_state);#endif /* CONFIG_X86_64 */ int fpu_swap_kvm_fpstate (struct fpu_guest *guest_fpu,bool enter_guest)
PATCH v4 21/21 kvm: x86: Disable interception for IA32_XFD on demand
In reply to Yang ZhongFrom: Kevin Tian <[email protected]>
Always intercepting IA32_XFD causes non-negligible overhead when this register is updated frequently in the guest.
Disable r/w emulation after intercepting the first WRMSR(IA32_XFD) with a non-zero value.
Disable WRMSR emulation implies that IA32_XFD becomes out-of-sync with the software states in fpstate and the per-cpu xfd cache. Call fpu_sync_guest_vmexit_xfd_state() to bring them back in-sync, before preemption is enabled.
Always trap #NM once write interception is disabled for IA32_XFD. The #NM exception is rare if the guest doesn't use dynamic features. Otherwise, there is at most one exception per guest task given a dynamic feature.
p.s. We have confirmed that SDM is being revised to say that when setting IA32_XFD[18] the AMX register state is not guaranteed to be preserved. This clarification avoids adding mess for a creative guest which sets IA32_XFD[18]=1 before saving active AMX state to its own storage.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/vmx.c | 23 ++++++++++++++++++----- arch/x86/kvm/vmx/vmx.h | 2 +- arch/x86/kvm/x86.c | 3 +++ 4 files changed, 23 insertions(+), 6 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -640,6 +640,7 @@ struct kvm_vcpu_arch {u64 smi_count;bool tpr_access_reporting;bool xsaves_enabled; +bool xfd_no_write_intercept;u64 ia32_xss;u64 microcode_version;u64 arch_capabilities;
arch/x86/kvm/vmx/vmx.c +18 -5
@@ -162,6 +162,7 @@ static u32 vmx_possible_passthrough_msrs[MAX_POSSIBLE_PASSTHROUGH_MSRS] = {MSR_FS_BASE ,MSR_GS_BASE ,MSR_KERNEL_GS_BASE , +MSR_IA32_XFD ,MSR_IA32_XFD_ERR , #endif MSR_IA32_SYSENTER_CS,
@@ -766,10 +767,11 @@ void vmx_update_exception_bitmap(struct kvm_vcpu *vcpu) }/* -* Trap #NM if guest xfd contains a non-zero value so guest XFD_ERR -* can be saved timely. +* Disabling xfd interception indicates that dynamically-enabled +* features might be used in the guest. Always trap #NM in this case +* for proper virtualization of guest xfd_err. */ -if (vcpu->arch.guest_fpu.fpstate->xfd) +if (vcpu->arch.xfd_no_write_intercept) eb |= (1u <<NM_VECTOR );vmcs_write32 (EXCEPTION_BITMAP , eb);
@@ -1971,9 +1973,20 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)break ;case MSR_IA32_XFD : ret =kvm_set_msr_common (vcpu, msr_info); -/* Update #NM interception according to guest xfd */ -if (!ret) +/* +* Always intercepting WRMSR could incur non-negligible +* overhead given xfd might be touched in guest context +* switch. Disable write interception upon the first write +* with a non-zero value (indicate potential guest usage +* on dynamic xfeatures). Also update exception bitmap +* to trap #NM for proper virtualization of guest xfd_err. +*/ +if (!ret && data) { +vmx_disable_intercept_for_msr (vcpu,MSR_IA32_XFD , +MSR_TYPE_RW ); + vcpu->arch.xfd_no_write_intercept = true;vmx_update_exception_bitmap (vcpu); + }break ;#endif case MSR_IA32_SYSENTER_CS :
arch/x86/kvm/vmx/vmx.h +1 -1
@@ -340,7 +340,7 @@ struct vcpu_vmx {struct lbr_desc lbr_desc;/* Save desired MSR intercept (read: pass-through) state */ -#define MAX_POSSIBLE_PASSTHROUGH_MSRS 14 +#define MAX_POSSIBLE_PASSTHROUGH_MSRS 15struct { DECLARE_BITMAP(read, MAX_POSSIBLE_PASSTHROUGH_MSRS); DECLARE_BITMAP(write, MAX_POSSIBLE_PASSTHROUGH_MSRS);
arch/x86/kvm/x86.c +3 -0
@@ -10027,6 +10027,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (vcpu->arch.guest_fpu.xfd_err)wrmsrl (MSR_IA32_XFD_ERR ,0 ); +if (vcpu->arch.xfd_no_write_intercept) +fpu_sync_guest_vmexit_xfd_state (); +/* * Consume any pending interrupts, including the possible source of * VM-Exit on SVM and any ticks that occur between VM-Exit and now.
Quoted 13 lines
From: Zhong, Yang <[email protected]>
Sent: Wednesday, December 29, 2021 9:13 PMHighly appreciate for your review. This version mostly addressed the comments
from Sean. Most comments are adopted except three which are not closed and
need more discussions:- Move the entire xfd write emulation code to
x86.c. Doing so requires introducing a newkvm_x86_opscallback to disable msr write bitmap. According to Paolo's earlier comment he prefers to handle it invmx.c.
Quoted 9 lines
- Directly check
msr_bitmapinupdate_exception_bitmap()(for trapping #NM) andvcpu_enter_guest()(for syncing guest xfd after vm-exit) instead of introducing an extra flag in the last patch. However, doing so requires another newkvm_x86_opscallback for checkingmsr_bitmapsincevcpu_enter_guest()is x86 common code. Having an extra flag sounds simpler here (at least for the initial AMX support). It does penalize nested guest with one xfd sync per exit, but it's not worse than a normal guest which initializes xfd but doesn't run AMX applications at all. Those could be improved afterwards.
Another option is to move xfd sync into vmx_vcpu_run(), given that disabling xfd interception is vmx specific code thus it makes some sense to also handle related side-effect in vmx.c. If this can be agreed then yes there is no need of an extra flag and just checking msr_bitmap is sufficient (and more accurate).
Paolo, how is your opinion?
Quoted 9 lines
- Disable #NM trap for nested guest. This version still chooses to always trap #NM (regardless in L1 or L2) as long as xfd write interception is
disabled.
In reality #NM is rare if nested guest doesn't intend to run AMX
applications
and always-trap is safer than dynamic trap for the basic support in case of any oversight here.
Sean just pointed out some potential issues in current logic, about handling #NM raised in L2 guest (could happen just due to L1 interception).
It's being discussed here:
https://lore.kernel.org/all/[email protected]/
Thanks
Kevin
Paolo Bonzini
maintainerQuoted 7 lines · Yang Zhong
Highly appreciate for your review. This version mostly addressed the comments from Sean. Most comments are adopted except three which are not closed and need more discussions:
- Move the entire xfd write emulation code to
x86.c. Doing so requires introducing a newkvm_x86_opscallback to disable msr write bitmap. According to Paolo's earlier comment he prefers to handle it invmx.c.
Yes, I do.
Quoted 9 lines
- Directly check
msr_bitmapinupdate_exception_bitmap()(for trapping #NM) andvcpu_enter_guest()(for syncing guest xfd after vm-exit) instead of introducing an extra flag in the last patch. However, doing so requires another newkvm_x86_opscallback for checkingmsr_bitmapsincevcpu_enter_guest()is x86 common code. Having an extra flag sounds simpler here (at least for the initial AMX support). It does penalize nested guest with one xfd sync per exit, but it's not worse than a normal guest which initializes xfd but doesn't run AMX applications at all. Those could be improved afterwards.
The thing to do here would be to move MAX_POSSIBLE_PASSTHROUGH_MSRS/MAX_DIRECT_ACCESS_MSRS from VMX/SVM to core code. For now we can keep the flag.
Quoted 5 lines
- Disable #NM trap for nested guest. This version still chooses to always trap #NM (regardless in L1 or L2) as long as xfd write interception is disabled. In reality #NM is rare if nested guest doesn't intend to run AMX applications and always-trap is safer than dynamic trap for the basic support in case of any oversight here.
Sean was justifying this with lack of support for nested AMX, but I'm not sure actually what is missing at all. That is, an L1 hypervisor could expose AMX to L2, and then an L2->L0->L2 exit/reentry would have to trap #NM. Otherwise it would miss an XFD_ERR update.
So the patches look good now.
Paolo
Quoted 221 lines
(Jing is temporarily leave for family reason, Yang helped work out this version)
----
v3->v4:
- Verify kvm selftest for AMX (Paolo)
- Move fpstate buffer expansion fromkvm_vcpu_after_set_cpuid() tokvm_check_cpuid()and improve patch description (Sean)
- Drop 'preemption' word in #NM interception patch (Sean)
- Remove 'trap_nm' flag. Replace it by: (Sean)
* Trapping #NM according toguest_fpu::xfdwhen write to xfd is intercepted.
* Always trapping #NM when xfd write interception is disabled
- Use better name for #NM related functions (Sean)
- Drop '#ifdefCONFIG_X86_64' in__kvm_set_xcr(Sean)
- Update description forKVM_CAP_XSAVE2and prevent the guest from using the wrong ioctl (Sean)
- Replace 'xfd_out_of_sync' with a better name (Sean)
v2->v3:
- Trap #NM until writeIA32_XFDwith a non-zero value (Thomas)
- Revise return value in__xstate_request_perm()(Thomas)
- Revise doc forKVM_GET_SUPPORTED_CPUID(Paolo)
- Add Thomas's reviewed-by on one patch
- Reorder disabling read interception ofXFD_ERRpatch (Paolo)
- Move disabling r/w interception of XFD fromx86.ctovmx.c(Paolo)
- Provide the API doc together with the newKVM_GET_XSAVE2ioctl (Paolo)
- MakeKVM_CHECK_EXTENSION(KVM_CAP_XSAVE2)return minimum size of structkvm_xsave (4K) (Paolo)- Request permission at the start of
vm_create_with_vcpus()in selftest
- Request permission conditionally when XFD is supported (Paolo)
v1->v2:
- Live migration supported and verified with a selftest
- Rebase to Thomas's new series for guest fpstate reallocation [1]
- Expand fpstate atKVM_SET_CPUID2instead of when emulating XCR0 andIA32_XFD(Thomas/Paolo)
- Accordingly remove all exit-to-userspace stuff
- Intercept #NM to save guestXFD_ERRand restore host/guest value at preemption on/off boundary (Thomas)
- Accordingly remove allxfd_errlogic in preemption callback andfpu_swap_kvm_fpstate()
- ReuseKVM_SET_XSAVEto handle both legacy and expanded buffer (Paolo)
- Don't return dynamic bits w/oprctl()inKVM_GET_SUPPORTED_CPUID(Paolo)
- Check guest permissions for dynamic features in CPUID[0xD] instead of only for AMX atKVM_SET_CPUID(Paolo)
- Remove dynamic bit check for 32-bit guest in__kvm_set_xcr()(Paolo)
- Fix CPUID emulation for0x1dand0x1e(Paolo)
- Move "disable interception" to the end of the series (Paolo)This series brings AMX (Advanced Matrix eXtensions) virtualization support to KVM. The preparatory series from Thomas [1] is also included.
A large portion of the changes in this series is to deal with eXtended Feature Disable (XFD) which allows resizing of the fpstate buffer to support dynamically-enabled XSTATE features with large state component (e.g. 8K for AMX).
There are a lot of simplications when comparing v2/v3 to the original proposal [2] and the first version [3]. Thanks to Thomas and Paolo for many good suggestions.
The support is based on following key changes:
- Guest permissions for dynamically-enabled XSAVE features
Native tasks have to request permission via
prctl()before touching a dynamic-resized XSTATE compoenent. Introduce guest permissions for the similar purpose. Userspace VMM is expected to request guest permission only once when the first vCPU is created.KVM checks guest permission in
KVM_SET_CPUID2. Setting XFD in guest cpuid w/o proper permissions fails this operation. In the meantime, unpermitted features are also excluded inKVM_GET_SUPPORTED_CPUID.- Extend fpstate reallocation mechanism to cover guest fpu
Unlike native tasks which have reallocation triggered from #NM handler, guest fpstate reallocation is requested by KVM when it identifies the intention on using dynamically-enabled XSAVE features inside guest.
Extend fpu core to allow KVM request fpstate buffer expansion for a guest fpu containter.
- Trigger fpstate reallocation in KVM
This could be done either statically (before guest runs) or dynamically (in the emulation path). According to discussion [1] we decide to statically enable all xfeatures allowed by guest perm in
KVM_SET_CPUID2, with fpstate buffer sized accordingly. This spares a lot of code and also avoid imposing an ordered restore sequence (XCR0, XFD and XSTATE) to userspace VMM.- RDMSR/WRMSR emulation for
IA32_XFDBecause fpstate expansion is completed in
KVM_SET_CPUID2, emulating r/w access toIA32_XFDsimply involves the xfd field in the guest fpu container. If write and guest fpu is currently active, the software state (guest_fpstate::xfdand per-cpu xfd cache) is also updated.- RDMSR/WRMSR emulation for
XFD_ERRWhen XFD causes an instruction to generate #NM,
XFD_ERRcontains information about which disabled state components are being accessed. It'd be problematic if theXFD_ERRvalue generated in guest is consumed/clobbered by the host before the guest itself doing so.Intercept #NM exception to save the guest
XFD_ERRvalue when writeIA32_XFDwith a non-zero value for 1st time. There is at most one interception per guest task given a dynamic feature.RDMSR/WRMSR emulation uses the saved value. The host value (always ZERO outside of the host #NM handler) is restored before enabling preemption. The saved guest value is restored right before entering the guest (with preemption disabled).
- Get/set dynamic xfeature state for migration
Introduce new capability (
KVM_CAP_XSAVE2) to deal with >4KB fpstate buffer. Reading this capability returns the size of the current guest fpstate (e.g. after expansion). Userspace VMM uses a new ioctl (KVM_GET_XSAVE2) to read guest fpstate from the kernel and reuses the existing ioctl (KVM_SET_XSAVE) to update guest fpsate to the kernel.KVM_SET_XSAVEis extended to doproperly_sizedmemdup_user()based on the guest fpstate.- Expose related cpuid bits to guest
The last step is to allow exposing XFD,
AMX_TILE,AMX_INT8andAMX_BF16in guest cpuid. Adding those bits intokvm_cpu_capsfinally activates all previous logics in this series- Optimization: disable interception for
IA32_XFD
IA32_XFDcan be frequently updated by the guest, as it is part of the task state and swapped in context switch when prev and next have different XFD setting. Always intercepting WRMSR can easily cause non-negligible overhead.Disable r/w emulation for
IA32_XFDafter intercepting the firstWRMSR(IA32_XFD)with a non-zero value. However MSR passthrough implies the software state (guest_fpstate::xfdand per-cpu xfd cache) might be out of sync with MSR. This suggests KVM needs to re-sync them at VM-exit before preemption is enabled.Thanks Jun Nakajima and Kevin Tian for the design suggestions when this version is being internally worked on.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://www.spinics.net/lists/kvm/msg259015.html
[3] https://lore.kernel.org/lkml/[email protected]/Thanks,
Yang---
Guang Zeng (1):
kvm: x86: Add support for getting/setting expanded xstate bufferJing Liu (11):
kvm: x86: Fixxstate_required_size()to follow XSTATE alignment rule
kvm: x86: Exclude unpermitted xfeatures atKVM_GET_SUPPORTED_CPUIDx86/fpu: Make XFD initialization in__fpstate_reset()a function
argument
kvm: x86: Check and enable permitted dynamic xfeatures at
KVM_SET_CPUID2
kvm: x86: Add emulation forIA32_XFD
x86/fpu: Preparexfd_errin structfpu_guest
kvm: x86: Intercept #NM for savingIA32_XFD_ERR
kvm: x86: EmulateIA32_XFD_ERRfor guest
kvm: x86: Disable RDMSR interception ofIA32_XFD_ERR
kvm: x86: Add XCR0 support for Intel AMX
kvm: x86: Add CPUID support for Intel AMXKevin Tian (3):
x86/fpu: Providefpu_update_guest_perm_features()for guest x86/fpu: Providefpu_update_guest_xfd()forIA32_XFDemulation
kvm: x86: Disable interception forIA32_XFDon demandThomas Gleixner (5):
x86/fpu: Extendfpu_xstate_prctl()with guest permissions x86/fpu: Prepare guest FPU for dynamically enabled FPU features x86/fpu: Add guest support toxfd_enable_feature()x86/fpu: Adduabi_sizetoguest_fpu
x86/fpu: Providefpu_sync_guest_vmexit_xfd_state()Wei Wang (1):
kvm: selftests: Add support forKVM_CAP_XSAVE2Documentation/virt/kvm/api.rst | 46 +++++- arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/fpu/api.h | 11 ++ arch/x86/include/asm/fpu/types.h | 32 ++++ arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 16 +- arch/x86/include/uapi/asm/prctl.h | 26 ++-- arch/x86/kernel/fpu/core.c | 104 ++++++++++++- arch/x86/kernel/fpu/xstate.c | 147 +++++++++++------- arch/x86/kernel/fpu/xstate.h | 15 +- arch/x86/kernel/process.c | 2 + arch/x86/kvm/cpuid.c | 99 +++++++++--- arch/x86/kvm/vmx/vmcs.h | 5 + arch/x86/kvm/vmx/vmx.c | 45 +++++- arch/x86/kvm/vmx/vmx.h | 2 +- arch/x86/kvm/x86.c | 105 ++++++++++++- include/uapi/linux/kvm.h | 4 + tools/arch/x86/include/uapi/asm/kvm.h | 16 +- tools/include/uapi/linux/kvm.h | 3 + .../testing/selftests/kvm/include/kvm_util.h | 2 + .../selftests/kvm/include/x86_64/processor.h | 10 ++ tools/testing/selftests/kvm/lib/kvm_util.c | 32 ++++ .../selftests/kvm/lib/x86_64/processor.c | 67 +++++++- .../testing/selftests/kvm/x86_64/evmcs_test.c | 2 +- tools/testing/selftests/kvm/x86_64/smm_test.c | 2 +- .../testing/selftests/kvm/x86_64/state_test.c | 2 +- .../kvm/x86_64/vmx_preemption_timer_test.c | 2 +- 27 files changed, 691 insertions(+), 109 deletions(-)
Sean Christopherson
maintainerQuoted 10 lines · Paolo Bonzini
Quoted 7 lines · Yang Zhong
Highly appreciate for your review. This version mostly addressed the comments from Sean. Most comments are adopted except three which are not closed and need more discussions:
- Move the entire xfd write emulation code to
x86.c. Doing so requires introducing a newkvm_x86_opscallback to disable msr write bitmap. According to Paolo's earlier comment he prefers to handle it invmx.c.Yes, I do.
No objection, my comments were prior to seeing the patches that manipulated the bitmap, e.g. in the earlier patches, having anything in vmx.c is unnecessary.
Quoted 20 lines
Quoted 9 lines
- Directly check
msr_bitmapinupdate_exception_bitmap()(for trapping #NM) andvcpu_enter_guest()(for syncing guest xfd after vm-exit) instead of introducing an extra flag in the last patch. However, doing so requires another newkvm_x86_opscallback for checkingmsr_bitmapsincevcpu_enter_guest()is x86 common code. Having an extra flag sounds simpler here (at least for the initial AMX support). It does penalize nested guest with one xfd sync per exit, but it's not worse than a normal guest which initializes xfd but doesn't run AMX applications at all. Those could be improved afterwards.The thing to do here would be to move MAX_POSSIBLE_PASSTHROUGH_MSRS/MAX_DIRECT_ACCESS_MSRS from VMX/SVM to core code. For now we can keep the flag.
Quoted 5 lines
- Disable #NM trap for nested guest. This version still chooses to always trap #NM (regardless in L1 or L2) as long as xfd write interception is disabled. In reality #NM is rare if nested guest doesn't intend to run AMX applications and always-trap is safer than dynamic trap for the basic support in case of any oversight here.
Quoted 4 lines
Sean was justifying this with lack of support for nested AMX, but I'm not sure actually what is missing at all. That is, an L1 hypervisor could expose AMX to L2, and then an
L2->L0->L2exit/reentry would have to trap #NM. Otherwise it would miss anXFD_ERRupdate.
Ya, I was assuming there was something L0 needed to do to supported nested AMX, but as Paolo pointed out there are no VMCS bits, so L0 just needs to correctly handle #NM and MSR interceptions according to vmcs12.
Quoted 31 lines
From: Sean Christopherson <[email protected]>
Sent: Wednesday, January 5, 2022 2:55 AMQuoted 2 lines · Paolo Bonzini
Quoted 1 line · Yang Zhong
Highly appreciate for your review. This version mostly addressed the
comments
Quoted 1 line
Quoted 1 line
from Sean. Most comments are adopted except three which are not
closed and
Quoted 7 lines
Quoted 5 lines
need more discussions:
- Move the entire xfd write emulation code to
x86.c. Doing so requires introducing a newkvm_x86_opscallback to disable msr write bitmap. According to Paolo's earlier comment he prefers to handle it invmx.c.Yes, I do.
No objection, my comments were prior to seeing the patches that manipulated the
bitmap, e.g. in the earlier patches, having anything invmx.cis unnecessary.Quoted 10 lines
Quoted 9 lines
- Directly check
msr_bitmapinupdate_exception_bitmap()(for trapping #NM) andvcpu_enter_guest()(for syncing guest xfd after vm-exit) instead of introducing an extra flag in the last patch. However, doing so requires another newkvm_x86_opscallback for checkingmsr_bitmapsincevcpu_enter_guest()is x86 common code. Having an extra flag sounds simpler here (at least for the initial AMX support). It does penalize nested guest with one xfd sync per exit, but it's not worse than a normal guest which initializes xfd but doesn't run AMX applications at all. Those could be improved afterwards.
Quoted 4 lines
Quoted 2 lines
The thing to do here would be to move MAX_POSSIBLE_PASSTHROUGH_MSRS/MAX_DIRECT_ACCESS_MSRS from
VMX/SVM to core
Quoted 1 line
code. For now we can keep the flag.
sounds good.
Quoted 15 lines
Quoted 3 lines
Quoted 3 lines
- Disable #NM trap for nested guest. This version still chooses to always trap #NM (regardless in L1 or L2) as long as xfd write interception is
disabled.
Quoted 1 line
Quoted 1 line
In reality #NM is rare if nested guest doesn't intend to run AMX
applications
Quoted 1 line
Quoted 1 line
and always-trap is safer than dynamic trap for the basic support in
case
Quoted 6 lines
Quoted 1 line
of any oversight here.
Sean was justifying this with lack of support for nested AMX, but I'm not sure actually what is missing at all. That is, an L1 hypervisor could expose AMX to L2, and then an
L2->L0->L2exit/reentry would have to trap #NM. Otherwise it would miss anXFD_ERRupdate.
Quoted 4 lines
Ya, I was assuming there was something L0 needed to do to supported nested AMX,
but as Paolo pointed out there are no VMCS bits, so L0 just needs to correctly handle #NM and MSR interceptions according to vmcs12.
btw Sean still made a good point on exception queuing part. Current version blindly queues a #NM even when L1 wants to intercept #NM itself. We had that fixed internally and will send out a new version very soon.
Thanks
Kevin