AMX Support in KVM
Subsystems (5)
cover AMX Support in KVM
(send on behalf of Jing who is currently on leave)
This series brings AMX (Advanced Matrix eXtensions) virtualization support to KVM. The three preparation patches in fpu core from Thomas [1] are also included.
A large portion of the changes in this series is to deal with eXtended Feature Disable (XFD) which allows resizing of the fpstate buffer to support dynamically-enabled XSTATE features with large state component (e.g. 8K for AMX).
The support is based on several key changes (design discussions can be found in [2]):
- Guest permissions for dynamically-enabled XSAVE features
Native tasks have to request permission via prctl() before touching a dynamic-resized XSTATE compoenent. Introduce guest permissions for the similar purpose. Userspace VMM is expected to request guest permission only once when the first vCPU is created.
KVM checks guest permission in KVM_SET_CPUID2. Setting XFD in guest cpuid w/o proper permissions fails this operation.
- Extend fpstate reallocation mechanism to cover guest fpu
Unlike native tasks which have reallocation triggered from #NM handler, guest fpstate reallocation is requested by KVM when it detects the guest intention to use dynamically-enabled XSAVE features.
The reallocation request is handled when exiting to userspace VMM. This implies that KVM must break vcpu_run() loop and exit to userspace VMM instead of immediately resuming back to the guest when reallocation is required.
- Detect fpstate reallocation in the emulation code
Because guest #NM is not trapped in KVM (costly), the guest intention of using a dynamically-enabled XSAVE feature[i] can be indirectly represented by guest XCR0[i]=1 and XFD[i]=0. This requires the emulation logic of both WRMSR(IA32_XFD) and XSETBV to check reallocation requirement when one of the two conditions is changed.
- Disable WRMSR interception for IA32_XFD
IA32_XFD can be frequently updated by the guest, as it is part of the task state and swapped in context switch when prev and next have different XFD setting. Always intercepting WRMSR can easily cause non-negligible overhead.
Disable WRMSR interception for IA32_XFD after fpstate reallocation succeeds. After that point the guest direct writes IA32_XFD without causing VM-exits.
However MSR passthrough implies that guest_fpstate::xfd and per-cpu xfd cache might be out of sync with the current IA32_XFD value set by the guest. This suggests KVM needs to re-sync the software state with IA32_XFD before the vCPU thread might be preempted or interrupted.
- Save/restore guest XFD_ERR
When XFD causes an instruction to generate #NM, XFD_ERR contains information about which disabled state components are being accessed. The #NM handler is expected to check this information and then enable the state components by clearing IA32_XFD for the faulting task (if having permission).
#NM can be triggered in both host and guest. It'd be problematic if the XFD_ERR value generated in guest is consumed/clobbered by the host before the guest itself doing so. This may lead to non-XFD- related #NM treated as XFD #NM in host (due to guest XFD_ERR value), or XFD-related #NM treated as non-XFD #NM in guest (XFD_ERR cleared by the host #NM handler).
KVM needs to save the guest XFD_ERR value before this register might be accessed by the host and restore it before entering the guest.
One open remains in this area about when to start saving/restoring guest XFD_ERR. Several options are discussed in patch 15.
- Expose related cpuid bits to guest
The last step is to allow exposing XFD, AMX_TILE, AMX_INT8 and AMX_BF16 in guest cpuid. Adding those bits into kvm_cpu_caps finally activates all previous logics in this series
To verify AMX virtualization overhead on non-AMX usages, we run the Phoronix kernel build test in the guest w/ and w/o AMX in cpuid. The result shows no observable difference between two configurations.
Live migration support is still being worked on. Userspace VMM needs to use the new KVM_{G|S}SET_XSAVE2 ioctl in this series to migrate state for dynamically-enabled XSAVE features.
Thanks Thomas for the thoughts and patches on the KVM FPU and AMX support. Thanks Jun Nakajima for the design suggestions.
[1] git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git x86/fpu-kvm
[2] https://www.spinics.net/lists/kvm/msg259015.html
Thanks,
Yang
---
Jing Liu (13):
kvm: x86: Fix xstate_required_size() to follow XSTATE alignment rule
kvm: x86: Check guest xstate permissions when KVM_SET_CPUID2 x86/fpu: Move xfd initialization out of __fpstate_reset() to the
callers
kvm: x86: Propagate fpstate reallocation error to userspace x86/fpu: Move xfd_update_state() to xstate.c and export symbol
kvm: x86: Prepare reallocation check
kvm: x86: Emulate WRMSR of guest IA32_XFD
kvm: x86: Disable WRMSR interception for IA32_XFD on demand x86/fpu: Prepare for KVM XFD_ERR handling
kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
docs: virt: api.rst: Document the new KVM_{G, S}ET_XSAVE2 ioctls
kvm: x86: AMX XCR0 support for guest
kvm: x86: Add AMX CPUIDs support
Thomas Gleixner (4):
x86/fpu: Extend prctl() with guest permissions x86/fpu: Prepare KVM for dynamically enabled states x86/fpu: Add reallocation mechanims for KVM x86/fpu: Prepare KVM for bringing XFD state back in-sync
Yang Zhong (2):
kvm: x86: Check fpstate reallocation in XSETBV emulation
kvm: x86: Save and restore guest XFD_ERR properly
Documentation/virt/kvm/api.rst | 47 +++++++ arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/fpu/api.h | 12 ++ arch/x86/include/asm/fpu/types.h | 56 +++++++++ arch/x86/include/asm/fpu/xstate.h | 2 + arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 + arch/x86/include/uapi/asm/kvm.h | 6 + arch/x86/include/uapi/asm/prctl.h | 26 ++-- arch/x86/kernel/fpu/core.c | 109 ++++++++++++++++- arch/x86/kernel/fpu/xstate.c | 119 +++++++++++++++--- arch/x86/kernel/fpu/xstate.h | 29 +++-- arch/x86/kernel/process.c | 2 + arch/x86/kvm/cpuid.c | 36 +++++- arch/x86/kvm/vmx/vmx.c | 20 +++ arch/x86/kvm/vmx/vmx.h | 2 +- arch/x86/kvm/x86.c | 189 ++++++++++++++++++++++++++++- arch/x86/kvm/x86.h | 2 + include/uapi/linux/kvm.h | 8 +- 19 files changed, 607 insertions(+), 63 deletions(-)
PATCH 1/19 x86/fpu: Extend prctl() with guest permissions
From: Thomas Gleixner <[email protected]>
Add guest permission control for dynamic XSTATE components, including extension to prctl() with two new options (ARCH_GET_XCOMP_GUEST_PERM and ARCH_REQ_XCOMP_GUEST_PERM) and to struct fpu with a new member (guest_perm).
Userspace VMM has to request guest permissions before it exposes any XSAVE feature using dynamic XSTATE components. The permission can be set only once when the first vCPU is created. A new flag FPU_GUEST_PERM_LOCKED is introduced to lock the change for this purpose
Similar to native permissions this doesn't actually enable the permitted feature. KVM is expected to install a larger kernel buffer and enable the feature when detecting the intention from the guest.
---
(To Thomas) We change the definition of xstate_get_guest_group_perm() from xstate.h to api.h since this will be called by KVM.
arch/x86/include/asm/fpu/api.h | 2 ++ arch/x86/include/asm/fpu/types.h | 9 ++++++ arch/x86/include/uapi/asm/prctl.h | 26 ++++++++-------- arch/x86/kernel/fpu/core.c | 3 ++ arch/x86/kernel/fpu/xstate.c | 50 +++++++++++++++++++++++-------- arch/x86/kernel/fpu/xstate.h | 13 ++++++-- arch/x86/kernel/process.c | 2 ++ 7 files changed, 78 insertions(+), 27 deletions(-)
arch/x86/include/asm/fpu/api.h +2 -0
@@ -138,6 +138,8 @@ static inline void fpstate_free(struct fpu *fpu) { }/* fpstate-related functions which are exported to KVM */ extern void fpstate_clear_xstate_component (struct fpstate *fps,unsigned int xfeature); +extern inline u64 xstate_get_guest_group_perm (void ); +/* KVM specific functions */ extern bool fpu_alloc_guest_fpstate (struct fpu_guest *gfpu);extern void fpu_free_guest_fpstate (struct fpu_guest *gfpu);
arch/x86/include/asm/fpu/types.h +9 -0
@@ -387,6 +387,8 @@ struct fpstate {/* @regs is dynamically sized! Don't add anything after @regs! */ }__aligned(64) ; +#define FPU_GUEST_PERM_LOCKED BIT_ULL(63) +struct fpu_state_perm {/* * @__state_perm:
@@ -476,6 +478,13 @@ struct fpu {*/ struct fpu_state_perm perm; +/* +* @guest_perm: +* +* Permission related information for guest pseudo FPUs +*/ +struct fpu_state_perm guest_perm; +/* * @__fpstate: *
arch/x86/include/uapi/asm/prctl.h +14 -12
@@ -2,20 +2,22 @@#ifndef _ASM_X86_PRCTL_H#define _ASM_X86_PRCTL_H -#define ARCH_SET_GS 0x1001 -#define ARCH_SET_FS 0x1002 -#define ARCH_GET_FS 0x1003 -#define ARCH_GET_GS 0x1004 +#define ARCH_SET_GS 0x1001 +#define ARCH_SET_FS 0x1002 +#define ARCH_GET_FS 0x1003 +#define ARCH_GET_GS 0x1004 -#define ARCH_GET_CPUID 0x1011 -#define ARCH_SET_CPUID 0x1012 +#define ARCH_GET_CPUID 0x1011 +#define ARCH_SET_CPUID 0x1012 -#define ARCH_GET_XCOMP_SUPP 0x1021 -#define ARCH_GET_XCOMP_PERM 0x1022 -#define ARCH_REQ_XCOMP_PERM 0x1023 +#define ARCH_GET_XCOMP_SUPP 0x1021 +#define ARCH_GET_XCOMP_PERM 0x1022 +#define ARCH_REQ_XCOMP_PERM 0x1023 +#define ARCH_GET_XCOMP_GUEST_PERM 0x1024 +#define ARCH_REQ_XCOMP_GUEST_PERM 0x1025 -#define ARCH_MAP_VDSO_X32 0x2001 -#define ARCH_MAP_VDSO_32 0x2002 -#define ARCH_MAP_VDSO_64 0x2003 +#define ARCH_MAP_VDSO_X32 0x2001 +#define ARCH_MAP_VDSO_32 0x2002 +#define ARCH_MAP_VDSO_64 0x2003#endif /* _ASM_X86_PRCTL_H */
arch/x86/kernel/fpu/core.c +3 -0
@@ -450,6 +450,8 @@ void fpstate_reset(struct fpu *fpu) fpu->perm.__state_perm = fpu_kernel_cfg.default_features; fpu->perm.__state_size = fpu_kernel_cfg.default_size; fpu->perm.__user_state_size = fpu_user_cfg.default_size; +/* Same defaults for guests */ + fpu->guest_perm = fpu->perm; }static inline void fpu_inherit_perms (struct fpu *dst_fpu)
@@ -460,6 +462,7 @@ static inline void fpu_inherit_perms(struct fpu *dst_fpu)spin_lock_irq (¤t->sighand->siglock);/* Fork also inherits the permissions of the parent */ dst_fpu->perm = src_fpu->perm; + dst_fpu->guest_perm = src_fpu->guest_perm;spin_unlock_irq (¤t->sighand->siglock); } }
arch/x86/kernel/fpu/xstate.c +37 -13
115 lines not on this page · raw message
arch/x86/kernel/fpu/xstate.h +11 -2
@@ -20,10 +20,19 @@ static inline void xstate_init_xcomp_bv(struct xregs_state *xsave, u64 mask) xsave->header.xcomp_bv = mask |XCOMP_BV_COMPACTED_FORMAT ; } -static inline u64 xstate_get_host_group_perm (void ) +static inline u64 xstate_get_group_perm (bool guest) { +struct fpu *fpu = ¤t->group_leader->thread.fpu; +struct fpu_state_perm *perm; +/* Pairs with WRITE_ONCE() in xstate_request_perm() */ -return READ_ONCE (current->group_leader->thread.fpu.perm.__state_perm); + perm = guest ? &fpu->guest_perm : &fpu->perm; +return READ_ONCE (perm->__state_perm); +} + +static inline u64 xstate_get_host_group_perm (void ) +{ +return xstate_get_group_perm (false); }enum xstate_copy_mode {
arch/x86/kernel/process.c +2 -0
@@ -993,6 +993,8 @@ long do_arch_prctl_common(struct task_struct *task, int option,case ARCH_GET_XCOMP_SUPP :case ARCH_GET_XCOMP_PERM :case ARCH_REQ_XCOMP_PERM : +case ARCH_GET_XCOMP_GUEST_PERM : +case ARCH_REQ_XCOMP_GUEST_PERM :return fpu_xstate_prctl (task, option, arg2); }
Re: [PATCH 01/19] x86/fpu: Extend prctl() with guest permissions
Quoted 10 lines · Yang Zhong
Similar to native permissions this doesn't actually enable the permitted feature. KVM is expected to install a larger kernel buffer and enable the feature when detecting the intention from the guest.
Signed-off-by Thomas Gleixner <[email protected]>Signed-off-by Jing Liu <[email protected]>Signed-off-by Yang Zhong <[email protected]>---
(To Thomas) We change the definition ofxstate_get_guest_group_perm()fromxstate.htoapi.hsince this will be called by KVM.
No.
There is absolutely no need for that. After creating a vCPU the permissions are frozen and readily available via vcpu->arch.guest_fpu.perm.
Thanks,
tglx
PATCH 2/19 x86/fpu: Prepare KVM for dynamically enabled states
In reply to Yang ZhongFrom: Thomas Gleixner <[email protected]>
Add more fields for tracking per-vCPU permissions for dynamic XSTATE components:
- user_xfeatures
Track which features are currently enabled for the vCPU
- user_perm
Copied from guest_perm of the group leader thread. The first vCPU which does the copy locks the guest_perm
- realloc_request
KVM sets this field to request dynamically-enabled features which require reallocation of @fpstate
Initialize those fields properly.
--- arch/x86/include/asm/fpu/types.h | 23 +++++++++++++++++++++++ arch/x86/kernel/fpu/core.c | 26 +++++++++++++++++++++++++- 2 files changed, 48 insertions(+), 1 deletion(-)
arch/x86/include/asm/fpu/types.h +23 -0
@@ -504,6 +504,29 @@ struct fpu {* Guest pseudo FPU container */ struct fpu_guest { +/* +* @user_xfeatures: xfeature bitmap of features which are +* currently enabled for the guest vCPU. +*/ +u64 user_xfeatures; + +/* +* @user_perm: xfeature bitmap of features which are +* permitted to be enabled for the guest +* vCPU. +*/ +u64 user_perm; + +/* +* @realloc_request: xfeature bitmap of features which are +* requested to be enabled dynamically +* which requires reallocation of @fpstate +* +* Set by an intercept handler and +* evaluated in fpu_swap_kvm_fpstate() +*/ +u64 realloc_request; +/* * @fpstate: Pointer to the allocated guest fpstate */
arch/x86/kernel/fpu/core.c +25 -1
@@ -201,6 +201,26 @@ void fpu_reset_from_exception_fixup(void)#if IS_ENABLED (CONFIG_KVM )static void __fpstate_reset (struct fpstate *fpstate); +static void fpu_init_guest_permissions (struct fpu_guest *gfpu) +{ +struct fpu_state_perm *fpuperm; +u64 perm; + +if (!IS_ENABLED (CONFIG_X86_64 )) +return ; + +spin_lock_irq (¤t->sighand->siglock); + fpuperm = ¤t->group_leader->thread.fpu.guest_perm; + perm = fpuperm->__state_perm; + +/* First fpstate allocation locks down permissions. */ +WRITE_ONCE (fpuperm->__state_perm, perm |FPU_GUEST_PERM_LOCKED ); + +spin_unlock_irq (¤t->sighand->siglock); + + gfpu->user_perm = perm & ~FPU_GUEST_PERM_LOCKED ; +} +bool fpu_alloc_guest_fpstate (struct fpu_guest *gfpu) {struct fpstate *fpstate;
@@ -216,7 +236,11 @@ bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) fpstate->is_valloc = true; fpstate->is_guest = true; - gfpu->fpstate = fpstate; + gfpu->fpstate = fpstate; + gfpu->user_xfeatures = fpu_user_cfg.default_features; + gfpu->user_perm = fpu_user_cfg.default_features; +fpu_init_guest_permissions (gfpu); +return true; }EXPORT_SYMBOL_GPL (fpu_alloc_guest_fpstate);
Paolo Bonzini
maintainerRe: [PATCH 02/19] x86/fpu: Prepare KVM for dynamically enabled states
Quoted 3 lines · Yang Zhong
-
user_xfeaturesTrack which features are currently enabled for the vCPU
Please rename to alloc_xfeatures
Quoted 4 lines
-
user_permCopied from
guest_permof the group leader thread. The first vCPU which does the copy locks theguest_perm
Please rename to perm_xfeatures.
Quoted 4 lines
-
realloc_requestKVM sets this field to request dynamically-enabled features which require reallocation of @fpstate
This field should be in vcpu->arch, and there is no need for fpu_guest_realloc_fpstate. Rename __xfd_enable_feature to fpu_enable_xfd_feature and add it to the public API, then just do
if (unlikely(vcpu->arch.xfd_realloc_request)) {
u64 request = vcpu->arch.xfd_realloc_request;
ret = fpu_enable_xfd(request, enter_guest);
}to kvm_put_guest_fpu.
Paolo
Re: [PATCH 02/19] x86/fpu: Prepare KVM for dynamically enabled states
Quoted 6 lines · Paolo Bonzini
Quoted 3 lines · Yang Zhong
-
user_xfeaturesTrack which features are currently enabled for the vCPU
Please rename to
alloc_xfeatures
That name makes no sense at all. This has nothing to do with alloc.
Quoted 6 lines
Quoted 4 lines
-
user_permCopied from
guest_permof the group leader thread. The first vCPU which does the copy locks theguest_permPlease rename to
perm_xfeatures.
All of that is following the naming conventions in the FPU code related to permissions etc.
Quoted 9 lines
Quoted 4 lines
-
realloc_requestKVM sets this field to request dynamically-enabled features which require reallocation of @fpstate
This field should be in
vcpu->arch, and there is no need forfpu_guest_realloc_fpstate. Rename__xfd_enable_featuretofpu_enable_xfd_featureand add it to the public API, then just do
Quoted 6 lines
if (unlikely(vcpu->arch.xfd_realloc_request)) { u64 request = vcpu->arch.xfd_realloc_request; ret = fpu_enable_xfd(request, enter_guest); }to
kvm_put_guest_fpu.
Why? Yet another export of FPU internals just because?
Also what clears the reallocation request and what is the @enter_guest argument supposed to help with?
I have no idea what you are trying to achieve.
Thanks,
tglx
Paolo Bonzini
maintainerRe: [PATCH 02/19] x86/fpu: Prepare KVM for dynamically enabled states
Quoted 9 lines · Thomas Gleixner
Quoted 6 lines · Paolo Bonzini
Quoted 3 lines · Yang Zhong
-
user_xfeaturesTrack which features are currently enabled for the vCPU
Please rename to
alloc_xfeaturesThat name makes no sense at all. This has nothing to do with alloc.
Isn't that the features for which space is currently allocated?
fpstate_realloc does
+ if (guest_fpu) {
+ newfps->is_guest = true;
+ newfps->is_confidential = curfps->is_confidential;
+ guest_fpu->user_xfeatures |= xfeatures;
+ }
+
and kvm_check_guest_realloc_fpstate does
+ if ((guest_fpu->user_xfeatures & request) != request) {
+ vcpu->arch.guest_fpu.realloc_request |= request;
+ return true;
+ }
Reading "user_xfeatures" in there is cryptic, it seems like it's something related to the userspace thread or group that has invoked the KVM ioctl. If it's renamed to alloc_xfeatures, then this:
+ missing = request & ~guest_fpu->alloc_xfeatures;
+ if (missing) {
+ vcpu->arch.guest_fpu.realloc_request |= missing;
+ return true;
+ }
makes it obvious that the allocation is for features that are requested but haven't been allocated in the xstate yet.
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
-
user_permCopied from
guest_permof the group leader thread. The first vCPU which does the copy locks theguest_permPlease rename to
perm_xfeatures.All of that is following the naming conventions in the FPU code related to permissions etc.
perm or guest_perm is just fine as well; but user_perm is not (there's no preexisting reference to user_perm in the code, in fact, as far as I can see).
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines
-
realloc_requestKVM sets this field to request dynamically-enabled features which require reallocation of @fpstate
This field should be in
vcpu->arch, and there is no need forfpu_guest_realloc_fpstate. Rename__xfd_enable_featuretofpu_enable_xfd_featureand add it to the public API, then just do
Quoted 8 lines
Quoted 6 lines
if (unlikely(vcpu->arch.xfd_realloc_request)) { u64 request = vcpu->arch.xfd_realloc_request; ret = fpu_enable_xfd(request, enter_guest); }to
kvm_put_guest_fpu.Why? Yet another export of FPU internals just because?
It's one function more and one field less. I prefer another export of FPU internals, to a write to a random field with undocumented invariants.
For example, why WARN_ON_ONCE if enter_guest == true? If you enter the guest after the host has restored MSR_IA32_XFD with KVM_SET_MSR, the WARN_ON_ONCE can actually fire. It would be just fine to skip the reallocation until enter_guest == false, which is you actually XSAVE into the guest_fpu.
As an aside, realloc_request (if it survives, see below) shouldn't be added until patch 6.
Quoted 2 lines
Also what clears the reallocation request and what is the @enter_guest argument supposed to help with?
Nothing---make it
if (unlikely(vcpu->arch.xfd_realloc_request)) {
u64 request = vcpu->arch.xfd_realloc_request;
ret = fpu_enable_xfd(request, &vcpu->arch.guest_fpu); if (!ret) vcpu->arch.xfd_realloc_request = 0;
}
but in fact, I'm not sure why the request has to be delayed at all. The obvious implementation of a write to XFD, after all the validity checks, is just
/* This function just calls xfd_enable_feature. */ r = fpu_guest_alloc_xfeatures(&vcpu->arch.guest_fpu, vcpu->arch.xcr0 & ~xfd); /* * An error means that userspace has screwed up by not doing * arch_prctl(ARCH_REQ_XCOMP_GUEST_PERM). If we are here coming * from a ioctl fail it, if the guest has done WRMSR or XSETBV * it will inject a #GP. */ if (r < 0) return 1; vcpu->arch.xfd = xfd;
with none of the kvm_check_guest_realloc_fpstate or KVM_EXIT_FPU_REALLOC business. No field and a simple, self-contained external API. The same code works for __kvm_set_xcr as well, just with "xcr0 & ~vcpu->arch.xfd" as the second argument instead.
(Maybe I'm missing why KVM_EXIT_FPU_REALLOC is needed, but I'm not ashamed to say that, given that new userspace API was added with documentation or tests. The only comment in the code is:
+ /*
+ * Check if fpstate reallocate is required. If yes, then
+ * let the fpu core do reallocation and update xfd;
+ * otherwise, update xfd here.
+ */
+ if (kvm_check_guest_realloc_fpstate(vcpu, data)) {
+ vcpu->run->exit_reason = KVM_EXIT_FPU_REALLOC;
+ vcpu->arch.complete_userspace_io =
+ kvm_skip_emulated_instruction;
+ return KVM_MSR_RET_USERSPACE;
+ }
+
which has nothing to do with the actual content of either kvm_check_guest_realloc_fpstate or the "then" branch. Userspace can just do ARCH_REQ_XCOMP_GUEST_PERM based on the guest CPUID, before KVM_SET_CPUID2, KVM_SET_MSR or KVM_SET_XCR).
Paolo
Re: [PATCH 02/19] x86/fpu: Prepare KVM for dynamically enabled states
Paolo,
Quoted 7 lines · Paolo Bonzini
Quoted 4 lines · Thomas Gleixner
Quoted 1 line · Paolo Bonzini
Please rename to
alloc_xfeaturesThat name makes no sense at all. This has nothing to do with alloc.
Isn't that the features for which space is currently allocated?
It is, but from the kernel POV this is user. :)
Quoted 10 lines
Reading "
user_xfeatures" in there is cryptic, it seems like it's something related to the userspace thread or group that has invoked the KVM ioctl. If it's renamed toalloc_xfeatures, then this:(quoted diff) +5
-0+ missing = request & ~guest_fpu->alloc_xfeatures; + if (missing) { + vcpu->arch.guest_fpu.realloc_request |= missing; + return true; + }
Quoted 2 lines
makes it obvious that the allocation is for features that are requested but haven't been allocated in the xstate yet.
Let's rename it to xfeatures and perm and be done with it.
Quoted 5 lines
Quoted 1 line
Why? Yet another export of FPU internals just because?
It's one function more and one field less. I prefer another export of FPU internals, to a write to a random field with undocumented invariants.
We want less not more exports. :)
Quoted 2 lines
For example, why
WARN_ON_ONCEifenter_guest== true? If you enter the guest after the host has restoredMSR_IA32_XFDwithKVM_SET_MSR, the
Indeed restoring a guest might require buffer reallocation, I missed that, duh!
On restore the following components are involved:
XCR0, XFD, XSTATE
XCR0 and XFD have to be restored _before_ XSTATE and that needs to be enforced.
But independent of the ordering of XCR0 and XFD restore the following check applies to both the restore and the runtime logic:
int kvm_fpu_realloc(struct kvm_vcpu *vcpu, u64 xcr0, u64 xfd) {
u64 expand, enabled = xcr0 & ~xfd;
expand = enabled & ~vcpu->arch.guest_fpu.xfeatures;
if (!expand)
return 0;
return fpu_enable_guest_features(&vcpu->arch.guest_fpu, expand);
}int fpu_enable_guest_features(struct guest_fpu *gfpu, u64 which)
{
permission_checks();
...
return fpstate_realloc(.....)
}fpstate_realloc() needs to be careful about flipping the pointers depending on the question whether guest_fpu->fpstate is actually active, i.e.:
current->thread.fpu.fpstate == gfpu->fpstate
I'm halfways done with that. Will send something soonish.
Thanks,
tglx
PATCH 3/19 kvm: x86: Fix xstate_required_size() to follow XSTATE alignment rule
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
CPUID.0xD.1.EBX enumerates the size of the XSAVE area (in compacted format) required by XSAVES. If CPUID.0xD.i.ECX[1] is set for a state component (i), this state component should be located on the next 64-bytes boundary following the preceding state component in the compacted layout.
Fix xstate_required_size() to follow the alignment rule. AMX is the first state component with 64-bytes alignment to catch this bug.
--- arch/x86/kvm/cpuid.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/cpuid.c +2 -1
@@ -42,7 +42,8 @@ static u32 xstate_required_size(u64 xstate_bv, bool compacted)if (xstate_bv &0x1 ) {u32 eax, ebx, ecx, edx, offset;cpuid_count (0xD , feature_bit, &eax, &ebx, &ecx, &edx); - offset = compacted ? ret : ebx; +/* ECX[1]: 64B alignment in compacted form */ + offset = compacted ? ((ecx &0x2 ) ?ALIGN (ret,64 ) : ret) : ebx; ret =max (ret, offset + eax); }
PATCH 4/19 kvm: x86: Check guest xstate permissions when KVM_SET_CPUID2
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Guest xstate permissions should be set by userspace VMM before vcpu creation. This patch extends KVM to check the guest permissions in KVM_SET_CPUID2 ioctl to avoid permission failure at guest run-time (e.g. when reallocation path is triggered).
--- arch/x86/kvm/cpuid.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/x86/kvm/cpuid.c +12 -0
@@ -18,6 +18,7 @@#include <asm/processor.h> #include <asm/user.h> #include <asm/fpu/xstate.h> +#include <asm/fpu/api.h> #include <asm/sgx.h> #include "cpuid.h" #include "lapic.h"
@@ -97,6 +98,17 @@ static int kvm_check_cpuid(struct kvm_cpuid_entry2 *entries, int nent)return -EINVAL ; } +/* +* Check guest permissions for XSTATE features which must +* be enabled dynamically. +*/ + best =cpuid_entry2_find (entries, nent,7 ,0 ); +if (best &&cpuid_entry_has (best,X86_FEATURE_AMX_TILE )) { +if (!(xstate_get_guest_group_perm () & +XFEATURE_MASK_XTILE_DATA )) +return -EINVAL ; + } +return 0 ; }
PATCH 5/19 x86/fpu: Move xfd initialization out of __fpstate_reset() to the callers
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
vCPU threads are different from native tasks regarding to the initial xfd value. While all native tasks follow a fixed value (init_fpstate::xfd) defined by fpu core, vCPU threads need to obey the reset value (i.e. ZERO) defined by the spec, to meet the expectation of the guest.
Move xfd initialization out of __fpstate_reset() to the callers for choosing a specific value.
--- arch/x86/kernel/fpu/core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kernel/fpu/core.c +2 -1
@@ -231,6 +231,7 @@ bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu)if (!fpstate)return false; +/* Leave xfd to 0 (the reset value defined by spec) */ __fpstate_reset (fpstate);fpstate_init_user (fpstate); fpstate->is_valloc = true;
@@ -461,7 +462,6 @@ static void __fpstate_reset(struct fpstate *fpstate) fpstate->user_size = fpu_user_cfg.default_size; fpstate->xfeatures = fpu_kernel_cfg.default_features; fpstate->user_xfeatures = fpu_user_cfg.default_features; - fpstate->xfd = init_fpstate.xfd; }void fpstate_reset (struct fpu *fpu)
@@ -469,6 +469,7 @@ void fpstate_reset(struct fpu *fpu)/* Set the fpstate pointer to the default fpstate */ fpu->fpstate = &fpu->__fpstate;__fpstate_reset (fpu->fpstate); + fpu->fpstate->xfd = init_fpstate.xfd;/* Initialize the permission related info in fpu */ fpu->perm.__state_perm = fpu_kernel_cfg.default_features;
Re: [PATCH 05/19] x86/fpu: Move xfd initialization out of __fpstate_reset() to the callers
Yang, Jing,
arch/x86/kernel/fpu/core.c bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) · line 235
Quoted 10 lines · Yang Zhong
arch/x86/kernel/fpu/core.c +1
-0@@ -231,6 +231,7 @@ bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu)if (!fpstate)return false; +/* Leave xfd to 0 (the reset value defined by spec) */ __fpstate_reset (fpstate);
That change makes me a bit wary simply because the comment here is above __fpstate_reset() which makes no sense. It does make sense to you at the time, but does it make sense to you when you look at it 6 month down the road?
So I'd rather make this very obvious what's going. See below.
Thanks,
tglx
---
arch/x86/kernel/fpu/core.c +6 -5
@@ -199,7 +199,7 @@ void fpu_reset_from_exception_fixup(void }#if IS_ENABLED (CONFIG_KVM ) -static void __fpstate_reset (struct fpstate *fpstate); +static void __fpstate_reset (struct fpstate *fpstate,u64 xfd);static void fpu_init_guest_permissions (struct fpu_guest *gfpu) {
@@ -231,7 +231,8 @@ bool fpu_alloc_guest_fpstate(struct fpu_if (!fpstate)return false; -__fpstate_reset (fpstate); +/* Leave xfd to 0 (the reset value defined by spec) */ +__fpstate_reset (fpstate,0 );fpstate_init_user (fpstate); fpstate->is_valloc = true; fpstate->is_guest = true;
@@ -454,21 +455,21 @@ void fpstate_init_user(struct fpstate *ffpstate_init_fstate (fpstate); } -static void __fpstate_reset (struct fpstate *fpstate) +static void __fpstate_reset (struct fpstate *fpstate,u64 xfd) {/* Initialize sizes and feature masks */ fpstate->size = fpu_kernel_cfg.default_size; fpstate->user_size = fpu_user_cfg.default_size; fpstate->xfeatures = fpu_kernel_cfg.default_features; fpstate->user_xfeatures = fpu_user_cfg.default_features; - fpstate->xfd = init_fpstate.xfd; + fpstate->xfd = xfd; }void fpstate_reset (struct fpu *fpu) {/* Set the fpstate pointer to the default fpstate */ fpu->fpstate = &fpu->__fpstate; -__fpstate_reset (fpu->fpstate); +__fpstate_reset (fpu->fpstate, init_fpstate.xfd);/* Initialize the permission related info in fpu */ fpu->perm.__state_perm = fpu_kernel_cfg.default_features;
PATCH 6/19 x86/fpu: Add reallocation mechanims for KVM
In reply to Yang ZhongFrom: Thomas Gleixner <[email protected]>
Extend fpstate reallocation mechanism to cover guest fpu. Unlike native tasks which have reallocation triggered from #NM handler, guest fpstate reallocation is requested by KVM when detecting the guest intention on using a dynamically-enabled XSAVE feature.
Since KVM currently swaps host/guest fpstate when exiting to userspace VMM (see fpu_swap_kvm_fpstate()), deal with fpstate reallocation also at this point.
The implication - KVM must break vcpu_run() loop to exit to userspace VMM instead of immediately returning back to the guest when fpstate requires reallocation. In this case KVM should set guest_fpu::realloc_request to mark those features in related VM exit handlers.
--- arch/x86/kernel/fpu/core.c | 26 +++++++++++++++++++--- arch/x86/kernel/fpu/xstate.c | 43 ++++++++++++++++++++++++++++++------ arch/x86/kernel/fpu/xstate.h | 2 ++ 3 files changed, 61 insertions(+), 10 deletions(-)
arch/x86/kernel/fpu/core.c +23 -3
@@ -261,11 +261,31 @@ void fpu_free_guest_fpstate(struct fpu_guest *gfpu) }EXPORT_SYMBOL_GPL (fpu_free_guest_fpstate); +static int fpu_guest_realloc_fpstate (struct fpu_guest *guest_fpu, +bool enter_guest) +{ +/* +* Reallocation requests can only be handled when +* switching from guest to host mode. +*/ +if (WARN_ON_ONCE (enter_guest || !IS_ENABLED (CONFIG_X86_64 ))) { + guest_fpu->realloc_request =0 ; +return -EUNATCH ; + } +return xfd_enable_guest_features (guest_fpu); +} +int fpu_swap_kvm_fpstate (struct fpu_guest *guest_fpu,bool enter_guest) { -struct fpstate *guest_fps = guest_fpu->fpstate; +struct fpstate *guest_fps, *cur_fps;struct fpu *fpu = ¤t->thread.fpu; -struct fpstate *cur_fps = fpu->fpstate; +int ret =0 ; + +if (unlikely (guest_fpu->realloc_request)) + ret =fpu_guest_realloc_fpstate (guest_fpu, enter_guest); + + guest_fps = guest_fpu->fpstate; + cur_fps = fpu->fpstate;fpregs_lock ();if (!cur_fps->is_confidential && !test_thread_flag (TIF_NEED_FPU_LOAD ))
@@ -298,7 +318,7 @@ int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest)fpregs_mark_activate ();fpregs_unlock (); -return 0 ; +return ret; }EXPORT_SYMBOL_GPL (fpu_swap_kvm_fpstate);
arch/x86/kernel/fpu/xstate.c +36 -7
100 lines not on this page · raw message
arch/x86/kernel/fpu/xstate.h +2 -0
@@ -55,6 +55,8 @@ extern void fpu__init_system_xstate(unsigned int legacy_size);extern void *get_xsave_addr (struct xregs_state *xsave,int xfeature_nr); +extern int xfd_enable_guest_features (struct fpu_guest *guest_fpu); +static inline u64 xfeatures_mask_supervisor (void ) {return fpu_kernel_cfg.max_features &XFEATURE_MASK_SUPERVISOR_SUPPORTED ;
PATCH 7/19 kvm: x86: Propagate fpstate reallocation error to userspace
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
fpstate reallocation is handled when the vCPU thread returns to userspace. As reallocation could fail (e.g. lack of memory), this patch extends kvm_put_guest_fpu() to return an integer value to carry error code to userspace VMM. The userspace VMM is expected to handle any error caused by fpstate reallocation.
--- arch/x86/kvm/x86.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
arch/x86/kvm/x86.c +11 -4
@@ -10171,17 +10171,21 @@ static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu) }/* When vcpu_run ends, restore user space FPU context. */ -static void kvm_put_guest_fpu (struct kvm_vcpu *vcpu) +static int kvm_put_guest_fpu (struct kvm_vcpu *vcpu) { -fpu_swap_kvm_fpstate (&vcpu->arch.guest_fpu, false); +int ret; + + ret =fpu_swap_kvm_fpstate (&vcpu->arch.guest_fpu, false); ++vcpu->stat.fpu_reload;trace_kvm_fpu (0 ); + +return ret; }int kvm_arch_vcpu_ioctl_run (struct kvm_vcpu *vcpu) {struct kvm_run *kvm_run = vcpu->run; -int r; +int r, ret;vcpu_load (vcpu);kvm_sigset_activate (vcpu);
@@ -10243,7 +10247,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) r =vcpu_run (vcpu); out: -kvm_put_guest_fpu (vcpu); + ret =kvm_put_guest_fpu (vcpu); +if ((r >=0 ) && (ret <0 )) + r = ret; +if (kvm_run->kvm_valid_regs)store_regs (vcpu);post_kvm_run_save (vcpu);
Paolo Bonzini
maintainerRe: [PATCH 07/19] kvm: x86: Propagate fpstate reallocation error to userspace
arch/x86/kvm/x86.c int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · line 10253
Quoted 6 lines · Yang Zhong
out:
arch/x86/kvm/x86.c +4
-1-kvm_put_guest_fpu (vcpu); + ret =kvm_put_guest_fpu (vcpu); +if ((r >=0 ) && (ret <0 )) + r = ret; +
No extra parentheses.
Paolo
PATCH 8/19 x86/fpu: Move xfd_update_state() to xstate.c and export symbol
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
xfd_update_state() is the interface to update IA32_XFD and its per-cpu cache. All callers of this interface are currently in fpu core. KVM only indirectly triggers IA32_XFD update via a helper function (fpu_swap_kvm_fpstate()) when switching between user fpu and guest fpu.
Supporting AMX in guest now requires KVM to directly update IA32_XFD with the guest value (when emulating WRMSR) so XSAVE/XRSTOR can manage XSTATE components correctly inside guest.
This patch moves xfd_update_state() from fpu/xstate.h to fpu/xstate.c and export it for reference outside of fpu core.
--- arch/x86/include/asm/fpu/api.h | 2 ++ arch/x86/kernel/fpu/xstate.c | 12 ++++++++++++ arch/x86/kernel/fpu/xstate.h | 14 +------------- 3 files changed, 15 insertions(+), 13 deletions(-)
arch/x86/include/asm/fpu/api.h +2 -0
@@ -131,8 +131,10 @@ DECLARE_PER_CPU(struct fpu *, fpu_fpregs_owner_ctx);/* Process cleanup */ #ifdef CONFIG_X86_64 extern void fpstate_free (struct fpu *fpu); +extern void xfd_update_state (struct fpstate *fpstate);#else static inline void fpstate_free (struct fpu *fpu) { } +static void xfd_update_state (struct fpstate *fpstate) { }#endif /* fpstate-related functions which are exported to KVM */
arch/x86/kernel/fpu/xstate.c +12 -0
@@ -1750,6 +1750,18 @@ int xfd_enable_guest_features(struct fpu_guest *guest_fpu) return __xfd_enable_feature(xfd_err, guest_fpu); } +void xfd_update_state (struct fpstate *fpstate) +{ +if (fpu_state_size_dynamic ()) { +u64 xfd = fpstate->xfd; + +if (__this_cpu_read (xfd_state) != xfd) { +wrmsrl (MSR_IA32_XFD , xfd); +__this_cpu_write (xfd_state, xfd); + } + } +} +EXPORT_SYMBOL_GPL (xfd_update_state);#else /* CONFIG_X86_64 */ static inline int xstate_request_perm (unsigned long idx,bool guest) {
arch/x86/kernel/fpu/xstate.h +1 -13
@@ -149,19 +149,7 @@ static inline void xfd_validate_state(struct fpstate *fpstate, u64 mask, bool rs#endif #ifdef CONFIG_X86_64 -static inline void xfd_update_state (struct fpstate *fpstate) -{ -if (fpu_state_size_dynamic ()) { -u64 xfd = fpstate->xfd; - -if (__this_cpu_read (xfd_state) != xfd) { -wrmsrl (MSR_IA32_XFD , xfd); -__this_cpu_write (xfd_state, xfd); - } - } -} -#else -static inline void xfd_update_state (struct fpstate *fpstate) { } +extern void xfd_update_state (struct fpstate *fpstate);#endif /*
Re: [PATCH 08/19] x86/fpu: Move xfd_update_state() to xstate.c and export symbol
Quoted 7 lines · Yang Zhong
From: Jing Liu <[email protected]>
xfd_update_state()is the interface to updateIA32_XFDand its per-cpu cache. All callers of this interface are currently in fpu core. KVM only indirectly triggersIA32_XFDupdate via a helper function (fpu_swap_kvm_fpstate()) when switching between user fpu and guest fpu.
Quoted 5 lines
Supporting AMX in guest now requires KVM to directly update
IA32_XFDwith the guest value (when emulating WRMSR) so XSAVE/XRSTOR can manage XSTATE components correctly inside guest.This patch moves
xfd_update_state()fromfpu/xstate.htofpu/xstate.c
s/This patch moves/Move/
please. See Documentation/process/submitting-patches.rst and search for 'This patch'
Quoted 15 lines
and export it for reference outside of fpu core.
Signed-off-by Jing Liu <[email protected]>Signed-off-by Yang Zhong <[email protected]>--- arch/x86/include/asm/fpu/api.h | 2 ++ arch/x86/kernel/fpu/xstate.c | 12 ++++++++++++ arch/x86/kernel/fpu/xstate.h | 14 +------------- 3 files changed, 15 insertions(+), 13 deletions(-)arch/x86/include/asm/fpu/api.h +0
-0
arch/x86/include/asm/fpu/api.h DECLARE_PER_CPU(struct fpu *, fpu_fpregs_owner_ctx); · line 137
Quoted 7 lines
arch/x86/include/asm/fpu/api.h +2
-0@@ -131,8 +131,10 @@ DECLARE_PER_CPU(struct fpu *, fpu_fpregs_owner_ctx);/* Process cleanup */ #ifdef CONFIG_X86_64 extern void fpstate_free (struct fpu *fpu); +extern void xfd_update_state (struct fpstate *fpstate);#else static inline void fpstate_free (struct fpu *fpu) { } +static void xfd_update_state (struct fpstate *fpstate) { }
Try a 32bit build to see the warnings this causes. That wants to be 'static inline void' obviously.
Quoted 5 lines
arch/x86/include/asm/fpu/api.h +0
-4#ifdef CONFIG_X86_64 -static inline void xfd_update_state (struct fpstate *fpstate) -{ -if (fpu_state_size_dynamic ()) { -u64 xfd = fpstate->xfd;
arch/x86/include/asm/fpu/api.h DECLARE_PER_CPU(struct fpu *, fpu_fpregs_owner_ctx);
Quoted 10 lines
arch/x86/include/asm/fpu/api.h +1
-9- -if (__this_cpu_read (xfd_state) != xfd) { -wrmsrl (MSR_IA32_XFD , xfd); -__this_cpu_write (xfd_state, xfd); - } - } -} -#else -static inline void xfd_update_state (struct fpstate *fpstate) { } +extern void xfd_update_state (struct fpstate *fpstate);
Why? It's already declared in the global header. So all of this has to be simply removed, no?
Thanks,
tglx
PATCH 9/19 kvm: x86: Prepare reallocation check
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
On native fpstate reallocation is triggered by #NM because IA32_XFD is initialized to 1 for all native tasks.
However #NM in guest is not trapped by KVM. Instead, guest enabling of a dynamic extended feature can be captured via emulation of IA32_XFD and XSETBV. Basically having guest XCR0[i]=1 and XFD[i]=0 indicates that the feature[i] is activated by the guest.
This patch provides a helper function for such check, invoked when either XCR0 or XFD is changed in the emulation path.
--- arch/x86/kvm/x86.c | 24 ++++++++++++++++++++++++ arch/x86/kvm/x86.h | 1 + 2 files changed, 25 insertions(+)
arch/x86/kvm/x86.c +24 -0
@@ -956,6 +956,30 @@ void kvm_load_host_xsave_state(struct kvm_vcpu *vcpu) }EXPORT_SYMBOL_GPL (kvm_load_host_xsave_state); +bool kvm_check_guest_realloc_fpstate (struct kvm_vcpu *vcpu,u64 xfd) +{ +u64 xcr0 = vcpu->arch.xcr0 &XFEATURE_MASK_USER_DYNAMIC ; + +/* For any state which is enabled dynamically */ +if ((xfd & xcr0) != xcr0) { +u64 request = (xcr0 ^ xfd) & xcr0; +struct fpu_guest *guest_fpu = &vcpu->arch.guest_fpu; + +/* +* If requested features haven't been enabled, update +* the request bitmap and tell the caller to request +* dynamic buffer reallocation. +*/ +if ((guest_fpu->user_xfeatures & request) != request) { + vcpu->arch.guest_fpu.realloc_request = request; +return true; + } + } + +return false; +} +EXPORT_SYMBOL_GPL (kvm_check_guest_realloc_fpstate); +static int __kvm_set_xcr (struct kvm_vcpu *vcpu,u32 index,u64 xcr) {u64 xcr0 = xcr;
arch/x86/kvm/x86.h +1 -0
@@ -445,6 +445,7 @@ static inline void kvm_machine_check(void)void kvm_load_guest_xsave_state (struct kvm_vcpu *vcpu);void kvm_load_host_xsave_state (struct kvm_vcpu *vcpu); +bool kvm_check_guest_realloc_fpstate (struct kvm_vcpu *vcpu,u64 new_xfd);int kvm_spec_ctrl_test_value (u64 value);bool kvm_is_valid_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4); int kvm_handle_memory_failure(struct kvm_vcpu *vcpu, int r,
Paolo Bonzini
maintainerRe: [PATCH 09/19] kvm: x86: Prepare reallocation check
Quoted 4 lines · Yang Zhong
arch/x86/kvm/x86.c +4
-0+u64 xcr0 = vcpu->arch.xcr0 &XFEATURE_MASK_USER_DYNAMIC ; + +/* For any state which is enabled dynamically */ +if ((xfd & xcr0) != xcr0) {
arch/x86/kvm/x86.c void kvm_load_host_xsave_state(struct kvm_vcpu *vcpu) · line 974
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+u64 request = (xcr0 ^ xfd) & xcr0; +struct fpu_guest *guest_fpu = &vcpu->arch.guest_fpu; + +/* +* If requested features haven't been enabled, update +* the request bitmap and tell the caller to request +* dynamic buffer reallocation. +*/ +if ((guest_fpu->user_xfeatures & request) != request) { + vcpu->arch.guest_fpu.realloc_request = request;
This should be "|=". If you have
wrmsr(XFD, dynamic-feature-1); ... wrmsr(XFD, dynamic-feature-2);
then the space for both features has to be allocated.
arch/x86/kvm/x86.c void kvm_load_host_xsave_state(struct kvm_vcpu *vcpu) · line 978
Quoted 4 lines
arch/x86/kvm/x86.c +4
-0+return true; + } + } +
This is just:
struct fpu_guest *guest_fpu = &vcpu->arch.guest_fpu; u64 xcr0 = vcpu->arch.xcr0 & XFEATURE_MASK_USER_DYNAMIC; u64 dynamic_enabled = xcr0 & ~xfd; if (!(dynamic_enabled & ~guest_fpu->user_xfeatures)) return false; /* * This should actually not be in guest_fpu, see review of * patch 2. Also see above regarding "=" vs "|=". */ vcpu->arch.guest_fpu.realloc_request |= dynamic_enabled; return true;
But without documentation I'm not sure why this exit-to-userspace step is needed:
- if (dynamic_enabled & ~guest_fpu->user_perm) != 0, then this is a userspace error and you can #GP the guest without any issue. Userspace is buggy
- if (dynamic_enabled & ~guest_fpu->user_xfeatures) != 0, but the feature *is* permitted, then it is okay to just go on and reallocate the guest FPU.
Paolo
RE: [PATCH 09/19] kvm: x86: Prepare reallocation check
Quoted 8 lines
From: Paolo Bonzini
Sent: Monday, December 13, 2021 5:16 PM- if (
dynamic_enabled& ~guest_fpu->user_perm) != 0, then this is a userspace error and you can #GP the guest without any issue. Userspace is buggy
Is it a general guideline that an error caused by emulation itself (e.g. due to no memory) can be reflected into the guest as #GP, even when from guest p.o.v there is nothing wrong with its setting?
Thanks
Kevin
Paolo Bonzini
maintainerRe: [PATCH 09/19] kvm: x86: Prepare reallocation check
Quoted 7 lines · Kevin
Quoted 3 lines
- if (
dynamic_enabled& ~guest_fpu->user_perm) != 0, then this is a userspace error and you can #GP the guest without any issue. Userspace is buggyIs it a general guideline that an error caused by emulation itself (e.g. due to no memory) can be reflected into the guest as #GP, even when from guest p.o.v there is nothing wrong with its setting?
No memory is a tricky one, if possible it should propagate -ENOMEM up to KVM_RUN or KVM_SET_MSR. But it's basically an impossible case anyway, because even with 8K TILEDATA we're within the limit of PAGE_ALLOC_COSTLY_ORDER.
So, since it's not easy to do it right now, we can look at it later.
Paolo
RE: [PATCH 09/19] kvm: x86: Prepare reallocation check
Quoted 10 lines · Paolo Bonzini
Quoted 7 lines · Kevin
Quoted 3 lines
- if (
dynamic_enabled& ~guest_fpu->user_perm) != 0, then this is a userspace error and you can #GP the guest without any issue. Userspace is buggyIs it a general guideline that an error caused by emulation itself (e.g. due to no memory) can be reflected into the guest as #GP, even when from guest p.o.v there is nothing wrong with its setting?
Quoted 6 lines
No memory is a tricky one, if possible it should propagate
-ENOMEMup toKVM_RUNorKVM_SET_MSR. But it's basically an impossible case anyway, because even with 8K TILEDATA we're within the limit ofPAGE_ALLOC_COSTLY_ORDER.So, since it's not easy to do it right now, we can look at it later.
For the way handling xcr0 and xfd ioctl failure, xcr0 and xfd have different handlings. Current KVM_SET_XCRS returns -EINVAL to userspace. KVM_SET_MSR is always allowed as the discussion in another thread.
So I'm thinking if reallocation failure in KVM_SET_XCRS and KVM_SET_MSR (may due to NOMEM or EPERM or ENOTSUPP), what is the way we would like to choose?
Thanks,
Jing
Quoted 1 line
Paolo
RE: [PATCH 09/19] kvm: x86: Prepare reallocation check
Quoted 21 lines
From: Liu, Jing2 <[email protected]>
Sent: Tuesday, December 14, 2021 10:42 PMQuoted 10 lines · Paolo Bonzini
Quoted 7 lines · Kevin
Quoted 3 lines
- if (
dynamic_enabled& ~guest_fpu->user_perm) != 0, then this is a userspace error and you can #GP the guest without any issue. Userspace is buggyIs it a general guideline that an error caused by emulation itself (e.g. due to no memory) can be reflected into the guest as #GP, even when from guest p.o.v there is nothing wrong with its setting?
Quoted 6 lines
No memory is a tricky one, if possible it should propagate
-ENOMEMup toKVM_RUNorKVM_SET_MSR. But it's basically an impossible case anyway, because even with 8K TILEDATA we're within the limit ofPAGE_ALLOC_COSTLY_ORDER.So, since it's not easy to do it right now, we can look at it later.
Quoted 9 lines
For the way handling xcr0 and xfd ioctl failure, xcr0 and xfd have different handlings. Current
KVM_SET_XCRSreturns-EINVALto userspace.KVM_SET_MSRis always allowed as the discussion in another thread.So I'm thinking if reallocation failure in
KVM_SET_XCRSandKVM_SET_MSR(may due to NOMEM or EPERM or ENOTSUPP), what is the way we would like to choose?
KVM_SET_MSRS can definitely accept failure according to msr_io(). I think Paolo's point is more about that the restore path should not inherit any check related to vCPU capability. It's a different matter if the error is caused by other host kernel errors.
Given that we don't need any special handling between the two scenarios (set by guest vs. set by host) in those emulation paths. Just return '1' to indicate error and whatever error policy exists in those scenarios is just applied.
Thanks
Kevin
PATCH 10/19 kvm: x86: Emulate WRMSR of guest IA32_XFD
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Intel's eXtended Feature Disable (XFD) feature allows the software to dynamically adjust fpstate buffer size for XSAVE features which have large state.
WRMSR to IA32_XFD is intercepted so if the written value enables a dynamic XSAVE feature the emulation code can exit to userspace to trigger fpstate reallocation for the state.
Introduce a new KVM exit reason (KVM_EXIT_FPU_REALLOC) for this purpose. If reallocation succeeds in fpu_swap_kvm_fpstate(), this exit just bounces to userspace and then back. Otherwise the userspace VMM should handle the error properly.
Use a new exit reason (instead of KVM_EXIT_X86_WRMSR) is clearer and can be shared between WRMSR(IA32_XFD) and XSETBV. This also avoids mixing with the userspace MSR machinery which is tied to KVM_EXIT_X86_WRMSR today.
Also introduce a new MSR return type (KVM_MSR_RET_USERSPACE). Currently MSR emulation returns to userspace only upon error or per certain filtering rules via the userspace MSR mechinary. This new return type indicates that emulation of certain MSR has its own specific reason to bounce to userspace.
IA32_XFD is updated in two ways:
- If reallocation is not required, the emulation code directly updates guest_fpu::xfd and then calls xfd_update_state() to update IA32_XFD and per-cpu cache;
- If reallocation is triggered, above updates are completed as
part of the fpstate reallocation process if succeeds;
RDMSR to IA32_XFD is not intercepted. fpu_swap_kvm_fpstate() ensures the guest XFD value loaded into MSR before re-entering the guest. Just save an unnecessary VM-exit here
--- arch/x86/kvm/vmx/vmx.c | 8 +++++++ arch/x86/kvm/x86.c | 48 ++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.h | 1 + include/uapi/linux/kvm.h | 1 + 4 files changed, 58 insertions(+)
arch/x86/kvm/vmx/vmx.c +8 -0
@@ -7141,6 +7141,11 @@ static void update_intel_pt_cfg(struct kvm_vcpu *vcpu) vmx->pt_desc.ctl_bitmask &= ~(0xfULL << (32 + i *4 )); } +static void vmx_update_intercept_xfd (struct kvm_vcpu *vcpu) +{ +vmx_set_intercept_for_msr (vcpu,MSR_IA32_XFD ,MSR_TYPE_R , false); +} +static void vmx_vcpu_after_set_cpuid (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);
@@ -7181,6 +7186,9 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) } } +if (cpu_feature_enabled (X86_FEATURE_XFD ) &&guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +vmx_update_intercept_xfd (vcpu); +set_cr4_guest_host_mask (vmx);vmx_write_encls_bitmap (vcpu,NULL );
arch/x86/kvm/x86.c +48 -0
60 lines not on this page · raw message
arch/x86/kvm/x86.h +1 -0
@@ -460,6 +460,7 @@ bool kvm_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u32 type);*/ #define KVM_MSR_RET_INVALID 2/* in-kernel MSR emulation #GP condition */ #define KVM_MSR_RET_FILTERED 3/* #GP due to userspace MSR filter */ +#define KVM_MSR_RET_USERSPACE 4/* Userspace handling */ #define __cr4_reserved_bits (__cpu_has, __c) \ ({ \
include/uapi/linux/kvm.h +1 -0
@@ -270,6 +270,7 @@ struct kvm_xen_exit {#define KVM_EXIT_X86_BUS_LOCK 33#define KVM_EXIT_XEN 34#define KVM_EXIT_RISCV_SBI 35 +#define KVM_EXIT_FPU_REALLOC 36/* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */
Paolo Bonzini
maintainerRe: [PATCH 10/19] kvm: x86: Emulate WRMSR of guest IA32_XFD
First, the MSR should be added to msrs_to_save_all and kvm_cpu_cap_has(X86_FEATURE_XFD) should be checked in kvm_init_msr_list.
It seems that RDMSR support is missing, too.
More important, please include:
- documentation for the new KVM_EXIT_* value
- a selftest that explains how userspace should react to it.
This is a strong requirement for any new API (the first has been for years; but the latter is also almost always respected these days). This series should not have been submitted without documentation.
Also:
arch/x86/kvm/x86.c int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 3708
Quoted 3 lines · Yang Zhong
arch/x86/kvm/x86.c +2
-0+if (!guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ;
This should allow msr->host_initiated always (even if XFD is not part of CPUID). However, if XFD is nonzero and kvm_check_guest_realloc_fpstate returns true, then it should return 1.
The selftest should also cover using KVM_GET_MSR/KVM_SET_MSR.
arch/x86/kvm/x86.c int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 3714
Quoted 5 lines
arch/x86/kvm/x86.c +5
-0+/* Setting unsupported bits causes #GP */ +if (~XFEATURE_MASK_USER_DYNAMIC & data) { +kvm_inject_gp (vcpu,0 ); +break ; + }
This should check
if (data & ~(XFEATURE_MASK_USER_DYNAMIC & vcpu->arch.guest_supported_xcr0))
instead.
Paolo
RE: [PATCH 10/19] kvm: x86: Emulate WRMSR of guest IA32_XFD
Quoted 10 lines · Paolo Bonzini
Also:
arch/x86/kvm/x86.c int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 3708
Quoted 3 lines · Yang Zhong
arch/x86/kvm/x86.c +2
-0+if (!guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ;This should allow
msr->host_initiatedalways (even if XFD is not part of CPUID).
Thanks Paolo.
msr->host_initiated handling would be added in next version.
I'd like to ask why always allow msr->host_initiated even if XFD is not part of CPUID, although guest doesn't care that MSR? We found some MSRs
(e.g. MSR_AMD64_OSVW_STATUS and MSR_AMD64_OSVW_ID_LENGTH )
are specially handled so would like to know the consideration of allowing msr->host_initiated.
if (!msr_info->host_initiated && !guest_cpuid_has(vcpu, X86_FEATURE_XFD))
return 1;
However, if XFD is nonzero and kvm_check_guest_realloc_fpstate
Quoted 2 lines
returns true, then it should return 1.
If XFD is nonzero, kvm_check_guest_realloc_fpstate() won't return true. So may not need this check here?
Thanks,
Jing
Quoted 2 lines
Paolo
Paolo Bonzini
maintainerRe: [PATCH 10/19] kvm: x86: Emulate WRMSR of guest IA32_XFD
Quoted 15 lines · On 12/13/21 08:51, Liu, Jing2 wrote:
Quoted 10 lines · Paolo Bonzini
Also:
arch/x86/kvm/x86.c int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 3708
Quoted 3 lines · Yang Zhong
arch/x86/kvm/x86.c +2
-0+if (!guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ;This should allow
msr->host_initiatedalways (even if XFD is not part of CPUID).Thanks Paolo.
msr->host_initiatedhandling would be added in next version.
Quoted 8 lines
I'd like to ask why always allow
msr->host_initiatedeven if XFD is not part of CPUID, although guest doesn't care that MSR? We found some MSRs
(e.g.MSR_AMD64_OSVW_STATUSandMSR_AMD64_OSVW_ID_LENGTH)
are specially handled so would like to know the consideration of allowingmsr->host_initiated.if (!msr_info->host_initiated && !guest_cpuid_has(vcpu,
X86_FEATURE_XFD))return 1;
Because it's simpler if userspace can just take the entire list from KVM_GET_MSR_INDEX_LIST and pass it to KVM_GET/SET_MSR. See for example vcpu_save_state and vcpu_load_state in tools/testing/selftests/kvm/lib/x86_64/processor.c.
Quoted 5 lines
Quoted 2 lines
However, if XFD is nonzero and
kvm_check_guest_realloc_fpstate
returns true, then it should return 1.If XFD is nonzero,
kvm_check_guest_realloc_fpstate()won't return true. So may not need this check here?
It can't for now, because there's a single dynamic feature, but here:
+ if ((xfd & xcr0) != xcr0) {
+ u64 request = (xcr0 ^ xfd) & xcr0;
+ struct fpu_guest *guest_fpu = &vcpu->arch.guest_fpu;
+
+ /*
+ * If requested features haven't been enabled, update
+ * the request bitmap and tell the caller to request
+ * dynamic buffer reallocation.
+ */
+ if ((guest_fpu->user_xfeatures & request) != request) {
+ vcpu->arch.guest_fpu.realloc_request = request;
+ return true;
+ }
+ }
it is certainly possible to return true with nonzero XFD.
Paolo
Re: [PATCH 10/19] kvm: x86: Emulate WRMSR of guest IA32_XFD
In reply to Paolo BonziniQuoted 28 lines · Paolo Bonzini
First, the MSR should be added to
msrs_to_save_allandkvm_cpu_cap_has(X86_FEATURE_XFD)should be checked inkvm_init_msr_list.It seems that RDMSR support is missing, too.
More important, please include:
- documentation for the new KVM_EXIT_* value
- a selftest that explains how userspace should react to it.
This is a strong requirement for any new API (the first has been for years; but the latter is also almost always respected these days). This series should not have been submitted without documentation.
Also:
arch/x86/kvm/x86.c int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 3708
Quoted 3 lines · Yang Zhong
arch/x86/kvm/x86.c +2
-0+if (!guest_cpuid_has (vcpu,X86_FEATURE_XFD )) +return 1 ;This should allow
msr->host_initiatedalways (even if XFD is not part of CPUID). However, if XFD is nonzero andkvm_check_guest_realloc_fpstatereturns true, then it should return 1.
Quoted 2 lines
The selftest should also cover using KVM_GET_MSR/KVM_SET_MSR.
Paolo, Seems we do not need new KVM_EXIT_* again from below thomas' new patchset:
git://git.kernel.org/pub/scm/linux/kernel/git/people/tglx/devel.git x86/fpu-kvm
So the selftest stll need support KVM_GET_MSR/KVM_SET_MSR for MSR_IA32_XFD and MSR_IA32_XFD_ERR? If yes, we only do some read/write test with vcpu_set_msr()/ vcpu_get_msr() from new selftest tool? or do wrmsr from guest side and check this value from selftest side?
I checked some msr selftest reference code, tsc_msrs_test.c, which maybe better for this reference. If you have better suggestion, please share it to me. thanks!
Yang
Paolo Bonzini
maintainerRe: [PATCH 10/19] kvm: x86: Emulate WRMSR of guest IA32_XFD
Quoted 7 lines · Yang Zhong
Paolo, Seems we do not need new KVM_EXIT_* again from below thomas' new patchset:
git://git.kernel.org/pub/scm/linux/kernel/git/people/tglx/devel.gitx86/fpu-kvmSo the selftest stll need support KVM_GET_MSR/KVM_SET_MSR for
MSR_IA32_XFDandMSR_IA32_XFD_ERR? If yes, we only do some read/write test with vcpu_set_msr()/vcpu_get_msr()from new selftest tool? or do wrmsr from guest side and check this value from selftest side?
You can write a test similar to state_test.c to cover XCR0, XFD and the new XSAVE extensions. The test can:
- initialize AMX and write a nonzero value to XFD
- load a matrix into TMM0
- check that #NM is delivered (search for vm_install_exception_handler) and that XFD_ERR is correct
- write 0 to XFD
- load again the matrix, and check that #NM is not delivered
- store it back into memory
- compare it with the original data
All of this can be done with a full save&restore after every step (though I suggest that you first get it working without save&restore, the relevant code in state_test.c is easy to identify and comment out).
You will have to modify vcpu_load_state, so that it does first KVM_SET_MSRS, then KVM_SET_XCRS, then KVM_SET_XSAVE. See patch below.
Paolo
Quoted 2 lines
I checked some msr selftest reference code,
tsc_msrs_test.c, which maybe better for this reference. If you have better suggestion, please share it to me. thanks!
------------------ 8< -----------------
From: Paolo Bonzini <[email protected]>
Subject: [PATCH] selftest: kvm: Reorder vcpu_load_state steps for AMX
For AMX support it is recommended to load XCR0 after XFD, so that KVM does not see XFD=0, XCR=1 for a save state that will eventually be disabled (which would lead to premature allocation of the space required for that save state).
It is also required to load XSAVE data after XCR0 and XFD, so that KVM can trigger allocation of the extra space required to store AMX state.
Adjust vcpu_load_state to obey these new requirements.
tools/testing/selftests/kvm/lib/x86_64/processor.c +10 -10
@@ -1157,16 +1157,6 @@ void vcpu_load_state(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_x86_state *sstruct vcpu *vcpu =vcpu_find (vm, vcpuid);int r; - r =ioctl (vcpu->fd,KVM_SET_XSAVE , &state->xsave); -TEST_ASSERT (r ==0 ,"Unexpected result from KVM_SET_XSAVE, r: %i" , - r); - -if (kvm_check_cap (KVM_CAP_XCRS )) { - r =ioctl (vcpu->fd,KVM_SET_XCRS , &state->xcrs); -TEST_ASSERT (r ==0 ,"Unexpected result from KVM_SET_XCRS, r: %i" , - r); - } - r =ioctl (vcpu->fd,KVM_SET_SREGS , &state->sregs);TEST_ASSERT (r ==0 ,"Unexpected result from KVM_SET_SREGS, r: %i" , r);
@@ -1175,6 +1165,16 @@ void vcpu_load_state(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_x86_state *sTEST_ASSERT (r == state->msrs.nmsrs,"Unexpected result from KVM_SET_MSRS, r: %i (failed at %x)" , r, r == state->msrs.nmsrs ?-1 : state->msrs.entries[r].index); +if (kvm_check_cap (KVM_CAP_XCRS )) { + r =ioctl (vcpu->fd,KVM_SET_XCRS , &state->xcrs); +TEST_ASSERT (r ==0 ,"Unexpected result from KVM_SET_XCRS, r: %i" , + r); + } + + r =ioctl (vcpu->fd,KVM_SET_XSAVE , &state->xsave); +TEST_ASSERT (r ==0 ,"Unexpected result from KVM_SET_XSAVE, r: %i" , + r); + r =ioctl (vcpu->fd,KVM_SET_VCPU_EVENTS , &state->events);TEST_ASSERT (r ==0 ,"Unexpected result from KVM_SET_VCPU_EVENTS, r: %i" , r);
Re: [PATCH 10/19] kvm: x86: Emulate WRMSR of guest IA32_XFD
In reply to Yang ZhongReview
Quoted 12 lines · Yang Zhong
(quoted diff) +12
-0+ + /* + * Update IA32_XFD to the guest value so #NM can be + * raised properly in the guest. Instead of directly + * writing the MSR, call a helper to avoid breaking + * per-cpu cached value in fpu core. + */ + fpregs_lock(); + current->thread.fpu.fpstate->xfd = data; + xfd_update_state(current->thread.fpu.fpstate); + fpregs_unlock(); + break;
Now looking at the actual callsite the previous patch really should be something like the below. Why?
It preserves the inline which allows the compiler to generate better code in the other hotpathes and it keeps the FPU internals to the core code. Hmm?
Thanks,
tglx
arch/x86/include/asm/fpu/api.h +2 -0
@@ -125,8 +125,10 @@ DECLARE_PER_CPU(struct fpu *, fpu_fpregs/* Process cleanup */ #ifdef CONFIG_X86_64 extern void fpstate_free (struct fpu *fpu); +extern void fpu_update_xfd_state (u64 xfd);#else static inline void fpstate_free (struct fpu *fpu) { } +static inline void fpu_update_xfd_state (u64 xfd) { }#endif /* fpstate-related functions which are exported to KVM */
arch/x86/kernel/fpu/core.c +13 -0
@@ -322,6 +322,19 @@ int fpu_swap_kvm_fpstate(struct fpu_gues }EXPORT_SYMBOL_GPL (fpu_swap_kvm_fpstate); +#ifdef CONFIG_X86_64 +void fpu_update_xfd_state (u64 xfd) +{ +struct fpstate *fps = current->thread.fpu.fpstate; + +fpregs_lock (); + fps->xfd = xfd; +xfd_update_state (fps); +fpregs_unlock (); +} +EXPORT_SYMBOL_GPL (fpu_update_xfd_state); +#endif +void fpu_copy_guest_fpstate_to_uabi (struct fpu_guest *gfpu,void *buf,unsigned int size,u32 pkru) {
Paolo Bonzini
maintainerRe: [PATCH 10/19] kvm: x86: Emulate WRMSR of guest IA32_XFD
In reply to Yang Zhongarch/x86/kvm/x86.c int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 3738
Quoted 8 lines · Yang Zhong
arch/x86/kvm/x86.c +8
-0+/* +* Update IA32_XFD to the guest value so #NM can be +* raised properly in the guest. Instead of directly +* writing the MSR, call a helper to avoid breaking +* per-cpu cached value in fpu core. +*/ +fpregs_lock (); + current->thread.fpu.fpstate->xfd = data;
This is wrong, it should be written in vcpu->arch.guest_fpu.
arch/x86/kvm/x86.c int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 3739
Quoted 1 line
arch/x86/kvm/x86.c +1
-0+xfd_update_state (current->thread.fpu.fpstate);
This is okay though, so that KVM_SET_MSR will not write XFD and WRMSR will.
That said, I think xfd_update_state should not have an argument. current->thread.fpu.fpstate->xfd is the only fpstate that should be synced with the xfd_state per-CPU variable.
Paolo
Quoted 1 line
arch/x86/kvm/x86.c +1
-0+fpregs_unlock ();
Re: [PATCH 10/19] kvm: x86: Emulate WRMSR of guest IA32_XFD
Quoted 12 lines · Paolo Bonzini
arch/x86/kvm/x86.c int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 3738
Quoted 8 lines · Yang Zhong
arch/x86/kvm/x86.c +8
-0+/* +* Update IA32_XFD to the guest value so #NM can be +* raised properly in the guest. Instead of directly +* writing the MSR, call a helper to avoid breaking +* per-cpu cached value in fpu core. +*/ +fpregs_lock (); + current->thread.fpu.fpstate->xfd = data;This is wrong, it should be written in
vcpu->arch.guest_fpu.
Quoted 8 lines
arch/x86/kvm/x86.c int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 3739
Quoted 1 line
arch/x86/kvm/x86.c +1
-0+xfd_update_state (current->thread.fpu.fpstate);This is okay though, so that
KVM_SET_MSRwill not write XFD and WRMSR will.That said, I think
xfd_update_stateshould not have an argument.current->thread.fpu.fpstate->xfdis the only fpstate that should be synced with thexfd_stateper-CPU variable.
I'm looking into this right now. The whole restore versus runtime thing needs to be handled differently.
Thanks,
tglx
Re: [PATCH 10/19] kvm: x86: Emulate WRMSR of guest IA32_XFD
Paolo,
Quoted 7 lines · Thomas Gleixner
Quoted 3 lines · Paolo Bonzini
That said, I think
xfd_update_stateshould not have an argument.current->thread.fpu.fpstate->xfdis the only fpstate that should be synced with thexfd_stateper-CPU variable.I'm looking into this right now. The whole restore versus runtime thing needs to be handled differently.
We need to look at different things here:
1) XFD MSR write emulation
2) XFD MSR synchronization when write emulation is disabled
3) Guest restore
#1 and #2 are in the context of vcpu_run() and
vcpu->arch.guest_fpu.fpstate == current->thread.fpu.fpstate
while #3 has:
vcpu->arch.guest_fpu.fpstate != current->thread.fpu.fpstate
#2 is only updating fpstate->xfd and the per CPU shadow.
So the state synchronization wants to be something like this:
void fpu_sync_guest_xfd_state(void)
{
struct fpstate *fps = current->thread.fpu.fpstate;
lockdep_assert_irqs_disabled();
if (fpu_state_size_dynamic()) {
rdmsrl(MSR_IA32_XFD, fps->xfd);
__this_cpu_write(xfd_state, fps->xfd);
}
}EXPORT_SYMBOL_GPL(fpu_sync_guest_xfd_state);
No wrmsrl() because the MSR is already up do date. The important part is that fpstate->xfd and the shadow state are updated so that after reenabling preemption the context switch FPU logic works correctly.
#1 and #3 can trigger a reallocation of guest_fpu.fpstate and can fail. But this is also true for XSETBV emulation and XCR0 restore.
For #1 modifying fps->xfd in the KVM code before calling into the FPU code is just _wrong_ because if the guest removes the XFD restriction then it must be ensured that the buffer is sized correctly _before_ this is updated.
For #3 it's not really important, but I still try to wrap my head around the whole picture vs. XCR0.
There are two options:
1) Require strict ordering of XFD and XCR0 update to avoid pointless buffer expansion, i.e. XFD before XCR0.
Because if XCR0 is updated while guest_fpu->fpstate.xfd is still in init state (0) and XCR0 contains extended features, then the buffer would be expanded because XFD does not mask the extended features out. When XFD is restored with a non-zero value, it's too late already.
2) Ignore buffer expansion up to the point where XSTATE restore happens
and evaluate guest XCR0 and guest_fpu->fpstate.xfd there.
I'm leaning towards #1 because that means we have exactly _ONE_ place where we need to deal with buffer expansion. If Qemu gets the ordering wrong it wastes memory per vCPU, *shrug*.
Thanks,
tglx
RE: [PATCH 10/19] kvm: x86: Emulate WRMSR of guest IA32_XFD
Hi, Thomas,
Quoted 14 lines
From: Thomas Gleixner <[email protected]>
Sent: Tuesday, December 14, 2021 5:23 AMPaolo,
Quoted 7 lines · Thomas Gleixner
Quoted 3 lines · Paolo Bonzini
That said, I think
xfd_update_stateshould not have an argument.current->thread.fpu.fpstate->xfdis the only fpstate that should be synced with thexfd_stateper-CPU variable.I'm looking into this right now. The whole restore versus runtime thing needs to be handled differently.
After looking at your series, I think it missed Paolo's comment about changing xfd_update_state() to accept no argument.
Thanks
Kevin
PATCH 11/19 kvm: x86: Check fpstate reallocation in XSETBV emulation
In reply to Yang ZhongXSETBV allows the software to write the extended control register XCR0, thus its emulation handler also needs to check fpstate reallocation when the changed XCR0 value enables certain dynamically-enabled features.
--- arch/x86/kvm/x86.c | 9 +++++++++ 1 file changed, 9 insertions(+)
arch/x86/kvm/x86.c +9 -0
@@ -1028,6 +1028,15 @@ int kvm_emulate_xsetbv(struct kvm_vcpu *vcpu)return 1 ; } +if (guest_cpuid_has (vcpu,X86_FEATURE_XFD )) { +if (kvm_check_guest_realloc_fpstate (vcpu, vcpu->arch.guest_fpu.fpstate->xfd)) { + vcpu->run->exit_reason =KVM_EXIT_FPU_REALLOC ; + vcpu->arch.complete_userspace_io = + kvm_skip_emulated_instruction; +return 0 ; + } + } +return kvm_skip_emulated_instruction (vcpu); }EXPORT_SYMBOL_GPL (kvm_emulate_xsetbv);
PATCH 12/19 x86/fpu: Prepare KVM for bringing XFD state back in-sync
In reply to Yang ZhongFrom: Thomas Gleixner <[email protected]>
Guest may toggle IA32_XFD in high frequency as it is part of the fpstate information (features, sizes, xfd) and swapped in task context switch.
To minimize the trap overhead of writes to this MSR, one optimization is to allow guest direct write thus eliminate traps. However MSR passthrough implies that guest_fpstate::xfd and per-cpu xfd cache might be out of sync with the current IA32_XFD value by the guest.
This suggests KVM needs to re-sync guest_fpstate::xfd and per-cpu cache with IA32_XFD before the vCPU thread might be preempted or interrupted.
This patch provides a helper function for the re-sync purpose.
---
(To Thomas): the original name kvm_update_guest_xfd_state() in your sample code is renamed to xfd_sync_state() in this patch. In concept it is a general helper to bring software values in-sync with the MSR value after they become out-of-sync. KVM is just the first out-of-sync usage on this helper, so a neutral name may make more sense. But if you prefer to the original name we can also change back.
arch/x86/include/asm/fpu/xstate.h | 2 ++ arch/x86/kernel/fpu/xstate.c | 14 ++++++++++++++ 2 files changed, 16 insertions(+)
arch/x86/include/asm/fpu/xstate.h +2 -0
@@ -129,4 +129,6 @@ static __always_inline __pure bool fpu_state_size_dynamic(void) }#endif +extern void xfd_sync_state (void ); +#endif
arch/x86/kernel/fpu/xstate.c +14 -0
@@ -1762,11 +1762,25 @@ void xfd_update_state(struct fpstate *fpstate) } }EXPORT_SYMBOL_GPL (xfd_update_state); + +/* Bring software state in sync with the current MSR value */ +void xfd_sync_state (void ) +{ +if (fpu_state_size_dynamic ()) { +u64 xfd; + +rdmsrl (MSR_IA32_XFD , xfd); + current->thread.fpu.fpstate->xfd = xfd; +__this_cpu_write (xfd_state, xfd); + } +} +EXPORT_SYMBOL_GPL (xfd_sync_state);#else /* CONFIG_X86_64 */ static inline int xstate_request_perm (unsigned long idx,bool guest) {return -EPERM ; } +void xfd_sync_state (void ) {}#endif /* !CONFIG_X86_64 */ inline u64 xstate_get_guest_group_perm (void )
Re: [PATCH 12/19] x86/fpu: Prepare KVM for bringing XFD state back in-sync
Quoted 10 lines · Yang Zhong
From: Thomas Gleixner <[email protected]>
Guest may toggle
IA32_XFDin high frequency as it is part of the fpstate information (features, sizes, xfd) and swapped in task context switch.To minimize the trap overhead of writes to this MSR, one optimization is to allow guest direct write thus eliminate traps. However MSR passthrough implies that
guest_fpstate::xfdand per-cpu xfd cache might be out of sync with the currentIA32_XFDvalue by the guest.
Quoted 4 lines
This suggests KVM needs to re-sync
guest_fpstate::xfdand per-cpu cache withIA32_XFDbefore the vCPU thread might be preempted or interrupted.This patch provides a helper function for the re-sync purpose.
Provide a ....
Quoted 11 lines
Signed-off-by Thomas Gleixner <[email protected]>Signed-off-by Jing Liu <[email protected]>Signed-off-by Yang Zhong <[email protected]>---
(To Thomas): the original namekvm_update_guest_xfd_state()in your sample code is renamed toxfd_sync_state()in this patch. In concept it is a general helper to bring software values in-sync with the MSR value after they become out-of-sync. KVM is just the first out-of-sync usage on this helper, so a neutral name may make more sense. But if you prefer to the original name we can also change back.
There is no need for a general helper, really.
It's KVM specific and should go into KVM section in core.c next to the other thing vs. the XFD update.
Thanks,
tglx
PATCH 13/19 kvm: x86: Disable WRMSR interception for IA32_XFD on demand
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Always intercepting IA32_XFD causes non-negligible overhead when this register is updated frequently in the guest.
Disable WRMSR interception to IA32_XFD after fpstate reallocation is completed. There are three options for when to disable the interception:
1) When emulating the 1st WRMSR which requires reallocation, disable interception before exiting to userapce with the assumption that the userspace VMM should not bounch back to the kernel if reallocation fails. However it's not good to design kernel based on application behavior. If due to bug the vCPU thread comes back to the kernel after reallocation fails, XFD passthrough may lead to host memory corruption when doing XSAVES for guest fpstate which has a smaller size than what guest XFD allows.
2) Disable interception when coming back from the userspace VMM (for the 1st WRMSR which triggers reallocation). Re-check whether fpstate size can serve the new guest XFD value. Disable interception only when the check succeeds. This requires KVM to store guest XFD value in some place and then compare it to guest_fpu::user_xfeatures in the completion handler.
3) Disable interception at the 2nd WRMSR which enables dynamic XSTATE features. If guest_fpu::user_xfeatures already includes bits for dynamic features set in guest XFD value, disable interception.
Currently 3) is implemented, with a flow like below:
(G) WRMSR(IA32_XFD) which enables AMX for the FIRST time --trap to host--
(HK) Emulate WRMSR and find fpstate size too small (HK) Reallocate fpstate
--exit to userspace--
(HU) do nothing
--back to kernel via kvm_run--
(HK) complete WRMSR emulation
--enter guest--
(G) do something
(G) WRMSR(IA32_XFD) which disables AMX
--trap to host--
(HK) Emulate WRMSR and disable AMX in IA32_XFD --enter guest--
(G) do something
(G) WRMSR(IA32_XFD) which enables AMX for the SECOND time --trap to host--
(HK) Emulate WRMSR and find fpstate size sufficient for AMX (HK) Disable WRMSR interception for IA32_XFD --enter guest--
(G) WRMSR(IA32_XFD)
(G) WRMSR(IA32_XFD)
(G) WRMSR(IA32_XFD)
...
After disabling WRMSR interception, the guest directly updates IA32_XFD which becomes out-of-sync with the host-side software state (guest_fpstate::xfd and per-cpu xfd cache). This requires KVM to call xfd_sync_state() to bring the software state in sync with IA32_XFD register after VM-exit (before preemption happens or exiting to userspace).
p.s. We have confirmed that SDM is being revised to say that when setting IA32_XFD[18] the AMX register state is not guaranteed to be preserved. This clarification avoids adding mess for a creative guest which sets IA32_XFD[18]=1 before saving active AMX state to its own storage.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/vmx.c | 10 ++++++++++ arch/x86/kvm/vmx/vmx.h | 2 +- arch/x86/kvm/x86.c | 7 +++++++ 5 files changed, 21 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -30,6 +30,7 @@ KVM_X86_OP(update_exception_bitmap) KVM_X86_OP(get_msr) KVM_X86_OP(set_msr) KVM_X86_OP(get_segment_base) +KVM_X86_OP_NULL(set_xfd_passthrough) KVM_X86_OP(get_segment) KVM_X86_OP(get_cpl) KVM_X86_OP(set_segment)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -640,6 +640,7 @@ struct kvm_vcpu_arch {u64 smi_count;bool tpr_access_reporting;bool xsaves_enabled; +bool xfd_out_of_sync;u64 ia32_xss;u64 microcode_version;u64 arch_capabilities;
@@ -1328,6 +1329,7 @@ struct kvm_x86_ops {void (*update_exception_bitmap)(struct kvm_vcpu *vcpu);int (*get_msr)(struct kvm_vcpu *vcpu,struct msr_data *msr);int (*set_msr)(struct kvm_vcpu *vcpu,struct msr_data *msr); +void (*set_xfd_passthrough)(struct kvm_vcpu *vcpu);u64 (*get_segment_base)(struct kvm_vcpu *vcpu,int seg);void (*get_segment)(struct kvm_vcpu *vcpu,struct kvm_segment *var,int seg);
arch/x86/kvm/vmx/vmx.c +10 -0
@@ -160,6 +160,7 @@ static u32 vmx_possible_passthrough_msrs[MAX_POSSIBLE_PASSTHROUGH_MSRS] = {MSR_FS_BASE ,MSR_GS_BASE ,MSR_KERNEL_GS_BASE , +MSR_IA32_XFD , #endifMSR_IA32_SYSENTER_CS , MSR_IA32_SYSENTER_ESP,
@@ -1924,6 +1925,14 @@ static u64 vcpu_supported_debugctl(struct kvm_vcpu *vcpu)return debugctl; } +#ifdef CONFIG_X86_64 +static void vmx_set_xfd_passthrough (struct kvm_vcpu *vcpu) +{ +vmx_disable_intercept_for_msr (vcpu,MSR_IA32_XFD ,MSR_TYPE_RW ); + vcpu->arch.xfd_out_of_sync = true; +} +#endif +/* * Writes msr value into the appropriate "register". * Returns 0 on success, non-0 otherwise.
@@ -7657,6 +7666,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = {#ifdef CONFIG_X86_64 .set_hv_timer = vmx_set_hv_timer, .cancel_hv_timer = vmx_cancel_hv_timer, + .set_xfd_passthrough = vmx_set_xfd_passthrough, #endif .setup_mce = vmx_setup_mce,
arch/x86/kvm/vmx/vmx.h +1 -1
@@ -340,7 +340,7 @@ struct vcpu_vmx {struct lbr_desc lbr_desc;/* Save desired MSR intercept (read: pass-through) state */ -#define MAX_POSSIBLE_PASSTHROUGH_MSRS 13 +#define MAX_POSSIBLE_PASSTHROUGH_MSRS 14struct { DECLARE_BITMAP(read, MAX_POSSIBLE_PASSTHROUGH_MSRS); DECLARE_BITMAP(write, MAX_POSSIBLE_PASSTHROUGH_MSRS);
arch/x86/kvm/x86.c +7 -0
@@ -974,6 +974,10 @@ bool kvm_check_guest_realloc_fpstate(struct kvm_vcpu *vcpu, u64 xfd) vcpu->arch.guest_fpu.realloc_request = request;return true; } + +/* Disable WRMSR interception if possible */ +if (kvm_x86_ops.set_xfd_passthrough) +static_call (kvm_x86_set_xfd_passthrough)(vcpu); }return false;
@@ -10002,6 +10006,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (hw_breakpoint_active ())hw_breakpoint_restore (); +if (vcpu->arch.xfd_out_of_sync) +xfd_sync_state (); + vcpu->arch.last_vmentry_cpu = vcpu->cpu; vcpu->arch.last_guest_tsc =kvm_read_l1_tsc (vcpu,rdtsc ());
RE: [PATCH 13/19] kvm: x86: Disable WRMSR interception for IA32_XFD on demand
Quoted 19 lines · Yang Zhong
From: Jing Liu <[email protected]>
Always intercepting
IA32_XFDcauses non-negligible overhead when this register is updated frequently in the guest.Disable WRMSR interception to
IA32_XFDafter fpstate reallocation is completed. There are three options for when to disable the interception:1) When emulating the 1st WRMSR which requires reallocation, disable interception before exiting to userapce with the assumption that the userspace VMM should not bounch back to the kernel if reallocation fails. However it's not good to design kernel based on application behavior. If due to bug the vCPU thread comes back to the kernel after reallocation fails, XFD passthrough may lead to host memory corruption when doing XSAVES for guest fpstate which has a smaller size than what guest XFD allows.
Quoted 6 lines
2) Disable interception when coming back from the userspace VMM (for the 1st WRMSR which triggers reallocation). Re-check whether fpstate size can serve the new guest XFD value. Disable interception only when the check succeeds. This requires KVM to store guest XFD value in some place and then compare it to
guest_fpu::user_xfeaturesin the completion handler.
For option 2), we are considering that fpstate->size can be used to indicate if reallocation is successful. Because once one of the XFD features (today, it's AMX) is enabled, kernel need reallocate full size, otherwise, KVM has no chance to reallocate for other XFD features later since it's non-trapped (to avoid WRMSR VM EXITs due to guest toggling XFD).
Then KVM doesn't need to store guest XFD value in some place. And kernel fpu core may need an API to tell guest permitted size for KVM.
Thanks,
Jing
Quoted 6 lines
3) Disable interception at the 2nd WRMSR which enables dynamic XSTATE features. If
guest_fpu::user_xfeaturesalready includes bits for dynamic features set in guest XFD value, disable interception.
PATCH 14/19 x86/fpu: Prepare for KVM XFD_ERR handling
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
When XFD causes an instruction to generate #NM, IA32_XFD_ERR contains information about which disabled state components are being accessed. The #NM handler is expected to check this information and then enable the state components by clearing IA32_XFD for the faulting task (if having permission).
if the XFD_ERR value generated in guest is consumed/clobbered by the host before the guest itself doing so. This may lead to non-XFD-related #NM treated as XFD #NM in host (due to non-zero value in XFD_ERR), or XFD-related #NM treated as non-XFD #NM in guest (XFD_ERR cleared by the host #NM handler).
This patch provides two helpers to swap the guest XFD_ERR and host XFD_ERR. Where to call them in KVM will be discussed thoroughly in next patch.
The guest XFD_ERR value is saved in fpu_guest::xfd_err. There is no need to save host XFD_ERR because it's always cleared to ZERO by the host #NM handler (which cannot be preempted by a vCPU thread to observe a non-zero value).
The lower two bits in fpu_guest::xfd_err is borrowed for special purposes. The state components (FP and SSE) covered by the two bits are not XSAVE-enabled feature, thus not XFD-enabled either. It's impossible to see hardware setting them in XFD_ERR:
- XFD_ERR_GUEST_DISABLED (bit 0)
Indicate that XFD extension is not exposed to the guest thus no need to save/restore it.
- XFD_ERR_GUEST_SAVED (bit 1)
Indicate fpu_guest::xfd_err already contains a saved value thus no need for duplicated saving (e.g. when the vCPU thread is preempted multiple times before re-enter the guest).
--- arch/x86/include/asm/fpu/api.h | 8 ++++++ arch/x86/include/asm/fpu/types.h | 24 ++++++++++++++++ arch/x86/kernel/fpu/core.c | 49 ++++++++++++++++++++++++++++++++ 3 files changed, 81 insertions(+)
arch/x86/include/asm/fpu/api.h +8 -0
@@ -147,6 +147,14 @@ extern bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu);extern void fpu_free_guest_fpstate (struct fpu_guest *gfpu);extern int fpu_swap_kvm_fpstate (struct fpu_guest *gfpu,bool enter_guest); +#ifdef CONFIG_X86_64 +extern void fpu_save_guest_xfd_err (struct fpu_guest *guest_fpu); +extern void fpu_restore_guest_xfd_err (struct fpu_guest *guest_fpu); +#else +static inline void fpu_save_guest_xfd_err (struct fpu_guest *guest_fpu) { } +static inline void fpu_restore_guest_xfd_err (struct fpu_guest *guest_fpu) { } +#endif +extern void fpu_copy_guest_fpstate_to_uabi (struct fpu_guest *gfpu,void *buf,unsigned int size,u32 pkru);extern int fpu_copy_uabi_to_guest_fpstate (struct fpu_guest *gfpu,const void *buf,u64 xcr0,u32 *vpkru);
arch/x86/include/asm/fpu/types.h +24 -0
@@ -500,6 +500,22 @@ struct fpu {*/ }; +/* +* Use @xfd_err:bit0 to indicate whether guest XFD_ERR should be +* saved/restored. The x87 state covered by bit 0 is not a +* XSAVE-enabled feature, thus is not XFD-enabled either (won't +* occur in XFD_ERR). +*/ +#define XFD_ERR_GUEST_DISABLED (1 << XFEATURE_FP) + +/* +* Use @xfd_err:bit1 to indicate the validity of @xfd_err. Used to +* avoid duplicated savings in case the vCPU is preempted multiple +* times before it re-enters the guest. The SSE state covered by +* bit 1 is neither XSAVE-enabled nor XFD-enabled. +*/ +#define XFD_ERR_GUEST_SAVED (1 << XFEATURE_SSE) +/* * Guest pseudo FPU container */
@@ -527,6 +543,14 @@ struct fpu_guest {*/ u64 realloc_request; +/* +* @xfd_err: save the guest value. bit 0 and bit1 +* have special meaning to indicate the +* requirement of saving and the validity +* of the saved value. +*/ +u64 xfd_err; +/* * @fpstate: Pointer to the allocated guest fpstate */
arch/x86/kernel/fpu/core.c +49 -0
55 lines not on this page · raw message
Paolo Bonzini
maintainerRe: [PATCH 14/19] x86/fpu: Prepare for KVM XFD_ERR handling
Quoted 11 lines · Yang Zhong
The guest
XFD_ERRvalue is saved infpu_guest::xfd_err. There is no need to save hostXFD_ERRbecause it's always cleared to ZERO by the host #NM handler (which cannot be preempted by a vCPU thread to observe a non-zero value).The lower two bits in
fpu_guest::xfd_erris borrowed for special purposes. The state components (FP and SSE) covered by the two bits are not XSAVE-enabled feature, thus not XFD-enabled either. It's impossible to see hardware setting them inXFD_ERR:
Quoted 4 lines
-
XFD_ERR_GUEST_DISABLED(bit 0)Indicate that XFD extension is not exposed to the guest thus no need to save/restore it.
Can this instead just check if xfeatures includes any dynamic-save features?
Paolo
Quoted 5 lines
-
XFD_ERR_GUEST_SAVED(bit 1)Indicate
fpu_guest::xfd_erralready contains a saved value thus no need for duplicated saving (e.g. when the vCPU thread is preempted multiple times before re-enter the guest).
Re: [PATCH 14/19] x86/fpu: Prepare for KVM XFD_ERR handling
In reply to Yang ZhongQuoted 8 lines · Yang Zhong
arch/x86/kernel/fpu/core.c +2
-0@@ -322,6 +322,55 @@ int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest) }EXPORT_SYMBOL_GPL (fpu_swap_kvm_fpstate); +#ifdef CONFIG_X86_64 +void fpu_save_guest_xfd_err (struct fpu_guest *guest_fpu)
arch/x86/kernel/fpu/core.c int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest) · line 336
Quoted 10 lines
arch/x86/kernel/fpu/core.c +10
-0+{ +if (guest_fpu->xfd_err &XFD_ERR_GUEST_DISABLED ) +return ; + +/* A non-zero value indicates guest XFD_ERR already saved */ +if (guest_fpu->xfd_err) +return ; + +/* Guest XFD_ERR must be saved before switching to host fpstate */ +WARN_ON_ONCE (!current->thread.fpu.fpstate->is_guest);
Warn and proceed?
Quoted 23 lines
arch/x86/kernel/fpu/core.c +23
-0+rdmsrl (MSR_IA32_XFD_ERR , guest_fpu->xfd_err); + +/* +* Restore to the host value if guest xfd_err is non-zero. +* Except in #NM handler, all other places in the kernel +* should just see xfd_err=0. So just restore to 0. +*/ +if (guest_fpu->xfd_err) +wrmsrl (MSR_IA32_XFD_ERR ,0 ); + + guest_fpu->xfd_err |=XFD_ERR_GUEST_SAVED ; +} +EXPORT_SYMBOL_GPL (fpu_save_guest_xfd_err); + +void fpu_restore_guest_xfd_err (struct fpu_guest *guest_fpu) +{ +u64 xfd_err = guest_fpu->xfd_err; + +if (xfd_err &XFD_ERR_GUEST_DISABLED ) +return ; + + xfd_err &= ~XFD_ERR_GUEST_SAVED ; +
arch/x86/kernel/fpu/core.c int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest) · line 370
Quoted 10 lines
arch/x86/kernel/fpu/core.c +10
-0+/* +* No need to restore a zero value since XFD_ERR +* is always zero outside of #NM handler in the host. +*/ +if (!xfd_err) +return ; + +wrmsrl (MSR_IA32_XFD_ERR , xfd_err); + guest_fpu->xfd_err =0 ; +}
Why should any pf this be in the FPU core?
It's a pure guest issue as all of this is related to struct fpu_guest and not struct fpu or any other core FPU state.
Thanks,
tglx
PATCH 15/19 kvm: x86: Save and restore guest XFD_ERR properly
In reply to Yang ZhongKVM needs to save the guest XFD_ERR value before this register might be accessed by the host and restore it before entering the guest.
This implementation saves guest XFD_ERR in two transition points:
- When the vCPU thread exits to the userspace VMM;
- When the vCPU thread is preempted;
XFD_ERR is cleared to ZERO right after saving the previous guest value. Otherwise a stale guest value may confuse the host #NM handler to misinterpret a non-XFD-related #NM as XFD related.
There is no need to save the host XFD_ERR value because the only place where XFD_ERR is consumed outside of KVM is in #NM handler (which can not be preempted by a vCPU thread). XFD_ERR should always be observed as ZER0 outside of #NM hanlder, thus clearing XFD_ERR meets the host expectation here.
The saved guest value is restored to XFD_ERR right before entering the guest (with preemption disabled).
Current implementation still has two opens which we would like to hear suggestions:
1) Will #NM be triggered in host kernel?
Now the code is written assuming above is true, and it's the only reason for saving guest XFD_ERR at preemption time. Otherwise the save is only required when the CPU enters ring-3 (either from the vCPU itself or other threads), by leveraging the "user-return notifier" machinery as suggested by Paolo.
2) When to enable XFD_ERR save/restore?
There are four options on the table:
a) As long as guest cpuid has xfd enabled
XFD_ERR save/restore is enabled in every VM-exit (if preemption or ret-to-userspace happens)
b) When the guest sets IA32_XFD to 1 for the first time
Indicate that guest OS supports XFD features. Because guest OS usually initializes IA32_XFD at boot time, XFD_ERR save/restore is enabled for almost every VM-exit (if preemption or ret-to- userspace happens).
No save/restore for legacy guest OS which doesn't support XFD features at all (thus won't touch IA32_XFD).
c) When the guest sets IA32_XFD to 0 for the first time
Lazily enabling XFD_ERR save/restore until XFD features are used inside guest. However, this option doesn't work because XFD_ERR is set when #NM is raised. An VM-exit could happen between CPU raising #NM and guest #NM handler reading XFD_ERR (before setting XFD to 0). The very first XFD_ERR might be already clobbered by the host due to no save/restore in that small window.
d) When the 1st guest #NM with non-zero XFD_ERR occurs
Lazily enabling XFD_ERR save/restore until XFD features are used inside guest. This requires intercepting guest #NM until non-zero XFD_ERR occurs. If a guest with XFD in cpuid never launches an AMX application, it implies that #NM is always trapped thus adding a constant overhead which may be even higher than doing RDMSR in preemption path in a) and b):
#preempts < #VMEXITS (no #NM trap) < #VMEXITS (#NM trap)
The number of preemptions and ret-to-userspaces should be a small portion of total #VMEXITs in a healthy virtualization environment. Our gut-feeling is that adding at most one MSR read and one MSR write to the preempt/user-ret paths is possibly more efficient than increasing #VMEXITs due to trapping #NM.
For above analysis we plan to go option b), although this version currently implements a). But we would like to hear other suggestions before making this change.
--- arch/x86/kernel/fpu/core.c | 2 ++ arch/x86/kvm/cpuid.c | 5 +++++ arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/vmx/vmx.h | 2 +- arch/x86/kvm/x86.c | 5 +++++ 5 files changed, 15 insertions(+), 1 deletion(-)
arch/x86/kernel/fpu/core.c +2 -0
@@ -238,6 +238,7 @@ bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) fpstate->is_guest = true; gfpu->fpstate = fpstate; + gfpu->xfd_err =XFD_ERR_GUEST_DISABLED ; gfpu->user_xfeatures = fpu_user_cfg.default_features; gfpu->user_perm = fpu_user_cfg.default_features;fpu_init_guest_permissions (gfpu);
@@ -297,6 +298,7 @@ int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest) fpu->fpstate = guest_fps; guest_fps->in_use = true; }else { +fpu_save_guest_xfd_err (guest_fpu); guest_fps->in_use = false; fpu->fpstate = fpu->__task_fpstate; fpu->__task_fpstate =NULL ;
arch/x86/kvm/cpuid.c +5 -0
@@ -219,6 +219,11 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)kvm_apic_set_version (vcpu); } +/* Enable saving guest XFD_ERR */ + best =kvm_find_cpuid_entry (vcpu,7 ,0 ); +if (best &&cpuid_entry_has (best,X86_FEATURE_AMX_TILE )) + vcpu->arch.guest_fpu.xfd_err =0 ; + best =kvm_find_cpuid_entry (vcpu,0xD ,0 );if (!best) vcpu->arch.guest_supported_xcr0 =0 ;
arch/x86/kvm/vmx/vmx.c +2 -0
@@ -161,6 +161,7 @@ static u32 vmx_possible_passthrough_msrs[MAX_POSSIBLE_PASSTHROUGH_MSRS] = {MSR_GS_BASE ,MSR_KERNEL_GS_BASE ,MSR_IA32_XFD , +MSR_IA32_XFD_ERR , #endifMSR_IA32_SYSENTER_CS , MSR_IA32_SYSENTER_ESP,
@@ -7153,6 +7154,7 @@ static void update_intel_pt_cfg(struct kvm_vcpu *vcpu)static void vmx_update_intercept_xfd (struct kvm_vcpu *vcpu) {vmx_set_intercept_for_msr (vcpu,MSR_IA32_XFD ,MSR_TYPE_R , false); +vmx_set_intercept_for_msr (vcpu,MSR_IA32_XFD_ERR ,MSR_TYPE_RW , false); }static void vmx_vcpu_after_set_cpuid (struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/vmx.h +1 -1
@@ -340,7 +340,7 @@ struct vcpu_vmx {struct lbr_desc lbr_desc;/* Save desired MSR intercept (read: pass-through) state */ -#define MAX_POSSIBLE_PASSTHROUGH_MSRS 14 +#define MAX_POSSIBLE_PASSTHROUGH_MSRS 15struct { DECLARE_BITMAP(read, MAX_POSSIBLE_PASSTHROUGH_MSRS); DECLARE_BITMAP(write, MAX_POSSIBLE_PASSTHROUGH_MSRS);
arch/x86/kvm/x86.c +5 -0
@@ -4550,6 +4550,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)kvm_steal_time_set_preempted (vcpu);srcu_read_unlock (&vcpu->kvm->srcu, idx); +if (vcpu->preempted) +fpu_save_guest_xfd_err (&vcpu->arch.guest_fpu); +static_call (kvm_x86_vcpu_put)(vcpu); vcpu->arch.last_host_tsc =rdtsc (); }
@@ -9951,6 +9954,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (test_thread_flag (TIF_NEED_FPU_LOAD ))switch_fpu_return (); +fpu_restore_guest_xfd_err (&vcpu->arch.guest_fpu); +if (unlikely (vcpu->arch.switch_db_regs)) {set_debugreg (0 ,7 );set_debugreg (vcpu->arch.eff_db[0 ],0 );
Paolo Bonzini
maintainerRe: [PATCH 15/19] kvm: x86: Save and restore guest XFD_ERR properly
arch/x86/kvm/x86.c void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) · line 4555
Quoted 6 lines · Yang Zhong
kvm_steal_time_set_preempted(vcpu); srcu_read_unlock(&vcpu->kvm->srcu, idx);arch/x86/kvm/x86.c +3
-0+if (vcpu->preempted) +fpu_save_guest_xfd_err (&vcpu->arch.guest_fpu); +
Instead of checking vcpu->preempted, can you instead check if the active FPU is the guest FPU? That is, save if current->thread.fpu->fpstate->is_guest?
Paolo
Paolo Bonzini
maintainerRe: [PATCH 15/19] kvm: x86: Save and restore guest XFD_ERR properly
In reply to Yang Zhongarch/x86/kvm/cpuid.c static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 226
Quoted 11 lines · Yang Zhong
arch/x86/kvm/cpuid.c +5
-0@@ -219,6 +219,11 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)kvm_apic_set_version (vcpu); } +/* Enable saving guest XFD_ERR */ + best =kvm_find_cpuid_entry (vcpu,7 ,0 ); +if (best &&cpuid_entry_has (best,X86_FEATURE_AMX_TILE )) + vcpu->arch.guest_fpu.xfd_err =0 ; +
This is incorrect. Instead it should check whether leaf 0xD includes any dynamic features.
Paolo
Re: [PATCH 15/19] kvm: x86: Save and restore guest XFD_ERR properly
Quoted 15 lines · Paolo Bonzini
arch/x86/kvm/cpuid.c static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 225
Quoted 10 lines · Yang Zhong
arch/x86/kvm/cpuid.c +5
-0@@ -219,6 +219,11 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)kvm_apic_set_version (vcpu); } +/* Enable saving guest XFD_ERR */ + best =kvm_find_cpuid_entry (vcpu,7 ,0 ); +if (best &&cpuid_entry_has (best,X86_FEATURE_AMX_TILE )) + vcpu->arch.guest_fpu.xfd_err =0 ; +This is incorrect. Instead it should check whether leaf
0xDincludes any dynamic features.
Thanks Paolo, So ditto for "[PATCH 04/19] kvm: x86: Check guest xstate permissions when KVM_SET_CPUID2".
Yang
Quoted 1 line
Paolo
Re: [PATCH 15/19] kvm: x86: Save and restore guest XFD_ERR properly
In reply to Yang Zhongarch/x86/kernel/fpu/core.c bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) · line 241
Quoted 9 lines · Yang Zhong
arch/x86/kernel/fpu/core.c +1
-0@@ -238,6 +238,7 @@ bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) fpstate->is_guest = true; gfpu->fpstate = fpstate; + gfpu->xfd_err =XFD_ERR_GUEST_DISABLED ;
This wants to be part of the previous patch, which introduces the field.
arch/x86/kernel/fpu/core.c int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest) · line 301
Quoted 8 lines
arch/x86/kernel/fpu/core.c +1
-0gfpu->user_xfeatures = fpu_user_cfg.default_features; gfpu->user_perm = fpu_user_cfg.default_features;fpu_init_guest_permissions (gfpu);@@ -297,6 +298,7 @@ int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest) fpu->fpstate = guest_fps; guest_fps->in_use = true; }else { +fpu_save_guest_xfd_err (guest_fpu);
Hmm. See below.
arch/x86/kernel/fpu/core.c void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) · line 4554
Quoted 9 lines
arch/x86/kernel/fpu/core.c +2
-0guest_fps->in_use = false; fpu->fpstate = fpu->__task_fpstate; fpu->__task_fpstate =NULL ;@@ -4550,6 +4550,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)kvm_steal_time_set_preempted (vcpu);srcu_read_unlock (&vcpu->kvm->srcu, idx); +if (vcpu->preempted) +fpu_save_guest_xfd_err (&vcpu->arch.guest_fpu);
I'm not really exited about the thought of an exception cause register in guest clobbered state.
Aside of that I really have to ask the question why all this is needed?
#NM in the guest is slow path, right? So why are you trying to optimize for it?
The straight forward solution to this is:
1) Trap #NM and MSR_XFD_ERR write
2) When the guest triggers #NM is takes an VMEXIT and the host
does:
rdmsrl(MSR_XFD_ERR, vcpu->arch.guest_fpu.xfd_err);
injects the #NM and goes on.
3) When the guest writes to MSR_XFD_ERR it takes an VMEXIT and
the host does:
vcpu->arch.guest_fpu.xfd_err = msrval;
wrmsrl(MSR_XFD_ERR, msrval);
and goes back.
4) Before entering the preemption disabled section of the VCPU loop
do:
if (vcpu->arch.guest_fpu.xfd_err)
wrmsrl(MSR_XFD_ERR, vcpu->arch.guest_fpu.xfd_err);
5) Before leaving the preemption disabled section of the VCPU loop
do:
if (vcpu->arch.guest_fpu.xfd_err)
wrmsrl(MSR_XFD_ERR, 0);It's really that simple and pretty much 0 overhead for the regular case.
If the guest triggers #NM with a high frequency then taking the VMEXITs is the least of the problems. That's not a realistic use case, really.
Hmm?
Thanks,
tglx
Paolo Bonzini
maintainerRe: [PATCH 15/19] kvm: x86: Save and restore guest XFD_ERR properly
Quoted 5 lines · Thomas Gleixner
2) When the guest triggers #NM is takes an VMEXIT and the host does:
rdmsrl(MSR_XFD_ERR, vcpu->arch.guest_fpu.xfd_err);
Quoted 7 lines
injects the #NM and goes on.
3) When the guest writes to
MSR_XFD_ERRit takes an VMEXIT and the host does:
vcpu->arch.guest_fpu.xfd_err= msrval;wrmsrl(MSR_XFD_ERR, msrval);
No wrmsrl here I think, the host value is 0 and should stay so. Instead the wrmsrl will happen the next time the VCPU loop is entred.
Paolo
RE: [PATCH 15/19] kvm: x86: Save and restore guest XFD_ERR properly
Quoted 11 lines
From: Paolo Bonzini
Sent: Saturday, December 11, 2021 9:32 AMQuoted 7 lines · Thomas Gleixner
2) When the guest triggers #NM is takes an VMEXIT and the host does:
rdmsrl(MSR_XFD_ERR, vcpu->arch.guest_fpu.xfd_err);injects the #NM and goes on.
Quoted 9 lines
Quoted 5 lines
3) When the guest writes to
MSR_XFD_ERRit takes an VMEXIT and the host does:
vcpu->arch.guest_fpu.xfd_err= msrval;wrmsrl(MSR_XFD_ERR, msrval);No wrmsrl here I think, the host value is 0 and should stay so. Instead the wrmsrl will happen the next time the VCPU loop is entred.
To elaborate I guess the reason is because MSR_XFD_ERR should always contain host value 0 after preemption is enabled, while WRMSR emulation is called with preemption enabled. Then we just need wait for the next time the vcpu loop is entered to restore the guest value after preemption is disabled. 😊
Thanks
Kevin
Re: [PATCH 15/19] kvm: x86: Save and restore guest XFD_ERR properly
In reply to Paolo BonziniQuoted 8 lines · Paolo Bonzini
Quoted 7 lines · Thomas Gleixner
2) When the guest triggers #NM is takes an VMEXIT and the host does:
rdmsrl(MSR_XFD_ERR, vcpu->arch.guest_fpu.xfd_err);injects the #NM and goes on.
Quoted 8 lines
Quoted 5 lines
3) When the guest writes to
MSR_XFD_ERRit takes an VMEXIT and the host does:
vcpu->arch.guest_fpu.xfd_err= msrval;wrmsrl(MSR_XFD_ERR, msrval);No wrmsrl here I think, the host value is 0 and should stay so. Instead the wrmsrl will happen the next time the VCPU loop is entred.
I assumed this can be handled in the fast path, but either way.
RE: [PATCH 15/19] kvm: x86: Save and restore guest XFD_ERR properly
In reply to Thomas GleixnerQuoted 42 lines
From: Thomas Gleixner <[email protected]>
Sent: Saturday, December 11, 2021 8:11 AMarch/x86/kernel/fpu/core.c bool fpu_alloc_guest_fpstate(struct fpu_guest
Quoted 5 lines · Yang Zhong
arch/x86/kernel/fpu/core.c +0
-0@@ -238,6 +238,7 @@ bool fpu_alloc_guest_fpstate(struct fpu_guest*gfpu)
arch/x86/kernel/fpu/core.c bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) · line 241
Quoted 4 lines
arch/x86/kernel/fpu/core.c +1
-0fpstate->is_guest = true; gfpu->fpstate = fpstate; + gfpu->xfd_err =XFD_ERR_GUEST_DISABLED ;This wants to be part of the previous patch, which introduces the field.
arch/x86/kernel/fpu/core.c int fpu_swap_kvm_fpstate(struct fpu_guest
Quoted 4 lines
arch/x86/kernel/fpu/core.c +0
-0gfpu->user_xfeatures = fpu_user_cfg.default_features; gfpu->user_perm = fpu_user_cfg.default_features;fpu_init_guest_permissions (gfpu);@@ -297,6 +298,7 @@ int fpu_swap_kvm_fpstate(struct fpu_guest
*guest_fpu, boolenter_guest)arch/x86/kernel/fpu/core.c int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest) · line 301
Quoted 4 lines
arch/x86/kernel/fpu/core.c +1
-0fpu->fpstate = guest_fps; guest_fps->in_use = true; }else { +fpu_save_guest_xfd_err (guest_fpu);Hmm. See below.
arch/x86/kernel/fpu/core.c void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) · line 4554
Quoted 9 lines
arch/x86/kernel/fpu/core.c +2
-0guest_fps->in_use = false; fpu->fpstate = fpu->__task_fpstate; fpu->__task_fpstate =NULL ;@@ -4550,6 +4550,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)kvm_steal_time_set_preempted (vcpu);srcu_read_unlock (&vcpu->kvm->srcu, idx); +if (vcpu->preempted) +fpu_save_guest_xfd_err (&vcpu->arch.guest_fpu);I'm not really exited about the thought of an exception cause register in guest clobbered state.
Quoted 4 lines
Aside of that I really have to ask the question why all this is needed?
#NM in the guest is slow path, right? So why are you trying to optimize for it?
This is really good information. The current logic is obviously based on the assumption that #NM is frequently triggered.
Quoted 4 lines
The straight forward solution to this is:
1) Trap #NM and
MSR_XFD_ERRwrite
and #NM vmexit handler should be called in kvm_x86_handle_exit_irqoff() before preemption is enabled, otherwise there is still a small window where MSR_XFD_ERR might be clobbered after preemption enable and before #NM handler is actually called.
Quoted 28 lines
2) When the guest triggers #NM is takes an VMEXIT and the host
does:
rdmsrl(MSR_XFD_ERR, vcpu->arch.guest_fpu.xfd_err);
injects the #NM and goes on.
3) When the guest writes to MSR_XFD_ERR it takes an VMEXIT and
the host does:
vcpu->arch.guest_fpu.xfd_err = msrval;
wrmsrl(MSR_XFD_ERR, msrval);
and goes back.
4) Before entering the preemption disabled section of the VCPU loop
do:
if (vcpu->arch.guest_fpu.xfd_err)
wrmsrl(MSR_XFD_ERR, vcpu->arch.guest_fpu.xfd_err);
5) Before leaving the preemption disabled section of the VCPU loop
do:
if (vcpu->arch.guest_fpu.xfd_err)
wrmsrl(MSR_XFD_ERR, 0);Quoted 1 line
It's really that simple and pretty much 0 overhead for the regular case.
Much cleaner.
Quoted 9 lines
If the guest triggers #NM with a high frequency then taking the VMEXITs is the least of the problems. That's not a realistic use case, really.
Hmm?
Thanks,
tglx
Thanks
Kevin
RE: [PATCH 15/19] kvm: x86: Save and restore guest XFD_ERR properly
Kevin,
Quoted 6 lines · Kevin Tian
Quoted 3 lines
From: Thomas Gleixner <[email protected]>
#NM in the guest is slow path, right? So why are you trying to optimize for it?This is really good information. The current logic is obviously based on the assumption that #NM is frequently triggered.
More context.
When an application want's to use AMX, it invokes the prctl() which grants permission. If permission is granted then still the kernel FPU state buffers are default size and XFD is armed.
When a thread of that process issues the first AMX (tile) instruction, then #NM is raised.
The #NM handler does:
1) Read MSR_XFD_ERR. If 0, goto regular #NM
2) Write MSR_XFD_ERR to 0
3) Check whether the process has permission granted. If not, raise SIGILL and return.
4) Allocate and install a larger FPU state buffer for the task. If allocation fails, raise SIGSEGV and return.
5) Disarm XFD for that task
That means one thread takes at max. one AMX/XFD related #NM during its lifetime, which means two VMEXITs.
If there are other XFD controlled facilities in the future, then it will be NR_USED_XFD_CONTROLLED_FACILITIES * 2 VMEXITs per thread which uses them. Not the end of the world either.
Looking at the targeted application space it's pretty unlikely that tasks which utilize AMX are going to be so short lived that the overhead of these VMEXITs really matters.
This of course can be revisited when there is a sane use case, but optimizing for it prematurely does not buy us anything else than pointless complexity.
Quoted 8 lines
Quoted 3 lines
The straight forward solution to this is:
1) Trap #NM and
MSR_XFD_ERRwriteand #NM vmexit handler should be called in
kvm_x86_handle_exit_irqoff()before preemption is enabled, otherwise there is still a small window whereMSR_XFD_ERRmight be clobbered after preemption enable and before #NM handler is actually called.
Yes.
Thanks,
tglx
RE: [PATCH 15/19] kvm: x86: Save and restore guest XFD_ERR properly
Quoted 44 lines
From: Thomas Gleixner <[email protected]>
Sent: Saturday, December 11, 2021 9:29 PMKevin,
Quoted 6 lines · Kevin Tian
Quoted 3 lines
From: Thomas Gleixner <[email protected]>
#NM in the guest is slow path, right? So why are you trying to optimize for it?This is really good information. The current logic is obviously based on the assumption that #NM is frequently triggered.
More context.
When an application want's to use AMX, it invokes the
prctl()which grants permission. If permission is granted then still the kernel FPU state buffers are default size and XFD is armed.When a thread of that process issues the first AMX (tile) instruction, then #NM is raised.
The #NM handler does:
1) Read
MSR_XFD_ERR. If 0, goto regular #NM2) Write
MSR_XFD_ERRto 03) Check whether the process has permission granted. If not, raise SIGILL and return.
4) Allocate and install a larger FPU state buffer for the task. If allocation fails, raise SIGSEGV and return.
5) Disarm XFD for that task
That means one thread takes at max. one AMX/XFD related #NM during its lifetime, which means two VMEXITs.
If there are other XFD controlled facilities in the future, then it will be
NR_USED_XFD_CONTROLLED_FACILITIES* 2 VMEXITs per thread which uses
them. Not the end of the world either.
Quoted 7 lines
Looking at the targeted application space it's pretty unlikely that tasks which utilize AMX are going to be so short lived that the overhead of these VMEXITs really matters.
This of course can be revisited when there is a sane use case, but optimizing for it prematurely does not buy us anything else than pointless complexity.
I get all above.
I guess the original open is also about the frequency of #NM not due to XFD. For Linux guest looks it's not a problem since CR0.TS is not set now when math emulation is not required:
DEFINE_IDTENTRY(exc_device_not_available)
{
...
/* This should not happen. */
if (WARN(cr0 & X86_CR0_TS, "CR0.TS was set")) {
/* Try to fix it up and carry on. */
write_cr0(cr0 & ~X86_CR0_TS);
} else {
/*
* Something terrible happened, and we're better off trying
* to kill the task than getting stuck in a never-ending
* loop of #NM faults.
*/
die("unexpected #NM exception", regs, 0);
}
}It may affect guest which still uses CR0.TS to do lazy save. But likely modern OSes all move to eager save approach so always trapping #NM should be fine.
Is this understanding correct?
Thanks
Kevin
Paolo Bonzini
maintainerRe: [PATCH 15/19] kvm: x86: Save and restore guest XFD_ERR properly
Quoted 10 lines · Kevin
Quoted 10 lines
If there are other XFD controlled facilities in the future, then it will be
NR_USED_XFD_CONTROLLED_FACILITIES* 2 VMEXITs per thread which uses
them. Not the end of the world either.Looking at the targeted application space it's pretty unlikely that tasks which utilize AMX are going to be so short lived that the overhead of these VMEXITs really matters.
Quoted 6 lines
Quoted 3 lines
This of course can be revisited when there is a sane use case, but optimizing for it prematurely does not buy us anything else than pointless complexity.
It may affect guest which still uses CR0.TS to do lazy save. But likely modern OSes all move to eager save approach so always trapping #NM should be fine.
You also don't need to trap #NM if CPUID includes no dynamic bits, because then XFD will never be nonzero.
Paolo
PATCH 16/19 kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
When dynamic XSTATE features are supported, the xsave states are beyond 4KB. The current kvm_xsave structure and related KVM_{G, S}ET_XSAVE only allows 4KB which is not enough for full states.
Introduce a new kvm_xsave2 structure and the corresponding KVM_GET_XSAVE2 and KVM_SET_XSAVE2 ioctls so that userspace VMM can get and set the full xsave states.
--- arch/x86/include/uapi/asm/kvm.h | 6 ++++ arch/x86/kvm/x86.c | 62 +++++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 7 +++- 3 files changed, 74 insertions(+), 1 deletion(-)
arch/x86/include/uapi/asm/kvm.h +6 -0
@@ -47,6 +47,7 @@#define __KVM_HAVE_VCPU_EVENTS#define __KVM_HAVE_DEBUGREGS#define __KVM_HAVE_XSAVE +#define __KVM_HAVE_XSAVE2#define __KVM_HAVE_XCRS#define __KVM_HAVE_READONLY_MEM
@@ -378,6 +379,11 @@ struct kvm_xsave {__u32 region[1024 ]; }; +struct kvm_xsave2 { +__u32 size; +__u8 state[0 ]; +}; +#define KVM_MAX_XCRS 16struct kvm_xcr {
arch/x86/kvm/x86.c +62 -0
86 lines not on this page · raw message
include/uapi/linux/kvm.h +6 -1
@@ -1132,7 +1132,9 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204#define KVM_CAP_ARM_MTE 205#define KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM 206 - +#ifdef __KVM_HAVE_XSAVE2 +#define KVM_CAP_XSAVE2 207 +#endif #ifdef KVM_CAP_IRQ_ROUTING struct kvm_irq_routing_irqchip {
@@ -1679,6 +1681,9 @@ struct kvm_xen_hvm_attr {#define KVM_GET_SREGS2 _IOR(KVMIO, 0xcc, struct kvm_sregs2)#define KVM_SET_SREGS2 _IOW(KVMIO, 0xcd, struct kvm_sregs2) +#define KVM_GET_XSAVE2 _IOR(KVMIO, 0xcf, struct kvm_xsave2) +#define KVM_SET_XSAVE2 _IOW(KVMIO, 0xd0, struct kvm_xsave2) +struct kvm_xen_vcpu_attr {__u16 type;__u16 pad[3 ];
Paolo Bonzini
maintainerRe: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 154 lines · Yang Zhong
From: Jing Liu <[email protected]>
When dynamic XSTATE features are supported, the xsave states are beyond 4KB. The current
kvm_xsavestructure and related KVM_{G, S}ET_XSAVE only allows 4KB which is not enough for full states.Introduce a new
kvm_xsave2structure and the correspondingKVM_GET_XSAVE2andKVM_SET_XSAVE2ioctls so that userspace VMM can get and set the full xsave states.Signed-off-by Jing Liu <[email protected]>Signed-off-by Yang Zhong <[email protected]>--- arch/x86/include/uapi/asm/kvm.h | 6 ++++ arch/x86/kvm/x86.c | 62 +++++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 7 +++- 3 files changed, 74 insertions(+), 1 deletion(-)arch/x86/include/uapi/asm/kvm.h +6
-0@@ -47,6 +47,7 @@#define __KVM_HAVE_VCPU_EVENTS#define __KVM_HAVE_DEBUGREGS#define __KVM_HAVE_XSAVE +#define __KVM_HAVE_XSAVE2#define __KVM_HAVE_XCRS#define __KVM_HAVE_READONLY_MEM@@ -378,6 +379,11 @@ struct kvm_xsave {__u32 region[1024 ]; }; +struct kvm_xsave2 { +__u32 size; +__u8 state[0 ]; +}; +#define KVM_MAX_XCRS 16struct kvm_xcr {arch/x86/kvm/x86.c +62
-0@@ -4216,6 +4216,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_DEBUGREGS :case KVM_CAP_X86_ROBUST_SINGLESTEP :case KVM_CAP_XSAVE : +case KVM_CAP_XSAVE2 :case KVM_CAP_ASYNC_PF :case KVM_CAP_ASYNC_PF_INT :case KVM_CAP_GET_TSC_KHZ :@@ -4940,6 +4941,17 @@ static void kvm_vcpu_ioctl_x86_get_xsave(struct kvm_vcpu *vcpu, vcpu->arch.pkru); } +static void kvm_vcpu_ioctl_x86_get_xsave2 (struct kvm_vcpu *vcpu, +u8 *state,u32 size) +{ +if (fpstate_is_confidential (&vcpu->arch.guest_fpu)) +return ; + +fpu_copy_guest_fpstate_to_uabi (&vcpu->arch.guest_fpu, + state, size, + vcpu->arch.pkru); +} +static int kvm_vcpu_ioctl_x86_set_xsave (struct kvm_vcpu *vcpu,struct kvm_xsave *guest_xsave) {@@ -4951,6 +4963,15 @@ static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu, supported_xcr0, &vcpu->arch.pkru); } +static int kvm_vcpu_ioctl_x86_set_xsave2 (struct kvm_vcpu *vcpu,u8 *state) +{ +if (fpstate_is_confidential (&vcpu->arch.guest_fpu)) +return 0 ; + +return fpu_copy_uabi_to_guest_fpstate (&vcpu->arch.guest_fpu, state, + supported_xcr0, &vcpu->arch.pkru); +} +static void kvm_vcpu_ioctl_x86_get_xcrs (struct kvm_vcpu *vcpu,struct kvm_xcrs *guest_xcrs) {@@ -5416,6 +5437,47 @@ long kvm_arch_vcpu_ioctl(struct file *filp, r =kvm_vcpu_ioctl_x86_set_xsave (vcpu, u.xsave);break ; } +case KVM_GET_XSAVE2 : { +struct kvm_xsave2 __user *xsave2_arg = argp; +struct kvm_xsave2 xsave2; + + r = -EFAULT ; +if (copy_from_user (&xsave2, xsave2_arg,sizeof (struct kvm_xsave2 ))) +break ; + + u.buffer =kzalloc (xsave2.size,GFP_KERNEL_ACCOUNT ); + + r = -ENOMEM ; +if (!u.buffer) +break ; + +kvm_vcpu_ioctl_x86_get_xsave2 (vcpu, u.buffer, xsave2.size); + + r = -EFAULT ; +if (copy_to_user (xsave2_arg->state, u.buffer, xsave2.size)) +break ; + + r =0 ; +break ; + } +case KVM_SET_XSAVE2 : { +struct kvm_xsave2 __user *xsave2_arg = argp; +struct kvm_xsave2 xsave2; + + r = -EFAULT ; +if (copy_from_user (&xsave2, xsave2_arg,sizeof (struct kvm_xsave2 ))) +break ; + + u.buffer =memdup_user (xsave2_arg->state, xsave2.size); + +if (IS_ERR (u.buffer)) { + r =PTR_ERR (u.buffer); + goto out_nofree; + } + + r =kvm_vcpu_ioctl_x86_set_xsave2 (vcpu, u.buffer); +break ; + }case KVM_GET_XCRS : { u.xcrs =kzalloc (sizeof (struct kvm_xcrs ),GFP_KERNEL_ACCOUNT ); r = -ENOMEM ;include/uapi/linux/kvm.h +3
-1@@ -1132,7 +1132,9 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204#define KVM_CAP_ARM_MTE 205#define KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM 206 - +#ifdef __KVM_HAVE_XSAVE2 +#define KVM_CAP_XSAVE2 207 +#endif #ifdef KVM_CAP_IRQ_ROUTING struct kvm_irq_routing_irqchip {
include/uapi/linux/kvm.h struct kvm_xen_hvm_attr { · line 1689
Quoted 9 lines
include/uapi/linux/kvm.h +3
-0@@ -1679,6 +1681,9 @@ struct kvm_xen_hvm_attr {#define KVM_GET_SREGS2 _IOR(KVMIO, 0xcc, struct kvm_sregs2)#define KVM_SET_SREGS2 _IOW(KVMIO, 0xcd, struct kvm_sregs2) +#define KVM_GET_XSAVE2 _IOR(KVMIO, 0xcf, struct kvm_xsave2) +#define KVM_SET_XSAVE2 _IOW(KVMIO, 0xd0, struct kvm_xsave2) +struct kvm_xen_vcpu_attr {__u16 type;__u16 pad[3 ];
Please also modify KVM_GET/SET_XSAVE to fail with ENOSPC if the requested size is bigger than 4096.
Paolo
Paolo Bonzini
maintainerRe: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
In reply to Yang ZhongQuoted 17 lines · Yang Zhong
arch/x86/kvm/x86.c +11
-0+static void kvm_vcpu_ioctl_x86_get_xsave2 (struct kvm_vcpu *vcpu, +u8 *state,u32 size) +{ +if (fpstate_is_confidential (&vcpu->arch.guest_fpu)) +return ; + +fpu_copy_guest_fpstate_to_uabi (&vcpu->arch.guest_fpu, + state, size, + vcpu->arch.pkru); +} +static int kvm_vcpu_ioctl_x86_set_xsave (struct kvm_vcpu *vcpu,struct kvm_xsave *guest_xsave) {@@ -4951,6 +4963,15 @@ static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu, supported_xcr0, &vcpu->arch.pkru); }
arch/x86/kvm/x86.c static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu, · line 4974
Quoted 10 lines
arch/x86/kvm/x86.c +9
-0+static int kvm_vcpu_ioctl_x86_set_xsave2 (struct kvm_vcpu *vcpu,u8 *state) +{ +if (fpstate_is_confidential (&vcpu->arch.guest_fpu)) +return 0 ; + +return fpu_copy_uabi_to_guest_fpstate (&vcpu->arch.guest_fpu, state, + supported_xcr0, &vcpu->arch.pkru); +} +
I think fpu_copy_uabi_to_guest_fpstate (and therefore copy_uabi_from_kernel_to_xstate) needs to check that the size is compatible with the components in the input.
Also, IIUC the size of the AMX state will vary in different processors.
Is this correct? If so, this should be handled already by
KVM_GET/SET_XSAVE2 and therefore should be part of the arch/x86/kernel/fpu APIs. In the future we want to support migrating a "small AMX" host to a "large AMX" host; and also migrating from a "large AMX" host to a "small AMX" host if the guest CPUID is compatible with the destination of the migration.
Paolo
Paolo Bonzini
maintainerRe: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 14 lines · Paolo Bonzini
arch/x86/kvm/x86.c static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu, · line 4974
Quoted 11 lines
+static int
kvm_vcpu_ioctl_x86_set_xsave2(struct kvm_vcpu *vcpu, u8 *state)arch/x86/kvm/x86.c +8
-0+{ + if (fpstate_is_confidential(&vcpu->arch.guest_fpu)) + return 0; + + return fpu_copy_uabi_to_guest_fpstate(&vcpu->arch.guest_fpu, state, + supported_xcr0, &vcpu->arch.pkru); +} +I think
fpu_copy_uabi_to_guest_fpstate(and thereforecopy_uabi_from_kernel_to_xstate) needs to check that the size is compatible with the components in the input.
Quoted 3 lines
Also, IIUC the size of the AMX state will vary in different processors.
Is this correct? If so, this should be handled already by
KVM_GET/SET_XSAVE2 and therefore should be part of thearch/x86/kernel/fpuAPIs. In the future we want to support migrating a "small AMX" host to a "large AMX" host; and also migrating from a "large AMX" host to a "small AMX" host if the guest CPUID is compatible with the destination of the migration.
So, the size of the AMX state will depend on the active "palette" in TILECONFIG, and on the CPUID information. I have a few questions on how Intel intends to handle future extensions to AMX:
- can we assume that, in the future, palette 1 will always have the same value (bytes_per_row=64, max_names=8, max_rows=16), and basically that the only variable value is really the number of palettes?
- how does Intel plan to handle bigger TILEDATA? Will it use more XCR0 bits or will it rather enlarge save state 18?
If it will use more XCR0 bits, I suppose that XCR0 bits will control which palettes can be chosen by LDTILECFG.
If not, on the other hand, this will be a first case of one system's XSAVE data not being XRSTOR-able on another system even if the destination system can set XCR0 to the same value as the source system.
Likewise, if the size and offsets for save state 18 were to vary depending on the selected palette, then this would be novel, in that the save state size and offsets would not be in CPUID anymore. It would be particularly interesting for non-compacted format, where all save states after 18 would also move forward.
So, I hope that save state 18 will be frozen to 8k. In that case, and if palette 1 is frozen to the same values as today, implementing migration will not be a problem; it will be essentially the same as SSE->AVX (horizontal extension of existing registers) and/or AVX->AVX512 (both horizontal and vertical extension).
By the way, I think KVM_SET_XSAVE2 is not needed. Instead:
- KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2) should return the size of the buffer that is passed to KVM_GET_XSAVE2
- KVM_GET_XSAVE2 should fill in the buffer expecting that its size is whatever KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2) passes
- KVM_SET_XSAVE can just expect a buffer that is bigger than 4k if the save states recorded in the header point to offsets larger than 4k.
Paolo
RE: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 11 lines · Paolo Bonzini
By the way, I think
KVM_SET_XSAVE2is not needed. Instead:-
KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2)should return the size of the buffer that is passed toKVM_GET_XSAVE2-
KVM_GET_XSAVE2should fill in the buffer expecting that its size is whateverKVM_CHECK_EXTENSION(KVM_CAP_XSAVE2)passes-
KVM_SET_XSAVEcan just expect a buffer that is bigger than 4k if the save states recorded in the header point to offsets larger than 4k.
I think one issue is that KVM_SET_XSAVE works with "struct kvm_xsave" (hardcoded 4KB buffer), including kvm_vcpu_ioctl_x86_set_xsave. The states obtained via KVM_GET_XSAVE2 will be made using "struct kvm_xsave2".
Did you mean that we could add a new code path under KVM_SET_XSAVE to make it work with the new "struct kvm_xsave2"?
e.g.:
(xsave2_enabled below is set when userspace calls to get KVM_CAP_XSAVE2) if (kvm->xsave2_enabled) {
new implementation using "struct kvm_xsave2"
} else {
current implementation using "struct kvm_xsave"
}
(this seems like a new implementation which might deserve a new ioctl)
Thanks,
Wei
Paolo Bonzini
maintainerRe: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 13 lines · Wei W
Quoted 11 lines · Paolo Bonzini
By the way, I think
KVM_SET_XSAVE2is not needed. Instead:-
KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2)should return the size of the buffer that is passed toKVM_GET_XSAVE2-
KVM_GET_XSAVE2should fill in the buffer expecting that its size is whateverKVM_CHECK_EXTENSION(KVM_CAP_XSAVE2)passes-
KVM_SET_XSAVEcan just expect a buffer that is bigger than 4k if the save states recorded in the header point to offsets larger than 4k.
Quoted 6 lines
I think one issue is that
KVM_SET_XSAVEworks with "structkvm_xsave" (hardcoded 4KB buffer), includingkvm_vcpu_ioctl_x86_set_xsave. The states obtained viaKVM_GET_XSAVE2will be made using "structkvm_xsave2".Did you mean that we could add a new code path under
KVM_SET_XSAVEto make it work with the new "structkvm_xsave2"?
There is no need for struct kvm_xsave2, because there is no need for a "size" argument.
- KVM_GET_XSAVE2 *is* needed, and it can expect a buffer as big as the return value of KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2)
- but KVM_SET_XSAVE2 is not needed, because KVM_SET_XSAVE can use copy_from_user to read the XSTATE_BV, use it to deduce the size of the buffer, and use copy_from_user to read the full size of the buffer.
For this to work you can redefine struct kvm_xsave to
struct kvm_xsave {
__u32 region[1024];
/*
* KVM_GET_XSAVE only uses 4096 bytes and only returns
* user save states up to save state 17 (TILECFG).
*
* For KVM_GET_XSAVE2, the total size of region + extra
* must be the size that is communicated by
* KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2).
*
* KVM_SET_XSAVE uses the extra field if the struct was
* returned by KVM_GET_XSAVE2.
*/
__u32 extra[];
}Paolo
RE: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 5 lines · Paolo Bonzini
There is no need for struct
kvm_xsave2, because there is no need for a "size" argument.-
KVM_GET_XSAVE2*is* needed, and it can expect a buffer as big as the return value ofKVM_CHECK_EXTENSION(KVM_CAP_XSAVE2)
Why would KVM_GET_XSAVE2 still be needed in this case?
I'm thinking it would also be possible to reuse KVM_GET_XSAVE:
- If userspace calls to KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2),
then KVM knows that the userspace is a new version and it works with larger xsave buffer using the "size" that it returns via KVM_CAP_XSAVE2. So we can add a flag "kvm->xsave2_enabled", which gets set upon userspace checks KVM_CAP_XSAVE2.
- On KVM_GET_XSAVE, if "kvm->xsave2_enabled" is set,
then KVM allocates buffer to load xstates and copies the loaded xstates data to the userspace buffer using the "size" that was returned to userspace on KVM_CAP_XSAVE2.
If "kvm->xsave2_enabled" isn't set, using the legacy "4KB" size.
Thanks,
Wei
Paolo Bonzini
maintainerRe: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 7 lines · Wei W
Quoted 5 lines · Paolo Bonzini
There is no need for struct
kvm_xsave2, because there is no need for a "size" argument.-
KVM_GET_XSAVE2*is* needed, and it can expect a buffer as big as the return value ofKVM_CHECK_EXTENSION(KVM_CAP_XSAVE2)
Quoted 7 lines
Why would
KVM_GET_XSAVE2still be needed in this case?I'm thinking it would also be possible to reuse
KVM_GET_XSAVE:- If userspace calls to
KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2),
then KVM knows that the userspace is a new version and it works with larger xsave buffer using the "size" that it returns viaKVM_CAP_XSAVE2. So we can add a flag "kvm->xsave2_enabled", which gets set upon userspace checksKVM_CAP_XSAVE2.
You can use KVM_ENABLE_CAP(KVM_CAP_XSAVE2) for that, yes. In that case you don't need KVM_GET_XSAVE2.
Paolo
Quoted 8 lines
- On
KVM_GET_XSAVE, if "kvm->xsave2_enabled" is set,
then KVM allocates buffer to load xstates and copies the loaded xstates data to the userspace buffer using the "size" that was returned to userspace onKVM_CAP_XSAVE2.
If "kvm->xsave2_enabled" isn't set, using the legacy "4KB" size.Thanks,
Wei
RE: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 22 lines · Paolo Bonzini
To: Wang, Wei W <[email protected]>; Zhong, Yang <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; Nakajima, Jun <[email protected]>; Tian, Kevin <[email protected]>; [email protected]; Liu, Jing2 <[email protected]>; Zeng, Guang <[email protected]>
Subject: Re: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctlQuoted 2 lines · Wei W
Quoted 1 line · Paolo Bonzini
There is no need for struct
kvm_xsave2, because there is no need for a"size"
Quoted 9 lines
Quoted 4 lines
argument.
-
KVM_GET_XSAVE2*is* needed, and it can expect a buffer as big as the return value ofKVM_CHECK_EXTENSION(KVM_CAP_XSAVE2)Why would
KVM_GET_XSAVE2still be needed in this case?I'm thinking it would also be possible to reuse
KVM_GET_XSAVE:
Quoted 8 lines
Quoted 2 lines
- If userspace calls to
KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2), then KVM knows that the userspace is a new version and it works withlarger xsave buffer using the "size" that it returns via
KVM_CAP_XSAVE2.Quoted 1 line
So we can add a flag "
kvm->xsave2_enabled", which gets set uponuserspace checks
KVM_CAP_XSAVE2.You can use
KVM_ENABLE_CAP(KVM_CAP_XSAVE2)for that, yes. In that case you don't needKVM_GET_XSAVE2.
On more thing here, what size should KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2) return? If the size still comes from the guest CPUID(0xd, 0)::RCX, would it be better to just return 1? This requires that the QEMU CPUID info has been set to KVM before checking the cap. QEMU already has this CPUID info to get the size (seems no need to inquire KVM for it).
Thanks,
Wei
Paolo Bonzini
maintainerRe: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 10 lines · Wei W
Quoted 10 lines
Quoted 6 lines
Why would
KVM_GET_XSAVE2still be needed in this case?I'm thinking it would also be possible to reuse
KVM_GET_XSAVE:- If userspace calls to
KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2), then KVM knows that the userspace is a new version and it works withlarger xsave buffer using the "size" that it returns via
KVM_CAP_XSAVE2.Quoted 1 line
So we can add a flag "
kvm->xsave2_enabled", which gets set uponuserspace checks
KVM_CAP_XSAVE2.
Quoted 7 lines
Quoted 2 lines
You can use
KVM_ENABLE_CAP(KVM_CAP_XSAVE2)for that, yes. In that case you don't needKVM_GET_XSAVE2.On more thing here, what size should
KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2)return? If the size still comes from the guestCPUID(0xd, 0)::RCX, would it be better to just return 1? This requires that the QEMU CPUID info has been set to KVM before checking the cap. QEMU already has this CPUID info to get the size (seems no need to inquire KVM for it).
It's still easier to return the full size of the buffer from KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2). It makes the userspace code a bit easier.
I'm also thinking that I prefer KVM_GET_XSAVE2 to KVM_ENABLE_CAP(KVM_CAP_XSAVE2), after all. Since it would be a backwards-incompatible change to an _old_ ioctl (KVM_GET_XSAVE), I prefer to limit the ways that userspace can shoot itself in the foot.
Paolo
RE: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 3 lines · Paolo Bonzini
It's still easier to return the full size of the buffer from
KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2). It makes the userspace code a bit easier.
OK. For the "full size" returned to userspace, would you prefer to directly use the value retrieved from guest CPUID(0xd), or get it from guest_fpu (i.e. fpstate->user_size)?
(retrieved from CPUID will be the max size and should work fine as well)
Quoted 5 lines
I'm also thinking that I prefer
KVM_GET_XSAVE2toKVM_ENABLE_CAP(KVM_CAP_XSAVE2), after all. Since it would be a backwards-incompatible change to an _old_ ioctl (KVM_GET_XSAVE), I prefer to limit the ways that userspace can shoot itself in the foot.
OK, we will use KVM_GET_XSAVE2.
Thanks,
Wei
Paolo Bonzini
maintainerRe: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 7 lines · Wei W
Quoted 2 lines
It's still easier to return the full size of the buffer from
KVM_CHECK_EXTENSION(KVM_CAP_XSAVE2). It makes the userspace code a bit easier.OK. For the "full size" returned to userspace, would you prefer to directly use the value retrieved from guest CPUID(0xd), or get it from
guest_fpu(i.e.fpstate->user_size)? (retrieved from CPUID will be the max size and should work fine as well)
It is okay to reflect only the bits that were enabled in prctl, but please document it in api.rst as well.
Paolo
State Component 18 and Palette 1 (Re: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl)
In reply to Paolo BonziniQuoted 23 lines · Paolo Bonzini
Quoted 12 lines · Paolo Bonzini
arch/x86/kvm/x86.c static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu, · line 4974
Quoted 10 lines
arch/x86/kvm/x86.c +9
-0+static int kvm_vcpu_ioctl_x86_set_xsave2 (struct kvm_vcpu *vcpu,u8 *state) +{ +if (fpstate_is_confidential (&vcpu->arch.guest_fpu)) +return 0 ; + +return fpu_copy_uabi_to_guest_fpstate (&vcpu->arch.guest_fpu, state, + supported_xcr0, &vcpu->arch.pkru); +} +I think
fpu_copy_uabi_to_guest_fpstate(and thereforecopy_uabi_from_kernel_to_xstate) needs to check that the size is compatible with the components in the input.
Also, IIUC the size of the AMX state will vary in different processors. Is this correct? If so, this should be handled already by KVM_GET/SET_XSAVE2 and therefore should be part of thearch/x86/kernel/fpuAPIs. In the future we want to support migrating a "small AMX" host to a "large AMX" host; and also migrating from a "large AMX" host to a "small AMX" host if the guest CPUID is compatible with the destination of the migration.So, the size of the AMX state will depend on the active "palette" in TILECONFIG, and on the CPUID information. I have a few questions on how Intel intends to handle future extensions to AMX:
- can we assume that, in the future, palette 1 will always have the same value (bytes_per_row=64, max_names=8, max_rows=16), and basically that the only variable value is really the number of palettes?
- how does Intel plan to handle bigger TILEDATA? Will it use more XCR0 bits or will it rather enlarge save state 18?
Quoted 7 lines
If it will use more XCR0 bits, I suppose that XCR0 bits will control which palettes can be chosen by LDTILECFG.
If not, on the other hand, this will be a first case of one system's XSAVE data not being XRSTOR-able on another system even if the destination system can set XCR0 to the same value as the source system.
Likewise, if the size and offsets for save state 18 were to vary depending on the selected palette, then this would be novel, in that the save state size and offsets would not be in CPUID anymore. It would be particularly interesting for non-compacted format, where all save states after 18 would also move forward.
So, I hope that save state 18 will be frozen to 8k. In that case, and if palette 1 is frozen to the same values as today, implementing migration will not be a problem; it will be essentially the same as
SSE->AVX(horizontal extension of existing registers) and/orAVX->AVX512(both horizontal and vertical extension).
Hi Paolo,
I would like to confirm that the state component 18 will remain 8KB and palette 1 will remain the same.
Thanks,
---
Jun
Paolo Bonzini
maintainerRe: State Component 18 and Palette 1 (Re: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl)
Quoted 3 lines · Jun
Hi Paolo,
I would like to confirm that the state component 18 will remain 8KB and palette 1 will remain the same.
Great! Can you also confirm that XCR0 bits will control which palettes can be chosen by LDTILECFG?
Thanks,
Paolo
Re: State Component 18 and Palette 1 (Re: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl)
Quoted 8 lines · Paolo Bonzini
Quoted 2 lines · Jun
Hi Paolo,
I would like to confirm that the state component 18 will remain 8KB and palette 1 will remain the same.Great! Can you also confirm that XCR0 bits will control which palettes can be chosen by LDTILECFG?
I need to discuss with the H/W team more and come back. I think this request is plausible; I will suggest it.
Thanks,
---
Jun
Re: State Component 18 and Palette 1 (Re: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl)
In reply to Nakajima, JunQuoted 9 lines · Jun
Quoted 6 lines
So, I hope that save state 18 will be frozen to 8k. In that case, and if palette 1 is frozen to the same values as today, implementing migration will not be a problem; it will be essentially the same as
SSE->AVX(horizontal extension of existing registers) and/orAVX->AVX512(both horizontal and vertical extension).I would like to confirm that the state component 18 will remain 8KB and palette 1 will remain the same.
Is that an architectural statement that will soon be making its way into the SDM?
Re: State Component 18 and Palette 1 (Re: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl)
Quoted 11 lines · Dave
Quoted 7 lines · Jun
Quoted 6 lines
So, I hope that save state 18 will be frozen to 8k. In that case, and if palette 1 is frozen to the same values as today, implementing migration will not be a problem; it will be essentially the same as
SSE->AVX(horizontal extension of existing registers) and/orAVX->AVX512(both horizontal and vertical extension).
Quoted 5 lines
Quoted 2 lines
I would like to confirm that the state component 18 will remain 8KB and palette 1 will remain the same.
Is that an architectural statement that will soon be making its way into the SDM?
Yes, with the other clarifications (e.g. setting IA32_XFD[18]).
Thanks,
---
Jun
Re: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
In reply to Paolo BonziniQuoted 14 lines · Paolo Bonzini
arch/x86/kvm/x86.c static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu, · line 4974
Quoted 9 lines · Yang Zhong
arch/x86/kvm/x86.c +9
-0+static int kvm_vcpu_ioctl_x86_set_xsave2 (struct kvm_vcpu *vcpu,u8 *state) +{ +if (fpstate_is_confidential (&vcpu->arch.guest_fpu)) +return 0 ; + +return fpu_copy_uabi_to_guest_fpstate (&vcpu->arch.guest_fpu, state, + supported_xcr0, &vcpu->arch.pkru); +} +I think
fpu_copy_uabi_to_guest_fpstate(and thereforecopy_uabi_from_kernel_to_xstate) needs to check that the size is compatible with the components in the input.
fpu_copy_uabi_to_guest_fpstate() expects that the input buffer is correctly sized. We surely can add a size check there.
Quoted 7 lines
Also, IIUC the size of the AMX state will vary in different processors.
Is this correct? If so, this should be handled already by
KVM_GET/SET_XSAVE2 and therefore should be part of thearch/x86/kernel/fpuAPIs. In the future we want to support migrating a "small AMX" host to a "large AMX" host; and also migrating from a "large AMX" host to a "small AMX" host if the guest CPUID is compatible with the destination of the migration.
How is that supposed to work? If the AMX state size differs then the hosts are not compatible.
Thanks,
tglx
Paolo Bonzini
maintainerRe: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 16 lines · Thomas Gleixner
Quoted 14 lines · Paolo Bonzini
Review
Quoted 9 lines · Yang Zhong
(quoted diff) +9
-0+static int kvm_vcpu_ioctl_x86_set_xsave2(struct kvm_vcpu *vcpu, u8 *state) +{ + if (fpstate_is_confidential(&vcpu->arch.guest_fpu)) + return 0; + + return fpu_copy_uabi_to_guest_fpstate(&vcpu->arch.guest_fpu, state, + supported_xcr0, &vcpu->arch.pkru); +} +I think
fpu_copy_uabi_to_guest_fpstate(and thereforecopy_uabi_from_kernel_to_xstate) needs to check that the size is compatible with the components in the input.
Quoted 2 lines
fpu_copy_uabi_to_guest_fpstate()expects that the input buffer is correctly sized. We surely can add a size check there.
fpu_copy_guest_fpstate_to_uabi is more problematic because that one writes memory. For fpu_copy_uabi_to_guest_fpstate, we know the input buffer size from the components and we can use it to do a properly-sized memdup_user.
For fpu_copy_guest_fpstate_to_uabi we can just decide that KVM_GET_XSAVE will only save up to the first 4K. Something like the following might actually be good for 5.16-rc; right now, header.xfeatures might lead userspace into reading uninitialized or unmapped memory:
arch/x86/kernel/fpu/xstate.c +16 -2
@@ -1138,8 +1138,10 @@ void __copy_xstate_to_uabi_buf(struct membuf to, struct fpstate *fpstate,struct xstate_header header;unsigned int zerofrom;u64 mask; +u64 size;int i; + size = to->left;memset (&header,0 ,sizeof (header)); header.xfeatures = xsave->header.xfeatures;
@@ -1186,7 +1188,20 @@ void __copy_xstate_to_uabi_buf(struct membuf to, struct fpstate *fpstate,/* Copy xsave->i387.sw_reserved */ membuf_write (&to, xstate_fx_sw_bytes,sizeof (xsave->i387.sw_reserved)); -/* Copy the user space relevant state of @xsave->header */ +/* +* Copy the user space relevant state of @xsave->header. +* If not all features fit in the buffer, drop them from the +* saved state so that userspace does not read uninitialized or +* unmapped memory. +*/ + mask = fpstate->user_xfeatures; +for_each_extended_xfeature (i, mask) { +if (xstate_offsets[i] + xstate_size[i] > size) { + header.xfeatures &=BIT (i) -1 ; + mask &=BIT (i) -1 ; +break ; + } + }membuf_write (&to, &header,sizeof (header)); zerofrom = offsetof(struct xregs_state , extended_state_area);
@@ -1197,7 +1212,6 @@ void __copy_xstate_to_uabi_buf(struct membuf to, struct fpstate *fpstate,* but there is no state to copy from in the compacted * init_fpstate. The gap tracking will zero these states. */ - mask = fpstate->user_xfeatures;for_each_extended_xfeature (i, mask) {/*
Quoted 10 lines
Quoted 7 lines
Also, IIUC the size of the AMX state will vary in different processors.
Is this correct? If so, this should be handled already by
KVM_GET/SET_XSAVE2 and therefore should be part of thearch/x86/kernel/fpuAPIs. In the future we want to support migrating a "small AMX" host to a "large AMX" host; and also migrating from a "large AMX" host to a "small AMX" host if the guest CPUID is compatible with the destination of the migration.How is that supposed to work? If the AMX state size differs then the hosts are not compatible.
I replied with some more questions later. Basically it depends on how Intel will define palettes that aren't part of the first implementation of AMX.
Paolo
Re: [PATCH 16/19] kvm: x86: Introduce KVM_{G|S}ET_XSAVE2 ioctl
Quoted 14 lines · Paolo Bonzini
Quoted 7 lines · Thomas Gleixner
Quoted 3 lines · Paolo Bonzini
I think
fpu_copy_uabi_to_guest_fpstate(and thereforecopy_uabi_from_kernel_to_xstate) needs to check that the size is compatible with the components in the input.
fpu_copy_uabi_to_guest_fpstate()expects that the input buffer is correctly sized. We surely can add a size check there.
fpu_copy_guest_fpstate_to_uabiis more problematic because that one writes memory. Forfpu_copy_uabi_to_guest_fpstate, we know the input buffer size from the components and we can use it to do a properly-sizedmemdup_user.
Quoted 4 lines
For
fpu_copy_guest_fpstate_to_uabiwe can just decide thatKVM_GET_XSAVEwill only save up to the first 4K. Something like the following might actually be good for 5.16-rc; right now, header.xfeatures might lead userspace into reading uninitialized or unmapped memory:
If user space supplies a 4k buffer and reads beyond the end of the buffer then it's hardly a kernel problem.
That function allows to provide a short buffer and fill it up to the point where the buffer ends with the real information.
Thanks,
tglx
PATCH 17/19 docs: virt: api.rst: Document the new KVM_{G, S}ET_XSAVE2 ioctls
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Document the detailed information of the new KVM_{G, S}ET_XSAVE2 ioctls.
--- Documentation/virt/kvm/api.rst | 47 ++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)
Documentation/virt/kvm/api.rst +47 -0
@@ -1569,6 +1569,8 @@ otherwise it will return EBUSY error. }; This ioctl would copy current vcpu's xsave struct to the userspace. +Application should use KVM_GET_XSAVE2 if xsave states are larger than +4KB. 4.43 KVM_SET_XSAVE
@@ -1588,6 +1590,8 @@ This ioctl would copy current vcpu's xsave struct to the userspace. }; This ioctl would copy userspace's xsave struct to the kernel. +Application should use KVM_SET_XSAVE2 if xsave states are larger than +4KB. 4.44 KVM_GET_XCRS
@@ -7484,3 +7488,46 @@ The argument to KVM_ENABLE_CAP is also a bitmask, and must be a subset of the result of KVM_CHECK_EXTENSION. KVM will forward to userspace the hypercalls whose corresponding bit is in the argument, and return ENOSYS for the others. + +8.35 KVM_GET_XSAVE2 +------------------- + +:Capability: KVM_CAP_XSAVE2 +:Architectures: x86 +:Type: vcpu ioctl +:Parameters: struct kvm_xsave2 (in) +:Returns: 0 on success, -1 on error + + +:: + + struct kvm_xsave2 { + __u32 size; + __u8 state[0]; + }; + +This ioctl is used for copying current vcpu's xsave struct to the +userspace when xsave state size is larger than 4KB. Application code +should set the 'size' member which indicates the size of xsave state +and KVM copies the xsave state into the 'state' region. + +8.36 KVM_SET_XSAVE2 +------------------- + +:Capability: KVM_CAP_XSAVE2 +:Architectures: x86 +:Type: vcpu ioctl +:Parameters: struct kvm_xsave2 (out) +:Returns: 0 on success, -1 on error + + +:: + + struct kvm_xsave2 { + __u32 size; + __u8 state[0]; + }; + +This ioctl is used for copying userspace's xsave struct to the kernel +when xsave size is larger than 4KB. Application code should set the +'size' member which indicates the size of xsave state.
PATCH 18/19 kvm: x86: AMX XCR0 support for guest
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Two XCR0 bits are defined for AMX to support XSAVE mechanism. Bit 17 is for tilecfg and bit 18 is for tiledata.
The value of XCR0[17:18] is always either 00b or 11b. Also, SDM recommends that only 64-bit operating systems enable Intel AMX by setting XCR0[18:17].
--- arch/x86/kvm/x86.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +18 -1
@@ -210,7 +210,7 @@ static struct kvm_user_return_msrs __percpu *user_return_msrs;#define KVM_SUPPORTED_XCR0 (XFEATURE_MASK_FP | XFEATURE_MASK_SSE \ | XFEATURE_MASK_YMM | XFEATURE_MASK_BNDREGS \ | XFEATURE_MASK_BNDCSR | XFEATURE_MASK_AVX512 \ - | XFEATURE_MASK_PKRU) + | XFEATURE_MASK_PKRU | XFEATURE_MASK_XTILE)u64 __read_mostly host_efer;EXPORT_SYMBOL_GPL (host_efer);
@@ -1017,6 +1017,23 @@ static int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr)if ((xcr0 &XFEATURE_MASK_AVX512 ) !=XFEATURE_MASK_AVX512 )return 1 ; } + +#ifdef CONFIG_X86_64 +if ((xcr0 &XFEATURE_MASK_XTILE ) && + ((xcr0 &XFEATURE_MASK_XTILE ) !=XFEATURE_MASK_XTILE )) +return 1 ; +#else +/* +* Intel AMX instructions can be executed only in 64-bit mode but +* XSAVE can operate on XTILECFG and XTILEDATA in any mode. +* Since the FPU core follows SDM recommendation to set +* XCR[18:17] only in 64-bit environment, here also prevent any +* guest OS from setting the two bits when host is 32-bit. +* +* XFEATURE_MASK_XTILE cannot be used since it is 0 in this case. +*/ + xcr0 &= ~(XFEATURE_MASK_XTILE_DATA |XFEATURE_MASK_XTILE_CFG ); +#endif vcpu->arch.xcr0 = xcr0;if ((xcr0 ^ old_xcr0) &XFEATURE_MASK_EXTEND )
Paolo Bonzini
maintainerRe: [PATCH 18/19] kvm: x86: AMX XCR0 support for guest
Quoted 7 lines · Yang Zhong
arch/x86/kvm/x86.c +7
-0+ +#ifdef CONFIG_X86_64 +if ((xcr0 &XFEATURE_MASK_XTILE ) && + ((xcr0 &XFEATURE_MASK_XTILE ) !=XFEATURE_MASK_XTILE )) +return 1 ; +#else +/*
arch/x86/kvm/x86.c static int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr) · line 1036
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+* Intel AMX instructions can be executed only in 64-bit mode but +* XSAVE can operate on XTILECFG and XTILEDATA in any mode. +* Since the FPU core follows SDM recommendation to set +* XCR[18:17] only in 64-bit environment, here also prevent any +* guest OS from setting the two bits when host is 32-bit. +* +* XFEATURE_MASK_XTILE cannot be used since it is 0 in this case. +*/ + xcr0 &= ~(XFEATURE_MASK_XTILE_DATA |XFEATURE_MASK_XTILE_CFG ); +#endif
This should not be necessary, because on a 32-bit system the bits won't be part of supported_xcr0.
Paolo
PATCH 19/19 kvm: x86: Add AMX CPUIDs support
In reply to Yang ZhongFrom: Jing Liu <[email protected]>
Extend CPUID emulation to support XFD, AMX_TILE, AMX_INT8 and AMX_BF16. Adding those bits into kvm_cpu_caps finally activates all previous logics in this series.
--- arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/kvm/cpuid.c | 16 +++++++++++++--- 2 files changed, 15 insertions(+), 3 deletions(-)
arch/x86/include/asm/cpufeatures.h +2 -0
@@ -299,7 +299,9 @@/* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4)/* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5)/* AVX512 BFLOAT16 instructions */ +#define X86_FEATURE_AMX_BF16 (18*32+22)/* AMX bf16 Support */ #define X86_FEATURE_AMX_TILE (18*32+24)/* AMX tile Support */ +#define X86_FEATURE_AMX_INT8 (18*32+25)/* AMX int8 Support */ /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */
arch/x86/kvm/cpuid.c +13 -3
@@ -510,7 +510,8 @@ void kvm_set_cpu_caps(void)F (AVX512_4VNNIW ) |F (AVX512_4FMAPS ) |F (SPEC_CTRL ) |F (SPEC_CTRL_SSBD ) |F (ARCH_CAPABILITIES ) |F (INTEL_STIBP ) |F (MD_CLEAR ) |F (AVX512_VP2INTERSECT ) |F (FSRM ) | -F (SERIALIZE ) |F (TSXLDTRK ) |F (AVX512_FP16 ) +F (SERIALIZE ) |F (TSXLDTRK ) |F (AVX512_FP16 ) | +F (AMX_TILE ) |F (AMX_INT8 ) |F (AMX_BF16 ) );/* TSC_ADJUST and ARCH_CAPABILITIES are emulated in software. */
@@ -529,7 +530,7 @@ void kvm_set_cpu_caps(void) );kvm_cpu_cap_mask (CPUID_D_1_EAX , -F (XSAVEOPT ) |F (XSAVEC ) |F (XGETBV1 ) |F (XSAVES ) +F (XSAVEOPT ) |F (XSAVEC ) |F (XGETBV1 ) |F (XSAVES ) |F (XFD ) ); kvm_cpu_cap_init_scattered(CPUID_12_EAX,
@@ -655,6 +656,8 @@ static struct kvm_cpuid_entry2 *do_host_cpuid(struct kvm_cpuid_array *array,case 0x14 :case 0x17 :case 0x18 : +case 0x1d : +case 0x1e :case 0x1f :case 0x8000001d : entry->flags |=KVM_CPUID_FLAG_SIGNIFCANT_INDEX ;
@@ -779,6 +782,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) }break ;case 9 : +case 0x1e :/* TMUL information */ break ;case 0xa : {/* Architectural Performance Monitoring */ struct x86_pmu_capability cap;
@@ -914,7 +918,8 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)break ;/* Intel PT */ case 0x14 : -if (!kvm_cpu_cap_has (X86_FEATURE_INTEL_PT )) { +if ((function ==0x14 && !kvm_cpu_cap_has (X86_FEATURE_INTEL_PT )) || + (function ==0x1d && !kvm_cpu_cap_has (X86_FEATURE_AMX_TILE ))) { entry->eax = entry->ebx = entry->ecx = entry->edx =0 ;break ; }
@@ -924,6 +929,11 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) goto out; }break ; +/* Intel AMX TILE */ +case 0x1d : +if (!kvm_cpu_cap_has (X86_FEATURE_AMX_TILE )) + entry->eax = entry->ebx = entry->ecx = entry->edx =0 ; +break ;case KVM_CPUID_SIGNATURE : {const u32 *sigptr = (const u32 *)KVM_SIGNATURE ; entry->eax =KVM_CPUID_FEATURES ;
Paolo Bonzini
maintainerRe: [PATCH 19/19] kvm: x86: Add AMX CPUIDs support
arch/x86/kvm/cpuid.c static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) · line 922
Quoted 7 lines · Yang Zhong
arch/x86/kvm/cpuid.c +2
-1@@ -914,7 +918,8 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)break ;/* Intel PT */ case 0x14 : -if (!kvm_cpu_cap_has (X86_FEATURE_INTEL_PT )) { +if ((function ==0x14 && !kvm_cpu_cap_has (X86_FEATURE_INTEL_PT )) || + (function ==0x1d && !kvm_cpu_cap_has (X86_FEATURE_AMX_TILE ))) {
This hunk is wrong.
arch/x86/kvm/cpuid.c static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) · line 936
Quoted 12 lines
arch/x86/kvm/cpuid.c +5
-0entry->eax = entry->ebx = entry->ecx = entry->edx =0 ;break ; }@@ -924,6 +929,11 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) goto out; }break ; +/* Intel AMX TILE */ +case 0x1d : +if (!kvm_cpu_cap_has (X86_FEATURE_AMX_TILE )) + entry->eax = entry->ebx = entry->ecx = entry->edx =0 ; +break ;
This also needs a loop similar to the one in case 0x14; so the "break" goes inside the "if" and then you have
for (i = 1, max_idx = entry->eax; i <= max_idx; ++i) {
if (!do_host_cpuid(array, function, i))
goto out;
}Same for 0x1e, which also needs to be marked conditional.
Paolo
Jing, Yang,
Quoted 3 lines · Yang Zhong
Thanks Thomas for the thoughts and patches on the KVM FPU and AMX support.
welcome. The overall impression of that series is that it is heading into the right direction. Keep up the good work!
Thanks,
tglx