x86: Add CPUID KVM support for new instruction WBNOINVD
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): CALMp9eRZCoZbeyttZdvaCUpOFKygTNVF_x7+TWh6MktmF-ZK9A@mail.gmail.com
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 41 lines · Paolo Bonzini
Quoted 2 lines · Jim Mattson
Is this an instruction that kvm has to be able to emulate before it can enumerate its existence?
It doesn't have any operands, so no.
Paolo
Quoted 32 lines · Robert Hoo
Quoted 31 lines
Signed-off-by Robert Hoo <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)arch/x86/include/asm/cpufeatures.h +1
-0@@ -280,6 +280,7 @@/* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */ #define X86_FEATURE_IRPERF (13*32+ 1)/* Instructions Retired Count */ +#define X86_FEATURE_WBNOINVD (13*32+ 9)/* Writeback and Don't invalid cache */ #define X86_FEATURE_XSAVEERPTR (13*32+ 2)/* Always save/restore FP error pointers */ #define X86_FEATURE_AMD_IBPB (13*32+12)/* "" Indirect Branch Prediction Barrier */ #define X86_FEATURE_AMD_IBRS (13*32+14)/* "" Indirect Branch Restricted Speculation */ arch/x86/kvm/cpuid.c +1
-1@@ -380,7 +380,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,/* cpuid 0x80000008.ebx */ const u32 kvm_cpuid_8000_0008_ebx_x86_features = -F (AMD_IBPB ) |F (AMD_IBRS ) |F (AMD_SSBD ) |F (VIRT_SSBD ) | +F (WBNOINVD ) |F (AMD_IBPB ) |F (AMD_IBRS ) |F (AMD_SSBD ) |F (VIRT_SSBD ) |F (AMD_SSB_NO ) |F (AMD_STIBP );
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
arch/x86/kvm/cpuid.c +0
-1/* cpuid 0xC0000001.edx */ --1.8.3.1
What is the point of enumerating support for WBNOINVD if kvm is going to implement it as WBINVD?
CCing qemu-devel.
Quoted 42 lines · Jim Mattson
Quoted 41 lines · Paolo Bonzini
Quoted 2 lines · Jim Mattson
Is this an instruction that kvm has to be able to emulate before it can enumerate its existence?
It doesn't have any operands, so no.
Paolo
Quoted 32 lines · Robert Hoo
Quoted 31 lines
Signed-off-by Robert Hoo <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)arch/x86/include/asm/cpufeatures.h +1
-0@@ -280,6 +280,7 @@/* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */ #define X86_FEATURE_IRPERF (13*32+ 1)/* Instructions Retired Count */ +#define X86_FEATURE_WBNOINVD (13*32+ 9)/* Writeback and Don't invalid cache */ #define X86_FEATURE_XSAVEERPTR (13*32+ 2)/* Always save/restore FP error pointers */ #define X86_FEATURE_AMD_IBPB (13*32+12)/* "" Indirect Branch Prediction Barrier */ #define X86_FEATURE_AMD_IBRS (13*32+14)/* "" Indirect Branch Restricted Speculation */ arch/x86/kvm/cpuid.c +1
-1@@ -380,7 +380,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,/* cpuid 0x80000008.ebx */ const u32 kvm_cpuid_8000_0008_ebx_x86_features = -F (AMD_IBPB ) |F (AMD_IBRS ) |F (AMD_SSBD ) |F (VIRT_SSBD ) | +F (WBNOINVD ) |F (AMD_IBPB ) |F (AMD_IBRS ) |F (AMD_SSBD ) |F (VIRT_SSBD ) |F (AMD_SSB_NO ) |F (AMD_STIBP );
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
arch/x86/kvm/cpuid.c +0
-1/* cpuid 0xC0000001.edx */ --1.8.3.1
What is the point of enumerating support for WBNOINVD if kvm is going to implement it as WBINVD?
I expect GET_SUPPORTED_CPUID to return WBNOINVD, because it indicates to userspace what is supported by KVM. Are there any expectations that GET_SUPPORTED_CPUID will also dictate what is enabled by default in some cases?
In either case, your question applies to QEMU: why do we want WBNOINVD to be enabled by "-cpu host" by default and be part of QEMU's Icelake-* CPU model definitions?
Signature · 1 line
Eduardo
Quoted 54 lines · Eduardo Habkost
CCing qemu-devel.
Quoted 42 lines · Jim Mattson
Quoted 41 lines · Paolo Bonzini
Quoted 2 lines · Jim Mattson
Is this an instruction that kvm has to be able to emulate before it can enumerate its existence?
It doesn't have any operands, so no.
Paolo
Quoted 32 lines · Robert Hoo
Quoted 31 lines
Signed-off-by Robert Hoo <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)arch/x86/include/asm/cpufeatures.h +1
-0@@ -280,6 +280,7 @@/* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */ #define X86_FEATURE_IRPERF (13*32+ 1)/* Instructions Retired Count */ +#define X86_FEATURE_WBNOINVD (13*32+ 9)/* Writeback and Don't invalid cache */ #define X86_FEATURE_XSAVEERPTR (13*32+ 2)/* Always save/restore FP error pointers */ #define X86_FEATURE_AMD_IBPB (13*32+12)/* "" Indirect Branch Prediction Barrier */ #define X86_FEATURE_AMD_IBRS (13*32+14)/* "" Indirect Branch Restricted Speculation */ arch/x86/kvm/cpuid.c +1
-1@@ -380,7 +380,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,/* cpuid 0x80000008.ebx */ const u32 kvm_cpuid_8000_0008_ebx_x86_features = -F (AMD_IBPB ) |F (AMD_IBRS ) |F (AMD_SSBD ) |F (VIRT_SSBD ) | +F (WBNOINVD ) |F (AMD_IBPB ) |F (AMD_IBRS ) |F (AMD_SSBD ) |F (VIRT_SSBD ) |F (AMD_SSB_NO ) |F (AMD_STIBP );Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
arch/x86/kvm/cpuid.c +0
-1/* cpuid 0xC0000001.edx */ --1.8.3.1
What is the point of enumerating support for WBNOINVD if kvm is going to implement it as WBINVD?
Quoted 8 lines
I expect
GET_SUPPORTED_CPUIDto return WBNOINVD, because it indicates to userspace what is supported by KVM. Are there any expectations thatGET_SUPPORTED_CPUIDwill also dictate what is enabled by default in some cases?In either case, your question applies to QEMU: why do we want WBNOINVD to be enabled by "-cpu host" by default and be part of QEMU's Icelake-* CPU model definitions?
I had only looked at the SVM implementation of WBNOINVD, which is exactly the same as the SVM implementation of WBINVD. So, the question is, "why enumerate WBNOINVD if its implementation is exactly the same as WBINVD?"
WBNOINVD appears to be only partially documented in Intel document 319433-037, "Intel® Architecture Instruction Set Extensions and Future Features Programming Reference." In particular, there is no documentation regarding the instruction's behavior in VMX non-root mode. Does WBNOINVD cause a VM-exit when the VM-execution control, "WBINVD exiting," is set? If so, does it have the same VM-exit reason as WBINVD (54), or a different one? If it does have the same VM-exit reason (a la SVM), how does one distinguish a WBINVD VM-exit from a WBNOINVD VM-exit? If one can't distinguish (a la SVM), then it would seem that the VMX implementation also implements WBNOINVD as WBINVD. If that's the case, the question for VMX is the same as for SVM.
Quoted 78 lines · Jim Mattson
Quoted 63 lines · Eduardo Habkost
CCing qemu-devel.
Quoted 52 lines · Jim Mattson
Quoted 51 lines · Paolo Bonzini
Quoted 2 lines · Jim Mattson
Is this an instruction that kvm has to be able to emulate before it can enumerate its existence?
It doesn't have any operands, so no.
Paolo
Quoted 43 lines
Quoted 41 lines · Robert Hoo
Signed-off-by Robert Hoo <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)arch/x86/include/asm/cpufeatures.h +0
-0@@ -280,6 +280,7 @@/* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */
#defineX86_FEATURE_IRPERF(13*32+ 1) /* Instructions
Retired Count */
+#defineX86_FEATURE_WBNOINVD(13*32+ 9) /* Writeback and
Don't invalid cache */
#defineX86_FEATURE_XSAVEERPTR(13*32+ 2) /* Always
save/restore FP error pointers */
#defineX86_FEATURE_AMD_IBPB(13*32+12) /* "" Indirect
Branch Prediction Barrier */
#defineX86_FEATURE_AMD_IBRS(13*32+14) /* "" Indirect
Branch Restricted Speculation */arch/x86/kvm/cpuid.c +0
-1@@ -380,7 +380,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,/* cpuid 0x80000008.ebx */ const u32 kvm_cpuid_8000_0008_ebx_x86_features = - F(AMD_IBPB) | F(AMD_IBRS) | F(AMD_SSBD) |
F(VIRT_SSBD)|
+ F(WBNOINVD) |F(AMD_IBPB)|F(AMD_IBRS)|F(AMD_SSBD)|F(VIRT_SSBD)|F(AMD_SSB_NO) | F(AMD_STIBP);Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
/* cpuid 0xC0000001.edx */--
1.8.3.1What is the point of enumerating support for WBNOINVD if kvm is going to implement it as WBINVD?
Quoted 8 lines
I expect
GET_SUPPORTED_CPUIDto return WBNOINVD, because it indicates to userspace what is supported by KVM. Are there any expectations thatGET_SUPPORTED_CPUIDwill also dictate what is enabled by default in some cases?In either case, your question applies to QEMU: why do we want WBNOINVD to be enabled by "-cpu host" by default and be part of QEMU's Icelake-* CPU model definitions?
I had only looked at the SVM implementation of WBNOINVD, which is exactly the same as the SVM implementation of WBINVD. So, the question is, "why enumerate WBNOINVD if its implementation is exactly the same as WBINVD?"
Quoted 11 lines
WBNOINVD appears to be only partially documented in Intel document 319433-037, "Intel® Architecture Instruction Set Extensions and Future Features Programming Reference." In particular, there is no documentation regarding the instruction's behavior in VMX non-root mode. Does WBNOINVD cause a VM-exit when the VM-execution control, "WBINVD exiting," is set? If so, does it have the same VM-exit reason as WBINVD (54), or a different one? If it does have the same VM-exit reason (a la SVM), how does one distinguish a WBINVD VM-exit from a WBNOINVD VM-exit? If one can't distinguish (a la SVM), then it would seem that the VMX implementation also implements WBNOINVD as WBINVD. If that's the case, the question for VMX is the same as for SVM.
Unfortunately WBNOINVD interaction with VMX has not been made to public yet. I am reaching out internally to see when it can be done. I agree it may not be necessary to expose WBNOINVD if its implementation is exactly the same as WBINVD, but it also doesn't have any harm, right?
Thanks,
-Kai
Quoted 92 lines · Kai
Quoted 78 lines · Jim Mattson
Quoted 63 lines · Eduardo Habkost
CCing qemu-devel.
Quoted 52 lines · Jim Mattson
Quoted 51 lines · Paolo Bonzini
Quoted 2 lines · Jim Mattson
Is this an instruction that kvm has to be able to emulate before it can enumerate its existence?
It doesn't have any operands, so no.
Paolo
Quoted 43 lines
Quoted 41 lines · Robert Hoo
Signed-off-by Robert Hoo <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kvm/cpuid.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)arch/x86/include/asm/cpufeatures.h +0
-0@@ -280,6 +280,7 @@/* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */
#defineX86_FEATURE_IRPERF(13*32+ 1) /* Instructions
Retired Count */
+#defineX86_FEATURE_WBNOINVD(13*32+ 9) /* Writeback and
Don't invalid cache */
#defineX86_FEATURE_XSAVEERPTR(13*32+ 2) /* Always
save/restore FP error pointers */
#defineX86_FEATURE_AMD_IBPB(13*32+12) /* "" Indirect
Branch Prediction Barrier */
#defineX86_FEATURE_AMD_IBRS(13*32+14) /* "" Indirect
Branch Restricted Speculation */arch/x86/kvm/cpuid.c +0
-1@@ -380,7 +380,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,/* cpuid 0x80000008.ebx */ const u32 kvm_cpuid_8000_0008_ebx_x86_features = - F(AMD_IBPB) | F(AMD_IBRS) | F(AMD_SSBD) |
F(VIRT_SSBD)|
+ F(WBNOINVD) |F(AMD_IBPB)|F(AMD_IBRS)|F(AMD_SSBD)|F(VIRT_SSBD)|F(AMD_SSB_NO) | F(AMD_STIBP);Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
/* cpuid 0xC0000001.edx */--
1.8.3.1What is the point of enumerating support for WBNOINVD if kvm is going to implement it as WBINVD?
Quoted 8 lines
I expect
GET_SUPPORTED_CPUIDto return WBNOINVD, because it indicates to userspace what is supported by KVM. Are there any expectations thatGET_SUPPORTED_CPUIDwill also dictate what is enabled by default in some cases?In either case, your question applies to QEMU: why do we want WBNOINVD to be enabled by "-cpu host" by default and be part of QEMU's Icelake-* CPU model definitions?
I had only looked at the SVM implementation of WBNOINVD, which is exactly the same as the SVM implementation of WBINVD. So, the question is, "why enumerate WBNOINVD if its implementation is exactly the same as WBINVD?"
Quoted 11 lines
WBNOINVD appears to be only partially documented in Intel document 319433-037, "Intel® Architecture Instruction Set Extensions and Future Features Programming Reference." In particular, there is no documentation regarding the instruction's behavior in VMX non-root mode. Does WBNOINVD cause a VM-exit when the VM-execution control, "WBINVD exiting," is set? If so, does it have the same VM-exit reason as WBINVD (54), or a different one? If it does have the same VM-exit reason (a la SVM), how does one distinguish a WBINVD VM-exit from a WBNOINVD VM-exit? If one can't distinguish (a la SVM), then it would seem that the VMX implementation also implements WBNOINVD as WBINVD. If that's the case, the question for VMX is the same as for SVM.
Quoted 4 lines
Unfortunately WBNOINVD interaction with VMX has not been made to public yet. I am reaching out internally to see when it can be done. I agree it may not be necessary to expose WBNOINVD if its implementation is exactly the same as WBINVD, but it also doesn't have any harm, right?
If nested VMX changes are necessary to be consistent with hardware, then enumerating WBNOINVD support in the guest CPUID information at this time--without the attendant nested VMX changes--is premature. No changes to nested SVM are necessary, so it's fine for AMD systems.
If no changes to nested VMX are necessary, then it is true that WBNOINVD can be emulated by WBINVD. However, it provides no value to specifically enumerate the instruction.
If there is some value that I'm missing, then why make guest support for the instruction contingent on host support for the instruction? KVM can implement WBNOINVD as WBINVD on any host with WBINVD, regardless of whether or not the host supports WBNOINVD.
Quoted 2 lines
Thanks,
-Kai
Quoted 21 lines · Jim Mattson
Quoted 20 lines · Kai
Quoted 17 lines · Jim Mattson
On Mon, Sep 30, 2019 at 10:54 AM Eduardo Habkost <[email protected]> wrote:
I had only looked at the SVM implementation of WBNOINVD, which is exactly the same as the SVM implementation of WBINVD. So, the question is, "why enumerate WBNOINVD if its implementation is exactly the same as WBINVD?"WBNOINVD appears to be only partially documented in Intel document 319433-037, "Intel® Architecture Instruction Set Extensions and Future Features Programming Reference." In particular, there is no documentation regarding the instruction's behavior in VMX non-root mode. Does WBNOINVD cause a VM-exit when the VM-execution control, "WBINVD exiting," is set? If so, does it have the same VM-exit reason as WBINVD (54), or a different one? If it does have the same VM-exit reason (a la SVM), how does one distinguish a WBINVD VM-exit from a WBNOINVD VM-exit? If one can't distinguish (a la SVM), then it would seem that the VMX implementation also implements WBNOINVD as WBINVD. If that's the case, the question for VMX is the same as for SVM.
Quoted 1 line
Quoted 1 line
Unfortunately WBNOINVD interaction with VMX has not been made to public yet.
Hint: WBNOINVD uses a previously ignored prefix, i.e. it looks a *lot*
like WBINVD...
Quoted 9 lines
Quoted 3 lines
I am reaching out internally to see when it can be done. I agree it may not be necessary to expose WBNOINVD if its implementation is exactly the same as WBINVD, but it also doesn't have any harm, right?
If nested VMX changes are necessary to be consistent with hardware, then enumerating WBNOINVD support in the guest CPUID information at this time--without the attendant nested VMX changes--is premature. No changes to nested SVM are necessary, so it's fine for AMD systems.
Quoted 8 lines
If no changes to nested VMX are necessary, then it is true that WBNOINVD can be emulated by WBINVD. However, it provides no value to specifically enumerate the instruction.
If there is some value that I'm missing, then why make guest support for the instruction contingent on host support for the instruction? KVM can implement WBNOINVD as WBINVD on any host with WBINVD, regardless of whether or not the host supports WBNOINVD.
Agreed. To play nice with live migration, KVM should enumerate WBNOINVD regardless of host support. Since WBNOINVD uses an ignored prefix, it will simply look like a regular WBINVD on platforms without WBNOINVD.
Let's assume the WBNOINVD VM-Exit behavior is sane, i.e. allows software to easily differentiate between WBINVD and WBNOINVD. In that case, the value added would be that KVM can do WBNOINVD instead of WBINVD in the unlikely event that (a) KVM needs to executed WBINVD on behalf of the guest (because the guest has non-coherent DMA), (b) WBNOINVD is supported on the host, and (c) WBNOINVD is used by the guest (I don't think it would be safe to assume that the guest doesn't need the caches invalidated on WBINVD).
Quoted 23 lines · Sean Christopherson
Quoted 21 lines · Jim Mattson
Quoted 20 lines · Kai
Quoted 17 lines · Jim Mattson
On Mon, Sep 30, 2019 at 10:54 AM Eduardo Habkost <[email protected]> wrote:
I had only looked at the SVM implementation of WBNOINVD, which is exactly the same as the SVM implementation of WBINVD. So, the question is, "why enumerate WBNOINVD if its implementation is exactly the same as WBINVD?"WBNOINVD appears to be only partially documented in Intel document 319433-037, "Intel® Architecture Instruction Set Extensions and Future Features Programming Reference." In particular, there is no documentation regarding the instruction's behavior in VMX non-root mode. Does WBNOINVD cause a VM-exit when the VM-execution control, "WBINVD exiting," is set? If so, does it have the same VM-exit reason as WBINVD (54), or a different one? If it does have the same VM-exit reason (a la SVM), how does one distinguish a WBINVD VM-exit from a WBNOINVD VM-exit? If one can't distinguish (a la SVM), then it would seem that the VMX implementation also implements WBNOINVD as WBINVD. If that's the case, the question for VMX is the same as for SVM.
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Unfortunately WBNOINVD interaction with VMX has not been made to public yet.
Hint: WBNOINVD uses a previously ignored prefix, i.e. it looks a *lot*
like WBINVD...
Because of the opcode selection, I would assume that we're not going to see a VM-execution control for "enable WBNOINVD." To avoid breaking legacy hypervisors, then, I would expect the "enable WBINVD exiting" control to apply to WBNOINVD as well, and I would expect the exit reason to be the same for both instructions. The exit qualification field is cleared for WBINVD exits, so perhaps we will see a bit in that field set to one for WBNOINVD. If so, will this new behavior be indicated by a bit in one of the VMX capability MSRs? That seems to be a closely guarded secret, for some reason.
Quoted 18 lines
Quoted 9 lines
Quoted 3 lines
I am reaching out internally to see when it can be done. I agree it may not be necessary to expose WBNOINVD if its implementation is exactly the same as WBINVD, but it also doesn't have any harm, right?
If nested VMX changes are necessary to be consistent with hardware, then enumerating WBNOINVD support in the guest CPUID information at this time--without the attendant nested VMX changes--is premature. No changes to nested SVM are necessary, so it's fine for AMD systems.
Quoted 8 lines
If no changes to nested VMX are necessary, then it is true that WBNOINVD can be emulated by WBINVD. However, it provides no value to specifically enumerate the instruction.
If there is some value that I'm missing, then why make guest support for the instruction contingent on host support for the instruction? KVM can implement WBNOINVD as WBINVD on any host with WBINVD, regardless of whether or not the host supports WBNOINVD.
Quoted 6 lines
Agreed. To play nice with live migration, KVM should enumerate WBNOINVD regardless of host support. Since WBNOINVD uses an ignored prefix, it will simply look like a regular WBINVD on platforms without WBNOINVD.
Let's assume the WBNOINVD VM-Exit behavior is sane, i.e. allows software to easily differentiate between WBINVD and WBNOINVD.
That isn't the case with SVM, oddly.
Quoted 7 lines
In that case, the
value added would be that KVM can do WBNOINVD instead of WBINVD in the unlikely event that (a) KVM needs to executed WBINVD on behalf of the guest (because the guest has non-coherent DMA), (b) WBNOINVD is supported on the host, and (c) WBNOINVD is used by the guest (I don't think it would be safe to assume that the guest doesn't need the caches invalidated on WBINVD).
I agree that there would be value if KVM implemented WBNOINVD using WBNOINVD, but that isn't what this change does. My question was, "What is the value in enumerating WBNOINVD if KVM is just going to implement it with WBINVD anyway?"
Quoted 30 lines · Jim Mattson
Quoted 23 lines · Sean Christopherson
Quoted 21 lines · Jim Mattson
Quoted 20 lines · Kai
Quoted 17 lines · Jim Mattson
On Mon, Sep 30, 2019 at 10:54 AM Eduardo Habkost <[email protected]> wrote:
I had only looked at the SVM implementation of WBNOINVD, which is exactly the same as the SVM implementation of WBINVD. So, the question is, "why enumerate WBNOINVD if its implementation is exactly the same as WBINVD?"WBNOINVD appears to be only partially documented in Intel document 319433-037, "Intel® Architecture Instruction Set Extensions and Future Features Programming Reference." In particular, there is no documentation regarding the instruction's behavior in VMX non-root mode. Does WBNOINVD cause a VM-exit when the VM-execution control, "WBINVD exiting," is set? If so, does it have the same VM-exit reason as WBINVD (54), or a different one? If it does have the same VM-exit reason (a la SVM), how does one distinguish a WBINVD VM-exit from a WBNOINVD VM-exit? If one can't distinguish (a la SVM), then it would seem that the VMX implementation also implements WBNOINVD as WBINVD. If that's the case, the question for VMX is the same as for SVM.
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Unfortunately WBNOINVD interaction with VMX has not been made to public yet.
Hint: WBNOINVD uses a previously ignored prefix, i.e. it looks a *lot*
like WBINVD...
Quoted 7 lines
Because of the opcode selection, I would assume that we're not going to see a VM-execution control for "enable WBNOINVD." To avoid breaking legacy hypervisors, then, I would expect the "enable WBINVD exiting" control to apply to WBNOINVD as well, and I would expect the exit reason to be the same for both instructions. The exit qualification field is cleared for WBINVD exits, so perhaps we will see a bit in that field set to one for WBNOINVD.
Those are all excellent assumptions.
Quoted 2 lines
If so, will this new behavior be indicated by a bit in one of the VMX capability MSRs?
My crystal ball came up blank on this one.
Quoted 1 line
That seems to be a closely guarded secret, for some reason.
Not a closely guarded secret, just poor documentation.
Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
Quoted 3 lines
I am reaching out internally to see when it can be done. I agree it may not be necessary to expose WBNOINVD if its implementation is exactly the same as WBINVD, but it also doesn't have any harm, right?
If nested VMX changes are necessary to be consistent with hardware, then enumerating WBNOINVD support in the guest CPUID information at this time--without the attendant nested VMX changes--is premature. No changes to nested SVM are necessary, so it's fine for AMD systems.
If no changes to nested VMX are necessary, then it is true that WBNOINVD can be emulated by WBINVD. However, it provides no value to specifically enumerate the instruction.
If there is some value that I'm missing, then why make guest support for the instruction contingent on host support for the instruction? KVM can implement WBNOINVD as WBINVD on any host with WBINVD, regardless of whether or not the host supports WBNOINVD.
Quoted 8 lines
Quoted 6 lines
Agreed. To play nice with live migration, KVM should enumerate WBNOINVD regardless of host support. Since WBNOINVD uses an ignored prefix, it will simply look like a regular WBINVD on platforms without WBNOINVD.
Let's assume the WBNOINVD VM-Exit behavior is sane, i.e. allows software to easily differentiate between WBINVD and WBNOINVD.
That isn't the case with SVM, oddly.
Assuming AMD uses the same opcode as Intel, maybe they're expecting VMMs to use the decode assist feature to check for the prefix?
Quoted 8 lines
Quoted 7 lines
In that case, the
value added would be that KVM can do WBNOINVD instead of WBINVD in the unlikely event that (a) KVM needs to executed WBINVD on behalf of the guest (because the guest has non-coherent DMA), (b) WBNOINVD is supported on the host, and (c) WBNOINVD is used by the guest (I don't think it would be safe to assume that the guest doesn't need the caches invalidated on WBINVD).
Quoted 4 lines
I agree that there would be value if KVM implemented WBNOINVD using WBNOINVD, but that isn't what this change does. My question was, "What is the value in enumerating WBNOINVD if KVM is just going to implement it with WBINVD anyway?"
Ah, I was stating what I would expect the KVM change to be, I didn't realize this patch was merged almost a year ago.
I suppose theoretically it would allow live migrating from an old kernel to a new kernel and gaining actual WBNOINVD support along the way?
Quoted 73 lines · Sean Christopherson
Quoted 30 lines · Jim Mattson
Quoted 23 lines · Sean Christopherson
Quoted 21 lines · Jim Mattson
Quoted 20 lines · Kai
Quoted 17 lines · Jim Mattson
On Mon, Sep 30, 2019 at 10:54 AM Eduardo Habkost <[email protected]> wrote:
I had only looked at the SVM implementation of WBNOINVD, which is exactly the same as the SVM implementation of WBINVD. So, the question is, "why enumerate WBNOINVD if its implementation is exactly the same as WBINVD?"WBNOINVD appears to be only partially documented in Intel document 319433-037, "Intel® Architecture Instruction Set Extensions and Future Features Programming Reference." In particular, there is no documentation regarding the instruction's behavior in VMX non-root mode. Does WBNOINVD cause a VM-exit when the VM-execution control, "WBINVD exiting," is set? If so, does it have the same VM-exit reason as WBINVD (54), or a different one? If it does have the same VM-exit reason (a la SVM), how does one distinguish a WBINVD VM-exit from a WBNOINVD VM-exit? If one can't distinguish (a la SVM), then it would seem that the VMX implementation also implements WBNOINVD as WBINVD. If that's the case, the question for VMX is the same as for SVM.
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Unfortunately WBNOINVD interaction with VMX has not been made to public yet.
Hint: WBNOINVD uses a previously ignored prefix, i.e. it looks a *lot*
like WBINVD...Quoted 7 lines
Because of the opcode selection, I would assume that we're not going to see a VM-execution control for "enable WBNOINVD." To avoid breaking legacy hypervisors, then, I would expect the "enable WBINVD exiting" control to apply to WBNOINVD as well, and I would expect the exit reason to be the same for both instructions. The exit qualification field is cleared for WBINVD exits, so perhaps we will see a bit in that field set to one for WBNOINVD.
Those are all excellent assumptions.
Quoted 2 lines
If so, will this new behavior be indicated by a bit in one of the VMX capability MSRs?
My crystal ball came up blank on this one.
Quoted 1 line
That seems to be a closely guarded secret, for some reason.
Not a closely guarded secret, just poor documentation.
Quoted 22 lines
Quoted 22 lines
Quoted 9 lines
Quoted 3 lines
I am reaching out internally to see when it can be done. I agree it may not be necessary to expose WBNOINVD if its implementation is exactly the same as WBINVD, but it also doesn't have any harm, right?
If nested VMX changes are necessary to be consistent with hardware, then enumerating WBNOINVD support in the guest CPUID information at this time--without the attendant nested VMX changes--is premature. No changes to nested SVM are necessary, so it's fine for AMD systems.
Quoted 8 lines
If no changes to nested VMX are necessary, then it is true that WBNOINVD can be emulated by WBINVD. However, it provides no value to specifically enumerate the instruction.
If there is some value that I'm missing, then why make guest support for the instruction contingent on host support for the instruction? KVM can implement WBNOINVD as WBINVD on any host with WBINVD, regardless of whether or not the host supports WBNOINVD.
Agreed. To play nice with live migration, KVM should enumerate WBNOINVD regardless of host support. Since WBNOINVD uses an ignored prefix, it will simply look like a regular WBINVD on platforms without WBNOINVD.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Let's assume the WBNOINVD VM-Exit behavior is sane, i.e. allows software to easily differentiate between WBINVD and WBNOINVD.
That isn't the case with SVM, oddly.
Assuming AMD uses the same opcode as Intel, maybe they're expecting VMMs to use the decode assist feature to check for the prefix?
There are no specific decode assists for WBINVD/WBNOINVD in the EXITINFO* fields. The "generic" decode assist, where the instruction bytes are stored in the VMCB, only applies to nested page faults and #PF intercepts.
Quoted 18 lines
Quoted 8 lines
Quoted 7 lines
In that case, the
value added would be that KVM can do WBNOINVD instead of WBINVD in the unlikely event that (a) KVM needs to executed WBINVD on behalf of the guest (because the guest has non-coherent DMA), (b) WBNOINVD is supported on the host, and (c) WBNOINVD is used by the guest (I don't think it would be safe to assume that the guest doesn't need the caches invalidated on WBINVD).Quoted 4 lines
I agree that there would be value if KVM implemented WBNOINVD using WBNOINVD, but that isn't what this change does. My question was, "What is the value in enumerating WBNOINVD if KVM is just going to implement it with WBINVD anyway?"
Ah, I was stating what I would expect the KVM change to be, I didn't realize this patch was merged almost a year ago.
I suppose theoretically it would allow live migrating from an old kernel to a new kernel and gaining actual WBNOINVD support along the way?