KVM: x86: Clean up kvm_<reg>_{read,write}() mess
Subsystems (6)
Sean Christopherson
maintainerv2 cover KVM: x86: Clean up kvm_<reg>_{read,write}() mess
Add proper, explicit "raw" versions of kvm_<reg>_{read,write}(), along with "e" versions (for hardcoded 32-bit accesses), and convert the existing kvm_<reg>_{read,write}() APIs into mode-aware variants.
This was prompted by commit 435741a4e766 ("KVM: SVM: Properly check RAX on #GP intercept of SVM instructions"), where using kvm_rax_read() to get EAX/RAX would have (*very* surprisingly) been wrong as it's actually a "raw" variant that doesn't truncate accesses when the guest is in 32-bit mode.
Aside from my dislike of inconsistent APIs, I really want to avoid carrying code that's subtly relying on using kvm_register_read(...) when accessing a hardcoded register.
Fix a handful of minor warts along the way.
Oh, and introduce regs.{c,h}, which just a "minor" addendum. Yosry pointed out that moving _more_ code into x86.h was rather gross (especially since the code split was super arbitrary), and it turns out that create regs.{c,h} isn't all that hard. In the future, I think we can also add msr.{c,h}, so I very deliberately didn't include that functionality in regs.{c,h}.
v2:
- Collect tags. [Yosry, Kai
- Fix some truly egregious goofs. [Binbin]
- Rename kvm_cache_regs.h => regs.h, add regs.c. [Yosry, though he'll probably yell at me for saying this was his suggestion :-) ]
- Drop superfluous casting/masking of e*x() usage. [Kai]
v1: https://lore.kernel.org/all/[email protected]
Sean Christopherson (15):
KVM: SVM: Truncate INVLPGA address in compatibility mode
KVM: x86/xen: Bug the VM if 32-bit KVM observes a 64-bit mode
hypercall
KVM: x86/xen: Don't truncate RAX when handling hypercall from
protected guest
KVM: VMX: Read 32-bit GPR values for ENCLS instructions outside of
64-bit mode
KVM: x86: Trace hypercall register *after* truncating values for
32-bit KVM: x86: Rename kvm_cache_regs.h => regs.h
KVM: x86: Move inlined CR and DR helpers from x86.h to regs.h
KVM: x86: Add mode-aware versions of kvm_<reg>_{read,write}() helpers
KVM: x86: Drop non-raw kvm_<reg>_write() helpers
KVM: nSVM: Use kvm_rax_read() now that it's mode-aware Revert "KVM: VMX: Read 32-bit GPR values for ENCLS instructions
outside of 64-bit mode"
KVM: x86: Harden is_64_bit_hypercall() against bugs on 32-bit kernels
KVM: x86: Move update_cr8_intercept() to lapic.c
KVM: x86: Move kvm_pv_async_pf_enabled() to x86.h (as an inline)
KVM: x86: Move the bulk of register specific code from x86.c to regs.c
arch/x86/include/asm/kvm_host.h | 2 -
arch/x86/kvm/Makefile | 4 +-
arch/x86/kvm/cpuid.c | 12 +-
arch/x86/kvm/emulate.c | 2 +-
arch/x86/kvm/hyperv.c | 21 +-
arch/x86/kvm/hyperv.h | 4 +-
arch/x86/kvm/lapic.c | 28 +-
arch/x86/kvm/lapic.h | 1 +
arch/x86/kvm/mmu.h | 2 +-
arch/x86/kvm/mmu/mmu.c | 2 +-
arch/x86/kvm/regs.c | 829 +++++++++++++++++++
arch/x86/kvm/{kvm_cache_regs.h => regs.h} | 203 ++++-
arch/x86/kvm/smm.c | 2 +-
arch/x86/kvm/svm/nested.c | 8 +-
arch/x86/kvm/svm/svm.c | 19 +-
arch/x86/kvm/svm/svm.h | 2 +-
arch/x86/kvm/vmx/nested.c | 8 +-
arch/x86/kvm/vmx/nested.h | 2 +-
arch/x86/kvm/vmx/sgx.c | 6 +-
arch/x86/kvm/vmx/tdx.c | 18 +-
arch/x86/kvm/vmx/vmx.c | 2 +-
arch/x86/kvm/vmx/vmx.h | 2 +-
arch/x86/kvm/x86.c | 935 +---------------------
arch/x86/kvm/x86.h | 116 +--
arch/x86/kvm/xen.c | 39 +-
25 files changed, 1162 insertions(+), 1107 deletions(-)
create mode 100644 arch/x86/kvm/regs.c
rename arch/x86/kvm/{kvm_cache_regs.h => regs.h} (58%)base-commit: a9512a611bd030088f13477258d1f8103cceaa40
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerPATCH v2 1/15 KVM: SVM: Truncate INVLPGA address in compatibility mode
Check for full 64-bit mode, not just long mode, when truncating the virtual address as part of INVLPGA emulation. Compatibility mode doesn't support 64-bit addressing.
Note, the FIXME still applies, e.g. if the guest deliberately targeted EAX while in 64-bit via an address size override. That flaw isn't worth fixing as it would require decoding the code stream, which would open a an entirely different can of worms, and in practice no sane guest would shove garbage into RAX[63:32] and execute INVLPGA.
Note #2, VMSAVE, VMLOAD, and VMRUN all suffer from the same architectural flaw of not providing the full linear address in a VMCB exit information field, because, quoting the APM verbatim:
the linear address is available directly from the guest rAX register
(VMSAVE, VMLOAD, and VMRUN take a physical address, but they're behavior with respect to rAX is otherwise identical).
--- arch/x86/kvm/svm/svm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/svm/svm.c +1 -1
@@ -2415,7 +2415,7 @@ static int invlpga_interception(struct kvm_vcpu *vcpu)return 1 ;/* FIXME: Handle an address size prefix. */ -if (!is_long_mode (vcpu)) +if (!is_64_bit_mode (vcpu)) gva = (u32 )gva;trace_kvm_invlpga (to_svm (vcpu)->vmcb->save.rip, asid, gva);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 01/15] KVM: SVM: Truncate INVLPGA address in compatibility mode
Quoted 7 lines · Sean Christopherson
Check for full 64-bit mode, not just long mode, when truncating the virtual address as part of INVLPGA emulation. Compatibility mode doesn't support 64-bit addressing.
Note, the FIXME still applies, e.g. if the guest deliberately targeted EAX while in 64-bit via an address size override. That flaw isn't worth fixing as it would require decoding the code stream, which would open a
^
an extra 'a'
Quoted 10 lines
an entirely different can of worms, and in practice no sane guest would shove garbage into RAX[63:32] and execute INVLPGA.
Note #2, VMSAVE, VMLOAD, and VMRUN all suffer from the same architectural flaw of not providing the full linear address in a VMCB exit information field, because, quoting the APM verbatim:
the linear address is available directly from the guest rAX register
(VMSAVE, VMLOAD, and VMRUN take a physical address, but they're behavior
^
their > with respect to rAX is otherwise identical).
Quoted 4 lines
Fixes bc9eff67fc35 ("KVM: SVM: Use default rAX size for INVLPGA emulation")Reviewed-by Yosry Ahmed <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 17 lines
--- arch/x86/kvm/svm/svm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kvm/svm/svm.c +1
-1@@ -2415,7 +2415,7 @@ static int invlpga_interception(struct kvm_vcpu *vcpu)return 1 ;/* FIXME: Handle an address size prefix. */ -if (!is_long_mode (vcpu)) +if (!is_64_bit_mode (vcpu)) gva = (u32 )gva;trace_kvm_invlpga (to_svm (vcpu)->vmcb->save.rip, asid, gva);
Sean Christopherson
maintainerPATCH v2 2/15 KVM: x86/xen: Bug the VM if 32-bit KVM observes a 64-bit mode hypercall
In reply to Sean ChristophersonBug the VM if 32-bit KVM attempts to handle a 64-bit hypercall, primarily so that a future change to set "input" in mode-specific code doesn't trigger a false positive warn=>error:
arch/x86/kvm/xen.c:1687:6: error: variable 'input' is used uninitialized
whenever 'if' condition is false [-Werror,-Wsometimes-uninitialized]
1687 | if (!longmode) {
| ^~~~~~~~~
arch/x86/kvm/xen.c:1708:31: note: uninitialized use occurs here
1708 | trace_kvm_xen_hypercall(cpl, input, params[0], params[1], params[2],
| ^~~~~
x86/kvm/xen.c:1687:2: note: remove the 'if' if its condition is always true
1687 | if (!longmode) {
| ^~~~~~~~~~~~~~ arch/x86/kvm/xen.c:1677:11: note: initialize the variable 'input' to silence this warning
1677 | u64 input, params[6], r = -ENOSYS;
| ^
1 error generated.
Note, params[] also has the same flaw, but -Wsometimes-uninitialized doesn't seem to be enforced for arrays, presumably because it's difficult to avoid false positives on specific entries.
--- arch/x86/kvm/xen.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
arch/x86/kvm/xen.c +5 -2
@@ -1694,16 +1694,19 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) params[4 ] = (u32 )kvm_rdi_read (vcpu); params[5 ] = (u32 )kvm_rbp_read (vcpu); } -#ifdef CONFIG_X86_64 else { +#ifdef CONFIG_X86_64 params[0 ] = (u64 )kvm_rdi_read (vcpu); params[1 ] = (u64 )kvm_rsi_read (vcpu); params[2 ] = (u64 )kvm_rdx_read (vcpu); params[3 ] = (u64 )kvm_r10_read (vcpu); params[4 ] = (u64 )kvm_r8_read (vcpu); params[5 ] = (u64 )kvm_r9_read (vcpu); - } +#else +KVM_BUG_ON (1 , vcpu->kvm); +return -EIO ;#endif + } cpl =kvm_x86_call (get_cpl)(vcpu);trace_kvm_xen_hypercall (cpl, input, params[0 ], params[1 ], params[2 ], params[3 ], params[4 ], params[5 ]);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 02/15] KVM: x86/xen: Bug the VM if 32-bit KVM observes a 64-bit mode hypercall
Quoted 19 lines · Sean Christopherson
Bug the VM if 32-bit KVM attempts to handle a 64-bit hypercall, primarily so that a future change to set "input" in mode-specific code doesn't trigger a false positive warn=>error:
arch/x86/kvm/xen.c:1687:6: error: variable 'input' is used uninitialized
whenever 'if' condition is false [-Werror,-Wsometimes-uninitialized]1687 | if (!longmode) { | ^~~~~~~~~ arch/x86/kvm/xen.c:1708:31: note: uninitialized use occurs here 1708 | trace_kvm_xen_hypercall(cpl, input, params[0], params[1], params[2], | ^~~~~ x86/kvm/xen.c:1687:2: note: remove the 'if' if its condition is always true 1687 | if (!longmode) { | ^~~~~~~~~~~~~~
arch/x86/kvm/xen.c:1677:11: note: initialize the variable 'input' to silence this warning
1677 | u64 input, params[6], r =-ENOSYS;
| ^
1 error generated.
Quoted 5 lines
Note, params[] also has the same flaw, but -Wsometimes-uninitialized doesn't seem to be enforced for arrays, presumably because it's difficult to avoid false positives on specific entries.
Signed-off-by Sean Christopherson <[email protected]>
Quoted 30 lines
--- arch/x86/kvm/xen.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)arch/x86/kvm/xen.c +5
-2@@ -1694,16 +1694,19 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) params[4 ] = (u32 )kvm_rdi_read (vcpu); params[5 ] = (u32 )kvm_rbp_read (vcpu); } -#ifdef CONFIG_X86_64 else { +#ifdef CONFIG_X86_64 params[0 ] = (u64 )kvm_rdi_read (vcpu); params[1 ] = (u64 )kvm_rsi_read (vcpu); params[2 ] = (u64 )kvm_rdx_read (vcpu); params[3 ] = (u64 )kvm_r10_read (vcpu); params[4 ] = (u64 )kvm_r8_read (vcpu); params[5 ] = (u64 )kvm_r9_read (vcpu); - } +#else +KVM_BUG_ON (1 , vcpu->kvm); +return -EIO ;#endif + } cpl =kvm_x86_call (get_cpl)(vcpu);trace_kvm_xen_hypercall (cpl, input, params[0 ], params[1 ], params[2 ], params[3 ], params[4 ], params[5 ]);
Sean Christopherson
maintainerPATCH v2 3/15 KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
In reply to Sean ChristophersonDon't truncate RAX when handling a Xen hypercall for a guest with protected state, as KVM's ABI is to assume the guest is in 64-bit for such cases (the guest leaving garbage in 63:32 after a transition to 32-bit mode is far less likely than 63:32 being necessary to complete the hypercall).
--- arch/x86/kvm/xen.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
arch/x86/kvm/xen.c +3 -3
@@ -1678,15 +1678,14 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu)bool handled = false;u8 cpl; - input = (u64 )kvm_register_read (vcpu,VCPU_REGS_RAX ); -/* Hyper-V hypercalls get bit 31 set in EAX */ -if ((input &0x80000000 ) && +if ((kvm_rax_read (vcpu) &0x80000000 ) &&kvm_hv_hypercall_enabled (vcpu))return kvm_hv_hypercall (vcpu); longmode =is_64_bit_hypercall (vcpu);if (!longmode) { + input = (u32 )kvm_rax_read (vcpu); params[0 ] = (u32 )kvm_rbx_read (vcpu); params[1 ] = (u32 )kvm_rcx_read (vcpu); params[2 ] = (u32 )kvm_rdx_read (vcpu);
@@ -1696,6 +1695,7 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) }else {#ifdef CONFIG_X86_64 + input = (u64 )kvm_rax_read (vcpu); params[0 ] = (u64 )kvm_rdi_read (vcpu); params[1 ] = (u64 )kvm_rsi_read (vcpu); params[2 ] = (u64 )kvm_rdx_read (vcpu);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
Quoted 7 lines · Sean Christopherson
Don't truncate RAX when handling a Xen hypercall for a guest with protected state, as KVM's ABI is to assume the guest is in 64-bit for such cases (the guest leaving garbage in 63:32 after a transition to 32-bit mode is far less likely than 63:32 being necessary to complete the hypercall).
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")Signed-off-by Sean Christopherson <[email protected]>
The patch looks good to me, but one question below.
Quoted 11 lines
--- arch/x86/kvm/xen.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)arch/x86/kvm/xen.c +0
-0@@ -1678,15 +1678,14 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu)bool handled = false;u8 cpl;
arch/x86/kvm/xen.c int kvm_xen_hypercall(struct kvm_vcpu *vcpu) · line 1686
Quoted 10 lines
arch/x86/kvm/xen.c +1
-3- input = (u64 )kvm_register_read (vcpu,VCPU_REGS_RAX ); -/* Hyper-V hypercalls get bit 31 set in EAX */ -if ((input &0x80000000 ) && +if ((kvm_rax_read (vcpu) &0x80000000 ) &&kvm_hv_hypercall_enabled (vcpu))return kvm_hv_hypercall (vcpu); longmode =is_64_bit_hypercall (vcpu);
Is the variable name misleading?
If the vcpu is in compatible mode (when guest state is not protected), it's in long mode, but the code goes to !longmode path.
Quoted 13 lines
arch/x86/kvm/xen.c +2
-0if (!longmode) { + input = (u32 )kvm_rax_read (vcpu); params[0 ] = (u32 )kvm_rbx_read (vcpu); params[1 ] = (u32 )kvm_rcx_read (vcpu); params[2 ] = (u32 )kvm_rdx_read (vcpu);@@ -1696,6 +1695,7 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) }else {#ifdef CONFIG_X86_64 + input = (u64 )kvm_rax_read (vcpu); params[0 ] = (u64 )kvm_rdi_read (vcpu); params[1 ] = (u64 )kvm_rsi_read (vcpu); params[2 ] = (u64 )kvm_rdx_read (vcpu);
Sean Christopherson
maintainerRe: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
Quoted 33 lines · Binbin Wu
Quoted 7 lines · Sean Christopherson
Don't truncate RAX when handling a Xen hypercall for a guest with protected state, as KVM's ABI is to assume the guest is in 64-bit for such cases (the guest leaving garbage in 63:32 after a transition to 32-bit mode is far less likely than 63:32 being necessary to complete the hypercall).
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")Signed-off-by Sean Christopherson <[email protected]>The patch looks good to me, but one question below.
Quoted 20 lines
--- arch/x86/kvm/xen.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)arch/x86/kvm/xen.c +1
-3@@ -1678,15 +1678,14 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu)bool handled = false;u8 cpl; - input = (u64 )kvm_register_read (vcpu,VCPU_REGS_RAX ); -/* Hyper-V hypercalls get bit 31 set in EAX */ -if ((input &0x80000000 ) && +if ((kvm_rax_read (vcpu) &0x80000000 ) &&kvm_hv_hypercall_enabled (vcpu))return kvm_hv_hypercall (vcpu);
Quoted 3 lines
arch/x86/kvm/xen.c int kvm_xen_hypercall(struct kvm_vcpu *vcpu) · line 1686
Quoted 1 line
arch/x86/kvm/xen.c +0
-0longmode =is_64_bit_hypercall (vcpu);Is the variable name misleading?
It most definitely is. However, @longmode is passed around quite a few locations in xen.c, and so I don't want to opportunistically fix this one variable. Though I'm definitely not opposed to a separate patch to rename them all to is_64bit or something.
Quoted 17 lines
If the vcpu is in compatible mode (when guest state is not protected), it's in long mode, but the code goes to !longmode path.
Quoted 13 lines
arch/x86/kvm/xen.c +2
-0if (!longmode) { + input = (u32 )kvm_rax_read (vcpu); params[0 ] = (u32 )kvm_rbx_read (vcpu); params[1 ] = (u32 )kvm_rcx_read (vcpu); params[2 ] = (u32 )kvm_rdx_read (vcpu);@@ -1696,6 +1695,7 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) }else {#ifdef CONFIG_X86_64 + input = (u64 )kvm_rax_read (vcpu); params[0 ] = (u64 )kvm_rdi_read (vcpu); params[1 ] = (u64 )kvm_rsi_read (vcpu); params[2 ] = (u64 )kvm_rdx_read (vcpu);
Re: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
Quoted 34 lines · Sean Christopherson
Quoted 33 lines · Binbin Wu
Quoted 7 lines · Sean Christopherson
Don't truncate RAX when handling a Xen hypercall for a guest with protected state, as KVM's ABI is to assume the guest is in 64-bit for such cases (the guest leaving garbage in 63:32 after a transition to 32-bit mode is far less likely than 63:32 being necessary to complete the hypercall).
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")Signed-off-by Sean Christopherson <[email protected]>The patch looks good to me, but one question below.
Quoted 20 lines
--- arch/x86/kvm/xen.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)arch/x86/kvm/xen.c +1
-3@@ -1678,15 +1678,14 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu)bool handled = false;u8 cpl; - input = (u64 )kvm_register_read (vcpu,VCPU_REGS_RAX ); -/* Hyper-V hypercalls get bit 31 set in EAX */ -if ((input &0x80000000 ) && +if ((kvm_rax_read (vcpu) &0x80000000 ) &&kvm_hv_hypercall_enabled (vcpu))return kvm_hv_hypercall (vcpu);
Quoted 8 lines
Quoted 3 lines
arch/x86/kvm/xen.c int kvm_xen_hypercall(struct kvm_vcpu *vcpu) · line 1686
Quoted 1 line
arch/x86/kvm/xen.c +0
-0longmode =is_64_bit_hypercall (vcpu);Is the variable name misleading?
It most definitely is. However, @longmode is passed around quite a few locations in
xen.c, and so I don't want to opportunistically fix this one variable. Though I'm definitely not opposed to a separate patch to rename them all tois_64bitor something.
OK, I can do it.
Quoted 19 lines
Quoted 18 lines
If the vcpu is in compatible mode (when guest state is not protected), it's in long mode, but the code goes to !longmode path.
Quoted 13 lines
arch/x86/kvm/xen.c +2
-0if (!longmode) { + input = (u32 )kvm_rax_read (vcpu); params[0 ] = (u32 )kvm_rbx_read (vcpu); params[1 ] = (u32 )kvm_rcx_read (vcpu); params[2 ] = (u32 )kvm_rdx_read (vcpu);@@ -1696,6 +1695,7 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) }else {#ifdef CONFIG_X86_64 + input = (u64 )kvm_rax_read (vcpu); params[0 ] = (u64 )kvm_rdi_read (vcpu); params[1 ] = (u64 )kvm_rsi_read (vcpu); params[2 ] = (u64 )kvm_rdx_read (vcpu);
David Woodhouse
maintainerRe: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
Quoted 11 lines · Binbin Wu
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
longmode =
is_64_bit_hypercall(vcpu);Is the variable name misleading?
It most definitely is. However, @longmode is passed around quite a few locations in
xen.c, and so I don't want to opportunistically fix this one variable. Though I'm definitely not opposed to a separate patch to rename them all tois_64bitor something.OK, I can do it.
This one (as shown above) is clearly indicating whether this particular vCPU is in 64-bit mode for this particular hypercall. Changing that to is_64bit makes sense.
However, there is a separate overall mode for the VM, which is stored in 'kvm->arch.xen.long_mode' and accessed by userspace using the KVM_XEN_ATTR_TYPE_LONG_MODE attribute. It affects the datatypes used by shared memory data structures, and is also latched by the kernel when the guest writes the MSR for the hypercall page. That one should probably keep its name.
Re: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
Quoted 17 lines · David Woodhouse
Quoted 11 lines · Binbin Wu
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
longmode =
is_64_bit_hypercall(vcpu);Is the variable name misleading?
It most definitely is. However, @longmode is passed around quite a few locations in
xen.c, and so I don't want to opportunistically fix this one variable. Though I'm definitely not opposed to a separate patch to rename them all tois_64bitor something.OK, I can do it.
This one (as shown above) is clearly indicating whether this particular vCPU is in 64-bit mode for this particular hypercall. Changing that to
is_64bitmakes sense.
Quoted 6 lines
However, there is a separate overall mode for the VM, which is stored in '
kvm->arch.xen.long_mode' and accessed by userspace using theKVM_XEN_ATTR_TYPE_LONG_MODEattribute. It affects the datatypes used by shared memory data structures, and is also latched by the kernel when the guest writes the MSR for the hypercall page. That one should probably keep its name.
For this one, I think the current KVM code is consistent.
The format is determined by EFER.LMA, whether the guest is running in 64 bit or compatible mode doesn't change the ABI.
struct compat_shared_info is used only when the guest is running natively in a 32-bit build.
Quoted 2 lines
David Woodhouse
maintainerRe: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
Quoted 27 lines · Binbin Wu
Quoted 17 lines · David Woodhouse
Quoted 11 lines · Binbin Wu
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
longmode =
is_64_bit_hypercall(vcpu);Is the variable name misleading?
It most definitely is. However, @longmode is passed around quite a few locations in
xen.c, and so I don't want to opportunistically fix this one variable. Though I'm definitely not opposed to a separate patch to rename them all tois_64bitor something.OK, I can do it.
This one (as shown above) is clearly indicating whether this particular vCPU is in 64-bit mode for this particular hypercall. Changing that to
is_64bitmakes sense.Quoted 6 lines
However, there is a separate overall mode for the VM, which is stored in '
kvm->arch.xen.long_mode' and accessed by userspace using theKVM_XEN_ATTR_TYPE_LONG_MODEattribute. It affects the datatypes used by shared memory data structures, and is also latched by the kernel when the guest writes the MSR for the hypercall page. That one should probably keep its name.
Quoted 3 lines
For this one, I think the current KVM code is consistent.
The format is determined by EFER.LMA, whether the guest is running in 64 bit or compatible mode doesn't change the ABI.
Agreed. For the hypercall case you're looking at, switching the name to is_64bit makes sense.
Quoted 2 lines
struct
compat_shared_infois used only when the guest is running natively in a 32-bit build.
The struct compat_shared_info is also used in !kvm->arch.xen.long_mode on a 64-bit host, as that's what means the guest is considered to be a 32-bit guest.
It's somewhat orthogonal from whether any given vCPU is making any given hypercall while in 64-bit mode. The 'long_mode' is *latched* at certain specific times which are defined by Xen's historical behaviour.
I'm suggesting that you clean up longmode→is_64bit for the *hypercalls* but leave 'long_mode' as is.
Re: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
Quoted 42 lines · David Woodhouse
Quoted 27 lines · Binbin Wu
Quoted 17 lines · David Woodhouse
Quoted 11 lines · Binbin Wu
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
longmode =
is_64_bit_hypercall(vcpu);Is the variable name misleading?
It most definitely is. However, @longmode is passed around quite a few locations in
xen.c, and so I don't want to opportunistically fix this one variable. Though I'm definitely not opposed to a separate patch to rename them all tois_64bitor something.OK, I can do it.
This one (as shown above) is clearly indicating whether this particular vCPU is in 64-bit mode for this particular hypercall. Changing that to
is_64bitmakes sense.Quoted 6 lines
However, there is a separate overall mode for the VM, which is stored in '
kvm->arch.xen.long_mode' and accessed by userspace using theKVM_XEN_ATTR_TYPE_LONG_MODEattribute. It affects the datatypes used by shared memory data structures, and is also latched by the kernel when the guest writes the MSR for the hypercall page. That one should probably keep its name.Quoted 3 lines
For this one, I think the current KVM code is consistent.
The format is determined by EFER.LMA, whether the guest is running in 64 bit or compatible mode doesn't change the ABI.Agreed. For the hypercall case you're looking at, switching the name to
is_64bitmakes sense.Quoted 2 lines
struct
compat_shared_infois used only when the guest is running natively in a 32-bit build.The struct
compat_shared_infois also used in !kvm->arch.xen.long_mode on a 64-bit host, as that's what means the guest is considered to be a 32-bit guest.
Quoted 7 lines
It's somewhat orthogonal from whether any given vCPU is making any given hypercall while in 64-bit mode. The '
long_mode' is *latched* at certain specific times which are defined by Xen's historical behaviour.I'm suggesting that you clean up longmode→is_64bit for the *hypercalls* but leave '
long_mode' as is.
Yes, will only do it for is_64_bit_hypercall().
Quoted 1 line
Re: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
Quoted 31 lines · Binbin Wu
Quoted 28 lines · David Woodhouse
Quoted 27 lines · Binbin Wu
Quoted 23 lines · David Woodhouse
Quoted 11 lines · Binbin Wu
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
longmode =
is_64_bit_hypercall(vcpu);Is the variable name misleading?
It most definitely is. However, @longmode is passed around quite a few locations in
xen.c, and so I don't want to opportunistically fix this one variable. Though I'm definitely not opposed to a separate patch to rename them all tois_64bitor something.OK, I can do it.
This one (as shown above) is clearly indicating whether this particular vCPU is in 64-bit mode for this particular hypercall. Changing that to
is_64bitmakes sense.However, there is a separate overall mode for the VM, which is stored in '
kvm->arch.xen.long_mode' and accessed by userspace using theKVM_XEN_ATTR_TYPE_LONG_MODEattribute. It affects the datatypes used by shared memory data structures, and is also latched by the kernel when the guest writes the MSR for the hypercall page. That one should probably keep its name.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
For this one, I think the current KVM code is consistent.
The format is determined by EFER.LMA, whether the guest is running in 64 bit or compatible mode doesn't change the ABI.
I still have a point of confusion.
I noticed a behavioral mismatch between KVM and Xen regarding when they switch to the standard/compat shared info.
- In Xen: The 32-bit shared info structure is latched if the current vCPU is not in 64-bit mode:
hvm_latch_shinfo_size
d->arch.has_32bit_shinfo = hvm_guest_x86_mode(current) != X86_MODE_64BIT
- In KVM: It evaluates is_long_mode(vcpu) instead. E.g., kvm_xen_write_hypercall_page
bool lm = is_long_mode(vcpu);
...
kvm->arch.xen.long_mode = lm;In theory, these two checks could differ when the guest kernel is running in a 32-bit compatibility mode. However, I believe this mismatch is fine in practice for two reasons:
- Mainstream 64-bit OSes don't run in compatibility mode for kernel code after the early init.
- By default, HVM guests cannot issue hypercalls from userspace. The only one exception HVMOP_guest_request_vm_event is not related to the share info.
So the vCPU will never be in compatibility mode when a related hypercall occurs. In this specific operational context, evaluating is_long_mode() yields the exact same functional outcome as checking for 64-bit execution mode. Am I missing anything here?
Quoted 23 lines
Quoted 11 lines
Agreed. For the hypercall case you're looking at, switching the name to
is_64bitmakes sense.Quoted 2 lines
struct
compat_shared_infois used only when the guest is running natively in a 32-bit build.The struct
compat_shared_infois also used in !kvm->arch.xen.long_mode on a 64-bit host, as that's what means the guest is considered to be a 32-bit guest.Quoted 7 lines
It's somewhat orthogonal from whether any given vCPU is making any given hypercall while in 64-bit mode. The '
long_mode' is *latched* at certain specific times which are defined by Xen's historical behaviour.I'm suggesting that you clean up longmode→is_64bit for the *hypercalls* but leave '
long_mode' as is.Yes, will only do it for
is_64_bit_hypercall().Quoted 1 line
David Woodhouse
maintainerRe: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
Quoted 13 lines · Binbin Wu
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
For this one, I think the current KVM code is consistent.
The format is determined by EFER.LMA, whether the guest is running in 64 bit or compatible mode doesn't change the ABI.I still have a point of confusion.
I noticed a behavioral mismatch between KVM and Xen regarding when they switch to the standard/compat shared info.
- In Xen: The 32-bit shared info structure is latched if the current vCPU is not in 64-bit mode:
hvm_latch_shinfo_size
d->arch.has_32bit_shinfo=hvm_guest_x86_mode(current)!=X86_MODE_64BIT
Quoted 5 lines
- In KVM: It evaluates
is_long_mode(vcpu)instead. E.g.,kvm_xen_write_hypercall_page
bool lm =is_long_mode(vcpu);
...
kvm->arch.xen.long_mode= lm;
Nice catch. That should probably use is_64_bit_hypercall() too, yes?
Quoted 5 lines
In theory, these two checks could differ when the guest kernel is running in a 32-bit compatibility mode. However, I believe this mismatch is fine in practice for two reasons:
- Mainstream 64-bit OSes don't run in compatibility mode for kernel code after the early init.
Although... some of the early init does involve 32-bit mode and it wouldn't be impossible for the guest to build the hypercall page from 32-bit mode during startup.
Quoted 2 lines
- By default, HVM guests cannot issue hypercalls from userspace. The only one exception
HVMOP_guest_request_vm_eventis not related to the share info.
A third reason: This is only one of *two* places where the guest mode gets latched. The guest's mode is also latched when it sets the HVM_PARAM_CALLBACK_IRQ parameter. When running under KVM, the VMM handles this and tells the kernel via the KVM_XEN_ATTR_TYPE_LONG_MODE attribute.
So even if it doesn't latch correctly when setting the hypercall page, it will later.
And Linux at least will set KVM_PARAM_CALLBACK_IRQ even if it isn't using it, in xen_set_upcall_vector() with a comment saying 'Trick toolstack to think we are enlightened'.
Quoted 4 lines
So the vCPU will never be in compatibility mode when a related hypercall occurs. In this specific operational context, evaluating
is_long_mode()yields the exact same functional outcome as checking for 64-bit execution mode. Am I missing anything here?
I think you're right. We have thus far launched about 5 billion Xen guests with this, and I've never heard any report about guests latching the wrong mode. And there are a *lot* of random home-grown and network appliance operating systems out there, and basic things like event channels would fail to work if the shared_info was in the wrong mode.
That said, I would not be averse to fixing it. We could just use is_64_bit_hypercall() in kvm_xen_write_hypercall_page(), right?
Or at the very least a comment saying that it *should* be doing so, but that we're too nervous to change it now?
David Woodhouse
maintainerRe: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
In reply to Binbin WuQuoted 6 lines · Binbin Wu
Quoted 3 lines
I'm suggesting that you clean up longmode→is_64bit for the *hypercalls* but leave '
long_mode' as is.Yes, will only do it for
is_64_bit_hypercall().
If you did this, I'm not sure I saw it?
In response to https://lore.kernel.org/all/aiHPPUk5DY7rH-zL@v4bel/#r I now find myself with both 'longmode' (current vCPU mode, should be called is_64bit), and 'long_mode' (latched VM-wide mode) in the *same* function.
I cannot live with that; I'm going to do the longmode→is_64bit change locally.
Re: [PATCH v2 03/15] KVM: x86/xen: Don't truncate RAX when handling hypercall from protected guest
In reply to Sean ChristophersonQuoted 5 lines · Sean Christopherson
Don't truncate RAX when handling a Xen hypercall for a guest with protected state, as KVM's ABI is to assume the guest is in 64-bit for such cases (the guest leaving garbage in 63:32 after a transition to 32-bit mode is far less likely than 63:32 being necessary to complete the hypercall).
The latter isn't likely either. RAX is the system call number, and those numbers are only in double digits; it'll be a while before we need more than 8 bits for those, let alone 32 :)
But sure, as a cleanup it makes sense. Thanks.
Quoted 36 lines
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")Signed-off-by Sean Christopherson <[email protected]>---
arch/x86/kvm/xen.c | 6 +++---1 file changed, 3 insertions(+), 3 deletions(-)arch/x86/kvm/xen.c +0
-0@@ -1678,15 +1678,14 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) bool handled = false; u8 cpl;arch/x86/kvm/xen.c +0
-2- input = (u64 )kvm_register_read (vcpu,VCPU_REGS_RAX ); -/* Hyper-V hypercalls get bit 31 set in EAX */
arch/x86/kvm/xen.c +1
-1- if ((input & 0x80000000) && + if ((kvm_rax_read(vcpu) & 0x80000000) &&
kvm_hv_hypercall_enabled(vcpu))
returnkvm_hv_hypercall(vcpu);
longmode =is_64_bit_hypercall(vcpu);
if (!longmode) {arch/x86/kvm/xen.c +1
-0+ input = (u32 )kvm_rax_read (vcpu);params[0] = (u32)kvm_rbx_read(vcpu); params[1] = (u32)kvm_rcx_read(vcpu); params[2] = (u32)kvm_rdx_read(vcpu);
arch/x86/kvm/xen.c +1
-0@@ -1696,6 +1695,7 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) } else { #ifdef CONFIG_X86_64 + input = (u64 )kvm_rax_read (vcpu); params[0] = (u64)kvm_rdi_read(vcpu); params[1] = (u64)kvm_rsi_read(vcpu); params[2] = (u64)kvm_rdx_read(vcpu);
Sean Christopherson
maintainerPATCH v2 4/15 KVM: VMX: Read 32-bit GPR values for ENCLS instructions outside of 64-bit mode
In reply to Sean ChristophersonWhen getting register values for ENCLS emulation, use kvm_register_read() instead of kvm_<reg>_read() so that bits 63:32 of the register are dropped if the guest is in 32-bit mode.
Note, the misleading/surprising behavior of kvm_<reg>_read() being "raw" variants under the hood will be addressed once all non-benign bugs are fixed.
--- arch/x86/kvm/vmx/sgx.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
arch/x86/kvm/vmx/sgx.c +5 -5
@@ -225,8 +225,8 @@ static int handle_encls_ecreate(struct kvm_vcpu *vcpu)struct x86_exception ex;int r; -if (sgx_get_encls_gva (vcpu,kvm_rbx_read (vcpu),32 ,32 , &pageinfo_gva) || -sgx_get_encls_gva (vcpu,kvm_rcx_read (vcpu),4096 ,4096 , &secs_gva)) +if (sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RBX ),32 ,32 , &pageinfo_gva) || +sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RCX ),4096 ,4096 , &secs_gva))return 1 ;/*
@@ -302,9 +302,9 @@ static int handle_encls_einit(struct kvm_vcpu *vcpu)gpa_t sig_gpa, secs_gpa, token_gpa;int ret, trapnr; -if (sgx_get_encls_gva (vcpu,kvm_rbx_read (vcpu),1808 ,4096 , &sig_gva) || -sgx_get_encls_gva (vcpu,kvm_rcx_read (vcpu),4096 ,4096 , &secs_gva) || -sgx_get_encls_gva (vcpu,kvm_rdx_read (vcpu),304 ,512 , &token_gva)) +if (sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RBX ),1808 ,4096 , &sig_gva) || +sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RCX ),4096 ,4096 , &secs_gva) || +sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RDX ),304 ,512 , &token_gva))return 1 ;/*
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 04/15] KVM: VMX: Read 32-bit GPR values for ENCLS instructions outside of 64-bit mode
Quoted 4 lines · Sean Christopherson
When getting register values for ENCLS emulation, use
kvm_register_read()instead of kvm_<reg>_read() so that bits 63:32 of the register are dropped if the guest is in 32-bit mode.
Quoted 8 lines
Note, the misleading/surprising behavior of kvm_<reg>_read() being "raw" variants under the hood will be addressed once all non-benign bugs are fixed.
Fixes 70210c044b4e ("KVM: VMX: Add SGX ENCLS[ECREATE] handler to enforce CPUID restrictions")Fixes b6f084ca5538 ("KVM: VMX: Add ENCLS[EINIT] handler to support SGX Launch Control (LC)")Acked-by Kai Huang <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 32 lines
--- arch/x86/kvm/vmx/sgx.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)arch/x86/kvm/vmx/sgx.c +5
-5@@ -225,8 +225,8 @@ static int handle_encls_ecreate(struct kvm_vcpu *vcpu)struct x86_exception ex;int r; -if (sgx_get_encls_gva (vcpu,kvm_rbx_read (vcpu),32 ,32 , &pageinfo_gva) || -sgx_get_encls_gva (vcpu,kvm_rcx_read (vcpu),4096 ,4096 , &secs_gva)) +if (sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RBX ),32 ,32 , &pageinfo_gva) || +sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RCX ),4096 ,4096 , &secs_gva))return 1 ;/* @@ -302,9 +302,9 @@ static int handle_encls_einit(struct kvm_vcpu *vcpu)gpa_t sig_gpa, secs_gpa, token_gpa;int ret, trapnr; -if (sgx_get_encls_gva (vcpu,kvm_rbx_read (vcpu),1808 ,4096 , &sig_gva) || -sgx_get_encls_gva (vcpu,kvm_rcx_read (vcpu),4096 ,4096 , &secs_gva) || -sgx_get_encls_gva (vcpu,kvm_rdx_read (vcpu),304 ,512 , &token_gva)) +if (sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RBX ),1808 ,4096 , &sig_gva) || +sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RCX ),4096 ,4096 , &secs_gva) || +sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RDX ),304 ,512 , &token_gva))return 1 ;/*
Sean Christopherson
maintainerPATCH v2 5/15 KVM: x86: Trace hypercall register *after* truncating values for 32-bit
In reply to Sean ChristophersonWhen tracing hypercalls, invoke the tracepoint *after* truncating the register values for 32-bit guests so as not to record unused garbage (in the extremely unlikely scenario that the guest left garbage in a register after transitioning from 64-bit mode to 32-bit mode).
--- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +2 -2
@@ -10430,8 +10430,6 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, int cpl, ++vcpu->stat.hypercalls; -trace_kvm_hypercall (nr, a0, a1, a2, a3); -if (!op_64_bit) { nr &=0xFFFFFFFF ; a0 &=0xFFFFFFFF ;
@@ -10440,6 +10438,8 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, int cpl, a3 &=0xFFFFFFFF ; } +trace_kvm_hypercall (nr, a0, a1, a2, a3); +if (cpl) { ret = -KVM_EPERM ; goto out;
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 05/15] KVM: x86: Trace hypercall register *after* truncating values for 32-bit
Quoted 8 lines · Sean Christopherson
When tracing hypercalls, invoke the tracepoint *after* truncating the register values for 32-bit guests so as not to record unused garbage (in the extremely unlikely scenario that the guest left garbage in a register after transitioning from 64-bit mode to 32-bit mode).
Fixes 229456fc34b1 ("KVM: convert custom marker based tracing to event traces")Reviewed-by Yosry Ahmed <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 26 lines
--- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/x86.c +2
-2@@ -10430,8 +10430,6 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, int cpl, ++vcpu->stat.hypercalls; -trace_kvm_hypercall (nr, a0, a1, a2, a3); -if (!op_64_bit) { nr &=0xFFFFFFFF ; a0 &=0xFFFFFFFF ;@@ -10440,6 +10438,8 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, int cpl, a3 &=0xFFFFFFFF ; } +trace_kvm_hypercall (nr, a0, a1, a2, a3); +if (cpl) { ret = -KVM_EPERM ; goto out;
Sean Christopherson
maintainerPATCH v2 6/15 KVM: x86: Rename kvm_cache_regs.h => regs.h
In reply to Sean ChristophersonRename kvm_cache_regs.h to simply regs.h, as the "cache" nomenclature is already a lie (the file deals with state/registers that aren't cached per se), and so that more code/functionality can be landed in the header without making it a truly horrible misnomer.
Deliberately drop the kvm_ prefix/namespace to align with other "local" headers, and to further differentiate regs.h from the public/global arch/x86/include/asm/kvm_vcpu_regs.h, which sadly needs to stay in asm/ so that the number of registers can be referenced by kvm_vcpu_arch.
No functional change intended.
---
arch/x86/kvm/emulate.c | 2 +-
arch/x86/kvm/lapic.c | 2 +-
arch/x86/kvm/mmu.h | 2 +-
arch/x86/kvm/mmu/mmu.c | 2 +-
arch/x86/kvm/{kvm_cache_regs.h => regs.h} | 4 ++--
arch/x86/kvm/smm.c | 2 +-
arch/x86/kvm/svm/svm.c | 2 +-
arch/x86/kvm/svm/svm.h | 2 +-
arch/x86/kvm/vmx/nested.h | 2 +-
arch/x86/kvm/vmx/sgx.c | 2 +-
arch/x86/kvm/vmx/vmx.c | 2 +-
arch/x86/kvm/vmx/vmx.h | 2 +-
arch/x86/kvm/x86.c | 2 +-
arch/x86/kvm/x86.h | 2 +-
14 files changed, 15 insertions(+), 15 deletions(-)
rename arch/x86/kvm/{kvm_cache_regs.h => regs.h} (99%)arch/x86/kvm/emulate.c +1 -1
@@ -20,7 +20,7 @@#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt#include <linux/kvm_host.h> -#include "kvm_cache_regs.h" +#include "regs.h" #include "kvm_emulate.h" #include <linux/stringify.h> #include <asm/debugreg.h>
arch/x86/kvm/lapic.c +1 -1
@@ -37,7 +37,7 @@#include <asm/delay.h> #include <linux/atomic.h> #include <linux/jump_label.h> -#include "kvm_cache_regs.h" +#include "regs.h" #include "irq.h" #include "ioapic.h" #include "trace.h"
arch/x86/kvm/mmu.h +1 -1
@@ -3,7 +3,7 @@#define __KVM_X86_MMU_H#include <linux/kvm_host.h> -#include "kvm_cache_regs.h" +#include "regs.h" #include "x86.h" #include "cpuid.h"
arch/x86/kvm/mmu/mmu.c +1 -1
@@ -22,7 +22,7 @@#include "mmu_internal.h" #include "tdp_mmu.h" #include "x86.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "smm.h" #include "kvm_emulate.h" #include "page_track.h"
arch/x86/kvm/regs.h renamed from arch/x86/kvm/kvm_cache_regs.h +2 -2
@@ -1,6 +1,6 @@/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef ASM_KVM_CACHE_REGS_H -#define ASM_KVM_CACHE_REGS_H +#ifndef ARCH_X86_KVM_REGS_H +#define ARCH_X86_KVM_REGS_H #include <linux/kvm_host.h>
arch/x86/kvm/smm.c +1 -1
@@ -3,7 +3,7 @@#include <linux/kvm_host.h> #include "x86.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "kvm_emulate.h" #include "smm.h" #include "cpuid.h"
arch/x86/kvm/svm/svm.c +1 -1
@@ -4,7 +4,7 @@#include "irq.h" #include "mmu.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "x86.h" #include "smm.h" #include "cpuid.h"
arch/x86/kvm/svm/svm.h +1 -1
@@ -23,7 +23,7 @@#include <asm/sev-common.h> #include "cpuid.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "x86.h" /*
arch/x86/kvm/vmx/nested.h +1 -1
@@ -2,7 +2,7 @@#ifndef __KVM_X86_VMX_NESTED_H#define __KVM_X86_VMX_NESTED_H -#include "kvm_cache_regs.h" +#include "regs.h" #include "hyperv.h" #include "vmcs12.h" #include "vmx.h"
arch/x86/kvm/vmx/sgx.c +1 -1
@@ -6,7 +6,7 @@#include <asm/sgx.h> #include "x86.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "nested.h" #include "sgx.h" #include "vmx.h"
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -59,7 +59,7 @@#include "hyperv.h" #include "kvm_onhyperv.h" #include "irq.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "lapic.h" #include "mmu.h" #include "nested.h"
arch/x86/kvm/vmx/vmx.h +1 -1
@@ -10,7 +10,7 @@#include <asm/posted_intr.h> #include "capabilities.h" -#include "../kvm_cache_regs.h" +#include "../regs.h" #include "pmu_intel.h" #include "vmcs.h" #include "vmx_ops.h"
arch/x86/kvm/x86.c +1 -1
@@ -23,7 +23,7 @@#include "mmu.h" #include "i8254.h" #include "tss.h" -#include "kvm_cache_regs.h" +#include "regs.h" #include "kvm_emulate.h" #include "mmu/page_track.h" #include "x86.h"
arch/x86/kvm/x86.h +1 -1
@@ -6,7 +6,7 @@#include <asm/fpu/xstate.h> #include <asm/mce.h> #include <asm/pvclock.h> -#include "kvm_cache_regs.h" +#include "regs.h" #include "kvm_emulate.h" #include "cpuid.h"
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 06/15] KVM: x86: Rename kvm_cache_regs.h => regs.h
Quoted 6 lines · Sean Christopherson
Rename
kvm_cache_regs.hto simplyregs.h, as the "cache" nomenclature is already a lie (the file deals with state/registers that aren't cached per se), and so that more code/functionality can be landed in the header without making it a truly horrible misnomer.
Quoted 8 lines
Deliberately drop the
kvm_prefix/namespace to align with other "local" headers, and to further differentiateregs.hfrom the public/globalarch/x86/include/asm/kvm_vcpu_regs.h, which sadly needs to stay in asm/ so that the number of registers can be referenced bykvm_vcpu_arch.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Re: [PATCH v2 06/15] KVM: x86: Rename kvm_cache_regs.h => regs.h
In reply to Sean ChristophersonQuoted 5 lines · Sean Christopherson
Rename
kvm_cache_regs.hto simplyregs.h, as the "cache" nomenclature is already a lie (the file deals with state/registers that aren't cached per se), and so that more code/functionality can be landed in the header without making it a truly horrible misnomer.
Quoted 8 lines
Deliberately drop the
kvm_prefix/namespace to align with other "local" headers, and to further differentiateregs.hfrom the public/globalarch/x86/include/asm/kvm_vcpu_regs.h, which sadly needs to stay in asm/ so that the number of registers can be referenced bykvm_vcpu_arch.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Sean Christopherson
maintainerPATCH v2 7/15 KVM: x86: Move inlined CR and DR helpers from x86.h to regs.h
In reply to Sean ChristophersonMove inlined Control Register and Debug Register helpers from x86.h to the aptly named regs.h, to help trim down x86.h (and x86.c in the future).
Move select EFER functionality, but leave behind all other MSR handling, There is more than enough MSR code to carve out msr.{c,h} in the future. Give EFER special treatment as it's an "MSR" in name only, e.g. it's has far more in common with CR4 than it does with any MSR.
No functional change intended.
--- arch/x86/kvm/regs.h | 108 ++++++++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/x86.h | 102 ----------------------------------------- 2 files changed, 105 insertions(+), 105 deletions(-)
arch/x86/kvm/regs.h +105 -3
@@ -16,6 +16,37 @@static_assert (!(KVM_POSSIBLE_CR0_GUEST_BITS &X86_CR0_PDPTR_BITS )); +static inline bool is_long_mode (struct kvm_vcpu *vcpu) +{ +#ifdef CONFIG_X86_64 +return !!(vcpu->arch.efer &EFER_LMA ); +#else +return false; +#endif +} + +static inline bool is_64_bit_mode (struct kvm_vcpu *vcpu) +{ +int cs_db, cs_l; + +WARN_ON_ONCE (vcpu->arch.guest_state_protected); + +if (!is_long_mode (vcpu)) +return false; +kvm_x86_call (get_cs_db_l_bits)(vcpu, &cs_db, &cs_l); +return cs_l; +} + +static inline bool is_64_bit_hypercall (struct kvm_vcpu *vcpu) +{ +/* +* If running with protected guest state, the CS register is not +* accessible. The hypercall register values will have had to been +* provided in 64-bit mode, so assume the guest is in 64-bit. +*/ +return vcpu->arch.guest_state_protected ||is_64_bit_mode (vcpu); +} +#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu)\ { \
@@ -177,6 +208,12 @@ static inline void kvm_rsp_write(struct kvm_vcpu *vcpu, unsigned long val)kvm_register_write_raw (vcpu,VCPU_REGS_RSP , val); } +static inline u64 kvm_read_edx_eax (struct kvm_vcpu *vcpu) +{ +return (kvm_rax_read (vcpu) &-1u ) + | ((u64)(kvm_rdx_read (vcpu) &-1u ) <<32 ); +} +static inline u64 kvm_pdptr_read (struct kvm_vcpu *vcpu,int index) {might_sleep ();/* on svm */
@@ -243,10 +280,75 @@ static inline ulong kvm_read_cr4(struct kvm_vcpu *vcpu)return kvm_read_cr4_bits (vcpu, ~0UL ); } -static inline u64 kvm_read_edx_eax (struct kvm_vcpu *vcpu) +static inline bool __kvm_is_valid_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4) { -return (kvm_rax_read (vcpu) &-1u ) - | ((u64)(kvm_rdx_read (vcpu) &-1u ) <<32 ); +return !(cr4 & vcpu->arch.cr4_guest_rsvd_bits); +} + +#define __cr4_reserved_bits (__cpu_has, __c) \ +({ \ + u64 __reserved_bits = CR4_RESERVED_BITS; \ + \ + if (!__cpu_has(__c, X86_FEATURE_XSAVE)) \ + __reserved_bits |= X86_CR4_OSXSAVE; \ + if (!__cpu_has(__c, X86_FEATURE_SMEP)) \ + __reserved_bits |= X86_CR4_SMEP; \ + if (!__cpu_has(__c, X86_FEATURE_SMAP)) \ + __reserved_bits |= X86_CR4_SMAP; \ + if (!__cpu_has(__c, X86_FEATURE_FSGSBASE)) \ + __reserved_bits |= X86_CR4_FSGSBASE; \ + if (!__cpu_has(__c, X86_FEATURE_PKU)) \ + __reserved_bits |= X86_CR4_PKE; \ + if (!__cpu_has(__c, X86_FEATURE_LA57)) \ + __reserved_bits |= X86_CR4_LA57; \ + if (!__cpu_has(__c, X86_FEATURE_UMIP)) \ + __reserved_bits |= X86_CR4_UMIP; \ + if (!__cpu_has(__c, X86_FEATURE_VMX)) \ + __reserved_bits |= X86_CR4_VMXE; \ + if (!__cpu_has(__c, X86_FEATURE_PCID)) \ + __reserved_bits |= X86_CR4_PCIDE; \ + if (!__cpu_has(__c, X86_FEATURE_LAM)) \ + __reserved_bits |= X86_CR4_LAM_SUP; \ + if (!__cpu_has(__c, X86_FEATURE_SHSTK) && \ + !__cpu_has(__c, X86_FEATURE_IBT)) \ + __reserved_bits |= X86_CR4_CET; \ + __reserved_bits; \ +}) + +static inline bool is_protmode (struct kvm_vcpu *vcpu) +{ +return kvm_is_cr0_bit_set (vcpu,X86_CR0_PE ); +} + +static inline bool is_pae (struct kvm_vcpu *vcpu) +{ +return kvm_is_cr4_bit_set (vcpu,X86_CR4_PAE ); +} + +static inline bool is_pse (struct kvm_vcpu *vcpu) +{ +return kvm_is_cr4_bit_set (vcpu,X86_CR4_PSE ); +} + +static inline bool is_paging (struct kvm_vcpu *vcpu) +{ +return likely (kvm_is_cr0_bit_set (vcpu,X86_CR0_PG )); +} + +static inline bool is_pae_paging (struct kvm_vcpu *vcpu) +{ +return !is_long_mode (vcpu) &&is_pae (vcpu) &&is_paging (vcpu); +} + +static inline bool kvm_dr7_valid (u64 data) +{ +/* Bits [63:32] are reserved */ +return !(data >>32 ); +} +static inline bool kvm_dr6_valid (u64 data) +{ +/* Bits [63:32] are reserved */ +return !(data >>32 ); }static inline void enter_guest_mode (struct kvm_vcpu *vcpu)
arch/x86/kvm/x86.h +0 -102
126 lines not on this page · raw message
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 07/15] KVM: x86: Move inlined CR and DR helpers from x86.h to regs.h
Quoted 45 lines · Sean Christopherson
Move inlined Control Register and Debug Register helpers from
x86.hto the aptly namedregs.h, to help trim downx86.h(andx86.cin the future).Move select EFER functionality, but leave behind all other MSR handling, There is more than enough MSR code to carve out msr.{c,h} in the future. Give EFER special treatment as it's an "MSR" in name only, e.g. it's has far more in common with CR4 than it does with any MSR.
No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/regs.h | 108 ++++++++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/x86.h | 102 ----------------------------------------- 2 files changed, 105 insertions(+), 105 deletions(-)arch/x86/kvm/regs.h +20
-0@@ -16,6 +16,37 @@static_assert (!(KVM_POSSIBLE_CR0_GUEST_BITS &X86_CR0_PDPTR_BITS )); +static inline bool is_long_mode (struct kvm_vcpu *vcpu) +{ +#ifdef CONFIG_X86_64 +return !!(vcpu->arch.efer &EFER_LMA ); +#else +return false; +#endif +} + +static inline bool is_64_bit_mode (struct kvm_vcpu *vcpu) +{ +int cs_db, cs_l; + +WARN_ON_ONCE (vcpu->arch.guest_state_protected); + +if (!is_long_mode (vcpu)) +return false; +kvm_x86_call (get_cs_db_l_bits)(vcpu, &cs_db, &cs_l); +return cs_l; +}
arch/x86/kvm/regs.h line 48
Quoted 10 lines
arch/x86/kvm/regs.h +10
-0+ +static inline bool is_64_bit_hypercall (struct kvm_vcpu *vcpu) +{ +/* +* If running with protected guest state, the CS register is not +* accessible. The hypercall register values will have had to been +* provided in 64-bit mode, so assume the guest is in 64-bit. +*/ +return vcpu->arch.guest_state_protected ||is_64_bit_mode (vcpu); +}
This is really stretching the meaning of 'regs', but it's not that much worse than 'x86'..
Re: [PATCH v2 07/15] KVM: x86: Move inlined CR and DR helpers from x86.h to regs.h
In reply to Sean ChristophersonQuoted 6 lines · Sean Christopherson
Move inlined Control Register and Debug Register helpers from
x86.hto the aptly namedregs.h, to help trim downx86.h(andx86.cin the future).Move select EFER functionality, but leave behind all other MSR handling, There is more than enough MSR code to carve out msr.{c,h} in the future. Give EFER special treatment as it's an "MSR" in name only, e.g. it's has
^
itQuoted 5 lines
far more in common with CR4 than it does with any MSR.
No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Nit: The shortlog doesn't cover EFER.
Sean Christopherson
maintainerPATCH v2 8/15 KVM: x86: Add mode-aware versions of kvm_<reg>_{read,write}() helpers
In reply to Sean ChristophersonMake kvm_<reg>_{read,write}() mode-aware (where the value is truncated to 32 bits if the vCPU isn't in 64-bit mode), and convert all the intentional "raw" accesses to kvm_<reg>_{read,write}_raw() versions. To avoid confusion and bikeshedding over whether or not explicit 32-bit accesses should use the "raw" or mode-aware variants, add and use "e" versions, e.g. for things like RDMSR, WRMSR, and CPUID, where the instruction uses only only bits 31:0, regardless of mode.
No functional change intended (all use of "e" versions is for cases where the value is already truncated due to bouncing through a u32).
--- arch/x86/kvm/cpuid.c | 12 ++-- arch/x86/kvm/hyperv.c | 21 +++---- arch/x86/kvm/hyperv.h | 4 +- arch/x86/kvm/regs.h | 80 +++++++++++++++++-------- arch/x86/kvm/svm/nested.c | 6 +- arch/x86/kvm/svm/svm.c | 13 ++-- arch/x86/kvm/vmx/nested.c | 8 +-- arch/x86/kvm/vmx/sgx.c | 4 +- arch/x86/kvm/vmx/tdx.c | 18 +++--- arch/x86/kvm/x86.c | 121 +++++++++++++++++++------------------- arch/x86/kvm/x86.h | 8 +-- arch/x86/kvm/xen.c | 32 +++++----- 12 files changed, 173 insertions(+), 154 deletions(-)
arch/x86/kvm/cpuid.c +6 -6
@@ -2165,13 +2165,13 @@ int kvm_emulate_cpuid(struct kvm_vcpu *vcpu) !kvm_require_cpl(vcpu, 0))return 1 ; - eax =kvm_rax_read (vcpu); - ecx =kvm_rcx_read (vcpu); + eax =kvm_eax_read (vcpu); + ecx =kvm_ecx_read (vcpu);kvm_cpuid (vcpu, &eax, &ebx, &ecx, &edx, false); -kvm_rax_write (vcpu, eax); -kvm_rbx_write (vcpu, ebx); -kvm_rcx_write (vcpu, ecx); -kvm_rdx_write (vcpu, edx); +kvm_eax_write (vcpu, eax); +kvm_ebx_write (vcpu, ebx); +kvm_ecx_write (vcpu, ecx); +kvm_edx_write (vcpu, edx);return kvm_skip_emulated_instruction (vcpu); }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_emulate_cpuid);
arch/x86/kvm/hyperv.c +9 -12
@@ -2377,10 +2377,10 @@ static void kvm_hv_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result) longmode =is_64_bit_hypercall (vcpu);if (longmode) -kvm_rax_write (vcpu, result); +kvm_rax_write_raw (vcpu, result);else { -kvm_rdx_write (vcpu, result >>32 ); -kvm_rax_write (vcpu, result &0xffffffff ); +kvm_edx_write (vcpu, result >>32 ); +kvm_eax_write (vcpu, result); } }
@@ -2544,18 +2544,15 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)#ifdef CONFIG_X86_64 if (is_64_bit_hypercall (vcpu)) { - hc.param =kvm_rcx_read (vcpu); - hc.ingpa =kvm_rdx_read (vcpu); - hc.outgpa =kvm_r8_read (vcpu); + hc.param =kvm_rcx_read_raw (vcpu); + hc.ingpa =kvm_rdx_read_raw (vcpu); + hc.outgpa =kvm_r8_read_raw (vcpu); } else#endif { - hc.param = ((u64 )kvm_rdx_read (vcpu) <<32 ) | - (kvm_rax_read (vcpu) &0xffffffff ); - hc.ingpa = ((u64 )kvm_rbx_read (vcpu) <<32 ) | - (kvm_rcx_read (vcpu) &0xffffffff ); - hc.outgpa = ((u64 )kvm_rdi_read (vcpu) <<32 ) | - (kvm_rsi_read (vcpu) &0xffffffff ); + hc.param = ((u64 )kvm_edx_read (vcpu) <<32 ) |kvm_eax_read (vcpu); + hc.ingpa = ((u64 )kvm_ebx_read (vcpu) <<32 ) |kvm_ecx_read (vcpu); + hc.outgpa = ((u64 )kvm_edi_read (vcpu) <<32 ) |kvm_esi_read (vcpu); } hc.code = hc.param &0xffff ;
arch/x86/kvm/hyperv.h +2 -2
@@ -232,8 +232,8 @@ static inline bool kvm_hv_is_tlb_flush_hcall(struct kvm_vcpu *vcpu)if (!hv_vcpu)return false; - code =is_64_bit_hypercall (vcpu) ?kvm_rcx_read (vcpu) : -kvm_rax_read (vcpu); + code =is_64_bit_hypercall (vcpu) ?kvm_rcx_read_raw (vcpu) : +kvm_eax_read (vcpu); return (code == HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE || code == HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST ||
arch/x86/kvm/regs.h +54 -26
94 lines not on this page · raw message
arch/x86/kvm/svm/nested.c +3 -3
@@ -778,7 +778,7 @@ static void nested_vmcb02_prepare_save(struct vcpu_svm *svm) svm->vcpu.arch.cr2 = save->cr2; -kvm_rax_write (vcpu, save->rax); +kvm_rax_write_raw (vcpu, save->rax);kvm_rsp_write (vcpu, save->rsp);kvm_rip_write (vcpu, save->rip);
@@ -1244,7 +1244,7 @@ static int nested_svm_vmexit_update_vmcb12(struct kvm_vcpu *vcpu) vmcb12->save.rflags =kvm_get_rflags (vcpu); vmcb12->save.rip =kvm_rip_read (vcpu); vmcb12->save.rsp =kvm_rsp_read (vcpu); - vmcb12->save.rax =kvm_rax_read (vcpu); + vmcb12->save.rax =kvm_rax_read_raw (vcpu); vmcb12->save.dr7 = vmcb02->save.dr7; vmcb12->save.dr6 = svm->vcpu.arch.dr6; vmcb12->save.cpl = vmcb02->save.cpl;
@@ -1394,7 +1394,7 @@ void nested_svm_vmexit(struct vcpu_svm *svm)svm_set_efer (vcpu, vmcb01->save.efer);svm_set_cr0 (vcpu, vmcb01->save.cr0 |X86_CR0_PE );svm_set_cr4 (vcpu, vmcb01->save.cr4); -kvm_rax_write (vcpu, vmcb01->save.rax); +kvm_rax_write_raw (vcpu, vmcb01->save.rax);kvm_rsp_write (vcpu, vmcb01->save.rsp);kvm_rip_write (vcpu, vmcb01->save.rip);
arch/x86/kvm/svm/svm.c +5 -8
@@ -2408,15 +2408,12 @@ static int clgi_interception(struct kvm_vcpu *vcpu)static int invlpga_interception (struct kvm_vcpu *vcpu) { -gva_t gva =kvm_rax_read (vcpu); -u32 asid =kvm_rcx_read (vcpu); - -if (nested_svm_check_permissions (vcpu)) -return 1 ; -/* FIXME: Handle an address size prefix. */ -if (!is_64_bit_mode (vcpu)) - gva = (u32 )gva; +gva_t gva =kvm_rax_read (vcpu); +u32 asid =kvm_ecx_read (vcpu); + +if (nested_svm_check_permissions (vcpu)) +return 1 ;trace_kvm_invlpga (to_svm (vcpu)->vmcb->save.rip, asid, gva);
arch/x86/kvm/vmx/nested.c +4 -4
@@ -6148,7 +6148,7 @@ static int handle_invvpid(struct kvm_vcpu *vcpu)static int nested_vmx_eptp_switching (struct kvm_vcpu *vcpu,struct vmcs12 *vmcs12) { -u32 index =kvm_rcx_read (vcpu); +u32 index =kvm_ecx_read (vcpu);u64 new_eptp;if (WARN_ON_ONCE (!nested_cpu_has_ept (vmcs12)))
@@ -6182,7 +6182,7 @@ static int handle_vmfunc(struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);struct vmcs12 *vmcs12; -u32 function =kvm_rax_read (vcpu); +u32 function =kvm_eax_read (vcpu);/* * VMFUNC should never execute cleanly while L1 is active; KVM supports
@@ -6304,7 +6304,7 @@ static bool nested_vmx_exit_handled_msr(struct kvm_vcpu *vcpu, exit_reason.basic == EXIT_REASON_MSR_WRITE_IMM) msr_index =vmx_get_exit_qual (vcpu); else - msr_index =kvm_rcx_read (vcpu); + msr_index =kvm_ecx_read (vcpu);/* * The MSR_BITMAP page is divided into four 1024-byte bitmaps,
@@ -6414,7 +6414,7 @@ static bool nested_vmx_exit_handled_encls(struct kvm_vcpu *vcpu, !nested_cpu_has2(vmcs12, SECONDARY_EXEC_ENCLS_EXITING))return false; - encls_leaf =kvm_rax_read (vcpu); + encls_leaf =kvm_eax_read (vcpu);if (encls_leaf >62 ) encls_leaf =63 ;return vmcs12->encls_exiting_bitmap &BIT_ULL (encls_leaf);
arch/x86/kvm/vmx/sgx.c +2 -2
@@ -352,7 +352,7 @@ static int handle_encls_einit(struct kvm_vcpu *vcpu) rflags &= ~X86_EFLAGS_ZF ;vmx_set_rflags (vcpu, rflags); -kvm_rax_write (vcpu, ret); +kvm_eax_write (vcpu, ret);return kvm_skip_emulated_instruction (vcpu); }
@@ -380,7 +380,7 @@ static inline bool sgx_enabled_in_guest_bios(struct kvm_vcpu *vcpu)int handle_encls (struct kvm_vcpu *vcpu) { -u32 leaf = (u32 )kvm_rax_read (vcpu); +u32 leaf =kvm_eax_read (vcpu);if (!enable_sgx || !guest_cpu_cap_has (vcpu,X86_FEATURE_SGX ) || !guest_cpu_cap_has (vcpu,X86_FEATURE_SGX1 )) {
arch/x86/kvm/vmx/tdx.c +9 -9
@@ -1163,11 +1163,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)static int tdx_emulate_vmcall (struct kvm_vcpu *vcpu) { -kvm_rax_write (vcpu,to_tdx (vcpu)->vp_enter_args.r10); -kvm_rbx_write (vcpu,to_tdx (vcpu)->vp_enter_args.r11); -kvm_rcx_write (vcpu,to_tdx (vcpu)->vp_enter_args.r12); -kvm_rdx_write (vcpu,to_tdx (vcpu)->vp_enter_args.r13); -kvm_rsi_write (vcpu,to_tdx (vcpu)->vp_enter_args.r14); +kvm_rax_write_raw (vcpu,to_tdx (vcpu)->vp_enter_args.r10); +kvm_rbx_write_raw (vcpu,to_tdx (vcpu)->vp_enter_args.r11); +kvm_rcx_write_raw (vcpu,to_tdx (vcpu)->vp_enter_args.r12); +kvm_rdx_write_raw (vcpu,to_tdx (vcpu)->vp_enter_args.r13); +kvm_rsi_write_raw (vcpu,to_tdx (vcpu)->vp_enter_args.r14);return __kvm_emulate_hypercall (vcpu,0 , complete_hypercall_exit); }
@@ -2028,12 +2028,12 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath)case EXIT_REASON_IO_INSTRUCTION :return tdx_emulate_io (vcpu);case EXIT_REASON_MSR_READ : -kvm_rcx_write (vcpu, tdx->vp_enter_args.r12); +kvm_ecx_write (vcpu, tdx->vp_enter_args.r12);return kvm_emulate_rdmsr (vcpu);case EXIT_REASON_MSR_WRITE : -kvm_rcx_write (vcpu, tdx->vp_enter_args.r12); -kvm_rax_write (vcpu, tdx->vp_enter_args.r13 &-1u ); -kvm_rdx_write (vcpu, tdx->vp_enter_args.r13 >>32 ); +kvm_ecx_write (vcpu, tdx->vp_enter_args.r12); +kvm_eax_write (vcpu, tdx->vp_enter_args.r13); +kvm_edx_write (vcpu, tdx->vp_enter_args.r13 >>32 );return kvm_emulate_wrmsr (vcpu);case EXIT_REASON_EPT_MISCONFIG :return tdx_emulate_mmio (vcpu);
arch/x86/kvm/x86.c +61 -60
230 lines not on this page · raw message
arch/x86/kvm/x86.h +2 -6
@@ -367,17 +367,13 @@ static inline bool vcpu_match_mmio_gpa(struct kvm_vcpu *vcpu, gpa_t gpa)static inline unsigned long kvm_register_read (struct kvm_vcpu *vcpu,int reg) { -unsigned long val =kvm_register_read_raw (vcpu, reg); - -return is_64_bit_mode (vcpu) ? val : (u32 )val; +return kvm_register_read_raw (vcpu, reg) &kvm_reg_mode_mask (vcpu); }static inline void kvm_register_write (struct kvm_vcpu *vcpu,int reg,unsigned long val) { -if (!is_64_bit_mode (vcpu)) - val = (u32 )val; -return kvm_register_write_raw (vcpu, reg, val); +return kvm_register_write_raw (vcpu, reg, val &kvm_reg_mode_mask (vcpu)); }static inline bool kvm_check_has_quirk (struct kvm *kvm,u64 quirk)
arch/x86/kvm/xen.c +16 -16
@@ -1408,7 +1408,7 @@ int kvm_xen_hvm_config(struct kvm *kvm, struct kvm_xen_hvm_config *xhc)static int kvm_xen_hypercall_set_result (struct kvm_vcpu *vcpu,u64 result) { -kvm_rax_write (vcpu, result); +kvm_rax_write_raw (vcpu, result);return kvm_skip_emulated_instruction (vcpu); }
@@ -1679,29 +1679,29 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu)u8 cpl;/* Hyper-V hypercalls get bit 31 set in EAX */ -if ((kvm_rax_read (vcpu) &0x80000000 ) && +if ((kvm_rax_read_raw (vcpu) &0x80000000 ) &&kvm_hv_hypercall_enabled (vcpu))return kvm_hv_hypercall (vcpu); longmode =is_64_bit_hypercall (vcpu);if (!longmode) { - input = (u32 )kvm_rax_read (vcpu); - params[0 ] = (u32 )kvm_rbx_read (vcpu); - params[1 ] = (u32 )kvm_rcx_read (vcpu); - params[2 ] = (u32 )kvm_rdx_read (vcpu); - params[3 ] = (u32 )kvm_rsi_read (vcpu); - params[4 ] = (u32 )kvm_rdi_read (vcpu); - params[5 ] = (u32 )kvm_rbp_read (vcpu); + input =kvm_eax_read (vcpu); + params[0 ] =kvm_ebx_read (vcpu); + params[1 ] =kvm_ecx_read (vcpu); + params[2 ] =kvm_edx_read (vcpu); + params[3 ] =kvm_esi_read (vcpu); + params[4 ] =kvm_edi_read (vcpu); + params[5 ] =kvm_ebp_read (vcpu); }else {#ifdef CONFIG_X86_64 - input = (u64 )kvm_rax_read (vcpu); - params[0 ] = (u64 )kvm_rdi_read (vcpu); - params[1 ] = (u64 )kvm_rsi_read (vcpu); - params[2 ] = (u64 )kvm_rdx_read (vcpu); - params[3 ] = (u64 )kvm_r10_read (vcpu); - params[4 ] = (u64 )kvm_r8_read (vcpu); - params[5 ] = (u64 )kvm_r9_read (vcpu); + input = (u64 )kvm_rax_read_raw (vcpu); + params[0 ] = (u64 )kvm_rdi_read_raw (vcpu); + params[1 ] = (u64 )kvm_rsi_read_raw (vcpu); + params[2 ] = (u64 )kvm_rdx_read_raw (vcpu); + params[3 ] = (u64 )kvm_r10_read_raw (vcpu); + params[4 ] = (u64 )kvm_r8_read_raw (vcpu); + params[5 ] = (u64 )kvm_r9_read_raw (vcpu);#else KVM_BUG_ON (1 , vcpu->kvm);return -EIO ;
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 08/15] KVM: x86: Add mode-aware versions of kvm_<reg>_{read,write}() helpers
Quoted 7 lines · Sean Christopherson
Make kvm_<reg>_{read,write}() mode-aware (where the value is truncated to 32 bits if the vCPU isn't in 64-bit mode), and convert all the intentional "raw" accesses to kvm_<reg>_{read,write}_raw() versions. To avoid confusion and bikeshedding over whether or not explicit 32-bit accesses should use the "raw" or mode-aware variants, add and use "e" versions, e.g. for things like RDMSR, WRMSR, and CPUID, where the instruction uses only only bits 31:0, regardless of mode.
^
double "only"
Quoted 7 lines
No functional change intended (all use of "e" versions is for cases where the value is already truncated due to bouncing through a u32).
Cc Binbin Wu <[email protected]>Cc Kai Huang <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Re: [PATCH v2 08/15] KVM: x86: Add mode-aware versions of kvm_<reg>_{read,write}() helpers
In reply to Sean ChristophersonQuoted 10 lines
(quoted diff) +0
-0@@ -10413,29 +10413,30 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
if (!is_64_bit_hypercall(vcpu))
ret = (u32)ret;arch/x86/kvm/x86.c +1
-1-kvm_rax_write (vcpu, ret); +kvm_rax_write_raw (vcpu, ret);return
kvm_skip_emulated_instruction(vcpu); }
Nit: AFAICT if we use kvm_rax_write(vcpu, ret) instead of the "raw" version here, we can then remove the
if (!is_64_bit_hypercall(vcpu)) ret = (u32)ret;
But I saw in your next patch you are going to remove the non-raw write helpers.
Sean Christopherson
maintainerRe: [PATCH v2 08/15] KVM: x86: Add mode-aware versions of kvm_<reg>_{read,write}() helpers
Quoted 14 lines · Kai Huang
Quoted 10 lines
(quoted diff) +0
-0@@ -10413,29 +10413,30 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
if (!is_64_bit_hypercall(vcpu))
ret = (u32)ret;arch/x86/kvm/x86.c +1
-1-kvm_rax_write (vcpu, ret); +kvm_rax_write_raw (vcpu, ret);return
kvm_skip_emulated_instruction(vcpu); }Nit: AFAICT if we use
kvm_rax_write(vcpu, ret)instead of the "raw" version here, we can then remove the
Quoted 2 lines
if (!is_64_bit_hypercall(vcpu)) ret = (u32)ret;
No, because sneakily, is_64_bit_hypercall() != is_64_bit_mode(vcpu). And because we also need to avoid calling is_64_bit_mode(). If we use kvm_rax_write(), then the unpacked code will be:
WARN_ON_ONCE(vcpu->arch.guest_state_protected); if (is_long_mode(vcpu)) kvm_x86_call(get_cs_db_l_bits)(vcpu, &cs_db, &cs_l); else cs_l = 0; if (cs_l) vcpu->arch.regs[VCPU_REGS_RAX] = ret; else vcpu->arch.regs[VCPU_REGS_RAX] = (u32)ret;
whereas the (correct) behavior here is:
if (vcpu->arch.guest_state_protected) cs_l = 1; else if (is_long_mode(vcpu)) kvm_x86_call(get_cs_db_l_bits)(vcpu, &cs_db, &cs_l); else cs_l = 0; if (cs_l) vcpu->arch.regs[VCPU_REGS_RAX] = ret; else vcpu->arch.regs[VCPU_REGS_RAX] = (u32)ret;
I.e. using the non-raw version will trigger the WARN_ON_ONCE(), and will incorrectly truncate "ret" whenever cs_l is stale (which might be always?).
Re: [PATCH v2 08/15] KVM: x86: Add mode-aware versions of kvm_<reg>_{read,write}() helpers
Quoted 15 lines · Sean Christopherson
Quoted 14 lines · Kai Huang
Quoted 10 lines
(quoted diff) +0
-0@@ -10413,29 +10413,30 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
if (!is_64_bit_hypercall(vcpu))
ret = (u32)ret;arch/x86/kvm/x86.c +1
-1-kvm_rax_write (vcpu, ret); +kvm_rax_write_raw (vcpu, ret);return
kvm_skip_emulated_instruction(vcpu); }Nit: AFAICT if we use
kvm_rax_write(vcpu, ret)instead of the "raw" version here, we can then remove the
Quoted 4 lines
Quoted 2 lines
if (!is_64_bit_hypercall(vcpu)) ret = (u32)ret;No, because sneakily,
is_64_bit_hypercall()!=is_64_bit_mode(vcpu).
Oh I missed this :-( sorry for the noise.
Re: [PATCH v2 08/15] KVM: x86: Add mode-aware versions of kvm_<reg>_{read,write}() helpers
In reply to Sean ChristophersonQuoted 48 lines · Sean Christopherson
Quoted 14 lines · Kai Huang
Quoted 10 lines
(quoted diff) +0
-0@@ -10413,29 +10413,30 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
if (!is_64_bit_hypercall(vcpu))
ret = (u32)ret;arch/x86/kvm/x86.c +1
-1-kvm_rax_write (vcpu, ret); +kvm_rax_write_raw (vcpu, ret);return
kvm_skip_emulated_instruction(vcpu); }Nit: AFAICT if we use
kvm_rax_write(vcpu, ret)instead of the "raw" version here, we can then remove theQuoted 2 lines
if (!is_64_bit_hypercall(vcpu)) ret = (u32)ret;No, because sneakily,
is_64_bit_hypercall()!=is_64_bit_mode(vcpu). And because we also need to avoid callingis_64_bit_mode(). If we usekvm_rax_write(), then the unpacked code will be:WARN_ON_ONCE(vcpu->arch.guest_state_protected); if (is_long_mode(vcpu)) kvm_x86_call(get_cs_db_l_bits)(vcpu, &cs_db, &cs_l); else cs_l = 0; if (cs_l) vcpu->arch.regs[VCPU_REGS_RAX] = ret; else vcpu->arch.regs[VCPU_REGS_RAX] = (u32)ret;whereas the (correct) behavior here is:
if (vcpu->arch.guest_state_protected) cs_l = 1; else if (is_long_mode(vcpu)) kvm_x86_call(get_cs_db_l_bits)(vcpu, &cs_db, &cs_l); else cs_l = 0; if (cs_l) vcpu->arch.regs[VCPU_REGS_RAX] = ret; else vcpu->arch.regs[VCPU_REGS_RAX] = (u32)ret;
Quoted 2 lines
I.e. using the non-raw version will trigger the
WARN_ON_ONCE(), and will incorrectly truncate "ret" whenevercs_lis stale (which might be always?).
FWIW, I sanity tested that booting/destroying both TD and VMX guests worked fine. I have no environment to test SVM and Xen related parts, though.
Sean Christopherson
maintainerPATCH v2 9/15 KVM: x86: Drop non-raw kvm_<reg>_write() helpers
In reply to Sean ChristophersonDrop the non-raw, mode-aware kvm_<reg>_write() helpers as there is no usage in KVM, and in all likelihood there will never be usage in KVM as use of hardcoded registers in instructions is uncommon, and *modifying* hardcoded registers is practically unheard of. While there are a few instructions that modify registers in mode-aware ways, e.g. REP string and some ENCLS varieties, the odds of KVM needing to emulate such instructions (outside of the fully emulator) are vanishingly small.
Drop kvm_<reg>_write() to prevent incorrect usage; _if_ a new instruction comes along that needs to modify a hardcoded register, this can be reverted.
No functional change intended.
--- arch/x86/kvm/regs.h | 5 ----- 1 file changed, 5 deletions(-)
arch/x86/kvm/regs.h +0 -5
@@ -61,11 +61,6 @@ static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu) { \ return vcpu->arch.regs[VCPU_REGS_##uname] & kvm_reg_mode_mask(vcpu); \ } \ -static __always_inline void kvm_##lname##_write(struct kvm_vcpu *vcpu, \ - unsigned long val) \ -{ \ - vcpu->arch.regs[VCPU_REGS_##uname] = val & kvm_reg_mode_mask(vcpu); \ -} \ static __always_inline unsigned long kvm_##lname##_read_raw(struct kvm_vcpu *vcpu) \ { \ return vcpu->arch.regs[VCPU_REGS_##uname]; \
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 09/15] KVM: x86: Drop non-raw kvm_<reg>_write() helpers
Quoted 8 lines · Sean Christopherson
Drop the non-raw, mode-aware kvm_<reg>_write() helpers as there is no usage in KVM, and in all likelihood there will never be usage in KVM as use of hardcoded registers in instructions is uncommon, and *modifying* hardcoded registers is practically unheard of. While there are a few instructions that modify registers in mode-aware ways, e.g. REP string and some ENCLS varieties, the odds of KVM needing to emulate such instructions (outside of the fully emulator) are vanishingly small.
Quoted 7 lines
Drop kvm_<reg>_write() to prevent incorrect usage; _if_ a new instruction comes along that needs to modify a hardcoded register, this can be reverted.
No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Quoted 20 lines
--- arch/x86/kvm/regs.h | 5 ----- 1 file changed, 5 deletions(-)arch/x86/kvm/regs.h +0
-5@@ -61,11 +61,6 @@ static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu) { \ return vcpu->arch.regs[VCPU_REGS_##uname] & kvm_reg_mode_mask(vcpu); \ } \ -static __always_inline void kvm_##lname##_write(struct kvm_vcpu *vcpu, \ - unsigned long val) \ -{ \ - vcpu->arch.regs[VCPU_REGS_##uname] = val & kvm_reg_mode_mask(vcpu); \ -} \ static __always_inline unsigned long kvm_##lname##_read_raw(struct kvm_vcpu *vcpu) \ { \ return vcpu->arch.regs[VCPU_REGS_##uname]; \
Sean Christopherson
maintainerPATCH v2 10/15 KVM: nSVM: Use kvm_rax_read() now that it's mode-aware
In reply to Sean ChristophersonNow that kvm_rax_read() truncates the output value to 32 bits if the vCPU isn't in 64-bit mode, use it instead of the more verbose (and very technically slower) kvm_register_read().
Note! VMLOAD, VMSAVE, and VMRUN emulation are still technically buggy, as they can use EAX (versus RAX) in 64-bit mode via an operand size prefix. Don't bother trying to handle that case, as it would require decoding the code stream, which would open an entirely different can of worms, and in practice no sane guest would shove garbage into RAX[63:32] and then execute VMLOAD/VMSAVE/VMRUN with just EAX.
No functional change intended.
--- arch/x86/kvm/svm/nested.c | 2 +- arch/x86/kvm/svm/svm.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/nested.c +1 -1
@@ -1119,7 +1119,7 @@ int nested_svm_vmrun(struct kvm_vcpu *vcpu)if (WARN_ON_ONCE (!svm->nested.initialized))return -EINVAL ; - vmcb12_gpa =kvm_register_read (vcpu,VCPU_REGS_RAX ); + vmcb12_gpa =kvm_rax_read (vcpu);if (!page_address_valid (vcpu, vmcb12_gpa)) {kvm_inject_gp (vcpu,0 );return 1 ;
arch/x86/kvm/svm/svm.c +2 -2
@@ -2217,7 +2217,7 @@ static int intr_interception(struct kvm_vcpu *vcpu)static int vmload_vmsave_interception (struct kvm_vcpu *vcpu,bool vmload) { -u64 vmcb12_gpa =kvm_register_read (vcpu,VCPU_REGS_RAX ); +u64 vmcb12_gpa =kvm_rax_read (vcpu);struct vcpu_svm *svm =to_svm (vcpu);struct vmcb *vmcb12;struct kvm_host_map map;
@@ -2325,7 +2325,7 @@ static int gp_interception(struct kvm_vcpu *vcpu)if (nested_svm_check_permissions (vcpu))return 1 ; -if (!page_address_valid (vcpu,kvm_register_read (vcpu,VCPU_REGS_RAX ))) +if (!page_address_valid (vcpu,kvm_rax_read (vcpu))) goto reinject;/*
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerPATCH v2 11/15 Revert "KVM: VMX: Read 32-bit GPR values for ENCLS instructions outside of 64-bit mode"
In reply to Sean ChristophersonNow that kvm_<reg>_read() are mode aware, i.e. are functionally equivalent to kvm_register_read(), revert aback to the less verbose versions.
No functional change intended.
This reverts commit 60919eccf6764c71cef31a1afeaa1a36b8e5ab85.
--- arch/x86/kvm/vmx/sgx.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
arch/x86/kvm/vmx/sgx.c +5 -5
@@ -225,8 +225,8 @@ static int handle_encls_ecreate(struct kvm_vcpu *vcpu)struct x86_exception ex;int r; -if (sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RBX ),32 ,32 , &pageinfo_gva) || -sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RCX ),4096 ,4096 , &secs_gva)) +if (sgx_get_encls_gva (vcpu,kvm_rbx_read (vcpu),32 ,32 , &pageinfo_gva) || +sgx_get_encls_gva (vcpu,kvm_rcx_read (vcpu),4096 ,4096 , &secs_gva))return 1 ;/*
@@ -302,9 +302,9 @@ static int handle_encls_einit(struct kvm_vcpu *vcpu)gpa_t sig_gpa, secs_gpa, token_gpa;int ret, trapnr; -if (sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RBX ),1808 ,4096 , &sig_gva) || -sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RCX ),4096 ,4096 , &secs_gva) || -sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RDX ),304 ,512 , &token_gva)) +if (sgx_get_encls_gva (vcpu,kvm_rbx_read (vcpu),1808 ,4096 , &sig_gva) || +sgx_get_encls_gva (vcpu,kvm_rcx_read (vcpu),4096 ,4096 , &secs_gva) || +sgx_get_encls_gva (vcpu,kvm_rdx_read (vcpu),304 ,512 , &token_gva))return 1 ;/*
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 11/15] Revert "KVM: VMX: Read 32-bit GPR values for ENCLS instructions outside of 64-bit mode"
Quoted 9 lines · Sean Christopherson
Now that kvm_<reg>_read() are mode aware, i.e. are functionally equivalent to
kvm_register_read(), revert aback to the less verbose versions.No functional change intended.
This reverts commit
60919eccf6764c71cef31a1afeaa1a36b8e5ab85.Acked-by Kai Huang <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 32 lines
--- arch/x86/kvm/vmx/sgx.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)arch/x86/kvm/vmx/sgx.c +5
-5@@ -225,8 +225,8 @@ static int handle_encls_ecreate(struct kvm_vcpu *vcpu)struct x86_exception ex;int r; -if (sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RBX ),32 ,32 , &pageinfo_gva) || -sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RCX ),4096 ,4096 , &secs_gva)) +if (sgx_get_encls_gva (vcpu,kvm_rbx_read (vcpu),32 ,32 , &pageinfo_gva) || +sgx_get_encls_gva (vcpu,kvm_rcx_read (vcpu),4096 ,4096 , &secs_gva))return 1 ;/* @@ -302,9 +302,9 @@ static int handle_encls_einit(struct kvm_vcpu *vcpu)gpa_t sig_gpa, secs_gpa, token_gpa;int ret, trapnr; -if (sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RBX ),1808 ,4096 , &sig_gva) || -sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RCX ),4096 ,4096 , &secs_gva) || -sgx_get_encls_gva (vcpu,kvm_register_read (vcpu,VCPU_REGS_RDX ),304 ,512 , &token_gva)) +if (sgx_get_encls_gva (vcpu,kvm_rbx_read (vcpu),1808 ,4096 , &sig_gva) || +sgx_get_encls_gva (vcpu,kvm_rcx_read (vcpu),4096 ,4096 , &secs_gva) || +sgx_get_encls_gva (vcpu,kvm_rdx_read (vcpu),304 ,512 , &token_gva))return 1 ;/*
Sean Christopherson
maintainerPATCH v2 12/15 KVM: x86: Harden is_64_bit_hypercall() against bugs on 32-bit kernels
In reply to Sean ChristophersonUnconditionally return %false for is_64_bit_hypercall() on 32-bit kernels to guard against incorrectly setting guest_state_protected, and because in a (very) hypothetical world where 32-bit KVM supports protected guests, assuming a hypercall was made in 64-bit mode is flat out wrong.
--- arch/x86/kvm/regs.h | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/kvm/regs.h +4 -0
@@ -39,12 +39,16 @@ static inline bool is_64_bit_mode(struct kvm_vcpu *vcpu)static inline bool is_64_bit_hypercall (struct kvm_vcpu *vcpu) { +#ifdef CONFIG_X86_64 /* * If running with protected guest state, the CS register is not * accessible. The hypercall register values will have had to been * provided in 64-bit mode, so assume the guest is in 64-bit. */ return vcpu->arch.guest_state_protected ||is_64_bit_mode (vcpu); +#else +return false; +#endif }static __always_inline unsigned long kvm_reg_mode_mask (struct kvm_vcpu *vcpu)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 12/15] KVM: x86: Harden is_64_bit_hypercall() against bugs on 32-bit kernels
Quoted 7 lines · Sean Christopherson
Unconditionally return %false for
is_64_bit_hypercall()on 32-bit kernels to guard against incorrectly settingguest_state_protected, and because in a (very) hypothetical world where 32-bit KVM supports protected guests, assuming a hypercall was made in 64-bit mode is flat out wrong.Reviewed-by Kai Huang <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 25 lines
--- arch/x86/kvm/regs.h | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/regs.h +4
-0@@ -39,12 +39,16 @@ static inline bool is_64_bit_mode(struct kvm_vcpu *vcpu)static inline bool is_64_bit_hypercall (struct kvm_vcpu *vcpu) { +#ifdef CONFIG_X86_64 /* * If running with protected guest state, the CS register is not * accessible. The hypercall register values will have had to been * provided in 64-bit mode, so assume the guest is in 64-bit. */ return vcpu->arch.guest_state_protected ||is_64_bit_mode (vcpu); +#else +return false; +#endif }static __always_inline unsigned long kvm_reg_mode_mask (struct kvm_vcpu *vcpu)
Sean Christopherson
maintainerPATCH v2 13/15 KVM: x86: Move update_cr8_intercept() to lapic.c
In reply to Sean ChristophersonMove update_cr8_intercept() to lapic.c so that it's globally visible in anticipation of extracting most of the register-specific code out of x86.c and into a new compilation unit. Opportunistically prefix the helper kvm_lapic_ to make its role/scope more obvious.
No functional change intended.
--- arch/x86/kvm/lapic.c | 26 ++++++++++++++++++++++++++ arch/x86/kvm/lapic.h | 1 + arch/x86/kvm/x86.c | 34 +++------------------------------- 3 files changed, 30 insertions(+), 31 deletions(-)
arch/x86/kvm/lapic.c +26 -0
@@ -2744,6 +2744,32 @@ u64 kvm_lapic_get_cr8(struct kvm_vcpu *vcpu)return (tpr &0xf0 ) >>4 ; } +void kvm_lapic_update_cr8_intercept (struct kvm_vcpu *vcpu) +{ +int max_irr, tpr; + +if (!kvm_x86_ops.update_cr8_intercept) +return ; + +if (!lapic_in_kernel (vcpu)) +return ; + +if (vcpu->arch.apic->apicv_active) +return ; + +if (!vcpu->arch.apic->vapic_addr) + max_irr =kvm_lapic_find_highest_irr (vcpu); +else + max_irr =-1 ; + +if (max_irr !=-1 ) + max_irr >>=4 ; + + tpr =kvm_lapic_get_cr8 (vcpu); + +kvm_x86_call (update_cr8_intercept)(vcpu, tpr, max_irr); +} +static void __kvm_apic_set_base (struct kvm_vcpu *vcpu,u64 value) {u64 old_value = vcpu->arch.apic_base;
arch/x86/kvm/lapic.h +1 -0
@@ -100,6 +100,7 @@ int kvm_apic_accept_events(struct kvm_vcpu *vcpu);void kvm_lapic_reset (struct kvm_vcpu *vcpu,bool init_event);u64 kvm_lapic_get_cr8 (struct kvm_vcpu *vcpu);void kvm_lapic_set_tpr (struct kvm_vcpu *vcpu,unsigned long cr8); +void kvm_lapic_update_cr8_intercept (struct kvm_vcpu *vcpu);void kvm_lapic_set_eoi (struct kvm_vcpu *vcpu);void kvm_apic_set_version (struct kvm_vcpu *vcpu);void kvm_apic_after_set_mcg_cap (struct kvm_vcpu *vcpu);
arch/x86/kvm/x86.c +3 -31
@@ -128,7 +128,6 @@ static u64 __read_mostly efer_reserved_bits = ~((u64)EFER_SCE); KVM_X2APIC_ENABLE_SUPPRESS_EOI_BROADCAST | \ KVM_X2APIC_DISABLE_SUPPRESS_EOI_BROADCAST) -static void update_cr8_intercept (struct kvm_vcpu *vcpu);static void process_nmi (struct kvm_vcpu *vcpu);static void __kvm_set_rflags (struct kvm_vcpu *vcpu,unsigned long rflags);static void store_regs (struct kvm_vcpu *vcpu);
@@ -5342,7 +5341,7 @@ static int kvm_vcpu_ioctl_set_lapic(struct kvm_vcpu *vcpu, r =kvm_apic_set_state (vcpu, s);if (r)return r; -update_cr8_intercept (vcpu); +kvm_lapic_update_cr8_intercept (vcpu);return 0 ; }
@@ -10583,33 +10582,6 @@ static void post_kvm_run_save(struct kvm_vcpu *vcpu) kvm_run->flags |=KVM_RUN_X86_GUEST_MODE ; } -static void update_cr8_intercept (struct kvm_vcpu *vcpu) -{ -int max_irr, tpr; - -if (!kvm_x86_ops.update_cr8_intercept) -return ; - -if (!lapic_in_kernel (vcpu)) -return ; - -if (vcpu->arch.apic->apicv_active) -return ; - -if (!vcpu->arch.apic->vapic_addr) - max_irr =kvm_lapic_find_highest_irr (vcpu); -else - max_irr =-1 ; - -if (max_irr !=-1 ) - max_irr >>=4 ; - - tpr =kvm_lapic_get_cr8 (vcpu); - -kvm_x86_call (update_cr8_intercept)(vcpu, tpr, max_irr); -} - -int kvm_check_nested_events (struct kvm_vcpu *vcpu) {if (kvm_test_request (KVM_REQ_TRIPLE_FAULT , vcpu)) {
@@ -11350,7 +11322,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)kvm_x86_call (enable_irq_window)(vcpu);if (kvm_lapic_enabled (vcpu)) { -update_cr8_intercept (vcpu); +kvm_lapic_update_cr8_intercept (vcpu);kvm_lapic_sync_to_vapic (vcpu); } }
@@ -12496,7 +12468,7 @@ static int __set_sregs_common(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs,kvm_set_segment (vcpu, &sregs->tr,VCPU_SREG_TR );kvm_set_segment (vcpu, &sregs->ldt,VCPU_SREG_LDTR ); -update_cr8_intercept (vcpu); +kvm_lapic_update_cr8_intercept (vcpu);/* Older userspace won't unhalt the vcpu on reset. */ if (kvm_vcpu_is_bsp(vcpu) && kvm_rip_read(vcpu) == 0xfff0 &&
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerPATCH v2 14/15 KVM: x86: Move kvm_pv_async_pf_enabled() to x86.h (as an inline)
In reply to Sean ChristophersonMove kvm_pv_async_pf_enabled() in anticipation of extracting the majority of register specific code out of x86.c.
No functional change intended.
--- arch/x86/kvm/x86.c | 12 ------------ arch/x86/kvm/x86.h | 12 ++++++++++++ 2 files changed, 12 insertions(+), 12 deletions(-)
arch/x86/kvm/x86.c +0 -12
@@ -1042,18 +1042,6 @@ bool kvm_require_dr(struct kvm_vcpu *vcpu, int dr) }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_require_dr); -static bool __kvm_pv_async_pf_enabled (u64 data) -{ -u64 mask =KVM_ASYNC_PF_ENABLED |KVM_ASYNC_PF_DELIVERY_AS_INT ; - -return (data & mask) == mask; -} - -static bool kvm_pv_async_pf_enabled (struct kvm_vcpu *vcpu) -{ -return __kvm_pv_async_pf_enabled (vcpu->arch.apf.msr_en_val); -} -static inline u64 pdptr_rsvd_bits (struct kvm_vcpu *vcpu) {return vcpu->arch.reserved_gpa_bits |rsvd_bits (5 ,8 ) |rsvd_bits (1 ,2 );
arch/x86/kvm/x86.h +12 -0
@@ -570,6 +570,18 @@ static inline bool kvm_pat_valid(u64 data)return (data | ((data &0x0202020202020202ull ) <<1 )) == data; } +static inline bool __kvm_pv_async_pf_enabled (u64 data) +{ +u64 mask =KVM_ASYNC_PF_ENABLED |KVM_ASYNC_PF_DELIVERY_AS_INT ; + +return (data & mask) == mask; +} + +static inline bool kvm_pv_async_pf_enabled (struct kvm_vcpu *vcpu) +{ +return __kvm_pv_async_pf_enabled (vcpu->arch.apf.msr_en_val); +} +/* * Trigger machine check on the host. We assume all the MSRs are already set up * by the CPU and that we still run on the same CPU as the MCE occurred on.
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerPATCH v2 15/15 KVM: x86: Move the bulk of register specific code from x86.c to regs.c
In reply to Sean ChristophersonIntroduce regs.c, and move the vast majority of register specific code out of x86.c and into regs.c. Deliberately leave behind MSR code (except for EFER, which can hardly be called an MSR), as KVM's MSR support is complex enough to warrant its own compilation unit, and doesn't have much in common with the other register code.
No functional change intended.
--- arch/x86/include/asm/kvm_host.h | 2 - arch/x86/kvm/Makefile | 4 +- arch/x86/kvm/regs.c | 829 ++++++++++++++++++++++++++++++++ arch/x86/kvm/regs.h | 16 + arch/x86/kvm/x86.c | 824 +------------------------------ arch/x86/kvm/x86.h | 2 + 6 files changed, 856 insertions(+), 821 deletions(-) create mode 100644 arch/x86/kvm/regs.c
arch/x86/include/asm/kvm_host.h +0 -2
@@ -2326,8 +2326,6 @@ static inline int __kvm_irq_line_state(unsigned long *irq_state,void kvm_inject_nmi (struct kvm_vcpu *vcpu);int kvm_get_nr_pending_nmis (struct kvm_vcpu *vcpu); -void kvm_update_dr7 (struct kvm_vcpu *vcpu); -bool __kvm_mmu_unprotect_gfn_and_retry (struct kvm_vcpu *vcpu,gpa_t cr2_or_gpa,bool always_retry);
arch/x86/kvm/Makefile +2 -2
@@ -5,8 +5,8 @@ ccflags-$(CONFIG_KVM_WERROR) += -Werror include $(srctree)/virt/kvm/Makefile.kvm -kvm-y += x86.o emulate.o irq.o lapic.o cpuid.o pmu.o mtrr.o \ - debugfs.o mmu/mmu.o mmu/page_track.o mmu/spte.o +kvm-y += x86.o emulate.o irq.o lapic.o cpuid.o pmu.o regs.o \ + mtrr.o debugfs.o mmu/mmu.o mmu/page_track.o mmu/spte.o kvm-$(CONFIG_X86_64) += mmu/tdp_iter.o mmu/tdp_mmu.o kvm-$(CONFIG_KVM_IOAPIC) += i8259.o i8254.o ioapic.o
arch/x86/kvm/regs.c new file +829 -0
829 lines not on this page · raw message
arch/x86/kvm/regs.h +16 -0
@@ -401,4 +401,20 @@ static inline bool is_guest_mode(struct kvm_vcpu *vcpu) return vcpu->arch.hflags & HF_GUEST_MASK; } +void kvm_x86_vcpu_ioctl_get_sregs2 (struct kvm_vcpu *vcpu, +struct kvm_sregs2 *sregs2); +int kvm_x86_vcpu_ioctl_set_sregs2 (struct kvm_vcpu *vcpu, +struct kvm_sregs2 *sregs2); + +void kvm_run_get_regs (struct kvm_vcpu *vcpu); +int kvm_run_set_regs (struct kvm_vcpu *vcpu); + +void kvm_update_dr0123 (struct kvm_vcpu *vcpu); +void kvm_update_dr7 (struct kvm_vcpu *vcpu); +int kvm_vcpu_ioctl_x86_get_debugregs (struct kvm_vcpu *vcpu, +struct kvm_debugregs *dbgregs); +int kvm_vcpu_ioctl_x86_set_debugregs (struct kvm_vcpu *vcpu, +struct kvm_debugregs *dbgregs); + +#endif
arch/x86/kvm/x86.c +7 -817
896 lines not on this page · raw message
arch/x86/kvm/x86.h +2 -0
@@ -414,6 +414,7 @@ int handle_ud(struct kvm_vcpu *vcpu);void kvm_deliver_exception_payload (struct kvm_vcpu *vcpu,struct kvm_queued_exception *ex); +void kvm_handle_exception_payload_quirk (struct kvm_vcpu *vcpu);int kvm_mtrr_set_msr (struct kvm_vcpu *vcpu,u32 msr,u64 data);int kvm_mtrr_get_msr (struct kvm_vcpu *vcpu,u32 msr,u64 *pdata);
@@ -604,6 +605,7 @@ static inline void kvm_machine_check(void)int kvm_spec_ctrl_test_value (u64 value);int kvm_handle_memory_failure (struct kvm_vcpu *vcpu,int r,struct x86_exception *e); +void kvm_invalidate_pcid (struct kvm_vcpu *vcpu,unsigned long pcid);int kvm_handle_invpcid (struct kvm_vcpu *vcpu,unsigned long type,gva_t gva);bool kvm_msr_allowed (struct kvm_vcpu *vcpu,u32 index,u32 type);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 15/15] KVM: x86: Move the bulk of register specific code from x86.c to regs.c
Quoted 31 lines
arch/x86/kvm/regs.c new file +29
-0+void kvm_run_get_regs (struct kvm_vcpu *vcpu) +{ +BUILD_BUG_ON (sizeof (struct kvm_sync_regs ) >SYNC_REGS_SIZE_BYTES ); + +if (vcpu->run->kvm_valid_regs &KVM_SYNC_X86_REGS ) +__get_regs (vcpu, &vcpu->run->s.regs.regs); + +if (vcpu->run->kvm_valid_regs &KVM_SYNC_X86_SREGS ) +__get_sregs (vcpu, &vcpu->run->s.regs.sregs); +} + +int kvm_run_set_regs (struct kvm_vcpu *vcpu) +{ +if (vcpu->run->kvm_dirty_regs &KVM_SYNC_X86_REGS ) { +__set_regs (vcpu, &vcpu->run->s.regs.regs); + vcpu->run->kvm_dirty_regs &= ~KVM_SYNC_X86_REGS ; + } + +if (vcpu->run->kvm_dirty_regs &KVM_SYNC_X86_SREGS ) { +struct kvm_sregs sregs = vcpu->run->s.regs.sregs; + +if (__set_sregs (vcpu, &sregs)) +return -EINVAL ; + + vcpu->run->kvm_dirty_regs &= ~KVM_SYNC_X86_SREGS ; + } + +return 0 ; +}
[...]
Quoted 22 lines
arch/x86/kvm/x86.c +1
-10@@ -12699,11 +11904,7 @@ static void store_regs(struct kvm_vcpu *vcpu) {BUILD_BUG_ON (sizeof (struct kvm_sync_regs ) >SYNC_REGS_SIZE_BYTES ); -if (vcpu->run->kvm_valid_regs &KVM_SYNC_X86_REGS ) -__get_regs (vcpu, &vcpu->run->s.regs.regs); - -if (vcpu->run->kvm_valid_regs &KVM_SYNC_X86_SREGS ) -__get_sregs (vcpu, &vcpu->run->s.regs.sregs); +kvm_run_get_regs (vcpu);if (vcpu->run->kvm_valid_regs &KVM_SYNC_X86_EVENTS ) kvm_vcpu_ioctl_x86_get_vcpu_events(@@ -12712,19 +11913,8 @@ static void store_regs(struct kvm_vcpu *vcpu)static int sync_regs (struct kvm_vcpu *vcpu) { -if (vcpu->run->kvm_dirty_regs &KVM_SYNC_X86_REGS ) { -__set_regs (vcpu, &vcpu->run->s.regs.regs); - vcpu->run->kvm_dirty_regs &= ~KVM_SYNC_X86_REGS ; - } -
arch/x86/kvm/x86.c static void store_regs(struct kvm_vcpu *vcpu) · line 11917
Quoted 10 lines
arch/x86/kvm/x86.c +2
-8-if (vcpu->run->kvm_dirty_regs &KVM_SYNC_X86_SREGS ) { -struct kvm_sregs sregs = vcpu->run->s.regs.sregs; - -if (__set_sregs (vcpu, &sregs)) -return -EINVAL ; - - vcpu->run->kvm_dirty_regs &= ~KVM_SYNC_X86_SREGS ; - } +if (kvm_run_set_regs (vcpu)) +return -EINVAL ;
Nit:
Do you think 'kvm_run_sync_regs()' is better than 'kvm_run_set_regs()'?
Because I think "sync" reflects better that vcpu->run->kvm_dirty_regs is cleared after the "set" operation.
arch/x86/kvm/x86.c static void store_regs(struct kvm_vcpu *vcpu) · line 11920
Quoted 3 lines
arch/x86/kvm/x86.c +0
-0if (vcpu->run->kvm_dirty_regs &KVM_SYNC_X86_EVENTS ) {struct kvm_vcpu_events events = vcpu->run->s.regs.events;
Also, I wonder whether it's better to add a helper for events so sync_regs() and store_regs() can be simplified to:
static int sync_regs(struct kvm_vcpu *vcpu)
{
if (kvm_run_sync_regs(vcpu))
return -EINVAL;
return kvm_run_sync_events(vcpu);
}static void store_regs(struct kvm_vcpu *vcpu)
{
kvm_run_get_regs(vcpu);
kvm_run_get_events(vcpu);
}And maybe 'kvm_run_get_regs()' could be 'kvm_run_store_regs()' too , so that the store_regs() could be:
static void store_regs(struct kvm_vcpu *vcpu)
{
kvm_run_store_regs(vcpu);
kvm_run_store_events(vcpu);
}Quoted 13 lines
arch/x86/kvm/x86.h +1
-0@@ -414,6 +414,7 @@ int handle_ud(struct kvm_vcpu *vcpu);void kvm_deliver_exception_payload (struct kvm_vcpu *vcpu,struct kvm_queued_exception *ex); +void kvm_handle_exception_payload_quirk (struct kvm_vcpu *vcpu);int kvm_mtrr_set_msr (struct kvm_vcpu *vcpu,u32 msr,u64 data);int kvm_mtrr_get_msr (struct kvm_vcpu *vcpu,u32 msr,u64 *pdata);
arch/x86/kvm/x86.h static inline void kvm_machine_check(void) · line 609
Quoted 5 lines
arch/x86/kvm/x86.h +1
-0@@ -604,6 +605,7 @@ static inline void kvm_machine_check(void)int kvm_spec_ctrl_test_value (u64 value);int kvm_handle_memory_failure (struct kvm_vcpu *vcpu,int r,struct x86_exception *e); +void kvm_invalidate_pcid (struct kvm_vcpu *vcpu,unsigned long pcid);
If I read correct, this is because "regs.c" calls kvm_invalidate_pcid() but you want to keep it in x86.c. But it seems the "x86.h" isn't included by "regs.c" directly but via other headers ("mmu.h" does include "x86.h").
Should the "regs.c" include "x86.h" directly?
Btw, I am a bit confused the relationship between "x86.h" and other headers like "mmu.h" and the new "regs.h". That is, headers like "mmu.h" include "x86.h", but headers like "regs.h" do not (instead, "x86.h" includes them).
Sean Christopherson
maintainerRe: [PATCH v2 15/15] KVM: x86: Move the bulk of register specific code from x86.c to regs.c
Quoted 22 lines · Kai Huang
Quoted 9 lines
(quoted diff) +0
-5@@ -12712,19 +11913,8 @@ static void store_regs(struct kvm_vcpu *vcpu) static int sync_regs(struct kvm_vcpu *vcpu) { - if (vcpu->run->kvm_dirty_regs & KVM_SYNC_X86_REGS) { - __set_regs(vcpu, &vcpu->run->s.regs.regs); - vcpu->run->kvm_dirty_regs &= ~KVM_SYNC_X86_REGS; - } -Review static void store_regs(struct kvm_vcpu *vcpu) · line 11917
Quoted 10 lines
(quoted diff) +2
-8- if (vcpu->run->kvm_dirty_regs & KVM_SYNC_X86_SREGS) { - struct kvm_sregs sregs = vcpu->run->s.regs.sregs; - - if (__set_sregs(vcpu, &sregs)) - return -EINVAL; - - vcpu->run->kvm_dirty_regs &= ~KVM_SYNC_X86_SREGS; - } + if (kvm_run_set_regs(vcpu)) + return -EINVAL;Nit:
Quoted 4 lines
Do you think '
kvm_run_sync_regs()' is better than 'kvm_run_set_regs()'?Because I think "sync" reflects better that
vcpu->run->kvm_dirty_regsis cleared after the "set" operation.
The problem I have with "sync" is that it doesn't communicate the direction of the sync. What about kvm_run_sync_regs_{to,from}_user()?
Quoted 20 lines
Review static void store_regs(struct kvm_vcpu *vcpu) · line 11919
Quoted 3 lines
(quoted diff) +0
-0if (vcpu->run->kvm_dirty_regs & KVM_SYNC_X86_EVENTS) { struct kvm_vcpu_events events = vcpu->run->s.regs.events;Also, I wonder whether it's better to add a helper for events so
sync_regs()andstore_regs()can be simplified to:static int
sync_regs(struct kvm_vcpu *vcpu){ if (kvm_run_sync_regs(vcpu)) return -EINVAL; return kvm_run_sync_events(vcpu); }static void
store_regs(struct kvm_vcpu *vcpu){ kvm_run_get_regs(vcpu); kvm_run_get_events(vcpu); }
Quoted 8 lines
And maybe '
kvm_run_get_regs()' could be 'kvm_run_store_regs()' too , so that thestore_regs()could be:static void
store_regs(struct kvm_vcpu *vcpu){ kvm_run_store_regs(vcpu); kvm_run_store_events(vcpu); }
{store,sync}_regs() look pretty, but IMO the overall code is uglier because we end up with super small helpers that have one caller, e.g.
static void kvm_run_sync_events_to_user(struct kvm_vcpu *vcpu)
{
if (vcpu->run->kvm_valid_regs & KVM_SYNC_X86_EVENTS)
kvm_vcpu_ioctl_x86_get_vcpu_events(vcpu, &vcpu->run->s.regs.events);}
static void store_regs(struct kvm_vcpu *vcpu)
{
BUILD_BUG_ON(sizeof(struct kvm_sync_regs) > SYNC_REGS_SIZE_BYTES);
kvm_run_sync_regs_to_user(vcpu);
kvm_run_sync_events_to_user(vcpu);}
For me, the extra "jump" is undesirable, but it allows burying __{g,s}et_{s,}regs() in regs.c, and so is a net positive for registers. But for events, it's pure overhead.
Quoted 23 lines
Quoted 13 lines
arch/x86/kvm/x86.h +1
-0@@ -414,6 +414,7 @@ int handle_ud(struct kvm_vcpu *vcpu);void kvm_deliver_exception_payload (struct kvm_vcpu *vcpu,struct kvm_queued_exception *ex); +void kvm_handle_exception_payload_quirk (struct kvm_vcpu *vcpu);int kvm_mtrr_set_msr (struct kvm_vcpu *vcpu,u32 msr,u64 data);int kvm_mtrr_get_msr (struct kvm_vcpu *vcpu,u32 msr,u64 *pdata);arch/x86/kvm/x86.h static inline void kvm_machine_check(void) · line 609
Quoted 5 lines
arch/x86/kvm/x86.h +1
-0@@ -604,6 +605,7 @@ static inline void kvm_machine_check(void)int kvm_spec_ctrl_test_value (u64 value);int kvm_handle_memory_failure (struct kvm_vcpu *vcpu,int r,struct x86_exception *e); +void kvm_invalidate_pcid (struct kvm_vcpu *vcpu,unsigned long pcid);If I read correct, this is because "
regs.c" callskvm_invalidate_pcid()but you want to keep it inx86.c. But it seems the "x86.h" isn't included by "regs.c" directly but via other headers ("mmu.h" does include "x86.h").
Quoted 1 line
Should the "
regs.c" include "x86.h" directly?
Oh, yeah, I just goofed that.
Quoted 3 lines
Btw, I am a bit confused the relationship between "
x86.h" and other headers like "mmu.h" and the new "regs.h". That is, headers like "mmu.h" include "x86.h", but headers like "regs.h" do not (instead, "x86.h" includes them).
Heh, don't look for a theme/plan, because there isn't one. Over the years, x86.h and x86.c became dumping grounds for everything that didn't have an obvious home, and so there aren't real "rules".
Hmm, though looking at all of this again, I think we're actually quite close to having somewhat sane rules. Over the past few years, I've tried multiple times to move what I felt should be KVM-internal structures from asm/kvm_host.h to x86.h, and I've failed miserably every time because inevitably even the most innocuous struct manages to have usage that leads to cyclical header dependencies and/or is used by arch-neutral KVM code.
I think it's probably time to admit I've been looking at the asm/kvm_host.h vs. x86.h split all wrong, i.e. finally give up on moving structures out of kvm_host.h, and do the exact opposite: commit to using kvm_host.h to define and declare widely used structures.
Because literally the only reason that x86.h doesn't include mmu.h is that mmu.h references struct kvm_host, which is currently defined in x86.h. If we "fix" that, then (a) we can make x86.h the "central" include everyone expects it to be, and (b) it can be the start of a cleanup of asm/kvm_host.h and a big step towards defining maintainable "rules" for what goes where. E.g. there are a pile of functional declarations in asm/kvm_host.h that can live elsewhere; if we trim those down, then the rules become:
- asm/kvm_host.h holds "common" structure definitions and associated key global variables, and things that are referenced by arch-neutral KVM.
- <area>.{c,h} holds relevant declarations and definitions.
- x86.{c,h} is the kitchen sink for everything else.
E.g. this compiles for at least one config:
--- arch/x86/include/asm/kvm_host.h | 50 ++++++++++++++++++++++++---- arch/x86/kvm/mmu.h | 20 +++++++---- arch/x86/kvm/x86.h | 59 ++++----------------------------- 3 files changed, 64 insertions(+), 65 deletions(-)
arch/x86/include/asm/kvm_host.h +44 -6
@@ -313,6 +313,50 @@ enum x86_intercept_stage;struct kvm_kernel_irqfd ;struct kvm_kernel_irq_routing_entry ; +struct kvm_caps { +/* control of guest tsc rate supported? */ +bool has_tsc_control; +/* maximum supported tsc_khz for guests */ +u32 max_guest_tsc_khz; +/* number of bits of the fractional part of the TSC scaling ratio */ +u8 tsc_scaling_ratio_frac_bits; +/* maximum allowed value of TSC scaling ratio */ +u64 max_tsc_scaling_ratio; +/* 1ull << kvm_caps.tsc_scaling_ratio_frac_bits */ +u64 default_tsc_scaling_ratio; +/* bus lock detection supported? */ +bool has_bus_lock_exit; +/* notify VM exit supported? */ +bool has_notify_vmexit; +/* bit mask of VM types */ +u32 supported_vm_types; + +u64 supported_mce_cap; +u64 supported_xcr0; +u64 supported_xss; +u64 supported_perf_cap; + +u64 supported_quirks; +u64 inapplicable_quirks; +}; +extern struct kvm_caps kvm_caps; + +struct kvm_host_values { +/* +* The host's raw MAXPHYADDR, i.e. the number of non-reserved physical +* address bits irrespective of features that repurpose legal bits, +* e.g. MKTME. +*/ +u8 maxphyaddr; + +u64 efer; +u64 xcr0; +u64 xss; +u64 s_cet; +u64 arch_capabilities; +}; +extern struct kvm_host_values kvm_host; +/* * kvm_mmu_page_role tracks the properties of a shadow page (where shadow page * also includes TDP pages) to determine whether or not a page can be used in
@@ -2056,10 +2100,6 @@ struct kvm_arch_async_pf {u64 error_code; }; -extern u32 __read_mostly kvm_nr_uret_msrs; -extern bool __read_mostly allow_smaller_maxphyaddr; -extern bool __read_mostly enable_apicv; -extern bool __read_mostly enable_ipiv;extern bool __read_mostly enable_device_posted_irqs;extern struct kvm_x86_ops kvm_x86_ops;
@@ -2151,8 +2191,6 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end);int load_pdptrs (struct kvm_vcpu *vcpu,unsigned long cr3); -extern bool tdp_enabled; -u64 vcpu_tsc_khz (struct kvm_vcpu *vcpu);/*
arch/x86/kvm/mmu.h +13 -7
@@ -4,9 +4,14 @@#include <linux/kvm_host.h> #include "regs.h" -#include "x86.h" #include "cpuid.h" +extern bool tdp_enabled; +#ifdef CONFIG_X86_64 +extern bool tdp_mmu_enabled; +#else +#define tdp_mmu_enabled false +#endif extern bool __read_mostly enable_mmio_caching;#define PT_WRITABLE_SHIFT 1
@@ -261,14 +266,10 @@ static inline bool kvm_shadow_root_allocated(struct kvm *kvm)return smp_load_acquire (&kvm->arch.shadow_root_allocated); } -#ifdef CONFIG_X86_64 -extern bool tdp_mmu_enabled; -#else -#define tdp_mmu_enabled false -#endif -int kvm_tdp_mmu_map_private_pfn (struct kvm_vcpu *vcpu,gfn_t gfn,kvm_pfn_t pfn); +bool kvm_mmu_is_mappable_memslot (const struct kvm_memory_slot *slot); +static inline bool kvm_memslots_have_rmaps (struct kvm *kvm) {return !tdp_mmu_enabled ||kvm_shadow_root_allocated (kvm);
@@ -300,6 +301,11 @@ static inline void kvm_update_page_stats(struct kvm *kvm, int level, int count)atomic64_add (count, &kvm->stat.pages[level -1 ]); } +static inline bool mmu_is_nested (struct kvm_vcpu *vcpu) +{ +return vcpu->arch.walk_mmu == &vcpu->arch.nested_mmu; +} + static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, gpa_t gpa, u64 access,
arch/x86/kvm/x86.h +7 -52
83 lines not on this page · raw message
base-commit: b99808a11a42edc2cecced7adf57c2ac231bdb68
Re: [PATCH v2 15/15] KVM: x86: Move the bulk of register specific code from x86.c to regs.c
Quoted 23 lines · Sean Christopherson
Quoted 22 lines · Kai Huang
Quoted 9 lines
(quoted diff) +0
-5@@ -12712,19 +11913,8 @@ static void store_regs(struct kvm_vcpu *vcpu) static int sync_regs(struct kvm_vcpu *vcpu) { - if (vcpu->run->kvm_dirty_regs & KVM_SYNC_X86_REGS) { - __set_regs(vcpu, &vcpu->run->s.regs.regs); - vcpu->run->kvm_dirty_regs &= ~KVM_SYNC_X86_REGS; - } -Review static void store_regs(struct kvm_vcpu *vcpu) · line 11917
Quoted 10 lines
(quoted diff) +2
-8- if (vcpu->run->kvm_dirty_regs & KVM_SYNC_X86_SREGS) { - struct kvm_sregs sregs = vcpu->run->s.regs.sregs; - - if (__set_sregs(vcpu, &sregs)) - return -EINVAL; - - vcpu->run->kvm_dirty_regs &= ~KVM_SYNC_X86_SREGS; - } + if (kvm_run_set_regs(vcpu)) + return -EINVAL;Nit:
Quoted 7 lines
Quoted 4 lines
Do you think '
kvm_run_sync_regs()' is better than 'kvm_run_set_regs()'?Because I think "sync" reflects better that
vcpu->run->kvm_dirty_regsis cleared after the "set" operation.The problem I have with "sync" is that it doesn't communicate the direction of the sync. What about kvm_run_sync_regs_{to,from}_user()?
Yeah that's better to me too.
Quoted 45 lines
Quoted 21 lines
Review static void store_regs(struct kvm_vcpu *vcpu) · line 11919
Quoted 4 lines
(quoted diff) +0
-0if (vcpu->run->kvm_dirty_regs & KVM_SYNC_X86_EVENTS) { struct kvm_vcpu_events events = vcpu->run->s.regs.events;Also, I wonder whether it's better to add a helper for events so
sync_regs()andstore_regs()can be simplified to:static int
sync_regs(struct kvm_vcpu *vcpu){ if (kvm_run_sync_regs(vcpu)) return -EINVAL; return kvm_run_sync_events(vcpu); }static void
store_regs(struct kvm_vcpu *vcpu){ kvm_run_get_regs(vcpu); kvm_run_get_events(vcpu); }Quoted 8 lines
And maybe '
kvm_run_get_regs()' could be 'kvm_run_store_regs()' too , so that thestore_regs()could be:static void
store_regs(struct kvm_vcpu *vcpu){ kvm_run_store_regs(vcpu); kvm_run_store_events(vcpu); }{store,sync}_regs() look pretty, but IMO the overall code is uglier because we end up with super small helpers that have one caller, e.g.
static void
kvm_run_sync_events_to_user(struct kvm_vcpu *vcpu){ if (vcpu->run->kvm_valid_regs & KVM_SYNC_X86_EVENTS) kvm_vcpu_ioctl_x86_get_vcpu_events(vcpu, &vcpu->run->s.regs.events);}
static void
store_regs(struct kvm_vcpu *vcpu){ BUILD_BUG_ON(sizeof(struct kvm_sync_regs) > SYNC_REGS_SIZE_BYTES); kvm_run_sync_regs_to_user(vcpu); kvm_run_sync_events_to_user(vcpu);
Quoted 5 lines
}
For me, the extra "jump" is undesirable, but it allows burying __{g,s}et_{s,}regs() in
regs.c, and so is a net positive for registers. But for events, it's pure overhead.
Sure.
Just wondering is it possible we might want to move events handling to some other C file since you are cleanup x86.c? But we can deal with this when it happens.
Quoted 28 lines
Quoted 24 lines
Quoted 14 lines
arch/x86/kvm/x86.h +1
-0@@ -414,6 +414,7 @@ int handle_ud(struct kvm_vcpu *vcpu);void kvm_deliver_exception_payload (struct kvm_vcpu *vcpu,struct kvm_queued_exception *ex); +void kvm_handle_exception_payload_quirk (struct kvm_vcpu *vcpu);int kvm_mtrr_set_msr (struct kvm_vcpu *vcpu,u32 msr,u64 data);int kvm_mtrr_get_msr (struct kvm_vcpu *vcpu,u32 msr,u64 *pdata);arch/x86/kvm/x86.h static inline void kvm_machine_check(void) · line 609
Quoted 5 lines
arch/x86/kvm/x86.h +1
-0@@ -604,6 +605,7 @@ static inline void kvm_machine_check(void)int kvm_spec_ctrl_test_value (u64 value);int kvm_handle_memory_failure (struct kvm_vcpu *vcpu,int r,struct x86_exception *e); +void kvm_invalidate_pcid (struct kvm_vcpu *vcpu,unsigned long pcid);If I read correct, this is because "
regs.c" callskvm_invalidate_pcid()but you want to keep it inx86.c. But it seems the "x86.h" isn't included by "regs.c" directly but via other headers ("mmu.h" does include "x86.h").Quoted 1 line
Should the "
regs.c" include "x86.h" directly?Oh, yeah, I just goofed that.
Quoted 7 lines
Quoted 3 lines
Btw, I am a bit confused the relationship between "
x86.h" and other headers like "mmu.h" and the new "regs.h". That is, headers like "mmu.h" include "x86.h", but headers like "regs.h" do not (instead, "x86.h" includes them).Heh, don't look for a theme/plan, because there isn't one. Over the years,
x86.handx86.cbecame dumping grounds for everything that didn't have an obvious home, and so there aren't real "rules".
My guess too.
Quoted 7 lines
Hmm, though looking at all of this again, I think we're actually quite close to having somewhat sane rules. Over the past few years, I've tried multiple times to move what I felt should be KVM-internal structures from
asm/kvm_host.htox86.h, and I've failed miserably every time because inevitably even the most innocuous struct manages to have usage that leads to cyclical header dependencies and/or is used by arch-neutral KVM code.
The problem is some other kernel code includes <linux/kvm_host.h> (which in turn includes <asm/kvm_host.h>) but the KVM internal structures have nothing to do with them.
E.g., some drivers are using <linux/kvm_host.h>:
#$ grep kvm_host.h drivers/ -Rn drivers/vfio/pci/vfio_pci_zdev.c:14:#include <linux/kvm_host.h> drivers/vfio/vfio_main.c:20:#include <linux/kvm_host.h> drivers/firmware/arm_sdei.c:19:#include <linux/kvm_host.h> drivers/hwtracing/coresight/coresight-trbe.c:20:#include <linux/kvm_host.h> drivers/hwtracing/coresight/coresight-etm4x-core.c:10:#include <linux/kvm_host.h> drivers/s390/crypto/vfio_ap_ops.c:17:#include <linux/kvm_host.h> drivers/s390/crypto/vfio_ap_private.h:20:#include <linux/kvm_host.h>
But looking at them, AFAICT what they need is only some structure declarations (e.g., 'struct kvm;') for type safety (plus some function declarations), but don't actually need to see the actual structure.
For x86, AFAICT there's (only) "arch/x86/events/intel/core.c" actually uses the 'struct kvm_pmu', though. I haven't checked other ARCHs whether there's cases actually need to use any structure.
Quoted 5 lines
I think it's probably time to admit I've been looking at the
asm/kvm_host.hvs.x86.hsplit all wrong, i.e. finally give up on moving structures out ofkvm_host.h, and do the exact opposite: commit to usingkvm_host.hto define and declare widely used structures.
If the structure(s) are only used within arch/x86/kvm/, it doesn't seem right to define them in asm/kvm_host.h?
Quoted 4 lines
Because literally the only reason that
x86.hdoesn't includemmu.his thatmmu.hreferences structkvm_host, which is currently defined inx86.h.
Yes. But I wouldn't worry about this too much since it's a small thing we can always find a way to fix. E.g., we can move kvm_mmu_max_gfn() out of "mmu.h" (with a renaming perhaps).
Quoted 9 lines
If we "fix"
that, then (a) we can makex86.hthe "central" include everyone expects it to be, and (b) it can be the start of a cleanup ofasm/kvm_host.hand a big step towards defining maintainable "rules" for what goes where. E.g. there are a pile of functional declarations inasm/kvm_host.hthat can live elsewhere; if we trim those down, then the rules become:-
asm/kvm_host.hholds "common" structure definitions and associated key global variables, and things that are referenced by arch-neutral KVM.
It's a bit weird the arch-neutral KVM code needs to reference variables in asm/kvm_host.h, and I am afraid the "common" structure definitions will effectively be a lot of structures only used by arch/x86/kvm/.
Which isn't necessarily a bad thing, from the perspective we might finally clean this up by a giant move.
E.g., <linux/kvm_types.h> is already used by other kernel components where they don't need <linux/kvm_host.h>. Ideally, maybe eventually we can use <linux/kvm_types.h> and <asm/kvm_types.h> for things needed by other kernel components, or keep <linux/kvm_host.h> and <asm/kvm_host.h> minimal after moving majority things to some KVM internal headers.
E.g., maybe:
virt/kvm/include/kvm_host.h arch/x86/kvm/kvm_host.h (can even be merged to x86.h)
I think the problem is "struct kvm_arch" and "struct kvm_vcpu_arch", that they are not a pointer but a fully embedded structure in "struct kvm" and "struct kvm_vcpu" respectively. That caused that you need to keep the actual structure definition of "struct kvm_arch" and "kvm_vcpu_arch" in asm/kvm_host.h, which in turns makes a lot of structures only used by arch/x86/kvm/ need to stay in asm/kvm_host.h.
I am not sure whether there's a mandatory requirement that "struct kvm_arch" and "struct kvm_vcpu_arch" must be fully embedded, and it would be kinda painful to covert to a pointer (e.g., there's kvm_x86_ops::vm_size), but perhaps that is also an option to consider?
Quoted 2 lines
- <area>.{c,h} holds relevant declarations and definitions.
- x86.{c,h} is the kitchen sink for everything else.
Yeah the two are reasonable to me.
Sean Christopherson
maintainerRe: [PATCH v2 15/15] KVM: x86: Move the bulk of register specific code from x86.c to regs.c
Quoted 5 lines · Kai Huang
Quoted 1 line · Sean Christopherson
On Tue, May 19, 2026, Kai Huang wrote:
Just wondering is it possible we might want to move events handling to some other C file since you are cleanup
x86.c? But we can deal with this when it happens.
Events are a hard one. There's a decent amount of code, but not _so_ much that it's a no-brainer to move them out of x86.c. And there's no super clear cut boundary, e.g. events can mean exceptions, INIT+SIPI, IRQs, APIC stuff, etc., several of which already have substantial amounts of code outside of x86.c.
Quoted 23 lines
Quoted 6 lines
Hmm, though looking at all of this again, I think we're actually quite close to having somewhat sane rules. Over the past few years, I've tried multiple times to move what I felt should be KVM-internal structures from
asm/kvm_host.htox86.h, and I've failed miserably every time because inevitably even the most innocuous struct manages to have usage that leads to cyclical header dependencies and/or is used by arch-neutral KVM code.The problem is some other kernel code includes <
linux/kvm_host.h> (which in turn includes <asm/kvm_host.h>) but the KVM internal structures have nothing to do with them.E.g., some drivers are using <
linux/kvm_host.h>:#$ grep
kvm_host.hdrivers/-Rn drivers/vfio/pci/vfio_pci_zdev.c:14:#include <linux/kvm_host.h> drivers/vfio/vfio_main.c:20:#include <linux/kvm_host.h> drivers/firmware/arm_sdei.c:19:#include <linux/kvm_host.h> drivers/hwtracing/coresight/coresight-trbe.c:20:#include <linux/kvm_host.h> drivers/hwtracing/coresight/coresight-etm4x-core.c:10:#include <linux/kvm_host.h> drivers/s390/crypto/vfio_ap_ops.c:17:#include <linux/kvm_host.h> drivers/s390/crypto/vfio_ap_private.h:20:#include <linux/kvm_host.h>
Quoted 3 lines
But looking at them, AFAICT what they need is only some structure declarations (e.g., 'struct kvm;') for type safety (plus some function declarations), but don't actually need to see the actual structure.
Ya.
Quoted 2 lines
For x86, AFAICT there's (only) "
arch/x86/events/intel/core.c" actually uses the 'structkvm_pmu', though.
I have a patch to fix that :-)
https://lore.kernel.org/all/[email protected]
Quoted 2 lines
I haven't checked other ARCHs whether there's cases actually need to use any structure.
PPC, arm64, and IIRC s390 all have assets defined by KVM that are consumed by the kernel at-large. E.g. because KVM for arm64 can't be built as a module, the kernel calls directly into KVM during boot. IIRC, PPC has similar code.
A few years ago (wow, time flies), I was able to hide KVM internals, using #ifdef shenanigans to deal with cases where non-KVM really truly needed to get at things defined in kvm_host.h
https://lore.kernel.org/all/[email protected]
More recently, I tried to standardize KVM arch=>common includes[1], to help pave the way to splitting up kvm_host.h, but then s390's crazy arm64 support killed that (at least for now).
[1] https://lore.kernel.org/all/[email protected]
[2] https://lore.kernel.org/all/[email protected]
Quoted 7 lines
Quoted 4 lines
I think it's probably time to admit I've been looking at the
asm/kvm_host.hvs.x86.hsplit all wrong, i.e. finally give up on moving structures out ofkvm_host.h, and do the exact opposite: commit to usingkvm_host.hto define and declare widely used structures.If the structure(s) are only used within
arch/x86/kvm/, it doesn't seem right to define them inasm/kvm_host.h?
The problem is that anything that feeds into kvm_vcpu_arch needs to be visible to virt/kvm. And burying kvm_x86_ops in arch/kvm/x86 would mean one-liners like kvm_arch_vcpu_blocking() couldn't be inlined.
I've looked at this far too many times :-)
Quoted 7 lines
Quoted 3 lines
Because literally the only reason that
x86.hdoesn't includemmu.his thatmmu.hreferences structkvm_host, which is currently defined inx86.h.Yes. But I wouldn't worry about this too much since it's a small thing we can always find a way to fix. E.g., we can move
kvm_mmu_max_gfn()out of "mmu.h" (with a renaming perhaps).
I hacked on moving more stuff out of x86.{c,h} and kvm_host.h. The diff stats are quite promising :-)
arch/x86/include/asm/kvm_host.h | 444 ++------------- arch/x86/kvm/x86.c | 3784 +++----------------------------------------------------------------------------------------------------------------------- arch/x86/kvm/x86.h | 474 ++++++++--------
Quoted 35 lines
Quoted 9 lines
If we "fix"
that, then (a) we can makex86.hthe "central" include everyone expects it to be, and (b) it can be the start of a cleanup ofasm/kvm_host.hand a big step towards defining maintainable "rules" for what goes where. E.g. there are a pile of functional declarations inasm/kvm_host.hthat can live elsewhere; if we trim those down, then the rules become:-
asm/kvm_host.hholds "common" structure definitions and associated key global variables, and things that are referenced by arch-neutral KVM.It's a bit weird the arch-neutral KVM code needs to reference variables in
asm/kvm_host.h, and I am afraid the "common" structure definitions will effectively be a lot of structures only used byarch/x86/kvm/.Which isn't necessarily a bad thing, from the perspective we might finally clean this up by a giant move.
E.g., <
linux/kvm_types.h> is already used by other kernel components where they don't need <linux/kvm_host.h>. Ideally, maybe eventually we can use <linux/kvm_types.h> and <asm/kvm_types.h> for things needed by other kernel components, or keep <linux/kvm_host.h> and <asm/kvm_host.h> minimal after moving majority things to some KVM internal headers.E.g., maybe:
virt/kvm/include/kvm_host.harch/x86/kvm/kvm_host.h(can even be merged tox86.h)I think the problem is "struct
kvm_arch" and "structkvm_vcpu_arch", that they are not a pointer but a fully embedded structure in "struct kvm" and "structkvm_vcpu" respectively. That caused that you need to keep the actual structure definition of "structkvm_arch" and "kvm_vcpu_arch" inasm/kvm_host.h, which in turns makes a lot of structures only used byarch/x86/kvm/need to stay inasm/kvm_host.h.
Quoted 4 lines
I am not sure whether there's a mandatory requirement that "struct
kvm_arch" and "structkvm_vcpu_arch" must be fully embedded, and it would be kinda painful to covert to a pointer (e.g., there'skvm_x86_ops::vm_size), but perhaps that is also an option to consider?
The idea I had in the past, and where I was going with things before s390's love for arm64 came along, was to add a kvm_arch.h in arch/<arch>/kvm, and have virt/kvm include _that_ instead of kvm_host.h. That way we don't need to make any fundamental changes to structures, but we can still significantly cut down on what's exposed via kvm_host.h. At some point I'll try to take another look; it's really the s390+arm64 combo that's problematic :-/
Re: [PATCH v2 15/15] KVM: x86: Move the bulk of register specific code from x86.c to regs.c
Quoted 11 lines · Sean Christopherson
Quoted 5 lines · Kai Huang
Quoted 1 line · Sean Christopherson
On Tue, May 19, 2026, Kai Huang wrote:
Just wondering is it possible we might want to move events handling to some other C file since you are cleanup
x86.c? But we can deal with this when it happens.Events are a hard one. There's a decent amount of code, but not _so_ much that it's a no-brainer to move them out of
x86.c. And there's no super clear cut boundary, e.g. events can mean exceptions, INIT+SIPI, IRQs, APIC stuff, etc., several of which already have substantial amounts of code outside ofx86.c.
Yes agreed.
Quoted 28 lines
Quoted 24 lines
Quoted 7 lines
Hmm, though looking at all of this again, I think we're actually quite close to having somewhat sane rules. Over the past few years, I've tried multiple times to move what I felt should be KVM-internal structures from
asm/kvm_host.htox86.h, and I've failed miserably every time because inevitably even the most innocuous struct manages to have usage that leads to cyclical header dependencies and/or is used by arch-neutral KVM code.The problem is some other kernel code includes <
linux/kvm_host.h> (which in turn includes <asm/kvm_host.h>) but the KVM internal structures have nothing to do with them.E.g., some drivers are using <
linux/kvm_host.h>:#$ grep
kvm_host.hdrivers/-Rn drivers/vfio/pci/vfio_pci_zdev.c:14:#include <linux/kvm_host.h> drivers/vfio/vfio_main.c:20:#include <linux/kvm_host.h> drivers/firmware/arm_sdei.c:19:#include <linux/kvm_host.h> drivers/hwtracing/coresight/coresight-trbe.c:20:#include <linux/kvm_host.h> drivers/hwtracing/coresight/coresight-etm4x-core.c:10:#include <linux/kvm_host.h> drivers/s390/crypto/vfio_ap_ops.c:17:#include <linux/kvm_host.h> drivers/s390/crypto/vfio_ap_private.h:20:#include <linux/kvm_host.h>Quoted 3 lines
But looking at them, AFAICT what they need is only some structure declarations (e.g., 'struct kvm;') for type safety (plus some function declarations), but don't actually need to see the actual structure.
Quoted 8 lines
Ya.
Quoted 2 lines
For x86, AFAICT there's (only) "
arch/x86/events/intel/core.c" actually uses the 'structkvm_pmu', though.I have a patch to fix that :-)
Oh great!
Quoted 8 lines
Quoted 3 lines
I haven't checked other ARCHs whether there's cases actually need to use any structure.
PPC, arm64, and IIRC s390 all have assets defined by KVM that are consumed by the kernel at-large. E.g. because KVM for arm64 can't be built as a module, the kernel calls directly into KVM during boot. IIRC, PPC has similar code.
Quoted 5 lines
A few years ago (wow, time flies), I was able to hide KVM internals, using #ifdef shenanigans to deal with cases where non-KVM really truly needed to get at things defined in
kvm_host.h
Oh I never thought from this perspective (thanks for the info):
--
Hiding KVM details for all architectures will, in the very distant future, allow loading a new (or old) KVM module without needing to rebuild and reboot the entire kernel, or to even allow loading and running multiple versions of KVM simultaneously on a single host.
--
Quoted 7 lines
More recently, I tried to standardize KVM arch=>common includes[1], to help pave the way to splitting up
kvm_host.h, but then s390's crazy arm64 support killed that (at least for now).[1] https://lore.kernel.org/all/[email protected]
[2] https://lore.kernel.org/all/[email protected]
:-)
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
I think it's probably time to admit I've been looking at the
asm/kvm_host.hvs.x86.hsplit all wrong, i.e. finally give up on moving structures out ofkvm_host.h, and do the exact opposite: commit to usingkvm_host.hto define and declare widely used structures.
Quoted 6 lines
Quoted 2 lines
If the structure(s) are only used within
arch/x86/kvm/, it doesn't seem right to define them inasm/kvm_host.h?The problem is that anything that feeds into
kvm_vcpu_archneeds to be visible tovirt/kvm.
Yeah that's the problem.
Quoted 2 lines
And burying
kvm_x86_opsinarch/kvm/x86would mean one-liners likekvm_arch_vcpu_blocking()couldn't be inlined.
Oh right, sad but acceptable tradeoff I guess.
Quoted 11 lines
I've looked at this far too many times :-)
Quoted 7 lines
Quoted 3 lines
Because literally the only reason that
x86.hdoesn't includemmu.his thatmmu.hreferences structkvm_host, which is currently defined inx86.h.Yes. But I wouldn't worry about this too much since it's a small thing we can always find a way to fix. E.g., we can move
kvm_mmu_max_gfn()out of "mmu.h" (with a renaming perhaps).
Quoted 7 lines
I hacked on moving more stuff out of x86.{c,h} and
kvm_host.h. The diff stats are quite promising :-)arch/x86/include/asm/kvm_host.h | 444 ++------------- arch/x86/kvm/x86.c | 3784 +++----------------------------------------------------------------------------------------------------------------------- arch/x86/kvm/x86.h | 474 ++++++++--------
Indeed!
Quoted 35 lines
Quoted 35 lines
Quoted 9 lines
If we "fix"
that, then (a) we can makex86.hthe "central" include everyone expects it to be, and (b) it can be the start of a cleanup ofasm/kvm_host.hand a big step towards defining maintainable "rules" for what goes where. E.g. there are a pile of functional declarations inasm/kvm_host.hthat can live elsewhere; if we trim those down, then the rules become:-
asm/kvm_host.hholds "common" structure definitions and associated key global variables, and things that are referenced by arch-neutral KVM.It's a bit weird the arch-neutral KVM code needs to reference variables in
asm/kvm_host.h, and I am afraid the "common" structure definitions will effectively be a lot of structures only used byarch/x86/kvm/.Which isn't necessarily a bad thing, from the perspective we might finally clean this up by a giant move.
E.g., <
linux/kvm_types.h> is already used by other kernel components where they don't need <linux/kvm_host.h>. Ideally, maybe eventually we can use <linux/kvm_types.h> and <asm/kvm_types.h> for things needed by other kernel components, or keep <linux/kvm_host.h> and <asm/kvm_host.h> minimal after moving majority things to some KVM internal headers.E.g., maybe:
virt/kvm/include/kvm_host.harch/x86/kvm/kvm_host.h(can even be merged tox86.h)I think the problem is "struct
kvm_arch" and "structkvm_vcpu_arch", that they are not a pointer but a fully embedded structure in "struct kvm" and "structkvm_vcpu" respectively. That caused that you need to keep the actual structure definition of "structkvm_arch" and "kvm_vcpu_arch" inasm/kvm_host.h, which in turns makes a lot of structures only used byarch/x86/kvm/need to stay inasm/kvm_host.h.
Quoted 9 lines
Quoted 4 lines
I am not sure whether there's a mandatory requirement that "struct
kvm_arch" and "structkvm_vcpu_arch" must be fully embedded, and it would be kinda painful to covert to a pointer (e.g., there'skvm_x86_ops::vm_size), but perhaps that is also an option to consider?The idea I had in the past, and where I was going with things before s390's love for arm64 came along, was to add a
kvm_arch.hin arch/<arch>/kvm, and havevirt/kvminclude _that_ instead ofkvm_host.h.
Not sure whether there's other code doing so? :-)
Quoted 4 lines
That way we don't need to make any fundamental
changes to structures, but we can still significantly cut down on what's exposed viakvm_host.h.
Yeah.
I saw below from you in [1]:
--
We've explore several alternatives to the #ifdef __KVM__ approach, and they all sucked, hard. What I really wanted (and still want) to do, is to bury the bulk of kvm_host.h (and other KVM headers) in virt/kvm, but every attempt to do that ended in flames. Even with the __KVM__ guards in place, each architecture's kvm_host.h is too intertwined with the common kvm_host.h, and trying to extract small-ish pieces just doesn't work (each patch inevitably snowballed into a gigantic beast).
The other idea we considered (which I thought of, and feel dirty for even proposing it internally), is to move all headers under virt/kvm, add virt/kvm/include to the global header path, and then have KVM x86 omit virt/kvm/include when configured to hide KVM internals. I hate this idea because it sets a bad precedent, and requires a lot of file movement without providing any benefit to other architectures. E.g. I hope that guarding KVM internals with #ifdef __KVM__ will allow us to slowly clean things up so that some day KVM only exposes a handful of APIs to the rest of the kernel (probably a pipe dream).
--
I haven't looked into details of your #ifdef __KVM__ approach yet, but seems you don't quite like moving KVM internal staff to virt/kvm/include/ ?
But if we want to hide KVM internal structures, I don't see any other options except virt/kvm/include/ is the place to go?
Btw, have you considered reverting the inclusion of "strut kvm" and "struct kvm_arch" (and the vcpu structure), i.e., to make "struct kvm_arch" include "struct kvm"? I don't have any clue of whether it is feasible or how much effort it needs, though -- it's just something came to mind when replying.
[1]: https://lore.kernel.org/all/[email protected]/
Quoted 2 lines
At some point I'll try to take another look; it's really the s390+arm64 combo that's problematic :-/
If you want, I can take a look. I think I'll have bandwidth in near feature.
Given you have tried multiple times so I am not sure what I can achieve, though.
Anyway, seems "allow loading a new (or old) KVM module without needing to rebuild and reboot the entire kernel" is a good reason to do this.
Sean Christopherson
maintainerRe: [PATCH v2 15/15] KVM: x86: Move the bulk of register specific code from x86.c to regs.c
Quoted 43 lines · Kai Huang
Quoted 10 lines · Sean Christopherson
Quoted 4 lines · Kai Huang
I am not sure whether there's a mandatory requirement that "struct
kvm_arch" and "structkvm_vcpu_arch" must be fully embedded, and it would be kinda painful to covert to a pointer (e.g., there'skvm_x86_ops::vm_size), but perhaps that is also an option to consider?The idea I had in the past, and where I was going with things before s390's love for arm64 came along, was to add a
kvm_arch.hin arch/<arch>/kvm, and havevirt/kvminclude _that_ instead ofkvm_host.h.Not sure whether there's other code doing so? :-)
Quoted 4 lines
That way we don't need to make any fundamental
changes to structures, but we can still significantly cut down on what's exposed viakvm_host.h.Yeah.
I saw below from you in [1]:
--
We've explore several alternatives to the #ifdef__KVM__approach, and they all sucked, hard. What I really wanted (and still want) to do, is to bury the bulk ofkvm_host.h(and other KVM headers) invirt/kvm, but every attempt to do that ended in flames. Even with the__KVM__guards in place, each architecture'skvm_host.his too intertwined with the commonkvm_host.h, and trying to extract small-ish pieces just doesn't work (each patch inevitably snowballed into a gigantic beast).The other idea we considered (which I thought of, and feel dirty for even proposing it internally), is to move all headers under
virt/kvm, addvirt/kvm/includeto the global header path, and then have KVM x86 omitvirt/kvm/includewhen configured to hide KVM internals. I hate this idea because it sets a bad precedent, and requires a lot of file movement without providing any benefit to other architectures. E.g. I hope that guarding KVM internals with #ifdef__KVM__will allow us to slowly clean things up so that some day KVM only exposes a handful of APIs to the rest of the kernel (probably a pipe dream).
--
Quoted 2 lines
I haven't looked into details of your #ifdef
__KVM__approach yet, but seems you don't quite like moving KVM internal staff tovirt/kvm/include/?
Not for arch code, which is the trickiest bit to handle.
Quoted 2 lines
But if we want to hide KVM internal structures, I don't see any other options except
virt/kvm/include/is the place to go?
arch/$(ARCH)/kvm/kvm_arch.h is the obvious approach. Code in virt/kvm can reach arch/$(ARCH)/kvm, we just need to add it to the include path. That's why I was working on unifying the include definitions.
Quoted 5 lines
Btw, have you considered reverting the inclusion of "strut kvm" and "struct
kvm_arch" (and the vcpu structure), i.e., to make "structkvm_arch" include "struct kvm"? I don't have any clue of whether it is feasible or how much effort it needs, though -- it's just something came to mind when replying.
Quoted 8 lines
[1]: https://lore.kernel.org/all/[email protected]/
Quoted 2 lines
At some point I'll try to take another look; it's really the s390+arm64 combo that's problematic :-/
If you want, I can take a look. I think I'll have bandwidth in near feature.
Given you have tried multiple times so I am not sure what I can achieve, though.
Consolidating includes and creating arch/$(ARCH)/kvm/kvm_arch.h should be more doable, what has failed spectacularly over and over is effectively trying to hide most of asm/kvm_host.h, which sounds *really* stupid when I phrase it that way :-)
Quoted 2 lines
Anyway, seems "allow loading a new (or old) KVM module without needing to rebuild and reboot the entire kernel" is a good reason to do this.
Note, we've scrapped upstreaming multi-KVM, and I don't see it ever getting accepted upstream, as live update is far superior, e.g. isn't limited to just KVM, doesn't have the same orchestration or testing problems, etc.
And FWIW, today, you _can_ reload a new (or old) KVM module without rebuilding the kernel or rebooting the host, it's just that you can't modify certain structures.
Re: [PATCH v2 15/15] KVM: x86: Move the bulk of register specific code from x86.c to regs.c
Quoted 47 lines · Sean Christopherson
Quoted 43 lines · Kai Huang
Quoted 10 lines · Sean Christopherson
Quoted 4 lines · Kai Huang
I am not sure whether there's a mandatory requirement that "struct
kvm_arch" and "structkvm_vcpu_arch" must be fully embedded, and it would be kinda painful to covert to a pointer (e.g., there'skvm_x86_ops::vm_size), but perhaps that is also an option to consider?The idea I had in the past, and where I was going with things before s390's love for arm64 came along, was to add a
kvm_arch.hin arch/<arch>/kvm, and havevirt/kvminclude _that_ instead ofkvm_host.h.Not sure whether there's other code doing so? :-)
Quoted 4 lines
That way we don't need to make any fundamental
changes to structures, but we can still significantly cut down on what's exposed viakvm_host.h.Yeah.
I saw below from you in [1]:
--
We've explore several alternatives to the #ifdef__KVM__approach, and they all sucked, hard. What I really wanted (and still want) to do, is to bury the bulk ofkvm_host.h(and other KVM headers) invirt/kvm, but every attempt to do that ended in flames. Even with the__KVM__guards in place, each architecture'skvm_host.his too intertwined with the commonkvm_host.h, and trying to extract small-ish pieces just doesn't work (each patch inevitably snowballed into a gigantic beast).The other idea we considered (which I thought of, and feel dirty for even proposing it internally), is to move all headers under
virt/kvm, addvirt/kvm/includeto the global header path, and then have KVM x86 omitvirt/kvm/includewhen configured to hide KVM internals. I hate this idea because it sets a bad precedent, and requires a lot of file movement without providing any benefit to other architectures. E.g. I hope that guarding KVM internals with #ifdef__KVM__will allow us to slowly clean things up so that some day KVM only exposes a handful of APIs to the rest of the kernel (probably a pipe dream).
--Quoted 2 lines
I haven't looked into details of your #ifdef
__KVM__approach yet, but seems you don't quite like moving KVM internal staff tovirt/kvm/include/?
Quoted 8 lines
Not for arch code, which is the trickiest bit to handle.
Quoted 2 lines
But if we want to hide KVM internal structures, I don't see any other options except
virt/kvm/include/is the place to go?arch/$(ARCH)/kvm/kvm_arch.h is the obvious approach. Code in
virt/kvmcan reach arch/$(ARCH)/kvm, we just need to add it to the include path. That's why I was working on unifying the include definitions.
Yeah, for asm/kvm_host.h.
But if I am still following you we still need a place for linux/kvm_host.h, for which I thought virt/kvm/include/ would be the place at first glance.
Quoted 11 lines
Quoted 11 lines
Btw, have you considered reverting the inclusion of "strut kvm" and "struct
kvm_arch" (and the vcpu structure), i.e., to make "structkvm_arch" include "struct kvm"? I don't have any clue of whether it is feasible or how much effort it needs, though -- it's just something came to mind when replying.[1]: https://lore.kernel.org/all/[email protected]/
Quoted 2 lines
At some point I'll try to take another look; it's really the s390+arm64 combo that's problematic :-/
Quoted 7 lines
Quoted 3 lines
If you want, I can take a look. I think I'll have bandwidth in near feature.
Given you have tried multiple times so I am not sure what I can achieve, though.
Consolidating includes and creating arch/$(ARCH)/kvm/kvm_arch.h should be more doable,
Right.
Quoted 2 lines
what has failed spectacularly over and over is effectively trying to hide most of
asm/kvm_host.h, which sounds *really* stupid when I phrase it that way :-)
I see :-)
Quoted 10 lines
Quoted 3 lines
Anyway, seems "allow loading a new (or old) KVM module without needing to rebuild and reboot the entire kernel" is a good reason to do this.
Note, we've scrapped upstreaming multi-KVM, and I don't see it ever getting accepted upstream, as live update is far superior, e.g. isn't limited to just KVM, doesn't have the same orchestration or testing problems, etc.
And FWIW, today, you _can_ reload a new (or old) KVM module without rebuilding the kernel or rebooting the host, it's just that you can't modify certain structures.
Right.
Sean Christopherson
maintainerRe: [PATCH v2 15/15] KVM: x86: Move the bulk of register specific code from x86.c to regs.c
Quoted 5 lines · Kai Huang
Quoted 4 lines · Sean Christopherson
Quoted 2 lines · Kai Huang
But if we want to hide KVM internal structures, I don't see any other options except
virt/kvm/include/is the place to go?
Quoted 8 lines
Quoted 3 lines
arch/$(ARCH)/kvm/kvm_arch.h is the obvious approach. Code in
virt/kvmcan reach arch/$(ARCH)/kvm, we just need to add it to the include path. That's why I was working on unifying the include definitions.Yeah, for
asm/kvm_host.h.But if I am still following you we still need a place for
linux/kvm_host.h, for which I thoughtvirt/kvm/include/would be the place at first glance.
Oh, yes, the KVM-internal pieces of linuy/kvm_host.h would live in virt/kvm.
Quoted 23 lines · Sean Christopherson
Add proper, explicit "raw" versions of kvm_<reg>_{read,write}(), along with "e" versions (for hardcoded 32-bit accesses), and convert the existing kvm_<reg>_{read,write}() APIs into mode-aware variants.
This was prompted by commit
435741a4e766("KVM: SVM: Properly check RAX on #GP intercept of SVM instructions"), where usingkvm_rax_read()to get EAX/RAX would have (*very* surprisingly) been wrong as it's actually a "raw" variant that doesn't truncate accesses when the guest is in 32-bit mode.Aside from my dislike of inconsistent APIs, I really want to avoid carrying code that's subtly relying on using
kvm_register_read(...)when accessing a hardcoded register.Fix a handful of minor warts along the way.
Oh, and introduce regs.{c,h}, which just a "minor" addendum. Yosry pointed out that moving _more_ code into
x86.hwas rather gross (especially since the code split was super arbitrary), and it turns out that create regs.{c,h} isn't all that hard. In the future, I think we can also add msr.{c,h}, so I very deliberately didn't include that functionality in regs.{c,h}.
Quoted 5 lines
v2:
- Collect tags. [Yosry, Kai
- Fix some truly egregious goofs. [Binbin]
- Renamekvm_cache_regs.h=>regs.h, addregs.c. [Yosry, though he'll probably yell at me for saying this was his suggestion :-) ]
This is kinda sorta the opposite of what I suggested, but sure :P