KVM: x86: Prep KVM hypercall handling for TDX
Subsystems (4)
Sean Christopherson
maintainerv4 cover KVM: x86: Prep KVM hypercall handling for TDX
Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX
Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.
As for abusing vcpu->run->hypercall.ret... It's obviously a bit gross, but I think it's a lesser evil than having multiple a one-line wrappers just to trampoline in the return code.
v4:
- Fix an SEV-* bug where KVM trips the WARN in is_64_bit_mode().
- Add a pile of reworks to (hopefully) avoid as much duplicate code when TDX comes along.
v3: https://lore.kernel.org/all/[email protected]
Binbin Wu (1):
KVM: x86: Add a helper to check for user interception of KVM
hypercalls
Sean Christopherson (5):
KVM: x86: Play nice with protected guests in complete_hypercall_exit()
KVM: x86: Move "emulate hypercall" function declarations to x86.h
KVM: x86: Bump hypercall stat prior to fully completing hypercall
KVM: x86: Always complete hypercall via function callback
KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
arch/x86/include/asm/kvm_host.h | 6 ---- arch/x86/kvm/svm/sev.c | 4 +-- arch/x86/kvm/x86.c | 50 +++++++++++---------------------- arch/x86/kvm/x86.h | 28 ++++++++++++++++++ 4 files changed, 47 insertions(+), 41 deletions(-)
base-commit: 4d911c7abee56771b0219a9fbf0120d06bdc9c14
Signature · 1 line
2.47.0.338.g60cca15819-goog
Sean Christopherson
maintainerPATCH v4 1/6 KVM: x86: Play nice with protected guests in complete_hypercall_exit()
Use is_64_bit_hypercall() instead of is_64_bit_mode() to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable.
Hacking the sev_smoke_test selftest to generate a KVM_HC_MAP_GPA_RANGE hypercall via VMGEXIT trips the WARN:
------------[ cut here ]------------ WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercall_exit+0x44/0xe0 [kvm] Modules linked in: kvm_amd kvm ... [last unloaded: kvm] CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--392e932fa0f3-feat #470 Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024 RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm] Call Trace: <TASK> kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm] kvm_vcpu_ioctl+0x54f/0x630 [kvm] __se_sys_ioctl+0x6b/0xc0 do_syscall_64+0x83/0x160 entry_SYSCALL_64_after_hwframe+0x76/0x7e </TASK> ---[ end trace 0000000000000000 ]---
--- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/x86.c +1 -1
@@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu) {u64 ret = vcpu->run->hypercall.ret; -if (!is_64_bit_mode (vcpu)) +if (!is_64_bit_hypercall (vcpu)) ret = (u32 )ret;kvm_rax_write (vcpu, ret); ++vcpu->stat.hypercalls;
Signature · 1 line
2.47.0.338.g60cca15819-goog
Re: [PATCH v4 1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit()
Quoted 8 lines · Sean Christopherson
Use
is_64_bit_hypercall()instead ofis_64_bit_mode()to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable.Hacking the
sev_smoke_testselftest to generate aKVM_HC_MAP_GPA_RANGEhypercall via VMGEXIT trips the WARN:
Quoted 15 lines
------------[ cut here ]------------
WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercall_exit+0x44/0xe0 [kvm]
Modules linked in: kvm_amd kvm ... [last unloaded: kvm]
CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--392e932fa0f3-feat #470
Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024
RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm]
Call Trace:
<TASK>
kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm]
kvm_vcpu_ioctl+0x54f/0x630 [kvm]
__se_sys_ioctl+0x6b/0xc0
do_syscall_64+0x83/0x160
entry_SYSCALL_64_after_hwframe+0x76/0x7e
</TASK>
---[ end trace 0000000000000000 ]---Quoted 21 lines
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")Cc Tom Lendacky <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kvm/x86.c +1
-1@@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu) {u64 ret = vcpu->run->hypercall.ret; -if (!is_64_bit_mode (vcpu)) +if (!is_64_bit_hypercall (vcpu)) ret = (u32 )ret;kvm_rax_write (vcpu, ret); ++vcpu->stat.hypercalls;
Re: [PATCH v4 1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit()
In reply to Sean ChristophersonQuoted 24 lines · Sean Christopherson
Use
is_64_bit_hypercall()instead ofis_64_bit_mode()to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable.Hacking the
sev_smoke_testselftest to generate aKVM_HC_MAP_GPA_RANGEhypercall via VMGEXIT trips the WARN:------------[ cut here ]------------ WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercall_exit+0x44/0xe0 [kvm] Modules linked in: kvm_amd kvm ... [last unloaded: kvm] CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--392e932fa0f3-feat #470 Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024 RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm] Call Trace: <TASK> kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm] kvm_vcpu_ioctl+0x54f/0x630 [kvm] __se_sys_ioctl+0x6b/0xc0 do_syscall_64+0x83/0x160 entry_SYSCALL_64_after_hwframe+0x76/0x7e </TASK> ---[ end trace 0000000000000000 ]---
Quoted 4 lines
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")Cc Tom Lendacky <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 17 lines
--- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kvm/x86.c +1
-1@@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu) {u64 ret = vcpu->run->hypercall.ret; -if (!is_64_bit_mode (vcpu)) +if (!is_64_bit_hypercall (vcpu)) ret = (u32 )ret;kvm_rax_write (vcpu, ret); ++vcpu->stat.hypercalls;
Re: [PATCH v4 1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit()
In reply to Sean ChristophersonQuoted 24 lines · Sean Christopherson
Use
is_64_bit_hypercall()instead ofis_64_bit_mode()to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable.Hacking the
sev_smoke_testselftest to generate aKVM_HC_MAP_GPA_RANGEhypercall via VMGEXIT trips the WARN:------------[ cut here ]------------ WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercall_exit+0x44/0xe0 [kvm] Modules linked in: kvm_amd kvm ... [last unloaded: kvm] CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--392e932fa0f3-feat #470 Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024 RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm] Call Trace: <TASK> kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm] kvm_vcpu_ioctl+0x54f/0x630 [kvm] __se_sys_ioctl+0x6b/0xc0 do_syscall_64+0x83/0x160 entry_SYSCALL_64_after_hwframe+0x76/0x7e </TASK> ---[ end trace 0000000000000000 ]---
Quoted 4 lines
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")Cc Tom Lendacky <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 17 lines
--- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kvm/x86.c +1
-1@@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu) {u64 ret = vcpu->run->hypercall.ret; -if (!is_64_bit_mode (vcpu)) +if (!is_64_bit_hypercall (vcpu)) ret = (u32 )ret;kvm_rax_write (vcpu, ret); ++vcpu->stat.hypercalls;
Re: [PATCH v4 1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit()
In reply to Sean ChristophersonQuoted 24 lines · Sean Christopherson
Use
is_64_bit_hypercall()instead ofis_64_bit_mode()to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable.Hacking the
sev_smoke_testselftest to generate aKVM_HC_MAP_GPA_RANGEhypercall via VMGEXIT trips the WARN:------------[ cut here ]------------ WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercall_exit+0x44/0xe0 [kvm] Modules linked in: kvm_amd kvm ... [last unloaded: kvm] CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--392e932fa0f3-feat #470 Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024 RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm] Call Trace: <TASK> kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm] kvm_vcpu_ioctl+0x54f/0x630 [kvm] __se_sys_ioctl+0x6b/0xc0 do_syscall_64+0x83/0x160 entry_SYSCALL_64_after_hwframe+0x76/0x7e </TASK> ---[ end trace 0000000000000000 ]---
Quoted 4 lines
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")Cc Tom Lendacky <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 17 lines
--- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kvm/x86.c +1
-1@@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu) {u64 ret = vcpu->run->hypercall.ret; -if (!is_64_bit_mode (vcpu)) +if (!is_64_bit_hypercall (vcpu)) ret = (u32 )ret;kvm_rax_write (vcpu, ret); ++vcpu->stat.hypercalls;
Sean Christopherson
maintainerPATCH v4 2/6 KVM: x86: Add a helper to check for user interception of KVM hypercalls
In reply to Sean ChristophersonFrom: Binbin Wu <[email protected]>
Add and use user_exit_on_hypercall() to check if userspace wants to handle a KVM hypercall instead of open-coding the logic everywhere.
No functional change intended.
--- arch/x86/kvm/svm/sev.c | 4 ++-- arch/x86/kvm/x86.c | 2 +- arch/x86/kvm/x86.h | 5 +++++ 3 files changed, 8 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/sev.c +2 -2
@@ -3640,7 +3640,7 @@ static int snp_begin_psc_msr(struct vcpu_svm *svm, u64 ghcb_msr)return 1 ;/* resume guest */ } -if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 <<KVM_HC_MAP_GPA_RANGE ))) { +if (!user_exit_on_hypercall (vcpu->kvm,KVM_HC_MAP_GPA_RANGE )) {set_ghcb_msr (svm,GHCB_MSR_PSC_RESP_ERROR );return 1 ;/* resume guest */ }
@@ -3723,7 +3723,7 @@ static int snp_begin_psc(struct vcpu_svm *svm, struct psc_buffer *psc)bool huge;u64 gfn; -if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 <<KVM_HC_MAP_GPA_RANGE ))) { +if (!user_exit_on_hypercall (vcpu->kvm,KVM_HC_MAP_GPA_RANGE )) {snp_complete_psc (svm,VMGEXIT_PSC_ERROR_GENERIC );return 1 ; }
arch/x86/kvm/x86.c +1 -1
@@ -10041,7 +10041,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,u64 gpa = a0, npages = a1, attrs = a2; ret = -KVM_ENOSYS ; -if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 <<KVM_HC_MAP_GPA_RANGE ))) +if (!user_exit_on_hypercall (vcpu->kvm,KVM_HC_MAP_GPA_RANGE ))break ; if (!PAGE_ALIGNED(gpa) || !npages ||
arch/x86/kvm/x86.h +5 -0
@@ -612,4 +612,9 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size, unsigned int port, void *data, unsigned int count, int in); +static inline bool user_exit_on_hypercall (struct kvm *kvm,unsigned long hc_nr) +{ +return kvm->arch.hypercall_exit_enabled &BIT (hc_nr); +} +#endif
Signature · 1 line
2.47.0.338.g60cca15819-goog
Re: [PATCH v4 2/6] KVM: x86: Add a helper to check for user interception of KVM hypercalls
Quoted 5 lines · Sean Christopherson
From: Binbin Wu <[email protected]>
Add and use
user_exit_on_hypercall()to check if userspace wants to handle a KVM hypercall instead of open-coding the logic everywhere.
Quoted 8 lines
No functional change intended.
Signed-off-by Binbin Wu <[email protected]>Reviewed-by Isaku Yamahata <[email protected]>Reviewed-by Kai Huang <[email protected]>Reviewed-by Xiaoyao Li <[email protected]>[sean: squash into one patch, keep explicit KVM_HC_MAP_GPA_RANGE check]Signed-off-by Sean Christopherson <[email protected]>
Quoted 55 lines
--- arch/x86/kvm/svm/sev.c | 4 ++-- arch/x86/kvm/x86.c | 2 +- arch/x86/kvm/x86.h | 5 +++++ 3 files changed, 8 insertions(+), 3 deletions(-)arch/x86/kvm/svm/sev.c +2
-2@@ -3640,7 +3640,7 @@ static int snp_begin_psc_msr(struct vcpu_svm *svm, u64 ghcb_msr)return 1 ;/* resume guest */ } -if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 <<KVM_HC_MAP_GPA_RANGE ))) { +if (!user_exit_on_hypercall (vcpu->kvm,KVM_HC_MAP_GPA_RANGE )) {set_ghcb_msr (svm,GHCB_MSR_PSC_RESP_ERROR );return 1 ;/* resume guest */ }@@ -3723,7 +3723,7 @@ static int snp_begin_psc(struct vcpu_svm *svm, struct psc_buffer *psc)bool huge;u64 gfn; -if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 <<KVM_HC_MAP_GPA_RANGE ))) { +if (!user_exit_on_hypercall (vcpu->kvm,KVM_HC_MAP_GPA_RANGE )) {snp_complete_psc (svm,VMGEXIT_PSC_ERROR_GENERIC );return 1 ; }arch/x86/kvm/x86.c +1
-1@@ -10041,7 +10041,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,u64 gpa = a0, npages = a1, attrs = a2; ret = -KVM_ENOSYS ; -if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 <<KVM_HC_MAP_GPA_RANGE ))) +if (!user_exit_on_hypercall (vcpu->kvm,KVM_HC_MAP_GPA_RANGE ))break ; if (!PAGE_ALIGNED(gpa) || !npages ||arch/x86/kvm/x86.h +5
-0@@ -612,4 +612,9 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size, unsigned int port, void *data, unsigned int count, int in); +static inline bool user_exit_on_hypercall (struct kvm *kvm,unsigned long hc_nr) +{ +return kvm->arch.hypercall_exit_enabled &BIT (hc_nr); +} +#endif
Sean Christopherson
maintainerPATCH v4 3/6 KVM: x86: Move "emulate hypercall" function declarations to x86.h
In reply to Sean ChristophersonMove the declarations for the hypercall emulation APIs to x86.h. While the helpers are exported, they are intended to be consumed only KVM vendor modules, i.e. don't need to exposed to the kernel at-large.
No functional change intended.
--- arch/x86/include/asm/kvm_host.h | 6 ------ arch/x86/kvm/x86.h | 6 ++++++ 2 files changed, 6 insertions(+), 6 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -6
@@ -2181,12 +2181,6 @@ static inline void kvm_clear_apicv_inhibit(struct kvm *kvm,kvm_set_or_clear_apicv_inhibit (kvm, reason, false); } -unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl); -int kvm_emulate_hypercall (struct kvm_vcpu *vcpu); -int kvm_mmu_page_fault (struct kvm_vcpu *vcpu,gpa_t cr2_or_gpa,u64 error_code,void *insn,int insn_len);void kvm_mmu_print_sptes (struct kvm_vcpu *vcpu,gpa_t gpa,const char *msg);
arch/x86/kvm/x86.h +6 -0
@@ -617,4 +617,10 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } +unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl); +int kvm_emulate_hypercall (struct kvm_vcpu *vcpu); +#endif
Signature · 1 line
2.47.0.338.g60cca15819-goog
Re: [PATCH v4 3/6] KVM: x86: Move "emulate hypercall" function declarations to x86.h
Quoted 5 lines · Sean Christopherson
Move the declarations for the hypercall emulation APIs to
x86.h. While the helpers are exported, they are intended to be consumed only KVM vendor modules, i.e. don't need to exposed to the kernel at-large.No functional change intended.
Quoted 38 lines
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/include/asm/kvm_host.h | 6 ------ arch/x86/kvm/x86.h | 6 ++++++ 2 files changed, 6 insertions(+), 6 deletions(-)arch/x86/include/asm/kvm_host.h +0
-6@@ -2181,12 +2181,6 @@ static inline void kvm_clear_apicv_inhibit(struct kvm *kvm,kvm_set_or_clear_apicv_inhibit (kvm, reason, false); } -unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl); -int kvm_emulate_hypercall (struct kvm_vcpu *vcpu); -int kvm_mmu_page_fault (struct kvm_vcpu *vcpu,gpa_t cr2_or_gpa,u64 error_code,void *insn,int insn_len);void kvm_mmu_print_sptes (struct kvm_vcpu *vcpu,gpa_t gpa,const char *msg);arch/x86/kvm/x86.h +6
-0@@ -617,4 +617,10 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } +unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl); +int kvm_emulate_hypercall (struct kvm_vcpu *vcpu); +#endif
Re: [PATCH v4 3/6] KVM: x86: Move "emulate hypercall" function declarations to x86.h
In reply to Sean ChristophersonQuoted 2 lines · Sean Christopherson
Move the declarations for the hypercall emulation APIs to
x86.h. While the helpers are exported, they are intended to be consumed only KVM vendor
s/only/only by/
Quoted 5 lines
modules, i.e. don't need to exposed to the kernel at-large.
No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Quoted 37 lines
--- arch/x86/include/asm/kvm_host.h | 6 ------ arch/x86/kvm/x86.h | 6 ++++++ 2 files changed, 6 insertions(+), 6 deletions(-)arch/x86/include/asm/kvm_host.h +0
-6@@ -2181,12 +2181,6 @@ static inline void kvm_clear_apicv_inhibit(struct kvm *kvm,kvm_set_or_clear_apicv_inhibit (kvm, reason, false); } -unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl); -int kvm_emulate_hypercall (struct kvm_vcpu *vcpu); -int kvm_mmu_page_fault (struct kvm_vcpu *vcpu,gpa_t cr2_or_gpa,u64 error_code,void *insn,int insn_len);void kvm_mmu_print_sptes (struct kvm_vcpu *vcpu,gpa_t gpa,const char *msg);arch/x86/kvm/x86.h +6
-0@@ -617,4 +617,10 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } +unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl); +int kvm_emulate_hypercall (struct kvm_vcpu *vcpu); +#endif
Re: [PATCH v4 3/6] KVM: x86: Move "emulate hypercall" function declarations to x86.h
In reply to Sean ChristophersonQuoted 2 lines · Sean Christopherson
Move the declarations for the hypercall emulation APIs to
x86.h. While the helpers are exported, they are intended to be consumed only KVM vendor
only -> only by
Quoted 1 line
modules, i.e. don't need to exposed to the kernel at-large.
to exposed -> to be exposed
Quoted 4 lines
No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Quoted 37 lines
--- arch/x86/include/asm/kvm_host.h | 6 ------ arch/x86/kvm/x86.h | 6 ++++++ 2 files changed, 6 insertions(+), 6 deletions(-)arch/x86/include/asm/kvm_host.h +0
-6@@ -2181,12 +2181,6 @@ static inline void kvm_clear_apicv_inhibit(struct kvm *kvm,kvm_set_or_clear_apicv_inhibit (kvm, reason, false); } -unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl); -int kvm_emulate_hypercall (struct kvm_vcpu *vcpu); -int kvm_mmu_page_fault (struct kvm_vcpu *vcpu,gpa_t cr2_or_gpa,u64 error_code,void *insn,int insn_len);void kvm_mmu_print_sptes (struct kvm_vcpu *vcpu,gpa_t gpa,const char *msg);arch/x86/kvm/x86.h +6
-0@@ -617,4 +617,10 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } +unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl); +int kvm_emulate_hypercall (struct kvm_vcpu *vcpu); +#endif
Sean Christopherson
maintainerPATCH v4 4/6 KVM: x86: Bump hypercall stat prior to fully completing hypercall
In reply to Sean ChristophersonIncrement the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits.
Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences).
--- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +2 -2
@@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)if (!is_64_bit_hypercall (vcpu)) ret = (u32 )ret;kvm_rax_write (vcpu, ret); - ++vcpu->stat.hypercalls;return kvm_skip_emulated_instruction (vcpu); }
@@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, {unsigned long ret; + ++vcpu->stat.hypercalls; +trace_kvm_hypercall (nr, a0, a1, a2, a3);if (!op_64_bit) {
@@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, } out: - ++vcpu->stat.hypercalls;return ret; }EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall);
Signature · 1 line
2.47.0.338.g60cca15819-goog
Re: [PATCH v4 4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall
Quoted 10 lines · Sean Christopherson
Increment the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits.
Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences).
Quoted 34 lines
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/x86.c +2
-2@@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)if (!is_64_bit_hypercall (vcpu)) ret = (u32 )ret;kvm_rax_write (vcpu, ret); - ++vcpu->stat.hypercalls;return kvm_skip_emulated_instruction (vcpu); }@@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, {unsigned long ret; + ++vcpu->stat.hypercalls; +trace_kvm_hypercall (nr, a0, a1, a2, a3);if (!op_64_bit) {@@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, } out: - ++vcpu->stat.hypercalls;return ret; }EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall);
Re: [PATCH v4 4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall
In reply to Sean ChristophersonQuoted 7 lines · Sean Christopherson
Increment the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits.
Quoted 5 lines
Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences).
Signed-off-by Sean Christopherson <[email protected]>
There's a comment in the KVM_HC_MAP_GPA_RANGE case that reads:
/* stat is incremented on completion. */
that should probably be deleted, but otherwise:
Also, if you want, you could get rid of the 'out' label, too, by doing:
if (cpl) return -KVM_EPERM;
Quoted 33 lines
--- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/x86.c +2
-2@@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)if (!is_64_bit_hypercall (vcpu)) ret = (u32 )ret;kvm_rax_write (vcpu, ret); - ++vcpu->stat.hypercalls;return kvm_skip_emulated_instruction (vcpu); }@@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, {unsigned long ret; + ++vcpu->stat.hypercalls; +trace_kvm_hypercall (nr, a0, a1, a2, a3);if (!op_64_bit) {@@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, } out: - ++vcpu->stat.hypercalls;return ret; }EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall);
Re: [PATCH v4 4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall
Quoted 18 lines · Tom Lendacky
Quoted 7 lines · Sean Christopherson
Increment the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits.
Quoted 5 lines
Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences).
Signed-off-by Sean Christopherson <[email protected]>There's a comment in the
KVM_HC_MAP_GPA_RANGEcase that reads:/* stat is incremented on completion. */
Quoted 8 lines
that should probably be deleted, but otherwise:
Reviewed-by Tom Lendacky <[email protected]>Also, if you want, you could get rid of the 'out' label, too, by doing:
if (cpl) return -KVM_EPERM;
Until I saw the next patch... nevermind.
Thanks,
Tom
Quoted 34 lines
Quoted 34 lines
--- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/x86.c +2
-2@@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)if (!is_64_bit_hypercall (vcpu)) ret = (u32 )ret;kvm_rax_write (vcpu, ret); - ++vcpu->stat.hypercalls;return kvm_skip_emulated_instruction (vcpu); }@@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, {unsigned long ret; + ++vcpu->stat.hypercalls; +trace_kvm_hypercall (nr, a0, a1, a2, a3);if (!op_64_bit) {@@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, } out: - ++vcpu->stat.hypercalls;return ret; }EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall);
Re: [PATCH v4 4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall
In reply to Sean ChristophersonQuoted 7 lines · Sean Christopherson
Increment the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits.
Quoted 5 lines
Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences).
Signed-off-by Sean Christopherson <[email protected]>
Quoted 33 lines
--- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/x86.c +2
-2@@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)if (!is_64_bit_hypercall (vcpu)) ret = (u32 )ret;kvm_rax_write (vcpu, ret); - ++vcpu->stat.hypercalls;return kvm_skip_emulated_instruction (vcpu); }@@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, {unsigned long ret; + ++vcpu->stat.hypercalls; +trace_kvm_hypercall (nr, a0, a1, a2, a3);if (!op_64_bit) {@@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, } out: - ++vcpu->stat.hypercalls;return ret; }EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall);
Sean Christopherson
maintainerPATCH v4 5/6 KVM: x86: Always complete hypercall via function callback
In reply to Sean ChristophersonFinish "emulation" of KVM hypercalls by function callback, even when the hypercall is handled entirely within KVM, i.e. doesn't require an exit to userspace, and refactor __kvm_emulate_hypercall()'s return value to *only* communicate whether or not KVM should exit to userspace or resume the guest.
(Ab)Use vcpu->run->hypercall.ret to propagate the return value to the callback, purely to avoid having to add a trampoline for every completion callback.
Using the function return value for KVM's control flow eliminates the multiplexed return value, where '0' for KVM_HC_MAP_GPA_RANGE (and only that hypercall) means "exit to userspace".
Note, the unnecessary extra indirect call and thus potential retpoline will be eliminated in the near future by converting the intermediate layer to a macro.
--- arch/x86/kvm/x86.c | 29 ++++++++++++----------------- arch/x86/kvm/x86.h | 10 ++++++---- 2 files changed, 18 insertions(+), 21 deletions(-)
arch/x86/kvm/x86.c +12 -17
@@ -9982,10 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)return kvm_skip_emulated_instruction (vcpu); } -unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl) +int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)) {unsigned long ret;
@@ -10061,7 +10062,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, vcpu->run->hypercall.flags |=KVM_EXIT_HYPERCALL_LONG_MODE ;WARN_ON_ONCE (vcpu->run->hypercall.flags &KVM_EXIT_HYPERCALL_MBZ ); - vcpu->arch.complete_userspace_io = complete_hypercall_exit; + vcpu->arch.complete_userspace_io = complete_hypercall;/* stat is incremented on completion. */ return 0 ; }
@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, } out: -return ret; + vcpu->run->hypercall.ret = ret; +complete_hypercall (vcpu); +return 1 ; }EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall);int kvm_emulate_hypercall (struct kvm_vcpu *vcpu) { -unsigned long nr, a0, a1, a2, a3, ret; +unsigned long nr, a0, a1, a2, a3;int op_64_bit;int cpl;
@@ -10095,16 +10098,8 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) op_64_bit =is_64_bit_hypercall (vcpu); cpl =kvm_x86_call (get_cpl)(vcpu); - ret =__kvm_emulate_hypercall (vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl); -if (nr ==KVM_HC_MAP_GPA_RANGE && !ret) -/* MAP_GPA tosses the request to the user space. */ -return 0 ; - -if (!op_64_bit) - ret = (u32 )ret; -kvm_rax_write (vcpu, ret); - -return kvm_skip_emulated_instruction (vcpu); +return __kvm_emulate_hypercall (vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, + complete_hypercall_exit); }EXPORT_SYMBOL_GPL (kvm_emulate_hypercall);
arch/x86/kvm/x86.h +6 -4
@@ -617,10 +617,12 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } -unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl); +int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)); +int kvm_emulate_hypercall (struct kvm_vcpu *vcpu);#endif
Signature · 1 line
2.47.0.338.g60cca15819-goog
Re: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback
Quoted 56 lines · Sean Christopherson
Finish "emulation" of KVM hypercalls by function callback, even when the hypercall is handled entirely within KVM, i.e. doesn't require an exit to userspace, and refactor __kvm_emulate_hypercall()'s return value to *only* communicate whether or not KVM should exit to userspace or resume the guest.
(Ab)Use
vcpu->run->hypercall.retto propagate the return value to the callback, purely to avoid having to add a trampoline for every completion callback.Using the function return value for KVM's control flow eliminates the multiplexed return value, where '0' for
KVM_HC_MAP_GPA_RANGE(and only that hypercall) means "exit to userspace".Note, the unnecessary extra indirect call and thus potential retpoline will be eliminated in the near future by converting the intermediate layer to a macro.
Suggested-by Binbin Wu <[email protected]>Suggested-by Kai Huang <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/x86.c | 29 ++++++++++++----------------- arch/x86/kvm/x86.h | 10 ++++++---- 2 files changed, 18 insertions(+), 21 deletions(-)arch/x86/kvm/x86.c +6
-5@@ -9982,10 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)return kvm_skip_emulated_instruction (vcpu); } -unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl) +int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)) {unsigned long ret;@@ -10061,7 +10062,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, vcpu->run->hypercall.flags |=KVM_EXIT_HYPERCALL_LONG_MODE ;WARN_ON_ONCE (vcpu->run->hypercall.flags &KVM_EXIT_HYPERCALL_MBZ ); - vcpu->arch.complete_userspace_io = complete_hypercall_exit; + vcpu->arch.complete_userspace_io = complete_hypercall;/* stat is incremented on completion. */ return 0 ; }
arch/x86/kvm/x86.c unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, · line 10077
Quoted 7 lines
arch/x86/kvm/x86.c +3
-1@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, } out: -return ret; + vcpu->run->hypercall.ret = ret; +complete_hypercall (vcpu); +return 1 ;
shouldn't it be
return complete_hypercall(vcpu);
?
Originally, kvm_emulate_hypercall() returns kvm_skip_emulated_instruction(). Now it becomes
kvm_skip_emulated_instruction(); return 1;
I don't go deep to see if kvm_skip_emulated_instruction() always return 1 for this case. But logically,
return complete_hypercall(vcpu);
looks more reasonable.
Quoted 50 lines
arch/x86/kvm/x86.c +3
-11}EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall);int kvm_emulate_hypercall (struct kvm_vcpu *vcpu) { -unsigned long nr, a0, a1, a2, a3, ret; +unsigned long nr, a0, a1, a2, a3;int op_64_bit;int cpl;@@ -10095,16 +10098,8 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) op_64_bit =is_64_bit_hypercall (vcpu); cpl =kvm_x86_call (get_cpl)(vcpu); - ret =__kvm_emulate_hypercall (vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl); -if (nr ==KVM_HC_MAP_GPA_RANGE && !ret) -/* MAP_GPA tosses the request to the user space. */ -return 0 ; - -if (!op_64_bit) - ret = (u32 )ret; -kvm_rax_write (vcpu, ret); - -return kvm_skip_emulated_instruction (vcpu); +return __kvm_emulate_hypercall (vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, + complete_hypercall_exit); }EXPORT_SYMBOL_GPL (kvm_emulate_hypercall);arch/x86/kvm/x86.h +6
-4@@ -617,10 +617,12 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } -unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl); +int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)); +int kvm_emulate_hypercall (struct kvm_vcpu *vcpu);#endif
Sean Christopherson
maintainerRe: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback
Quoted 37 lines · Xiaoyao Li
Quoted 27 lines · Sean Christopherson
arch/x86/kvm/x86.c +6
-5@@ -9982,10 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)return kvm_skip_emulated_instruction (vcpu); } -unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl) +int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)) {unsigned long ret;@@ -10061,7 +10062,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, vcpu->run->hypercall.flags |=KVM_EXIT_HYPERCALL_LONG_MODE ;WARN_ON_ONCE (vcpu->run->hypercall.flags &KVM_EXIT_HYPERCALL_MBZ ); - vcpu->arch.complete_userspace_io = complete_hypercall_exit; + vcpu->arch.complete_userspace_io = complete_hypercall;/* stat is incremented on completion. */ return 0 ; }arch/x86/kvm/x86.c unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, · line 10076
Quoted 6 lines
arch/x86/kvm/x86.c +3
-1@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, } out: -return ret; + vcpu->run->hypercall.ret = ret; +complete_hypercall (vcpu); +return 1 ;shouldn't it be
Quoted 3 lines
return complete_hypercall(vcpu);?
Ouch. Yes, it most definitely should be.
Quoted 8 lines
Originally,
kvm_emulate_hypercall()returnskvm_skip_emulated_instruction(). Now it becomeskvm_skip_emulated_instruction(); return 1;I don't go deep to see if
kvm_skip_emulated_instruction()always return 1 for this case.
It doesn't. KVM needs to exit to userspace if userspace is single-stepping, or in the extremely unlikely scenario that KVM can't skip the emulated instruction (which can very theoretically happen on older AMD CPUs).
Re: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback
In reply to Sean ChristophersonQuoted 56 lines · Sean Christopherson
Finish "emulation" of KVM hypercalls by function callback, even when the hypercall is handled entirely within KVM, i.e. doesn't require an exit to userspace, and refactor __kvm_emulate_hypercall()'s return value to *only* communicate whether or not KVM should exit to userspace or resume the guest.
(Ab)Use
vcpu->run->hypercall.retto propagate the return value to the callback, purely to avoid having to add a trampoline for every completion callback.Using the function return value for KVM's control flow eliminates the multiplexed return value, where '0' for
KVM_HC_MAP_GPA_RANGE(and only that hypercall) means "exit to userspace".Note, the unnecessary extra indirect call and thus potential retpoline will be eliminated in the near future by converting the intermediate layer to a macro.
Suggested-by Binbin Wu <[email protected]>Suggested-by Kai Huang <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/x86.c | 29 ++++++++++++----------------- arch/x86/kvm/x86.h | 10 ++++++---- 2 files changed, 18 insertions(+), 21 deletions(-)arch/x86/kvm/x86.c +6
-5@@ -9982,10 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)return kvm_skip_emulated_instruction (vcpu); } -unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl) +int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)) {unsigned long ret;@@ -10061,7 +10062,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, vcpu->run->hypercall.flags |=KVM_EXIT_HYPERCALL_LONG_MODE ;WARN_ON_ONCE (vcpu->run->hypercall.flags &KVM_EXIT_HYPERCALL_MBZ ); - vcpu->arch.complete_userspace_io = complete_hypercall_exit; + vcpu->arch.complete_userspace_io = complete_hypercall;/* stat is incremented on completion. */ return 0 ; }
arch/x86/kvm/x86.c unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, · line 10077
Quoted 7 lines
arch/x86/kvm/x86.c +3
-1@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, } out: -return ret; + vcpu->run->hypercall.ret = ret; +complete_hypercall (vcpu); +return 1 ;
Should this do return complete_hypercall(vcpu) so that you get the return code from kvm_skip_emulated_instruction()?
Thanks,
Tom
Quoted 50 lines
arch/x86/kvm/x86.c +3
-11}EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall);int kvm_emulate_hypercall (struct kvm_vcpu *vcpu) { -unsigned long nr, a0, a1, a2, a3, ret; +unsigned long nr, a0, a1, a2, a3;int op_64_bit;int cpl;@@ -10095,16 +10098,8 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) op_64_bit =is_64_bit_hypercall (vcpu); cpl =kvm_x86_call (get_cpl)(vcpu); - ret =__kvm_emulate_hypercall (vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl); -if (nr ==KVM_HC_MAP_GPA_RANGE && !ret) -/* MAP_GPA tosses the request to the user space. */ -return 0 ; - -if (!op_64_bit) - ret = (u32 )ret; -kvm_rax_write (vcpu, ret); - -return kvm_skip_emulated_instruction (vcpu); +return __kvm_emulate_hypercall (vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, + complete_hypercall_exit); }EXPORT_SYMBOL_GPL (kvm_emulate_hypercall);arch/x86/kvm/x86.h +6
-4@@ -617,10 +617,12 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } -unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl); +int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)); +int kvm_emulate_hypercall (struct kvm_vcpu *vcpu);#endif
Re: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback
Quoted 5 lines · Tom Lendacky
Quoted 3 lines · Sean Christopherson
(quoted diff) +0
-0@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, }
Quoted 8 lines
arch/x86/kvm/x86.c unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, · line 10077
Quoted 5 lines
arch/x86/kvm/x86.c +3
-1out: -return ret; + vcpu->run->hypercall.ret = ret; +complete_hypercall (vcpu); +return 1 ;Should this do return
complete_hypercall(vcpu)so that you get the return code fromkvm_skip_emulated_instruction()?
Bah, ignore... already commented on by Xiaoyao.
Thanks,
Tom
Quoted 4 lines
Thanks,
Tom
Sean Christopherson
maintainerRe: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback
Quoted 15 lines · Tom Lendacky
Quoted 5 lines · Tom Lendacky
Quoted 3 lines · Sean Christopherson
(quoted diff) +0
-0@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, }Quoted 8 lines
arch/x86/kvm/x86.c unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, · line 10077
Quoted 5 lines
arch/x86/kvm/x86.c +3
-1out: -return ret; + vcpu->run->hypercall.ret = ret; +complete_hypercall (vcpu); +return 1 ;Should this do return
complete_hypercall(vcpu)so that you get the return code fromkvm_skip_emulated_instruction()?
Quoted 1 line
Bah, ignore... already commented on by Xiaoyao.
Reviewers: 2, Sean: 0
:-)
Sean Christopherson
maintainerPATCH v4 6/6 KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
In reply to Sean ChristophersonRework __kvm_emulate_hypercall() into a macro so that completion of hypercalls that don't exit to userspace use direct function calls to the completion helper, i.e. don't trigger a retpoline when RETPOLINE=y.
Opportunistically take the names of the input registers, as opposed to taking the input values, to preemptively dedup more of the calling code (TDX needs to use different registers). Use the direct GPR accessors to read values to avoid the pointless marking of the registers as available (KVM requires GPRs to always be available).
--- arch/x86/kvm/x86.c | 29 +++++++++-------------------- arch/x86/kvm/x86.h | 25 ++++++++++++++++++++----- 2 files changed, 29 insertions(+), 25 deletions(-)
arch/x86/kvm/x86.c +9 -20
@@ -9982,11 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)return kvm_skip_emulated_instruction (vcpu); } -int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl, -int (*complete_hypercall)(struct kvm_vcpu *)) +int ____kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)) {unsigned long ret;
@@ -10073,32 +10073,21 @@ int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, out: vcpu->run->hypercall.ret = ret; -complete_hypercall (vcpu);return 1 ; } -EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall); +EXPORT_SYMBOL_GPL (____kvm_emulate_hypercall);int kvm_emulate_hypercall (struct kvm_vcpu *vcpu) { -unsigned long nr, a0, a1, a2, a3; -int op_64_bit; -int cpl; -if (kvm_xen_hypercall_enabled (vcpu->kvm))return kvm_xen_hypercall (vcpu);if (kvm_hv_hypercall_enabled (vcpu))return kvm_hv_hypercall (vcpu); - nr =kvm_rax_read (vcpu); - a0 =kvm_rbx_read (vcpu); - a1 =kvm_rcx_read (vcpu); - a2 =kvm_rdx_read (vcpu); - a3 =kvm_rsi_read (vcpu); - op_64_bit =is_64_bit_hypercall (vcpu); - cpl =kvm_x86_call (get_cpl)(vcpu); - -return __kvm_emulate_hypercall (vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, +return __kvm_emulate_hypercall (vcpu, rax, rbx, rcx, rdx, rsi, +is_64_bit_hypercall (vcpu), +kvm_x86_call (get_cpl)(vcpu), complete_hypercall_exit); }EXPORT_SYMBOL_GPL (kvm_emulate_hypercall);
arch/x86/kvm/x86.h +20 -5
@@ -617,11 +617,26 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)return kvm->arch.hypercall_exit_enabled &BIT (hc_nr); } -int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl, -int (*complete_hypercall)(struct kvm_vcpu *)); +int ____kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)); + +#define __kvm_emulate_hypercall (_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \ +({ \ + int __ret; \ + \ + __ret = ____kvm_emulate_hypercall(_vcpu, \ + kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu), \ + kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu), \ + kvm_##a3##_read(_vcpu), op_64_bit, cpl, \ + complete_hypercall); \ + \ + if (__ret > 0) \ + complete_hypercall(_vcpu); \ + __ret; \ +})int kvm_emulate_hypercall (struct kvm_vcpu *vcpu);
Signature · 1 line
2.47.0.338.g60cca15819-goog
Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
Quoted 9 lines · Sean Christopherson
Rework
__kvm_emulate_hypercall()into a macro so that completion of hypercalls that don't exit to userspace use direct function calls to the completion helper, i.e. don't trigger a retpoline when RETPOLINE=y.Opportunistically take the names of the input registers, as opposed to taking the input values, to preemptively dedup more of the calling code (TDX needs to use different registers). Use the direct GPR accessors to read values to avoid the pointless marking of the registers as available (KVM requires GPRs to always be available).
For TDX, there is an RFC relating to using descriptively named parameters instead of register names for tdh_vp_enter():
https://lore.kernel.org/all/[email protected]/
Please do give some feedback on that approach. Note we need both KVM and x86 maintainer approval for SEAMCALL wrappers like tdh_vp_enter().
As proposed, that ends up with putting the values back into vcpu->arch.regs[] for __kvm_emulate_hypercall() which is not pretty:
static int tdx_emulate_vmcall(struct kvm_vcpu *vcpu) {
+ struct vcpu_tdx *tdx = to_tdx(vcpu);
int r;
+ kvm_r10_write(vcpu, tdx->vp_enter_args.tdcall.fn);
+ kvm_r11_write(vcpu, tdx->vp_enter_args.tdcall.subfn);
+ kvm_r12_write(vcpu, tdx->vp_enter_args.tdcall.vmcall.p2);
+ kvm_r13_write(vcpu, tdx->vp_enter_args.tdcall.vmcall.p3);
+ kvm_r14_write(vcpu, tdx->vp_enter_args.tdcall.vmcall.p4);
+
/*
* ABI for KVM tdvmcall argument:
* In Guest-Hypervisor Communication Interface(GHCI) specification,
@@ -1092,13 +1042,12 @@ static int tdx_emulate_vmcall(struct kvm_vcpu *vcpu)
* vendor-specific. KVM uses this for KVM hypercall. NOTE: KVM
* hypercall number starts from one. Zero isn't used for KVM hypercall
* number.
- *
- * R10: KVM hypercall number
- * arguments: R11, R12, R13, R14.
*/
r = __kvm_emulate_hypercall(vcpu, r10, r11, r12, r13, r14, true, 0,
R10, complete_hypercall_exit);
+ tdvmcall_set_return_code(vcpu, kvm_r10_read(vcpu));
+
return r > 0;
}
Quoted 101 lines
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/x86.c | 29 +++++++++-------------------- arch/x86/kvm/x86.h | 25 ++++++++++++++++++++----- 2 files changed, 29 insertions(+), 25 deletions(-)arch/x86/kvm/x86.c +9
-20@@ -9982,11 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)return kvm_skip_emulated_instruction (vcpu); } -int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl, -int (*complete_hypercall)(struct kvm_vcpu *)) +int ____kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)) {unsigned long ret;@@ -10073,32 +10073,21 @@ int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, out: vcpu->run->hypercall.ret = ret; -complete_hypercall (vcpu);return 1 ; } -EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall); +EXPORT_SYMBOL_GPL (____kvm_emulate_hypercall);int kvm_emulate_hypercall (struct kvm_vcpu *vcpu) { -unsigned long nr, a0, a1, a2, a3; -int op_64_bit; -int cpl; -if (kvm_xen_hypercall_enabled (vcpu->kvm))return kvm_xen_hypercall (vcpu);if (kvm_hv_hypercall_enabled (vcpu))return kvm_hv_hypercall (vcpu); - nr =kvm_rax_read (vcpu); - a0 =kvm_rbx_read (vcpu); - a1 =kvm_rcx_read (vcpu); - a2 =kvm_rdx_read (vcpu); - a3 =kvm_rsi_read (vcpu); - op_64_bit =is_64_bit_hypercall (vcpu); - cpl =kvm_x86_call (get_cpl)(vcpu); - -return __kvm_emulate_hypercall (vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, +return __kvm_emulate_hypercall (vcpu, rax, rbx, rcx, rdx, rsi, +is_64_bit_hypercall (vcpu), +kvm_x86_call (get_cpl)(vcpu), complete_hypercall_exit); }EXPORT_SYMBOL_GPL (kvm_emulate_hypercall);arch/x86/kvm/x86.h +20
-5@@ -617,11 +617,26 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)return kvm->arch.hypercall_exit_enabled &BIT (hc_nr); } -int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl, -int (*complete_hypercall)(struct kvm_vcpu *)); +int ____kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)); + +#define __kvm_emulate_hypercall (_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \ +({ \ + int __ret; \ + \ + __ret = ____kvm_emulate_hypercall(_vcpu, \ + kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu), \ + kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu), \ + kvm_##a3##_read(_vcpu), op_64_bit, cpl, \ + complete_hypercall); \ + \ + if (__ret > 0) \ + complete_hypercall(_vcpu); \ + __ret; \ +})int kvm_emulate_hypercall (struct kvm_vcpu *vcpu);
Paolo Bonzini
maintainerRe: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
Quoted 6 lines · Adrian Hunter
For TDX, there is an RFC relating to using descriptively named parameters instead of register names for
tdh_vp_enter():https://lore.kernel.org/all/[email protected]/
Quoted 7 lines
Please do give some feedback on that approach. Note we need both KVM and x86 maintainer approval for SEAMCALL wrappers like
tdh_vp_enter().As proposed, that ends up with putting the values back into vcpu->arch.regs[] for
__kvm_emulate_hypercall()which is not pretty:
If needed we can revert this patch, it's not a big problem.
Paolo
Sean Christopherson
maintainerRe: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
Quoted 11 lines · Paolo Bonzini
Quoted 10 lines · Adrian Hunter
For TDX, there is an RFC relating to using descriptively named parameters instead of register names for
tdh_vp_enter():https://lore.kernel.org/all/[email protected]/Please do give some feedback on that approach. Note we need both KVM and x86 maintainer approval for SEAMCALL wrappers like
tdh_vp_enter().
Quoted 5 lines
Quoted 3 lines
As proposed, that ends up with putting the values back into vcpu->arch.regs[] for
__kvm_emulate_hypercall()which is not pretty:If needed we can revert this patch, it's not a big problem.
I don't care terribly about the SEAMCALL interfaces. I have opinions on what would I think would be ideal, but I can live with whatever.
What I do deeply care about though is consistency within KVM, across vendors and VM flavors. And that means that guest registers absolutely need to be captured in vcpu->arch.regs[]. TDX already requires too much special cased code in KVM, there is zero reason to make TDX even more different and thus more difficult to maintain.
Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
Quoted 16 lines · Sean Christopherson
Quoted 15 lines · Paolo Bonzini
Quoted 13 lines · Adrian Hunter
For TDX, there is an RFC relating to using descriptively named parameters instead of register names for
tdh_vp_enter():https://lore.kernel.org/all/[email protected]/Please do give some feedback on that approach. Note we need both KVM and x86 maintainer approval for SEAMCALL wrappers like
tdh_vp_enter().As proposed, that ends up with putting the values back into vcpu->arch.regs[] for
__kvm_emulate_hypercall()which is not pretty:
Quoted 8 lines
Quoted 1 line
If needed we can revert this patch, it's not a big problem.
I don't care terribly about the SEAMCALL interfaces. I have opinions on what would I think would be ideal, but I can live with whatever.
What I do deeply care about though is consistency within KVM, across vendors and VM flavors. And that means that guest registers absolutely need to be captured in vcpu->arch.regs[].
In general, TDX host VMM does not know what guest register values are.
This case, where some GPRs are passed to the host VMM via arguments of the TDG.VP.VMCALL TDCALL, is really just a side effect of the choice of argument passing rather than any attempt to share guest registers with the host VMM.
It could be regarded as more consistent to never use vcpu->arch.regs[] for confidential guests.
Paolo Bonzini
maintainerRe: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
Quoted 8 lines · Adrian Hunter
Quoted 3 lines · Sean Christopherson
What I do deeply care about though is consistency within KVM, across vendors and VM flavors. And that means that guest registers absolutely need to be captured in vcpu->arch.regs[].
In general, TDX host VMM does not know what guest register values are.
Quoted 7 lines
This case, where some GPRs are passed to the host VMM via arguments of the TDG.VP.VMCALL TDCALL, is really just a side effect of the choice of argument passing rather than any attempt to share guest registers with the host VMM.
It could be regarded as more consistent to never use vcpu->arch.regs[] for confidential guests.
Yes, that's where I stand as well. There's reasons to use vcpu->arch.regs[] when "decrypted" values are available, and reasons to not use it at all. Both of them could be considered the more consistent choice, and I think I prefer slightly the latter, but it's definitely not a hill to die on...
Paolo
Sean Christopherson
maintainerRe: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
In reply to Adrian HunterQuoted 26 lines · Adrian Hunter
Quoted 16 lines · Sean Christopherson
Quoted 15 lines · Paolo Bonzini
Quoted 13 lines · Adrian Hunter
For TDX, there is an RFC relating to using descriptively named parameters instead of register names for
tdh_vp_enter():https://lore.kernel.org/all/[email protected]/Please do give some feedback on that approach. Note we need both KVM and x86 maintainer approval for SEAMCALL wrappers like
tdh_vp_enter().As proposed, that ends up with putting the values back into vcpu->arch.regs[] for
__kvm_emulate_hypercall()which is not pretty:Quoted 8 lines
Quoted 1 line
If needed we can revert this patch, it's not a big problem.
I don't care terribly about the SEAMCALL interfaces. I have opinions on what would I think would be ideal, but I can live with whatever.
What I do deeply care about though is consistency within KVM, across vendors and VM flavors. And that means that guest registers absolutely need to be captured in vcpu->arch.regs[].
Quoted 8 lines
In general, TDX host VMM does not know what guest register values are.
This case, where some GPRs are passed to the host VMM via arguments of the TDG.VP.VMCALL TDCALL, is really just a side effect of the choice of argument passing rather than any attempt to share guest registers with the host VMM.
It could be regarded as more consistent to never use vcpu->arch.regs[] for confidential guests.
SEV-ES+ marshalls data to/from the GHCB to KVM's register array, because the GHCB spec was intentionally crafted to allow hypervisors to reuse exit-handling code. Granted, that's only for R{A,B,C,D}X and RSI, but the other GPRs should never be used and thus their data is irrelevant.
Which applies to TDX as well. For regs[], it's really only TDVMCALL that I care about, i.e. cases where GPRs hold guest values, versus things like EXIT_QUALIFICATION where the GPR is simply TDX's way of communicating information to the hypervisor.
Argh. The reason I care about putting vCPU state into regs[] is because it helps share code between vendors. Looking at kvm-coco-queue, TDX support wandered in the opposite direction. E.g. TDX rolls its own RDMSR, WRMSR, CPUID, and HYPERCALL implementations, which is quite frustrating. Ditto for things like EXIT_REASON, EXIT_QUALIFICATION, EXIT_INTR_INFO, etc.
For EXIT_REASON in particular, I think maintaining the guest-requested exit reason (via TDMVCALL) in a separate field is a mistake. Readers shouldn't have to care that a HLT exit technically was requested via TDVMCALL. If KVM instead immediately morphs the requested exit reason to KVM's tracked exit_reason, then there's no need to deal with the TDVMCALL layer in flows that don't care. The only danger is a collision with a EXIT_REASON_EPT_MISCONFIG from the TDX module, but that's easy enough to handle.
And even where TDX and VMX have shared some code, IMO it doesn't go far enough. E.g. having vcpu_tdx and vcpu_vmx open code their own version of the posted interrupt fields, just to avoid minimal churn in the VMX code, is beyond gross.
Even concepts like guest_state_loaded are unnecessarily different for TDX. Yes, I get that that host state doesn't need to be reloaded if KVM doesn't actually enter the guest. But holy moly, we're talking about avoid _one_ WRMSR in an extremely rare path (late abort of entry), at the cost of making TDX frustratingly different from VMX.
Making TDX look more like everything else isn't just about code sharing. It's also about providing a familiar setting so that readers who know almost nothing about TDX can find their way around without having to effectively learn an entirely new "architecture" *and* code base.
Hacking around, I think the attached half-baked diff will provide a middle ground for the regs[] vs. struct/union issue. The basic gist is to essentially treat TDX's register ABI as a faster version of VMREAD/VMWRITE, e.g. marshall state to/from the appropriate x86 registers as needed. That way, regs[] holds the correct state and so TDX can reuse much of KVM's existing code verbatim, while allowing the kernel's VP_ENTER API to evolve independently.
From 0319082fc23089f516618e193d94da18c837e35a Mon Sep 17 00:00:00 2001
From: Sean Christopherson <[email protected]>
Date: Thu, 12 Dec 2024 10:22:25 -0800
Subject: [PATCH] Make TDX look more like VMX
--- arch/x86/kvm/vmx/common.h | 64 +++++- arch/x86/kvm/vmx/nested.c | 7 +- arch/x86/kvm/vmx/posted_intr.h | 11 - arch/x86/kvm/vmx/tdx.c | 375 +++++++++++---------------------- arch/x86/kvm/vmx/tdx.h | 11 +- arch/x86/kvm/vmx/vmx.c | 26 +-- arch/x86/kvm/vmx/vmx.h | 42 +--- 7 files changed, 201 insertions(+), 335 deletions(-)
arch/x86/kvm/vmx/common.h +60 -4
@@ -12,6 +12,61 @@#include "vmcs.h" #include "x86.h" +struct vcpu_vt { +/* Posted interrupt descriptor */ +struct pi_desc pi_desc; + +/* Used if this vCPU is waiting for PI notification wakeup. */ +struct list_head pi_wakeup_list; + +union vmx_exit_reason exit_reason; + +unsigned long exit_qualification; +u64 ext_qualification; +gpa_t exit_gpa; +u32 exit_intr_info; +u32 idt_vectoring_info; + + +/* +* If true, guest state has been loaded into hardware, and host state +* saved into vcpu_{vt,vmx,tdx}. If false, host state is loaded into +* hardware. +*/ +bool guest_state_loaded; + +#ifdef CONFIG_X86_64 +u64 msr_host_kernel_gs_base; +#endif +}; + +static __always_inline unsigned long vmx_get_exit_reason (struct kvm_vcpu *vcpu) +{ +return to_vt (vcpu)->exit_reason; +} + +static __always_inline unsigned long vmx_get_exit_qual (struct kvm_vcpu *vcpu) +{ +struct vcpu_vt *vt =to_vcpu_vt (vcpu); + +if (!kvm_register_test_and_mark_available (vcpu,VCPU_EXREG_EXIT_INFO_1 ) && + !WARN_ON_ONCE(is_td_vcpu(vcpu)))) + vt->exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); + +return vt->exit_qualification; +} + +static __always_inline u32 vmx_get_intr_info(struct kvm_vcpu *vcpu) +{ +struct vcpu_vt *vt =to_vcpu_vt (vcpu); + +if (!kvm_register_test_and_mark_available (vcpu,VCPU_EXREG_EXIT_INFO_2 ) && + !WARN_ON_ONCE (is_td_vcpu (vcpu))) + vt->exit_intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); + +return vt->exit_intr_info; +} +extern unsigned long vmx_host_idt_base;void vmx_do_interrupt_irqoff (unsigned long entry);void vmx_do_nmi_irqoff (void );
@@ -36,9 +91,10 @@ static inline void vmx_handle_nm_fault_irqoff(struct kvm_vcpu *vcpu)rdmsrl (MSR_IA32_XFD_ERR , vcpu->arch.guest_fpu.xfd_err); } -static inline void vmx_handle_exception_irqoff (struct kvm_vcpu *vcpu, -u32 intr_info) +static inline void vmx_handle_exception_irqoff (struct kvm_vcpu *vcpu) { +u32 intr_info =vmx_get_intr_info (vcpu); +/* if exit due to PF check for async PF */ if (is_page_fault (intr_info)) vcpu->arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags ();
@@ -50,9 +106,9 @@ static inline void vmx_handle_exception_irqoff(struct kvm_vcpu *vcpu,kvm_machine_check (); } -static inline void vmx_handle_external_interrupt_irqoff (struct kvm_vcpu *vcpu, -u32 intr_info) +static inline void vmx_handle_external_interrupt_irqoff (struct kvm_vcpu *vcpu) { +u32 intr_info =vmx_get_intr_info (vcpu);unsigned int vector = intr_info &INTR_INFO_VECTOR_MASK ; if (KVM_BUG(!is_external_intr(intr_info), vcpu->kvm,
arch/x86/kvm/vmx/nested.c +4 -3
@@ -410,6 +410,7 @@ static void nested_ept_inject_page_fault(struct kvm_vcpu *vcpu, {struct vmcs12 *vmcs12 =get_vmcs12 (vcpu);struct vcpu_vmx *vmx =to_vmx (vcpu); +struct vcpu_vt *vt =to_vt (vcpu);unsigned long exit_qualification;u32 vm_exit_reason;
@@ -425,7 +426,7 @@ static void nested_ept_inject_page_fault(struct kvm_vcpu *vcpu,* tables also changed, but KVM should not treat EPT Misconfig * VM-Exits as writes. */ -WARN_ON_ONCE (vmx->exit_reason.basic !=EXIT_REASON_EPT_VIOLATION ); +WARN_ON_ONCE (vt->exit_reason.basic !=EXIT_REASON_EPT_VIOLATION );/* * PML Full and EPT Violation VM-Exits both use bit 12 to report
@@ -6099,7 +6100,7 @@ static int handle_vmfunc(struct kvm_vcpu *vcpu)* nested VM-Exit. Pass the original exit reason, i.e. don't hardcode * EXIT_REASON_VMFUNC as the exit reason. */ -nested_vmx_vmexit (vcpu, vmx->exit_reason.full, +nested_vmx_vmexit (vcpu,vmx_get_exit_reason (vcpu).full,vmx_get_intr_info (vcpu),vmx_get_exit_qual (vcpu));return 1 ;
@@ -6544,7 +6545,7 @@ static bool nested_vmx_l1_wants_exit(struct kvm_vcpu *vcpu,bool nested_vmx_reflect_vmexit (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); -union vmx_exit_reason exit_reason = vmx->exit_reason; +union vmx_exit_reason exit_reason =vmx_get_exit_reason (vcpu);unsigned long exit_qual;u32 exit_intr_info;
arch/x86/kvm/vmx/posted_intr.h +0 -11
@@ -5,17 +5,6 @@#include <linux/find.h> #include <asm/posted_intr.h> -struct vcpu_pi { -struct kvm_vcpu vcpu; - -/* Posted interrupt descriptor */ -struct pi_desc pi_desc; - -/* Used if this vCPU is waiting for PI notification wakeup. */ -struct list_head pi_wakeup_list; -/* Until here common layout between vcpu_vmx and vcpu_tdx. */ -}; -struct pi_desc *vcpu_to_pi_desc (struct kvm_vcpu *vcpu);void vmx_vcpu_pi_load (struct kvm_vcpu *vcpu,int cpu);
arch/x86/kvm/vmx/tdx.c +121 -254
622 lines not on this page · raw message
arch/x86/kvm/vmx/tdx.h +1 -10
@@ -59,12 +59,7 @@ enum vcpu_tdx_state {struct vcpu_tdx {struct kvm_vcpu vcpu; -/* Posted interrupt descriptor */ -struct pi_desc pi_desc; - -/* Used if this vCPU is waiting for PI notification wakeup. */ -struct list_head pi_wakeup_list; -/* Until here same layout to struct vcpu_pi. */ +struct vcpu_vt vt;unsigned long tdvpr_pa;unsigned long *tdcx_pa;
@@ -75,10 +70,6 @@ struct vcpu_tdx {enum vcpu_tdx_state state; -bool host_state_need_save; -bool host_state_need_restore; -u64 msr_host_kernel_gs_base; -u64 map_gpa_next;u64 map_gpa_end; };
arch/x86/kvm/vmx/vmx.c +13 -13
@@ -6099,9 +6099,9 @@ void vmx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, {struct vcpu_vmx *vmx =to_vmx (vcpu); - *reason = vmx->exit_reason.full; + *reason =vmx_get_exit_reason (vcpu).full; *info1 =vmx_get_exit_qual (vcpu); -if (!(vmx->exit_reason.failed_vmentry)) { +if (!(vmx_get_exit_reason (vcpu).failed_vmentry)) { *info2 = vmx->idt_vectoring_info; *intr_info =vmx_get_intr_info (vcpu);if (is_exception_with_error_code (*intr_info))
@@ -6380,7 +6380,7 @@ void dump_vmcs(struct kvm_vcpu *vcpu)static int __vmx_handle_exit (struct kvm_vcpu *vcpu,fastpath_t exit_fastpath) {struct vcpu_vmx *vmx =to_vmx (vcpu); -union vmx_exit_reason exit_reason = vmx->exit_reason; +union vmx_exit_reason exit_reason =vmx_get_exit_reason (vcpu);u32 vectoring_info = vmx->idt_vectoring_info;u16 exit_handler_index;
@@ -6901,11 +6901,10 @@ void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu)if (vmx->emulation_required)return ; -if (vmx->exit_reason.basic ==EXIT_REASON_EXTERNAL_INTERRUPT ) -vmx_handle_external_interrupt_irqoff (vcpu, -vmx_get_intr_info (vcpu)); -else if (vmx->exit_reason.basic ==EXIT_REASON_EXCEPTION_NMI ) -vmx_handle_exception_irqoff (vcpu,vmx_get_intr_info (vcpu)); +if (vmx_get_exit_reason (vcpu).basic ==EXIT_REASON_EXTERNAL_INTERRUPT ) +vmx_handle_external_interrupt_irqoff (vcpu); +else if (vmx_get_exit_reason (vcpu).basic ==EXIT_REASON_EXCEPTION_NMI ) +vmx_handle_exception_irqoff (vcpu); }/*
@@ -7154,6 +7153,7 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, unsigned int flags) {struct vcpu_vmx *vmx =to_vmx (vcpu); +struct vcpu_vt *vt =to_vt (vcpu);guest_state_enter_irqoff ();
@@ -7185,15 +7185,15 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu,vmx_enable_fb_clear (vmx);if (unlikely (vmx->fail)) { - vmx->exit_reason.full =0xdead ; + vt->exit_reason.full =0xdead ; goto out; } - vmx->exit_reason.full =vmcs_read32 (VM_EXIT_REASON ); -if (likely (!vmx->exit_reason.failed_vmentry)) + vt->exit_reason.full =vmcs_read32 (VM_EXIT_REASON ); +if (likely (!vt->exit_reason.failed_vmentry)) vmx->idt_vectoring_info =vmcs_read32 (IDT_VECTORING_INFO_FIELD ); -if ((u16 )vmx->exit_reason.basic ==EXIT_REASON_EXCEPTION_NMI && +if ((u16 )vt->exit_reason.basic ==EXIT_REASON_EXCEPTION_NMI &&is_nmi (vmx_get_intr_info (vcpu)))__vmx_handle_nmi (vcpu);
@@ -7331,7 +7331,7 @@ fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit)* checking. */ if (vmx->nested.nested_run_pending && - !vmx->exit_reason.failed_vmentry) + !vmx_get_exit_reason (vcpu).failed_vmentry) ++vcpu->stat.nested_run; vmx->nested.nested_run_pending =0 ;
arch/x86/kvm/vmx/vmx.h +2 -40
67 lines not on this page · raw message
base-commit: 14cfaed7621d53af608fd96aa36188064937ca44
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
In reply to Sean ChristophersonQuoted 11 lines · Sean Christopherson
Rework
__kvm_emulate_hypercall()into a macro so that completion of hypercalls that don't exit to userspace use direct function calls to the completion helper, i.e. don't trigger a retpoline when RETPOLINE=y.Opportunistically take the names of the input registers, as opposed to taking the input values, to preemptively dedup more of the calling code (TDX needs to use different registers). Use the direct GPR accessors to read values to avoid the pointless marking of the registers as available (KVM requires GPRs to always be available).
Signed-off-by Sean Christopherson <[email protected]>
Quoted 99 lines
--- arch/x86/kvm/x86.c | 29 +++++++++-------------------- arch/x86/kvm/x86.h | 25 ++++++++++++++++++++----- 2 files changed, 29 insertions(+), 25 deletions(-)arch/x86/kvm/x86.c +9
-20@@ -9982,11 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)return kvm_skip_emulated_instruction (vcpu); } -int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl, -int (*complete_hypercall)(struct kvm_vcpu *)) +int ____kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)) {unsigned long ret;@@ -10073,32 +10073,21 @@ int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, out: vcpu->run->hypercall.ret = ret; -complete_hypercall (vcpu);return 1 ; } -EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall); +EXPORT_SYMBOL_GPL (____kvm_emulate_hypercall);int kvm_emulate_hypercall (struct kvm_vcpu *vcpu) { -unsigned long nr, a0, a1, a2, a3; -int op_64_bit; -int cpl; -if (kvm_xen_hypercall_enabled (vcpu->kvm))return kvm_xen_hypercall (vcpu);if (kvm_hv_hypercall_enabled (vcpu))return kvm_hv_hypercall (vcpu); - nr =kvm_rax_read (vcpu); - a0 =kvm_rbx_read (vcpu); - a1 =kvm_rcx_read (vcpu); - a2 =kvm_rdx_read (vcpu); - a3 =kvm_rsi_read (vcpu); - op_64_bit =is_64_bit_hypercall (vcpu); - cpl =kvm_x86_call (get_cpl)(vcpu); - -return __kvm_emulate_hypercall (vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, +return __kvm_emulate_hypercall (vcpu, rax, rbx, rcx, rdx, rsi, +is_64_bit_hypercall (vcpu), +kvm_x86_call (get_cpl)(vcpu), complete_hypercall_exit); }EXPORT_SYMBOL_GPL (kvm_emulate_hypercall);arch/x86/kvm/x86.h +20
-5@@ -617,11 +617,26 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)return kvm->arch.hypercall_exit_enabled &BIT (hc_nr); } -int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl, -int (*complete_hypercall)(struct kvm_vcpu *)); +int ____kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)); + +#define __kvm_emulate_hypercall (_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \ +({ \ + int __ret; \ + \ + __ret = ____kvm_emulate_hypercall(_vcpu, \ + kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu), \ + kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu), \ + kvm_##a3##_read(_vcpu), op_64_bit, cpl, \ + complete_hypercall); \ + \ + if (__ret > 0) \ + complete_hypercall(_vcpu); \ + __ret; \ +})int kvm_emulate_hypercall (struct kvm_vcpu *vcpu);
Paolo Bonzini
maintainerRe: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
In reply to Sean ChristophersonQuoted 96 lines · Sean Christopherson
Rework
__kvm_emulate_hypercall()into a macro so that completion of hypercalls that don't exit to userspace use direct function calls to the completion helper, i.e. don't trigger a retpoline when RETPOLINE=y.Opportunistically take the names of the input registers, as opposed to taking the input values, to preemptively dedup more of the calling code (TDX needs to use different registers). Use the direct GPR accessors to read values to avoid the pointless marking of the registers as available (KVM requires GPRs to always be available).
Signed-off-by Sean Christopherson <[email protected]>Reviewed-by Binbin Wu <[email protected]>--- arch/x86/kvm/x86.c | 29 +++++++++-------------------- arch/x86/kvm/x86.h | 25 ++++++++++++++++++++----- 2 files changed, 29 insertions(+), 25 deletions(-)arch/x86/kvm/x86.c +9
-20@@ -9982,11 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)return kvm_skip_emulated_instruction (vcpu); } -int __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, -unsigned long a0,unsigned long a1, -unsigned long a2,unsigned long a3, -int op_64_bit,int cpl, -int (*complete_hypercall)(struct kvm_vcpu *)) +int ____kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit,int cpl, +int (*complete_hypercall)(struct kvm_vcpu *)) {unsigned long ret;@@ -10073,32 +10073,21 @@ int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, out: vcpu->run->hypercall.ret = ret; -complete_hypercall (vcpu);return 1 ; } -EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall); +EXPORT_SYMBOL_GPL (____kvm_emulate_hypercall);int kvm_emulate_hypercall (struct kvm_vcpu *vcpu) { -unsigned long nr, a0, a1, a2, a3; -int op_64_bit; -int cpl; -if (kvm_xen_hypercall_enabled (vcpu->kvm))return kvm_xen_hypercall (vcpu);if (kvm_hv_hypercall_enabled (vcpu))return kvm_hv_hypercall (vcpu); - nr =kvm_rax_read (vcpu); - a0 =kvm_rbx_read (vcpu); - a1 =kvm_rcx_read (vcpu); - a2 =kvm_rdx_read (vcpu); - a3 =kvm_rsi_read (vcpu); - op_64_bit =is_64_bit_hypercall (vcpu); - cpl =kvm_x86_call (get_cpl)(vcpu); - -return __kvm_emulate_hypercall (vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, +return __kvm_emulate_hypercall (vcpu, rax, rbx, rcx, rdx, rsi, +is_64_bit_hypercall (vcpu), +kvm_x86_call (get_cpl)(vcpu), complete_hypercall_exit); }EXPORT_SYMBOL_GPL (kvm_emulate_hypercall);arch/x86/kvm/x86.h +8
-5@@ -617,11 +617,26 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)return kvm->arch.hypercall_exit_enabled &BIT (hc_nr); } d - -int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl, - int (*complete_hypercall)(struct kvm_vcpu *)); +int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl, + int (*complete_hypercall)(struct kvm_vcpu *)); + +#define __kvm_emulate_hypercall (_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \ +({ \
arch/x86/kvm/x86.h static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) · line 637
Quoted 10 lines
arch/x86/kvm/x86.h +10
-0+int __ret; \ + \ + __ret = ____kvm_emulate_hypercall(_vcpu, \ + kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu), \ + kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu), \ + kvm_##a3##_read(_vcpu), op_64_bit, cpl, \ + complete_hypercall); \ + \ + if (__ret > 0) \ + complete_hypercall(_vcpu); \
So based on the review of the previous patch this should become
__ret = complete_hypercall(_vcpu);
Applied with this change to kvm-coco-queue, thanks.
Paolo
Quoted 5 lines
arch/x86/kvm/x86.h +2
-0+ __ret; \ +})int kvm_emulate_hypercall (struct kvm_vcpu *vcpu);
Sean Christopherson
maintainerRe: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
Quoted 16 lines · Paolo Bonzini
arch/x86/kvm/x86.h static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) · line 637
Quoted 12 lines · Sean Christopherson
arch/x86/kvm/x86.h +12
-0+#define __kvm_emulate_hypercall (_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \ +({ \ + int __ret; \ + \ + __ret = ____kvm_emulate_hypercall(_vcpu, \ + kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu), \ + kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu), \ + kvm_##a3##_read(_vcpu), op_64_bit, cpl, \ + complete_hypercall); \ + \ + if (__ret > 0) \ + complete_hypercall(_vcpu); \So based on the review of the previous patch this should become
Quoted 3 lines
__ret = complete_hypercall(_vcpu);Applied with this change to kvm-coco-queue, thanks.
I was planning on applying this for 6.14. Should I still do that, or do you want to take the bulk of the series through kvm/next, or maybe let it set in kvm-coco-queue? I can't think of any potential conflicts off the top of my head, and the refactoring is really only useful for TDX.
Patch 1 should go in sooner than later though.
Quoted 9 lines · Sean Christopherson
Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX
Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.
Quoted 3 lines
As for abusing
vcpu->run->hypercall.ret... It's obviously a bit gross, but I think it's a lesser evil than having multiple a one-line wrappers just to trampoline in the return code.
Doesn't seem to be "gross" to me, and AFAICT now for TDX we just need to play with __kvm_emulate_hypercall() with a TDX-specific completion callback.
Which is nice. Thanks!
For this series:
Quoted 10 lines · Sean Christopherson
Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX
Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.
[...]
Applied patch 1 to kvm-x86 fixes. I'm going to hold off on the rest until the dust settles on the SEAMCALL interfaces, e.g. in case TDX ends up marshalling state into the "normal" GPRs.
[1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit()
https://github.com/kvm-x86/linux/commit/a317794eefd0
[2/6] KVM: x86: Add a helper to check for user interception of KVM hypercalls
(no commit info)
[3/6] KVM: x86: Move "emulate hypercall" function declarations to x86.h
(no commit info)
[4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall
(no commit info)
[5/6] KVM: x86: Always complete hypercall via function callback
(no commit info)
[6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
(no commit info)
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Sean Christopherson
Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX
Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.
Quoted 4 lines
Quoted 1 line
[...]
Applied patch 1 to kvm-x86 fixes. I'm going to hold off on the rest until the dust settles on the SEAMCALL interfaces, e.g. in case TDX ends up marshalling state into the "normal" GPRs.
Hi Sean, Based on your suggestions in the link https://lore.kernel.org/kvm/[email protected], the v2 of "KVM: TDX: TDX hypercalls may exit to userspace" is planned to morph the TDG.VP.VMCALL with KVM hypercall to EXIT_REASON_VMCALL and marshall r10~r14 from vp_enter_args in struct vcpu_tdx to the appropriate x86 registers for KVM hypercall handling.
arch/x86/kvm/vmx/tdx.c +0 -0
@@ -935,6 +935,23 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit)
return tdx_exit_handlers_fastpath(vcpu);
}
+static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
+{
+ tdvmcall_set_return_code(vcpu, vcpu->run->hypercall.ret);
+ return 1;
+}
+
+static int tdx_emulate_vmcall(struct kvm_vcpu *vcpu)
+{
+ kvm_rax_write(vcpu, to_tdx(vcpu)->vp_enter_args.r10);
+ kvm_rbx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r11);
+ kvm_rcx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r12);
+ kvm_rdx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r13);
+ kvm_rsi_write(vcpu, to_tdx(vcpu)->vp_enter_args.r14);
+
+ return __kvm_emulate_hypercall(vcpu, 0, complete_hypercall_exit); +}
+
static int handle_tdvmcall(struct kvm_vcpu *vcpu)
{
switch (tdvmcall_leaf(vcpu)) {
@@ -1286,6 +1303,8 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) return 0;
case EXIT_REASON_TDCALL:
return handle_tdvmcall(vcpu);
+ case EXIT_REASON_VMCALL:
+ return tdx_emulate_vmcall(vcpu);
default:
break;
}
To test TDX, I made some modifications to your patch "KVM: x86: Refactor __kvm_emulate_hypercall() into a macro" Are the following changes make sense to you?
arch/x86/kvm/x86.c +0 -0
@@ -10088,9 +10088,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
if (kvm_hv_hypercall_enabled(vcpu))
return kvm_hv_hypercall(vcpu);
- return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi,
- is_64_bit_hypercall(vcpu),
- kvm_x86_call(get_cpl)(vcpu),
+ return __kvm_emulate_hypercall(vcpu, kvm_x86_call(get_cpl)(vcpu), complete_hypercall_exit); }
EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);
arch/x86/kvm/x86.h +0 -0
@@ -623,19 +623,18 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
int op_64_bit, int cpl,
int (*complete_hypercall)(struct kvm_vcpu *)); -#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \
-({ \
- int __ret; \
- \
- __ret = ____kvm_emulate_hypercall(_vcpu, \
- kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu), \
- kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu), \
- kvm_##a3##_read(_vcpu), op_64_bit, cpl, \
- complete_hypercall); \
- \
- if (__ret > 0) \
- __ret = complete_hypercall(_vcpu); \
- __ret; \
+#define __kvm_emulate_hypercall(_vcpu, cpl, complete_hypercall) \
+({ \
+ int __ret; \
+ __ret = ____kvm_emulate_hypercall(_vcpu, kvm_rax_read(_vcpu), \
+ kvm_rbx_read(_vcpu), kvm_rcx_read(_vcpu), \
+ kvm_rdx_read(_vcpu), kvm_rsi_read(_vcpu), \
+ is_64_bit_hypercall(_vcpu), cpl, \
+ complete_hypercall); \
+ \
+ if (__ret > 0) \
+ __ret = complete_hypercall(_vcpu); \
+ __ret; \
})
int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
Sean Christopherson
maintainerQuoted 21 lines · Binbin Wu
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Sean Christopherson
Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX
Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.
Quoted 4 lines
Quoted 1 line
[...]
Applied patch 1 to kvm-x86 fixes. I'm going to hold off on the rest until the dust settles on the SEAMCALL interfaces, e.g. in case TDX ends up marshalling state into the "normal" GPRs.
Hi Sean, Based on your suggestions in the link
https://lore.kernel.org/kvm/[email protected], the v2 of "KVM: TDX:
TDX hypercalls may exit to userspace" is planned to morph the TDG.VP.VMCALL with KVM hypercall toEXIT_REASON_VMCALLand marshall r10~r14 fromvp_enter_argsin structvcpu_tdxto the appropriate x86 registers for KVM hypercall handling.
...
Quoted 3 lines
To test TDX, I made some modifications to your patch "KVM: x86: Refactor
__kvm_emulate_hypercall()into a macro" Are the following changes make sense to you?
Yes, but I think we can go a step further and effectively revert the bulk of commit e913ef159fad ("KVM: x86: Split core of hypercall emulation to helper function"), i.e. have ____kvm_emulate_hypercall() read the GPRs instead of passing them in via the macro.
Quoted 50 lines
arch/x86/kvm/x86.c +1
-3@@ -10088,9 +10088,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) if (kvm_hv_hypercall_enabled(vcpu)) return kvm_hv_hypercall(vcpu); - return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi, - is_64_bit_hypercall(vcpu), - kvm_x86_call(get_cpl)(vcpu), + return __kvm_emulate_hypercall(vcpu, kvm_x86_call(get_cpl)(vcpu), complete_hypercall_exit); } EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);arch/x86/kvm/x86.h +12
-13@@ -623,19 +623,18 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, int op_64_bit, int cpl, int (*complete_hypercall)(struct kvm_vcpu *)); -#define __kvm_emulate_hypercall (_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \ -({ \ - int __ret; \ - \ - __ret = ____kvm_emulate_hypercall(_vcpu, \ - kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu), \ - kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu), \ - kvm_##a3##_read(_vcpu), op_64_bit, cpl, \ - complete_hypercall); \ - \ - if (__ret > 0) \ - __ret = complete_hypercall(_vcpu); \ - __ret; \ +#define __kvm_emulate_hypercall (_vcpu, cpl, complete_hypercall) \ +({ \ + int __ret; \ + __ret = ____kvm_emulate_hypercall(_vcpu, kvm_rax_read(_vcpu), \ + kvm_rbx_read(_vcpu), kvm_rcx_read(_vcpu), \ + kvm_rdx_read(_vcpu), kvm_rsi_read(_vcpu), \ + is_64_bit_hypercall(_vcpu), cpl, \ + complete_hypercall); \ + \ + if (__ret > 0) \ + __ret = complete_hypercall(_vcpu); \ + __ret; \ }) int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
Quoted 24 lines · Sean Christopherson
Quoted 21 lines · Binbin Wu
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Sean Christopherson
Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX
Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.
Quoted 4 lines
Quoted 1 line
[...]
Applied patch 1 to kvm-x86 fixes. I'm going to hold off on the rest until the dust settles on the SEAMCALL interfaces, e.g. in case TDX ends up marshalling state into the "normal" GPRs.
Hi Sean, Based on your suggestions in the link
https://lore.kernel.org/kvm/[email protected], the v2 of "KVM: TDX:
TDX hypercalls may exit to userspace" is planned to morph the TDG.VP.VMCALL with KVM hypercall toEXIT_REASON_VMCALLand marshall r10~r14 fromvp_enter_argsin structvcpu_tdxto the appropriate x86 registers for KVM hypercall handling....
Quoted 7 lines
Quoted 3 lines
To test TDX, I made some modifications to your patch "KVM: x86: Refactor
__kvm_emulate_hypercall()into a macro" Are the following changes make sense to you?Yes, but I think we can go a step further and effectively revert the bulk of commit
e913ef159fad("KVM: x86: Split core of hypercall emulation to helper function"), i.e. have____kvm_emulate_hypercall()read the GPRs instead of passing them in via the macro.
Sure.
Are you OK if I sent the change (as a prep patch) along with v2 of "TDX hypercalls may exit to userspace"?
Quoted 51 lines
Quoted 51 lines
arch/x86/kvm/x86.c +1
-3@@ -10088,9 +10088,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) if (kvm_hv_hypercall_enabled(vcpu)) return kvm_hv_hypercall(vcpu); - return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi, - is_64_bit_hypercall(vcpu), - kvm_x86_call(get_cpl)(vcpu), + return __kvm_emulate_hypercall(vcpu, kvm_x86_call(get_cpl)(vcpu), complete_hypercall_exit); } EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);arch/x86/kvm/x86.h +12
-13@@ -623,19 +623,18 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, int op_64_bit, int cpl, int (*complete_hypercall)(struct kvm_vcpu *)); -#define __kvm_emulate_hypercall (_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \ -({ \ - int __ret; \ - \ - __ret = ____kvm_emulate_hypercall(_vcpu, \ - kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu), \ - kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu), \ - kvm_##a3##_read(_vcpu), op_64_bit, cpl, \ - complete_hypercall); \ - \ - if (__ret > 0) \ - __ret = complete_hypercall(_vcpu); \ - __ret; \ +#define __kvm_emulate_hypercall (_vcpu, cpl, complete_hypercall) \ +({ \ + int __ret; \ + __ret = ____kvm_emulate_hypercall(_vcpu, kvm_rax_read(_vcpu), \ + kvm_rbx_read(_vcpu), kvm_rcx_read(_vcpu), \ + kvm_rdx_read(_vcpu), kvm_rsi_read(_vcpu), \ + is_64_bit_hypercall(_vcpu), cpl, \ + complete_hypercall); \ + \ + if (__ret > 0) \ + __ret = complete_hypercall(_vcpu); \ + __ret; \ }) int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
Sean Christopherson
maintainerQuoted 33 lines · Binbin Wu
Quoted 24 lines · Sean Christopherson
Quoted 21 lines · Binbin Wu
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Sean Christopherson
Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX
Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.
Quoted 4 lines
Quoted 1 line
[...]
Applied patch 1 to kvm-x86 fixes. I'm going to hold off on the rest until the dust settles on the SEAMCALL interfaces, e.g. in case TDX ends up marshalling state into the "normal" GPRs.
Hi Sean, Based on your suggestions in the link
https://lore.kernel.org/kvm/[email protected], the v2 of "KVM: TDX:
TDX hypercalls may exit to userspace" is planned to morph the TDG.VP.VMCALL with KVM hypercall toEXIT_REASON_VMCALLand marshall r10~r14 fromvp_enter_argsin structvcpu_tdxto the appropriate x86 registers for KVM hypercall handling....
Quoted 7 lines
Quoted 3 lines
To test TDX, I made some modifications to your patch "KVM: x86: Refactor
__kvm_emulate_hypercall()into a macro" Are the following changes make sense to you?Yes, but I think we can go a step further and effectively revert the bulk of commit
e913ef159fad("KVM: x86: Split core of hypercall emulation to helper function"), i.e. have____kvm_emulate_hypercall()read the GPRs instead of passing them in via the macro.
Quoted 4 lines
Sure.
Are you OK if I sent the change (as a prep patch) along with v2 of "TDX hypercalls may exit to userspace"?
Ya, go for it.