Skip to contentSkip to conversation

KVM: x86: Prep KVM hypercall handling for TDX

PATCH Applied Sean Christopherson 8 participants 39 messages Nov 28, 2024 – Jan 21, 2025

Subsystems (4)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

v4 cover KVM: x86: Prep KVM hypercall handling for TDX

Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX

Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.

As for abusing vcpu->run->hypercall.ret... It's obviously a bit gross, but I think it's a lesser evil than having multiple a one-line wrappers just to trampoline in the return code.

v4:
- Fix an SEV-* bug where KVM trips the WARN in is_64_bit_mode().
- Add a pile of reworks to (hopefully) avoid as much duplicate code when TDX comes along.

v3: https://lore.kernel.org/all/[email protected]

Binbin Wu (1):
KVM: x86: Add a helper to check for user interception of KVM
hypercalls

Sean Christopherson (5):
KVM: x86: Play nice with protected guests in complete_hypercall_exit()
KVM: x86: Move "emulate hypercall" function declarations to x86.h
KVM: x86: Bump hypercall stat prior to fully completing hypercall
KVM: x86: Always complete hypercall via function callback
KVM: x86: Refactor __kvm_emulate_hypercall() into a macro

 arch/x86/include/asm/kvm_host.h |  6 ----
 arch/x86/kvm/svm/sev.c          |  4 +--
 arch/x86/kvm/x86.c              | 50 +++++++++++----------------------
 arch/x86/kvm/x86.h              | 28 ++++++++++++++++++
 4 files changed, 47 insertions(+), 41 deletions(-)

base-commit: 4d911c7abee56771b0219a9fbf0120d06bdc9c14

Signature · 1 line
2.47.0.338.g60cca15819-goog
author maintainer
#1 lore raw

PATCH v4 1/6 KVM: x86: Play nice with protected guests in complete_hypercall_exit()

Use is_64_bit_hypercall() instead of is_64_bit_mode() to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable.

Hacking the sev_smoke_test selftest to generate a KVM_HC_MAP_GPA_RANGE hypercall via VMGEXIT trips the WARN:

  ------------[ cut here ]------------
  WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercall_exit+0x44/0xe0 [kvm]
  Modules linked in: kvm_amd kvm ... [last unloaded: kvm]
  CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--392e932fa0f3-feat #470
  Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024
  RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm]
  Call Trace:
   <TASK>
   kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm]
   kvm_vcpu_ioctl+0x54f/0x630 [kvm]
   __se_sys_ioctl+0x6b/0xc0
   do_syscall_64+0x83/0x160
   entry_SYSCALL_64_after_hwframe+0x76/0x7e
   </TASK>
  ---[ end trace 0000000000000000 ]---
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")
Cc Tom Lendacky <[email protected]>
---
 arch/x86/kvm/x86.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/x86.c +1 -1
index 2e713480933a..0b2fe4aa04a2 100644
@@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
 {
 	u64 ret = vcpu->run->hypercall.ret;
 
-	if (!is_64_bit_mode(vcpu))
+	if (!is_64_bit_hypercall(vcpu))
 		ret = (u32)ret;
 	kvm_rax_write(vcpu, ret);
 	++vcpu->stat.hypercalls;
Signature · 1 line
2.47.0.338.g60cca15819-goog
Reviewed-by on v4 1/6
#2 lore raw

Re: [PATCH v4 1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit()

Quoted 8 lines · Sean Christopherson

Use is_64_bit_hypercall() instead of is_64_bit_mode() to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable.

Hacking the sev_smoke_test selftest to generate a KVM_HC_MAP_GPA_RANGE hypercall via VMGEXIT trips the WARN:

Quoted 15 lines
   ------------[ cut here ]------------
   WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercall_exit+0x44/0xe0 [kvm]
   Modules linked in: kvm_amd kvm ... [last unloaded: kvm]
   CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--392e932fa0f3-feat #470
   Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024
   RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm]
   Call Trace:
    <TASK>
    kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm]
    kvm_vcpu_ioctl+0x54f/0x630 [kvm]
    __se_sys_ioctl+0x6b/0xc0
    do_syscall_64+0x83/0x160
    entry_SYSCALL_64_after_hwframe+0x76/0x7e
    </TASK>
   ---[ end trace 0000000000000000 ]---
Quoted 21 lines
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")
Cc Tom Lendacky <[email protected]>
---
  arch/x86/kvm/x86.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/x86.c +1 -1
index 2e713480933a..0b2fe4aa04a2 100644
@@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
  {
  	u64 ret = vcpu->run->hypercall.ret;
 
-	if (!is_64_bit_mode(vcpu))
+	if (!is_64_bit_hypercall(vcpu))
  		ret = (u32)ret;
  	kvm_rax_write(vcpu, ret);
  	++vcpu->stat.hypercalls;
Reviewed-by on v4 1/6
#3 lore raw

Re: [PATCH v4 1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit()

In reply to Sean Christopherson
Quoted 24 lines · Sean Christopherson

Use is_64_bit_hypercall() instead of is_64_bit_mode() to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable.

Hacking the sev_smoke_test selftest to generate a KVM_HC_MAP_GPA_RANGE hypercall via VMGEXIT trips the WARN:

  ------------[ cut here ]------------
  WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercall_exit+0x44/0xe0 [kvm]
  Modules linked in: kvm_amd kvm ... [last unloaded: kvm]
  CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--392e932fa0f3-feat #470
  Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024
  RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm]
  Call Trace:
   <TASK>
   kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm]
   kvm_vcpu_ioctl+0x54f/0x630 [kvm]
   __se_sys_ioctl+0x6b/0xc0
   do_syscall_64+0x83/0x160
   entry_SYSCALL_64_after_hwframe+0x76/0x7e
   </TASK>
  ---[ end trace 0000000000000000 ]---
Quoted 4 lines
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")
Cc Tom Lendacky <[email protected]>
Quoted 17 lines
---
 arch/x86/kvm/x86.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/x86.c +1 -1
index 2e713480933a..0b2fe4aa04a2 100644
@@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
 {
 	u64 ret = vcpu->run->hypercall.ret;
 
-	if (!is_64_bit_mode(vcpu))
+	if (!is_64_bit_hypercall(vcpu))
 		ret = (u32)ret;
 	kvm_rax_write(vcpu, ret);
 	++vcpu->stat.hypercalls;
Reviewed-by on v4 1/6
#4 lore raw

Re: [PATCH v4 1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit()

In reply to Sean Christopherson
Quoted 24 lines · Sean Christopherson

Use is_64_bit_hypercall() instead of is_64_bit_mode() to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable.

Hacking the sev_smoke_test selftest to generate a KVM_HC_MAP_GPA_RANGE hypercall via VMGEXIT trips the WARN:

  ------------[ cut here ]------------
  WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercall_exit+0x44/0xe0 [kvm]
  Modules linked in: kvm_amd kvm ... [last unloaded: kvm]
  CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--392e932fa0f3-feat #470
  Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024
  RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm]
  Call Trace:
   <TASK>
   kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm]
   kvm_vcpu_ioctl+0x54f/0x630 [kvm]
   __se_sys_ioctl+0x6b/0xc0
   do_syscall_64+0x83/0x160
   entry_SYSCALL_64_after_hwframe+0x76/0x7e
   </TASK>
  ---[ end trace 0000000000000000 ]---
Quoted 4 lines
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")
Cc Tom Lendacky <[email protected]>
Quoted 17 lines
---
 arch/x86/kvm/x86.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/x86.c +1 -1
index 2e713480933a..0b2fe4aa04a2 100644
@@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
 {
 	u64 ret = vcpu->run->hypercall.ret;
 
-	if (!is_64_bit_mode(vcpu))
+	if (!is_64_bit_hypercall(vcpu))
 		ret = (u32)ret;
 	kvm_rax_write(vcpu, ret);
 	++vcpu->stat.hypercalls;
Reviewed-by on v4 1/6
#5 lore raw

Re: [PATCH v4 1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit()

In reply to Sean Christopherson
Quoted 24 lines · Sean Christopherson

Use is_64_bit_hypercall() instead of is_64_bit_mode() to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable.

Hacking the sev_smoke_test selftest to generate a KVM_HC_MAP_GPA_RANGE hypercall via VMGEXIT trips the WARN:

   ------------[ cut here ]------------
   WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercall_exit+0x44/0xe0 [kvm]
   Modules linked in: kvm_amd kvm ... [last unloaded: kvm]
   CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--392e932fa0f3-feat #470
   Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024
   RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm]
   Call Trace:
    <TASK>
    kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm]
    kvm_vcpu_ioctl+0x54f/0x630 [kvm]
    __se_sys_ioctl+0x6b/0xc0
    do_syscall_64+0x83/0x160
    entry_SYSCALL_64_after_hwframe+0x76/0x7e
    </TASK>
   ---[ end trace 0000000000000000 ]---
Quoted 4 lines
Fixes b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state")
Cc Tom Lendacky <[email protected]>
Quoted 17 lines
---
  arch/x86/kvm/x86.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/x86.c +1 -1
index 2e713480933a..0b2fe4aa04a2 100644
@@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
  {
  	u64 ret = vcpu->run->hypercall.ret;
 
-	if (!is_64_bit_mode(vcpu))
+	if (!is_64_bit_hypercall(vcpu))
  		ret = (u32)ret;
  	kvm_rax_write(vcpu, ret);
  	++vcpu->stat.hypercalls;
author maintainer
#6 lore raw

PATCH v4 2/6 KVM: x86: Add a helper to check for user interception of KVM hypercalls

In reply to Sean Christopherson

From: Binbin Wu <[email protected]>

Add and use user_exit_on_hypercall() to check if userspace wants to handle a KVM hypercall instead of open-coding the logic everywhere.

No functional change intended.

[sean: squash into one patch, keep explicit KVM_HC_MAP_GPA_RANGE check]
---
 arch/x86/kvm/svm/sev.c | 4 ++--
 arch/x86/kvm/x86.c     | 2 +-
 arch/x86/kvm/x86.h     | 5 +++++
 3 files changed, 8 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/sev.c +2 -2
index 72674b8825c4..6ac6312c4d57 100644
@@ -3640,7 +3640,7 @@ static int snp_begin_psc_msr(struct vcpu_svm *svm, u64 ghcb_msr)
 		return 1; /* resume guest */
 	}
 
-	if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANGE))) {
+	if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) {
 		set_ghcb_msr(svm, GHCB_MSR_PSC_RESP_ERROR);
 		return 1; /* resume guest */
 	}
@@ -3723,7 +3723,7 @@ static int snp_begin_psc(struct vcpu_svm *svm, struct psc_buffer *psc)
 	bool huge;
 	u64 gfn;
 
-	if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANGE))) {
+	if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) {
 		snp_complete_psc(svm, VMGEXIT_PSC_ERROR_GENERIC);
 		return 1;
 	}
arch/x86/kvm/x86.c +1 -1
index 0b2fe4aa04a2..13fe5d6eb8f3 100644
@@ -10041,7 +10041,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 		u64 gpa = a0, npages = a1, attrs = a2;
 
 		ret = -KVM_ENOSYS;
-		if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANGE)))
+		if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE))
 			break;
 
 		if (!PAGE_ALIGNED(gpa) || !npages ||
arch/x86/kvm/x86.h +5 -0
index ec623d23d13d..45dd53284dbd 100644
@@ -612,4 +612,9 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size,
 			 unsigned int port, void *data,  unsigned int count,
 			 int in);
 
+static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
+{
+	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
+}
+
 #endif
Signature · 1 line
2.47.0.338.g60cca15819-goog
Reviewed-by on v4 2/6
#7 lore raw

Re: [PATCH v4 2/6] KVM: x86: Add a helper to check for user interception of KVM hypercalls

Quoted 5 lines · Sean Christopherson

From: Binbin Wu <[email protected]>

Add and use user_exit_on_hypercall() to check if userspace wants to handle a KVM hypercall instead of open-coding the logic everywhere.

Quoted 8 lines

No functional change intended.

[sean: squash into one patch, keep explicit KVM_HC_MAP_GPA_RANGE check]
Quoted 55 lines
---
 arch/x86/kvm/svm/sev.c | 4 ++--
 arch/x86/kvm/x86.c     | 2 +-
 arch/x86/kvm/x86.h     | 5 +++++
 3 files changed, 8 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/sev.c +2 -2
index 72674b8825c4..6ac6312c4d57 100644
@@ -3640,7 +3640,7 @@ static int snp_begin_psc_msr(struct vcpu_svm *svm, u64 ghcb_msr)
 		return 1; /* resume guest */
 	}
 
-	if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANGE))) {
+	if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) {
 		set_ghcb_msr(svm, GHCB_MSR_PSC_RESP_ERROR);
 		return 1; /* resume guest */
 	}
@@ -3723,7 +3723,7 @@ static int snp_begin_psc(struct vcpu_svm *svm, struct psc_buffer *psc)
 	bool huge;
 	u64 gfn;
 
-	if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANGE))) {
+	if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) {
 		snp_complete_psc(svm, VMGEXIT_PSC_ERROR_GENERIC);
 		return 1;
 	}
arch/x86/kvm/x86.c +1 -1
index 0b2fe4aa04a2..13fe5d6eb8f3 100644
@@ -10041,7 +10041,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 		u64 gpa = a0, npages = a1, attrs = a2;
 
 		ret = -KVM_ENOSYS;
-		if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANGE)))
+		if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE))
 			break;
 
 		if (!PAGE_ALIGNED(gpa) || !npages ||
arch/x86/kvm/x86.h +5 -0
index ec623d23d13d..45dd53284dbd 100644
@@ -612,4 +612,9 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size,
 			 unsigned int port, void *data,  unsigned int count,
 			 int in);
 
+static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
+{
+	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
+}
+
 #endif
author maintainer
#8 lore raw

PATCH v4 3/6 KVM: x86: Move "emulate hypercall" function declarations to x86.h

In reply to Sean Christopherson

Move the declarations for the hypercall emulation APIs to x86.h. While the helpers are exported, they are intended to be consumed only KVM vendor modules, i.e. don't need to exposed to the kernel at-large.

No functional change intended.

---
 arch/x86/include/asm/kvm_host.h | 6 ------
 arch/x86/kvm/x86.h              | 6 ++++++
 2 files changed, 6 insertions(+), 6 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -6
index e159e44a6a1b..c1251b371421 100644
@@ -2181,12 +2181,6 @@ static inline void kvm_clear_apicv_inhibit(struct kvm *kvm,
 	kvm_set_or_clear_apicv_inhibit(kvm, reason, false);
 }
 
-unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-				      unsigned long a0, unsigned long a1,
-				      unsigned long a2, unsigned long a3,
-				      int op_64_bit, int cpl);
-int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
-
 int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,
 		       void *insn, int insn_len);
 void kvm_mmu_print_sptes(struct kvm_vcpu *vcpu, gpa_t gpa, const char *msg);
arch/x86/kvm/x86.h +6 -0
index 45dd53284dbd..6db13b696468 100644
@@ -617,4 +617,10 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
 	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
 }
 
+unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+				      unsigned long a0, unsigned long a1,
+				      unsigned long a2, unsigned long a3,
+				      int op_64_bit, int cpl);
+int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
+
 #endif
Signature · 1 line
2.47.0.338.g60cca15819-goog
Reviewed-by on v4 3/6
#9 lore raw

Re: [PATCH v4 3/6] KVM: x86: Move "emulate hypercall" function declarations to x86.h

Quoted 5 lines · Sean Christopherson

Move the declarations for the hypercall emulation APIs to x86.h. While the helpers are exported, they are intended to be consumed only KVM vendor modules, i.e. don't need to exposed to the kernel at-large.

No functional change intended.

Quoted 38 lines
---
  arch/x86/include/asm/kvm_host.h | 6 ------
  arch/x86/kvm/x86.h              | 6 ++++++
  2 files changed, 6 insertions(+), 6 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -6
index e159e44a6a1b..c1251b371421 100644
@@ -2181,12 +2181,6 @@ static inline void kvm_clear_apicv_inhibit(struct kvm *kvm,
  	kvm_set_or_clear_apicv_inhibit(kvm, reason, false);
  }
 
-unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-				      unsigned long a0, unsigned long a1,
-				      unsigned long a2, unsigned long a3,
-				      int op_64_bit, int cpl);
-int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
-
  int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,
  		       void *insn, int insn_len);
  void kvm_mmu_print_sptes(struct kvm_vcpu *vcpu, gpa_t gpa, const char *msg);
arch/x86/kvm/x86.h +6 -0
index 45dd53284dbd..6db13b696468 100644
@@ -617,4 +617,10 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
  	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
  }
 
+unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+				      unsigned long a0, unsigned long a1,
+				      unsigned long a2, unsigned long a3,
+				      int op_64_bit, int cpl);
+int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
+
  #endif
Reviewed-by on v4 3/6
#10 lore raw

Re: [PATCH v4 3/6] KVM: x86: Move "emulate hypercall" function declarations to x86.h

In reply to Sean Christopherson
Quoted 2 lines · Sean Christopherson

Move the declarations for the hypercall emulation APIs to x86.h. While the helpers are exported, they are intended to be consumed only KVM vendor

s/only/only by/

Quoted 5 lines

modules, i.e. don't need to exposed to the kernel at-large.

No functional change intended.

Quoted 37 lines
---
 arch/x86/include/asm/kvm_host.h | 6 ------
 arch/x86/kvm/x86.h              | 6 ++++++
 2 files changed, 6 insertions(+), 6 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -6
index e159e44a6a1b..c1251b371421 100644
@@ -2181,12 +2181,6 @@ static inline void kvm_clear_apicv_inhibit(struct kvm *kvm,
 	kvm_set_or_clear_apicv_inhibit(kvm, reason, false);
 }
 
-unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-				      unsigned long a0, unsigned long a1,
-				      unsigned long a2, unsigned long a3,
-				      int op_64_bit, int cpl);
-int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
-
 int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,
 		       void *insn, int insn_len);
 void kvm_mmu_print_sptes(struct kvm_vcpu *vcpu, gpa_t gpa, const char *msg);
arch/x86/kvm/x86.h +6 -0
index 45dd53284dbd..6db13b696468 100644
@@ -617,4 +617,10 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
 	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
 }
 
+unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+				      unsigned long a0, unsigned long a1,
+				      unsigned long a2, unsigned long a3,
+				      int op_64_bit, int cpl);
+int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
+
 #endif
Reviewed-by on v4 3/6
#11 lore raw

Re: [PATCH v4 3/6] KVM: x86: Move "emulate hypercall" function declarations to x86.h

In reply to Sean Christopherson
Quoted 2 lines · Sean Christopherson

Move the declarations for the hypercall emulation APIs to x86.h. While the helpers are exported, they are intended to be consumed only KVM vendor

only -> only by

Quoted 1 line

modules, i.e. don't need to exposed to the kernel at-large.

to exposed -> to be exposed

Quoted 4 lines

No functional change intended.

Quoted 37 lines
---
  arch/x86/include/asm/kvm_host.h | 6 ------
  arch/x86/kvm/x86.h              | 6 ++++++
  2 files changed, 6 insertions(+), 6 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -6
index e159e44a6a1b..c1251b371421 100644
@@ -2181,12 +2181,6 @@ static inline void kvm_clear_apicv_inhibit(struct kvm *kvm,
  	kvm_set_or_clear_apicv_inhibit(kvm, reason, false);
  }
 
-unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-				      unsigned long a0, unsigned long a1,
-				      unsigned long a2, unsigned long a3,
-				      int op_64_bit, int cpl);
-int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
-
  int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,
  		       void *insn, int insn_len);
  void kvm_mmu_print_sptes(struct kvm_vcpu *vcpu, gpa_t gpa, const char *msg);
arch/x86/kvm/x86.h +6 -0
index 45dd53284dbd..6db13b696468 100644
@@ -617,4 +617,10 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
  	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
  }
 
+unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+				      unsigned long a0, unsigned long a1,
+				      unsigned long a2, unsigned long a3,
+				      int op_64_bit, int cpl);
+int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
+
  #endif
author maintainer
#12 lore raw

PATCH v4 4/6 KVM: x86: Bump hypercall stat prior to fully completing hypercall

In reply to Sean Christopherson

Increment the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits.

Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences).

---
 arch/x86/kvm/x86.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +2 -2
index 13fe5d6eb8f3..11434752b467 100644
@@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
 	if (!is_64_bit_hypercall(vcpu))
 		ret = (u32)ret;
 	kvm_rax_write(vcpu, ret);
-	++vcpu->stat.hypercalls;
 	return kvm_skip_emulated_instruction(vcpu);
 }
 
@@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 {
 	unsigned long ret;
 
+	++vcpu->stat.hypercalls;
+
 	trace_kvm_hypercall(nr, a0, a1, a2, a3);
 
 	if (!op_64_bit) {
@@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 	}
 
 out:
-	++vcpu->stat.hypercalls;
 	return ret;
 }
 EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
Signature · 1 line
2.47.0.338.g60cca15819-goog
Reviewed-by on v4 4/6
#13 lore raw

Re: [PATCH v4 4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall

Quoted 10 lines · Sean Christopherson

Increment the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits.

Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences).

Quoted 34 lines
---
  arch/x86/kvm/x86.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +2 -2
index 13fe5d6eb8f3..11434752b467 100644
@@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
  	if (!is_64_bit_hypercall(vcpu))
  		ret = (u32)ret;
  	kvm_rax_write(vcpu, ret);
-	++vcpu->stat.hypercalls;
  	return kvm_skip_emulated_instruction(vcpu);
  }
 
@@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
  {
  	unsigned long ret;
 
+	++vcpu->stat.hypercalls;
+
  	trace_kvm_hypercall(nr, a0, a1, a2, a3);
 
  	if (!op_64_bit) {
@@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
  	}
 
  out:
-	++vcpu->stat.hypercalls;
  	return ret;
  }
  EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
Reviewed-by on v4 4/6
#14 lore raw

Re: [PATCH v4 4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall

In reply to Sean Christopherson
Quoted 7 lines · Sean Christopherson

Increment the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits.

Quoted 5 lines

Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences).

There's a comment in the KVM_HC_MAP_GPA_RANGE case that reads:

	/* stat is incremented on completion. */

that should probably be deleted, but otherwise:

Also, if you want, you could get rid of the 'out' label, too, by doing:

	if (cpl)
		return -KVM_EPERM;
Quoted 33 lines
---
 arch/x86/kvm/x86.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +2 -2
index 13fe5d6eb8f3..11434752b467 100644
@@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
 	if (!is_64_bit_hypercall(vcpu))
 		ret = (u32)ret;
 	kvm_rax_write(vcpu, ret);
-	++vcpu->stat.hypercalls;
 	return kvm_skip_emulated_instruction(vcpu);
 }
 
@@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 {
 	unsigned long ret;
 
+	++vcpu->stat.hypercalls;
+
 	trace_kvm_hypercall(nr, a0, a1, a2, a3);
 
 	if (!op_64_bit) {
@@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 	}
 
 out:
-	++vcpu->stat.hypercalls;
 	return ret;
 }
 EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
#15 lore raw

Re: [PATCH v4 4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall

Quoted 18 lines · Tom Lendacky
Quoted 7 lines · Sean Christopherson

Increment the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits.

Quoted 5 lines

Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences).

There's a comment in the KVM_HC_MAP_GPA_RANGE case that reads:

	/* stat is incremented on completion. */
Quoted 8 lines

that should probably be deleted, but otherwise:

Also, if you want, you could get rid of the 'out' label, too, by doing:

	if (cpl)
		return -KVM_EPERM;

Until I saw the next patch... nevermind.

Thanks,
Tom

Quoted 34 lines
Quoted 34 lines
---
 arch/x86/kvm/x86.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +2 -2
index 13fe5d6eb8f3..11434752b467 100644
@@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
 	if (!is_64_bit_hypercall(vcpu))
 		ret = (u32)ret;
 	kvm_rax_write(vcpu, ret);
-	++vcpu->stat.hypercalls;
 	return kvm_skip_emulated_instruction(vcpu);
 }
 
@@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 {
 	unsigned long ret;
 
+	++vcpu->stat.hypercalls;
+
 	trace_kvm_hypercall(nr, a0, a1, a2, a3);
 
 	if (!op_64_bit) {
@@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 	}
 
 out:
-	++vcpu->stat.hypercalls;
 	return ret;
 }
 EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
Reviewed-by on v4 4/6
#16 lore raw

Re: [PATCH v4 4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall

In reply to Sean Christopherson
Quoted 7 lines · Sean Christopherson

Increment the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits.

Quoted 5 lines

Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences).

Quoted 33 lines
---
  arch/x86/kvm/x86.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +2 -2
index 13fe5d6eb8f3..11434752b467 100644
@@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
  	if (!is_64_bit_hypercall(vcpu))
  		ret = (u32)ret;
  	kvm_rax_write(vcpu, ret);
-	++vcpu->stat.hypercalls;
  	return kvm_skip_emulated_instruction(vcpu);
  }
 
@@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
  {
  	unsigned long ret;
 
+	++vcpu->stat.hypercalls;
+
  	trace_kvm_hypercall(nr, a0, a1, a2, a3);
 
  	if (!op_64_bit) {
@@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
  	}
 
  out:
-	++vcpu->stat.hypercalls;
  	return ret;
  }
  EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
author maintainer
#17 lore raw

PATCH v4 5/6 KVM: x86: Always complete hypercall via function callback

In reply to Sean Christopherson

Finish "emulation" of KVM hypercalls by function callback, even when the hypercall is handled entirely within KVM, i.e. doesn't require an exit to userspace, and refactor __kvm_emulate_hypercall()'s return value to *only* communicate whether or not KVM should exit to userspace or resume the guest.

(Ab)Use vcpu->run->hypercall.ret to propagate the return value to the callback, purely to avoid having to add a trampoline for every completion callback.

Using the function return value for KVM's control flow eliminates the multiplexed return value, where '0' for KVM_HC_MAP_GPA_RANGE (and only that hypercall) means "exit to userspace".

Note, the unnecessary extra indirect call and thus potential retpoline will be eliminated in the near future by converting the intermediate layer to a macro.

---
 arch/x86/kvm/x86.c | 29 ++++++++++++-----------------
 arch/x86/kvm/x86.h | 10 ++++++----
 2 files changed, 18 insertions(+), 21 deletions(-)
arch/x86/kvm/x86.c +12 -17
index 11434752b467..39be2a891ab4 100644
@@ -9982,10 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
 	return kvm_skip_emulated_instruction(vcpu);
 }
 
-unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-				      unsigned long a0, unsigned long a1,
-				      unsigned long a2, unsigned long a3,
-				      int op_64_bit, int cpl)
+int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			    unsigned long a0, unsigned long a1,
+			    unsigned long a2, unsigned long a3,
+			    int op_64_bit, int cpl,
+			    int (*complete_hypercall)(struct kvm_vcpu *))
 {
 	unsigned long ret;
 
@@ -10061,7 +10062,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 			vcpu->run->hypercall.flags |= KVM_EXIT_HYPERCALL_LONG_MODE;
 
 		WARN_ON_ONCE(vcpu->run->hypercall.flags & KVM_EXIT_HYPERCALL_MBZ);
-		vcpu->arch.complete_userspace_io = complete_hypercall_exit;
+		vcpu->arch.complete_userspace_io = complete_hypercall;
 		/* stat is incremented on completion. */
 		return 0;
 	}
@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 	}
 
 out:
-	return ret;
+	vcpu->run->hypercall.ret = ret;
+	complete_hypercall(vcpu);
+	return 1;
 }
 EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
 
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 {
-	unsigned long nr, a0, a1, a2, a3, ret;
+	unsigned long nr, a0, a1, a2, a3;
 	int op_64_bit;
 	int cpl;
 
@@ -10095,16 +10098,8 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 	op_64_bit = is_64_bit_hypercall(vcpu);
 	cpl = kvm_x86_call(get_cpl)(vcpu);
 
-	ret = __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl);
-	if (nr == KVM_HC_MAP_GPA_RANGE && !ret)
-		/* MAP_GPA tosses the request to the user space. */
-		return 0;
-
-	if (!op_64_bit)
-		ret = (u32)ret;
-	kvm_rax_write(vcpu, ret);
-
-	return kvm_skip_emulated_instruction(vcpu);
+	return __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl,
+				       complete_hypercall_exit);
 }
 EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);
 
arch/x86/kvm/x86.h +6 -4
index 6db13b696468..28adc8ea04bf 100644
@@ -617,10 +617,12 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
 	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
 }
 
-unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-				      unsigned long a0, unsigned long a1,
-				      unsigned long a2, unsigned long a3,
-				      int op_64_bit, int cpl);
+int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			    unsigned long a0, unsigned long a1,
+			    unsigned long a2, unsigned long a3,
+			    int op_64_bit, int cpl,
+			    int (*complete_hypercall)(struct kvm_vcpu *));
+
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
 
 #endif
Signature · 1 line
2.47.0.338.g60cca15819-goog
#18 lore raw

Re: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback

Quoted 56 lines · Sean Christopherson

Finish "emulation" of KVM hypercalls by function callback, even when the hypercall is handled entirely within KVM, i.e. doesn't require an exit to userspace, and refactor __kvm_emulate_hypercall()'s return value to *only* communicate whether or not KVM should exit to userspace or resume the guest.

(Ab)Use vcpu->run->hypercall.ret to propagate the return value to the callback, purely to avoid having to add a trampoline for every completion callback.

Using the function return value for KVM's control flow eliminates the multiplexed return value, where '0' for KVM_HC_MAP_GPA_RANGE (and only that hypercall) means "exit to userspace".

Note, the unnecessary extra indirect call and thus potential retpoline will be eliminated in the near future by converting the intermediate layer to a macro.

---
  arch/x86/kvm/x86.c | 29 ++++++++++++-----------------
  arch/x86/kvm/x86.h | 10 ++++++----
  2 files changed, 18 insertions(+), 21 deletions(-)
arch/x86/kvm/x86.c +6 -5
index 11434752b467..39be2a891ab4 100644
@@ -9982,10 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
  	return kvm_skip_emulated_instruction(vcpu);
  }
 
-unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-				      unsigned long a0, unsigned long a1,
-				      unsigned long a2, unsigned long a3,
-				      int op_64_bit, int cpl)
+int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			    unsigned long a0, unsigned long a1,
+			    unsigned long a2, unsigned long a3,
+			    int op_64_bit, int cpl,
+			    int (*complete_hypercall)(struct kvm_vcpu *))
  {
  	unsigned long ret;
 
@@ -10061,7 +10062,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
  			vcpu->run->hypercall.flags |= KVM_EXIT_HYPERCALL_LONG_MODE;
 
  		WARN_ON_ONCE(vcpu->run->hypercall.flags & KVM_EXIT_HYPERCALL_MBZ);
-		vcpu->arch.complete_userspace_io = complete_hypercall_exit;
+		vcpu->arch.complete_userspace_io = complete_hypercall;
  		/* stat is incremented on completion. */
  		return 0;
  	}

arch/x86/kvm/x86.c unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, · line 10077

Quoted 7 lines
arch/x86/kvm/x86.c +3 -1
@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
  	}
 
  out:
-	return ret;
+	vcpu->run->hypercall.ret = ret;
+	complete_hypercall(vcpu);
+	return 1;

shouldn't it be

	return complete_hypercall(vcpu);

?

Originally, kvm_emulate_hypercall() returns kvm_skip_emulated_instruction(). Now it becomes

	kvm_skip_emulated_instruction();
	return 1;

I don't go deep to see if kvm_skip_emulated_instruction() always return 1 for this case. But logically,

	return complete_hypercall(vcpu);

looks more reasonable.

Quoted 50 lines
arch/x86/kvm/x86.c +3 -11
  }
  EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
 
  int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
  {
-	unsigned long nr, a0, a1, a2, a3, ret;
+	unsigned long nr, a0, a1, a2, a3;
  	int op_64_bit;
  	int cpl;
 
@@ -10095,16 +10098,8 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
  	op_64_bit = is_64_bit_hypercall(vcpu);
  	cpl = kvm_x86_call(get_cpl)(vcpu);
 
-	ret = __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl);
-	if (nr == KVM_HC_MAP_GPA_RANGE && !ret)
-		/* MAP_GPA tosses the request to the user space. */
-		return 0;
-
-	if (!op_64_bit)
-		ret = (u32)ret;
-	kvm_rax_write(vcpu, ret);
-
-	return kvm_skip_emulated_instruction(vcpu);
+	return __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl,
+				       complete_hypercall_exit);
  }
  EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);
 
arch/x86/kvm/x86.h +6 -4
index 6db13b696468..28adc8ea04bf 100644
@@ -617,10 +617,12 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
  	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
  }
 
-unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-				      unsigned long a0, unsigned long a1,
-				      unsigned long a2, unsigned long a3,
-				      int op_64_bit, int cpl);
+int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			    unsigned long a0, unsigned long a1,
+			    unsigned long a2, unsigned long a3,
+			    int op_64_bit, int cpl,
+			    int (*complete_hypercall)(struct kvm_vcpu *));
+
  int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
 
  #endif
author maintainer
#19 lore raw

Re: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback

Quoted 37 lines · Xiaoyao Li
Quoted 27 lines · Sean Christopherson
arch/x86/kvm/x86.c +6 -5
index 11434752b467..39be2a891ab4 100644
@@ -9982,10 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
  	return kvm_skip_emulated_instruction(vcpu);
  }
-unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-				      unsigned long a0, unsigned long a1,
-				      unsigned long a2, unsigned long a3,
-				      int op_64_bit, int cpl)
+int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			    unsigned long a0, unsigned long a1,
+			    unsigned long a2, unsigned long a3,
+			    int op_64_bit, int cpl,
+			    int (*complete_hypercall)(struct kvm_vcpu *))
  {
  	unsigned long ret;
@@ -10061,7 +10062,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
  			vcpu->run->hypercall.flags |= KVM_EXIT_HYPERCALL_LONG_MODE;
  		WARN_ON_ONCE(vcpu->run->hypercall.flags & KVM_EXIT_HYPERCALL_MBZ);
-		vcpu->arch.complete_userspace_io = complete_hypercall_exit;
+		vcpu->arch.complete_userspace_io = complete_hypercall;
  		/* stat is incremented on completion. */
  		return 0;
  	}

arch/x86/kvm/x86.c unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, · line 10076

Quoted 6 lines
arch/x86/kvm/x86.c +3 -1
@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
  	}
  out:
-	return ret;
+	vcpu->run->hypercall.ret = ret;
+	complete_hypercall(vcpu);
+	return 1;

shouldn't it be

Quoted 3 lines
	return complete_hypercall(vcpu);

?

Ouch. Yes, it most definitely should be.

Quoted 8 lines

Originally, kvm_emulate_hypercall() returns kvm_skip_emulated_instruction(). Now it becomes

	kvm_skip_emulated_instruction();
	return 1;

I don't go deep to see if kvm_skip_emulated_instruction() always return 1 for this case.

It doesn't. KVM needs to exit to userspace if userspace is single-stepping, or in the extremely unlikely scenario that KVM can't skip the emulated instruction (which can very theoretically happen on older AMD CPUs).

#20 lore raw

Re: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback

In reply to Sean Christopherson
Quoted 56 lines · Sean Christopherson

Finish "emulation" of KVM hypercalls by function callback, even when the hypercall is handled entirely within KVM, i.e. doesn't require an exit to userspace, and refactor __kvm_emulate_hypercall()'s return value to *only* communicate whether or not KVM should exit to userspace or resume the guest.

(Ab)Use vcpu->run->hypercall.ret to propagate the return value to the callback, purely to avoid having to add a trampoline for every completion callback.

Using the function return value for KVM's control flow eliminates the multiplexed return value, where '0' for KVM_HC_MAP_GPA_RANGE (and only that hypercall) means "exit to userspace".

Note, the unnecessary extra indirect call and thus potential retpoline will be eliminated in the near future by converting the intermediate layer to a macro.

---
 arch/x86/kvm/x86.c | 29 ++++++++++++-----------------
 arch/x86/kvm/x86.h | 10 ++++++----
 2 files changed, 18 insertions(+), 21 deletions(-)
arch/x86/kvm/x86.c +6 -5
index 11434752b467..39be2a891ab4 100644
@@ -9982,10 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
 	return kvm_skip_emulated_instruction(vcpu);
 }
 
-unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-				      unsigned long a0, unsigned long a1,
-				      unsigned long a2, unsigned long a3,
-				      int op_64_bit, int cpl)
+int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			    unsigned long a0, unsigned long a1,
+			    unsigned long a2, unsigned long a3,
+			    int op_64_bit, int cpl,
+			    int (*complete_hypercall)(struct kvm_vcpu *))
 {
 	unsigned long ret;
 
@@ -10061,7 +10062,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 			vcpu->run->hypercall.flags |= KVM_EXIT_HYPERCALL_LONG_MODE;
 
 		WARN_ON_ONCE(vcpu->run->hypercall.flags & KVM_EXIT_HYPERCALL_MBZ);
-		vcpu->arch.complete_userspace_io = complete_hypercall_exit;
+		vcpu->arch.complete_userspace_io = complete_hypercall;
 		/* stat is incremented on completion. */
 		return 0;
 	}

arch/x86/kvm/x86.c unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, · line 10077

Quoted 7 lines
arch/x86/kvm/x86.c +3 -1
@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 	}
 
 out:
-	return ret;
+	vcpu->run->hypercall.ret = ret;
+	complete_hypercall(vcpu);
+	return 1;

Should this do return complete_hypercall(vcpu) so that you get the return code from kvm_skip_emulated_instruction()?

Thanks,
Tom

Quoted 50 lines
arch/x86/kvm/x86.c +3 -11
 }
 EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
 
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 {
-	unsigned long nr, a0, a1, a2, a3, ret;
+	unsigned long nr, a0, a1, a2, a3;
 	int op_64_bit;
 	int cpl;
 
@@ -10095,16 +10098,8 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 	op_64_bit = is_64_bit_hypercall(vcpu);
 	cpl = kvm_x86_call(get_cpl)(vcpu);
 
-	ret = __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl);
-	if (nr == KVM_HC_MAP_GPA_RANGE && !ret)
-		/* MAP_GPA tosses the request to the user space. */
-		return 0;
-
-	if (!op_64_bit)
-		ret = (u32)ret;
-	kvm_rax_write(vcpu, ret);
-
-	return kvm_skip_emulated_instruction(vcpu);
+	return __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl,
+				       complete_hypercall_exit);
 }
 EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);
 
arch/x86/kvm/x86.h +6 -4
index 6db13b696468..28adc8ea04bf 100644
@@ -617,10 +617,12 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
 	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
 }
 
-unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-				      unsigned long a0, unsigned long a1,
-				      unsigned long a2, unsigned long a3,
-				      int op_64_bit, int cpl);
+int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			    unsigned long a0, unsigned long a1,
+			    unsigned long a2, unsigned long a3,
+			    int op_64_bit, int cpl,
+			    int (*complete_hypercall)(struct kvm_vcpu *));
+
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
 
 #endif
#21 lore raw

Re: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback

Quoted 5 lines · Tom Lendacky
Quoted 3 lines · Sean Christopherson
(quoted diff) +0 -0
@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 	}
 
Quoted 8 lines

arch/x86/kvm/x86.c unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, · line 10077

Quoted 5 lines
arch/x86/kvm/x86.c +3 -1
 out:
-	return ret;
+	vcpu->run->hypercall.ret = ret;
+	complete_hypercall(vcpu);
+	return 1;

Should this do return complete_hypercall(vcpu) so that you get the return code from kvm_skip_emulated_instruction()?

Bah, ignore... already commented on by Xiaoyao.

Thanks,
Tom

Quoted 4 lines

Thanks,
Tom

author maintainer
#22 lore raw

Re: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback

Quoted 15 lines · Tom Lendacky
Quoted 5 lines · Tom Lendacky
Quoted 3 lines · Sean Christopherson
(quoted diff) +0 -0
@@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 	}
 
Quoted 8 lines

arch/x86/kvm/x86.c unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, · line 10077

Quoted 5 lines
arch/x86/kvm/x86.c +3 -1
 out:
-	return ret;
+	vcpu->run->hypercall.ret = ret;
+	complete_hypercall(vcpu);
+	return 1;

Should this do return complete_hypercall(vcpu) so that you get the return code from kvm_skip_emulated_instruction()?

Quoted 1 line

Bah, ignore... already commented on by Xiaoyao.

Reviewers: 2, Sean: 0

:-)

author maintainer
#23 lore raw

PATCH v4 6/6 KVM: x86: Refactor __kvm_emulate_hypercall() into a macro

In reply to Sean Christopherson

Rework __kvm_emulate_hypercall() into a macro so that completion of hypercalls that don't exit to userspace use direct function calls to the completion helper, i.e. don't trigger a retpoline when RETPOLINE=y.

Opportunistically take the names of the input registers, as opposed to taking the input values, to preemptively dedup more of the calling code (TDX needs to use different registers). Use the direct GPR accessors to read values to avoid the pointless marking of the registers as available (KVM requires GPRs to always be available).

---
 arch/x86/kvm/x86.c | 29 +++++++++--------------------
 arch/x86/kvm/x86.h | 25 ++++++++++++++++++++-----
 2 files changed, 29 insertions(+), 25 deletions(-)
arch/x86/kvm/x86.c +9 -20
index 39be2a891ab4..fef8b4e63d25 100644
@@ -9982,11 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
 	return kvm_skip_emulated_instruction(vcpu);
 }
 
-int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-			    unsigned long a0, unsigned long a1,
-			    unsigned long a2, unsigned long a3,
-			    int op_64_bit, int cpl,
-			    int (*complete_hypercall)(struct kvm_vcpu *))
+int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			      unsigned long a0, unsigned long a1,
+			      unsigned long a2, unsigned long a3,
+			      int op_64_bit, int cpl,
+			      int (*complete_hypercall)(struct kvm_vcpu *))
 {
 	unsigned long ret;
 
@@ -10073,32 +10073,21 @@ int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 
 out:
 	vcpu->run->hypercall.ret = ret;
-	complete_hypercall(vcpu);
 	return 1;
 }
-EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
+EXPORT_SYMBOL_GPL(____kvm_emulate_hypercall);
 
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 {
-	unsigned long nr, a0, a1, a2, a3;
-	int op_64_bit;
-	int cpl;
-
 	if (kvm_xen_hypercall_enabled(vcpu->kvm))
 		return kvm_xen_hypercall(vcpu);
 
 	if (kvm_hv_hypercall_enabled(vcpu))
 		return kvm_hv_hypercall(vcpu);
 
-	nr = kvm_rax_read(vcpu);
-	a0 = kvm_rbx_read(vcpu);
-	a1 = kvm_rcx_read(vcpu);
-	a2 = kvm_rdx_read(vcpu);
-	a3 = kvm_rsi_read(vcpu);
-	op_64_bit = is_64_bit_hypercall(vcpu);
-	cpl = kvm_x86_call(get_cpl)(vcpu);
-
-	return __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl,
+	return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi,
+				       is_64_bit_hypercall(vcpu),
+				       kvm_x86_call(get_cpl)(vcpu),
 				       complete_hypercall_exit);
 }
 EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);
arch/x86/kvm/x86.h +20 -5
index 28adc8ea04bf..ad6fe6159dea 100644
@@ -617,11 +617,26 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
 	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
 }
 
-int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-			    unsigned long a0, unsigned long a1,
-			    unsigned long a2, unsigned long a3,
-			    int op_64_bit, int cpl,
-			    int (*complete_hypercall)(struct kvm_vcpu *));
+int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			      unsigned long a0, unsigned long a1,
+			      unsigned long a2, unsigned long a3,
+			      int op_64_bit, int cpl,
+			      int (*complete_hypercall)(struct kvm_vcpu *));
+
+#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall)	\
+({												\
+	int __ret;										\
+												\
+	__ret = ____kvm_emulate_hypercall(_vcpu,						\
+					  kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu),	\
+					  kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu),	\
+					  kvm_##a3##_read(_vcpu), op_64_bit, cpl,		\
+					  complete_hypercall);					\
+												\
+	if (__ret > 0)										\
+		complete_hypercall(_vcpu);							\
+	__ret;											\
+})
 
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
 
Signature · 1 line
2.47.0.338.g60cca15819-goog
#24 lore raw

Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro

Quoted 9 lines · Sean Christopherson

Rework __kvm_emulate_hypercall() into a macro so that completion of hypercalls that don't exit to userspace use direct function calls to the completion helper, i.e. don't trigger a retpoline when RETPOLINE=y.

Opportunistically take the names of the input registers, as opposed to taking the input values, to preemptively dedup more of the calling code (TDX needs to use different registers). Use the direct GPR accessors to read values to avoid the pointless marking of the registers as available (KVM requires GPRs to always be available).

For TDX, there is an RFC relating to using descriptively named parameters instead of register names for tdh_vp_enter():

	https://lore.kernel.org/all/[email protected]/

Please do give some feedback on that approach. Note we need both KVM and x86 maintainer approval for SEAMCALL wrappers like tdh_vp_enter().

As proposed, that ends up with putting the values back into vcpu->arch.regs[] for __kvm_emulate_hypercall() which is not pretty:

static int tdx_emulate_vmcall(struct kvm_vcpu *vcpu) {
+ struct vcpu_tdx *tdx = to_tdx(vcpu);
int r;

+ kvm_r10_write(vcpu, tdx->vp_enter_args.tdcall.fn);
+ kvm_r11_write(vcpu, tdx->vp_enter_args.tdcall.subfn);
+ kvm_r12_write(vcpu, tdx->vp_enter_args.tdcall.vmcall.p2);
+ kvm_r13_write(vcpu, tdx->vp_enter_args.tdcall.vmcall.p3);
+ kvm_r14_write(vcpu, tdx->vp_enter_args.tdcall.vmcall.p4);
+
/*
* ABI for KVM tdvmcall argument:
* In Guest-Hypervisor Communication Interface(GHCI) specification,
@@ -1092,13 +1042,12 @@ static int tdx_emulate_vmcall(struct kvm_vcpu *vcpu)
* vendor-specific. KVM uses this for KVM hypercall. NOTE: KVM
* hypercall number starts from one. Zero isn't used for KVM hypercall
* number.
- *
- * R10: KVM hypercall number
- * arguments: R11, R12, R13, R14.
*/
r = __kvm_emulate_hypercall(vcpu, r10, r11, r12, r13, r14, true, 0,
R10, complete_hypercall_exit);

+ tdvmcall_set_return_code(vcpu, kvm_r10_read(vcpu));
+
return r > 0;
}

Quoted 101 lines
---
 arch/x86/kvm/x86.c | 29 +++++++++--------------------
 arch/x86/kvm/x86.h | 25 ++++++++++++++++++++-----
 2 files changed, 29 insertions(+), 25 deletions(-)
arch/x86/kvm/x86.c +9 -20
index 39be2a891ab4..fef8b4e63d25 100644
@@ -9982,11 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
 	return kvm_skip_emulated_instruction(vcpu);
 }
 
-int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-			    unsigned long a0, unsigned long a1,
-			    unsigned long a2, unsigned long a3,
-			    int op_64_bit, int cpl,
-			    int (*complete_hypercall)(struct kvm_vcpu *))
+int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			      unsigned long a0, unsigned long a1,
+			      unsigned long a2, unsigned long a3,
+			      int op_64_bit, int cpl,
+			      int (*complete_hypercall)(struct kvm_vcpu *))
 {
 	unsigned long ret;
 
@@ -10073,32 +10073,21 @@ int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 
 out:
 	vcpu->run->hypercall.ret = ret;
-	complete_hypercall(vcpu);
 	return 1;
 }
-EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
+EXPORT_SYMBOL_GPL(____kvm_emulate_hypercall);
 
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 {
-	unsigned long nr, a0, a1, a2, a3;
-	int op_64_bit;
-	int cpl;
-
 	if (kvm_xen_hypercall_enabled(vcpu->kvm))
 		return kvm_xen_hypercall(vcpu);
 
 	if (kvm_hv_hypercall_enabled(vcpu))
 		return kvm_hv_hypercall(vcpu);
 
-	nr = kvm_rax_read(vcpu);
-	a0 = kvm_rbx_read(vcpu);
-	a1 = kvm_rcx_read(vcpu);
-	a2 = kvm_rdx_read(vcpu);
-	a3 = kvm_rsi_read(vcpu);
-	op_64_bit = is_64_bit_hypercall(vcpu);
-	cpl = kvm_x86_call(get_cpl)(vcpu);
-
-	return __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl,
+	return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi,
+				       is_64_bit_hypercall(vcpu),
+				       kvm_x86_call(get_cpl)(vcpu),
 				       complete_hypercall_exit);
 }
 EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);
arch/x86/kvm/x86.h +20 -5
index 28adc8ea04bf..ad6fe6159dea 100644
@@ -617,11 +617,26 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
 	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
 }
 
-int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-			    unsigned long a0, unsigned long a1,
-			    unsigned long a2, unsigned long a3,
-			    int op_64_bit, int cpl,
-			    int (*complete_hypercall)(struct kvm_vcpu *));
+int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			      unsigned long a0, unsigned long a1,
+			      unsigned long a2, unsigned long a3,
+			      int op_64_bit, int cpl,
+			      int (*complete_hypercall)(struct kvm_vcpu *));
+
+#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall)	\
+({												\
+	int __ret;										\
+												\
+	__ret = ____kvm_emulate_hypercall(_vcpu,						\
+					  kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu),	\
+					  kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu),	\
+					  kvm_##a3##_read(_vcpu), op_64_bit, cpl,		\
+					  complete_hypercall);					\
+												\
+	if (__ret > 0)										\
+		complete_hypercall(_vcpu);							\
+	__ret;											\
+})
 
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
 
maintainer
#25 lore raw

Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro

Quoted 6 lines · Adrian Hunter

For TDX, there is an RFC relating to using descriptively named parameters instead of register names for tdh_vp_enter():

	https://lore.kernel.org/all/[email protected]/
Quoted 7 lines

Please do give some feedback on that approach. Note we need both KVM and x86 maintainer approval for SEAMCALL wrappers like tdh_vp_enter().

As proposed, that ends up with putting the values back into vcpu->arch.regs[] for __kvm_emulate_hypercall() which is not pretty:

If needed we can revert this patch, it's not a big problem.

Paolo

author maintainer
#26 lore raw

Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro

Quoted 11 lines · Paolo Bonzini
Quoted 10 lines · Adrian Hunter

For TDX, there is an RFC relating to using descriptively named parameters instead of register names for tdh_vp_enter():

	https://lore.kernel.org/all/[email protected]/

Please do give some feedback on that approach. Note we need both KVM and x86 maintainer approval for SEAMCALL wrappers like tdh_vp_enter().

Quoted 5 lines
Quoted 3 lines

As proposed, that ends up with putting the values back into vcpu->arch.regs[] for __kvm_emulate_hypercall() which is not pretty:

If needed we can revert this patch, it's not a big problem.

I don't care terribly about the SEAMCALL interfaces. I have opinions on what would I think would be ideal, but I can live with whatever.

What I do deeply care about though is consistency within KVM, across vendors and VM flavors. And that means that guest registers absolutely need to be captured in vcpu->arch.regs[]. TDX already requires too much special cased code in KVM, there is zero reason to make TDX even more different and thus more difficult to maintain.

#27 lore raw

Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro

Quoted 16 lines · Sean Christopherson
Quoted 15 lines · Paolo Bonzini
Quoted 13 lines · Adrian Hunter

For TDX, there is an RFC relating to using descriptively named parameters instead of register names for tdh_vp_enter():

	https://lore.kernel.org/all/[email protected]/

Please do give some feedback on that approach. Note we need both KVM and x86 maintainer approval for SEAMCALL wrappers like tdh_vp_enter().

As proposed, that ends up with putting the values back into vcpu->arch.regs[] for __kvm_emulate_hypercall() which is not pretty:

Quoted 8 lines
Quoted 1 line

If needed we can revert this patch, it's not a big problem.

I don't care terribly about the SEAMCALL interfaces. I have opinions on what would I think would be ideal, but I can live with whatever.

What I do deeply care about though is consistency within KVM, across vendors and VM flavors. And that means that guest registers absolutely need to be captured in vcpu->arch.regs[].

In general, TDX host VMM does not know what guest register values are.

This case, where some GPRs are passed to the host VMM via arguments of the TDG.VP.VMCALL TDCALL, is really just a side effect of the choice of argument passing rather than any attempt to share guest registers with the host VMM.

It could be regarded as more consistent to never use vcpu->arch.regs[] for confidential guests.

maintainer
#28 lore raw

Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro

Quoted 8 lines · Adrian Hunter
Quoted 3 lines · Sean Christopherson

What I do deeply care about though is consistency within KVM, across vendors and VM flavors. And that means that guest registers absolutely need to be captured in vcpu->arch.regs[].

In general, TDX host VMM does not know what guest register values are.

Quoted 7 lines

This case, where some GPRs are passed to the host VMM via arguments of the TDG.VP.VMCALL TDCALL, is really just a side effect of the choice of argument passing rather than any attempt to share guest registers with the host VMM.

It could be regarded as more consistent to never use vcpu->arch.regs[] for confidential guests.

Yes, that's where I stand as well. There's reasons to use vcpu->arch.regs[] when "decrypted" values are available, and reasons to not use it at all. Both of them could be considered the more consistent choice, and I think I prefer slightly the latter, but it's definitely not a hill to die on...

Paolo

author maintainer
#29 lore raw

Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro

In reply to Adrian Hunter
Quoted 26 lines · Adrian Hunter
Quoted 16 lines · Sean Christopherson
Quoted 15 lines · Paolo Bonzini
Quoted 13 lines · Adrian Hunter

For TDX, there is an RFC relating to using descriptively named parameters instead of register names for tdh_vp_enter():

	https://lore.kernel.org/all/[email protected]/

Please do give some feedback on that approach. Note we need both KVM and x86 maintainer approval for SEAMCALL wrappers like tdh_vp_enter().

As proposed, that ends up with putting the values back into vcpu->arch.regs[] for __kvm_emulate_hypercall() which is not pretty:

Quoted 8 lines
Quoted 1 line

If needed we can revert this patch, it's not a big problem.

I don't care terribly about the SEAMCALL interfaces. I have opinions on what would I think would be ideal, but I can live with whatever.

What I do deeply care about though is consistency within KVM, across vendors and VM flavors. And that means that guest registers absolutely need to be captured in vcpu->arch.regs[].

Quoted 8 lines

In general, TDX host VMM does not know what guest register values are.

This case, where some GPRs are passed to the host VMM via arguments of the TDG.VP.VMCALL TDCALL, is really just a side effect of the choice of argument passing rather than any attempt to share guest registers with the host VMM.

It could be regarded as more consistent to never use vcpu->arch.regs[] for confidential guests.

SEV-ES+ marshalls data to/from the GHCB to KVM's register array, because the GHCB spec was intentionally crafted to allow hypervisors to reuse exit-handling code. Granted, that's only for R{A,B,C,D}X and RSI, but the other GPRs should never be used and thus their data is irrelevant.

Which applies to TDX as well. For regs[], it's really only TDVMCALL that I care about, i.e. cases where GPRs hold guest values, versus things like EXIT_QUALIFICATION where the GPR is simply TDX's way of communicating information to the hypervisor.

Argh. The reason I care about putting vCPU state into regs[] is because it helps share code between vendors. Looking at kvm-coco-queue, TDX support wandered in the opposite direction. E.g. TDX rolls its own RDMSR, WRMSR, CPUID, and HYPERCALL implementations, which is quite frustrating. Ditto for things like EXIT_REASON, EXIT_QUALIFICATION, EXIT_INTR_INFO, etc.

For EXIT_REASON in particular, I think maintaining the guest-requested exit reason (via TDMVCALL) in a separate field is a mistake. Readers shouldn't have to care that a HLT exit technically was requested via TDVMCALL. If KVM instead immediately morphs the requested exit reason to KVM's tracked exit_reason, then there's no need to deal with the TDVMCALL layer in flows that don't care. The only danger is a collision with a EXIT_REASON_EPT_MISCONFIG from the TDX module, but that's easy enough to handle.

And even where TDX and VMX have shared some code, IMO it doesn't go far enough. E.g. having vcpu_tdx and vcpu_vmx open code their own version of the posted interrupt fields, just to avoid minimal churn in the VMX code, is beyond gross.

Even concepts like guest_state_loaded are unnecessarily different for TDX. Yes, I get that that host state doesn't need to be reloaded if KVM doesn't actually enter the guest. But holy moly, we're talking about avoid _one_ WRMSR in an extremely rare path (late abort of entry), at the cost of making TDX frustratingly different from VMX.

Making TDX look more like everything else isn't just about code sharing. It's also about providing a familiar setting so that readers who know almost nothing about TDX can find their way around without having to effectively learn an entirely new "architecture" *and* code base.

Hacking around, I think the attached half-baked diff will provide a middle ground for the regs[] vs. struct/union issue. The basic gist is to essentially treat TDX's register ABI as a faster version of VMREAD/VMWRITE, e.g. marshall state to/from the appropriate x86 registers as needed. That way, regs[] holds the correct state and so TDX can reuse much of KVM's existing code verbatim, while allowing the kernel's VP_ENTER API to evolve independently.

From 0319082fc23089f516618e193d94da18c837e35a Mon Sep 17 00:00:00 2001
From: Sean Christopherson <[email protected]>
Date: Thu, 12 Dec 2024 10:22:25 -0800
Subject: [PATCH] Make TDX look more like VMX

---
 arch/x86/kvm/vmx/common.h      |  64 +++++-
 arch/x86/kvm/vmx/nested.c      |   7 +-
 arch/x86/kvm/vmx/posted_intr.h |  11 -
 arch/x86/kvm/vmx/tdx.c         | 375 +++++++++++----------------------
 arch/x86/kvm/vmx/tdx.h         |  11 +-
 arch/x86/kvm/vmx/vmx.c         |  26 +--
 arch/x86/kvm/vmx/vmx.h         |  42 +---
 7 files changed, 201 insertions(+), 335 deletions(-)
arch/x86/kvm/vmx/common.h +60 -4
index 809ced4c6cd8..f1679e53cb4b 100644
@@ -12,6 +12,61 @@
 #include "vmcs.h"
 #include "x86.h"
 
+struct vcpu_vt {
+	/* Posted interrupt descriptor */
+	struct pi_desc pi_desc;
+
+	/* Used if this vCPU is waiting for PI notification wakeup. */
+	struct list_head pi_wakeup_list;
+
+	union vmx_exit_reason exit_reason;
+
+	unsigned long	exit_qualification;
+	u64		ext_qualification;
+	gpa_t		exit_gpa;
+	u32		exit_intr_info;
+	u32		idt_vectoring_info;
+
+
+	/*
+	 * If true, guest state has been loaded into hardware, and host state
+	 * saved into vcpu_{vt,vmx,tdx}.  If false, host state is loaded into
+	 * hardware.
+	 */
+	bool		guest_state_loaded;
+
+#ifdef CONFIG_X86_64
+	u64		msr_host_kernel_gs_base;
+#endif
+};
+
+static __always_inline unsigned long vmx_get_exit_reason(struct kvm_vcpu *vcpu)
+{
+	return to_vt(vcpu)->exit_reason;
+}
+
+static __always_inline unsigned long vmx_get_exit_qual(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_vt *vt = to_vcpu_vt(vcpu);
+
+	if (!kvm_register_test_and_mark_available(vcpu, VCPU_EXREG_EXIT_INFO_1) &&
+	    !WARN_ON_ONCE(is_td_vcpu(vcpu))))
+		vt->exit_qualification = vmcs_readl(EXIT_QUALIFICATION);
+
+	return vt->exit_qualification;
+}
+
+static __always_inline u32 vmx_get_intr_info(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_vt *vt = to_vcpu_vt(vcpu);
+
+	if (!kvm_register_test_and_mark_available(vcpu, VCPU_EXREG_EXIT_INFO_2) &&
+	    !WARN_ON_ONCE(is_td_vcpu(vcpu)))
+		vt->exit_intr_info = vmcs_read32(VM_EXIT_INTR_INFO);
+
+	return vt->exit_intr_info;
+}
+
 extern unsigned long vmx_host_idt_base;
 void vmx_do_interrupt_irqoff(unsigned long entry);
 void vmx_do_nmi_irqoff(void);
@@ -36,9 +91,10 @@ static inline void vmx_handle_nm_fault_irqoff(struct kvm_vcpu *vcpu)
 		rdmsrl(MSR_IA32_XFD_ERR, vcpu->arch.guest_fpu.xfd_err);
 }
 
-static inline void vmx_handle_exception_irqoff(struct kvm_vcpu *vcpu,
-					       u32 intr_info)
+static inline void vmx_handle_exception_irqoff(struct kvm_vcpu *vcpu)
 {
+	u32 intr_info = vmx_get_intr_info(vcpu);
+
 	/* if exit due to PF check for async PF */
 	if (is_page_fault(intr_info))
 		vcpu->arch.apf.host_apf_flags = kvm_read_and_reset_apf_flags();
@@ -50,9 +106,9 @@ static inline void vmx_handle_exception_irqoff(struct kvm_vcpu *vcpu,
 		kvm_machine_check();
 }
 
-static inline void vmx_handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu,
-							u32 intr_info)
+static inline void vmx_handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu)
 {
+	u32 intr_info = vmx_get_intr_info(vcpu);
 	unsigned int vector = intr_info & INTR_INFO_VECTOR_MASK;
 
 	if (KVM_BUG(!is_external_intr(intr_info), vcpu->kvm,
arch/x86/kvm/vmx/nested.c +4 -3
index aa78b6f38dfe..056b6ff1503e 100644
@@ -410,6 +410,7 @@ static void nested_ept_inject_page_fault(struct kvm_vcpu *vcpu,
 {
 	struct vmcs12 *vmcs12 = get_vmcs12(vcpu);
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	struct vcpu_vt *vt = to_vt(vcpu);
 	unsigned long exit_qualification;
 	u32 vm_exit_reason;
 
@@ -425,7 +426,7 @@ static void nested_ept_inject_page_fault(struct kvm_vcpu *vcpu,
 		 * tables also changed, but KVM should not treat EPT Misconfig
 		 * VM-Exits as writes.
 		 */
-		WARN_ON_ONCE(vmx->exit_reason.basic != EXIT_REASON_EPT_VIOLATION);
+		WARN_ON_ONCE(vt->exit_reason.basic != EXIT_REASON_EPT_VIOLATION);
 
 		/*
 		 * PML Full and EPT Violation VM-Exits both use bit 12 to report
@@ -6099,7 +6100,7 @@ static int handle_vmfunc(struct kvm_vcpu *vcpu)
 	 * nested VM-Exit.  Pass the original exit reason, i.e. don't hardcode
 	 * EXIT_REASON_VMFUNC as the exit reason.
 	 */
-	nested_vmx_vmexit(vcpu, vmx->exit_reason.full,
+	nested_vmx_vmexit(vcpu, vmx_get_exit_reason(vcpu).full,
 			  vmx_get_intr_info(vcpu),
 			  vmx_get_exit_qual(vcpu));
 	return 1;
@@ -6544,7 +6545,7 @@ static bool nested_vmx_l1_wants_exit(struct kvm_vcpu *vcpu,
 bool nested_vmx_reflect_vmexit(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
-	union vmx_exit_reason exit_reason = vmx->exit_reason;
+	union vmx_exit_reason exit_reason = vmx_get_exit_reason(vcpu);
 	unsigned long exit_qual;
 	u32 exit_intr_info;
 
arch/x86/kvm/vmx/posted_intr.h +0 -11
index 8b1dccfe4885..9ac4f6eafac5 100644
@@ -5,17 +5,6 @@
 #include <linux/find.h>
 #include <asm/posted_intr.h>
 
-struct vcpu_pi {
-	struct kvm_vcpu	vcpu;
-
-	/* Posted interrupt descriptor */
-	struct pi_desc pi_desc;
-
-	/* Used if this vCPU is waiting for PI notification wakeup. */
-	struct list_head pi_wakeup_list;
-	/* Until here common layout between vcpu_vmx and vcpu_tdx. */
-};
-
 struct pi_desc *vcpu_to_pi_desc(struct kvm_vcpu *vcpu);
 
 void vmx_vcpu_pi_load(struct kvm_vcpu *vcpu, int cpu);
arch/x86/kvm/vmx/tdx.c +121 -254

622 lines not on this page · raw message

arch/x86/kvm/vmx/tdx.h +1 -10
index 0833d1084331..33d316e81a7e 100644
@@ -59,12 +59,7 @@ enum vcpu_tdx_state {
 struct vcpu_tdx {
 	struct kvm_vcpu	vcpu;
 
-	/* Posted interrupt descriptor */
-	struct pi_desc pi_desc;
-
-	/* Used if this vCPU is waiting for PI notification wakeup. */
-	struct list_head pi_wakeup_list;
-	/* Until here same layout to struct vcpu_pi. */
+	struct vcpu_vt vt;
 
 	unsigned long tdvpr_pa;
 	unsigned long *tdcx_pa;
@@ -75,10 +70,6 @@ struct vcpu_tdx {
 
 	enum vcpu_tdx_state state;
 
-	bool host_state_need_save;
-	bool host_state_need_restore;
-	u64 msr_host_kernel_gs_base;
-
 	u64 map_gpa_next;
 	u64 map_gpa_end;
 };
arch/x86/kvm/vmx/vmx.c +13 -13
index 832387bea753..8302e429c82a 100644
@@ -6099,9 +6099,9 @@ void vmx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason,
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
 
-	*reason = vmx->exit_reason.full;
+	*reason = vmx_get_exit_reason(vcpu).full;
 	*info1 = vmx_get_exit_qual(vcpu);
-	if (!(vmx->exit_reason.failed_vmentry)) {
+	if (!(vmx_get_exit_reason(vcpu).failed_vmentry)) {
 		*info2 = vmx->idt_vectoring_info;
 		*intr_info = vmx_get_intr_info(vcpu);
 		if (is_exception_with_error_code(*intr_info))
@@ -6380,7 +6380,7 @@ void dump_vmcs(struct kvm_vcpu *vcpu)
 static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
-	union vmx_exit_reason exit_reason = vmx->exit_reason;
+	union vmx_exit_reason exit_reason = vmx_get_exit_reason(vcpu);
 	u32 vectoring_info = vmx->idt_vectoring_info;
 	u16 exit_handler_index;
 
@@ -6901,11 +6901,10 @@ void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu)
 	if (vmx->emulation_required)
 		return;
 
-	if (vmx->exit_reason.basic == EXIT_REASON_EXTERNAL_INTERRUPT)
-		vmx_handle_external_interrupt_irqoff(vcpu,
-						     vmx_get_intr_info(vcpu));
-	else if (vmx->exit_reason.basic == EXIT_REASON_EXCEPTION_NMI)
-		vmx_handle_exception_irqoff(vcpu, vmx_get_intr_info(vcpu));
+	if (vmx_get_exit_reason(vcpu).basic == EXIT_REASON_EXTERNAL_INTERRUPT)
+		vmx_handle_external_interrupt_irqoff(vcpu);
+	else if (vmx_get_exit_reason(vcpu).basic == EXIT_REASON_EXCEPTION_NMI)
+		vmx_handle_exception_irqoff(vcpu);
 }
 
 /*
@@ -7154,6 +7153,7 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu,
 					unsigned int flags)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	struct vcpu_vt *vt = to_vt(vcpu);
 
 	guest_state_enter_irqoff();
 
@@ -7185,15 +7185,15 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu,
 	vmx_enable_fb_clear(vmx);
 
 	if (unlikely(vmx->fail)) {
-		vmx->exit_reason.full = 0xdead;
+		vt->exit_reason.full = 0xdead;
 		goto out;
 	}
 
-	vmx->exit_reason.full = vmcs_read32(VM_EXIT_REASON);
-	if (likely(!vmx->exit_reason.failed_vmentry))
+	vt->exit_reason.full = vmcs_read32(VM_EXIT_REASON);
+	if (likely(!vt->exit_reason.failed_vmentry))
 		vmx->idt_vectoring_info = vmcs_read32(IDT_VECTORING_INFO_FIELD);
 
-	if ((u16)vmx->exit_reason.basic == EXIT_REASON_EXCEPTION_NMI &&
+	if ((u16)vt->exit_reason.basic == EXIT_REASON_EXCEPTION_NMI &&
 	    is_nmi(vmx_get_intr_info(vcpu)))
 		__vmx_handle_nmi(vcpu);
 
@@ -7331,7 +7331,7 @@ fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit)
 		 * checking.
 		 */
 		if (vmx->nested.nested_run_pending &&
-		    !vmx->exit_reason.failed_vmentry)
+		    !vmx_get_exit_reason(vcpu).failed_vmentry)
 			++vcpu->stat.nested_run;
 
 		vmx->nested.nested_run_pending = 0;
arch/x86/kvm/vmx/vmx.h +2 -40

67 lines not on this page · raw message

base-commit: 14cfaed7621d53af608fd96aa36188064937ca44

Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Reviewed-by on v4 6/6
#30 lore raw

Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro

In reply to Sean Christopherson
Quoted 11 lines · Sean Christopherson

Rework __kvm_emulate_hypercall() into a macro so that completion of hypercalls that don't exit to userspace use direct function calls to the completion helper, i.e. don't trigger a retpoline when RETPOLINE=y.

Opportunistically take the names of the input registers, as opposed to taking the input values, to preemptively dedup more of the calling code (TDX needs to use different registers). Use the direct GPR accessors to read values to avoid the pointless marking of the registers as available (KVM requires GPRs to always be available).

Quoted 99 lines
---
  arch/x86/kvm/x86.c | 29 +++++++++--------------------
  arch/x86/kvm/x86.h | 25 ++++++++++++++++++++-----
  2 files changed, 29 insertions(+), 25 deletions(-)
arch/x86/kvm/x86.c +9 -20
index 39be2a891ab4..fef8b4e63d25 100644
@@ -9982,11 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
  	return kvm_skip_emulated_instruction(vcpu);
  }
 
-int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-			    unsigned long a0, unsigned long a1,
-			    unsigned long a2, unsigned long a3,
-			    int op_64_bit, int cpl,
-			    int (*complete_hypercall)(struct kvm_vcpu *))
+int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			      unsigned long a0, unsigned long a1,
+			      unsigned long a2, unsigned long a3,
+			      int op_64_bit, int cpl,
+			      int (*complete_hypercall)(struct kvm_vcpu *))
  {
  	unsigned long ret;
 
@@ -10073,32 +10073,21 @@ int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 
  out:
  	vcpu->run->hypercall.ret = ret;
-	complete_hypercall(vcpu);
  	return 1;
  }
-EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
+EXPORT_SYMBOL_GPL(____kvm_emulate_hypercall);
 
  int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
  {
-	unsigned long nr, a0, a1, a2, a3;
-	int op_64_bit;
-	int cpl;
-
  	if (kvm_xen_hypercall_enabled(vcpu->kvm))
  		return kvm_xen_hypercall(vcpu);
 
  	if (kvm_hv_hypercall_enabled(vcpu))
  		return kvm_hv_hypercall(vcpu);
 
-	nr = kvm_rax_read(vcpu);
-	a0 = kvm_rbx_read(vcpu);
-	a1 = kvm_rcx_read(vcpu);
-	a2 = kvm_rdx_read(vcpu);
-	a3 = kvm_rsi_read(vcpu);
-	op_64_bit = is_64_bit_hypercall(vcpu);
-	cpl = kvm_x86_call(get_cpl)(vcpu);
-
-	return __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl,
+	return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi,
+				       is_64_bit_hypercall(vcpu),
+				       kvm_x86_call(get_cpl)(vcpu),
  				       complete_hypercall_exit);
  }
  EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);
arch/x86/kvm/x86.h +20 -5
index 28adc8ea04bf..ad6fe6159dea 100644
@@ -617,11 +617,26 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
  	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
  }
 
-int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-			    unsigned long a0, unsigned long a1,
-			    unsigned long a2, unsigned long a3,
-			    int op_64_bit, int cpl,
-			    int (*complete_hypercall)(struct kvm_vcpu *));
+int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			      unsigned long a0, unsigned long a1,
+			      unsigned long a2, unsigned long a3,
+			      int op_64_bit, int cpl,
+			      int (*complete_hypercall)(struct kvm_vcpu *));
+
+#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall)	\
+({												\
+	int __ret;										\
+												\
+	__ret = ____kvm_emulate_hypercall(_vcpu,						\
+					  kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu),	\
+					  kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu),	\
+					  kvm_##a3##_read(_vcpu), op_64_bit, cpl,		\
+					  complete_hypercall);					\
+												\
+	if (__ret > 0)										\
+		complete_hypercall(_vcpu);							\
+	__ret;											\
+})
 
  int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
 
maintainer
#31 lore raw

Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro

In reply to Sean Christopherson
Quoted 96 lines · Sean Christopherson

Rework __kvm_emulate_hypercall() into a macro so that completion of hypercalls that don't exit to userspace use direct function calls to the completion helper, i.e. don't trigger a retpoline when RETPOLINE=y.

Opportunistically take the names of the input registers, as opposed to taking the input values, to preemptively dedup more of the calling code (TDX needs to use different registers). Use the direct GPR accessors to read values to avoid the pointless marking of the registers as available (KVM requires GPRs to always be available).

---
  arch/x86/kvm/x86.c | 29 +++++++++--------------------
  arch/x86/kvm/x86.h | 25 ++++++++++++++++++++-----
  2 files changed, 29 insertions(+), 25 deletions(-)
arch/x86/kvm/x86.c +9 -20
index 39be2a891ab4..fef8b4e63d25 100644
@@ -9982,11 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
  	return kvm_skip_emulated_instruction(vcpu);
  }
 
-int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-			    unsigned long a0, unsigned long a1,
-			    unsigned long a2, unsigned long a3,
-			    int op_64_bit, int cpl,
-			    int (*complete_hypercall)(struct kvm_vcpu *))
+int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			      unsigned long a0, unsigned long a1,
+			      unsigned long a2, unsigned long a3,
+			      int op_64_bit, int cpl,
+			      int (*complete_hypercall)(struct kvm_vcpu *))
  {
  	unsigned long ret;
 
@@ -10073,32 +10073,21 @@ int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
 
  out:
  	vcpu->run->hypercall.ret = ret;
-	complete_hypercall(vcpu);
  	return 1;
  }
-EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall);
+EXPORT_SYMBOL_GPL(____kvm_emulate_hypercall);
 
  int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
  {
-	unsigned long nr, a0, a1, a2, a3;
-	int op_64_bit;
-	int cpl;
-
  	if (kvm_xen_hypercall_enabled(vcpu->kvm))
  		return kvm_xen_hypercall(vcpu);
 
  	if (kvm_hv_hypercall_enabled(vcpu))
  		return kvm_hv_hypercall(vcpu);
 
-	nr = kvm_rax_read(vcpu);
-	a0 = kvm_rbx_read(vcpu);
-	a1 = kvm_rcx_read(vcpu);
-	a2 = kvm_rdx_read(vcpu);
-	a3 = kvm_rsi_read(vcpu);
-	op_64_bit = is_64_bit_hypercall(vcpu);
-	cpl = kvm_x86_call(get_cpl)(vcpu);
-
-	return __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl,
+	return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi,
+				       is_64_bit_hypercall(vcpu),
+				       kvm_x86_call(get_cpl)(vcpu),
  				       complete_hypercall_exit);
  }
  EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);
arch/x86/kvm/x86.h +8 -5
index 28adc8ea04bf..ad6fe6159dea 100644
@@ -617,11 +617,26 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
  	return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
  }
  d -
-int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
-			    unsigned long a0, unsigned long a1,
-			    unsigned long a2, unsigned long a3,
-			    int op_64_bit, int cpl,
-			    int (*complete_hypercall)(struct kvm_vcpu *));
+int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+			      unsigned long a0, unsigned long a1,
+			      unsigned long a2, unsigned long a3,
+			      int op_64_bit, int cpl,
+			      int (*complete_hypercall)(struct kvm_vcpu *));
+
+#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall)	\
+({												\

arch/x86/kvm/x86.h static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) · line 637

Quoted 10 lines
arch/x86/kvm/x86.h +10 -0
+	int __ret;										\
+												\
+	__ret = ____kvm_emulate_hypercall(_vcpu,						\
+					  kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu),	\
+					  kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu),	\
+					  kvm_##a3##_read(_vcpu), op_64_bit, cpl,		\
+					  complete_hypercall);					\
+												\
+	if (__ret > 0)										\
+		complete_hypercall(_vcpu);							\

So based on the review of the previous patch this should become

	__ret = complete_hypercall(_vcpu);

Applied with this change to kvm-coco-queue, thanks.

Paolo

Quoted 5 lines
arch/x86/kvm/x86.h +2 -0
+	__ret;											\
+})
 
  int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
 
author maintainer
#32 lore raw

Re: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro

Quoted 16 lines · Paolo Bonzini

arch/x86/kvm/x86.h static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) · line 637

Quoted 12 lines · Sean Christopherson
arch/x86/kvm/x86.h +12 -0
+#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall)	\
+({												\
+	int __ret;										\
+												\
+	__ret = ____kvm_emulate_hypercall(_vcpu,						\
+					  kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu),	\
+					  kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu),	\
+					  kvm_##a3##_read(_vcpu), op_64_bit, cpl,		\
+					  complete_hypercall);					\
+												\
+	if (__ret > 0)										\
+		complete_hypercall(_vcpu);							\

So based on the review of the previous patch this should become

Quoted 3 lines
	__ret = complete_hypercall(_vcpu);

Applied with this change to kvm-coco-queue, thanks.

I was planning on applying this for 6.14. Should I still do that, or do you want to take the bulk of the series through kvm/next, or maybe let it set in kvm-coco-queue? I can't think of any potential conflicts off the top of my head, and the refactoring is really only useful for TDX.

Patch 1 should go in sooner than later though.

Reviewed-by on the v4 series
#33 lore raw
In reply to Sean Christopherson
Quoted 9 lines · Sean Christopherson

Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX

Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.

Quoted 3 lines

As for abusing vcpu->run->hypercall.ret... It's obviously a bit gross, but I think it's a lesser evil than having multiple a one-line wrappers just to trampoline in the return code.

Doesn't seem to be "gross" to me, and AFAICT now for TDX we just need to play with __kvm_emulate_hypercall() with a TDX-specific completion callback.

Which is nice. Thanks!

For this series:

author maintainer Applied
#34 lore raw
In reply to Sean Christopherson
Quoted 10 lines · Sean Christopherson

Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX

Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.

[...]

Applied patch 1 to kvm-x86 fixes. I'm going to hold off on the rest until the dust settles on the SEAMCALL interfaces, e.g. in case TDX ends up marshalling state into the "normal" GPRs.

[1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit()

      https://github.com/kvm-x86/linux/commit/a317794eefd0

[2/6] KVM: x86: Add a helper to check for user interception of KVM hypercalls
(no commit info)
[3/6] KVM: x86: Move "emulate hypercall" function declarations to x86.h
(no commit info)
[4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall
(no commit info)
[5/6] KVM: x86: Always complete hypercall via function callback
(no commit info)
[6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro
(no commit info)

--
https://github.com/kvm-x86/linux/tree/next

#35 lore raw
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Sean Christopherson

Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX

Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.

Quoted 4 lines
Quoted 1 line

[...]

Applied patch 1 to kvm-x86 fixes. I'm going to hold off on the rest until the dust settles on the SEAMCALL interfaces, e.g. in case TDX ends up marshalling state into the "normal" GPRs.

Hi Sean, Based on your suggestions in the link https://lore.kernel.org/kvm/[email protected], the v2 of "KVM: TDX: TDX hypercalls may exit to userspace" is planned to morph the TDG.VP.VMCALL with KVM hypercall to EXIT_REASON_VMCALL and marshall r10~r14 from vp_enter_args in struct vcpu_tdx to the appropriate x86 registers for KVM hypercall handling.

arch/x86/kvm/vmx/tdx.c +0 -0
index ef66985ddc91..d5aaf66af835 100644
@@ -935,6 +935,23 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit)

        return tdx_exit_handlers_fastpath(vcpu);
 }
+static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
+{
+       tdvmcall_set_return_code(vcpu, vcpu->run->hypercall.ret);
+       return 1;
+}
+
+static int tdx_emulate_vmcall(struct kvm_vcpu *vcpu)
+{
+       kvm_rax_write(vcpu, to_tdx(vcpu)->vp_enter_args.r10);
+       kvm_rbx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r11);
+       kvm_rcx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r12);
+       kvm_rdx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r13);
+       kvm_rsi_write(vcpu, to_tdx(vcpu)->vp_enter_args.r14);
+
+       return __kvm_emulate_hypercall(vcpu, 0, complete_hypercall_exit); +}
+
 static int handle_tdvmcall(struct kvm_vcpu *vcpu)
 {
        switch (tdvmcall_leaf(vcpu)) {
@@ -1286,6 +1303,8 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) return 0;
        case EXIT_REASON_TDCALL:
                return handle_tdvmcall(vcpu);
+       case EXIT_REASON_VMCALL:
+               return tdx_emulate_vmcall(vcpu);
        default:
                break;
        }

To test TDX, I made some modifications to your patch "KVM: x86: Refactor __kvm_emulate_hypercall() into a macro" Are the following changes make sense to you?

arch/x86/kvm/x86.c +0 -0
index a2198807290b..2c5df57ad799 100644
@@ -10088,9 +10088,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)

        if (kvm_hv_hypercall_enabled(vcpu))
                return kvm_hv_hypercall(vcpu);
-       return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi,
-                                      is_64_bit_hypercall(vcpu),
-                                      kvm_x86_call(get_cpl)(vcpu),
+       return __kvm_emulate_hypercall(vcpu, kvm_x86_call(get_cpl)(vcpu), complete_hypercall_exit); }
 EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);

arch/x86/kvm/x86.h +0 -0
index b00ecbfef000..989bed5b48b0 100644
@@ -623,19 +623,18 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,

                              int op_64_bit, int cpl,
                              int (*complete_hypercall)(struct kvm_vcpu *)); -#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \
-({                                                                                             \
-       int __ret;                                                                              \
-                                                                                               \
-       __ret = ____kvm_emulate_hypercall(_vcpu,                                                \
-                                         kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu),       \
-                                         kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu),       \
-                                         kvm_##a3##_read(_vcpu), op_64_bit, cpl,               \
-                                         complete_hypercall);                                  \
-                                                                                               \
-       if (__ret > 0)                                                                          \
-               __ret = complete_hypercall(_vcpu);                                              \
-       __ret;                                                                                  \
+#define __kvm_emulate_hypercall(_vcpu, cpl, complete_hypercall)                                \
+({                                                                                     \
+       int __ret;                                                                      \
+       __ret = ____kvm_emulate_hypercall(_vcpu, kvm_rax_read(_vcpu),                   \
+                                         kvm_rbx_read(_vcpu), kvm_rcx_read(_vcpu),     \
+                                         kvm_rdx_read(_vcpu), kvm_rsi_read(_vcpu),     \
+                                         is_64_bit_hypercall(_vcpu), cpl,              \
+                                         complete_hypercall);                          \
+                                                                                       \
+       if (__ret > 0)                                                                  \
+               __ret = complete_hypercall(_vcpu);                                      \
+       __ret;                                                                          \
 })
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);

author maintainer
#36 lore raw
Quoted 21 lines · Binbin Wu
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Sean Christopherson

Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX

Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.

Quoted 4 lines
Quoted 1 line

[...]

Applied patch 1 to kvm-x86 fixes. I'm going to hold off on the rest until the dust settles on the SEAMCALL interfaces, e.g. in case TDX ends up marshalling state into the "normal" GPRs.

Hi Sean, Based on your suggestions in the link
https://lore.kernel.org/kvm/[email protected], the v2 of "KVM: TDX:
TDX hypercalls may exit to userspace" is planned to morph the TDG.VP.VMCALL with KVM hypercall to EXIT_REASON_VMCALL and marshall r10~r14 from vp_enter_args in struct vcpu_tdx to the appropriate x86 registers for KVM hypercall handling.

...

Quoted 3 lines

To test TDX, I made some modifications to your patch "KVM: x86: Refactor __kvm_emulate_hypercall() into a macro" Are the following changes make sense to you?

Yes, but I think we can go a step further and effectively revert the bulk of commit e913ef159fad ("KVM: x86: Split core of hypercall emulation to helper function"), i.e. have ____kvm_emulate_hypercall() read the GPRs instead of passing them in via the macro.

Quoted 50 lines
arch/x86/kvm/x86.c +1 -3
index a2198807290b..2c5df57ad799 100644
@@ -10088,9 +10088,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
         if (kvm_hv_hypercall_enabled(vcpu))
                 return kvm_hv_hypercall(vcpu);
-       return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi,
-                                      is_64_bit_hypercall(vcpu),
-                                      kvm_x86_call(get_cpl)(vcpu),
+       return __kvm_emulate_hypercall(vcpu, kvm_x86_call(get_cpl)(vcpu),
                                        complete_hypercall_exit);
  }
  EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);
arch/x86/kvm/x86.h +12 -13
index b00ecbfef000..989bed5b48b0 100644
@@ -623,19 +623,18 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
                               int op_64_bit, int cpl,
                               int (*complete_hypercall)(struct kvm_vcpu *));
-#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \
-({                                                                                             \
-       int __ret;                                                                              \
-                                                                                               \
-       __ret = ____kvm_emulate_hypercall(_vcpu,                                                \
-                                         kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu),       \
-                                         kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu),       \
-                                         kvm_##a3##_read(_vcpu), op_64_bit, cpl,               \
-                                         complete_hypercall);                                  \
-                                                                                               \
-       if (__ret > 0)                                                                          \
-               __ret = complete_hypercall(_vcpu);                                              \
-       __ret;                                                                                  \
+#define __kvm_emulate_hypercall(_vcpu, cpl, complete_hypercall)                                \
+({                                                                                     \
+       int __ret;                                                                      \
+       __ret = ____kvm_emulate_hypercall(_vcpu, kvm_rax_read(_vcpu),                   \
+                                         kvm_rbx_read(_vcpu), kvm_rcx_read(_vcpu),     \
+                                         kvm_rdx_read(_vcpu), kvm_rsi_read(_vcpu),     \
+                                         is_64_bit_hypercall(_vcpu), cpl,              \
+                                         complete_hypercall);                          \
+                                                                                       \
+       if (__ret > 0)                                                                  \
+               __ret = complete_hypercall(_vcpu);                                      \
+       __ret;                                                                          \
  })
  int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
 
 
#37 lore raw
Quoted 24 lines · Sean Christopherson
Quoted 21 lines · Binbin Wu
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Sean Christopherson

Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX

Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.

Quoted 4 lines
Quoted 1 line

[...]

Applied patch 1 to kvm-x86 fixes. I'm going to hold off on the rest until the dust settles on the SEAMCALL interfaces, e.g. in case TDX ends up marshalling state into the "normal" GPRs.

Hi Sean, Based on your suggestions in the link
https://lore.kernel.org/kvm/[email protected], the v2 of "KVM: TDX:
TDX hypercalls may exit to userspace" is planned to morph the TDG.VP.VMCALL with KVM hypercall to EXIT_REASON_VMCALL and marshall r10~r14 from vp_enter_args in struct vcpu_tdx to the appropriate x86 registers for KVM hypercall handling.

...

Quoted 7 lines
Quoted 3 lines

To test TDX, I made some modifications to your patch "KVM: x86: Refactor __kvm_emulate_hypercall() into a macro" Are the following changes make sense to you?

Yes, but I think we can go a step further and effectively revert the bulk of commit e913ef159fad ("KVM: x86: Split core of hypercall emulation to helper function"), i.e. have ____kvm_emulate_hypercall() read the GPRs instead of passing them in via the macro.

Sure.

Are you OK if I sent the change (as a prep patch) along with v2 of "TDX hypercalls may exit to userspace"?

Quoted 51 lines
Quoted 51 lines
arch/x86/kvm/x86.c +1 -3
index a2198807290b..2c5df57ad799 100644
@@ -10088,9 +10088,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
         if (kvm_hv_hypercall_enabled(vcpu))
                 return kvm_hv_hypercall(vcpu);
-       return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi,
-                                      is_64_bit_hypercall(vcpu),
-                                      kvm_x86_call(get_cpl)(vcpu),
+       return __kvm_emulate_hypercall(vcpu, kvm_x86_call(get_cpl)(vcpu),
                                        complete_hypercall_exit);
  }
  EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);
arch/x86/kvm/x86.h +12 -13
index b00ecbfef000..989bed5b48b0 100644
@@ -623,19 +623,18 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
                               int op_64_bit, int cpl,
                               int (*complete_hypercall)(struct kvm_vcpu *));
-#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \
-({                                                                                             \
-       int __ret;                                                                              \
-                                                                                               \
-       __ret = ____kvm_emulate_hypercall(_vcpu,                                                \
-                                         kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu),       \
-                                         kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu),       \
-                                         kvm_##a3##_read(_vcpu), op_64_bit, cpl,               \
-                                         complete_hypercall);                                  \
-                                                                                               \
-       if (__ret > 0)                                                                          \
-               __ret = complete_hypercall(_vcpu);                                              \
-       __ret;                                                                                  \
+#define __kvm_emulate_hypercall(_vcpu, cpl, complete_hypercall)                                \
+({                                                                                     \
+       int __ret;                                                                      \
+       __ret = ____kvm_emulate_hypercall(_vcpu, kvm_rax_read(_vcpu),                   \
+                                         kvm_rbx_read(_vcpu), kvm_rcx_read(_vcpu),     \
+                                         kvm_rdx_read(_vcpu), kvm_rsi_read(_vcpu),     \
+                                         is_64_bit_hypercall(_vcpu), cpl,              \
+                                         complete_hypercall);                          \
+                                                                                       \
+       if (__ret > 0)                                                                  \
+               __ret = complete_hypercall(_vcpu);                                      \
+       __ret;                                                                          \
  })
  int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
 
 
author maintainer
#38 lore raw
Quoted 33 lines · Binbin Wu
Quoted 24 lines · Sean Christopherson
Quoted 21 lines · Binbin Wu
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Sean Christopherson

Effectively v4 of Binbin's series to handle hypercall exits to userspace in a generic manner, so that TDX

Binbin and Kai, this is fairly different that what we last discussed. While sorting through Binbin's latest patch, I stumbled on what I think/hope is an approach that will make life easier for TDX. Rather than have common code set the return value, _and_ have TDX implement a callback to do the same for user return MSRs, just use the callback for all paths.

Quoted 4 lines
Quoted 1 line

[...]

Applied patch 1 to kvm-x86 fixes. I'm going to hold off on the rest until the dust settles on the SEAMCALL interfaces, e.g. in case TDX ends up marshalling state into the "normal" GPRs.

Hi Sean, Based on your suggestions in the link
https://lore.kernel.org/kvm/[email protected], the v2 of "KVM: TDX:
TDX hypercalls may exit to userspace" is planned to morph the TDG.VP.VMCALL with KVM hypercall to EXIT_REASON_VMCALL and marshall r10~r14 from vp_enter_args in struct vcpu_tdx to the appropriate x86 registers for KVM hypercall handling.

...

Quoted 7 lines
Quoted 3 lines

To test TDX, I made some modifications to your patch "KVM: x86: Refactor __kvm_emulate_hypercall() into a macro" Are the following changes make sense to you?

Yes, but I think we can go a step further and effectively revert the bulk of commit e913ef159fad ("KVM: x86: Split core of hypercall emulation to helper function"), i.e. have ____kvm_emulate_hypercall() read the GPRs instead of passing them in via the macro.

Quoted 4 lines

Sure.

Are you OK if I sent the change (as a prep patch) along with v2 of "TDX hypercalls may exit to userspace"?

Ya, go for it.