KVM: arm64: Userspace SMCCC call filtering

In anticipation of user hypercall filters, add the necessary plumbing to get SMCCC calls out to userspace. Even though the exit structure has space for KVM to pass register arguments, let's just avoid it altogether and let userspace poke at the registers via KVM_GET_ONE_REG.

This deliberately stretches the definition of a 'hypercall' to cover SMCs from EL1 in addition to the HVCs we know and love. KVM doesn't support EL1 calls into secure services, but now we can paint that as a userspace problem and be done with it.

Finally, we need a flag to let userspace know what conduit instruction was used (i.e. SMC vs. HVC).

Signed-off-by Oliver Upton <[email protected]>

---
 Documentation/virt/kvm/api.rst    | 18 ++++++++++++++++--
 arch/arm64/include/uapi/asm/kvm.h |  4 ++++
 arch/arm64/kvm/handle_exit.c      |  4 +++-
 arch/arm64/kvm/hypercalls.c       | 16 ++++++++++++++++
 4 files changed, 39 insertions(+), 3 deletions(-)

Documentation/virt/kvm/api.rst +16 -2

index 9b01e3d0e757..9497792c4ee5 100644
@@ -6221,11 +6221,25 @@ to the byte array.
 			__u64 flags;
 		} hypercall;
 
-Unused.  This was once used for 'hypercall to userspace'.  To implement
-such functionality, use KVM_EXIT_IO (x86) or KVM_EXIT_MMIO (all except s390).
+
+It is strongly recommended that userspace use ``KVM_EXIT_IO`` (x86) or
+``KVM_EXIT_MMIO`` (all except s390) to implement functionality that
+requires a guest to interact with host userpace.
 
 .. note:: KVM_EXIT_IO is significantly faster than KVM_EXIT_MMIO.
 
+For arm64:
+----------
+
+``nr`` contains the function ID of the guest's SMCCC call. Userspace is
+expected to use the ``KVM_GET_ONE_REG`` ioctl to retrieve the call
+parameters from the vCPU's GPRs.
+
+Definition of ``flags``:
+ - ``KVM_HYPERCALL_EXIT_SMC``: Indicates that the guest used the SMC
+   conduit to initiate the SMCCC call. If this bit is 0 then the guest
+   used the HVC conduit for the SMCCC call.
+
 ::
 
 		/* KVM_EXIT_TPR_ACCESS */

arch/arm64/include/uapi/asm/kvm.h +4 -0

index f9672ef1159a..f86446c5a7e3 100644
@@ -472,12 +472,16 @@ enum {
 enum kvm_smccc_filter_action {
 	KVM_SMCCC_FILTER_HANDLE = 0,
 	KVM_SMCCC_FILTER_DENY,
+	KVM_SMCCC_FILTER_FWD_TO_USER,
 
 #ifdef __KERNEL__
 	NR_SMCCC_FILTER_ACTIONS
 #endif
 };
 
+/* arm64-specific KVM_EXIT_HYPERCALL flags */
+#define KVM_HYPERCALL_EXIT_SMC	(1U << 0)
+
 #endif
 
 #endif /* __ARM_KVM_H__ */

arch/arm64/kvm/handle_exit.c +0 -0

index 68f95dcd41a1..3f43e20c48b6 100644

Quoted 80 lines · Oliver Upton

In anticipation of user hypercall filters, add the necessary plumbing to get SMCCC calls out to userspace. Even though the exit structure has space for KVM to pass register arguments, let's just avoid it altogether and let userspace poke at the registers via KVM_GET_ONE_REG.

This deliberately stretches the definition of a 'hypercall' to cover SMCs from EL1 in addition to the HVCs we know and love. KVM doesn't support EL1 calls into secure services, but now we can paint that as a userspace problem and be done with it.

Finally, we need a flag to let userspace know what conduit instruction was used (i.e. SMC vs. HVC).

Signed-off-by Oliver Upton <[email protected]>

---
 Documentation/virt/kvm/api.rst    | 18 ++++++++++++++++--
 arch/arm64/include/uapi/asm/kvm.h |  4 ++++
 arch/arm64/kvm/handle_exit.c      |  4 +++-
 arch/arm64/kvm/hypercalls.c       | 16 ++++++++++++++++
 4 files changed, 39 insertions(+), 3 deletions(-)

Documentation/virt/kvm/api.rst +16 -2

index 9b01e3d0e757..9497792c4ee5 100644
@@ -6221,11 +6221,25 @@ to the byte array.
 			__u64 flags;
 		} hypercall;
 
-Unused.  This was once used for 'hypercall to userspace'.  To implement
-such functionality, use KVM_EXIT_IO (x86) or KVM_EXIT_MMIO (all except s390).
+
+It is strongly recommended that userspace use ``KVM_EXIT_IO`` (x86) or
+``KVM_EXIT_MMIO`` (all except s390) to implement functionality that
+requires a guest to interact with host userpace.
 
 .. note:: KVM_EXIT_IO is significantly faster than KVM_EXIT_MMIO.
 
+For arm64:
+----------
+
+``nr`` contains the function ID of the guest's SMCCC call. Userspace is
+expected to use the ``KVM_GET_ONE_REG`` ioctl to retrieve the call
+parameters from the vCPU's GPRs.
+
+Definition of ``flags``:
+ - ``KVM_HYPERCALL_EXIT_SMC``: Indicates that the guest used the SMC
+   conduit to initiate the SMCCC call. If this bit is 0 then the guest
+   used the HVC conduit for the SMCCC call.
+
 ::
 
 		/* KVM_EXIT_TPR_ACCESS */

arch/arm64/include/uapi/asm/kvm.h +4 -0

index f9672ef1159a..f86446c5a7e3 100644
@@ -472,12 +472,16 @@ enum {
 enum kvm_smccc_filter_action {
 	KVM_SMCCC_FILTER_HANDLE = 0,
 	KVM_SMCCC_FILTER_DENY,
+	KVM_SMCCC_FILTER_FWD_TO_USER,
 
 #ifdef __KERNEL__
 	NR_SMCCC_FILTER_ACTIONS
 #endif
 };
 
+/* arm64-specific KVM_EXIT_HYPERCALL flags */
+#define KVM_HYPERCALL_EXIT_SMC	(1U << 0)
+
 #endif
 
 #endif /* __ARM_KVM_H__ */

arch/arm64/kvm/handle_exit.c +0 -0

index 68f95dcd41a1..3f43e20c48b6 100644

arch/arm64/kvm/handle_exit.c static int handle_smc(struct kvm_vcpu *vcpu) · line 76

Quoted 7 lines

arch/arm64/kvm/handle_exit.c +3 -1

@@ -71,7 +71,9 @@ static int handle_smc(struct kvm_vcpu *vcpu)
 	 * Trap exception, not a Secure Monitor Call exception [...]"
 	 *
 	 * We need to advance the PC after the trap, as it would
-	 * otherwise return to the same address...
+	 * otherwise return to the same address. Furthermore, pre-incrementing
+	 * the PC before potentially exiting to userspace maintains the same
+	 * abstraction for both SMCs and HVCs.

nit: this comment really needs to find its way in the documentation so that a VMM author can determine the PC of the SMC/HVC. This is specially important for 32bit, which has a 16bit encodings for SMC/HVC.

And thinking of it, this outlines a small flaw in this API. If luserspace needs to find out about the address of the HVC/SMC, it needs to know the *size* of the instruction. But we don't propagate the ESR value. I think this still works by construction (userspace can check PSTATE and work out whether we're in ARM or Thumb mode), but this feels fragile.

Should we expose the ESR, or at least ESR_EL2.IL as an additional flag?

Quoted 25 lines

arch/arm64/kvm/handle_exit.c static int handle_smc(struct kvm_vcpu *vcpu) · line 76

Quoted 12 lines

arch/arm64/kvm/handle_exit.c +3 -1

index 68f95dcd41a1..3f43e20c48b6 100644
@@ -71,7 +71,9 @@ static int handle_smc(struct kvm_vcpu *vcpu)
 	 * Trap exception, not a Secure Monitor Call exception [...]"
 	 *
 	 * We need to advance the PC after the trap, as it would
-	 * otherwise return to the same address...
+	 * otherwise return to the same address. Furthermore, pre-incrementing
+	 * the PC before potentially exiting to userspace maintains the same
+	 * abstraction for both SMCs and HVCs.

nit: this comment really needs to find its way in the documentation so that a VMM author can determine the PC of the SMC/HVC. This is specially important for 32bit, which has a 16bit encodings for SMC/HVC.

And thinking of it, this outlines a small flaw in this API. If luserspace needs to find out about the address of the HVC/SMC, it needs to know the *size* of the instruction. But we don't propagate the ESR value. I think this still works by construction (userspace can check PSTATE and work out whether we're in ARM or Thumb mode), but this feels fragile.

Quoted 2 lines

Should we expose the ESR, or at least ESR_EL2.IL as an additional flag?

Just to make this a quicker round trip, I hacked the following together. If you agree with it, I'll stick it on top and get the ball rolling.

From 9b830e7a3819c2771074bebe66c1d5f20394e3cc Mon Sep 17 00:00:00 2001
From: Marc Zyngier <[email protected]>
Date: Wed, 5 Apr 2023 12:48:58 +0100
Subject: [PATCH] KVM: arm64: Expose SMC/HVC width to userspace

When returning to userspace to handle a SMCCC call, we consistently set PC to point to the instruction immediately after the HVC/SMC.

However, should userspace need to know the exact address of the trapping instruction, it needs to know about the *size* of that instruction. For AArch64, this is pretty easy. For AArch32, this is a bit more funky, as Thumb has 16bit encodings for both HVC and SMC.

Expose this to userspace with a new flag that directly derives from ESR_EL2.IL. Also update the documentation to reflect the PC state at the point of exit.

Finally, this fixes a small buglet where the hypercall.{args,ret} fields would not be cleared on exit, and could contain some random junk.

Signed-off-by Marc Zyngier <[email protected]>

---
 Documentation/virt/kvm/api.rst    |  8 ++++++++
 arch/arm64/include/uapi/asm/kvm.h |  3 ++-
 arch/arm64/kvm/hypercalls.c       | 16 +++++++++++-----
 3 files changed, 21 insertions(+), 6 deletions(-)

Documentation/virt/kvm/api.rst +8 -0

index c8ab2f730945..103f945959ed 100644
@@ -6244,6 +6244,14 @@ Definition of ``flags``:
    conduit to initiate the SMCCC call. If this bit is 0 then the guest
    used the HVC conduit for the SMCCC call.
 
+ - ``KVM_HYPERCALL_EXIT_16BIT``: Indicates that the guest used a 16bit
+   instruction to initiate the SMCCC call. If this bit is 0 then the
+   guest used a 32bit instruction. An AArch64 guest always has this
+   bit set to 0.
+
+At the point of exit, PC points to the instruction immediately following
+the trapping instruction.
+
 ::
 
 		/* KVM_EXIT_TPR_ACCESS */

arch/arm64/include/uapi/asm/kvm.h +2 -1

index 3dcfa4bfdf83..b1c1edf85480 100644
@@ -491,7 +491,8 @@ struct kvm_smccc_filter {
 };
 
 /* arm64-specific KVM_EXIT_HYPERCALL flags */
-#define KVM_HYPERCALL_EXIT_SMC	(1U << 0)
+#define KVM_HYPERCALL_EXIT_SMC		(1U << 0)
+#define KVM_HYPERCALL_EXIT_16BIT	(1U << 1)
 
 #endif
 

arch/arm64/kvm/hypercalls.c +11 -5

index 9a35d6d18193..3b6523f25afc 100644
@@ -222,13 +222,19 @@ static void kvm_prepare_hypercall_exit(struct kvm_vcpu *vcpu, u32 func_id)
 {
 	u8 ec = ESR_ELx_EC(kvm_vcpu_get_esr(vcpu));
 	struct kvm_run *run = vcpu->run;
-
-	run->exit_reason = KVM_EXIT_HYPERCALL;
-	run->hypercall.nr = func_id;
-	run->hypercall.flags = 0;
+	u64 flags = 0;
 
 	if (ec == ESR_ELx_EC_SMC32 || ec == ESR_ELx_EC_SMC64)
-		run->hypercall.flags |= KVM_HYPERCALL_EXIT_SMC;
+		flags |= KVM_HYPERCALL_EXIT_SMC;
+
+	if (!kvm_vcpu_trap_il_is32bit(vcpu))
+		flags |= KVM_HYPERCALL_EXIT_16BIT;
+
+	run->exit_reason = KVM_EXIT_HYPERCALL;
+	run->hypercall = (typeof(run->hypercall)) {
+		.nr	= func_id,
+		.flags	= flags,
+	};
 }
 
 int kvm_smccc_call_handler(struct kvm_vcpu *vcpu)

Signature · 5 lines

2.34.1


-- 
Without deviation from the norm, progress is not possible.

From: Marc Zyngier <[email protected]>
Sent: Wednesday, April 5, 2023 12:59 PM
To: Oliver Upton <[email protected]>
Cc: [email protected]; [email protected]; Paolo Bonzini <[email protected]>; James Morse <[email protected]>; Suzuki K Poulose <[email protected]>; yuzenghui <[email protected]>; Sean Christopherson <[email protected]>; Salil Mehta <[email protected]>
Subject: Re: [PATCH v3 08/13] KVM: arm64: Add support for KVM_EXIT_HYPERCALL

Quoted 105 lines · Marc Zyngier

Quoted 80 lines · Oliver Upton

In anticipation of user hypercall filters, add the necessary plumbing to get SMCCC calls out to userspace. Even though the exit structure has space for KVM to pass register arguments, let's just avoid it altogether and let userspace poke at the registers via KVM_GET_ONE_REG.

This deliberately stretches the definition of a 'hypercall' to cover SMCs from EL1 in addition to the HVCs we know and love. KVM doesn't support EL1 calls into secure services, but now we can paint that as a userspace problem and be done with it.

Finally, we need a flag to let userspace know what conduit instruction was used (i.e. SMC vs. HVC).

Signed-off-by Oliver Upton <[email protected]>

---
 Documentation/virt/kvm/api.rst    | 18 ++++++++++++++++--
 arch/arm64/include/uapi/asm/kvm.h |  4 ++++
 arch/arm64/kvm/handle_exit.c      |  4 +++-
 arch/arm64/kvm/hypercalls.c       | 16 ++++++++++++++++
 4 files changed, 39 insertions(+), 3 deletions(-)

Documentation/virt/kvm/api.rst +16 -2

index 9b01e3d0e757..9497792c4ee5 100644
@@ -6221,11 +6221,25 @@ to the byte array.
 			__u64 flags;
 		} hypercall;
 
-Unused.  This was once used for 'hypercall to userspace'.  To  implement
-such functionality, use KVM_EXIT_IO (x86) or KVM_EXIT_MMIO (all except s390).
+
+It is strongly recommended that userspace use ``KVM_EXIT_IO`` (x86) or
+``KVM_EXIT_MMIO`` (all except s390) to implement functionality that
+requires a guest to interact with host userpace.
 
 .. note:: KVM_EXIT_IO is significantly faster than KVM_EXIT_MMIO.
 
+For arm64:
+----------
+
+``nr`` contains the function ID of the guest's SMCCC call. Userspace is
+expected to use the ``KVM_GET_ONE_REG`` ioctl to retrieve the call
+parameters from the vCPU's GPRs.
+
+Definition of ``flags``:
+ - ``KVM_HYPERCALL_EXIT_SMC``: Indicates that the guest used the SMC
+   conduit to initiate the SMCCC call. If this bit is 0 then the guest
+   used the HVC conduit for the SMCCC call.
+
 ::
 
 		/* KVM_EXIT_TPR_ACCESS */

arch/arm64/include/uapi/asm/kvm.h +4 -0

index f9672ef1159a..f86446c5a7e3 100644
@@ -472,12 +472,16 @@ enum {
 enum kvm_smccc_filter_action {
 	KVM_SMCCC_FILTER_HANDLE = 0,
 	KVM_SMCCC_FILTER_DENY,
+	KVM_SMCCC_FILTER_FWD_TO_USER,
 
 #ifdef __KERNEL__
 	NR_SMCCC_FILTER_ACTIONS
 #endif
 };
 
+/* arm64-specific KVM_EXIT_HYPERCALL flags */
+#define KVM_HYPERCALL_EXIT_SMC	(1U << 0)
+
 #endif
 
 #endif /* __ARM_KVM_H__ */

arch/arm64/kvm/handle_exit.c +0 -0

index 68f95dcd41a1..3f43e20c48b6 100644

arch/arm64/kvm/handle_exit.c static int handle_smc(struct kvm_vcpu *vcpu) · line 76

Quoted 7 lines

arch/arm64/kvm/handle_exit.c +3 -1

@@ -71,7 +71,9 @@ static int handle_smc(struct kvm_vcpu *vcpu)
 	 * Trap exception, not a Secure Monitor Call exception [...]"
 	 *
 	 * We need to advance the PC after the trap, as it would
-	 * otherwise return to the same address...
+	 * otherwise return to the same address. Furthermore, pre-incrementing
+	 * the PC before potentially exiting to userspace maintains the same
+	 * abstraction for both SMCs and HVCs.

nit: this comment really needs to find its way in the documentation so that a VMM author can determine the PC of the SMC/HVC. This is specially important for 32bit, which has a 16bit encodings for SMC/HVC.

And thinking of it, this outlines a small flaw in this API. If luserspace needs to find out about the address of the HVC/SMC, it needs to know the *size* of the instruction. But we don't propagate the ESR value. I think this still works by construction (userspace can check PSTATE and work out whether we're in ARM or Thumb mode), but this feels fragile.

Should we expose the ESR, or at least ESR_EL2.IL as an additional flag?

Just to make this a quicker round trip, I hacked the following together. If you agree with it, I'll stick it on top and get the ball rolling.

Thanks,

	M.

From 9b830e7a3819c2771074bebe66c1d5f20394e3cc Mon Sep 17 00:00:00 2001
From: Marc Zyngier <[email protected]>
Date: Wed, 5 Apr 2023 12:48:58 +0100
Subject: [PATCH] KVM: arm64: Expose SMC/HVC width to userspace

When returning to userspace to handle a SMCCC call, we consistently set PC to point to the instruction immediately after the HVC/SMC.

However, should userspace need to know the exact address of the trapping instruction, it needs to know about the *size* of that instruction. For AArch64, this is pretty easy. For AArch32, this is a bit more funky, as Thumb has 16bit encodings for both HVC and SMC.

Expose this to userspace with a new flag that directly derives from ESR_EL2.IL. Also update the documentation to reflect the PC state at the point of exit.

Finally, this fixes a small buglet where the hypercall.{args,ret} fields would not be cleared on exit, and could contain some random junk.

Signed-off-by Marc Zyngier <[email protected]>

---
 Documentation/virt/kvm/api.rst    |  8 ++++++++
 arch/arm64/include/uapi/asm/kvm.h |  3 ++-
 arch/arm64/kvm/hypercalls.c       | 16 +++++++++++-----
 3 files changed, 21 insertions(+), 6 deletions(-)

Documentation/virt/kvm/api.rst +8 -0

index c8ab2f730945..103f945959ed 100644
@@ -6244,6 +6244,14 @@ Definition of ``flags``:
    conduit to initiate the SMCCC call. If this bit is 0 then the guest
    used the HVC conduit for the SMCCC call.
 
+ - ``KVM_HYPERCALL_EXIT_16BIT``: Indicates that the guest used a 16bit
+   instruction to initiate the SMCCC call. If this bit is 0 then the
+   guest used a 32bit instruction. An AArch64 guest always has this
+   bit set to 0.
+
+At the point of exit, PC points to the instruction immediately following
+the trapping instruction.
+
 ::
 
 		/* KVM_EXIT_TPR_ACCESS */

arch/arm64/include/uapi/asm/kvm.h +2 -1

index 3dcfa4bfdf83..b1c1edf85480 100644
@@ -491,7 +491,8 @@ struct kvm_smccc_filter {
 };
 
 /* arm64-specific KVM_EXIT_HYPERCALL flags */
-#define KVM_HYPERCALL_EXIT_SMC	(1U << 0)
+#define KVM_HYPERCALL_EXIT_SMC		(1U << 0)
+#define KVM_HYPERCALL_EXIT_16BIT	(1U << 1)
 
 #endif
 

arch/arm64/kvm/hypercalls.c +1 -4

index 9a35d6d18193..3b6523f25afc 100644
@@ -222,13 +222,19 @@ static void kvm_prepare_hypercall_exit(struct
 kvm_vcpu *vcpu, u32 func_id)
 {
 	u8 ec = ESR_ELx_EC(kvm_vcpu_get_esr(vcpu));
 	struct kvm_run *run = vcpu->run;
-
-	run->exit_reason = KVM_EXIT_HYPERCALL;
-	run->hypercall.nr = func_id;
-	run->hypercall.flags = 0;
+	u64 flags = 0;
 

Quoted 2 lines

Quoted 2 lines

Should we expose the ESR, or at least ESR_EL2.IL as an additional flag?

I think we would need "Immediate value" of the ESR_EL2 register in the user-space/VMM to be able to construct the syndrome value. I cannot see where it is being sent?

Hi Salil,

On Wed, May 17, 2023 at 06:00:18PM +0000, Salil Mehta wrote:

[...]

Quoted 7 lines

Quoted 2 lines

Quoted 2 lines

Should we expose the ESR, or at least ESR_EL2.IL as an additional flag?

I think we would need "Immediate value" of the ESR_EL2 register in the user-space/VMM to be able to construct the syndrome value. I cannot see where it is being sent?

The immediate value is not exposed to userspace, although by definition the immediate value must be zero. The SMCCC spec requires all compliant calls to use an immediate of zero (DEN0028E 2.9).

Is there a legitimate use case for hypercalls with a nonzero immediate? They would no longer be considered SMCCC calls at that point, so they wouldn't work with the new UAPI.

From: Marc Zyngier <[email protected]>
Sent: Thursday, May 18, 2023 9:06 AM
To: Oliver Upton <[email protected]>
Cc: Salil Mehta <[email protected]>; [email protected]; [email protected]; Paolo Bonzini <[email protected]>; James Morse <[email protected]>; Suzuki K Poulose <[email protected]>; yuzenghui <[email protected]>; Sean Christopherson <[email protected]>
Subject: Re: [PATCH v3 08/13] KVM: arm64: Add support for KVM_EXIT_HYPERCALL

Quoted 19 lines · Oliver Upton

Hi Salil,

On Wed, May 17, 2023 at 06:00:18PM +0000, Salil Mehta wrote:

[...]

Quoted 7 lines

Quoted 2 lines

Quoted 2 lines

Should we expose the ESR, or at least ESR_EL2.IL as an additional flag?

I think we would need "Immediate value" of the ESR_EL2 register in the user-space/VMM to be able to construct the syndrome value. I cannot see where it is being sent?

The immediate value is not exposed to userspace, although by definition the immediate value must be zero. The SMCCC spec requires all compliant calls to use an immediate of zero (DEN0028E 2.9).

Quoted 3 lines

Is there a legitimate use case for hypercalls with a nonzero immediate? They would no longer be considered SMCCC calls at that point, so they wouldn't work with the new UAPI.

I agree. The use of non-zero immediate has long been deprecated. I guess we should actually reject non-zero immediate for HVC just like we do for SMC.

If there is an actual need for a non-zero immediate to be propagated to userspace (want to emulate Xen's infamous 'HVC #0xEA1'?), then this should be an extension to the current API.

Hi Marc,

Quoted 35 lines

From: Marc Zyngier <[email protected]>
Sent: Thursday, May 18, 2023 9:06 AM
To: Oliver Upton <[email protected]>
Cc: Salil Mehta <[email protected]>; [email protected]; [email protected]; Paolo Bonzini <[email protected]>; James Morse <[email protected]>; Suzuki K Poulose <[email protected]>; yuzenghui <[email protected]>; Sean Christopherson <[email protected]>
Subject: Re: [PATCH v3 08/13] KVM: arm64: Add support for KVM_EXIT_HYPERCALL

Quoted 22 lines · Oliver Upton

Hi Salil,

On Wed, May 17, 2023 at 06:00:18PM +0000, Salil Mehta wrote:

[...]

Quoted 7 lines

Quoted 2 lines

Quoted 2 lines

Should we expose the ESR, or at least ESR_EL2.IL as an additional flag?

I think we would need "Immediate value" of the ESR_EL2 register in the user-space/VMM to be able to construct the syndrome value. I cannot see where it is being sent?

The immediate value is not exposed to userspace, although by definition the immediate value must be zero. The SMCCC spec requires all compliant calls to use an immediate of zero (DEN0028E 2.9).

Is there a legitimate use case for hypercalls with a nonzero immediate? They would no longer be considered SMCCC calls at that point, so they wouldn't work with the new UAPI.

Quoted 3 lines

I agree. The use of non-zero immediate has long been deprecated. I guess we should actually reject non-zero immediate for HVC just like we do for SMC.

Ok. Maybe I will hard code Immediate value as 0 to create a syndrome value at the VMM/Qemu and will also put a note stating non-zero immediate for HVC/SVC are not supported/deprecated.

Quoted 3 lines

If there is an actual need for a non-zero immediate to be propagated to userspace (want to emulate Xen's infamous 'HVC #0xEA1'?), then this should be an extension to the current API.

Oh ok, then perhaps this new extension change should be simultaneously committed to avoid breaking Xen?

From: Marc Zyngier <[email protected]>
Sent: Thursday, May 18, 2023 10:43 AM
To: Salil Mehta <[email protected]>
Cc: Oliver Upton <[email protected]>; [email protected]; [email protected]; Paolo Bonzini <[email protected]>; James Morse <[email protected]>; Suzuki K Poulose <[email protected]>; yuzenghui <[email protected]>; Sean Christopherson <[email protected]>
Subject: Re: [PATCH v3 08/13] KVM: arm64: Add support for KVM_EXIT_HYPERCALL

Quoted 16 lines

Quoted 16 lines

Quoted 15 lines

Quoted 7 lines

Quoted 2 lines

Quoted 2 lines

Should we expose the ESR, or at least ESR_EL2.IL as an additional flag?

I think we would need "Immediate value" of the ESR_EL2 register in the user-space/VMM to be able to construct the syndrome value. I cannot see where it is being sent?

The immediate value is not exposed to userspace, although by definition the immediate value must be zero. The SMCCC spec requires all compliant calls to use an immediate of zero (DEN0028E 2.9).

Is there a legitimate use case for hypercalls with a nonzero immediate? They would no longer be considered SMCCC calls at that point, so they wouldn't work with the new UAPI.

Quoted 8 lines

Quoted 3 lines

I agree. The use of non-zero immediate has long been deprecated. I guess we should actually reject non-zero immediate for HVC just like we do for SMC.

Ok. Maybe I will hard code Immediate value as 0 to create a syndrome value at the VMM/Qemu and will also put a note stating non-zero immediate for HVC/SVC are not supported/deprecated.

Yes, because this should be the only situation where you should see such an exit to userspace.

Quoted 4 lines

Quoted 3 lines

If there is an actual need for a non-zero immediate to be propagated to userspace (want to emulate Xen's infamous 'HVC #0xEA1'?), then this should be an extension to the current API.

Quoted 2 lines

Oh ok, then perhaps this new extension change should be simultaneously committed to avoid breaking Xen?

How would that break Xen? I don't have any plan to emulate Xen in any shape or form, and I don't think anyone want to do that in userspace either.

I really want to see an actual use case to expand this stuff. Because so far, we follow the strict SMCCC spec, and nothing else. But if we admit deviations, do we also have to expose SMC and HVC as different instructions?

From: Oliver Upton <[email protected]>
Sent: Wednesday, May 17, 2023 7:38 PM
To: Salil Mehta <[email protected]>
Cc: Marc Zyngier <[email protected]>; [email protected]; [email protected]; Paolo Bonzini <[email protected]>; James Morse <[email protected]>; Suzuki K Poulose <[email protected]>; yuzenghui <[email protected]>; Sean Christopherson <[email protected]>
Subject: Re: [PATCH v3 08/13] KVM: arm64: Add support for KVM_EXIT_HYPERCALL

Hi Salil,

On Wed, May 17, 2023 at 06:00:18PM +0000, Salil Mehta wrote:

[...]

Quoted 4 lines

Quoted 2 lines

Quoted 2 lines

Should we expose the ESR, or at least ESR_EL2.IL as an additional flag?

Quoted 3 lines

I think we would need "Immediate value" of the ESR_EL2 register in the user-space/VMM to be able to construct the syndrome value. I cannot see where it is being sent?

The immediate value is not exposed to userspace, although by definition the immediate value must be zero. The SMCCC spec requires all compliant calls to use an immediate of zero (DEN0028E 2.9).

Is there a legitimate use case for hypercalls with a nonzero immediate?

They would no longer be considered SMCCC calls at that point, so they wouldn't work with the new UAPI.

The Arm SMCCC is rather prescriptive in regards to the allocation of SMCCC function ID ranges. Many of the hypercall ranges have an associated specification from Arm (FF-A, PSCI, SDEI, etc.) with some room for vendor-specific implementations.

The ever-expanding SMCCC surface leaves a lot of work within KVM for providing new features. Furthermore, KVM implements its own vendor-specific ABI, with little room for other implementations (like Hyper-V, for example). Rather than cramming it all into the kernel we should provide a way for userspace to handle hypercalls.

[...]