Skip to contentSkip to conversation

Support SDEI Virtualization

PATCH In review Gavin Shan 5 participants 31 messages May 27, 2022 – Jun 4, 2023 1 message not in the archive

Subsystems (7)
Documentation KVM Kselftest arm64 +3 more subsystems
lore
Share & tools
lore

Gavin Shan

author
#0 lore raw

v7 cover Support SDEI Virtualization

This series intends to virtualize Software Delegated Exception Interface (SDEI), which is defined by DEN0054C (v1.1). It allows the hypervisor to deliver NMI-alike SDEI event to guest and it's needed by Async PF to deliver page-not-present notification from hypervisor to guest. The code and the required qemu changes can be found from:

https://developer.arm.com/documentation/den0054/c
https://github.com/gwshan/linux ("kvm/arm64_sdei")
https://github.com/gwshan/qemu ("kvm/arm64_sdei")

The design is quite strightforward by following the specification. The (SDEI) events are classified into the shared and private ones according to their scope. The shared event is system or VM scoped, but the private event is vcpu scoped. This implementation doesn't support the shared event because all the needed events are private. Besides, the critial events aren't supported by the implementation either. It means all events are normal in terms of priority.

There are several objects (data structures) introduced to help on the event registration, enablement, disablement, unregistration, reset, delivery and handling.

* kvm_sdei_event_handler
SDEI event handler, which is provided through EVENT_REGISTER hypercall, is called when the SDEI event is delivered from host to guest.

* kvm_sdei_event_context
The saved (preempted) context when SDEI event is delivered for handling.

* kvm_sdei_vcpu
SDEI events and their states.

The patches are organized as below:

PATCH[01-02] Preparatory work to extend smccc_get_argx() and refactor
hypercall routing mechanism
PATCH[03] Adds SDEI virtualization infrastructure
PATCH[04-16] Supports various SDEI hypercalls and event handling
PATCH[17] Exposes SDEI capability
PATCH[18-19] Support SDEI migration
PATCH[20] Adds document about SDEI
PATCH[21-22] SDEI related selftest cases

The previous revisions can be found:

v6: https://lore.kernel.org/lkml/[email protected]/T/
v5: https://lore.kernel.org/kvmarm/[email protected]/
v4: https://lore.kernel.org/kvmarm/[email protected]/
v3: https://lore.kernel.org/kvmarm/[email protected]/
v2: https://lore.kernel.org/kvmarm/[email protected]/
v1: https://lore.kernel.org/kvmarm/[email protected]/

Testing
=======
[1] The selftest case included in this series works fine. The default SDEI

    event, whose number is zero, can be registered, enabled, raised. The
    SDEI event handler can be invoked.

    [host]# pwd
    /home/gavin/sandbox/linux.main/tools/testing/selftests/kvm
    [root@virtlab-arm01 kvm]# ./aarch64/sdei 

        NR_VCPUS: 2    SDEI Event: 0x00000000

    --- VERSION
        Version:              1.1 (vendor: 0x4b564d)
    --- FEATURES
        Shared event slots:   0
        Private event slots:  0
        Relative mode:        No
    --- PRIVATE_RESET
    --- SHARED_RESET
    --- PE_UNMASK
    --- EVENT_GET_INFO
        Type:                 Private
        Priority:             Normal
        Signaled:             Yes
    --- EVENT_REGISTER
    --- EVENT_ENABLE
    --- EVENT_SIGNAL
        Handled:              Yes
        IRQ:                  No
        Status:               Registered-Enabled-Running
        PC/PSTATE:            000000000040232c 00000000600003c5
        Regs:                 0000000000000000 0000000000000000
                              0000000000000000 0000000000000000
    --- PE_MASK
    --- EVENT_DISABLE
    --- EVENT_UNREGISTER

        Result: OK

[2] There are additional patches in the following repositories to create

    procfs entries, allowing to inject SDEI event from host side. The
    SDEI client in the guest side registers the SDEI default event, whose
    number is zero. Also, the QEMU exports SDEI ACPI table and supports
    migration for SDEI.

    https://github.com/gwshan/linux    ("kvm/arm64_sdei")
    https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

    [2.1] Start the guests and migrate the source VM to the destination
          VM.

    [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
            -accel kvm -machine virt,gic-version=host                     \
            -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
            -m 1024M,slots=16,maxmem=64G                                  \
               :                                                          \
            -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
            -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
            -append earlycon=pl011,mmio,0x9000000                         \
               :

    [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
            -accel kvm -machine virt,gic-version=host                     \
            -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
            -m 1024M,slots=16,maxmem=64G                                  \
               :                                                          \
            -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
            -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
            -append earlycon=pl011,mmio,0x9000000                         \
            -incoming tcp:0:4444                                          \
               :

    [2.2] Check kernel log on the source VM. The SDEI service is enabled
          and the default SDEI event (0x0) is enabled.

     [guest-src]# dmesg | grep -i sdei
     ACPI: SDEI 0x000000005BC80000 000024 \
                (v00 BOCHS  BXPC     00000001 BXPC 00000001)
     sdei: SDEIv1.1 (0x4b564d) detected in firmware.
     SDEI TEST: Version 1.1, Vendor 0x4b564d
     sdei_init: SDEI event (0x0) registered
     sdei_init: SDEI event (0x0) enabled

 
     (qemu) migrate -d tcp:localhost:4444

    [2.3] Migrate the source VM to the destination VM. Inject SDEI event
          to the destination VM. The event is raised and handled.

    (qemu) migrate -d tcp:localhost:4444

    [host]# echo 0 > /proc/kvm/kvm-5360/vcpu-1

    [guest-dst]#
    =========== SDEI Event (CPU#1) ===========
    Event: 0000000000000000  Parameter: 00000000dabfdabf
    PC:    ffff800008cbb554  PSTATE: 00000000604000c5  SP: ffff800009c7bde0
    Regs:    00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001 
             ffff800016c28000 0000000000000000 0000000000000000 0000000000000000 
             0000000000000000 0000000000000000 0000000000000000 0000000000000000 
             0000000000000000 0000000000000000 0000000000000000 0000000000000000 
             0000000000000000 0000000000000000 0000000000000000 ffff800009399008 
             ffff8000097d9af0 ffff8000097d99f8 ffff8000093a8db8 ffff8000097d9b18 
             0000000000000000 0000000000000000 ffff000000339d00 0000000000000000 
             0000000000000000 ffff800009c7bde0 ffff800008cbb5c4 
    Context: 00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001 
             ffff800016c28000 03ffffffffffffff 000000024325db59 ffff8000097de190 
             ffff00000033a790 ffff800008cbb814 0000000000000a30 0000000000000000 

Changelog
=========
v7:
* Rebased to v5.19.rc1 (Gavin)
* Add hypercall ranges for routing (Oliver)
* Remove support to the critical event and redesigned data structures. Function names are also modified
as Oliver suggested (Oliver)
* Deliver event when it's enabled or the specific PE
is unmasked (Oliver)
* Improve EVENT_COMPLETE_AND_RESUME hypercall to resume

     from the specified address                               (Oliver) 
   * Add patches for SDEI migration and documentation         (Gavin)
   * Misc comments from Oliver Upon                           (Oliver)

v6:
* Rebased to v5.18.rc1 (Gavin)
* Pass additional argument to smccc_get_arg() (Oliver)
* Add preparatory patch to route hypercalls based on their
owners (Oliver)
* Remove the support for shared event. (Oliver/Gavin)
* Remove the support for migration and add-on patches to
support it in future (Oliver)
* The events are exposed by KVM instead of VMM (Oliver)
* kvm_sdei_state.h is dropped and all the structures are
folded into the corresponding ones in kvm_sdei.h (Oliver)
* Rename 'struct kvm_sdei_registered_event' to
'struct kvm_sdei_event' (Oliver)
* Misc comments from Oliver Upon (Oliver)
v5/v4/v3/v2/v1:
* Skipped here and please visit the history by

     https://lore.kernel.org/lkml/[email protected]/T/

Gavin Shan (22):
KVM: arm64: Extend smccc_get_argx()
KVM: arm64: Route hypercalls based on their owner
KVM: arm64: Add SDEI virtualization infrastructure
KVM: arm64: Support EVENT_REGISTER hypercall
KVM: arm64: Support EVENT_{ENABLE, DISABLE} hypercall
KVM: arm64: Support EVENT_CONTEXT hypercall
KVM: arm64: Support EVENT_UNREGISTER hypercall
KVM: arm64: Support EVENT_STATUS hypercall
KVM: arm64: Support EVENT_GET_INFO hypercall
KVM: arm64: Support PE_{MASK, UNMASK} hypercall
KVM: arm64: Support {PRIVATE, SHARED}_RESET hypercall
KVM: arm64: Support event injection and delivery
KVM: arm64: Support EVENT_{COMPLETE, COMPLETE_AND_RESUME} hypercall
KVM: arm64: Support EVENT_SIGNAL hypercall
KVM: arm64: Support SDEI_FEATURES hypercall
KVM: arm64: Support SDEI_VERSION hypercall
KVM: arm64: Expose SDEI capbility and service
KVM: arm64: Allow large sized pseudo firmware registers
KVM: arm64: Support SDEI event migration
KVM: arm64: Add SDEI document
selftests: KVM: aarch64: Add SDEI case in hypercall tests
selftests: KVM: aarch64: Add SDEI test case

 Documentation/virt/kvm/api.rst                |  11 +
 Documentation/virt/kvm/arm/hypercalls.rst     |   4 +
 Documentation/virt/kvm/arm/sdei.rst           |  64 ++
 arch/arm64/include/asm/kvm_host.h             |   3 +
 arch/arm64/include/asm/kvm_sdei.h             |  81 +++
 arch/arm64/include/uapi/asm/kvm.h             |  18 +
 arch/arm64/kvm/Makefile                       |   2 +-
 arch/arm64/kvm/arm.c                          |   8 +
 arch/arm64/kvm/hypercalls.c                   | 182 +++--
 arch/arm64/kvm/psci.c                         |  14 +-
 arch/arm64/kvm/pvtime.c                       |   2 +-
 arch/arm64/kvm/sdei.c                         | 676 ++++++++++++++++++
 arch/arm64/kvm/trng.c                         |   4 +-
 include/kvm/arm_hypercalls.h                  |  19 +-
 include/linux/arm-smccc.h                     |   7 +
 include/uapi/linux/arm_sdei.h                 |   8 +
 include/uapi/linux/kvm.h                      |   1 +
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/aarch64/hypercalls.c        |  11 +-
 tools/testing/selftests/kvm/aarch64/sdei.c    | 450 ++++++++++++
 20 files changed, 1499 insertions(+), 67 deletions(-)
 create mode 100644 Documentation/virt/kvm/arm/sdei.rst
 create mode 100644 arch/arm64/include/asm/kvm_sdei.h
 create mode 100644 arch/arm64/kvm/sdei.c
 create mode 100644 tools/testing/selftests/kvm/aarch64/sdei.c
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#1 lore raw

PATCH v7 1/22 KVM: arm64: Extend smccc_get_argx()

Currently, there are 3 inline functions to retrieve SMCCC arguments, but the number of arguments is limited to 3. We need to retrieve more SMCCC arguments when SDEI virtualization is supported.

This introduces smccc_get_arg(), which accepts @index to indicate the SMCCC argument to be retrieved. Besides, smccc_get_function() also calls into this newly introduced helper. Further more, we also mechanically replace smccc_get_{arg1, arg2, arg3}() using the newly introduced helper.

---
 arch/arm64/kvm/hypercalls.c  |  4 ++--
 arch/arm64/kvm/psci.c        | 14 +++++++-------
 arch/arm64/kvm/pvtime.c      |  2 +-
 arch/arm64/kvm/trng.c        |  4 ++--
 include/kvm/arm_hypercalls.h | 19 +++++--------------
 5 files changed, 17 insertions(+), 26 deletions(-)
arch/arm64/kvm/hypercalls.c +2 -2
index c9f401fa01a9..69f437f0d7c7 100644
@@ -41,7 +41,7 @@ static void kvm_ptp_get_time(struct kvm_vcpu *vcpu, u64 *val)
 	 * (virtual or physical) with the first argument of the SMCCC
 	 * call. In case the identifier is not supported, error out.
 	 */
-	feature = smccc_get_arg1(vcpu);
+	feature = smccc_get_arg(vcpu, 1);
 	switch (feature) {
 	case KVM_PTP_VIRT_COUNTER:
 		cycles = systime_snapshot.cycles - vcpu_read_sys_reg(vcpu, CNTVOFF_EL2);
@@ -137,7 +137,7 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
 		val[0] = ARM_SMCCC_VERSION_1_1;
 		break;
 	case ARM_SMCCC_ARCH_FEATURES_FUNC_ID:
-		feature = smccc_get_arg1(vcpu);
+		feature = smccc_get_arg(vcpu, 1);
 		switch (feature) {
 		case ARM_SMCCC_ARCH_WORKAROUND_1:
 			switch (arm64_get_spectre_v2_state()) {
arch/arm64/kvm/psci.c +7 -7
index 7fbc4c1b9df0..5394b8a69e10 100644
@@ -64,7 +64,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)
 	struct kvm_vcpu *vcpu = NULL;
 	unsigned long cpu_id;
 
-	cpu_id = smccc_get_arg1(source_vcpu);
+	cpu_id = smccc_get_arg(source_vcpu, 1);
 	if (!kvm_psci_valid_affinity(source_vcpu, cpu_id))
 		return PSCI_RET_INVALID_PARAMS;
 
@@ -85,7 +85,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)
 
 	reset_state = &vcpu->arch.reset_state;
 
-	reset_state->pc = smccc_get_arg2(source_vcpu);
+	reset_state->pc = smccc_get_arg(source_vcpu, 2);
 
 	/* Propagate caller endianness */
 	reset_state->be = kvm_vcpu_is_be(source_vcpu);
@@ -94,7 +94,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)
 	 * NOTE: We always update r0 (or x0) because for PSCI v0.1
 	 * the general purpose registers are undefined upon CPU_ON.
 	 */
-	reset_state->r0 = smccc_get_arg3(source_vcpu);
+	reset_state->r0 = smccc_get_arg(source_vcpu, 3);
 
 	WRITE_ONCE(reset_state->reset, true);
 	kvm_make_request(KVM_REQ_VCPU_RESET, vcpu);
@@ -121,8 +121,8 @@ static unsigned long kvm_psci_vcpu_affinity_info(struct kvm_vcpu *vcpu)
 	struct kvm *kvm = vcpu->kvm;
 	struct kvm_vcpu *tmp;
 
-	target_affinity = smccc_get_arg1(vcpu);
-	lowest_affinity_level = smccc_get_arg2(vcpu);
+	target_affinity = smccc_get_arg(vcpu, 1);
+	lowest_affinity_level = smccc_get_arg(vcpu, 2);
 
 	if (!kvm_psci_valid_affinity(vcpu, target_affinity))
 		return PSCI_RET_INVALID_PARAMS;
@@ -318,7 +318,7 @@ static int kvm_psci_1_x_call(struct kvm_vcpu *vcpu, u32 minor)
 		val = minor == 0 ? KVM_ARM_PSCI_1_0 : KVM_ARM_PSCI_1_1;
 		break;
 	case PSCI_1_0_FN_PSCI_FEATURES:
-		arg = smccc_get_arg1(vcpu);
+		arg = smccc_get_arg(vcpu, 1);
 		val = kvm_psci_check_allowed_function(vcpu, arg);
 		if (val)
 			break;
@@ -372,7 +372,7 @@ static int kvm_psci_1_x_call(struct kvm_vcpu *vcpu, u32 minor)
 		fallthrough;
 	case PSCI_1_1_FN64_SYSTEM_RESET2:
 		if (minor >= 1) {
-			arg = smccc_get_arg1(vcpu);
+			arg = smccc_get_arg(vcpu, 1);
 
 			if (arg <= PSCI_1_1_RESET_TYPE_SYSTEM_WARM_RESET ||
 			    arg >= PSCI_1_1_RESET_TYPE_VENDOR_START) {
arch/arm64/kvm/pvtime.c +1 -1
index 78a09f7a6637..05e775fc9e8b 100644
@@ -34,7 +34,7 @@ void kvm_update_stolen_time(struct kvm_vcpu *vcpu)
 
 long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu)
 {
-	u32 feature = smccc_get_arg1(vcpu);
+	u32 feature = smccc_get_arg(vcpu, 1);
 	long val = SMCCC_RET_NOT_SUPPORTED;
 
 	switch (feature) {
arch/arm64/kvm/trng.c +2 -2
index 99bdd7103c9c..89911b724a26 100644
@@ -24,7 +24,7 @@ static const uuid_t arm_smc_trng_uuid __aligned(4) = UUID_INIT(
 static int kvm_trng_do_rnd(struct kvm_vcpu *vcpu, int size)
 {
 	DECLARE_BITMAP(bits, TRNG_MAX_BITS64);
-	u32 num_bits = smccc_get_arg1(vcpu);
+	u32 num_bits = smccc_get_arg(vcpu, 1);
 	int i;
 
 	if (num_bits > 3 * size) {
@@ -60,7 +60,7 @@ int kvm_trng_call(struct kvm_vcpu *vcpu)
 		val = ARM_SMCCC_TRNG_VERSION_1_0;
 		break;
 	case ARM_SMCCC_TRNG_FEATURES:
-		switch (smccc_get_arg1(vcpu)) {
+		switch (smccc_get_arg(vcpu, 1)) {
 		case ARM_SMCCC_TRNG_VERSION:
 		case ARM_SMCCC_TRNG_FEATURES:
 		case ARM_SMCCC_TRNG_GET_UUID:
include/kvm/arm_hypercalls.h +5 -14
index 1188f116cf4e..ffd06387dfdf 100644
@@ -8,24 +8,15 @@
 
 int kvm_hvc_call_handler(struct kvm_vcpu *vcpu);
 
-static inline u32 smccc_get_function(struct kvm_vcpu *vcpu)
-{
-	return vcpu_get_reg(vcpu, 0);
-}
-
-static inline unsigned long smccc_get_arg1(struct kvm_vcpu *vcpu)
+static inline unsigned long smccc_get_arg(struct kvm_vcpu *vcpu,
+					  unsigned char index)
 {
-	return vcpu_get_reg(vcpu, 1);
+	return vcpu_get_reg(vcpu, index);
 }
 
-static inline unsigned long smccc_get_arg2(struct kvm_vcpu *vcpu)
-{
-	return vcpu_get_reg(vcpu, 2);
-}
-
-static inline unsigned long smccc_get_arg3(struct kvm_vcpu *vcpu)
+static inline u32 smccc_get_function(struct kvm_vcpu *vcpu)
 {
-	return vcpu_get_reg(vcpu, 3);
+	return smccc_get_arg(vcpu, 0);
 }
 
 static inline void smccc_set_retval(struct kvm_vcpu *vcpu,
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#2 lore raw

PATCH v7 2/22 KVM: arm64: Route hypercalls based on their owner

In reply to Gavin Shan

kvm_hvc_call_handler() directly handles the incoming hypercall, or routes it based on its (function) ID. kvm_psci_call() becomes the gate keeper to handle the hypercall that can't be handled by any one else. It makes kvm_hvc_call_handler() a bit messy.

This reorgnizes the code to route hypercalls to the corresponding handler based on their owners. A hypercall may be handled directly inside the owner's handler or routed to the associated functionality for further handling. By the way, @func_id is renamed to @func, consistent with smccc_get_function().

The only exception is 0.1 PSCI hypercalls. Their owners aren't Standard Secure Service and can't be routed by their owners. So those 0.1 PSCI hypercalls are filtered and routed to kvm_psci_call() in kvm_hvc_call_handler().

---
 arch/arm64/kvm/hypercalls.c | 116 ++++++++++++++++++++++++++----------
 include/linux/arm-smccc.h   |   5 ++
 2 files changed, 88 insertions(+), 33 deletions(-)
arch/arm64/kvm/hypercalls.c +83 -33
index 69f437f0d7c7..d66f464e0eb5 100644
@@ -65,9 +65,9 @@ static void kvm_ptp_get_time(struct kvm_vcpu *vcpu, u64 *val)
 	val[3] = lower_32_bits(cycles);
 }
 
-static bool kvm_hvc_call_default_allowed(u32 func_id)
+static bool kvm_hvc_call_default_allowed(u32 func)
 {
-	switch (func_id) {
+	switch (func) {
 	/*
 	 * List of function-ids that are not gated with the bitmapped
 	 * feature firmware registers, and are to be allowed for
@@ -78,26 +78,26 @@ static bool kvm_hvc_call_default_allowed(u32 func_id)
 		return true;
 	default:
 		/* PSCI 0.2 and up is in the 0:0x1f range */
-		if (ARM_SMCCC_OWNER_NUM(func_id) == ARM_SMCCC_OWNER_STANDARD &&
-		    ARM_SMCCC_FUNC_NUM(func_id) <= 0x1f)
+		if (ARM_SMCCC_OWNER_NUM(func) == ARM_SMCCC_OWNER_STANDARD &&
+		    ARM_SMCCC_FUNC_NUM(func) <= 0x1f)
 			return true;
 
 		/*
 		 * KVM's PSCI 0.1 doesn't comply with SMCCC, and has
 		 * its own function-id base and range
 		 */
-		if (func_id >= KVM_PSCI_FN(0) && func_id <= KVM_PSCI_FN(3))
+		if (func >= KVM_PSCI_FN(0) && func <= KVM_PSCI_FN(3))
 			return true;
 
 		return false;
 	}
 }
 
-static bool kvm_hvc_call_allowed(struct kvm_vcpu *vcpu, u32 func_id)
+static bool kvm_hvc_call_allowed(struct kvm_vcpu *vcpu, u32 func)
 {
 	struct kvm_smccc_features *smccc_feat = &vcpu->kvm->arch.smccc_feat;
 
-	switch (func_id) {
+	switch (func) {
 	case ARM_SMCCC_TRNG_VERSION:
 	case ARM_SMCCC_TRNG_FEATURES:
 	case ARM_SMCCC_TRNG_GET_UUID:
@@ -117,24 +117,19 @@ static bool kvm_hvc_call_allowed(struct kvm_vcpu *vcpu, u32 func_id)
 		return test_bit(KVM_REG_ARM_VENDOR_HYP_BIT_PTP,
 				&smccc_feat->vendor_hyp_bmap);
 	default:
-		return kvm_hvc_call_default_allowed(func_id);
+		return kvm_hvc_call_default_allowed(func);
 	}
 }
 
-int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
+static int kvm_hvc_arch(struct kvm_vcpu *vcpu, u32 func)
 {
 	struct kvm_smccc_features *smccc_feat = &vcpu->kvm->arch.smccc_feat;
-	u32 func_id = smccc_get_function(vcpu);
-	u64 val[4] = {SMCCC_RET_NOT_SUPPORTED};
+	u64 val = SMCCC_RET_NOT_SUPPORTED;
 	u32 feature;
-	gpa_t gpa;
-
-	if (!kvm_hvc_call_allowed(vcpu, func_id))
-		goto out;
 
-	switch (func_id) {
+	switch (func) {
 	case ARM_SMCCC_VERSION_FUNC_ID:
-		val[0] = ARM_SMCCC_VERSION_1_1;
+		val = ARM_SMCCC_VERSION_1_1;
 		break;
 	case ARM_SMCCC_ARCH_FEATURES_FUNC_ID:
 		feature = smccc_get_arg(vcpu, 1);
@@ -144,10 +139,10 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
 			case SPECTRE_VULNERABLE:
 				break;
 			case SPECTRE_MITIGATED:
-				val[0] = SMCCC_RET_SUCCESS;
+				val = SMCCC_RET_SUCCESS;
 				break;
 			case SPECTRE_UNAFFECTED:
-				val[0] = SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED;
+				val = SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED;
 				break;
 			}
 			break;
@@ -170,7 +165,7 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
 					break;
 				fallthrough;
 			case SPECTRE_UNAFFECTED:
-				val[0] = SMCCC_RET_NOT_REQUIRED;
+				val = SMCCC_RET_NOT_REQUIRED;
 				break;
 			}
 			break;
@@ -179,28 +174,65 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
 			case SPECTRE_VULNERABLE:
 				break;
 			case SPECTRE_MITIGATED:
-				val[0] = SMCCC_RET_SUCCESS;
+				val = SMCCC_RET_SUCCESS;
 				break;
 			case SPECTRE_UNAFFECTED:
-				val[0] = SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED;
+				val = SMCCC_ARCH_WORKAROUND_RET_UNAFFECTED;
 				break;
 			}
 			break;
 		case ARM_SMCCC_HV_PV_TIME_FEATURES:
 			if (test_bit(KVM_REG_ARM_STD_HYP_BIT_PV_TIME,
 				     &smccc_feat->std_hyp_bmap))
-				val[0] = SMCCC_RET_SUCCESS;
+				val = SMCCC_RET_SUCCESS;
 			break;
 		}
 		break;
+	}
+
+	smccc_set_retval(vcpu, val, 0, 0, 0);
+	return 1;
+}
+
+static int kvm_hvc_standard(struct kvm_vcpu *vcpu, u32 func)
+{
+	switch (ARM_SMCCC_FUNC_NUM(func)) {
+	case ARM_SMCCC_STANDARD_PSCI_START ... ARM_SMCCC_STANDARD_PSCI_END:
+		return kvm_psci_call(vcpu);
+	case ARM_SMCCC_STANDARD_TRNG_START ... ARM_SMCCC_STANDARD_TRNG_END:
+		return kvm_trng_call(vcpu);
+	}
+
+	smccc_set_retval(vcpu, SMCCC_RET_NOT_SUPPORTED, 0, 0, 0);
+	return 1;
+}
+
+static int kvm_hvc_standard_hyp(struct kvm_vcpu *vcpu, u32 func)
+{
+	u64 val = SMCCC_RET_NOT_SUPPORTED;
+	gpa_t gpa;
+
+	switch (func) {
 	case ARM_SMCCC_HV_PV_TIME_FEATURES:
-		val[0] = kvm_hypercall_pv_features(vcpu);
+		val = kvm_hypercall_pv_features(vcpu);
 		break;
 	case ARM_SMCCC_HV_PV_TIME_ST:
 		gpa = kvm_init_stolen_time(vcpu);
 		if (gpa != GPA_INVALID)
-			val[0] = gpa;
+			val = gpa;
 		break;
+	}
+
+	smccc_set_retval(vcpu, val, 0, 0, 0);
+	return 1;
+}
+
+static int kvm_hvc_vendor_hyp(struct kvm_vcpu *vcpu, u32 func)
+{
+	struct kvm_smccc_features *smccc_feat = &vcpu->kvm->arch.smccc_feat;
+	u64 val[4] = { SMCCC_RET_NOT_SUPPORTED };
+
+	switch (func) {
 	case ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID:
 		val[0] = ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_0;
 		val[1] = ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_1;
@@ -213,18 +245,36 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
 	case ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID:
 		kvm_ptp_get_time(vcpu, val);
 		break;
-	case ARM_SMCCC_TRNG_VERSION:
-	case ARM_SMCCC_TRNG_FEATURES:
-	case ARM_SMCCC_TRNG_GET_UUID:
-	case ARM_SMCCC_TRNG_RND32:
-	case ARM_SMCCC_TRNG_RND64:
-		return kvm_trng_call(vcpu);
-	default:
+	}
+
+	smccc_set_retval(vcpu, val[0], val[1], val[2], val[3]);
+	return 1;
+}
+
+int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
+{
+	u32 func = smccc_get_function(vcpu);
+
+	if (!kvm_hvc_call_allowed(vcpu, func))
+		goto out;
+
+	/* Filter these calls that aren't documented in the specification */
+	if (func >= KVM_PSCI_FN_CPU_SUSPEND && func <= KVM_PSCI_FN_MIGRATE)
 		return kvm_psci_call(vcpu);
+
+	switch (ARM_SMCCC_OWNER_NUM(func)) {
+	case ARM_SMCCC_OWNER_ARCH:
+		return kvm_hvc_arch(vcpu, func);
+	case ARM_SMCCC_OWNER_STANDARD:
+		return kvm_hvc_standard(vcpu, func);
+	case ARM_SMCCC_OWNER_STANDARD_HYP:
+		return kvm_hvc_standard_hyp(vcpu, func);
+	case ARM_SMCCC_OWNER_VENDOR_HYP:
+		return kvm_hvc_vendor_hyp(vcpu, func);
 	}
 
 out:
-	smccc_set_retval(vcpu, val[0], val[1], val[2], val[3]);
+	smccc_set_retval(vcpu, SMCCC_RET_NOT_SUPPORTED, 0, 0, 0);
 	return 1;
 }
 
include/linux/arm-smccc.h +5 -0
index 220c8c60e021..834fb963a689 100644
@@ -55,6 +55,11 @@
 #define ARM_SMCCC_OWNER_TRUSTED_OS	50
 #define ARM_SMCCC_OWNER_TRUSTED_OS_END	63
 
+#define ARM_SMCCC_STANDARD_PSCI_START	0x0000
+#define ARM_SMCCC_STANDARD_PSCI_END	0x001f
+#define ARM_SMCCC_STANDARD_TRNG_START	0x0050
+#define ARM_SMCCC_STANDARD_TRNG_END	0x005f
+
 #define ARM_SMCCC_FUNC_QUERY_CALL_UID  0xff01
 
 #define ARM_SMCCC_QUIRK_NONE		0
Signature · 1 line
2.23.0
author
#3 lore raw

PATCH v7 3/22 KVM: arm64: Add SDEI virtualization infrastructure

In reply to Gavin Shan

Software Delegated Exception Interface (SDEI) provides a mechanism for registering and servicing system events, as defined by ARM DEN0054C specification. One of these events will be used by Asynchronous Page Fault (Async PF) to deliver notifications from host to guest.

The events are classified into shared and private ones according to their scopes. The shared events are system or VM scoped, but the private events are CPU or VCPU scoped. The shared events can be registered, enabled, unregistered and reset through hypercalls issued from any VCPU. However, the private events are registered, enabled, unregistered and reset on the calling VCPU through hypercalls. Besides, the events are also classified into critical and normal events according their priority. During event delivery and handling, the normal event can be preempted by another critical event, but not in reverse way. The critical event can't be preempted by another normal event.

In the implementation, both shared event and critical event aren't supported. It means all supported events are private and normal. There are several objects introduced to manage events and their states.

* kvm_sdei_event_handler
SDEI event handler, which is provided through EVENT_REGISTER hypercall, is called when the SDEI event is delivered from host to guest.

* kvm_sdei_event_context
The saved (preempted) context when SDEI event is delivered for handling.

* kvm_sdei_vcpu
SDEI events and their states.

The flag KVM_ARM64_SDEI_MASKED is set to mask vCPU from SDEI events by default. Besides, SDEI_NOT_SUPPORTED is returned for all hypercalls at present. The hypercalls will be supported one by one in subsequent patches.

---
 arch/arm64/include/asm/kvm_host.h |  2 +
 arch/arm64/include/asm/kvm_sdei.h | 70 +++++++++++++++++++++++++++++++
 arch/arm64/kvm/Makefile           |  2 +-
 arch/arm64/kvm/arm.c              |  4 ++
 arch/arm64/kvm/hypercalls.c       |  3 ++
 arch/arm64/kvm/sdei.c             | 59 ++++++++++++++++++++++++++
 include/linux/arm-smccc.h         |  2 +
 7 files changed, 141 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm64/include/asm/kvm_sdei.h
 create mode 100644 arch/arm64/kvm/sdei.c
arch/arm64/include/asm/kvm_host.h +2 -0
index 47a1e25e25bb..c9d053c2569e 100644
@@ -366,6 +366,7 @@ struct kvm_vcpu_arch {
 	 * Anything that is not used directly from assembly code goes
 	 * here.
 	 */
+	struct kvm_sdei_vcpu *sdei;
 
 	/*
 	 * Guest registers we preserve during guest debugging.
@@ -466,6 +467,7 @@ struct kvm_vcpu_arch {
 #define KVM_ARM64_ON_UNSUPPORTED_CPU	(1 << 15) /* Physical CPU not in supported_cpus */
 #define KVM_ARM64_HOST_SME_ENABLED	(1 << 16) /* SME enabled for EL0 */
 #define KVM_ARM64_WFIT			(1 << 17) /* WFIT instruction trapped */
+#define KVM_ARM64_SDEI_MASKED		(1 << 18) /* SDEI is masked */
 
 #define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \
 				 KVM_GUESTDBG_USE_SW_BP | \
arch/arm64/include/asm/kvm_sdei.h new file +70 -0

70 lines not on this page · raw message

arch/arm64/kvm/Makefile +1 -1
index aa127ae9f675..26e9d62875a8 100644
@@ -14,7 +14,7 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
 	 inject_fault.o va_layout.o handle_exit.o \
 	 guest.o debug.o reset.o sys_regs.o \
 	 vgic-sys-reg-v3.o fpsimd.o pkvm.o \
-	 arch_timer.o trng.o vmid.o \
+	 arch_timer.o trng.o vmid.o sdei.o \
 	 vgic/vgic.o vgic/vgic-init.o \
 	 vgic/vgic-irqfd.o vgic/vgic-v2.o \
 	 vgic/vgic-v3.o vgic/vgic-v4.o \
arch/arm64/kvm/arm.c +4 -0
index 400bb0fe2745..e9516f951e7b 100644
@@ -38,6 +38,7 @@
 #include <asm/kvm_asm.h>
 #include <asm/kvm_mmu.h>
 #include <asm/kvm_emulate.h>
+#include <asm/kvm_sdei.h>
 #include <asm/sections.h>
 
 #include <kvm/arm_hypercalls.h>
@@ -337,6 +338,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
 
 	kvm_arm_pvtime_vcpu_init(&vcpu->arch);
 
+	kvm_sdei_create_vcpu(vcpu);
+
 	vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu;
 
 	err = kvm_vgic_vcpu_init(vcpu);
@@ -358,6 +361,7 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
 	kvm_mmu_free_memory_cache(&vcpu->arch.mmu_page_cache);
 	kvm_timer_vcpu_terminate(vcpu);
 	kvm_pmu_vcpu_destroy(vcpu);
+	kvm_sdei_destroy_vcpu(vcpu);
 
 	kvm_arm_vcpu_destroy(vcpu);
 }
arch/arm64/kvm/hypercalls.c +3 -0
index d66f464e0eb5..88561ac47301 100644
@@ -5,6 +5,7 @@
 #include <linux/kvm_host.h>
 
 #include <asm/kvm_emulate.h>
+#include <asm/kvm_sdei.h>
 
 #include <kvm/arm_hypercalls.h>
 #include <kvm/arm_psci.h>
@@ -199,6 +200,8 @@ static int kvm_hvc_standard(struct kvm_vcpu *vcpu, u32 func)
 	switch (ARM_SMCCC_FUNC_NUM(func)) {
 	case ARM_SMCCC_STANDARD_PSCI_START ... ARM_SMCCC_STANDARD_PSCI_END:
 		return kvm_psci_call(vcpu);
+	case ARM_SMCCC_STANDARD_SDEI_START ... ARM_SMCCC_STANDARD_SDEI_END:
+		return kvm_sdei_call(vcpu);
 	case ARM_SMCCC_STANDARD_TRNG_START ... ARM_SMCCC_STANDARD_TRNG_END:
 		return kvm_trng_call(vcpu);
 	}
arch/arm64/kvm/sdei.c new file +59 -0

59 lines not on this page · raw message

include/linux/arm-smccc.h +2 -0
index 834fb963a689..18159a9c6703 100644
@@ -57,6 +57,8 @@
 
 #define ARM_SMCCC_STANDARD_PSCI_START	0x0000
 #define ARM_SMCCC_STANDARD_PSCI_END	0x001f
+#define ARM_SMCCC_STANDARD_SDEI_START	0x0020
+#define ARM_SMCCC_STANDARD_SDEI_END	0x003f
 #define ARM_SMCCC_STANDARD_TRNG_START	0x0050
 #define ARM_SMCCC_STANDARD_TRNG_END	0x005f
 
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#4 lore raw

PATCH v7 4/22 KVM: arm64: Support EVENT_REGISTER hypercall

In reply to Gavin Shan

This supports EVENT_REGISTER hypercall, which is used by guest to register event. The event won't be raised until it's registered and enabled. There are several cases where the request is injected:

* The event isn't supported by KVM

* The event has been registered or pending for unregistration

* The reserved bits or relative mode are set in @flags argument

---
 arch/arm64/kvm/sdei.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
arch/arm64/kvm/sdei.c +31 -0
index 0c4415fcad16..40c2d585470d 100644
@@ -13,6 +13,34 @@
 #include <kvm/arm_hypercalls.h>
 #include <asm/kvm_sdei.h>
 
+static unsigned long event_register(struct kvm_vcpu *vcpu)
+{
+	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
+	unsigned int num = smccc_get_arg(vcpu, 1);
+	unsigned long flags = smccc_get_arg(vcpu, 4);
+
+	if (num >= KVM_NR_SDEI_EVENTS)
+		return SDEI_INVALID_PARAMETERS;
+
+	/* Reject if the reserved bits or relative mode are set */
+	if (flags & ~0x1UL)
+		return SDEI_INVALID_PARAMETERS;
+
+	/*
+	 * Reject if the event has been registered or pending for
+	 * unregistration.
+	 */
+	if (test_bit(num, &vsdei->registered) ||
+	    test_bit(num, &vsdei->running))
+		return SDEI_DENIED;
+
+	vsdei->handlers[num].ep_addr = smccc_get_arg(vcpu, 2);
+	vsdei->handlers[num].ep_arg = smccc_get_arg(vcpu, 3);
+	set_bit(num, &vsdei->registered);
+
+	return SDEI_SUCCESS;
+}
+
 int kvm_sdei_call(struct kvm_vcpu *vcpu)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -31,6 +59,9 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	}
 
 	switch (func) {
+	case SDEI_1_0_FN_SDEI_EVENT_REGISTER:
+		ret = event_register(vcpu);
+		break;
 	default:
 		ret = SDEI_NOT_SUPPORTED;
 	}
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#5 lore raw

PATCH v7 5/22 KVM: arm64: Support EVENT_{ENABLE, DISABLE} hypercall

In reply to Gavin Shan

This supports EVENT_{ENABLE, DISABLE} hypercall. The event can be pending, but never be delivered to guest until it's enabled.

* The request is rejected if the event isn't registered or pending for unregistration.

* When the event is enabled, KVM_REQ_SDEI is set to so that the event can be delivered to guest, if it has been pending. The delivery logic will be implemented in subsequent patches. For now, it's nothing other than a flag.

---
 arch/arm64/include/asm/kvm_host.h |  1 +
 arch/arm64/kvm/sdei.c             | 33 +++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+)
arch/arm64/include/asm/kvm_host.h +1 -0
index c9d053c2569e..79a0043a908c 100644
@@ -47,6 +47,7 @@
 #define KVM_REQ_RELOAD_GICv4	KVM_ARCH_REQ(4)
 #define KVM_REQ_RELOAD_PMU	KVM_ARCH_REQ(5)
 #define KVM_REQ_SUSPEND		KVM_ARCH_REQ(6)
+#define KVM_REQ_SDEI		KVM_ARCH_REQ(7)
 
 #define KVM_DIRTY_LOG_MANUAL_CAPS   (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \
 				     KVM_DIRTY_LOG_INITIALLY_SET)
arch/arm64/kvm/sdei.c +33 -0
index 40c2d585470d..a4046e7b21d8 100644
@@ -41,6 +41,33 @@ static unsigned long event_register(struct kvm_vcpu *vcpu)
 	return SDEI_SUCCESS;
 }
 
+static unsigned long event_enable(struct kvm_vcpu *vcpu, bool enable)
+{
+	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
+	unsigned int num = smccc_get_arg(vcpu, 1);
+
+	if (num >= KVM_NR_SDEI_EVENTS)
+		return SDEI_INVALID_PARAMETERS;
+
+	/*
+	 * Reject if the event hasn't been registered or pending
+	 * for unregistration.
+	 */
+	if (!test_bit(num, &vsdei->registered))
+		return SDEI_DENIED;
+
+	if (enable) {
+		set_bit(num, &vsdei->enabled);
+		if (!(vcpu->arch.flags & KVM_ARM64_SDEI_MASKED) &&
+		    test_bit(num, &vsdei->pending))
+			kvm_make_request(KVM_REQ_SDEI, vcpu);
+	} else {
+		clear_bit(num, &vsdei->enabled);
+	}
+
+	return SDEI_SUCCESS;
+}
+
 int kvm_sdei_call(struct kvm_vcpu *vcpu)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -62,6 +89,12 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	case SDEI_1_0_FN_SDEI_EVENT_REGISTER:
 		ret = event_register(vcpu);
 		break;
+	case SDEI_1_0_FN_SDEI_EVENT_ENABLE:
+		ret = event_enable(vcpu, true);
+		break;
+	case SDEI_1_0_FN_SDEI_EVENT_DISABLE:
+		ret = event_enable(vcpu, false);
+		break;
 	default:
 		ret = SDEI_NOT_SUPPORTED;
 	}
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#6 lore raw

PATCH v7 6/22 KVM: arm64: Support EVENT_CONTEXT hypercall

In reply to Gavin Shan

This supports EVENT_CONTEXT hypercall. It's called inside the event handler to retrieve the specified register in the saved or preempted context.

* The request is rejected if no running event handler exists or the parameter ID is out of range.

---
 arch/arm64/kvm/sdei.c | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)
arch/arm64/kvm/sdei.c +21 -1
index a4046e7b21d8..2fec2dcd02b0 100644
@@ -44,7 +44,7 @@ static unsigned long event_register(struct kvm_vcpu *vcpu)
 static unsigned long event_enable(struct kvm_vcpu *vcpu, bool enable)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
-	unsigned int num = smccc_get_arg(vcpu, 1);
+	int num = smccc_get_arg(vcpu, 1);
 
 	if (num >= KVM_NR_SDEI_EVENTS)
 		return SDEI_INVALID_PARAMETERS;
@@ -68,6 +68,23 @@ static unsigned long event_enable(struct kvm_vcpu *vcpu, bool enable)
 	return SDEI_SUCCESS;
 }
 
+static unsigned long event_context(struct kvm_vcpu *vcpu)
+{
+	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
+	struct kvm_sdei_event_context *ctxt = &vsdei->ctxt;
+	unsigned int param_id = smccc_get_arg(vcpu, 1);
+
+	/* Reject if event handler isn't running */
+	if (!vsdei->running)
+		return SDEI_DENIED;
+
+	/* Reject if the parameter ID is out of range */
+	if (param_id >= ARRAY_SIZE(ctxt->regs))
+		return SDEI_INVALID_PARAMETERS;
+
+	return ctxt->regs[param_id];
+}
+
 int kvm_sdei_call(struct kvm_vcpu *vcpu)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -95,6 +112,9 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	case SDEI_1_0_FN_SDEI_EVENT_DISABLE:
 		ret = event_enable(vcpu, false);
 		break;
+	case SDEI_1_0_FN_SDEI_EVENT_CONTEXT:
+		ret = event_context(vcpu);
+		break;
 	default:
 		ret = SDEI_NOT_SUPPORTED;
 	}
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#7 lore raw

PATCH v7 7/22 KVM: arm64: Support EVENT_UNREGISTER hypercall

In reply to Gavin Shan

This supports EVENT_UNREGISTER hypercall. No event will be delivered to guest after it's unregistered.

* Reject if the event hasn't been registered.

* Return SDEI_PENDING if the event handler is running.

* The event is disabled automatically on unregistration.

---
 arch/arm64/kvm/sdei.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
arch/arm64/kvm/sdei.c +36 -0
index 2fec2dcd02b0..b44ab302732d 100644
@@ -85,6 +85,39 @@ static unsigned long event_context(struct kvm_vcpu *vcpu)
 	return ctxt->regs[param_id];
 }
 
+static unsigned long event_unregister(struct kvm_vcpu *vcpu)
+{
+	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
+	unsigned int num = smccc_get_arg(vcpu, 1);
+
+	if (num >= KVM_NR_SDEI_EVENTS)
+		return SDEI_INVALID_PARAMETERS;
+
+	/*
+	 * Reject if the event isn't registered. It's allowed to
+	 * unregister event which has been pending for that.
+	 */
+	if (!test_bit(num, &vsdei->registered)) {
+		if (test_bit(num, &vsdei->running))
+			return SDEI_PENDING;
+		else
+			return SDEI_DENIED;
+	}
+
+	/*
+	 * The event is disabled automatically on unregistration, even
+	 * pending for that.
+	 */
+	clear_bit(num, &vsdei->enabled);
+	clear_bit(num, &vsdei->registered);
+
+	/* Pending for unreigstration if the event handler is running */
+	if (test_bit(num, &vsdei->running))
+		return SDEI_PENDING;
+
+	return SDEI_SUCCESS;
+}
+
 int kvm_sdei_call(struct kvm_vcpu *vcpu)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -115,6 +148,9 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	case SDEI_1_0_FN_SDEI_EVENT_CONTEXT:
 		ret = event_context(vcpu);
 		break;
+	case SDEI_1_0_FN_SDEI_EVENT_UNREGISTER:
+		ret = event_unregister(vcpu);
+		break;
 	default:
 		ret = SDEI_NOT_SUPPORTED;
 	}
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#8 lore raw

PATCH v7 8/22 KVM: arm64: Support EVENT_STATUS hypercall

In reply to Gavin Shan

This supports EVENT_STATUS hypercall. It's used to retrieve the status about the specified event. A bitmap is returned to represent the event status, which includes state of registration, enablement, and running.

---
 arch/arm64/kvm/sdei.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
arch/arm64/kvm/sdei.c +22 -0
index b44ab302732d..377341f229da 100644
@@ -118,6 +118,25 @@ static unsigned long event_unregister(struct kvm_vcpu *vcpu)
 	return SDEI_SUCCESS;
 }
 
+static unsigned long event_status(struct kvm_vcpu *vcpu)
+{
+	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
+	unsigned int num = smccc_get_arg(vcpu, 1);
+	unsigned long ret = 0;
+
+	if (num >= KVM_NR_SDEI_EVENTS)
+		return SDEI_INVALID_PARAMETERS;
+
+	if (test_bit(num, &vsdei->registered))
+		ret |= (1UL << SDEI_EVENT_STATUS_REGISTERED);
+	if (test_bit(num, &vsdei->enabled))
+		ret |= (1UL << SDEI_EVENT_STATUS_ENABLED);
+	if (test_bit(num, &vsdei->running))
+		ret |= (1UL << SDEI_EVENT_STATUS_RUNNING);
+
+	return ret;
+}
+
 int kvm_sdei_call(struct kvm_vcpu *vcpu)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -151,6 +170,9 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	case SDEI_1_0_FN_SDEI_EVENT_UNREGISTER:
 		ret = event_unregister(vcpu);
 		break;
+	case SDEI_1_0_FN_SDEI_EVENT_STATUS:
+		ret = event_status(vcpu);
+		break;
 	default:
 		ret = SDEI_NOT_SUPPORTED;
 	}
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#9 lore raw

PATCH v7 9/22 KVM: arm64: Support EVENT_GET_INFO hypercall

In reply to Gavin Shan

This supports EVENT_GET_INFO hypercall. It's used to retrieve information about the event: type, signaled, priority, routing mode and affinity.

All supported events are private and have normal priority. Besides, all supported events can be signaled from software. It means we just return fixed value for type/signaled/priority.

As the routing mode and affinity are only available for the shared event, which is unsupported. So SDEI_INVALID_PARAMETERS is returned when they're requested.

---
 arch/arm64/kvm/sdei.c | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
arch/arm64/kvm/sdei.c +35 -0
index 377341f229da..cea523418c75 100644
@@ -137,6 +137,38 @@ static unsigned long event_status(struct kvm_vcpu *vcpu)
 	return ret;
 }
 
+static unsigned long event_info(struct kvm_vcpu *vcpu)
+{
+	unsigned int num = smccc_get_arg(vcpu, 1);
+	unsigned int info = smccc_get_arg(vcpu, 2);
+	unsigned long ret = 0;
+
+	if (num >= KVM_NR_SDEI_EVENTS)
+		return SDEI_INVALID_PARAMETERS;
+
+	/*
+	 * All supported events are private and have normal priority.
+	 * Besides, all supported events can be signaled by software
+	 */
+	switch (info) {
+	case SDEI_EVENT_INFO_EV_TYPE:
+		ret = SDEI_EVENT_TYPE_PRIVATE;
+		break;
+	case SDEI_EVENT_INFO_EV_SIGNALED:
+		ret = 1;
+		break;
+	case SDEI_EVENT_INFO_EV_PRIORITY:
+		ret = SDEI_EVENT_PRIORITY_NORMAL;
+		break;
+	case SDEI_EVENT_INFO_EV_ROUTING_MODE:
+	case SDEI_EVENT_INFO_EV_ROUTING_AFF:
+	default:
+		ret = SDEI_INVALID_PARAMETERS;
+	}
+
+	return ret;
+}
+
 int kvm_sdei_call(struct kvm_vcpu *vcpu)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -173,6 +205,9 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	case SDEI_1_0_FN_SDEI_EVENT_STATUS:
 		ret = event_status(vcpu);
 		break;
+	case SDEI_1_0_FN_SDEI_EVENT_GET_INFO:
+		ret = event_info(vcpu);
+		break;
 	default:
 		ret = SDEI_NOT_SUPPORTED;
 	}
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#10 lore raw

PATCH v7 10/22 KVM: arm64: Support PE_{MASK, UNMASK} hypercall

In reply to Gavin Shan

This supports PE_{MASK, UNMASK} hypercall. When PE_UNMASK is called, KVM_REQ_SDEI request becomes pending if there are any events need to be delivered.

---
 arch/arm64/kvm/sdei.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
arch/arm64/kvm/sdei.c +21 -0
index cea523418c75..6afa49b4f3c7 100644
@@ -169,6 +169,21 @@ static unsigned long event_info(struct kvm_vcpu *vcpu)
 	return ret;
 }
 
+static unsigned long pe_mask(struct kvm_vcpu *vcpu, bool mask)
+{
+	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
+
+	if (mask) {
+		vcpu->arch.flags |= KVM_ARM64_SDEI_MASKED;
+	} else {
+		vcpu->arch.flags &= ~KVM_ARM64_SDEI_MASKED;
+		if (vsdei->pending)
+			kvm_make_request(KVM_REQ_SDEI, vcpu);
+	}
+
+	return SDEI_SUCCESS;
+}
+
 int kvm_sdei_call(struct kvm_vcpu *vcpu)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -208,6 +223,12 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	case SDEI_1_0_FN_SDEI_EVENT_GET_INFO:
 		ret = event_info(vcpu);
 		break;
+	case SDEI_1_0_FN_SDEI_PE_MASK:
+		ret = pe_mask(vcpu, true);
+		break;
+	case SDEI_1_0_FN_SDEI_PE_UNMASK:
+		ret = pe_mask(vcpu, false);
+		break;
 	default:
 		ret = SDEI_NOT_SUPPORTED;
 	}
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#11 lore raw

PATCH v7 11/22 KVM: arm64: Support {PRIVATE, SHARED}_RESET hypercall

In reply to Gavin Shan

This supports {PRIVATE, SHARED}_RESET hypercall. There is nothing to do if we're going to reset the shared events, which are not supported. For the private events, their registered and enabled state are cleared.

---
 arch/arm64/kvm/sdei.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
arch/arm64/kvm/sdei.c +26 -0
index 6afa49b4f3c7..42ba6f97b168 100644
@@ -184,6 +184,26 @@ static unsigned long pe_mask(struct kvm_vcpu *vcpu, bool mask)
 	return SDEI_SUCCESS;
 }
 
+static unsigned long event_reset(struct kvm_vcpu *vcpu, bool private)
+{
+	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
+	unsigned int num;
+
+	/*
+	 * Nothing to do if we're going to reset the shared events,
+	 * which are unsupported.
+	 */
+	if (!private)
+		return SDEI_SUCCESS;
+
+	for (num = 0; num < KVM_NR_SDEI_EVENTS; num++) {
+		clear_bit(num, &vsdei->registered);
+		clear_bit(num, &vsdei->enabled);
+	}
+
+	return SDEI_SUCCESS;
+}
+
 int kvm_sdei_call(struct kvm_vcpu *vcpu)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -229,6 +249,12 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	case SDEI_1_0_FN_SDEI_PE_UNMASK:
 		ret = pe_mask(vcpu, false);
 		break;
+	case SDEI_1_0_FN_SDEI_PRIVATE_RESET:
+		ret = event_reset(vcpu, true);
+		break;
+	case SDEI_1_0_FN_SDEI_SHARED_RESET:
+		ret = event_reset(vcpu, false);
+		break;
 	default:
 		ret = SDEI_NOT_SUPPORTED;
 	}
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#12 lore raw

PATCH v7 12/22 KVM: arm64: Support event injection and delivery

In reply to Gavin Shan

This supports event injection, delivery and cancellation. The event is injected and cancelled by kvm_sdei_{inject, cancel}_event(). For event delivery, kvm_sdei_deliver_event() is added to accommodate KVM_REQ_SDEI request.

The KVM_REQ_SDEI request can be raised in several situation:

* PE is unmasked

* Event is enabled

* Completion of currently running event or handler on receiving EVENT_COMPLETE or EVENT_COMPLETE_AND_RESUME hypercall, which will be supported in the subsequent patch.

---
 arch/arm64/include/asm/kvm_sdei.h |   4 +
 arch/arm64/kvm/arm.c              |   3 +
 arch/arm64/kvm/sdei.c             | 123 ++++++++++++++++++++++++++++++
 3 files changed, 130 insertions(+)
arch/arm64/include/asm/kvm_sdei.h +4 -0
index 609338b17478..735d9ac1a5a2 100644
@@ -64,6 +64,10 @@ struct kvm_sdei_vcpu {
 
 /* APIs */
 int kvm_sdei_call(struct kvm_vcpu *vcpu);
+int kvm_sdei_inject_event(struct kvm_vcpu *vcpu,
+			  unsigned int num, bool immediate);
+int kvm_sdei_cancel_event(struct kvm_vcpu *vcpu, unsigned int num);
+void kvm_sdei_deliver_event(struct kvm_vcpu *vcpu);
 void kvm_sdei_create_vcpu(struct kvm_vcpu *vcpu);
 void kvm_sdei_destroy_vcpu(struct kvm_vcpu *vcpu);
 
arch/arm64/kvm/arm.c +3 -0
index e9516f951e7b..06cb5e38634e 100644
@@ -720,6 +720,9 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu)
 		if (kvm_check_request(KVM_REQ_VCPU_RESET, vcpu))
 			kvm_reset_vcpu(vcpu);
 
+		if (kvm_check_request(KVM_REQ_SDEI, vcpu))
+			kvm_sdei_deliver_event(vcpu);
+
 		/*
 		 * Clear IRQ_PENDING requests that were made to guarantee
 		 * that a VCPU sees new virtual interrupts.
arch/arm64/kvm/sdei.c +123 -0

129 lines not on this page · raw message

Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#13 lore raw

PATCH v7 13/22 KVM: arm64: Support EVENT_{COMPLETE, COMPLETE_AND_RESUME} hypercall

In reply to Gavin Shan

This supports EVENT_{COMPLETE, COMPLETE_AND_RESUME} hypercall. The execution resumes from previously interrupted context when EVENT_COMPLETE hypercall is received.

However, the execution resumes from the specified address when EVENT_COMPLETE_AND_RESUME is received. In this case, context switches like below.

* x0 to x17 are restored from the interrupted context.

* SPSR_EL1 is set to PSTATE of the interrupted context.

* ELR_EL1 is set to PC of the interrupted context.

* PSTATE has nRW cleared, but D/A/I/F set.

* PC is set to the resume address, specified in the first argument of EVENT_COMPLETE_AND_RESUME hypercall.

---
 arch/arm64/kvm/sdei.c | 54 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)
arch/arm64/kvm/sdei.c +54 -0
index 36a72c1750fc..0bea1b2f9452 100644
@@ -85,6 +85,54 @@ static unsigned long event_context(struct kvm_vcpu *vcpu)
 	return ctxt->regs[param_id];
 }
 
+static void event_complete(struct kvm_vcpu *vcpu, bool resume)
+{
+	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
+	struct kvm_sdei_event_context *ctxt = &vsdei->ctxt;
+	unsigned long pstate, resume_addr = smccc_get_arg(vcpu, 1);
+	unsigned int num, i;
+
+	num = find_next_bit(&vsdei->running, KVM_NR_SDEI_EVENTS, 0);
+	if (num >= KVM_NR_SDEI_EVENTS)
+		return;
+
+	/* Restore registers: x0 -> x17 */
+	for (i = 0; i < ARRAY_SIZE(ctxt->regs); i++)
+		vcpu_set_reg(vcpu, i, ctxt->regs[i]);
+
+	/*
+	 * The registers are modified accordingly if the execution resumes
+	 * from the specified address.
+	 *
+	 * SPSR_EL1: PSTATE of the interrupted context
+	 * ELR_EL1:  PC of the interrupted context
+	 * PSTATE:   cleared nRW bit, but D/A/I/F bits are set
+	 * PC:       the resume address
+	 */
+	if (resume) {
+		if (has_vhe()) {
+			write_sysreg_el1(ctxt->pstate, SYS_SPSR);
+			write_sysreg_s(ctxt->pc, SYS_ELR_EL12);
+		} else {
+			__vcpu_sys_reg(vcpu, SPSR_EL1) = ctxt->pstate;
+			__vcpu_sys_reg(vcpu, ELR_EL1) = ctxt->pc;
+		}
+
+		pstate = ctxt->pstate;
+		pstate &= ~(PSR_MODE32_BIT | PSR_MODE_MASK);
+		pstate |= (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT |
+			   PSR_F_BIT | PSR_MODE_EL1h);
+		*vcpu_cpsr(vcpu) = pstate;
+		*vcpu_pc(vcpu) = resume_addr;
+	} else {
+		*vcpu_cpsr(vcpu) = ctxt->pstate;
+		*vcpu_pc(vcpu) = ctxt->pc;
+	}
+
+	/* Update event state */
+	clear_bit(num, &vsdei->running);
+}
+
 static unsigned long event_unregister(struct kvm_vcpu *vcpu)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -234,6 +282,12 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	case SDEI_1_0_FN_SDEI_EVENT_CONTEXT:
 		ret = event_context(vcpu);
 		break;
+	case SDEI_1_0_FN_SDEI_EVENT_COMPLETE:
+		event_complete(vcpu, false);
+		break;
+	case SDEI_1_0_FN_SDEI_EVENT_COMPLETE_AND_RESUME:
+		event_complete(vcpu, true);
+		break;
 	case SDEI_1_0_FN_SDEI_EVENT_UNREGISTER:
 		ret = event_unregister(vcpu);
 		break;
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#14 lore raw

PATCH v7 14/22 KVM: arm64: Support EVENT_SIGNAL hypercall

In reply to Gavin Shan

This supports EVENT_SIGNAL hypercall. It's used to inject the software signaled event, whose number is zero. The KVM_REQ_SDEI request is raised if the PE is unmasked and the event has been enabled.

---
 arch/arm64/kvm/sdei.c         | 28 ++++++++++++++++++++++++++++
 include/uapi/linux/arm_sdei.h |  3 +++
 2 files changed, 31 insertions(+)
arch/arm64/kvm/sdei.c +28 -0
index 0bea1b2f9452..a55797fce06a 100644
@@ -232,6 +232,31 @@ static unsigned long pe_mask(struct kvm_vcpu *vcpu, bool mask)
 	return SDEI_SUCCESS;
 }
 
+static unsigned long event_signal(struct kvm_vcpu *vcpu)
+{
+	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
+	unsigned int num = smccc_get_arg(vcpu, 1);
+
+	if (num >= KVM_NR_SDEI_EVENTS)
+		return SDEI_INVALID_PARAMETERS;
+
+	/*
+	 * The event must be software signaled event, whose
+	 * number is zero.
+	 */
+	if (num != SDEI_SW_SIGNALED_EVENT ||
+	    !test_bit(num, &vsdei->registered))
+		return SDEI_INVALID_PARAMETERS;
+
+	if (!test_and_set_bit(num, &vsdei->pending)) {
+		if (!(vcpu->arch.flags & KVM_ARM64_SDEI_MASKED) &&
+		    test_bit(num, &vsdei->enabled))
+			kvm_make_request(KVM_REQ_SDEI, vcpu);
+	}
+
+	return SDEI_SUCCESS;
+}
+
 static unsigned long event_reset(struct kvm_vcpu *vcpu, bool private)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -303,6 +328,9 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	case SDEI_1_0_FN_SDEI_PE_UNMASK:
 		ret = pe_mask(vcpu, false);
 		break;
+	case SDEI_1_1_FN_SDEI_EVENT_SIGNAL:
+		ret = event_signal(vcpu);
+		break;
 	case SDEI_1_0_FN_SDEI_PRIVATE_RESET:
 		ret = event_reset(vcpu, true);
 		break;
include/uapi/linux/arm_sdei.h +3 -0
index af0630ba5437..72b07114bbfa 100644
@@ -22,9 +22,12 @@
 #define SDEI_1_0_FN_SDEI_PE_UNMASK			SDEI_1_0_FN(0x0C)
 #define SDEI_1_0_FN_SDEI_INTERRUPT_BIND			SDEI_1_0_FN(0x0D)
 #define SDEI_1_0_FN_SDEI_INTERRUPT_RELEASE		SDEI_1_0_FN(0x0E)
+#define SDEI_1_1_FN_SDEI_EVENT_SIGNAL			SDEI_1_0_FN(0x0F)
 #define SDEI_1_0_FN_SDEI_PRIVATE_RESET			SDEI_1_0_FN(0x11)
 #define SDEI_1_0_FN_SDEI_SHARED_RESET			SDEI_1_0_FN(0x12)
 
+#define SDEI_SW_SIGNALED_EVENT		0
+
 #define SDEI_VERSION_MAJOR_SHIFT			48
 #define SDEI_VERSION_MAJOR_MASK				0x7fff
 #define SDEI_VERSION_MINOR_SHIFT			32
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#15 lore raw

PATCH v7 15/22 KVM: arm64: Support SDEI_FEATURES hypercall

In reply to Gavin Shan

This supports SDEI_FEATURES hypercall. It's used to retrieve the supported features, including number of slots for the interrupt binding events and relative mode for event handler. Currently, we simply return zero, meaning none of them is supported.

---
 arch/arm64/kvm/sdei.c         | 16 ++++++++++++++++
 include/uapi/linux/arm_sdei.h |  5 +++++
 2 files changed, 21 insertions(+)
arch/arm64/kvm/sdei.c +16 -0
index a55797fce06a..d295f84f6940 100644
@@ -277,6 +277,19 @@ static unsigned long event_reset(struct kvm_vcpu *vcpu, bool private)
 	return SDEI_SUCCESS;
 }
 
+static unsigned long sdei_features(struct kvm_vcpu *vcpu)
+{
+	unsigned int feature = smccc_get_arg(vcpu, 1);
+
+	switch (feature) {
+	case SDEI_FEATURE_BIND_SLOTS:
+	case SDEI_FEATURE_RELATIVE_MODE:
+		return 0;
+	}
+
+	return SDEI_INVALID_PARAMETERS;
+}
+
 int kvm_sdei_call(struct kvm_vcpu *vcpu)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -337,6 +350,9 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	case SDEI_1_0_FN_SDEI_SHARED_RESET:
 		ret = event_reset(vcpu, false);
 		break;
+	case SDEI_1_1_FN_SDEI_FEATURES:
+		ret = sdei_features(vcpu);
+		break;
 	default:
 		ret = SDEI_NOT_SUPPORTED;
 	}
include/uapi/linux/arm_sdei.h +5 -0
index 72b07114bbfa..5b783cc64617 100644
@@ -23,6 +23,7 @@
 #define SDEI_1_0_FN_SDEI_INTERRUPT_BIND			SDEI_1_0_FN(0x0D)
 #define SDEI_1_0_FN_SDEI_INTERRUPT_RELEASE		SDEI_1_0_FN(0x0E)
 #define SDEI_1_1_FN_SDEI_EVENT_SIGNAL			SDEI_1_0_FN(0x0F)
+#define SDEI_1_1_FN_SDEI_FEATURES			SDEI_1_0_FN(0x10)
 #define SDEI_1_0_FN_SDEI_PRIVATE_RESET			SDEI_1_0_FN(0x11)
 #define SDEI_1_0_FN_SDEI_SHARED_RESET			SDEI_1_0_FN(0x12)
 
@@ -73,4 +74,8 @@
 #define SDEI_EVENT_PRIORITY_NORMAL		0
 #define SDEI_EVENT_PRIORITY_CRITICAL		1
 
+/* SDEI_FEATURES */
+#define SDEI_FEATURE_BIND_SLOTS			0
+#define SDEI_FEATURE_RELATIVE_MODE		1
+
 #endif /* _UAPI_LINUX_ARM_SDEI_H */
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#16 lore raw

PATCH v7 16/22 KVM: arm64: Support SDEI_VERSION hypercall

In reply to Gavin Shan

This supports SDEI_VERSION hypercall by returning v1.1, which is the specification version we're following. The vendor is set to one of the values returned from ARM_SMCCC_VENDOR_HYP_CALL_UID hypercall.

---
 arch/arm64/include/asm/kvm_sdei.h |  3 +++
 arch/arm64/kvm/sdei.c             | 11 +++++++++++
 2 files changed, 14 insertions(+)
arch/arm64/include/asm/kvm_sdei.h +3 -0
index 735d9ac1a5a2..a1e960943515 100644
@@ -62,6 +62,9 @@ struct kvm_sdei_vcpu {
 	struct kvm_sdei_event_context	ctxt;
 };
 
+/* Returned as vendor through SDEI_VERSION hypercall */
+#define KVM_SDEI_VENDOR	ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_2
+
 /* APIs */
 int kvm_sdei_call(struct kvm_vcpu *vcpu);
 int kvm_sdei_inject_event(struct kvm_vcpu *vcpu,
arch/arm64/kvm/sdei.c +11 -0
index d295f84f6940..f95b9bcce13c 100644
@@ -13,6 +13,14 @@
 #include <kvm/arm_hypercalls.h>
 #include <asm/kvm_sdei.h>
 
+static unsigned long sdei_version(struct kvm_vcpu *vcpu)
+{
+	/* v1.1 and vendor ID */
+	return (1UL << SDEI_VERSION_MAJOR_SHIFT) |
+	       (1UL << SDEI_VERSION_MINOR_SHIFT) |
+	       KVM_SDEI_VENDOR;
+}
+
 static unsigned long event_register(struct kvm_vcpu *vcpu)
 {
 	struct kvm_sdei_vcpu *vsdei = vcpu->arch.sdei;
@@ -308,6 +316,9 @@ int kvm_sdei_call(struct kvm_vcpu *vcpu)
 	}
 
 	switch (func) {
+	case SDEI_1_0_FN_SDEI_VERSION:
+		ret = sdei_version(vcpu);
+		break;
 	case SDEI_1_0_FN_SDEI_EVENT_REGISTER:
 		ret = event_register(vcpu);
 		break;
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#17 lore raw

PATCH v7 17/22 KVM: arm64: Expose SDEI capbility and service

In reply to Gavin Shan

The SDEI functionality is ready and lets exposes the capability and service.

* KVM_CAP_ARM_SDEI

* KVM_REG_ARM_STD_BIT_SDEI in pseudo-firmware bitmap register KVM_REG_ARM_STD_BMAP

* The events for software signaled one (0x0) and Async PF (0x1) are exposed.

---
 Documentation/virt/kvm/api.rst            | 11 +++++++++++
 Documentation/virt/kvm/arm/hypercalls.rst |  4 ++++
 arch/arm64/include/asm/kvm_sdei.h         |  2 ++
 arch/arm64/include/uapi/asm/kvm.h         |  1 +
 arch/arm64/kvm/arm.c                      |  1 +
 arch/arm64/kvm/hypercalls.c               |  4 ++++
 include/uapi/linux/kvm.h                  |  1 +
 7 files changed, 24 insertions(+)
Documentation/virt/kvm/api.rst +11 -0
index 11e00a46c610..011ec0987d71 100644
@@ -7956,6 +7956,17 @@ should adjust CPUID leaf 0xA to reflect that the PMU is disabled.
 When enabled, KVM will exit to userspace with KVM_EXIT_SYSTEM_EVENT of
 type KVM_SYSTEM_EVENT_SUSPEND to process the guest suspend request.
 
+8.37 KVM_CAP_ARM_SDEI
+---------------------
+
+:Capability: KVM_CAP_ARM_SDEI
+:Architectures: arm64
+:Type: vm
+
+This capability indicates that the SDEI virtual service is supported
+in the host. A VMM can check whether the service is available to enable
+it.
+
 9. Known KVM API problems
 =========================
 
Documentation/virt/kvm/arm/hypercalls.rst +4 -0
index 3e23084644ba..8b8159988934 100644
@@ -106,6 +106,10 @@ The pseudo-firmware bitmap register are as follows:
       The bit represents the services offered under v1.0 of ARM True Random
       Number Generator (TRNG) specification, ARM DEN0098.
 
+    Bit-1: KVM_REG_ARM_STD_BIT_SDEI:
+      The bit represents the services offered under v1.1 of ARM Software
+      Delegated Exception Interface (SDEI) specification, ARM DEN0054C.
+
 * KVM_REG_ARM_STD_HYP_BMAP:
     Controls the bitmap of the ARM Standard Hypervisor Service Calls.
 
arch/arm64/include/asm/kvm_sdei.h +2 -0
index a1e960943515..a68d40137a88 100644
@@ -15,6 +15,8 @@
 #include <linux/bits.h>
 
 enum {
+	KVM_SDEI_EVENT_SW_SIGNALED = 0,
+	KVM_SDEI_EVENT_ASYNC_PF,
 	KVM_NR_SDEI_EVENTS,
 };
 
arch/arm64/include/uapi/asm/kvm.h +1 -0
index 3bb134355874..cc3251381960 100644
@@ -344,6 +344,7 @@ struct kvm_arm_copy_mte_tags {
 
 enum {
 	KVM_REG_ARM_STD_BIT_TRNG_V1_0	= 0,
+	KVM_REG_ARM_STD_BIT_SDEI,
 #ifdef __KERNEL__
 	KVM_REG_ARM_STD_BMAP_BIT_COUNT,
 #endif
arch/arm64/kvm/arm.c +1 -0
index 06cb5e38634e..15ead92ad26d 100644
@@ -217,6 +217,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_VCPU_ATTRIBUTES:
 	case KVM_CAP_PTP_KVM:
 	case KVM_CAP_ARM_SYSTEM_SUSPEND:
+	case KVM_CAP_ARM_SDEI:
 		r = 1;
 		break;
 	case KVM_CAP_SET_GUEST_DEBUG2:
arch/arm64/kvm/hypercalls.c +4 -0
index 88561ac47301..aaa5197ffe83 100644
@@ -99,6 +99,10 @@ static bool kvm_hvc_call_allowed(struct kvm_vcpu *vcpu, u32 func)
 	struct kvm_smccc_features *smccc_feat = &vcpu->kvm->arch.smccc_feat;
 
 	switch (func) {
+	case SDEI_1_0_FN(ARM_SMCCC_STANDARD_SDEI_START - 0x20) ...
+	     SDEI_1_0_FN(ARM_SMCCC_STANDARD_SDEI_END - 0x20):
+		return test_bit(KVM_REG_ARM_STD_BIT_SDEI,
+				&smccc_feat->std_bmap);
 	case ARM_SMCCC_TRNG_VERSION:
 	case ARM_SMCCC_TRNG_FEATURES:
 	case ARM_SMCCC_TRNG_GET_UUID:
include/uapi/linux/kvm.h +1 -0
index 5088bd9f1922..65622bcaeec3 100644
@@ -1157,6 +1157,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_VM_TSC_CONTROL 214
 #define KVM_CAP_SYSTEM_EVENT_DATA 215
 #define KVM_CAP_ARM_SYSTEM_SUSPEND 216
+#define KVM_CAP_ARM_SDEI 217
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#18 lore raw

PATCH v7 18/22 KVM: arm64: Allow large sized pseudo firmware registers

In reply to Gavin Shan

Currently, the sizes of all pseudo firmware registers are 64 bits. The value from user space is copied to a local 64-bits variable, which is passed on in kvm_arm_set_fw_reg(). It works perfectly until large sized pseudo firmware registers are added for SDEI event handlers, states and context.

This copies the user space's value to local 64-bits variable and passes the local variable on, only if the pseudo firmware register is 64-bits in size. No functional change intended.

---
 arch/arm64/kvm/hypercalls.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)
arch/arm64/kvm/hypercalls.c +17 -6
index aaa5197ffe83..8e0df54d1422 100644
@@ -406,14 +406,20 @@ int kvm_arm_get_fw_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
 	return 0;
 }
 
-static int kvm_arm_set_fw_reg_bmap(struct kvm_vcpu *vcpu, u64 reg_id, u64 val)
+static int kvm_arm_set_fw_reg_bmap(struct kvm_vcpu *vcpu,
+				   const struct kvm_one_reg *reg)
 {
 	int ret = 0;
 	struct kvm *kvm = vcpu->kvm;
 	struct kvm_smccc_features *smccc_feat = &kvm->arch.smccc_feat;
 	unsigned long *fw_reg_bmap, fw_reg_features;
+	void __user *uaddr = (void __user *)(long)reg->addr;
+	u64 val;
 
-	switch (reg_id) {
+	if (copy_from_user(&val, uaddr, KVM_REG_SIZE(reg->id)))
+		return -EFAULT;
+
+	switch (reg->id) {
 	case KVM_REG_ARM_STD_BMAP:
 		fw_reg_bmap = &smccc_feat->std_bmap;
 		fw_reg_features = KVM_ARM_SMCCC_STD_FEATURES;
@@ -454,15 +460,14 @@ int kvm_arm_set_fw_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
 	u64 val;
 	int wa_level;
 
-	if (copy_from_user(&val, uaddr, KVM_REG_SIZE(reg->id)))
-		return -EFAULT;
-
 	switch (reg->id) {
 	case KVM_REG_ARM_PSCI_VERSION:
 	{
 		bool wants_02;
 
 		wants_02 = test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features);
+		if (copy_from_user(&val, uaddr, KVM_REG_SIZE(reg->id)))
+			return -EFAULT;
 
 		switch (val) {
 		case KVM_ARM_PSCI_0_1:
@@ -483,6 +488,9 @@ int kvm_arm_set_fw_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
 
 	case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1:
 	case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3:
+		if (copy_from_user(&val, uaddr, KVM_REG_SIZE(reg->id)))
+			return -EFAULT;
+
 		if (val & ~KVM_REG_FEATURE_LEVEL_MASK)
 			return -EINVAL;
 
@@ -492,6 +500,9 @@ int kvm_arm_set_fw_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
 		return 0;
 
 	case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2:
+		if (copy_from_user(&val, uaddr, KVM_REG_SIZE(reg->id)))
+			return -EFAULT;
+
 		if (val & ~(KVM_REG_FEATURE_LEVEL_MASK |
 			    KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_ENABLED))
 			return -EINVAL;
@@ -529,7 +540,7 @@ int kvm_arm_set_fw_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
 	case KVM_REG_ARM_STD_BMAP:
 	case KVM_REG_ARM_STD_HYP_BMAP:
 	case KVM_REG_ARM_VENDOR_HYP_BMAP:
-		return kvm_arm_set_fw_reg_bmap(vcpu, reg->id, val);
+		return kvm_arm_set_fw_reg_bmap(vcpu, reg);
 	default:
 		return -ENOENT;
 	}
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#19 lore raw

PATCH v7 19/22 KVM: arm64: Support SDEI event migration

In reply to Gavin Shan

This supports migration for SDEI event handlers, states and context. Several pseudo firmware registers are added to assist the migration work.

* KVM_REG_ARM_SDEI_EVENT_HANDLER_0
KVM_REG_ARM_SDEI_EVENT_HANDLER_1
KVM_REG_ARM_SDEI_EVENT_HANDLER_2
KVM_REG_ARM_SDEI_EVENT_HANDLER_3

128-bits in length. They're mapped to the handler's address and argument for Software Signaled event and Async PF event. Additinal two events are reserved for future needs without too much considerations to the compatible issue. Not too much bandwidth needed to migrate those two additional registers.

* KVM_REG_ARM_SDEI_EVENT_REGISTERED
KVM_REG_ARM_SDEI_EVENT_ENABLED
KVM_REG_ARM_SDEI_EVENT_RUNNING
KVM_REG_ARM_SDEI_EVENT_PENDING

64-bits in length. They're mapped to registered, enabled, running and pending bitmap.

* KVM_REG_ARM_SDEI_EVENT_CONTEXT

2048-bits in length. It's mapped to saved or interrupted context.

* KVM_REG_ARM_SDEI_PE_STATE

64-bits in length. It's mapped to PE's states, which is masked or unmasked.

---
 arch/arm64/include/asm/kvm_sdei.h |   2 +
 arch/arm64/include/uapi/asm/kvm.h |  17 ++++
 arch/arm64/kvm/hypercalls.c       |  32 ++++++
 arch/arm64/kvm/sdei.c             | 161 ++++++++++++++++++++++++++++++
 4 files changed, 212 insertions(+)
arch/arm64/include/asm/kvm_sdei.h +2 -0
index a68d40137a88..d11964b88b58 100644
@@ -73,6 +73,8 @@ int kvm_sdei_inject_event(struct kvm_vcpu *vcpu,
 			  unsigned int num, bool immediate);
 int kvm_sdei_cancel_event(struct kvm_vcpu *vcpu, unsigned int num);
 void kvm_sdei_deliver_event(struct kvm_vcpu *vcpu);
+int kvm_sdei_get_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg);
+int kvm_sdei_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg);
 void kvm_sdei_create_vcpu(struct kvm_vcpu *vcpu);
 void kvm_sdei_destroy_vcpu(struct kvm_vcpu *vcpu);
 
arch/arm64/include/uapi/asm/kvm.h +17 -0
index cc3251381960..90247ce8de59 100644
@@ -262,6 +262,11 @@ struct kvm_arm_copy_mte_tags {
 #define KVM_REG_ARM_FW			(0x0014 << KVM_REG_ARM_COPROC_SHIFT)
 #define KVM_REG_ARM_FW_REG(r)		(KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
 					 KVM_REG_ARM_FW | ((r) & 0xffff))
+#define KVM_REG_ARM_FW_REG_128(r)	(KVM_REG_ARM64 | KVM_REG_SIZE_U128 | \
+					 KVM_REG_ARM_FW | ((r) & 0xffff))
+#define KVM_REG_ARM_FW_REG_2048(r)	(KVM_REG_ARM64 | KVM_REG_SIZE_U2048 | \
+					 KVM_REG_ARM_FW | ((r) & 0xffff))
+
 #define KVM_REG_ARM_PSCI_VERSION	KVM_REG_ARM_FW_REG(0)
 #define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1	KVM_REG_ARM_FW_REG(1)
 #define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1_NOT_AVAIL		0
@@ -288,6 +293,18 @@ struct kvm_arm_copy_mte_tags {
 #define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_AVAIL		1
 #define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_REQUIRED	2
 
+/* SDEI registers */
+#define KVM_REG_ARM_SDEI_EVENT_HANDLER_0	KVM_REG_ARM_FW_REG_128(4)
+#define KVM_REG_ARM_SDEI_EVENT_HANDLER_1	KVM_REG_ARM_FW_REG_128(5)
+#define KVM_REG_ARM_SDEI_EVENT_HANDLER_2	KVM_REG_ARM_FW_REG_128(6)
+#define KVM_REG_ARM_SDEI_EVENT_HANDLER_3	KVM_REG_ARM_FW_REG_128(7)
+#define KVM_REG_ARM_SDEI_EVENT_REGISTERED	KVM_REG_ARM_FW_REG(8)
+#define KVM_REG_ARM_SDEI_EVENT_ENABLED		KVM_REG_ARM_FW_REG(9)
+#define KVM_REG_ARM_SDEI_EVENT_RUNNING		KVM_REG_ARM_FW_REG(10)
+#define KVM_REG_ARM_SDEI_EVENT_PENDING		KVM_REG_ARM_FW_REG(11)
+#define KVM_REG_ARM_SDEI_EVENT_CONTEXT		KVM_REG_ARM_FW_REG_2048(12)
+#define KVM_REG_ARM_SDEI_PE_STATE		KVM_REG_ARM_FW_REG(13)
+
 /* SVE registers */
 #define KVM_REG_ARM64_SVE		(0x15 << KVM_REG_ARM_COPROC_SHIFT)
 
arch/arm64/kvm/hypercalls.c +32 -0
index 8e0df54d1422..d330aee968a9 100644
@@ -290,6 +290,16 @@ static const u64 kvm_arm_fw_reg_ids[] = {
 	KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1,
 	KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2,
 	KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3,
+	KVM_REG_ARM_SDEI_EVENT_HANDLER_0,
+	KVM_REG_ARM_SDEI_EVENT_HANDLER_1,
+	KVM_REG_ARM_SDEI_EVENT_HANDLER_2,
+	KVM_REG_ARM_SDEI_EVENT_HANDLER_3,
+	KVM_REG_ARM_SDEI_EVENT_REGISTERED,
+	KVM_REG_ARM_SDEI_EVENT_ENABLED,
+	KVM_REG_ARM_SDEI_EVENT_RUNNING,
+	KVM_REG_ARM_SDEI_EVENT_PENDING,
+	KVM_REG_ARM_SDEI_EVENT_CONTEXT,
+	KVM_REG_ARM_SDEI_PE_STATE,
 	KVM_REG_ARM_STD_BMAP,
 	KVM_REG_ARM_STD_HYP_BMAP,
 	KVM_REG_ARM_VENDOR_HYP_BMAP,
@@ -387,6 +397,17 @@ int kvm_arm_get_fw_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
 	case KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3:
 		val = get_kernel_wa_level(reg->id) & KVM_REG_FEATURE_LEVEL_MASK;
 		break;
+	case KVM_REG_ARM_SDEI_EVENT_HANDLER_0:
+	case KVM_REG_ARM_SDEI_EVENT_HANDLER_1:
+	case KVM_REG_ARM_SDEI_EVENT_HANDLER_2:
+	case KVM_REG_ARM_SDEI_EVENT_HANDLER_3:
+	case KVM_REG_ARM_SDEI_EVENT_REGISTERED:
+	case KVM_REG_ARM_SDEI_EVENT_ENABLED:
+	case KVM_REG_ARM_SDEI_EVENT_RUNNING:
+	case KVM_REG_ARM_SDEI_EVENT_PENDING:
+	case KVM_REG_ARM_SDEI_EVENT_CONTEXT:
+	case KVM_REG_ARM_SDEI_PE_STATE:
+		return kvm_sdei_get_reg(vcpu, reg);
 	case KVM_REG_ARM_STD_BMAP:
 		val = READ_ONCE(smccc_feat->std_bmap);
 		break;
@@ -537,6 +558,17 @@ int kvm_arm_set_fw_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
 			return -EINVAL;
 
 		return 0;
+	case KVM_REG_ARM_SDEI_EVENT_HANDLER_0:
+	case KVM_REG_ARM_SDEI_EVENT_HANDLER_1:
+	case KVM_REG_ARM_SDEI_EVENT_HANDLER_2:
+	case KVM_REG_ARM_SDEI_EVENT_HANDLER_3:
+	case KVM_REG_ARM_SDEI_EVENT_REGISTERED:
+	case KVM_REG_ARM_SDEI_EVENT_ENABLED:
+	case KVM_REG_ARM_SDEI_EVENT_RUNNING:
+	case KVM_REG_ARM_SDEI_EVENT_PENDING:
+	case KVM_REG_ARM_SDEI_EVENT_CONTEXT:
+	case KVM_REG_ARM_SDEI_PE_STATE:
+		return kvm_sdei_set_reg(vcpu, reg);
 	case KVM_REG_ARM_STD_BMAP:
 	case KVM_REG_ARM_STD_HYP_BMAP:
 	case KVM_REG_ARM_VENDOR_HYP_BMAP:
arch/arm64/kvm/sdei.c +161 -0

167 lines not on this page · raw message

Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#20 lore raw

PATCH v7 20/22 KVM: arm64: Add SDEI document

In reply to Gavin Shan

This adds SDEI document, to explain what has been missed from the specification and the pesudo firmware registers used for migration.

---
 Documentation/virt/kvm/arm/sdei.rst | 64 +++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
 create mode 100644 Documentation/virt/kvm/arm/sdei.rst
Documentation/virt/kvm/arm/sdei.rst new file +64 -0
new file mode 100644
index 000000000000..ba1539978b2c
@@ -0,0 +1,64 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=====================================
+SDEI Virtualization Support for ARM64
+=====================================
+
+ARM specification DEN0054/C defines Software Delegated Exception Interface
+(SDEI). It provides a mechanism for registering and servicing system events
+from system firmware. The interface is offered by a higher exception level
+to a lower exception level, in other words by a secure platform firmware
+to hypervisor or hypervisor to OS or both.
+
+https://developer.arm.com/documentation/den0054/c
+
+KVM/arm64 implementation follows the specification to support the defined
+hypercalls so that the system events can be registered and serviced from
+KVM hypervisor to the guest OS. However, some of specified functionalities
+are missed by the implementation.
+
+  * The shared event is not supported. It means all supported events are
+    private. They need to be registered, enabled, disabled, unregistered
+    and reset from one particular PE (vCPU).
+
+  * The critical priority is not supported. It means all supported events
+    have normal priority. So there is no preemption between two events
+    in the critical and normal priorities. One event can be running at
+    once on one particular PE (vCPU).
+
+  * Interrupt binding event is not supported. It means all supported
+    events triggered by software.
+
+  * Relative mode for the event handler's address is not supported.
+    The event handler address is always an absolute address.
+
+The event handlers, states and context need to be migrated. Several pseudo
+firmware registers are added for this.
+
+  * KVM_REG_ARM_SDEI_EVENT_HANDLER_0
+    KVM_REG_ARM_SDEI_EVENT_HANDLER_1
+    KVM_REG_ARM_SDEI_EVENT_HANDLER_2
+    KVM_REG_ARM_SDEI_EVENT_HANDLER_3
+
+    The event handler's address and argument for events, whose number
+    are 0, 1, 2 and 3 respectively. Currently, there are only two
+    supported events, which are Software Signaled Event (0) and Async
+    PF Event (1). So the first two registers are only used. zeroes are
+    returned on reading KVM_REG_ARM_SDEI_EVENT_HANDLER_{2, 3}, and the
+    values written to them are ignored.
+
+  * KVM_REG_ARM_SDEI_EVENT_REGISTERED
+    KVM_REG_ARM_SDEI_EVENT_ENABLED
+    KVM_REG_ARM_SDEI_EVENT_RUNNING
+    KVM_REG_ARM_SDEI_EVENT_PENDING
+
+    They are mapped to the registered, enabled, running and pending
+    bitmap respectively.
+
+  * KVM_REG_ARM_SDEI_EVENT_CONTEXT
+
+    The interrupted context.
+
+  * KVM_REG_ARM_SDEI_PE_STATE
+
+    Relect the SDEI masked state on the PE (vCPU).
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#21 lore raw

PATCH v7 21/22 selftests: KVM: aarch64: Add SDEI case in hypercall tests

In reply to Gavin Shan

This adds SDEI case in hypercall tests. The added hypercalls are issued to mimic what guest kernel does: reset the shared and private events, unmask PE, register/enable/disable/unregister event, and then mask PE.

---
 tools/testing/selftests/kvm/aarch64/hypercalls.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/aarch64/hypercalls.c +10 -1
index 41e0210b7a5e..1377d2ec8d4b 100644
@@ -11,6 +11,7 @@
 
 #include <errno.h>
 #include <linux/arm-smccc.h>
+#include <linux/arm_sdei.h>
 #include <asm/kvm.h>
 #include <kvm_util.h>
 
@@ -19,7 +20,7 @@
 #define FW_REG_ULIMIT_VAL(max_feat_bit) (GENMASK(max_feat_bit, 0))
 
 /* Last valid bits of the bitmapped firmware registers */
-#define KVM_REG_ARM_STD_BMAP_BIT_MAX		0
+#define KVM_REG_ARM_STD_BMAP_BIT_MAX		1
 #define KVM_REG_ARM_STD_HYP_BMAP_BIT_MAX	0
 #define KVM_REG_ARM_VENDOR_HYP_BMAP_BIT_MAX	1
 
@@ -68,6 +69,14 @@ static const struct test_hvc_info hvc_info[] = {
 	TEST_HVC_INFO(ARM_SMCCC_TRNG_GET_UUID, 0),
 	TEST_HVC_INFO(ARM_SMCCC_TRNG_RND32, 0),
 	TEST_HVC_INFO(ARM_SMCCC_TRNG_RND64, 0),
+	TEST_HVC_INFO(SDEI_1_0_FN_SDEI_SHARED_RESET, 0),
+	TEST_HVC_INFO(SDEI_1_0_FN_SDEI_PRIVATE_RESET, 0),
+	TEST_HVC_INFO(SDEI_1_0_FN_SDEI_PE_UNMASK, 0),
+	TEST_HVC_INFO(SDEI_1_0_FN_SDEI_EVENT_REGISTER, SDEI_SW_SIGNALED_EVENT),
+	TEST_HVC_INFO(SDEI_1_0_FN_SDEI_EVENT_ENABLE, SDEI_SW_SIGNALED_EVENT),
+	TEST_HVC_INFO(SDEI_1_0_FN_SDEI_EVENT_DISABLE, SDEI_SW_SIGNALED_EVENT),
+	TEST_HVC_INFO(SDEI_1_0_FN_SDEI_EVENT_UNREGISTER, SDEI_SW_SIGNALED_EVENT),
+	TEST_HVC_INFO(SDEI_1_0_FN_SDEI_PE_MASK, 0),
 
 	/* KVM_REG_ARM_STD_HYP_BMAP */
 	TEST_HVC_INFO(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, ARM_SMCCC_HV_PV_TIME_FEATURES),
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#22 lore raw

PATCH v7 22/22 selftests: KVM: aarch64: Add SDEI test case

In reply to Gavin Shan

This adds SDEI selftest case where the various hypercalls are issued to the software signaled event, whose number is zero. The event is private, signaled and in normal priority.

By default, two vCPUs are started and the following ioctl commands or hypercalls are sent to them in sequence, to simulate how they are used in VMM and the linux guest:

kvm_check_cap(KVM_CAP_ARM_SDEI) (Available functionality)

SDEI_1_0_FN_SDEI_VERSION

   SDEI_1_1_FN_SDEI_FEATURES           (SDEI capability probing)
   SDEI_1_0_FN_SDEI_SHARED_RESET       (restart SDEI)
   SDEI_1_0_FN_SDEI_PE_UNMASK          (CPU online)

SDEI_1_0_FN_SDEI_EVENT_GET_INFO

   SDEI_1_0_FN_SDEI_EVENT_REGISTER     (register event)
   SDEI_1_0_FN_SDEI_EVENT_ENABLE       (enable event)
   SDEI_1_1_FN_SDEI_EVENT_SIGNAL       (event injection)
   SDEI_1_0_FN_SDEI_EVENT_DISABLE      (disable event)
   SDEI_1_0_FN_SDEI_EVENT_UNREGISTER   (unregister event)
   SDEI_1_0_FN_SDEI_PE_MASK            (CPU offline)
---
 tools/testing/selftests/kvm/Makefile       |   1 +
 tools/testing/selftests/kvm/aarch64/sdei.c | 450 +++++++++++++++++++++
 2 files changed, 451 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/aarch64/sdei.c
tools/testing/selftests/kvm/Makefile +1 -0
index 81470a99ed1c..39b976a6568f 100644
@@ -109,6 +109,7 @@ TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
 TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
 TEST_GEN_PROGS_aarch64 += aarch64/hypercalls
 TEST_GEN_PROGS_aarch64 += aarch64/psci_test
+TEST_GEN_PROGS_aarch64 += aarch64/sdei
 TEST_GEN_PROGS_aarch64 += aarch64/vcpu_width_config
 TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
 TEST_GEN_PROGS_aarch64 += aarch64/vgic_irq
tools/testing/selftests/kvm/aarch64/sdei.c new file +450 -0

450 lines not on this page · raw message

Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Tested-by Huang Shijie on the v7 series
#23 lore raw
In reply to Gavin Shan

Hi Gavin,

Quoted 158 lines · Gavin Shan

[EXTERNAL EMAIL NOTICE: This email originated from an external sender. Please be mindful of safe email handling and proprietary information protection practices.]

This series intends to virtualize Software Delegated Exception Interface (SDEI), which is defined by DEN0054C (v1.1). It allows the hypervisor to deliver NMI-alike SDEI event to guest and it's needed by Async PF to deliver page-not-present notification from hypervisor to guest. The code and the required qemu changes can be found from:

    https://developer.arm.com/documentation/den0054/c
    https://github.com/gwshan/linux    ("kvm/arm64_sdei")
    https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

The design is quite strightforward by following the specification. The (SDEI) events are classified into the shared and private ones according to their scope. The shared event is system or VM scoped, but the private event is vcpu scoped. This implementation doesn't support the shared event because all the needed events are private. Besides, the critial events aren't supported by the implementation either. It means all events are normal in terms of priority.

There are several objects (data structures) introduced to help on the event registration, enablement, disablement, unregistration, reset, delivery and handling.

* kvm_sdei_event_handler
SDEI event handler, which is provided through EVENT_REGISTER hypercall, is called when the SDEI event is delivered from host to guest.

* kvm_sdei_event_context
The saved (preempted) context when SDEI event is delivered for handling.

* kvm_sdei_vcpu
SDEI events and their states.

The patches are organized as below:

PATCH[01-02] Preparatory work to extend smccc_get_argx() and refactor
hypercall routing mechanism
PATCH[03] Adds SDEI virtualization infrastructure
PATCH[04-16] Supports various SDEI hypercalls and event handling
PATCH[17] Exposes SDEI capability
PATCH[18-19] Support SDEI migration
PATCH[20] Adds document about SDEI
PATCH[21-22] SDEI related selftest cases

The previous revisions can be found:

   v6: https://lore.kernel.org/lkml/[email protected]/T/
   v5: https://lore.kernel.org/kvmarm/[email protected]/
   v4: https://lore.kernel.org/kvmarm/[email protected]/
   v3: https://lore.kernel.org/kvmarm/[email protected]/
   v2: https://lore.kernel.org/kvmarm/[email protected]/
   v1: https://lore.kernel.org/kvmarm/[email protected]/

Testing
=======
[1] The selftest case included in this series works fine. The default SDEI

     event, whose number is zero, can be registered, enabled, raised. The
     SDEI event handler can be invoked.

     [host]# pwd
     /home/gavin/sandbox/linux.main/tools/testing/selftests/kvm
     [root@virtlab-arm01 kvm]# ./aarch64/sdei

         NR_VCPUS: 2    SDEI Event: 0x00000000

     --- VERSION
         Version:              1.1 (vendor: 0x4b564d)
     --- FEATURES
         Shared event slots:   0
         Private event slots:  0
         Relative mode:        No
     --- PRIVATE_RESET
     --- SHARED_RESET
     --- PE_UNMASK
     --- EVENT_GET_INFO
         Type:                 Private
         Priority:             Normal
         Signaled:             Yes
     --- EVENT_REGISTER
     --- EVENT_ENABLE
     --- EVENT_SIGNAL
         Handled:              Yes
         IRQ:                  No
         Status:               Registered-Enabled-Running
         PC/PSTATE:            000000000040232c 00000000600003c5
         Regs:                 0000000000000000 0000000000000000
                               0000000000000000 0000000000000000
     --- PE_MASK
     --- EVENT_DISABLE
     --- EVENT_UNREGISTER

         Result: OK

[2] There are additional patches in the following repositories to create

     procfs entries, allowing to inject SDEI event from host side. The
     SDEI client in the guest side registers the SDEI default event, whose
     number is zero. Also, the QEMU exports SDEI ACPI table and supports
     migration for SDEI.

     https://github.com/gwshan/linux    ("kvm/arm64_sdei")
     https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

     [2.1] Start the guests and migrate the source VM to the destination
           VM.

     [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
             -accel kvm -machine virt,gic-version=host                     \
             -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
             -m 1024M,slots=16,maxmem=64G                                  \
                :                                                          \
             -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
             -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
             -append earlycon=pl011,mmio,0x9000000                         \
                :

     [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
             -accel kvm -machine virt,gic-version=host                     \
             -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
             -m 1024M,slots=16,maxmem=64G                                  \
                :                                                          \
             -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
             -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
             -append earlycon=pl011,mmio,0x9000000                         \
             -incoming tcp:0:4444                                          \
                :

     [2.2] Check kernel log on the source VM. The SDEI service is enabled
           and the default SDEI event (0x0) is enabled.

      [guest-src]# dmesg | grep -i sdei
      ACPI: SDEI 0x000000005BC80000 000024 \
                 (v00 BOCHS  BXPC     00000001 BXPC 00000001)
      sdei: SDEIv1.1 (0x4b564d) detected in firmware.
      SDEI TEST: Version 1.1, Vendor 0x4b564d
      sdei_init: SDEI event (0x0) registered
      sdei_init: SDEI event (0x0) enabled


      (qemu) migrate -d tcp:localhost:4444

     [2.3] Migrate the source VM to the destination VM. Inject SDEI event
           to the destination VM. The event is raised and handled.

     (qemu) migrate -d tcp:localhost:4444

     [host]# echo 0 > /proc/kvm/kvm-5360/vcpu-1

     [guest-dst]#
     =========== SDEI Event (CPU#1) ===========
     Event: 0000000000000000  Parameter: 00000000dabfdabf
     PC:    ffff800008cbb554  PSTATE: 00000000604000c5  SP: ffff800009c7bde0
     Regs:    00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001
              ffff800016c28000 0000000000000000 0000000000000000 0000000000000000
              0000000000000000 0000000000000000 0000000000000000 0000000000000000
Quoted 8 lines
              0000000000000000 0000000000000000 0000000000000000 0000000000000000
              0000000000000000 0000000000000000 0000000000000000 ffff800009399008
              ffff8000097d9af0 ffff8000097d99f8 ffff8000093a8db8 ffff8000097d9b18
              0000000000000000 0000000000000000 ffff000000339d00 0000000000000000
              0000000000000000 ffff800009c7bde0 ffff800008cbb5c4
     Context: 00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001
              ffff800016c28000 03ffffffffffffff 000000024325db59 ffff8000097de190
              ffff00000033a790 ffff800008cbb814 0000000000000a30 0000000000000000

I tested this patch set. It's okay.

Thanks

Huang Shijie

Quoted 78 lines

Changelog
=========
v7:
* Rebased to v5.19.rc1 (Gavin)
* Add hypercall ranges for routing (Oliver)
* Remove support to the critical event and redesigned data structures. Function names are also modified
as Oliver suggested (Oliver)
* Deliver event when it's enabled or the specific PE
is unmasked (Oliver)
* Improve EVENT_COMPLETE_AND_RESUME hypercall to resume
from the specified address (Oliver)
* Add patches for SDEI migration and documentation (Gavin)
* Misc comments from Oliver Upon (Oliver)
v6:
* Rebased to v5.18.rc1 (Gavin)
* Pass additional argument to smccc_get_arg() (Oliver)
* Add preparatory patch to route hypercalls based on their
owners (Oliver)
* Remove the support for shared event. (Oliver/Gavin)
* Remove the support for migration and add-on patches to
support it in future (Oliver)
* The events are exposed by KVM instead of VMM (Oliver)
* kvm_sdei_state.h is dropped and all the structures are
folded into the corresponding ones in kvm_sdei.h (Oliver)
* Rename 'struct kvm_sdei_registered_event' to
'struct kvm_sdei_event' (Oliver)
* Misc comments from Oliver Upon (Oliver)
v5/v4/v3/v2/v1:
* Skipped here and please visit the history by
https://lore.kernel.org/lkml/[email protected]/T/

Gavin Shan (22):
KVM: arm64: Extend smccc_get_argx()
KVM: arm64: Route hypercalls based on their owner
KVM: arm64: Add SDEI virtualization infrastructure
KVM: arm64: Support EVENT_REGISTER hypercall
KVM: arm64: Support EVENT_{ENABLE, DISABLE} hypercall
KVM: arm64: Support EVENT_CONTEXT hypercall
KVM: arm64: Support EVENT_UNREGISTER hypercall
KVM: arm64: Support EVENT_STATUS hypercall
KVM: arm64: Support EVENT_GET_INFO hypercall
KVM: arm64: Support PE_{MASK, UNMASK} hypercall
KVM: arm64: Support {PRIVATE, SHARED}_RESET hypercall
KVM: arm64: Support event injection and delivery
KVM: arm64: Support EVENT_{COMPLETE, COMPLETE_AND_RESUME} hypercall
KVM: arm64: Support EVENT_SIGNAL hypercall
KVM: arm64: Support SDEI_FEATURES hypercall
KVM: arm64: Support SDEI_VERSION hypercall
KVM: arm64: Expose SDEI capbility and service
KVM: arm64: Allow large sized pseudo firmware registers
KVM: arm64: Support SDEI event migration
KVM: arm64: Add SDEI document
selftests: KVM: aarch64: Add SDEI case in hypercall tests
selftests: KVM: aarch64: Add SDEI test case

  Documentation/virt/kvm/api.rst                |  11 +
  Documentation/virt/kvm/arm/hypercalls.rst     |   4 +
  Documentation/virt/kvm/arm/sdei.rst           |  64 ++
  arch/arm64/include/asm/kvm_host.h             |   3 +
  arch/arm64/include/asm/kvm_sdei.h             |  81 +++
  arch/arm64/include/uapi/asm/kvm.h             |  18 +
  arch/arm64/kvm/Makefile                       |   2 +-
  arch/arm64/kvm/arm.c                          |   8 +
  arch/arm64/kvm/hypercalls.c                   | 182 +++--
  arch/arm64/kvm/psci.c                         |  14 +-
  arch/arm64/kvm/pvtime.c                       |   2 +-
  arch/arm64/kvm/sdei.c                         | 676 ++++++++++++++++++
  arch/arm64/kvm/trng.c                         |   4 +-
  include/kvm/arm_hypercalls.h                  |  19 +-
  include/linux/arm-smccc.h                     |   7 +
  include/uapi/linux/arm_sdei.h                 |   8 +
  include/uapi/linux/kvm.h                      |   1 +
  tools/testing/selftests/kvm/Makefile          |   1 +
  .../selftests/kvm/aarch64/hypercalls.c        |  11 +-
  tools/testing/selftests/kvm/aarch64/sdei.c    | 450 ++++++++++++
  20 files changed, 1499 insertions(+), 67 deletions(-)
Quoted 8 lines

create mode 100644 Documentation/virt/kvm/arm/sdei.rst create mode 100644 arch/arm64/include/asm/kvm_sdei.h create mode 100644 arch/arm64/kvm/sdei.c
create mode 100644 tools/testing/selftests/kvm/aarch64/sdei.c

--
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author
#24 lore raw

Hi Shijie,

Quoted 166 lines · Shijie Huang
Quoted 149 lines · Gavin Shan

This series intends to virtualize Software Delegated Exception Interface (SDEI), which is defined by DEN0054C (v1.1). It allows the hypervisor to deliver NMI-alike SDEI event to guest and it's needed by Async PF to deliver page-not-present notification from hypervisor to guest. The code and the required qemu changes can be found from:

    https://developer.arm.com/documentation/den0054/c
    https://github.com/gwshan/linux    ("kvm/arm64_sdei")
    https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

The design is quite strightforward by following the specification. The (SDEI) events are classified into the shared and private ones according to their scope. The shared event is system or VM scoped, but the private event is vcpu scoped. This implementation doesn't support the shared event because all the needed events are private. Besides, the critial events aren't supported by the implementation either. It means all events are normal in terms of priority.

There are several objects (data structures) introduced to help on the event registration, enablement, disablement, unregistration, reset, delivery and handling.

   * kvm_sdei_event_handler
     SDEI event handler, which is provided through EVENT_REGISTER hypercall, is called when the SDEI event is delivered from host to guest.

   * kvm_sdei_event_context
     The saved (preempted) context when SDEI event is delivered for handling.

   * kvm_sdei_vcpu
     SDEI events and their states.

The patches are organized as below:

   PATCH[01-02] Preparatory work to extend smccc_get_argx() and refactor hypercall routing mechanism
   PATCH[03]    Adds SDEI virtualization infrastructure PATCH[04-16] Supports various SDEI hypercalls and event handling PATCH[17]    Exposes SDEI capability
   PATCH[18-19] Support SDEI migration
   PATCH[20]    Adds document about SDEI
   PATCH[21-22] SDEI related selftest cases

The previous revisions can be found:

   v6: https://lore.kernel.org/lkml/[email protected]/T/
   v5: https://lore.kernel.org/kvmarm/[email protected]/
   v4: https://lore.kernel.org/kvmarm/[email protected]/
   v3: https://lore.kernel.org/kvmarm/[email protected]/
   v2: https://lore.kernel.org/kvmarm/[email protected]/
   v1: https://lore.kernel.org/kvmarm/[email protected]/

Testing
=======
[1] The selftest case included in this series works fine. The default SDEI event, whose number is zero, can be registered, enabled, raised. The SDEI event handler can be invoked.

     [host]# pwd /home/gavin/sandbox/linux.main/tools/testing/selftests/kvm [root@virtlab-arm01 kvm]# ./aarch64/sdei

         NR_VCPUS: 2    SDEI Event: 0x00000000

     --- VERSION
         Version:              1.1 (vendor: 0x4b564d) --- FEATURES
         Shared event slots:   0
         Private event slots:  0
         Relative mode:        No
     --- PRIVATE_RESET
     --- SHARED_RESET
     --- PE_UNMASK
     --- EVENT_GET_INFO
         Type:                 Private
         Priority:             Normal
         Signaled:             Yes
     --- EVENT_REGISTER
     --- EVENT_ENABLE
     --- EVENT_SIGNAL
         Handled:              Yes
         IRQ:                  No
         Status:               Registered-Enabled-Running PC/PSTATE:            000000000040232c 00000000600003c5 Regs:                 0000000000000000 0000000000000000 0000000000000000 0000000000000000 --- PE_MASK
     --- EVENT_DISABLE
     --- EVENT_UNREGISTER

         Result: OK

[2] There are additional patches in the following repositories to create procfs entries, allowing to inject SDEI event from host side. The SDEI client in the guest side registers the SDEI default event, whose number is zero. Also, the QEMU exports SDEI ACPI table and supports migration for SDEI.

     https://github.com/gwshan/linux    ("kvm/arm64_sdei")
     https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

     [2.1] Start the guests and migrate the source VM to the destination VM.

     [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
             -accel kvm -machine virt,gic-version=host                     \
             -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
             -m 1024M,slots=16,maxmem=64G                                  \
                :                                                          \
             -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
             -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
             -append earlycon=pl011,mmio,0x9000000                         \
                :

     [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
             -accel kvm -machine virt,gic-version=host                     \
             -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
             -m 1024M,slots=16,maxmem=64G                                  \
                :                                                          \
             -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
             -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
             -append earlycon=pl011,mmio,0x9000000                         \
             -incoming tcp:0:4444                                          \
                :

     [2.2] Check kernel log on the source VM. The SDEI service is enabled and the default SDEI event (0x0) is enabled.

      [guest-src]# dmesg | grep -i sdei
      ACPI: SDEI 0x000000005BC80000 000024 \
                 (v00 BOCHS  BXPC     00000001 BXPC 00000001)
      sdei: SDEIv1.1 (0x4b564d) detected in firmware. SDEI TEST: Version 1.1, Vendor 0x4b564d
      sdei_init: SDEI event (0x0) registered
      sdei_init: SDEI event (0x0) enabled

      (qemu) migrate -d tcp:localhost:4444

     [2.3] Migrate the source VM to the destination VM. Inject SDEI event to the destination VM. The event is raised and handled.

     (qemu) migrate -d tcp:localhost:4444

     [host]# echo 0 > /proc/kvm/kvm-5360/vcpu-1

Quoted 15 lines

     [guest-dst]#
     =========== SDEI Event (CPU#1) ===========
     Event: 0000000000000000  Parameter: 00000000dabfdabf
     PC:    ffff800008cbb554  PSTATE: 00000000604000c5  SP: ffff800009c7bde0 Regs:    00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001 ffff800016c28000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 ffff800009399008 ffff8000097d9af0 ffff8000097d99f8 ffff8000093a8db8 ffff8000097d9b18 0000000000000000 0000000000000000 ffff000000339d00 0000000000000000 0000000000000000 ffff800009c7bde0 ffff800008cbb5c4
     Context: 00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001 ffff800016c28000 03ffffffffffffff 000000024325db59 ffff8000097de190 ffff00000033a790 ffff800008cbb814 0000000000000a30 0000000000000000

Quoted 4 lines

I tested this patch set. It's okay.

[...]

Appreciate your efforts to test it through. I will have your tested-by if respin is needed. Thank you for your time on this.

Thanks,
Gavin

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#26 lore raw

Hi Vishnu,

Quoted 188 lines · Vishnu Pajjuri
Quoted 169 lines · Gavin Shan
Quoted 168 lines · Shijie Huang
Quoted 149 lines · Gavin Shan

This series intends to virtualize Software Delegated Exception Interface (SDEI), which is defined by DEN0054C (v1.1). It allows the hypervisor to deliver NMI-alike SDEI event to guest and it's needed by Async PF to deliver page-not-present notification from hypervisor to guest. The code and the required qemu changes can be found from:

https://developer.arm.com/documentation/den0054/c
https://github.com/gwshan/linux    ("kvm/arm64_sdei")
https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

The design is quite strightforward by following the specification. The (SDEI) events are classified into the shared and private ones according to their scope. The shared event is system or VM scoped, but the private event is vcpu scoped. This implementation doesn't support the shared event because all the needed events are private. Besides, the critial events aren't supported by the implementation either. It means all events are normal in terms of priority.

There are several objects (data structures) introduced to help on the event registration, enablement, disablement, unregistration, reset, delivery and handling.

   * kvm_sdei_event_handler
     SDEI event handler, which is provided through EVENT_REGISTER hypercall, is called when the SDEI event is delivered from host to guest.

   * kvm_sdei_event_context
     The saved (preempted) context when SDEI event is delivered for handling.

   * kvm_sdei_vcpu
     SDEI events and their states.

The patches are organized as below:

   PATCH[01-02] Preparatory work to extend smccc_get_argx() and refactor hypercall routing mechanism
   PATCH[03]    Adds SDEI virtualization infrastructure PATCH[04-16] Supports various SDEI hypercalls and event handling PATCH[17]    Exposes SDEI capability
   PATCH[18-19] Support SDEI migration
   PATCH[20]    Adds document about SDEI
   PATCH[21-22] SDEI related selftest cases

The previous revisions can be found:

   v6: https://lore.kernel.org/lkml/[email protected]/T/
   v5: https://lore.kernel.org/kvmarm/[email protected]/
   v4: https://lore.kernel.org/kvmarm/[email protected]/
   v3: https://lore.kernel.org/kvmarm/[email protected]/
   v2: https://lore.kernel.org/kvmarm/[email protected]/
   v1: https://lore.kernel.org/kvmarm/[email protected]/

Testing
=======
[1] The selftest case included in this series works fine. The default SDEI event, whose number is zero, can be registered, enabled, raised. The SDEI event handler can be invoked.

     [host]# pwd /home/gavin/sandbox/linux.main/tools/testing/selftests/kvm [root@virtlab-arm01 kvm]# ./aarch64/sdei

         NR_VCPUS: 2    SDEI Event: 0x00000000

     --- VERSION
         Version:              1.1 (vendor: 0x4b564d) --- FEATURES
         Shared event slots:   0
         Private event slots:  0
         Relative mode:        No
     --- PRIVATE_RESET
     --- SHARED_RESET
     --- PE_UNMASK
     --- EVENT_GET_INFO
         Type:                 Private
         Priority:             Normal
         Signaled:             Yes
     --- EVENT_REGISTER
     --- EVENT_ENABLE
     --- EVENT_SIGNAL
         Handled:              Yes
         IRQ:                  No
         Status:               Registered-Enabled-Running PC/PSTATE:            000000000040232c 00000000600003c5 Regs:                 0000000000000000 0000000000000000 0000000000000000 0000000000000000 --- PE_MASK
     --- EVENT_DISABLE
     --- EVENT_UNREGISTER

         Result: OK

[2] There are additional patches in the following repositories to create procfs entries, allowing to inject SDEI event from host side. The SDEI client in the guest side registers the SDEI default event, whose number is zero. Also, the QEMU exports SDEI ACPI table and supports migration for SDEI.

https://github.com/gwshan/linux    ("kvm/arm64_sdei")
https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

     [2.1] Start the guests and migrate the source VM to the destination VM.

     [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64 \
             -accel kvm -machine virt,gic-version=host                     \
             -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
             -m 1024M,slots=16,maxmem=64G                                  \
:                                                          \
             -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
             -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
             -append earlycon=pl011,mmio,0x9000000                         \
                :

     [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64 \
             -accel kvm -machine virt,gic-version=host                     \
             -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
             -m 1024M,slots=16,maxmem=64G                                  \
:                                                          \
             -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
             -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
             -append earlycon=pl011,mmio,0x9000000                         \
             -incoming tcp:0:4444                                          \
                :

     [2.2] Check kernel log on the source VM. The SDEI service is enabled and the default SDEI event (0x0) is enabled.

      [guest-src]# dmesg | grep -i sdei
      ACPI: SDEI 0x000000005BC80000 000024 \
                 (v00 BOCHS  BXPC     00000001 BXPC 00000001)
      sdei: SDEIv1.1 (0x4b564d) detected in firmware. SDEI TEST: Version 1.1, Vendor 0x4b564d
      sdei_init: SDEI event (0x0) registered
      sdei_init: SDEI event (0x0) enabled

      (qemu) migrate -d tcp:localhost:4444

     [2.3] Migrate the source VM to the destination VM. Inject SDEI event to the destination VM. The event is raised and handled.

     (qemu) migrate -d tcp:localhost:4444

     [host]# echo 0 > /proc/kvm/kvm-5360/vcpu-1

Quoted 15 lines

     [guest-dst]#
     =========== SDEI Event (CPU#1) ===========
     Event: 0000000000000000  Parameter: 00000000dabfdabf
     PC:    ffff800008cbb554  PSTATE: 00000000604000c5  SP: ffff800009c7bde0 Regs:    00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001 ffff800016c28000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 ffff800009399008 ffff8000097d9af0 ffff8000097d99f8 ffff8000093a8db8 ffff8000097d9b18 0000000000000000 0000000000000000 ffff000000339d00 0000000000000000 0000000000000000 ffff800009c7bde0 ffff800008cbb5c4
     Context: 00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001 ffff800016c28000 03ffffffffffffff 000000024325db59 ffff8000097de190 ffff00000033a790 ffff800008cbb814 0000000000000a30 0000000000000000

I tested this patch set. It's okay.

Quoted 8 lines
Quoted 2 lines

[...]

Appreciate your efforts to test it through. I will have your tested-by if respin is needed. Thank you for your time on this.

I would like to know the current latest status of this patch series,

Since I didn't find any latest spin for this patch series.

Also, I didn't find any active development branch at

https://github.com/gwshan/linux.

And I observed that the kernel development branch

Quoted 8 lines

"https://github.com/gwshan/linuxkvm/arm64_sdei" moved to

"https://github.com/gwshan/linuxbackup/kvm/arm64_sdei"

I'm curious that what is required to spin this patch series with latest kernel versions

Appreciate any other insights on this...

Thanks for raising concerns. The SDEI events were used to deliver signals, required by the asynchronous page fault (Aync PF). I had several discussions with Marc and Paolo, and we reached to the temporary conclusion that Async PF isn't used in production enviornments like google cloud. I suspended the efforts since then. The SDEI virtualization support won't be needed if we needn't Async PF, unless there are other signals needed to be delivered by SDEI events.

Thanks,
Gavin

author
#27 lore raw
In reply to Gavin Shan

Hi Oliver,

Quoted 45 lines · Gavin Shan

This series intends to virtualize Software Delegated Exception Interface (SDEI), which is defined by DEN0054C (v1.1). It allows the hypervisor to deliver NMI-alike SDEI event to guest and it's needed by Async PF to deliver page-not-present notification from hypervisor to guest. The code and the required qemu changes can be found from:

    https://developer.arm.com/documentation/den0054/c
    https://github.com/gwshan/linux    ("kvm/arm64_sdei")
    https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

The design is quite strightforward by following the specification. The (SDEI) events are classified into the shared and private ones according to their scope. The shared event is system or VM scoped, but the private event is vcpu scoped. This implementation doesn't support the shared event because all the needed events are private. Besides, the critial events aren't supported by the implementation either. It means all events are normal in terms of priority.

There are several objects (data structures) introduced to help on the event registration, enablement, disablement, unregistration, reset, delivery and handling.

* kvm_sdei_event_handler
SDEI event handler, which is provided through EVENT_REGISTER hypercall, is called when the SDEI event is delivered from host to guest.

* kvm_sdei_event_context
The saved (preempted) context when SDEI event is delivered for handling.

* kvm_sdei_vcpu
SDEI events and their states.

The patches are organized as below:

PATCH[01-02] Preparatory work to extend smccc_get_argx() and refactor
hypercall routing mechanism
PATCH[03] Adds SDEI virtualization infrastructure
PATCH[04-16] Supports various SDEI hypercalls and event handling
PATCH[17] Exposes SDEI capability
PATCH[18-19] Support SDEI migration
PATCH[20] Adds document about SDEI
PATCH[21-22] SDEI related selftest cases

Quoted 9 lines

The previous revisions can be found:

   v6: https://lore.kernel.org/lkml/[email protected]/T/
   v5: https://lore.kernel.org/kvmarm/[email protected]/
   v4: https://lore.kernel.org/kvmarm/[email protected]/
   v3: https://lore.kernel.org/kvmarm/[email protected]/
   v2: https://lore.kernel.org/kvmarm/[email protected]/
   v1: https://lore.kernel.org/kvmarm/[email protected]/

Copying Oliver's new email address ([email protected]).

Please let me know if I need to rebase and repost the series.

Thanks,
Gavin

Quoted 187 lines

Testing
=======
[1] The selftest case included in this series works fine. The default SDEI

     event, whose number is zero, can be registered, enabled, raised. The
     SDEI event handler can be invoked.

     [host]# pwd
     /home/gavin/sandbox/linux.main/tools/testing/selftests/kvm
     [root@virtlab-arm01 kvm]# ./aarch64/sdei

         NR_VCPUS: 2    SDEI Event: 0x00000000

     --- VERSION
         Version:              1.1 (vendor: 0x4b564d)
     --- FEATURES
         Shared event slots:   0
         Private event slots:  0
         Relative mode:        No
     --- PRIVATE_RESET
     --- SHARED_RESET
     --- PE_UNMASK
     --- EVENT_GET_INFO
         Type:                 Private
         Priority:             Normal
         Signaled:             Yes
     --- EVENT_REGISTER
     --- EVENT_ENABLE
     --- EVENT_SIGNAL
         Handled:              Yes
         IRQ:                  No
         Status:               Registered-Enabled-Running
         PC/PSTATE:            000000000040232c 00000000600003c5
         Regs:                 0000000000000000 0000000000000000
                               0000000000000000 0000000000000000
     --- PE_MASK
     --- EVENT_DISABLE
     --- EVENT_UNREGISTER

         Result: OK

[2] There are additional patches in the following repositories to create

     procfs entries, allowing to inject SDEI event from host side. The
     SDEI client in the guest side registers the SDEI default event, whose
     number is zero. Also, the QEMU exports SDEI ACPI table and supports
     migration for SDEI.

     https://github.com/gwshan/linux    ("kvm/arm64_sdei")
     https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

     [2.1] Start the guests and migrate the source VM to the destination
           VM.

     [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
             -accel kvm -machine virt,gic-version=host                     \
             -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
             -m 1024M,slots=16,maxmem=64G                                  \
                :                                                          \
             -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
             -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
             -append earlycon=pl011,mmio,0x9000000                         \
                :

     [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
             -accel kvm -machine virt,gic-version=host                     \
             -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
             -m 1024M,slots=16,maxmem=64G                                  \
                :                                                          \
             -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
             -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
             -append earlycon=pl011,mmio,0x9000000                         \
             -incoming tcp:0:4444                                          \
                :

     [2.2] Check kernel log on the source VM. The SDEI service is enabled
           and the default SDEI event (0x0) is enabled.

      [guest-src]# dmesg | grep -i sdei
      ACPI: SDEI 0x000000005BC80000 000024 \
                 (v00 BOCHS  BXPC     00000001 BXPC 00000001)
      sdei: SDEIv1.1 (0x4b564d) detected in firmware.
      SDEI TEST: Version 1.1, Vendor 0x4b564d
      sdei_init: SDEI event (0x0) registered
      sdei_init: SDEI event (0x0) enabled

  
      (qemu) migrate -d tcp:localhost:4444

     [2.3] Migrate the source VM to the destination VM. Inject SDEI event
           to the destination VM. The event is raised and handled.

     (qemu) migrate -d tcp:localhost:4444

     [host]# echo 0 > /proc/kvm/kvm-5360/vcpu-1

     [guest-dst]#
     =========== SDEI Event (CPU#1) ===========
     Event: 0000000000000000  Parameter: 00000000dabfdabf
     PC:    ffff800008cbb554  PSTATE: 00000000604000c5  SP: ffff800009c7bde0
     Regs:    00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001
              ffff800016c28000 0000000000000000 0000000000000000 0000000000000000
              0000000000000000 0000000000000000 0000000000000000 0000000000000000
              0000000000000000 0000000000000000 0000000000000000 0000000000000000
              0000000000000000 0000000000000000 0000000000000000 ffff800009399008
              ffff8000097d9af0 ffff8000097d99f8 ffff8000093a8db8 ffff8000097d9b18
              0000000000000000 0000000000000000 ffff000000339d00 0000000000000000
              0000000000000000 ffff800009c7bde0 ffff800008cbb5c4
     Context: 00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001
              ffff800016c28000 03ffffffffffffff 000000024325db59 ffff8000097de190
              ffff00000033a790 ffff800008cbb814 0000000000000a30 0000000000000000

Changelog
=========
v7:
* Rebased to v5.19.rc1 (Gavin)
* Add hypercall ranges for routing (Oliver)
* Remove support to the critical event and redesigned data structures. Function names are also modified
as Oliver suggested (Oliver)
* Deliver event when it's enabled or the specific PE
is unmasked (Oliver)
* Improve EVENT_COMPLETE_AND_RESUME hypercall to resume
from the specified address (Oliver)
* Add patches for SDEI migration and documentation (Gavin)
* Misc comments from Oliver Upon (Oliver)
v6:
* Rebased to v5.18.rc1 (Gavin)
* Pass additional argument to smccc_get_arg() (Oliver)
* Add preparatory patch to route hypercalls based on their
owners (Oliver)
* Remove the support for shared event. (Oliver/Gavin)
* Remove the support for migration and add-on patches to
support it in future (Oliver)
* The events are exposed by KVM instead of VMM (Oliver)
* kvm_sdei_state.h is dropped and all the structures are
folded into the corresponding ones in kvm_sdei.h (Oliver)
* Rename 'struct kvm_sdei_registered_event' to
'struct kvm_sdei_event' (Oliver)
* Misc comments from Oliver Upon (Oliver)
v5/v4/v3/v2/v1:
* Skipped here and please visit the history by
https://lore.kernel.org/lkml/[email protected]/T/

Gavin Shan (22):
KVM: arm64: Extend smccc_get_argx()
KVM: arm64: Route hypercalls based on their owner
KVM: arm64: Add SDEI virtualization infrastructure
KVM: arm64: Support EVENT_REGISTER hypercall
KVM: arm64: Support EVENT_{ENABLE, DISABLE} hypercall
KVM: arm64: Support EVENT_CONTEXT hypercall
KVM: arm64: Support EVENT_UNREGISTER hypercall
KVM: arm64: Support EVENT_STATUS hypercall
KVM: arm64: Support EVENT_GET_INFO hypercall
KVM: arm64: Support PE_{MASK, UNMASK} hypercall
KVM: arm64: Support {PRIVATE, SHARED}_RESET hypercall
KVM: arm64: Support event injection and delivery
KVM: arm64: Support EVENT_{COMPLETE, COMPLETE_AND_RESUME} hypercall
KVM: arm64: Support EVENT_SIGNAL hypercall
KVM: arm64: Support SDEI_FEATURES hypercall
KVM: arm64: Support SDEI_VERSION hypercall
KVM: arm64: Expose SDEI capbility and service
KVM: arm64: Allow large sized pseudo firmware registers
KVM: arm64: Support SDEI event migration
KVM: arm64: Add SDEI document
selftests: KVM: aarch64: Add SDEI case in hypercall tests
selftests: KVM: aarch64: Add SDEI test case

  Documentation/virt/kvm/api.rst                |  11 +
  Documentation/virt/kvm/arm/hypercalls.rst     |   4 +
  Documentation/virt/kvm/arm/sdei.rst           |  64 ++
  arch/arm64/include/asm/kvm_host.h             |   3 +
  arch/arm64/include/asm/kvm_sdei.h             |  81 +++
  arch/arm64/include/uapi/asm/kvm.h             |  18 +
  arch/arm64/kvm/Makefile                       |   2 +-
  arch/arm64/kvm/arm.c                          |   8 +
  arch/arm64/kvm/hypercalls.c                   | 182 +++--
  arch/arm64/kvm/psci.c                         |  14 +-
  arch/arm64/kvm/pvtime.c                       |   2 +-
  arch/arm64/kvm/sdei.c                         | 676 ++++++++++++++++++
  arch/arm64/kvm/trng.c                         |   4 +-
  include/kvm/arm_hypercalls.h                  |  19 +-
  include/linux/arm-smccc.h                     |   7 +
  include/uapi/linux/arm_sdei.h                 |   8 +
  include/uapi/linux/kvm.h                      |   1 +
  tools/testing/selftests/kvm/Makefile          |   1 +
  .../selftests/kvm/aarch64/hypercalls.c        |  11 +-
  tools/testing/selftests/kvm/aarch64/sdei.c    | 450 ++++++++++++
  20 files changed, 1499 insertions(+), 67 deletions(-)
Quoted 5 lines

create mode 100644 Documentation/virt/kvm/arm/sdei.rst create mode 100644 arch/arm64/include/asm/kvm_sdei.h create mode 100644 arch/arm64/kvm/sdei.c
create mode 100644 tools/testing/selftests/kvm/aarch64/sdei.c

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

maintainer
#28 lore raw

Hi Gavin,

Quoted 57 lines · Gavin Shan

Hi Oliver,

Quoted 43 lines · Gavin Shan

This series intends to virtualize Software Delegated Exception Interface (SDEI), which is defined by DEN0054C (v1.1). It allows the hypervisor to deliver NMI-alike SDEI event to guest and it's needed by Async PF to deliver page-not-present notification from hypervisor to guest. The code and the required qemu changes can be found from:

    https://developer.arm.com/documentation/den0054/c
    https://github.com/gwshan/linux    ("kvm/arm64_sdei")
    https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

The design is quite strightforward by following the specification. The (SDEI) events are classified into the shared and private ones according to their scope. The shared event is system or VM scoped, but the private event is vcpu scoped. This implementation doesn't support the shared event because all the needed events are private. Besides, the critial events aren't supported by the implementation either. It means all events are normal in terms of priority.

There are several objects (data structures) introduced to help on the event registration, enablement, disablement, unregistration, reset, delivery and handling.

* kvm_sdei_event_handler
SDEI event handler, which is provided through EVENT_REGISTER hypercall, is called when the SDEI event is delivered from host to guest.
* kvm_sdei_event_context
The saved (preempted) context when SDEI event is delivered for handling.
* kvm_sdei_vcpu
SDEI events and their states.

The patches are organized as below:

PATCH[01-02] Preparatory work to extend smccc_get_argx() and refactor
hypercall routing mechanism
PATCH[03] Adds SDEI virtualization infrastructure
PATCH[04-16] Supports various SDEI hypercalls and event handling
PATCH[17] Exposes SDEI capability
PATCH[18-19] Support SDEI migration
PATCH[20] Adds document about SDEI
PATCH[21-22] SDEI related selftest cases

Quoted 9 lines

The previous revisions can be found:

   v6: https://lore.kernel.org/lkml/[email protected]/T/
   v5: https://lore.kernel.org/kvmarm/[email protected]/
   v4: https://lore.kernel.org/kvmarm/[email protected]/
   v3: https://lore.kernel.org/kvmarm/[email protected]/
   v2: https://lore.kernel.org/kvmarm/[email protected]/
   v1: https://lore.kernel.org/kvmarm/[email protected]/
Quoted 3 lines

Copying Oliver's new email address ([email protected]).

Please let me know if I need to rebase and repost the series.

My main issue with this series is that it is a solution in search of a problem. It is only an enabler for Asynchronous Page Fault support, and:

- as far as I know, the core Linux/arm64 maintainers have no plan to support APF. Without it, this is a pointless exercise. And even with it, this introduces a Linux specific behaviour in an otherwise architectural hypervisor (something I'm quite keen on avoiding)

- It gives an incentive to other hypervisor vendors to add random crap to the Linux mm subsystem, which is even worse. At this stage, we might as well go back to the Xen PV days altogether.

- I haven't seen any of the KVM/arm64 users actually asking for the APF horror, and the cloud vendors I directly asked had no plan to use it, and not using it on their x86 systems either

- no performance data nor workloads that could help making an informed decision have been disclosed, and the only argument in its favour seems to be "but x86 has it" (hardly a compelling one)

Given the above, I don't see how to justify this series, as it has no purpose on its own, no matter how well written it is.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author
#29 lore raw

Hi Marc,

Quoted 77 lines · Marc Zyngier
Quoted 54 lines · Gavin Shan
Quoted 43 lines · Gavin Shan

This series intends to virtualize Software Delegated Exception Interface (SDEI), which is defined by DEN0054C (v1.1). It allows the hypervisor to deliver NMI-alike SDEI event to guest and it's needed by Async PF to deliver page-not-present notification from hypervisor to guest. The code and the required qemu changes can be found from:

     https://developer.arm.com/documentation/den0054/c
     https://github.com/gwshan/linux    ("kvm/arm64_sdei")
     https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

The design is quite strightforward by following the specification. The (SDEI) events are classified into the shared and private ones according to their scope. The shared event is system or VM scoped, but the private event is vcpu scoped. This implementation doesn't support the shared event because all the needed events are private. Besides, the critial events aren't supported by the implementation either. It means all events are normal in terms of priority.

There are several objects (data structures) introduced to help on the event registration, enablement, disablement, unregistration, reset, delivery and handling.

* kvm_sdei_event_handler
SDEI event handler, which is provided through EVENT_REGISTER hypercall, is called when the SDEI event is delivered from host to guest.
* kvm_sdei_event_context
The saved (preempted) context when SDEI event is delivered for handling.
* kvm_sdei_vcpu
SDEI events and their states.

The patches are organized as below:

PATCH[01-02] Preparatory work to extend smccc_get_argx() and refactor
hypercall routing mechanism
PATCH[03] Adds SDEI virtualization infrastructure
PATCH[04-16] Supports various SDEI hypercalls and event handling
PATCH[17] Exposes SDEI capability
PATCH[18-19] Support SDEI migration
PATCH[20] Adds document about SDEI
PATCH[21-22] SDEI related selftest cases

Quoted 9 lines

The previous revisions can be found:

    v6: https://lore.kernel.org/lkml/[email protected]/T/
    v5: https://lore.kernel.org/kvmarm/[email protected]/
    v4: https://lore.kernel.org/kvmarm/[email protected]/
    v3: https://lore.kernel.org/kvmarm/[email protected]/
    v2: https://lore.kernel.org/kvmarm/[email protected]/
    v1: https://lore.kernel.org/kvmarm/[email protected]/
Quoted 3 lines

Copying Oliver's new email address ([email protected]).

Please let me know if I need to rebase and repost the series.

My main issue with this series is that it is a solution in search of a problem. It is only an enabler for Asynchronous Page Fault support, and:

- as far as I know, the core Linux/arm64 maintainers have no plan to support APF. Without it, this is a pointless exercise. And even with it, this introduces a Linux specific behaviour in an otherwise architectural hypervisor (something I'm quite keen on avoiding)

- It gives an incentive to other hypervisor vendors to add random crap to the Linux mm subsystem, which is even worse. At this stage, we might as well go back to the Xen PV days altogether.

- I haven't seen any of the KVM/arm64 users actually asking for the APF horror, and the cloud vendors I directly asked had no plan to use it, and not using it on their x86 systems either

Quoted 7 lines

- no performance data nor workloads that could help making an informed decision have been disclosed, and the only argument in its favour seems to be "but x86 has it" (hardly a compelling one)

Given the above, I don't see how to justify this series, as it has no purpose on its own, no matter how well written it is.

Thank you for your time to review the series and provide comments. Long time ago, I compare the features supported on x86 and arm64, to sort out the gaps. Async page fault is one of the missed features. From that on, I started to investigate x86's implementation and work on arm64's implementation. It's the history why I continue to work on Async page fault for arm64. It means there is no customer request, asking to support Async page fault on arm64, on my side.

In order to support Async PF on arm64, there are two parts of changes, which are related to kvm/arm64 and guest kernel. The service of Async page fault won't be enabled if either kvm/arm64 or guest kernel doesn't support it. The service is negotiated between host and guest. So I don't think it would be a problem. It's true that Async page fault is only beneficial to Linux host and Linux guest, until it gets supported on other guest kernels.

SDEI implementation is following the specification. It's true that Async PF isn't specified by arm64 architecture. However, it's also not a architectural feature to x86 either. I guess the benefits count here. The reason we need Async PF (and SDEI virtualization) is the benefit.

If I'm correct, Async PF has been used broadly on x86 because of
'post-copy live migration', which relies on userfaultfd. 'Async page fault' is explicitly mentioned in its document (linux/Documentation/admin-guide/mm/userfaultfd.rst) like below. It's the most important motivation to support Async PF.

Yeah, performance data is definitely helpful to measure the benefit, especially for Async page fault on arm64. I used to revise both serieses (SDEI virtualization and Async page fault) together, meaning 'Async page fault' series is revised if there are any code changes to the series of 'SDEI virtualization', until I found it would be practical to finialize 'SDEI virtualization' before working on 'Async page fault'. It's why I don't post revised series of 'Async page fault' recently. However, I think the performance data released in last year's KVM forum is still relative. I certainly need to regain the performance data when I continue to work on 'Async page fault' series after 'SDEI virutalization' is finalized.

https://static.sched.com/hosted_files/kvmforum2021/cb/sdei_apf_for_arm64_gavin.pdf
(In page 14 and 15, 41% to 68% improvement in live post-copy migration)

Extracted from linux/documentation/admin-guide/mm/userfaultfd.rst
------------------------------------------------------------------

QEMU/KVM
========

QEMU/KVM is using the ``userfaultfd`` syscall to implement postcopy live migration. Postcopy live migration is one form of memory externalization consisting of a virtual machine running with part or all of its memory residing on a different node in the cloud. The ``userfaultfd`` abstraction is generic enough that not a single line of KVM kernel code had to be modified in order to add postcopy live migration to QEMU.

Guest async page faults, ``FOLL_NOWAIT`` and all other ``GUP*`` features work just fine in combination with userfaults. Userfaults trigger async page faults in the guest scheduler so those guest processes that aren't waiting for userfaults (i.e. network bound) can keep running in the guest vcpus.

Thanks,
Gavin

maintainer
#30 lore raw
In reply to Marc Zyngier
Quoted 4 lines · Marc Zyngier

- as far as I know, the core Linux/arm64 maintainers have no plan to support APF. Without it, this is a pointless exercise. And even with it, this introduces a Linux specific behaviour in an otherwise architectural hypervisor (something I'm quite keen on avoiding)

Regarding non-architectural behavior, isn't that the same already for PTP? I understand that the PTP hypercall is a much smaller implementation than SDEI+APF, but it goes to show that KVM is already not "architectural".

There are other cases where paravirtualized solutions can be useful. PTP is one but there are more where KVM/ARM does not have a solution yet, for example lock holder preemption. Unless ARM (the company) has a way to receive input from developers and standardize the interface, similar to the RISC-V SIGs, vendor-specific hypercalls are a sad fact of life. It just happened that until now KVM/ARM hasn't seen much use in some cases (such as desktop virtualization) where overcommitted hosts are more common.

Async page faults per se are not KVM specific, in fact Linux supported them for the IBM s390 hypervisor long before KVM added support. They didn't exist on x86 and ARM, so the developers came up with a new hypercall API and for x86 honestly it wasn't great. For ARM we learnt from the mistakes and it seems to me that SDEI is a good match for the feature. If ARM wants to produce a standard interface for APF, whether based on SDEI or something else, we're all ears.

Regarding plans of core arm64 maintainers to support async page fault, can you provide a pointer to the discussion? I agree that if there's a hard NACK for APF for whatever reason, the whole host-side code is pointless (including SDEI virtualization); but I would like to read more about it.

Quoted 3 lines

- It gives an incentive to other hypervisor vendors to add random crap to the Linux mm subsystem, which is even worse. At this stage, we might as well go back to the Xen PV days altogether.

	return -EGREGIOUS;

Since you mention hypervisor vendors and there's only one hypervisor in Linux, I guess you're not talking about the host mm/ subsystem (otherwise yeah, FOLL_NOWAIT is only used by KVM async page faults).

So I suppose you're talking about the guest, and then yeah, it sucks to have multiple hypervisors providing the same functionality in different ways (or multiple hypervisors providing different subsets of PV functionality). It happens on x86 with Hyper-V and KVM, and to a lesser extent Xen and VMware.

But again, KVM/ARM has already crossed that bridge with PTP support, and the guest needs exactly zero code in the Linux mm subsystem (both generic and arch-specific) to support asynchronous page faults. There are 20 lines of code in do_notify_resume(), and the rest is just SDEI gunk. Again, I would be happy to get a pointer to concrete objections from the Linux ARM64 maintainers. Maybe a different implementation is possible, I don't know.

In any case it's absolutely not comparable to Xen PV, and you know it.

Quoted 3 lines

- I haven't seen any of the KVM/arm64 users actually asking for the APF horror, and the cloud vendors I directly asked had no plan to use it, and not using it on their x86 systems either

Please define "horror" in more technical terms. And since this is the second time I'm calling you out on this, I'm also asking you to avoid hyperboles and similar rhetorical gimmicks in the future.

That said: Peter, Sean, Google uses or used postcopy extensively on GCE (https://dl.acm.org/doi/pdf/10.1145/3296975.3186415). If it doesn't use it on x86, do you have any insights on why?

Quoted 3 lines

- no performance data nor workloads that could help making an informed decision have been disclosed, and the only argument in its favour seems to be "but x86 has it" (hardly a compelling one)

Again this is just false, numbers have been posted (https://lwn.net/ml/linux-kernel/[email protected]/ was the first result that came up from a quick mailing list search). If they are not enough, please be more specific.

Thanks,

Paolo

#31 lore raw
Quoted 3 lines · Paolo Bonzini

That said: Peter, Sean, Google uses or used postcopy extensively on GCE
(https://dl.acm.org/doi/pdf/10.1145/3296975.3186415). If it doesn't use it on x86, do you have any insights on why?

We still use postcopy, but we don't use async #PF. Async #PF is disabled (mostly?) because the x86 implementation was such a mess prior to switching to IRQ-based delivery and AFAIK we haven't re-evaluated it since that update.