Skip to content

Support SDEI Virtualization

Gavin Shan kvmarm lkml 7 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v7 Latest22 patches 8 repliesIn review
  2. v618 patches 34 repliesSuperseded
  3. v522 patches 26 repliesSuperseded
Earlier versions · 4
VersionPostedPatchesRepliesStatus
v421 patches57 repliesSuperseded
v321 patches0 repliesSuperseded
v221 patches0 repliesSuperseded
v118 patches1 replySuperseded

Changes in v7 from the cover letter

* Rebased to v5.19.rc1 (Gavin)
* Add hypercall ranges for routing (Oliver)
* Remove support to the critical event and redesigned data structures. Function names are also modified
as Oliver suggested (Oliver)
* Deliver event when it's enabled or the specific PE
is unmasked (Oliver)
* Improve EVENT_COMPLETE_AND_RESUME hypercall to resume

Patches in v7 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/22KVM: arm64: Extend smccc_get_argx()arch/arm64/kvm/hypercalls.c arch/arm64/kvm/psci.c arch/arm64/kvm/pvtime.c +2 more+17 -260 replies
2/22KVM: arm64: Route hypercalls based on their ownerarch/arm64/kvm/hypercalls.c include/linux/arm-smccc.h+88 -330 replies
3/22KVM: arm64: Add SDEI virtualization infrastructurearch/arm64/include/asm/kvm_host.h arch/arm64/include/asm/kvm_sdei.h arch/arm64/kvm/Makefile +4 more+141 -10 replies
4/22KVM: arm64: Support EVENT_REGISTER hypercallarch/arm64/kvm/sdei.c+31 -00 replies
5/22KVM: arm64: Support EVENT_{ENABLE, DISABLE} hypercallarch/arm64/include/asm/kvm_host.h arch/arm64/kvm/sdei.c+34 -00 replies
6/22KVM: arm64: Support EVENT_CONTEXT hypercallarch/arm64/kvm/sdei.c+21 -10 replies
7/22KVM: arm64: Support EVENT_UNREGISTER hypercallarch/arm64/kvm/sdei.c+36 -00 replies
8/22KVM: arm64: Support EVENT_STATUS hypercallarch/arm64/kvm/sdei.c+22 -00 replies
9/22KVM: arm64: Support EVENT_GET_INFO hypercallarch/arm64/kvm/sdei.c+35 -00 replies
10/22KVM: arm64: Support PE_{MASK, UNMASK} hypercallarch/arm64/kvm/sdei.c+21 -00 replies
11/22KVM: arm64: Support {PRIVATE, SHARED}_RESET hypercallarch/arm64/kvm/sdei.c+26 -00 replies
12/22KVM: arm64: Support event injection and deliveryarch/arm64/include/asm/kvm_sdei.h arch/arm64/kvm/arm.c arch/arm64/kvm/sdei.c+130 -00 replies
13/22KVM: arm64: Support EVENT_{COMPLETE, COMPLETE_AND_RESUME} hypercallarch/arm64/kvm/sdei.c+54 -00 replies
14/22KVM: arm64: Support EVENT_SIGNAL hypercallarch/arm64/kvm/sdei.c include/uapi/linux/arm_sdei.h+31 -00 replies
15/22KVM: arm64: Support SDEI_FEATURES hypercallarch/arm64/kvm/sdei.c include/uapi/linux/arm_sdei.h+21 -00 replies
16/22KVM: arm64: Support SDEI_VERSION hypercallarch/arm64/include/asm/kvm_sdei.h arch/arm64/kvm/sdei.c+14 -00 replies
17/22KVM: arm64: Expose SDEI capbility and serviceDocumentation/virt/kvm/api.rst Documentation/virt/kvm/arm/hypercalls.rst arch/arm64/include/asm/kvm_sdei.h +4 more+24 -00 replies
18/22KVM: arm64: Allow large sized pseudo firmware registersarch/arm64/kvm/hypercalls.c+17 -60 replies
19/22KVM: arm64: Support SDEI event migrationarch/arm64/include/asm/kvm_sdei.h arch/arm64/include/uapi/asm/kvm.h arch/arm64/kvm/hypercalls.c +1 more+212 -00 replies
20/22KVM: arm64: Add SDEI documentDocumentation/virt/kvm/arm/sdei.rst+64 -00 replies
21/22selftests: KVM: aarch64: Add SDEI case in hypercall teststools/testing/selftests/kvm/aarch64/hypercalls.c+10 -10 replies
22/22selftests: KVM: aarch64: Add SDEI test casetools/testing/selftests/kvm/Makefile tools/testing/selftests/kvm/aarch64/sdei.c+451 -00 replies

Reviews trailers per patch

None of the 22 patches has a Reviewed-by or Acked-by.

PatchHuang Shijie
SeriesT-b
1/22
2/22
3/22
4/22
5/22
6/22
7/22
8/22
9/22
10/22
11/22
12/22
13/22
14/22
15/22
16/22
17/22
18/22
19/22
20/22
21/22
22/22

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v7 thread

Cover letter [PATCH v7 00/22] Support SDEI Virtualization

This series intends to virtualize Software Delegated Exception Interface (SDEI), which is defined by DEN0054C (v1.1). It allows the hypervisor to deliver NMI-alike SDEI event to guest and it's needed by Async PF to deliver page-not-present notification from hypervisor to guest. The code and the required qemu changes can be found from:

https://developer.arm.com/documentation/den0054/c
https://github.com/gwshan/linux ("kvm/arm64_sdei")
https://github.com/gwshan/qemu ("kvm/arm64_sdei")

The design is quite strightforward by following the specification. The (SDEI) events are classified into the shared and private ones according to their scope. The shared event is system or VM scoped, but the private event is vcpu scoped. This implementation doesn't support the shared event because all the needed events are private. Besides, the critial events aren't supported by the implementation either. It means all events are normal in terms of priority.

There are several objects (data structures) introduced to help on the event registration, enablement, disablement, unregistration, reset, delivery and handling.

* kvm_sdei_event_handler
SDEI event handler, which is provided through EVENT_REGISTER hypercall, is called when the SDEI event is delivered from host to guest.

* kvm_sdei_event_context
The saved (preempted) context when SDEI event is delivered for handling.

* kvm_sdei_vcpu
SDEI events and their states.

The patches are organized as below:

PATCH[01-02] Preparatory work to extend smccc_get_argx() and refactor
hypercall routing mechanism
PATCH[03] Adds SDEI virtualization infrastructure
PATCH[04-16] Supports various SDEI hypercalls and event handling
PATCH[17] Exposes SDEI capability
PATCH[18-19] Support SDEI migration
PATCH[20] Adds document about SDEI
PATCH[21-22] SDEI related selftest cases

The previous revisions can be found:

v6: https://lore.kernel.org/lkml/[email protected]/T/
v5: https://lore.kernel.org/kvmarm/[email protected]/
v4: https://lore.kernel.org/kvmarm/[email protected]/
v3: https://lore.kernel.org/kvmarm/[email protected]/
v2: https://lore.kernel.org/kvmarm/[email protected]/
v1: https://lore.kernel.org/kvmarm/[email protected]/

Testing
=======
[1] The selftest case included in this series works fine. The default SDEI

    event, whose number is zero, can be registered, enabled, raised. The
    SDEI event handler can be invoked.

    [host]# pwd
    /home/gavin/sandbox/linux.main/tools/testing/selftests/kvm
    [root@virtlab-arm01 kvm]# ./aarch64/sdei 

        NR_VCPUS: 2    SDEI Event: 0x00000000

    --- VERSION
        Version:              1.1 (vendor: 0x4b564d)
    --- FEATURES
        Shared event slots:   0
        Private event slots:  0
        Relative mode:        No
    --- PRIVATE_RESET
    --- SHARED_RESET
    --- PE_UNMASK
    --- EVENT_GET_INFO
        Type:                 Private
        Priority:             Normal
        Signaled:             Yes
    --- EVENT_REGISTER
    --- EVENT_ENABLE
    --- EVENT_SIGNAL
        Handled:              Yes
        IRQ:                  No
        Status:               Registered-Enabled-Running
        PC/PSTATE:            000000000040232c 00000000600003c5
        Regs:                 0000000000000000 0000000000000000
                              0000000000000000 0000000000000000
    --- PE_MASK
    --- EVENT_DISABLE
    --- EVENT_UNREGISTER

        Result: OK

[2] There are additional patches in the following repositories to create

    procfs entries, allowing to inject SDEI event from host side. The
    SDEI client in the guest side registers the SDEI default event, whose
    number is zero. Also, the QEMU exports SDEI ACPI table and supports
    migration for SDEI.

    https://github.com/gwshan/linux    ("kvm/arm64_sdei")
    https://github.com/gwshan/qemu     ("kvm/arm64_sdei")

    [2.1] Start the guests and migrate the source VM to the destination
          VM.

    [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
            -accel kvm -machine virt,gic-version=host                     \
            -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
            -m 1024M,slots=16,maxmem=64G                                  \
               :                                                          \
            -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
            -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
            -append earlycon=pl011,mmio,0x9000000                         \
               :

    [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
            -accel kvm -machine virt,gic-version=host                     \
            -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
            -m 1024M,slots=16,maxmem=64G                                  \
               :                                                          \
            -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
            -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
            -append earlycon=pl011,mmio,0x9000000                         \
            -incoming tcp:0:4444                                          \
               :

    [2.2] Check kernel log on the source VM. The SDEI service is enabled
          and the default SDEI event (0x0) is enabled.

     [guest-src]# dmesg | grep -i sdei
     ACPI: SDEI 0x000000005BC80000 000024 \
                (v00 BOCHS  BXPC     00000001 BXPC 00000001)
     sdei: SDEIv1.1 (0x4b564d) detected in firmware.
     SDEI TEST: Version 1.1, Vendor 0x4b564d
     sdei_init: SDEI event (0x0) registered
     sdei_init: SDEI event (0x0) enabled

 
     (qemu) migrate -d tcp:localhost:4444

    [2.3] Migrate the source VM to the destination VM. Inject SDEI event
          to the destination VM. The event is raised and handled.

    (qemu) migrate -d tcp:localhost:4444

    [host]# echo 0 > /proc/kvm/kvm-5360/vcpu-1

    [guest-dst]#
    =========== SDEI Event (CPU#1) ===========
    Event: 0000000000000000  Parameter: 00000000dabfdabf
    PC:    ffff800008cbb554  PSTATE: 00000000604000c5  SP: ffff800009c7bde0
    Regs:    00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001 
             ffff800016c28000 0000000000000000 0000000000000000 0000000000000000 
             0000000000000000 0000000000000000 0000000000000000 0000000000000000 
             0000000000000000 0000000000000000 0000000000000000 0000000000000000 
             0000000000000000 0000000000000000 0000000000000000 ffff800009399008 
             ffff8000097d9af0 ffff8000097d99f8 ffff8000093a8db8 ffff8000097d9b18 
             0000000000000000 0000000000000000 ffff000000339d00 0000000000000000 
             0000000000000000 ffff800009c7bde0 ffff800008cbb5c4 
    Context: 00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001 
             ffff800016c28000 03ffffffffffffff 000000024325db59 ffff8000097de190 
             ffff00000033a790 ffff800008cbb814 0000000000000a30 0000000000000000 

Changelog
=========
v7:
* Rebased to v5.19.rc1 (Gavin)
* Add hypercall ranges for routing (Oliver)
* Remove support to the critical event and redesigned data structures. Function names are also modified
as Oliver suggested (Oliver)
* Deliver event when it's enabled or the specific PE
is unmasked (Oliver)
* Improve EVENT_COMPLETE_AND_RESUME hypercall to resume

     from the specified address                               (Oliver) 
   * Add patches for SDEI migration and documentation         (Gavin)
   * Misc comments from Oliver Upon                           (Oliver)

v6:
* Rebased to v5.18.rc1 (Gavin)
* Pass additional argument to smccc_get_arg() (Oliver)
* Add preparatory patch to route hypercalls based on their
owners (Oliver)
* Remove the support for shared event. (Oliver/Gavin)
* Remove the support for migration and add-on patches to
support it in future (Oliver)
* The events are exposed by KVM instead of VMM (Oliver)
* kvm_sdei_state.h is dropped and all the structures are
folded into the corresponding ones in kvm_sdei.h (Oliver)
* Rename 'struct kvm_sdei_registered_event' to
'struct kvm_sdei_event' (Oliver)
* Misc comments from Oliver Upon (Oliver)
v5/v4/v3/v2/v1:
* Skipped here and please visit the history by

     https://lore.kernel.org/lkml/[email protected]/T/

Gavin Shan (22):
KVM: arm64: Extend smccc_get_argx()
KVM: arm64: Route hypercalls based on their owner
KVM: arm64: Add SDEI virtualization infrastructure
KVM: arm64: Support EVENT_REGISTER hypercall
KVM: arm64: Support EVENT_{ENABLE, DISABLE} hypercall
KVM: arm64: Support EVENT_CONTEXT hypercall
KVM: arm64: Support EVENT_UNREGISTER hypercall
KVM: arm64: Support EVENT_STATUS hypercall
KVM: arm64: Support EVENT_GET_INFO hypercall
KVM: arm64: Support PE_{MASK, UNMASK} hypercall
KVM: arm64: Support {PRIVATE, SHARED}_RESET hypercall
KVM: arm64: Support event injection and delivery
KVM: arm64: Support EVENT_{COMPLETE, COMPLETE_AND_RESUME} hypercall
KVM: arm64: Support EVENT_SIGNAL hypercall
KVM: arm64: Support SDEI_FEATURES hypercall
KVM: arm64: Support SDEI_VERSION hypercall
KVM: arm64: Expose SDEI capbility and service
KVM: arm64: Allow large sized pseudo firmware registers
KVM: arm64: Support SDEI event migration
KVM: arm64: Add SDEI document
selftests: KVM: aarch64: Add SDEI case in hypercall tests
selftests: KVM: aarch64: Add SDEI test case

 Documentation/virt/kvm/api.rst                |  11 +
 Documentation/virt/kvm/arm/hypercalls.rst     |   4 +
 Documentation/virt/kvm/arm/sdei.rst           |  64 ++
 arch/arm64/include/asm/kvm_host.h             |   3 +
 arch/arm64/include/asm/kvm_sdei.h             |  81 +++
 arch/arm64/include/uapi/asm/kvm.h             |  18 +
 arch/arm64/kvm/Makefile                       |   2 +-
 arch/arm64/kvm/arm.c                          |   8 +
 arch/arm64/kvm/hypercalls.c                   | 182 +++--
 arch/arm64/kvm/psci.c                         |  14 +-
 arch/arm64/kvm/pvtime.c                       |   2 +-
 arch/arm64/kvm/sdei.c                         | 676 ++++++++++++++++++
 arch/arm64/kvm/trng.c                         |   4 +-
 include/kvm/arm_hypercalls.h                  |  19 +-
 include/linux/arm-smccc.h                     |   7 +
 include/uapi/linux/arm_sdei.h                 |   8 +
 include/uapi/linux/kvm.h                      |   1 +
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/aarch64/hypercalls.c        |  11 +-
 tools/testing/selftests/kvm/aarch64/sdei.c    | 450 ++++++++++++
 20 files changed, 1499 insertions(+), 67 deletions(-)
 create mode 100644 Documentation/virt/kvm/arm/sdei.rst
 create mode 100644 arch/arm64/include/asm/kvm_sdei.h
 create mode 100644 arch/arm64/kvm/sdei.c
 create mode 100644 tools/testing/selftests/kvm/aarch64/sdei.c
Signature · 6 lines
2.23.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm