Skip to content

Add AMD SEV and SEV-ES intra host migration support

Peter Gonda kvm lkml 13 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v11 Latest5 patches 8 repliesIn review
  2. v105 patches 5 repliesSuperseded
  3. v94 patches 3 repliesSuperseded
Earlier versions · 10
VersionPostedPatchesRepliesStatus
v84 patches11 repliesSuperseded
v7 posting 2/23 patches13 repliesSuperseded
v7 posting 1/20 of 2 patches2 repliesSuperseded
v6 posting 2/23 patches2 repliesSuperseded
v6 posting 1/20 of 2 patches0 repliesSuperseded
v52 patches1 replySuperseded
v42 patches9 repliesSuperseded
v33 patches2 repliesSuperseded
v23 patches2 repliesSuperseded
v13 patches10 repliesSuperseded

V5 from the cover letter

* Fix up locking scheme
* Address marcorr@ comments.

Patches in v11 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/5KVM: SEV: Refactor out sev_es_state structarch/x86/kvm/svm/sev.c arch/x86/kvm/svm/svm.c arch/x86/kvm/svm/svm.h+61 -561 reply
2/5KVM: SEV: Add support for SEV intra host migrationDocumentation/virt/kvm/api.rst arch/x86/include/asm/kvm_host.h arch/x86/kvm/svm/sev.c +4 more+163 -05 replies
3/5KVM: SEV: Add support for SEV-ES intra host migrationarch/x86/kvm/svm/sev.c+49 -11 reply
4/5selftest: KVM: Add open sev dev helpertools/testing/selftests/kvm/include/kvm_util.h tools/testing/selftests/kvm/include/x86_64/svm_util.h tools/testing/selftests/kvm/lib/kvm_util.c +1 more+30 -101 reply
5/5selftest: KVM: Add intra host migration teststools/testing/selftests/kvm/Makefile tools/testing/selftests/kvm/x86_64/sev_vm_tests.c+205 -10 replies

Reviews trailers per patch

3 of 5 patches have a Reviewed-by or Acked-by.

PatchTom LendackyMarc OrrSean Christopherson
1/5A-b R-b
2/5R-b
3/5R-b
4/5
5/5

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v11 thread

Cover letter [PATCH 0/5 V11] Add AMD SEV and SEV-ES intra host migration support

Intra host migration provides a low-cost mechanism for userspace VMM upgrades. It is an alternative to traditional (i.e., remote) live migration. Whereas remote migration handles moving a guest to a new host, intra host migration only handles moving a guest to a new userspace VMM within a host. This can be used to update, rollback, change flags of the VMM, etc. The lower cost compared to live migration comes from the fact that the guest's memory does not need to be copied between processes. A handle to the guest memory simply gets passed to the new VMM, this could be done via /dev/shm with share=on or similar feature.

The guest state can be transferred from an old VMM to a new VMM as follows:
1. Export guest state from KVM to the old user-space VMM via a getter user-space/kernel API 2. Transfer guest state from old VMM to new VMM via IPC communication 3. Import guest state into KVM from the new user-space VMM via a setter user-space/kernel API VMMs by exporting from KVM using getters, sending that data to the new VMM, then setting it again in KVM.

In the common case for intra host migration, we can rely on the normal ioctls for passing data from one VMM to the next. SEV, SEV-ES, and other confidential compute environments make most of this information opaque, and render KVM ioctls such as "KVM_GET_REGS" irrelevant. As a result, we need the ability to pass this opaque metadata from one VMM to the next. The easiest way to do this is to leave this data in the kernel, and transfer ownership of the metadata from one KVM VM (or vCPU) to the next. For example, we need to move the SEV enabled ASID, VMSAs, and GHCB metadata from one VMM to the next. In general, we need to be able to hand off any data that would be unsafe/impossible for the kernel to hand directly to userspace (and cannot be reproduced using data that can be handed safely to userspace).

V11
* Zero SEV-ES vCPU state on source.
* Rebase onto SEV-ES fixes.

V10
* Add new starting patch to refactor all SEV-ES related vCPU data into for easier copying.

V9
* Fix sev_lock_vcpus_for_migration from unlocking the vCPU mutex it failed to unlock.

V8
* Update to require that @dst is not SEV or SEV-ES enabled.
* Address selftest feedback.

V7
* Address selftest feedback.

V6
* Add selftest.

V5:
* Fix up locking scheme
* Address marcorr@ comments.

V4:
* Move to seanjc@'s suggestion of source VM FD based single ioctl design.

v3:
* Fix memory leak found by dan.carpenter@

v2:
* Added marcorr@ reviewed by tag
* Renamed function introduced in 1/3
* Edited with seanjc@'s review comments
** Cleaned up WARN usage
** Userspace makes random token now
* Edited with brijesh.singh@'s review comments ** Checks for different LAUNCH_* states in send function

v1: https://lore.kernel.org/kvm/[email protected]/

base-commit: 9f1ee7b169af

Cc Marc Orr <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc David Rientjes <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Tom Lendacky <[email protected]>
Cc Vitaly Kuznetsov <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Jim Mattson <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc "H. Peter Anvin" <[email protected]>

Peter Gonda (5):
Refactor out sev_es_state struct
KVM: SEV: Add support for SEV intra host migration
KVM: SEV: Add support for SEV-ES intra host migration
selftest: KVM: Add open sev dev helper
selftest: KVM: Add intra host migration tests

 Documentation/virt/kvm/api.rst                |  15 +
 arch/x86/include/asm/kvm_host.h               |   1 +
 arch/x86/kvm/svm/sev.c                        | 268 +++++++++++++++---
 arch/x86/kvm/svm/svm.c                        |   9 +-
 arch/x86/kvm/svm/svm.h                        |  28 +-
 arch/x86/kvm/x86.c                            |   6 +
 include/uapi/linux/kvm.h                      |   1 +
 tools/testing/selftests/kvm/Makefile          |   3 +-
 .../testing/selftests/kvm/include/kvm_util.h  |   1 +
 .../selftests/kvm/include/x86_64/svm_util.h   |   2 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |  24 +-
 tools/testing/selftests/kvm/lib/x86_64/svm.c  |  13 +
 .../selftests/kvm/x86_64/sev_vm_tests.c       | 203 +++++++++++++
 13 files changed, 507 insertions(+), 67 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86_64/sev_vm_tests.c
Signature · 1 line
2.33.0.1079.g6e70778dc9-goog