Skip to contentSkip to conversation

Add AMD Secure Nested Paging (SEV-SNP) support

RFC PATCH Applied Michael Roth 9 participants 110 messages Mar 20, 2024 – Jan 21, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v3 cover Add AMD Secure Nested Paging (SEV-SNP) support

These patches implement SEV-SNP base support along with CPUID enforcement support for QEMU, and are also available at:

https://github.com/amdese/qemu/commits/snp-v3-rfc

they are based on top of the following patchset from Paolo:

"[PATCH 0/7] target/i386: VM type infrastructure and KVM_SEV_INIT2 support"
https://lists.gnu.org/archive/html/qemu-devel/2024-03/msg04663.html

Patch Layout
------------

01-05: Various changes needed to handle new header files in kvm-next tree
and some hacks to get a functional header sync in place for building this series.
06-18: These are patches directly plucked from Xiaoyao's TDX v5 patchset[1]
that implement common dependencies between SNP/TDX like base guest_memfd, KVM_EXIT_MEMORY_FAULT handling (with a small FIXUP), and mechanisms to disable SMM. We would've also needed some of the basic infrastructure for handling specifying VM types for KVM_CREATE, but much of that is now part of the sevinit2 series this patchset is based on. Ideally all these patches, once stable, could be maintained in a common tree so that future SNP/TDX patchsets can be more easily iterated on/reviewed.
19-20: Patches introduced by this series that are possible candidate for a
common tree.
shared/private pages when things like VFIO are in use.
21-32: Introduction of sev-snp-guest object and various configuration
requirements for SNP.
33-36: Handling for various KVM_EXIT_VMGEXIT events that are handled in
userspace.
37-49: Support for creating a cryptographic "launch" context and populating
various OVMF metadata pages, BIOS regions, and vCPU/VMSA pages with the initial encrypted/measured/validated launch data prior to launching the SNP guest.

Testing
-------

This series has been tested against the following host kernel tree, which is a snapshot of the latest WIP SNP hypervisor tree at the time of this posting. It will likely not be kept up to date afterward, so please keep an eye upstream or official AMDESE github if you are looking for the latest some time after this posting:

https://github.com/mdroth/linux/commits/snp-host-v12-wip40/

A patched OVMF is also needed due to upstream KVM no longer supporting MMIO ranges that are mapped as private. It is recommended you build the AmdSevX64 variant as it provides the kernel-hashing support present in this series:

https://github.com/mdroth/edk2/commits/apic-mmio-fix1c/

A basic command-line invocation for SNP would be:

qemu-system-x86_64 -smp 32,maxcpus=255 -cpu EPYC-Milan-v2
-machine q35,confidential-guest-support=sev0,memory-backend=ram1 -object memory-backend-memfd,id=ram1,size=4G,share=true,reserve=false -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,id-auth= -bios /home/mroth/ovmf/OVMF_CODE-upstream-20240228-apicfix-1c-AmdSevX64.fd

With kernel-hashing and certificate data supplied:

qemu-system-x86_64 -smp 32,maxcpus=255 -cpu EPYC-Milan-v2
-machine q35,confidential-guest-support=sev0,memory-backend=ram1
-object memory-backend-memfd,id=ram1,size=4G,share=true,reserve=false
-object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,id-auth=,certs-path=/home/mroth/cert.blob,kernel-hashes=on -bios /home/mroth/ovmf/OVMF_CODE-upstream-20240228-apicfix-1c-AmdSevX64.fd
-kernel /boot/vmlinuz-6.8.0-snp-host-v12-wip40+
-initrd /boot/initrd.img-6.8.0-snp-host-v12-wip40+
-append "root=UUID=d72a6d1c-06cf-4b79-af43-f1bac4f620f9 ro console=ttyS0,115200n8"

Any comments/feedback would be very much appreciated.

[1] https://github.com/amdese/linux
https://github.com/amdese/amdsev/tree/snp-latest

Changes since rfc2:

- reworked on top of guest_memfd support
- added handling for various KVM_EXIT_VMGEXIT events
- various changes/considerations for PCI passthrough support
- general bugfixes/hardening/cleanups
- qapi cmdline doc fixes/rework (Dov, Markus)
- switch to qbase64_decode, more error-checking for cmdline opts (Dov)
- unset id_block_en for 0 input (Dov)
- use error_setg in snp init (Dov)
- report more info in trace_kvm_sev_init (Dov)
- rework bounds-checking for kvm_cpuid_info, rework existing checks for readability, add additional checks (Dov)
- fixups for validated_ranges handling (Dov)
- rename 'policy' field to 'snp-policy' in query-sev when sev-type is SNP

Changes since rfc1:

- rebased onto latest master
- drop SNP config file in favor of a new 'sev-snp-guest' object where all SNP-related params are passed as strings/integers via command-line
- report specific error if BIOS reports invalid address/len for reserved/pre-validated regions (Connor)
- use Range helpers for handling validated region overlaps (Dave)
- simplify error handling in sev_snp_launch_start, and report the correct return code when handling LAUNCH_START failures (Dov)
- add SEV-SNP bit to CPUID 0x8000001f when SNP enabled
- updated query-sev to handle differences between SEV and SEV-SNP
- updated to work against v5 of SEV-SNP host kernel / hypervisor patches

----------------------------------------------------------------
Brijesh Singh (5):
i386/sev: Introduce 'sev-snp-guest' object i386/sev: Add the SNP launch start context i386/sev: Add handling to encrypt/finalize guest launch data hw/i386/sev: Add function to get SEV metadata from OVMF header i386/sev: Add support for populating OVMF metadata pages

Chao Peng (2):
kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
kvm: handle KVM_EXIT_MEMORY_FAULT

Dov Murik (4):
qapi, i386: Move kernel-hashes to SevCommonProperties i386/sev: Extract build_kernel_loader_hashes i386/sev: Reorder struct declarations i386/sev: Allow measured direct kernel boot on SNP

Isaku Yamahata (2):
pci-host/q35: Move PAM initialization above SMRAM initialization
q35: Introduce smm_ranges property for q35-pci-host

Michael Roth (30):
Revert "linux-headers hack" from sevinit2 base tree scripts/update-linux-headers: Add setup_data.h to import list scripts/update-linux-headers: Add bits.h to file imports [HACK] linux-headers: Update headers for 6.8 + kvm-coco-queue + SNP [TEMP] hw/i386: Remove redeclaration of struct setup_data
RAMBlock: Add support of KVM private guest memfd
[FIXUP] "kvm: handle KVM_EXIT_MEMORY_FAULT": drop qemu_host_page_size trace/kvm: Add trace for page convertion between shared and private
kvm: Make kvm_convert_memory() obey ram_block_discard_is_enabled() trace/kvm: Add trace for KVM_EXIT_MEMORY_FAULT
i386/sev: Introduce "sev-common" type to encapsulate common SEV state i386/sev: Add a sev_snp_enabled() helper
target/i386: Add handling for KVM_X86_SNP_VM VM type i386/sev: Skip RAMBlock notifiers for SNP
i386/sev: Skip machine-init-done notifiers for SNP
i386/sev: Set ms->require_guest_memfd for SNP
i386/sev: Disable SMM for SNP
i386/sev: Don't disable block discarding for SNP
i386/cpu: Set SEV-SNP CPUID bit when SNP enabled
i386/sev: Update query-sev QAPI format to handle SEV-SNP i386/sev: Don't return launch measurements for SEV-SNP guests
kvm: Make kvm_convert_memory() non-static
i386/sev: Add KVM_EXIT_VMGEXIT handling for Page State Changes i386/sev: Add KVM_EXIT_VMGEXIT handling for Page State Changes (MSR-based) i386/sev: Add KVM_EXIT_VMGEXIT handling for Extended Guest Requests i386/sev: Set CPU state to protected once SNP guest payload is finalized i386/sev: Add support for SNP CPUID validation
hw/i386/sev: Add support to encrypt BIOS when SEV-SNP is enabled hw/i386/sev: Use guest_memfd for legacy ROMs
hw/i386: Add support for loading BIOS using guest_memfd

Xiaoyao Li (6):
HostMem: Add mechanism to opt in kvm guest memfd via MachineState trace/kvm: Split address space and slot id in trace_kvm_set_user_memory()
kvm: Introduce support for memory_attributes
physmem: Introduce ram_block_discard_guest_memfd_range() kvm/memory: Make memory type private by default if it has guest memfd backend
memory: Introduce memory_region_init_ram_guest_memfd()

 accel/kvm/kvm-all.c                                |  241 ++-
 accel/kvm/trace-events                             |    4 +-
 accel/stubs/kvm-stub.c                             |    5 +
 backends/hostmem-file.c                            |    1 +
 backends/hostmem-memfd.c                           |    1 +
 backends/hostmem-ram.c                             |    1 +
 backends/hostmem.c                                 |    1 +
 docs/system/i386/amd-memory-encryption.rst         |   78 +-
 hw/core/machine.c                                  |    5 +
 hw/i386/pc.c                                       |   13 +-
 hw/i386/pc_q35.c                                   |    2 +
 hw/i386/pc_sysfw.c                                 |   25 +-
 hw/i386/pc_sysfw_ovmf.c                            |   33 +
 hw/i386/x86.c                                      |   46 +-
 hw/pci-host/q35.c                                  |   61 +-
 include/exec/cpu-common.h                          |    2 +
 include/exec/memory.h                              |   26 +-
 include/exec/ram_addr.h                            |    2 +-
 include/exec/ramblock.h                            |    1 +
 include/hw/boards.h                                |    2 +
 include/hw/i386/pc.h                               |   31 +-
 include/hw/i386/x86.h                              |    2 +-
 include/hw/pci-host/q35.h                          |    1 +
 include/standard-headers/asm-x86/bootparam.h       |   17 +-
 include/standard-headers/asm-x86/kvm_para.h        |    3 +-
 include/standard-headers/linux/ethtool.h           |   48 +
 include/standard-headers/linux/fuse.h              |   39 +-
 include/standard-headers/linux/input-event-codes.h |    1 +
 include/standard-headers/linux/virtio_gpu.h        |    2 +
 include/standard-headers/linux/virtio_snd.h        |  154 ++
 include/sysemu/hostmem.h                           |    1 +
 include/sysemu/kvm.h                               |    7 +
 include/sysemu/kvm_int.h                           |    2 +
 linux-headers/asm-arm64/kvm.h                      |   15 +-
 linux-headers/asm-arm64/sve_context.h              |   11 +
 linux-headers/asm-generic/bitsperlong.h            |    4 +
 linux-headers/asm-loongarch/kvm.h                  |    2 -
 linux-headers/asm-mips/kvm.h                       |    2 -
 linux-headers/asm-powerpc/kvm.h                    |   45 +-
 linux-headers/asm-riscv/kvm.h                      |    3 +-
 linux-headers/asm-s390/kvm.h                       |  315 +++-
 linux-headers/asm-x86/kvm.h                        |  372 ++++-
 linux-headers/asm-x86/setup_data.h                 |   83 +
 linux-headers/linux/bits.h                         |   15 +
 linux-headers/linux/kvm.h                          |  719 +--------
 linux-headers/linux/psp-sev.h                      |   71 +
 qapi/misc-target.json                              |   71 +-
 qapi/qom.json                                      |   96 +-
 scripts/update-linux-headers.sh                    |    5 +-
 system/memory.c                                    |   30 +
 system/physmem.c                                   |   47 +-
 target/i386/cpu.c                                  |    1 +
 target/i386/kvm/kvm.c                              |    4 +
 target/i386/sev-sysemu-stub.c                      |    2 +-
 target/i386/sev.c                                  | 1631 ++++++++++++++++----
 target/i386/sev.h                                  |   13 +-
 target/i386/trace-events                           |    3 +
 57 files changed, 3272 insertions(+), 1146 deletions(-)
 create mode 100644 linux-headers/asm-x86/setup_data.h
 create mode 100644 linux-headers/linux/bits.h
#1 lore raw

PATCH v3 1/49 Revert "linux-headers hack" from sevinit2 base tree

TODO: Either apply this in advance of sevinit2 patches, or drop this in favor of a separate preceeding sync of 6.8 kvm-next.

A separate standalone linux-headers sync will be used instead.

---
 linux-headers/asm-x86/kvm.h | 8 --------
 linux-headers/linux/kvm.h   | 2 --
 2 files changed, 10 deletions(-)
linux-headers/asm-x86/kvm.h +0 -8
index 8f58c32d37..003fb74534 100644
@@ -562,13 +562,5 @@ struct kvm_pmu_event_filter {
 
 #define KVM_X86_DEFAULT_VM	0
 #define KVM_X86_SW_PROTECTED_VM	1
-#define KVM_X86_SEV_VM          2
-#define KVM_X86_SEV_ES_VM       3
-
-struct kvm_sev_init {
-        __u64 vmsa_features;
-        __u32 flags;
-        __u32 pad[9];
-};
 
 #endif /* _ASM_X86_KVM_H */
linux-headers/linux/kvm.h +0 -2
index 5fd84fd7d0..17839229b2 100644
@@ -1865,8 +1865,6 @@ enum sev_cmd_id {
 	/* Guest Migration Extension */
 	KVM_SEV_SEND_CANCEL,
 
-	KVM_SEV_INIT2,
-
 	KVM_SEV_NR_MAX,
 };
 
Signature · 1 line
2.25.1
#2 lore raw

PATCH v3 2/49 scripts/update-linux-headers: Add setup_data.h to import list

In reply to Michael Roth

Data structures like struct setup_data have been moved to a separate setup_data.h header which bootparam.h relies on. Add setup_data.h to the cp_portable() list and sync it along with the other header files.

Note that currently struct setup_data is stripped away as part of generating bootparam.h, but that handling is no currently needed for setup_data.h since it doesn't pull in many external headers/dependencies. However, QEMU currently redefines struct setup_data in hw/i386/x86.c, so that will need to be removed as part of any header update that pulls in the new setup_data.h to avoid build bisect breakage.

---
 scripts/update-linux-headers.sh | 3 +++
 1 file changed, 3 insertions(+)
scripts/update-linux-headers.sh +3 -0
index a0006eec6f..579b03dc82 100755
@@ -62,6 +62,7 @@ cp_portable() {
                                      -e 'linux/kernel' \
                                      -e 'linux/sysinfo' \
                                      -e 'asm-generic/kvm_para' \
+                                     -e 'asm/setup_data.h' \
                                      > /dev/null
     then
         echo "Unexpected #include in input file $f".
@@ -155,6 +156,8 @@ for arch in $ARCHLIST; do
                "$tmpdir/include/asm/bootparam.h" > "$tmpdir/bootparam.h"
         cp_portable "$tmpdir/bootparam.h" \
                     "$output/include/standard-headers/asm-$arch"
+        cp_portable "$tmpdir/include/asm/setup_data.h" \
+                    "$output/linux-headers/asm-x86"
     fi
     if [ $arch = riscv ]; then
         cp "$tmpdir/include/asm/ptrace.h" "$output/linux-headers/asm-riscv/"
Signature · 1 line
2.25.1
maintainer Applied
#3 lore raw

Re: [PATCH v3 02/49] scripts/update-linux-headers: Add setup_data.h to import list

Quoted 12 lines · Michael Roth

Data structures like struct setup_data have been moved to a separate setup_data.h header which bootparam.h relies on. Add setup_data.h to the cp_portable() list and sync it along with the other header files.

Note that currently struct setup_data is stripped away as part of generating bootparam.h, but that handling is no currently needed for setup_data.h since it doesn't pull in many external headers/dependencies. However, QEMU currently redefines struct setup_data in hw/i386/x86.c, so that will need to be removed as part of any header update that pulls in the new setup_data.h to avoid build bisect breakage.

Quoted 1 line

Including Linux headers from standard-headers breaks build on non-Linux systems, and <asm/setup_data.h> is the first architecture specific #include in include/standard-headers/.

So this needs a small fixup, to rewrite asm/ include to the standard-headers/asm-* subdirectory for the current architecture.

While at it, we should remove asm-generic/kvm_para.h from the list of allowed includes: it does not have a matching substitution, so if it appeared it would break the build it on non-Linux systems where there is no /usr/include/asm-generic/ directory.

Applied patches 2-5 to my QEMU coco tree - still temporary, but certainly better than the hack that I posted yesterday. By the time QEMU 9.1 opens there will be something more stable to import from.

Paolo

scripts/update-linux-headers.sh +3 -2
index 579b03dc824..d48856f9e24 100755
@@ -61,7 +61,6 @@ cp_portable() {
                                      -e 'linux/const' \
                                      -e 'linux/kernel' \
                                      -e 'linux/sysinfo' \
-                                     -e 'asm-generic/kvm_para' \
                                      -e 'asm/setup_data.h' \
                                      > /dev/null
     then
@@ -78,6 +77,7 @@ cp_portable() {
         -e 's/__be\([0-9][0-9]*\)/uint\1_t/g' \
         -e 's/"\(input-event-codes\.h\)"/"standard-headers\/linux\/\1"/' \
         -e 's/<linux\/\([^>]*\)>/"standard-headers\/linux\/\1"/' \
+        -e 's/<asm\/\([^>]*\)>/"standard-headers\/asm-'$arch'\/\1"/' \
         -e 's/__bitwise//' \
         -e 's/__attribute__((packed))/QEMU_PACKED/' \
         -e 's/__inline__/inline/' \
@@ -157,12 +157,13 @@ for arch in $ARCHLIST; do
         cp_portable "$tmpdir/bootparam.h" \
                     "$output/include/standard-headers/asm-$arch"
         cp_portable "$tmpdir/include/asm/setup_data.h" \
-                    "$output/linux-headers/asm-x86"
+                    "$output/standard-headers/asm-x86"
     fi
     if [ $arch = riscv ]; then
         cp "$tmpdir/include/asm/ptrace.h" "$output/linux-headers/asm-riscv/"
     fi
 done
+arch=
 
 rm -rf "$output/linux-headers/linux"
 mkdir -p "$output/linux-headers/linux"
#4 lore raw

PATCH v3 3/49 scripts/update-linux-headers: Add bits.h to file imports

In reply to Michael Roth
---
 scripts/update-linux-headers.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/update-linux-headers.sh +1 -1
index 579b03dc82..b992ed7b15 100755
@@ -168,7 +168,7 @@ rm -rf "$output/linux-headers/linux"
 mkdir -p "$output/linux-headers/linux"
 for header in const.h stddef.h kvm.h vfio.h vfio_ccw.h vfio_zdev.h vhost.h \
               psci.h psp-sev.h userfaultfd.h memfd.h mman.h nvme_ioctl.h \
-              vduse.h iommufd.h; do
+              vduse.h iommufd.h bits.h; do
     cp "$tmpdir/include/linux/$header" "$output/linux-headers/linux"
 done
 
Signature · 1 line
2.25.1
#5 lore raw

PATCH v3 4/49 linux-headers: Update headers for 6.8 + kvm-coco-queue + SNP

In reply to Michael Roth

Pull in 6.8 kvm-next + kvm-coco-queue + SNP headers.

Be careful to omit removing the following virtio_pci.h definitions which are no longer present upstream, since QEMU still relies on them:

  #define LM_LOGGING_CTRL                 0
  #define LM_BASE_ADDR_LOW                4
  #define LM_BASE_ADDR_HIGH               8
  #define LM_END_ADDR_LOW                 12
  #define LM_END_ADDR_HIGH                16
  #define LM_VRING_STATE_OFFSET           0x20
---
 include/standard-headers/asm-x86/bootparam.h  |  17 +-
 include/standard-headers/asm-x86/kvm_para.h   |   3 +-
 include/standard-headers/linux/ethtool.h      |  48 ++
 include/standard-headers/linux/fuse.h         |  39 +-
 .../linux/input-event-codes.h                 |   1 +
 include/standard-headers/linux/virtio_gpu.h   |   2 +
 include/standard-headers/linux/virtio_snd.h   | 154 ++++
 linux-headers/asm-arm64/kvm.h                 |  15 +-
 linux-headers/asm-arm64/sve_context.h         |  11 +
 linux-headers/asm-generic/bitsperlong.h       |   4 +
 linux-headers/asm-loongarch/kvm.h             |   2 -
 linux-headers/asm-mips/kvm.h                  |   2 -
 linux-headers/asm-powerpc/kvm.h               |  45 +-
 linux-headers/asm-riscv/kvm.h                 |   3 +-
 linux-headers/asm-s390/kvm.h                  | 315 +++++++-
 linux-headers/asm-x86/kvm.h                   | 364 ++++++++-
 linux-headers/asm-x86/setup_data.h            |  83 ++
 linux-headers/linux/bits.h                    |  15 +
 linux-headers/linux/kvm.h                     | 717 +-----------------
 linux-headers/linux/psp-sev.h                 |  71 ++
 20 files changed, 1186 insertions(+), 725 deletions(-)
 create mode 100644 linux-headers/asm-x86/setup_data.h
 create mode 100644 linux-headers/linux/bits.h
include/standard-headers/asm-x86/bootparam.h +2 -15
index 0b06d2bff1..62e4cd5390 100644
@@ -2,21 +2,7 @@
 #ifndef _ASM_X86_BOOTPARAM_H
 #define _ASM_X86_BOOTPARAM_H
 
-/* setup_data/setup_indirect types */
-#define SETUP_NONE			0
-#define SETUP_E820_EXT			1
-#define SETUP_DTB			2
-#define SETUP_PCI			3
-#define SETUP_EFI			4
-#define SETUP_APPLE_PROPERTIES		5
-#define SETUP_JAILHOUSE			6
-#define SETUP_CC_BLOB			7
-#define SETUP_IMA			8
-#define SETUP_RNG_SEED			9
-#define SETUP_ENUM_MAX			SETUP_RNG_SEED
-
-#define SETUP_INDIRECT			(1<<31)
-#define SETUP_TYPE_MAX			(SETUP_ENUM_MAX | SETUP_INDIRECT)
+#include <asm/setup_data.h>
 
 /* ram_size flags */
 #define RAMDISK_IMAGE_START_MASK	0x07FF
@@ -38,6 +24,7 @@
 #define XLF_EFI_KEXEC			(1<<4)
 #define XLF_5LEVEL			(1<<5)
 #define XLF_5LEVEL_ENABLED		(1<<6)
+#define XLF_MEM_ENCRYPTION		(1<<7)
 
 
 #endif /* _ASM_X86_BOOTPARAM_H */
include/standard-headers/asm-x86/kvm_para.h +1 -2
index f0235e58a1..9a011d20f0 100644
@@ -92,7 +92,7 @@ struct kvm_clock_pairing {
 #define KVM_ASYNC_PF_DELIVERY_AS_INT		(1 << 3)
 
 /* MSR_KVM_ASYNC_PF_INT */
-#define KVM_ASYNC_PF_VEC_MASK			GENMASK(7, 0)
+#define KVM_ASYNC_PF_VEC_MASK			__GENMASK(7, 0)
 
 /* MSR_KVM_MIGRATION_CONTROL */
 #define KVM_MIGRATION_READY		(1 << 0)
@@ -142,7 +142,6 @@ struct kvm_vcpu_pv_apf_data {
 	uint32_t token;
 
 	uint8_t pad[56];
-	uint32_t enabled;
 };
 
 #define KVM_PV_EOI_BIT 0
include/standard-headers/linux/ethtool.h +48 -0

60 lines not on this page · raw message

include/standard-headers/linux/fuse.h +36 -3
index fc0dcd10ae..bac9dbc49f 100644
@@ -211,6 +211,12 @@
  *  7.39
  *  - add FUSE_DIRECT_IO_ALLOW_MMAP
  *  - add FUSE_STATX and related structures
+ *
+ *  7.40
+ *  - add max_stack_depth to fuse_init_out, add FUSE_PASSTHROUGH init flag
+ *  - add backing_id to fuse_open_out, add FOPEN_PASSTHROUGH open flag
+ *  - add FUSE_NO_EXPORT_SUPPORT init flag
+ *  - add FUSE_NOTIFY_RESEND, add FUSE_HAS_RESEND init flag
  */
 
 #ifndef _LINUX_FUSE_H
@@ -242,7 +248,7 @@
 #define FUSE_KERNEL_VERSION 7
 
 /** Minor version number of this interface */
-#define FUSE_KERNEL_MINOR_VERSION 39
+#define FUSE_KERNEL_MINOR_VERSION 40
 
 /** The node ID of the root inode */
 #define FUSE_ROOT_ID 1
@@ -349,6 +355,7 @@ struct fuse_file_lock {
  * FOPEN_STREAM: the file is stream-like (no file position at all)
  * FOPEN_NOFLUSH: don't flush data cache on close (unless FUSE_WRITEBACK_CACHE)
  * FOPEN_PARALLEL_DIRECT_WRITES: Allow concurrent direct writes on the same inode
+ * FOPEN_PASSTHROUGH: passthrough read/write io for this open file
  */
 #define FOPEN_DIRECT_IO		(1 << 0)
 #define FOPEN_KEEP_CACHE	(1 << 1)
@@ -357,6 +364,7 @@ struct fuse_file_lock {
 #define FOPEN_STREAM		(1 << 4)
 #define FOPEN_NOFLUSH		(1 << 5)
 #define FOPEN_PARALLEL_DIRECT_WRITES	(1 << 6)
+#define FOPEN_PASSTHROUGH	(1 << 7)
 
 /**
  * INIT request/reply flags
@@ -406,6 +414,9 @@ struct fuse_file_lock {
  *			symlink and mknod (single group that matches parent)
  * FUSE_HAS_EXPIRE_ONLY: kernel supports expiry-only entry invalidation
  * FUSE_DIRECT_IO_ALLOW_MMAP: allow shared mmap in FOPEN_DIRECT_IO mode.
+ * FUSE_NO_EXPORT_SUPPORT: explicitly disable export support
+ * FUSE_HAS_RESEND: kernel supports resending pending requests, and the high bit
+ *		    of the request ID indicates resend requests
  */
 #define FUSE_ASYNC_READ		(1 << 0)
 #define FUSE_POSIX_LOCKS	(1 << 1)
@@ -445,6 +456,9 @@ struct fuse_file_lock {
 #define FUSE_CREATE_SUPP_GROUP	(1ULL << 34)
 #define FUSE_HAS_EXPIRE_ONLY	(1ULL << 35)
 #define FUSE_DIRECT_IO_ALLOW_MMAP (1ULL << 36)
+#define FUSE_PASSTHROUGH	(1ULL << 37)
+#define FUSE_NO_EXPORT_SUPPORT	(1ULL << 38)
+#define FUSE_HAS_RESEND		(1ULL << 39)
 
 /* Obsolete alias for FUSE_DIRECT_IO_ALLOW_MMAP */
 #define FUSE_DIRECT_IO_RELAX	FUSE_DIRECT_IO_ALLOW_MMAP
@@ -631,6 +645,7 @@ enum fuse_notify_code {
 	FUSE_NOTIFY_STORE = 4,
 	FUSE_NOTIFY_RETRIEVE = 5,
 	FUSE_NOTIFY_DELETE = 6,
+	FUSE_NOTIFY_RESEND = 7,
 	FUSE_NOTIFY_CODE_MAX,
 };
 
@@ -757,7 +772,7 @@ struct fuse_create_in {
 struct fuse_open_out {
 	uint64_t	fh;
 	uint32_t	open_flags;
-	uint32_t	padding;
+	int32_t		backing_id;
 };
 
 struct fuse_release_in {
@@ -873,7 +888,8 @@ struct fuse_init_out {
 	uint16_t	max_pages;
 	uint16_t	map_alignment;
 	uint32_t	flags2;
-	uint32_t	unused[7];
+	uint32_t	max_stack_depth;
+	uint32_t	unused[6];
 };
 
 #define CUSE_INIT_INFO_MAX 4096
@@ -956,6 +972,14 @@ struct fuse_fallocate_in {
 	uint32_t	padding;
 };
 
+/**
+ * FUSE request unique ID flag
+ *
+ * Indicates whether this is a resend request. The receiver should handle this
+ * request accordingly.
+ */
+#define FUSE_UNIQUE_RESEND (1ULL << 63)
+
 struct fuse_in_header {
 	uint32_t	len;
 	uint32_t	opcode;
@@ -1045,9 +1069,18 @@ struct fuse_notify_retrieve_in {
 	uint64_t	dummy4;
 };
 
+struct fuse_backing_map {
+	int32_t		fd;
+	uint32_t	flags;
+	uint64_t	padding;
+};
+
 /* Device ioctls: */
 #define FUSE_DEV_IOC_MAGIC		229
 #define FUSE_DEV_IOC_CLONE		_IOR(FUSE_DEV_IOC_MAGIC, 0, uint32_t)
+#define FUSE_DEV_IOC_BACKING_OPEN	_IOW(FUSE_DEV_IOC_MAGIC, 1, \
+					     struct fuse_backing_map)
+#define FUSE_DEV_IOC_BACKING_CLOSE	_IOW(FUSE_DEV_IOC_MAGIC, 2, uint32_t)
 
 struct fuse_lseek_in {
 	uint64_t	fh;
include/standard-headers/linux/input-event-codes.h +1 -0
index f6bab08540..2221b0c383 100644
@@ -602,6 +602,7 @@
 
 #define KEY_ALS_TOGGLE		0x230	/* Ambient light sensor */
 #define KEY_ROTATE_LOCK_TOGGLE	0x231	/* Display rotation lock */
+#define KEY_REFRESH_RATE_TOGGLE	0x232	/* Display refresh rate toggle */
 
 #define KEY_BUTTONCONFIG		0x240	/* AL Button Configuration */
 #define KEY_TASKMANAGER		0x241	/* AL Task/Project Manager */
include/standard-headers/linux/virtio_gpu.h +2 -0
index 2da48d3d4c..2db643ed8f 100644
@@ -309,6 +309,8 @@ struct virtio_gpu_cmd_submit {
 
 #define VIRTIO_GPU_CAPSET_VIRGL 1
 #define VIRTIO_GPU_CAPSET_VIRGL2 2
+/* 3 is reserved for gfxstream */
+#define VIRTIO_GPU_CAPSET_VENUS 4
 
 /* VIRTIO_GPU_CMD_GET_CAPSET_INFO */
 struct virtio_gpu_get_capset_info {
include/standard-headers/linux/virtio_snd.h +154 -0

182 lines not on this page · raw message

linux-headers/asm-arm64/kvm.h +9 -6
index c59ea55cd8..2af9931ae9 100644
@@ -37,9 +37,7 @@
 #include <asm/ptrace.h>
 #include <asm/sve_context.h>
 
-#define __KVM_HAVE_GUEST_DEBUG
 #define __KVM_HAVE_IRQ_LINE
-#define __KVM_HAVE_READONLY_MEM
 #define __KVM_HAVE_VCPU_EVENTS
 
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
@@ -76,11 +74,11 @@ struct kvm_regs {
 
 /* KVM_ARM_SET_DEVICE_ADDR ioctl id encoding */
 #define KVM_ARM_DEVICE_TYPE_SHIFT	0
-#define KVM_ARM_DEVICE_TYPE_MASK	GENMASK(KVM_ARM_DEVICE_TYPE_SHIFT + 15, \
-						KVM_ARM_DEVICE_TYPE_SHIFT)
+#define KVM_ARM_DEVICE_TYPE_MASK	__GENMASK(KVM_ARM_DEVICE_TYPE_SHIFT + 15, \
+						  KVM_ARM_DEVICE_TYPE_SHIFT)
 #define KVM_ARM_DEVICE_ID_SHIFT		16
-#define KVM_ARM_DEVICE_ID_MASK		GENMASK(KVM_ARM_DEVICE_ID_SHIFT + 15, \
-						KVM_ARM_DEVICE_ID_SHIFT)
+#define KVM_ARM_DEVICE_ID_MASK		__GENMASK(KVM_ARM_DEVICE_ID_SHIFT + 15, \
+						  KVM_ARM_DEVICE_ID_SHIFT)
 
 /* Supported device IDs */
 #define KVM_ARM_DEVICE_VGIC_V2		0
@@ -162,6 +160,11 @@ struct kvm_sync_regs {
 	__u64 device_irq_level;
 };
 
+/* Bits for run->s.regs.device_irq_level */
+#define KVM_ARM_DEV_EL1_VTIMER		(1 << 0)
+#define KVM_ARM_DEV_EL1_PTIMER		(1 << 1)
+#define KVM_ARM_DEV_PMU			(1 << 2)
+
 /*
  * PMU filter structure. Describe a range of events with a particular
  * action. To be used with KVM_ARM_VCPU_PMU_V3_FILTER.
linux-headers/asm-arm64/sve_context.h +11 -0
index 1d0e3e1d09..d1b1ec8cb1 100644
@@ -13,6 +13,17 @@
 
 #define __SVE_VQ_BYTES		16	/* number of bytes per quadword */
 
+/*
+ * Yes, __SVE_VQ_MAX is 512 QUADWORDS.
+ *
+ * To help ensure forward portability, this is much larger than the
+ * current maximum value defined by the SVE architecture.  While arrays
+ * or static allocations can be sized based on this value, watch out!
+ * It will waste a surprisingly large amount of memory.
+ *
+ * Dynamic sizing based on the actual runtime vector length is likely to
+ * be preferable for most purposes.
+ */
 #define __SVE_VQ_MIN		1
 #define __SVE_VQ_MAX		512
 
linux-headers/asm-generic/bitsperlong.h +4 -0
index 75f320fa91..1fb4f0c9f2 100644
@@ -24,4 +24,8 @@
 #endif
 #endif
 
+#ifndef __BITS_PER_LONG_LONG
+#define __BITS_PER_LONG_LONG 64
+#endif
+
 #endif /* __ASM_GENERIC_BITS_PER_LONG */
linux-headers/asm-loongarch/kvm.h +0 -2
index 923d0bd382..109785922c 100644
@@ -14,8 +14,6 @@
  * Some parts derived from the x86 version of this file.
  */
 
-#define __KVM_HAVE_READONLY_MEM
-
 #define KVM_COALESCED_MMIO_PAGE_OFFSET	1
 #define KVM_DIRTY_LOG_PAGE_OFFSET	64
 
linux-headers/asm-mips/kvm.h +0 -2
index edcf717c43..9673dc9cb3 100644
@@ -20,8 +20,6 @@
  * Some parts derived from the x86 version of this file.
  */
 
-#define __KVM_HAVE_READONLY_MEM
-
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
 
 /*
linux-headers/asm-powerpc/kvm.h +44 -1

55 lines not on this page · raw message

linux-headers/asm-riscv/kvm.h +2 -1
index 7499e88a94..b1c503c295 100644
@@ -16,7 +16,6 @@
 #include <asm/ptrace.h>
 
 #define __KVM_HAVE_IRQ_LINE
-#define __KVM_HAVE_READONLY_MEM
 
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
 
@@ -166,6 +165,8 @@ enum KVM_RISCV_ISA_EXT_ID {
 	KVM_RISCV_ISA_EXT_ZVFH,
 	KVM_RISCV_ISA_EXT_ZVFHMIN,
 	KVM_RISCV_ISA_EXT_ZFA,
+	KVM_RISCV_ISA_EXT_ZTSO,
+	KVM_RISCV_ISA_EXT_ZACAS,
 	KVM_RISCV_ISA_EXT_MAX,
 };
 
linux-headers/asm-s390/kvm.h +314 -1

321 lines not on this page · raw message

linux-headers/asm-x86/kvm.h +356 -8

410 lines not on this page · raw message

linux-headers/asm-x86/setup_data.h new file +83 -0

83 lines not on this page · raw message

linux-headers/linux/bits.h new file +15 -0
new file mode 100644
index 0000000000..d9897771be
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/* bits.h: Macros for dealing with bitmasks.  */
+
+#ifndef _LINUX_BITS_H
+#define _LINUX_BITS_H
+
+#define __GENMASK(h, l) \
+        (((~_UL(0)) - (_UL(1) << (l)) + 1) & \
+         (~_UL(0) >> (__BITS_PER_LONG - 1 - (h))))
+
+#define __GENMASK_ULL(h, l) \
+        (((~_ULL(0)) - (_ULL(1) << (l)) + 1) & \
+         (~_ULL(0) >> (__BITS_PER_LONG_LONG - 1 - (h))))
+
+#endif /* _LINUX_BITS_H */
linux-headers/linux/kvm.h +33 -684

820 lines not on this page · raw message

linux-headers/linux/psp-sev.h +71 -0

89 lines not on this page · raw message

Signature · 1 line
2.25.1
#6 lore raw

PATCH v3 5/49 hw/i386: Remove redeclaration of struct setup_data

In reply to Michael Roth

TODO: this needs to be done as part of header update to avoid temporary build bisect breakage. Keeping it separate for reference.

It is now provided by kernel headers.

---
 hw/i386/x86.c | 8 --------
 1 file changed, 8 deletions(-)
hw/i386/x86.c +0 -8
index 2d4b148cd2..825dc4c735 100644
@@ -679,14 +679,6 @@ DeviceState *ioapic_init_secondary(GSIState *gsi_state)
     return dev;
 }
 
-struct setup_data {
-    uint64_t next;
-    uint32_t type;
-    uint32_t len;
-    uint8_t data[];
-} __attribute__((packed));
-
-
 /*
  * The entry point into the kernel for PVH boot is different from
  * the native entry point.  The PVH entry is defined by the x86/HVM
Signature · 1 line
2.25.1
#7 lore raw

PATCH v3 6/49 RAMBlock: Add support of KVM private guest memfd

In reply to Michael Roth

Add KVM guest_memfd support to RAMBlock so both normal hva based memory and kvm guest memfd based private memory can be associated in one RAMBlock.

Introduce new flag RAM_GUEST_MEMFD. When it's set, it calls KVM ioctl to create private guest_memfd during RAMBlock setup.

Allocating a new RAM_GUEST_MEMFD flag to instruct the setup of guest memfd is more flexible and extensible than simply relying on the VM type because in the future we may have the case that not all the memory of a VM need guest memfd. As a benefit, it also avoid getting MachineState in memory subsystem.

Note, RAM_GUEST_MEMFD is supposed to be set for memory backends of confidential guests, such as TDX VM. How and when to set it for memory backends will be implemented in the following patches.

Introduce memory_region_has_guest_memfd() to query if the MemoryRegion has KVM guest_memfd allocated.

---
Changes in v5:
- Use assert(new_block->guest_memfd < 0) instead of condition check;
- Collect Reviewed-by tag from David;

Changes in v4:
- Add clarification of RAM_GUEST_MEMFD in commit message; (David Hildenbrand)
- refine the return value and error message; (Daniel P. Berrangé)
- remove flags in ram_block_add(); (David Hildenbrand)

Changes in v3:
- rename gmem to guest_memfd;
- close(guest_memfd) when RAMBlock is released; (Daniel P. Berrangé)
- Suqash the patch that introduces memory_region_has_guest_memfd().

---
 accel/kvm/kvm-all.c     | 25 +++++++++++++++++++++++++
 accel/stubs/kvm-stub.c  |  5 +++++
 include/exec/memory.h   | 20 +++++++++++++++++---
 include/exec/ram_addr.h |  2 +-
 include/exec/ramblock.h |  1 +
 include/sysemu/kvm.h    |  2 ++
 system/memory.c         |  5 +++++
 system/physmem.c        | 24 +++++++++++++++++++++---
 8 files changed, 77 insertions(+), 7 deletions(-)
accel/kvm/kvm-all.c +25 -0
index a05dea2313..132ab65df5 100644
@@ -91,6 +91,7 @@ bool kvm_msi_use_devid;
 static bool kvm_has_guest_debug;
 static int kvm_sstep_flags;
 static bool kvm_immediate_exit;
+static bool kvm_guest_memfd_supported;
 static hwaddr kvm_max_slot_size = ~0;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -2395,6 +2396,8 @@ static int kvm_init(MachineState *ms)
     }
     s->as = g_new0(struct KVMAs, s->nr_as);
 
+    kvm_guest_memfd_supported = kvm_check_extension(s, KVM_CAP_GUEST_MEMFD);
+
     if (object_property_find(OBJECT(current_machine), "kvm-type")) {
         g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine),
                                                             "kvm-type",
@@ -4099,3 +4102,25 @@ void kvm_mark_guest_state_protected(void)
 {
     kvm_state->guest_state_protected = true;
 }
+
+int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp)
+{
+    int fd;
+    struct kvm_create_guest_memfd guest_memfd = {
+        .size = size,
+        .flags = flags,
+    };
+
+    if (!kvm_guest_memfd_supported) {
+        error_setg(errp, "KVM doesn't support guest memfd\n");
+        return -1;
+    }
+
+    fd = kvm_vm_ioctl(kvm_state, KVM_CREATE_GUEST_MEMFD, &guest_memfd);
+    if (fd < 0) {
+        error_setg_errno(errp, errno, "Error creating kvm guest memfd");
+        return -1;
+    }
+
+    return fd;
+}
accel/stubs/kvm-stub.c +5 -0
index ca38172884..8e0eb22e61 100644
@@ -129,3 +129,8 @@ bool kvm_hwpoisoned_mem(void)
 {
     return false;
 }
+
+int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp)
+{
+    return -ENOSYS;
+}
include/exec/memory.h +17 -3
index 8626a355b3..679a847685 100644
@@ -243,6 +243,9 @@ typedef struct IOMMUTLBEvent {
 /* RAM FD is opened read-only */
 #define RAM_READONLY_FD (1 << 11)
 
+/* RAM can be private that has kvm guest memfd backend */
+#define RAM_GUEST_MEMFD   (1 << 12)
+
 static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn,
                                        IOMMUNotifierFlag flags,
                                        hwaddr start, hwaddr end,
@@ -1307,7 +1310,8 @@ bool memory_region_init_ram_nomigrate(MemoryRegion *mr,
  * @name: Region name, becomes part of RAMBlock name used in migration stream
  *        must be unique within any device
  * @size: size of the region.
- * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_NORESERVE.
+ * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_NORESERVE,
+ *             RAM_GUEST_MEMFD.
  * @errp: pointer to Error*, to store an error if it happens.
  *
  * Note that this function does not do anything to cause the data in the
@@ -1369,7 +1373,7 @@ bool memory_region_init_resizeable_ram(MemoryRegion *mr,
  *         (getpagesize()) will be used.
  * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM,
  *             RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY,
- *             RAM_READONLY_FD
+ *             RAM_READONLY_FD, RAM_GUEST_MEMFD
  * @path: the path in which to allocate the RAM.
  * @offset: offset within the file referenced by path
  * @errp: pointer to Error*, to store an error if it happens.
@@ -1399,7 +1403,7 @@ bool memory_region_init_ram_from_file(MemoryRegion *mr,
  * @size: size of the region.
  * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM,
  *             RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY,
- *             RAM_READONLY_FD
+ *             RAM_READONLY_FD, RAM_GUEST_MEMFD
  * @fd: the fd to mmap.
  * @offset: offset within the file referenced by fd
  * @errp: pointer to Error*, to store an error if it happens.
@@ -1722,6 +1726,16 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)
  */
 bool memory_region_is_protected(MemoryRegion *mr);
 
+/**
+ * memory_region_has_guest_memfd: check whether a memory region has guest_memfd
+ *     associated
+ *
+ * Returns %true if a memory region's ram_block has valid guest_memfd assigned.
+ *
+ * @mr: the memory region being queried
+ */
+bool memory_region_has_guest_memfd(MemoryRegion *mr);
+
 /**
  * memory_region_get_iommu: check whether a memory region is an iommu
  *
include/exec/ram_addr.h +1 -1
index 90676093f5..4ebd9ded5e 100644
@@ -109,7 +109,7 @@ long qemu_maxrampagesize(void);
  *  @mr: the memory region where the ram block is
  *  @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM,
  *              RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY,
- *              RAM_READONLY_FD
+ *              RAM_READONLY_FD, RAM_GUEST_MEMFD
  *  @mem_path or @fd: specify the backing file or device
  *  @offset: Offset into target file
  *  @errp: pointer to Error*, to store an error if it happens
include/exec/ramblock.h +1 -0
index 848915ea5b..459c8917de 100644
@@ -41,6 +41,7 @@ struct RAMBlock {
     QLIST_HEAD(, RAMBlockNotifier) ramblock_notifiers;
     int fd;
     uint64_t fd_offset;
+    int guest_memfd;
     size_t page_size;
     /* dirty bitmap used during migration */
     unsigned long *bmap;
include/sysemu/kvm.h +2 -0
index 54f4d83a37..b4913281e2 100644
@@ -536,4 +536,6 @@ void kvm_mark_guest_state_protected(void);
  * reported for the VM.
  */
 bool kvm_hwpoisoned_mem(void);
+
+int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp);
 #endif
system/memory.c +5 -0
index a229a79988..c756950c0c 100644
@@ -1850,6 +1850,11 @@ bool memory_region_is_protected(MemoryRegion *mr)
     return mr->ram && (mr->ram_block->flags & RAM_PROTECTED);
 }
 
+bool memory_region_has_guest_memfd(MemoryRegion *mr)
+{
+    return mr->ram_block && mr->ram_block->guest_memfd >= 0;
+}
+
 uint8_t memory_region_get_dirty_log_mask(MemoryRegion *mr)
 {
     uint8_t mask = mr->dirty_log_mask;
system/physmem.c +21 -3
index 6cfb7a80ab..3a4a3f10d5 100644
@@ -1842,6 +1842,17 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
         }
     }
 
+    if (kvm_enabled() && (new_block->flags & RAM_GUEST_MEMFD)) {
+        assert(new_block->guest_memfd < 0);
+
+        new_block->guest_memfd = kvm_create_guest_memfd(new_block->max_length,
+                                                        0, errp);
+        if (new_block->guest_memfd < 0) {
+            qemu_mutex_unlock_ramlist();
+            return;
+        }
+    }
+
     new_ram_size = MAX(old_ram_size,
               (new_block->offset + new_block->max_length) >> TARGET_PAGE_BITS);
     if (new_ram_size > old_ram_size) {
@@ -1904,7 +1915,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,
     /* Just support these ram flags by now. */
     assert((ram_flags & ~(RAM_SHARED | RAM_PMEM | RAM_NORESERVE |
                           RAM_PROTECTED | RAM_NAMED_FILE | RAM_READONLY |
-                          RAM_READONLY_FD)) == 0);
+                          RAM_READONLY_FD | RAM_GUEST_MEMFD)) == 0);
 
     if (xen_enabled()) {
         error_setg(errp, "-mem-path not supported with Xen");
@@ -1941,6 +1952,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,
     new_block->used_length = size;
     new_block->max_length = size;
     new_block->flags = ram_flags;
+    new_block->guest_memfd = -1;
     new_block->host = file_ram_alloc(new_block, size, fd, !file_size, offset,
                                      errp);
     if (!new_block->host) {
@@ -2020,7 +2032,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
     int align;
 
     assert((ram_flags & ~(RAM_SHARED | RAM_RESIZEABLE | RAM_PREALLOC |
-                          RAM_NORESERVE)) == 0);
+                          RAM_NORESERVE| RAM_GUEST_MEMFD)) == 0);
     assert(!host ^ (ram_flags & RAM_PREALLOC));
 
     align = qemu_real_host_page_size();
@@ -2035,6 +2047,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
     new_block->max_length = max_size;
     assert(max_size >= size);
     new_block->fd = -1;
+    new_block->guest_memfd = -1;
     new_block->page_size = qemu_real_host_page_size();
     new_block->host = host;
     new_block->flags = ram_flags;
@@ -2057,7 +2070,7 @@ RAMBlock *qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
 RAMBlock *qemu_ram_alloc(ram_addr_t size, uint32_t ram_flags,
                          MemoryRegion *mr, Error **errp)
 {
-    assert((ram_flags & ~(RAM_SHARED | RAM_NORESERVE)) == 0);
+    assert((ram_flags & ~(RAM_SHARED | RAM_NORESERVE | RAM_GUEST_MEMFD)) == 0);
     return qemu_ram_alloc_internal(size, size, NULL, NULL, ram_flags, mr, errp);
 }
 
@@ -2085,6 +2098,11 @@ static void reclaim_ramblock(RAMBlock *block)
     } else {
         qemu_anon_ram_free(block->host, block->max_length);
     }
+
+    if (block->guest_memfd >= 0) {
+        close(block->guest_memfd);
+    }
+
     g_free(block);
 }
 
Signature · 1 line
2.25.1
maintainer
#8 lore raw

Re: [PATCH v3 06/49] RAMBlock: Add support of KVM private guest memfd

Quoted 5 lines · Michael Roth
(quoted diff) +1 -0
@@ -1842,6 +1842,17 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
          }
      }
 
+    if (kvm_enabled() && (new_block->flags & RAM_GUEST_MEMFD)) {

Review static void ram_block_add(RAMBlock *new_block, Error **errp) · line 1855

Quoted 10 lines
(quoted diff) +10 -0
+        assert(new_block->guest_memfd < 0);
+
+        new_block->guest_memfd = kvm_create_guest_memfd(new_block->max_length,
+                                                        0, errp);
+        if (new_block->guest_memfd < 0) {
+            qemu_mutex_unlock_ramlist();
+            return;
+        }
+    }
+

This potentially leaks new_block->host. This can be squashed into the patch:

system/physmem.c +10 -1
index 3a4a3f10d5a..0836aff190e 100644
@@ -1810,6 +1810,7 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
     const bool shared = qemu_ram_is_shared(new_block);
     RAMBlock *block;
     RAMBlock *last_block = NULL;
+    bool free_on_error = false;
     ram_addr_t old_ram_size, new_ram_size;
     Error *err = NULL;
 
@@ -1839,6 +1841,7 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
                 return;
             }
             memory_try_enable_merging(new_block->host, new_block->max_length);
+            free_on_error = true;
         }
     }
 
@@ -1849,7 +1852,7 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
                                                         0, errp);
         if (new_block->guest_memfd < 0) {
             qemu_mutex_unlock_ramlist();
-            return;
+            goto out_free;
         }
     }
 
@@ -1901,6 +1904,13 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
         ram_block_notify_add(new_block->host, new_block->used_length,
                              new_block->max_length);
     }
+    return;
+
+out_free:
+    if (free_on_error) {
+        qemu_anon_ram_free(new_block->host, new_block->max_length);
+        new_block->host = NULL;
+    }
 }
 
 #ifdef CONFIG_POSIX
#9 lore raw

PATCH v3 7/49 HostMem: Add mechanism to opt in kvm guest memfd via MachineState

In reply to Michael Roth

From: Xiaoyao Li <[email protected]>

Add a new member "guest_memfd" to memory backends. When it's set to true, it enables RAM_GUEST_MEMFD in ram_flags, thus private kvm guest_memfd will be allocated during RAMBlock allocation.

Memory backend's @guest_memfd is wired with @require_guest_memfd field of MachineState. It avoid looking up the machine in phymem.c.

MachineState::require_guest_memfd is supposed to be set by any VMs that requires KVM guest memfd as private memory, e.g., TDX VM.

---
Changes in v4:
- rename "require_guest_memfd" to "guest_memfd" in struct
HostMemoryBackend; (David Hildenbrand)

---
 backends/hostmem-file.c  | 1 +
 backends/hostmem-memfd.c | 1 +
 backends/hostmem-ram.c   | 1 +
 backends/hostmem.c       | 1 +
 hw/core/machine.c        | 5 +++++
 include/hw/boards.h      | 2 ++
 include/sysemu/hostmem.h | 1 +
 7 files changed, 12 insertions(+)
backends/hostmem-file.c +1 -0
index ac3e433cbd..3c69db7946 100644
@@ -85,6 +85,7 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
     ram_flags |= fb->readonly ? RAM_READONLY_FD : 0;
     ram_flags |= fb->rom == ON_OFF_AUTO_ON ? RAM_READONLY : 0;
     ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
     ram_flags |= fb->is_pmem ? RAM_PMEM : 0;
     ram_flags |= RAM_NAMED_FILE;
     return memory_region_init_ram_from_file(&backend->mr, OBJECT(backend), name,
backends/hostmem-memfd.c +1 -0
index 3923ea9364..745ead0034 100644
@@ -55,6 +55,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
     name = host_memory_backend_get_name(backend);
     ram_flags = backend->share ? RAM_SHARED : 0;
     ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
     return memory_region_init_ram_from_fd(&backend->mr, OBJECT(backend), name,
                                           backend->size, ram_flags, fd, 0, errp);
 }
backends/hostmem-ram.c +1 -0
index d121249f0f..f7d81af783 100644
@@ -30,6 +30,7 @@ ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
     name = host_memory_backend_get_name(backend);
     ram_flags = backend->share ? RAM_SHARED : 0;
     ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
     return memory_region_init_ram_flags_nomigrate(&backend->mr, OBJECT(backend),
                                                   name, backend->size,
                                                   ram_flags, errp);
backends/hostmem.c +1 -0
index 81a72ce40b..eb9682b4a8 100644
@@ -277,6 +277,7 @@ static void host_memory_backend_init(Object *obj)
     /* TODO: convert access to globals to compat properties */
     backend->merge = machine_mem_merge(machine);
     backend->dump = machine_dump_guest_core(machine);
+    backend->guest_memfd = machine_require_guest_memfd(machine);
     backend->reserve = true;
     backend->prealloc_threads = machine->smp.cpus;
 }
hw/core/machine.c +5 -0
index 37ede0e7d4..73ce9da835 100644
@@ -1198,6 +1198,11 @@ bool machine_mem_merge(MachineState *machine)
     return machine->mem_merge;
 }
 
+bool machine_require_guest_memfd(MachineState *machine)
+{
+    return machine->require_guest_memfd;
+}
+
 static char *cpu_slot_to_string(const CPUArchId *cpu)
 {
     GString *s = g_string_new(NULL);
include/hw/boards.h +2 -0
index 8b8f6d5c00..44c2a4e1ec 100644
@@ -36,6 +36,7 @@ bool machine_usb(MachineState *machine);
 int machine_phandle_start(MachineState *machine);
 bool machine_dump_guest_core(MachineState *machine);
 bool machine_mem_merge(MachineState *machine);
+bool machine_require_guest_memfd(MachineState *machine);
 HotpluggableCPUList *machine_query_hotpluggable_cpus(MachineState *machine);
 void machine_set_cpu_numa_node(MachineState *machine,
                                const CpuInstanceProperties *props,
@@ -370,6 +371,7 @@ struct MachineState {
     char *dt_compatible;
     bool dump_guest_core;
     bool mem_merge;
+    bool require_guest_memfd;
     bool usb;
     bool usb_disabled;
     char *firmware;
include/sysemu/hostmem.h +1 -0
index 0e411aaa29..04b884bf42 100644
@@ -74,6 +74,7 @@ struct HostMemoryBackend {
     uint64_t size;
     bool merge, dump, use_canonical_path;
     bool prealloc, is_mapped, share, reserve;
+    bool guest_memfd;
     uint32_t prealloc_threads;
     ThreadContext *prealloc_context;
     DECLARE_BITMAP(host_nodes, MAX_NODES + 1);
Signature · 1 line
2.25.1
#10 lore raw

Re: [PATCH v3 07/49] HostMem: Add mechanism to opt in kvm guest memfd via MachineState

Quoted 58 lines · Michael Roth

From: Xiaoyao Li <[email protected]>

Add a new member "guest_memfd" to memory backends. When it's set to true, it enables RAM_GUEST_MEMFD in ram_flags, thus private kvm guest_memfd will be allocated during RAMBlock allocation.

Memory backend's @guest_memfd is wired with @require_guest_memfd field of MachineState. It avoid looking up the machine in phymem.c.

MachineState::require_guest_memfd is supposed to be set by any VMs that requires KVM guest memfd as private memory, e.g., TDX VM.

---
Changes in v4:
- rename "require_guest_memfd" to "guest_memfd" in struct
HostMemoryBackend; (David Hildenbrand)

---
 backends/hostmem-file.c  | 1 +
 backends/hostmem-memfd.c | 1 +
 backends/hostmem-ram.c   | 1 +
 backends/hostmem.c       | 1 +
 hw/core/machine.c        | 5 +++++
 include/hw/boards.h      | 2 ++
 include/sysemu/hostmem.h | 1 +
 7 files changed, 12 insertions(+)
backends/hostmem-file.c +1 -0
index ac3e433cbd..3c69db7946 100644
@@ -85,6 +85,7 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
     ram_flags |= fb->readonly ? RAM_READONLY_FD : 0;
     ram_flags |= fb->rom == ON_OFF_AUTO_ON ? RAM_READONLY : 0;
     ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
     ram_flags |= fb->is_pmem ? RAM_PMEM : 0;
     ram_flags |= RAM_NAMED_FILE;
     return memory_region_init_ram_from_file(&backend->mr, OBJECT(backend), name,
backends/hostmem-memfd.c +1 -0
index 3923ea9364..745ead0034 100644
@@ -55,6 +55,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
     name = host_memory_backend_get_name(backend);
     ram_flags = backend->share ? RAM_SHARED : 0;
     ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
     return memory_region_init_ram_from_fd(&backend->mr, OBJECT(backend), name,
                                           backend->size, ram_flags, fd, 0, errp);
 }
backends/hostmem-ram.c +0 -0
index d121249f0f..f7d81af783 100644

backends/hostmem-ram.c ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) · line 36

Quoted 7 lines
backends/hostmem-ram.c +1 -0
@@ -30,6 +30,7 @@ ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
     name = host_memory_backend_get_name(backend);
     ram_flags = backend->share ? RAM_SHARED : 0;
     ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
     return memory_region_init_ram_flags_nomigrate(&backend->mr, OBJECT(backend),
                                                   name, backend->size,
                                                   ram_flags, errp);

These change look a bit confusing to me, as I don't see how gmemfd can be used with either file or ram typed memory backends..

When specified gmemfd=on with those, IIUC it'll allocate both the memory (ramblock->host) and gmemfd, but without using ->host. Meanwhile AFAIU the ramblock->host will start to conflict with gmemfd in the future when it might be able to be mapp-able (having valid ->host).

I have a local fix for this (and actually more than below.. but starting from it), I'm not sure whether I overlooked something, but from reading the cover letter it's only using memfd backend which makes perfect sense to me so far. I also don't know the planning of coco patches merging so I don't think even if valid this is urgent - I don't want to mess up on merging plans.. but still want to collect some comments on whether it's valid:

===8<===

From edfdf019ab01e99fb4ff417e30bb3692b4e3b922 Mon Sep 17 00:00:00 2001
From: Peter Xu <[email protected]>
Date: Tue, 21 Jan 2025 12:31:19 -0500
Subject: [PATCH] hostmem: Disallow guest memfd for FILE or RAM typed backends

Guest memfd has very special semantics, which by default doesn't have a path at all, meanwhile it won't proactively allocate anonymous memory.

Currently:

- memory-backend-file: it is about creating a memory object based on a path in the file system. It doesn't apply to gmemfd.

- memory-backend-ram: it is about (mostly) trying to allocate anonymous memories from the system (private or shared). It also doesn't apply to gmemfd.

Forbid the two types of memory backends to gmemfd, but only allow memory-backend-memfd for it as of now.

---
 backends/hostmem-file.c | 8 +++++++-
 backends/hostmem-ram.c  | 7 ++++++-
 2 files changed, 13 insertions(+), 2 deletions(-)
backends/hostmem-file.c +7 -1
index 46321fda84..c94cf8441b 100644
@@ -52,11 +52,18 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
         error_setg(errp, "can't create backend with size 0");
         return false;
     }
+
     if (!fb->mem_path) {
         error_setg(errp, "mem-path property not set");
         return false;
     }
 
+    if (backend->guest_memfd) {
+        error_setg(errp, "File backends do not support guest memfd. "
+                   "Please use memfd backend");
+        return false;
+    }
+
     switch (fb->rom) {
     case ON_OFF_AUTO_AUTO:
         /* Traditionally, opening the file readonly always resulted in ROM. */
@@ -86,7 +93,6 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
     ram_flags |= fb->readonly ? RAM_READONLY_FD : 0;
     ram_flags |= fb->rom == ON_OFF_AUTO_ON ? RAM_READONLY : 0;
     ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
-    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
     ram_flags |= fb->is_pmem ? RAM_PMEM : 0;
     ram_flags |= RAM_NAMED_FILE;
     return memory_region_init_ram_from_file(&backend->mr, OBJECT(backend), name,
backends/hostmem-ram.c +6 -1
index 39aac6bf35..8125be217c 100644
@@ -27,10 +27,15 @@ ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
         return false;
     }
 
+    if (backend->guest_memfd) {
+        error_setg(errp, "File backends do not support guest memfd. "
+                   "Please use memfd backend");
+        return false;
+    }
+
     name = host_memory_backend_get_name(backend);
     ram_flags = backend->share ? RAM_SHARED : 0;
     ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
-    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
     return memory_region_init_ram_flags_nomigrate(&backend->mr, OBJECT(backend),
                                                   name, backend->size,
                                                   ram_flags, errp);
Signature · 7 lines
2.47.0
===8<===

Thanks,

-- 
Peter Xu
#11 lore raw

Re: [PATCH v3 07/49] HostMem: Add mechanism to opt in kvm guest memfd via MachineState

Quoted 61 lines · Peter Xu
Quoted 60 lines · Michael Roth

From: Xiaoyao Li <[email protected]>

Add a new member "guest_memfd" to memory backends. When it's set to true, it enables RAM_GUEST_MEMFD in ram_flags, thus private kvm guest_memfd will be allocated during RAMBlock allocation.

Memory backend's @guest_memfd is wired with @require_guest_memfd field of MachineState. It avoid looking up the machine in phymem.c.

MachineState::require_guest_memfd is supposed to be set by any VMs that requires KVM guest memfd as private memory, e.g., TDX VM.

---
Changes in v4:
- rename "require_guest_memfd" to "guest_memfd" in struct
HostMemoryBackend; (David Hildenbrand)

---
  backends/hostmem-file.c  | 1 +
  backends/hostmem-memfd.c | 1 +
  backends/hostmem-ram.c   | 1 +
  backends/hostmem.c       | 1 +
  hw/core/machine.c        | 5 +++++
  include/hw/boards.h      | 2 ++
  include/sysemu/hostmem.h | 1 +
  7 files changed, 12 insertions(+)
backends/hostmem-file.c +1 -0
index ac3e433cbd..3c69db7946 100644
@@ -85,6 +85,7 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
      ram_flags |= fb->readonly ? RAM_READONLY_FD : 0;
      ram_flags |= fb->rom == ON_OFF_AUTO_ON ? RAM_READONLY : 0;
      ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
      ram_flags |= fb->is_pmem ? RAM_PMEM : 0;
      ram_flags |= RAM_NAMED_FILE;
      return memory_region_init_ram_from_file(&backend->mr, OBJECT(backend), name,
backends/hostmem-memfd.c +1 -0
index 3923ea9364..745ead0034 100644
@@ -55,6 +55,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
      name = host_memory_backend_get_name(backend);
      ram_flags = backend->share ? RAM_SHARED : 0;
      ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
      return memory_region_init_ram_from_fd(&backend->mr, OBJECT(backend), name,
                                            backend->size, ram_flags, fd, 0, errp);
  }
backends/hostmem-ram.c +0 -0
index d121249f0f..f7d81af783 100644
@@ -30,6 +30,7 @@ ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
      name = host_memory_backend_get_name(backend);
      ram_flags = backend->share ? RAM_SHARED : 0;
Quoted 8 lines

backends/hostmem-ram.c ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) · line 41

Quoted 5 lines
backends/hostmem-ram.c +1 -0
      ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
      return memory_region_init_ram_flags_nomigrate(&backend->mr, OBJECT(backend),
                                                    name, backend->size,
                                                    ram_flags, errp);

These change look a bit confusing to me, as I don't see how gmemfd can be used with either file or ram typed memory backends..

I recall that the following should work:

"private" memory will come from guest_memfd, "shared" (as in, accessible by the host) will come from anonymous memory.

This "anon" memory cannot be "shared" with other processes, but virtio-kernel etc. can just use it.

To "share" the memory with other processes, we'd need memfd/file.

Quoted 5 lines

When specified gmemfd=on with those, IIUC it'll allocate both the memory (ramblock->host) and gmemfd, but without using ->host. Meanwhile AFAIU the ramblock->host will start to conflict with gmemfd in the future when it might be able to be mapp-able (having valid ->host).

These will require a new guest_memfd memory backend (I recall that was discussed a couple of times).

Quoted 5 lines

I have a local fix for this (and actually more than below.. but starting from it), I'm not sure whether I overlooked something, but from reading the cover letter it's only using memfd backend which makes perfect sense to me so far.

Does the anon+guest_memfd combination not work or are you speculating about the usability (which I hopefully addressed above).

Signature · 3 lines
Cheers,

David / dhildenb
#12 lore raw

Re: [PATCH v3 07/49] HostMem: Add mechanism to opt in kvm guest memfd via MachineState

Quoted 73 lines · David Hildenbrand
Quoted 61 lines · Peter Xu
Quoted 60 lines · Michael Roth

From: Xiaoyao Li <[email protected]>

Add a new member "guest_memfd" to memory backends. When it's set to true, it enables RAM_GUEST_MEMFD in ram_flags, thus private kvm guest_memfd will be allocated during RAMBlock allocation.

Memory backend's @guest_memfd is wired with @require_guest_memfd field of MachineState. It avoid looking up the machine in phymem.c.

MachineState::require_guest_memfd is supposed to be set by any VMs that requires KVM guest memfd as private memory, e.g., TDX VM.

---
Changes in v4:
- rename "require_guest_memfd" to "guest_memfd" in struct
HostMemoryBackend; (David Hildenbrand)

---
  backends/hostmem-file.c  | 1 +
  backends/hostmem-memfd.c | 1 +
  backends/hostmem-ram.c   | 1 +
  backends/hostmem.c       | 1 +
  hw/core/machine.c        | 5 +++++
  include/hw/boards.h      | 2 ++
  include/sysemu/hostmem.h | 1 +
  7 files changed, 12 insertions(+)
backends/hostmem-file.c +1 -0
index ac3e433cbd..3c69db7946 100644
@@ -85,6 +85,7 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
      ram_flags |= fb->readonly ? RAM_READONLY_FD : 0;
      ram_flags |= fb->rom == ON_OFF_AUTO_ON ? RAM_READONLY : 0;
      ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
      ram_flags |= fb->is_pmem ? RAM_PMEM : 0;
      ram_flags |= RAM_NAMED_FILE;
      return memory_region_init_ram_from_file(&backend->mr, OBJECT(backend), name,
backends/hostmem-memfd.c +1 -0
index 3923ea9364..745ead0034 100644
@@ -55,6 +55,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
      name = host_memory_backend_get_name(backend);
      ram_flags = backend->share ? RAM_SHARED : 0;
      ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
      return memory_region_init_ram_from_fd(&backend->mr, OBJECT(backend), name,
                                            backend->size, ram_flags, fd, 0, errp);
  }
backends/hostmem-ram.c +0 -0
index d121249f0f..f7d81af783 100644
@@ -30,6 +30,7 @@ ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
      name = host_memory_backend_get_name(backend);
      ram_flags = backend->share ? RAM_SHARED : 0;
Quoted 8 lines

backends/hostmem-ram.c ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) · line 41

Quoted 5 lines
backends/hostmem-ram.c +1 -0
      ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+    ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
      return memory_region_init_ram_flags_nomigrate(&backend->mr, OBJECT(backend),
                                                    name, backend->size,
                                                    ram_flags, errp);

These change look a bit confusing to me, as I don't see how gmemfd can be used with either file or ram typed memory backends..

I recall that the following should work:

Quoted 7 lines

"private" memory will come from guest_memfd, "shared" (as in, accessible by the host) will come from anonymous memory.

This "anon" memory cannot be "shared" with other processes, but virtio-kernel etc. can just use it.

To "share" the memory with other processes, we'd need memfd/file.

Ah OK, thanks David. Is this the planned long term solution for vhost-kernel?

I wonder what happens if vhost tries to DMA to a region that is private with this setup.

AFAIU, it'll try to DMA to the fake address of ramblock->host that is pointing to by the memory backend (either anon, shmem, file, etc.). The ideal case IIUC is it should crash QEMU because it's trying to access an illegal page which is private. But if with this model, it won't crash but silently populate some page in the non-gmemfd backend.

Is that expected?

Quoted 9 lines
Quoted 6 lines

When specified gmemfd=on with those, IIUC it'll allocate both the memory (ramblock->host) and gmemfd, but without using ->host. Meanwhile AFAIU the ramblock->host will start to conflict with gmemfd in the future when it might be able to be mapp-able (having valid ->host).

These will require a new guest_memfd memory backend (I recall that was discussed a couple of times).

Do you know if anyone is working on this one?

Quoted 9 lines
Quoted 6 lines

I have a local fix for this (and actually more than below.. but starting from it), I'm not sure whether I overlooked something, but from reading the cover letter it's only using memfd backend which makes perfect sense to me so far.

Does the anon+guest_memfd combination not work or are you speculating about the usability (which I hopefully addressed above).

IIUC, if with above solution and with how QEMU interacts memory convertions right now, at least hugetlb pages will suffer from double allocation, as kvm_convert_memory() won't free hugetlb pages even if converted to private.

It sounds like also doable (and also preferrable..) that for each of the VM we always stich with pages in the gmemfd page cache, no matter if it's shared or private. For private, we could zap all pgtables and sigbus any faults afterwards. I thought that was always the plan, but I could lose many latest informations..

Thanks,

Signature · 1 line
Peter Xu
#13 lore raw

Re: [PATCH v3 07/49] HostMem: Add mechanism to opt in kvm guest memfd via MachineState

Quoted 7 lines
Quoted 4 lines

This "anon" memory cannot be "shared" with other processes, but virtio-kernel etc. can just use it.

To "share" the memory with other processes, we'd need memfd/file.

Ah OK, thanks David. Is this the planned long term solution for vhost-kernel?

I think the basic idea was that the memory backend defines how the "non-private" memory is backed, which is the same just like for any other non-CC VM.

The "private" memory always comes from guest_memfd.

So for the time being using anon+guest_memfd coresponds to "just a simple VM".

Long-term I expect that we use guest_memfd for shared+private, and use in-place conversion. Access to "private" memory using the mmap() will result in a SIGBUS.

Quoted 9 lines
Quoted 1 line

I wonder what happens if vhost tries to DMA to a region that is private

with this setup.

Quoted 1 line

AFAIU, it'll try to DMA to the fake address of ramblock->host that is

pointing to by the memory backend (either anon, shmem, file, etc.). The ideal case IIUC is it should crash QEMU because it's trying to access an illegal page which is private. But if with this model, it won't crash but silently populate some page in the non-gmemfd backend.

Is that expected?

Yes, it's all just a big mmap() which will populate memory on access -- independent of using anon/file/memfd.

Similar to virtio-mem, long-term we'd want a mechanism to check/enforce that some memory in there will not be populated on access from QEMU (well, and vhost-user processes ...).

In memory_get_xlat_addr() we perform such checks, but it's only used for iommu. vhost-kernel likely has no such checks, just like vhost-user etc does not.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

When specified gmemfd=on with those, IIUC it'll allocate both the memory (ramblock->host) and gmemfd, but without using ->host. Meanwhile AFAIU the ramblock->host will start to conflict with gmemfd in the future when it might be able to be mapp-able (having valid ->host).

Quoted 4 lines
Quoted 2 lines

These will require a new guest_memfd memory backend (I recall that was discussed a couple of times).

Do you know if anyone is working on this one?

So far my understanding is that Google that does shared+private guest_memfd kernel part won't be working on QEMU patches. I raised that to our management recently, that this would be a good project for RH to focus on.

I am not aware of real implementations of the guest_memfd backend (yet).

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

I have a local fix for this (and actually more than below.. but starting from it), I'm not sure whether I overlooked something, but from reading the cover letter it's only using memfd backend which makes perfect sense to me so far.

Quoted 6 lines
Quoted 2 lines

Does the anon+guest_memfd combination not work or are you speculating about the usability (which I hopefully addressed above).

IIUC, if with above solution and with how QEMU interacts memory convertions right now, at least hugetlb pages will suffer from double allocation, as kvm_convert_memory() won't free hugetlb pages even if converted to private.

Yes, that's why I'm invested in teaching guest_memfd in-place conversion alongside huge page support (which fortunately Google engineers are doing great work on).

Quoted 6 lines

It sounds like also doable (and also preferrable..) that for each of the VM we always stich with pages in the gmemfd page cache, no matter if it's shared or private. For private, we could zap all pgtables and sigbus any faults afterwards. I thought that was always the plan, but I could lose many latest informations..

Yes, with the guest_memfd backend (shared+private) that's the plan: SIGBUS on invalid access.

Signature · 3 lines
Cheers,

David / dhildenb
#14 lore raw

Re: [PATCH v3 07/49] HostMem: Add mechanism to opt in kvm guest memfd via MachineState

Quoted 5 lines · David Hildenbrand

So far my understanding is that Google that does shared+private guest_memfd kernel part won't be working on QEMU patches. I raised that to our management recently, that this would be a good project for RH to focus on.

I am not aware of real implementations of the guest_memfd backend (yet).

I see, thanks, those are pretty useful information to me.

I think I have a rough picture now. Since I've already had some patches going that direction, I'll give it a shot. I'll keep you updated if I get something.

Signature · 1 line
Peter Xu
#15 lore raw

Re: [PATCH v3 07/49] HostMem: Add mechanism to opt in kvm guest memfd via MachineState

Quoted 5 lines · Peter Xu
Quoted 4 lines · David Hildenbrand

So far my understanding is that Google that does shared+private guest_memfd kernel part won't be working on QEMU patches. I raised that to our management recently, that this would be a good project for RH to focus on.

Quoted 7 lines
Quoted 1 line

I am not aware of real implementations of the guest_memfd backend (yet).

I see, thanks, those are pretty useful information to me.

I think I have a rough picture now. Since I've already had some patches going that direction, I'll give it a shot. I'll keep you updated if I get something.

Nice!

Signature · 3 lines
Cheers,

David / dhildenb
#16 lore raw

PATCH v3 8/49 trace/kvm: Split address space and slot id in trace_kvm_set_user_memory()

In reply to Michael Roth

From: Xiaoyao Li <[email protected]>

The upper 16 bits of kvm_userspace_memory_region::slot are address space id. Parse it separately in trace_kvm_set_user_memory().

---
 accel/kvm/kvm-all.c    | 5 +++--
 accel/kvm/trace-events | 2 +-
 2 files changed, 4 insertions(+), 3 deletions(-)
accel/kvm/kvm-all.c +3 -2
index 132ab65df5..d2856dd736 100644
@@ -304,8 +304,9 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo
     ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
     slot->old_flags = mem.flags;
 err:
-    trace_kvm_set_user_memory(mem.slot, mem.flags, mem.guest_phys_addr,
-                              mem.memory_size, mem.userspace_addr, ret);
+    trace_kvm_set_user_memory(mem.slot >> 16, (uint16_t)mem.slot, mem.flags,
+                              mem.guest_phys_addr, mem.memory_size,
+                              mem.userspace_addr, ret);
     if (ret < 0) {
         error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
                      " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
accel/kvm/trace-events +1 -1
index a25902597b..9f599abc17 100644
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
 kvm_irqchip_release_virq(int virq) "virq %d"
 kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d"
 kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d"
-kvm_set_user_memory(uint32_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
+kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
 kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32
 kvm_resample_fd_notify(int gsi) "gsi %d"
 kvm_dirty_ring_full(int id) "vcpu %d"
Signature · 1 line
2.25.1
#17 lore raw

PATCH v3 9/49 kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot

In reply to Michael Roth

From: Chao Peng <[email protected]>

Switch to KVM_SET_USER_MEMORY_REGION2 when supported by KVM.

With KVM_SET_USER_MEMORY_REGION2, QEMU can set up memory region that backend'ed both by hva-based shared memory and guest memfd based private memory.

---
Changes in v4:
- update KVM_MEM_PRIVATE to KVM_MEM_GUEST_MEMFD; (Isaku)

---
 accel/kvm/kvm-all.c      | 56 ++++++++++++++++++++++++++++++++++------
 accel/kvm/trace-events   |  2 +-
 include/sysemu/kvm_int.h |  2 ++
 3 files changed, 51 insertions(+), 9 deletions(-)
accel/kvm/kvm-all.c +48 -8
index d2856dd736..e83429b31e 100644
@@ -283,35 +283,69 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,
 static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, bool new)
 {
     KVMState *s = kvm_state;
-    struct kvm_userspace_memory_region mem;
+    struct kvm_userspace_memory_region2 mem;
+    static int cap_user_memory2 = -1;
     int ret;
 
+    if (cap_user_memory2 == -1) {
+        cap_user_memory2 = kvm_check_extension(s, KVM_CAP_USER_MEMORY2);
+    }
+
+    if (!cap_user_memory2 && slot->guest_memfd >= 0) {
+        error_report("%s, KVM doesn't support KVM_CAP_USER_MEMORY2,"
+                     " which is required by guest memfd!", __func__);
+        exit(1);
+    }
+
     mem.slot = slot->slot | (kml->as_id << 16);
     mem.guest_phys_addr = slot->start_addr;
     mem.userspace_addr = (unsigned long)slot->ram;
     mem.flags = slot->flags;
+    mem.guest_memfd = slot->guest_memfd;
+    mem.guest_memfd_offset = slot->guest_memfd_offset;
 
     if (slot->memory_size && !new && (mem.flags ^ slot->old_flags) & KVM_MEM_READONLY) {
         /* Set the slot size to 0 before setting the slot to the desired
          * value. This is needed based on KVM commit 75d61fbc. */
         mem.memory_size = 0;
-        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+
+        if (cap_user_memory2) {
+            ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
+        } else {
+            ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+        }
         if (ret < 0) {
             goto err;
         }
     }
     mem.memory_size = slot->memory_size;
-    ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+    if (cap_user_memory2) {
+        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
+    } else {
+        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+    }
     slot->old_flags = mem.flags;
 err:
     trace_kvm_set_user_memory(mem.slot >> 16, (uint16_t)mem.slot, mem.flags,
                               mem.guest_phys_addr, mem.memory_size,
-                              mem.userspace_addr, ret);
+                              mem.userspace_addr, mem.guest_memfd,
+                              mem.guest_memfd_offset, ret);
     if (ret < 0) {
-        error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
-                     " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
-                     __func__, mem.slot, slot->start_addr,
-                     (uint64_t)mem.memory_size, strerror(errno));
+        if (cap_user_memory2) {
+                error_report("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d,"
+                        " start=0x%" PRIx64 ", size=0x%" PRIx64 ","
+                        " flags=0x%" PRIx32 ", guest_memfd=%" PRId32 ","
+                        " guest_memfd_offset=0x%" PRIx64 ": %s",
+                        __func__, mem.slot, slot->start_addr,
+                        (uint64_t)mem.memory_size, mem.flags,
+                        mem.guest_memfd, (uint64_t)mem.guest_memfd_offset,
+                        strerror(errno));
+        } else {
+                error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
+                            " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
+                            __func__, mem.slot, slot->start_addr,
+                            (uint64_t)mem.memory_size, strerror(errno));
+        }
     }
     return ret;
 }
@@ -466,6 +500,9 @@ static int kvm_mem_flags(MemoryRegion *mr)
     if (readonly && kvm_readonly_mem_allowed) {
         flags |= KVM_MEM_READONLY;
     }
+    if (memory_region_has_guest_memfd(mr)) {
+        flags |= KVM_MEM_GUEST_MEMFD;
+    }
     return flags;
 }
 
@@ -1363,6 +1400,9 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
         mem->ram_start_offset = ram_start_offset;
         mem->ram = ram;
         mem->flags = kvm_mem_flags(mr);
+        mem->guest_memfd = mr->ram_block->guest_memfd;
+        mem->guest_memfd_offset = (uint8_t*)ram - mr->ram_block->host;
+
         kvm_slot_init_dirty_bitmap(mem);
         err = kvm_set_user_memory_region(kml, mem, true);
         if (err) {
accel/kvm/trace-events +1 -1
index 9f599abc17..e8c52cb9e7 100644
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
 kvm_irqchip_release_virq(int virq) "virq %d"
 kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d"
 kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d"
-kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
+kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, uint32_t fd, uint64_t fd_offset, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " guest_memfd=%d" " guest_memfd_offset=0x%" PRIx64 " ret=%d"
 kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32
 kvm_resample_fd_notify(int gsi) "gsi %d"
 kvm_dirty_ring_full(int id) "vcpu %d"
include/sysemu/kvm_int.h +2 -0
index 3496be7997..a5a3fee411 100644
@@ -30,6 +30,8 @@ typedef struct KVMSlot
     int as_id;
     /* Cache of the offset in ram address space */
     ram_addr_t ram_start_offset;
+    int guest_memfd;
+    hwaddr guest_memfd_offset;
 } KVMSlot;
 
 typedef struct KVMMemoryUpdate {
Signature · 1 line
2.25.1
maintainer
#18 lore raw

Re: [PATCH v3 09/49] kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot

Review

Quoted 9 lines · Michael Roth
(quoted diff) +9 -0
+    if (cap_user_memory2 == -1) {
+        cap_user_memory2 = kvm_check_extension(s, KVM_CAP_USER_MEMORY2);
+    }
+
+    if (!cap_user_memory2 && slot->guest_memfd >= 0) {
+        error_report("%s, KVM doesn't support KVM_CAP_USER_MEMORY2,"
+                     " which is required by guest memfd!", __func__);
+        exit(1);
+    }

It's easier and more robust (for error reporting purposes) to check both KVM_CAP_GUEST_MEMFD and KVM_CAP_USER_MEMORY2 at once in the earlier patches:

- kvm_guest_memfd_supported = kvm_check_extension(s, KVM_CAP_GUEST_MEMFD);
+ kvm_guest_memfd_supported =
+ kvm_check_extension(s, KVM_CAP_GUEST_MEMFD) &&
+ kvm_check_extension(s, KVM_CAP_USER_MEMORY2);

since KVM cannot really support guest_memfd if it cannot then use it to create private memory slots.

And then, this one can be changed to also use kvm_guest_memfd_supported:

accel/kvm/kvm-all.c +4 -14
index e83429b31eb..afcf6f87045 100644
@@ -284,19 +284,8 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo
 {
     KVMState *s = kvm_state;
     struct kvm_userspace_memory_region2 mem;
-    static int cap_user_memory2 = -1;
     int ret;
 
-    if (cap_user_memory2 == -1) {
-        cap_user_memory2 = kvm_check_extension(s, KVM_CAP_USER_MEMORY2);
-    }
-
-    if (!cap_user_memory2 && slot->guest_memfd >= 0) {
-        error_report("%s, KVM doesn't support KVM_CAP_USER_MEMORY2,"
-                     " which is required by guest memfd!", __func__);
-        exit(1);
-    }
-
     mem.slot = slot->slot | (kml->as_id << 16);
     mem.guest_phys_addr = slot->start_addr;
     mem.userspace_addr = (unsigned long)slot->ram;
@@ -309,7 +298,7 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo
          * value. This is needed based on KVM commit 75d61fbc. */
         mem.memory_size = 0;
 
-        if (cap_user_memory2) {
+        if (kvm_guest_memfd_supported) {
             ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
         } else {
             ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
@@ -319,7 +308,7 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo
         }
     }
     mem.memory_size = slot->memory_size;
-    if (cap_user_memory2) {
+    if (kvm_guest_memfd_supported) {
         ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
     } else {
         ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
@@ -331,7 +320,7 @@ err:
                               mem.userspace_addr, mem.guest_memfd,
                               mem.guest_memfd_offset, ret);
     if (ret < 0) {
-        if (cap_user_memory2) {
+        if (kvm_guest_memfd_supported) {
                 error_report("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d,"
                         " start=0x%" PRIx64 ", size=0x%" PRIx64 ","
                         " flags=0x%" PRIx32 ", guest_memfd=%" PRId32 ","
@@ -501,6 +490,7 @@ static int kvm_mem_flags(MemoryRegion *mr)
         flags |= KVM_MEM_READONLY;
     }
     if (memory_region_has_guest_memfd(mr)) {
+        assert(kvm_guest_memfd_supported);
         flags |= KVM_MEM_GUEST_MEMFD;
     }
     return flags;
#19 lore raw

PATCH v3 10/49 kvm: Introduce support for memory_attributes

In reply to Michael Roth

From: Xiaoyao Li <[email protected]>

Introduce the helper functions to set the attributes of a range of memory to private or shared.

This is necessary to notify KVM the private/shared attribute of each gpa range. KVM needs the information to decide the GPA needs to be mapped at hva-based shared memory or guest_memfd based private memory.

---
Changes in v4:
- move the check of kvm_supported_memory_attributes to the common kvm_set_memory_attributes(); (Wang Wei)
- change warn_report() to error_report() in kvm_set_memory_attributes() and drop the __func__; (Daniel)

---
 accel/kvm/kvm-all.c  | 44 ++++++++++++++++++++++++++++++++++++++++++++
 include/sysemu/kvm.h |  3 +++
 2 files changed, 47 insertions(+)
accel/kvm/kvm-all.c +44 -0
index e83429b31e..df7a32735a 100644
@@ -92,6 +92,7 @@ static bool kvm_has_guest_debug;
 static int kvm_sstep_flags;
 static bool kvm_immediate_exit;
 static bool kvm_guest_memfd_supported;
+static uint64_t kvm_supported_memory_attributes;
 static hwaddr kvm_max_slot_size = ~0;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -1304,6 +1305,46 @@ void kvm_set_max_memslot_size(hwaddr max_slot_size)
     kvm_max_slot_size = max_slot_size;
 }
 
+static int kvm_set_memory_attributes(hwaddr start, hwaddr size, uint64_t attr)
+{
+    struct kvm_memory_attributes attrs;
+    int r;
+
+    if (kvm_supported_memory_attributes == 0) {
+        error_report("No memory attribute supported by KVM\n");
+        return -EINVAL;
+    }
+
+    if ((attr & kvm_supported_memory_attributes) != attr) {
+        error_report("memory attribute 0x%lx not supported by KVM,"
+                     " supported bits are 0x%lx\n",
+                     attr, kvm_supported_memory_attributes);
+        return -EINVAL;
+    }
+
+    attrs.attributes = attr;
+    attrs.address = start;
+    attrs.size = size;
+    attrs.flags = 0;
+
+    r = kvm_vm_ioctl(kvm_state, KVM_SET_MEMORY_ATTRIBUTES, &attrs);
+    if (r) {
+        error_report("failed to set memory (0x%lx+%#zx) with attr 0x%lx error '%s'",
+                     start, size, attr, strerror(errno));
+    }
+    return r;
+}
+
+int kvm_set_memory_attributes_private(hwaddr start, hwaddr size)
+{
+    return kvm_set_memory_attributes(start, size, KVM_MEMORY_ATTRIBUTE_PRIVATE);
+}
+
+int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size)
+{
+    return kvm_set_memory_attributes(start, size, 0);
+}
+
 /* Called with KVMMemoryListener.slots_lock held */
 static void kvm_set_phys_mem(KVMMemoryListener *kml,
                              MemoryRegionSection *section, bool add)
@@ -2439,6 +2480,9 @@ static int kvm_init(MachineState *ms)
 
     kvm_guest_memfd_supported = kvm_check_extension(s, KVM_CAP_GUEST_MEMFD);
 
+    ret = kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
+    kvm_supported_memory_attributes = ret > 0 ? ret : 0;
+
     if (object_property_find(OBJECT(current_machine), "kvm-type")) {
         g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine),
                                                             "kvm-type",
include/sysemu/kvm.h +3 -0
index b4913281e2..2cb3192509 100644
@@ -538,4 +538,7 @@ void kvm_mark_guest_state_protected(void);
 bool kvm_hwpoisoned_mem(void);
 
 int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp);
+
+int kvm_set_memory_attributes_private(hwaddr start, hwaddr size);
+int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size);
 #endif
Signature · 1 line
2.25.1
maintainer
#20 lore raw

Re: [PATCH v3 10/49] kvm: Introduce support for memory_attributes

Quoted 45 lines · Michael Roth

From: Xiaoyao Li <[email protected]>

Introduce the helper functions to set the attributes of a range of memory to private or shared.

This is necessary to notify KVM the private/shared attribute of each gpa range. KVM needs the information to decide the GPA needs to be mapped at hva-based shared memory or guest_memfd based private memory.

---
Changes in v4:
- move the check of kvm_supported_memory_attributes to the common kvm_set_memory_attributes(); (Wang Wei)
- change warn_report() to error_report() in kvm_set_memory_attributes() and drop the __func__; (Daniel)

---
  accel/kvm/kvm-all.c  | 44 ++++++++++++++++++++++++++++++++++++++++++++
  include/sysemu/kvm.h |  3 +++
  2 files changed, 47 insertions(+)
accel/kvm/kvm-all.c +7 -0
index e83429b31e..df7a32735a 100644
@@ -92,6 +92,7 @@ static bool kvm_has_guest_debug;
  static int kvm_sstep_flags;
  static bool kvm_immediate_exit;
  static bool kvm_guest_memfd_supported;
+static uint64_t kvm_supported_memory_attributes;
  static hwaddr kvm_max_slot_size = ~0;
 
  static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -1304,6 +1305,46 @@ void kvm_set_max_memslot_size(hwaddr max_slot_size)
      kvm_max_slot_size = max_slot_size;
  }
 
+static int kvm_set_memory_attributes(hwaddr start, hwaddr size, uint64_t attr)
+{
+    struct kvm_memory_attributes attrs;
+    int r;
+
+    if (kvm_supported_memory_attributes == 0) {

accel/kvm/kvm-all.c void kvm_set_max_memslot_size(hwaddr max_slot_size) · line 1323

Quoted 10 lines
accel/kvm/kvm-all.c +10 -0
+        error_report("No memory attribute supported by KVM\n");
+        return -EINVAL;
+    }
+
+    if ((attr & kvm_supported_memory_attributes) != attr) {
+        error_report("memory attribute 0x%lx not supported by KVM,"
+                     " supported bits are 0x%lx\n",
+                     attr, kvm_supported_memory_attributes);
+        return -EINVAL;
+    }

This should also be tested at the same time as kvm_guest_memfd_supported.

Paolo

Quoted 41 lines
accel/kvm/kvm-all.c +26 -0
+    attrs.attributes = attr;
+    attrs.address = start;
+    attrs.size = size;
+    attrs.flags = 0;
+
+    r = kvm_vm_ioctl(kvm_state, KVM_SET_MEMORY_ATTRIBUTES, &attrs);
+    if (r) {
+        error_report("failed to set memory (0x%lx+%#zx) with attr 0x%lx error '%s'",
+                     start, size, attr, strerror(errno));
+    }
+    return r;
+}
+
+int kvm_set_memory_attributes_private(hwaddr start, hwaddr size)
+{
+    return kvm_set_memory_attributes(start, size, KVM_MEMORY_ATTRIBUTE_PRIVATE);
+}
+
+int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size)
+{
+    return kvm_set_memory_attributes(start, size, 0);
+}
+
  /* Called with KVMMemoryListener.slots_lock held */
  static void kvm_set_phys_mem(KVMMemoryListener *kml,
                               MemoryRegionSection *section, bool add)
@@ -2439,6 +2480,9 @@ static int kvm_init(MachineState *ms)
 
      kvm_guest_memfd_supported = kvm_check_extension(s, KVM_CAP_GUEST_MEMFD);
 
+    ret = kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
+    kvm_supported_memory_attributes = ret > 0 ? ret : 0;
+
      if (object_property_find(OBJECT(current_machine), "kvm-type")) {
          g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine),
                                                              "kvm-type",
include/sysemu/kvm.h +0 -0
index b4913281e2..2cb3192509 100644

include/sysemu/kvm.h void kvm_mark_guest_state_protected(void); · line 544

Quoted 7 lines
include/sysemu/kvm.h +3 -0
@@ -538,4 +538,7 @@ void kvm_mark_guest_state_protected(void);
  bool kvm_hwpoisoned_mem(void);
 
  int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp);
+
+int kvm_set_memory_attributes_private(hwaddr start, hwaddr size);
+int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size);
  #endif

This suggests that

#21 lore raw

PATCH v3 11/49 physmem: Introduce ram_block_discard_guest_memfd_range()

In reply to Michael Roth

From: Xiaoyao Li <[email protected]>

When memory page is converted from private to shared, the original private memory is back'ed by guest_memfd. Introduce ram_block_discard_guest_memfd_range() for discarding memory in guest_memfd.

Originally-from: Isaku Yamahata <[email protected]>

---
Changes in v5:
- Collect Reviewed-by from David;

Changes in in v4:
- Drop ram_block_convert_range() and open code its implementation in the next Patch.

---
 include/exec/cpu-common.h |  2 ++
 system/physmem.c          | 23 +++++++++++++++++++++++
 2 files changed, 25 insertions(+)
include/exec/cpu-common.h +2 -0
index 6346df17ce..6d5318895a 100644
@@ -159,6 +159,8 @@ typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque);
 
 int qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque);
 int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length);
+int ram_block_discard_guest_memfd_range(RAMBlock *rb, uint64_t start,
+                                        size_t length);
 
 #endif
 
system/physmem.c +23 -0
index 3a4a3f10d5..8be8053cf7 100644
@@ -3705,6 +3705,29 @@ err:
     return ret;
 }
 
+int ram_block_discard_guest_memfd_range(RAMBlock *rb, uint64_t start,
+                                        size_t length)
+{
+    int ret = -1;
+
+#ifdef CONFIG_FALLOCATE_PUNCH_HOLE
+    ret = fallocate(rb->guest_memfd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
+                    start, length);
+
+    if (ret) {
+        ret = -errno;
+        error_report("%s: Failed to fallocate %s:%" PRIx64 " +%zx (%d)",
+                     __func__, rb->idstr, start, length, ret);
+    }
+#else
+    ret = -ENOSYS;
+    error_report("%s: fallocate not available %s:%" PRIx64 " +%zx (%d)",
+                 __func__, rb->idstr, start, length, ret);
+#endif
+
+    return ret;
+}
+
 bool ramblock_is_pmem(RAMBlock *rb)
 {
     return rb->flags & RAM_PMEM;
Signature · 1 line
2.25.1
#22 lore raw

Re: [PATCH v3 11/49] physmem: Introduce ram_block_discard_guest_memfd_range()

Quoted 9 lines · Michael Roth

From: Xiaoyao Li <[email protected]>

When memory page is converted from private to shared, the original private memory is back'ed by guest_memfd. Introduce ram_block_discard_guest_memfd_range() for discarding memory in guest_memfd.

Originally-from: Isaku Yamahata <[email protected]>

"Co-developed-by"

Quoted 2 lines

Your SOB should go here.

Quoted 9 lines

---
Changes in v5:
- Collect Reviewed-by from David;

Changes in in v4:
- Drop ram_block_convert_range() and open code its implementation in the next Patch.

I only received 3 patches from this series, and now I am confused: changelog talks about v5 and this is "PATCH v3"

Please make sure to send at least the cover letter along (I might not need the other 46 patches :D ).

Signature · 3 lines
Cheers,

David / dhildenb
#23 lore raw

Re: [PATCH v3 11/49] physmem: Introduce ram_block_discard_guest_memfd_range()

Quoted 8 lines · David Hildenbrand
Quoted 7 lines · Michael Roth

From: Xiaoyao Li <[email protected]>

When memory page is converted from private to shared, the original private memory is back'ed by guest_memfd. Introduce ram_block_discard_guest_memfd_range() for discarding memory in guest_memfd.

Quoted 4 lines
Quoted 2 lines

Originally-from: Isaku Yamahata <[email protected]>

"Co-developed-by"

Michael is using the patch from my TDX-QEMU v5 series[1]. I need to fix it.

Quoted 9 lines
Quoted 2 lines

Your SOB should go here.

Quoted 4 lines

---
Changes in v5:
- Collect Reviewed-by from David;

Quoted 7 lines
Quoted 5 lines

Changes in in v4:
- Drop ram_block_convert_range() and open code its implementation in the next Patch.

I only received 3 patches from this series, and now I am confused: changelog talks about v5 and this is "PATCH v3"

As above, because the guest_memfd patches in my TDX-QEMU v5[1] were directly picked for this series, so the change history says v5. They are needed by SEV-SNP as well.

I want to raise the question, how do we want to proceed with the guest memfd patches (patch 2 to 10 in [1])? Can they be merged separately before TDX/SNP patches?

Quoted 2 lines

Please make sure to send at least the cover letter along (I might not need the other 46 patches :D ).

[1] https://lore.kernel.org/qemu-devel/[email protected]/

#24 lore raw

Re: [PATCH v3 11/49] physmem: Introduce ram_block_discard_guest_memfd_range()

Quoted 29 lines · Xiaoyao Li
Quoted 8 lines · David Hildenbrand
Quoted 7 lines · Michael Roth

From: Xiaoyao Li <[email protected]>

When memory page is converted from private to shared, the original private memory is back'ed by guest_memfd. Introduce ram_block_discard_guest_memfd_range() for discarding memory in guest_memfd.

Quoted 4 lines
Quoted 2 lines

Originally-from: Isaku Yamahata <[email protected]>

"Co-developed-by"

Michael is using the patch from my TDX-QEMU v5 series[1]. I need to fix it.

Quoted 13 lines
Quoted 2 lines

Your SOB should go here.

Quoted 8 lines

---
Changes in v5:
- Collect Reviewed-by from David;

Changes in in v4:
- Drop ram_block_convert_range() and open code its implementation in the next Patch.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

I only received 3 patches from this series, and now I am confused:
changelog talks about v5 and this is "PATCH v3"

As above, because the guest_memfd patches in my TDX-QEMU v5[1] were directly picked for this series, so the change history says v5. They are needed by SEV-SNP as well.

Thanks, I was missing the context without a cover letter. These comments here likely should be dropped here.

Signature · 3 lines
Cheers,

David / dhildenb
#25 lore raw

Re: [PATCH v3 11/49] physmem: Introduce ram_block_discard_guest_memfd_range()

In reply to David Hildenbrand
Quoted 26 lines · David Hildenbrand
Quoted 9 lines · Michael Roth

From: Xiaoyao Li <[email protected]>

When memory page is converted from private to shared, the original private memory is back'ed by guest_memfd. Introduce ram_block_discard_guest_memfd_range() for discarding memory in guest_memfd.

Originally-from: Isaku Yamahata <[email protected]>

"Co-developed-by"

Quoted 2 lines

Your SOB should go here.

Quoted 8 lines

---
Changes in v5:
- Collect Reviewed-by from David;

Changes in in v4:
- Drop ram_block_convert_range() and open code its implementation in the next Patch.

Quoted 7 lines
Quoted 1 line

I only received 3 patches from this series, and now I am confused: changelog talks about v5 and this is "PATCH v3"

Please make sure to send at least the cover letter along (I might not need the other 46 patches :D ).

Sorry for the confusion, you got auto-Cc'd by git, which is good, but not sure there's a good way to make sure everyone gets a copy of the cover letter. I could see how it would help useful to potential reviewers though. I'll try to come up with a script for it and take that approach in the future.

-Mike

Quoted 6 lines
Signature · 4 lines
-- 
Cheers,

David / dhildenb
#26 lore raw

Re: [PATCH v3 11/49] physmem: Introduce ram_block_discard_guest_memfd_range()

Quoted 32 lines · Michael Roth
Quoted 31 lines · David Hildenbrand
Quoted 9 lines · Michael Roth

From: Xiaoyao Li <[email protected]>

When memory page is converted from private to shared, the original private memory is back'ed by guest_memfd. Introduce ram_block_discard_guest_memfd_range() for discarding memory in guest_memfd.

Originally-from: Isaku Yamahata <[email protected]>

"Co-developed-by"

Quoted 2 lines

Your SOB should go here.

Quoted 9 lines

---
Changes in v5:
- Collect Reviewed-by from David;

Changes in in v4:
- Drop ram_block_convert_range() and open code its implementation in the next Patch.

I only received 3 patches from this series, and now I am confused: changelog talks about v5 and this is "PATCH v3"

Quoted 8 lines
Quoted 2 lines

Please make sure to send at least the cover letter along (I might not need the other 46 patches :D ).

Sorry for the confusion, you got auto-Cc'd by git, which is good, but not sure there's a good way to make sure everyone gets a copy of the cover letter. I could see how it would help useful to potential reviewers though. I'll try to come up with a script for it and take that approach in the future.

A script shared with me in the past to achieve that in most cases:

$ cat cc-cmd.sh
#!/bin/bash

if [[ $1 == *gitsendemail.msg* || $1 == *cover-letter* ]]; then

        grep ': .* <.*@.*>' -h *.patch | sed 's/^.*: //' | sort | uniq

fi

And attach to "git send-email ... *.patch": --cc-cmd=./cc-cmd.sh

Signature · 3 lines
Cheers,

David / dhildenb
#27 lore raw

Re: [PATCH v3 11/49] physmem: Introduce ram_block_discard_guest_memfd_range()

Quoted 47 lines · David Hildenbrand
Quoted 32 lines · Michael Roth
Quoted 31 lines · David Hildenbrand
Quoted 9 lines · Michael Roth

From: Xiaoyao Li <[email protected]>

When memory page is converted from private to shared, the original private memory is back'ed by guest_memfd. Introduce ram_block_discard_guest_memfd_range() for discarding memory in guest_memfd.

Originally-from: Isaku Yamahata <[email protected]>

"Co-developed-by"

Quoted 2 lines

Your SOB should go here.

Quoted 9 lines

---
Changes in v5:
- Collect Reviewed-by from David;

Changes in in v4:
- Drop ram_block_convert_range() and open code its implementation in the next Patch.

I only received 3 patches from this series, and now I am confused: changelog talks about v5 and this is "PATCH v3"

Quoted 8 lines
Quoted 2 lines

Please make sure to send at least the cover letter along (I might not need the other 46 patches :D ).

Sorry for the confusion, you got auto-Cc'd by git, which is good, but not sure there's a good way to make sure everyone gets a copy of the cover letter. I could see how it would help useful to potential reviewers though. I'll try to come up with a script for it and take that approach in the future.

A script shared with me in the past to achieve that in most cases:

$ cat cc-cmd.sh
#!/bin/bash
Quoted 6 lines

if [[ $1 == *gitsendemail.msg* || $1 == *cover-letter* ]]; then

        grep ': .* <.*@.*>' -h *.patch | sed 's/^.*: //' | sort | uniq

fi

And attach to "git send-email ... *.patch": --cc-cmd=./cc-cmd.sh

That should do the trick nicely. Thanks!

-Mike

Quoted 6 lines
Signature · 4 lines
-- 
Cheers,

David / dhildenb
#28 lore raw

PATCH v3 12/49 kvm: handle KVM_EXIT_MEMORY_FAULT

In reply to Michael Roth

From: Chao Peng <[email protected]>

When geeting KVM_EXIT_MEMORY_FAULT exit, it indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.

Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when KVM_EXIT_MEMORY_FAULT happens.

Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has guest_memfd memory backend.

Note, KVM_EXIT_MEMORY_FAULT returns with -EFAULT, so special handling is added.

When page is converted from shared to private, the original shared memory can be discarded via ram_block_discard_range(). Note, shared memory can be discarded only when it's not back'ed by hugetlb because hugetlb is supposed to be pre-allocated and no need for discarding.

---
Changes in v4:
- open-coded ram_block_discard logic;
- change warn_report() to error_report(); (Daniel)

---
 accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 84 insertions(+), 10 deletions(-)
accel/kvm/kvm-all.c +84 -10
index df7a32735a..2fdc07a472 100644
@@ -2903,6 +2903,68 @@ static void kvm_eat_signals(CPUState *cpu)
     } while (sigismember(&chkset, SIG_IPI));
 }
 
+static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
+{
+    MemoryRegionSection section;
+    ram_addr_t offset;
+    MemoryRegion *mr;
+    RAMBlock *rb;
+    void *addr;
+    int ret = -1;
+
+    if (!QEMU_PTR_IS_ALIGNED(start, qemu_host_page_size) ||
+        !QEMU_PTR_IS_ALIGNED(size, qemu_host_page_size)) {
+        return -1;
+    }
+
+    if (!size) {
+        return -1;
+    }
+
+    section = memory_region_find(get_system_memory(), start, size);
+    mr = section.mr;
+    if (!mr) {
+        return -1;
+    }
+
+    if (memory_region_has_guest_memfd(mr)) {
+        if (to_private) {
+            ret = kvm_set_memory_attributes_private(start, size);
+        } else {
+            ret = kvm_set_memory_attributes_shared(start, size);
+        }
+
+        if (ret) {
+            memory_region_unref(section.mr);
+            return ret;
+        }
+
+        addr = memory_region_get_ram_ptr(mr) + section.offset_within_region;
+        rb = qemu_ram_block_from_host(addr, false, &offset);
+
+        if (to_private) {
+            if (rb->page_size != qemu_host_page_size) {
+                /*
+                * shared memory is back'ed by  hugetlb, which is supposed to be
+                * pre-allocated and doesn't need to be discarded
+                */
+                return 0;
+            } else {
+                ret = ram_block_discard_range(rb, offset, size);
+            }
+        } else {
+            ret = ram_block_discard_guest_memfd_range(rb, offset, size);
+        }
+    } else {
+        error_report("Convert non guest_memfd backed memory region "
+                    "(0x%"HWADDR_PRIx" ,+ 0x%"HWADDR_PRIx") to %s",
+                    start, size, to_private ? "private" : "shared");
+    }
+
+    memory_region_unref(section.mr);
+    return ret;
+}
+
 int kvm_cpu_exec(CPUState *cpu)
 {
     struct kvm_run *run = cpu->kvm_run;
@@ -2970,18 +3032,20 @@ int kvm_cpu_exec(CPUState *cpu)
                 ret = EXCP_INTERRUPT;
                 break;
             }
-            fprintf(stderr, "error: kvm run failed %s\n",
-                    strerror(-run_ret));
+            if (!(run_ret == -EFAULT && run->exit_reason == KVM_EXIT_MEMORY_FAULT)) {
+                fprintf(stderr, "error: kvm run failed %s\n",
+                        strerror(-run_ret));
 #ifdef TARGET_PPC
-            if (run_ret == -EBUSY) {
-                fprintf(stderr,
-                        "This is probably because your SMT is enabled.\n"
-                        "VCPU can only run on primary threads with all "
-                        "secondary threads offline.\n");
-            }
+                if (run_ret == -EBUSY) {
+                    fprintf(stderr,
+                            "This is probably because your SMT is enabled.\n"
+                            "VCPU can only run on primary threads with all "
+                            "secondary threads offline.\n");
+                }
 #endif
-            ret = -1;
-            break;
+                ret = -1;
+                break;
+            }
         }
 
         trace_kvm_run_exit(cpu->cpu_index, run->exit_reason);
@@ -3064,6 +3128,16 @@ int kvm_cpu_exec(CPUState *cpu)
                 break;
             }
             break;
+        case KVM_EXIT_MEMORY_FAULT:
+            if (run->memory_fault.flags & ~KVM_MEMORY_EXIT_FLAG_PRIVATE) {
+                error_report("KVM_EXIT_MEMORY_FAULT: Unknown flag 0x%" PRIx64,
+                             (uint64_t)run->memory_fault.flags);
+                ret = -1;
+                break;
+            }
+            ret = kvm_convert_memory(run->memory_fault.gpa, run->memory_fault.size,
+                                     run->memory_fault.flags & KVM_MEMORY_EXIT_FLAG_PRIVATE);
+            break;
         default:
             ret = kvm_arch_handle_exit(cpu, run);
             break;
Signature · 1 line
2.25.1
#29 lore raw

PATCH v3 13/49 "kvm: handle KVM_EXIT_MEMORY_FAULT": drop qemu_host_page_size

In reply to Michael Roth

TODO: squash into "kvm: handle KVM_EXIT_MEMORY_FAULT"

qemu_host_page_size has been superseded by qemu_real_host_page_size() in newer QEMU, so update the patch accordingly.

---
 accel/kvm/kvm-all.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
accel/kvm/kvm-all.c +3 -3
index 2fdc07a472..a9c19ab9a1 100644
@@ -2912,8 +2912,8 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
     void *addr;
     int ret = -1;
 
-    if (!QEMU_PTR_IS_ALIGNED(start, qemu_host_page_size) ||
-        !QEMU_PTR_IS_ALIGNED(size, qemu_host_page_size)) {
+    if (!QEMU_PTR_IS_ALIGNED(start, qemu_real_host_page_size()) ||
+        !QEMU_PTR_IS_ALIGNED(size, qemu_real_host_page_size())) {
         return -1;
     }
 
@@ -2943,7 +2943,7 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
         rb = qemu_ram_block_from_host(addr, false, &offset);
 
         if (to_private) {
-            if (rb->page_size != qemu_host_page_size) {
+            if (rb->page_size != qemu_real_host_page_size()) {
                 /*
                 * shared memory is back'ed by  hugetlb, which is supposed to be
                 * pre-allocated and doesn't need to be discarded
Signature · 1 line
2.25.1
#30 lore raw

Re: [PATCH v3 13/49] [FIXUP] "kvm: handle KVM_EXIT_MEMORY_FAULT": drop qemu_host_page_size

Quoted 4 lines · Michael Roth

TODO: squash into "kvm: handle KVM_EXIT_MEMORY_FAULT"

qemu_host_page_size has been superseded by qemu_real_host_page_size() in newer QEMU, so update the patch accordingly.

I found it today as well when rebase to qemu v9.0.0-rc0.

Fix it locally, will show up on my next post of TDX-QEMU patches. :)

Quoted 29 lines
---
  accel/kvm/kvm-all.c | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)
accel/kvm/kvm-all.c +3 -3
index 2fdc07a472..a9c19ab9a1 100644
@@ -2912,8 +2912,8 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
      void *addr;
      int ret = -1;
 
-    if (!QEMU_PTR_IS_ALIGNED(start, qemu_host_page_size) ||
-        !QEMU_PTR_IS_ALIGNED(size, qemu_host_page_size)) {
+    if (!QEMU_PTR_IS_ALIGNED(start, qemu_real_host_page_size()) ||
+        !QEMU_PTR_IS_ALIGNED(size, qemu_real_host_page_size())) {
          return -1;
      }
 
@@ -2943,7 +2943,7 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
          rb = qemu_ram_block_from_host(addr, false, &offset);
 
          if (to_private) {
-            if (rb->page_size != qemu_host_page_size) {
+            if (rb->page_size != qemu_real_host_page_size()) {
                  /*
                  * shared memory is back'ed by  hugetlb, which is supposed to be
                  * pre-allocated and doesn't need to be discarded
#31 lore raw

PATCH v3 14/49 trace/kvm: Add trace for page convertion between shared and private

In reply to Michael Roth
---
 accel/kvm/kvm-all.c    | 2 ++
 accel/kvm/trace-events | 1 +
 2 files changed, 3 insertions(+)
accel/kvm/kvm-all.c +2 -0
index a9c19ab9a1..9a8b365a69 100644
@@ -2912,6 +2912,8 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
     void *addr;
     int ret = -1;
 
+    trace_kvm_convert_memory(start, size, to_private ? "shared_to_private" : "private_to_shared");
+
     if (!QEMU_PTR_IS_ALIGNED(start, qemu_real_host_page_size()) ||
         !QEMU_PTR_IS_ALIGNED(size, qemu_real_host_page_size())) {
         return -1;
accel/kvm/trace-events +1 -0
index e8c52cb9e7..31175fed97 100644
@@ -31,3 +31,4 @@ kvm_cpu_exec(void) ""
 kvm_interrupt_exit_request(void) ""
 kvm_io_window_exit(void) ""
 kvm_run_exit_system_event(int cpu_index, uint32_t event_type) "cpu_index %d, system_even_type %"PRIu32
+kvm_convert_memory(uint64_t start, uint64_t size, const char *msg) "start 0x%" PRIx64 " size 0x%" PRIx64 " %s"
Signature · 1 line
2.25.1
#32 lore raw

PATCH v3 15/49 kvm/memory: Make memory type private by default if it has guest memfd backend

In reply to Michael Roth

From: Xiaoyao Li <[email protected]>

KVM side leaves the memory to shared by default, while may incur the overhead of paging conversion on the first visit of each page. Because the expectation is that page is likely to private for the VMs that require private memory (has guest memfd).

Explicitly set the memory to private when memory region has valid guest memfd backend.

---
 accel/kvm/kvm-all.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
accel/kvm/kvm-all.c +10 -0
index 9a8b365a69..53ce4f091e 100644
@@ -1451,6 +1451,16 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
                     strerror(-err));
             abort();
         }
+
+        if (memory_region_has_guest_memfd(mr)) {
+            err = kvm_set_memory_attributes_private(start_addr, slot_size);
+            if (err) {
+                error_report("%s: failed to set memory attribute private: %s\n",
+                             __func__, strerror(-err));
+                exit(1);
+            }
+        }
+
         start_addr += slot_size;
         ram_start_offset += slot_size;
         ram += slot_size;
Signature · 1 line
2.25.1
#33 lore raw

PATCH v3 16/49 memory: Introduce memory_region_init_ram_guest_memfd()

In reply to Michael Roth

From: Xiaoyao Li <[email protected]>

Introduce memory_region_init_ram_guest_memfd() to allocate private guset memfd on the MemoryRegion initialization. It's for the use case of TDVF, which must be private on TDX case.

---
Changes in v5:
- drop memory_region_set_default_private() because this function is dropped in this v5 series;

---
 include/exec/memory.h |  6 ++++++
 system/memory.c       | 25 +++++++++++++++++++++++++
 2 files changed, 31 insertions(+)
include/exec/memory.h +6 -0
index 679a847685..1e351f6fc8 100644
@@ -1603,6 +1603,12 @@ bool memory_region_init_ram(MemoryRegion *mr,
                             uint64_t size,
                             Error **errp);
 
+bool memory_region_init_ram_guest_memfd(MemoryRegion *mr,
+                                        Object *owner,
+                                        const char *name,
+                                        uint64_t size,
+                                        Error **errp);
+
 /**
  * memory_region_init_rom: Initialize a ROM memory region.
  *
system/memory.c +25 -0
index c756950c0c..85a22408e9 100644
@@ -3606,6 +3606,31 @@ bool memory_region_init_ram(MemoryRegion *mr,
     return true;
 }
 
+bool memory_region_init_ram_guest_memfd(MemoryRegion *mr,
+                                        Object *owner,
+                                        const char *name,
+                                        uint64_t size,
+                                        Error **errp)
+{
+    DeviceState *owner_dev;
+
+    if (!memory_region_init_ram_flags_nomigrate(mr, owner, name, size,
+                                                RAM_GUEST_MEMFD, errp)) {
+        return false;
+    }
+
+    /* This will assert if owner is neither NULL nor a DeviceState.
+     * We only want the owner here for the purposes of defining a
+     * unique name for migration. TODO: Ideally we should implement
+     * a naming scheme for Objects which are not DeviceStates, in
+     * which case we can relax this restriction.
+     */
+    owner_dev = DEVICE(owner);
+    vmstate_register_ram(mr, owner_dev);
+
+    return true;
+}
+
 bool memory_region_init_rom(MemoryRegion *mr,
                             Object *owner,
                             const char *name,
Signature · 1 line
2.25.1
#34 lore raw

PATCH v3 17/49 pci-host/q35: Move PAM initialization above SMRAM initialization

In reply to Michael Roth

From: Isaku Yamahata <[email protected]>

In mch_realize(), process PAM initialization before SMRAM initialization so that later patch can skill all the SMRAM related with a single check.

---
 hw/pci-host/q35.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)
hw/pci-host/q35.c +10 -9
index 0d7d4e3f08..98d4a7c253 100644
@@ -568,6 +568,16 @@ static void mch_realize(PCIDevice *d, Error **errp)
     /* setup pci memory mapping */
     pc_pci_as_mapping_init(mch->system_memory, mch->pci_address_space);
 
+    /* PAM */
+    init_pam(&mch->pam_regions[0], OBJECT(mch), mch->ram_memory,
+             mch->system_memory, mch->pci_address_space,
+             PAM_BIOS_BASE, PAM_BIOS_SIZE);
+    for (i = 0; i < ARRAY_SIZE(mch->pam_regions) - 1; ++i) {
+        init_pam(&mch->pam_regions[i + 1], OBJECT(mch), mch->ram_memory,
+                 mch->system_memory, mch->pci_address_space,
+                 PAM_EXPAN_BASE + i * PAM_EXPAN_SIZE, PAM_EXPAN_SIZE);
+    }
+
     /* if *disabled* show SMRAM to all CPUs */
     memory_region_init_alias(&mch->smram_region, OBJECT(mch), "smram-region",
                              mch->pci_address_space, MCH_HOST_BRIDGE_SMRAM_C_BASE,
@@ -634,15 +644,6 @@ static void mch_realize(PCIDevice *d, Error **errp)
 
     object_property_add_const_link(qdev_get_machine(), "smram",
                                    OBJECT(&mch->smram));
-
-    init_pam(&mch->pam_regions[0], OBJECT(mch), mch->ram_memory,
-             mch->system_memory, mch->pci_address_space,
-             PAM_BIOS_BASE, PAM_BIOS_SIZE);
-    for (i = 0; i < ARRAY_SIZE(mch->pam_regions) - 1; ++i) {
-        init_pam(&mch->pam_regions[i + 1], OBJECT(mch), mch->ram_memory,
-                 mch->system_memory, mch->pci_address_space,
-                 PAM_EXPAN_BASE + i * PAM_EXPAN_SIZE, PAM_EXPAN_SIZE);
-    }
 }
 
 uint64_t mch_mcfg_base(void)
Signature · 1 line
2.25.1
#35 lore raw

PATCH v3 18/49 q35: Introduce smm_ranges property for q35-pci-host

In reply to Michael Roth

From: Isaku Yamahata <[email protected]>

Add a q35 property to check whether or not SMM ranges, e.g. SMRAM, TSEG, etc... exist for the target platform. TDX doesn't support SMM and doesn't play nice with QEMU modifying related guest memory ranges.

---
 hw/i386/pc_q35.c          |  2 ++
 hw/pci-host/q35.c         | 42 +++++++++++++++++++++++++++------------
 include/hw/i386/pc.h      |  1 +
 include/hw/pci-host/q35.h |  1 +
 4 files changed, 33 insertions(+), 13 deletions(-)
hw/i386/pc_q35.c +2 -0
index 8a427c4647..42324448d7 100644
@@ -219,6 +219,8 @@ static void pc_q35_init(MachineState *machine)
                             x86ms->above_4g_mem_size, NULL);
     object_property_set_bool(phb, PCI_HOST_BYPASS_IOMMU,
                              pcms->default_bus_bypass_iommu, NULL);
+    object_property_set_bool(phb, PCI_HOST_PROP_SMM_RANGES,
+                             x86_machine_is_smm_enabled(x86ms), NULL);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(phb), &error_fatal);
 
     /* pci */
hw/pci-host/q35.c +29 -13

81 lines not on this page · raw message

include/hw/i386/pc.h +1 -0
index 27a68071d7..fb1d4106e5 100644
@@ -161,6 +161,7 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level);
 #define PCI_HOST_PROP_PCI_HOLE64_SIZE  "pci-hole64-size"
 #define PCI_HOST_BELOW_4G_MEM_SIZE     "below-4g-mem-size"
 #define PCI_HOST_ABOVE_4G_MEM_SIZE     "above-4g-mem-size"
+#define PCI_HOST_PROP_SMM_RANGES       "smm-ranges"
 
 
 void pc_pci_as_mapping_init(MemoryRegion *system_memory,
include/hw/pci-host/q35.h +1 -0
index bafcbe6752..22fadfa3ed 100644
@@ -50,6 +50,7 @@ struct MCHPCIState {
     MemoryRegion tseg_blackhole, tseg_window;
     MemoryRegion smbase_blackhole, smbase_window;
     bool has_smram_at_smbase;
+    bool has_smm_ranges;
     Range pci_hole;
     uint64_t below_4g_mem_size;
     uint64_t above_4g_mem_size;
Signature · 1 line
2.25.1
#36 lore raw

PATCH v3 19/49 kvm: Make kvm_convert_memory() obey ram_block_discard_is_enabled()

In reply to Michael Roth

Some subsystems like VFIO might disable ram block discard for uncoordinated cases. Since kvm_convert_memory()/guest_memfd don't implement a RamDiscardManager handler to convey discard operations to various listeners like VFIO. Because of this, sequences like the following can result due to stale IOMMU mappings:

- convert page shared->private
- discard shared page
- convert page private->shared
- new page is allocated
- issue DMA operations against that shared page

Address this by taking ram_block_discard_is_enabled() into account when deciding whether or not to discard pages.

---
 accel/kvm/kvm-all.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
accel/kvm/kvm-all.c +6 -2
index 53ce4f091e..6ae03c880f 100644
@@ -2962,10 +2962,14 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
                 */
                 return 0;
             } else {
-                ret = ram_block_discard_range(rb, offset, size);
+                ret = ram_block_discard_is_disabled()
+                      ? ram_block_discard_range(rb, offset, size)
+                      : 0;
             }
         } else {
-            ret = ram_block_discard_guest_memfd_range(rb, offset, size);
+            ret = ram_block_discard_is_disabled()
+                  ? ram_block_discard_guest_memfd_range(rb, offset, size)
+                  : 0;
         }
     } else {
         error_report("Convert non guest_memfd backed memory region "
Signature · 1 line
2.25.1
maintainer
#37 lore raw

Re: [PATCH v3 19/49] kvm: Make kvm_convert_memory() obey ram_block_discard_is_enabled()

Quoted 5 lines · Michael Roth

Some subsystems like VFIO might disable ram block discard for uncoordinated cases. Since kvm_convert_memory()/guest_memfd don't implement a RamDiscardManager handler to convey discard operations to various listeners like VFIO. > Because of this, sequences like the following can result due to stale IOMMU mappings:

Alternatively, should guest-memfd memory regions call ram_block_discard_require(true)? This will prevent VFIO from operating, but it will avoid consuming twice the memory.

If desirable, guest-memfd support can be changed to implement an extension of RamDiscardManager that notifies about private/shared memory changes, and then guest-memfd would be able to support coordinated discard. But I wonder if that's doable at all - how common are shared<->private flips, and is it feasible to change the IOMMU page tables every time?

If the real solution is SEV-TIO (which means essentially guest_memfd support for VFIO), calling ram_block_discard_require(true) may be the simplest stopgap solution.

Paolo

Quoted 35 lines

- convert page shared->private
- discard shared page
- convert page private->shared
- new page is allocated
- issue DMA operations against that shared page

Address this by taking ram_block_discard_is_enabled() into account when deciding whether or not to discard pages.

---
  accel/kvm/kvm-all.c | 8 ++++++--
  1 file changed, 6 insertions(+), 2 deletions(-)
accel/kvm/kvm-all.c +6 -2
index 53ce4f091e..6ae03c880f 100644
@@ -2962,10 +2962,14 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
                  */
                  return 0;
              } else {
-                ret = ram_block_discard_range(rb, offset, size);
+                ret = ram_block_discard_is_disabled()
+                      ? ram_block_discard_range(rb, offset, size)
+                      : 0;
              }
          } else {
-            ret = ram_block_discard_guest_memfd_range(rb, offset, size);
+            ret = ram_block_discard_is_disabled()
+                  ? ram_block_discard_guest_memfd_range(rb, offset, size)
+                  : 0;
          }
      } else {
          error_report("Convert non guest_memfd backed memory region "
#38 lore raw

Re: [PATCH v3 19/49] kvm: Make kvm_convert_memory() obey ram_block_discard_is_enabled()

Quoted 7 lines · Paolo Bonzini
Quoted 5 lines · Michael Roth

Some subsystems like VFIO might disable ram block discard for uncoordinated cases. Since kvm_convert_memory()/guest_memfd don't implement a RamDiscardManager handler to convey discard operations to various listeners like VFIO. > Because of this, sequences like the following can result due to stale IOMMU mappings:

Quoted 7 lines

Alternatively, should guest-memfd memory regions call ram_block_discard_require(true)? This will prevent VFIO from operating, but it will avoid consuming twice the memory.

If desirable, guest-memfd support can be changed to implement an extension of RamDiscardManager that notifies about private/shared memory changes, and then guest-memfd would be able to support coordinated discard. But I wonder

In an earlier/internal version of the SNP+gmem patches (when there was still a dedicated hostmem-memfd-private backend for restrictedmem/gmem), we had a rough implementation of RamDiscardManager that did this:

https://github.com/AMDESE/qemu/blob/snp-latest-gmem-v12/backends/hostmem-memfd-private.c#L75

Now that gmem handling is mostly done transparently to the HostMem backend in use I'm not sure what the right place would be to implement something similar, but maybe it can be done in a more generic way.

There were some notable downsides to that approach though that I'm a little hazy on now, but I think they were both kernel limitations:

- VFIO seemed to have some limitation where it expects that the DMA mapping for a particular iova will be unmapped/mapped with the same granularity, but for an SNP guest there's no guarantee that if you flip a 2MB page from shared->private, that it won't later be flipped private->shared again but this time with a 4K granularity/sub-range. I think the current code still treats this as an -EINVAL case. So we end up needing to do everything with 4K granularity, which I *think* results in 4K IOMMU page table mappings, but I'd need to confirm.

- VFIO doesn't seem to be optimized for this sort of use case and generally expects a much larger granularity and defaults to 64K max DMA entries, so for a 16GB guest you need to configure VFIO with something like:

vfio_iommu_type1.dma_entry_limit=4194304

I didn't see any reason to suggest that's problematic but it makes we wonder if there's other stuff me might run into.

Quoted 2 lines

if that's doable at all - how common are shared<->private flips, and is it feasible to change the IOMMU page tables every time?

- For OVMF+guest kernel that don't do lazy-acceptance:

I think the bulk of the flipping is during boot where most of shared GPA ranges get converted to private memory, and then later on the guest kernel switches memory back to to shared for stuff like SWIOTLB, and after that I think DMA mappings would be fairly stable.

- For OVMF+guest kernel that support lazy-acceptance:

The first 4GB get converted to private, and the rest remains shared until guest kernel needs to allocate memory from it. I'm not sure if SWIOTLB allocation is optimized to avoid unecessary flipping if it's allocated from that pool of still-shared memory, but normal/private allocations will result in a steady stream of DMA unmap operations as the guest faults in its working set.

Quoted 4 lines

If the real solution is SEV-TIO (which means essentially guest_memfd support for VFIO), calling ram_block_discard_require(true) may be the simplest stopgap solution.

Hard to guess how cloud vendors will feel about waiting for trusted I/O. It does make sense in the context of CoCo to expect them to wait, but would be nice to have a stop-gap to offer like disabling discard, since it has minimal requirements on the QEMU/VFIO side and might be enough to get early adopters up and running at least.

All that said, if you think something based around RamDiscardManager seems tenable given all above then we can re-visit that approach as well.

-Mike

Quoted 39 lines

Paolo

Quoted 35 lines

- convert page shared->private
- discard shared page
- convert page private->shared
- new page is allocated
- issue DMA operations against that shared page

Address this by taking ram_block_discard_is_enabled() into account when deciding whether or not to discard pages.

---
  accel/kvm/kvm-all.c | 8 ++++++--
  1 file changed, 6 insertions(+), 2 deletions(-)
accel/kvm/kvm-all.c +6 -2
index 53ce4f091e..6ae03c880f 100644
@@ -2962,10 +2962,14 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
                  */
                  return 0;
              } else {
-                ret = ram_block_discard_range(rb, offset, size);
+                ret = ram_block_discard_is_disabled()
+                      ? ram_block_discard_range(rb, offset, size)
+                      : 0;
              }
          } else {
-            ret = ram_block_discard_guest_memfd_range(rb, offset, size);
+            ret = ram_block_discard_is_disabled()
+                  ? ram_block_discard_guest_memfd_range(rb, offset, size)
+                  : 0;
          }
      } else {
          error_report("Convert non guest_memfd backed memory region "
#39 lore raw

PATCH v3 20/49 trace/kvm: Add trace for KVM_EXIT_MEMORY_FAULT

In reply to Michael Roth
---
 accel/kvm/kvm-all.c    | 3 +++
 accel/kvm/trace-events | 1 +
 2 files changed, 4 insertions(+)
accel/kvm/kvm-all.c +3 -0
index 6ae03c880f..b5872fdc07 100644
@@ -3145,6 +3145,9 @@ int kvm_cpu_exec(CPUState *cpu)
             }
             break;
         case KVM_EXIT_MEMORY_FAULT:
+            trace_kvm_memory_fault(run->memory_fault.gpa,
+                                   run->memory_fault.size,
+                                   run->memory_fault.flags);
             if (run->memory_fault.flags & ~KVM_MEMORY_EXIT_FLAG_PRIVATE) {
                 error_report("KVM_EXIT_MEMORY_FAULT: Unknown flag 0x%" PRIx64,
                              (uint64_t)run->memory_fault.flags);
accel/kvm/trace-events +1 -0
index 31175fed97..681ccb667d 100644
@@ -32,3 +32,4 @@ kvm_interrupt_exit_request(void) ""
 kvm_io_window_exit(void) ""
 kvm_run_exit_system_event(int cpu_index, uint32_t event_type) "cpu_index %d, system_even_type %"PRIu32
 kvm_convert_memory(uint64_t start, uint64_t size, const char *msg) "start 0x%" PRIx64 " size 0x%" PRIx64 " %s"
+kvm_memory_fault(uint64_t start, uint64_t size, uint64_t flags) "start 0x%" PRIx64 " size 0x%" PRIx64 " flags 0x%" PRIx64
Signature · 1 line
2.25.1
#40 lore raw

PATCH v3 21/49 i386/sev: Introduce "sev-common" type to encapsulate common SEV state

In reply to Michael Roth

Currently all SEV/SEV-ES functionality is managed through a single 'sev-guest' QOM type. With upcoming support for SEV-SNP, taking this same approach won't work well since some of the properties/state managed by 'sev-guest' is not applicable to SEV-SNP, which will instead rely on a new QOM type with its own set of properties/state.

To prepare for this, this patch moves common state into an abstract 'sev-common' parent type to encapsulate properties/state that are common to both SEV/SEV-ES and SEV-SNP, leaving only SEV/SEV-ES-specific properties/state in the current 'sev-guest' type. This should not affect current behavior or command-line options.

As part of this patch, some related changes are also made:

- a static 'sev_guest' variable is currently used to keep track of the 'sev-guest' instance. SEV-SNP would similarly introduce an 'sev_snp_guest' static variable. But these instances are now available via qdev_get_machine()->cgs, so switch to using that instead and drop the static variable.

- 'sev_guest' is currently used as the name for the static variable holding a pointer to the 'sev-guest' instance. Re-purpose the name as a local variable referring the 'sev-guest' instance, and use that consistently throughout the code so it can be easily distinguished from sev-common/sev-snp-guest instances.

- 'sev' is generally used as the name for local variables holding a pointer to the 'sev-guest' instance. In cases where that now points to common state, use the name 'sev_common'; in cases where that now points to state specific to 'sev-guest' instance, use the name 'sev_guest'

---
 qapi/qom.json     |  32 ++--
 target/i386/sev.c | 457 ++++++++++++++++++++++++++--------------------
 target/i386/sev.h |   3 +
 3 files changed, 281 insertions(+), 211 deletions(-)
qapi/qom.json +21 -11
index baae3a183f..66b5781ca6 100644
@@ -875,12 +875,29 @@
   'data': { '*filename': 'str' } }
 
 ##
-# @SevGuestProperties:
+# @SevCommonProperties:
 #
-# Properties for sev-guest objects.
+# Properties common to objects that are derivatives of sev-common.
 #
 # @sev-device: SEV device to use (default: "/dev/sev")
 #
+# @cbitpos: C-bit location in page table entry (default: 0)
+#
+# @reduced-phys-bits: number of bits in physical addresses that become
+#     unavailable when SEV is enabled
+#
+# Since: 2.12
+##
+{ 'struct': 'SevCommonProperties',
+  'data': { '*sev-device': 'str',
+            '*cbitpos': 'uint32',
+            'reduced-phys-bits': 'uint32' } }
+
+##
+# @SevGuestProperties:
+#
+# Properties for sev-guest objects.
+#
 # @dh-cert-file: guest owners DH certificate (encoded with base64)
 #
 # @session-file: guest owners session parameters (encoded with base64)
@@ -889,11 +906,6 @@
 #
 # @handle: SEV firmware handle (default: 0)
 #
-# @cbitpos: C-bit location in page table entry (default: 0)
-#
-# @reduced-phys-bits: number of bits in physical addresses that become
-#     unavailable when SEV is enabled
-#
 # @kernel-hashes: if true, add hashes of kernel/initrd/cmdline to a
 #     designated guest firmware page for measured boot with -kernel
 #     (default: false) (since 6.2)
@@ -901,13 +913,11 @@
 # Since: 2.12
 ##
 { 'struct': 'SevGuestProperties',
-  'data': { '*sev-device': 'str',
-            '*dh-cert-file': 'str',
+  'base': 'SevCommonProperties',
+  'data': { '*dh-cert-file': 'str',
             '*session-file': 'str',
             '*policy': 'uint32',
             '*handle': 'uint32',
-            '*cbitpos': 'uint32',
-            'reduced-phys-bits': 'uint32',
             '*kernel-hashes': 'bool' } }
 
 ##
target/i386/sev.c +257 -200

880 lines not on this page · raw message

target/i386/sev.h +3 -0
index 9e10d09539..668374eef3 100644
@@ -20,6 +20,9 @@
 
 #include "exec/confidential-guest-support.h"
 
+#define TYPE_SEV_COMMON "sev-common"
+#define TYPE_SEV_GUEST "sev-guest"
+
 #define SEV_POLICY_NODBG        0x1
 #define SEV_POLICY_NOKS         0x2
 #define SEV_POLICY_ES           0x4
Signature · 1 line
2.25.1
#41 lore raw

Re: [PATCH v3 21/49] i386/sev: Introduce "sev-common" type to encapsulate common SEV state

Quoted 51 lines · Michael Roth

Currently all SEV/SEV-ES functionality is managed through a single 'sev-guest' QOM type. With upcoming support for SEV-SNP, taking this same approach won't work well since some of the properties/state managed by 'sev-guest' is not applicable to SEV-SNP, which will instead rely on a new QOM type with its own set of properties/state.

To prepare for this, this patch moves common state into an abstract 'sev-common' parent type to encapsulate properties/state that are common to both SEV/SEV-ES and SEV-SNP, leaving only SEV/SEV-ES-specific properties/state in the current 'sev-guest' type. This should not affect current behavior or command-line options.

As part of this patch, some related changes are also made:

- a static 'sev_guest' variable is currently used to keep track of the 'sev-guest' instance. SEV-SNP would similarly introduce an 'sev_snp_guest' static variable. But these instances are now available via qdev_get_machine()->cgs, so switch to using that instead and drop the static variable.

- 'sev_guest' is currently used as the name for the static variable holding a pointer to the 'sev-guest' instance. Re-purpose the name as a local variable referring the 'sev-guest' instance, and use that consistently throughout the code so it can be easily distinguished from sev-common/sev-snp-guest instances.

- 'sev' is generally used as the name for local variables holding a pointer to the 'sev-guest' instance. In cases where that now points to common state, use the name 'sev_common'; in cases where that now points to state specific to 'sev-guest' instance, use the name 'sev_guest'

---
 qapi/qom.json     |  32 ++--
 target/i386/sev.c | 457 ++++++++++++++++++++++++++--------------------
 target/i386/sev.h |   3 +
 3 files changed, 281 insertions(+), 211 deletions(-)
qapi/qom.json +1 -2
index baae3a183f..66b5781ca6 100644
@@ -875,12 +875,29 @@
   'data': { '*filename': 'str' } }
 
 ##
-# @SevGuestProperties:
+# @SevCommonProperties:
 #
-# Properties for sev-guest objects.

qapi/qom.json line 889

Quoted 10 lines
qapi/qom.json +7 -0
+# Properties common to objects that are derivatives of sev-common.
 #
 # @sev-device: SEV device to use (default: "/dev/sev")
 #
+# @cbitpos: C-bit location in page table entry (default: 0)
+#
+# @reduced-phys-bits: number of bits in physical addresses that become
+#     unavailable when SEV is enabled
+#
+# Since: 2.12

Not quite sure what we've done in this scenario before. It feels wierd to use '2.12' for the new base type, even though in effect the properties all existed since 2.12 in the sub-class.

Perhaps 'Since: 9.1' for the type, but 'Since: 2.12' for the properties, along with an explanatory comment about stuff moving into the new base type ?

Markus, opinions ?

Quoted 284 lines
qapi/qom.json +13 -9
+##
+{ 'struct': 'SevCommonProperties',
+  'data': { '*sev-device': 'str',
+            '*cbitpos': 'uint32',
+            'reduced-phys-bits': 'uint32' } }
+
+##
+# @SevGuestProperties:
+#
+# Properties for sev-guest objects.
+#
 # @dh-cert-file: guest owners DH certificate (encoded with base64)
 #
 # @session-file: guest owners session parameters (encoded with base64)
@@ -889,11 +906,6 @@
 #
 # @handle: SEV firmware handle (default: 0)
 #
-# @cbitpos: C-bit location in page table entry (default: 0)
-#
-# @reduced-phys-bits: number of bits in physical addresses that become
-#     unavailable when SEV is enabled
-#
 # @kernel-hashes: if true, add hashes of kernel/initrd/cmdline to a
 #     designated guest firmware page for measured boot with -kernel
 #     (default: false) (since 6.2)
@@ -901,13 +913,11 @@
 # Since: 2.12
 ##
 { 'struct': 'SevGuestProperties',
-  'data': { '*sev-device': 'str',
-            '*dh-cert-file': 'str',
+  'base': 'SevCommonProperties',
+  'data': { '*dh-cert-file': 'str',
             '*session-file': 'str',
             '*policy': 'uint32',
             '*handle': 'uint32',
-            '*cbitpos': 'uint32',
-            'reduced-phys-bits': 'uint32',
             '*kernel-hashes': 'bool' } }
 
 ##
target/i386/sev.c +49 -104
index 9dab4060b8..63a220de5e 100644
@@ -40,48 +40,53 @@
 #include "hw/i386/pc.h"
 #include "exec/address-spaces.h"
 
-#define TYPE_SEV_GUEST "sev-guest"
+OBJECT_DECLARE_SIMPLE_TYPE(SevCommonState, SEV_COMMON)
 OBJECT_DECLARE_SIMPLE_TYPE(SevGuestState, SEV_GUEST)
 
-
-/**
- * SevGuestState:
- *
- * The SevGuestState object is used for creating and managing a SEV
- * guest.
- *
- * # $QEMU \
- *         -object sev-guest,id=sev0 \
- *         -machine ...,memory-encryption=sev0
- */
-struct SevGuestState {
+struct SevCommonState {
     X86ConfidentialGuest parent_obj;
 
     int kvm_type;
 
     /* configuration parameters */
     char *sev_device;
-    uint32_t policy;
-    char *dh_cert_file;
-    char *session_file;
     uint32_t cbitpos;
     uint32_t reduced_phys_bits;
-    bool kernel_hashes;
 
     /* runtime state */
-    uint32_t handle;
     uint8_t api_major;
     uint8_t api_minor;
     uint8_t build_id;
     int sev_fd;
     SevState state;
-    gchar *measurement;
 
     uint32_t reset_cs;
     uint32_t reset_ip;
     bool reset_data_valid;
 };
 
+/**
+ * SevGuestState:
+ *
+ * The SevGuestState object is used for creating and managing a SEV
+ * guest.
+ *
+ * # $QEMU \
+ *         -object sev-guest,id=sev0 \
+ *         -machine ...,memory-encryption=sev0
+ */
+struct SevGuestState {
+    SevCommonState sev_common;
+    gchar *measurement;
+
+    /* configuration parameters */
+    uint32_t handle;
+    uint32_t policy;
+    char *dh_cert_file;
+    char *session_file;
+    bool kernel_hashes;
+};
+
 #define DEFAULT_GUEST_POLICY    0x1 /* disable debug */
 #define DEFAULT_SEV_DEVICE      "/dev/sev"
 
@@ -127,7 +132,6 @@ typedef struct QEMU_PACKED PaddedSevHashTable {
 
 QEMU_BUILD_BUG_ON(sizeof(PaddedSevHashTable) % 16 != 0);
 
-static SevGuestState *sev_guest;
 static Error *sev_mig_blocker;
 
 static const char *const sev_fw_errlist[] = {
@@ -208,21 +212,21 @@ fw_error_to_str(int code)
 }
 
 static bool
-sev_check_state(const SevGuestState *sev, SevState state)
+sev_check_state(const SevCommonState *sev_common, SevState state)
 {
-    assert(sev);
-    return sev->state == state ? true : false;
+    assert(sev_common);
+    return sev_common->state == state ? true : false;
 }
 
 static void
-sev_set_guest_state(SevGuestState *sev, SevState new_state)
+sev_set_guest_state(SevCommonState *sev_common, SevState new_state)
 {
     assert(new_state < SEV_STATE__MAX);
-    assert(sev);
+    assert(sev_common);
 
-    trace_kvm_sev_change_state(SevState_str(sev->state),
+    trace_kvm_sev_change_state(SevState_str(sev_common->state),
                                SevState_str(new_state));
-    sev->state = new_state;
+    sev_common->state = new_state;
 }
 
 static void
@@ -289,111 +293,61 @@ static struct RAMBlockNotifier sev_ram_notifier = {
     .ram_block_removed = sev_ram_block_removed,
 };
 
-static void
-sev_guest_finalize(Object *obj)
-{
-}
-
-static char *
-sev_guest_get_session_file(Object *obj, Error **errp)
-{
-    SevGuestState *s = SEV_GUEST(obj);
-
-    return s->session_file ? g_strdup(s->session_file) : NULL;
-}
-
-static void
-sev_guest_set_session_file(Object *obj, const char *value, Error **errp)
-{
-    SevGuestState *s = SEV_GUEST(obj);
-
-    s->session_file = g_strdup(value);
-}
-
-static char *
-sev_guest_get_dh_cert_file(Object *obj, Error **errp)
-{
-    SevGuestState *s = SEV_GUEST(obj);
-
-    return g_strdup(s->dh_cert_file);
-}
-
-static void
-sev_guest_set_dh_cert_file(Object *obj, const char *value, Error **errp)
-{
-    SevGuestState *s = SEV_GUEST(obj);
-
-    s->dh_cert_file = g_strdup(value);
-}
-
-static char *
-sev_guest_get_sev_device(Object *obj, Error **errp)
-{
-    SevGuestState *sev = SEV_GUEST(obj);
-
-    return g_strdup(sev->sev_device);
-}
-
-static void
-sev_guest_set_sev_device(Object *obj, const char *value, Error **errp)
-{
-    SevGuestState *sev = SEV_GUEST(obj);
-
-    sev->sev_device = g_strdup(value);
-}
-
-static bool sev_guest_get_kernel_hashes(Object *obj, Error **errp)
-{
-    SevGuestState *sev = SEV_GUEST(obj);
-
-    return sev->kernel_hashes;
-}
-
-static void sev_guest_set_kernel_hashes(Object *obj, bool value, Error **errp)
-{
-    SevGuestState *sev = SEV_GUEST(obj);
-
-    sev->kernel_hashes = value;
-}
-
 bool
 sev_enabled(void)
 {
-    return !!sev_guest;
+    ConfidentialGuestSupport *cgs = MACHINE(qdev_get_machine())->cgs;
+
+    return !!object_dynamic_cast(OBJECT(cgs), TYPE_SEV_COMMON);
 }
 
 bool
 sev_es_enabled(void)
 {
-    return sev_enabled() && (sev_guest->policy & SEV_POLICY_ES);
+    ConfidentialGuestSupport *cgs = MACHINE(qdev_get_machine())->cgs;
+
+    return sev_enabled() && (SEV_GUEST(cgs)->policy & SEV_POLICY_ES);
 }
 
 uint32_t
 sev_get_cbit_position(void)
 {
-    return sev_guest ? sev_guest->cbitpos : 0;
+    SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+
+    return sev_common ? sev_common->cbitpos : 0;
 }
 
 uint32_t
 sev_get_reduced_phys_bits(void)
 {
-    return sev_guest ? sev_guest->reduced_phys_bits : 0;
+    SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+
+    return sev_common ? sev_common->reduced_phys_bits : 0;
 }
 
 static SevInfo *sev_get_info(void)
 {
     SevInfo *info;
+    SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+    SevGuestState *sev_guest =
+        (SevGuestState *)object_dynamic_cast(OBJECT(sev_common),
+                                             TYPE_SEV_GUEST);
 
     info = g_new0(SevInfo, 1);
     info->enabled = sev_enabled();
 
     if (info->enabled) {
-        info->api_major = sev_guest->api_major;
-        info->api_minor = sev_guest->api_minor;
-        info->build_id = sev_guest->build_id;
-        info->policy = sev_guest->policy;
-        info->state = sev_guest->state;
-        info->handle = sev_guest->handle;
+        if (sev_guest) {
+            info->handle = sev_guest->handle;

target/i386/sev.c static struct RAMBlockNotifier sev_ram_notifier = { · line 351

Quoted 10 lines
target/i386/sev.c +9 -0
+        }
+        info->api_major = sev_common->api_major;
+        info->api_minor = sev_common->api_minor;
+        info->build_id = sev_common->build_id;
+        info->state = sev_common->state;
+        /* we only report the lower 32-bits of policy for SNP, ok for now... */
+        info->policy =
+            (uint32_t)object_property_get_uint(OBJECT(sev_common),
+                                               "policy", NULL);
     }

I think we can change this 'policy' field to 'int64'.

Going from int32 to int64 doesn't change the encoding in JSON or cli properites. SEV/SEV-ES guests will still only use values that fit within int32, so existing users of QEMU won't notice a change.

Apps that want to use SEV-SNP will know that they can have policy values exceeding int32, but since that's net new code to suupport SEV-SNP there's no back compat issue.

target/i386/sev.c static SevCapability *sev_get_capabilities(Error **errp) · line 477

Quoted 6 lines
target/i386/sev.c +2 -0
@@ -519,6 +473,8 @@ static SevCapability *sev_get_capabilities(Error **errp)
     size_t pdh_len = 0, cert_chain_len = 0, cpu0_id_len = 0;
     uint32_t ebx;
     int fd;
+    SevCommonState *sev_common;
+    char *sev_device;

Declare 'g_autofree char *sev_device = NULL;'

target/i386/sev.c static SevCapability *sev_get_capabilities(Error **errp) · line 491

Quoted 12 lines
target/i386/sev.c +4 -1
     if (!kvm_enabled()) {
         error_setg(errp, "KVM not enabled");
@@ -529,12 +485,21 @@ static SevCapability *sev_get_capabilities(Error **errp)
         return NULL;
     }
 
-    fd = open(DEFAULT_SEV_DEVICE, O_RDWR);
+    sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+    if (!sev_common) {
+        error_setg(errp, "SEV is not configured");
+    }

Missing 'return' ?

target/i386/sev.c static SevCapability *sev_get_capabilities(Error **errp) · line 502

Quoted 11 lines
target/i386/sev.c +6 -0
+
+    sev_device = object_property_get_str(OBJECT(sev_common), "sev-device",
+                                         &error_abort);
+    fd = open(sev_device, O_RDWR);
     if (fd < 0) {
         error_setg_errno(errp, errno, "SEV: Failed to open %s",
                          DEFAULT_SEV_DEVICE);
+        g_free(sev_device);
         return NULL;
     }
+    g_free(sev_device);

These 'g_free' are redundant with g_autofree usage on the declaration.

target/i386/sev.c static SevAttestationReport *sev_get_attestation_report(const char *mnonce, · line 545

Quoted 9 lines
target/i386/sev.c +1 -1
     if (sev_get_pdh_info(fd, &pdh_data, &pdh_len,
                          &cert_chain_data, &cert_chain_len, errp)) {
@@ -577,7 +542,7 @@ static SevAttestationReport *sev_get_attestation_report(const char *mnonce,
 {
     struct kvm_sev_attestation_report input = {};
     SevAttestationReport *report = NULL;
-    SevGuestState *sev = sev_guest;
+    SevCommonState *sev_common;

I think it would have been nicer to just keep the variable just called 'sev', except in the few cases where you needed to have variables for both parent & subclass in the same method. This diff would be much smaller too.

That's a bit bikeshedding though, so not too bothered either way.

Quoted 29 lines
target/i386/sev.c +8 -2
     g_autofree guchar *data = NULL;
     g_autofree guchar *buf = NULL;
     gsize len;
@@ -602,8 +567,10 @@ static SevAttestationReport *sev_get_attestation_report(const char *mnonce,
         return NULL;
     }
 
+    sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+
     /* Query the report length */
-    ret = sev_ioctl(sev->sev_fd, KVM_SEV_GET_ATTESTATION_REPORT,
+    ret = sev_ioctl(sev_common->sev_fd, KVM_SEV_GET_ATTESTATION_REPORT,
             &input, &err);
     if (ret < 0) {
         if (err != SEV_RET_INVALID_LEN) {
@@ -619,7 +586,7 @@ static SevAttestationReport *sev_get_attestation_report(const char *mnonce,
     memcpy(input.mnonce, buf, sizeof(input.mnonce));
 
     /* Query the report */
-    ret = sev_ioctl(sev->sev_fd, KVM_SEV_GET_ATTESTATION_REPORT,
+    ret = sev_ioctl(sev_common->sev_fd, KVM_SEV_GET_ATTESTATION_REPORT,
             &input, &err);
     if (ret) {
         error_setg_errno(errp, errno, "SEV: Failed to get attestation report"
 
+
+/* sev guest info common to sev/sev-es/sev-snp */
+static const TypeInfo sev_common_info = {
+    .parent = TYPE_X86_CONFIDENTIAL_GUEST,

target/i386/sev.c static SevAttestationReport *sev_get_attestation_report(const char *mnonce, · line 1393

Quoted 10 lines
target/i386/sev.c +10 -0
+    .name = TYPE_SEV_COMMON,
+    .instance_size = sizeof(SevCommonState),
+    .class_init = sev_common_class_init,
+    .instance_init = sev_common_instance_init,
+    .abstract = true,
+    .interfaces = (InterfaceInfo[]) {
+        { TYPE_USER_CREATABLE },
+        { }
+    }
+};

It feels wierd to declare a type as "abstract", and at the same time declare it "user creatable". I know this was a simple short-cut to avoid repeating the .interfaces on every sub-class, but I still think it would be better to put the "user creatable" marker on the concrete impls instead.

Also how about using OBJECT_DEFINE_ABSTRACT_TYPE here and also converting the subclasses to use OBJECT_DEFINE_TYPE_WITH_INTERFACES ?

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#42 lore raw

Re: [PATCH v3 21/49] i386/sev: Introduce "sev-common" type to encapsulate common SEV state

Quoted 68 lines · Daniel P. Berrangé
Quoted 51 lines · Michael Roth

Currently all SEV/SEV-ES functionality is managed through a single 'sev-guest' QOM type. With upcoming support for SEV-SNP, taking this same approach won't work well since some of the properties/state managed by 'sev-guest' is not applicable to SEV-SNP, which will instead rely on a new QOM type with its own set of properties/state.

To prepare for this, this patch moves common state into an abstract 'sev-common' parent type to encapsulate properties/state that are common to both SEV/SEV-ES and SEV-SNP, leaving only SEV/SEV-ES-specific properties/state in the current 'sev-guest' type. This should not affect current behavior or command-line options.

As part of this patch, some related changes are also made:

- a static 'sev_guest' variable is currently used to keep track of the 'sev-guest' instance. SEV-SNP would similarly introduce an 'sev_snp_guest' static variable. But these instances are now available via qdev_get_machine()->cgs, so switch to using that instead and drop the static variable.

- 'sev_guest' is currently used as the name for the static variable holding a pointer to the 'sev-guest' instance. Re-purpose the name as a local variable referring the 'sev-guest' instance, and use that consistently throughout the code so it can be easily distinguished from sev-common/sev-snp-guest instances.

- 'sev' is generally used as the name for local variables holding a pointer to the 'sev-guest' instance. In cases where that now points to common state, use the name 'sev_common'; in cases where that now points to state specific to 'sev-guest' instance, use the name 'sev_guest'

---
 qapi/qom.json     |  32 ++--
 target/i386/sev.c | 457 ++++++++++++++++++++++++++--------------------
 target/i386/sev.h |   3 +
 3 files changed, 281 insertions(+), 211 deletions(-)
qapi/qom.json +1 -2
index baae3a183f..66b5781ca6 100644
@@ -875,12 +875,29 @@
   'data': { '*filename': 'str' } }
 
 ##
-# @SevGuestProperties:
+# @SevCommonProperties:
 #
-# Properties for sev-guest objects.

qapi/qom.json line 889

Quoted 10 lines
qapi/qom.json +7 -0
+# Properties common to objects that are derivatives of sev-common.
 #
 # @sev-device: SEV device to use (default: "/dev/sev")
 #
+# @cbitpos: C-bit location in page table entry (default: 0)
+#
+# @reduced-phys-bits: number of bits in physical addresses that become
+#     unavailable when SEV is enabled
+#
+# Since: 2.12

Not quite sure what we've done in this scenario before. It feels wierd to use '2.12' for the new base type, even though in effect the properties all existed since 2.12 in the sub-class.

Quoted 5 lines

Perhaps 'Since: 9.1' for the type, but 'Since: 2.12' for the properties, along with an explanatory comment about stuff moving into the new base type ?

Markus, opinions ?

My thinking is that the internal details are less important than what's actually exposed to users in the form of command-line options/etc. So in that context the "Since: 2.12" sort of becomes the "default" for when those properties were first made available to users, and then anything we add after would then get special treatment with the per-property versioning. But no issue with taking a different approach if that's preferred.

Quoted 298 lines
Quoted 285 lines
qapi/qom.json +13 -9
+##
+{ 'struct': 'SevCommonProperties',
+  'data': { '*sev-device': 'str',
+            '*cbitpos': 'uint32',
+            'reduced-phys-bits': 'uint32' } }
+
+##
+# @SevGuestProperties:
+#
+# Properties for sev-guest objects.
+#
 # @dh-cert-file: guest owners DH certificate (encoded with base64)
 #
 # @session-file: guest owners session parameters (encoded with base64)
@@ -889,11 +906,6 @@
 #
 # @handle: SEV firmware handle (default: 0)
 #
-# @cbitpos: C-bit location in page table entry (default: 0)
-#
-# @reduced-phys-bits: number of bits in physical addresses that become
-#     unavailable when SEV is enabled
-#
 # @kernel-hashes: if true, add hashes of kernel/initrd/cmdline to a
 #     designated guest firmware page for measured boot with -kernel
 #     (default: false) (since 6.2)
@@ -901,13 +913,11 @@
 # Since: 2.12
 ##
 { 'struct': 'SevGuestProperties',
-  'data': { '*sev-device': 'str',
-            '*dh-cert-file': 'str',
+  'base': 'SevCommonProperties',
+  'data': { '*dh-cert-file': 'str',
             '*session-file': 'str',
             '*policy': 'uint32',
             '*handle': 'uint32',
-            '*cbitpos': 'uint32',
-            'reduced-phys-bits': 'uint32',
             '*kernel-hashes': 'bool' } }
 
 ##
target/i386/sev.c +49 -104
index 9dab4060b8..63a220de5e 100644
@@ -40,48 +40,53 @@
 #include "hw/i386/pc.h"
 #include "exec/address-spaces.h"
 
-#define TYPE_SEV_GUEST "sev-guest"
+OBJECT_DECLARE_SIMPLE_TYPE(SevCommonState, SEV_COMMON)
 OBJECT_DECLARE_SIMPLE_TYPE(SevGuestState, SEV_GUEST)
 
-
-/**
- * SevGuestState:
- *
- * The SevGuestState object is used for creating and managing a SEV
- * guest.
- *
- * # $QEMU \
- *         -object sev-guest,id=sev0 \
- *         -machine ...,memory-encryption=sev0
- */
-struct SevGuestState {
+struct SevCommonState {
     X86ConfidentialGuest parent_obj;
 
     int kvm_type;
 
     /* configuration parameters */
     char *sev_device;
-    uint32_t policy;
-    char *dh_cert_file;
-    char *session_file;
     uint32_t cbitpos;
     uint32_t reduced_phys_bits;
-    bool kernel_hashes;
 
     /* runtime state */
-    uint32_t handle;
     uint8_t api_major;
     uint8_t api_minor;
     uint8_t build_id;
     int sev_fd;
     SevState state;
-    gchar *measurement;
 
     uint32_t reset_cs;
     uint32_t reset_ip;
     bool reset_data_valid;
 };
 
+/**
+ * SevGuestState:
+ *
+ * The SevGuestState object is used for creating and managing a SEV
+ * guest.
+ *
+ * # $QEMU \
+ *         -object sev-guest,id=sev0 \
+ *         -machine ...,memory-encryption=sev0
+ */
+struct SevGuestState {
+    SevCommonState sev_common;
+    gchar *measurement;
+
+    /* configuration parameters */
+    uint32_t handle;
+    uint32_t policy;
+    char *dh_cert_file;
+    char *session_file;
+    bool kernel_hashes;
+};
+
 #define DEFAULT_GUEST_POLICY    0x1 /* disable debug */
 #define DEFAULT_SEV_DEVICE      "/dev/sev"
 
@@ -127,7 +132,6 @@ typedef struct QEMU_PACKED PaddedSevHashTable {
 
 QEMU_BUILD_BUG_ON(sizeof(PaddedSevHashTable) % 16 != 0);
 
-static SevGuestState *sev_guest;
 static Error *sev_mig_blocker;
 
 static const char *const sev_fw_errlist[] = {
@@ -208,21 +212,21 @@ fw_error_to_str(int code)
 }
 
 static bool
-sev_check_state(const SevGuestState *sev, SevState state)
+sev_check_state(const SevCommonState *sev_common, SevState state)
 {
-    assert(sev);
-    return sev->state == state ? true : false;
+    assert(sev_common);
+    return sev_common->state == state ? true : false;
 }
 
 static void
-sev_set_guest_state(SevGuestState *sev, SevState new_state)
+sev_set_guest_state(SevCommonState *sev_common, SevState new_state)
 {
     assert(new_state < SEV_STATE__MAX);
-    assert(sev);
+    assert(sev_common);
 
-    trace_kvm_sev_change_state(SevState_str(sev->state),
+    trace_kvm_sev_change_state(SevState_str(sev_common->state),
                                SevState_str(new_state));
-    sev->state = new_state;
+    sev_common->state = new_state;
 }
 
 static void
@@ -289,111 +293,61 @@ static struct RAMBlockNotifier sev_ram_notifier = {
     .ram_block_removed = sev_ram_block_removed,
 };
 
-static void
-sev_guest_finalize(Object *obj)
-{
-}
-
-static char *
-sev_guest_get_session_file(Object *obj, Error **errp)
-{
-    SevGuestState *s = SEV_GUEST(obj);
-
-    return s->session_file ? g_strdup(s->session_file) : NULL;
-}
-
-static void
-sev_guest_set_session_file(Object *obj, const char *value, Error **errp)
-{
-    SevGuestState *s = SEV_GUEST(obj);
-
-    s->session_file = g_strdup(value);
-}
-
-static char *
-sev_guest_get_dh_cert_file(Object *obj, Error **errp)
-{
-    SevGuestState *s = SEV_GUEST(obj);
-
-    return g_strdup(s->dh_cert_file);
-}
-
-static void
-sev_guest_set_dh_cert_file(Object *obj, const char *value, Error **errp)
-{
-    SevGuestState *s = SEV_GUEST(obj);
-
-    s->dh_cert_file = g_strdup(value);
-}
-
-static char *
-sev_guest_get_sev_device(Object *obj, Error **errp)
-{
-    SevGuestState *sev = SEV_GUEST(obj);
-
-    return g_strdup(sev->sev_device);
-}
-
-static void
-sev_guest_set_sev_device(Object *obj, const char *value, Error **errp)
-{
-    SevGuestState *sev = SEV_GUEST(obj);
-
-    sev->sev_device = g_strdup(value);
-}
-
-static bool sev_guest_get_kernel_hashes(Object *obj, Error **errp)
-{
-    SevGuestState *sev = SEV_GUEST(obj);
-
-    return sev->kernel_hashes;
-}
-
-static void sev_guest_set_kernel_hashes(Object *obj, bool value, Error **errp)
-{
-    SevGuestState *sev = SEV_GUEST(obj);
-
-    sev->kernel_hashes = value;
-}
-
 bool
 sev_enabled(void)
 {
-    return !!sev_guest;
+    ConfidentialGuestSupport *cgs = MACHINE(qdev_get_machine())->cgs;
+
+    return !!object_dynamic_cast(OBJECT(cgs), TYPE_SEV_COMMON);
 }
 
 bool
 sev_es_enabled(void)
 {
-    return sev_enabled() && (sev_guest->policy & SEV_POLICY_ES);
+    ConfidentialGuestSupport *cgs = MACHINE(qdev_get_machine())->cgs;
+
+    return sev_enabled() && (SEV_GUEST(cgs)->policy & SEV_POLICY_ES);
 }
 
 uint32_t
 sev_get_cbit_position(void)
 {
-    return sev_guest ? sev_guest->cbitpos : 0;
+    SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+
+    return sev_common ? sev_common->cbitpos : 0;
 }
 
 uint32_t
 sev_get_reduced_phys_bits(void)
 {
-    return sev_guest ? sev_guest->reduced_phys_bits : 0;
+    SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+
+    return sev_common ? sev_common->reduced_phys_bits : 0;
 }
 
 static SevInfo *sev_get_info(void)
 {
     SevInfo *info;
+    SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+    SevGuestState *sev_guest =
+        (SevGuestState *)object_dynamic_cast(OBJECT(sev_common),
+                                             TYPE_SEV_GUEST);
 
     info = g_new0(SevInfo, 1);
     info->enabled = sev_enabled();
 
     if (info->enabled) {
-        info->api_major = sev_guest->api_major;
-        info->api_minor = sev_guest->api_minor;
-        info->build_id = sev_guest->build_id;
-        info->policy = sev_guest->policy;
-        info->state = sev_guest->state;
-        info->handle = sev_guest->handle;
+        if (sev_guest) {
+            info->handle = sev_guest->handle;

target/i386/sev.c static struct RAMBlockNotifier sev_ram_notifier = { · line 351

Quoted 10 lines
target/i386/sev.c +9 -0
+        }
+        info->api_major = sev_common->api_major;
+        info->api_minor = sev_common->api_minor;
+        info->build_id = sev_common->build_id;
+        info->state = sev_common->state;
+        /* we only report the lower 32-bits of policy for SNP, ok for now... */
+        info->policy =
+            (uint32_t)object_property_get_uint(OBJECT(sev_common),
+                                               "policy", NULL);
     }

I think we can change this 'policy' field to 'int64'.

Quoted 8 lines

Going from int32 to int64 doesn't change the encoding in JSON or cli properites. SEV/SEV-ES guests will still only use values that fit within int32, so existing users of QEMU won't notice a change.

Apps that want to use SEV-SNP will know that they can have policy values exceeding int32, but since that's net new code to suupport SEV-SNP there's no back compat issue.

In subsequent patch:

"i386/sev: Update query-sev QAPI format to handle SEV-SNP"

we end up reporting SNP policy via a new 64-bit field, 'snp_policy', based on a discussion:

https://lore.kernel.org/kvm/YTdSlg5NymDQex5T@work-vm/T/#mac6758af9bfc41ee49ff3ef5c3ec3779ec275ff9

I think the concern was some 'old_mgmt' tool trying to interact with an SNP guest launched through other means and misinterpreting SNP-specific policy bits as SEV ones.

So because if that, there isn't really a need to make the policy bit 64-bit as part of this patch.

One thing that does need to be addressed here in the confusing comment:

/* we only report the lower 32-bits of policy for SNP, ok for now... */

which I think was a relic from v2 that can be dropped now.

Quoted 55 lines

target/i386/sev.c static SevCapability *sev_get_capabilities(Error **errp) · line 477

Quoted 8 lines
target/i386/sev.c +2 -0
@@ -519,6 +473,8 @@ static SevCapability *sev_get_capabilities(Error **errp)
     size_t pdh_len = 0, cert_chain_len = 0, cpu0_id_len = 0;
     uint32_t ebx;
     int fd;
+    SevCommonState *sev_common;
+    char *sev_device;

Declare 'g_autofree char *sev_device = NULL;'

target/i386/sev.c static SevCapability *sev_get_capabilities(Error **errp) · line 491

Quoted 12 lines
target/i386/sev.c +4 -1
     if (!kvm_enabled()) {
         error_setg(errp, "KVM not enabled");
@@ -529,12 +485,21 @@ static SevCapability *sev_get_capabilities(Error **errp)
         return NULL;
     }
 
-    fd = open(DEFAULT_SEV_DEVICE, O_RDWR);
+    sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+    if (!sev_common) {
+        error_setg(errp, "SEV is not configured");
+    }

Missing 'return' ?

target/i386/sev.c static SevCapability *sev_get_capabilities(Error **errp) · line 502

Quoted 11 lines
target/i386/sev.c +6 -0
+
+    sev_device = object_property_get_str(OBJECT(sev_common), "sev-device",
+                                         &error_abort);
+    fd = open(sev_device, O_RDWR);
     if (fd < 0) {
         error_setg_errno(errp, errno, "SEV: Failed to open %s",
                          DEFAULT_SEV_DEVICE);
+        g_free(sev_device);
         return NULL;
     }
+    g_free(sev_device);

These 'g_free' are redundant with g_autofree usage on the declaration.

target/i386/sev.c static SevAttestationReport *sev_get_attestation_report(const char *mnonce, · line 545

Quoted 9 lines
target/i386/sev.c +1 -1
     if (sev_get_pdh_info(fd, &pdh_data, &pdh_len,
                          &cert_chain_data, &cert_chain_len, errp)) {
@@ -577,7 +542,7 @@ static SevAttestationReport *sev_get_attestation_report(const char *mnonce,
 {
     struct kvm_sev_attestation_report input = {};
     SevAttestationReport *report = NULL;
-    SevGuestState *sev = sev_guest;
+    SevCommonState *sev_common;

I think it would have been nicer to just keep the variable just called 'sev', except in the few cases where you needed to have variables for both parent & subclass in the same method. This diff would be much smaller too.

Quoted 2 lines

That's a bit bikeshedding though, so not too bothered either way.

Yah, I think at the time I'd considered that, but always anchoring the variable name to the underlying type makes it easy to always know what type of instance you're working with which I'm hoping is worth the initial investment of doing the conversion consistently.

Quoted 48 lines
Quoted 30 lines
target/i386/sev.c +8 -2
     g_autofree guchar *data = NULL;
     g_autofree guchar *buf = NULL;
     gsize len;
@@ -602,8 +567,10 @@ static SevAttestationReport *sev_get_attestation_report(const char *mnonce,
         return NULL;
     }
 
+    sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+
     /* Query the report length */
-    ret = sev_ioctl(sev->sev_fd, KVM_SEV_GET_ATTESTATION_REPORT,
+    ret = sev_ioctl(sev_common->sev_fd, KVM_SEV_GET_ATTESTATION_REPORT,
             &input, &err);
     if (ret < 0) {
         if (err != SEV_RET_INVALID_LEN) {
@@ -619,7 +586,7 @@ static SevAttestationReport *sev_get_attestation_report(const char *mnonce,
     memcpy(input.mnonce, buf, sizeof(input.mnonce));
 
     /* Query the report */
-    ret = sev_ioctl(sev->sev_fd, KVM_SEV_GET_ATTESTATION_REPORT,
+    ret = sev_ioctl(sev_common->sev_fd, KVM_SEV_GET_ATTESTATION_REPORT,
             &input, &err);
     if (ret) {
         error_setg_errno(errp, errno, "SEV: Failed to get attestation report"
 
+
+/* sev guest info common to sev/sev-es/sev-snp */
+static const TypeInfo sev_common_info = {
+    .parent = TYPE_X86_CONFIDENTIAL_GUEST,

target/i386/sev.c static SevAttestationReport *sev_get_attestation_report(const char *mnonce, · line 1393

Quoted 10 lines
target/i386/sev.c +10 -0
+    .name = TYPE_SEV_COMMON,
+    .instance_size = sizeof(SevCommonState),
+    .class_init = sev_common_class_init,
+    .instance_init = sev_common_instance_init,
+    .abstract = true,
+    .interfaces = (InterfaceInfo[]) {
+        { TYPE_USER_CREATABLE },
+        { }
+    }
+};

It feels wierd to declare a type as "abstract", and at the same time declare it "user creatable". I know this was a simple short-cut to avoid repeating the .interfaces on every sub-class, but I still think it would be better to put the "user creatable" marker on the concrete impls instead.

Quoted 3 lines

Also how about using OBJECT_DEFINE_ABSTRACT_TYPE here and also converting the subclasses to use OBJECT_DEFINE_TYPE_WITH_INTERFACES ?

Makes sense, will implement these for v4.

-Mike

Quoted 10 lines

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#43 lore raw

Re: [PATCH v3 21/49] i386/sev: Introduce "sev-common" type to encapsulate common SEV state

In reply to Daniel P. Berrangé
Quoted 68 lines · Daniel P. Berrangé
Quoted 51 lines · Michael Roth

Currently all SEV/SEV-ES functionality is managed through a single 'sev-guest' QOM type. With upcoming support for SEV-SNP, taking this same approach won't work well since some of the properties/state managed by 'sev-guest' is not applicable to SEV-SNP, which will instead rely on a new QOM type with its own set of properties/state.

To prepare for this, this patch moves common state into an abstract 'sev-common' parent type to encapsulate properties/state that are common to both SEV/SEV-ES and SEV-SNP, leaving only SEV/SEV-ES-specific properties/state in the current 'sev-guest' type. This should not affect current behavior or command-line options.

As part of this patch, some related changes are also made:

- a static 'sev_guest' variable is currently used to keep track of the 'sev-guest' instance. SEV-SNP would similarly introduce an 'sev_snp_guest' static variable. But these instances are now available via qdev_get_machine()->cgs, so switch to using that instead and drop the static variable.

- 'sev_guest' is currently used as the name for the static variable holding a pointer to the 'sev-guest' instance. Re-purpose the name as a local variable referring the 'sev-guest' instance, and use that consistently throughout the code so it can be easily distinguished from sev-common/sev-snp-guest instances.

- 'sev' is generally used as the name for local variables holding a pointer to the 'sev-guest' instance. In cases where that now points to common state, use the name 'sev_common'; in cases where that now points to state specific to 'sev-guest' instance, use the name 'sev_guest'

---
 qapi/qom.json     |  32 ++--
 target/i386/sev.c | 457 ++++++++++++++++++++++++++--------------------
 target/i386/sev.h |   3 +
 3 files changed, 281 insertions(+), 211 deletions(-)
qapi/qom.json +1 -2
index baae3a183f..66b5781ca6 100644
@@ -875,12 +875,29 @@
   'data': { '*filename': 'str' } }
 
 ##
-# @SevGuestProperties:
+# @SevCommonProperties:
 #
-# Properties for sev-guest objects.

qapi/qom.json line 889

Quoted 10 lines
qapi/qom.json +7 -0
+# Properties common to objects that are derivatives of sev-common.
 #
 # @sev-device: SEV device to use (default: "/dev/sev")
 #
+# @cbitpos: C-bit location in page table entry (default: 0)
+#
+# @reduced-phys-bits: number of bits in physical addresses that become
+#     unavailable when SEV is enabled
+#
+# Since: 2.12

Not quite sure what we've done in this scenario before. It feels wierd to use '2.12' for the new base type, even though in effect the properties all existed since 2.12 in the sub-class.

Quoted 5 lines

Perhaps 'Since: 9.1' for the type, but 'Since: 2.12' for the properties, along with an explanatory comment about stuff moving into the new base type ?

Markus, opinions ?

The confusion is due to us documenting the schema instead of the external interface defined by it. Let me explain.

The external interface is commands and their arguments, ignoring results, errors and events for brevity's sake.

We use types to define the arguments. How exactly we use types is not part of the interface. This permits refactorings. However, since the documentation is attached to the types, refactorings can easily mess it up.

I'd like to demonstrate this for a simpler command first, then return to object-add.

Consider nbd-server-add. It is documented to be since 1.3.

From now on, I'm abbreviating "documented to be since X.Y" to "since X.Y".

Its arguments are the members of struct NbdServerAddOptions.

NbdServerAddOptions is since 5.0. Its base BlockExportOptionsNbdBase is since 5.2.

BlockExportOptionsNbdBase member @name is since 2.12, and @description is since 5.0.

NbdServerAddOptions member @bitmap is since 4.0. Members @device and @writable have no "since" documented, so they inherit it from the struct, i.e. 5.0.

So, it looks like the command is since 1.3, argument @name since 2.12, @bitmap since 4.0, @description, @device, and @writable since 5.0.

Wrong! Arguments @device and @writable have always been there, i.e. since 1.3. We ended up with documentation incorrectly claiming 5.0 via several refactorings.

Initially, the command arguments were defined right with the command. They simply inherited the command's since 1.3.

Commit c62d24e906e (blockdev-nbd: Boxed argument type for nbd-server-add) moved them to a struct type BlockExportNbd. The new struct type was since 5.0. Newer arguments retained their "since" tags, but the initial arguments @device and @writable remained without one. Their documented "since" changed from 1.3 to 5.0.

Aside: the new struct was then used as a branch of union BlockExport, which was incorrectly documented to be since 4.2.

Messing up "since" when factoring out arguments into a new type was
avoidable: either lie and claim the new type is as old as its members, or add suitable since tags to its members.

Having a struct with members older than itself looks weird. Of course, when a struct came to be is *immaterial*. How exactly we assemble the arguments from types is not part of the interface. We could omit "since" for the struct, and then require it for all members. We don't, because having to think (and then argue!) whether we want a "since" or not would be a waste of mental capacity.

Here's another refactoring where that may not be possible. Say you discover two structs share several members. You decide to factor them out into a common base type. Won't affect the external interface. But what if one of these common members has conflicting "since"? Either we refrain from the refactoring, or we resort to something like "since X1.Y1 when used for USER1, since X1.Y2 when used for USER2". Which *sucks* as external interface documentation.

Aside: documentation text could clash similarly. Same code, different meaning.

I've come to the conclusion that manually recording "since" in the documentation is dumb. One, because we mess it up. Two, because not messing it up involves either lies or oddities, or too much thought. Three, because keeping it correct can interfere with refactorings.

Some time ago, Michael Tsirkin suggested to generate "since" information automatically. I like the idea. We'd have to record the external interface at release time. To fully replace existing "since" tags, we'd have to record for old versions, too. I'd expect this to fix quite a few doc bugs.

I hope "The confusion is due to us documenting the schema instead of the external interface defined by it" is now more clear. The external interface is derived from the types. How exactly we construct it from types is invisible at the interface. But since we document the interface by documenting the types, the structure of our interface documentation mirrors our use of types. We succeed at shielding the interface from how we use types, but we fail to shield the documentation.

Back to your problem at hand. The external interface is command object-add. The command is since 2.0.

It has common arguments and variant arguments depending on the value of common argument @type. We specify all that via union ObjectOptions, and the variant arguments for @type value "sev-guest" via struct SevGuestProperties.

Union ObjectOptions is since 6.0, but that's immaterial; the type isn't part of the external interface, only its members are.

Its members are the common arguments. Since they don't have their own "since" tag, they inherit it from ObjectOptions, i.e. since 6.0. That's simply wrong; they exist since 2.0 just like object-add.

Struct SevGuestProperties is since 2.12, but that's also immaterial.

The members of SevGuestProperties are the variant arguments for @type value "sev-guest". Since they don't have their own "since" tag, they inherit it from SevGuestProperties, i.e. since 2.12.

Your patch moves some of the members to new base type SevCommonProperties. As Daniel observed, you can either claim SevCommonProperties is since 2.12 (which is a lie), or you claim 9.1 for the type and 2.12 for all its members (which is odd).

I prefer oddities to lies.

I'm not sure we need a comment explaining the oddity. If you think it's useful, please make it a non-doc comment. Reminder:

##
# This is a doc comment. It goes into generated documentation. ##

# This is is not a doc comment. It does not go into generated # documentation.

Comments?

[...]

#44 lore raw

Re: [PATCH v3 21/49] i386/sev: Introduce "sev-common" type to encapsulate common SEV state

In reply to Michael Roth
Quoted 51 lines · Michael Roth

Currently all SEV/SEV-ES functionality is managed through a single 'sev-guest' QOM type. With upcoming support for SEV-SNP, taking this same approach won't work well since some of the properties/state managed by 'sev-guest' is not applicable to SEV-SNP, which will instead rely on a new QOM type with its own set of properties/state.

To prepare for this, this patch moves common state into an abstract 'sev-common' parent type to encapsulate properties/state that are common to both SEV/SEV-ES and SEV-SNP, leaving only SEV/SEV-ES-specific properties/state in the current 'sev-guest' type. This should not affect current behavior or command-line options.

As part of this patch, some related changes are also made:

- a static 'sev_guest' variable is currently used to keep track of the 'sev-guest' instance. SEV-SNP would similarly introduce an 'sev_snp_guest' static variable. But these instances are now available via qdev_get_machine()->cgs, so switch to using that instead and drop the static variable.

- 'sev_guest' is currently used as the name for the static variable holding a pointer to the 'sev-guest' instance. Re-purpose the name as a local variable referring the 'sev-guest' instance, and use that consistently throughout the code so it can be easily distinguished from sev-common/sev-snp-guest instances.

- 'sev' is generally used as the name for local variables holding a pointer to the 'sev-guest' instance. In cases where that now points to common state, use the name 'sev_common'; in cases where that now points to state specific to 'sev-guest' instance, use the name 'sev_guest'

---
 qapi/qom.json     |  32 ++--
 target/i386/sev.c | 457 ++++++++++++++++++++++++++--------------------
 target/i386/sev.h |   3 +
 3 files changed, 281 insertions(+), 211 deletions(-)

static SevInfo *sev_get_info(void)

 {
     SevInfo *info;
target/i386/sev.c +4 -0
+    SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+    SevGuestState *sev_guest =
+        (SevGuestState *)object_dynamic_cast(OBJECT(sev_common),
+                                             TYPE_SEV_GUEST);
 
     info = g_new0(SevInfo, 1);
     info->enabled = sev_enabled();
 

target/i386/sev.c static struct RAMBlockNotifier sev_ram_notifier = { · line 342

Quoted 10 lines
target/i386/sev.c +3 -6
     if (info->enabled) {
-        info->api_major = sev_guest->api_major;
-        info->api_minor = sev_guest->api_minor;
-        info->build_id = sev_guest->build_id;
-        info->policy = sev_guest->policy;
-        info->state = sev_guest->state;
-        info->handle = sev_guest->handle;
+        if (sev_guest) {
+            info->handle = sev_guest->handle;
+        }

If we're not going to provide a value for 'handle', then we should update the QAPI for this to mark the property as optional, which would then require doing

  info->has_handle = true;

inside this 'if' block.

target/i386/sev.c static struct RAMBlockNotifier sev_ram_notifier = { · line 351

Quoted 9 lines
target/i386/sev.c +8 -0
+        info->api_major = sev_common->api_major;
+        info->api_minor = sev_common->api_minor;
+        info->build_id = sev_common->build_id;
+        info->state = sev_common->state;
+        /* we only report the lower 32-bits of policy for SNP, ok for now... */
+        info->policy =
+            (uint32_t)object_property_get_uint(OBJECT(sev_common),
+                                               "policy", NULL);
     }

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#45 lore raw

Re: [PATCH v3 21/49] i386/sev: Introduce "sev-common" type to encapsulate common SEV state

Quoted 63 lines · Daniel P. Berrangé
Quoted 51 lines · Michael Roth

Currently all SEV/SEV-ES functionality is managed through a single 'sev-guest' QOM type. With upcoming support for SEV-SNP, taking this same approach won't work well since some of the properties/state managed by 'sev-guest' is not applicable to SEV-SNP, which will instead rely on a new QOM type with its own set of properties/state.

To prepare for this, this patch moves common state into an abstract 'sev-common' parent type to encapsulate properties/state that are common to both SEV/SEV-ES and SEV-SNP, leaving only SEV/SEV-ES-specific properties/state in the current 'sev-guest' type. This should not affect current behavior or command-line options.

As part of this patch, some related changes are also made:

- a static 'sev_guest' variable is currently used to keep track of the 'sev-guest' instance. SEV-SNP would similarly introduce an 'sev_snp_guest' static variable. But these instances are now available via qdev_get_machine()->cgs, so switch to using that instead and drop the static variable.

- 'sev_guest' is currently used as the name for the static variable holding a pointer to the 'sev-guest' instance. Re-purpose the name as a local variable referring the 'sev-guest' instance, and use that consistently throughout the code so it can be easily distinguished from sev-common/sev-snp-guest instances.

- 'sev' is generally used as the name for local variables holding a pointer to the 'sev-guest' instance. In cases where that now points to common state, use the name 'sev_common'; in cases where that now points to state specific to 'sev-guest' instance, use the name 'sev_guest'

---
 qapi/qom.json     |  32 ++--
 target/i386/sev.c | 457 ++++++++++++++++++++++++++--------------------
 target/i386/sev.h |   3 +
 3 files changed, 281 insertions(+), 211 deletions(-)

static SevInfo *sev_get_info(void)

 {
     SevInfo *info;
target/i386/sev.c +4 -0
+    SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+    SevGuestState *sev_guest =
+        (SevGuestState *)object_dynamic_cast(OBJECT(sev_common),
+                                             TYPE_SEV_GUEST);
 
     info = g_new0(SevInfo, 1);
     info->enabled = sev_enabled();
 

target/i386/sev.c static struct RAMBlockNotifier sev_ram_notifier = { · line 342

Quoted 10 lines
target/i386/sev.c +3 -6
     if (info->enabled) {
-        info->api_major = sev_guest->api_major;
-        info->api_minor = sev_guest->api_minor;
-        info->build_id = sev_guest->build_id;
-        info->policy = sev_guest->policy;
-        info->state = sev_guest->state;
-        info->handle = sev_guest->handle;
+        if (sev_guest) {
+            info->handle = sev_guest->handle;
+        }
Quoted 7 lines

If we're not going to provide a value for 'handle', then we should update the QAPI for this to mark the property as optional, which would then require doing

  info->has_handle = true;

inside this 'if' block.

I think this is another temporarily-awkward case that gets resolved with:

i386/sev: Update query-sev QAPI format to handle SEV-SNP

With that patch 'handle' is always available for SEV guests, and never available for SNP, and that's managed through a discriminated union type. I think that info->handle should be treated the same as the other fields as part of this patch and any changes in how they are reported should be kept in the above-mentioned patch.

This might be another artifact from v2's handling. Will get this fixed up.

-Mike

Quoted 20 lines

target/i386/sev.c static struct RAMBlockNotifier sev_ram_notifier = { · line 351

Quoted 12 lines
target/i386/sev.c +9 -0
+        }
 
 
+        info->api_major = sev_common->api_major;
+        info->api_minor = sev_common->api_minor;
+        info->build_id = sev_common->build_id;
+        info->state = sev_common->state;
+        /* we only report the lower 32-bits of policy for SNP, ok for now... */
+        info->policy =
+            (uint32_t)object_property_get_uint(OBJECT(sev_common),
+                                               "policy", NULL);
     }

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Acked-by on v3 21/49
#46 lore raw

Re: [PATCH v3 21/49] i386/sev: Introduce "sev-common" type to encapsulate common SEV state

In reply to Michael Roth
Quoted 11 lines · Michael Roth

Currently all SEV/SEV-ES functionality is managed through a single 'sev-guest' QOM type. With upcoming support for SEV-SNP, taking this same approach won't work well since some of the properties/state managed by 'sev-guest' is not applicable to SEV-SNP, which will instead rely on a new QOM type with its own set of properties/state.

To prepare for this, this patch moves common state into an abstract 'sev-common' parent type to encapsulate properties/state that are common to both SEV/SEV-ES and SEV-SNP, leaving only SEV/SEV-ES-specific properties/state in the current 'sev-guest' type. This should not affect current behavior or command-line options.

QAPI schema refactoring except for the misleading "since" documentation pointed out by Daniel

[...]

#47 lore raw

PATCH v3 22/49 i386/sev: Introduce 'sev-snp-guest' object

In reply to Michael Roth

From: Brijesh Singh <[email protected]>

SEV-SNP support relies on a different set of properties/state than the existing 'sev-guest' object. This patch introduces the 'sev-snp-guest' object, which can be used to configure an SEV-SNP guest. For example, a default-configured SEV-SNP guest with no additional information passed in for use with attestation:

-object sev-snp-guest,id=sev0

or a fully-specified SEV-SNP guest where all spec-defined binary blobs are passed in as base64-encoded strings:

  -object sev-snp-guest,id=sev0, \
    policy=0x30000, \
    init-flags=0, \
    id-block=YWFhYWFhYWFhYWFhYWFhCg==, \
    id-auth=CxHK/OKLkXGn/KpAC7Wl1FSiisWDbGTEKz..., \
    auth-key-enabled=on, \
    host-data=LNkCWBRC5CcdGXirbNUV1OrsR28s..., \
    guest-visible-workarounds=AA==, \

See the QAPI schema updates included in this patch for more usage details.

In some cases these blobs may be up to 4096 characters, but this is generally well below the default limit for linux hosts where command-line sizes are defined by the sysconf-configurable ARG_MAX value, which defaults to 2097152 characters for Ubuntu hosts, for example.

---
 docs/system/i386/amd-memory-encryption.rst |  78 ++++++-
 qapi/qom.json                              |  51 +++++
 target/i386/sev.c                          | 241 +++++++++++++++++++++
 target/i386/sev.h                          |   1 +
 4 files changed, 369 insertions(+), 2 deletions(-)
docs/system/i386/amd-memory-encryption.rst +76 -2
index e9bc142bc1..9d6b63acd9 100644
@@ -25,8 +25,8 @@ support for notifying a guest's operating system when certain types of VMEXITs
 are about to occur. This allows the guest to selectively share information with
 the hypervisor to satisfy the requested function.
 
-Launching
----------
+Launching (SEV and SEV-ES)
+--------------------------
 
 Boot images (such as bios) must be encrypted before a guest can be booted. The
 ``MEMORY_ENCRYPT_OP`` ioctl provides commands to encrypt the images: ``LAUNCH_START``,
@@ -161,6 +161,80 @@ The value of GCTX.LD is
 If kernel hashes are not used, or SEV-ES is disabled, use empty blobs for
 ``kernel_hashes_blob`` and ``vmsas_blob`` as needed.
 
+Launching (SEV-SNP)
+-------------------
+Boot images (such as bios) must be encrypted before a guest can be booted. The
+``MEMORY_ENCRYPT_OP`` ioctl provides commands to encrypt the images:
+``KVM_SNP_INIT``, ``SNP_LAUNCH_START``, ``SNP_LAUNCH_UPDATE``, and
+``SNP_LAUNCH_FINISH``. These four commands together generate a fresh memory
+encryption key for the VM, encrypt the boot images for a successful launch.
+
+KVM_SNP_INIT is called first to initialize the SEV-SNP firmware and SNP
+features in the KVM. The feature flags value can be provided through the
+init-flags property of the sev-snp-guest object.
+
++------------+-------+----------+---------------------------------+
+| key        | type  | default  | meaning                         |
++------------+-------+----------+---------------------------------+
+| init_flags | hex   | 0        | SNP feature flags               |
++-----------------------------------------------------------------+
+
+Note: currently the init_flags must be zero.
+
+``SNP_LAUNCH_START`` is called first to create a cryptographic launch context
+within the firmware. To create this context, guest owner must provide a guest
+policy and other parameters as described in the SEV-SNP firmware
+specification. The launch parameters should be specified as described in the
+QAPI schema for the sev-snp-guest object.
+
+The ``SNP_LAUNCH_START`` uses the following parameters (see the SEV-SNP
+specification for more details):
+
++--------+-------+----------+----------------------------------------------+
+| key    | type  | default  | meaning                                      |
++--------+-------+----------+----------------------------------------------+
+| policy | hex   | 0x30000  | a 64-bit guest policy                        |
+| imi_en | bool  | 0        | 1 when IMI is enabled                        |
+| ma_end | bool  | 0        | 1 when migration agent is used               |
+| gosvw  | string| 0        | 16-byte base64 encoded string for the guest  |
+|        |       |          | OS visible workaround.                       |
++--------+-------+----------+----------------------------------------------+
+
+``SNP_LAUNCH_UPDATE`` encrypts the memory region using the cryptographic context
+created via the ``SNP_LAUNCH_START`` command. If required, this command can be
+called multiple times to encrypt different memory regions. The command also
+calculates the measurement of the memory contents as it encrypts.
+
+``SNP_LAUNCH_FINISH`` finalizes the guest launch flow. Optionally, while
+finalizing the launch the firmware can perform checks on the launch digest
+computing through the ``SNP_LAUNCH_UPDATE``. To perform the check the user must
+supply the id block, authentication blob and host data that should be included
+in the attestation report. See the SEV-SNP spec for further details.
+
+The ``SNP_LAUNCH_FINISH`` uses the following parameters, which can be configured
+by the corresponding parameters documented in the QAPI schema for the
+'sev-snp-guest' object.
+
++------------+-------+----------+----------------------------------------------+
+| key        | type  | default  | meaning                                      |
++------------+-------+----------+----------------------------------------------+
+| id_block   | string| none     | base64 encoded ID block                      |
++------------+-------+----------+----------------------------------------------+
+| id_auth    | string| none     | base64 encoded authentication information    |
++------------+-------+----------+----------------------------------------------+
+| auth_key_en| bool  | 0        | auth block contains author key               |
++------------+-------+----------+----------------------------------------------+
+| host_data  | string| none     | host provided data                           |
++------------+-------+----------+----------------------------------------------+
+
+To launch a SEV-SNP guest (additional parameters are documented in the QAPI
+schema for the 'sev-snp-guest' object)::
+
+  # ${QEMU} \
+    -machine ...,confidential-guest-support=sev0 \
+    -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1
+
+
 Debugging
 ---------
 
qapi/qom.json +51 -0

69 lines not on this page · raw message

target/i386/sev.c +241 -0

266 lines not on this page · raw message

target/i386/sev.h +1 -0
index 668374eef3..bedc667eeb 100644
@@ -22,6 +22,7 @@
 
 #define TYPE_SEV_COMMON "sev-common"
 #define TYPE_SEV_GUEST "sev-guest"
+#define TYPE_SEV_SNP_GUEST "sev-snp-guest"
 
 #define SEV_POLICY_NODBG        0x1
 #define SEV_POLICY_NOKS         0x2
Signature · 1 line
2.25.1
#48 lore raw

Re: [PATCH v3 22/49] i386/sev: Introduce 'sev-snp-guest' object

Quoted 71 lines · Michael Roth

From: Brijesh Singh <[email protected]>

SEV-SNP support relies on a different set of properties/state than the existing 'sev-guest' object. This patch introduces the 'sev-snp-guest' object, which can be used to configure an SEV-SNP guest. For example, a default-configured SEV-SNP guest with no additional information passed in for use with attestation:

-object sev-snp-guest,id=sev0

or a fully-specified SEV-SNP guest where all spec-defined binary blobs are passed in as base64-encoded strings:

  -object sev-snp-guest,id=sev0, \
    policy=0x30000, \
    init-flags=0, \
    id-block=YWFhYWFhYWFhYWFhYWFhCg==, \
    id-auth=CxHK/OKLkXGn/KpAC7Wl1FSiisWDbGTEKz..., \
    auth-key-enabled=on, \
    host-data=LNkCWBRC5CcdGXirbNUV1OrsR28s..., \
    guest-visible-workarounds=AA==, \

See the QAPI schema updates included in this patch for more usage details.

In some cases these blobs may be up to 4096 characters, but this is generally well below the default limit for linux hosts where command-line sizes are defined by the sysconf-configurable ARG_MAX value, which defaults to 2097152 characters for Ubuntu hosts, for example.

---
 docs/system/i386/amd-memory-encryption.rst |  78 ++++++-
 qapi/qom.json                              |  51 +++++
 target/i386/sev.c                          | 241 +++++++++++++++++++++
 target/i386/sev.h                          |   1 +
 4 files changed, 369 insertions(+), 2 deletions(-)
qapi/qom.json +28 -0
+##
+# @SevSnpGuestProperties:
+#
+# Properties for sev-snp-guest objects. Most of these are direct arguments
+# for the KVM_SNP_* interfaces documented in the linux kernel source
+# under Documentation/virt/kvm/amd-memory-encryption.rst, which are in
+# turn closely coupled with the SNP_INIT/SNP_LAUNCH_* firmware commands
+# documented in the SEV-SNP Firmware ABI Specification (Rev 0.9).
+#
+# More usage information is also available in the QEMU source tree under
+# docs/amd-memory-encryption.
+#
+# @policy: the 'POLICY' parameter to the SNP_LAUNCH_START command, as
+#          defined in the SEV-SNP firmware ABI (default: 0x30000)
+#
+# @guest-visible-workarounds: 16-byte, base64-encoded blob to report
+#                             hypervisor-defined workarounds, corresponding
+#                             to the 'GOSVW' parameter of the
+#                             SNP_LAUNCH_START command defined in the
+#                             SEV-SNP firmware ABI (default: all-zero)
+#
+# @id-block: 96-byte, base64-encoded blob to provide the 'ID Block'
+#            structure for the SNP_LAUNCH_FINISH command defined in the
+#            SEV-SNP firmware ABI (default: all-zero)
+#
+# @id-auth: 4096-byte, base64-encoded blob to provide the 'ID Authentication
+#           Information Structure' for the SNP_LAUNCH_FINISH command defined
+#           in the SEV-SNP firmware ABI (default: all-zero)

qapi/qom.json line 960

Quoted 10 lines
qapi/qom.json +10 -0
+#
+# @auth-key-enabled: true if 'id-auth' blob contains the 'AUTHOR_KEY' field
+#                    defined SEV-SNP firmware ABI (default: false)
+#
+# @host-data: 32-byte, base64-encoded, user-defined blob to provide to the
+#             guest, as documented for the 'HOST_DATA' parameter of the
+#             SNP_LAUNCH_FINISH command in the SEV-SNP firmware ABI
+#             (default: all-zero)
+#
+# Since: 7.2

This will be 9.1 at the earliest now.

Quoted 59 lines
qapi/qom.json +11 -0
+##
+{ 'struct': 'SevSnpGuestProperties',
+  'base': 'SevCommonProperties',
+  'data': {
+            '*policy': 'uint64',
+            '*guest-visible-workarounds': 'str',
+            '*id-block': 'str',
+            '*id-auth': 'str',
+            '*auth-key-enabled': 'bool',
+            '*host-data': 'str' } }
+
target/i386/sev.c +24 -0
index 63a220de5e..7e6dab642a 100644
@@ -42,6 +42,7 @@
 
 OBJECT_DECLARE_SIMPLE_TYPE(SevCommonState, SEV_COMMON)
 OBJECT_DECLARE_SIMPLE_TYPE(SevGuestState, SEV_GUEST)
+OBJECT_DECLARE_SIMPLE_TYPE(SevSnpGuestState, SEV_SNP_GUEST)
 
 struct SevCommonState {
     X86ConfidentialGuest parent_obj;
@@ -87,8 +88,22 @@ struct SevGuestState {
     bool kernel_hashes;
 };
 
+struct SevSnpGuestState {
+    SevCommonState sev_common;
+
+    /* configuration parameters */
+    char *guest_visible_workarounds;
+    char *id_block;
+    char *id_auth;
+    char *host_data;
+
+    struct kvm_sev_snp_launch_start kvm_start_conf;
+    struct kvm_sev_snp_launch_finish kvm_finish_conf;
+};
+
 #define DEFAULT_GUEST_POLICY    0x1 /* disable debug */
 #define DEFAULT_SEV_DEVICE      "/dev/sev"
+#define DEFAULT_SEV_SNP_POLICY  0x30000
 
 #define SEV_INFO_BLOCK_GUID     "00f771de-1a7e-4fcb-890e-68c77e2fb44e"
 typedef struct __attribute__((__packed__)) SevInfoBlock {
@@ -1473,11 +1488,237 @@ static const TypeInfo sev_guest_info = {
     .class_init = sev_guest_class_init,
 
+
+static char *
+sev_snp_guest_get_guest_visible_workarounds(Object *obj, Error **errp)
+{
+    return g_strdup(SEV_SNP_GUEST(obj)->guest_visible_workarounds);
+}
+
+static void
+sev_snp_guest_set_guest_visible_workarounds(Object *obj, const char *value,

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1508

Quoted 10 lines
target/i386/sev.c +10 -0
+                                            Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+    struct kvm_sev_snp_launch_start *start = &sev_snp_guest->kvm_start_conf;
+    g_autofree guchar *blob;
+    gsize len;
+
+    if (sev_snp_guest->guest_visible_workarounds) {
+        g_free(sev_snp_guest->guest_visible_workarounds);
+    }

Redundant 'if' test - g_free is happy with NULL

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1536

Quoted 10 lines
target/i386/sev.c +10 -0
+
+    /* store the base64 str so we don't need to re-encode in getter */
+    sev_snp_guest->guest_visible_workarounds = g_strdup(value);
+
+    blob = qbase64_decode(sev_snp_guest->guest_visible_workarounds, -1, &len, errp);
+    if (!blob) {
+        return;
+    }
+
+    if (len > sizeof(start->gosvw)) {

The QAPI docs said this property must be '16 bytes', so I'd suggest we do a strict equality test, rather than min size test to catch a wider set of mistakes.

Quoted 17 lines
target/i386/sev.c +17 -0
+        error_setg(errp, "parameter length of %lu exceeds max of %lu",
+                   len, sizeof(start->gosvw));
+        return;
+    }
+
+    memcpy(start->gosvw, blob, len);
+}
+
+static char *
+sev_snp_guest_get_id_block(Object *obj, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    return g_strdup(sev_snp_guest->id_block);
+}
+
+static void

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1563

Quoted 10 lines
target/i386/sev.c +10 -0
+sev_snp_guest_set_id_block(Object *obj, const char *value, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+    struct kvm_sev_snp_launch_finish *finish = &sev_snp_guest->kvm_finish_conf;
+    gsize len;
+
+    if (sev_snp_guest->id_block) {
+        g_free(sev_snp_guest->id_block);
+        g_free((guchar *)finish->id_block_uaddr);
+    }

Assuming 'id_block_uaddr' is also initialized to 0, when id_block is NULL, then you can remove the 'if' conditional.

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1575

Quoted 12 lines
target/i386/sev.c +12 -0
+
+    /* store the base64 str so we don't need to re-encode in getter */
+    sev_snp_guest->id_block = g_strdup(value);
+
+    finish->id_block_uaddr =
+        (uint64_t)qbase64_decode(sev_snp_guest->id_block, -1, &len, errp);
+
+    if (!finish->id_block_uaddr) {
+        return;
+    }
+
+    if (len > KVM_SEV_SNP_ID_BLOCK_SIZE) {

Again, lets do a strict equality test to match the documented required size.

Quoted 17 lines
target/i386/sev.c +17 -0
+        error_setg(errp, "parameter length of %lu exceeds max of %u",
+                   len, KVM_SEV_SNP_ID_BLOCK_SIZE);
+        return;
+    }
+
+    finish->id_block_en = (len) ? 1 : 0;
+}
+
+static char *
+sev_snp_guest_get_id_auth(Object *obj, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    return g_strdup(sev_snp_guest->id_auth);
+}
+
+static void

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1602

Quoted 10 lines
target/i386/sev.c +10 -0
+sev_snp_guest_set_id_auth(Object *obj, const char *value, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+    struct kvm_sev_snp_launch_finish *finish = &sev_snp_guest->kvm_finish_conf;
+    gsize len;
+
+    if (sev_snp_guest->id_auth) {
+        g_free(sev_snp_guest->id_auth);
+        g_free((guchar *)finish->id_auth_uaddr);
+    }

Same probably redundant 'if'

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1614

Quoted 12 lines
target/i386/sev.c +12 -0
+
+    /* store the base64 str so we don't need to re-encode in getter */
+    sev_snp_guest->id_auth = g_strdup(value);
+
+    finish->id_auth_uaddr =
+        (uint64_t)qbase64_decode(sev_snp_guest->id_auth, -1, &len, errp);
+
+    if (!finish->id_auth_uaddr) {
+        return;
+    }
+
+    if (len > KVM_SEV_SNP_ID_AUTH_SIZE) {

Equality test.

Quoted 31 lines
target/i386/sev.c +31 -0
+        error_setg(errp, "parameter length of %lu exceeds max of %u",
+                   len, KVM_SEV_SNP_ID_AUTH_SIZE);
+        return;
+    }
+}
+
+static bool
+sev_snp_guest_get_auth_key_en(Object *obj, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    return !!sev_snp_guest->kvm_finish_conf.auth_key_en;
+}
+
+static void
+sev_snp_guest_set_auth_key_en(Object *obj, bool value, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    sev_snp_guest->kvm_finish_conf.auth_key_en = value;
+}
+
+static char *
+sev_snp_guest_get_host_data(Object *obj, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    return g_strdup(sev_snp_guest->host_data);
+}
+
+static void

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1655

Quoted 10 lines
target/i386/sev.c +10 -0
+sev_snp_guest_set_host_data(Object *obj, const char *value, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+    struct kvm_sev_snp_launch_finish *finish = &sev_snp_guest->kvm_finish_conf;
+    g_autofree guchar *blob;
+    gsize len;
+
+    if (sev_snp_guest->host_data) {
+        g_free(sev_snp_guest->host_data);
+    }

Redundant 'if'

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1666

Quoted 11 lines
target/i386/sev.c +11 -0
+
+    /* store the base64 str so we don't need to re-encode in getter */
+    sev_snp_guest->host_data = g_strdup(value);
+
+    blob = qbase64_decode(sev_snp_guest->host_data, -1, &len, errp);
+
+    if (!blob) {
+        return;
+    }
+
+    if (len > sizeof(finish->host_data)) {

Equality test

Quoted 38 lines
target/i386/sev.c +38 -0
+        error_setg(errp, "parameter length of %lu exceeds max of %lu",
+                   len, sizeof(finish->host_data));
+        return;
+    }
+
+    memcpy(finish->host_data, blob, len);
+}
+
+static void
+sev_snp_guest_class_init(ObjectClass *oc, void *data)
+{
+    object_class_property_add(oc, "policy", "uint64",
+                              sev_snp_guest_get_policy,
+                              sev_snp_guest_set_policy, NULL, NULL);
+    object_class_property_add_str(oc, "guest-visible-workarounds",
+                                  sev_snp_guest_get_guest_visible_workarounds,
+                                  sev_snp_guest_set_guest_visible_workarounds);
+    object_class_property_add_str(oc, "id-block",
+                                  sev_snp_guest_get_id_block,
+                                  sev_snp_guest_set_id_block);
+    object_class_property_add_str(oc, "id-auth",
+                                  sev_snp_guest_get_id_auth,
+                                  sev_snp_guest_set_id_auth);
+    object_class_property_add_bool(oc, "auth-key-enabled",
+                                   sev_snp_guest_get_auth_key_en,
+                                   sev_snp_guest_set_auth_key_en);
+    object_class_property_add_str(oc, "host-data",
+                                  sev_snp_guest_get_host_data,
+                                  sev_snp_guest_set_host_data);
+}
+
+static void
+sev_snp_guest_instance_init(Object *obj)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    /* default init/start/finish params for kvm */
+    sev_snp_guest->kvm_start_conf.policy = DEFAULT_SEV_SNP_POLICY;

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1714

Quoted 10 lines
target/i386/sev.c +10 -0
+}
+
+/* guest info specific to sev-snp */
+static const TypeInfo sev_snp_guest_info = {
+    .parent = TYPE_SEV_COMMON,
+    .name = TYPE_SEV_SNP_GUEST,
+    .instance_size = sizeof(SevSnpGuestState),
+    .class_init = sev_snp_guest_class_init,
+    .instance_init = sev_snp_guest_instance_init,
+};

Use the OBJECT_DEFINE_TYPE_WITH_INTERFACES macro here.

Quoted 25 lines
target/i386/sev.c +2 -0
+
 static void
 sev_register_types(void)
 {
     type_register_static(&sev_common_info);
     type_register_static(&sev_guest_info);
+    type_register_static(&sev_snp_guest_info);
 }
 
 type_init(sev_register_types);
target/i386/sev.h +1 -0
index 668374eef3..bedc667eeb 100644
@@ -22,6 +22,7 @@
 
 #define TYPE_SEV_COMMON "sev-common"
 #define TYPE_SEV_GUEST "sev-guest"
+#define TYPE_SEV_SNP_GUEST "sev-snp-guest"
 
 #define SEV_POLICY_NODBG        0x1
 #define SEV_POLICY_NOKS         0x2
Signature · 1 line
2.25.1

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#49 lore raw

Re: [PATCH v3 22/49] i386/sev: Introduce 'sev-snp-guest' object

Quoted 76 lines · Daniel P. Berrangé
Quoted 75 lines · Michael Roth

From: Brijesh Singh <[email protected]>

SEV-SNP support relies on a different set of properties/state than the existing 'sev-guest' object. This patch introduces the 'sev-snp-guest' object, which can be used to configure an SEV-SNP guest. For example, a default-configured SEV-SNP guest with no additional information passed in for use with attestation:

-object sev-snp-guest,id=sev0

or a fully-specified SEV-SNP guest where all spec-defined binary blobs are passed in as base64-encoded strings:

  -object sev-snp-guest,id=sev0, \
    policy=0x30000, \
    init-flags=0, \
    id-block=YWFhYWFhYWFhYWFhYWFhCg==, \
    id-auth=CxHK/OKLkXGn/KpAC7Wl1FSiisWDbGTEKz..., \
    auth-key-enabled=on, \
    host-data=LNkCWBRC5CcdGXirbNUV1OrsR28s..., \
    guest-visible-workarounds=AA==, \

See the QAPI schema updates included in this patch for more usage details.

In some cases these blobs may be up to 4096 characters, but this is generally well below the default limit for linux hosts where command-line sizes are defined by the sysconf-configurable ARG_MAX value, which defaults to 2097152 characters for Ubuntu hosts, for example.

---
 docs/system/i386/amd-memory-encryption.rst |  78 ++++++-
 qapi/qom.json                              |  51 +++++
 target/i386/sev.c                          | 241 +++++++++++++++++++++
 target/i386/sev.h                          |   1 +
 4 files changed, 369 insertions(+), 2 deletions(-)
qapi/qom.json +32 -0
+##
+# @SevSnpGuestProperties:
+#
+# Properties for sev-snp-guest objects. Most of these are direct arguments
+# for the KVM_SNP_* interfaces documented in the linux kernel source
+# under Documentation/virt/kvm/amd-memory-encryption.rst, which are in
+# turn closely coupled with the SNP_INIT/SNP_LAUNCH_* firmware commands
+# documented in the SEV-SNP Firmware ABI Specification (Rev 0.9).
+#
+# More usage information is also available in the QEMU source tree under
+# docs/amd-memory-encryption.
+#
+# @policy: the 'POLICY' parameter to the SNP_LAUNCH_START command, as
+#          defined in the SEV-SNP firmware ABI (default: 0x30000)
+#
+# @guest-visible-workarounds: 16-byte, base64-encoded blob to report
+#                             hypervisor-defined workarounds, corresponding
+#                             to the 'GOSVW' parameter of the
+#                             SNP_LAUNCH_START command defined in the
+#                             SEV-SNP firmware ABI (default: all-zero)
+#
+# @id-block: 96-byte, base64-encoded blob to provide the 'ID Block'
+#            structure for the SNP_LAUNCH_FINISH command defined in the
+#            SEV-SNP firmware ABI (default: all-zero)
+#
+# @id-auth: 4096-byte, base64-encoded blob to provide the 'ID Authentication
+#           Information Structure' for the SNP_LAUNCH_FINISH command defined
+#           in the SEV-SNP firmware ABI (default: all-zero)
+#
+# @auth-key-enabled: true if 'id-auth' blob contains the 'AUTHOR_KEY' field
+#                    defined SEV-SNP firmware ABI (default: false)
+#
Quoted 8 lines

qapi/qom.json line 960

Quoted 6 lines
qapi/qom.json +6 -0
+# @host-data: 32-byte, base64-encoded, user-defined blob to provide to the
+#             guest, as documented for the 'HOST_DATA' parameter of the
+#             SNP_LAUNCH_FINISH command in the SEV-SNP firmware ABI
+#             (default: all-zero)
+#
+# Since: 7.2

This will be 9.1 at the earliest now.

Amazing how good I am at remembering these once I see a reply to a schema patch I'd already hit 'send' on :)

Quoted 73 lines
Quoted 60 lines
qapi/qom.json +11 -0
+##
+{ 'struct': 'SevSnpGuestProperties',
+  'base': 'SevCommonProperties',
+  'data': {
+            '*policy': 'uint64',
+            '*guest-visible-workarounds': 'str',
+            '*id-block': 'str',
+            '*id-auth': 'str',
+            '*auth-key-enabled': 'bool',
+            '*host-data': 'str' } }
+
target/i386/sev.c +24 -0
index 63a220de5e..7e6dab642a 100644
@@ -42,6 +42,7 @@
 
 OBJECT_DECLARE_SIMPLE_TYPE(SevCommonState, SEV_COMMON)
 OBJECT_DECLARE_SIMPLE_TYPE(SevGuestState, SEV_GUEST)
+OBJECT_DECLARE_SIMPLE_TYPE(SevSnpGuestState, SEV_SNP_GUEST)
 
 struct SevCommonState {
     X86ConfidentialGuest parent_obj;
@@ -87,8 +88,22 @@ struct SevGuestState {
     bool kernel_hashes;
 };
 
+struct SevSnpGuestState {
+    SevCommonState sev_common;
+
+    /* configuration parameters */
+    char *guest_visible_workarounds;
+    char *id_block;
+    char *id_auth;
+    char *host_data;
+
+    struct kvm_sev_snp_launch_start kvm_start_conf;
+    struct kvm_sev_snp_launch_finish kvm_finish_conf;
+};
+
 #define DEFAULT_GUEST_POLICY    0x1 /* disable debug */
 #define DEFAULT_SEV_DEVICE      "/dev/sev"
+#define DEFAULT_SEV_SNP_POLICY  0x30000
 
 #define SEV_INFO_BLOCK_GUID     "00f771de-1a7e-4fcb-890e-68c77e2fb44e"
 typedef struct __attribute__((__packed__)) SevInfoBlock {
@@ -1473,11 +1488,237 @@ static const TypeInfo sev_guest_info = {
     .class_init = sev_guest_class_init,
 
+
+static char *
+sev_snp_guest_get_guest_visible_workarounds(Object *obj, Error **errp)
+{
+    return g_strdup(SEV_SNP_GUEST(obj)->guest_visible_workarounds);
+}
+
+static void
+sev_snp_guest_set_guest_visible_workarounds(Object *obj, const char *value,

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1508

Quoted 10 lines
target/i386/sev.c +10 -0
+                                            Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+    struct kvm_sev_snp_launch_start *start = &sev_snp_guest->kvm_start_conf;
+    g_autofree guchar *blob;
+    gsize len;
+
+    if (sev_snp_guest->guest_visible_workarounds) {
+        g_free(sev_snp_guest->guest_visible_workarounds);
+    }

Redundant 'if' test - g_free is happy with NULL

Quoted 14 lines

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1536

Quoted 10 lines
target/i386/sev.c +10 -0
+
+    /* store the base64 str so we don't need to re-encode in getter */
+    sev_snp_guest->guest_visible_workarounds = g_strdup(value);
+
+    blob = qbase64_decode(sev_snp_guest->guest_visible_workarounds, -1, &len, errp);
+    if (!blob) {
+        return;
+    }
+
+    if (len > sizeof(start->gosvw)) {

The QAPI docs said this property must be '16 bytes', so I'd suggest we do a strict equality test, rather than min size test to catch a wider set of mistakes.

Makes sense.

Quoted 23 lines
Quoted 23 lines
target/i386/sev.c +22 -0
+        error_setg(errp, "parameter length of %lu exceeds max of %lu",
+                   len, sizeof(start->gosvw));
+        return;
+    }
+
+    memcpy(start->gosvw, blob, len);
+}
+
+static char *
+sev_snp_guest_get_id_block(Object *obj, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    return g_strdup(sev_snp_guest->id_block);
+}
+
+static void
+sev_snp_guest_set_id_block(Object *obj, const char *value, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+    struct kvm_sev_snp_launch_finish *finish = &sev_snp_guest->kvm_finish_conf;
+    gsize len;
Quoted 8 lines

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1563

Quoted 5 lines
target/i386/sev.c +5 -0
+
+    if (sev_snp_guest->id_block) {
+        g_free(sev_snp_guest->id_block);
+        g_free((guchar *)finish->id_block_uaddr);
+    }

Assuming 'id_block_uaddr' is also initialized to 0, when id_block is NULL, then you can remove the 'if' conditional.

id_block_uaddr only ever gets initialized after id_block gets initialized via g_strdup() below, and the SevSnpGuestState struct is zero'd during creation time so no need to worry about uninitialized values. So I think we can indeed drop the if check.

Quoted 38 lines

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1575

Quoted 13 lines
target/i386/sev.c +12 -0
+
+    /* store the base64 str so we don't need to re-encode in getter */
+    sev_snp_guest->id_block = g_strdup(value);
+
+    finish->id_block_uaddr =
+        (uint64_t)qbase64_decode(sev_snp_guest->id_block, -1, &len, errp);
+
+    if (!finish->id_block_uaddr) {
+        return;
+    }
+
+    if (len > KVM_SEV_SNP_ID_BLOCK_SIZE) {

Again, lets do a strict equality test to match the documented required size.

Quoted 21 lines
target/i386/sev.c +21 -0
+        error_setg(errp, "parameter length of %lu exceeds max of %u",
+                   len, KVM_SEV_SNP_ID_BLOCK_SIZE);
+        return;
+    }
+
+    finish->id_block_en = (len) ? 1 : 0;
+}
+
+static char *
+sev_snp_guest_get_id_auth(Object *obj, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    return g_strdup(sev_snp_guest->id_auth);
+}
+
+static void
+sev_snp_guest_set_id_auth(Object *obj, const char *value, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+    struct kvm_sev_snp_launch_finish *finish = &sev_snp_guest->kvm_finish_conf;
Quoted 8 lines

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1602

Quoted 6 lines
target/i386/sev.c +6 -0
+    gsize len;
+
+    if (sev_snp_guest->id_auth) {
+        g_free(sev_snp_guest->id_auth);
+        g_free((guchar *)finish->id_auth_uaddr);
+    }

Same probably redundant 'if'

Looks like it. Will address these and recurring cases below.

-Mike

Quoted 158 lines

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1614

Quoted 13 lines
target/i386/sev.c +12 -0
+
+    /* store the base64 str so we don't need to re-encode in getter */
+    sev_snp_guest->id_auth = g_strdup(value);
+
+    finish->id_auth_uaddr =
+        (uint64_t)qbase64_decode(sev_snp_guest->id_auth, -1, &len, errp);
+
+    if (!finish->id_auth_uaddr) {
+        return;
+    }
+
+    if (len > KVM_SEV_SNP_ID_AUTH_SIZE) {

Equality test.

Quoted 31 lines
target/i386/sev.c +31 -0
+        error_setg(errp, "parameter length of %lu exceeds max of %u",
+                   len, KVM_SEV_SNP_ID_AUTH_SIZE);
+        return;
+    }
+}
+
+static bool
+sev_snp_guest_get_auth_key_en(Object *obj, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    return !!sev_snp_guest->kvm_finish_conf.auth_key_en;
+}
+
+static void
+sev_snp_guest_set_auth_key_en(Object *obj, bool value, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    sev_snp_guest->kvm_finish_conf.auth_key_en = value;
+}
+
+static char *
+sev_snp_guest_get_host_data(Object *obj, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    return g_strdup(sev_snp_guest->host_data);
+}
+
+static void

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1655

Quoted 10 lines
target/i386/sev.c +10 -0
+sev_snp_guest_set_host_data(Object *obj, const char *value, Error **errp)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+    struct kvm_sev_snp_launch_finish *finish = &sev_snp_guest->kvm_finish_conf;
+    g_autofree guchar *blob;
+    gsize len;
+
+    if (sev_snp_guest->host_data) {
+        g_free(sev_snp_guest->host_data);
+    }

Redundant 'if'

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1666

Quoted 11 lines
target/i386/sev.c +11 -0
+
+    /* store the base64 str so we don't need to re-encode in getter */
+    sev_snp_guest->host_data = g_strdup(value);
+
+    blob = qbase64_decode(sev_snp_guest->host_data, -1, &len, errp);
+
+    if (!blob) {
+        return;
+    }
+
+    if (len > sizeof(finish->host_data)) {

Equality test

Quoted 38 lines
target/i386/sev.c +38 -0
+        error_setg(errp, "parameter length of %lu exceeds max of %lu",
+                   len, sizeof(finish->host_data));
+        return;
+    }
+
+    memcpy(finish->host_data, blob, len);
+}
+
+static void
+sev_snp_guest_class_init(ObjectClass *oc, void *data)
+{
+    object_class_property_add(oc, "policy", "uint64",
+                              sev_snp_guest_get_policy,
+                              sev_snp_guest_set_policy, NULL, NULL);
+    object_class_property_add_str(oc, "guest-visible-workarounds",
+                                  sev_snp_guest_get_guest_visible_workarounds,
+                                  sev_snp_guest_set_guest_visible_workarounds);
+    object_class_property_add_str(oc, "id-block",
+                                  sev_snp_guest_get_id_block,
+                                  sev_snp_guest_set_id_block);
+    object_class_property_add_str(oc, "id-auth",
+                                  sev_snp_guest_get_id_auth,
+                                  sev_snp_guest_set_id_auth);
+    object_class_property_add_bool(oc, "auth-key-enabled",
+                                   sev_snp_guest_get_auth_key_en,
+                                   sev_snp_guest_set_auth_key_en);
+    object_class_property_add_str(oc, "host-data",
+                                  sev_snp_guest_get_host_data,
+                                  sev_snp_guest_set_host_data);
+}
+
+static void
+sev_snp_guest_instance_init(Object *obj)
+{
+    SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(obj);
+
+    /* default init/start/finish params for kvm */
+    sev_snp_guest->kvm_start_conf.policy = DEFAULT_SEV_SNP_POLICY;

target/i386/sev.c static const TypeInfo sev_guest_info = { · line 1714

Quoted 10 lines
target/i386/sev.c +10 -0
+}
+
+/* guest info specific to sev-snp */
+static const TypeInfo sev_snp_guest_info = {
+    .parent = TYPE_SEV_COMMON,
+    .name = TYPE_SEV_SNP_GUEST,
+    .instance_size = sizeof(SevSnpGuestState),
+    .class_init = sev_snp_guest_class_init,
+    .instance_init = sev_snp_guest_instance_init,
+};

Use the OBJECT_DEFINE_TYPE_WITH_INTERFACES macro here.

Quoted 25 lines
target/i386/sev.c +2 -0
+
 static void
 sev_register_types(void)
 {
     type_register_static(&sev_common_info);
     type_register_static(&sev_guest_info);
+    type_register_static(&sev_snp_guest_info);
 }
 
 type_init(sev_register_types);
target/i386/sev.h +1 -0
index 668374eef3..bedc667eeb 100644
@@ -22,6 +22,7 @@
 
 #define TYPE_SEV_COMMON "sev-common"
 #define TYPE_SEV_GUEST "sev-guest"
+#define TYPE_SEV_SNP_GUEST "sev-snp-guest"
 
 #define SEV_POLICY_NODBG        0x1
 #define SEV_POLICY_NOKS         0x2
Signature · 1 line
2.25.1

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#50 lore raw

Re: [PATCH v3 22/49] i386/sev: Introduce 'sev-snp-guest' object

In reply to Michael Roth
Quoted 35 lines · Michael Roth

From: Brijesh Singh <[email protected]>

SEV-SNP support relies on a different set of properties/state than the existing 'sev-guest' object. This patch introduces the 'sev-snp-guest' object, which can be used to configure an SEV-SNP guest. For example, a default-configured SEV-SNP guest with no additional information passed in for use with attestation:

-object sev-snp-guest,id=sev0

or a fully-specified SEV-SNP guest where all spec-defined binary blobs are passed in as base64-encoded strings:

  -object sev-snp-guest,id=sev0, \
    policy=0x30000, \
    init-flags=0, \
    id-block=YWFhYWFhYWFhYWFhYWFhCg==, \
    id-auth=CxHK/OKLkXGn/KpAC7Wl1FSiisWDbGTEKz..., \
    auth-key-enabled=on, \
    host-data=LNkCWBRC5CcdGXirbNUV1OrsR28s..., \
    guest-visible-workarounds=AA==, \

See the QAPI schema updates included in this patch for more usage details.

In some cases these blobs may be up to 4096 characters, but this is generally well below the default limit for linux hosts where command-line sizes are defined by the sysconf-configurable ARG_MAX value, which defaults to 2097152 characters for Ubuntu hosts, for example.

[...]

qapi/qom.json line 927

Quoted 13 lines
qapi/qom.json +5 -0
index 66b5781ca6..b25a3043da 100644
@@ -920,6 +920,55 @@
             '*handle': 'uint32',
             '*kernel-hashes': 'bool' } }
 
+##
+# @SevSnpGuestProperties:
+#
+# Properties for sev-snp-guest objects. Most of these are direct arguments
+# for the KVM_SNP_* interfaces documented in the linux kernel source

"Linux", please.

qapi/qom.json line 928

Quoted 1 line
qapi/qom.json +1 -0
+# under Documentation/virt/kvm/amd-memory-encryption.rst, which are in

Does not seem to exist. Do you mean
Documentation/arch/x86/amd-memory-encryption.rst?

qapi/qom.json line 930

Quoted 2 lines
qapi/qom.json +2 -0
+# turn closely coupled with the SNP_INIT/SNP_LAUNCH_* firmware commands
+# documented in the SEV-SNP Firmware ABI Specification (Rev 0.9).

docs/devel/qapi-code-gen.rst:

For legibility, wrap text paragraphs so every line is at most 70 characters long.

Separate sentences with two spaces.

qapi/qom.json line 936

Quoted 6 lines
qapi/qom.json +6 -0
+#
+# More usage information is also available in the QEMU source tree under
+# docs/amd-memory-encryption.
+#
+# @policy: the 'POLICY' parameter to the SNP_LAUNCH_START command, as
+#          defined in the SEV-SNP firmware ABI (default: 0x30000)

docs/devel/qapi-code-gen.rst:

Descriptions start with '\@name:'. The description text must be indented like this::

# @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
# do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Quoted 14 lines
qapi/qom.json +14 -0
+#
+# @guest-visible-workarounds: 16-byte, base64-encoded blob to report
+#                             hypervisor-defined workarounds, corresponding
+#                             to the 'GOSVW' parameter of the
+#                             SNP_LAUNCH_START command defined in the
+#                             SEV-SNP firmware ABI (default: all-zero)
+#
+# @id-block: 96-byte, base64-encoded blob to provide the 'ID Block'
+#            structure for the SNP_LAUNCH_FINISH command defined in the
+#            SEV-SNP firmware ABI (default: all-zero)
+#
+# @id-auth: 4096-byte, base64-encoded blob to provide the 'ID Authentication
+#           Information Structure' for the SNP_LAUNCH_FINISH command defined
+#           in the SEV-SNP firmware ABI (default: all-zero)

qapi/qom.json line 960

Quoted 10 lines
qapi/qom.json +10 -0
+#
+# @auth-key-enabled: true if 'id-auth' blob contains the 'AUTHOR_KEY' field
+#                    defined SEV-SNP firmware ABI (default: false)
+#
+# @host-data: 32-byte, base64-encoded, user-defined blob to provide to the
+#             guest, as documented for the 'HOST_DATA' parameter of the
+#             SNP_LAUNCH_FINISH command in the SEV-SNP firmware ABI
+#             (default: all-zero)
+#
+# Since: 7.2

9.1

qapi/qom.json line 961

Quoted 1 line
qapi/qom.json +1 -0
+##

Together:

##
# @SevSnpGuestProperties:
#
# Properties for sev-snp-guest objects. Most of these are direct # arguments for the KVM_SNP_* interfaces documented in the Linux # kernel source under
# Documentation/arch/x86/amd-memory-encryption.rst, which are in turn # closely coupled with the SNP_INIT/SNP_LAUNCH_* firmware commands # documented in the SEV-SNP Firmware ABI Specification (Rev 0.9). #
# More usage information is also available in the QEMU source tree # under docs/amd-memory-encryption.
#
# @policy: the 'POLICY' parameter to the SNP_LAUNCH_START command, as
# defined in the SEV-SNP firmware ABI (default: 0x30000)
#
# @guest-visible-workarounds: 16-byte, base64-encoded blob to report
# hypervisor-defined workarounds, corresponding to the 'GOSVW'
# parameter of the SNP_LAUNCH_START command defined in the SEV-SNP
# firmware ABI (default: all-zero)
#
# @id-block: 96-byte, base64-encoded blob to provide the 'ID Block'
# structure for the SNP_LAUNCH_FINISH command defined in the
# SEV-SNP firmware ABI (default: all-zero)
#
# @id-auth: 4096-byte, base64-encoded blob to provide the 'ID
# Authentication Information Structure' for the SNP_LAUNCH_FINISH
# command defined in the SEV-SNP firmware ABI (default: all-zero)
#
# @auth-key-enabled: true if 'id-auth' blob contains the 'AUTHOR_KEY'
# field defined SEV-SNP firmware ABI (default: false)
#
# @host-data: 32-byte, base64-encoded, user-defined blob to provide to
# the guest, as documented for the 'HOST_DATA' parameter of the
# SNP_LAUNCH_FINISH command in the SEV-SNP firmware ABI (default:
# all-zero)
#
# @certs-path: path to certificate data that can be passed to guests
# via SNP Extended Guest Requests. File should be in the format
# described in the GHCB specification. (default: none)
#
# Since: 9.1
##

We generally prefer symbolic to numeric / binary encoding in QMP. Can you explain briefly why you choose numeric and binary here?

Quoted 29 lines
qapi/qom.json +12 -0
+{ 'struct': 'SevSnpGuestProperties',
+  'base': 'SevCommonProperties',
+  'data': {
+            '*policy': 'uint64',
+            '*guest-visible-workarounds': 'str',
+            '*id-block': 'str',
+            '*id-auth': 'str',
+            '*auth-key-enabled': 'bool',
+            '*host-data': 'str' } }
+
 ##
 # @ThreadContextProperties:
 #
@@ -998,6 +1047,7 @@
     { 'name': 'secret_keyring',
       'if': 'CONFIG_SECRET_KEYRING' },
     'sev-guest',
+    'sev-snp-guest',
     'thread-context',
     's390-pv-guest',
     'throttle-group',
@@ -1068,6 +1118,7 @@
       'secret_keyring':             { 'type': 'SecretKeyringProperties',
                                       'if': 'CONFIG_SECRET_KEYRING' },
       'sev-guest':                  'SevGuestProperties',
+      'sev-snp-guest':              'SevSnpGuestProperties',
       'thread-context':             'ThreadContextProperties',
       'throttle-group':             'ThrottleGroupProperties',
       'tls-creds-anon':             'TlsCredsAnonProperties',

[...]

#51 lore raw

PATCH v3 23/49 i386/sev: Add a sev_snp_enabled() helper

In reply to Michael Roth

Add a simple helper to check if the current guest type is SNP. Also have SNP-enabled imply that SEV-ES is enabled as well, and fix up any places where the sev_es_enabled() check is expecting a pure/non-SNP guest.

---
 target/i386/sev.c | 13 ++++++++++++-
 target/i386/sev.h |  2 ++
 2 files changed, 14 insertions(+), 1 deletion(-)
target/i386/sev.c +12 -1
index 7e6dab642a..2eb13ba639 100644
@@ -316,12 +316,21 @@ sev_enabled(void)
     return !!object_dynamic_cast(OBJECT(cgs), TYPE_SEV_COMMON);
 }
 
+bool
+sev_snp_enabled(void)
+{
+    ConfidentialGuestSupport *cgs = MACHINE(qdev_get_machine())->cgs;
+
+    return !!object_dynamic_cast(OBJECT(cgs), TYPE_SEV_SNP_GUEST);
+}
+
 bool
 sev_es_enabled(void)
 {
     ConfidentialGuestSupport *cgs = MACHINE(qdev_get_machine())->cgs;
 
-    return sev_enabled() && (SEV_GUEST(cgs)->policy & SEV_POLICY_ES);
+    return sev_snp_enabled() ||
+            (sev_enabled() && SEV_GUEST(cgs)->policy & SEV_POLICY_ES);
 }
 
 uint32_t
@@ -933,7 +942,9 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
                          __func__);
             goto err;
         }
+    }
 
+    if (sev_es_enabled() && !sev_snp_enabled()) {
         if (!(status.flags & SEV_STATUS_FLAGS_CONFIG_ES)) {
             error_report("%s: guest policy requires SEV-ES, but "
                          "host SEV-ES support unavailable",
target/i386/sev.h +2 -0
index bedc667eeb..94295ee74f 100644
@@ -45,9 +45,11 @@ typedef struct SevKernelLoaderContext {
 #ifdef CONFIG_SEV
 bool sev_enabled(void);
 bool sev_es_enabled(void);
+bool sev_snp_enabled(void);
 #else
 #define sev_enabled() 0
 #define sev_es_enabled() 0
+#define sev_snp_enabled() 0
 #endif
 
 uint32_t sev_get_cbit_position(void);
Signature · 1 line
2.25.1
#52 lore raw

Re: [PATCH v3 23/49] i386/sev: Add a sev_snp_enabled() helper

Quoted 17 lines · Michael Roth

Add a simple helper to check if the current guest type is SNP. Also have SNP-enabled imply that SEV-ES is enabled as well, and fix up any places where the sev_es_enabled() check is expecting a pure/non-SNP guest.

---
 target/i386/sev.c | 13 ++++++++++++-
 target/i386/sev.h |  2 ++
 2 files changed, 14 insertions(+), 1 deletion(-)
target/i386/sev.c +0 -0
index 7e6dab642a..2eb13ba639 100644

target/i386/sev.c static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 950

Quoted 9 lines
target/i386/sev.c +2 -0
@@ -933,7 +942,9 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
                          __func__);
             goto err;
         }
+    }
 
+    if (sev_es_enabled() && !sev_snp_enabled()) {
         if (!(status.flags & SEV_STATUS_FLAGS_CONFIG_ES)) {
             error_report("%s: guest policy requires SEV-ES, but "
                          "host SEV-ES support unavailable",

Opps, pre-existing bug here - this method has an 'Error **errp' parameter, so should be using 'error_report'.

There are several more examples of this in this method that predate your patch series. Can you put a patch at the start of this series that fixes them before introducing SNP.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#53 lore raw

Re: [PATCH v3 23/49] i386/sev: Add a sev_snp_enabled() helper

Quoted 31 lines · Daniel P. Berrangé
Quoted 17 lines · Michael Roth

Add a simple helper to check if the current guest type is SNP. Also have SNP-enabled imply that SEV-ES is enabled as well, and fix up any places where the sev_es_enabled() check is expecting a pure/non-SNP guest.

---
 target/i386/sev.c | 13 ++++++++++++-
 target/i386/sev.h |  2 ++
 2 files changed, 14 insertions(+), 1 deletion(-)
target/i386/sev.c +0 -0
index 7e6dab642a..2eb13ba639 100644

target/i386/sev.c static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 950

Quoted 9 lines
target/i386/sev.c +2 -0
@@ -933,7 +942,9 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
                          __func__);
             goto err;
         }
+    }
 
+    if (sev_es_enabled() && !sev_snp_enabled()) {
         if (!(status.flags & SEV_STATUS_FLAGS_CONFIG_ES)) {
             error_report("%s: guest policy requires SEV-ES, but "
                          "host SEV-ES support unavailable",

Opps, pre-existing bug here - this method has an 'Error **errp' parameter, so should be using 'error_report'.

Quoted 3 lines

There are several more examples of this in this method that predate your patch series. Can you put a patch at the start of this series that fixes them before introducing SNP.

Sure, will add a pre-patch to fix up all the pre-existing issues you've noted.

-Mike

Quoted 9 lines

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#54 lore raw

PATCH v3 24/49 target/i386: Add handling for KVM_X86_SNP_VM VM type

In reply to Michael Roth

An SNP VM requires VM type KVM_X86_SNP_VM to be passed to kvm_ioctl(KVM_CREATE_VM). Add it to the list of supported VM types, and return it appropriately via X86ConfidentialGuestClass->kvm_type().

---
 target/i386/kvm/kvm.c |  1 +
 target/i386/sev.c     | 10 ++++++++--
 2 files changed, 9 insertions(+), 2 deletions(-)
target/i386/kvm/kvm.c +1 -0
index e109648f26..59e9048e61 100644
@@ -164,6 +164,7 @@ static int kvm_get_one_msr(X86CPU *cpu, int index, uint64_t *value);
 
 static const char *vm_type_name[] = {
     [KVM_X86_DEFAULT_VM] = "default",
+    [KVM_X86_SNP_VM] = "snp"
 };
 
 bool kvm_is_vm_type_supported(int type)
target/i386/sev.c +8 -2
index 2eb13ba639..61af312a11 100644
@@ -853,14 +853,20 @@ sev_vm_state_change(void *opaque, bool running, RunState state)
 static int sev_kvm_type(X86ConfidentialGuest *cg)
 {
     SevCommonState *sev_common = SEV_COMMON(cg);
-    SevGuestState *sev_guest = SEV_GUEST(sev_common);
     int kvm_type;
 
     if (sev_common->kvm_type != -1) {
         goto out;
     }
 
-    kvm_type = (sev_guest->policy & SEV_POLICY_ES) ? KVM_X86_SEV_ES_VM : KVM_X86_SEV_VM;
+    if (sev_snp_enabled()) {
+        kvm_type = KVM_X86_SNP_VM;
+    } else if (sev_es_enabled()) {
+        kvm_type = KVM_X86_SEV_ES_VM;
+    } else {
+        kvm_type = KVM_X86_SEV_VM;
+    }
+
     if (kvm_is_vm_type_supported(kvm_type)) {
         sev_common->kvm_type = kvm_type;
     } else {
Signature · 1 line
2.25.1
maintainer
#55 lore raw

Re: [PATCH v3 24/49] target/i386: Add handling for KVM_X86_SNP_VM VM type

Quoted 35 lines · Michael Roth

An SNP VM requires VM type KVM_X86_SNP_VM to be passed to kvm_ioctl(KVM_CREATE_VM). Add it to the list of supported VM types, and return it appropriately via X86ConfidentialGuestClass->kvm_type().

---
  target/i386/kvm/kvm.c |  1 +
  target/i386/sev.c     | 10 ++++++++--
  2 files changed, 9 insertions(+), 2 deletions(-)
target/i386/kvm/kvm.c +1 -0
index e109648f26..59e9048e61 100644
@@ -164,6 +164,7 @@ static int kvm_get_one_msr(X86CPU *cpu, int index, uint64_t *value);
 
  static const char *vm_type_name[] = {
      [KVM_X86_DEFAULT_VM] = "default",
+    [KVM_X86_SNP_VM] = "snp"
  };
 
  bool kvm_is_vm_type_supported(int type)
target/i386/sev.c +0 -1
index 2eb13ba639..61af312a11 100644
@@ -853,14 +853,20 @@ sev_vm_state_change(void *opaque, bool running, RunState state)
  static int sev_kvm_type(X86ConfidentialGuest *cg)
  {
      SevCommonState *sev_common = SEV_COMMON(cg);
-    SevGuestState *sev_guest = SEV_GUEST(sev_common);
      int kvm_type;
 
      if (sev_common->kvm_type != -1) {
          goto out;

target/i386/sev.c sev_vm_state_change(void *opaque, bool running, RunState state) · line 868

Quoted 10 lines
target/i386/sev.c +7 -1
      }
 
-    kvm_type = (sev_guest->policy & SEV_POLICY_ES) ? KVM_X86_SEV_ES_VM : KVM_X86_SEV_VM;
+    if (sev_snp_enabled()) {
+        kvm_type = KVM_X86_SNP_VM;
+    } else if (sev_es_enabled()) {
+        kvm_type = KVM_X86_SEV_ES_VM;
+    } else {
+        kvm_type = KVM_X86_SEV_VM;
+    }

I don't really like this, the kvm_type method can be implemented separately in sev-guest and sev-snp-guest.

This is for two reasons:

1) it makes sev_kvm_type self-contained, instead of poking in current_machine via sev_*_enabled()

2) sev-snp-guest can just return KVM_X86_SNP_VM, relying on the "vm type supported" checks from common target/i386/kvm/kvm.c code. Instead, sev-guest has to check kvm_is_vm_type_supported() in order to support the legacy ioctls (and that is why my patches didn't add the SEV/SEV-ES types to vm_type_name[], too).

Thanks,

Paolo

Quoted 4 lines
target/i386/sev.c +1 -0
+
      if (kvm_is_vm_type_supported(kvm_type)) {
          sev_common->kvm_type = kvm_type;
      } else {
#56 lore raw

PATCH v3 25/49 i386/sev: Skip RAMBlock notifiers for SNP

In reply to Michael Roth

SEV uses these notifiers to register/pin pages prior to guest use, since they could potentially be used for private memory where page migration is not supported. But SNP only uses guest_memfd-provided pages for private memory, which has its own kernel-internal mechanisms for registering/pinning memory.

---
 target/i386/sev.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
target/i386/sev.c +9 -1
index 61af312a11..774262d834 100644
@@ -982,7 +982,15 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         goto err;
     }
 
-    ram_block_notifier_add(&sev_ram_notifier);
+    if (!sev_snp_enabled()) {
+        /*
+         * SEV uses these notifiers to register/pin pages prior to guest use,
+         * but SNP relies on guest_memfd for private pages, which has it's
+         * own internal mechanisms for registering/pinning private memory.
+         */
+        ram_block_notifier_add(&sev_ram_notifier);
+    }
+
     qemu_add_machine_init_done_notifier(&sev_machine_done_notify);
     qemu_add_vm_change_state_handler(sev_vm_state_change, sev_common);
 
Signature · 1 line
2.25.1
maintainer
#57 lore raw

Re: [PATCH v3 25/49] i386/sev: Skip RAMBlock notifiers for SNP

Quoted 22 lines · Michael Roth

SEV uses these notifiers to register/pin pages prior to guest use, since they could potentially be used for private memory where page migration is not supported. But SNP only uses guest_memfd-provided pages for private memory, which has its own kernel-internal mechanisms for registering/pinning memory.

---
  target/i386/sev.c | 10 +++++++++-
  1 file changed, 9 insertions(+), 1 deletion(-)
target/i386/sev.c +2 -1
index 61af312a11..774262d834 100644
@@ -982,7 +982,15 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
          goto err;
      }
 
-    ram_block_notifier_add(&sev_ram_notifier);
+    if (!sev_snp_enabled()) {
+        /*

target/i386/sev.c static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 996

Quoted 10 lines
target/i386/sev.c +7 -0
+         * SEV uses these notifiers to register/pin pages prior to guest use,
+         * but SNP relies on guest_memfd for private pages, which has it's
+         * own internal mechanisms for registering/pinning private memory.
+         */
+        ram_block_notifier_add(&sev_ram_notifier);
+    }
+
      qemu_add_machine_init_done_notifier(&sev_machine_done_notify);
      qemu_add_vm_change_state_handler(sev_vm_state_change, sev_common);
 

These three lines can be done in any order, so I suggest removing ram_block_notifier_add + qemu_add_machine_init_done_notifier from the sev-common implementation of kvm_init (let's call it sev_common_kvm_init); and add an override in sev-guest that calls them if sev_common_kvm_init() succeeds.

(treat this as a review for 25/26/29).

Paolo

#58 lore raw

Re: [PATCH v3 25/49] i386/sev: Skip RAMBlock notifiers for SNP

Quoted 38 lines · Paolo Bonzini
Quoted 20 lines · Michael Roth

SEV uses these notifiers to register/pin pages prior to guest use, since they could potentially be used for private memory where page migration is not supported. But SNP only uses guest_memfd-provided pages for private memory, which has its own kernel-internal mechanisms for registering/pinning memory.

---
  target/i386/sev.c | 10 +++++++++-
  1 file changed, 9 insertions(+), 1 deletion(-)
target/i386/sev.c +1 -1
index 61af312a11..774262d834 100644
@@ -982,7 +982,15 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
          goto err;
      }
-    ram_block_notifier_add(&sev_ram_notifier);
+    if (!sev_snp_enabled()) {

target/i386/sev.c static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 994

Quoted 10 lines
target/i386/sev.c +8 -0
+        /*
+         * SEV uses these notifiers to register/pin pages prior to guest use,
+         * but SNP relies on guest_memfd for private pages, which has it's
+         * own internal mechanisms for registering/pinning private memory.
+         */
+        ram_block_notifier_add(&sev_ram_notifier);
+    }
+
      qemu_add_machine_init_done_notifier(&sev_machine_done_notify);
      qemu_add_vm_change_state_handler(sev_vm_state_change, sev_common);

These three lines can be done in any order, so I suggest removing ram_block_notifier_add + qemu_add_machine_init_done_notifier from the sev-common implementation of kvm_init (let's call it sev_common_kvm_init); and add an override in sev-guest that calls them if sev_common_kvm_init() succeeds.

Quoted 1 line

(treat this as a review for 25/26/29).

Makes sense. Will split out the common bits of sev_kvm_init() and use class methods for initialization specific to sev-guest and sev-snp-guest.

-Mike

Quoted 3 lines

Paolo

#59 lore raw

PATCH v3 26/49 i386/sev: Skip machine-init-done notifiers for SNP

In reply to Michael Roth

The machine done notify event is used for SEV guests to get the measurement of the encrypted images. When SEV-SNP is enabled, the measurement is part of the guest attestation process where it can be collected without any reliance on the VMM. So skip registering the notifier for SNP in favor of using guest attestation instead.

---
 target/i386/sev.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
target/i386/sev.c +9 -1
index 774262d834..e4deb7b41e 100644
@@ -989,9 +989,17 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
          * own internal mechanisms for registering/pinning private memory.
          */
         ram_block_notifier_add(&sev_ram_notifier);
+
+        /*
+         * The machine done notify event is used for SEV guests to get the
+         * measurement of the encrypted images. When SEV-SNP is enabled, the
+         * measurement is part of the guest attestation process where it can
+         * be collected without any reliance on the VMM. So skip registering
+         * the notifier for SNP in favor of using guest attestation instead.
+         */
+        qemu_add_machine_init_done_notifier(&sev_machine_done_notify);
     }
 
-    qemu_add_machine_init_done_notifier(&sev_machine_done_notify);
     qemu_add_vm_change_state_handler(sev_vm_state_change, sev_common);
 
     cgs->ready = true;
Signature · 1 line
2.25.1
#60 lore raw

PATCH v3 27/49 i386/sev: Set ms->require_guest_memfd for SNP

In reply to Michael Roth

SNP requires guest_memfd for private guest memory, so enable it so that the appropriate guest_memfd backend will be available for normal RAM regions.

---
 target/i386/sev.c | 5 +++++
 1 file changed, 5 insertions(+)
target/i386/sev.c +5 -0
index e4deb7b41e..b06c796aae 100644
@@ -880,6 +880,7 @@ out:
 static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
     SevCommonState *sev_common = SEV_COMMON(cgs);
+    MachineState *ms = MACHINE(qdev_get_machine());
     char *devname;
     int ret, fw_error, cmd;
     uint32_t ebx;
@@ -1000,6 +1001,10 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         qemu_add_machine_init_done_notifier(&sev_machine_done_notify);
     }
 
+    if (sev_snp_enabled()) {
+        ms->require_guest_memfd = true;
+    }
+
     qemu_add_vm_change_state_handler(sev_vm_state_change, sev_common);
 
     cgs->ready = true;
Signature · 1 line
2.25.1
maintainer
#61 lore raw

Re: [PATCH v3 27/49] i386/sev: Set ms->require_guest_memfd for SNP

Quoted 22 lines · Michael Roth

SNP requires guest_memfd for private guest memory, so enable it so that the appropriate guest_memfd backend will be available for normal RAM regions.

---
  target/i386/sev.c | 5 +++++
  1 file changed, 5 insertions(+)
target/i386/sev.c +1 -0
index e4deb7b41e..b06c796aae 100644
@@ -880,6 +880,7 @@ out:
  static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
  {
      SevCommonState *sev_common = SEV_COMMON(cgs);
+    MachineState *ms = MACHINE(qdev_get_machine());
      char *devname;
      int ret, fw_error, cmd;
      uint32_t ebx;

target/i386/sev.c static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 1006

Quoted 6 lines
target/i386/sev.c +3 -0
@@ -1000,6 +1001,10 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
          qemu_add_machine_init_done_notifier(&sev_machine_done_notify);
      }
 
+    if (sev_snp_enabled()) {
+        ms->require_guest_memfd = true;
+    }

Likewise, this and the following patch should be done in the sev-snp-guest's override of kvm_init.

Paolo

Quoted 3 lines
target/i386/sev.c +0 -0
      qemu_add_vm_change_state_handler(sev_vm_state_change, sev_common);
 
      cgs->ready = true;
#62 lore raw

PATCH v3 28/49 i386/sev: Disable SMM for SNP

In reply to Michael Roth

SNP does not support SMM.

---
 target/i386/sev.c | 8 ++++++++
 1 file changed, 8 insertions(+)
target/i386/sev.c +8 -0
index b06c796aae..134e8f7c22 100644
@@ -881,6 +881,7 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
     SevCommonState *sev_common = SEV_COMMON(cgs);
     MachineState *ms = MACHINE(qdev_get_machine());
+    X86MachineState *x86ms = X86_MACHINE(ms);
     char *devname;
     int ret, fw_error, cmd;
     uint32_t ebx;
@@ -1003,6 +1004,13 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 
     if (sev_snp_enabled()) {
         ms->require_guest_memfd = true;
+
+        if (x86ms->smm == ON_OFF_AUTO_AUTO) {
+            x86ms->smm = ON_OFF_AUTO_OFF;
+        } else if (x86ms->smm == ON_OFF_AUTO_ON) {
+            error_report("SEV-SNP does not support SMM.");
+            goto err;
+        }
     }
 
     qemu_add_vm_change_state_handler(sev_vm_state_change, sev_common);
Signature · 1 line
2.25.1
#63 lore raw

Re: [PATCH v3 28/49] i386/sev: Disable SMM for SNP

Quoted 21 lines · Michael Roth

SNP does not support SMM.

---
 target/i386/sev.c | 8 ++++++++
 1 file changed, 8 insertions(+)
target/i386/sev.c +1 -0
index b06c796aae..134e8f7c22 100644
@@ -881,6 +881,7 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
     SevCommonState *sev_common = SEV_COMMON(cgs);
     MachineState *ms = MACHINE(qdev_get_machine());
+    X86MachineState *x86ms = X86_MACHINE(ms);
     char *devname;
     int ret, fw_error, cmd;
     uint32_t ebx;
@@ -1003,6 +1004,13 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 

target/i386/sev.c static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 1014

Quoted 10 lines
target/i386/sev.c +7 -0
     if (sev_snp_enabled()) {
         ms->require_guest_memfd = true;
+
+        if (x86ms->smm == ON_OFF_AUTO_AUTO) {
+            x86ms->smm = ON_OFF_AUTO_OFF;
+        } else if (x86ms->smm == ON_OFF_AUTO_ON) {
+            error_report("SEV-SNP does not support SMM.");
+            goto err;
+        }
     }

This method has a 'Error **errp' parameter, so you must use error_setg, not error_report.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#64 lore raw

PATCH v3 29/49 i386/sev: Don't disable block discarding for SNP

In reply to Michael Roth

SEV/SEV-ES rely on pinned memory to back guest RAM so discarding isn't actually possible. With SNP, only guest_memfd pages are used for private guest memory, so discarding of shared memory is still possible, so only disable discard for SEV/SEV-ES.

---
 target/i386/sev.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
target/i386/sev.c +12 -4
index 134e8f7c22..43e6c0172f 100644
@@ -888,10 +888,18 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
     uint32_t host_cbitpos;
     struct sev_user_data_status status = {};
 
-    ret = ram_block_discard_disable(true);
-    if (ret) {
-        error_report("%s: cannot disable RAM discard", __func__);
-        return -1;
+    /*
+     * SEV/SEV-ES rely on pinned memory to back guest RAM so discarding
+     * isn't actually possible. With SNP, only guest_memfd pages are used
+     * for private guest memory, so discarding of shared memory is still
+     * possible..
+     */
+    if (!sev_snp_enabled()) {
+        ret = ram_block_discard_disable(true);
+        if (ret) {
+            error_report("%s: cannot disable RAM discard", __func__);
+            return -1;
+        }
     }
 
     sev_common->state = SEV_STATE_UNINIT;
Signature · 1 line
2.25.1
#65 lore raw

Re: [PATCH v3 29/49] i386/sev: Don't disable block discarding for SNP

Quoted 25 lines · Michael Roth

SEV/SEV-ES rely on pinned memory to back guest RAM so discarding isn't actually possible. With SNP, only guest_memfd pages are used for private guest memory, so discarding of shared memory is still possible, so only disable discard for SEV/SEV-ES.

---
 target/i386/sev.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
target/i386/sev.c +3 -4
index 134e8f7c22..43e6c0172f 100644
@@ -888,10 +888,18 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
     uint32_t host_cbitpos;
     struct sev_user_data_status status = {};
 
-    ret = ram_block_discard_disable(true);
-    if (ret) {
-        error_report("%s: cannot disable RAM discard", __func__);
-        return -1;
+    /*
+     * SEV/SEV-ES rely on pinned memory to back guest RAM so discarding
+     * isn't actually possible. With SNP, only guest_memfd pages are used

target/i386/sev.c static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 903

Quoted 10 lines
target/i386/sev.c +9 -0
+     * for private guest memory, so discarding of shared memory is still
+     * possible..
+     */
+    if (!sev_snp_enabled()) {
+        ret = ram_block_discard_disable(true);
+        if (ret) {
+            error_report("%s: cannot disable RAM discard", __func__);
+            return -1;
+        }
     }

Pre-existing code bug, but this method must use 'error_setg' to fill the 'Error **errp' parameter.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#66 lore raw

PATCH v3 30/49 i386/cpu: Set SEV-SNP CPUID bit when SNP enabled

In reply to Michael Roth

SNP guests will rely on this bit to determine certain feature support.

---
 target/i386/cpu.c | 1 +
 1 file changed, 1 insertion(+)
target/i386/cpu.c +1 -0
index 33760a2ee1..3fdaac3472 100644
@@ -6664,6 +6664,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         if (sev_enabled()) {
             *eax = 0x2;
             *eax |= sev_es_enabled() ? 0x8 : 0;
+            *eax |= sev_snp_enabled() ? 0x10 : 0;
             *ebx = sev_get_cbit_position() & 0x3f; /* EBX[5:0] */
             *ebx |= (sev_get_reduced_phys_bits() & 0x3f) << 6; /* EBX[11:6] */
         }
Signature · 1 line
2.25.1
#67 lore raw

PATCH v3 31/49 i386/sev: Update query-sev QAPI format to handle SEV-SNP

In reply to Michael Roth

Most of the current 'query-sev' command is relevant to both legacy SEV/SEV-ES guests and SEV-SNP guests, with 2 exceptions:

- 'policy' is a 64-bit field for SEV-SNP, not 32-bit, and the meaning of the bit positions has changed
- 'handle' is not relevant to SEV-SNP

To address this, this patch adds a new 'sev-type' field that can be used as a discriminator to select between SEV and SEV-SNP-specific fields/formats without breaking compatibility for existing management tools (so long as management tools that add support for launching SEV-SNP guest update their handling of query-sev appropriately).

The corresponding HMP command has also been fixed up similarly.

---
 qapi/misc-target.json | 71 ++++++++++++++++++++++++++++++++++---------
 target/i386/sev.c     | 50 ++++++++++++++++++++----------
 target/i386/sev.h     |  3 ++
 3 files changed, 94 insertions(+), 30 deletions(-)
qapi/misc-target.json +57 -14
index 4e0a6492a9..daceb85d95 100644
@@ -47,6 +47,49 @@
            'send-update', 'receive-update' ],
   'if': 'TARGET_I386' }
 
+##
+# @SevGuestType:
+#
+# An enumeration indicating the type of SEV guest being run.
+#
+# @sev:     The guest is a legacy SEV or SEV-ES guest.
+# @sev-snp: The guest is an SEV-SNP guest.
+#
+# Since: 6.2
+##
+{ 'enum': 'SevGuestType',
+  'data': [ 'sev', 'sev-snp' ],
+  'if': 'TARGET_I386' }
+
+##
+# @SevGuestInfo:
+#
+# Information specific to legacy SEV/SEV-ES guests.
+#
+# @policy: SEV policy value
+#
+# @handle: SEV firmware handle
+#
+# Since: 2.12
+##
+{ 'struct': 'SevGuestInfo',
+  'data': { 'policy': 'uint32',
+            'handle': 'uint32' },
+  'if': 'TARGET_I386' }
+
+##
+# @SevSnpGuestInfo:
+#
+# Information specific to SEV-SNP guests.
+#
+# @snp-policy: SEV-SNP policy value
+#
+# Since: 6.2
+##
+{ 'struct': 'SevSnpGuestInfo',
+  'data': { 'snp-policy': 'uint64' },
+  'if': 'TARGET_I386' }
+
 ##
 # @SevInfo:
 #
@@ -60,25 +103,25 @@
 #
 # @build-id: SEV FW build id
 #
-# @policy: SEV policy value
-#
 # @state: SEV guest state
 #
-# @handle: SEV firmware handle
+# @sev-type: Type of SEV guest being run
 #
 # Since: 2.12
 ##
-{ 'struct': 'SevInfo',
-    'data': { 'enabled': 'bool',
-              'api-major': 'uint8',
-              'api-minor' : 'uint8',
-              'build-id' : 'uint8',
-              'policy' : 'uint32',
-              'state' : 'SevState',
-              'handle' : 'uint32'
-            },
-  'if': 'TARGET_I386'
-}
+{ 'union': 'SevInfo',
+  'base': { 'enabled': 'bool',
+            'api-major': 'uint8',
+            'api-minor' : 'uint8',
+            'build-id' : 'uint8',
+            'state' : 'SevState',
+            'sev-type' : 'SevGuestType' },
+  'discriminator': 'sev-type',
+  'data': {
+      'sev': 'SevGuestInfo',
+      'sev-snp': 'SevSnpGuestInfo' },
+  'if': 'TARGET_I386' }
+
 
 ##
 # @query-sev:
target/i386/sev.c +34 -16

79 lines not on this page · raw message

target/i386/sev.h +3 -0
index 94295ee74f..5dc4767b1e 100644
@@ -31,6 +31,9 @@
 #define SEV_POLICY_DOMAIN       0x10
 #define SEV_POLICY_SEV          0x20
 
+#define SEV_SNP_POLICY_SMT      0x10000
+#define SEV_SNP_POLICY_DBG      0x80000
+
 typedef struct SevKernelLoaderContext {
     char *setup_data;
     size_t setup_size;
Signature · 1 line
2.25.1
#68 lore raw

Re: [PATCH v3 31/49] i386/sev: Update query-sev QAPI format to handle SEV-SNP

Quoted 29 lines · Michael Roth

Most of the current 'query-sev' command is relevant to both legacy SEV/SEV-ES guests and SEV-SNP guests, with 2 exceptions:

- 'policy' is a 64-bit field for SEV-SNP, not 32-bit, and the meaning of the bit positions has changed
- 'handle' is not relevant to SEV-SNP

To address this, this patch adds a new 'sev-type' field that can be used as a discriminator to select between SEV and SEV-SNP-specific fields/formats without breaking compatibility for existing management tools (so long as management tools that add support for launching SEV-SNP guest update their handling of query-sev appropriately).

The corresponding HMP command has also been fixed up similarly.

---
 qapi/misc-target.json | 71 ++++++++++++++++++++++++++++++++++---------
 target/i386/sev.c     | 50 ++++++++++++++++++++----------
 target/i386/sev.h     |  3 ++
 3 files changed, 94 insertions(+), 30 deletions(-)
qapi/misc-target.json +0 -0
index 4e0a6492a9..daceb85d95 100644
@@ -47,6 +47,49 @@
            'send-update', 'receive-update' ],
   'if': 'TARGET_I386' }

qapi/misc-target.json line 58

Quoted 10 lines
qapi/misc-target.json +9 -0
 
+##
+# @SevGuestType:
+#
+# An enumeration indicating the type of SEV guest being run.
+#
+# @sev:     The guest is a legacy SEV or SEV-ES guest.
+# @sev-snp: The guest is an SEV-SNP guest.
+#
+# Since: 6.2

Now 9.1 at the earliest.

Quoted 23 lines
qapi/misc-target.json +23 -0
+##
+{ 'enum': 'SevGuestType',
+  'data': [ 'sev', 'sev-snp' ],
+  'if': 'TARGET_I386' }
+
+##
+# @SevGuestInfo:
+#
+# Information specific to legacy SEV/SEV-ES guests.
+#
+# @policy: SEV policy value
+#
+# @handle: SEV firmware handle
+#
+# Since: 2.12
+##
+{ 'struct': 'SevGuestInfo',
+  'data': { 'policy': 'uint32',
+            'handle': 'uint32' },
+  'if': 'TARGET_I386' }
+
+##
+# @SevSnpGuestInfo:

qapi/misc-target.json line 91

Quoted 10 lines
qapi/misc-target.json +10 -0
+#
+# Information specific to SEV-SNP guests.
+#
+# @snp-policy: SEV-SNP policy value
+#
+# Since: 6.2
+##
+{ 'struct': 'SevSnpGuestInfo',
+  'data': { 'snp-policy': 'uint64' },
+  'if': 'TARGET_I386' }

IMHO it can just be called 'policy' still, since
it is implicitly within a 'Snp' specific type.

Quoted 74 lines
qapi/misc-target.json +15 -14
+
 ##
 # @SevInfo:
 #
@@ -60,25 +103,25 @@
 #
 # @build-id: SEV FW build id
 #
-# @policy: SEV policy value
-#
 # @state: SEV guest state
 #
-# @handle: SEV firmware handle
+# @sev-type: Type of SEV guest being run
 #
 # Since: 2.12
 ##
-{ 'struct': 'SevInfo',
-    'data': { 'enabled': 'bool',
-              'api-major': 'uint8',
-              'api-minor' : 'uint8',
-              'build-id' : 'uint8',
-              'policy' : 'uint32',
-              'state' : 'SevState',
-              'handle' : 'uint32'
-            },
-  'if': 'TARGET_I386'
-}
+{ 'union': 'SevInfo',
+  'base': { 'enabled': 'bool',
+            'api-major': 'uint8',
+            'api-minor' : 'uint8',
+            'build-id' : 'uint8',
+            'state' : 'SevState',
+            'sev-type' : 'SevGuestType' },
+  'discriminator': 'sev-type',
+  'data': {
+      'sev': 'SevGuestInfo',
+      'sev-snp': 'SevSnpGuestInfo' },
+  'if': 'TARGET_I386' }
+
 
 ##
 # @query-sev:
target/i386/sev.c +3 -10
index 43e6c0172f..b03d70a3d1 100644
@@ -353,25 +353,27 @@ static SevInfo *sev_get_info(void)
 {
     SevInfo *info;
     SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
-    SevGuestState *sev_guest =
-        (SevGuestState *)object_dynamic_cast(OBJECT(sev_common),
-                                             TYPE_SEV_GUEST);
 
     info = g_new0(SevInfo, 1);
     info->enabled = sev_enabled();
 
     if (info->enabled) {
-        if (sev_guest) {
-            info->handle = sev_guest->handle;
-        }
         info->api_major = sev_common->api_major;
         info->api_minor = sev_common->api_minor;
         info->build_id = sev_common->build_id;
         info->state = sev_common->state;
-        /* we only report the lower 32-bits of policy for SNP, ok for now... */
-        info->policy =
-            (uint32_t)object_property_get_uint(OBJECT(sev_common),
-                                               "policy", NULL);
+
+        if (sev_snp_enabled()) {
+            info->sev_type = SEV_GUEST_TYPE_SEV_SNP;

target/i386/sev.c static SevInfo *sev_get_info(void) · line 377

Quoted 10 lines
target/i386/sev.c +9 -0
+            info->u.sev_snp.snp_policy =
+                object_property_get_uint(OBJECT(sev_common), "policy", NULL);
+        } else {
+            info->sev_type = SEV_GUEST_TYPE_SEV;
+            info->u.sev.handle = SEV_GUEST(sev_common)->handle;
+            info->u.sev.policy =
+                (uint32_t)object_property_get_uint(OBJECT(sev_common),
+                                                   "policy", NULL);
+        }
     }

Ok, this is fixing the issues I reported earlier.

For 'policy' I do wonder if we really need to push it into the type specific part of the union, as oppposed to having a common 'policy' field that is uint64 in size.

As mentioned earlier, on the wire there's no distinction between int32/int64s, so there's no compat issues with changing it from int32 to int64.

Quoted 24 lines
target/i386/sev.c +8 -5
     return info;
@@ -394,20 +396,36 @@ void hmp_info_sev(Monitor *mon, const QDict *qdict)
 {
     SevInfo *info = sev_get_info();
 
-    if (info && info->enabled) {
-        monitor_printf(mon, "handle: %d\n", info->handle);
+    if (!info || !info->enabled) {
+        monitor_printf(mon, "SEV is not enabled\n");
+        goto out;
+    }
+
+    if (sev_snp_enabled()) {
         monitor_printf(mon, "state: %s\n", SevState_str(info->state));
         monitor_printf(mon, "build: %d\n", info->build_id);
         monitor_printf(mon, "api version: %d.%d\n",
                        info->api_major, info->api_minor);
         monitor_printf(mon, "debug: %s\n",
-                       info->policy & SEV_POLICY_NODBG ? "off" : "on");
-        monitor_printf(mon, "key-sharing: %s\n",
-                       info->policy & SEV_POLICY_NOKS ? "off" : "on");
+                       info->u.sev_snp.snp_policy & SEV_SNP_POLICY_DBG ? "on"
+                                                                       : "off");

target/i386/sev.c void hmp_info_sev(Monitor *mon, const QDict *qdict) · line 420

Quoted 10 lines
target/i386/sev.c +8 -1
+        monitor_printf(mon, "SMT allowed: %s\n",
+                       info->u.sev_snp.snp_policy & SEV_SNP_POLICY_SMT ? "on"
+                                                                       : "off");
     } else {
-        monitor_printf(mon, "SEV is not enabled\n");
+        monitor_printf(mon, "handle: %d\n", info->u.sev.handle);
+        monitor_printf(mon, "state: %s\n", SevState_str(info->state));
+        monitor_printf(mon, "build: %d\n", info->build_id);
+        monitor_printf(mon, "api version: %d.%d\n",
+                       info->api_major, info->api_minor);

This set of three fields is identical in both branches, so the duplication in printing it should be eliminated.

target/i386/sev.c void hmp_info_sev(Monitor *mon, const QDict *qdict) · line 426

Quoted 6 lines
target/i386/sev.c +5 -0
+        monitor_printf(mon, "debug: %s\n",
+                       info->u.sev.policy & SEV_POLICY_NODBG ? "off" : "on");
+        monitor_printf(mon, "key-sharing: %s\n",
+                       info->u.sev.policy & SEV_POLICY_NOKS ? "off" : "on");
     }
+    monitor_printf(mon, "SEV type: %s\n", SevGuestType_str(info->sev_type));

I'd say 'SEV type' should be printed before everything else, since that value is the discriminator for interpreting the other data that is printed.

target/i386/sev.c void hmp_info_sev(Monitor *mon, const QDict *qdict) · line 430

Quoted 4 lines
target/i386/sev.c +1 -0
+out:
     qapi_free_SevInfo(info);
 }

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#69 lore raw

Re: [PATCH v3 31/49] i386/sev: Update query-sev QAPI format to handle SEV-SNP

Quoted 168 lines · Daniel P. Berrangé
Quoted 29 lines · Michael Roth

Most of the current 'query-sev' command is relevant to both legacy SEV/SEV-ES guests and SEV-SNP guests, with 2 exceptions:

- 'policy' is a 64-bit field for SEV-SNP, not 32-bit, and the meaning of the bit positions has changed
- 'handle' is not relevant to SEV-SNP

To address this, this patch adds a new 'sev-type' field that can be used as a discriminator to select between SEV and SEV-SNP-specific fields/formats without breaking compatibility for existing management tools (so long as management tools that add support for launching SEV-SNP guest update their handling of query-sev appropriately).

The corresponding HMP command has also been fixed up similarly.

---
 qapi/misc-target.json | 71 ++++++++++++++++++++++++++++++++++---------
 target/i386/sev.c     | 50 ++++++++++++++++++++----------
 target/i386/sev.h     |  3 ++
 3 files changed, 94 insertions(+), 30 deletions(-)
qapi/misc-target.json +0 -0
index 4e0a6492a9..daceb85d95 100644
@@ -47,6 +47,49 @@
            'send-update', 'receive-update' ],
   'if': 'TARGET_I386' }

qapi/misc-target.json line 58

Quoted 10 lines
qapi/misc-target.json +9 -0
 
+##
+# @SevGuestType:
+#
+# An enumeration indicating the type of SEV guest being run.
+#
+# @sev:     The guest is a legacy SEV or SEV-ES guest.
+# @sev-snp: The guest is an SEV-SNP guest.
+#
+# Since: 6.2

Now 9.1 at the earliest.

Quoted 23 lines
qapi/misc-target.json +23 -0
+##
+{ 'enum': 'SevGuestType',
+  'data': [ 'sev', 'sev-snp' ],
+  'if': 'TARGET_I386' }
+
+##
+# @SevGuestInfo:
+#
+# Information specific to legacy SEV/SEV-ES guests.
+#
+# @policy: SEV policy value
+#
+# @handle: SEV firmware handle
+#
+# Since: 2.12
+##
+{ 'struct': 'SevGuestInfo',
+  'data': { 'policy': 'uint32',
+            'handle': 'uint32' },
+  'if': 'TARGET_I386' }
+
+##
+# @SevSnpGuestInfo:

qapi/misc-target.json line 91

Quoted 10 lines
qapi/misc-target.json +10 -0
+#
+# Information specific to SEV-SNP guests.
+#
+# @snp-policy: SEV-SNP policy value
+#
+# Since: 6.2
+##
+{ 'struct': 'SevSnpGuestInfo',
+  'data': { 'snp-policy': 'uint64' },
+  'if': 'TARGET_I386' }

IMHO it can just be called 'policy' still, since
it is implicitly within a 'Snp' specific type.

Quoted 74 lines
qapi/misc-target.json +15 -14
+
 ##
 # @SevInfo:
 #
@@ -60,25 +103,25 @@
 #
 # @build-id: SEV FW build id
 #
-# @policy: SEV policy value
-#
 # @state: SEV guest state
 #
-# @handle: SEV firmware handle
+# @sev-type: Type of SEV guest being run
 #
 # Since: 2.12
 ##
-{ 'struct': 'SevInfo',
-    'data': { 'enabled': 'bool',
-              'api-major': 'uint8',
-              'api-minor' : 'uint8',
-              'build-id' : 'uint8',
-              'policy' : 'uint32',
-              'state' : 'SevState',
-              'handle' : 'uint32'
-            },
-  'if': 'TARGET_I386'
-}
+{ 'union': 'SevInfo',
+  'base': { 'enabled': 'bool',
+            'api-major': 'uint8',
+            'api-minor' : 'uint8',
+            'build-id' : 'uint8',
+            'state' : 'SevState',
+            'sev-type' : 'SevGuestType' },
+  'discriminator': 'sev-type',
+  'data': {
+      'sev': 'SevGuestInfo',
+      'sev-snp': 'SevSnpGuestInfo' },
+  'if': 'TARGET_I386' }
+
 
 ##
 # @query-sev:
target/i386/sev.c +3 -10
index 43e6c0172f..b03d70a3d1 100644
@@ -353,25 +353,27 @@ static SevInfo *sev_get_info(void)
 {
     SevInfo *info;
     SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
-    SevGuestState *sev_guest =
-        (SevGuestState *)object_dynamic_cast(OBJECT(sev_common),
-                                             TYPE_SEV_GUEST);
 
     info = g_new0(SevInfo, 1);
     info->enabled = sev_enabled();
 
     if (info->enabled) {
-        if (sev_guest) {
-            info->handle = sev_guest->handle;
-        }
         info->api_major = sev_common->api_major;
         info->api_minor = sev_common->api_minor;
         info->build_id = sev_common->build_id;
         info->state = sev_common->state;
-        /* we only report the lower 32-bits of policy for SNP, ok for now... */
-        info->policy =
-            (uint32_t)object_property_get_uint(OBJECT(sev_common),
-                                               "policy", NULL);
+
+        if (sev_snp_enabled()) {
+            info->sev_type = SEV_GUEST_TYPE_SEV_SNP;

target/i386/sev.c static SevInfo *sev_get_info(void) · line 377

Quoted 10 lines
target/i386/sev.c +9 -0
+            info->u.sev_snp.snp_policy =
+                object_property_get_uint(OBJECT(sev_common), "policy", NULL);
+        } else {
+            info->sev_type = SEV_GUEST_TYPE_SEV;
+            info->u.sev.handle = SEV_GUEST(sev_common)->handle;
+            info->u.sev.policy =
+                (uint32_t)object_property_get_uint(OBJECT(sev_common),
+                                                   "policy", NULL);
+        }
     }

Ok, this is fixing the issues I reported earlier.

Quoted 7 lines

For 'policy' I do wonder if we really need to push it into the type specific part of the union, as oppposed to having a common 'policy' field that is uint64 in size.

As mentioned earlier, on the wire there's no distinction between int32/int64s, so there's no compat issues with changing it from int32 to int64.

Mentioned this in response to your earlier reply, but the renaming of 'policy' to 'snp_policy' was in response to Dave's query about old management tools who don't understand/parse the new sev-type field and blinding misinterpret an SNP 'policy' as an SEV one:

https://lore.kernel.org/kvm/YTdSlg5NymDQex5T@work-vm/T/#mac6758af9bfc41ee49ff3ef5c3ec3779ec275ff9

I think the thinking is that it's better that they see nothing at all (likely treating as policy 0x0) versus getting unexpected values and reporting random policy features.

Quoted 30 lines
Quoted 30 lines
target/i386/sev.c +11 -6
     return info;
@@ -394,20 +396,36 @@ void hmp_info_sev(Monitor *mon, const QDict *qdict)
 {
     SevInfo *info = sev_get_info();
 
-    if (info && info->enabled) {
-        monitor_printf(mon, "handle: %d\n", info->handle);
+    if (!info || !info->enabled) {
+        monitor_printf(mon, "SEV is not enabled\n");
+        goto out;
+    }
+
+    if (sev_snp_enabled()) {
         monitor_printf(mon, "state: %s\n", SevState_str(info->state));
         monitor_printf(mon, "build: %d\n", info->build_id);
         monitor_printf(mon, "api version: %d.%d\n",
                        info->api_major, info->api_minor);
         monitor_printf(mon, "debug: %s\n",
-                       info->policy & SEV_POLICY_NODBG ? "off" : "on");
-        monitor_printf(mon, "key-sharing: %s\n",
-                       info->policy & SEV_POLICY_NOKS ? "off" : "on");
+                       info->u.sev_snp.snp_policy & SEV_SNP_POLICY_DBG ? "on"
+                                                                       : "off");
+        monitor_printf(mon, "SMT allowed: %s\n",
+                       info->u.sev_snp.snp_policy & SEV_SNP_POLICY_SMT ? "on"
+                                                                       : "off");
     } else {
-        monitor_printf(mon, "SEV is not enabled\n");
Quoted 8 lines

target/i386/sev.c void hmp_info_sev(Monitor *mon, const QDict *qdict) · line 420

Quoted 5 lines
target/i386/sev.c +5 -0
+        monitor_printf(mon, "handle: %d\n", info->u.sev.handle);
+        monitor_printf(mon, "state: %s\n", SevState_str(info->state));
+        monitor_printf(mon, "build: %d\n", info->build_id);
+        monitor_printf(mon, "api version: %d.%d\n",
+                       info->api_major, info->api_minor);

This set of three fields is identical in both branches, so the duplication in printing it should be eliminated.

Makes sense.

Quoted 11 lines

target/i386/sev.c void hmp_info_sev(Monitor *mon, const QDict *qdict) · line 426

Quoted 7 lines
target/i386/sev.c +5 -0
+        monitor_printf(mon, "debug: %s\n",
+                       info->u.sev.policy & SEV_POLICY_NODBG ? "off" : "on");
+        monitor_printf(mon, "key-sharing: %s\n",
+                       info->u.sev.policy & SEV_POLICY_NOKS ? "off" : "on");
     }
+    monitor_printf(mon, "SEV type: %s\n", SevGuestType_str(info->sev_type));

I'd say 'SEV type' should be printed before everything else, since that value is the discriminator for interpreting the other data that is printed.

Will do.

Thanks,

Mike

Quoted 13 lines

target/i386/sev.c void hmp_info_sev(Monitor *mon, const QDict *qdict) · line 430

Quoted 5 lines
target/i386/sev.c +1 -0
+out:
     qapi_free_SevInfo(info);
 }

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#70 lore raw

Re: [PATCH v3 31/49] i386/sev: Update query-sev QAPI format to handle SEV-SNP

In reply to Michael Roth
Quoted 27 lines · Michael Roth

Most of the current 'query-sev' command is relevant to both legacy SEV/SEV-ES guests and SEV-SNP guests, with 2 exceptions:

- 'policy' is a 64-bit field for SEV-SNP, not 32-bit, and the meaning of the bit positions has changed
- 'handle' is not relevant to SEV-SNP

To address this, this patch adds a new 'sev-type' field that can be used as a discriminator to select between SEV and SEV-SNP-specific fields/formats without breaking compatibility for existing management tools (so long as management tools that add support for launching SEV-SNP guest update their handling of query-sev appropriately).

The corresponding HMP command has also been fixed up similarly.

---
 qapi/misc-target.json | 71 ++++++++++++++++++++++++++++++++++---------
 target/i386/sev.c     | 50 ++++++++++++++++++++----------
 target/i386/sev.h     |  3 ++
 3 files changed, 94 insertions(+), 30 deletions(-)
qapi/misc-target.json +0 -0
index 4e0a6492a9..daceb85d95 100644

qapi/misc-target.json line 55

Quoted 9 lines
qapi/misc-target.json +6 -0
@@ -47,6 +47,49 @@
            'send-update', 'receive-update' ],
   'if': 'TARGET_I386' }
 
+##
+# @SevGuestType:
+#
+# An enumeration indicating the type of SEV guest being run.
+#
+# @sev:     The guest is a legacy SEV or SEV-ES guest.

Single space after ':', please.

Recommend a blank line between argument descriptions.

qapi/misc-target.json line 58

Quoted 3 lines
qapi/misc-target.json +3 -0
+# @sev-snp: The guest is an SEV-SNP guest.
+#
+# Since: 6.2

The type is since 9.1, but its members become results of query-sev, where they are since 2.12. See also my reply to Daniel's question on PATCH 21.

qapi/misc-target.json line 69

Quoted 11 lines
qapi/misc-target.json +11 -0
+##
+{ 'enum': 'SevGuestType',
+  'data': [ 'sev', 'sev-snp' ],
+  'if': 'TARGET_I386' }
+
+##
+# @SevGuestInfo:
+#
+# Information specific to legacy SEV/SEV-ES guests.
+#
+# @policy: SEV policy value

I know you're just moving existing documentation. Still: this is rather sparse. Where would I find what numbers to pass for @policy?

Quoted 6 lines
qapi/misc-target.json +6 -0
+#
+# @handle: SEV firmware handle
+#
+# Since: 2.12
+##
+{ 'struct': 'SevGuestInfo',

qapi/misc-target.json line 85

Quoted 10 lines
qapi/misc-target.json +10 -0
+  'data': { 'policy': 'uint32',
+            'handle': 'uint32' },
+  'if': 'TARGET_I386' }
+
+##
+# @SevSnpGuestInfo:
+#
+# Information specific to SEV-SNP guests.
+#
+# @snp-policy: SEV-SNP policy value

Same question.

qapi/misc-target.json line 87

Quoted 2 lines
qapi/misc-target.json +2 -0
+#
+# Since: 6.2

9.1

Quoted 48 lines
qapi/misc-target.json +19 -14
+##
+{ 'struct': 'SevSnpGuestInfo',
+  'data': { 'snp-policy': 'uint64' },
+  'if': 'TARGET_I386' }
+
 ##
 # @SevInfo:
 #
@@ -60,25 +103,25 @@
 #
 # @build-id: SEV FW build id
 #
-# @policy: SEV policy value
-#
 # @state: SEV guest state
 #
-# @handle: SEV firmware handle
+# @sev-type: Type of SEV guest being run
 #
 # Since: 2.12
 ##
-{ 'struct': 'SevInfo',
-    'data': { 'enabled': 'bool',
-              'api-major': 'uint8',
-              'api-minor' : 'uint8',
-              'build-id' : 'uint8',
-              'policy' : 'uint32',
-              'state' : 'SevState',
-              'handle' : 'uint32'
-            },
-  'if': 'TARGET_I386'
-}
+{ 'union': 'SevInfo',
+  'base': { 'enabled': 'bool',
+            'api-major': 'uint8',
+            'api-minor' : 'uint8',
+            'build-id' : 'uint8',
+            'state' : 'SevState',
+            'sev-type' : 'SevGuestType' },
+  'discriminator': 'sev-type',
+  'data': {
+      'sev': 'SevGuestInfo',
+      'sev-snp': 'SevSnpGuestInfo' },
+  'if': 'TARGET_I386' }
+
 
 ##
 # @query-sev:

[...]

#71 lore raw

PATCH v3 32/49 i386/sev: Don't return launch measurements for SEV-SNP guests

In reply to Michael Roth

For SEV-SNP guests, launch measurement is queried from within the guest during attestation, so don't attempt to return it as part of query-sev-launch-measure.

---
 target/i386/sev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
target/i386/sev.c +3 -1
index b03d70a3d1..0c8e4bdb4c 100644
@@ -803,7 +803,9 @@ sev_launch_get_measure(Notifier *notifier, void *unused)
 
 static char *sev_get_launch_measurement(void)
 {
-    SevGuestState *sev_guest = SEV_GUEST(MACHINE(qdev_get_machine())->cgs);
+    ConfidentialGuestSupport *cgs = MACHINE(qdev_get_machine())->cgs;
+    SevGuestState *sev_guest =
+        (SevGuestState *)object_dynamic_cast(OBJECT(cgs), TYPE_SEV_GUEST);
 
     if (sev_guest &&
         SEV_COMMON(sev_guest)->state >= SEV_STATE_LAUNCH_SECRET) {
Signature · 1 line
2.25.1
#72 lore raw

Re: [PATCH v3 32/49] i386/sev: Don't return launch measurements for SEV-SNP guests

Quoted 14 lines · Michael Roth

For SEV-SNP guests, launch measurement is queried from within the guest during attestation, so don't attempt to return it as part of query-sev-launch-measure.

---
 target/i386/sev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
target/i386/sev.c +0 -0
index b03d70a3d1..0c8e4bdb4c 100644

target/i386/sev.c sev_launch_get_measure(Notifier *notifier, void *unused) · line 811

Quoted 10 lines
target/i386/sev.c +3 -1
@@ -803,7 +803,9 @@ sev_launch_get_measure(Notifier *notifier, void *unused)
 
 static char *sev_get_launch_measurement(void)
 {
-    SevGuestState *sev_guest = SEV_GUEST(MACHINE(qdev_get_machine())->cgs);
+    ConfidentialGuestSupport *cgs = MACHINE(qdev_get_machine())->cgs;
+    SevGuestState *sev_guest =
+        (SevGuestState *)object_dynamic_cast(OBJECT(cgs), TYPE_SEV_GUEST);
 
     if (sev_guest &&
         SEV_COMMON(sev_guest)->state >= SEV_STATE_LAUNCH_SECRET) {

The QAPI docs for query-sev-launch-measurement should be updated to reflect that this command is only valid to call for SEV/SEV-ES, not SNP.

The error reoprting in qmp_query_sev_launch_measure leaves a little to be desired just giving a generic message

"SEV launch measurement is not available"

I think that this sev_get_launch_measurement method should report a more fine grained error, to distinguish

* Unavailable because we're not a SEV/SEV-ES guest
* Unavailable because the guest hasn't reached launch state

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#73 lore raw

Re: [PATCH v3 32/49] i386/sev: Don't return launch measurements for SEV-SNP guests

Quoted 23 lines · Daniel P. Berrangé
Quoted 22 lines · Michael Roth

For SEV-SNP guests, launch measurement is queried from within the guest during attestation, so don't attempt to return it as part of query-sev-launch-measure.

---
 target/i386/sev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
target/i386/sev.c +3 -1
index b03d70a3d1..0c8e4bdb4c 100644
@@ -803,7 +803,9 @@ sev_launch_get_measure(Notifier *notifier, void *unused)
 
 static char *sev_get_launch_measurement(void)
 {
-    SevGuestState *sev_guest = SEV_GUEST(MACHINE(qdev_get_machine())->cgs);
+    ConfidentialGuestSupport *cgs = MACHINE(qdev_get_machine())->cgs;
+    SevGuestState *sev_guest =
+        (SevGuestState *)object_dynamic_cast(OBJECT(cgs), TYPE_SEV_GUEST);
 
Quoted 6 lines

target/i386/sev.c sev_launch_get_measure(Notifier *notifier, void *unused) · line 811

Quoted 2 lines
target/i386/sev.c +0 -0
     if (sev_guest &&
         SEV_COMMON(sev_guest)->state >= SEV_STATE_LAUNCH_SECRET) {

The QAPI docs for query-sev-launch-measurement should be updated to reflect that this command is only valid to call for SEV/SEV-ES, not SNP.

Also, the same question about whether query-sev-attestation-report and sev-inject-launch-secret need updating to declare them SEV/SEV-ES only, or if they are expected work with SNP too ?

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#74 lore raw

PATCH v3 33/49 kvm: Make kvm_convert_memory() non-static

In reply to Michael Roth
---
 accel/kvm/kvm-all.c  | 2 +-
 include/sysemu/kvm.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
accel/kvm/kvm-all.c +1 -1
index b5872fdc07..bf0ae0c8ad 100644
@@ -2913,7 +2913,7 @@ static void kvm_eat_signals(CPUState *cpu)
     } while (sigismember(&chkset, SIG_IPI));
 }
 
-static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
+int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
 {
     MemoryRegionSection section;
     ram_addr_t offset;
include/sysemu/kvm.h +2 -0
index 2cb3192509..698f1640fe 100644
@@ -541,4 +541,6 @@ int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp);
 
 int kvm_set_memory_attributes_private(hwaddr start, hwaddr size);
 int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size);
+
+int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private);
 #endif
Signature · 1 line
2.25.1
#75 lore raw

PATCH v3 34/49 i386/sev: Add KVM_EXIT_VMGEXIT handling for Page State Changes

In reply to Michael Roth

When running SEV-SNP guests, the kernel may forward some subset of VMGEXIT-based guest hypercalls to userspace. One of these is for Page State Change requests, as documented by the GHCB specification[1].

Userspace does not directly have control over the SNP RMP table to actually satisfy these requests, but will instead make use of the kvm_convert_memory() interface, which makes use of the KVM_SET_MEMORY_ATTRIBUTES ioctl to instruct KVM to map these these GPAs using private/shared memory and make the appropriate RMP changes via the associated kernel hooks.

Add the basic infrastructure for handling KVM_EXIT_VMGEXIT events, and then implement handling for Page State Change requests on top of that.

[1] https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/specifications/56421.pdf

---
 target/i386/kvm/kvm.c |   3 +
 target/i386/sev.c     | 152 ++++++++++++++++++++++++++++++++++++++++++
 target/i386/sev.h     |   2 +
 3 files changed, 157 insertions(+)
target/i386/kvm/kvm.c +3 -0
index 59e9048e61..22eb21a2f3 100644
@@ -5409,6 +5409,9 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
         ret = kvm_xen_handle_exit(cpu, &run->xen);
         break;
 #endif
+    case KVM_EXIT_VMGEXIT:
+        ret = kvm_handle_vmgexit(run);
+        break;
     default:
         fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
         ret = -1;
target/i386/sev.c +152 -0

158 lines not on this page · raw message

target/i386/sev.h +2 -0
index 5dc4767b1e..5cbfc3365b 100644
@@ -66,4 +66,6 @@ int sev_inject_launch_secret(const char *hdr, const char *secret,
 int sev_es_save_reset_vector(void *flash_ptr, uint64_t flash_size);
 void sev_es_set_reset_vector(CPUState *cpu);
 
+int kvm_handle_vmgexit(struct kvm_run *run);
+
 #endif
Signature · 1 line
2.25.1
#76 lore raw

PATCH v3 35/49 i386/sev: Add KVM_EXIT_VMGEXIT handling for Page State Changes (MSR-based)

In reply to Michael Roth

SEV-SNP guests might issue MSR-based Page State Changes for situations like early boot where it might not be easily able to make use of a GHCB page to issue the request. Just as with GHCB-based Page State Changes, these are forwarded to userspace as KVM_EXIT_VMGEXITs. Add handling for these.

---
 target/i386/sev.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
target/i386/sev.c +16 -0
index 0c6a253138..b54422b28e 100644
@@ -1560,6 +1560,18 @@ out_unmap:
     return 0;
 }
 
+static int kvm_handle_vmgexit_psc_msr_protocol(__u64 gpa, __u8 op, __u32 *psc_ret)
+{
+    int ret;
+
+    ret = kvm_convert_memory(gpa, TARGET_PAGE_SIZE,
+                             op == KVM_USER_VMGEXIT_PSC_MSR_OP_PRIVATE);
+
+    *psc_ret = ret;
+
+    return ret;
+}
+
 int kvm_handle_vmgexit(struct kvm_run *run)
 {
     int ret;
@@ -1567,6 +1579,10 @@ int kvm_handle_vmgexit(struct kvm_run *run)
     if (run->vmgexit.type == KVM_USER_VMGEXIT_PSC) {
         ret = kvm_handle_vmgexit_psc(run->vmgexit.psc.shared_gpa,
                                      &run->vmgexit.psc.ret);
+    } else if (run->vmgexit.type == KVM_USER_VMGEXIT_PSC_MSR) {
+        ret = kvm_handle_vmgexit_psc_msr_protocol(run->vmgexit.psc_msr.gpa,
+                                                  run->vmgexit.psc_msr.op,
+                                                  &run->vmgexit.psc_msr.ret);
     } else {
         warn_report("KVM: unknown vmgexit type: %d", run->vmgexit.type);
         ret = -1;
Signature · 1 line
2.25.1
#77 lore raw

PATCH v3 36/49 i386/sev: Add KVM_EXIT_VMGEXIT handling for Extended Guest Requests

In reply to Michael Roth

The GHCB specification[1] defines a VMGEXIT-based Guest Request hypercall to allow an SNP guest to issue encrypted requests directly to SNP firmware to do things like query the attestation report for the guest. These are generally handled purely in the kernel.

In some some cases, it's useful for the host to be able to additionally supply the certificate chain for the signing key that SNP firmware uses to sign these attestation reports. To allow for, the GHCB specification defines an Extended Guest Request where this certificate data can be provided in a special format described in the GHCB spec. This certificate data may be global or guest-specific depending on how the guest was configured. Rather than providing interfaces to manage these within the kernel, KVM handles this by forward the Extended Guest Requests on to userspace so the certificate data can be provided in the expected format.

Add a certs-path parameter to the sev-snp-guest object so that it can be used to inject any certificate data into these Extended Guest Requests.

---
 qapi/qom.json     |  7 +++-
 target/i386/sev.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 91 insertions(+), 1 deletion(-)
qapi/qom.json +6 -1
index b25a3043da..7ba778af91 100644
@@ -957,6 +957,10 @@
 #             SNP_LAUNCH_FINISH command in the SEV-SNP firmware ABI
 #             (default: all-zero)
 #
+# @certs-path: path to certificate data that can be passed to guests via
+#              SNP Extended Guest Requests. File should be in the format
+#              described in the GHCB specification. (default: none)
+#
 # Since: 7.2
 ##
 { 'struct': 'SevSnpGuestProperties',
@@ -967,7 +971,8 @@
             '*id-block': 'str',
             '*id-auth': 'str',
             '*auth-key-enabled': 'bool',
-            '*host-data': 'str' } }
+            '*host-data': 'str',
+            '*certs-path': 'str' } }
 
 ##
 # @ThreadContextProperties:
target/i386/sev.c +85 -0

115 lines not on this page · raw message

Signature · 1 line
2.25.1
#78 lore raw

Re: [PATCH v3 36/49] i386/sev: Add KVM_EXIT_VMGEXIT handling for Extended Guest Requests

Quoted 31 lines · Michael Roth

The GHCB specification[1] defines a VMGEXIT-based Guest Request hypercall to allow an SNP guest to issue encrypted requests directly to SNP firmware to do things like query the attestation report for the guest. These are generally handled purely in the kernel.

In some some cases, it's useful for the host to be able to additionally supply the certificate chain for the signing key that SNP firmware uses to sign these attestation reports. To allow for, the GHCB specification defines an Extended Guest Request where this certificate data can be provided in a special format described in the GHCB spec. This certificate data may be global or guest-specific depending on how the guest was configured. Rather than providing interfaces to manage these within the kernel, KVM handles this by forward the Extended Guest Requests on to userspace so the certificate data can be provided in the expected format.

Add a certs-path parameter to the sev-snp-guest object so that it can be used to inject any certificate data into these Extended Guest Requests.

---
 qapi/qom.json     |  7 +++-
 target/i386/sev.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 91 insertions(+), 1 deletion(-)
qapi/qom.json +0 -0
index b25a3043da..7ba778af91 100644

qapi/qom.json line 962

Quoted 6 lines
qapi/qom.json +3 -0
@@ -957,6 +957,10 @@
 #             SNP_LAUNCH_FINISH command in the SEV-SNP firmware ABI
 #             (default: all-zero)
 #
+# @certs-path: path to certificate data that can be passed to guests via
+#              SNP Extended Guest Requests. File should be in the format
+#              described in the GHCB specification. (default: none)

Is this a filename, or is it a search path of sorts?

Quoted 14 lines
qapi/qom.json +3 -1
+#
 # Since: 7.2
 ##
 { 'struct': 'SevSnpGuestProperties',
@@ -967,7 +971,8 @@
             '*id-block': 'str',
             '*id-auth': 'str',
             '*auth-key-enabled': 'bool',
-            '*host-data': 'str' } }
+            '*host-data': 'str',
+            '*certs-path': 'str' } }
 
 ##
 # @ThreadContextProperties:

[...]

#79 lore raw

PATCH v3 37/49 i386/sev: Add the SNP launch start context

In reply to Michael Roth

From: Brijesh Singh <[email protected]>

The SNP_LAUNCH_START is called first to create a cryptographic launch context within the firmware.

---
 target/i386/sev.c        | 42 +++++++++++++++++++++++++++++++++++++++-
 target/i386/trace-events |  1 +
 2 files changed, 42 insertions(+), 1 deletion(-)
target/i386/sev.c +41 -1
index 3b4dbc63b1..9f63a41f08 100644
@@ -39,6 +39,7 @@
 #include "confidential-guest.h"
 #include "hw/i386/pc.h"
 #include "exec/address-spaces.h"
+#include "qemu/queue.h"
 
 OBJECT_DECLARE_SIMPLE_TYPE(SevCommonState, SEV_COMMON)
 OBJECT_DECLARE_SIMPLE_TYPE(SevGuestState, SEV_GUEST)
@@ -106,6 +107,16 @@ struct SevSnpGuestState {
 #define DEFAULT_SEV_DEVICE      "/dev/sev"
 #define DEFAULT_SEV_SNP_POLICY  0x30000
 
+typedef struct SevLaunchUpdateData {
+    QTAILQ_ENTRY(SevLaunchUpdateData) next;
+    hwaddr gpa;
+    void *hva;
+    uint64_t len;
+    int type;
+} SevLaunchUpdateData;
+
+static QTAILQ_HEAD(, SevLaunchUpdateData) launch_update;
+
 #define SEV_INFO_BLOCK_GUID     "00f771de-1a7e-4fcb-890e-68c77e2fb44e"
 typedef struct __attribute__((__packed__)) SevInfoBlock {
     /* SEV-ES Reset Vector Address */
@@ -668,6 +679,30 @@ sev_read_file_base64(const char *filename, guchar **data, gsize *len)
     return 0;
 }
 
+static int
+sev_snp_launch_start(SevSnpGuestState *sev_snp_guest)
+{
+    int fw_error, rc;
+    SevCommonState *sev_common = SEV_COMMON(sev_snp_guest);
+    struct kvm_sev_snp_launch_start *start = &sev_snp_guest->kvm_start_conf;
+
+    trace_kvm_sev_snp_launch_start(start->policy, sev_snp_guest->guest_visible_workarounds);
+
+    rc = sev_ioctl(sev_common->sev_fd, KVM_SEV_SNP_LAUNCH_START,
+                   start, &fw_error);
+    if (rc < 0) {
+        error_report("%s: SNP_LAUNCH_START ret=%d fw_error=%d '%s'",
+                __func__, rc, fw_error, fw_error_to_str(fw_error));
+        return 1;
+    }
+
+    QTAILQ_INIT(&launch_update);
+
+    sev_set_guest_state(sev_common, SEV_STATE_LAUNCH_UPDATE);
+
+    return 0;
+}
+
 static int
 sev_launch_start(SevGuestState *sev_guest)
 {
@@ -1007,7 +1042,12 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         goto err;
     }
 
-    ret = sev_launch_start(SEV_GUEST(sev_common));
+    if (sev_snp_enabled()) {
+        ret = sev_snp_launch_start(SEV_SNP_GUEST(sev_common));
+    } else {
+        ret = sev_launch_start(SEV_GUEST(sev_common));
+    }
+
     if (ret) {
         error_setg(errp, "%s: failed to create encryption context", __func__);
         goto err;
target/i386/trace-events +1 -0
index 2cd8726eeb..cb26d8a925 100644
@@ -11,3 +11,4 @@ kvm_sev_launch_measurement(const char *value) "data %s"
 kvm_sev_launch_finish(void) ""
 kvm_sev_launch_secret(uint64_t hpa, uint64_t hva, uint64_t secret, int len) "hpa 0x%" PRIx64 " hva 0x%" PRIx64 " data 0x%" PRIx64 " len %d"
 kvm_sev_attestation_report(const char *mnonce, const char *data) "mnonce %s data %s"
+kvm_sev_snp_launch_start(uint64_t policy, char *gosvw) "policy 0x%" PRIx64 " gosvw %s"
Signature · 1 line
2.25.1
maintainer
#80 lore raw

Re: [PATCH v3 37/49] i386/sev: Add the SNP launch start context

Quoted 73 lines · Michael Roth

From: Brijesh Singh <[email protected]>

The SNP_LAUNCH_START is called first to create a cryptographic launch context within the firmware.

---
  target/i386/sev.c        | 42 +++++++++++++++++++++++++++++++++++++++-
  target/i386/trace-events |  1 +
  2 files changed, 42 insertions(+), 1 deletion(-)
target/i386/sev.c +35 -0
index 3b4dbc63b1..9f63a41f08 100644
@@ -39,6 +39,7 @@
  #include "confidential-guest.h"
  #include "hw/i386/pc.h"
  #include "exec/address-spaces.h"
+#include "qemu/queue.h"
 
  OBJECT_DECLARE_SIMPLE_TYPE(SevCommonState, SEV_COMMON)
  OBJECT_DECLARE_SIMPLE_TYPE(SevGuestState, SEV_GUEST)
@@ -106,6 +107,16 @@ struct SevSnpGuestState {
  #define DEFAULT_SEV_DEVICE      "/dev/sev"
  #define DEFAULT_SEV_SNP_POLICY  0x30000
 
+typedef struct SevLaunchUpdateData {
+    QTAILQ_ENTRY(SevLaunchUpdateData) next;
+    hwaddr gpa;
+    void *hva;
+    uint64_t len;
+    int type;
+} SevLaunchUpdateData;
+
+static QTAILQ_HEAD(, SevLaunchUpdateData) launch_update;
+
  #define SEV_INFO_BLOCK_GUID     "00f771de-1a7e-4fcb-890e-68c77e2fb44e"
  typedef struct __attribute__((__packed__)) SevInfoBlock {
      /* SEV-ES Reset Vector Address */
@@ -668,6 +679,30 @@ sev_read_file_base64(const char *filename, guchar **data, gsize *len)
      return 0;
  }
 
+static int
+sev_snp_launch_start(SevSnpGuestState *sev_snp_guest)
+{
+    int fw_error, rc;
+    SevCommonState *sev_common = SEV_COMMON(sev_snp_guest);
+    struct kvm_sev_snp_launch_start *start = &sev_snp_guest->kvm_start_conf;
+
+    trace_kvm_sev_snp_launch_start(start->policy, sev_snp_guest->guest_visible_workarounds);
+
+    rc = sev_ioctl(sev_common->sev_fd, KVM_SEV_SNP_LAUNCH_START,
+                   start, &fw_error);
+    if (rc < 0) {
+        error_report("%s: SNP_LAUNCH_START ret=%d fw_error=%d '%s'",
+                __func__, rc, fw_error, fw_error_to_str(fw_error));
+        return 1;
+    }
+
+    QTAILQ_INIT(&launch_update);
+
+    sev_set_guest_state(sev_common, SEV_STATE_LAUNCH_UPDATE);
+
+    return 0;
+}
+
  static int
  sev_launch_start(SevGuestState *sev_guest)
  {

target/i386/sev.c static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 1049

Quoted 9 lines
target/i386/sev.c +5 -1
@@ -1007,7 +1042,12 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
          goto err;
      }
 
-    ret = sev_launch_start(SEV_GUEST(sev_common));
+    if (sev_snp_enabled()) {
+        ret = sev_snp_launch_start(SEV_SNP_GUEST(sev_common));
+    } else {
+        ret = sev_launch_start(SEV_GUEST(sev_common));
+    }

Instead of an "if", this should be a method in sev-common. Likewise for launch_finish in the next patch.

Also, patch 47 should introduce an "int (*launch_update_data)(hwaddr gpa, uint8_t *ptr, uint64_t len)" method whose implementation is either the existing sev_launch_update_data() for sev-guest, or a wrapper around snp_launch_update_data() (to add KVM_SEV_SNP_PAGE_TYPE_NORMAL) for sev-snp-guest.

In general, the only uses of sev_snp_enabled() should be in sev_add_kernel_loader_hashes() and kvm_handle_vmgexit_ext_req(). I would not be that strict for the QMP and HMP functions, but if you want to make those methods of sev-common I wouldn't complain.

Paolo

Quoted 12 lines
target/i386/sev.c +0 -0
      if (ret) {
          error_setg(errp, "%s: failed to create encryption context", __func__);
          goto err;
target/i386/trace-events +1 -0
index 2cd8726eeb..cb26d8a925 100644
@@ -11,3 +11,4 @@ kvm_sev_launch_measurement(const char *value) "data %s"
  kvm_sev_launch_finish(void) ""
  kvm_sev_launch_secret(uint64_t hpa, uint64_t hva, uint64_t secret, int len) "hpa 0x%" PRIx64 " hva 0x%" PRIx64 " data 0x%" PRIx64 " len %d"
  kvm_sev_attestation_report(const char *mnonce, const char *data) "mnonce %s data %s"
+kvm_sev_snp_launch_start(uint64_t policy, char *gosvw) "policy 0x%" PRIx64 " gosvw %s"
#81 lore raw

Re: [PATCH v3 37/49] i386/sev: Add the SNP launch start context

Quoted 74 lines · Paolo Bonzini
Quoted 73 lines · Michael Roth

From: Brijesh Singh <[email protected]>

The SNP_LAUNCH_START is called first to create a cryptographic launch context within the firmware.

---
  target/i386/sev.c        | 42 +++++++++++++++++++++++++++++++++++++++-
  target/i386/trace-events |  1 +
  2 files changed, 42 insertions(+), 1 deletion(-)
target/i386/sev.c +35 -1
index 3b4dbc63b1..9f63a41f08 100644
@@ -39,6 +39,7 @@
  #include "confidential-guest.h"
  #include "hw/i386/pc.h"
  #include "exec/address-spaces.h"
+#include "qemu/queue.h"
  OBJECT_DECLARE_SIMPLE_TYPE(SevCommonState, SEV_COMMON)
  OBJECT_DECLARE_SIMPLE_TYPE(SevGuestState, SEV_GUEST)
@@ -106,6 +107,16 @@ struct SevSnpGuestState {
  #define DEFAULT_SEV_DEVICE      "/dev/sev"
  #define DEFAULT_SEV_SNP_POLICY  0x30000
+typedef struct SevLaunchUpdateData {
+    QTAILQ_ENTRY(SevLaunchUpdateData) next;
+    hwaddr gpa;
+    void *hva;
+    uint64_t len;
+    int type;
+} SevLaunchUpdateData;
+
+static QTAILQ_HEAD(, SevLaunchUpdateData) launch_update;
+
  #define SEV_INFO_BLOCK_GUID     "00f771de-1a7e-4fcb-890e-68c77e2fb44e"
  typedef struct __attribute__((__packed__)) SevInfoBlock {
      /* SEV-ES Reset Vector Address */
@@ -668,6 +679,30 @@ sev_read_file_base64(const char *filename, guchar **data, gsize *len)
      return 0;
  }
+static int
+sev_snp_launch_start(SevSnpGuestState *sev_snp_guest)
+{
+    int fw_error, rc;
+    SevCommonState *sev_common = SEV_COMMON(sev_snp_guest);
+    struct kvm_sev_snp_launch_start *start = &sev_snp_guest->kvm_start_conf;
+
+    trace_kvm_sev_snp_launch_start(start->policy, sev_snp_guest->guest_visible_workarounds);
+
+    rc = sev_ioctl(sev_common->sev_fd, KVM_SEV_SNP_LAUNCH_START,
+                   start, &fw_error);
+    if (rc < 0) {
+        error_report("%s: SNP_LAUNCH_START ret=%d fw_error=%d '%s'",
+                __func__, rc, fw_error, fw_error_to_str(fw_error));
+        return 1;
+    }
+
+    QTAILQ_INIT(&launch_update);
+
+    sev_set_guest_state(sev_common, SEV_STATE_LAUNCH_UPDATE);
+
+    return 0;
+}
+
  static int
  sev_launch_start(SevGuestState *sev_guest)
  {
@@ -1007,7 +1042,12 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
          goto err;
      }
-    ret = sev_launch_start(SEV_GUEST(sev_common));
Quoted 8 lines

target/i386/sev.c static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 1049

Quoted 5 lines
target/i386/sev.c +5 -0
+    if (sev_snp_enabled()) {
+        ret = sev_snp_launch_start(SEV_SNP_GUEST(sev_common));
+    } else {
+        ret = sev_launch_start(SEV_GUEST(sev_common));
+    }

Instead of an "if", this should be a method in sev-common. Likewise for launch_finish in the next patch.

Makes sense.

Quoted 6 lines

Also, patch 47 should introduce an "int (*launch_update_data)(hwaddr gpa, uint8_t *ptr, uint64_t len)" method whose implementation is either the existing sev_launch_update_data() for sev-guest, or a wrapper around snp_launch_update_data() (to add KVM_SEV_SNP_PAGE_TYPE_NORMAL) for sev-snp-guest.

I suppose if we end up introducing an unused 'gpa' parameter in the case of sev_launch_update_data() that's still worth the change? Seems reasonable to me.

Quoted 5 lines

In general, the only uses of sev_snp_enabled() should be in sev_add_kernel_loader_hashes() and kvm_handle_vmgexit_ext_req(). I would not be that strict for the QMP and HMP functions, but if you want to make those methods of sev-common I wouldn't complain.

There's a good bit of duplication in those cases which is a little awkward to break out into a common helper. Will consider these as well though.

Thanks,

Mike

Quoted 16 lines

Paolo

Quoted 12 lines
target/i386/sev.c +0 -0
      if (ret) {
          error_setg(errp, "%s: failed to create encryption context", __func__);
          goto err;
target/i386/trace-events +1 -0
index 2cd8726eeb..cb26d8a925 100644
@@ -11,3 +11,4 @@ kvm_sev_launch_measurement(const char *value) "data %s"
  kvm_sev_launch_finish(void) ""
  kvm_sev_launch_secret(uint64_t hpa, uint64_t hva, uint64_t secret, int len) "hpa 0x%" PRIx64 " hva 0x%" PRIx64 " data 0x%" PRIx64 " len %d"
  kvm_sev_attestation_report(const char *mnonce, const char *data) "mnonce %s data %s"
+kvm_sev_snp_launch_start(uint64_t policy, char *gosvw) "policy 0x%" PRIx64 " gosvw %s"
maintainer
#82 lore raw

Re: [PATCH v3 37/49] i386/sev: Add the SNP launch start context

Il mer 20 mar 2024, 23:33 Michael Roth <[email protected]> ha scritto:

Quoted 99 lines · Paolo Bonzini
Quoted 10 lines · Michael Roth
Quoted 9 lines

From: Brijesh Singh <[email protected]>

The SNP_LAUNCH_START is called first to create a cryptographic launch context within the firmware.

---
  target/i386/sev.c        | 42

+++++++++++++++++++++++++++++++++++++++-

Quoted 20 lines
Quoted 20 lines
  target/i386/trace-events |  1 +
  2 files changed, 42 insertions(+), 1 deletion(-)
target/i386/sev.c +4 -0
index 3b4dbc63b1..9f63a41f08 100644
@@ -39,6 +39,7 @@
  #include "confidential-guest.h"
  #include "hw/i386/pc.h"
  #include "exec/address-spaces.h"
+#include "qemu/queue.h"
  OBJECT_DECLARE_SIMPLE_TYPE(SevCommonState, SEV_COMMON)
  OBJECT_DECLARE_SIMPLE_TYPE(SevGuestState, SEV_GUEST)
@@ -106,6 +107,16 @@ struct SevSnpGuestState {
  #define DEFAULT_SEV_DEVICE      "/dev/sev"
  #define DEFAULT_SEV_SNP_POLICY  0x30000
+typedef struct SevLaunchUpdateData {
+    QTAILQ_ENTRY(SevLaunchUpdateData) next;
+    hwaddr gpa;
Quoted 8 lines

target/i386/sev.c struct SevSnpGuestState {

Quoted 8 lines
target/i386/sev.c +7 -0
+    void *hva;
+    uint64_t len;
+    int type;
+} SevLaunchUpdateData;
+
+static QTAILQ_HEAD(, SevLaunchUpdateData) launch_update;
+
  #define SEV_INFO_BLOCK_GUID

"00f771de-1a7e-4fcb-890e-68c77e2fb44e"

Quoted 3 lines

target/i386/sev.c sev_read_file_base64(const char *filename, guchar

Quoted 3 lines
target/i386/sev.c +0 -0
  typedef struct __attribute__((__packed__)) SevInfoBlock {
      /* SEV-ES Reset Vector Address */
@@ -668,6 +679,30 @@ sev_read_file_base64(const char *filename, guchar

**data, gsize *len)

Quoted 8 lines

target/i386/sev.c sev_read_file_base64(const char *filename, guchar **data, gsize *len)

Quoted 8 lines
target/i386/sev.c +6 -0
      return 0;
  }
+static int
+sev_snp_launch_start(SevSnpGuestState *sev_snp_guest)
+{
+    int fw_error, rc;
+    SevCommonState *sev_common = SEV_COMMON(sev_snp_guest);
+    struct kvm_sev_snp_launch_start *start =

&sev_snp_guest->kvm_start_conf;

Quoted 2 lines

target/i386/sev.c sev_read_file_base64(const char *filename, guchar · line 688

Quoted 2 lines
target/i386/sev.c +2 -0
+
+    trace_kvm_sev_snp_launch_start(start->policy,

sev_snp_guest->guest_visible_workarounds);

Quoted 12 lines
Quoted 12 lines
target/i386/sev.c +12 -0
+
+    rc = sev_ioctl(sev_common->sev_fd, KVM_SEV_SNP_LAUNCH_START,
+                   start, &fw_error);
+    if (rc < 0) {
+        error_report("%s: SNP_LAUNCH_START ret=%d fw_error=%d '%s'",
+                __func__, rc, fw_error, fw_error_to_str(fw_error));
+        return 1;
+    }
+
+    QTAILQ_INIT(&launch_update);
+
+    sev_set_guest_state(sev_common, SEV_STATE_LAUNCH_UPDATE);
Quoted 8 lines

target/i386/sev.c static int

Quoted 8 lines
target/i386/sev.c +4 -0
+
+    return 0;
+}
+
  static int
  sev_launch_start(SevGuestState *sev_guest)
  {
@@ -1007,7 +1042,12 @@ static int

sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)

Quoted 11 lines

target/i386/sev.c static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 1049

Quoted 8 lines
target/i386/sev.c +5 -1
          goto err;
      }
-    ret = sev_launch_start(SEV_GUEST(sev_common));
+    if (sev_snp_enabled()) {
+        ret = sev_snp_launch_start(SEV_SNP_GUEST(sev_common));
+    } else {
+        ret = sev_launch_start(SEV_GUEST(sev_common));
+    }

Instead of an "if", this should be a method in sev-common. Likewise for launch_finish in the next patch.

Makes sense.

Quoted 6 lines

Also, patch 47 should introduce an "int (*launch_update_data)(hwaddr gpa, uint8_t *ptr, uint64_t len)" method whose implementation is either the existing sev_launch_update_data() for sev-guest, or a wrapper around snp_launch_update_data() (to add KVM_SEV_SNP_PAGE_TYPE_NORMAL) for sev-snp-guest.

Quoted 4 lines

I suppose if we end up introducing an unused 'gpa' parameter in the case of sev_launch_update_data() that's still worth the change? Seems reasonable to me.

Yeah, you most likely have the gpa anyway in the kind of board code that calls UPDATE_DATA. It would put some challenges with migration to use gpas everywhere instead of ram_addrs, but that doesn't use UPDATE_DATA and there's no SEV/SEV-ES migration anyway.

Quoted 11 lines
Quoted 6 lines

In general, the only uses of sev_snp_enabled() should be in sev_add_kernel_loader_hashes() and kvm_handle_vmgexit_ext_req(). I would not be that strict for the QMP and HMP functions, but if you want to make those methods of sev-common I wouldn't complain.

There's a good bit of duplication in those cases which is a little awkward to break out into a common helper. Will consider these as well though.

I would say especially in HMP do not bother since you have to start from a QAPI union and not QOM objects. For QMP I am not sure but don't waste too much effort in it. All I wanted to say is that the monitor is a fine place to draw the line.

Paolo

Quoted 27 lines

Thanks,

Mike

Quoted 5 lines

Paolo

Quoted 2 lines
target/i386/sev.c +0 -0
      if (ret) {
          error_setg(errp, "%s: failed to create encryption context",

__func__);

Quoted 6 lines

target/i386/trace-events kvm_sev_launch_measurement(const char *value) "data

Quoted 6 lines
target/i386/sev.c +0 -0
          goto err;
target/i386/trace-events +0 -0
index 2cd8726eeb..cb26d8a925 100644
@@ -11,3 +11,4 @@ kvm_sev_launch_measurement(const char *value) "data

%s"

Quoted 2 lines

target/i386/trace-events kvm_sev_launch_measurement(const char *value) "data · line 12

Quoted 2 lines
target/i386/trace-events +0 -0
  kvm_sev_launch_finish(void) ""
  kvm_sev_launch_secret(uint64_t hpa, uint64_t hva, uint64_t secret,

int len) "hpa 0x%" PRIx64 " hva 0x%" PRIx64 " data 0x%" PRIx64 " len %d"

Quoted 1 line

target/i386/trace-events kvm_sev_launch_measurement(const char *value) "data · line 13

Quoted 1 line
target/i386/trace-events +0 -0
  kvm_sev_attestation_report(const char *mnonce, const char *data)

"mnonce %s data %s"

Quoted 1 line

target/i386/trace-events kvm_sev_launch_measurement(const char *value) "data · line 14

Quoted 1 line
target/i386/trace-events +1 -0
+kvm_sev_snp_launch_start(uint64_t policy, char *gosvw) "policy 0x%"

PRIx64 " gosvw %s"

Quoted 1 line
#83 lore raw

PATCH v3 38/49 i386/sev: Add handling to encrypt/finalize guest launch data

In reply to Michael Roth

From: Brijesh Singh <[email protected]>

Process any queued up launch data and encrypt/measure it into the SNP guest instance prior to initial guest launch.

---
 target/i386/sev.c        | 101 ++++++++++++++++++++++++++++++++++++++-
 target/i386/trace-events |   2 +
 2 files changed, 102 insertions(+), 1 deletion(-)
target/i386/sev.c +100 -1
index 9f63a41f08..4155342e72 100644
@@ -749,6 +749,61 @@ out:
     return ret;
 }
 
+static const char *
+snp_page_type_to_str(int type)
+{
+    switch (type) {
+    case KVM_SEV_SNP_PAGE_TYPE_NORMAL: return "Normal";
+    case KVM_SEV_SNP_PAGE_TYPE_ZERO: return "Zero";
+    case KVM_SEV_SNP_PAGE_TYPE_UNMEASURED: return "Unmeasured";
+    case KVM_SEV_SNP_PAGE_TYPE_SECRETS: return "Secrets";
+    case KVM_SEV_SNP_PAGE_TYPE_CPUID: return "Cpuid";
+    default: return "unknown";
+    }
+}
+
+static int
+sev_snp_launch_update(SevSnpGuestState *sev_snp_guest, SevLaunchUpdateData *data)
+{
+    int ret, fw_error;
+    struct kvm_sev_snp_launch_update update = {0};
+
+    if (!data->hva || !data->len) {
+        error_report("SNP_LAUNCH_UPDATE called with invalid address / length: %p / %lx",
+                     data->hva, data->len);
+        return 1;
+    }
+
+    update.uaddr = (__u64)(unsigned long)data->hva;
+    update.gfn_start = data->gpa >> TARGET_PAGE_BITS;
+    update.len = data->len;
+    update.type = data->type;
+
+    trace_kvm_sev_snp_launch_update(data->hva, data->gpa, data->len,
+                                    snp_page_type_to_str(data->type));
+
+    /*
+     * KVM_SEV_SNP_LAUNCH_UPDATE requires that GPA ranges have the private
+     * memory attribute set in advance.
+     */
+    ret = kvm_set_memory_attributes_private(data->gpa, data->len);
+    if (ret) {
+        error_report("SEV-SNP: failed to configure initial private guest memory");
+        goto out;
+    }
+
+    ret = sev_ioctl(SEV_COMMON(sev_snp_guest)->sev_fd,
+                    KVM_SEV_SNP_LAUNCH_UPDATE,
+                    &update, &fw_error);
+    if (ret) {
+        error_report("SNP_LAUNCH_UPDATE ret=%d fw_error=%d '%s'",
+                     ret, fw_error, fw_error_to_str(fw_error));
+    }
+
+out:
+    return ret;
+}
+
 static int
 sev_launch_update_data(SevGuestState *sev_guest, uint8_t *addr, uint64_t len)
 {
@@ -894,6 +949,46 @@ sev_launch_finish(SevGuestState *sev_guest)
     migrate_add_blocker(&sev_mig_blocker, &error_fatal);
 }
 
+static void
+sev_snp_launch_finish(SevSnpGuestState *sev_snp)
+{
+    int ret, error;
+    Error *local_err = NULL;
+    OvmfSevMetadata *metadata;
+    SevLaunchUpdateData *data;
+    struct kvm_sev_snp_launch_finish *finish = &sev_snp->kvm_finish_conf;
+
+    QTAILQ_FOREACH(data, &launch_update, next) {
+        ret = sev_snp_launch_update(sev_snp, data);
+        if (ret) {
+            exit(1);
+        }
+    }
+
+    trace_kvm_sev_snp_launch_finish(sev_snp->id_block, sev_snp->id_auth,
+                                    sev_snp->host_data);
+    ret = sev_ioctl(SEV_COMMON(sev_snp)->sev_fd, KVM_SEV_SNP_LAUNCH_FINISH,
+                    finish, &error);
+    if (ret) {
+        error_report("SNP_LAUNCH_FINISH ret=%d fw_error=%d '%s'",
+                     ret, error, fw_error_to_str(error));
+        exit(1);
+    }
+
+    sev_set_guest_state(SEV_COMMON(sev_snp), SEV_STATE_RUNNING);
+
+    /* add migration blocker */
+    error_setg(&sev_mig_blocker,
+               "SEV-SNP: Migration is not implemented");
+    ret = migrate_add_blocker(&sev_mig_blocker, &local_err);
+    if (local_err) {
+        error_report_err(local_err);
+        error_free(sev_mig_blocker);
+        exit(1);
+    }
+}
+
+
 static void
 sev_vm_state_change(void *opaque, bool running, RunState state)
 {
@@ -901,7 +996,11 @@ sev_vm_state_change(void *opaque, bool running, RunState state)
 
     if (running) {
         if (!sev_check_state(sev_common, SEV_STATE_RUNNING)) {
-            sev_launch_finish(SEV_GUEST(sev_common));
+            if (sev_snp_enabled()) {
+                sev_snp_launch_finish(SEV_SNP_GUEST(sev_common));
+            } else {
+                sev_launch_finish(SEV_GUEST(sev_common));
+            }
         }
     }
 }
target/i386/trace-events +2 -0
index cb26d8a925..873a7e424e 100644
@@ -12,3 +12,5 @@ kvm_sev_launch_finish(void) ""
 kvm_sev_launch_secret(uint64_t hpa, uint64_t hva, uint64_t secret, int len) "hpa 0x%" PRIx64 " hva 0x%" PRIx64 " data 0x%" PRIx64 " len %d"
 kvm_sev_attestation_report(const char *mnonce, const char *data) "mnonce %s data %s"
 kvm_sev_snp_launch_start(uint64_t policy, char *gosvw) "policy 0x%" PRIx64 " gosvw %s"
+kvm_sev_snp_launch_update(void *addr, uint32_t gpa, uint64_t len, const char *type) "addr %p gpa 0x%x len 0x%" PRIx64 " (%s page)"
+kvm_sev_snp_launch_finish(char *id_block, char *id_auth, char *host_data) "id_block %s id_auth %s host_data %s"
Signature · 1 line
2.25.1
#84 lore raw

PATCH v3 39/49 i386/sev: Set CPU state to protected once SNP guest payload is finalized

In reply to Michael Roth

Once KVM_SNP_LAUNCH_FINISH is called the vCPU state is copied into the vCPU's VMSA page and measured/encrypted. Any attempt to read/write CPU state afterward will only be acting on the initial data and so are effectively no-ops.

Set the vCPU state to protected at this point so that QEMU don't continue trying to re-sync vCPU data during guest runtime.

---
 target/i386/sev.c | 1 +
 1 file changed, 1 insertion(+)
target/i386/sev.c +1 -0
index 4155342e72..4d862eef78 100644
@@ -975,6 +975,7 @@ sev_snp_launch_finish(SevSnpGuestState *sev_snp)
         exit(1);
     }
 
+    kvm_mark_guest_state_protected();
     sev_set_guest_state(SEV_COMMON(sev_snp), SEV_STATE_RUNNING);
 
     /* add migration blocker */
Signature · 1 line
2.25.1
#85 lore raw

PATCH v3 40/49 hw/i386/sev: Add function to get SEV metadata from OVMF header

In reply to Michael Roth

From: Brijesh Singh <[email protected]>

A recent version of OVMF expanded the reset vector GUID list to add SEV-specific metadata GUID. The SEV metadata describes the reserved memory regions such as the secrets and CPUID page used during the SEV-SNP guest launch.

The pc_system_get_ovmf_sev_metadata_ptr() is used to retieve the SEV metadata pointer from the OVMF GUID list.

---
 hw/i386/pc_sysfw_ovmf.c | 33 +++++++++++++++++++++++++++++++++
 include/hw/i386/pc.h    | 26 ++++++++++++++++++++++++++
 2 files changed, 59 insertions(+)
hw/i386/pc_sysfw_ovmf.c +33 -0
index 07a4c267fa..32efa34614 100644
@@ -35,6 +35,31 @@ static const int bytes_after_table_footer = 32;
 static bool ovmf_flash_parsed;
 static uint8_t *ovmf_table;
 static int ovmf_table_len;
+static OvmfSevMetadata *ovmf_sev_metadata_table;
+
+#define OVMF_SEV_META_DATA_GUID "dc886566-984a-4798-A75e-5585a7bf67cc"
+typedef struct __attribute__((__packed__)) OvmfSevMetadataOffset {
+    uint32_t offset;
+} OvmfSevMetadataOffset;
+
+static void pc_system_parse_sev_metadata(uint8_t *flash_ptr, size_t flash_size)
+{
+    OvmfSevMetadata     *metadata;
+    OvmfSevMetadataOffset  *data;
+
+    if (!pc_system_ovmf_table_find(OVMF_SEV_META_DATA_GUID, (uint8_t **)&data,
+                                   NULL)) {
+        return;
+    }
+
+    metadata = (OvmfSevMetadata *)(flash_ptr + flash_size - data->offset);
+    if (memcmp(metadata->signature, "ASEV", 4) != 0) {
+        return;
+    }
+
+    ovmf_sev_metadata_table = g_malloc(metadata->len);
+    memcpy(ovmf_sev_metadata_table, metadata, metadata->len);
+}
 
 void pc_system_parse_ovmf_flash(uint8_t *flash_ptr, size_t flash_size)
 {
@@ -90,6 +115,9 @@ void pc_system_parse_ovmf_flash(uint8_t *flash_ptr, size_t flash_size)
      */
     memcpy(ovmf_table, ptr - tot_len, tot_len);
     ovmf_table += tot_len;
+
+    /* Copy the SEV metadata table (if exist) */
+    pc_system_parse_sev_metadata(flash_ptr, flash_size);
 }
 
 /**
@@ -159,3 +187,8 @@ bool pc_system_ovmf_table_find(const char *entry, uint8_t **data,
     }
     return false;
 }
+
+OvmfSevMetadata *pc_system_get_ovmf_sev_metadata_ptr(void)
+{
+    return ovmf_sev_metadata_table;
+}
include/hw/i386/pc.h +26 -0
index fb1d4106e5..df9a61540d 100644
@@ -163,6 +163,32 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level);
 #define PCI_HOST_ABOVE_4G_MEM_SIZE     "above-4g-mem-size"
 #define PCI_HOST_PROP_SMM_RANGES       "smm-ranges"
 
+typedef enum {
+    SEV_DESC_TYPE_UNDEF,
+    /* The section contains the region that must be validated by the VMM. */
+    SEV_DESC_TYPE_SNP_SEC_MEM,
+    /* The section contains the SNP secrets page */
+    SEV_DESC_TYPE_SNP_SECRETS,
+    /* The section contains address that can be used as a CPUID page */
+    SEV_DESC_TYPE_CPUID,
+
+} ovmf_sev_metadata_desc_type;
+
+typedef struct __attribute__((__packed__)) OvmfSevMetadataDesc {
+    uint32_t base;
+    uint32_t len;
+    ovmf_sev_metadata_desc_type type;
+} OvmfSevMetadataDesc;
+
+typedef struct __attribute__((__packed__)) OvmfSevMetadata {
+    uint8_t signature[4];
+    uint32_t len;
+    uint32_t version;
+    uint32_t num_desc;
+    OvmfSevMetadataDesc descs[];
+} OvmfSevMetadata;
+
+OvmfSevMetadata *pc_system_get_ovmf_sev_metadata_ptr(void);
 
 void pc_pci_as_mapping_init(MemoryRegion *system_memory,
                             MemoryRegion *pci_address_space);
Signature · 1 line
2.25.1
#86 lore raw

Re: [PATCH v3 40/49] hw/i386/sev: Add function to get SEV metadata from OVMF header

Quoted 54 lines · Michael Roth

From: Brijesh Singh <[email protected]>

A recent version of OVMF expanded the reset vector GUID list to add SEV-specific metadata GUID. The SEV metadata describes the reserved memory regions such as the secrets and CPUID page used during the SEV-SNP guest launch.

The pc_system_get_ovmf_sev_metadata_ptr() is used to retieve the SEV metadata pointer from the OVMF GUID list.

---
 hw/i386/pc_sysfw_ovmf.c | 33 +++++++++++++++++++++++++++++++++
 include/hw/i386/pc.h    | 26 ++++++++++++++++++++++++++
 2 files changed, 59 insertions(+)
hw/i386/pc_sysfw_ovmf.c +25 -0
index 07a4c267fa..32efa34614 100644
@@ -35,6 +35,31 @@ static const int bytes_after_table_footer = 32;
 static bool ovmf_flash_parsed;
 static uint8_t *ovmf_table;
 static int ovmf_table_len;
+static OvmfSevMetadata *ovmf_sev_metadata_table;
+
+#define OVMF_SEV_META_DATA_GUID "dc886566-984a-4798-A75e-5585a7bf67cc"
+typedef struct __attribute__((__packed__)) OvmfSevMetadataOffset {
+    uint32_t offset;
+} OvmfSevMetadataOffset;
+
+static void pc_system_parse_sev_metadata(uint8_t *flash_ptr, size_t flash_size)
+{
+    OvmfSevMetadata     *metadata;
+    OvmfSevMetadataOffset  *data;
+
+    if (!pc_system_ovmf_table_find(OVMF_SEV_META_DATA_GUID, (uint8_t **)&data,
+                                   NULL)) {
+        return;
+    }
+
+    metadata = (OvmfSevMetadata *)(flash_ptr + flash_size - data->offset);
+    if (memcmp(metadata->signature, "ASEV", 4) != 0) {
+        return;
+    }
+
+    ovmf_sev_metadata_table = g_malloc(metadata->len);
+    memcpy(ovmf_sev_metadata_table, metadata, metadata->len);
+}
 
 void pc_system_parse_ovmf_flash(uint8_t *flash_ptr, size_t flash_size)
 {

hw/i386/pc_sysfw_ovmf.c void pc_system_parse_ovmf_flash(uint8_t *flash_ptr, size_t flash_size) · line 121

Quoted 7 lines
hw/i386/pc_sysfw_ovmf.c +3 -0
@@ -90,6 +115,9 @@ void pc_system_parse_ovmf_flash(uint8_t *flash_ptr, size_t flash_size)
      */
     memcpy(ovmf_table, ptr - tot_len, tot_len);
     ovmf_table += tot_len;
+
+    /* Copy the SEV metadata table (if exist) */
+    pc_system_parse_sev_metadata(flash_ptr, flash_size);
 }

Can we move this call to x86_firmware_configure() @ pc_sysfw.c, and move sev specific bits to somewhere to sev specific file? We don't have to parse sev metadata for non-SEV case, right?

We don't have to touch common ovmf file. It also will be consistent with tdx case. TDX patch series adds tdx_parse_tdvf() to x86_firmware_configure().

thanks,

Quoted 52 lines
hw/i386/pc_sysfw_ovmf.c +5 -0
 /**
@@ -159,3 +187,8 @@ bool pc_system_ovmf_table_find(const char *entry, uint8_t **data,
     }
     return false;
 }
+
+OvmfSevMetadata *pc_system_get_ovmf_sev_metadata_ptr(void)
+{
+    return ovmf_sev_metadata_table;
+}
include/hw/i386/pc.h +26 -0
index fb1d4106e5..df9a61540d 100644
@@ -163,6 +163,32 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level);
 #define PCI_HOST_ABOVE_4G_MEM_SIZE     "above-4g-mem-size"
 #define PCI_HOST_PROP_SMM_RANGES       "smm-ranges"
 
+typedef enum {
+    SEV_DESC_TYPE_UNDEF,
+    /* The section contains the region that must be validated by the VMM. */
+    SEV_DESC_TYPE_SNP_SEC_MEM,
+    /* The section contains the SNP secrets page */
+    SEV_DESC_TYPE_SNP_SECRETS,
+    /* The section contains address that can be used as a CPUID page */
+    SEV_DESC_TYPE_CPUID,
+
+} ovmf_sev_metadata_desc_type;
+
+typedef struct __attribute__((__packed__)) OvmfSevMetadataDesc {
+    uint32_t base;
+    uint32_t len;
+    ovmf_sev_metadata_desc_type type;
+} OvmfSevMetadataDesc;
+
+typedef struct __attribute__((__packed__)) OvmfSevMetadata {
+    uint8_t signature[4];
+    uint32_t len;
+    uint32_t version;
+    uint32_t num_desc;
+    OvmfSevMetadataDesc descs[];
+} OvmfSevMetadata;
+
+OvmfSevMetadata *pc_system_get_ovmf_sev_metadata_ptr(void);
 
 void pc_pci_as_mapping_init(MemoryRegion *system_memory,
                             MemoryRegion *pci_address_space);
Signature · 1 line
2.25.1
Signature · 1 line
Isaku Yamahata <[email protected]>
#87 lore raw

Re: [PATCH v3 40/49] hw/i386/sev: Add function to get SEV metadata from OVMF header

Quoted 69 lines · Isaku Yamahata
Quoted 54 lines · Michael Roth

From: Brijesh Singh <[email protected]>

A recent version of OVMF expanded the reset vector GUID list to add SEV-specific metadata GUID. The SEV metadata describes the reserved memory regions such as the secrets and CPUID page used during the SEV-SNP guest launch.

The pc_system_get_ovmf_sev_metadata_ptr() is used to retieve the SEV metadata pointer from the OVMF GUID list.

---
 hw/i386/pc_sysfw_ovmf.c | 33 +++++++++++++++++++++++++++++++++
 include/hw/i386/pc.h    | 26 ++++++++++++++++++++++++++
 2 files changed, 59 insertions(+)
hw/i386/pc_sysfw_ovmf.c +25 -0
index 07a4c267fa..32efa34614 100644
@@ -35,6 +35,31 @@ static const int bytes_after_table_footer = 32;
 static bool ovmf_flash_parsed;
 static uint8_t *ovmf_table;
 static int ovmf_table_len;
+static OvmfSevMetadata *ovmf_sev_metadata_table;
+
+#define OVMF_SEV_META_DATA_GUID "dc886566-984a-4798-A75e-5585a7bf67cc"
+typedef struct __attribute__((__packed__)) OvmfSevMetadataOffset {
+    uint32_t offset;
+} OvmfSevMetadataOffset;
+
+static void pc_system_parse_sev_metadata(uint8_t *flash_ptr, size_t flash_size)
+{
+    OvmfSevMetadata     *metadata;
+    OvmfSevMetadataOffset  *data;
+
+    if (!pc_system_ovmf_table_find(OVMF_SEV_META_DATA_GUID, (uint8_t **)&data,
+                                   NULL)) {
+        return;
+    }
+
+    metadata = (OvmfSevMetadata *)(flash_ptr + flash_size - data->offset);
+    if (memcmp(metadata->signature, "ASEV", 4) != 0) {
+        return;
+    }
+
+    ovmf_sev_metadata_table = g_malloc(metadata->len);
+    memcpy(ovmf_sev_metadata_table, metadata, metadata->len);
+}
 
 void pc_system_parse_ovmf_flash(uint8_t *flash_ptr, size_t flash_size)
 {

hw/i386/pc_sysfw_ovmf.c void pc_system_parse_ovmf_flash(uint8_t *flash_ptr, size_t flash_size) · line 121

Quoted 7 lines
hw/i386/pc_sysfw_ovmf.c +3 -0
@@ -90,6 +115,9 @@ void pc_system_parse_ovmf_flash(uint8_t *flash_ptr, size_t flash_size)
      */
     memcpy(ovmf_table, ptr - tot_len, tot_len);
     ovmf_table += tot_len;
+
+    /* Copy the SEV metadata table (if exist) */
+    pc_system_parse_sev_metadata(flash_ptr, flash_size);
 }

Can we move this call to x86_firmware_configure() @ pc_sysfw.c, and move sev specific bits to somewhere to sev specific file? We don't have to parse sev metadata for non-SEV case, right?

Quoted 2 lines

We don't have to touch common ovmf file. It also will be consistent with tdx case. TDX patch series adds tdx_parse_tdvf() to x86_firmware_configure().

Yep, makes sense to handle it similarly for SNP.

Thanks,

Mike

Quoted 58 lines

thanks,

Quoted 52 lines
hw/i386/pc_sysfw_ovmf.c +5 -0
 /**
@@ -159,3 +187,8 @@ bool pc_system_ovmf_table_find(const char *entry, uint8_t **data,
     }
     return false;
 }
+
+OvmfSevMetadata *pc_system_get_ovmf_sev_metadata_ptr(void)
+{
+    return ovmf_sev_metadata_table;
+}
include/hw/i386/pc.h +26 -0
index fb1d4106e5..df9a61540d 100644
@@ -163,6 +163,32 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level);
 #define PCI_HOST_ABOVE_4G_MEM_SIZE     "above-4g-mem-size"
 #define PCI_HOST_PROP_SMM_RANGES       "smm-ranges"
 
+typedef enum {
+    SEV_DESC_TYPE_UNDEF,
+    /* The section contains the region that must be validated by the VMM. */
+    SEV_DESC_TYPE_SNP_SEC_MEM,
+    /* The section contains the SNP secrets page */
+    SEV_DESC_TYPE_SNP_SECRETS,
+    /* The section contains address that can be used as a CPUID page */
+    SEV_DESC_TYPE_CPUID,
+
+} ovmf_sev_metadata_desc_type;
+
+typedef struct __attribute__((__packed__)) OvmfSevMetadataDesc {
+    uint32_t base;
+    uint32_t len;
+    ovmf_sev_metadata_desc_type type;
+} OvmfSevMetadataDesc;
+
+typedef struct __attribute__((__packed__)) OvmfSevMetadata {
+    uint8_t signature[4];
+    uint32_t len;
+    uint32_t version;
+    uint32_t num_desc;
+    OvmfSevMetadataDesc descs[];
+} OvmfSevMetadata;
+
+OvmfSevMetadata *pc_system_get_ovmf_sev_metadata_ptr(void);
 
 void pc_pci_as_mapping_init(MemoryRegion *system_memory,
                             MemoryRegion *pci_address_space);
Signature · 1 line
2.25.1
Signature · 1 line
Isaku Yamahata <[email protected]>
#88 lore raw

PATCH v3 41/49 i386/sev: Add support for populating OVMF metadata pages

In reply to Michael Roth

From: Brijesh Singh <[email protected]>

OVMF reserves various pages so they can be pre-initialized/validated prior to launching the guest. Add support for populating these pages with the expected content.

---
 target/i386/sev.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)
target/i386/sev.c +75 -0
index 4d862eef78..6c5166c729 100644
@@ -949,6 +949,67 @@ sev_launch_finish(SevGuestState *sev_guest)
     migrate_add_blocker(&sev_mig_blocker, &error_fatal);
 }
 
+static int
+snp_launch_update_data(uint64_t gpa, void *hva, uint32_t len, int type)
+{
+    SevLaunchUpdateData *data;
+
+    data = g_new0(SevLaunchUpdateData, 1);
+    data->gpa = gpa;
+    data->hva = hva;
+    data->len = len;
+    data->type = type;
+
+    QTAILQ_INSERT_TAIL(&launch_update, data, next);
+
+    return 0;
+}
+
+static int
+snp_metadata_desc_to_page_type(int desc_type)
+{
+    switch(desc_type) {
+    /* Add the umeasured prevalidated pages as a zero page */
+    case SEV_DESC_TYPE_SNP_SEC_MEM: return KVM_SEV_SNP_PAGE_TYPE_ZERO;
+    case SEV_DESC_TYPE_SNP_SECRETS: return KVM_SEV_SNP_PAGE_TYPE_SECRETS;
+    case SEV_DESC_TYPE_CPUID: return KVM_SEV_SNP_PAGE_TYPE_CPUID;
+    default: return -1;
+    }
+}
+
+static void
+snp_populate_metadata_pages(SevSnpGuestState *sev_snp, OvmfSevMetadata *metadata)
+{
+    OvmfSevMetadataDesc *desc;
+    int type, ret, i;
+    void *hva;
+    MemoryRegion *mr = NULL;
+
+    for (i = 0; i < metadata->num_desc; i++) {
+        desc = &metadata->descs[i];
+
+        type = snp_metadata_desc_to_page_type(desc->type);
+        if (type < 0) {
+            error_report("%s: Invalid memory type '%d'\n", __func__, desc->type);
+            exit(1);
+        }
+
+        hva = gpa2hva(&mr, desc->base, desc->len, NULL);
+        if (!hva) {
+            error_report("%s: Failed to get HVA for GPA 0x%x sz 0x%x\n",
+                         __func__, desc->base, desc->len);
+            exit(1);
+        }
+
+        ret = snp_launch_update_data(desc->base, hva, desc->len, type);
+        if (ret) {
+            error_report("%s: Failed to add metadata page gpa 0x%x+%x type %d\n",
+                         __func__, desc->base, desc->len, desc->type);
+            exit(1);
+        }
+    }
+}
+
 static void
 sev_snp_launch_finish(SevSnpGuestState *sev_snp)
 {
@@ -958,6 +1019,20 @@ sev_snp_launch_finish(SevSnpGuestState *sev_snp)
     SevLaunchUpdateData *data;
     struct kvm_sev_snp_launch_finish *finish = &sev_snp->kvm_finish_conf;
 
+    /*
+     * To boot the SNP guest, the hypervisor is required to populate the CPUID
+     * and Secrets page before finalizing the launch flow. The location of
+     * the secrets and CPUID page is available through the OVMF metadata GUID.
+     */
+    metadata = pc_system_get_ovmf_sev_metadata_ptr();
+    if (metadata == NULL) {
+        error_report("%s: Failed to locate SEV metadata header\n", __func__);
+        exit(1);
+    }
+
+    /* Populate all the metadata pages */
+    snp_populate_metadata_pages(sev_snp, metadata);
+
     QTAILQ_FOREACH(data, &launch_update, next) {
         ret = sev_snp_launch_update(sev_snp, data);
         if (ret) {
Signature · 1 line
2.25.1
#89 lore raw

PATCH v3 42/49 i386/sev: Add support for SNP CPUID validation

In reply to Michael Roth

SEV-SNP firmware allows a special guest page to be populated with a table of guest CPUID values so that they can be validated through firmware before being loaded into encrypted guest memory where they can be used in place of hypervisor-provided values[1].

As part of SEV-SNP guest initialization, use this interface to validate the CPUID entries reported by KVM_GET_CPUID2 prior to initial guest start and populate the CPUID page reserved by OVMF with the resulting encrypted data.

[1] SEV SNP Firmware ABI Specification, Rev. 0.8, 8.13.2.6

---
 target/i386/sev.c | 159 +++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 158 insertions(+), 1 deletion(-)
target/i386/sev.c +158 -1
index 6c5166c729..db888afb53 100644
@@ -191,6 +191,36 @@ static const char *const sev_fw_errlist[] = {
 
 #define SEV_FW_MAX_ERROR      ARRAY_SIZE(sev_fw_errlist)
 
+/* <linux/kvm.h> doesn't expose this, so re-use the max from kvm.c */
+#define KVM_MAX_CPUID_ENTRIES 100
+
+typedef struct KvmCpuidInfo {
+    struct kvm_cpuid2 cpuid;
+    struct kvm_cpuid_entry2 entries[KVM_MAX_CPUID_ENTRIES];
+} KvmCpuidInfo;
+
+#define SNP_CPUID_FUNCTION_MAXCOUNT 64
+#define SNP_CPUID_FUNCTION_UNKNOWN 0xFFFFFFFF
+
+typedef struct {
+    uint32_t eax_in;
+    uint32_t ecx_in;
+    uint64_t xcr0_in;
+    uint64_t xss_in;
+    uint32_t eax;
+    uint32_t ebx;
+    uint32_t ecx;
+    uint32_t edx;
+    uint64_t reserved;
+} __attribute__((packed)) SnpCpuidFunc;
+
+typedef struct {
+    uint32_t count;
+    uint32_t reserved1;
+    uint64_t reserved2;
+    SnpCpuidFunc entries[SNP_CPUID_FUNCTION_MAXCOUNT];
+} __attribute__((packed)) SnpCpuidInfo;
+
 static int
 sev_ioctl(int fd, int cmd, void *data, int *error)
 {
@@ -749,6 +779,34 @@ out:
     return ret;
 }
 
+static void
+sev_snp_cpuid_report_mismatches(SnpCpuidInfo *old,
+                                SnpCpuidInfo *new)
+{
+    size_t i;
+
+    if (old->count != new->count) {
+        error_report("SEV-SNP: CPUID validation failed due to count mismatch, provided: %d, expected: %d",
+                     old->count, new->count);
+    }
+
+    for (i = 0; i < old->count; i++) {
+        SnpCpuidFunc *old_func, *new_func;
+
+        old_func = &old->entries[i];
+        new_func = &new->entries[i];
+
+        if (memcmp(old_func, new_func, sizeof(SnpCpuidFunc))) {
+            error_report("SEV-SNP: CPUID validation failed for function 0x%x, index: 0x%x.\n"
+                         "provided: eax:0x%08x, ebx: 0x%08x, ecx: 0x%08x, edx: 0x%08x\n"
+                         "expected: eax:0x%08x, ebx: 0x%08x, ecx: 0x%08x, edx: 0x%08x",
+                         old_func->eax_in, old_func->ecx_in,
+                         old_func->eax, old_func->ebx, old_func->ecx, old_func->edx,
+                         new_func->eax, new_func->ebx, new_func->ecx, new_func->edx);
+        }
+    }
+}
+
 static const char *
 snp_page_type_to_str(int type)
 {
@@ -766,6 +824,7 @@ static int
 sev_snp_launch_update(SevSnpGuestState *sev_snp_guest, SevLaunchUpdateData *data)
 {
     int ret, fw_error;
+    SnpCpuidInfo snp_cpuid_info;
     struct kvm_sev_snp_launch_update update = {0};
 
     if (!data->hva || !data->len) {
@@ -774,6 +833,11 @@ sev_snp_launch_update(SevSnpGuestState *sev_snp_guest, SevLaunchUpdateData *data
         return 1;
     }
 
+    if (data->type == KVM_SEV_SNP_PAGE_TYPE_CPUID) {
+        /* Save a copy for comparison in case the LAUNCH_UPDATE fails */
+        memcpy(&snp_cpuid_info, data->hva, sizeof(snp_cpuid_info));
+    }
+
     update.uaddr = (__u64)(unsigned long)data->hva;
     update.gfn_start = data->gpa >> TARGET_PAGE_BITS;
     update.len = data->len;
@@ -798,6 +862,11 @@ sev_snp_launch_update(SevSnpGuestState *sev_snp_guest, SevLaunchUpdateData *data
     if (ret) {
         error_report("SNP_LAUNCH_UPDATE ret=%d fw_error=%d '%s'",
                      ret, fw_error, fw_error_to_str(fw_error));
+
+        if (data->type == KVM_SEV_SNP_PAGE_TYPE_CPUID) {
+            sev_snp_cpuid_report_mismatches(&snp_cpuid_info, data->hva);
+            error_report("SEV-SNP: failed update CPUID page");
+        }
     }
 
 out:
@@ -965,6 +1034,89 @@ snp_launch_update_data(uint64_t gpa, void *hva, uint32_t len, int type)
     return 0;
 }
 
+static int
+sev_snp_cpuid_info_fill(SnpCpuidInfo *snp_cpuid_info,
+                        const KvmCpuidInfo *kvm_cpuid_info)
+{
+    size_t i;
+
+    if (kvm_cpuid_info->cpuid.nent > SNP_CPUID_FUNCTION_MAXCOUNT) {
+        error_report("SEV-SNP: CPUID entry count (%d) exceeds max (%d)",
+                     kvm_cpuid_info->cpuid.nent, SNP_CPUID_FUNCTION_MAXCOUNT);
+        return -1;
+    }
+
+    memset(snp_cpuid_info, 0, sizeof(*snp_cpuid_info));
+
+    for (i = 0; i < kvm_cpuid_info->cpuid.nent; i++) {
+        const struct kvm_cpuid_entry2 *kvm_cpuid_entry;
+        SnpCpuidFunc *snp_cpuid_entry;
+
+        kvm_cpuid_entry = &kvm_cpuid_info->entries[i];
+        snp_cpuid_entry = &snp_cpuid_info->entries[i];
+
+        snp_cpuid_entry->eax_in = kvm_cpuid_entry->function;
+        if (kvm_cpuid_entry->flags == KVM_CPUID_FLAG_SIGNIFCANT_INDEX) {
+            snp_cpuid_entry->ecx_in = kvm_cpuid_entry->index;
+        }
+        snp_cpuid_entry->eax = kvm_cpuid_entry->eax;
+        snp_cpuid_entry->ebx = kvm_cpuid_entry->ebx;
+        snp_cpuid_entry->ecx = kvm_cpuid_entry->ecx;
+        snp_cpuid_entry->edx = kvm_cpuid_entry->edx;
+
+        /*
+         * Guest kernels will calculate EBX themselves using the 0xD
+         * subfunctions corresponding to the individual XSAVE areas, so only
+         * encode the base XSAVE size in the initial leaves, corresponding
+         * to the initial XCR0=1 state.
+         */
+        if (snp_cpuid_entry->eax_in == 0xD &&
+            (snp_cpuid_entry->ecx_in == 0x0 || snp_cpuid_entry->ecx_in == 0x1)) {
+            snp_cpuid_entry->ebx = 0x240;
+            snp_cpuid_entry->xcr0_in = 1;
+            snp_cpuid_entry->xss_in = 0;
+        }
+    }
+
+    snp_cpuid_info->count = i;
+
+    return 0;
+}
+
+static int
+snp_launch_update_cpuid(uint32_t cpuid_addr, void *hva, uint32_t cpuid_len)
+{
+    KvmCpuidInfo kvm_cpuid_info = {0};
+    SnpCpuidInfo snp_cpuid_info;
+    CPUState *cs = first_cpu;
+    int ret;
+    uint32_t i = 0;
+
+    assert(sizeof(snp_cpuid_info) <= cpuid_len);
+
+    /* get the cpuid list from KVM */
+    do {
+        kvm_cpuid_info.cpuid.nent = ++i;
+        ret = kvm_vcpu_ioctl(cs, KVM_GET_CPUID2, &kvm_cpuid_info);
+    } while (ret == -E2BIG);
+
+    if (ret) {
+        error_report("SEV-SNP: unable to query CPUID values for CPU: '%s'",
+                     strerror(-ret));
+        return 1;
+    }
+
+    ret = sev_snp_cpuid_info_fill(&snp_cpuid_info, &kvm_cpuid_info);
+    if (ret) {
+        error_report("SEV-SNP: failed to generate CPUID table information");
+        return 1;
+    }
+
+    memcpy(hva, &snp_cpuid_info, sizeof(snp_cpuid_info));
+
+    return snp_launch_update_data(cpuid_addr, hva, cpuid_len, KVM_SEV_SNP_PAGE_TYPE_CPUID);
+}
+
 static int
 snp_metadata_desc_to_page_type(int desc_type)
 {
@@ -1001,7 +1153,12 @@ snp_populate_metadata_pages(SevSnpGuestState *sev_snp, OvmfSevMetadata *metadata
             exit(1);
         }
 
-        ret = snp_launch_update_data(desc->base, hva, desc->len, type);
+        if (type == KVM_SEV_SNP_PAGE_TYPE_CPUID) {
+            ret = snp_launch_update_cpuid(desc->base, hva, desc->len);
+        } else {
+            ret = snp_launch_update_data(desc->base, hva, desc->len, type);
+        }
+
         if (ret) {
             error_report("%s: Failed to add metadata page gpa 0x%x+%x type %d\n",
                          __func__, desc->base, desc->len, desc->type);
Signature · 1 line
2.25.1
#90 lore raw

Re: [PATCH v3 42/49] i386/sev: Add support for SNP CPUID validation

Quoted 18 lines · Michael Roth

SEV-SNP firmware allows a special guest page to be populated with a table of guest CPUID values so that they can be validated through firmware before being loaded into encrypted guest memory where they can be used in place of hypervisor-provided values[1].

As part of SEV-SNP guest initialization, use this interface to validate the CPUID entries reported by KVM_GET_CPUID2 prior to initial guest start and populate the CPUID page reserved by OVMF with the resulting encrypted data.

[1] SEV SNP Firmware ABI Specification, Rev. 0.8, 8.13.2.6

---
 target/i386/sev.c | 159 +++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 158 insertions(+), 1 deletion(-)

target/i386/sev.c out: · line 791

Quoted 10 lines
target/i386/sev.c +10 -0
+static void
+sev_snp_cpuid_report_mismatches(SnpCpuidInfo *old,
+                                SnpCpuidInfo *new)
+{
+    size_t i;
+
+    if (old->count != new->count) {
+        error_report("SEV-SNP: CPUID validation failed due to count mismatch, provided: %d, expected: %d",
+                     old->count, new->count);
+    }

Missing 'return' here, may result in array out of bounds on 'new->entries' in the next loop

Quoted 8 lines
target/i386/sev.c +8 -0
+
+    for (i = 0; i < old->count; i++) {
+        SnpCpuidFunc *old_func, *new_func;
+
+        old_func = &old->entries[i];
+        new_func = &new->entries[i];
+
+        if (memcmp(old_func, new_func, sizeof(SnpCpuidFunc))) {

target/i386/sev.c out: · line 809

Quoted 10 lines
target/i386/sev.c +10 -0
+            error_report("SEV-SNP: CPUID validation failed for function 0x%x, index: 0x%x.\n"
+                         "provided: eax:0x%08x, ebx: 0x%08x, ecx: 0x%08x, edx: 0x%08x\n"
+                         "expected: eax:0x%08x, ebx: 0x%08x, ecx: 0x%08x, edx: 0x%08x",
+                         old_func->eax_in, old_func->ecx_in,
+                         old_func->eax, old_func->ebx, old_func->ecx, old_func->edx,
+                         new_func->eax, new_func->ebx, new_func->ecx, new_func->edx);
+        }
+    }
+}
+

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#91 lore raw

PATCH v3 43/49 qapi, i386: Move kernel-hashes to SevCommonProperties

In reply to Michael Roth

From: Dov Murik <[email protected]>

In order to enable kernel-hashes for SNP, pull it from SevGuestProperties to its parent SevCommonProperties so it will be available for both SEV and SNP.

---
 qapi/qom.json     | 14 +++++++-------
 target/i386/sev.c | 44 ++++++++++++++++++--------------------------
 2 files changed, 25 insertions(+), 33 deletions(-)
qapi/qom.json +7 -7
index 7ba778af91..ea8832a8c3 100644
@@ -886,12 +886,17 @@
 # @reduced-phys-bits: number of bits in physical addresses that become
 #     unavailable when SEV is enabled
 #
+# @kernel-hashes: if true, add hashes of kernel/initrd/cmdline to a
+#     designated guest firmware page for measured boot with -kernel
+#     (default: false) (since 6.2)
+#
 # Since: 2.12
 ##
 { 'struct': 'SevCommonProperties',
   'data': { '*sev-device': 'str',
             '*cbitpos': 'uint32',
-            'reduced-phys-bits': 'uint32' } }
+            'reduced-phys-bits': 'uint32',
+            '*kernel-hashes': 'bool' } }
 
 ##
 # @SevGuestProperties:
@@ -906,10 +911,6 @@
 #
 # @handle: SEV firmware handle (default: 0)
 #
-# @kernel-hashes: if true, add hashes of kernel/initrd/cmdline to a
-#     designated guest firmware page for measured boot with -kernel
-#     (default: false) (since 6.2)
-#
 # Since: 2.12
 ##
 { 'struct': 'SevGuestProperties',
@@ -917,8 +918,7 @@
   'data': { '*dh-cert-file': 'str',
             '*session-file': 'str',
             '*policy': 'uint32',
-            '*handle': 'uint32',
-            '*kernel-hashes': 'bool' } }
+            '*handle': 'uint32' } }
 
 ##
 # @SevSnpGuestProperties:
target/i386/sev.c +18 -26

90 lines not on this page · raw message

Signature · 1 line
2.25.1
#92 lore raw

Re: [PATCH v3 43/49] qapi, i386: Move kernel-hashes to SevCommonProperties

Quoted 6 lines · Michael Roth

From: Dov Murik <[email protected]>

In order to enable kernel-hashes for SNP, pull it from SevGuestProperties to its parent SevCommonProperties so it will be available for both SEV and SNP.

Quoted 6 lines
---
 qapi/qom.json     | 14 +++++++-------
 target/i386/sev.c | 44 ++++++++++++++++++--------------------------
 2 files changed, 25 insertions(+), 33 deletions(-)

This change ought to be squashed into the earlier patch that introduce sev-guest-common.

Quoted 149 lines
qapi/qom.json +7 -7
index 7ba778af91..ea8832a8c3 100644
@@ -886,12 +886,17 @@
 # @reduced-phys-bits: number of bits in physical addresses that become
 #     unavailable when SEV is enabled
 #
+# @kernel-hashes: if true, add hashes of kernel/initrd/cmdline to a
+#     designated guest firmware page for measured boot with -kernel
+#     (default: false) (since 6.2)
+#
 # Since: 2.12
 ##
 { 'struct': 'SevCommonProperties',
   'data': { '*sev-device': 'str',
             '*cbitpos': 'uint32',
-            'reduced-phys-bits': 'uint32' } }
+            'reduced-phys-bits': 'uint32',
+            '*kernel-hashes': 'bool' } }
 
 ##
 # @SevGuestProperties:
@@ -906,10 +911,6 @@
 #
 # @handle: SEV firmware handle (default: 0)
 #
-# @kernel-hashes: if true, add hashes of kernel/initrd/cmdline to a
-#     designated guest firmware page for measured boot with -kernel
-#     (default: false) (since 6.2)
-#
 # Since: 2.12
 ##
 { 'struct': 'SevGuestProperties',
@@ -917,8 +918,7 @@
   'data': { '*dh-cert-file': 'str',
             '*session-file': 'str',
             '*policy': 'uint32',
-            '*handle': 'uint32',
-            '*kernel-hashes': 'bool' } }
+            '*handle': 'uint32' } }
 
 ##
 # @SevSnpGuestProperties:
target/i386/sev.c +18 -26
index db888afb53..3187b3dee8 100644
@@ -54,6 +54,7 @@ struct SevCommonState {
     char *sev_device;
     uint32_t cbitpos;
     uint32_t reduced_phys_bits;
+    bool kernel_hashes;
 
     /* runtime state */
     uint8_t api_major;
@@ -86,7 +87,6 @@ struct SevGuestState {
     uint32_t policy;
     char *dh_cert_file;
     char *session_file;
-    bool kernel_hashes;
 };
 
 struct SevSnpGuestState {
@@ -1696,16 +1696,12 @@ bool sev_add_kernel_loader_hashes(SevKernelLoaderContext *ctx, Error **errp)
     MemTxAttrs attrs = { 0 };
     bool ret = true;
     SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
-    SevGuestState *sev_guest =
-        (SevGuestState *)object_dynamic_cast(OBJECT(sev_common),
-                                             TYPE_SEV_GUEST);
 
     /*
      * Only add the kernel hashes if the sev-guest configuration explicitly
-     * stated kernel-hashes=on. Currently only enabled for SEV/SEV-ES guests,
-     * so check for TYPE_SEV_GUEST as well.
+     * stated kernel-hashes=on.
      */
-    if (sev_guest && !sev_guest->kernel_hashes) {
+    if (!sev_common->kernel_hashes) {
         return false;
     }
 
@@ -2037,6 +2033,16 @@ sev_common_set_sev_device(Object *obj, const char *value, Error **errp)
     SEV_COMMON(obj)->sev_device = g_strdup(value);
 }
 
+static bool sev_common_get_kernel_hashes(Object *obj, Error **errp)
+{
+    return SEV_COMMON(obj)->kernel_hashes;
+}
+
+static void sev_common_set_kernel_hashes(Object *obj, bool value, Error **errp)
+{
+    SEV_COMMON(obj)->kernel_hashes = value;
+}
+
 static void
 sev_common_class_init(ObjectClass *oc, void *data)
 {
@@ -2051,6 +2057,11 @@ sev_common_class_init(ObjectClass *oc, void *data)
                                   sev_common_set_sev_device);
     object_class_property_set_description(oc, "sev-device",
             "SEV device to use");
+    object_class_property_add_bool(oc, "kernel-hashes",
+                                   sev_common_get_kernel_hashes,
+                                   sev_common_set_kernel_hashes);
+    object_class_property_set_description(oc, "kernel-hashes",
+            "add kernel hashes to guest firmware for measured Linux boot");
 }
 
 static void
@@ -2109,20 +2120,6 @@ sev_guest_set_session_file(Object *obj, const char *value, Error **errp)
     SEV_GUEST(obj)->session_file = g_strdup(value);
 }
 
-static bool sev_guest_get_kernel_hashes(Object *obj, Error **errp)
-{
-    SevGuestState *sev_guest = SEV_GUEST(obj);
-
-    return sev_guest->kernel_hashes;
-}
-
-static void sev_guest_set_kernel_hashes(Object *obj, bool value, Error **errp)
-{
-    SevGuestState *sev = SEV_GUEST(obj);
-
-    sev->kernel_hashes = value;
-}
-
 static void
 sev_guest_class_init(ObjectClass *oc, void *data)
 {
@@ -2136,11 +2133,6 @@ sev_guest_class_init(ObjectClass *oc, void *data)
                                   sev_guest_set_session_file);
     object_class_property_set_description(oc, "session-file",
             "guest owners session parameters (encoded with base64)");
-    object_class_property_add_bool(oc, "kernel-hashes",
-                                   sev_guest_get_kernel_hashes,
-                                   sev_guest_set_kernel_hashes);
-    object_class_property_set_description(oc, "kernel-hashes",
-            "add kernel hashes to guest firmware for measured Linux boot");
 }
 
 static void
Signature · 1 line
2.25.1

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#93 lore raw

Re: [PATCH v3 43/49] qapi, i386: Move kernel-hashes to SevCommonProperties

Quoted 14 lines · Daniel P. Berrangé
Quoted 6 lines · Michael Roth

From: Dov Murik <[email protected]>

In order to enable kernel-hashes for SNP, pull it from SevGuestProperties to its parent SevCommonProperties so it will be available for both SEV and SNP.

Quoted 6 lines
---
 qapi/qom.json     | 14 +++++++-------
 target/i386/sev.c | 44 ++++++++++++++++++--------------------------
 2 files changed, 25 insertions(+), 33 deletions(-)
Quoted 2 lines

This change ought to be squashed into the earlier patch that introduce sev-guest-common.

Concur.

[...]

#94 lore raw

PATCH v3 44/49 i386/sev: Extract build_kernel_loader_hashes

In reply to Michael Roth

From: Dov Murik <[email protected]>

Extract the building of the kernel hashes table out from sev_add_kernel_loader_hashes() to allow building it in other memory areas (for SNP support).

No functional change intended.

---
 target/i386/sev.c | 101 ++++++++++++++++++++++++++--------------------
 1 file changed, 58 insertions(+), 43 deletions(-)
target/i386/sev.c +58 -43
index 3187b3dee8..0913cb7fed 100644
@@ -1677,45 +1677,16 @@ static const QemuUUID sev_cmdline_entry_guid = {
                     0x4d, 0x36, 0xab, 0x2a)
 };
 
-/*
- * Add the hashes of the linux kernel/initrd/cmdline to an encrypted guest page
- * which is included in SEV's initial memory measurement.
- */
-bool sev_add_kernel_loader_hashes(SevKernelLoaderContext *ctx, Error **errp)
+static bool build_kernel_loader_hashes(PaddedSevHashTable *padded_ht,
+                                       SevKernelLoaderContext *ctx,
+                                       Error **errp)
 {
-    uint8_t *data;
-    SevHashTableDescriptor *area;
     SevHashTable *ht;
-    PaddedSevHashTable *padded_ht;
     uint8_t cmdline_hash[HASH_SIZE];
     uint8_t initrd_hash[HASH_SIZE];
     uint8_t kernel_hash[HASH_SIZE];
     uint8_t *hashp;
     size_t hash_len = HASH_SIZE;
-    hwaddr mapped_len = sizeof(*padded_ht);
-    MemTxAttrs attrs = { 0 };
-    bool ret = true;
-    SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
-
-    /*
-     * Only add the kernel hashes if the sev-guest configuration explicitly
-     * stated kernel-hashes=on.
-     */
-    if (!sev_common->kernel_hashes) {
-        return false;
-    }
-
-    if (!pc_system_ovmf_table_find(SEV_HASH_TABLE_RV_GUID, &data, NULL)) {
-        error_setg(errp, "SEV: kernel specified but guest firmware "
-                         "has no hashes table GUID");
-        return false;
-    }
-    area = (SevHashTableDescriptor *)data;
-    if (!area->base || area->size < sizeof(PaddedSevHashTable)) {
-        error_setg(errp, "SEV: guest firmware hashes table area is invalid "
-                         "(base=0x%x size=0x%x)", area->base, area->size);
-        return false;
-    }
 
     /*
      * Calculate hash of kernel command-line with the terminating null byte. If
@@ -1752,16 +1723,6 @@ bool sev_add_kernel_loader_hashes(SevKernelLoaderContext *ctx, Error **errp)
     }
     assert(hash_len == HASH_SIZE);
 
-    /*
-     * Populate the hashes table in the guest's memory at the OVMF-designated
-     * area for the SEV hashes table
-     */
-    padded_ht = address_space_map(&address_space_memory, area->base,
-                                  &mapped_len, true, attrs);
-    if (!padded_ht || mapped_len != sizeof(*padded_ht)) {
-        error_setg(errp, "SEV: cannot map hashes table guest memory area");
-        return false;
-    }
     ht = &padded_ht->ht;
 
     ht->guid = sev_hash_table_header_guid;
@@ -1782,7 +1743,61 @@ bool sev_add_kernel_loader_hashes(SevKernelLoaderContext *ctx, Error **errp)
     /* zero the excess data so the measurement can be reliably calculated */
     memset(padded_ht->padding, 0, sizeof(padded_ht->padding));
 
-    if (sev_encrypt_flash((uint8_t *)padded_ht, sizeof(*padded_ht), errp) < 0) {
+    return true;
+}
+
+/*
+ * Add the hashes of the linux kernel/initrd/cmdline to an encrypted guest page
+ * which is included in SEV's initial memory measurement.
+ */
+bool sev_add_kernel_loader_hashes(SevKernelLoaderContext *ctx, Error **errp)
+{
+    uint8_t *data;
+    SevHashTableDescriptor *area;
+    PaddedSevHashTable *padded_ht;
+    hwaddr mapped_len = sizeof(*padded_ht);
+    MemTxAttrs attrs = { 0 };
+    bool ret = true;
+    SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
+
+    /*
+     * Only add the kernel hashes if the sev-guest configuration explicitly
+     * stated kernel-hashes=on.
+     */
+    if (!sev_common->kernel_hashes) {
+        return false;
+    }
+
+    if (!pc_system_ovmf_table_find(SEV_HASH_TABLE_RV_GUID, &data, NULL)) {
+        error_setg(errp, "SEV: kernel specified but guest firmware "
+                         "has no hashes table GUID");
+        return false;
+    }
+
+    area = (SevHashTableDescriptor *)data;
+    if (!area->base || area->size < sizeof(PaddedSevHashTable)) {
+        error_setg(errp, "SEV: guest firmware hashes table area is invalid "
+                         "(base=0x%x size=0x%x)", area->base, area->size);
+        return false;
+    }
+
+    /*
+     * Populate the hashes table in the guest's memory at the OVMF-designated
+     * area for the SEV hashes table
+     */
+    padded_ht = address_space_map(&address_space_memory, area->base,
+                                  &mapped_len, true, attrs);
+    if (!padded_ht || mapped_len != sizeof(*padded_ht)) {
+        error_setg(errp, "SEV: cannot map hashes table guest memory area");
+        return false;
+    }
+
+    if (build_kernel_loader_hashes(padded_ht, ctx, errp)) {
+        if (sev_encrypt_flash((uint8_t *)padded_ht, sizeof(*padded_ht),
+                              errp) < 0) {
+            ret = false;
+        }
+    } else {
         ret = false;
     }
 
Signature · 1 line
2.25.1
#95 lore raw

PATCH v3 45/49 i386/sev: Reorder struct declarations

In reply to Michael Roth

From: Dov Murik <[email protected]>

Move the declaration of PaddedSevHashTable before SevSnpGuest so we can add a new such field to the latter.

No functional change intended.

---
 target/i386/sev.c | 56 +++++++++++++++++++++++------------------------
 1 file changed, 28 insertions(+), 28 deletions(-)
target/i386/sev.c +28 -28
index 0913cb7fed..4bc6004037 100644
@@ -41,6 +41,34 @@
 #include "exec/address-spaces.h"
 #include "qemu/queue.h"
 
+/* hard code sha256 digest size */
+#define HASH_SIZE 32
+
+typedef struct QEMU_PACKED SevHashTableEntry {
+    QemuUUID guid;
+    uint16_t len;
+    uint8_t hash[HASH_SIZE];
+} SevHashTableEntry;
+
+typedef struct QEMU_PACKED SevHashTable {
+    QemuUUID guid;
+    uint16_t len;
+    SevHashTableEntry cmdline;
+    SevHashTableEntry initrd;
+    SevHashTableEntry kernel;
+} SevHashTable;
+
+/*
+ * Data encrypted by sev_encrypt_flash() must be padded to a multiple of
+ * 16 bytes.
+ */
+typedef struct QEMU_PACKED PaddedSevHashTable {
+    SevHashTable ht;
+    uint8_t padding[ROUND_UP(sizeof(SevHashTable), 16) - sizeof(SevHashTable)];
+} PaddedSevHashTable;
+
+QEMU_BUILD_BUG_ON(sizeof(PaddedSevHashTable) % 16 != 0);
+
 OBJECT_DECLARE_SIMPLE_TYPE(SevCommonState, SEV_COMMON)
 OBJECT_DECLARE_SIMPLE_TYPE(SevGuestState, SEV_GUEST)
 OBJECT_DECLARE_SIMPLE_TYPE(SevSnpGuestState, SEV_SNP_GUEST)
@@ -131,34 +159,6 @@ typedef struct QEMU_PACKED SevHashTableDescriptor {
     uint32_t size;
 } SevHashTableDescriptor;
 
-/* hard code sha256 digest size */
-#define HASH_SIZE 32
-
-typedef struct QEMU_PACKED SevHashTableEntry {
-    QemuUUID guid;
-    uint16_t len;
-    uint8_t hash[HASH_SIZE];
-} SevHashTableEntry;
-
-typedef struct QEMU_PACKED SevHashTable {
-    QemuUUID guid;
-    uint16_t len;
-    SevHashTableEntry cmdline;
-    SevHashTableEntry initrd;
-    SevHashTableEntry kernel;
-} SevHashTable;
-
-/*
- * Data encrypted by sev_encrypt_flash() must be padded to a multiple of
- * 16 bytes.
- */
-typedef struct QEMU_PACKED PaddedSevHashTable {
-    SevHashTable ht;
-    uint8_t padding[ROUND_UP(sizeof(SevHashTable), 16) - sizeof(SevHashTable)];
-} PaddedSevHashTable;
-
-QEMU_BUILD_BUG_ON(sizeof(PaddedSevHashTable) % 16 != 0);
-
 static Error *sev_mig_blocker;
 
 static const char *const sev_fw_errlist[] = {
Signature · 1 line
2.25.1
#96 lore raw

PATCH v3 46/49 i386/sev: Allow measured direct kernel boot on SNP

In reply to Michael Roth

From: Dov Murik <[email protected]>

In SNP, the hashes page designated with a specific metadata entry published in AmdSev OVMF.

Therefore, if the user enabled kernel hashes (for measured direct boot), QEMU should prepare the content of hashes table, and during the processing of the metadata entry it copy the content into the designated page and encrypt it.

Note that in SNP (unlike SEV and SEV-ES) the measurements is done in whole 4KB pages. Therefore QEMU zeros the whole page that includes the hashes table, and fills in the kernel hashes area in that page, and then encrypts the whole page. The rest of the page is reserved for SEV launch secrets which are not usable anyway on SNP.

If the user disabled kernel hashes, QEMU pre-validates the kernel hashes page as a zero page.

---
 include/hw/i386/pc.h |  2 ++
 target/i386/sev.c    | 36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+)
include/hw/i386/pc.h +2 -0
index df9a61540d..d9d3a5b5b8 100644
@@ -171,6 +171,8 @@ typedef enum {
     SEV_DESC_TYPE_SNP_SECRETS,
     /* The section contains address that can be used as a CPUID page */
     SEV_DESC_TYPE_CPUID,
+    /* The section contains the region for kernel hashes for measured direct boot */
+    SEV_DESC_TYPE_SNP_KERNEL_HASHES = 0x10,
 
 } ovmf_sev_metadata_desc_type;
 
target/i386/sev.c +36 -0
index 4bc6004037..e2506f74da 100644
@@ -129,6 +129,9 @@ struct SevSnpGuestState {
 
     struct kvm_sev_snp_launch_start kvm_start_conf;
     struct kvm_sev_snp_launch_finish kvm_finish_conf;
+
+    uint32_t kernel_hashes_offset;
+    PaddedSevHashTable *kernel_hashes_data;
 };
 
 #define DEFAULT_GUEST_POLICY    0x1 /* disable debug */
@@ -1117,6 +1120,23 @@ snp_launch_update_cpuid(uint32_t cpuid_addr, void *hva, uint32_t cpuid_len)
     return snp_launch_update_data(cpuid_addr, hva, cpuid_len, KVM_SEV_SNP_PAGE_TYPE_CPUID);
 }
 
+static int
+snp_launch_update_kernel_hashes(SevSnpGuestState *sev_snp, uint32_t addr,
+                                void *hva, uint32_t len)
+{
+    int type = KVM_SEV_SNP_PAGE_TYPE_ZERO;
+    if (sev_snp->sev_common.kernel_hashes) {
+        assert(sev_snp->kernel_hashes_data);
+        assert((sev_snp->kernel_hashes_offset +
+                sizeof(*sev_snp->kernel_hashes_data)) <= len);
+        memset(hva, 0, len);
+        memcpy(hva + sev_snp->kernel_hashes_offset, sev_snp->kernel_hashes_data,
+               sizeof(*sev_snp->kernel_hashes_data));
+        type = KVM_SEV_SNP_PAGE_TYPE_NORMAL;
+    }
+    return snp_launch_update_data(addr, hva, len, type);
+}
+
 static int
 snp_metadata_desc_to_page_type(int desc_type)
 {
@@ -1125,6 +1145,7 @@ snp_metadata_desc_to_page_type(int desc_type)
     case SEV_DESC_TYPE_SNP_SEC_MEM: return KVM_SEV_SNP_PAGE_TYPE_ZERO;
     case SEV_DESC_TYPE_SNP_SECRETS: return KVM_SEV_SNP_PAGE_TYPE_SECRETS;
     case SEV_DESC_TYPE_CPUID: return KVM_SEV_SNP_PAGE_TYPE_CPUID;
+    case SEV_DESC_TYPE_SNP_KERNEL_HASHES: return KVM_SEV_SNP_PAGE_TYPE_NORMAL;
     default: return -1;
     }
 }
@@ -1155,6 +1176,9 @@ snp_populate_metadata_pages(SevSnpGuestState *sev_snp, OvmfSevMetadata *metadata
 
         if (type == KVM_SEV_SNP_PAGE_TYPE_CPUID) {
             ret = snp_launch_update_cpuid(desc->base, hva, desc->len);
+        } else if (desc->type == SEV_DESC_TYPE_SNP_KERNEL_HASHES) {
+            ret = snp_launch_update_kernel_hashes(sev_snp, desc->base, hva,
+                                                  desc->len);
         } else {
             ret = snp_launch_update_data(desc->base, hva, desc->len, type);
         }
@@ -1781,6 +1805,18 @@ bool sev_add_kernel_loader_hashes(SevKernelLoaderContext *ctx, Error **errp)
         return false;
     }
 
+    if (sev_snp_enabled()) {
+        /*
+         * SNP: Populate the hashes table in an area that later in
+         * snp_launch_update_kernel_hashes() will be copied to the guest memory
+         * and encrypted.
+         */
+        SevSnpGuestState *sev_snp_guest = SEV_SNP_GUEST(sev_common);
+        sev_snp_guest->kernel_hashes_offset = area->base & ~TARGET_PAGE_MASK;
+        sev_snp_guest->kernel_hashes_data = g_new0(PaddedSevHashTable, 1);
+        return build_kernel_loader_hashes(sev_snp_guest->kernel_hashes_data, ctx, errp);
+    }
+
     /*
      * Populate the hashes table in the guest's memory at the OVMF-designated
      * area for the SEV hashes table
Signature · 1 line
2.25.1
#97 lore raw

PATCH v3 47/49 hw/i386/sev: Add support to encrypt BIOS when SEV-SNP is enabled

In reply to Michael Roth

TODO: Brijesh as author, me as co-author (vice-versa depending)
drop flash handling? we only support BIOS now

---
 hw/i386/pc_sysfw.c            | 12 +++++++-----
 hw/i386/x86.c                 |  2 +-
 include/hw/i386/x86.h         |  2 +-
 target/i386/sev-sysemu-stub.c |  2 +-
 target/i386/sev.c             | 15 +++++++++++----
 target/i386/sev.h             |  2 +-
 6 files changed, 22 insertions(+), 13 deletions(-)
hw/i386/pc_sysfw.c +7 -5
index 3efabbbab2..9dbb3f7337 100644
@@ -149,6 +149,8 @@ static void pc_system_flash_map(PCMachineState *pcms,
     assert(PC_MACHINE_GET_CLASS(pcms)->pci_enabled);
 
     for (i = 0; i < ARRAY_SIZE(pcms->flash); i++) {
+        hwaddr gpa;
+
         system_flash = pcms->flash[i];
         blk = pflash_cfi01_get_blk(system_flash);
         if (!blk) {
@@ -178,11 +180,11 @@ static void pc_system_flash_map(PCMachineState *pcms,
         }
 
         total_size += size;
+        gpa = 0x100000000ULL - total_size; /* where the flash is mapped */
         qdev_prop_set_uint32(DEVICE(system_flash), "num-blocks",
                              size / FLASH_SECTOR_SIZE);
         sysbus_realize_and_unref(SYS_BUS_DEVICE(system_flash), &error_fatal);
-        sysbus_mmio_map(SYS_BUS_DEVICE(system_flash), 0,
-                        0x100000000ULL - total_size);
+        sysbus_mmio_map(SYS_BUS_DEVICE(system_flash), 0, gpa);
 
         if (i == 0) {
             flash_mem = pflash_cfi01_get_memory(system_flash);
@@ -192,7 +194,7 @@ static void pc_system_flash_map(PCMachineState *pcms,
             if (sev_enabled()) {
                 flash_ptr = memory_region_get_ram_ptr(flash_mem);
                 flash_size = memory_region_size(flash_mem);
-                x86_firmware_configure(flash_ptr, flash_size);
+                x86_firmware_configure(gpa, flash_ptr, flash_size);
             }
         }
     }
@@ -245,7 +247,7 @@ void pc_system_firmware_init(PCMachineState *pcms,
     pc_system_flash_cleanup_unused(pcms);
 }
 
-void x86_firmware_configure(void *ptr, int size)
+void x86_firmware_configure(hwaddr gpa, void *ptr, int size)
 {
     int ret;
 
@@ -262,6 +264,6 @@ void x86_firmware_configure(void *ptr, int size)
             exit(1);
         }
 
-        sev_encrypt_flash(ptr, size, &error_fatal);
+        sev_encrypt_flash(gpa, ptr, size, &error_fatal);
     }
 }
hw/i386/x86.c +1 -1
index 825dc4c735..e3ddc39133 100644
@@ -1161,7 +1161,7 @@ void x86_bios_rom_init(MachineState *ms, const char *default_firmware,
          */
         void *ptr = memory_region_get_ram_ptr(bios);
         load_image_size(filename, ptr, bios_size);
-        x86_firmware_configure(ptr, bios_size);
+        x86_firmware_configure(0x100000000ULL - bios_size, ptr, bios_size);
     } else {
         if (!isapc_ram_fw) {
             memory_region_set_readonly(bios, true);
include/hw/i386/x86.h +1 -1
index 4dc30dcb4d..53dfd95cb2 100644
@@ -143,6 +143,6 @@ void ioapic_init_gsi(GSIState *gsi_state, Object *parent);
 DeviceState *ioapic_init_secondary(GSIState *gsi_state);
 
 /* pc_sysfw.c */
-void x86_firmware_configure(void *ptr, int size);
+void x86_firmware_configure(hwaddr gpa, void *ptr, int size);
 
 #endif
target/i386/sev-sysemu-stub.c +1 -1
index 96e1c15cc3..6af643e3a1 100644
@@ -42,7 +42,7 @@ void qmp_sev_inject_launch_secret(const char *packet_header, const char *secret,
     error_setg(errp, "SEV is not available in this QEMU");
 }
 
-int sev_encrypt_flash(uint8_t *ptr, uint64_t len, Error **errp)
+int sev_encrypt_flash(hwaddr gpa, uint8_t *ptr, uint64_t len, Error **errp)
 {
     g_assert_not_reached();
 }
target/i386/sev.c +11 -4
index e2506f74da..d8e6aba67c 100644
@@ -1449,7 +1449,7 @@ err:
 }
 
 int
-sev_encrypt_flash(uint8_t *ptr, uint64_t len, Error **errp)
+sev_encrypt_flash(hwaddr gpa, uint8_t *ptr, uint64_t len, Error **errp)
 {
     SevCommonState *sev_common = SEV_COMMON(MACHINE(qdev_get_machine())->cgs);
 
@@ -1459,7 +1459,14 @@ sev_encrypt_flash(uint8_t *ptr, uint64_t len, Error **errp)
 
     /* if SEV is in update state then encrypt the data else do nothing */
     if (sev_check_state(sev_common, SEV_STATE_LAUNCH_UPDATE)) {
-        int ret = sev_launch_update_data(SEV_GUEST(sev_common), ptr, len);
+        int ret;
+
+        if (sev_snp_enabled()) {
+            ret = snp_launch_update_data(gpa, ptr, len,
+                                         KVM_SEV_SNP_PAGE_TYPE_NORMAL);
+        } else {
+            ret = sev_launch_update_data(SEV_GUEST(sev_common), ptr, len);
+        }
         if (ret < 0) {
             error_setg(errp, "SEV: Failed to encrypt pflash rom");
             return ret;
@@ -1829,8 +1836,8 @@ bool sev_add_kernel_loader_hashes(SevKernelLoaderContext *ctx, Error **errp)
     }
 
     if (build_kernel_loader_hashes(padded_ht, ctx, errp)) {
-        if (sev_encrypt_flash((uint8_t *)padded_ht, sizeof(*padded_ht),
-                              errp) < 0) {
+        if (sev_encrypt_flash(area->base, (uint8_t *)padded_ht,
+                              sizeof(*padded_ht), errp) < 0) {
             ret = false;
         }
     } else {
target/i386/sev.h +1 -1
index 5cbfc3365b..d570777769 100644
@@ -59,7 +59,7 @@ uint32_t sev_get_cbit_position(void);
 uint32_t sev_get_reduced_phys_bits(void);
 bool sev_add_kernel_loader_hashes(SevKernelLoaderContext *ctx, Error **errp);
 
-int sev_encrypt_flash(uint8_t *ptr, uint64_t len, Error **errp);
+int sev_encrypt_flash(hwaddr gpa, uint8_t *ptr, uint64_t len, Error **errp);
 int sev_inject_launch_secret(const char *hdr, const char *secret,
                              uint64_t gpa, Error **errp);
 
Signature · 1 line
2.25.1
#98 lore raw

Re: [PATCH v3 47/49] hw/i386/sev: Add support to encrypt BIOS when SEV-SNP is enabled

Quoted 2 lines · Michael Roth

TODO: Brijesh as author, me as co-author (vice-versa depending)
drop flash handling? we only support BIOS now

A reminder that this commit message needs fixing.

Quoted 10 lines
---
 hw/i386/pc_sysfw.c            | 12 +++++++-----
 hw/i386/x86.c                 |  2 +-
 include/hw/i386/x86.h         |  2 +-
 target/i386/sev-sysemu-stub.c |  2 +-
 target/i386/sev.c             | 15 +++++++++++----
 target/i386/sev.h             |  2 +-
 6 files changed, 22 insertions(+), 13 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#99 lore raw

Re: [PATCH v3 47/49] hw/i386/sev: Add support to encrypt BIOS when SEV-SNP is enabled

Quoted 5 lines · Daniel P. Berrangé
Quoted 2 lines · Michael Roth

TODO: Brijesh as author, me as co-author (vice-versa depending)
drop flash handling? we only support BIOS now

A reminder that this commit message needs fixing.

Sorry, definitely meant to fix this one up before submitting. I've gone ahead and force-pushed an updated tree to same qemu-v3-rc branch. The only change is proper attribution/commit message for this patch:

https://github.com/AMDESE/qemu/commit/c54618a1cc23f2398e6c3af6f3cf140c4901347c

-Mike

Quoted 19 lines
Quoted 11 lines
---
 hw/i386/pc_sysfw.c            | 12 +++++++-----
 hw/i386/x86.c                 |  2 +-
 include/hw/i386/x86.h         |  2 +-
 target/i386/sev-sysemu-stub.c |  2 +-
 target/i386/sev.c             | 15 +++++++++++----
 target/i386/sev.h             |  2 +-
 6 files changed, 22 insertions(+), 13 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#100 lore raw

PATCH v3 48/49 hw/i386/sev: Use guest_memfd for legacy ROMs

In reply to Michael Roth

TODO: make this SNP-specific if TDX disables legacy ROMs in general

Current SNP guest kernels will attempt to access these regions with with C-bit set, so guest_memfd is needed to handle that. Otherwise, kvm_convert_memory() will fail when the guest kernel tries to access it and QEMU attempts to call KVM_SET_MEMORY_ATTRIBUTES to set these ranges to private.

Whether guests should actually try to access ROM regions in this way (or need to deal with legacy ROM regions at all), is a separate issue to be addressed on kernel side, but current SNP guest kernels will exhibit this behavior and so this handling is needed to allow QEMU to continue running existing SNP guest kernels.

---
 hw/i386/pc.c       | 13 +++++++++----
 hw/i386/pc_sysfw.c | 13 ++++++++++---
 2 files changed, 19 insertions(+), 7 deletions(-)
hw/i386/pc.c +9 -4
index feb7a93083..5feaeb43ee 100644
@@ -1011,10 +1011,15 @@ void pc_memory_init(PCMachineState *pcms,
     pc_system_firmware_init(pcms, rom_memory);
 
     option_rom_mr = g_malloc(sizeof(*option_rom_mr));
-    memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
-                           &error_fatal);
-    if (pcmc->pci_enabled) {
-        memory_region_set_readonly(option_rom_mr, true);
+    if (machine_require_guest_memfd(machine)) {
+        memory_region_init_ram_guest_memfd(option_rom_mr, NULL, "pc.rom",
+                                           PC_ROM_SIZE, &error_fatal);
+    } else {
+        memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
+                               &error_fatal);
+        if (pcmc->pci_enabled) {
+            memory_region_set_readonly(option_rom_mr, true);
+        }
     }
     memory_region_add_subregion_overlap(rom_memory,
                                         PC_ROM_MIN_VGA,
hw/i386/pc_sysfw.c +10 -3
index 9dbb3f7337..850f86edd4 100644
@@ -54,8 +54,13 @@ static void pc_isa_bios_init(MemoryRegion *rom_memory,
     /* map the last 128KB of the BIOS in ISA space */
     isa_bios_size = MIN(flash_size, 128 * KiB);
     isa_bios = g_malloc(sizeof(*isa_bios));
-    memory_region_init_ram(isa_bios, NULL, "isa-bios", isa_bios_size,
-                           &error_fatal);
+    if (machine_require_guest_memfd(current_machine)) {
+        memory_region_init_ram_guest_memfd(isa_bios, NULL, "isa-bios",
+                                           isa_bios_size, &error_fatal);
+    } else {
+        memory_region_init_ram(isa_bios, NULL, "isa-bios", isa_bios_size,
+                               &error_fatal);
+    }
     memory_region_add_subregion_overlap(rom_memory,
                                         0x100000 - isa_bios_size,
                                         isa_bios,
@@ -68,7 +73,9 @@ static void pc_isa_bios_init(MemoryRegion *rom_memory,
            ((uint8_t*)flash_ptr) + (flash_size - isa_bios_size),
            isa_bios_size);
 
-    memory_region_set_readonly(isa_bios, true);
+    if (!machine_require_guest_memfd(current_machine)) {
+        memory_region_set_readonly(isa_bios, true);
+    }
 }
 
 static PFlashCFI01 *pc_pflash_create(PCMachineState *pcms,
Signature · 1 line
2.25.1
#101 lore raw

Re: [PATCH v3 48/49] hw/i386/sev: Use guest_memfd for legacy ROMs

Quoted 1 line · Michael Roth

TODO: make this SNP-specific if TDX disables legacy ROMs in general

TDX disables pc.rom, not disable isa-bios. IIRC, TDX doesn't need pc pflash. Xiaoyao can chime in.

Thanks,

Quoted 78 lines

Current SNP guest kernels will attempt to access these regions with with C-bit set, so guest_memfd is needed to handle that. Otherwise, kvm_convert_memory() will fail when the guest kernel tries to access it and QEMU attempts to call KVM_SET_MEMORY_ATTRIBUTES to set these ranges to private.

Whether guests should actually try to access ROM regions in this way (or need to deal with legacy ROM regions at all), is a separate issue to be addressed on kernel side, but current SNP guest kernels will exhibit this behavior and so this handling is needed to allow QEMU to continue running existing SNP guest kernels.

---
 hw/i386/pc.c       | 13 +++++++++----
 hw/i386/pc_sysfw.c | 13 ++++++++++---
 2 files changed, 19 insertions(+), 7 deletions(-)
hw/i386/pc.c +9 -4
index feb7a93083..5feaeb43ee 100644
@@ -1011,10 +1011,15 @@ void pc_memory_init(PCMachineState *pcms,
     pc_system_firmware_init(pcms, rom_memory);
 
     option_rom_mr = g_malloc(sizeof(*option_rom_mr));
-    memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
-                           &error_fatal);
-    if (pcmc->pci_enabled) {
-        memory_region_set_readonly(option_rom_mr, true);
+    if (machine_require_guest_memfd(machine)) {
+        memory_region_init_ram_guest_memfd(option_rom_mr, NULL, "pc.rom",
+                                           PC_ROM_SIZE, &error_fatal);
+    } else {
+        memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
+                               &error_fatal);
+        if (pcmc->pci_enabled) {
+            memory_region_set_readonly(option_rom_mr, true);
+        }
     }
     memory_region_add_subregion_overlap(rom_memory,
                                         PC_ROM_MIN_VGA,
hw/i386/pc_sysfw.c +10 -3
index 9dbb3f7337..850f86edd4 100644
@@ -54,8 +54,13 @@ static void pc_isa_bios_init(MemoryRegion *rom_memory,
     /* map the last 128KB of the BIOS in ISA space */
     isa_bios_size = MIN(flash_size, 128 * KiB);
     isa_bios = g_malloc(sizeof(*isa_bios));
-    memory_region_init_ram(isa_bios, NULL, "isa-bios", isa_bios_size,
-                           &error_fatal);
+    if (machine_require_guest_memfd(current_machine)) {
+        memory_region_init_ram_guest_memfd(isa_bios, NULL, "isa-bios",
+                                           isa_bios_size, &error_fatal);
+    } else {
+        memory_region_init_ram(isa_bios, NULL, "isa-bios", isa_bios_size,
+                               &error_fatal);
+    }
     memory_region_add_subregion_overlap(rom_memory,
                                         0x100000 - isa_bios_size,
                                         isa_bios,
@@ -68,7 +73,9 @@ static void pc_isa_bios_init(MemoryRegion *rom_memory,
            ((uint8_t*)flash_ptr) + (flash_size - isa_bios_size),
            isa_bios_size);
 
-    memory_region_set_readonly(isa_bios, true);
+    if (!machine_require_guest_memfd(current_machine)) {
+        memory_region_set_readonly(isa_bios, true);
+    }
 }
 
 static PFlashCFI01 *pc_pflash_create(PCMachineState *pcms,
Signature · 1 line
2.25.1
Signature · 1 line
Isaku Yamahata <[email protected]>
#102 lore raw

Re: [PATCH v3 48/49] hw/i386/sev: Use guest_memfd for legacy ROMs

Quoted 6 lines · Isaku Yamahata
Quoted 1 line · Michael Roth

TODO: make this SNP-specific if TDX disables legacy ROMs in general

TDX disables pc.rom, not disable isa-bios. IIRC, TDX doesn't need pc pflash.

Not TDX doesn't need pc pflash, but TDX cannot support pflash.

Can SNP support the behavior of pflash? That what's got changed will be synced back to OVMF file?

Quoted 83 lines

Xiaoyao can chime in.

Thanks,

Quoted 78 lines

Current SNP guest kernels will attempt to access these regions with with C-bit set, so guest_memfd is needed to handle that. Otherwise, kvm_convert_memory() will fail when the guest kernel tries to access it and QEMU attempts to call KVM_SET_MEMORY_ATTRIBUTES to set these ranges to private.

Whether guests should actually try to access ROM regions in this way (or need to deal with legacy ROM regions at all), is a separate issue to be addressed on kernel side, but current SNP guest kernels will exhibit this behavior and so this handling is needed to allow QEMU to continue running existing SNP guest kernels.

---
  hw/i386/pc.c       | 13 +++++++++----
  hw/i386/pc_sysfw.c | 13 ++++++++++---
  2 files changed, 19 insertions(+), 7 deletions(-)
hw/i386/pc.c +9 -4
index feb7a93083..5feaeb43ee 100644
@@ -1011,10 +1011,15 @@ void pc_memory_init(PCMachineState *pcms,
      pc_system_firmware_init(pcms, rom_memory);
 
      option_rom_mr = g_malloc(sizeof(*option_rom_mr));
-    memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
-                           &error_fatal);
-    if (pcmc->pci_enabled) {
-        memory_region_set_readonly(option_rom_mr, true);
+    if (machine_require_guest_memfd(machine)) {
+        memory_region_init_ram_guest_memfd(option_rom_mr, NULL, "pc.rom",
+                                           PC_ROM_SIZE, &error_fatal);
+    } else {
+        memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
+                               &error_fatal);
+        if (pcmc->pci_enabled) {
+            memory_region_set_readonly(option_rom_mr, true);
+        }
      }
      memory_region_add_subregion_overlap(rom_memory,
                                          PC_ROM_MIN_VGA,
hw/i386/pc_sysfw.c +10 -3
index 9dbb3f7337..850f86edd4 100644
@@ -54,8 +54,13 @@ static void pc_isa_bios_init(MemoryRegion *rom_memory,
      /* map the last 128KB of the BIOS in ISA space */
      isa_bios_size = MIN(flash_size, 128 * KiB);
      isa_bios = g_malloc(sizeof(*isa_bios));
-    memory_region_init_ram(isa_bios, NULL, "isa-bios", isa_bios_size,
-                           &error_fatal);
+    if (machine_require_guest_memfd(current_machine)) {
+        memory_region_init_ram_guest_memfd(isa_bios, NULL, "isa-bios",
+                                           isa_bios_size, &error_fatal);
+    } else {
+        memory_region_init_ram(isa_bios, NULL, "isa-bios", isa_bios_size,
+                               &error_fatal);
+    }
      memory_region_add_subregion_overlap(rom_memory,
                                          0x100000 - isa_bios_size,
                                          isa_bios,
@@ -68,7 +73,9 @@ static void pc_isa_bios_init(MemoryRegion *rom_memory,
             ((uint8_t*)flash_ptr) + (flash_size - isa_bios_size),
             isa_bios_size);
 
-    memory_region_set_readonly(isa_bios, true);
+    if (!machine_require_guest_memfd(current_machine)) {
+        memory_region_set_readonly(isa_bios, true);
+    }
  }
 
  static PFlashCFI01 *pc_pflash_create(PCMachineState *pcms,
Signature · 1 line
2.25.1
#103 lore raw

Re: [PATCH v3 48/49] hw/i386/sev: Use guest_memfd for legacy ROMs

Quoted 4 lines · Xiaoyao Li
Quoted 3 lines · Isaku Yamahata

On Wed, Mar 20, 2024 at 03:39:44AM -0500,
Michael Roth <[email protected]> wrote:

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

TODO: make this SNP-specific if TDX disables legacy ROMs in general

TDX disables pc.rom, not disable isa-bios. IIRC, TDX doesn't need pc pflash.

Not TDX doesn't need pc pflash, but TDX cannot support pflash.

Can SNP support the behavior of pflash? That what's got changed will be synced back to OVMF file?

For split OVMF files (separate FD for CODE vs. VARS) it can, but it requires special handling in OVMF to handle MMIO to the VARS area using direct MMIO hypercalls rather than relying on MMIO emulation. Here's the relevant OVMF commit:

commit 437eb3f7a8db7681afe0e6064d3a8edb12abb766
Author: Tom Lendacky <[email protected]>
Date: Wed Aug 12 15:21:42 2020 -0500

OvmfPkg/QemuFlashFvbServicesRuntimeDxe: Bypass flash detection with SEV-ES

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2198

For SNP, the plan is to continue to use -bios to handle the actual CODE/BIOS FD, but to allow the use of pflash,unit=0 to handle the VARS fd if a separate/persistent store is desired. This allows us to continue using read-only memslots on the VARS/pflash side without being at odds with the fact that read-only memslots are no longer supported for private memslots (since VARS doesn't need to be measured/mapped as private), and limiting the special handling to -bios where TDX/SNP both need private memslots.

This is roughly how things will look with v4 of this series:

https://github.com/AMDESE/qemu/commit/21fff075372ad25b2d09c5e416349c2b353fdb4c

I think (if needed) TDX could in theory take a similar approach with similar modifications to OVMF and providing an option for a split CODE/VARS variant.

-Mike

Quoted 83 lines
Quoted 81 lines

Xiaoyao can chime in.

Thanks,

Quoted 75 lines

Current SNP guest kernels will attempt to access these regions with with C-bit set, so guest_memfd is needed to handle that. Otherwise, kvm_convert_memory() will fail when the guest kernel tries to access it and QEMU attempts to call KVM_SET_MEMORY_ATTRIBUTES to set these ranges to private.

Whether guests should actually try to access ROM regions in this way (or need to deal with legacy ROM regions at all), is a separate issue to be addressed on kernel side, but current SNP guest kernels will exhibit this behavior and so this handling is needed to allow QEMU to continue running existing SNP guest kernels.

---
  hw/i386/pc.c       | 13 +++++++++----
  hw/i386/pc_sysfw.c | 13 ++++++++++---
  2 files changed, 19 insertions(+), 7 deletions(-)
hw/i386/pc.c +9 -4
index feb7a93083..5feaeb43ee 100644
@@ -1011,10 +1011,15 @@ void pc_memory_init(PCMachineState *pcms,
      pc_system_firmware_init(pcms, rom_memory);
      option_rom_mr = g_malloc(sizeof(*option_rom_mr));
-    memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
-                           &error_fatal);
-    if (pcmc->pci_enabled) {
-        memory_region_set_readonly(option_rom_mr, true);
+    if (machine_require_guest_memfd(machine)) {
+        memory_region_init_ram_guest_memfd(option_rom_mr, NULL, "pc.rom",
+                                           PC_ROM_SIZE, &error_fatal);
+    } else {
+        memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
+                               &error_fatal);
+        if (pcmc->pci_enabled) {
+            memory_region_set_readonly(option_rom_mr, true);
+        }
      }
      memory_region_add_subregion_overlap(rom_memory,
                                          PC_ROM_MIN_VGA,
hw/i386/pc_sysfw.c +10 -3
index 9dbb3f7337..850f86edd4 100644
@@ -54,8 +54,13 @@ static void pc_isa_bios_init(MemoryRegion *rom_memory,
      /* map the last 128KB of the BIOS in ISA space */
      isa_bios_size = MIN(flash_size, 128 * KiB);
      isa_bios = g_malloc(sizeof(*isa_bios));
-    memory_region_init_ram(isa_bios, NULL, "isa-bios", isa_bios_size,
-                           &error_fatal);
+    if (machine_require_guest_memfd(current_machine)) {
+        memory_region_init_ram_guest_memfd(isa_bios, NULL, "isa-bios",
+                                           isa_bios_size, &error_fatal);
+    } else {
+        memory_region_init_ram(isa_bios, NULL, "isa-bios", isa_bios_size,
+                               &error_fatal);
+    }
      memory_region_add_subregion_overlap(rom_memory,
                                          0x100000 - isa_bios_size,
                                          isa_bios,
@@ -68,7 +73,9 @@ static void pc_isa_bios_init(MemoryRegion *rom_memory,
             ((uint8_t*)flash_ptr) + (flash_size - isa_bios_size),
             isa_bios_size);
-    memory_region_set_readonly(isa_bios, true);
+    if (!machine_require_guest_memfd(current_machine)) {
+        memory_region_set_readonly(isa_bios, true);
+    }
  }
  static PFlashCFI01 *pc_pflash_create(PCMachineState *pcms,
Signature · 1 line
2.25.1
#104 lore raw

PATCH v3 49/49 hw/i386: Add support for loading BIOS using guest_memfd

In reply to Michael Roth

TODO: - Add proper handling for non-64K-aligned BIOS images.
- Add proper handling for BIOS pflash area which should be initially mapped as shared, resulting in unecessary KVM_EXIT_MEMORY_FAULTs

When guest_memfd is enabled, the BIOS is generally part of the initial encrypted guest image and will be accessed as private guest memory. Add the necessary changes to set up the associated RAM region with a guest_memfd backend to allow for this.

Current support centers around using -bios to load the BIOS data. Support for loading the BIOS via pflash requires additional enablement since those interfaces rely on the use of ROM memory regions which make use of the KVM_MEM_READONLY memslot flag, which is not supported for guest_memfd-backed memslots.

---
 hw/i386/x86.c | 36 +++++++++++++++++++++++-------------
 1 file changed, 23 insertions(+), 13 deletions(-)
hw/i386/x86.c +23 -13
index e3ddc39133..ea2d03cc02 100644
@@ -1147,10 +1147,18 @@ void x86_bios_rom_init(MachineState *ms, const char *default_firmware,
     }
     if (bios_size <= 0 ||
         (bios_size % 65536) != 0) {
-        goto bios_error;
+        g_warning("%s: Unaligned BIOS size %d", __func__, bios_size);
+        if (!machine_require_guest_memfd(ms)) {
+            goto bios_error;
+        }
     }
     bios = g_malloc(sizeof(*bios));
-    memory_region_init_ram(bios, NULL, "pc.bios", bios_size, &error_fatal);
+    if (machine_require_guest_memfd(ms)) {
+        memory_region_init_ram_guest_memfd(bios, NULL, "pc.bios", bios_size,
+                                           &error_fatal);
+    } else {
+        memory_region_init_ram(bios, NULL, "pc.bios", bios_size, &error_fatal);
+    }
     if (sev_enabled()) {
         /*
          * The concept of a "reset" simply doesn't exist for
@@ -1173,17 +1181,19 @@ void x86_bios_rom_init(MachineState *ms, const char *default_firmware,
     }
     g_free(filename);
 
-    /* map the last 128KB of the BIOS in ISA space */
-    isa_bios_size = MIN(bios_size, 128 * KiB);
-    isa_bios = g_malloc(sizeof(*isa_bios));
-    memory_region_init_alias(isa_bios, NULL, "isa-bios", bios,
-                             bios_size - isa_bios_size, isa_bios_size);
-    memory_region_add_subregion_overlap(rom_memory,
-                                        0x100000 - isa_bios_size,
-                                        isa_bios,
-                                        1);
-    if (!isapc_ram_fw) {
-        memory_region_set_readonly(isa_bios, true);
+    if (!machine_require_guest_memfd(ms)) {
+        /* map the last 128KB of the BIOS in ISA space */
+        isa_bios_size = MIN(bios_size, 128 * KiB);
+        isa_bios = g_malloc(sizeof(*isa_bios));
+        memory_region_init_alias(isa_bios, NULL, "isa-bios", bios,
+                                 bios_size - isa_bios_size, isa_bios_size);
+        memory_region_add_subregion_overlap(rom_memory,
+                                            0x100000 - isa_bios_size,
+                                            isa_bios,
+                                            1);
+        if (!isapc_ram_fw) {
+            memory_region_set_readonly(isa_bios, true);
+        }
     }
 
     /* map all the bios at the top of memory */
Signature · 1 line
2.25.1
maintainer
#105 lore raw
In reply to Michael Roth
Quoted 29 lines · Michael Roth

These patches implement SEV-SNP base support along with CPUID enforcement support for QEMU, and are also available at:

https://github.com/amdese/qemu/commits/snp-v3-rfc

they are based on top of the following patchset from Paolo:

"[PATCH 0/7] target/i386: VM type infrastructure and KVM_SEV_INIT2 support"
https://lists.gnu.org/archive/html/qemu-devel/2024-03/msg04663.html

Patch Layout
------------

01-05: Various changes needed to handle new header files in kvm-next tree
and some hacks to get a functional header sync in place for building this series.
06-18: These are patches directly plucked from Xiaoyao's TDX v5 patchset[1]
that implement common dependencies between SNP/TDX like base guest_memfd, KVM_EXIT_MEMORY_FAULT handling (with a small FIXUP), and mechanisms to disable SMM. We would've also needed some of the basic infrastructure for handling specifying VM types for KVM_CREATE, but much of that is now part of the sevinit2 series this patchset is based on. Ideally all these patches, once stable, could be maintained in a common tree so that future SNP/TDX patchsets can be more easily iterated on/reviewed.
19-20: Patches introduced by this series that are possible candidate for a
common tree.
shared/private pages when things like VFIO are in use.

Quoted 8 lines

21-32: Introduction of sev-snp-guest object and various configuration
requirements for SNP.
33-36: Handling for various KVM_EXIT_VMGEXIT events that are handled in
userspace.
37-49: Support for creating a cryptographic "launch" context and populating
various OVMF metadata pages, BIOS regions, and vCPU/VMSA pages with the initial encrypted/measured/validated launch data prior to launching the SNP guest.

I reviewed the non-SEV bits of patches 21-46 and it looks nicely self-contained. That's pretty much expected but still good news.

I didn't look closely at the SEV-SNP code for obvious reasons (it's only been one hour :)), except for the object-oriented aesthetics which I have remarked upon. However, they seem to be in good shape.

I will now focus on reviewing patches 6-20. This way we can prepare a common tree for SEV_INIT2/SNP/TDX, for both vendors to build upon.

Thanks for posting this, and thanks to the Intel people too for the previous work on the guest_memfd parts!

Paolo

maintainer
#106 lore raw
Quoted 2 lines · Paolo Bonzini

I will now focus on reviewing patches 6-20. This way we can prepare a common tree for SEV_INIT2/SNP/TDX, for both vendors to build upon.

Ok, the attachment is the delta that I have. The only major change is requiring discard (thus effectively blocking VFIO support for SEV-SNP/TDX, at least for now).

I will push it shortly to the same sevinit2 branch, and will post the patches sometime soon.

Xiaoyao, you can use that branch too (it's on
https://gitlab.com/bonzini/qemu) as the basis for your TDX work.

Paolo

accel/kvm/kvm-all.c +12 -34
index bf0ae0c8adb..428468950d9 100644
@@ -285,19 +285,8 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo
 {
     KVMState *s = kvm_state;
     struct kvm_userspace_memory_region2 mem;
-    static int cap_user_memory2 = -1;
     int ret;
 
-    if (cap_user_memory2 == -1) {
-        cap_user_memory2 = kvm_check_extension(s, KVM_CAP_USER_MEMORY2);
-    }
-
-    if (!cap_user_memory2 && slot->guest_memfd >= 0) {
-        error_report("%s, KVM doesn't support KVM_CAP_USER_MEMORY2,"
-                     " which is required by guest memfd!", __func__);
-        exit(1);
-    }
-
     mem.slot = slot->slot | (kml->as_id << 16);
     mem.guest_phys_addr = slot->start_addr;
     mem.userspace_addr = (unsigned long)slot->ram;
@@ -310,7 +299,7 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo
          * value. This is needed based on KVM commit 75d61fbc. */
         mem.memory_size = 0;
 
-        if (cap_user_memory2) {
+        if (kvm_guest_memfd_supported) {
             ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
         } else {
             ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
@@ -320,7 +309,7 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo
         }
     }
     mem.memory_size = slot->memory_size;
-    if (cap_user_memory2) {
+    if (kvm_guest_memfd_supported) {
         ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
     } else {
         ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
@@ -332,7 +321,7 @@ err:
                               mem.userspace_addr, mem.guest_memfd,
                               mem.guest_memfd_offset, ret);
     if (ret < 0) {
-        if (cap_user_memory2) {
+        if (kvm_guest_memfd_supported) {
                 error_report("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d,"
                         " start=0x%" PRIx64 ", size=0x%" PRIx64 ","
                         " flags=0x%" PRIx32 ", guest_memfd=%" PRId32 ","
@@ -502,6 +491,7 @@ static int kvm_mem_flags(MemoryRegion *mr)
         flags |= KVM_MEM_READONLY;
     }
     if (memory_region_has_guest_memfd(mr)) {
+        assert(kvm_guest_memfd_supported);
         flags |= KVM_MEM_GUEST_MEMFD;
     }
     return flags;
@@ -1310,18 +1300,7 @@ static int kvm_set_memory_attributes(hwaddr start, hwaddr size, uint64_t attr)
     struct kvm_memory_attributes attrs;
     int r;
 
-    if (kvm_supported_memory_attributes == 0) {
-        error_report("No memory attribute supported by KVM\n");
-        return -EINVAL;
-    }
-
-    if ((attr & kvm_supported_memory_attributes) != attr) {
-        error_report("memory attribute 0x%lx not supported by KVM,"
-                     " supported bits are 0x%lx\n",
-                     attr, kvm_supported_memory_attributes);
-        return -EINVAL;
-    }
-
+    assert((attr & kvm_supported_memory_attributes) == attr);
     attrs.attributes = attr;
     attrs.address = start;
     attrs.size = size;
@@ -2488,11 +2467,14 @@ static int kvm_init(MachineState *ms)
     }
     s->as = g_new0(struct KVMAs, s->nr_as);
 
-    kvm_guest_memfd_supported = kvm_check_extension(s, KVM_CAP_GUEST_MEMFD);
-
     ret = kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
     kvm_supported_memory_attributes = ret > 0 ? ret : 0;
 
+    kvm_guest_memfd_supported =
+        kvm_check_extension(s, KVM_CAP_GUEST_MEMFD) &&
+        kvm_check_extension(s, KVM_CAP_USER_MEMORY2) &&
+        (kvm_supported_memory_attributes & KVM_MEMORY_ATTRIBUTE_PRIVATE);
+
     if (object_property_find(OBJECT(current_machine), "kvm-type")) {
         g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine),
                                                             "kvm-type",
@@ -2962,14 +2944,10 @@ int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
                 */
                 return 0;
             } else {
-                ret = ram_block_discard_is_disabled()
-                      ? ram_block_discard_range(rb, offset, size)
-                      : 0;
+                ret = ram_block_discard_range(rb, offset, size);
             }
         } else {
-            ret = ram_block_discard_is_disabled()
-                  ? ram_block_discard_guest_memfd_range(rb, offset, size)
-                  : 0;
+            ret = ram_block_discard_guest_memfd_range(rb, offset, size);
         }
     } else {
         error_report("Convert non guest_memfd backed memory region "
system/physmem.c +21 -3
index 8be8053cf77..b839be58538 100644
@@ -1810,6 +1810,7 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
     const bool shared = qemu_ram_is_shared(new_block);
     RAMBlock *block;
     RAMBlock *last_block = NULL;
+    bool free_on_error = false;
     ram_addr_t old_ram_size, new_ram_size;
     Error *err = NULL;
 
@@ -1839,17 +1840,26 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
                 return;
             }
             memory_try_enable_merging(new_block->host, new_block->max_length);
+            free_on_error = true;
         }
     }
 
-    if (kvm_enabled() && (new_block->flags & RAM_GUEST_MEMFD)) {
+    if (new_block->flags & RAM_GUEST_MEMFD) {
+        assert(kvm_enabled());
         assert(new_block->guest_memfd < 0);
 
+        if (ram_block_discard_require(true) < 0) {
+            error_setg_errno(errp, errno,
+                             "cannot set up private guest memory: discard currently blocked");
+            error_append_hint(errp, "Are you using assigned devices?\n");
+            goto out_free;
+        }
+
         new_block->guest_memfd = kvm_create_guest_memfd(new_block->max_length,
                                                         0, errp);
         if (new_block->guest_memfd < 0) {
             qemu_mutex_unlock_ramlist();
-            return;
+            goto out_free;
         }
     }
 
@@ -1901,6 +1911,13 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
         ram_block_notify_add(new_block->host, new_block->used_length,
                              new_block->max_length);
     }
+    return;
+
+out_free:
+    if (free_on_error) {
+        qemu_anon_ram_free(new_block->host, new_block->max_length);
+        new_block->host = NULL;
+    }
 }
 
 #ifdef CONFIG_POSIX
@@ -2032,7 +2049,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
     int align;
 
     assert((ram_flags & ~(RAM_SHARED | RAM_RESIZEABLE | RAM_PREALLOC |
-                          RAM_NORESERVE| RAM_GUEST_MEMFD)) == 0);
+                          RAM_NORESERVE | RAM_GUEST_MEMFD)) == 0);
     assert(!host ^ (ram_flags & RAM_PREALLOC));
 
     align = qemu_real_host_page_size();
@@ -2101,6 +2118,7 @@ static void reclaim_ramblock(RAMBlock *block)
 
     if (block->guest_memfd >= 0) {
         close(block->guest_memfd);
+        ram_block_discard_require(false);
     }
 
     g_free(block);
#107 lore raw
Quoted 8 lines · Paolo Bonzini
Quoted 2 lines · Paolo Bonzini

I will now focus on reviewing patches 6-20. This way we can prepare a common tree for SEV_INIT2/SNP/TDX, for both vendors to build upon.

Ok, the attachment is the delta that I have. The only major change is requiring discard (thus effectively blocking VFIO support for SEV-SNP/TDX, at least for now).

Quoted 5 lines

I will push it shortly to the same sevinit2 branch, and will post the patches sometime soon.

Xiaoyao, you can use that branch too (it's on
https://gitlab.com/bonzini/qemu) as the basis for your TDX work.

Sure, it's really a good news for us.

BTW, there are some minor comments on guest_memfd patches of my v5 post[*]. Could you please resolve them it your branch?

[*] https://lore.kernel.org/qemu-devel/[email protected]/

Quoted 1 line

Paolo

#108 lore raw
In reply to Michael Roth
Quoted 11 lines · Michael Roth

Testing
-------

This series has been tested against the following host kernel tree, which is a snapshot of the latest WIP SNP hypervisor tree at the time of this posting. It will likely not be kept up to date afterward, so please keep an eye upstream or official AMDESE github if you are looking for the latest some time after this posting:

https://github.com/mdroth/linux/commits/snp-host-v12-wip40/

I just noticed I had a necessary local change that wasn't included in the initial push of this branch. I've updated the branch now, but just wanted to post a heads-up in case anyone was having issues.

-Mike

#109 lore raw
In reply to Michael Roth
Quoted 7 lines · Michael Roth

These patches implement SEV-SNP base support along with CPUID enforcement support for QEMU, and are also available at:

Quoted 6 lines

https://github.com/amdese/qemu/commits/snp-v3-rfc

they are based on top of the following patchset from Paolo:

"[PATCH 0/7] target/i386: VM type infrastructure and KVM_SEV_INIT2 support"
https://lists.gnu.org/archive/html/qemu-devel/2024-03/msg04663.html

Can you please also CC me on future revisions of this patchset?

Thanks!

Quoted 226 lines

Patch Layout
------------

01-05: Various changes needed to handle new header files in kvm-next tree
and some hacks to get a functional header sync in place for building this series.
06-18: These are patches directly plucked from Xiaoyao's TDX v5 patchset[1]
that implement common dependencies between SNP/TDX like base guest_memfd, KVM_EXIT_MEMORY_FAULT handling (with a small FIXUP), and mechanisms to disable SMM. We would've also needed some of the basic infrastructure for handling specifying VM types for KVM_CREATE, but much of that is now part of the sevinit2 series this patchset is based on. Ideally all these patches, once stable, could be maintained in a common tree so that future SNP/TDX patchsets can be more easily iterated on/reviewed.
19-20: Patches introduced by this series that are possible candidate for a
common tree.
shared/private pages when things like VFIO are in use.
21-32: Introduction of sev-snp-guest object and various configuration
requirements for SNP.
33-36: Handling for various KVM_EXIT_VMGEXIT events that are handled in
userspace.
37-49: Support for creating a cryptographic "launch" context and populating
various OVMF metadata pages, BIOS regions, and vCPU/VMSA pages with the initial encrypted/measured/validated launch data prior to launching the SNP guest.

Testing
-------

This series has been tested against the following host kernel tree, which is a snapshot of the latest WIP SNP hypervisor tree at the time of this posting. It will likely not be kept up to date afterward, so please keep an eye upstream or official AMDESE github if you are looking for the latest some time after this posting:

https://github.com/mdroth/linux/commits/snp-host-v12-wip40/

A patched OVMF is also needed due to upstream KVM no longer supporting MMIO ranges that are mapped as private. It is recommended you build the AmdSevX64 variant as it provides the kernel-hashing support present in this series:

https://github.com/mdroth/edk2/commits/apic-mmio-fix1c/

A basic command-line invocation for SNP would be:

qemu-system-x86_64 -smp 32,maxcpus=255 -cpu EPYC-Milan-v2
-machine q35,confidential-guest-support=sev0,memory-backend=ram1 -object memory-backend-memfd,id=ram1,size=4G,share=true,reserve=false -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,id-auth= -bios /home/mroth/ovmf/OVMF_CODE-upstream-20240228-apicfix-1c-AmdSevX64.fd

With kernel-hashing and certificate data supplied:

qemu-system-x86_64 -smp 32,maxcpus=255 -cpu EPYC-Milan-v2
-machine q35,confidential-guest-support=sev0,memory-backend=ram1
-object memory-backend-memfd,id=ram1,size=4G,share=true,reserve=false
-object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,id-auth=,certs-path=/home/mroth/cert.blob,kernel-hashes=on -bios /home/mroth/ovmf/OVMF_CODE-upstream-20240228-apicfix-1c-AmdSevX64.fd
-kernel /boot/vmlinuz-6.8.0-snp-host-v12-wip40+
-initrd /boot/initrd.img-6.8.0-snp-host-v12-wip40+
-append "root=UUID=d72a6d1c-06cf-4b79-af43-f1bac4f620f9 ro console=ttyS0,115200n8"

Any comments/feedback would be very much appreciated.

[1] https://github.com/amdese/linux
https://github.com/amdese/amdsev/tree/snp-latest

Changes since rfc2:

- reworked on top of guest_memfd support
- added handling for various KVM_EXIT_VMGEXIT events
- various changes/considerations for PCI passthrough support
- general bugfixes/hardening/cleanups
- qapi cmdline doc fixes/rework (Dov, Markus)
- switch to qbase64_decode, more error-checking for cmdline opts (Dov)
- unset id_block_en for 0 input (Dov)
- use error_setg in snp init (Dov)
- report more info in trace_kvm_sev_init (Dov)
- rework bounds-checking for kvm_cpuid_info, rework existing checks for readability, add additional checks (Dov)
- fixups for validated_ranges handling (Dov)
- rename 'policy' field to 'snp-policy' in query-sev when sev-type is SNP

Changes since rfc1:

- rebased onto latest master
- drop SNP config file in favor of a new 'sev-snp-guest' object where all SNP-related params are passed as strings/integers via command-line
- report specific error if BIOS reports invalid address/len for reserved/pre-validated regions (Connor)
- use Range helpers for handling validated region overlaps (Dave)
- simplify error handling in sev_snp_launch_start, and report the correct return code when handling LAUNCH_START failures (Dov)
- add SEV-SNP bit to CPUID 0x8000001f when SNP enabled
- updated query-sev to handle differences between SEV and SEV-SNP
- updated to work against v5 of SEV-SNP host kernel / hypervisor patches

----------------------------------------------------------------
Brijesh Singh (5):
i386/sev: Introduce 'sev-snp-guest' object i386/sev: Add the SNP launch start context i386/sev: Add handling to encrypt/finalize guest launch data hw/i386/sev: Add function to get SEV metadata from OVMF header i386/sev: Add support for populating OVMF metadata pages

Chao Peng (2):
kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
kvm: handle KVM_EXIT_MEMORY_FAULT

Dov Murik (4):
qapi, i386: Move kernel-hashes to SevCommonProperties i386/sev: Extract build_kernel_loader_hashes i386/sev: Reorder struct declarations i386/sev: Allow measured direct kernel boot on SNP

Isaku Yamahata (2):
pci-host/q35: Move PAM initialization above SMRAM initialization
q35: Introduce smm_ranges property for q35-pci-host

Michael Roth (30):
Revert "linux-headers hack" from sevinit2 base tree scripts/update-linux-headers: Add setup_data.h to import list scripts/update-linux-headers: Add bits.h to file imports [HACK] linux-headers: Update headers for 6.8 + kvm-coco-queue + SNP [TEMP] hw/i386: Remove redeclaration of struct setup_data
RAMBlock: Add support of KVM private guest memfd
[FIXUP] "kvm: handle KVM_EXIT_MEMORY_FAULT": drop qemu_host_page_size trace/kvm: Add trace for page convertion between shared and private
kvm: Make kvm_convert_memory() obey ram_block_discard_is_enabled() trace/kvm: Add trace for KVM_EXIT_MEMORY_FAULT
i386/sev: Introduce "sev-common" type to encapsulate common SEV state i386/sev: Add a sev_snp_enabled() helper
target/i386: Add handling for KVM_X86_SNP_VM VM type i386/sev: Skip RAMBlock notifiers for SNP
i386/sev: Skip machine-init-done notifiers for SNP
i386/sev: Set ms->require_guest_memfd for SNP
i386/sev: Disable SMM for SNP
i386/sev: Don't disable block discarding for SNP
i386/cpu: Set SEV-SNP CPUID bit when SNP enabled
i386/sev: Update query-sev QAPI format to handle SEV-SNP i386/sev: Don't return launch measurements for SEV-SNP guests
kvm: Make kvm_convert_memory() non-static
i386/sev: Add KVM_EXIT_VMGEXIT handling for Page State Changes i386/sev: Add KVM_EXIT_VMGEXIT handling for Page State Changes (MSR-based) i386/sev: Add KVM_EXIT_VMGEXIT handling for Extended Guest Requests i386/sev: Set CPU state to protected once SNP guest payload is finalized i386/sev: Add support for SNP CPUID validation
hw/i386/sev: Add support to encrypt BIOS when SEV-SNP is enabled hw/i386/sev: Use guest_memfd for legacy ROMs
hw/i386: Add support for loading BIOS using guest_memfd

Xiaoyao Li (6):
HostMem: Add mechanism to opt in kvm guest memfd via MachineState trace/kvm: Split address space and slot id in trace_kvm_set_user_memory()
kvm: Introduce support for memory_attributes
physmem: Introduce ram_block_discard_guest_memfd_range() kvm/memory: Make memory type private by default if it has guest memfd backend
memory: Introduce memory_region_init_ram_guest_memfd()

accel/kvm/kvm-all.c                                |  241 ++-
accel/kvm/trace-events                             |    4 +-
accel/stubs/kvm-stub.c                             |    5 +
backends/hostmem-file.c                            |    1 +
backends/hostmem-memfd.c                           |    1 +
backends/hostmem-ram.c                             |    1 +
backends/hostmem.c                                 |    1 +
docs/system/i386/amd-memory-encryption.rst         |   78 +-
hw/core/machine.c                                  |    5 +
hw/i386/pc.c                                       |   13 +-
hw/i386/pc_q35.c                                   |    2 +
hw/i386/pc_sysfw.c                                 |   25 +-
hw/i386/pc_sysfw_ovmf.c                            |   33 +
hw/i386/x86.c                                      |   46 +-
hw/pci-host/q35.c                                  |   61 +-
include/exec/cpu-common.h                          |    2 +
include/exec/memory.h                              |   26 +-
include/exec/ram_addr.h                            |    2 +-
include/exec/ramblock.h                            |    1 +
include/hw/boards.h                                |    2 +
include/hw/i386/pc.h                               |   31 +-
include/hw/i386/x86.h                              |    2 +-
include/hw/pci-host/q35.h                          |    1 +
include/standard-headers/asm-x86/bootparam.h       |   17 +-
include/standard-headers/asm-x86/kvm_para.h        |    3 +-
include/standard-headers/linux/ethtool.h           |   48 +
include/standard-headers/linux/fuse.h              |   39 +-
include/standard-headers/linux/input-event-codes.h |    1 +
include/standard-headers/linux/virtio_gpu.h        |    2 +
include/standard-headers/linux/virtio_snd.h        |  154 ++
include/sysemu/hostmem.h                           |    1 +
include/sysemu/kvm.h                               |    7 +
include/sysemu/kvm_int.h                           |    2 +
linux-headers/asm-arm64/kvm.h                      |   15 +-
linux-headers/asm-arm64/sve_context.h              |   11 +
linux-headers/asm-generic/bitsperlong.h            |    4 +
linux-headers/asm-loongarch/kvm.h                  |    2 -
linux-headers/asm-mips/kvm.h                       |    2 -
linux-headers/asm-powerpc/kvm.h                    |   45 +-
linux-headers/asm-riscv/kvm.h                      |    3 +-
linux-headers/asm-s390/kvm.h                       |  315 +++-
linux-headers/asm-x86/kvm.h                        |  372 ++++-
linux-headers/asm-x86/setup_data.h                 |   83 +
linux-headers/linux/bits.h                         |   15 +
linux-headers/linux/kvm.h                          |  719 +--------
linux-headers/linux/psp-sev.h                      |   71 +
qapi/misc-target.json                              |   71 +-
qapi/qom.json                                      |   96 +-
scripts/update-linux-headers.sh                    |    5 +-
system/memory.c                                    |   30 +
system/physmem.c                                   |   47 +-
target/i386/cpu.c                                  |    1 +
target/i386/kvm/kvm.c                              |    4 +
target/i386/sev-sysemu-stub.c                      |    2 +-
target/i386/sev.c                                  | 1631 ++++++++++++++++----
target/i386/sev.h                                  |   13 +-
target/i386/trace-events                           |    3 +
57 files changed, 3272 insertions(+), 1146 deletions(-)

create mode 100644 linux-headers/asm-x86/setup_data.h create mode 100644 linux-headers/linux/bits.h