Skip to contentSkip to conversation

QEMU gmem implemention

RFC PATCH Superseded Xiaoyao Li 11 participants 53 messages Jul 31, 2023 – Sep 13, 2023

Subsystems (2)

v1 of 9 Superseded Latest is v9 →

lore
Share & tools
lore

Xiaoyao Li

author
#0 lore raw

cover QEMU gmem implemention

This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of KVM_X86_PROTECTED_VM.

It adds the support to create a specific KVM_X86_PROTECTED_VM type VM, and introduces 'private' property for memory backend. When the vm type is KVM_X86_PROTECTED_VM and memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.

    $qemu -object memory-backend-ram,id=mem0,size=1G,private=on \
          -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \
	  ...

Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.

There are following design opens that need to be discussed:

1. how to determine the vm type?

a. like this series, specify the vm type via machine property

      'kvm-type'

b. check the memory backend, if any backend has 'private' property
set, the vm-type is set to KVM_X86_PROTECTED_VM.

2. whether 'private' property is needed if we choose 1.b as design

with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.

3. What is KVM_X86_SW_PROTECTED_VM going to look like? What's the purose of it and what's the requirement on it. I think it's the questions for KVM folks than QEMU folks.

Any other idea/open/question is welcomed.

Beside, TDX QEMU implemetation is based on this series to provide private gmem for TD private memory, which can be found at [2]. And it can work corresponding KVM [3] to boot TDX guest.

[1] https://lore.kernel.org/all/[email protected]/
[2] https://github.com/intel/qemu-tdx/tree/tdx-upstream-wip
[3] https://github.com/intel/tdx/tree/kvm-upstream-2023.07.27-v6.5-rc2-workaround

Chao Peng (4):
RAMBlock: Support KVM gmemory
kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
physmem: Add ram_block_convert_range
kvm: handle KVM_EXIT_MEMORY_FAULT

Isaku Yamahata (4):
HostMem: Add private property to indicate to use kvm gmem trace/kvm: Add trace for page convertion between shared and private pci-host/q35: Move PAM initialization above SMRAM initialization
q35: Introduce smm_ranges property for q35-pci-host

Xiaoyao Li (11):
trace/kvm: Split address space and slot id in
trace_kvm_set_user_memory()
*** HACK *** linux-headers: Update headers to pull in gmem APIs
memory: Introduce memory_region_can_be_private() i386/pc: Drop pc_machine_kvm_type()
target/i386: Implement mc->kvm_type() to get VM type i386/kvm: Create gmem fd for KVM_X86_SW_PROTECTED_VM
kvm: Introduce support for memory_attributes
kvm/memory: Introduce the infrastructure to set the default
shared/private value
i386/kvm: Set memory to default private for KVM_X86_SW_PROTECTED_VM
physmem: replace function name with __func__ in
ram_block_discard_range()
i386: Disable SMM mode for X86_SW_PROTECTED_VM

 accel/kvm/kvm-all.c         | 166 +++++++++++++++++++++++++++++++++---
 accel/kvm/trace-events      |   4 +-
 backends/hostmem.c          |  18 ++++
 hw/i386/pc.c                |   5 --
 hw/i386/pc_q35.c            |   3 +-
 hw/i386/x86.c               |  27 ++++++
 hw/pci-host/q35.c           |  61 ++++++++-----
 include/exec/cpu-common.h   |   2 +
 include/exec/memory.h       |  24 ++++++
 include/exec/ramblock.h     |   1 +
 include/hw/i386/pc.h        |   4 +-
 include/hw/i386/x86.h       |   4 +
 include/hw/pci-host/q35.h   |   1 +
 include/sysemu/hostmem.h    |   2 +-
 include/sysemu/kvm.h        |   3 +
 include/sysemu/kvm_int.h    |   2 +
 linux-headers/asm-x86/kvm.h |   3 +
 linux-headers/linux/kvm.h   |  50 +++++++++++
 qapi/qom.json               |   4 +
 softmmu/memory.c            |  27 ++++++
 softmmu/physmem.c           |  97 ++++++++++++++-------
 target/i386/kvm/kvm.c       |  84 ++++++++++++++++++
 target/i386/kvm/kvm_i386.h  |   1 +
 23 files changed, 517 insertions(+), 76 deletions(-)
Signature · 1 line
2.34.1
author
#1 lore raw

RFC PATCH 1/19 trace/kvm: Split address space and slot id in trace_kvm_set_user_memory()

The upper 16 bits of kvm_userspace_memory_region::slot are address space id. Parse it separately in trace_kvm_set_user_memory().

---
 accel/kvm/kvm-all.c    | 5 +++--
 accel/kvm/trace-events | 2 +-
 2 files changed, 4 insertions(+), 3 deletions(-)
accel/kvm/kvm-all.c +3 -2
index 373d876c0580..d8eee405de24 100644
@@ -309,8 +309,9 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo
     ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
     slot->old_flags = mem.flags;
 err:
-    trace_kvm_set_user_memory(mem.slot, mem.flags, mem.guest_phys_addr,
-                              mem.memory_size, mem.userspace_addr, ret);
+    trace_kvm_set_user_memory(mem.slot >> 16, (uint16_t)mem.slot, mem.flags,
+                              mem.guest_phys_addr, mem.memory_size,
+                              mem.userspace_addr, ret);
     if (ret < 0) {
         error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
                      " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
accel/kvm/trace-events +1 -1
index 399aaeb0ec75..14ebfa1b991c 100644
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
 kvm_irqchip_release_virq(int virq) "virq %d"
 kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d"
 kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d"
-kvm_set_user_memory(uint32_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
+kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
 kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32
 kvm_resample_fd_notify(int gsi) "gsi %d"
 kvm_dirty_ring_full(int id) "vcpu %d"
Signature · 1 line
2.34.1
author
#2 lore raw

RFC PATCH 2/19 *** HACK *** linux-headers: Update headers to pull in gmem APIs

In reply to Xiaoyao Li

This patch needs to be updated by script

scripts/update-linux-headers.sh

once gmem fd support is upstreamed in Linux kernel.

---
 linux-headers/asm-x86/kvm.h |  3 +++
 linux-headers/linux/kvm.h   | 50 +++++++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
linux-headers/asm-x86/kvm.h +3 -0
index 2b3a8f7bd2c0..003fb745347c 100644
@@ -560,4 +560,7 @@ struct kvm_pmu_event_filter {
 /* x86-specific KVM_EXIT_HYPERCALL flags. */
 #define KVM_EXIT_HYPERCALL_LONG_MODE	BIT(0)
 
+#define KVM_X86_DEFAULT_VM	0
+#define KVM_X86_SW_PROTECTED_VM	1
+
 #endif /* _ASM_X86_KVM_H */
linux-headers/linux/kvm.h +50 -0

90 lines not on this page · raw message

Signature · 1 line
2.34.1
author
#3 lore raw

RFC PATCH 3/19 RAMBlock: Support KVM gmemory

In reply to Xiaoyao Li

From: Chao Peng <[email protected]>

Add KVM gmem support to RAMBlock so we can have both normal hva based memory and gmem fd based memory in one RAMBlock.

The gmem part is represented by the gmem_fd.

---
 include/exec/memory.h   | 8 ++++++++
 include/exec/ramblock.h | 1 +
 softmmu/memory.c        | 9 +++++++++
 softmmu/physmem.c       | 2 ++
 4 files changed, 20 insertions(+)
include/exec/memory.h +8 -0
index 7f5c11a0cc9e..61e31c7b9874 100644
@@ -1376,6 +1376,14 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr,
                                     int fd,
                                     ram_addr_t offset,
                                     Error **errp);
+/**
+ * memory_region_set_gmem_fd:  Set RAM memory region with a restricted fd.
+ *
+ * @mr: the #MemoryRegion to be set.
+ * @fd: the fd to provide restricted memory.
+ */
+void memory_region_set_gmem_fd(MemoryRegion *mr, int fd);
+
 #endif
 
 /**
include/exec/ramblock.h +1 -0
index 69c6a5390293..0d158b3909c9 100644
@@ -41,6 +41,7 @@ struct RAMBlock {
     QLIST_HEAD(, RAMBlockNotifier) ramblock_notifiers;
     int fd;
     uint64_t fd_offset;
+    int gmem_fd;
     size_t page_size;
     /* dirty bitmap used during migration */
     unsigned long *bmap;
softmmu/memory.c +9 -0
index 7d9494ce7028..4f8f8c0a02e6 100644
@@ -1661,6 +1661,15 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr,
         error_propagate(errp, err);
     }
 }
+
+void memory_region_set_gmem_fd(MemoryRegion *mr, int fd)
+{
+    if (mr->ram_block) {
+        assert(fd >= 0);
+        mr->ram_block->gmem_fd = fd;
+    }
+}
+
 #endif
 
 void memory_region_init_ram_ptr(MemoryRegion *mr,
softmmu/physmem.c +2 -0
index 3df73542e1fe..8f64128de0b5 100644
@@ -1920,6 +1920,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,
     new_block->used_length = size;
     new_block->max_length = size;
     new_block->flags = ram_flags;
+    new_block->gmem_fd = -1;
     new_block->host = file_ram_alloc(new_block, size, fd, readonly,
                                      !file_size, offset, errp);
     if (!new_block->host) {
@@ -1990,6 +1991,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
     new_block->max_length = max_size;
     assert(max_size >= size);
     new_block->fd = -1;
+    new_block->gmem_fd = -1;
     new_block->page_size = qemu_real_host_page_size();
     new_block->host = host;
     new_block->flags = ram_flags;
Signature · 1 line
2.34.1
#4 lore raw

Re: [RFC PATCH 03/19] RAMBlock: Support KVM gmemory

Quoted 9 lines · Xiaoyao Li

From: Chao Peng <[email protected]>

Add KVM gmem support to RAMBlock so we can have both normal hva based memory and gmem fd based memory in one RAMBlock.

The gmem part is represented by the gmem_fd.

So, *someone* creates a RAMBlock in QEMU.

Who'll squeeze a gmem_fd in there? When? How?

Shouldn't we create the RAM memory region / RAMBlock already with the gmem_fd, and set that when initializing these things?

Signature · 3 lines
Cheers,

David / dhildenb
author
#5 lore raw

RFC PATCH 4/19 memory: Introduce memory_region_can_be_private()

In reply to Xiaoyao Li
---
 include/exec/memory.h | 9 +++++++++
 softmmu/memory.c      | 5 +++++
 2 files changed, 14 insertions(+)
include/exec/memory.h +9 -0
index 61e31c7b9874..e119d3ce1a1d 100644
@@ -1679,6 +1679,15 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)
  */
 bool memory_region_is_protected(MemoryRegion *mr);
 
+/**
+ * memory_region_can_be_private: check whether a memory region can be private
+ *
+ * Returns %true if a memory region's ram_block has valid gmem fd assigned.
+ *
+ * @mr: the memory region being queried
+ */
+bool memory_region_can_be_private(MemoryRegion *mr);
+
 /**
  * memory_region_get_iommu: check whether a memory region is an iommu
  *
softmmu/memory.c +5 -0
index 4f8f8c0a02e6..336c76ede660 100644
@@ -1855,6 +1855,11 @@ bool memory_region_is_protected(MemoryRegion *mr)
     return mr->ram && (mr->ram_block->flags & RAM_PROTECTED);
 }
 
+bool memory_region_can_be_private(MemoryRegion *mr)
+{
+    return mr->ram_block && mr->ram_block->gmem_fd >= 0;
+}
+
 uint8_t memory_region_get_dirty_log_mask(MemoryRegion *mr)
 {
     uint8_t mask = mr->dirty_log_mask;
Signature · 1 line
2.34.1
#6 lore raw

Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()

softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861

Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4 -0
+bool memory_region_can_be_private(MemoryRegion *mr)
+{
+    return mr->ram_block && mr->ram_block->gmem_fd >= 0;
+}

This is not really MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?

I worry it can be very confusing in the future against MAP_PRIVATE / MAP_SHARED otherwise.

Thanks,

Signature · 1 line
Peter Xu
#7 lore raw

Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()

Quoted 8 lines · Peter Xu

softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861

Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4 -0
+bool memory_region_can_be_private(MemoryRegion *mr)
+{
+    return mr->ram_block && mr->ram_block->gmem_fd >= 0;
+}

This is not really MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?

Yes and yes.

Quoted 2 lines

I worry it can be very confusing in the future against MAP_PRIVATE / MAP_SHARED otherwise.

Heh, it's already quite confusing at times. I'm definitely open to naming that doesn't collide with MAP_{PRIVATE,SHARED}, especially if someone can come with a naming scheme that includes a succinct way to describe memory that is shared between two or more VMs, but is accessible to _only_ those VMs.

#8 lore raw

Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()

Quoted 12 lines · Sean Christopherson
Quoted 8 lines · Peter Xu

softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861

Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4 -0
+bool memory_region_can_be_private(MemoryRegion *mr)
+{
+    return mr->ram_block && mr->ram_block->gmem_fd >= 0;
+}

This is not really MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?

Yes and yes.

Quoted 7 lines
Quoted 2 lines

I worry it can be very confusing in the future against MAP_PRIVATE / MAP_SHARED otherwise.

Heh, it's already quite confusing at times. I'm definitely open to naming that doesn't collide with MAP_{PRIVATE,SHARED}, especially if someone can come with a naming scheme that includes a succinct way to describe memory that is shared between two or more VMs, but is accessible to _only_ those VMs.

Standard solution is a technology specific prefix. protect_shared, encrypt_shared etc.

Signature · 1 line
MST
#9 lore raw

Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()

Quoted 16 lines · Michael S. Tsirkin
Quoted 15 lines · Sean Christopherson
Quoted 8 lines · Peter Xu

softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861

Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4 -0
+bool memory_region_can_be_private(MemoryRegion *mr)
+{
+    return mr->ram_block && mr->ram_block->gmem_fd >= 0;
+}

This is not really MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?

Yes and yes.

Quoted 2 lines

I worry it can be very confusing in the future against MAP_PRIVATE / MAP_SHARED otherwise.

Quoted 7 lines
Quoted 4 lines

Heh, it's already quite confusing at times. I'm definitely open to naming that doesn't collide with MAP_{PRIVATE,SHARED}, especially if someone can come with a naming scheme that includes a succinct way to describe memory that is shared between two or more VMs, but is accessible to _only_ those VMs.

Standard solution is a technology specific prefix. protect_shared, encrypt_shared etc.

Agreed, a prefix could definitely help (if nothing better comes at last..). If e.g. "encrypted" too long to be applied everywhere in var names and functions, maybe it can also be "enc_{private|shared}".

Thanks,

Signature · 1 line
Peter Xu
#10 lore raw

Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()

Quoted 22 lines · Peter Xu
Quoted 21 lines · Michael S. Tsirkin
Quoted 19 lines · Sean Christopherson
Quoted 8 lines · Peter Xu

softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861

Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4 -0
+bool memory_region_can_be_private(MemoryRegion *mr)
+{
+    return mr->ram_block && mr->ram_block->gmem_fd >= 0;
+}

This is not really MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?

Yes and yes.

Quoted 2 lines

I worry it can be very confusing in the future against MAP_PRIVATE / MAP_SHARED otherwise.

Heh, it's already quite confusing at times. I'm definitely open to naming that doesn't collide with MAP_{PRIVATE,SHARED}, especially if someone can come with a naming scheme that includes a succinct way to describe memory that is shared between two or more VMs, but is accessible to _only_ those VMs.

Quoted 6 lines
Quoted 2 lines

Standard solution is a technology specific prefix. protect_shared, encrypt_shared etc.

Agreed, a prefix could definitely help (if nothing better comes at last..). If e.g. "encrypted" too long to be applied everywhere in var names and functions, maybe it can also be "enc_{private|shared}".

FWIW, I would stay away from "encrypted", there is no requirement that the memory actually be encrypted.

#11 lore raw

Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()

In reply to Peter Xu
Quoted 9 lines · Peter Xu

softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861

Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4 -0
+bool memory_region_can_be_private(MemoryRegion *mr)
+{
+    return mr->ram_block && mr->ram_block->gmem_fd >= 0;
+}

This is not really MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?

Quoted 4 lines

I worry it can be very confusing in the future against MAP_PRIVATE / MAP_SHARED otherwise.

Thanks,

It is - kernel calls this "map shared" but unfortunately qemu calls this just "shared". Maybe e.g. "protect_shared" would be a better name for qemu?

Signature · 1 line
MST
#12 lore raw

Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()

In reply to Xiaoyao Li
Quoted 11 lines · Xiaoyao Li
---
 include/exec/memory.h | 9 +++++++++
 softmmu/memory.c      | 5 +++++
 2 files changed, 14 insertions(+)
include/exec/memory.h +0 -0
index 61e31c7b9874..e119d3ce1a1d 100644

include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1683

Quoted 5 lines
include/exec/memory.h +2 -0
@@ -1679,6 +1679,15 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)
  */
 bool memory_region_is_protected(MemoryRegion *mr);
 
+/**
+ * memory_region_can_be_private: check whether a memory region can be private

The name of the function is not particularly informative,

include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1685

Quoted 2 lines
include/exec/memory.h +2 -0
+ *
+ * Returns %true if a memory region's ram_block has valid gmem fd assigned.

but in your comment you describe more accurately what it does, why not make it the function name?

bool memory_region_has_valid_gmem_fd()

include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1689

Quoted 4 lines
include/exec/memory.h +4 -0
+ *
+ * @mr: the memory region being queried
+ */
+bool memory_region_can_be_private(MemoryRegion *mr);

bool memory_region_has_valid_gmem_fd()

Thanks,

C

Quoted 20 lines
include/exec/memory.h +1 -0
+
 /**
  * memory_region_get_iommu: check whether a memory region is an iommu
  *
softmmu/memory.c +5 -0
index 4f8f8c0a02e6..336c76ede660 100644
@@ -1855,6 +1855,11 @@ bool memory_region_is_protected(MemoryRegion *mr)
     return mr->ram && (mr->ram_block->flags & RAM_PROTECTED);
 }
 
+bool memory_region_can_be_private(MemoryRegion *mr)
+{
+    return mr->ram_block && mr->ram_block->gmem_fd >= 0;
+}
+
 uint8_t memory_region_get_dirty_log_mask(MemoryRegion *mr)
 {
     uint8_t mask = mr->dirty_log_mask;
#13 lore raw

Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()

Quoted 20 lines · Claudio Fontana
Quoted 11 lines · Xiaoyao Li
---
 include/exec/memory.h | 9 +++++++++
 softmmu/memory.c      | 5 +++++
 2 files changed, 14 insertions(+)
include/exec/memory.h +0 -0
index 61e31c7b9874..e119d3ce1a1d 100644

include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1683

Quoted 5 lines
include/exec/memory.h +2 -0
@@ -1679,6 +1679,15 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)
  */
 bool memory_region_is_protected(MemoryRegion *mr);
 
+/**
+ * memory_region_can_be_private: check whether a memory region can be private

The name of the function is not particularly informative,

Quoted 6 lines

include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1685

Quoted 2 lines
include/exec/memory.h +2 -0
+ *
+ * Returns %true if a memory region's ram_block has valid gmem fd assigned.

but in your comment you describe more accurately what it does, why not make it the function name?

bool memory_region_has_valid_gmem_fd()

btw can a memory region have an invalid gmem_fd ?

If an invalid gmem_fd is just used to mark whether gmem_fd is present or not,

we could make it just:

bool memory_region_has_gmem_fd()

Thanks,

C

Quoted 35 lines

include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1689

Quoted 5 lines
include/exec/memory.h +4 -0
+ *
+ * @mr: the memory region being queried
+ */
+bool memory_region_can_be_private(MemoryRegion *mr);

bool memory_region_has_valid_gmem_fd()

Thanks,

C

Quoted 20 lines
include/exec/memory.h +1 -0
+
 /**
  * memory_region_get_iommu: check whether a memory region is an iommu
  *
softmmu/memory.c +5 -0
index 4f8f8c0a02e6..336c76ede660 100644
@@ -1855,6 +1855,11 @@ bool memory_region_is_protected(MemoryRegion *mr)
     return mr->ram && (mr->ram_block->flags & RAM_PROTECTED);
 }
 
+bool memory_region_can_be_private(MemoryRegion *mr)
+{
+    return mr->ram_block && mr->ram_block->gmem_fd >= 0;
+}
+
 uint8_t memory_region_get_dirty_log_mask(MemoryRegion *mr)
 {
     uint8_t mask = mr->dirty_log_mask;
author
#14 lore raw

Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()

Quoted 29 lines · Claudio Fontana
Quoted 20 lines · Claudio Fontana
Quoted 11 lines · Xiaoyao Li
---
  include/exec/memory.h | 9 +++++++++
  softmmu/memory.c      | 5 +++++
  2 files changed, 14 insertions(+)
include/exec/memory.h +0 -0
index 61e31c7b9874..e119d3ce1a1d 100644

include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1683

Quoted 5 lines
include/exec/memory.h +2 -0
@@ -1679,6 +1679,15 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)
   */
  bool memory_region_is_protected(MemoryRegion *mr);
 
+/**
+ * memory_region_can_be_private: check whether a memory region can be private

The name of the function is not particularly informative,

Quoted 6 lines

include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1685

Quoted 2 lines
include/exec/memory.h +2 -0
+ *
+ * Returns %true if a memory region's ram_block has valid gmem fd assigned.

but in your comment you describe more accurately what it does, why not make it the function name?

bool memory_region_has_valid_gmem_fd()

Quoted 7 lines

btw can a memory region have an invalid gmem_fd ?

If an invalid gmem_fd is just used to mark whether gmem_fd is present or not,

we could make it just:

bool memory_region_has_gmem_fd()

yes. It's a good suggestion!

I will use it in next version if no objection from others.

Thanks,
-Xiaoyao

author
#15 lore raw

RFC PATCH 5/19 kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot

In reply to Xiaoyao Li

From: Chao Peng <[email protected]>

Switch to KVM_SET_USER_MEMORY_REGION2 when supported by KVM.

With KVM_SET_USER_MEMORY_REGION2, QEMU can set up memory region that backen'ed both by hva-based shared memory and gmem fd based private memory.

---
 accel/kvm/kvm-all.c      | 57 +++++++++++++++++++++++++++++++++-------
 accel/kvm/trace-events   |  2 +-
 include/sysemu/kvm_int.h |  2 ++
 3 files changed, 51 insertions(+), 10 deletions(-)
accel/kvm/kvm-all.c +48 -9
index d8eee405de24..7b1818334ba7 100644
@@ -288,35 +288,68 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,
 static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, bool new)
 {
     KVMState *s = kvm_state;
-    struct kvm_userspace_memory_region mem;
+    struct kvm_userspace_memory_region2 mem;
+    static int cap_user_memory2 = -1;
     int ret;
 
+    if (cap_user_memory2 == -1) {
+        cap_user_memory2 = kvm_check_extension(s, KVM_CAP_USER_MEMORY2);
+    }
+
+    if (!cap_user_memory2 && slot->fd >= 0) {
+        error_report("%s, KVM doesn't support gmem!", __func__);
+        exit(1);
+    }
+
     mem.slot = slot->slot | (kml->as_id << 16);
     mem.guest_phys_addr = slot->start_addr;
     mem.userspace_addr = (unsigned long)slot->ram;
     mem.flags = slot->flags;
+    mem.gmem_fd = slot->fd;
+    mem.gmem_offset = slot->ofs;
 
-    if (slot->memory_size && !new && (mem.flags ^ slot->old_flags) & KVM_MEM_READONLY) {
+    if (slot->memory_size && !new && (slot->flags ^ slot->old_flags) & KVM_MEM_READONLY) {
         /* Set the slot size to 0 before setting the slot to the desired
          * value. This is needed based on KVM commit 75d61fbc. */
         mem.memory_size = 0;
-        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+
+        if (cap_user_memory2) {
+            ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
+        } else {
+            ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+	    }
         if (ret < 0) {
             goto err;
         }
     }
     mem.memory_size = slot->memory_size;
-    ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+    if (cap_user_memory2) {
+        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
+    } else {
+        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+    }
     slot->old_flags = mem.flags;
 err:
     trace_kvm_set_user_memory(mem.slot >> 16, (uint16_t)mem.slot, mem.flags,
                               mem.guest_phys_addr, mem.memory_size,
-                              mem.userspace_addr, ret);
+                              mem.userspace_addr, mem.gmem_fd,
+			      mem.gmem_offset, ret);
     if (ret < 0) {
-        error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
-                     " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
-                     __func__, mem.slot, slot->start_addr,
-                     (uint64_t)mem.memory_size, strerror(errno));
+        if (cap_user_memory2) {
+                error_report("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d,"
+                        " start=0x%" PRIx64 ", size=0x%" PRIx64 ","
+                        " flags=0x%" PRIx32 ","
+                        " gmem_fd=%" PRId32 ", gmem_offset=0x%" PRIx64 ": %s",
+                        __func__, mem.slot, slot->start_addr,
+                (uint64_t)mem.memory_size, mem.flags,
+                        mem.gmem_fd, (uint64_t)mem.gmem_offset,
+                        strerror(errno));
+        } else {
+                error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
+                            " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
+                __func__, mem.slot, slot->start_addr,
+                (uint64_t)mem.memory_size, strerror(errno));
+        }
     }
     return ret;
 }
@@ -472,6 +505,9 @@ static int kvm_mem_flags(MemoryRegion *mr)
     if (readonly && kvm_readonly_mem_allowed) {
         flags |= KVM_MEM_READONLY;
     }
+    if (memory_region_can_be_private(mr)) {
+        flags |= KVM_MEM_PRIVATE;
+    }
     return flags;
 }
 
@@ -1402,6 +1438,9 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
         mem->ram_start_offset = ram_start_offset;
         mem->ram = ram;
         mem->flags = kvm_mem_flags(mr);
+        mem->fd = mr->ram_block->gmem_fd;
+        mem->ofs = (uint8_t*)ram - mr->ram_block->host;
+
         kvm_slot_init_dirty_bitmap(mem);
         err = kvm_set_user_memory_region(kml, mem, true);
         if (err) {
accel/kvm/trace-events +1 -1
index 14ebfa1b991c..80694683acea 100644
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
 kvm_irqchip_release_virq(int virq) "virq %d"
 kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d"
 kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d"
-kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
+kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, uint32_t fd, uint64_t fd_offset, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " gmem_fd=%d" " gmem_fd_offset=0x%" PRIx64 " ret=%d"
 kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32
 kvm_resample_fd_notify(int gsi) "gsi %d"
 kvm_dirty_ring_full(int id) "vcpu %d"
include/sysemu/kvm_int.h +2 -0
index 511b42bde5c4..48220c0793ac 100644
@@ -30,6 +30,8 @@ typedef struct KVMSlot
     int as_id;
     /* Cache of the offset in ram address space */
     ram_addr_t ram_start_offset;
+    int fd;
+    hwaddr ofs;
 } KVMSlot;
 
 typedef struct KVMMemoryUpdate {
Signature · 1 line
2.34.1
#16 lore raw

Re: [RFC PATCH 05/19] kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot

Quoted 28 lines · Xiaoyao Li

From: Chao Peng <[email protected]>

Switch to KVM_SET_USER_MEMORY_REGION2 when supported by KVM.

With KVM_SET_USER_MEMORY_REGION2, QEMU can set up memory region that backen'ed both by hva-based shared memory and gmem fd based private memory.

---
 accel/kvm/kvm-all.c      | 57 +++++++++++++++++++++++++++++++++-------
 accel/kvm/trace-events   |  2 +-
 include/sysemu/kvm_int.h |  2 ++
 3 files changed, 51 insertions(+), 10 deletions(-)
accel/kvm/kvm-all.c +2 -1
index d8eee405de24..7b1818334ba7 100644
@@ -288,35 +288,68 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,
 static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, bool new)
 {
     KVMState *s = kvm_state;
-    struct kvm_userspace_memory_region mem;
+    struct kvm_userspace_memory_region2 mem;
+    static int cap_user_memory2 = -1;

accel/kvm/kvm-all.c int kvm_physical_memory_addr_from_host(KVMState *s, void *ram, · line 302

Quoted 10 lines
accel/kvm/kvm-all.c +8 -0
     int ret;
 
+    if (cap_user_memory2 == -1) {
+        cap_user_memory2 = kvm_check_extension(s, KVM_CAP_USER_MEMORY2);
+    }
+
+    if (!cap_user_memory2 && slot->fd >= 0) {
+        error_report("%s, KVM doesn't support gmem!", __func__);
+        exit(1);
+    }

We handle this special error case here,
while the existing callers of kvm_set_user_memory_region handle the other error cases in different places.

Not that the rest of kvm-all does an excellent job at error handling, but maybe we can avoid compounding on the issue.

accel/kvm/kvm-all.c int kvm_physical_memory_addr_from_host(KVMState *s, void *ram, · line 311

Quoted 10 lines
accel/kvm/kvm-all.c +4 -1
+
     mem.slot = slot->slot | (kml->as_id << 16);
     mem.guest_phys_addr = slot->start_addr;
     mem.userspace_addr = (unsigned long)slot->ram;
     mem.flags = slot->flags;
+    mem.gmem_fd = slot->fd;
+    mem.gmem_offset = slot->ofs;
 
-    if (slot->memory_size && !new && (mem.flags ^ slot->old_flags) & KVM_MEM_READONLY) {
+    if (slot->memory_size && !new && (slot->flags ^ slot->old_flags) & KVM_MEM_READONLY) {

Why the change if mem.flags == slot->flags ?

Quoted 97 lines
accel/kvm/kvm-all.c +34 -7
         /* Set the slot size to 0 before setting the slot to the desired
          * value. This is needed based on KVM commit 75d61fbc. */
         mem.memory_size = 0;
-        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+
+        if (cap_user_memory2) {
+            ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
+        } else {
+            ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+	    }
         if (ret < 0) {
             goto err;
         }
     }
     mem.memory_size = slot->memory_size;
-    ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+    if (cap_user_memory2) {
+        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
+    } else {
+        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+    }
     slot->old_flags = mem.flags;
 err:
     trace_kvm_set_user_memory(mem.slot >> 16, (uint16_t)mem.slot, mem.flags,
                               mem.guest_phys_addr, mem.memory_size,
-                              mem.userspace_addr, ret);
+                              mem.userspace_addr, mem.gmem_fd,
+			      mem.gmem_offset, ret);
     if (ret < 0) {
-        error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
-                     " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
-                     __func__, mem.slot, slot->start_addr,
-                     (uint64_t)mem.memory_size, strerror(errno));
+        if (cap_user_memory2) {
+                error_report("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d,"
+                        " start=0x%" PRIx64 ", size=0x%" PRIx64 ","
+                        " flags=0x%" PRIx32 ","
+                        " gmem_fd=%" PRId32 ", gmem_offset=0x%" PRIx64 ": %s",
+                        __func__, mem.slot, slot->start_addr,
+                (uint64_t)mem.memory_size, mem.flags,
+                        mem.gmem_fd, (uint64_t)mem.gmem_offset,
+                        strerror(errno));
+        } else {
+                error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
+                            " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
+                __func__, mem.slot, slot->start_addr,
+                (uint64_t)mem.memory_size, strerror(errno));
+        }
     }
     return ret;
 }
@@ -472,6 +505,9 @@ static int kvm_mem_flags(MemoryRegion *mr)
     if (readonly && kvm_readonly_mem_allowed) {
         flags |= KVM_MEM_READONLY;
     }
+    if (memory_region_can_be_private(mr)) {
+        flags |= KVM_MEM_PRIVATE;
+    }
     return flags;
 }
 
@@ -1402,6 +1438,9 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
         mem->ram_start_offset = ram_start_offset;
         mem->ram = ram;
         mem->flags = kvm_mem_flags(mr);
+        mem->fd = mr->ram_block->gmem_fd;
+        mem->ofs = (uint8_t*)ram - mr->ram_block->host;
+
         kvm_slot_init_dirty_bitmap(mem);
         err = kvm_set_user_memory_region(kml, mem, true);
         if (err) {
accel/kvm/trace-events +1 -1
index 14ebfa1b991c..80694683acea 100644
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
 kvm_irqchip_release_virq(int virq) "virq %d"
 kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d"
 kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d"
-kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
+kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, uint32_t fd, uint64_t fd_offset, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " gmem_fd=%d" " gmem_fd_offset=0x%" PRIx64 " ret=%d"
 kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32
 kvm_resample_fd_notify(int gsi) "gsi %d"
 kvm_dirty_ring_full(int id) "vcpu %d"
include/sysemu/kvm_int.h +2 -0
index 511b42bde5c4..48220c0793ac 100644
@@ -30,6 +30,8 @@ typedef struct KVMSlot
     int as_id;
     /* Cache of the offset in ram address space */
     ram_addr_t ram_start_offset;
+    int fd;
+    hwaddr ofs;
 } KVMSlot;
 
 typedef struct KVMMemoryUpdate {
author
#17 lore raw

Re: [RFC PATCH 05/19] kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot

Quoted 43 lines · Claudio Fontana
Quoted 28 lines · Xiaoyao Li

From: Chao Peng <[email protected]>

Switch to KVM_SET_USER_MEMORY_REGION2 when supported by KVM.

With KVM_SET_USER_MEMORY_REGION2, QEMU can set up memory region that backen'ed both by hva-based shared memory and gmem fd based private memory.

---
  accel/kvm/kvm-all.c      | 57 +++++++++++++++++++++++++++++++++-------
  accel/kvm/trace-events   |  2 +-
  include/sysemu/kvm_int.h |  2 ++
  3 files changed, 51 insertions(+), 10 deletions(-)
accel/kvm/kvm-all.c +2 -1
index d8eee405de24..7b1818334ba7 100644
@@ -288,35 +288,68 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,
  static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, bool new)
  {
      KVMState *s = kvm_state;
-    struct kvm_userspace_memory_region mem;
+    struct kvm_userspace_memory_region2 mem;
+    static int cap_user_memory2 = -1;

accel/kvm/kvm-all.c int kvm_physical_memory_addr_from_host(KVMState *s, void *ram, · line 302

Quoted 10 lines
accel/kvm/kvm-all.c +8 -0
      int ret;
 
+    if (cap_user_memory2 == -1) {
+        cap_user_memory2 = kvm_check_extension(s, KVM_CAP_USER_MEMORY2);
+    }
+
+    if (!cap_user_memory2 && slot->fd >= 0) {
+        error_report("%s, KVM doesn't support gmem!", __func__);
+        exit(1);
+    }

We handle this special error case here,
while the existing callers of kvm_set_user_memory_region handle the other error cases in different places.

Quoted 1 line

Not that the rest of kvm-all does an excellent job at error handling, but maybe we can avoid compounding on the issue.

I'm not sure how to align them. Do you have any suggestion?

Quoted 8 lines
Quoted 8 lines
accel/kvm/kvm-all.c +3 -0
+
      mem.slot = slot->slot | (kml->as_id << 16);
      mem.guest_phys_addr = slot->start_addr;
      mem.userspace_addr = (unsigned long)slot->ram;
      mem.flags = slot->flags;
+    mem.gmem_fd = slot->fd;
+    mem.gmem_offset = slot->ofs;
 
Quoted 4 lines

accel/kvm/kvm-all.c int kvm_physical_memory_addr_from_host(KVMState *s, void *ram, · line 311

Quoted 2 lines
accel/kvm/kvm-all.c +1 -1
-    if (slot->memory_size && !new && (mem.flags ^ slot->old_flags) & KVM_MEM_READONLY) {
+    if (slot->memory_size && !new && (slot->flags ^ slot->old_flags) & KVM_MEM_READONLY) {

Why the change if mem.flags == slot->flags ?

I guess the goal is to make it clearer that it's comparing the (new) flags with old_flags of the slot.

Anyway, if this change is annoying, I can drop it. :)

Quoted 98 lines
Quoted 97 lines
accel/kvm/kvm-all.c +34 -7
          /* Set the slot size to 0 before setting the slot to the desired
           * value. This is needed based on KVM commit 75d61fbc. */
          mem.memory_size = 0;
-        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+
+        if (cap_user_memory2) {
+            ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
+        } else {
+            ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+	    }
          if (ret < 0) {
              goto err;
          }
      }
      mem.memory_size = slot->memory_size;
-    ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+    if (cap_user_memory2) {
+        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
+    } else {
+        ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+    }
      slot->old_flags = mem.flags;
  err:
      trace_kvm_set_user_memory(mem.slot >> 16, (uint16_t)mem.slot, mem.flags,
                                mem.guest_phys_addr, mem.memory_size,
-                              mem.userspace_addr, ret);
+                              mem.userspace_addr, mem.gmem_fd,
+			      mem.gmem_offset, ret);
      if (ret < 0) {
-        error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
-                     " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
-                     __func__, mem.slot, slot->start_addr,
-                     (uint64_t)mem.memory_size, strerror(errno));
+        if (cap_user_memory2) {
+                error_report("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d,"
+                        " start=0x%" PRIx64 ", size=0x%" PRIx64 ","
+                        " flags=0x%" PRIx32 ","
+                        " gmem_fd=%" PRId32 ", gmem_offset=0x%" PRIx64 ": %s",
+                        __func__, mem.slot, slot->start_addr,
+                (uint64_t)mem.memory_size, mem.flags,
+                        mem.gmem_fd, (uint64_t)mem.gmem_offset,
+                        strerror(errno));
+        } else {
+                error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
+                            " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
+                __func__, mem.slot, slot->start_addr,
+                (uint64_t)mem.memory_size, strerror(errno));
+        }
      }
      return ret;
  }
@@ -472,6 +505,9 @@ static int kvm_mem_flags(MemoryRegion *mr)
      if (readonly && kvm_readonly_mem_allowed) {
          flags |= KVM_MEM_READONLY;
      }
+    if (memory_region_can_be_private(mr)) {
+        flags |= KVM_MEM_PRIVATE;
+    }
      return flags;
  }
 
@@ -1402,6 +1438,9 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
          mem->ram_start_offset = ram_start_offset;
          mem->ram = ram;
          mem->flags = kvm_mem_flags(mr);
+        mem->fd = mr->ram_block->gmem_fd;
+        mem->ofs = (uint8_t*)ram - mr->ram_block->host;
+
          kvm_slot_init_dirty_bitmap(mem);
          err = kvm_set_user_memory_region(kml, mem, true);
          if (err) {
accel/kvm/trace-events +1 -1
index 14ebfa1b991c..80694683acea 100644
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
  kvm_irqchip_release_virq(int virq) "virq %d"
  kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d"
  kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d"
-kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
+kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, uint32_t fd, uint64_t fd_offset, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " gmem_fd=%d" " gmem_fd_offset=0x%" PRIx64 " ret=%d"
  kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32
  kvm_resample_fd_notify(int gsi) "gsi %d"
  kvm_dirty_ring_full(int id) "vcpu %d"
include/sysemu/kvm_int.h +2 -0
index 511b42bde5c4..48220c0793ac 100644
@@ -30,6 +30,8 @@ typedef struct KVMSlot
      int as_id;
      /* Cache of the offset in ram address space */
      ram_addr_t ram_start_offset;
+    int fd;
+    hwaddr ofs;
  } KVMSlot;
 
  typedef struct KVMMemoryUpdate {
author
#18 lore raw

RFC PATCH 6/19 i386/pc: Drop pc_machine_kvm_type()

In reply to Xiaoyao Li

pc_machine_kvm_type() was introduced by commit e21be724eaf5 ("i386/xen:
add pc_machine_kvm_type to initialize XEN_EMULATE mode") to do Xen specific initialization by utilizing kvm_type method.

commit eeedfe6c6316 ("hw/xen: Simplify emulated Xen platform init") moves the Xen specific initialization to pc_basic_device_init().

There is no need to keep the PC specific kvm_type() implementation anymore. On the other hand, later patch will implement kvm_type() method for all x86/i386 machines to support KVM_X86_SW_PROTECTED_VM.

---
 hw/i386/pc.c         | 5 -----
 include/hw/i386/pc.h | 3 ---
 2 files changed, 8 deletions(-)
hw/i386/pc.c +0 -5
index 3109d5e0e035..abeadd903827 100644
@@ -1794,11 +1794,6 @@ static void pc_machine_initfn(Object *obj)
     cxl_machine_init(obj, &pcms->cxl_devices_state);
 }
 
-int pc_machine_kvm_type(MachineState *machine, const char *kvm_type)
-{
-    return 0;
-}
-
 static void pc_machine_reset(MachineState *machine, ShutdownCause reason)
 {
     CPUState *cs;
include/hw/i386/pc.h +0 -3
index d54e8b1101e4..c98d628a76f3 100644
@@ -296,15 +296,12 @@ extern const size_t pc_compat_1_5_len;
 extern GlobalProperty pc_compat_1_4[];
 extern const size_t pc_compat_1_4_len;
 
-int pc_machine_kvm_type(MachineState *machine, const char *vm_type);
-
 #define DEFINE_PC_MACHINE(suffix, namestr, initfn, optsfn) \
     static void pc_machine_##suffix##_class_init(ObjectClass *oc, void *data) \
     { \
         MachineClass *mc = MACHINE_CLASS(oc); \
         optsfn(mc); \
         mc->init = initfn; \
-        mc->kvm_type = pc_machine_kvm_type; \
     } \
     static const TypeInfo pc_machine_type_##suffix = { \
         .name       = namestr TYPE_MACHINE_SUFFIX, \
Signature · 1 line
2.34.1
Reviewed-by on v1 6/19
#19 lore raw

Re: [RFC PATCH 06/19] i386/pc: Drop pc_machine_kvm_type()

Quoted 56 lines · Xiaoyao Li

pc_machine_kvm_type() was introduced by commit e21be724eaf5 ("i386/xen:
add pc_machine_kvm_type to initialize XEN_EMULATE mode") to do Xen specific initialization by utilizing kvm_type method.

commit eeedfe6c6316 ("hw/xen: Simplify emulated Xen platform init") moves the Xen specific initialization to pc_basic_device_init().

There is no need to keep the PC specific kvm_type() implementation anymore. On the other hand, later patch will implement kvm_type() method for all x86/i386 machines to support KVM_X86_SW_PROTECTED_VM.

---
 hw/i386/pc.c         | 5 -----
 include/hw/i386/pc.h | 3 ---
 2 files changed, 8 deletions(-)
hw/i386/pc.c +0 -5
index 3109d5e0e035..abeadd903827 100644
@@ -1794,11 +1794,6 @@ static void pc_machine_initfn(Object *obj)
     cxl_machine_init(obj, &pcms->cxl_devices_state);
 }
 
-int pc_machine_kvm_type(MachineState *machine, const char *kvm_type)
-{
-    return 0;
-}
-
 static void pc_machine_reset(MachineState *machine, ShutdownCause reason)
 {
     CPUState *cs;
include/hw/i386/pc.h +0 -3
index d54e8b1101e4..c98d628a76f3 100644
@@ -296,15 +296,12 @@ extern const size_t pc_compat_1_5_len;
 extern GlobalProperty pc_compat_1_4[];
 extern const size_t pc_compat_1_4_len;
 
-int pc_machine_kvm_type(MachineState *machine, const char *vm_type);
-
 #define DEFINE_PC_MACHINE(suffix, namestr, initfn, optsfn) \
     static void pc_machine_##suffix##_class_init(ObjectClass *oc, void *data) \
     { \
         MachineClass *mc = MACHINE_CLASS(oc); \
         optsfn(mc); \
         mc->init = initfn; \
-        mc->kvm_type = pc_machine_kvm_type; \
     } \
     static const TypeInfo pc_machine_type_##suffix = { \
         .name       = namestr TYPE_MACHINE_SUFFIX, \
Signature · 1 line
2.34.1

It seems strange for MachineClass to have kvm_type(). Probably AccelClass. (struct KVMAccelClass?)

Anyway this is independent clean up.

Signature · 1 line
Isaku Yamahata <[email protected]>
author
#20 lore raw

RFC PATCH 7/19 target/i386: Implement mc->kvm_type() to get VM type

In reply to Xiaoyao Li

Implement mc->kvm_type() for i386 machines. It provides a way for user to create SW_PROTECTE_VM.

---
 hw/i386/x86.c              | 27 +++++++++++++++++++++++++++
 include/hw/i386/x86.h      |  4 ++++
 target/i386/kvm/kvm.c      | 38 ++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/kvm_i386.h |  1 +
 4 files changed, 70 insertions(+)
hw/i386/x86.c +27 -0
index a88a126123be..3ccd06154249 100644
@@ -1382,6 +1382,26 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,
     qapi_free_SgxEPCList(list);
 }
 
+static int x86_kvm_type(MachineState *ms, const char *vm_type)
+{
+    return kvm_get_vm_type(ms, vm_type);
+}
+
+static char *x86_machine_get_kvm_type(Object *obj, Error **errp)
+{
+    X86MachineState *x86ms = X86_MACHINE(obj);
+
+    return g_strdup(x86ms->kvm_type);
+}
+
+static void x86_machine_set_kvm_type(Object *obj, const char *value, Error **errp)
+{
+    X86MachineState *x86ms = X86_MACHINE(obj);
+
+    g_free(x86ms->kvm_type);
+    x86ms->kvm_type = g_strdup(value);
+}
+
 static void x86_machine_initfn(Object *obj)
 {
     X86MachineState *x86ms = X86_MACHINE(obj);
@@ -1406,6 +1426,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
     mc->cpu_index_to_instance_props = x86_cpu_index_to_props;
     mc->get_default_cpu_node_id = x86_get_default_cpu_node_id;
     mc->possible_cpu_arch_ids = x86_possible_cpu_arch_ids;
+    mc->kvm_type = x86_kvm_type;
     x86mc->save_tsc_khz = true;
     x86mc->fwcfg_dma_enabled = true;
     nc->nmi_monitor_handler = x86_nmi;
@@ -1464,6 +1485,12 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
         NULL, NULL);
     object_class_property_set_description(oc, "sgx-epc",
         "SGX EPC device");
+
+    object_class_property_add_str(oc, X86_MACHINE_KVM_TYPE,
+                                  x86_machine_get_kvm_type,
+                                  x86_machine_set_kvm_type);
+    object_class_property_set_description(oc, X86_MACHINE_KVM_TYPE,
+                                          "KVM guest type (default, sw_protected_vm)");
 }
 
 static const TypeInfo x86_machine_info = {
include/hw/i386/x86.h +4 -0
index da19ae15463a..a3d03f78cefe 100644
@@ -42,6 +42,9 @@ struct X86MachineState {
 
     /*< public >*/
 
+    char *kvm_type;
+    unsigned int vm_type;
+
     /* Pointers to devices and objects: */
     ISADevice *rtc;
     FWCfgState *fw_cfg;
@@ -91,6 +94,7 @@ struct X86MachineState {
 #define X86_MACHINE_OEM_ID           "x-oem-id"
 #define X86_MACHINE_OEM_TABLE_ID     "x-oem-table-id"
 #define X86_MACHINE_BUS_LOCK_RATELIMIT  "bus-lock-ratelimit"
+#define X86_MACHINE_KVM_TYPE         "kvm-type"
 
 #define TYPE_X86_MACHINE   MACHINE_TYPE_NAME("x86")
 OBJECT_DECLARE_TYPE(X86MachineState, X86MachineClass, X86_MACHINE)
target/i386/kvm/kvm.c +38 -0
index f8cc8eb1fe70..7971f0fd74b1 100644
@@ -154,6 +154,44 @@ static KVMMSRHandlers msr_handlers[KVM_MSR_FILTER_MAX_RANGES];
 static RateLimit bus_lock_ratelimit_ctrl;
 static int kvm_get_one_msr(X86CPU *cpu, int index, uint64_t *value);
 
+static const char* vm_type_name[] = {
+    [KVM_X86_DEFAULT_VM] = "default",
+    [KVM_X86_SW_PROTECTED_VM] = "sw-protected-vm",
+};
+
+int kvm_get_vm_type(MachineState *ms, const char *vm_type)
+{
+    X86MachineState *x86ms = X86_MACHINE(ms);
+    int kvm_type = KVM_X86_DEFAULT_VM;
+
+    if (vm_type) {
+        if (!g_ascii_strcasecmp(vm_type, "default") || !g_ascii_strcasecmp(vm_type, "")) {
+            kvm_type = KVM_X86_DEFAULT_VM;
+        } else if (!g_ascii_strcasecmp(vm_type, "sw-protected-vm")) {
+            kvm_type = KVM_X86_SW_PROTECTED_VM;
+        } else {
+            error_report("Unknown kvm-type specified '%s'", vm_type);
+            exit(1);
+        }
+    }
+
+    /*
+     * old KVM doesn't support KVM_CAP_VM_TYPES and KVM_X86_DEFAULT_VM
+     * is always supported
+     */
+    if (kvm_type == KVM_X86_DEFAULT_VM) {
+        return kvm_type;
+    }
+
+    if (!(kvm_check_extension(KVM_STATE(ms->accelerator), KVM_CAP_VM_TYPES) & BIT(kvm_type))) {
+        error_report("vm-type %s not supported by KVM", vm_type_name[kvm_type]);
+        exit(1);
+    }
+
+    x86ms->vm_type = kvm_type;
+    return kvm_type;
+}
+
 int kvm_has_pit_state2(void)
 {
     return has_pit_state2;
target/i386/kvm/kvm_i386.h +1 -0
index e24753abfe6a..ea3a5b174ac0 100644
@@ -37,6 +37,7 @@ bool kvm_has_adjust_clock(void);
 bool kvm_has_adjust_clock_stable(void);
 bool kvm_has_exception_payload(void);
 void kvm_synchronize_all_tsc(void);
+int kvm_get_vm_type(MachineState *ms, const char *vm_type);
 void kvm_arch_reset_vcpu(X86CPU *cs);
 void kvm_arch_after_reset_vcpu(X86CPU *cpu);
 void kvm_arch_do_init_vcpu(X86CPU *cs);
Signature · 1 line
2.34.1
author
#21 lore raw

RFC PATCH 8/19 HostMem: Add private property to indicate to use kvm gmem

In reply to Xiaoyao Li

From: Isaku Yamahata <[email protected]>

---
 backends/hostmem.c       | 18 ++++++++++++++++++
 include/sysemu/hostmem.h |  2 +-
 qapi/qom.json            |  4 ++++
 3 files changed, 23 insertions(+), 1 deletion(-)
backends/hostmem.c +18 -0
index 747e7838c031..dbdbb0aafd45 100644
@@ -461,6 +461,20 @@ static void host_memory_backend_set_reserve(Object *o, bool value, Error **errp)
     }
     backend->reserve = value;
 }
+
+static bool host_memory_backend_get_private(Object *o, Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    return backend->private;
+}
+
+static void host_memory_backend_set_private(Object *o, bool value, Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    backend->private = value;
+}
 #endif /* CONFIG_LINUX */
 
 static bool
@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc, void *data)
         host_memory_backend_get_reserve, host_memory_backend_set_reserve);
     object_class_property_set_description(oc, "reserve",
         "Reserve swap space (or huge pages) if applicable");
+    object_class_property_add_bool(oc, "private",
+        host_memory_backend_get_private, host_memory_backend_set_private);
+    object_class_property_set_description(oc, "private",
+        "Use KVM gmem private memory");
 #endif /* CONFIG_LINUX */
     /*
      * Do not delete/rename option. This option must be considered stable
include/sysemu/hostmem.h +1 -1
index 39326f1d4f9c..d88970395618 100644
@@ -65,7 +65,7 @@ struct HostMemoryBackend {
     /* protected */
     uint64_t size;
     bool merge, dump, use_canonical_path;
-    bool prealloc, is_mapped, share, reserve;
+    bool prealloc, is_mapped, share, reserve, private;
     uint32_t prealloc_threads;
     ThreadContext *prealloc_context;
     DECLARE_BITMAP(host_nodes, MAX_NODES + 1);
qapi/qom.json +4 -0
index 7f92ea43e8e1..e0b2044e3d20 100644
@@ -605,6 +605,9 @@
 # @reserve: if true, reserve swap space (or huge pages) if applicable
 #     (default: true) (since 6.1)
 #
+# @private: if true, use KVM gmem private memory
+#           (default: false) (since 8.1)
+#
 # @size: size of the memory region in bytes
 #
 # @x-use-canonical-path-for-ramblock-id: if true, the canonical path
@@ -631,6 +634,7 @@
             '*prealloc-context': 'str',
             '*share': 'bool',
             '*reserve': 'bool',
+            '*private': 'bool',
             'size': 'size',
             '*x-use-canonical-path-for-ramblock-id': 'bool' } }
 
Signature · 1 line
2.34.1
#22 lore raw

Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem

Quoted 4 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

[...]

qapi/qom.json line 610

Quoted 11 lines
qapi/qom.json +3 -0
index 7f92ea43e8e1..e0b2044e3d20 100644
@@ -605,6 +605,9 @@
 # @reserve: if true, reserve swap space (or huge pages) if applicable
 #     (default: true) (since 6.1)
 #
+# @private: if true, use KVM gmem private memory
+#           (default: false) (since 8.1)
+#

Please format like

# @private: if true, use KVM gmem private memory (default: false)
# (since 8.1)

to blend in with recent commit a937b6aa739 (qapi: Reformat doc comments to conform to current conventions).

Quoted 10 lines
qapi/qom.json +1 -0
 # @size: size of the memory region in bytes
 #
 # @x-use-canonical-path-for-ramblock-id: if true, the canonical path
@@ -631,6 +634,7 @@
             '*prealloc-context': 'str',
             '*share': 'bool',
             '*reserve': 'bool',
+            '*private': 'bool',
             'size': 'size',
             '*x-use-canonical-path-for-ramblock-id': 'bool' } }
author
#23 lore raw

Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem

Quoted 23 lines · Markus Armbruster
Quoted 4 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

[...]

qapi/qom.json line 610

Quoted 11 lines
qapi/qom.json +3 -0
index 7f92ea43e8e1..e0b2044e3d20 100644
@@ -605,6 +605,9 @@
  # @reserve: if true, reserve swap space (or huge pages) if applicable
  #     (default: true) (since 6.1)
  #
+# @private: if true, use KVM gmem private memory
+#           (default: false) (since 8.1)
+#

Please format like

Quoted 5 lines

# @private: if true, use KVM gmem private memory (default: false)
# (since 8.1)

to blend in with recent commit a937b6aa739 (qapi: Reformat doc comments to conform to current conventions).

will do it in next version.

Thanks!
-Xiaoyao

Quoted 11 lines
Quoted 10 lines
qapi/qom.json +1 -0
  # @size: size of the memory region in bytes
  #
  # @x-use-canonical-path-for-ramblock-id: if true, the canonical path
@@ -631,6 +634,7 @@
              '*prealloc-context': 'str',
              '*share': 'bool',
              '*reserve': 'bool',
+            '*private': 'bool',
              'size': 'size',
              '*x-use-canonical-path-for-ramblock-id': 'bool' } }
#24 lore raw

Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem

In reply to Markus Armbruster
Quoted 23 lines · Markus Armbruster
Quoted 4 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

[...]

qapi/qom.json line 610

Quoted 11 lines
qapi/qom.json +3 -0
index 7f92ea43e8e1..e0b2044e3d20 100644
@@ -605,6 +605,9 @@
 # @reserve: if true, reserve swap space (or huge pages) if applicable
 #     (default: true) (since 6.1)
 #
+# @private: if true, use KVM gmem private memory
+#           (default: false) (since 8.1)
+#

Please format like

Quoted 2 lines

# @private: if true, use KVM gmem private memory (default: false)
# (since 8.1)

Also QEMU 8.1.0 is in freeze right now, so there's no chance of these patches making 8.1.0. IOW, use "since 8.2" as the next release you might achieve merge for.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#25 lore raw

Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem

Quoted 24 lines · Daniel P. Berrangé
Quoted 23 lines · Markus Armbruster
Quoted 4 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

[...]

qapi/qom.json line 610

Quoted 11 lines
qapi/qom.json +3 -0
index 7f92ea43e8e1..e0b2044e3d20 100644
@@ -605,6 +605,9 @@
  # @reserve: if true, reserve swap space (or huge pages) if applicable
  #     (default: true) (since 6.1)
  #
+# @private: if true, use KVM gmem private memory
+#           (default: false) (since 8.1)
+#

Please format like

Quoted 6 lines
Quoted 2 lines

# @private: if true, use KVM gmem private memory (default: false)
# (since 8.1)

Also QEMU 8.1.0 is in freeze right now, so there's no chance of these patches making 8.1.0. IOW, use "since 8.2" as the next release you might achieve merge for.

Thanks for pointing it out. Will do it in next version.

Quoted 2 lines

With regards,
Daniel

#26 lore raw

Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem

In reply to Xiaoyao Li
Quoted 64 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

---
  backends/hostmem.c       | 18 ++++++++++++++++++
  include/sysemu/hostmem.h |  2 +-
  qapi/qom.json            |  4 ++++
  3 files changed, 23 insertions(+), 1 deletion(-)
backends/hostmem.c +18 -0
index 747e7838c031..dbdbb0aafd45 100644
@@ -461,6 +461,20 @@ static void host_memory_backend_set_reserve(Object *o, bool value, Error **errp)
      }
      backend->reserve = value;
  }
+
+static bool host_memory_backend_get_private(Object *o, Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    return backend->private;
+}
+
+static void host_memory_backend_set_private(Object *o, bool value, Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    backend->private = value;
+}
  #endif /* CONFIG_LINUX */
 
  static bool
@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc, void *data)
          host_memory_backend_get_reserve, host_memory_backend_set_reserve);
      object_class_property_set_description(oc, "reserve",
          "Reserve swap space (or huge pages) if applicable");
+    object_class_property_add_bool(oc, "private",
+        host_memory_backend_get_private, host_memory_backend_set_private);
+    object_class_property_set_description(oc, "private",
+        "Use KVM gmem private memory");
  #endif /* CONFIG_LINUX */
      /*
       * Do not delete/rename option. This option must be considered stable
include/sysemu/hostmem.h +1 -1
index 39326f1d4f9c..d88970395618 100644
@@ -65,7 +65,7 @@ struct HostMemoryBackend {
      /* protected */
      uint64_t size;
      bool merge, dump, use_canonical_path;
-    bool prealloc, is_mapped, share, reserve;
+    bool prealloc, is_mapped, share, reserve, private;
      uint32_t prealloc_threads;
      ThreadContext *prealloc_context;
      DECLARE_BITMAP(host_nodes, MAX_NODES + 1);
qapi/qom.json +0 -0
index 7f92ea43e8e1..e0b2044e3d20 100644

qapi/qom.json line 610

Quoted 6 lines
qapi/qom.json +3 -0
@@ -605,6 +605,9 @@
  # @reserve: if true, reserve swap space (or huge pages) if applicable
  #     (default: true) (since 6.1)
  #
+# @private: if true, use KVM gmem private memory
+#           (default: false) (since 8.1)
+#

But that's not what any of this does.

This patch only adds a property and doesn't even explain what it intends to achieve with that.

How will it be used from a user? What will it affect internally? What will it modify in regards of the memory backend?

That all should go into the surprisingly empty patch description.

Signature · 3 lines
Cheers,

David / dhildenb
author
#27 lore raw

Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem

Quoted 77 lines · David Hildenbrand
Quoted 67 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

---
  backends/hostmem.c       | 18 ++++++++++++++++++ include/sysemu/hostmem.h |  2 +- qapi/qom.json            |  4 ++++

  3 files changed, 23 insertions(+), 1 deletion(-)
backends/hostmem.c +14 -0
index 747e7838c031..dbdbb0aafd45 100644
@@ -461,6 +461,20 @@ static void 
 host_memory_backend_set_reserve(Object *o, bool value, Error **errp)
       }
       backend->reserve = value;
   }
+
+static bool host_memory_backend_get_private(Object *o, Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    return backend->private;
+}
+
+static void host_memory_backend_set_private(Object *o, bool value, 
 Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    backend->private = value;
+}
   #endif /* CONFIG_LINUX */

  static bool

backends/hostmem.c +4 -0
@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc, 
 void *data)
           host_memory_backend_get_reserve, host_memory_backend_set_reserve);
       object_class_property_set_description(oc, "reserve",
           "Reserve swap space (or huge pages) if applicable");
+    object_class_property_add_bool(oc, "private",
+        host_memory_backend_get_private, 
 host_memory_backend_set_private);
+    object_class_property_set_description(oc, "private",
+        "Use KVM gmem private memory");
   #endif /* CONFIG_LINUX */

      /*
       * Do not delete/rename option. This option must be considered stable

include/sysemu/hostmem.h +1 -1
index 39326f1d4f9c..d88970395618 100644
@@ -65,7 +65,7 @@ struct HostMemoryBackend {
       /* protected */
       uint64_t size;
       bool merge, dump, use_canonical_path;
-    bool prealloc, is_mapped, share, reserve;
+    bool prealloc, is_mapped, share, reserve, private;
       uint32_t prealloc_threads;
       ThreadContext *prealloc_context;
       DECLARE_BITMAP(host_nodes, MAX_NODES + 1);
qapi/qom.json +0 -0
index 7f92ea43e8e1..e0b2044e3d20 100644

qapi/qom.json line 610

Quoted 6 lines
qapi/qom.json +3 -0
@@ -605,6 +605,9 @@
   # @reserve: if true, reserve swap space (or huge pages) if applicable
   #     (default: true) (since 6.1)
   #
+# @private: if true, use KVM gmem private memory
+#           (default: false) (since 8.1)
+#

But that's not what any of this does.

Quoted 3 lines

This patch only adds a property and doesn't even explain what it intends to achieve with that.

How will it be used from a user? What will it affect internally? What will it modify in regards of the memory backend?

How it will be used is in the next patch, patch 09.

for kvm_x86_sw_protected_vm type VM, it will allocate private gmem with KVM ioctl if the memory backend has property "private" on.

Quoted 1 line

That all should go into the surprisingly empty patch description.

I'm sorry. I admit the empty commit message is really bad.

#28 lore raw

Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem

Quoted 83 lines · Xiaoyao Li
Quoted 82 lines · David Hildenbrand
Quoted 69 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

---
   backends/hostmem.c       | 18 ++++++++++++++++++
   include/sysemu/hostmem.h |  2 +-
   qapi/qom.json            |  4 ++++
   3 files changed, 23 insertions(+), 1 deletion(-)
backends/hostmem.c +14 -0
index 747e7838c031..dbdbb0aafd45 100644
@@ -461,6 +461,20 @@ static void
 host_memory_backend_set_reserve(Object *o, bool value, Error **errp)
       }
       backend->reserve = value;
   }
+
+static bool host_memory_backend_get_private(Object *o, Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    return backend->private;
+}
+
+static void host_memory_backend_set_private(Object *o, bool value,
 Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    backend->private = value;
+}
   #endif /* CONFIG_LINUX */

  static bool

backends/hostmem.c +4 -0
@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc,
 void *data)
           host_memory_backend_get_reserve,
 host_memory_backend_set_reserve);
       object_class_property_set_description(oc, "reserve",
           "Reserve swap space (or huge pages) if applicable");
+    object_class_property_add_bool(oc, "private",
+        host_memory_backend_get_private,
 host_memory_backend_set_private);
+    object_class_property_set_description(oc, "private",
+        "Use KVM gmem private memory");

  #endif /* CONFIG_LINUX */
      /*
       * Do not delete/rename option. This option must be considered
stable

include/sysemu/hostmem.h +1 -1
index 39326f1d4f9c..d88970395618 100644
@@ -65,7 +65,7 @@ struct HostMemoryBackend {
       /* protected */
       uint64_t size;
       bool merge, dump, use_canonical_path;
-    bool prealloc, is_mapped, share, reserve;
+    bool prealloc, is_mapped, share, reserve, private;
       uint32_t prealloc_threads;
       ThreadContext *prealloc_context;
       DECLARE_BITMAP(host_nodes, MAX_NODES + 1);
qapi/qom.json +0 -0
index 7f92ea43e8e1..e0b2044e3d20 100644

qapi/qom.json line 610

Quoted 6 lines
qapi/qom.json +3 -0
@@ -605,6 +605,9 @@
   # @reserve: if true, reserve swap space (or huge pages) if applicable
   #     (default: true) (since 6.1)
   #
+# @private: if true, use KVM gmem private memory
+#           (default: false) (since 8.1)
+#

But that's not what any of this does.

This patch only adds a property and doesn't even explain what it intends to achieve with that.

Quoted 7 lines
Quoted 2 lines

How will it be used from a user? What will it affect internally? What will it modify in regards of the memory backend?

How it will be used is in the next patch, patch 09.

for kvm_x86_sw_protected_vm type VM, it will allocate private gmem with KVM ioctl if the memory backend has property "private" on.

It feels wired up the wrong way.

When creating/initializing the memory backend, we should also take care of allocating the gmem_fd, for example, by doing some gmem allocation callback, ideally *internally* creating the RAM memory region / RAMBlock.

And we should fail if that is impossible (gmem does not apply to the VM) or creating the gmem_fd failed for other reason.

Like passing a RAM_GMEM flag to memory_region_init_ram_flags_nomigrate() in ram_backend_memory_alloc(), to then handle it internally, failing if there is an error.

Signature · 3 lines
Cheers,

David / dhildenb
#29 lore raw

Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem

Quoted 98 lines · David Hildenbrand
Quoted 83 lines · Xiaoyao Li
Quoted 82 lines · David Hildenbrand
Quoted 69 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

---
   backends/hostmem.c       | 18 ++++++++++++++++++
   include/sysemu/hostmem.h |  2 +-
   qapi/qom.json            |  4 ++++
   3 files changed, 23 insertions(+), 1 deletion(-)
backends/hostmem.c +14 -0
index 747e7838c031..dbdbb0aafd45 100644
@@ -461,6 +461,20 @@ static void
 host_memory_backend_set_reserve(Object *o, bool value, Error **errp)
       }
       backend->reserve = value;
   }
+
+static bool host_memory_backend_get_private(Object *o, Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    return backend->private;
+}
+
+static void host_memory_backend_set_private(Object *o, bool value,
 Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    backend->private = value;
+}
   #endif /* CONFIG_LINUX */

  static bool

backends/hostmem.c +4 -0
@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc,
 void *data)
           host_memory_backend_get_reserve,
 host_memory_backend_set_reserve);
       object_class_property_set_description(oc, "reserve",
           "Reserve swap space (or huge pages) if applicable");
+    object_class_property_add_bool(oc, "private",
+        host_memory_backend_get_private,
 host_memory_backend_set_private);
+    object_class_property_set_description(oc, "private",
+        "Use KVM gmem private memory");

  #endif /* CONFIG_LINUX */
      /*
       * Do not delete/rename option. This option must be considered
stable

include/sysemu/hostmem.h +1 -1
index 39326f1d4f9c..d88970395618 100644
@@ -65,7 +65,7 @@ struct HostMemoryBackend {
       /* protected */
       uint64_t size;
       bool merge, dump, use_canonical_path;
-    bool prealloc, is_mapped, share, reserve;
+    bool prealloc, is_mapped, share, reserve, private;
       uint32_t prealloc_threads;
       ThreadContext *prealloc_context;
       DECLARE_BITMAP(host_nodes, MAX_NODES + 1);
qapi/qom.json +0 -0
index 7f92ea43e8e1..e0b2044e3d20 100644

qapi/qom.json line 610

Quoted 6 lines
qapi/qom.json +3 -0
@@ -605,6 +605,9 @@
   # @reserve: if true, reserve swap space (or huge pages) if applicable
   #     (default: true) (since 6.1)
   #
+# @private: if true, use KVM gmem private memory
+#           (default: false) (since 8.1)
+#

But that's not what any of this does.

This patch only adds a property and doesn't even explain what it intends to achieve with that.

Quoted 7 lines
Quoted 2 lines

How will it be used from a user? What will it affect internally? What will it modify in regards of the memory backend?

How it will be used is in the next patch, patch 09.

for kvm_x86_sw_protected_vm type VM, it will allocate private gmem with KVM ioctl if the memory backend has property "private" on.

It feels wired up the wrong way.

When creating/initializing the memory backend, we should also take care of allocating the gmem_fd, for example, by doing some gmem allocation callback, ideally *internally* creating the RAM memory region / RAMBlock.

Quoted 6 lines

And we should fail if that is impossible (gmem does not apply to the VM) or creating the gmem_fd failed for other reason.

Like passing a RAM_GMEM flag to memory_region_init_ram_flags_nomigrate() in ram_backend_memory_alloc(), to then handle it internally, failing if there is an error.

KVM gmem is tied to VM. not before creating VM. We have to delay of the allocation of kvm gmem until VM initialization.

Hmm, one options is to move gmem_fd from RAMBlock to KVMSlot. Handle the allocation of KVM gmem (issuing KVM gmem ioctl) there. i.e. in kvm_set_phys_mem() or kvm_region_add() (or whatever functions of KVM memory listener). Maybe we can drop ram_block_convert_range() and can have KVM specific logic instead.

We still need a way for user to specify which guest memory region is subject to KVM gmem, though.

Signature · 1 line
Isaku Yamahata <[email protected]>
#30 lore raw

Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem

Quoted 104 lines · Isaku Yamahata
Quoted 101 lines · David Hildenbrand
Quoted 83 lines · Xiaoyao Li
Quoted 82 lines · David Hildenbrand
Quoted 69 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

---
    backends/hostmem.c       | 18 ++++++++++++++++++
    include/sysemu/hostmem.h |  2 +-
    qapi/qom.json            |  4 ++++
    3 files changed, 23 insertions(+), 1 deletion(-)
backends/hostmem.c +14 -0
index 747e7838c031..dbdbb0aafd45 100644
@@ -461,6 +461,20 @@ static void
 host_memory_backend_set_reserve(Object *o, bool value, Error **errp)
        }
        backend->reserve = value;
    }
+
+static bool host_memory_backend_get_private(Object *o, Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    return backend->private;
+}
+
+static void host_memory_backend_set_private(Object *o, bool value,
 Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    backend->private = value;
+}
    #endif /* CONFIG_LINUX */

  static bool

backends/hostmem.c +4 -0
@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc,
 void *data)
            host_memory_backend_get_reserve,
 host_memory_backend_set_reserve);
        object_class_property_set_description(oc, "reserve",
            "Reserve swap space (or huge pages) if applicable");
+    object_class_property_add_bool(oc, "private",
+        host_memory_backend_get_private,
 host_memory_backend_set_private);
+    object_class_property_set_description(oc, "private",
+        "Use KVM gmem private memory");

  #endif /* CONFIG_LINUX */
      /*
       * Do not delete/rename option. This option must be considered
stable

include/sysemu/hostmem.h +1 -1
index 39326f1d4f9c..d88970395618 100644
@@ -65,7 +65,7 @@ struct HostMemoryBackend {
        /* protected */
        uint64_t size;
        bool merge, dump, use_canonical_path;
-    bool prealloc, is_mapped, share, reserve;
+    bool prealloc, is_mapped, share, reserve, private;
        uint32_t prealloc_threads;
        ThreadContext *prealloc_context;
        DECLARE_BITMAP(host_nodes, MAX_NODES + 1);
qapi/qom.json +0 -0
index 7f92ea43e8e1..e0b2044e3d20 100644

qapi/qom.json line 610

Quoted 6 lines
qapi/qom.json +3 -0
@@ -605,6 +605,9 @@
    # @reserve: if true, reserve swap space (or huge pages) if applicable
    #     (default: true) (since 6.1)
    #
+# @private: if true, use KVM gmem private memory
+#           (default: false) (since 8.1)
+#

But that's not what any of this does.

This patch only adds a property and doesn't even explain what it intends to achieve with that.

Quoted 7 lines
Quoted 2 lines

How will it be used from a user? What will it affect internally? What will it modify in regards of the memory backend?

How it will be used is in the next patch, patch 09.

for kvm_x86_sw_protected_vm type VM, it will allocate private gmem with KVM ioctl if the memory backend has property "private" on.

It feels wired up the wrong way.

When creating/initializing the memory backend, we should also take care of allocating the gmem_fd, for example, by doing some gmem allocation callback, ideally *internally* creating the RAM memory region / RAMBlock.

And we should fail if that is impossible (gmem does not apply to the VM) or creating the gmem_fd failed for other reason.

Quoted 6 lines
Quoted 3 lines

Like passing a RAM_GMEM flag to memory_region_init_ram_flags_nomigrate() in ram_backend_memory_alloc(), to then handle it internally, failing if there is an error.

KVM gmem is tied to VM. not before creating VM. We have to delay of the allocation of kvm gmem until VM initialization.

In vl.c, the flow is

1) Create machine: qemu_create_machine()
2) Configure KVM: configure_accelerators()
3) Create backends: qemu_create_late_backends()

Staring at object_create_early(), "memory-backend-" area always late.

So maybe, at the time memory backends are created+initialized, everything you need is already in place.

Quoted 6 lines

Hmm, one options is to move gmem_fd from RAMBlock to KVMSlot. Handle the allocation of KVM gmem (issuing KVM gmem ioctl) there. i.e. in kvm_set_phys_mem() or kvm_region_add() (or whatever functions of KVM memory listener). Maybe we can drop ram_block_convert_range() and can have KVM specific logic instead.

Might be doable as well.

Quoted 3 lines

We still need a way for user to specify which guest memory region is subject to KVM gmem, though.

Let's minimize the gmem hacks and come up with something clean.

Signature · 3 lines
Cheers,

David / dhildenb
author
#31 lore raw

RFC PATCH 9/19 i386/kvm: Create gmem fd for KVM_X86_SW_PROTECTED_VM

In reply to Xiaoyao Li

Register a memory listener for KVM_X86_SW_PROVTED_VM. It creates gmem for the backend who sets the private property.

---
 include/exec/memory.h |  1 +
 target/i386/kvm/kvm.c | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 39 insertions(+)
include/exec/memory.h +1 -0
index e119d3ce1a1d..ddf0e14970b0 100644
@@ -814,6 +814,7 @@ struct IOMMUMemoryRegion {
 #define MEMORY_LISTENER_PRIORITY_MIN            0
 #define MEMORY_LISTENER_PRIORITY_ACCEL          10
 #define MEMORY_LISTENER_PRIORITY_DEV_BACKEND    10
+#define MEMORY_LISTENER_PRIORITY_ACCEL_HIGH     20
 
 /**
  * struct MemoryListener: callbacks structure for updates to the physical memory map
target/i386/kvm/kvm.c +38 -0
index 7971f0fd74b1..df3a5f89396e 100644
@@ -37,6 +37,7 @@
 #include "hyperv-proto.h"
 
 #include "exec/gdbstub.h"
+#include "exec/ramblock.h"
 #include "qemu/host-utils.h"
 #include "qemu/main-loop.h"
 #include "qemu/ratelimit.h"
@@ -2591,8 +2592,41 @@ static void register_smram_listener(Notifier *n, void *unused)
                                  &smram_address_space, 1, "kvm-smram");
 }
 
+static void kvm_x86_sw_protected_vm_region_add(MemoryListener *listenr,
+                                       MemoryRegionSection *section)
+{
+    MemoryRegion *mr = section->mr;
+    Object *owner = memory_region_owner(mr);
+    int fd;
+
+    if (owner && object_dynamic_cast(owner, TYPE_MEMORY_BACKEND) &&
+        object_property_get_bool(owner, "private", NULL) &&
+        mr->ram_block && mr->ram_block->gmem_fd < 0) {
+        struct kvm_create_guest_memfd gmem = {
+            .size = memory_region_size(mr),
+            /* TODO: to decide whether KVM_GUEST_MEMFD_ALLOW_HUGEPAGE is supported */
+            .flags = KVM_GUEST_MEMFD_ALLOW_HUGEPAGE,
+        };
+
+        fd = kvm_vm_ioctl(kvm_state, KVM_CREATE_GUEST_MEMFD, &gmem);
+        if (fd < 0) {
+            error_report("%s: error creating gmem: %s\n", __func__, strerror(-fd));
+            exit(1);
+        }
+        memory_region_set_gmem_fd(mr, fd);
+    }
+}
+
+static MemoryListener kvm_x86_sw_protected_vm_memory_listener = {
+    .name = "kvm_x86_sw_protected_vm_memory_listener",
+    .region_add = kvm_x86_sw_protected_vm_region_add,
+    /* Higher than KVM memory listener = 10. */
+    .priority = MEMORY_LISTENER_PRIORITY_ACCEL_HIGH,
+};
+
 int kvm_arch_init(MachineState *ms, KVMState *s)
 {
+    X86MachineState *x86ms = X86_MACHINE(ms);
     uint64_t identity_base = 0xfffbc000;
     uint64_t shadow_mem;
     int ret;
@@ -2617,6 +2651,10 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
         return ret;
     }
 
+    if (x86ms->vm_type == KVM_X86_SW_PROTECTED_VM) {
+        memory_listener_register(&kvm_x86_sw_protected_vm_memory_listener, &address_space_memory);
+    }
+
     if (!kvm_check_extension(s, KVM_CAP_IRQ_ROUTING)) {
         error_report("kvm: KVM_CAP_IRQ_ROUTING not supported by KVM");
         return -ENOTSUP;
Signature · 1 line
2.34.1
author
#32 lore raw

RFC PATCH 10/19 kvm: Introduce support for memory_attributes

In reply to Xiaoyao Li

Introcude the helper functions to set the attributes of a range of memory to private and shared.

---
 accel/kvm/kvm-all.c  | 43 +++++++++++++++++++++++++++++++++++++++++++
 include/sysemu/kvm.h |  3 +++
 2 files changed, 46 insertions(+)
accel/kvm/kvm-all.c +43 -0
index 7b1818334ba7..6dd22fa4fd6f 100644
@@ -105,6 +105,7 @@ bool kvm_msi_use_devid;
 bool kvm_has_guest_debug;
 static int kvm_sstep_flags;
 static bool kvm_immediate_exit;
+static uint64_t kvm_supported_memory_attributes;
 static hwaddr kvm_max_slot_size = ~0;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -1343,6 +1344,44 @@ void kvm_set_max_memslot_size(hwaddr max_slot_size)
     kvm_max_slot_size = max_slot_size;
 }
 
+static int kvm_set_memory_attributes(hwaddr start, hwaddr size, uint64_t attr)
+{
+    struct kvm_memory_attributes attrs;
+    int r;
+
+    attrs.attributes = attr;
+    attrs.address = start;
+    attrs.size = size;
+    attrs.flags = 0;
+
+    r = kvm_vm_ioctl(kvm_state, KVM_SET_MEMORY_ATTRIBUTES, &attrs);
+    if (r) {
+        warn_report("%s: failed to set memory (0x%lx+%#zx) with attr 0x%lx error '%s'",
+                     __func__, start, size, attr, strerror(errno));
+    }
+    return r;
+}
+
+int kvm_set_memory_attributes_private(hwaddr start, hwaddr size)
+{
+    if (!(kvm_supported_memory_attributes & KVM_MEMORY_ATTRIBUTE_PRIVATE)) {
+        error_report("KVM doesn't support PRIVATE memory attribute\n");
+        return -EINVAL;
+    }
+
+    return kvm_set_memory_attributes(start, size, KVM_MEMORY_ATTRIBUTE_PRIVATE);
+}
+
+int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size)
+{
+    if (!(kvm_supported_memory_attributes & KVM_MEMORY_ATTRIBUTE_PRIVATE)) {
+        error_report("KVM doesn't support PRIVATE memory attribute\n");
+        return -EINVAL;
+    }
+
+    return kvm_set_memory_attributes(start, size, 0);
+}
+
 /* Called with KVMMemoryListener.slots_lock held */
 static void kvm_set_phys_mem(KVMMemoryListener *kml,
                              MemoryRegionSection *section, bool add)
@@ -2556,6 +2595,10 @@ static int kvm_init(MachineState *ms)
     }
     s->as = g_new0(struct KVMAs, s->nr_as);
 
+    if (kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES)) {
+        kvm_supported_memory_attributes = kvm_ioctl(s, KVM_GET_SUPPORTED_MEMORY_ATTRIBUTES, 0);
+    }
+
     if (object_property_find(OBJECT(current_machine), "kvm-type")) {
         g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine),
                                                             "kvm-type",
include/sysemu/kvm.h +3 -0
index 115f0cca79d1..49c896d8a512 100644
@@ -580,4 +580,7 @@ bool kvm_arch_cpu_check_are_resettable(void);
 bool kvm_dirty_ring_enabled(void);
 
 uint32_t kvm_dirty_ring_size(void);
+
+int kvm_set_memory_attributes_private(hwaddr start, hwaddr size);
+int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size);
 #endif
Signature · 1 line
2.34.1
author
#33 lore raw

RFC PATCH 11/19 kvm/memory: Introduce the infrastructure to set the default shared/private value

In reply to Xiaoyao Li

Introduce new flags for MemoryRegion to indicate the default attribute, private or not. And update the attribute if default private.

Originated-from: Isaku Yamahata <[email protected]>

---
 accel/kvm/kvm-all.c   | 10 ++++++++++
 include/exec/memory.h |  6 ++++++
 softmmu/memory.c      | 13 +++++++++++++
 3 files changed, 29 insertions(+)
accel/kvm/kvm-all.c +10 -0
index 6dd22fa4fd6f..f9b5050b8885 100644
@@ -1487,6 +1487,16 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
                     strerror(-err));
             abort();
         }
+
+        if (memory_region_is_default_private(mr)) {
+            err = kvm_set_memory_attributes_private(start_addr, slot_size);
+            if (err) {
+                error_report("%s: failed to set memory attribute private: %s\n",
+                             __func__, strerror(-err));
+                exit(1);
+            }
+        }
+
         start_addr += slot_size;
         ram_start_offset += slot_size;
         ram += slot_size;
include/exec/memory.h +6 -0
index ddf0e14970b0..759f797b6acd 100644
@@ -235,6 +235,9 @@ typedef struct IOMMUTLBEvent {
 /* RAM is an mmap-ed named file */
 #define RAM_NAMED_FILE (1 << 9)
 
+/* RAM is default private */
+#define RAM_DEFAULT_PRIVATE     (1 << 10)
+
 static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn,
                                        IOMMUNotifierFlag flags,
                                        hwaddr start, hwaddr end,
@@ -1689,6 +1692,9 @@ bool memory_region_is_protected(MemoryRegion *mr);
  */
 bool memory_region_can_be_private(MemoryRegion *mr);
 
+void memory_region_set_default_private(MemoryRegion *mr);
+bool memory_region_is_default_private(MemoryRegion *mr);
+
 /**
  * memory_region_get_iommu: check whether a memory region is an iommu
  *
softmmu/memory.c +13 -0
index 336c76ede660..af6aa3c1e3c9 100644
@@ -1860,6 +1860,19 @@ bool memory_region_can_be_private(MemoryRegion *mr)
     return mr->ram_block && mr->ram_block->gmem_fd >= 0;
 }
 
+bool memory_region_is_default_private(MemoryRegion *mr)
+{
+    return mr->ram_block && mr->ram_block->gmem_fd >= 0 &&
+        (mr->ram_block->flags & RAM_DEFAULT_PRIVATE);
+}
+
+void memory_region_set_default_private(MemoryRegion *mr)
+{
+    if (mr->ram_block && mr->ram_block->gmem_fd >= 0) {
+        mr->ram_block->flags |= RAM_DEFAULT_PRIVATE;
+    }
+}
+
 uint8_t memory_region_get_dirty_log_mask(MemoryRegion *mr)
 {
     uint8_t mask = mr->dirty_log_mask;
Signature · 1 line
2.34.1
author
#34 lore raw

RFC PATCH 12/19 i386/kvm: Set memory to default private for KVM_X86_SW_PROTECTED_VM

In reply to Xiaoyao Li
---
 target/i386/kvm/kvm.c | 1 +
 1 file changed, 1 insertion(+)
target/i386/kvm/kvm.c +1 -0
index df3a5f89396e..a96640512dbc 100644
@@ -2614,6 +2614,7 @@ static void kvm_x86_sw_protected_vm_region_add(MemoryListener *listenr,
             exit(1);
         }
         memory_region_set_gmem_fd(mr, fd);
+        memory_region_set_default_private(mr);
     }
 }
 
Signature · 1 line
2.34.1
author
#35 lore raw

RFC PATCH 13/19 physmem: replace function name with __func__ in ram_block_discard_range()

In reply to Xiaoyao Li
---
 softmmu/physmem.c | 34 +++++++++++++++-------------------
 1 file changed, 15 insertions(+), 19 deletions(-)
softmmu/physmem.c +15 -19
index 8f64128de0b5..05c981e5c18e 100644
@@ -3430,16 +3430,15 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length)
     uint8_t *host_startaddr = rb->host + start;
 
     if (!QEMU_PTR_IS_ALIGNED(host_startaddr, rb->page_size)) {
-        error_report("ram_block_discard_range: Unaligned start address: %p",
-                     host_startaddr);
+        error_report("%s: Unaligned start address: %p",
+                     __func__, host_startaddr);
         goto err;
     }
 
     if ((start + length) <= rb->max_length) {
         bool need_madvise, need_fallocate;
         if (!QEMU_IS_ALIGNED(length, rb->page_size)) {
-            error_report("ram_block_discard_range: Unaligned length: %zx",
-                         length);
+            error_report("%s: Unaligned length: %zx", __func__, length);
             goto err;
         }
 
@@ -3469,27 +3468,26 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length)
              * file.
              */
             if (!qemu_ram_is_shared(rb)) {
-                warn_report_once("ram_block_discard_range: Discarding RAM"
+                warn_report_once("%s: Discarding RAM"
                                  " in private file mappings is possibly"
                                  " dangerous, because it will modify the"
                                  " underlying file and will affect other"
-                                 " users of the file");
+                                 " users of the file", __func__);
             }
 
             ret = fallocate(rb->fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
                             start, length);
             if (ret) {
                 ret = -errno;
-                error_report("ram_block_discard_range: Failed to fallocate "
-                             "%s:%" PRIx64 " +%zx (%d)",
-                             rb->idstr, start, length, ret);
+                error_report("%s: Failed to fallocate %s:%" PRIx64 " +%zx (%d)",
+                             __func__, rb->idstr, start, length, ret);
                 goto err;
             }
 #else
             ret = -ENOSYS;
-            error_report("ram_block_discard_range: fallocate not available/file"
+            error_report("%s: fallocate not available/file"
                          "%s:%" PRIx64 " +%zx (%d)",
-                         rb->idstr, start, length, ret);
+                         __func__, rb->idstr, start, length, ret);
             goto err;
 #endif
         }
@@ -3507,25 +3505,23 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length)
             }
             if (ret) {
                 ret = -errno;
-                error_report("ram_block_discard_range: Failed to discard range "
+                error_report("%s: Failed to discard range "
                              "%s:%" PRIx64 " +%zx (%d)",
-                             rb->idstr, start, length, ret);
+                             __func__, rb->idstr, start, length, ret);
                 goto err;
             }
 #else
             ret = -ENOSYS;
-            error_report("ram_block_discard_range: MADVISE not available"
-                         "%s:%" PRIx64 " +%zx (%d)",
-                         rb->idstr, start, length, ret);
+            error_report("%s: MADVISE not available %s:%" PRIx64 " +%zx (%d)",
+                         __func__, rb->idstr, start, length, ret);
             goto err;
 #endif
         }
         trace_ram_block_discard_range(rb->idstr, host_startaddr, length,
                                       need_madvise, need_fallocate, ret);
     } else {
-        error_report("ram_block_discard_range: Overrun block '%s' (%" PRIu64
-                     "/%zx/" RAM_ADDR_FMT")",
-                     rb->idstr, start, length, rb->max_length);
+        error_report("%s: Overrun block '%s' (%" PRIu64 "/%zx/" RAM_ADDR_FMT")",
+                     __func__, rb->idstr, start, length, rb->max_length);
     }
 
 err:
Signature · 1 line
2.34.1
author
#36 lore raw

RFC PATCH 14/19 physmem: Add ram_block_convert_range

In reply to Xiaoyao Li

From: Chao Peng <[email protected]>

This new routine adds support for memory conversion between shared/private memory for gmem fd based private ram_block.

---
 include/exec/cpu-common.h |  2 ++
 softmmu/physmem.c         | 61 ++++++++++++++++++++++++++++++---------
 2 files changed, 50 insertions(+), 13 deletions(-)
include/exec/cpu-common.h +2 -0
index 87dc9a752c9a..558684b9f246 100644
@@ -157,6 +157,8 @@ typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque);
 
 int qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque);
 int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length);
+int ram_block_convert_range(RAMBlock *rb, uint64_t start, size_t length,
+                            bool shared_to_private);
 
 #endif
 
softmmu/physmem.c +48 -13

94 lines not on this page · raw message

Signature · 1 line
2.34.1
author
#37 lore raw

RFC PATCH 15/19 kvm: handle KVM_EXIT_MEMORY_FAULT

In reply to Xiaoyao Li

From: Chao Peng <[email protected]>

Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when KVM_EXIT_MEMORY_FAULT happens. It indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.

Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has gmem memory backend.

---
 accel/kvm/kvm-all.c | 52 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
accel/kvm/kvm-all.c +52 -0
index f9b5050b8885..72d50b923bf2 100644
@@ -3040,6 +3040,48 @@ static void kvm_eat_signals(CPUState *cpu)
     } while (sigismember(&chkset, SIG_IPI));
 }
 
+static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
+{
+    MemoryRegionSection section;
+    void *addr;
+    RAMBlock *rb;
+    ram_addr_t offset;
+    int ret = -1;
+
+    section = memory_region_find(get_system_memory(), start, size);
+    if (!section.mr) {
+        return ret;
+    }
+
+    if (memory_region_can_be_private(section.mr)) {
+        if (to_private) {
+            ret = kvm_set_memory_attributes_private(start, size);
+        } else {
+            ret = kvm_set_memory_attributes_shared(start, size);
+        }
+
+        if (ret) {
+            return ret;
+        }
+
+        addr = memory_region_get_ram_ptr(section.mr) +
+               section.offset_within_region;
+        rb = qemu_ram_block_from_host(addr, false, &offset);
+        /*
+         * With KVM_SET_MEMORY_ATTRIBUTES by kvm_set_memory_attributes(),
+         * operation on underlying file descriptor is only for releasing
+         * unnecessary pages.
+         */
+        ram_block_convert_range(rb, offset, size, to_private);
+    } else {
+        warn_report("Convert non guest-memfd backed memory region (0x%"HWADDR_PRIx" ,+ 0x%"HWADDR_PRIx") to %s",
+                    start, size, to_private ? "private" : "shared");
+    }
+
+    memory_region_unref(section.mr);
+    return ret;
+}
+
 int kvm_cpu_exec(CPUState *cpu)
 {
     struct kvm_run *run = cpu->kvm_run;
@@ -3198,6 +3240,16 @@ int kvm_cpu_exec(CPUState *cpu)
                 break;
             }
             break;
+        case KVM_EXIT_MEMORY_FAULT:
+            if (run->memory.flags & ~KVM_MEMORY_EXIT_FLAG_PRIVATE) {
+                error_report("KVM_EXIT_MEMORY_FAULT: Unknown flag 0x%" PRIx64,
+                             (uint64_t)run->memory.flags);
+                ret = -1;
+                break;
+            }
+            ret = kvm_convert_memory(run->memory.gpa, run->memory.size,
+                                     run->memory.flags & KVM_MEMORY_EXIT_FLAG_PRIVATE);
+            break;
         default:
             DPRINTF("kvm_arch_handle_exit\n");
             ret = kvm_arch_handle_exit(cpu, run);
Signature · 1 line
2.34.1
Reviewed-by on v1 15/19
#38 lore raw

Re: [RFC PATCH 15/19] kvm: handle KVM_EXIT_MEMORY_FAULT

Quoted 41 lines · Xiaoyao Li

From: Chao Peng <[email protected]>

Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when KVM_EXIT_MEMORY_FAULT happens. It indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.

Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has gmem memory backend.

---
 accel/kvm/kvm-all.c | 52 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
accel/kvm/kvm-all.c +17 -0
index f9b5050b8885..72d50b923bf2 100644
@@ -3040,6 +3040,48 @@ static void kvm_eat_signals(CPUState *cpu)
     } while (sigismember(&chkset, SIG_IPI));
 }
 
+static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
+{
+    MemoryRegionSection section;
+    void *addr;
+    RAMBlock *rb;
+    ram_addr_t offset;
+    int ret = -1;
+
+    section = memory_region_find(get_system_memory(), start, size);
+    if (!section.mr) {
+        return ret;
+    }
+
+    if (memory_region_can_be_private(section.mr)) {
+        if (to_private) {
+            ret = kvm_set_memory_attributes_private(start, size);
+        } else {

accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 3069

Quoted 10 lines
accel/kvm/kvm-all.c +10 -0
+            ret = kvm_set_memory_attributes_shared(start, size);
+        }
+
+        if (ret) {
+            return ret;
+        }
+
+        addr = memory_region_get_ram_ptr(section.mr) +
+               section.offset_within_region;
+        rb = qemu_ram_block_from_host(addr, false, &offset);

Here we have already section. section.mr->ram_block. We don't have to scan the existing RAMBlocks.

Except that, looks good to me.

Signature · 1 line
Isaku Yamahata <[email protected]>
author
#39 lore raw

Re: [RFC PATCH 15/19] kvm: handle KVM_EXIT_MEMORY_FAULT

Quoted 49 lines · Isaku Yamahata
Quoted 46 lines · Xiaoyao Li

From: Chao Peng <[email protected]>

Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when KVM_EXIT_MEMORY_FAULT happens. It indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.

Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has gmem memory backend.

---
  accel/kvm/kvm-all.c | 52 +++++++++++++++++++++++++++++++++++++++++++++
  1 file changed, 52 insertions(+)
accel/kvm/kvm-all.c +22 -0
index f9b5050b8885..72d50b923bf2 100644
@@ -3040,6 +3040,48 @@ static void kvm_eat_signals(CPUState *cpu)
      } while (sigismember(&chkset, SIG_IPI));
  }
 
+static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
+{
+    MemoryRegionSection section;
+    void *addr;
+    RAMBlock *rb;
+    ram_addr_t offset;
+    int ret = -1;
+
+    section = memory_region_find(get_system_memory(), start, size);
+    if (!section.mr) {
+        return ret;
+    }
+
+    if (memory_region_can_be_private(section.mr)) {
+        if (to_private) {
+            ret = kvm_set_memory_attributes_private(start, size);
+        } else {
+            ret = kvm_set_memory_attributes_shared(start, size);
+        }
+
+        if (ret) {
+            return ret;
Quoted 8 lines

accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 3069

Quoted 5 lines
accel/kvm/kvm-all.c +5 -0
+        }
+
+        addr = memory_region_get_ram_ptr(section.mr) +
+               section.offset_within_region;
+        rb = qemu_ram_block_from_host(addr, false, &offset);

Here we have already section. section.mr->ram_block. We don't have to scan the existing RAMBlocks.

But we don't have the @offset, do we?

Quoted 2 lines

Except that, looks good to me.

#40 lore raw

Re: [RFC PATCH 15/19] kvm: handle KVM_EXIT_MEMORY_FAULT

In reply to Xiaoyao Li
Quoted 36 lines · Xiaoyao Li

From: Chao Peng <[email protected]>

Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when KVM_EXIT_MEMORY_FAULT happens. It indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.

Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has gmem memory backend.

---
 accel/kvm/kvm-all.c | 52 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
accel/kvm/kvm-all.c +12 -0
index f9b5050b8885..72d50b923bf2 100644
@@ -3040,6 +3040,48 @@ static void kvm_eat_signals(CPUState *cpu)
     } while (sigismember(&chkset, SIG_IPI));
 }
 
+static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
+{
+    MemoryRegionSection section;
+    void *addr;
+    RAMBlock *rb;
+    ram_addr_t offset;
+    int ret = -1;
+
+    section = memory_region_find(get_system_memory(), start, size);
+    if (!section.mr) {
+        return ret;
+    }

accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 3064

Quoted 10 lines
accel/kvm/kvm-all.c +10 -0
+
+    if (memory_region_can_be_private(section.mr)) {
+        if (to_private) {
+            ret = kvm_set_memory_attributes_private(start, size);
+        } else {
+            ret = kvm_set_memory_attributes_shared(start, size);
+        }
+
+        if (ret) {
+            return ret;

Should we unref the memory region before return?

Thanks,
Yilun

Quoted 19 lines
accel/kvm/kvm-all.c +19 -0
+        }
+
+        addr = memory_region_get_ram_ptr(section.mr) +
+               section.offset_within_region;
+        rb = qemu_ram_block_from_host(addr, false, &offset);
+        /*
+         * With KVM_SET_MEMORY_ATTRIBUTES by kvm_set_memory_attributes(),
+         * operation on underlying file descriptor is only for releasing
+         * unnecessary pages.
+         */
+        ram_block_convert_range(rb, offset, size, to_private);
+    } else {
+        warn_report("Convert non guest-memfd backed memory region (0x%"HWADDR_PRIx" ,+ 0x%"HWADDR_PRIx") to %s",
+                    start, size, to_private ? "private" : "shared");
+    }
+
+    memory_region_unref(section.mr);
+    return ret;
+}
author
#41 lore raw

Re: [RFC PATCH 15/19] kvm: handle KVM_EXIT_MEMORY_FAULT

Quoted 41 lines · Xu Yilun
Quoted 40 lines · Xiaoyao Li

From: Chao Peng <[email protected]>

Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when KVM_EXIT_MEMORY_FAULT happens. It indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.

Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has gmem memory backend.

---
  accel/kvm/kvm-all.c | 52 +++++++++++++++++++++++++++++++++++++++++++++
  1 file changed, 52 insertions(+)
accel/kvm/kvm-all.c +16 -0
index f9b5050b8885..72d50b923bf2 100644
@@ -3040,6 +3040,48 @@ static void kvm_eat_signals(CPUState *cpu)
      } while (sigismember(&chkset, SIG_IPI));
  }
 
+static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
+{
+    MemoryRegionSection section;
+    void *addr;
+    RAMBlock *rb;
+    ram_addr_t offset;
+    int ret = -1;
+
+    section = memory_region_find(get_system_memory(), start, size);
+    if (!section.mr) {
+        return ret;
+    }
+
+    if (memory_region_can_be_private(section.mr)) {
+        if (to_private) {
+            ret = kvm_set_memory_attributes_private(start, size);
Quoted 8 lines

accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 3064

Quoted 6 lines
accel/kvm/kvm-all.c +6 -0
+        } else {
+            ret = kvm_set_memory_attributes_shared(start, size);
+        }
+
+        if (ret) {
+            return ret;

Should we unref the memory region before return?

Thanks for catching this!

Quoted 22 lines

Thanks,
Yilun

Quoted 19 lines
accel/kvm/kvm-all.c +19 -0
+        }
+
+        addr = memory_region_get_ram_ptr(section.mr) +
+               section.offset_within_region;
+        rb = qemu_ram_block_from_host(addr, false, &offset);
+        /*
+         * With KVM_SET_MEMORY_ATTRIBUTES by kvm_set_memory_attributes(),
+         * operation on underlying file descriptor is only for releasing
+         * unnecessary pages.
+         */
+        ram_block_convert_range(rb, offset, size, to_private);
+    } else {
+        warn_report("Convert non guest-memfd backed memory region (0x%"HWADDR_PRIx" ,+ 0x%"HWADDR_PRIx") to %s",
+                    start, size, to_private ? "private" : "shared");
+    }
+
+    memory_region_unref(section.mr);
+    return ret;
+}
author
#42 lore raw

RFC PATCH 16/19 trace/kvm: Add trace for page convertion between shared and private

In reply to Xiaoyao Li

From: Isaku Yamahata <[email protected]>

---
 accel/kvm/kvm-all.c    | 1 +
 accel/kvm/trace-events | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
accel/kvm/kvm-all.c +1 -0
index 72d50b923bf2..c9f3aab5e587 100644
@@ -3048,6 +3048,7 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
     ram_addr_t offset;
     int ret = -1;
 
+    trace_kvm_convert_memory(start, size, to_private ? "shared_to_private" : "private_to_shared");
     section = memory_region_find(get_system_memory(), start, size);
     if (!section.mr) {
         return ret;
accel/kvm/trace-events +1 -1
index 80694683acea..4935c9c5cf0b 100644
@@ -25,4 +25,4 @@ kvm_dirty_ring_reaper(const char *s) "%s"
 kvm_dirty_ring_reap(uint64_t count, int64_t t) "reaped %"PRIu64" pages (took %"PRIi64" us)"
 kvm_dirty_ring_reaper_kick(const char *reason) "%s"
 kvm_dirty_ring_flush(int finished) "%d"
-
+kvm_convert_memory(uint64_t start, uint64_t size, const char *msg) "start 0x%" PRIx64 " size 0x%" PRIx64 " %s"
Signature · 1 line
2.34.1
author
#43 lore raw

RFC PATCH 17/19 pci-host/q35: Move PAM initialization above SMRAM initialization

In reply to Xiaoyao Li

From: Isaku Yamahata <[email protected]>

In mch_realize(), process PAM initialization before SMRAM initialization so that later patch can skill all the SMRAM related with a single check.

---
 hw/pci-host/q35.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)
hw/pci-host/q35.c +10 -9
index 91c46df9ae31..ac1518a94ee4 100644
@@ -575,6 +575,16 @@ static void mch_realize(PCIDevice *d, Error **errp)
     /* setup pci memory mapping */
     pc_pci_as_mapping_init(mch->system_memory, mch->pci_address_space);
 
+    /* PAM */
+    init_pam(&mch->pam_regions[0], OBJECT(mch), mch->ram_memory,
+             mch->system_memory, mch->pci_address_space,
+             PAM_BIOS_BASE, PAM_BIOS_SIZE);
+    for (i = 0; i < ARRAY_SIZE(mch->pam_regions) - 1; ++i) {
+        init_pam(&mch->pam_regions[i + 1], OBJECT(mch), mch->ram_memory,
+                 mch->system_memory, mch->pci_address_space,
+                 PAM_EXPAN_BASE + i * PAM_EXPAN_SIZE, PAM_EXPAN_SIZE);
+    }
+
     /* if *disabled* show SMRAM to all CPUs */
     memory_region_init_alias(&mch->smram_region, OBJECT(mch), "smram-region",
                              mch->pci_address_space, MCH_HOST_BRIDGE_SMRAM_C_BASE,
@@ -641,15 +651,6 @@ static void mch_realize(PCIDevice *d, Error **errp)
 
     object_property_add_const_link(qdev_get_machine(), "smram",
                                    OBJECT(&mch->smram));
-
-    init_pam(&mch->pam_regions[0], OBJECT(mch), mch->ram_memory,
-             mch->system_memory, mch->pci_address_space,
-             PAM_BIOS_BASE, PAM_BIOS_SIZE);
-    for (i = 0; i < ARRAY_SIZE(mch->pam_regions) - 1; ++i) {
-        init_pam(&mch->pam_regions[i + 1], OBJECT(mch), mch->ram_memory,
-                 mch->system_memory, mch->pci_address_space,
-                 PAM_EXPAN_BASE + i * PAM_EXPAN_SIZE, PAM_EXPAN_SIZE);
-    }
 }
 
 uint64_t mch_mcfg_base(void)
Signature · 1 line
2.34.1
author
#44 lore raw

RFC PATCH 18/19 q35: Introduce smm_ranges property for q35-pci-host

In reply to Xiaoyao Li

From: Isaku Yamahata <[email protected]>

Add a q35 property to check whether or not SMM ranges, e.g. SMRAM, TSEG, etc... exist for the target platform. TDX doesn't support SMM and doesn't play nice with QEMU modifying related guest memory ranges.

---
 hw/i386/pc_q35.c          |  3 ++-
 hw/pci-host/q35.c         | 42 +++++++++++++++++++++++++++------------
 include/hw/i386/pc.h      |  1 +
 include/hw/pci-host/q35.h |  1 +
 4 files changed, 33 insertions(+), 14 deletions(-)
hw/i386/pc_q35.c +2 -1
index dc27a9e223a2..73eb3bc5e826 100644
@@ -232,8 +232,9 @@ static void pc_q35_init(MachineState *machine)
                             x86ms->above_4g_mem_size, NULL);
     object_property_set_bool(phb, PCI_HOST_BYPASS_IOMMU,
                              pcms->default_bus_bypass_iommu, NULL);
+    object_property_set_bool(phb, PCI_HOST_PROP_SMM_RANGES,
+                             x86_machine_is_smm_enabled(x86ms), NULL);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(phb), &error_fatal);
-
     /* pci */
     host_bus = PCI_BUS(qdev_get_child_bus(DEVICE(phb), "pcie.0"));
     pcms->bus = host_bus;
hw/pci-host/q35.c +29 -13

82 lines not on this page · raw message

include/hw/i386/pc.h +1 -0
index c98d628a76f3..3c9ced22cea9 100644
@@ -156,6 +156,7 @@ void pc_guest_info_init(PCMachineState *pcms);
 #define PCI_HOST_PROP_PCI_HOLE64_SIZE  "pci-hole64-size"
 #define PCI_HOST_BELOW_4G_MEM_SIZE     "below-4g-mem-size"
 #define PCI_HOST_ABOVE_4G_MEM_SIZE     "above-4g-mem-size"
+#define PCI_HOST_PROP_SMM_RANGES       "smm-ranges"
 
 
 void pc_pci_as_mapping_init(MemoryRegion *system_memory,
include/hw/pci-host/q35.h +1 -0
index 1d98bbfe0dd2..930af3870e8a 100644
@@ -50,6 +50,7 @@ struct MCHPCIState {
     MemoryRegion tseg_blackhole, tseg_window;
     MemoryRegion smbase_blackhole, smbase_window;
     bool has_smram_at_smbase;
+    bool has_smm_ranges;
     Range pci_hole;
     uint64_t below_4g_mem_size;
     uint64_t above_4g_mem_size;
Signature · 1 line
2.34.1
author
#45 lore raw

RFC PATCH 19/19 i386: Disable SMM mode for X86_SW_PROTECTED_VM

In reply to Xiaoyao Li
---
 target/i386/kvm/kvm.c | 7 +++++++
 1 file changed, 7 insertions(+)
target/i386/kvm/kvm.c +7 -0
index a96640512dbc..62f237068a3a 100644
@@ -2654,6 +2654,13 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
 
     if (x86ms->vm_type == KVM_X86_SW_PROTECTED_VM) {
         memory_listener_register(&kvm_x86_sw_protected_vm_memory_listener, &address_space_memory);
+
+        if (x86ms->smm == ON_OFF_AUTO_AUTO) {
+            x86ms->smm = ON_OFF_AUTO_OFF;
+        } else if (x86ms->smm == ON_OFF_AUTO_ON) {
+            error_report("X86_SW_PROTECTED_VM doesn't support SMM");
+            return -EINVAL;
+        }
     }
 
     if (!kvm_check_extension(s, KVM_CAP_IRQ_ROUTING)) {
Signature · 1 line
2.34.1
#46 lore raw

Re: [RFC PATCH 19/19] i386: Disable SMM mode for X86_SW_PROTECTED_VM

Quoted 12 lines · Xiaoyao Li
---
 target/i386/kvm/kvm.c | 7 +++++++
 1 file changed, 7 insertions(+)
target/i386/kvm/kvm.c +0 -0
index a96640512dbc..62f237068a3a 100644
@@ -2654,6 +2654,13 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
 
     if (x86ms->vm_type == KVM_X86_SW_PROTECTED_VM) {

target/i386/kvm/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 2665

Quoted 10 lines
target/i386/kvm/kvm.c +7 -0
         memory_listener_register(&kvm_x86_sw_protected_vm_memory_listener, &address_space_memory);
+
+        if (x86ms->smm == ON_OFF_AUTO_AUTO) {
+            x86ms->smm = ON_OFF_AUTO_OFF;
+        } else if (x86ms->smm == ON_OFF_AUTO_ON) {
+            error_report("X86_SW_PROTECTED_VM doesn't support SMM");
+            return -EINVAL;
+        }
     }
 

If we use confidential guest support, this check should go to there.

Signature · 1 line
Isaku Yamahata <[email protected]>
#47 lore raw
In reply to Xiaoyao Li
Quoted 34 lines · Xiaoyao Li

This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of KVM_X86_PROTECTED_VM.

It adds the support to create a specific KVM_X86_PROTECTED_VM type VM, and introduces 'private' property for memory backend. When the vm type is KVM_X86_PROTECTED_VM and memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.

    $qemu -object memory-backend-ram,id=mem0,size=1G,private=on \
          -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \
	  ...

Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.

There are following design opens that need to be discussed:

1. how to determine the vm type?

a. like this series, specify the vm type via machine property

      'kvm-type'

b. check the memory backend, if any backend has 'private' property
set, the vm-type is set to KVM_X86_PROTECTED_VM.

2. whether 'private' property is needed if we choose 1.b as design

with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.

3. What is KVM_X86_SW_PROTECTED_VM going to look like? What's the purose of it and what's the requirement on it. I think it's the questions for KVM folks than QEMU folks.

Quoted 6 lines

Any other idea/open/question is welcomed.

Beside, TDX QEMU implemetation is based on this series to provide private gmem for TD private memory, which can be found at [2]. And it can work corresponding KVM [3] to boot TDX guest.

We already have a general purpose configuration mechanism for confidential guests. The -machine argument has a property confidential-guest-support=$OBJECT-ID, for pointing to an object that implements the TYPE_CONFIDENTIAL_GUEST_SUPPORT interface in QEMU. This is implemented with SEV, PPC PEF mode, and s390 protvirt.

I would expect TDX to follow this same design ie

    qemu-system-x86_64 \
      -object tdx-guest,id=tdx0,..... \
      -machine q35,confidential-guest-support=tdx0 \
      ...

and not require inventing the new 'kvm-type' attribute at least.

For the memory backend though, I'm not so sure - possibly that might be something that still wants an extra property to identify the type of memory to allocate, since we use memory-backend-ram for a variety of use cases. Or it could be an entirely new object type such as "memory-backend-gmem"

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#48 lore raw
Quoted 49 lines · Daniel P. Berrangé
Quoted 34 lines · Xiaoyao Li

This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of KVM_X86_PROTECTED_VM.

It adds the support to create a specific KVM_X86_PROTECTED_VM type VM, and introduces 'private' property for memory backend. When the vm type is KVM_X86_PROTECTED_VM and memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.

     $qemu -object memory-backend-ram,id=mem0,size=1G,private=on \
           -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \
	  ...

Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.

There are following design opens that need to be discussed:

1. how to determine the vm type?

a. like this series, specify the vm type via machine property
'kvm-type'
b. check the memory backend, if any backend has 'private' property
set, the vm-type is set to KVM_X86_PROTECTED_VM.

2. whether 'private' property is needed if we choose 1.b as design

with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.

3. What is KVM_X86_SW_PROTECTED_VM going to look like? What's the purose of it and what's the requirement on it. I think it's the questions for KVM folks than QEMU folks.

Quoted 6 lines

Any other idea/open/question is welcomed.

Beside, TDX QEMU implemetation is based on this series to provide private gmem for TD private memory, which can be found at [2]. And it can work corresponding KVM [3] to boot TDX guest.

We already have a general purpose configuration mechanism for confidential guests. The -machine argument has a property confidential-guest-support=$OBJECT-ID, for pointing to an object that implements the TYPE_CONFIDENTIAL_GUEST_SUPPORT interface in QEMU. This is implemented with SEV, PPC PEF mode, and s390 protvirt.

Quoted 8 lines

I would expect TDX to follow this same design ie

     qemu-system-x86_64 \
       -object tdx-guest,id=tdx0,..... \
       -machine q35,confidential-guest-support=tdx0 \
       ...

and not require inventing the new 'kvm-type' attribute at least.

yes.

TDX is initialized exactly as the above.

This RFC series introduces the 'kvm-type' for KVM_X86_SW_PROTECTED_VM. It's my fault that forgot to list the option of introducing sw_protected_vm object with CONFIDENTIAL_GUEST_SUPPORT interface. Thanks for Isaku to raise it https://lore.kernel.org/qemu-devel/[email protected]/

we can specify KVM_X86_SW_PROTECTED_VM this way:

qemu \
-object sw-protected,id=swp0,... \
-machine confidential-guest-support=swp0 \
...

Quoted 5 lines

For the memory backend though, I'm not so sure - possibly that might be something that still wants an extra property to identify the type of memory to allocate, since we use memory-backend-ram for a variety of use cases. Or it could be an entirely new object type such as "memory-backend-gmem"

What I want to discuss is whether providing the interface to users to allow them configuring which memory is/can be private. For example, QEMU can do it internally. If users wants a confidential guest, QEMU allocates private gmem for normal RAM automatically.

#49 lore raw
Quoted 81 lines · Xiaoyao Li
Quoted 49 lines · Daniel P. Berrangé
Quoted 34 lines · Xiaoyao Li

This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of KVM_X86_PROTECTED_VM.

It adds the support to create a specific KVM_X86_PROTECTED_VM type VM, and introduces 'private' property for memory backend. When the vm type is KVM_X86_PROTECTED_VM and memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.

     $qemu -object memory-backend-ram,id=mem0,size=1G,private=on \
           -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \
	  ...

Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.

There are following design opens that need to be discussed:

1. how to determine the vm type?

a. like this series, specify the vm type via machine property
'kvm-type'
b. check the memory backend, if any backend has 'private' property
set, the vm-type is set to KVM_X86_PROTECTED_VM.

2. whether 'private' property is needed if we choose 1.b as design

with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.

3. What is KVM_X86_SW_PROTECTED_VM going to look like? What's the purose of it and what's the requirement on it. I think it's the questions for KVM folks than QEMU folks.

Quoted 6 lines

Any other idea/open/question is welcomed.

Beside, TDX QEMU implemetation is based on this series to provide private gmem for TD private memory, which can be found at [2]. And it can work corresponding KVM [3] to boot TDX guest.

We already have a general purpose configuration mechanism for confidential guests. The -machine argument has a property confidential-guest-support=$OBJECT-ID, for pointing to an object that implements the TYPE_CONFIDENTIAL_GUEST_SUPPORT interface in QEMU. This is implemented with SEV, PPC PEF mode, and s390 protvirt.

Quoted 8 lines

I would expect TDX to follow this same design ie

     qemu-system-x86_64 \
       -object tdx-guest,id=tdx0,..... \
       -machine q35,confidential-guest-support=tdx0 \
       ...

and not require inventing the new 'kvm-type' attribute at least.

yes.

TDX is initialized exactly as the above.

This RFC series introduces the 'kvm-type' for KVM_X86_SW_PROTECTED_VM. It's
my fault that forgot to list the option of introducing sw_protected_vm
object with CONFIDENTIAL_GUEST_SUPPORT interface.
Thanks for Isaku to raise it https://lore.kernel.org/qemu-devel/[email protected]/

we can specify KVM_X86_SW_PROTECTED_VM this way:

qemu \
-object sw-protected,id=swp0,... \
-machine confidential-guest-support=swp0 \
...

Quoted 5 lines

For the memory backend though, I'm not so sure - possibly that might be something that still wants an extra property to identify the type of memory to allocate, since we use memory-backend-ram for a variety of use cases. Or it could be an entirely new object type such as "memory-backend-gmem"

Quoted 4 lines

What I want to discuss is whether providing the interface to users to allow them configuring which memory is/can be private. For example, QEMU can do it internally. If users wants a confidential guest, QEMU allocates private gmem for normal RAM automatically.

I think handling it automatically simplifies things a good deal on the QEMU side. I think it's still worthwhile to still allow:

-object memory-backend-memfd-private,...

because it provides a nice mechanism to set up a pair of shared/private memfd's to enable hole-punching via fallocate() to avoid doubling memory allocations for shared/private. It's also a nice place to control potentially-configurable things like:

- whether or not to enable discard/hole-punching
- if discard is enabled, whether or not to register the range via RamDiscardManager interface so that VFIO/IOMMU mappings get updated when doing PCI passthrough. SNP relies on this for PCI passthrough when discard is enabled, otherwise DMA occurs to stale mappings of discarded bounce-buffer pages:

https://github.com/AMDESE/qemu/blob/snp-latest/backends/hostmem-memfd-private.c#L449

But for other memory ranges, it doesn't do a lot of good to rely on users to control those via -object memory-backend-memfd-private, since QEMU will set up some regions internally, like the UEFI ROM.

It also isn't ideal for QEMU itself to internally control what should/shouldn't be set up with a backing guest_memfd, because some guest kernels do weird stuff, like scan for ROM regions in areas that guest kernels might have mapped as encrypted in guest page table. You can consider them to be guest bugs, but even current SNP-capable kernels exhibit this behavior and if the guest wants to do dumb stuff QEMU should let it.

But for these latter 2 cases, it doesn't make sense to attempt to do any sort of discarding of backing pages since it doesn't make sense to discard ROM pages.

So I think it makes sense to just set up the gmemfd automatically across the board internally, and keep memory-backend-memfd-private around purely as a way to control/configure discardable memory.

-Mike

Quoted 2 lines
#50 lore raw
Quoted 118 lines · Michael Roth via
Quoted 81 lines · Xiaoyao Li
Quoted 49 lines · Daniel P. Berrangé
Quoted 34 lines · Xiaoyao Li

This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of KVM_X86_PROTECTED_VM.

It adds the support to create a specific KVM_X86_PROTECTED_VM type VM, and introduces 'private' property for memory backend. When the vm type is KVM_X86_PROTECTED_VM and memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.

     $qemu -object memory-backend-ram,id=mem0,size=1G,private=on \
           -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \
	  ...

Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.

There are following design opens that need to be discussed:

1. how to determine the vm type?

a. like this series, specify the vm type via machine property
'kvm-type'
b. check the memory backend, if any backend has 'private' property
set, the vm-type is set to KVM_X86_PROTECTED_VM.

2. whether 'private' property is needed if we choose 1.b as design

with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.

3. What is KVM_X86_SW_PROTECTED_VM going to look like? What's the purose of it and what's the requirement on it. I think it's the questions for KVM folks than QEMU folks.

Quoted 6 lines

Any other idea/open/question is welcomed.

Beside, TDX QEMU implemetation is based on this series to provide private gmem for TD private memory, which can be found at [2]. And it can work corresponding KVM [3] to boot TDX guest.

We already have a general purpose configuration mechanism for confidential guests. The -machine argument has a property confidential-guest-support=$OBJECT-ID, for pointing to an object that implements the TYPE_CONFIDENTIAL_GUEST_SUPPORT interface in QEMU. This is implemented with SEV, PPC PEF mode, and s390 protvirt.

Quoted 8 lines

I would expect TDX to follow this same design ie

     qemu-system-x86_64 \
       -object tdx-guest,id=tdx0,..... \
       -machine q35,confidential-guest-support=tdx0 \
       ...

and not require inventing the new 'kvm-type' attribute at least.

yes.

TDX is initialized exactly as the above.

This RFC series introduces the 'kvm-type' for KVM_X86_SW_PROTECTED_VM. It's
my fault that forgot to list the option of introducing sw_protected_vm
object with CONFIDENTIAL_GUEST_SUPPORT interface.
Thanks for Isaku to raise it https://lore.kernel.org/qemu-devel/[email protected]/

we can specify KVM_X86_SW_PROTECTED_VM this way:

qemu \
-object sw-protected,id=swp0,... \
-machine confidential-guest-support=swp0 \
...

Quoted 5 lines

For the memory backend though, I'm not so sure - possibly that might be something that still wants an extra property to identify the type of memory to allocate, since we use memory-backend-ram for a variety of use cases. Or it could be an entirely new object type such as "memory-backend-gmem"

Quoted 4 lines

What I want to discuss is whether providing the interface to users to allow them configuring which memory is/can be private. For example, QEMU can do it internally. If users wants a confidential guest, QEMU allocates private gmem for normal RAM automatically.

I think handling it automatically simplifies things a good deal on the QEMU side. I think it's still worthwhile to still allow:

-object memory-backend-memfd-private,...

because it provides a nice mechanism to set up a pair of shared/private memfd's to enable hole-punching via fallocate() to avoid doubling memory allocations for shared/private. It's also a nice place to control potentially-configurable things like:

- whether or not to enable discard/hole-punching
- if discard is enabled, whether or not to register the range via RamDiscardManager interface so that VFIO/IOMMU mappings get updated when doing PCI passthrough. SNP relies on this for PCI passthrough when discard is enabled, otherwise DMA occurs to stale mappings of discarded bounce-buffer pages:

https://github.com/AMDESE/qemu/blob/snp-latest/backends/hostmem-memfd-private.c#L449

But for other memory ranges, it doesn't do a lot of good to rely on users to control those via -object memory-backend-memfd-private, since QEMU will set up some regions internally, like the UEFI ROM.

It also isn't ideal for QEMU itself to internally control what should/shouldn't be set up with a backing guest_memfd, because some guest kernels do weird stuff, like scan for ROM regions in areas that guest kernels might have mapped as encrypted in guest page table. You can consider them to be guest bugs, but even current SNP-capable kernels exhibit this behavior and if the guest wants to do dumb stuff QEMU should let it.

Quoted 7 lines

But for these latter 2 cases, it doesn't make sense to attempt to do any sort of discarding of backing pages since it doesn't make sense to discard ROM pages.

So I think it makes sense to just set up the gmemfd automatically across the board internally, and keep memory-backend-memfd-private around purely as a way to control/configure discardable memory.

I'm looking at the repo and
31a7c7e36684 ("*hostmem-memfd-private: Initial discard manager support")

Do we have to implement RAM_DISCARD_MANGER at memory-backend-memfd-private? Can't we implement it at host_mem? The interface callbacks can have check "if (!private) return". Then we can support any host-mem backend.

Signature · 1 line
Isaku Yamahata <[email protected]>
#51 lore raw
In reply to Xiaoyao Li
Quoted 16 lines · Xiaoyao Li

This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of KVM_X86_PROTECTED_VM.

It adds the support to create a specific KVM_X86_PROTECTED_VM type VM, and introduces 'private' property for memory backend. When the vm type is KVM_X86_PROTECTED_VM and memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.

    $qemu -object memory-backend-ram,id=mem0,size=1G,private=on \
          -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \
	  ...

Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.

Quoted 8 lines

There are following design opens that need to be discussed:

1. how to determine the vm type?

a. like this series, specify the vm type via machine property

      'kvm-type'

b. check the memory backend, if any backend has 'private' property
set, the vm-type is set to KVM_X86_PROTECTED_VM.

Hi Xiaoyao. Because qemu has already confidential guest support, we should utilize it. Say,
qemu \
-object sw-protected, id=swp0, <more options for KVM_X86_SW_PROTECTED_VM> \
-machine confidential-guest-support=swp0

Quoted 4 lines

2. whether 'private' property is needed if we choose 1.b as design

with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.

Memory region property (how to create KVM memory slot) should be independent from underlying VM type. Some (e.g. TDX) may require KVM private memory slot, some may not. Leave the decision to its vm type backend. They can use qemu memory listener.

Signature · 1 line
Isaku Yamahata <[email protected]>
author
#52 lore raw
Quoted 28 lines · Isaku Yamahata
Quoted 16 lines · Xiaoyao Li

This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of KVM_X86_PROTECTED_VM.

It adds the support to create a specific KVM_X86_PROTECTED_VM type VM, and introduces 'private' property for memory backend. When the vm type is KVM_X86_PROTECTED_VM and memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.

     $qemu -object memory-backend-ram,id=mem0,size=1G,private=on \
           -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \
	  ...

Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.

Quoted 8 lines

There are following design opens that need to be discussed:

1. how to determine the vm type?

a. like this series, specify the vm type via machine property
'kvm-type'
b. check the memory backend, if any backend has 'private' property
set, the vm-type is set to KVM_X86_PROTECTED_VM.

Quoted 5 lines

Hi Xiaoyao. Because qemu has already confidential guest support, we should utilize it. Say,

qemu  \
   -object sw-protected, id=swp0, <more options for KVM_X86_SW_PROTECTED_VM> \
   -machine confidential-guest-support=swp0

thanks for pointing out this option. I thought of it and forgot to list it as option.

It seems better and I'll go this direction if no one has different opinion.

Quoted 11 lines
Quoted 5 lines

2. whether 'private' property is needed if we choose 1.b as design

with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.

Memory region property (how to create KVM memory slot) should be independent from underlying VM type. Some (e.g. TDX) may require KVM private memory slot, some may not. Leave the decision to its vm type backend. They can use qemu memory listener.

As I replied to Daniel, the topic is whether 'private' property is needed. Is it essential to let users decide which memory can be private? It seems OK that QEMU can make the decision based on VM type.