QEMU gmem implemention
cover QEMU gmem implemention
This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of KVM_X86_PROTECTED_VM.
It adds the support to create a specific KVM_X86_PROTECTED_VM type VM, and introduces 'private' property for memory backend. When the vm type is KVM_X86_PROTECTED_VM and memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.
$qemu -object memory-backend-ram,id=mem0,size=1G,private=on \
-machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \
...Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.
There are following design opens that need to be discussed:
1. how to determine the vm type?
a. like this series, specify the vm type via machine property
'kvm-type'
b. check the memory backend, if any backend has 'private' property
set, the vm-type is set to KVM_X86_PROTECTED_VM.
2. whether 'private' property is needed if we choose 1.b as design
with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.
3. What is KVM_X86_SW_PROTECTED_VM going to look like? What's the purose of it and what's the requirement on it. I think it's the questions for KVM folks than QEMU folks.
Any other idea/open/question is welcomed.
Beside, TDX QEMU implemetation is based on this series to provide private gmem for TD private memory, which can be found at [2]. And it can work corresponding KVM [3] to boot TDX guest.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://github.com/intel/qemu-tdx/tree/tdx-upstream-wip
[3] https://github.com/intel/tdx/tree/kvm-upstream-2023.07.27-v6.5-rc2-workaround
Chao Peng (4):
RAMBlock: Support KVM gmemory
kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
physmem: Add ram_block_convert_range
kvm: handle KVM_EXIT_MEMORY_FAULT
Isaku Yamahata (4):
HostMem: Add private property to indicate to use kvm gmem trace/kvm: Add trace for page convertion between shared and private pci-host/q35: Move PAM initialization above SMRAM initialization
q35: Introduce smm_ranges property for q35-pci-host
Xiaoyao Li (11):
trace/kvm: Split address space and slot id in
trace_kvm_set_user_memory()
*** HACK *** linux-headers: Update headers to pull in gmem APIs
memory: Introduce memory_region_can_be_private() i386/pc: Drop pc_machine_kvm_type()
target/i386: Implement mc->kvm_type() to get VM type i386/kvm: Create gmem fd for KVM_X86_SW_PROTECTED_VM
kvm: Introduce support for memory_attributes
kvm/memory: Introduce the infrastructure to set the default
shared/private value
i386/kvm: Set memory to default private for KVM_X86_SW_PROTECTED_VM
physmem: replace function name with __func__ in
ram_block_discard_range()
i386: Disable SMM mode for X86_SW_PROTECTED_VM
accel/kvm/kvm-all.c | 166 +++++++++++++++++++++++++++++++++--- accel/kvm/trace-events | 4 +- backends/hostmem.c | 18 ++++ hw/i386/pc.c | 5 -- hw/i386/pc_q35.c | 3 +- hw/i386/x86.c | 27 ++++++ hw/pci-host/q35.c | 61 ++++++++----- include/exec/cpu-common.h | 2 + include/exec/memory.h | 24 ++++++ include/exec/ramblock.h | 1 + include/hw/i386/pc.h | 4 +- include/hw/i386/x86.h | 4 + include/hw/pci-host/q35.h | 1 + include/sysemu/hostmem.h | 2 +- include/sysemu/kvm.h | 3 + include/sysemu/kvm_int.h | 2 + linux-headers/asm-x86/kvm.h | 3 + linux-headers/linux/kvm.h | 50 +++++++++++ qapi/qom.json | 4 + softmmu/memory.c | 27 ++++++ softmmu/physmem.c | 97 ++++++++++++++------- target/i386/kvm/kvm.c | 84 ++++++++++++++++++ target/i386/kvm/kvm_i386.h | 1 + 23 files changed, 517 insertions(+), 76 deletions(-)
Signature · 1 line
2.34.1
RFC PATCH 1/19 trace/kvm: Split address space and slot id in trace_kvm_set_user_memory()
The upper 16 bits of kvm_userspace_memory_region::slot are address space id. Parse it separately in trace_kvm_set_user_memory().
--- accel/kvm/kvm-all.c | 5 +++-- accel/kvm/trace-events | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-)
accel/kvm/kvm-all.c +3 -2
@@ -309,8 +309,9 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); slot->old_flags = mem.flags; err: -trace_kvm_set_user_memory (mem.slot, mem.flags, mem.guest_phys_addr, - mem.memory_size, mem.userspace_addr, ret); +trace_kvm_set_user_memory (mem.slot >>16 , (uint16_t )mem.slot, mem.flags, + mem.guest_phys_addr, mem.memory_size, + mem.userspace_addr, ret);if (ret <0 ) { error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
accel/kvm/trace-events +1 -1
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d" kvm_irqchip_release_virq(int virq) "virq %d" kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d" kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d" -kvm_set_user_memory(uint32_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" +kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32 kvm_resample_fd_notify(int gsi) "gsi %d" kvm_dirty_ring_full(int id) "vcpu %d"
Signature · 1 line
2.34.1
RFC PATCH 2/19 *** HACK *** linux-headers: Update headers to pull in gmem APIs
In reply to Xiaoyao LiThis patch needs to be updated by script
scripts/update-linux-headers.sh
once gmem fd support is upstreamed in Linux kernel.
--- linux-headers/asm-x86/kvm.h | 3 +++ linux-headers/linux/kvm.h | 50 +++++++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+)
linux-headers/asm-x86/kvm.h +3 -0
@@ -560,4 +560,7 @@ struct kvm_pmu_event_filter {/* x86-specific KVM_EXIT_HYPERCALL flags. */ #define KVM_EXIT_HYPERCALL_LONG_MODE BIT(0) +#define KVM_X86_DEFAULT_VM 0 +#define KVM_X86_SW_PROTECTED_VM 1 +#endif /* _ASM_X86_KVM_H */
linux-headers/linux/kvm.h +50 -0
90 lines not on this page · raw message
Signature · 1 line
2.34.1
RFC PATCH 3/19 RAMBlock: Support KVM gmemory
In reply to Xiaoyao LiFrom: Chao Peng <[email protected]>
Add KVM gmem support to RAMBlock so we can have both normal hva based memory and gmem fd based memory in one RAMBlock.
The gmem part is represented by the gmem_fd.
--- include/exec/memory.h | 8 ++++++++ include/exec/ramblock.h | 1 + softmmu/memory.c | 9 +++++++++ softmmu/physmem.c | 2 ++ 4 files changed, 20 insertions(+)
include/exec/memory.h +8 -0
@@ -1376,6 +1376,14 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr,int fd, ram_addr_t offset, Error **errp); +/** +* memory_region_set_gmem_fd: Set RAM memory region with a restricted fd. +* +* @mr: the #MemoryRegion to be set. +* @fd: the fd to provide restricted memory. +*/ +void memory_region_set_gmem_fd (MemoryRegion *mr,int fd); +#endif /**
include/exec/ramblock.h +1 -0
@@ -41,6 +41,7 @@ struct RAMBlock { QLIST_HEAD(, RAMBlockNotifier) ramblock_notifiers;int fd;uint64_t fd_offset; +int gmem_fd;size_t page_size;/* dirty bitmap used during migration */ unsigned long *bmap;
softmmu/memory.c +9 -0
@@ -1661,6 +1661,15 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr, error_propagate(errp, err); } } + +void memory_region_set_gmem_fd (MemoryRegion *mr,int fd) +{ +if (mr->ram_block) { +assert (fd >=0 ); + mr->ram_block->gmem_fd = fd; + } +} +#endif void memory_region_init_ram_ptr(MemoryRegion *mr,
softmmu/physmem.c +2 -0
@@ -1920,6 +1920,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, new_block->used_length = size; new_block->max_length = size; new_block->flags = ram_flags; + new_block->gmem_fd =-1 ; new_block->host =file_ram_alloc (new_block, size, fd, readonly, !file_size, offset, errp);if (!new_block->host) {
@@ -1990,6 +1991,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size, new_block->max_length = max_size;assert (max_size >= size); new_block->fd =-1 ; + new_block->gmem_fd =-1 ; new_block->page_size =qemu_real_host_page_size (); new_block->host = host; new_block->flags = ram_flags;
Signature · 1 line
2.34.1
Re: [RFC PATCH 03/19] RAMBlock: Support KVM gmemory
Quoted 9 lines · Xiaoyao Li
From: Chao Peng <[email protected]>
Add KVM gmem support to RAMBlock so we can have both normal hva based memory and gmem fd based memory in one RAMBlock.
The gmem part is represented by the
gmem_fd.Signed-off-by Chao Peng <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>
So, *someone* creates a RAMBlock in QEMU.
Who'll squeeze a gmem_fd in there? When? How?
Shouldn't we create the RAM memory region / RAMBlock already with the gmem_fd, and set that when initializing these things?
Signature · 3 lines
Cheers, David / dhildenb
RFC PATCH 4/19 memory: Introduce memory_region_can_be_private()
In reply to Xiaoyao Li--- include/exec/memory.h | 9 +++++++++ softmmu/memory.c | 5 +++++ 2 files changed, 14 insertions(+)
include/exec/memory.h +9 -0
@@ -1679,6 +1679,15 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)*/ bool memory_region_is_protected (MemoryRegion *mr); +/** +* memory_region_can_be_private: check whether a memory region can be private +* +* Returns %true if a memory region's ram_block has valid gmem fd assigned. +* +* @mr: the memory region being queried +*/ +bool memory_region_can_be_private (MemoryRegion *mr); +/** * memory_region_get_iommu: check whether a memory region is an iommu *
softmmu/memory.c +5 -0
@@ -1855,6 +1855,11 @@ bool memory_region_is_protected(MemoryRegion *mr)return mr->ram && (mr->ram_block->flags &RAM_PROTECTED ); } +bool memory_region_can_be_private (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->gmem_fd >=0 ; +} +uint8_t memory_region_get_dirty_log_mask (MemoryRegion *mr) {uint8_t mask = mr->dirty_log_mask;
Signature · 1 line
2.34.1
Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()
softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861
Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4
-0+bool memory_region_can_be_private (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->gmem_fd >=0 ; +}
This is not really MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?
I worry it can be very confusing in the future against MAP_PRIVATE / MAP_SHARED otherwise.
Thanks,
Signature · 1 line
Peter Xu
Sean Christopherson
reviewerRe: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()
Quoted 8 lines · Peter Xu
softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861
Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4
-0+bool memory_region_can_be_private (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->gmem_fd >=0 ; +}This is not really
MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?
Yes and yes.
Quoted 2 lines
I worry it can be very confusing in the future against
MAP_PRIVATE/MAP_SHAREDotherwise.
Heh, it's already quite confusing at times. I'm definitely open to naming that doesn't collide with MAP_{PRIVATE,SHARED}, especially if someone can come with a naming scheme that includes a succinct way to describe memory that is shared between two or more VMs, but is accessible to _only_ those VMs.
Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()
Quoted 12 lines · Sean Christopherson
Quoted 8 lines · Peter Xu
softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861
Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4
-0+bool memory_region_can_be_private (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->gmem_fd >=0 ; +}This is not really
MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?Yes and yes.
Quoted 7 lines
Quoted 2 lines
I worry it can be very confusing in the future against
MAP_PRIVATE/MAP_SHAREDotherwise.Heh, it's already quite confusing at times. I'm definitely open to naming that doesn't collide with MAP_{PRIVATE,SHARED}, especially if someone can come with a naming scheme that includes a succinct way to describe memory that is shared between two or more VMs, but is accessible to _only_ those VMs.
Standard solution is a technology specific prefix. protect_shared, encrypt_shared etc.
Signature · 1 line
MST
Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()
Quoted 16 lines · Michael S. Tsirkin
Quoted 15 lines · Sean Christopherson
Quoted 8 lines · Peter Xu
softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861
Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4
-0+bool memory_region_can_be_private (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->gmem_fd >=0 ; +}This is not really
MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?Yes and yes.
Quoted 2 lines
I worry it can be very confusing in the future against
MAP_PRIVATE/MAP_SHAREDotherwise.
Quoted 7 lines
Quoted 4 lines
Heh, it's already quite confusing at times. I'm definitely open to naming that doesn't collide with MAP_{PRIVATE,SHARED}, especially if someone can come with a naming scheme that includes a succinct way to describe memory that is shared between two or more VMs, but is accessible to _only_ those VMs.
Standard solution is a technology specific prefix.
protect_shared,encrypt_sharedetc.
Agreed, a prefix could definitely help (if nothing better comes at last..). If e.g. "encrypted" too long to be applied everywhere in var names and functions, maybe it can also be "enc_{private|shared}".
Thanks,
Signature · 1 line
Peter Xu
Sean Christopherson
reviewerRe: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()
Quoted 22 lines · Peter Xu
Quoted 21 lines · Michael S. Tsirkin
Quoted 19 lines · Sean Christopherson
Quoted 8 lines · Peter Xu
softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861
Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4
-0+bool memory_region_can_be_private (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->gmem_fd >=0 ; +}This is not really
MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?Yes and yes.
Quoted 2 lines
I worry it can be very confusing in the future against
MAP_PRIVATE/MAP_SHAREDotherwise.Heh, it's already quite confusing at times. I'm definitely open to naming that doesn't collide with MAP_{PRIVATE,SHARED}, especially if someone can come with a naming scheme that includes a succinct way to describe memory that is shared between two or more VMs, but is accessible to _only_ those VMs.
Quoted 6 lines
Quoted 2 lines
Standard solution is a technology specific prefix.
protect_shared,encrypt_sharedetc.Agreed, a prefix could definitely help (if nothing better comes at last..). If e.g. "encrypted" too long to be applied everywhere in var names and functions, maybe it can also be "enc_{private|shared}".
FWIW, I would stay away from "encrypted", there is no requirement that the memory actually be encrypted.
Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()
In reply to Peter XuQuoted 9 lines · Peter Xu
softmmu/memory.c bool memory_region_is_protected(MemoryRegion *mr) · line 1861
Quoted 4 lines · Xiaoyao Li
softmmu/memory.c +4
-0+bool memory_region_can_be_private (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->gmem_fd >=0 ; +}This is not really
MAP_PRIVATE, am I right? If so, is there still chance we rename it (it seems to be also in the kernel proposal all across..)?
Quoted 4 lines
I worry it can be very confusing in the future against
MAP_PRIVATE/MAP_SHAREDotherwise.Thanks,
It is - kernel calls this "map shared" but unfortunately qemu calls this just "shared". Maybe e.g. "protect_shared" would be a better name for qemu?
Signature · 1 line
MST
Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()
In reply to Xiaoyao LiQuoted 11 lines · Xiaoyao Li
Signed-off-by Xiaoyao Li <[email protected]>--- include/exec/memory.h | 9 +++++++++ softmmu/memory.c | 5 +++++ 2 files changed, 14 insertions(+)include/exec/memory.h +0
-0
include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1683
Quoted 5 lines
include/exec/memory.h +2
-0@@ -1679,6 +1679,15 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)*/ bool memory_region_is_protected (MemoryRegion *mr); +/** +* memory_region_can_be_private: check whether a memory region can be private
The name of the function is not particularly informative,
include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1685
Quoted 2 lines
include/exec/memory.h +2
-0+* +* Returns %true if a memory region's ram_block has valid gmem fd assigned.
but in your comment you describe more accurately what it does, why not make it the function name?
bool memory_region_has_valid_gmem_fd()
include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1689
Quoted 4 lines
include/exec/memory.h +4
-0+* +* @mr: the memory region being queried +*/ +bool memory_region_can_be_private (MemoryRegion *mr);
bool memory_region_has_valid_gmem_fd()
Thanks,
C
Quoted 20 lines
include/exec/memory.h +1
-0+/** * memory_region_get_iommu: check whether a memory region is an iommu * softmmu/memory.c +5
-0@@ -1855,6 +1855,11 @@ bool memory_region_is_protected(MemoryRegion *mr)return mr->ram && (mr->ram_block->flags &RAM_PROTECTED ); } +bool memory_region_can_be_private (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->gmem_fd >=0 ; +} +uint8_t memory_region_get_dirty_log_mask (MemoryRegion *mr) {uint8_t mask = mr->dirty_log_mask;
Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()
Quoted 20 lines · Claudio Fontana
Quoted 11 lines · Xiaoyao Li
Signed-off-by Xiaoyao Li <[email protected]>--- include/exec/memory.h | 9 +++++++++ softmmu/memory.c | 5 +++++ 2 files changed, 14 insertions(+)include/exec/memory.h +0
-0include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1683
Quoted 5 lines
include/exec/memory.h +2
-0@@ -1679,6 +1679,15 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)*/ bool memory_region_is_protected (MemoryRegion *mr); +/** +* memory_region_can_be_private: check whether a memory region can be private The name of the function is not particularly informative,
Quoted 6 lines
include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1685
Quoted 2 lines
include/exec/memory.h +2
-0+* +* Returns %true if a memory region's ram_block has valid gmem fd assigned. but in your comment you describe more accurately what it does, why not make it the function name?
bool
memory_region_has_valid_gmem_fd()
btw can a memory region have an invalid gmem_fd ?
If an invalid gmem_fd is just used to mark whether gmem_fd is present or not,
we could make it just:
bool memory_region_has_gmem_fd()
Thanks,
C
Quoted 35 lines
include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1689
Quoted 5 lines
include/exec/memory.h +4
-0+* +* @mr: the memory region being queried +*/ +bool memory_region_can_be_private (MemoryRegion *mr);bool
memory_region_has_valid_gmem_fd()Thanks,
C
Quoted 20 lines
include/exec/memory.h +1
-0+/** * memory_region_get_iommu: check whether a memory region is an iommu * softmmu/memory.c +5
-0@@ -1855,6 +1855,11 @@ bool memory_region_is_protected(MemoryRegion *mr)return mr->ram && (mr->ram_block->flags &RAM_PROTECTED ); } +bool memory_region_can_be_private (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->gmem_fd >=0 ; +} +uint8_t memory_region_get_dirty_log_mask (MemoryRegion *mr) {uint8_t mask = mr->dirty_log_mask;
Re: [RFC PATCH 04/19] memory: Introduce memory_region_can_be_private()
Quoted 29 lines · Claudio Fontana
Quoted 20 lines · Claudio Fontana
Quoted 11 lines · Xiaoyao Li
Signed-off-by Xiaoyao Li <[email protected]>--- include/exec/memory.h | 9 +++++++++ softmmu/memory.c | 5 +++++ 2 files changed, 14 insertions(+)include/exec/memory.h +0
-0include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1683
Quoted 5 lines
include/exec/memory.h +2
-0@@ -1679,6 +1679,15 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)*/ bool memory_region_is_protected (MemoryRegion *mr); +/** +* memory_region_can_be_private: check whether a memory region can be private The name of the function is not particularly informative,
Quoted 6 lines
include/exec/memory.h static inline bool memory_region_is_romd(MemoryRegion *mr) · line 1685
Quoted 2 lines
include/exec/memory.h +2
-0+* +* Returns %true if a memory region's ram_block has valid gmem fd assigned. but in your comment you describe more accurately what it does, why not make it the function name?
bool
memory_region_has_valid_gmem_fd()
Quoted 7 lines
btw can a memory region have an invalid
gmem_fd?If an invalid
gmem_fdis just used to mark whethergmem_fdis present or not,we could make it just:
bool
memory_region_has_gmem_fd()
yes. It's a good suggestion!
I will use it in next version if no objection from others.
Thanks,
-Xiaoyao
RFC PATCH 5/19 kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
In reply to Xiaoyao LiFrom: Chao Peng <[email protected]>
Switch to KVM_SET_USER_MEMORY_REGION2 when supported by KVM.
With KVM_SET_USER_MEMORY_REGION2, QEMU can set up memory region that backen'ed both by hva-based shared memory and gmem fd based private memory.
--- accel/kvm/kvm-all.c | 57 +++++++++++++++++++++++++++++++++------- accel/kvm/trace-events | 2 +- include/sysemu/kvm_int.h | 2 ++ 3 files changed, 51 insertions(+), 10 deletions(-)
accel/kvm/kvm-all.c +48 -9
@@ -288,35 +288,68 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,static int kvm_set_user_memory_region (KVMMemoryListener *kml,KVMSlot *slot,bool new) {KVMState *s = kvm_state; -struct kvm_userspace_memory_region mem; +struct kvm_userspace_memory_region2 mem; +static int cap_user_memory2 =-1 ;int ret; +if (cap_user_memory2 ==-1 ) { + cap_user_memory2 =kvm_check_extension (s,KVM_CAP_USER_MEMORY2 ); + } + +if (!cap_user_memory2 && slot->fd >=0 ) { +error_report ("%s, KVM doesn't support gmem!" , __func__); +exit (1 ); + } + mem.slot = slot->slot | (kml->as_id <<16 ); mem.guest_phys_addr = slot->start_addr; mem.userspace_addr = (unsigned long )slot->ram; mem.flags = slot->flags; + mem.gmem_fd = slot->fd; + mem.gmem_offset = slot->ofs; -if (slot->memory_size && !new && (mem.flags ^ slot->old_flags) &KVM_MEM_READONLY ) { +if (slot->memory_size && !new && (slot->flags ^ slot->old_flags) &KVM_MEM_READONLY ) {/* Set the slot size to 0 before setting the slot to the desired * value. This is needed based on KVM commit 75d61fbc. */ mem.memory_size =0 ; - ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + +if (cap_user_memory2) { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION2 , &mem); + }else { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + }if (ret <0 ) { goto err; } } mem.memory_size = slot->memory_size; - ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); +if (cap_user_memory2) { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION2 , &mem); + }else { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + } slot->old_flags = mem.flags; err:trace_kvm_set_user_memory (mem.slot >>16 , (uint16_t )mem.slot, mem.flags, mem.guest_phys_addr, mem.memory_size, - mem.userspace_addr, ret); + mem.userspace_addr, mem.gmem_fd, + mem.gmem_offset, ret);if (ret <0 ) { -error_report ("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," -" start=0x%" PRIx64", size=0x%" PRIx64": %s" , - __func__, mem.slot, slot->start_addr, - (uint64_t )mem.memory_size,strerror (errno)); +if (cap_user_memory2) { +error_report ("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d," +" start=0x%" PRIx64", size=0x%" PRIx64"," +" flags=0x%" PRIx32"," +" gmem_fd=%" PRId32", gmem_offset=0x%" PRIx64": %s" , + __func__, mem.slot, slot->start_addr, + (uint64_t )mem.memory_size, mem.flags, + mem.gmem_fd, (uint64_t )mem.gmem_offset, +strerror (errno)); + }else { +error_report ("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," +" start=0x%" PRIx64", size=0x%" PRIx64": %s" , + __func__, mem.slot, slot->start_addr, + (uint64_t )mem.memory_size,strerror (errno)); + } }return ret; }
@@ -472,6 +505,9 @@ static int kvm_mem_flags(MemoryRegion *mr)if (readonly && kvm_readonly_mem_allowed) { flags |=KVM_MEM_READONLY ; } +if (memory_region_can_be_private (mr)) { + flags |=KVM_MEM_PRIVATE ; + }return flags; }
@@ -1402,6 +1438,9 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, mem->ram_start_offset = ram_start_offset; mem->ram = ram; mem->flags =kvm_mem_flags (mr); + mem->fd = mr->ram_block->gmem_fd; + mem->ofs = (uint8_t *)ram - mr->ram_block->host; +kvm_slot_init_dirty_bitmap (mem); err =kvm_set_user_memory_region (kml, mem, true);if (err) {
accel/kvm/trace-events +1 -1
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d" kvm_irqchip_release_virq(int virq) "virq %d" kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d" kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d" -kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" +kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, uint32_t fd, uint64_t fd_offset, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " gmem_fd=%d" " gmem_fd_offset=0x%" PRIx64 " ret=%d" kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32 kvm_resample_fd_notify(int gsi) "gsi %d" kvm_dirty_ring_full(int id) "vcpu %d"
include/sysemu/kvm_int.h +2 -0
@@ -30,6 +30,8 @@ typedef struct KVMSlotint as_id;/* Cache of the offset in ram address space */ ram_addr_t ram_start_offset; +int fd; +hwaddr ofs; } KVMSlot; typedef struct KVMMemoryUpdate {
Signature · 1 line
2.34.1
Re: [RFC PATCH 05/19] kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
Quoted 28 lines · Xiaoyao Li
From: Chao Peng <[email protected]>
Switch to
KVM_SET_USER_MEMORY_REGION2when supported by KVM.With
KVM_SET_USER_MEMORY_REGION2, QEMU can set up memory region that backen'ed both by hva-based shared memory and gmem fd based private memory.Signed-off-by Chao Peng <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>--- accel/kvm/kvm-all.c | 57 +++++++++++++++++++++++++++++++++------- accel/kvm/trace-events | 2 +- include/sysemu/kvm_int.h | 2 ++ 3 files changed, 51 insertions(+), 10 deletions(-)accel/kvm/kvm-all.c +2
-1@@ -288,35 +288,68 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,static int kvm_set_user_memory_region (KVMMemoryListener *kml,KVMSlot *slot,bool new) {KVMState *s = kvm_state; -struct kvm_userspace_memory_region mem; +struct kvm_userspace_memory_region2 mem; +static int cap_user_memory2 =-1 ;
accel/kvm/kvm-all.c int kvm_physical_memory_addr_from_host(KVMState *s, void *ram, · line 302
Quoted 10 lines
accel/kvm/kvm-all.c +8
-0int ret; +if (cap_user_memory2 ==-1 ) { + cap_user_memory2 =kvm_check_extension (s,KVM_CAP_USER_MEMORY2 ); + } + +if (!cap_user_memory2 && slot->fd >=0 ) { +error_report ("%s, KVM doesn't support gmem!" , __func__); +exit (1 ); + }
We handle this special error case here,
while the existing callers of kvm_set_user_memory_region handle the other error cases in different places.
Not that the rest of kvm-all does an excellent job at error handling, but maybe we can avoid compounding on the issue.
accel/kvm/kvm-all.c int kvm_physical_memory_addr_from_host(KVMState *s, void *ram, · line 311
Quoted 10 lines
accel/kvm/kvm-all.c +4
-1+ mem.slot = slot->slot | (kml->as_id <<16 ); mem.guest_phys_addr = slot->start_addr; mem.userspace_addr = (unsigned long )slot->ram; mem.flags = slot->flags; + mem.gmem_fd = slot->fd; + mem.gmem_offset = slot->ofs; -if (slot->memory_size && !new && (mem.flags ^ slot->old_flags) &KVM_MEM_READONLY ) { +if (slot->memory_size && !new && (slot->flags ^ slot->old_flags) &KVM_MEM_READONLY ) {
Why the change if mem.flags == slot->flags ?
Quoted 97 lines
accel/kvm/kvm-all.c +34
-7/* Set the slot size to 0 before setting the slot to the desired * value. This is needed based on KVM commit 75d61fbc. */ mem.memory_size =0 ; - ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + +if (cap_user_memory2) { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION2 , &mem); + }else { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + }if (ret <0 ) { goto err; } } mem.memory_size = slot->memory_size; - ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); +if (cap_user_memory2) { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION2 , &mem); + }else { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + } slot->old_flags = mem.flags; err:trace_kvm_set_user_memory (mem.slot >>16 , (uint16_t )mem.slot, mem.flags, mem.guest_phys_addr, mem.memory_size, - mem.userspace_addr, ret); + mem.userspace_addr, mem.gmem_fd, + mem.gmem_offset, ret);if (ret <0 ) { -error_report ("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," -" start=0x%" PRIx64", size=0x%" PRIx64": %s" , - __func__, mem.slot, slot->start_addr, - (uint64_t )mem.memory_size,strerror (errno)); +if (cap_user_memory2) { +error_report ("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d," +" start=0x%" PRIx64", size=0x%" PRIx64"," +" flags=0x%" PRIx32"," +" gmem_fd=%" PRId32", gmem_offset=0x%" PRIx64": %s" , + __func__, mem.slot, slot->start_addr, + (uint64_t )mem.memory_size, mem.flags, + mem.gmem_fd, (uint64_t )mem.gmem_offset, +strerror (errno)); + }else { +error_report ("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," +" start=0x%" PRIx64", size=0x%" PRIx64": %s" , + __func__, mem.slot, slot->start_addr, + (uint64_t )mem.memory_size,strerror (errno)); + } }return ret; }@@ -472,6 +505,9 @@ static int kvm_mem_flags(MemoryRegion *mr)if (readonly && kvm_readonly_mem_allowed) { flags |=KVM_MEM_READONLY ; } +if (memory_region_can_be_private (mr)) { + flags |=KVM_MEM_PRIVATE ; + }return flags; }@@ -1402,6 +1438,9 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, mem->ram_start_offset = ram_start_offset; mem->ram = ram; mem->flags =kvm_mem_flags (mr); + mem->fd = mr->ram_block->gmem_fd; + mem->ofs = (uint8_t *)ram - mr->ram_block->host; +kvm_slot_init_dirty_bitmap (mem); err =kvm_set_user_memory_region (kml, mem, true);if (err) {accel/kvm/trace-events +1
-1@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d" kvm_irqchip_release_virq(int virq) "virq %d" kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d" kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d" -kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" +kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, uint32_t fd, uint64_t fd_offset, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " gmem_fd=%d" " gmem_fd_offset=0x%" PRIx64 " ret=%d" kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32 kvm_resample_fd_notify(int gsi) "gsi %d" kvm_dirty_ring_full(int id) "vcpu %d"include/sysemu/kvm_int.h +2
-0@@ -30,6 +30,8 @@ typedef struct KVMSlotint as_id;/* Cache of the offset in ram address space */ ram_addr_t ram_start_offset; +int fd; +hwaddr ofs; } KVMSlot; typedef struct KVMMemoryUpdate {
Re: [RFC PATCH 05/19] kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
Quoted 43 lines · Claudio Fontana
Quoted 28 lines · Xiaoyao Li
From: Chao Peng <[email protected]>
Switch to
KVM_SET_USER_MEMORY_REGION2when supported by KVM.With
KVM_SET_USER_MEMORY_REGION2, QEMU can set up memory region that backen'ed both by hva-based shared memory and gmem fd based private memory.Signed-off-by Chao Peng <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>--- accel/kvm/kvm-all.c | 57 +++++++++++++++++++++++++++++++++------- accel/kvm/trace-events | 2 +- include/sysemu/kvm_int.h | 2 ++ 3 files changed, 51 insertions(+), 10 deletions(-)accel/kvm/kvm-all.c +2
-1@@ -288,35 +288,68 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,static int kvm_set_user_memory_region (KVMMemoryListener *kml,KVMSlot *slot,bool new) {KVMState *s = kvm_state; -struct kvm_userspace_memory_region mem; +struct kvm_userspace_memory_region2 mem; +static int cap_user_memory2 =-1 ;accel/kvm/kvm-all.c int kvm_physical_memory_addr_from_host(KVMState *s, void *ram, · line 302
Quoted 10 lines
accel/kvm/kvm-all.c +8
-0int ret; +if (cap_user_memory2 ==-1 ) { + cap_user_memory2 =kvm_check_extension (s,KVM_CAP_USER_MEMORY2 ); + } + +if (!cap_user_memory2 && slot->fd >=0 ) { +error_report ("%s, KVM doesn't support gmem!" , __func__); +exit (1 ); + }We handle this special error case here,
while the existing callers ofkvm_set_user_memory_regionhandle the other error cases in different places.
Quoted 1 line
Not that the rest of kvm-all does an excellent job at error handling, but maybe we can avoid compounding on the issue.
I'm not sure how to align them. Do you have any suggestion?
Quoted 8 lines
Quoted 8 lines
accel/kvm/kvm-all.c +3
-0+ mem.slot = slot->slot | (kml->as_id <<16 ); mem.guest_phys_addr = slot->start_addr; mem.userspace_addr = (unsigned long )slot->ram; mem.flags = slot->flags; + mem.gmem_fd = slot->fd; + mem.gmem_offset = slot->ofs;
Quoted 4 lines
accel/kvm/kvm-all.c int kvm_physical_memory_addr_from_host(KVMState *s, void *ram, · line 311
Quoted 2 lines
accel/kvm/kvm-all.c +1
-1-if (slot->memory_size && !new && (mem.flags ^ slot->old_flags) &KVM_MEM_READONLY ) { +if (slot->memory_size && !new && (slot->flags ^ slot->old_flags) &KVM_MEM_READONLY ) {Why the change if mem.flags ==
slot->flags?
I guess the goal is to make it clearer that it's comparing the (new) flags with old_flags of the slot.
Anyway, if this change is annoying, I can drop it. :)
Quoted 98 lines
Quoted 97 lines
accel/kvm/kvm-all.c +34
-7/* Set the slot size to 0 before setting the slot to the desired * value. This is needed based on KVM commit 75d61fbc. */ mem.memory_size =0 ; - ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + +if (cap_user_memory2) { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION2 , &mem); + }else { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + }if (ret <0 ) { goto err; } } mem.memory_size = slot->memory_size; - ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); +if (cap_user_memory2) { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION2 , &mem); + }else { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + } slot->old_flags = mem.flags; err:trace_kvm_set_user_memory (mem.slot >>16 , (uint16_t )mem.slot, mem.flags, mem.guest_phys_addr, mem.memory_size, - mem.userspace_addr, ret); + mem.userspace_addr, mem.gmem_fd, + mem.gmem_offset, ret);if (ret <0 ) { -error_report ("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," -" start=0x%" PRIx64", size=0x%" PRIx64": %s" , - __func__, mem.slot, slot->start_addr, - (uint64_t )mem.memory_size,strerror (errno)); +if (cap_user_memory2) { +error_report ("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d," +" start=0x%" PRIx64", size=0x%" PRIx64"," +" flags=0x%" PRIx32"," +" gmem_fd=%" PRId32", gmem_offset=0x%" PRIx64": %s" , + __func__, mem.slot, slot->start_addr, + (uint64_t )mem.memory_size, mem.flags, + mem.gmem_fd, (uint64_t )mem.gmem_offset, +strerror (errno)); + }else { +error_report ("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," +" start=0x%" PRIx64", size=0x%" PRIx64": %s" , + __func__, mem.slot, slot->start_addr, + (uint64_t )mem.memory_size,strerror (errno)); + } }return ret; }@@ -472,6 +505,9 @@ static int kvm_mem_flags(MemoryRegion *mr)if (readonly && kvm_readonly_mem_allowed) { flags |=KVM_MEM_READONLY ; } +if (memory_region_can_be_private (mr)) { + flags |=KVM_MEM_PRIVATE ; + }return flags; }@@ -1402,6 +1438,9 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, mem->ram_start_offset = ram_start_offset; mem->ram = ram; mem->flags =kvm_mem_flags (mr); + mem->fd = mr->ram_block->gmem_fd; + mem->ofs = (uint8_t *)ram - mr->ram_block->host; +kvm_slot_init_dirty_bitmap (mem); err =kvm_set_user_memory_region (kml, mem, true);if (err) {accel/kvm/trace-events +1
-1@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d" kvm_irqchip_release_virq(int virq) "virq %d" kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d" kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d" -kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" +kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, uint32_t fd, uint64_t fd_offset, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " gmem_fd=%d" " gmem_fd_offset=0x%" PRIx64 " ret=%d" kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32 kvm_resample_fd_notify(int gsi) "gsi %d" kvm_dirty_ring_full(int id) "vcpu %d"include/sysemu/kvm_int.h +2
-0@@ -30,6 +30,8 @@ typedef struct KVMSlotint as_id;/* Cache of the offset in ram address space */ ram_addr_t ram_start_offset; +int fd; +hwaddr ofs; } KVMSlot; typedef struct KVMMemoryUpdate {
RFC PATCH 6/19 i386/pc: Drop pc_machine_kvm_type()
In reply to Xiaoyao Lipc_machine_kvm_type() was introduced by commit e21be724eaf5 ("i386/xen:
add pc_machine_kvm_type to initialize XEN_EMULATE mode") to do Xen specific initialization by utilizing kvm_type method.
commit eeedfe6c6316 ("hw/xen: Simplify emulated Xen platform init") moves the Xen specific initialization to pc_basic_device_init().
There is no need to keep the PC specific kvm_type() implementation anymore. On the other hand, later patch will implement kvm_type() method for all x86/i386 machines to support KVM_X86_SW_PROTECTED_VM.
--- hw/i386/pc.c | 5 ----- include/hw/i386/pc.h | 3 --- 2 files changed, 8 deletions(-)
hw/i386/pc.c +0 -5
@@ -1794,11 +1794,6 @@ static void pc_machine_initfn(Object *obj)cxl_machine_init (obj, &pcms->cxl_devices_state); } -int pc_machine_kvm_type (MachineState *machine,const char *kvm_type) -{ -return 0 ; -} -static void pc_machine_reset (MachineState *machine,ShutdownCause reason) {CPUState *cs;
include/hw/i386/pc.h +0 -3
@@ -296,15 +296,12 @@ extern const size_t pc_compat_1_5_len;extern GlobalProperty pc_compat_1_4[];extern const size_t pc_compat_1_4_len; -int pc_machine_kvm_type (MachineState *machine,const char *vm_type); -#define DEFINE_PC_MACHINE (suffix, namestr, initfn, optsfn) \ static void pc_machine_##suffix##_class_init(ObjectClass *oc, void *data) \ { \ MachineClass *mc = MACHINE_CLASS(oc); \ optsfn(mc); \ mc->init = initfn; \ - mc->kvm_type = pc_machine_kvm_type; \ } \ static const TypeInfo pc_machine_type_##suffix = { \ .name = namestr TYPE_MACHINE_SUFFIX, \
Signature · 1 line
2.34.1
Re: [RFC PATCH 06/19] i386/pc: Drop pc_machine_kvm_type()
Quoted 56 lines · Xiaoyao Li
pc_machine_kvm_type()was introduced by commite21be724eaf5("i386/xen:
addpc_machine_kvm_typeto initializeXEN_EMULATEmode") to do Xen specific initialization by utilizingkvm_typemethod.commit
eeedfe6c6316("hw/xen: Simplify emulated Xen platform init") moves the Xen specific initialization topc_basic_device_init().There is no need to keep the PC specific
kvm_type()implementation anymore. On the other hand, later patch will implementkvm_type()method for all x86/i386 machines to supportKVM_X86_SW_PROTECTED_VM.Signed-off-by Xiaoyao Li <[email protected]>--- hw/i386/pc.c | 5 ----- include/hw/i386/pc.h | 3 --- 2 files changed, 8 deletions(-)hw/i386/pc.c +0
-5@@ -1794,11 +1794,6 @@ static void pc_machine_initfn(Object *obj)cxl_machine_init (obj, &pcms->cxl_devices_state); } -int pc_machine_kvm_type (MachineState *machine,const char *kvm_type) -{ -return 0 ; -} -static void pc_machine_reset (MachineState *machine,ShutdownCause reason) {CPUState *cs;include/hw/i386/pc.h +0
-3@@ -296,15 +296,12 @@ extern const size_t pc_compat_1_5_len;extern GlobalProperty pc_compat_1_4[];extern const size_t pc_compat_1_4_len; -int pc_machine_kvm_type (MachineState *machine,const char *vm_type); -#define DEFINE_PC_MACHINE (suffix, namestr, initfn, optsfn) \ static void pc_machine_##suffix##_class_init(ObjectClass *oc, void *data) \ { \ MachineClass *mc = MACHINE_CLASS(oc); \ optsfn(mc); \ mc->init = initfn; \ - mc->kvm_type = pc_machine_kvm_type; \ } \ static const TypeInfo pc_machine_type_##suffix = { \ .name = namestr TYPE_MACHINE_SUFFIX, \Signature · 1 line
2.34.1
It seems strange for MachineClass to have kvm_type(). Probably AccelClass. (struct KVMAccelClass?)
Anyway this is independent clean up.
Signature · 1 line
Isaku Yamahata <[email protected]>
RFC PATCH 7/19 target/i386: Implement mc->kvm_type() to get VM type
In reply to Xiaoyao LiImplement mc->kvm_type() for i386 machines. It provides a way for user to create SW_PROTECTE_VM.
--- hw/i386/x86.c | 27 +++++++++++++++++++++++++++ include/hw/i386/x86.h | 4 ++++ target/i386/kvm/kvm.c | 38 ++++++++++++++++++++++++++++++++++++++ target/i386/kvm/kvm_i386.h | 1 + 4 files changed, 70 insertions(+)
hw/i386/x86.c +27 -0
@@ -1382,6 +1382,26 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,qapi_free_SgxEPCList (list); } +static int x86_kvm_type (MachineState *ms,const char *vm_type) +{ +return kvm_get_vm_type (ms, vm_type); +} + +static char *x86_machine_get_kvm_type (Object *obj,Error **errp) +{ +X86MachineState *x86ms =X86_MACHINE (obj); + +return g_strdup (x86ms->kvm_type); +} + +static void x86_machine_set_kvm_type (Object *obj,const char *value,Error **errp) +{ +X86MachineState *x86ms =X86_MACHINE (obj); + +g_free (x86ms->kvm_type); + x86ms->kvm_type =g_strdup (value); +} +static void x86_machine_initfn (Object *obj) {X86MachineState *x86ms =X86_MACHINE (obj);
@@ -1406,6 +1426,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data) mc->cpu_index_to_instance_props = x86_cpu_index_to_props; mc->get_default_cpu_node_id = x86_get_default_cpu_node_id; mc->possible_cpu_arch_ids = x86_possible_cpu_arch_ids; + mc->kvm_type = x86_kvm_type; x86mc->save_tsc_khz = true; x86mc->fwcfg_dma_enabled = true; nc->nmi_monitor_handler = x86_nmi;
@@ -1464,6 +1485,12 @@ static void x86_machine_class_init(ObjectClass *oc, void *data) NULL, NULL);object_class_property_set_description (oc,"sgx-epc" ,"SGX EPC device" ); + +object_class_property_add_str (oc,X86_MACHINE_KVM_TYPE , + x86_machine_get_kvm_type, + x86_machine_set_kvm_type); +object_class_property_set_description (oc,X86_MACHINE_KVM_TYPE , +"KVM guest type (default, sw_protected_vm)" ); }static const TypeInfo x86_machine_info = {
include/hw/i386/x86.h +4 -0
@@ -42,6 +42,9 @@ struct X86MachineState {/*< public >*/ +char *kvm_type; +unsigned int vm_type; +/* Pointers to devices and objects: */ ISADevice *rtc;FWCfgState *fw_cfg;
@@ -91,6 +94,7 @@ struct X86MachineState {#define X86_MACHINE_OEM_ID "x-oem-id"#define X86_MACHINE_OEM_TABLE_ID "x-oem-table-id"#define X86_MACHINE_BUS_LOCK_RATELIMIT "bus-lock-ratelimit" +#define X86_MACHINE_KVM_TYPE "kvm-type"#define TYPE_X86_MACHINE MACHINE_TYPE_NAME("x86")OBJECT_DECLARE_TYPE (X86MachineState, X86MachineClass,X86_MACHINE )
target/i386/kvm/kvm.c +38 -0
@@ -154,6 +154,44 @@ static KVMMSRHandlers msr_handlers[KVM_MSR_FILTER_MAX_RANGES];static RateLimit bus_lock_ratelimit_ctrl;static int kvm_get_one_msr (X86CPU *cpu,int index,uint64_t *value); +static const char * vm_type_name[] = { + [KVM_X86_DEFAULT_VM ] ="default" , + [KVM_X86_SW_PROTECTED_VM ] ="sw-protected-vm" , +}; + +int kvm_get_vm_type (MachineState *ms,const char *vm_type) +{ +X86MachineState *x86ms =X86_MACHINE (ms); +int kvm_type =KVM_X86_DEFAULT_VM ; + +if (vm_type) { +if (!g_ascii_strcasecmp (vm_type,"default" ) || !g_ascii_strcasecmp (vm_type,"" )) { + kvm_type =KVM_X86_DEFAULT_VM ; + }else if (!g_ascii_strcasecmp (vm_type,"sw-protected-vm" )) { + kvm_type =KVM_X86_SW_PROTECTED_VM ; + }else { +error_report ("Unknown kvm-type specified '%s'" , vm_type); +exit (1 ); + } + } + +/* +* old KVM doesn't support KVM_CAP_VM_TYPES and KVM_X86_DEFAULT_VM +* is always supported +*/ +if (kvm_type ==KVM_X86_DEFAULT_VM ) { +return kvm_type; + } + +if (!(kvm_check_extension (KVM_STATE (ms->accelerator),KVM_CAP_VM_TYPES ) &BIT (kvm_type))) { +error_report ("vm-type %s not supported by KVM" , vm_type_name[kvm_type]); +exit (1 ); + } + + x86ms->vm_type = kvm_type; +return kvm_type; +} +int kvm_has_pit_state2 (void ) {return has_pit_state2;
target/i386/kvm/kvm_i386.h +1 -0
@@ -37,6 +37,7 @@ bool kvm_has_adjust_clock(void);bool kvm_has_adjust_clock_stable (void );bool kvm_has_exception_payload (void );void kvm_synchronize_all_tsc (void ); +int kvm_get_vm_type (MachineState *ms,const char *vm_type);void kvm_arch_reset_vcpu (X86CPU *cs);void kvm_arch_after_reset_vcpu (X86CPU *cpu);void kvm_arch_do_init_vcpu (X86CPU *cs);
Signature · 1 line
2.34.1
RFC PATCH 8/19 HostMem: Add private property to indicate to use kvm gmem
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
--- backends/hostmem.c | 18 ++++++++++++++++++ include/sysemu/hostmem.h | 2 +- qapi/qom.json | 4 ++++ 3 files changed, 23 insertions(+), 1 deletion(-)
backends/hostmem.c +18 -0
@@ -461,6 +461,20 @@ static void host_memory_backend_set_reserve(Object *o, bool value, Error **errp) } backend->reserve = value; } + +static bool host_memory_backend_get_private (Object *o,Error **errp) +{ +HostMemoryBackend *backend =MEMORY_BACKEND (o); + +return backend->private; +} + +static void host_memory_backend_set_private (Object *o,bool value,Error **errp) +{ +HostMemoryBackend *backend =MEMORY_BACKEND (o); + + backend->private = value; +}#endif /* CONFIG_LINUX */ static bool
@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc, void *data) host_memory_backend_get_reserve, host_memory_backend_set_reserve);object_class_property_set_description (oc,"reserve" ,"Reserve swap space (or huge pages) if applicable" ); +object_class_property_add_bool (oc,"private" , + host_memory_backend_get_private, host_memory_backend_set_private); +object_class_property_set_description (oc,"private" , +"Use KVM gmem private memory" );#endif /* CONFIG_LINUX */ /* * Do not delete/rename option. This option must be considered stable
include/sysemu/hostmem.h +1 -1
@@ -65,7 +65,7 @@ struct HostMemoryBackend {/* protected */ uint64_t size;bool merge, dump, use_canonical_path; -bool prealloc, is_mapped, share, reserve; +bool prealloc, is_mapped, share, reserve, private;uint32_t prealloc_threads;ThreadContext *prealloc_context;DECLARE_BITMAP (host_nodes,MAX_NODES +1 );
qapi/qom.json +4 -0
@@ -605,6 +605,9 @@ # @reserve: if true, reserve swap space (or huge pages) if applicable # (default: true) (since 6.1) # +# @private: if true, use KVM gmem private memory +# (default: false) (since 8.1) +# # @size: size of the memory region in bytes # # @x-use-canonical-path-for-ramblock-id: if true, the canonical path
@@ -631,6 +634,7 @@ '*prealloc-context': 'str', '*share': 'bool', '*reserve': 'bool', + '*private': 'bool', 'size': 'size', '*x-use-canonical-path-for-ramblock-id': 'bool' } }
Signature · 1 line
2.34.1
Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem
Quoted 4 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>
[...]
qapi/qom.json line 610
Quoted 11 lines
qapi/qom.json +3
-0@@ -605,6 +605,9 @@ # @reserve: if true, reserve swap space (or huge pages) if applicable # (default: true) (since 6.1) # +# @private: if true, use KVM gmem private memory +# (default: false) (since 8.1) +#
Please format like
# @private: if true, use KVM gmem private memory (default: false)
# (since 8.1)
to blend in with recent commit a937b6aa739 (qapi: Reformat doc comments to conform to current conventions).
Quoted 10 lines
qapi/qom.json +1
-0# @size: size of the memory region in bytes # # @x-use-canonical-path-for-ramblock-id: if true, the canonical path@@ -631,6 +634,7 @@ '*prealloc-context': 'str', '*share': 'bool', '*reserve': 'bool', + '*private': 'bool', 'size': 'size', '*x-use-canonical-path-for-ramblock-id': 'bool' } }
Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem
Quoted 23 lines · Markus Armbruster
Quoted 4 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>[...]
qapi/qom.json line 610
Quoted 11 lines
qapi/qom.json +3
-0@@ -605,6 +605,9 @@ # @reserve: if true, reserve swap space (or huge pages) if applicable # (default: true) (since 6.1) # +# @private: if true, use KVM gmem private memory +# (default: false) (since 8.1) +#Please format like
Quoted 5 lines
# @private: if true, use KVM gmem private memory (default: false)
# (since 8.1)to blend in with recent commit
a937b6aa739(qapi: Reformat doc comments to conform to current conventions).
will do it in next version.
Thanks!
-Xiaoyao
Quoted 11 lines
Quoted 10 lines
qapi/qom.json +1
-0# @size: size of the memory region in bytes # # @x-use-canonical-path-for-ramblock-id: if true, the canonical path@@ -631,6 +634,7 @@ '*prealloc-context': 'str', '*share': 'bool', '*reserve': 'bool', + '*private': 'bool', 'size': 'size', '*x-use-canonical-path-for-ramblock-id': 'bool' } }
Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem
In reply to Markus ArmbrusterQuoted 23 lines · Markus Armbruster
Quoted 4 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>[...]
qapi/qom.json line 610
Quoted 11 lines
qapi/qom.json +3
-0@@ -605,6 +605,9 @@ # @reserve: if true, reserve swap space (or huge pages) if applicable # (default: true) (since 6.1) # +# @private: if true, use KVM gmem private memory +# (default: false) (since 8.1) +#Please format like
Quoted 2 lines
# @private: if true, use KVM gmem private memory (default: false)
# (since 8.1)
Also QEMU 8.1.0 is in freeze right now, so there's no chance of these patches making 8.1.0. IOW, use "since 8.2" as the next release you might achieve merge for.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem
Quoted 24 lines · Daniel P. Berrangé
Quoted 23 lines · Markus Armbruster
Quoted 4 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>[...]
qapi/qom.json line 610
Quoted 11 lines
qapi/qom.json +3
-0@@ -605,6 +605,9 @@ # @reserve: if true, reserve swap space (or huge pages) if applicable # (default: true) (since 6.1) # +# @private: if true, use KVM gmem private memory +# (default: false) (since 8.1) +#Please format like
Quoted 6 lines
Quoted 2 lines
# @private: if true, use KVM gmem private memory (default: false)
# (since 8.1)Also QEMU 8.1.0 is in freeze right now, so there's no chance of these patches making 8.1.0. IOW, use "since 8.2" as the next release you might achieve merge for.
Thanks for pointing it out. Will do it in next version.
Quoted 2 lines
With regards,
Daniel
Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem
In reply to Xiaoyao LiQuoted 64 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>--- backends/hostmem.c | 18 ++++++++++++++++++ include/sysemu/hostmem.h | 2 +- qapi/qom.json | 4 ++++ 3 files changed, 23 insertions(+), 1 deletion(-)backends/hostmem.c +18
-0@@ -461,6 +461,20 @@ static void host_memory_backend_set_reserve(Object *o, bool value, Error **errp) } backend->reserve = value; } + +static bool host_memory_backend_get_private (Object *o,Error **errp) +{ +HostMemoryBackend *backend =MEMORY_BACKEND (o); + +return backend->private; +} + +static void host_memory_backend_set_private (Object *o,bool value,Error **errp) +{ +HostMemoryBackend *backend =MEMORY_BACKEND (o); + + backend->private = value; +}#endif /* CONFIG_LINUX */ static bool@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc, void *data) host_memory_backend_get_reserve, host_memory_backend_set_reserve);object_class_property_set_description (oc,"reserve" ,"Reserve swap space (or huge pages) if applicable" ); +object_class_property_add_bool (oc,"private" , + host_memory_backend_get_private, host_memory_backend_set_private); +object_class_property_set_description (oc,"private" , +"Use KVM gmem private memory" );#endif /* CONFIG_LINUX */ /* * Do not delete/rename option. This option must be considered stable include/sysemu/hostmem.h +1
-1@@ -65,7 +65,7 @@ struct HostMemoryBackend {/* protected */ uint64_t size;bool merge, dump, use_canonical_path; -bool prealloc, is_mapped, share, reserve; +bool prealloc, is_mapped, share, reserve, private;uint32_t prealloc_threads;ThreadContext *prealloc_context;DECLARE_BITMAP (host_nodes,MAX_NODES +1 );qapi/qom.json +0
-0
qapi/qom.json line 610
Quoted 6 lines
qapi/qom.json +3
-0@@ -605,6 +605,9 @@ # @reserve: if true, reserve swap space (or huge pages) if applicable # (default: true) (since 6.1) # +# @private: if true, use KVM gmem private memory +# (default: false) (since 8.1) +#
But that's not what any of this does.
This patch only adds a property and doesn't even explain what it intends to achieve with that.
How will it be used from a user? What will it affect internally? What will it modify in regards of the memory backend?
That all should go into the surprisingly empty patch description.
Signature · 3 lines
Cheers, David / dhildenb
Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem
Quoted 77 lines · David Hildenbrand
Quoted 67 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
backends/hostmem.c | 18 ++++++++++++++++++include/sysemu/hostmem.h| 2 +- qapi/qom.json | 4 ++++3 files changed, 23 insertions(+), 1 deletion(-)backends/hostmem.c +14
-0@@ -461,6 +461,20 @@ static void host_memory_backend_set_reserve(Object *o, bool value, Error **errp) } backend->reserve = value; } + +static bool host_memory_backend_get_private (Object *o,Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(o); + + return backend->private; +} + +static void host_memory_backend_set_private (Object *o,bool value,Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(o); + + backend->private = value; +} #endif /* CONFIG_LINUX */static bool
backends/hostmem.c +4
-0@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc, void *data) host_memory_backend_get_reserve, host_memory_backend_set_reserve); object_class_property_set_description(oc, "reserve", "Reserve swap space (or huge pages) if applicable"); + object_class_property_add_bool(oc, "private", + host_memory_backend_get_private, host_memory_backend_set_private); + object_class_property_set_description(oc, "private", + "Use KVM gmem private memory"); #endif /* CONFIG_LINUX *//*
* Do not delete/rename option. This option must be considered stableinclude/sysemu/hostmem.h +1
-1@@ -65,7 +65,7 @@ struct HostMemoryBackend { /* protected */ uint64_t size; bool merge, dump, use_canonical_path; - bool prealloc, is_mapped, share, reserve; + bool prealloc, is_mapped, share, reserve, private; uint32_t prealloc_threads; ThreadContext *prealloc_context; DECLARE_BITMAP(host_nodes, MAX_NODES + 1);qapi/qom.json +0
-0qapi/qom.json line 610
Quoted 6 lines
qapi/qom.json +3
-0@@ -605,6 +605,9 @@ # @reserve: if true, reserve swap space (or huge pages) if applicable # (default: true) (since 6.1) # +# @private: if true, use KVM gmem private memory +# (default: false) (since 8.1) +#But that's not what any of this does.
Quoted 3 lines
This patch only adds a property and doesn't even explain what it intends to achieve with that.
How will it be used from a user? What will it affect internally? What will it modify in regards of the memory backend?
How it will be used is in the next patch, patch 09.
for kvm_x86_sw_protected_vm type VM, it will allocate private gmem with KVM ioctl if the memory backend has property "private" on.
Quoted 1 line
That all should go into the surprisingly empty patch description.
I'm sorry. I admit the empty commit message is really bad.
Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem
Quoted 83 lines · Xiaoyao Li
Quoted 82 lines · David Hildenbrand
Quoted 69 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>--- backends/hostmem.c | 18 ++++++++++++++++++ include/sysemu/hostmem.h | 2 +- qapi/qom.json | 4 ++++ 3 files changed, 23 insertions(+), 1 deletion(-)backends/hostmem.c +14
-0@@ -461,6 +461,20 @@ static void host_memory_backend_set_reserve(Object *o, bool value, Error **errp) } backend->reserve = value; } + +static bool host_memory_backend_get_private (Object *o,Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(o); + + return backend->private; +} + +static void host_memory_backend_set_private (Object *o,bool value,Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(o); + + backend->private = value; +} #endif /* CONFIG_LINUX */static bool
backends/hostmem.c +4
-0@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc, void *data) host_memory_backend_get_reserve, host_memory_backend_set_reserve); object_class_property_set_description(oc, "reserve", "Reserve swap space (or huge pages) if applicable"); + object_class_property_add_bool(oc, "private", + host_memory_backend_get_private, host_memory_backend_set_private); + object_class_property_set_description(oc, "private", + "Use KVM gmem private memory");#endif /*
CONFIG_LINUX*/
/*
* Do not delete/rename option. This option must be considered
stableinclude/sysemu/hostmem.h +1
-1@@ -65,7 +65,7 @@ struct HostMemoryBackend { /* protected */ uint64_t size; bool merge, dump, use_canonical_path; - bool prealloc, is_mapped, share, reserve; + bool prealloc, is_mapped, share, reserve, private; uint32_t prealloc_threads; ThreadContext *prealloc_context; DECLARE_BITMAP(host_nodes, MAX_NODES + 1);qapi/qom.json +0
-0qapi/qom.json line 610
Quoted 6 lines
qapi/qom.json +3
-0@@ -605,6 +605,9 @@ # @reserve: if true, reserve swap space (or huge pages) if applicable # (default: true) (since 6.1) # +# @private: if true, use KVM gmem private memory +# (default: false) (since 8.1) +#But that's not what any of this does.
This patch only adds a property and doesn't even explain what it intends to achieve with that.
Quoted 7 lines
Quoted 2 lines
How will it be used from a user? What will it affect internally? What will it modify in regards of the memory backend?
How it will be used is in the next patch, patch 09.
for
kvm_x86_sw_protected_vmtype VM, it will allocate private gmem with KVM ioctl if the memory backend has property "private" on.
It feels wired up the wrong way.
When creating/initializing the memory backend, we should also take care of allocating the gmem_fd, for example, by doing some gmem allocation callback, ideally *internally* creating the RAM memory region / RAMBlock.
And we should fail if that is impossible (gmem does not apply to the VM) or creating the gmem_fd failed for other reason.
Like passing a RAM_GMEM flag to memory_region_init_ram_flags_nomigrate() in ram_backend_memory_alloc(), to then handle it internally, failing if there is an error.
Signature · 3 lines
Cheers, David / dhildenb
Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem
Quoted 98 lines · David Hildenbrand
Quoted 83 lines · Xiaoyao Li
Quoted 82 lines · David Hildenbrand
Quoted 69 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>--- backends/hostmem.c | 18 ++++++++++++++++++ include/sysemu/hostmem.h | 2 +- qapi/qom.json | 4 ++++ 3 files changed, 23 insertions(+), 1 deletion(-)backends/hostmem.c +14
-0@@ -461,6 +461,20 @@ static void host_memory_backend_set_reserve(Object *o, bool value, Error **errp) } backend->reserve = value; } + +static bool host_memory_backend_get_private (Object *o,Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(o); + + return backend->private; +} + +static void host_memory_backend_set_private (Object *o,bool value,Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(o); + + backend->private = value; +} #endif /* CONFIG_LINUX */static bool
backends/hostmem.c +4
-0@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc, void *data) host_memory_backend_get_reserve, host_memory_backend_set_reserve); object_class_property_set_description(oc, "reserve", "Reserve swap space (or huge pages) if applicable"); + object_class_property_add_bool(oc, "private", + host_memory_backend_get_private, host_memory_backend_set_private); + object_class_property_set_description(oc, "private", + "Use KVM gmem private memory");#endif /*
CONFIG_LINUX*/
/*
* Do not delete/rename option. This option must be considered
stableinclude/sysemu/hostmem.h +1
-1@@ -65,7 +65,7 @@ struct HostMemoryBackend { /* protected */ uint64_t size; bool merge, dump, use_canonical_path; - bool prealloc, is_mapped, share, reserve; + bool prealloc, is_mapped, share, reserve, private; uint32_t prealloc_threads; ThreadContext *prealloc_context; DECLARE_BITMAP(host_nodes, MAX_NODES + 1);qapi/qom.json +0
-0qapi/qom.json line 610
Quoted 6 lines
qapi/qom.json +3
-0@@ -605,6 +605,9 @@ # @reserve: if true, reserve swap space (or huge pages) if applicable # (default: true) (since 6.1) # +# @private: if true, use KVM gmem private memory +# (default: false) (since 8.1) +#But that's not what any of this does.
This patch only adds a property and doesn't even explain what it intends to achieve with that.
Quoted 7 lines
Quoted 2 lines
How will it be used from a user? What will it affect internally? What will it modify in regards of the memory backend?
How it will be used is in the next patch, patch 09.
for
kvm_x86_sw_protected_vmtype VM, it will allocate private gmem with KVM ioctl if the memory backend has property "private" on.It feels wired up the wrong way.
When creating/initializing the memory backend, we should also take care of allocating the
gmem_fd, for example, by doing some gmem allocation callback, ideally *internally* creating the RAM memory region / RAMBlock.
Quoted 6 lines
And we should fail if that is impossible (gmem does not apply to the VM) or creating the
gmem_fdfailed for other reason.Like passing a
RAM_GMEMflag tomemory_region_init_ram_flags_nomigrate()inram_backend_memory_alloc(), to then handle it internally, failing if there is an error.
KVM gmem is tied to VM. not before creating VM. We have to delay of the allocation of kvm gmem until VM initialization.
Hmm, one options is to move gmem_fd from RAMBlock to KVMSlot. Handle the allocation of KVM gmem (issuing KVM gmem ioctl) there. i.e. in kvm_set_phys_mem() or kvm_region_add() (or whatever functions of KVM memory listener). Maybe we can drop ram_block_convert_range() and can have KVM specific logic instead.
We still need a way for user to specify which guest memory region is subject to KVM gmem, though.
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem
Quoted 104 lines · Isaku Yamahata
Quoted 101 lines · David Hildenbrand
Quoted 83 lines · Xiaoyao Li
Quoted 82 lines · David Hildenbrand
Quoted 69 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>--- backends/hostmem.c | 18 ++++++++++++++++++ include/sysemu/hostmem.h | 2 +- qapi/qom.json | 4 ++++ 3 files changed, 23 insertions(+), 1 deletion(-)backends/hostmem.c +14
-0@@ -461,6 +461,20 @@ static void host_memory_backend_set_reserve(Object *o, bool value, Error **errp) } backend->reserve = value; } + +static bool host_memory_backend_get_private (Object *o,Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(o); + + return backend->private; +} + +static void host_memory_backend_set_private (Object *o,bool value,Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(o); + + backend->private = value; +} #endif /* CONFIG_LINUX */static bool
backends/hostmem.c +4
-0@@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc, void *data) host_memory_backend_get_reserve, host_memory_backend_set_reserve); object_class_property_set_description(oc, "reserve", "Reserve swap space (or huge pages) if applicable"); + object_class_property_add_bool(oc, "private", + host_memory_backend_get_private, host_memory_backend_set_private); + object_class_property_set_description(oc, "private", + "Use KVM gmem private memory");#endif /*
CONFIG_LINUX*/
/*
* Do not delete/rename option. This option must be considered
stableinclude/sysemu/hostmem.h +1
-1@@ -65,7 +65,7 @@ struct HostMemoryBackend { /* protected */ uint64_t size; bool merge, dump, use_canonical_path; - bool prealloc, is_mapped, share, reserve; + bool prealloc, is_mapped, share, reserve, private; uint32_t prealloc_threads; ThreadContext *prealloc_context; DECLARE_BITMAP(host_nodes, MAX_NODES + 1);qapi/qom.json +0
-0qapi/qom.json line 610
Quoted 6 lines
qapi/qom.json +3
-0@@ -605,6 +605,9 @@ # @reserve: if true, reserve swap space (or huge pages) if applicable # (default: true) (since 6.1) # +# @private: if true, use KVM gmem private memory +# (default: false) (since 8.1) +#But that's not what any of this does.
This patch only adds a property and doesn't even explain what it intends to achieve with that.
Quoted 7 lines
Quoted 2 lines
How will it be used from a user? What will it affect internally? What will it modify in regards of the memory backend?
How it will be used is in the next patch, patch 09.
for
kvm_x86_sw_protected_vmtype VM, it will allocate private gmem with KVM ioctl if the memory backend has property "private" on.It feels wired up the wrong way.
When creating/initializing the memory backend, we should also take care of allocating the
gmem_fd, for example, by doing some gmem allocation callback, ideally *internally* creating the RAM memory region / RAMBlock.And we should fail if that is impossible (gmem does not apply to the VM) or creating the
gmem_fdfailed for other reason.
Quoted 6 lines
Quoted 3 lines
Like passing a
RAM_GMEMflag tomemory_region_init_ram_flags_nomigrate()inram_backend_memory_alloc(), to then handle it internally, failing if there is an error.KVM gmem is tied to VM. not before creating VM. We have to delay of the allocation of kvm gmem until VM initialization.
In vl.c, the flow is
1) Create machine: qemu_create_machine()
2) Configure KVM: configure_accelerators()
3) Create backends: qemu_create_late_backends()
Staring at object_create_early(), "memory-backend-" area always late.
So maybe, at the time memory backends are created+initialized, everything you need is already in place.
Quoted 6 lines
Hmm, one options is to move
gmem_fdfrom RAMBlock to KVMSlot. Handle the allocation of KVM gmem (issuing KVM gmem ioctl) there. i.e. inkvm_set_phys_mem()orkvm_region_add()(or whatever functions of KVM memory listener). Maybe we can dropram_block_convert_range()and can have KVM specific logic instead.
Might be doable as well.
Quoted 3 lines
We still need a way for user to specify which guest memory region is subject to KVM gmem, though.
Let's minimize the gmem hacks and come up with something clean.
Signature · 3 lines
Cheers, David / dhildenb
RFC PATCH 9/19 i386/kvm: Create gmem fd for KVM_X86_SW_PROTECTED_VM
In reply to Xiaoyao LiRegister a memory listener for KVM_X86_SW_PROVTED_VM. It creates gmem for the backend who sets the private property.
--- include/exec/memory.h | 1 + target/i386/kvm/kvm.c | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+)
include/exec/memory.h +1 -0
@@ -814,6 +814,7 @@ struct IOMMUMemoryRegion {#define MEMORY_LISTENER_PRIORITY_MIN 0#define MEMORY_LISTENER_PRIORITY_ACCEL 10#define MEMORY_LISTENER_PRIORITY_DEV_BACKEND 10 +#define MEMORY_LISTENER_PRIORITY_ACCEL_HIGH 20/** * struct MemoryListener: callbacks structure for updates to the physical memory map
target/i386/kvm/kvm.c +38 -0
@@ -37,6 +37,7 @@#include "hyperv-proto.h" #include "exec/gdbstub.h" +#include "exec/ramblock.h" #include "qemu/host-utils.h" #include "qemu/main-loop.h" #include "qemu/ratelimit.h"
@@ -2591,8 +2592,41 @@ static void register_smram_listener(Notifier *n, void *unused) &smram_address_space, 1, "kvm-smram"); } +static void kvm_x86_sw_protected_vm_region_add (MemoryListener *listenr, +MemoryRegionSection *section) +{ +MemoryRegion *mr = section->mr; +Object *owner =memory_region_owner (mr); +int fd; + +if (owner &&object_dynamic_cast (owner,TYPE_MEMORY_BACKEND ) && +object_property_get_bool (owner,"private" ,NULL ) && + mr->ram_block && mr->ram_block->gmem_fd <0 ) { +struct kvm_create_guest_memfd gmem = { + .size =memory_region_size (mr), +/* TODO: to decide whether KVM_GUEST_MEMFD_ALLOW_HUGEPAGE is supported */ + .flags =KVM_GUEST_MEMFD_ALLOW_HUGEPAGE , + }; + + fd =kvm_vm_ioctl (kvm_state,KVM_CREATE_GUEST_MEMFD , &gmem); +if (fd <0 ) { +error_report ("%s: error creating gmem: %s\n" , __func__,strerror (-fd)); +exit (1 ); + } +memory_region_set_gmem_fd (mr, fd); + } +} + +static MemoryListener kvm_x86_sw_protected_vm_memory_listener = { + .name ="kvm_x86_sw_protected_vm_memory_listener" , + .region_add = kvm_x86_sw_protected_vm_region_add, +/* Higher than KVM memory listener = 10. */ + .priority =MEMORY_LISTENER_PRIORITY_ACCEL_HIGH , +}; +int kvm_arch_init (MachineState *ms,KVMState *s) { +X86MachineState *x86ms =X86_MACHINE (ms);uint64_t identity_base =0xfffbc000 ;uint64_t shadow_mem;int ret;
@@ -2617,6 +2651,10 @@ int kvm_arch_init(MachineState *ms, KVMState *s)return ret; } +if (x86ms->vm_type ==KVM_X86_SW_PROTECTED_VM ) { +memory_listener_register (&kvm_x86_sw_protected_vm_memory_listener, &address_space_memory); + } +if (!kvm_check_extension (s,KVM_CAP_IRQ_ROUTING )) {error_report ("kvm: KVM_CAP_IRQ_ROUTING not supported by KVM" );return -ENOTSUP ;
Signature · 1 line
2.34.1
RFC PATCH 10/19 kvm: Introduce support for memory_attributes
In reply to Xiaoyao LiIntrocude the helper functions to set the attributes of a range of memory to private and shared.
--- accel/kvm/kvm-all.c | 43 +++++++++++++++++++++++++++++++++++++++++++ include/sysemu/kvm.h | 3 +++ 2 files changed, 46 insertions(+)
accel/kvm/kvm-all.c +43 -0
@@ -105,6 +105,7 @@ bool kvm_msi_use_devid;bool kvm_has_guest_debug;static int kvm_sstep_flags;static bool kvm_immediate_exit; +static uint64_t kvm_supported_memory_attributes;static hwaddr kvm_max_slot_size = ~0 ;static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -1343,6 +1344,44 @@ void kvm_set_max_memslot_size(hwaddr max_slot_size) kvm_max_slot_size = max_slot_size; } +static int kvm_set_memory_attributes (hwaddr start,hwaddr size,uint64_t attr) +{ +struct kvm_memory_attributes attrs; +int r; + + attrs.attributes = attr; + attrs.address = start; + attrs.size = size; + attrs.flags =0 ; + + r =kvm_vm_ioctl (kvm_state,KVM_SET_MEMORY_ATTRIBUTES , &attrs); +if (r) { +warn_report ("%s: failed to set memory (0x%lx+%#zx) with attr 0x%lx error '%s'" , + __func__, start, size, attr,strerror (errno)); + } +return r; +} + +int kvm_set_memory_attributes_private (hwaddr start,hwaddr size) +{ +if (!(kvm_supported_memory_attributes &KVM_MEMORY_ATTRIBUTE_PRIVATE )) { +error_report ("KVM doesn't support PRIVATE memory attribute\n" ); +return -EINVAL ; + } + +return kvm_set_memory_attributes (start, size,KVM_MEMORY_ATTRIBUTE_PRIVATE ); +} + +int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size) +{ +if (!(kvm_supported_memory_attributes &KVM_MEMORY_ATTRIBUTE_PRIVATE )) { +error_report ("KVM doesn't support PRIVATE memory attribute\n" ); +return -EINVAL ; + } + +return kvm_set_memory_attributes (start, size,0 ); +} +/* Called with KVMMemoryListener.slots_lock held */ static void kvm_set_phys_mem (KVMMemoryListener *kml,MemoryRegionSection *section,bool add)
@@ -2556,6 +2595,10 @@ static int kvm_init(MachineState *ms) } s->as = g_new0(struct KVMAs, s->nr_as); +if (kvm_check_extension (s,KVM_CAP_MEMORY_ATTRIBUTES )) { + kvm_supported_memory_attributes =kvm_ioctl (s,KVM_GET_SUPPORTED_MEMORY_ATTRIBUTES ,0 ); + } +if (object_property_find (OBJECT (current_machine),"kvm-type" )) { g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine), "kvm-type",
include/sysemu/kvm.h +3 -0
@@ -580,4 +580,7 @@ bool kvm_arch_cpu_check_are_resettable(void);bool kvm_dirty_ring_enabled (void );uint32_t kvm_dirty_ring_size (void ); + +int kvm_set_memory_attributes_private (hwaddr start,hwaddr size); +int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size);#endif
Signature · 1 line
2.34.1
RFC PATCH 11/19 kvm/memory: Introduce the infrastructure to set the default shared/private value
In reply to Xiaoyao LiIntroduce new flags for MemoryRegion to indicate the default attribute, private or not. And update the attribute if default private.
Originated-from: Isaku Yamahata <[email protected]>
--- accel/kvm/kvm-all.c | 10 ++++++++++ include/exec/memory.h | 6 ++++++ softmmu/memory.c | 13 +++++++++++++ 3 files changed, 29 insertions(+)
accel/kvm/kvm-all.c +10 -0
@@ -1487,6 +1487,16 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, strerror(-err));abort (); } + +if (memory_region_is_default_private (mr)) { + err =kvm_set_memory_attributes_private (start_addr, slot_size); +if (err) { +error_report ("%s: failed to set memory attribute private: %s\n" , + __func__,strerror (-err)); +exit (1 ); + } + } + start_addr += slot_size; ram_start_offset += slot_size; ram += slot_size;
include/exec/memory.h +6 -0
@@ -235,6 +235,9 @@ typedef struct IOMMUTLBEvent {/* RAM is an mmap-ed named file */ #define RAM_NAMED_FILE (1 << 9) +/* RAM is default private */ +#define RAM_DEFAULT_PRIVATE (1 << 10) + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end,
@@ -1689,6 +1692,9 @@ bool memory_region_is_protected(MemoryRegion *mr);*/ bool memory_region_can_be_private (MemoryRegion *mr); +void memory_region_set_default_private (MemoryRegion *mr); +bool memory_region_is_default_private (MemoryRegion *mr); +/** * memory_region_get_iommu: check whether a memory region is an iommu *
softmmu/memory.c +13 -0
@@ -1860,6 +1860,19 @@ bool memory_region_can_be_private(MemoryRegion *mr)return mr->ram_block && mr->ram_block->gmem_fd >=0 ; } +bool memory_region_is_default_private (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->gmem_fd >=0 && + (mr->ram_block->flags &RAM_DEFAULT_PRIVATE ); +} + +void memory_region_set_default_private (MemoryRegion *mr) +{ +if (mr->ram_block && mr->ram_block->gmem_fd >=0 ) { + mr->ram_block->flags |=RAM_DEFAULT_PRIVATE ; + } +} +uint8_t memory_region_get_dirty_log_mask (MemoryRegion *mr) {uint8_t mask = mr->dirty_log_mask;
Signature · 1 line
2.34.1
RFC PATCH 12/19 i386/kvm: Set memory to default private for KVM_X86_SW_PROTECTED_VM
In reply to Xiaoyao Li--- target/i386/kvm/kvm.c | 1 + 1 file changed, 1 insertion(+)
target/i386/kvm/kvm.c +1 -0
@@ -2614,6 +2614,7 @@ static void kvm_x86_sw_protected_vm_region_add(MemoryListener *listenr,exit (1 ); }memory_region_set_gmem_fd (mr, fd); +memory_region_set_default_private (mr); } }
Signature · 1 line
2.34.1
RFC PATCH 13/19 physmem: replace function name with __func__ in ram_block_discard_range()
In reply to Xiaoyao Li--- softmmu/physmem.c | 34 +++++++++++++++------------------- 1 file changed, 15 insertions(+), 19 deletions(-)
softmmu/physmem.c +15 -19
@@ -3430,16 +3430,15 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length)uint8_t *host_startaddr = rb->host + start;if (!QEMU_PTR_IS_ALIGNED (host_startaddr, rb->page_size)) { -error_report ("ram_block_discard_range: Unaligned start address: %p" , - host_startaddr); +error_report ("%s: Unaligned start address: %p" , + __func__, host_startaddr); goto err; }if ((start + length) <= rb->max_length) {bool need_madvise, need_fallocate;if (!QEMU_IS_ALIGNED (length, rb->page_size)) { -error_report ("ram_block_discard_range: Unaligned length: %zx" , - length); +error_report ("%s: Unaligned length: %zx" , __func__, length); goto err; }
@@ -3469,27 +3468,26 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length)* file. */ if (!qemu_ram_is_shared (rb)) { -warn_report_once ("ram_block_discard_range: Discarding RAM" +warn_report_once ("%s: Discarding RAM" " in private file mappings is possibly" " dangerous, because it will modify the" " underlying file and will affect other" -" users of the file" ); +" users of the file" , __func__); } ret =fallocate (rb->fd,FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE , start, length);if (ret) { ret = -errno; -error_report ("ram_block_discard_range: Failed to fallocate " -"%s:%" PRIx64" +%zx (%d)" , - rb->idstr, start, length, ret); +error_report ("%s: Failed to fallocate %s:%" PRIx64" +%zx (%d)" , + __func__, rb->idstr, start, length, ret); goto err; }#else ret = -ENOSYS ; -error_report ("ram_block_discard_range: fallocate not available/file" +error_report ("%s: fallocate not available/file" "%s:%" PRIx64" +%zx (%d)" , - rb->idstr, start, length, ret); + __func__, rb->idstr, start, length, ret); goto err;#endif }
@@ -3507,25 +3505,23 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length) }if (ret) { ret = -errno; -error_report ("ram_block_discard_range: Failed to discard range " +error_report ("%s: Failed to discard range " "%s:%" PRIx64" +%zx (%d)" , - rb->idstr, start, length, ret); + __func__, rb->idstr, start, length, ret); goto err; }#else ret = -ENOSYS ; -error_report ("ram_block_discard_range: MADVISE not available" -"%s:%" PRIx64" +%zx (%d)" , - rb->idstr, start, length, ret); +error_report ("%s: MADVISE not available %s:%" PRIx64" +%zx (%d)" , + __func__, rb->idstr, start, length, ret); goto err;#endif }trace_ram_block_discard_range (rb->idstr, host_startaddr, length, need_madvise, need_fallocate, ret); }else { -error_report ("ram_block_discard_range: Overrun block '%s' (%" PRIu64 -"/%zx/" RAM_ADDR_FMT ")" , - rb->idstr, start, length, rb->max_length); +error_report ("%s: Overrun block '%s' (%" PRIu64"/%zx/" RAM_ADDR_FMT ")" , + __func__, rb->idstr, start, length, rb->max_length); } err:
Signature · 1 line
2.34.1
RFC PATCH 14/19 physmem: Add ram_block_convert_range
In reply to Xiaoyao LiFrom: Chao Peng <[email protected]>
This new routine adds support for memory conversion between shared/private memory for gmem fd based private ram_block.
--- include/exec/cpu-common.h | 2 ++ softmmu/physmem.c | 61 ++++++++++++++++++++++++++++++--------- 2 files changed, 50 insertions(+), 13 deletions(-)
include/exec/cpu-common.h +2 -0
@@ -157,6 +157,8 @@ typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque);int qemu_ram_foreach_block (RAMBlockIterFunc func,void *opaque);int ram_block_discard_range (RAMBlock *rb,uint64_t start,size_t length); +int ram_block_convert_range (RAMBlock *rb,uint64_t start,size_t length, +bool shared_to_private);#endif
softmmu/physmem.c +48 -13
94 lines not on this page · raw message
Signature · 1 line
2.34.1
RFC PATCH 15/19 kvm: handle KVM_EXIT_MEMORY_FAULT
In reply to Xiaoyao LiFrom: Chao Peng <[email protected]>
Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when KVM_EXIT_MEMORY_FAULT happens. It indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.
Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has gmem memory backend.
--- accel/kvm/kvm-all.c | 52 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)
accel/kvm/kvm-all.c +52 -0
@@ -3040,6 +3040,48 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } +static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +{ +MemoryRegionSection section; +void *addr; +RAMBlock *rb; +ram_addr_t offset; +int ret =-1 ; + + section =memory_region_find (get_system_memory (), start, size); +if (!section.mr) { +return ret; + } + +if (memory_region_can_be_private (section.mr)) { +if (to_private) { + ret =kvm_set_memory_attributes_private (start, size); + }else { + ret =kvm_set_memory_attributes_shared (start, size); + } + +if (ret) { +return ret; + } + + addr =memory_region_get_ram_ptr (section.mr) + + section.offset_within_region; + rb =qemu_ram_block_from_host (addr, false, &offset); +/* +* With KVM_SET_MEMORY_ATTRIBUTES by kvm_set_memory_attributes(), +* operation on underlying file descriptor is only for releasing +* unnecessary pages. +*/ +ram_block_convert_range (rb, offset, size, to_private); + }else { +warn_report ("Convert non guest-memfd backed memory region (0x%" HWADDR_PRIx" ,+ 0x%" HWADDR_PRIx") to %s" , + start, size, to_private ?"private" :"shared" ); + } + +memory_region_unref (section.mr); +return ret; +} +int kvm_cpu_exec (CPUState *cpu) {struct kvm_run *run = cpu->kvm_run;
@@ -3198,6 +3240,16 @@ int kvm_cpu_exec(CPUState *cpu)break ; }break ; +case KVM_EXIT_MEMORY_FAULT : +if (run->memory.flags & ~KVM_MEMORY_EXIT_FLAG_PRIVATE ) { +error_report ("KVM_EXIT_MEMORY_FAULT: Unknown flag 0x%" PRIx64, + (uint64_t )run->memory.flags); + ret =-1 ; +break ; + } + ret =kvm_convert_memory (run->memory.gpa, run->memory.size, + run->memory.flags &KVM_MEMORY_EXIT_FLAG_PRIVATE ); +break ;default :DPRINTF ("kvm_arch_handle_exit\n" ); ret =kvm_arch_handle_exit (cpu, run);
Signature · 1 line
2.34.1
Re: [RFC PATCH 15/19] kvm: handle KVM_EXIT_MEMORY_FAULT
Quoted 41 lines · Xiaoyao Li
From: Chao Peng <[email protected]>
Currently only
KVM_MEMORY_EXIT_FLAG_PRIVATEin flags is valid whenKVM_EXIT_MEMORY_FAULThappens. It indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.Note,
KVM_EXIT_MEMORY_FAULTmakes sense only when the RAMBlock has gmem memory backend.Signed-off-by Chao Peng <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>--- accel/kvm/kvm-all.c | 52 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)accel/kvm/kvm-all.c +17
-0@@ -3040,6 +3040,48 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } +static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +{ +MemoryRegionSection section; +void *addr; +RAMBlock *rb; +ram_addr_t offset; +int ret =-1 ; + + section =memory_region_find (get_system_memory (), start, size); +if (!section.mr) { +return ret; + } + +if (memory_region_can_be_private (section.mr)) { +if (to_private) { + ret =kvm_set_memory_attributes_private (start, size); + }else {
accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 3069
Quoted 10 lines
accel/kvm/kvm-all.c +10
-0+ ret =kvm_set_memory_attributes_shared (start, size); + } + +if (ret) { +return ret; + } + + addr =memory_region_get_ram_ptr (section.mr) + + section.offset_within_region; + rb =qemu_ram_block_from_host (addr, false, &offset);
Here we have already section. section.mr->ram_block. We don't have to scan the existing RAMBlocks.
Except that, looks good to me.
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [RFC PATCH 15/19] kvm: handle KVM_EXIT_MEMORY_FAULT
Quoted 49 lines · Isaku Yamahata
Quoted 46 lines · Xiaoyao Li
From: Chao Peng <[email protected]>
Currently only
KVM_MEMORY_EXIT_FLAG_PRIVATEin flags is valid whenKVM_EXIT_MEMORY_FAULThappens. It indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.Note,
KVM_EXIT_MEMORY_FAULTmakes sense only when the RAMBlock has gmem memory backend.Signed-off-by Chao Peng <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>--- accel/kvm/kvm-all.c | 52 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)accel/kvm/kvm-all.c +22
-0@@ -3040,6 +3040,48 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } +static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +{ +MemoryRegionSection section; +void *addr; +RAMBlock *rb; +ram_addr_t offset; +int ret =-1 ; + + section =memory_region_find (get_system_memory (), start, size); +if (!section.mr) { +return ret; + } + +if (memory_region_can_be_private (section.mr)) { +if (to_private) { + ret =kvm_set_memory_attributes_private (start, size); + }else { + ret =kvm_set_memory_attributes_shared (start, size); + } + +if (ret) { +return ret;
Quoted 8 lines
accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 3069
Quoted 5 lines
accel/kvm/kvm-all.c +5
-0+ } + + addr =memory_region_get_ram_ptr (section.mr) + + section.offset_within_region; + rb =qemu_ram_block_from_host (addr, false, &offset);Here we have already section.
section.mr->ram_block. We don't have to scan the existing RAMBlocks.
But we don't have the @offset, do we?
Quoted 2 lines
Except that, looks good to me.
Reviewed-by Isaku Yamahata <[email protected]>
Re: [RFC PATCH 15/19] kvm: handle KVM_EXIT_MEMORY_FAULT
In reply to Xiaoyao LiQuoted 36 lines · Xiaoyao Li
From: Chao Peng <[email protected]>
Currently only
KVM_MEMORY_EXIT_FLAG_PRIVATEin flags is valid whenKVM_EXIT_MEMORY_FAULThappens. It indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.Note,
KVM_EXIT_MEMORY_FAULTmakes sense only when the RAMBlock has gmem memory backend.Signed-off-by Chao Peng <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>--- accel/kvm/kvm-all.c | 52 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)accel/kvm/kvm-all.c +12
-0@@ -3040,6 +3040,48 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } +static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +{ +MemoryRegionSection section; +void *addr; +RAMBlock *rb; +ram_addr_t offset; +int ret =-1 ; + + section =memory_region_find (get_system_memory (), start, size); +if (!section.mr) { +return ret; + }
accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 3064
Quoted 10 lines
accel/kvm/kvm-all.c +10
-0+ +if (memory_region_can_be_private (section.mr)) { +if (to_private) { + ret =kvm_set_memory_attributes_private (start, size); + }else { + ret =kvm_set_memory_attributes_shared (start, size); + } + +if (ret) { +return ret;
Should we unref the memory region before return?
Thanks,
Yilun
Quoted 19 lines
accel/kvm/kvm-all.c +19
-0+ } + + addr =memory_region_get_ram_ptr (section.mr) + + section.offset_within_region; + rb =qemu_ram_block_from_host (addr, false, &offset); +/* +* With KVM_SET_MEMORY_ATTRIBUTES by kvm_set_memory_attributes(), +* operation on underlying file descriptor is only for releasing +* unnecessary pages. +*/ +ram_block_convert_range (rb, offset, size, to_private); + }else { +warn_report ("Convert non guest-memfd backed memory region (0x%" HWADDR_PRIx" ,+ 0x%" HWADDR_PRIx") to %s" , + start, size, to_private ?"private" :"shared" ); + } + +memory_region_unref (section.mr); +return ret; +}
Re: [RFC PATCH 15/19] kvm: handle KVM_EXIT_MEMORY_FAULT
Quoted 41 lines · Xu Yilun
Quoted 40 lines · Xiaoyao Li
From: Chao Peng <[email protected]>
Currently only
KVM_MEMORY_EXIT_FLAG_PRIVATEin flags is valid whenKVM_EXIT_MEMORY_FAULThappens. It indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.Note,
KVM_EXIT_MEMORY_FAULTmakes sense only when the RAMBlock has gmem memory backend.Signed-off-by Chao Peng <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>--- accel/kvm/kvm-all.c | 52 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)accel/kvm/kvm-all.c +16
-0@@ -3040,6 +3040,48 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } +static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +{ +MemoryRegionSection section; +void *addr; +RAMBlock *rb; +ram_addr_t offset; +int ret =-1 ; + + section =memory_region_find (get_system_memory (), start, size); +if (!section.mr) { +return ret; + } + +if (memory_region_can_be_private (section.mr)) { +if (to_private) { + ret =kvm_set_memory_attributes_private (start, size);
Quoted 8 lines
accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 3064
Quoted 6 lines
accel/kvm/kvm-all.c +6
-0+ }else { + ret =kvm_set_memory_attributes_shared (start, size); + } + +if (ret) { +return ret;Should we unref the memory region before return?
Thanks for catching this!
Quoted 22 lines
Thanks,
YilunQuoted 19 lines
accel/kvm/kvm-all.c +19
-0+ } + + addr =memory_region_get_ram_ptr (section.mr) + + section.offset_within_region; + rb =qemu_ram_block_from_host (addr, false, &offset); +/* +* With KVM_SET_MEMORY_ATTRIBUTES by kvm_set_memory_attributes(), +* operation on underlying file descriptor is only for releasing +* unnecessary pages. +*/ +ram_block_convert_range (rb, offset, size, to_private); + }else { +warn_report ("Convert non guest-memfd backed memory region (0x%" HWADDR_PRIx" ,+ 0x%" HWADDR_PRIx") to %s" , + start, size, to_private ?"private" :"shared" ); + } + +memory_region_unref (section.mr); +return ret; +}
RFC PATCH 16/19 trace/kvm: Add trace for page convertion between shared and private
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
--- accel/kvm/kvm-all.c | 1 + accel/kvm/trace-events | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
accel/kvm/kvm-all.c +1 -0
@@ -3048,6 +3048,7 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)ram_addr_t offset;int ret =-1 ; +trace_kvm_convert_memory (start, size, to_private ?"shared_to_private" :"private_to_shared" ); section =memory_region_find (get_system_memory (), start, size);if (!section.mr) {return ret;
accel/kvm/trace-events +1 -1
@@ -25,4 +25,4 @@ kvm_dirty_ring_reaper(const char *s) "%s" kvm_dirty_ring_reap(uint64_t count, int64_t t) "reaped %"PRIu64" pages (took %"PRIi64" us)" kvm_dirty_ring_reaper_kick(const char *reason) "%s" kvm_dirty_ring_flush(int finished) "%d" - +kvm_convert_memory(uint64_t start, uint64_t size, const char *msg) "start 0x%" PRIx64 " size 0x%" PRIx64 " %s"
Signature · 1 line
2.34.1
RFC PATCH 17/19 pci-host/q35: Move PAM initialization above SMRAM initialization
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
In mch_realize(), process PAM initialization before SMRAM initialization so that later patch can skill all the SMRAM related with a single check.
--- hw/pci-host/q35.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
hw/pci-host/q35.c +10 -9
@@ -575,6 +575,16 @@ static void mch_realize(PCIDevice *d, Error **errp)/* setup pci memory mapping */ pc_pci_as_mapping_init (mch->system_memory, mch->pci_address_space); +/* PAM */ +init_pam (&mch->pam_regions[0 ],OBJECT (mch), mch->ram_memory, + mch->system_memory, mch->pci_address_space, +PAM_BIOS_BASE ,PAM_BIOS_SIZE ); +for (i =0 ; i <ARRAY_SIZE (mch->pam_regions) -1 ; ++i) { +init_pam (&mch->pam_regions[i +1 ],OBJECT (mch), mch->ram_memory, + mch->system_memory, mch->pci_address_space, +PAM_EXPAN_BASE + i *PAM_EXPAN_SIZE ,PAM_EXPAN_SIZE ); + } +/* if *disabled* show SMRAM to all CPUs */ memory_region_init_alias(&mch->smram_region, OBJECT(mch), "smram-region", mch->pci_address_space, MCH_HOST_BRIDGE_SMRAM_C_BASE,
@@ -641,15 +651,6 @@ static void mch_realize(PCIDevice *d, Error **errp)object_property_add_const_link (qdev_get_machine (),"smram" ,OBJECT (&mch->smram)); - -init_pam (&mch->pam_regions[0 ],OBJECT (mch), mch->ram_memory, - mch->system_memory, mch->pci_address_space, -PAM_BIOS_BASE ,PAM_BIOS_SIZE ); -for (i =0 ; i <ARRAY_SIZE (mch->pam_regions) -1 ; ++i) { -init_pam (&mch->pam_regions[i +1 ],OBJECT (mch), mch->ram_memory, - mch->system_memory, mch->pci_address_space, -PAM_EXPAN_BASE + i *PAM_EXPAN_SIZE ,PAM_EXPAN_SIZE ); - } }uint64_t mch_mcfg_base (void )
Signature · 1 line
2.34.1
RFC PATCH 18/19 q35: Introduce smm_ranges property for q35-pci-host
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
Add a q35 property to check whether or not SMM ranges, e.g. SMRAM, TSEG, etc... exist for the target platform. TDX doesn't support SMM and doesn't play nice with QEMU modifying related guest memory ranges.
--- hw/i386/pc_q35.c | 3 ++- hw/pci-host/q35.c | 42 +++++++++++++++++++++++++++------------ include/hw/i386/pc.h | 1 + include/hw/pci-host/q35.h | 1 + 4 files changed, 33 insertions(+), 14 deletions(-)
hw/i386/pc_q35.c +2 -1
@@ -232,8 +232,9 @@ static void pc_q35_init(MachineState *machine) x86ms->above_4g_mem_size, NULL);object_property_set_bool (phb,PCI_HOST_BYPASS_IOMMU , pcms->default_bus_bypass_iommu,NULL ); +object_property_set_bool (phb,PCI_HOST_PROP_SMM_RANGES , +x86_machine_is_smm_enabled (x86ms),NULL );sysbus_realize_and_unref (SYS_BUS_DEVICE (phb), &error_fatal); -/* pci */ host_bus =PCI_BUS (qdev_get_child_bus (DEVICE (phb),"pcie.0" )); pcms->bus = host_bus;
hw/pci-host/q35.c +29 -13
82 lines not on this page · raw message
include/hw/i386/pc.h +1 -0
@@ -156,6 +156,7 @@ void pc_guest_info_init(PCMachineState *pcms);#define PCI_HOST_PROP_PCI_HOLE64_SIZE "pci-hole64-size"#define PCI_HOST_BELOW_4G_MEM_SIZE "below-4g-mem-size"#define PCI_HOST_ABOVE_4G_MEM_SIZE "above-4g-mem-size" +#define PCI_HOST_PROP_SMM_RANGES "smm-ranges" void pc_pci_as_mapping_init(MemoryRegion *system_memory,
include/hw/pci-host/q35.h +1 -0
@@ -50,6 +50,7 @@ struct MCHPCIState {MemoryRegion tseg_blackhole, tseg_window;MemoryRegion smbase_blackhole, smbase_window;bool has_smram_at_smbase; +bool has_smm_ranges;Range pci_hole;uint64_t below_4g_mem_size;uint64_t above_4g_mem_size;
Signature · 1 line
2.34.1
RFC PATCH 19/19 i386: Disable SMM mode for X86_SW_PROTECTED_VM
In reply to Xiaoyao Li--- target/i386/kvm/kvm.c | 7 +++++++ 1 file changed, 7 insertions(+)
target/i386/kvm/kvm.c +7 -0
@@ -2654,6 +2654,13 @@ int kvm_arch_init(MachineState *ms, KVMState *s)if (x86ms->vm_type ==KVM_X86_SW_PROTECTED_VM ) {memory_listener_register (&kvm_x86_sw_protected_vm_memory_listener, &address_space_memory); + +if (x86ms->smm ==ON_OFF_AUTO_AUTO ) { + x86ms->smm =ON_OFF_AUTO_OFF ; + }else if (x86ms->smm ==ON_OFF_AUTO_ON ) { +error_report ("X86_SW_PROTECTED_VM doesn't support SMM" ); +return -EINVAL ; + } }if (!kvm_check_extension (s,KVM_CAP_IRQ_ROUTING )) {
Signature · 1 line
2.34.1
Re: [RFC PATCH 19/19] i386: Disable SMM mode for X86_SW_PROTECTED_VM
Quoted 12 lines · Xiaoyao Li
Signed-off-by Xiaoyao Li <[email protected]>--- target/i386/kvm/kvm.c | 7 +++++++ 1 file changed, 7 insertions(+)target/i386/kvm/kvm.c +0
-0@@ -2654,6 +2654,13 @@ int kvm_arch_init(MachineState *ms, KVMState *s)if (x86ms->vm_type ==KVM_X86_SW_PROTECTED_VM ) {
target/i386/kvm/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 2665
Quoted 10 lines
target/i386/kvm/kvm.c +7
-0memory_listener_register (&kvm_x86_sw_protected_vm_memory_listener, &address_space_memory); + +if (x86ms->smm ==ON_OFF_AUTO_AUTO ) { + x86ms->smm =ON_OFF_AUTO_OFF ; + }else if (x86ms->smm ==ON_OFF_AUTO_ON ) { +error_report ("X86_SW_PROTECTED_VM doesn't support SMM" ); +return -EINVAL ; + } }
If we use confidential guest support, this check should go to there.
Signature · 1 line
Isaku Yamahata <[email protected]>
Quoted 34 lines · Xiaoyao Li
This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of
KVM_X86_PROTECTED_VM.It adds the support to create a specific
KVM_X86_PROTECTED_VMtype VM, and introduces 'private' property for memory backend. When the vm type isKVM_X86_PROTECTED_VMand memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.$qemu -object memory-backend-ram,id=mem0,size=1G,private=on \ -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \ ...Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.
There are following design opens that need to be discussed:
1. how to determine the vm type?
a. like this series, specify the vm type via machine property
'kvm-type'b. check the memory backend, if any backend has 'private' property
set, the vm-type is set toKVM_X86_PROTECTED_VM.2. whether 'private' property is needed if we choose 1.b as design
with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.
3. What is
KVM_X86_SW_PROTECTED_VMgoing to look like? What's the purose of it and what's the requirement on it. I think it's the questions for KVM folks than QEMU folks.
Quoted 6 lines
Any other idea/open/question is welcomed.
Beside, TDX QEMU implemetation is based on this series to provide private gmem for TD private memory, which can be found at [2]. And it can work corresponding KVM [3] to boot TDX guest.
We already have a general purpose configuration mechanism for confidential guests. The -machine argument has a property confidential-guest-support=$OBJECT-ID, for pointing to an object that implements the TYPE_CONFIDENTIAL_GUEST_SUPPORT interface in QEMU. This is implemented with SEV, PPC PEF mode, and s390 protvirt.
I would expect TDX to follow this same design ie
qemu-system-x86_64 \
-object tdx-guest,id=tdx0,..... \
-machine q35,confidential-guest-support=tdx0 \
...and not require inventing the new 'kvm-type' attribute at least.
For the memory backend though, I'm not so sure - possibly that might be something that still wants an extra property to identify the type of memory to allocate, since we use memory-backend-ram for a variety of use cases. Or it could be an entirely new object type such as "memory-backend-gmem"
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 49 lines · Daniel P. Berrangé
Quoted 34 lines · Xiaoyao Li
This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of
KVM_X86_PROTECTED_VM.It adds the support to create a specific
KVM_X86_PROTECTED_VMtype VM, and introduces 'private' property for memory backend. When the vm type isKVM_X86_PROTECTED_VMand memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.$qemu -object memory-backend-ram,id=mem0,size=1G,private=on \ -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \ ...Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.
There are following design opens that need to be discussed:
1. how to determine the vm type?
a. like this series, specify the vm type via machine property
'kvm-type'
b. check the memory backend, if any backend has 'private' property
set, the vm-type is set toKVM_X86_PROTECTED_VM.2. whether 'private' property is needed if we choose 1.b as design
with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.
3. What is
KVM_X86_SW_PROTECTED_VMgoing to look like? What's the purose of it and what's the requirement on it. I think it's the questions for KVM folks than QEMU folks.Quoted 6 lines
Any other idea/open/question is welcomed.
Beside, TDX QEMU implemetation is based on this series to provide private gmem for TD private memory, which can be found at [2]. And it can work corresponding KVM [3] to boot TDX guest.
We already have a general purpose configuration mechanism for confidential guests. The -machine argument has a property confidential-guest-support=$OBJECT-ID, for pointing to an object that implements the
TYPE_CONFIDENTIAL_GUEST_SUPPORTinterface in QEMU. This is implemented with SEV, PPC PEF mode, and s390 protvirt.
Quoted 8 lines
I would expect TDX to follow this same design ie
qemu-system-x86_64 \ -object tdx-guest,id=tdx0,..... \ -machine q35,confidential-guest-support=tdx0 \ ...and not require inventing the new 'kvm-type' attribute at least.
yes.
TDX is initialized exactly as the above.
This RFC series introduces the 'kvm-type' for KVM_X86_SW_PROTECTED_VM. It's my fault that forgot to list the option of introducing sw_protected_vm object with CONFIDENTIAL_GUEST_SUPPORT interface. Thanks for Isaku to raise it https://lore.kernel.org/qemu-devel/[email protected]/
we can specify KVM_X86_SW_PROTECTED_VM this way:
qemu \
-object sw-protected,id=swp0,... \
-machine confidential-guest-support=swp0 \
...
Quoted 5 lines
For the memory backend though, I'm not so sure - possibly that might be something that still wants an extra property to identify the type of memory to allocate, since we use memory-backend-ram for a variety of use cases. Or it could be an entirely new object type such as "memory-backend-gmem"
What I want to discuss is whether providing the interface to users to allow them configuring which memory is/can be private. For example, QEMU can do it internally. If users wants a confidential guest, QEMU allocates private gmem for normal RAM automatically.
Quoted 81 lines · Xiaoyao Li
Quoted 49 lines · Daniel P. Berrangé
Quoted 34 lines · Xiaoyao Li
This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of
KVM_X86_PROTECTED_VM.It adds the support to create a specific
KVM_X86_PROTECTED_VMtype VM, and introduces 'private' property for memory backend. When the vm type isKVM_X86_PROTECTED_VMand memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.$qemu -object memory-backend-ram,id=mem0,size=1G,private=on \ -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \ ...Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.
There are following design opens that need to be discussed:
1. how to determine the vm type?
a. like this series, specify the vm type via machine property
'kvm-type'
b. check the memory backend, if any backend has 'private' property
set, the vm-type is set toKVM_X86_PROTECTED_VM.2. whether 'private' property is needed if we choose 1.b as design
with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.
3. What is
KVM_X86_SW_PROTECTED_VMgoing to look like? What's the purose of it and what's the requirement on it. I think it's the questions for KVM folks than QEMU folks.Quoted 6 lines
Any other idea/open/question is welcomed.
Beside, TDX QEMU implemetation is based on this series to provide private gmem for TD private memory, which can be found at [2]. And it can work corresponding KVM [3] to boot TDX guest.
We already have a general purpose configuration mechanism for confidential guests. The -machine argument has a property confidential-guest-support=$OBJECT-ID, for pointing to an object that implements the
TYPE_CONFIDENTIAL_GUEST_SUPPORTinterface in QEMU. This is implemented with SEV, PPC PEF mode, and s390 protvirt.Quoted 8 lines
I would expect TDX to follow this same design ie
qemu-system-x86_64 \ -object tdx-guest,id=tdx0,..... \ -machine q35,confidential-guest-support=tdx0 \ ...and not require inventing the new 'kvm-type' attribute at least.
yes.
TDX is initialized exactly as the above.
This RFC series introduces the 'kvm-type' for
KVM_X86_SW_PROTECTED_VM. It's
my fault that forgot to list the option of introducingsw_protected_vm
object withCONFIDENTIAL_GUEST_SUPPORTinterface.
Thanks for Isaku to raise it https://lore.kernel.org/qemu-devel/[email protected]/we can specify
KVM_X86_SW_PROTECTED_VMthis way:qemu \
-object sw-protected,id=swp0,... \
-machine confidential-guest-support=swp0 \
...Quoted 5 lines
For the memory backend though, I'm not so sure - possibly that might be something that still wants an extra property to identify the type of memory to allocate, since we use memory-backend-ram for a variety of use cases. Or it could be an entirely new object type such as "memory-backend-gmem"
Quoted 4 lines
What I want to discuss is whether providing the interface to users to allow them configuring which memory is/can be private. For example, QEMU can do it internally. If users wants a confidential guest, QEMU allocates private gmem for normal RAM automatically.
I think handling it automatically simplifies things a good deal on the QEMU side. I think it's still worthwhile to still allow:
-object memory-backend-memfd-private,...
because it provides a nice mechanism to set up a pair of shared/private memfd's to enable hole-punching via fallocate() to avoid doubling memory allocations for shared/private. It's also a nice place to control potentially-configurable things like:
- whether or not to enable discard/hole-punching
- if discard is enabled, whether or not to register the range via RamDiscardManager interface so that VFIO/IOMMU mappings get updated when doing PCI passthrough. SNP relies on this for PCI passthrough when discard is enabled, otherwise DMA occurs to stale mappings of discarded bounce-buffer pages:
https://github.com/AMDESE/qemu/blob/snp-latest/backends/hostmem-memfd-private.c#L449
But for other memory ranges, it doesn't do a lot of good to rely on users to control those via -object memory-backend-memfd-private, since QEMU will set up some regions internally, like the UEFI ROM.
It also isn't ideal for QEMU itself to internally control what should/shouldn't be set up with a backing guest_memfd, because some guest kernels do weird stuff, like scan for ROM regions in areas that guest kernels might have mapped as encrypted in guest page table. You can consider them to be guest bugs, but even current SNP-capable kernels exhibit this behavior and if the guest wants to do dumb stuff QEMU should let it.
But for these latter 2 cases, it doesn't make sense to attempt to do any sort of discarding of backing pages since it doesn't make sense to discard ROM pages.
So I think it makes sense to just set up the gmemfd automatically across the board internally, and keep memory-backend-memfd-private around purely as a way to control/configure discardable memory.
-Mike
Quoted 2 lines
Quoted 118 lines · Michael Roth via
Quoted 81 lines · Xiaoyao Li
Quoted 49 lines · Daniel P. Berrangé
Quoted 34 lines · Xiaoyao Li
This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of
KVM_X86_PROTECTED_VM.It adds the support to create a specific
KVM_X86_PROTECTED_VMtype VM, and introduces 'private' property for memory backend. When the vm type isKVM_X86_PROTECTED_VMand memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.$qemu -object memory-backend-ram,id=mem0,size=1G,private=on \ -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \ ...Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.
There are following design opens that need to be discussed:
1. how to determine the vm type?
a. like this series, specify the vm type via machine property
'kvm-type'
b. check the memory backend, if any backend has 'private' property
set, the vm-type is set toKVM_X86_PROTECTED_VM.2. whether 'private' property is needed if we choose 1.b as design
with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.
3. What is
KVM_X86_SW_PROTECTED_VMgoing to look like? What's the purose of it and what's the requirement on it. I think it's the questions for KVM folks than QEMU folks.Quoted 6 lines
Any other idea/open/question is welcomed.
Beside, TDX QEMU implemetation is based on this series to provide private gmem for TD private memory, which can be found at [2]. And it can work corresponding KVM [3] to boot TDX guest.
We already have a general purpose configuration mechanism for confidential guests. The -machine argument has a property confidential-guest-support=$OBJECT-ID, for pointing to an object that implements the
TYPE_CONFIDENTIAL_GUEST_SUPPORTinterface in QEMU. This is implemented with SEV, PPC PEF mode, and s390 protvirt.Quoted 8 lines
I would expect TDX to follow this same design ie
qemu-system-x86_64 \ -object tdx-guest,id=tdx0,..... \ -machine q35,confidential-guest-support=tdx0 \ ...and not require inventing the new 'kvm-type' attribute at least.
yes.
TDX is initialized exactly as the above.
This RFC series introduces the 'kvm-type' for
KVM_X86_SW_PROTECTED_VM. It's
my fault that forgot to list the option of introducingsw_protected_vm
object withCONFIDENTIAL_GUEST_SUPPORTinterface.
Thanks for Isaku to raise it https://lore.kernel.org/qemu-devel/[email protected]/we can specify
KVM_X86_SW_PROTECTED_VMthis way:qemu \
-object sw-protected,id=swp0,... \
-machine confidential-guest-support=swp0 \
...Quoted 5 lines
For the memory backend though, I'm not so sure - possibly that might be something that still wants an extra property to identify the type of memory to allocate, since we use memory-backend-ram for a variety of use cases. Or it could be an entirely new object type such as "memory-backend-gmem"
Quoted 4 lines
What I want to discuss is whether providing the interface to users to allow them configuring which memory is/can be private. For example, QEMU can do it internally. If users wants a confidential guest, QEMU allocates private gmem for normal RAM automatically.
I think handling it automatically simplifies things a good deal on the QEMU side. I think it's still worthwhile to still allow:
-object memory-backend-memfd-private,...
because it provides a nice mechanism to set up a pair of shared/private memfd's to enable hole-punching via
fallocate()to avoid doubling memory allocations for shared/private. It's also a nice place to control potentially-configurable things like:- whether or not to enable discard/hole-punching
- if discard is enabled, whether or not to register the range via RamDiscardManager interface so that VFIO/IOMMU mappings get updated when doing PCI passthrough. SNP relies on this for PCI passthrough when discard is enabled, otherwise DMA occurs to stale mappings of discarded bounce-buffer pages:https://github.com/AMDESE/qemu/blob/snp-latest/backends/hostmem-memfd-private.c#L449
But for other memory ranges, it doesn't do a lot of good to rely on users to control those via -object memory-backend-memfd-private, since QEMU will set up some regions internally, like the UEFI ROM.
It also isn't ideal for QEMU itself to internally control what should/shouldn't be set up with a backing
guest_memfd, because some guest kernels do weird stuff, like scan for ROM regions in areas that guest kernels might have mapped as encrypted in guest page table. You can consider them to be guest bugs, but even current SNP-capable kernels exhibit this behavior and if the guest wants to do dumb stuff QEMU should let it.
Quoted 7 lines
But for these latter 2 cases, it doesn't make sense to attempt to do any sort of discarding of backing pages since it doesn't make sense to discard ROM pages.
So I think it makes sense to just set up the gmemfd automatically across the board internally, and keep memory-backend-memfd-private around purely as a way to control/configure discardable memory.
I'm looking at the repo and31a7c7e36684 ("*hostmem-memfd-private: Initial discard manager support")
Do we have to implement RAM_DISCARD_MANGER at memory-backend-memfd-private? Can't we implement it at host_mem? The interface callbacks can have check "if (!private) return". Then we can support any host-mem backend.
Signature · 1 line
Isaku Yamahata <[email protected]>
Quoted 16 lines · Xiaoyao Li
This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of
KVM_X86_PROTECTED_VM.It adds the support to create a specific
KVM_X86_PROTECTED_VMtype VM, and introduces 'private' property for memory backend. When the vm type isKVM_X86_PROTECTED_VMand memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.$qemu -object memory-backend-ram,id=mem0,size=1G,private=on \ -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \ ...Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.
Quoted 8 lines
There are following design opens that need to be discussed:
1. how to determine the vm type?
a. like this series, specify the vm type via machine property
'kvm-type'b. check the memory backend, if any backend has 'private' property
set, the vm-type is set toKVM_X86_PROTECTED_VM.
Hi Xiaoyao. Because qemu has already confidential guest support, we should utilize it. Say,
qemu \
-object sw-protected, id=swp0, <more options for KVM_X86_SW_PROTECTED_VM> \
-machine confidential-guest-support=swp0
Quoted 4 lines
2. whether 'private' property is needed if we choose 1.b as design
with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.
Memory region property (how to create KVM memory slot) should be independent from underlying VM type. Some (e.g. TDX) may require KVM private memory slot, some may not. Leave the decision to its vm type backend. They can use qemu memory listener.
Signature · 1 line
Isaku Yamahata <[email protected]>
Quoted 28 lines · Isaku Yamahata
Quoted 16 lines · Xiaoyao Li
This is the first RFC version of enabling KVM gmem[1] as the backend for private memory of
KVM_X86_PROTECTED_VM.It adds the support to create a specific
KVM_X86_PROTECTED_VMtype VM, and introduces 'private' property for memory backend. When the vm type isKVM_X86_PROTECTED_VMand memory backend has private enabled as below, it will call KVM gmem ioctl to allocate private memory for the backend.$qemu -object memory-backend-ram,id=mem0,size=1G,private=on \ -machine q35,kvm-type=sw-protected-vm,memory-backend=mem0 \ ...Unfortunately this patch series fails the boot of OVMF at very early stage due to triple fault because KVM doesn't support emulate string IO to private memory. We leave it as an open to be discussed.
Quoted 8 lines
There are following design opens that need to be discussed:
1. how to determine the vm type?
a. like this series, specify the vm type via machine property
'kvm-type'
b. check the memory backend, if any backend has 'private' property
set, the vm-type is set toKVM_X86_PROTECTED_VM.
Quoted 5 lines
Hi Xiaoyao. Because qemu has already confidential guest support, we should utilize it. Say,
qemu \ -object sw-protected, id=swp0, <more options for KVM_X86_SW_PROTECTED_VM> \ -machine confidential-guest-support=swp0
thanks for pointing out this option. I thought of it and forgot to list it as option.
It seems better and I'll go this direction if no one has different opinion.
Quoted 11 lines
Quoted 5 lines
2. whether 'private' property is needed if we choose 1.b as design
with 1.b, QEMU can decide whether the memory region needs to be private (allocates gmem fd for it) or not, on its own.
Memory region property (how to create KVM memory slot) should be independent from underlying VM type. Some (e.g. TDX) may require KVM private memory slot, some may not. Leave the decision to its vm type backend. They can use qemu memory listener.
As I replied to Daniel, the topic is whether 'private' property is needed. Is it essential to let users decide which memory can be private? It seems OK that QEMU can make the decision based on VM type.