Enable encrypted guest memory access in QEMU
Subsystems (2)
cover Enable encrypted guest memory access in QEMU
From: Yuan Yao <[email protected]>
This RFC series introduces the basic framework and a common implementation on x86 to handle encrypted guest memory reading/writing, to support QEMU's built-in guest debugging features, like the monitor command xp and gdbstub.
The encrypted guest which its memory and/or register context is encrypted by vendor specific technology(AMD SEV/INTEL TDX), is able to resist the attack from malicious VMM or other privileged components in host side, however, this ability also breaks down the QEMU's built-in guest debugging features, because it prohibits the direct guest memory accessing (memcpy() with HVA) from QEMU which is the base of these debugging features.
The framework part based on the previous patche set from AMD[1] and some discussion result in community[2]. The main idea is, introduce some new debug interfaces to handle the encrypted guest physical memory accessing, also introduce new interfaces in MemoryRegion to handle the actual accessing there with KVM, don't bother the exist memory access logic or callbacks as far as possible.
[1] https://lore.kernel.org/qemu-devel/
[email protected]/ [2] https://lore.kernel.org/qemu-devel/ 20200922201124.GA6606@ashkalra_ubuntu_server/
- The difference part in this patch series:
- We introduce another new vm level ioctl focus on the encrypted guest memory accessing:
KVM_MEMORY_ENCRYPT_{READ,WRITE}_MEMORY
struct kvm_rw_memory rw;
rw.addr = gpa_OR_hva;
rw.buf = (__u64)src;
rw.len = len;
kvm_vm_ioctl(kvm_state,
KVM_MEMORY_ENCRYPT_{READ,WRITE}_MEMORY, &rw);
This new ioctl has more neutral and general name for its purpose, the debugging support of AMD SEV and INTEL TDX can be covered by a unify QEMU implementation on x86 with this ioctl. Although only INTEL TD guest is supported in this series, AMD SEV could be also supported with implementation of this ioctl in KVM, plus small modifications in QEMU to enable the unify part.
- The MemoryRegion interface introduced by AMD before now has addtional GPA parameter(only HVA before).
This is for INTEL TDX which uses GPA to do guest memory accessing. This change won't impact AMD SEV which is using HVA to access the guest memory.
- New APIs in QEMU:
- Physical memory accessing:
- cpu_physical_memory_rw_debug().
- cpu_physical_memory_read_debug().
- cpu_physical_memory_write_debug().
- x86_ldl_phys_debug().
- x86_ldq_phys_debug().
- Access from address_space:
- address_space_read_debug().
- address_space_write_rom_debug().
- Virtual memory accessing and page table walking:
- cpu_memory_rw_debug().
- x86_cpu_get_phys_page_attrs_encrypted_debug().
- New intrfaces in QEMU:
- MemoryDebugOps *physical_memory_debug_op
- For normal guest:
Just call the old exist memory RW functions.
- For encrypted guest:
Forward the request to MemoryRegion->ram_debug_ops
- MemoryRegionRAMReadWriteOps MemoryRegion::*ram_debug_ops
- For normal guest:
NULL and nobody use it.
- For encrypted guest:
Forward the request to common/vendor specific implementation.
- The relationship diagram of the APIs and interfaces:
+---------------------------------------------+
|x86_cpu_get_phys_page_attrs_encrypted_debug()|
+----------------------------------+----------+
|
+---------------------------------+ |
|cpu_physical_memory_rw_debug() | |
|cpu_physical_memory_read_debug() | |
|cpu_physical_memory_write_debug()| |
+----------------------+----------+ |
| |
+---------------------+ | +---------v----------+
|cpu_memory_rw_debug()| | |x86_ldl_phys_debug()|
+-------------------+-+ | |x86_ldq_phys_debug()|
| | +-------+------------+
| | |
| | |
+--------------------v---------v----------------v------------+
| MemoryDebugOps *physical_memory_debug_op |
+----------------------+--------------------------+----------+
| |
|Encrypted guest |Normal guest
| |
+--------------------v-----------------------+ |
|address_space_encrypted_memory_read_debug() | |
|address_space_encrypted_rom_write_debug() | |
+--------------------+-----------------------+ |
| |
| +---------------v----------+
| |address_space_read() |
| |address_space_write_rom() |
| +--------------------------+
|
+----------------v----------------+
| address_space_read_debug() |
| address_space_write_rom_debug() |
+----------------+----------------+
|
|
|
+----------------v----------------+
| MemoryRegionRAMReadWriteOps |
| MemoryRegion::*ram_debug_ops |
+--------+--------------+---------+
| |
| |Normal guest
| |
Encrypted guest| +---v-------------------+
| | NULL(nobody using it) |
| +-----------------------+
|
+---------v----------------------------+
| kvm_encrypted_guest_read_memory() |
| kvm_encrypted_guest_write_memory() |
+--------------------------------------+Ashish Kalra (2):
Introduce new MemoryDebugOps which hook into guest virtual and
physical memory debug interfaces such as cpu_memory_rw_debug, to allow vendor specific assist/hooks for debugging and delegating accessing the guest memory. This is required for example in case of AMD SEV platform where the guest memory is encrypted and a SEV specific debug assist/hook will be required to access the guest memory.
Add new address_space_read and address_space_write debug helper
interfaces which can be invoked by vendor specific guest memory debug assist/hooks to do guest RAM memory accesses using the added MemoryRegion callbacks.
Brijesh Singh (2):
Extend the MemTxAttrs to include a 'debug' flag. The flag can be used
as general indicator that operation was triggered by the debugger.
Currently, guest memory access for debugging purposes is performed
using memcpy(). Extend the 'struct MemoryRegion' to include new callbacks that can be used to override the use of memcpy() with something else.
Yuan Yao (6):
Introduce new interface KVMState::set_mr_debug_ops and its wrapper Implements the common MemoryRegion::ram_debug_ops for encrypted guests Set the RAM's MemoryRegion::debug_ops for INTEL TD guests Introduce debug version of physical memory read/write API Change the monitor and other commands and gdbstub to use the debug API Introduce new CPUClass::get_phys_page_attrs_debug implementation for
encrypted guests
accel/kvm/kvm-all.c | 17 +++++ accel/stubs/kvm-stub.c | 11 +++ dump/dump.c | 2 +- gdbstub.c | 4 +- hw/i386/pc.c | 4 + include/exec/cpu-common.h | 14 ++++ include/exec/memattrs.h | 4 + include/exec/memory.h | 54 +++++++++++++ include/sysemu/kvm.h | 5 ++ include/sysemu/tdx.h | 3 + monitor/misc.c | 12 ++- softmmu/cpus.c | 2 +- softmmu/physmem.c | 154 +++++++++++++++++++++++++++++++++++++- target/i386/cpu.h | 4 + target/i386/helper.c | 64 +++++++++++++--- target/i386/kvm/kvm.c | 68 +++++++++++++++++ target/i386/kvm/tdx.c | 21 ++++++ target/i386/monitor.c | 52 ++++++------- 18 files changed, 447 insertions(+), 48 deletions(-)
Signature · 1 line
2.20.1
RFC PATCH 1/10 Extend the MemTxAttrs to include a 'debug' flag. The flag can be used as general indicator that operation was triggered by the debugger.
From: Brijesh Singh <[email protected]>
A subsequent patch will set the debug=1 when issuing a memory access from the gdbstub or HMP commands. This is a prerequisite to support debugging an encrypted guest. When a request with debug=1 is seen, the encryption APIs will be used to access the guest memory.
include/exec/memattrs.h +2 -0
@@ -49,6 +49,8 @@ typedef struct MemTxAttrs { unsigned int target_tlb_bit0 : 1; unsigned int target_tlb_bit1 : 1; unsigned int target_tlb_bit2 : 1; +/* Memory access request from the debugger */ + unsigned int debug:1; } MemTxAttrs;/* Bus masters which don't specify any attributes will get this,
Signature · 1 line
2.20.1
RFC PATCH 2/10 Currently, guest memory access for debugging purposes is performed using memcpy(). Extend the 'struct MemoryRegion' to include new callbacks that can be used to override the use of memcpy() with something else.
In reply to Yuan YaoFrom: Brijesh Singh <[email protected]>
The new callbacks can be used to display the guest memory of an SEV guest by registering callbacks to the SEV memory encryption/decryption APIs.
Typical usage:
mem_read(uint8_t *dest,
const uint8_t *hva_src, hwaddr gpa_src,
uint32_t len, MemTxAttrs attrs);
mem_write(uint8_t *hva_dest, hwaddr gpa_des,
const uint8_t *src, uint32_t len, MemTxAttrs attrs);
MemoryRegionRAMReadWriteOps ops;
ops.read = mem_read;
ops.write = mem_write;
memory_region_init_ram(mem, NULL, "memory", size, NULL); memory_region_set_ram_debug_ops(mem, ops);
Yuan Yao:
- Add the gpa_src/gpa_des for read/write interface
include/exec/memory.h +28 -0
@@ -444,6 +444,19 @@ struct IOMMUMemoryRegionClass {typedef struct CoalescedMemoryRange CoalescedMemoryRange ;typedef struct MemoryRegionIoeventfd MemoryRegionIoeventfd ; +/* Memory Region RAM debug callback */ +typedef struct MemoryRegionRAMReadWriteOps MemoryRegionRAMReadWriteOps ; + +struct MemoryRegionRAMReadWriteOps { +/* Write data into guest memory */ +int (*write) (uint8_t *hva_dest,hwaddr gpa_des, +const uint8_t *src,uint32_t len,MemTxAttrs attrs); +/* Read data from guest memory */ +int (*read) (uint8_t *dest, +const uint8_t *hva_src,hwaddr gpa_src, +uint32_t len,MemTxAttrs attrs); +}; +/** MemoryRegion: * * A struct representing a memory region.
@@ -487,6 +500,7 @@ struct MemoryRegion {const char *name;unsigned ioeventfd_nb;MemoryRegionIoeventfd *ioeventfds; +const MemoryRegionRAMReadWriteOps *ram_debug_ops; };struct IOMMUMemoryRegion {
@@ -1130,6 +1144,20 @@ void memory_region_init_rom_nomigrate(MemoryRegion *mr,uint64_t size, Error **errp); +/** +* memory_region_set_ram_debug_ops: Set access ops for a give memory region. +* +* @mr: the #MemoryRegion to be initialized +* @ops: a function that will be used when accessing @target region during +* debug +*/ +static inline void +memory_region_set_ram_debug_ops (MemoryRegion *mr, +const MemoryRegionRAMReadWriteOps *ops) +{ + mr->ram_debug_ops = ops; +} +/** * memory_region_init_rom_device_nomigrate: Initialize a ROM memory region. * Writes are handled via callbacks.
Signature · 1 line
2.20.1
RFC PATCH 3/10 Introduce new interface KVMState::set_mr_debug_ops and its wrapper
In reply to Yuan YaoFrom: Yuan Yao <[email protected]>
This interface is designed to setup the MemoryRegion::debug_ops.
Also introduced 2 wrapper functions for installing/calling the KVMState::set_mr_debug_ops from different targets easily.
accel/kvm/kvm-all.c +17 -0
@@ -129,6 +129,8 @@ struct KVMStateKVMMemoryListener *ml;AddressSpace *as; } *as; + +set_memory_region_debug_ops set_mr_debug_ops; };KVMState *kvm_state;
@@ -3157,6 +3159,21 @@ static void kvm_set_kernel_irqchip(Object *obj, Visitor *v, } } +void kvm_setup_memory_region_debug_ops (struct KVMState *s, +set_memory_region_debug_ops new_ops) +{ +if (s) + s->set_mr_debug_ops = new_ops; +} + +void kvm_set_memory_region_debug_ops (void *handle,MemoryRegion *mr) +{ +if (!kvm_state || !kvm_state->set_mr_debug_ops) +return ; + + kvm_state->set_mr_debug_ops (handle, mr); +} +bool kvm_kernel_irqchip_allowed (void ) {return kvm_state->kernel_irqchip_allowed;
accel/stubs/kvm-stub.c +11 -0
@@ -148,4 +148,15 @@ bool kvm_arm_supports_user_irq(void) {return false; } + +void kvm_setup_memory_region_debug_ops (struct KVMState *s, +set_memory_region_debug_ops new_ops) +{ + +} + +void kvm_set_memory_region_debug_ops (void *handle,MemoryRegion *mr) +{ + +}#endif
include/sysemu/kvm.h +5 -0
@@ -547,4 +547,9 @@ bool kvm_cpu_check_are_resettable(void);bool kvm_arch_cpu_check_are_resettable (void ); +typedef void (*set_memory_region_debug_ops )(void *handle,MemoryRegion *mr); +void kvm_setup_memory_region_debug_ops (struct KVMState *s,set_memory_region_debug_ops new_ops); + +void kvm_set_memory_region_debug_ops (void *handle,MemoryRegion *mr); +#endif
Signature · 1 line
2.20.1
RFC PATCH 4/10 Implements the common MemoryRegion::ram_debug_ops for encrypted guests
In reply to Yuan YaoFrom: Yuan Yao <[email protected]>
The new functions are added into target/i386/kvm/kvm.c as common functions to support encrypted guest for KVM on x86.
Now we enable these only for INTEL TD guests.
target/i386/kvm/kvm.c +66 -0
@@ -134,6 +134,9 @@ static struct kvm_msr_list *kvm_feature_msrs;static int vm_type; +void kvm_encrypted_guest_set_memory_region_debug_ops (void *handle, +MemoryRegion *mr); +int kvm_set_vm_type (MachineState *ms,int kvm_type) { if (kvm_type == KVM_X86_LEGACY_VM ||
@@ -2228,6 +2231,10 @@ int kvm_arch_init(MachineState *ms, KVMState *s)return ret; } +if (kvm_tdx_enabled ()) +kvm_setup_memory_region_debug_ops (s, + kvm_encrypted_guest_set_memory_region_debug_ops); +if (!kvm_check_extension (s,KVM_CAP_IRQ_ROUTING )) {error_report ("kvm: KVM_CAP_IRQ_ROUTING not supported by KVM" );return -ENOTSUP ;
@@ -4917,3 +4924,62 @@ bool kvm_arch_cpu_check_are_resettable(void) {return !sev_es_enabled (); } + +static int kvm_encrypted_guest_read_memory (uint8_t *dest, +const uint8_t *hva_src,hwaddr gpa_src, +uint32_t len,MemTxAttrs attrs) +{ +struct kvm_rw_memory rw; + +/* +TODO: +Can we check SEV/TDX state to decide use +gpa_dest or hva_dest here ? + +Also how shall we handle the kvm_vm_ioctl failure case ? +Some user like cpu_physical_memory_{read,write}() doesn't handle such +failure, because for non-encrypted guest these functions may do memory +reading/wrting with memcpy() dirctly before. +May memset() the buffer to a bad pattern (all 0x0 or 0xff) +for indicating this ? +*/ + rw.addr = gpa_src; + rw.buf = dest; + rw.len = len; + +return kvm_vm_ioctl (kvm_state,KVM_MEMORY_ENCRYPT_READ_MEMORY , &rw); +} + +static int kvm_encrypted_guest_write_memory (uint8_t *hva_dest,hwaddr gpa_dest, +const uint8_t *src, +uint32_t len,MemTxAttrs attrs) +{ +struct kvm_rw_memory rw; + +/* +TODO: +Can we check SEV/TDX state to decide use +gpa_dest or hva_dest here ? + +Also how shall we handle the kvm_vm_ioctl failure case ? +Some user like cpu_physical_memory_{read,write}() doesn't handle such +failure, because for non-encrypted guest these functions may do memory +reading/wrting with memcpy() dirctly before. +*/ + rw.addr = gpa_dest; + rw.buf = (void *)src; + rw.len = len; + +return kvm_vm_ioctl (kvm_state,KVM_MEMORY_ENCRYPT_WRITE_MEMORY , &rw); +} + +static MemoryRegionRAMReadWriteOps kvm_encrypted_guest_mr_debug_ops = { + .read = kvm_encrypted_guest_read_memory, + .write = kvm_encrypted_guest_write_memory, +}; + +void kvm_encrypted_guest_set_memory_region_debug_ops (void *handle, +MemoryRegion *mr) +{ +memory_region_set_ram_debug_ops (mr, &kvm_encrypted_guest_mr_debug_ops); +}
Signature · 1 line
2.20.1
RFC PATCH 5/10 Set the RAM's MemoryRegion::debug_ops for INTEL TD guests
In reply to Yuan YaoFrom: Yuan Yao <[email protected]>
Now only set the RAM's debug_ops for INTEL TD guests, SEV can also rely on the common part introduced in previous patch or introduce new debug_ops implementation if it's necessary.
hw/i386/pc.c +4 -0
@@ -60,6 +60,7 @@#include "sysemu/xen.h" #include "sysemu/reset.h" #include "sysemu/runstate.h" +#include "sysemu/tdx.h" #include "kvm/kvm_i386.h" #include "hw/xen/xen.h" #include "hw/xen/start_info.h"
@@ -992,6 +993,9 @@ void pc_memory_init(PCMachineState *pcms,/* Init ACPI memory hotplug IO base address */ pcms->memhp_io_base =ACPI_MEMORY_HOTPLUG_BASE ; + +if (tdx_debug_enabled (machine->cgs)) +kvm_set_memory_region_debug_ops (NULL , *ram_memory); }/*
include/sysemu/tdx.h +3 -0
@@ -16,4 +16,7 @@ void tdx_post_init_vcpu(CPUState *cpu);struct TDXCapability ;struct TDXCapability *tdx_get_capabilities (void ); +struct ConfidentialGuestSupport ; +bool tdx_debug_enabled (ConfidentialGuestSupport *cgs); +#endif
target/i386/kvm/tdx.c +15 -0
@@ -384,3 +384,18 @@ static void tdx_guest_finalize(Object *obj)static void tdx_guest_class_init (ObjectClass *oc,void *data) { } + +bool tdx_debug_enabled (ConfidentialGuestSupport *cgs) +{ +TdxGuest *tdx; + +if (!cgs) +return false; + + tdx = (TdxGuest *)object_dynamic_cast (OBJECT (cgs), +TYPE_TDX_GUEST ); +if (!tdx) +return false; + +return tdx->debug; +}
Signature · 1 line
2.20.1
RFC PATCH 6/10 Introduce new MemoryDebugOps which hook into guest virtual and physical memory debug interfaces such as cpu_memory_rw_debug, to allow vendor specific assist/hooks for debugging and delegating accessing the guest memory. This is required for example in case of AMD SEV platform where the guest memory is encrypted and a SEV specific debug assist/hook will be required to access the guest memory.
In reply to Yuan YaoFrom: Ashish Kalra <[email protected]>
The MemoryDebugOps are used by cpu_memory_rw_debug() and default to address_space_read and address_space_write_rom.
Yuan Yao: Exports the physical_memory_debug_ops variable for functions in target/i386/helper.c
include/exec/memory.h +14 -0
@@ -2498,6 +2498,20 @@ MemTxResult address_space_write_cached_slow(MemoryRegionCache *cache, hwaddr addr, const void *buf, hwaddr len); +typedef struct MemoryDebugOps { +MemTxResult (*read)(AddressSpace *as,hwaddr phys_addr, +MemTxAttrs attrs,void *buf, +hwaddr len); +MemTxResult (*write)(AddressSpace *as,hwaddr phys_addr, +MemTxAttrs attrs,const void *buf, +hwaddr len); +}MemoryDebugOps ; + +// Export for functions in target/i386/helper.c +extern const MemoryDebugOps *physical_memory_debug_ops; + +void address_space_set_debug_ops (const MemoryDebugOps *ops); +static inline bool memory_access_is_direct (MemoryRegion *mr,bool is_write) {if (is_write) {
softmmu/physmem.c +22 -4
@@ -171,6 +171,18 @@ struct DirtyBitmapSnapshot {unsigned long dirty[]; }; +static const MemoryDebugOps default_debug_ops = { + .read = address_space_read, + .write = address_space_write_rom +}; + +const MemoryDebugOps *physical_memory_debug_ops = &default_debug_ops; + +void address_space_set_debug_ops (const MemoryDebugOps *ops) +{ + physical_memory_debug_ops = ops; +} +static void phys_map_node_reserve (PhysPageMap *map,unsigned nodes) {static unsigned alloc_hint =16 ;
@@ -3396,6 +3408,10 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr, page = addr &TARGET_PAGE_MASK ; phys_addr =cpu_get_phys_page_attrs_debug (cpu, page, &attrs); asidx =cpu_asidx_from_attrs (cpu, attrs); + +/* set debug attrs to indicate memory access is from the debugger */ + attrs.debug =1 ; +/* if no physical page mapped, return an error */ if (phys_addr ==-1 )return -1 ;
@@ -3404,11 +3420,13 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr, l = len; phys_addr += (addr & ~TARGET_PAGE_MASK );if (is_write) { - res =address_space_write_rom (cpu->cpu_ases[asidx].as, phys_addr, - attrs, buf, l); + res = physical_memory_debug_ops->write (cpu->cpu_ases[asidx].as, + phys_addr, + attrs, buf, l); }else { - res =address_space_read (cpu->cpu_ases[asidx].as, phys_addr, - attrs, buf, l); + res = physical_memory_debug_ops->read (cpu->cpu_ases[asidx].as, + phys_addr, + attrs, buf, l); }if (res !=MEMTX_OK ) {return -1 ;
Signature · 1 line
2.20.1
RFC PATCH 7/10 Add new address_space_read and address_space_write debug helper interfaces which can be invoked by vendor specific guest memory debug assist/hooks to do guest RAM memory accesses using the added MemoryRegion callbacks.
In reply to Yuan YaoFrom: Ashish Kalra <[email protected]>
Yuan Yao:
- Fixed fuzz_dma_read_cb() parameter issue for QEMU 5.2.91.
- Move the caller of encrypted_memory_debug_ops into phymem.c as common callbacks for encrypted guests.
- Adapted address_space_read_debug/address_space_wirte_rom_debug with new definition of MemoryRegion::ram_debug_ops;
- Install the encrypted_memory_debug_ops/phymem.c for INTEL TD guest.
include/exec/memory.h +12 -0
@@ -2246,6 +2246,12 @@ MemTxResult address_space_write_rom(AddressSpace *as, hwaddr addr,MemTxAttrs attrs, const void *buf, hwaddr len); +MemTxResult address_space_write_rom_debug (AddressSpace *as, +hwaddr addr, +MemTxAttrs attrs, +const void *ptr, +hwaddr len); +/* address_space_ld*: load from an address space * address_space_st*: store to an address space *
@@ -2512,6 +2518,8 @@ extern const MemoryDebugOps *physical_memory_debug_ops;void address_space_set_debug_ops (const MemoryDebugOps *ops); +void set_encrypted_memory_debug_ops (void ); +static inline bool memory_access_is_direct (MemoryRegion *mr,bool is_write) {if (is_write) {
@@ -2567,6 +2575,10 @@ MemTxResult address_space_read(AddressSpace *as, hwaddr addr,return result; } +MemTxResult address_space_read_debug (AddressSpace *as,hwaddr addr, +MemTxAttrs attrs,void *buf, +hwaddr len); +/** * address_space_read_cached: read from a cached RAM region *
softmmu/physmem.c +115 -0
127 lines not on this page · raw message
target/i386/kvm/kvm.c +3 -1
@@ -2231,9 +2231,11 @@ int kvm_arch_init(MachineState *ms, KVMState *s)return ret; } -if (kvm_tdx_enabled ()) +if (kvm_tdx_enabled ()) {kvm_setup_memory_region_debug_ops (s, kvm_encrypted_guest_set_memory_region_debug_ops); +set_encrypted_memory_debug_ops (); + }if (!kvm_check_extension (s,KVM_CAP_IRQ_ROUTING )) {error_report ("kvm: KVM_CAP_IRQ_ROUTING not supported by KVM" );
Signature · 1 line
2.20.1
RFC PATCH 8/10 Introduce debug version of physical memory read/write API
In reply to Yuan YaoFrom: Yuan Yao <[email protected]>
Add below APIs for reading/writing the physical memory, subsequent patch will use them in monitor commands and gdbstub to support encrypted guest debugging.
uint32_t x86_ldl_phys_debug(CPUState *cs, hwaddr addr); uint64_t x86_ldq_phys_debug(CPUState *cs, hwaddr addr); void cpu_physical_memory_rw_debug(hwaddr addr, void *buf,
hwaddr len, bool is_write);
void cpu_physical_memory_read_debug(hwaddr addr,
void *buf,
hwaddr len);void cpu_physical_memory_write_debug(hwaddr addr,
const void *buf, hwaddr len);
include/exec/cpu-common.h +14 -0
@@ -69,6 +69,8 @@ size_t qemu_ram_pagesize_largest(void);void cpu_physical_memory_rw (hwaddr addr,void *buf,hwaddr len,bool is_write); +void cpu_physical_memory_rw_debug (hwaddr addr,void *buf, +hwaddr len,bool is_write);static inline void cpu_physical_memory_read (hwaddr addr,void *buf,hwaddr len) {
@@ -79,6 +81,18 @@ static inline void cpu_physical_memory_write(hwaddr addr, {cpu_physical_memory_rw (addr, (void *)buf, len, true); } + +static inline void cpu_physical_memory_read_debug (hwaddr addr, +void *buf,hwaddr len) +{ +cpu_physical_memory_rw_debug (addr, buf, len, false); +} +static inline void cpu_physical_memory_write_debug (hwaddr addr, +const void *buf,hwaddr len) +{ +cpu_physical_memory_rw_debug (addr, (void *)buf, len, true); +} +void *cpu_physical_memory_map (hwaddr addr,hwaddr *plen,bool is_write);
include/exec/memattrs.h +3 -0
@@ -60,6 +60,9 @@ typedef struct MemTxAttrs {*/ #define MEMTXATTRS_UNSPECIFIED ((MemTxAttrs) { .unspecified = 1 }) +// Same as MEMTXATTRS_UNSPECIFIED but enable debug +#define MEMTXATTRS_UNSPECIFIED_DEBUG ((MemTxAttrs) { .unspecified = 1, .debug = 1 }) +/* New-style MMIO accessors can indicate that the transaction failed. * A zero (MEMTX_OK) response means success; anything else is a failure * of some kind. The memory subsystem will bitwise-OR together results
softmmu/physmem.c +13 -0
@@ -2910,6 +2910,19 @@ void cpu_physical_memory_rw(hwaddr addr, void *buf, buf, len, is_write); } +void cpu_physical_memory_rw_debug (hwaddr addr,void *buf, +hwaddr len,bool is_write) +{ +if (is_write) + physical_memory_debug_ops->write (&address_space_memory, + addr,MEMTXATTRS_UNSPECIFIED_DEBUG , + buf, len); +else + physical_memory_debug_ops->read (&address_space_memory, + addr,MEMTXATTRS_UNSPECIFIED_DEBUG , + buf, len); +} +enum write_rom_type {WRITE_DATA ,FLUSH_CACHE ,
target/i386/cpu.h +2 -0
@@ -1960,6 +1960,8 @@ void x86_stl_phys_notdirty(CPUState *cs, hwaddr addr, uint32_t val);void x86_stw_phys (CPUState *cs,hwaddr addr,uint32_t val);void x86_stl_phys (CPUState *cs,hwaddr addr,uint32_t val);void x86_stq_phys (CPUState *cs,hwaddr addr,uint64_t val); +uint32_t x86_ldl_phys_debug (CPUState *cs,hwaddr addr); +uint64_t x86_ldq_phys_debug (CPUState *cs,hwaddr addr);#endif /* will be suppressed */
target/i386/helper.c +26 -0
@@ -663,4 +663,30 @@ void x86_stq_phys(CPUState *cs, hwaddr addr, uint64_t val)address_space_stq (as, addr, val, attrs,NULL ); } + +uint32_t x86_ldl_phys_debug (CPUState *cs,hwaddr addr) +{ +uint32_t ret; +MemTxAttrs attrs =MEMTXATTRS_UNSPECIFIED_DEBUG ; +int as_id =cpu_asidx_from_attrs (cs, attrs); +struct AddressSpace *as =cpu_get_address_space (cs, as_id); + + physical_memory_debug_ops->read (as, addr, attrs, + &ret,sizeof (ret)); + +return tswap32 (ret); +} + +uint64_t x86_ldq_phys_debug (CPUState *cs,hwaddr addr) +{ +uint64_t ret; +MemTxAttrs attrs =MEMTXATTRS_UNSPECIFIED_DEBUG ; +int as_id =cpu_asidx_from_attrs (cs, attrs); +struct AddressSpace *as =cpu_get_address_space (cs, as_id); + + physical_memory_debug_ops->read (as, addr, attrs, + &ret,sizeof (ret)); + +return tswap64 (ret); +}#endif
Signature · 1 line
2.20.1
RFC PATCH 9/10 Change the monitor and other commands and gdbstub to use the debug API
In reply to Yuan YaoFrom: Yuan Yao <[email protected]>
Please comment if some changes are incorrect or I missed something here.
dump/dump.c +1 -1
@@ -1746,7 +1746,7 @@ static void dump_init(DumpState *s, int fd, bool has_format,warn_report ("guest note format is unsupported: %" PRIu16, format); }else { s->guest_note =g_malloc (size +1 );/* +1 for adding \0 */ -cpu_physical_memory_read (addr, s->guest_note, size); +cpu_physical_memory_read_debug (addr, s->guest_note, size);get_note_sizes (s, s->guest_note,NULL , &name_size, &desc_size); s->guest_note_size = ELF_NOTE_SIZE(note_head_size, name_size,
gdbstub.c +2 -2
@@ -71,9 +71,9 @@ static inline int target_memory_rw_debug(CPUState *cpu, target_ulong addr,#ifndef CONFIG_USER_ONLY if (phy_memory_mode) {if (is_write) { -cpu_physical_memory_write (addr, buf, len); +cpu_physical_memory_write_debug (addr, buf, len); }else { -cpu_physical_memory_read (addr, buf, len); +cpu_physical_memory_read_debug (addr, buf, len); }return 0 ; }
include/exec/memattrs.h +0 -1
@@ -59,7 +59,6 @@ typedef struct MemTxAttrs {* from "didn't specify" if necessary). */ #define MEMTXATTRS_UNSPECIFIED ((MemTxAttrs) { .unspecified = 1 }) -// Same as MEMTXATTRS_UNSPECIFIED but enable debug #define MEMTXATTRS_UNSPECIFIED_DEBUG ((MemTxAttrs) { .unspecified = 1, .debug = 1 })
monitor/misc.c +8 -4
@@ -587,8 +587,9 @@ static void memory_dump(Monitor *mon, int count, int format, int wsize, l = line_size;if (is_physical) {AddressSpace *as = cs ? cs->as : &address_space_memory; -MemTxResult r =address_space_read (as, addr, -MEMTXATTRS_UNSPECIFIED , buf, l); +MemTxResult r =address_space_read_debug (as, addr, +MEMTXATTRS_UNSPECIFIED_DEBUG , + buf, l);if (r !=MEMTX_OK ) {monitor_printf (mon," Cannot access memory\n" );break ;
@@ -825,11 +826,14 @@ static void hmp_sum(Monitor *mon, const QDict *qdict)uint16_t sum;uint32_t start =qdict_get_int (qdict,"start" );uint32_t size =qdict_get_int (qdict,"size" ); +uint8_t val; sum =0 ;for (addr = start; addr < (start + size); addr++) { -uint8_t val =address_space_ldub (&address_space_memory, addr, -MEMTXATTRS_UNSPECIFIED ,NULL ); +address_space_read_debug (&address_space_memory, + addr, +MEMTXATTRS_UNSPECIFIED_DEBUG , + &val,sizeof (val));/* BSD sum algorithm ('sum' Unix command) */ sum = (sum >>1 ) | (sum <<15 ); sum += val;
softmmu/cpus.c +1 -1
@@ -785,7 +785,7 @@ void qmp_pmemsave(int64_t addr, int64_t size, const char *filename, l =sizeof (buf);if (l > size) l = size; -cpu_physical_memory_read (addr, buf, l); +cpu_physical_memory_read_debug (addr, buf, l);if (fwrite (buf,1 , l, f) != l) {error_setg (errp,QERR_IO_ERROR ); goto exit;
target/i386/monitor.c +26 -26
192 lines not on this page · raw message
Signature · 1 line
2.20.1
RFC PATCH 10/10 Introduce new CPUClass::get_phys_page_attrs_debug implementation for encrypted guests
In reply to Yuan YaoFrom: Yuan Yao <[email protected]>
Add new function x86_cpu_get_phys_page_attrs_encrypted_debug() to walking guset page tables to do VA -> PA translation for encrypted guests.
Now install this to cc->get_phys_page_attrs_debug for INTEL TD guests only.
target/i386/cpu.h +2 -0
@@ -1807,6 +1807,8 @@ void x86_cpu_dump_state(CPUState *cs, FILE *f, int flags);hwaddr x86_cpu_get_phys_page_attrs_debug (CPUState *cpu,vaddr addr,MemTxAttrs *attrs); +hwaddr x86_cpu_get_phys_page_attrs_encrypted_debug (CPUState *cs,vaddr addr, +MemTxAttrs *attrs);int x86_cpu_gdb_read_register (CPUState *cpu,GByteArray *buf,int reg);int x86_cpu_gdb_write_register (CPUState *cpu,uint8_t *buf,int reg);
target/i386/helper.c +28 -10
@@ -205,8 +205,10 @@ void cpu_x86_update_cr4(CPUX86State *env, uint32_t new_cr4) }#if !defined(CONFIG_USER_ONLY ) -hwaddr x86_cpu_get_phys_page_attrs_debug (CPUState *cs,vaddr addr, -MemTxAttrs *attrs) +static hwaddr x86_cpu_get_phys_page_attrs_debug_internal (CPUState *cs,vaddr addr, +MemTxAttrs *attrs, +uint64_t (*ldq_phys)(CPUState *,hwaddr ), +uint32_t (*ldl_phys)(CPUState *,hwaddr )) {X86CPU *cpu =X86_CPU (cs);CPUX86State *env = &cpu->env;
@@ -242,7 +244,7 @@ hwaddr x86_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,if (la57) { pml5e_addr = ((env->cr[3 ] & ~0xfff ) + (((addr >>48 ) &0x1ff ) <<3 )) & a20_mask; - pml5e =x86_ldq_phys (cs, pml5e_addr); + pml5e =ldq_phys (cs, pml5e_addr);if (!(pml5e &PG_PRESENT_MASK )) {return -1 ; }
@@ -252,13 +254,13 @@ hwaddr x86_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr, pml4e_addr = ((pml5e &PG_ADDRESS_MASK ) + (((addr >>39 ) &0x1ff ) <<3 )) & a20_mask; - pml4e =x86_ldq_phys (cs, pml4e_addr); + pml4e =ldq_phys (cs, pml4e_addr);if (!(pml4e &PG_PRESENT_MASK )) {return -1 ; } pdpe_addr = ((pml4e &PG_ADDRESS_MASK ) + (((addr >>30 ) &0x1ff ) <<3 )) & a20_mask; - pdpe =x86_ldq_phys (cs, pdpe_addr); + pdpe =ldq_phys (cs, pdpe_addr);if (!(pdpe &PG_PRESENT_MASK )) {return -1 ; }
@@ -273,14 +275,14 @@ hwaddr x86_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr, { pdpe_addr = ((env->cr[3 ] & ~0x1f ) + ((addr >>27 ) &0x18 )) & a20_mask; - pdpe =x86_ldq_phys (cs, pdpe_addr); + pdpe =ldq_phys (cs, pdpe_addr);if (!(pdpe &PG_PRESENT_MASK ))return -1 ; } pde_addr = ((pdpe &PG_ADDRESS_MASK ) + (((addr >>21 ) &0x1ff ) <<3 )) & a20_mask; - pde =x86_ldq_phys (cs, pde_addr); + pde =ldq_phys (cs, pde_addr);if (!(pde &PG_PRESENT_MASK )) {return -1 ; }
@@ -293,7 +295,7 @@ hwaddr x86_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr, pte_addr = ((pde &PG_ADDRESS_MASK ) + (((addr >>12 ) &0x1ff ) <<3 )) & a20_mask; page_size =4096 ; - pte =x86_ldq_phys (cs, pte_addr); + pte =ldq_phys (cs, pte_addr); }if (!(pte &PG_PRESENT_MASK )) {return -1 ;
@@ -303,7 +305,7 @@ hwaddr x86_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,/* page directory entry */ pde_addr = ((env->cr[3 ] & ~0xfff ) + ((addr >>20 ) &0xffc )) & a20_mask; - pde =x86_ldl_phys (cs, pde_addr); + pde =ldl_phys (cs, pde_addr);if (!(pde &PG_PRESENT_MASK ))return -1 ;if ((pde &PG_PSE_MASK ) && (env->cr[4 ] &CR4_PSE_MASK )) {
@@ -312,7 +314,7 @@ hwaddr x86_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr, }else {/* page directory entry */ pte_addr = ((pde & ~0xfff ) + ((addr >>10 ) &0xffc )) & a20_mask; - pte =x86_ldl_phys (cs, pte_addr); + pte =ldl_phys (cs, pte_addr);if (!(pte &PG_PRESENT_MASK )) {return -1 ; }
@@ -329,6 +331,22 @@ out:return pte | page_offset; } +hwaddr x86_cpu_get_phys_page_attrs_debug (CPUState *cs,vaddr addr, +MemTxAttrs *attrs) +{ +return x86_cpu_get_phys_page_attrs_debug_internal (cs, addr, attrs, + x86_ldq_phys, + x86_ldl_phys); +} + +hwaddr x86_cpu_get_phys_page_attrs_encrypted_debug (CPUState *cs,vaddr addr, +MemTxAttrs *attrs) +{ +return x86_cpu_get_phys_page_attrs_debug_internal (cs, addr, attrs, + x86_ldq_phys_debug, + x86_ldl_phys_debug); +} + typedef struct MCEInjectionParams { Monitor *mon; int bank;
target/i386/kvm/tdx.c +6 -0
@@ -273,6 +273,7 @@ void tdx_pre_create_vcpu(CPUState *cpu)MachineState *ms =MACHINE (qdev_get_machine ());X86CPU *x86cpu =X86_CPU (cpu); +CPUClass *cc =CPU_GET_CLASS (cpu);CPUX86State *env = &x86cpu->env;TdxGuest *tdx = (TdxGuest *)object_dynamic_cast (OBJECT (ms->cgs),TYPE_TDX_GUEST );
@@ -320,6 +321,11 @@ void tdx_pre_create_vcpu(CPUState *cpu) init_vm.cpuid = (__u64)(&cpuid_data);tdx_ioctl (KVM_TDX_INIT_VM ,0 , &init_vm); + +if (tdx->debug) { + cc->get_phys_page_attrs_debug + = x86_cpu_get_phys_page_attrs_encrypted_debug; + } out:qemu_mutex_unlock (&tdx->lock); }
Signature · 1 line
2.20.1
cover Enable encrypted guest memory access in QEMU
In reply to Yuan YaoQuoted 13 lines
- We introduce another new vm level ioctl focus on the encrypted guest memory accessing:
KVM_MEMORY_ENCRYPT_{READ,WRITE}_MEMORY
struct
kvm_rw_memoryrw;
rw.addr =gpa_OR_hva;
rw.buf = (__u64)src;
rw.len = len;
kvm_vm_ioctl(kvm_state,
KVM_MEMORY_ENCRYPT_{READ,WRITE}_MEMORY, &rw);
Quoted 7 lines
This new ioctl has more neutral and general name for its purpose, the debugging support of AMD SEV and INTEL TDX can be covered by a unify QEMU implementation on x86 with this ioctl. Although only INTEL TD guest is supported in this series, AMD SEV could be also supported with implementation of this ioctl in KVM, plus small modifications in QEMU to enable the unify part.
A general comment, we have sev_ioctl() interface for SEV guests and probably this new vm level ioctl will not work for us.
It probably makes more sense to do this TDX/SEV level abstraction using the Memory Region's ram_debug_ops, which can point these to TDX specific vm level ioctl and SEV specific ioctl at the lowest level of this interface.
Thanks,
Ashish
Quoted 31 lines
-----Original Message-----
From: Ashish Kalra <[email protected]>
Sent: Thursday, September 02, 2021 22:05
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yamahata, Isaku <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yao, Yuan <[email protected]>
Subject: [RFC][PATCH v1 00/10] Enable encrypted guest memory access in QEMUQuoted 13 lines
- We introduce another new vm level ioctl focus on the encrypted guest memory accessing:
KVM_MEMORY_ENCRYPT_{READ,WRITE}_MEMORY
struct
kvm_rw_memoryrw;
rw.addr =gpa_OR_hva;
rw.buf = (__u64)src;
rw.len = len;
kvm_vm_ioctl(kvm_state,
KVM_MEMORY_ENCRYPT_{READ,WRITE}_MEMORY, &rw);Quoted 7 lines
This new ioctl has more neutral and general name for its purpose, the debugging support of AMD SEV and INTEL TDX can be covered by a unify QEMU implementation on x86 with this ioctl. Although only INTEL TD guest is supported in this series, AMD SEV could be also supported with implementation of this ioctl in KVM, plus small modifications in QEMU to enable the unify part.
Quoted 8 lines
A general comment, we have
sev_ioctl()interface for SEV guests and probably this new vm level ioctl will not work for us.It probably makes more sense to do this TDX/SEV level abstraction using the Memory Region's
ram_debug_ops, which can point these to TDX specific vm level ioctl and SEV specific ioctl at the lowest level of this interface.
Hi Ashish,
Yes, this new ioctl is now working as the low-level interface for Memory Region's ram_debug_ops. SEV can use kvm_setup_set_memory_region_debug_ops() to install a new callback to KVM for installing SEV only low-level implementation, then call kvm_set_memory_region_debug_ops() to do Memory Region's ram_debug_ops installation later.
Quoted 2 lines
Thanks,
Ashish
Hello Yuan,
Quoted 39 lines · Yuan
Quoted 31 lines
-----Original Message-----
From: Ashish Kalra <[email protected]>
Sent: Thursday, September 02, 2021 22:05
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yamahata, Isaku <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yao, Yuan <[email protected]>
Subject: [RFC][PATCH v1 00/10] Enable encrypted guest memory access in QEMUQuoted 13 lines
- We introduce another new vm level ioctl focus on the encrypted guest memory accessing:
KVM_MEMORY_ENCRYPT_{READ,WRITE}_MEMORY
struct
kvm_rw_memoryrw;
rw.addr =gpa_OR_hva;
rw.buf = (__u64)src;
rw.len = len;
kvm_vm_ioctl(kvm_state,
KVM_MEMORY_ENCRYPT_{READ,WRITE}_MEMORY, &rw);Quoted 7 lines
This new ioctl has more neutral and general name for its purpose, the debugging support of AMD SEV and INTEL TDX can be covered by a unify QEMU implementation on x86 with this ioctl. Although only INTEL TD guest is supported in this series, AMD SEV could be also supported with implementation of this ioctl in KVM, plus small modifications in QEMU to enable the unify part.
Quoted 8 lines
A general comment, we have
sev_ioctl()interface for SEV guests and probably this new vm level ioctl will not work for us.It probably makes more sense to do this TDX/SEV level abstraction using the Memory Region's
ram_debug_ops, which can point these to TDX specific vm level ioctl and SEV specific ioctl at the lowest level of this interface.
Quoted 10 lines
Hi Ashish,
Yes, this new ioctl is now working as the low-level interface for Memory Region's
ram_debug_ops. SEV can usekvm_setup_set_memory_region_debug_ops()to install a new callback to KVM for installing SEV only low-level implementation, then callkvm_set_memory_region_debug_ops()to do Memory Region'sram_debug_opsinstallation later.
Ok. Yes i think that should work.
Thanks,
Ashish