Skip to contentSkip to conversation

Add QEMU debug support for SEV guests

PATCH In review Ashish Kalra 4 participants 27 messages Nov 16, 2020 – Dec 1, 2020 KVM

lore
Share & tools
lore
#0 lore raw

cover Add QEMU debug support for SEV guests

From: Ashish Kalra <[email protected]>

This patchset adds QEMU debug support for SEV guests. Debug requires access to the guest pages, which is encrypted when SEV is enabled.

KVM_SEV_DBG_DECRYPT and KVM_SEV_DBG_ENCRYPT commands are available to decrypt/encrypt the guest pages, if the guest policy allows for debugging.

Changes are made to the guest page table walker since SEV guest pte entries will have the C-bit set.

Also introduces new MemoryDebugOps which hook into guest virtual and physical memory debug interfaces such as cpu_memory_rw_debug, to allow vendor specific assist/hooks for debugging and delegating accessing the guest memory. This is used for example in case of AMD SEV platform where the guest memory is encrypted and a SEV specific debug assist/hook will be required to access the guest memory.

The MemoryDebugOps are used by cpu_memory_rw_debug() and default to address_space_read and address_space_write_rom as described below.

typedef struct MemoryDebugOps {
MemTxResult (*read)(AddressSpace *as, hwaddr phys_addr,
MemTxAttrs attrs, void *buf, hwaddr len);
MemTxResult (*write)(AddressSpace *as, hwaddr phys_addr,
MemTxAttrs attrs, const void *buf, hwaddr len);
} MemoryDebugOps;

These ops would be used only by cpu_memory_rw_debug and would default to

static const MemoryDebugOps default_debug_ops = {
.translate = cpu_get_phys_page_attrs_debug,
.read = address_space_read,
.write = address_space_write_rom
};

static const MemoryDebugOps *debug_ops = &default_debug_ops;

Ashish Kalra (3):
exec: Add new MemoryDebugOps.
exec: Add address_space_read and address_space_write debug helpers. sev/i386: add SEV specific MemoryDebugOps.

Brijesh Singh (8):
memattrs: add debug attribute
exec: add ram_debug_ops support
exec: add debug version of physical memory read and write API monitor/i386: use debug APIs when accessing guest memory
kvm: introduce debug memory encryption API sev/i386: add debug encrypt and decrypt commands hw/i386: set ram_debug_ops when memory encryption is enabled target/i386: clear C-bit when walking SEV guest page table

 accel/kvm/kvm-all.c       |  22 ++++
 accel/kvm/sev-stub.c      |   8 ++
 accel/stubs/kvm-stub.c    |   8 ++
 hw/i386/pc.c              |   9 ++
 hw/i386/pc_sysfw.c        |   6 +
 include/exec/cpu-common.h |  18 +++
 include/exec/memattrs.h   |   2 +
 include/exec/memory.h     |  49 ++++++++
 include/sysemu/kvm.h      |  15 +++
 include/sysemu/sev.h      |  12 ++
 monitor/misc.c            |   4 +-
 softmmu/cpus.c            |   2 +-
 softmmu/physmem.c         | 170 +++++++++++++++++++++++++-
 target/i386/kvm.c         |   4 +
 target/i386/monitor.c     | 124 +++++++++++--------
 target/i386/sev.c         | 244 ++++++++++++++++++++++++++++++++++++++
 target/i386/trace-events  |   1 +
 17 files changed, 642 insertions(+), 56 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/11 memattrs: add debug attribute

From: Brijesh Singh <[email protected]>

From: Brijesh Singh <[email protected]>

Extend the MemTxAttrs to include a 'debug' flag. The flag can be used as general indicator that operation was triggered by the debugger.

A subsequent patch will set the debug=1 when issuing a memory access from the gdbstub or HMP commands. This is a prerequisite to support debugging an encrypted guest. When a request with debug=1 is seen, the encryption APIs will be used to access the guest memory.

---
 include/exec/memattrs.h | 2 ++
 1 file changed, 2 insertions(+)
include/exec/memattrs.h +2 -0
index 95f2d20d55..c8b56389d6 100644
@@ -49,6 +49,8 @@ typedef struct MemTxAttrs {
     unsigned int target_tlb_bit0 : 1;
     unsigned int target_tlb_bit1 : 1;
     unsigned int target_tlb_bit2 : 1;
+    /* Memory access request from the debugger */
+    unsigned int debug:1;
 } MemTxAttrs;
 
 /* Bus masters which don't specify any attributes will get this,
Signature · 1 line
2.17.1
#7 lore raw

PATCH 2/11 exec: Add new MemoryDebugOps.

In reply to Ashish Kalra

From: Ashish Kalra <[email protected]>

Introduce new MemoryDebugOps which hook into guest virtual and physical memory debug interfaces such as cpu_memory_rw_debug, to allow vendor specific assist/hooks for debugging and delegating accessing the guest memory. This is required for example in case of AMD SEV platform where the guest memory is encrypted and a SEV specific debug assist/hook will be required to access the guest memory.

The MemoryDebugOps are used by cpu_memory_rw_debug() and default to address_space_read and address_space_write_rom.

---
 include/exec/memory.h | 11 +++++++++++
 softmmu/physmem.c     | 24 ++++++++++++++++++++----
 2 files changed, 31 insertions(+), 4 deletions(-)
include/exec/memory.h +11 -0
index aff6ef7605..73deb4b456 100644
@@ -2394,6 +2394,17 @@ MemTxResult address_space_write_cached_slow(MemoryRegionCache *cache,
                                             hwaddr addr, const void *buf,
                                             hwaddr len);
 
+typedef struct MemoryDebugOps {
+    MemTxResult (*read)(AddressSpace *as, hwaddr phys_addr,
+                        MemTxAttrs attrs, void *buf,
+                        hwaddr len);
+    MemTxResult (*write)(AddressSpace *as, hwaddr phys_addr,
+                         MemTxAttrs attrs, const void *buf,
+                         hwaddr len);
+} MemoryDebugOps;
+
+void address_space_set_debug_ops(const MemoryDebugOps *ops);
+
 static inline bool memory_access_is_direct(MemoryRegion *mr, bool is_write)
 {
     if (is_write) {
softmmu/physmem.c +20 -4
index a9adedb9f8..057d6d4ce1 100644
@@ -166,6 +166,18 @@ struct DirtyBitmapSnapshot {
     unsigned long dirty[];
 };
 
+static const MemoryDebugOps default_debug_ops = {
+    .read = address_space_read,
+    .write = address_space_write_rom
+};
+
+static const MemoryDebugOps *debug_ops = &default_debug_ops;
+
+void address_space_set_debug_ops(const MemoryDebugOps *ops)
+{
+    debug_ops = ops;
+}
+
 static void phys_map_node_reserve(PhysPageMap *map, unsigned nodes)
 {
     static unsigned alloc_hint = 16;
@@ -3407,6 +3419,10 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
         page = addr & TARGET_PAGE_MASK;
         phys_addr = cpu_get_phys_page_attrs_debug(cpu, page, &attrs);
         asidx = cpu_asidx_from_attrs(cpu, attrs);
+
+        /* set debug attrs to indicate memory access is from the debugger */
+        attrs.debug = 1;
+
         /* if no physical page mapped, return an error */
         if (phys_addr == -1)
             return -1;
@@ -3415,11 +3431,11 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
             l = len;
         phys_addr += (addr & ~TARGET_PAGE_MASK);
         if (is_write) {
-            res = address_space_write_rom(cpu->cpu_ases[asidx].as, phys_addr,
-                                          attrs, buf, l);
+            res = debug_ops->write(cpu->cpu_ases[asidx].as, phys_addr,
+                                   attrs, buf, l);
         } else {
-            res = address_space_read(cpu->cpu_ases[asidx].as, phys_addr,
-                                     attrs, buf, l);
+            res = debug_ops->read(cpu->cpu_ases[asidx].as, phys_addr,
+                                  attrs, buf, l);
         }
         if (res != MEMTX_OK) {
             return -1;
Signature · 1 line
2.17.1
#13 lore raw

PATCH 3/11 exec: add ram_debug_ops support

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

From: Brijesh Singh <[email protected]>

Currently, guest memory access for debugging purposes is performed using memcpy(). Extend the 'struct MemoryRegion' to include new callbacks that can be used to override the use of memcpy() with something else.

The new callbacks can be used to display the guest memory of an SEV guest by registering callbacks to the SEV memory encryption/decryption APIs.

Typical usage:

mem_read(uint8_t *dst, uint8_t *src, uint32_t len, MemTxAttrs *attrs); mem_write(uint8_t *dst, uint8_t *src, uint32_t len, MemTxAttrs *attrs);

MemoryRegionRAMReadWriteOps ops;
ops.read = mem_read;
ops.write = mem_write;

memory_region_init_ram(mem, NULL, "memory", size, NULL); memory_region_set_ram_debug_ops(mem, ops);

---
 include/exec/memory.h | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
include/exec/memory.h +27 -0
index 73deb4b456..2fb4193358 100644
@@ -402,6 +402,18 @@ struct IOMMUMemoryRegionClass {
 typedef struct CoalescedMemoryRange CoalescedMemoryRange;
 typedef struct MemoryRegionIoeventfd MemoryRegionIoeventfd;
 
+/* Memory Region RAM debug callback */
+typedef struct MemoryRegionRAMReadWriteOps MemoryRegionRAMReadWriteOps;
+
+struct MemoryRegionRAMReadWriteOps {
+    /* Write data into guest memory */
+    int (*write) (uint8_t *dest, const uint8_t *src,
+                  uint32_t len, MemTxAttrs attrs);
+    /* Read data from guest memory */
+    int (*read) (uint8_t *dest, const uint8_t *src,
+                 uint32_t len, MemTxAttrs attrs);
+};
+
 /** MemoryRegion:
  *
  * A struct representing a memory region.
@@ -445,6 +457,7 @@ struct MemoryRegion {
     const char *name;
     unsigned ioeventfd_nb;
     MemoryRegionIoeventfd *ioeventfds;
+    const MemoryRegionRAMReadWriteOps *ram_debug_ops;
 };
 
 struct IOMMUMemoryRegion {
@@ -1060,6 +1073,20 @@ void memory_region_init_rom_nomigrate(MemoryRegion *mr,
                                       uint64_t size,
                                       Error **errp);
 
+/**
+ * memory_region_set_ram_debug_ops: Set access ops for a give memory region.
+ *
+ * @mr: the #MemoryRegion to be initialized
+ * @ops: a function that will be used when accessing @target region during
+ *       debug
+ */
+static inline void
+memory_region_set_ram_debug_ops(MemoryRegion *mr,
+                                const MemoryRegionRAMReadWriteOps *ops)
+{
+    mr->ram_debug_ops = ops;
+}
+
 /**
  * memory_region_init_rom_device_nomigrate:  Initialize a ROM memory region.
  *                                 Writes are handled via callbacks.
Signature · 1 line
2.17.1
#16 lore raw

PATCH 4/11 exec: Add address_space_read and address_space_write debug helpers.

In reply to Ashish Kalra

From: Ashish Kalra <[email protected]>

Add new address_space_read and address_space_write debug helper interfaces which can be invoked by vendor specific guest memory debug assist/hooks to do guest RAM memory accesses using the added MemoryRegion callbacks.

---
 include/exec/memory.h | 10 +++++
 softmmu/physmem.c     | 88 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 98 insertions(+)
include/exec/memory.h +10 -0
index 2fb4193358..74f2dcec00 100644
@@ -2432,6 +2432,16 @@ typedef struct MemoryDebugOps {
 
 void address_space_set_debug_ops(const MemoryDebugOps *ops);
 
+MemTxResult address_space_write_rom_debug(AddressSpace *as,
+                                          hwaddr addr,
+                                          MemTxAttrs attrs,
+                                          const void *ptr,
+                                          hwaddr len);
+
+MemTxResult address_space_read_debug(AddressSpace *as, hwaddr addr,
+                                     MemTxAttrs attrs, void *buf,
+                                     hwaddr len);
+
 static inline bool memory_access_is_direct(MemoryRegion *mr, bool is_write)
 {
     if (is_write) {
softmmu/physmem.c +88 -0

94 lines not on this page · raw message

Signature · 1 line
2.17.1
#17 lore raw

PATCH 5/11 exec: add debug version of physical memory read and write API

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Adds the following new APIs
- cpu_physical_memory_read_debug
- cpu_physical_memory_write_debug
- cpu_physical_memory_rw_debug
- ldl_phys_debug
- ldq_phys_debug

The subsequent patch will make use of the API introduced, to ensure that the page table walks are handled correctly when debugging an SEV guest.

---
 include/exec/cpu-common.h | 15 +++++++++++++
 softmmu/physmem.c         | 47 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+)
include/exec/cpu-common.h +15 -0
index 19805ed6db..d2089e6873 100644
@@ -71,11 +71,26 @@ size_t qemu_ram_pagesize_largest(void);
 
 void cpu_physical_memory_rw(hwaddr addr, void *buf,
                             hwaddr len, bool is_write);
+void cpu_physical_memory_rw_debug(hwaddr addr, uint8_t *buf,
+                                  int len, int is_write);
 static inline void cpu_physical_memory_read(hwaddr addr,
                                             void *buf, hwaddr len)
 {
     cpu_physical_memory_rw(addr, buf, len, false);
 }
+static inline void cpu_physical_memory_read_debug(hwaddr addr,
+                                                  void *buf, int len)
+{
+    cpu_physical_memory_rw_debug(addr, buf, len, false);
+}
+static inline void cpu_physical_memory_write_debug(hwaddr addr,
+                                                   const void *buf, int len)
+{
+    cpu_physical_memory_rw_debug(addr, (void *)buf, len, true);
+}
+uint32_t ldl_phys_debug(CPUState *cpu, hwaddr addr);
+uint64_t ldq_phys_debug(CPUState *cpu, hwaddr addr);
+
 static inline void cpu_physical_memory_write(hwaddr addr,
                                              const void *buf, hwaddr len)
 {
softmmu/physmem.c +47 -0

53 lines not on this page · raw message

Signature · 1 line
2.17.1
#19 lore raw

PATCH 6/11 monitor/i386: use debug APIs when accessing guest memory

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Update the HMP commands to use the debug version of APIs when accessing guest memory.

---
 monitor/misc.c        |  4 ++--
 softmmu/cpus.c        |  2 +-
 target/i386/monitor.c | 54 ++++++++++++++++++++++++-------------------
 3 files changed, 33 insertions(+), 27 deletions(-)
monitor/misc.c +2 -2
index 32e6a8c13d..7eba3a6fce 100644
@@ -824,8 +824,8 @@ static void hmp_sum(Monitor *mon, const QDict *qdict)
 
     sum = 0;
     for(addr = start; addr < (start + size); addr++) {
-        uint8_t val = address_space_ldub(&address_space_memory, addr,
-                                         MEMTXATTRS_UNSPECIFIED, NULL);
+        uint8_t val;
+        cpu_physical_memory_read_debug(addr, &val, 1);
         /* BSD sum algorithm ('sum' Unix command) */
         sum = (sum >> 1) | (sum << 15);
         sum += val;
softmmu/cpus.c +1 -1
index e46ac68ad0..79817330b7 100644
@@ -779,7 +779,7 @@ void qmp_pmemsave(int64_t addr, int64_t size, const char *filename,
         l = sizeof(buf);
         if (l > size)
             l = size;
-        cpu_physical_memory_read(addr, buf, l);
+        cpu_physical_memory_read_debug(addr, buf, l);
         if (fwrite(buf, 1, l, f) != l) {
             error_setg(errp, QERR_IO_ERROR);
             goto exit;
target/i386/monitor.c +30 -24

194 lines not on this page · raw message

Signature · 1 line
2.17.1
#22 lore raw

PATCH 7/11 kvm: introduce debug memory encryption API

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

In order to support debugging with Secure Encrypted Virtualization (SEV), add a high-level memory encryption API.

Also add a new API interface to override any CPU class specific callbacks for supporting debugging with SEV, for example, overriding the guest MMU/ page-table walker callback.

---
 accel/kvm/kvm-all.c    | 19 +++++++++++++++++++
 accel/stubs/kvm-stub.c |  8 ++++++++
 include/sysemu/kvm.h   | 15 +++++++++++++++
 3 files changed, 42 insertions(+)
accel/kvm/kvm-all.c +19 -0
index 9ef5daf4c5..ae85f53e7d 100644
@@ -123,6 +123,8 @@ struct KVMState
     /* memory encryption */
     void *memcrypt_handle;
     int (*memcrypt_encrypt_data)(void *handle, uint8_t *ptr, uint64_t len);
+    void (*memcrypt_debug_ops_memory_region)(void *handle, MemoryRegion *mr);
+    void (*memcrypt_debug_ops_cpu_state)(void *handle, CPUState *cpu);
 
     /* For "info mtree -f" to tell if an MR is registered in KVM */
     int nr_as;
@@ -222,6 +224,23 @@ int kvm_get_max_memslots(void)
     return s->nr_slots;
 }
 
+void kvm_memcrypt_set_debug_ops_memory_region(MemoryRegion *mr)
+{
+    if (kvm_state->memcrypt_handle &&
+        kvm_state->memcrypt_debug_ops_memory_region) {
+        kvm_state->memcrypt_debug_ops_memory_region(kvm_state->memcrypt_handle,
+                                                    mr);
+    }
+}
+
+void kvm_memcrypt_set_debug_ops_cpu_state(CPUState *cs)
+{
+    if (kvm_state->memcrypt_handle &&
+        kvm_state->memcrypt_debug_ops_cpu_state) {
+        kvm_state->memcrypt_debug_ops_cpu_state(kvm_state->memcrypt_handle, cs);
+    }
+}
+
 bool kvm_memcrypt_enabled(void)
 {
     if (kvm_state && kvm_state->memcrypt_handle) {
accel/stubs/kvm-stub.c +8 -0
index 680e099463..bf93431e46 100644
@@ -91,6 +91,14 @@ int kvm_memcrypt_encrypt_data(uint8_t *ptr, uint64_t len)
   return 1;
 }
 
+void kvm_memcrypt_set_debug_ops_memory_region(MemoryRegion *mr)
+{
+}
+
+void kvm_memcrypt_set_debug_ops_cpu_state(CPUState *cs)
+{
+}
+
 #ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev)
 {
include/sysemu/kvm.h +15 -0
index bb5d5cf497..1bde2e3d71 100644
@@ -470,6 +470,21 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 uint64_t kvm_arch_get_supported_msr_feature(KVMState *s, uint32_t index);
 
+/**
+ * kvm_memcrypt_set_debug_ram_ops: set debug_ram_ops callback
+ *
+ * When debug_ram_ops is set, debug access to this memory region will use
+ * memory encryption APIs.
+ */
+void kvm_memcrypt_set_debug_ops_memory_region(MemoryRegion *mr);
+
+/**
+ * kvm_memcrypt_set_debug_ops_cpu_state: override cpu_class callbacks
+ *
+ * This interface allows vendor specific debug ops to override any
+ * cpu_class callbacks.
+ */
+void kvm_memcrypt_set_debug_ops_cpu_state(CPUState *cs);
 
 void kvm_set_sigmask_len(KVMState *s, unsigned int sigmask_len);
 
Signature · 1 line
2.17.1
#23 lore raw

PATCH 8/11 sev/i386: add debug encrypt and decrypt commands

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SEV_DBG_DECRYPT and KVM_SEV_DBG_ENCRYPT commands are used for decrypting and encrypting guest memory. The command works only if the guest policy allows the debugging.

---
 accel/kvm/kvm-all.c      |  2 ++
 accel/kvm/sev-stub.c     |  4 +++
 include/sysemu/sev.h     |  1 +
 target/i386/sev.c        | 58 ++++++++++++++++++++++++++++++++++++++++
 target/i386/trace-events |  1 +
 5 files changed, 66 insertions(+)
accel/kvm/kvm-all.c +2 -0
index ae85f53e7d..042205e3e1 100644
@@ -2232,6 +2232,8 @@ static int kvm_init(MachineState *ms)
         }
 
         kvm_state->memcrypt_encrypt_data = sev_encrypt_data;
+        kvm_state->memcrypt_debug_ops_memory_region =
+            sev_set_debug_ops_memory_region;
     }
 
     ret = kvm_arch_init(ms, s);
accel/kvm/sev-stub.c +4 -0
index 4f97452585..3f1f0ef217 100644
@@ -15,6 +15,10 @@
 #include "qemu-common.h"
 #include "sysemu/sev.h"
 
+void sev_set_debug_ops_memory_region(void *handle, MemoryRegion *mr)
+{
+}
+
 int sev_encrypt_data(void *handle, uint8_t *ptr, uint64_t len)
 {
     abort();
include/sysemu/sev.h +1 -0
index 98c1ec8d38..6c37247915 100644
@@ -18,4 +18,5 @@
 
 void *sev_guest_init(const char *id);
 int sev_encrypt_data(void *handle, uint8_t *ptr, uint64_t len);
+void sev_set_debug_ops_memory_region(void *handle, MemoryRegion *mr);
 #endif
target/i386/sev.c +58 -0

76 lines not on this page · raw message

target/i386/trace-events +1 -0
index 789c700d4a..f91213c5e9 100644
@@ -15,3 +15,4 @@ kvm_sev_launch_start(int policy, void *session, void *pdh) "policy 0x%x session
 kvm_sev_launch_update_data(void *addr, uint64_t len) "addr %p len 0x%" PRIu64
 kvm_sev_launch_measurement(const char *value) "data %s"
 kvm_sev_launch_finish(void) ""
+kvm_sev_debug(const char *op, const uint8_t *src, uint8_t *dst, int len) "(%s) src %p dst %p len %d"
Signature · 1 line
2.17.1
#24 lore raw

PATCH 9/11 hw/i386: set ram_debug_ops when memory encryption is enabled

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When memory encryption is enabled, the guest RAM and boot flash ROM will contain the encrypted data. By setting the debug ops allow us to invoke encryption APIs when accessing the memory for the debug purposes.

---
 hw/i386/pc.c       | 9 +++++++++
 hw/i386/pc_sysfw.c | 6 ++++++
 2 files changed, 15 insertions(+)
hw/i386/pc.c +9 -0
index 5e6c0023e0..dfb63cd686 100644
@@ -913,6 +913,15 @@ void pc_memory_init(PCMachineState *pcms,
         e820_add_entry(0x100000000ULL, x86ms->above_4g_mem_size, E820_RAM);
     }
 
+    /*
+     * When memory encryption is enabled, the guest RAM will be encrypted with
+     * a guest unique key. Set the debug ops so that any debug access to the
+     * guest RAM will go through the memory encryption APIs.
+     */
+    if (kvm_memcrypt_enabled()) {
+        kvm_memcrypt_set_debug_ops_memory_region(*ram_memory);
+    }
+
     if (!pcmc->has_reserved_memory &&
         (machine->ram_slots ||
          (machine->maxram_size > machine->ram_size))) {
hw/i386/pc_sysfw.c +6 -0
index b6c0822fe3..9f90c9d761 100644
@@ -209,6 +209,12 @@ static void pc_system_flash_map(PCMachineState *pcms,
                     error_report("failed to encrypt pflash rom");
                     exit(1);
                 }
+
+                /*
+                 * The pflash ROM is encrypted, set the debug ops so that any
+                 * debug accesses will use memory encryption APIs.
+                 */
+                kvm_memcrypt_set_debug_ops_memory_region(flash_mem);
             }
         }
     }
Signature · 1 line
2.17.1
#25 lore raw

PATCH 10/11 sev/i386: add SEV specific MemoryDebugOps.

In reply to Ashish Kalra

From: Ashish Kalra <[email protected]>

Add SEV specific MemoryDebugOps which override the default MemoryDebugOps when SEV memory encryption is enabled. The SEV specific MemoryDebugOps invoke the generic address_space_rw_debug helpers which will then invoke the memory region specific callbacks to handle and access encrypted memory when guest RAM is accessed.

Also invoke the memory encryption API to override any CPU class specific callbacks to handle memory encryption.

Specifically for SEV we override CPU class specific guest MMU/page-table walker to invoke a SEV specific handler which can handle guest encrypted memory and also clear C-bit when walking SEV guest page table.

---
 accel/kvm/kvm-all.c  |   1 +
 accel/kvm/sev-stub.c |   4 +
 include/sysemu/sev.h |  11 +++
 target/i386/kvm.c    |   4 +
 target/i386/sev.c    | 185 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 205 insertions(+)
accel/kvm/kvm-all.c +1 -0
index 042205e3e1..6d812d5b09 100644
@@ -2234,6 +2234,7 @@ static int kvm_init(MachineState *ms)
         kvm_state->memcrypt_encrypt_data = sev_encrypt_data;
         kvm_state->memcrypt_debug_ops_memory_region =
             sev_set_debug_ops_memory_region;
+        kvm_state->memcrypt_debug_ops_cpu_state = sev_set_debug_ops_cpu_state;
     }
 
     ret = kvm_arch_init(ms, s);
accel/kvm/sev-stub.c +4 -0
index 3f1f0ef217..ad27226058 100644
@@ -19,6 +19,10 @@ void sev_set_debug_ops_memory_region(void *handle, MemoryRegion *mr)
 {
 }
 
+void sev_set_debug_ops_cpu_state(void *handle, CPUState *cpu)
+{
+}
+
 int sev_encrypt_data(void *handle, uint8_t *ptr, uint64_t len)
 {
     abort();
include/sysemu/sev.h +11 -0
index 6c37247915..e6f176b85b 100644
@@ -19,4 +19,15 @@
 void *sev_guest_init(const char *id);
 int sev_encrypt_data(void *handle, uint8_t *ptr, uint64_t len);
 void sev_set_debug_ops_memory_region(void *handle, MemoryRegion *mr);
+void sev_set_debug_ops_cpu_state(void *handle, CPUState *cpu);
+hwaddr sev_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
+                                         MemTxAttrs *attrs);
+MemTxResult sev_address_space_read_debug(AddressSpace *as, hwaddr addr,
+                                         MemTxAttrs attrs, void *ptr,
+                                         hwaddr len);
+MemTxResult sev_address_space_write_rom_debug(AddressSpace *as,
+                                              hwaddr addr,
+                                              MemTxAttrs attrs,
+                                              const void *ptr,
+                                              hwaddr len);
 #endif
target/i386/kvm.c +4 -0
index cf46259534..7a2d10b745 100644
@@ -1838,6 +1838,10 @@ int kvm_arch_init_vcpu(CPUState *cs)
 
     kvm_init_msrs(cpu);
 
+    if (kvm_memcrypt_enabled()) {
+        kvm_memcrypt_set_debug_ops_cpu_state(cs);
+    }
+
     r = hyperv_init_vcpu(cpu);
     if (r) {
         goto fail;
target/i386/sev.c +185 -0

191 lines not on this page · raw message

Signature · 1 line
2.17.1
#26 lore raw

PATCH 11/11 target/i386: clear C-bit when walking SEV guest page table

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

In SEV-enabled guest the pte entry will have C-bit set, we need to clear the C-bit when walking the page table.

This ensures that the proper page address translation occurs and, with the C-bit reset, the true physical address is got.

The pte_mask to be used during guest page table walk is added as a vendor specific assist/hook as part of the new MemoryDebugOps and available via the new debug API interface cpu_physical_memory_pte_mask_debug().

---
 include/exec/cpu-common.h |  3 ++
 include/exec/memory.h     |  1 +
 softmmu/physmem.c         | 13 +++++++-
 target/i386/monitor.c     | 70 +++++++++++++++++++++++++--------------
 target/i386/sev.c         |  3 +-
 5 files changed, 63 insertions(+), 27 deletions(-)
include/exec/cpu-common.h +3 -0
index d2089e6873..3374573d39 100644
@@ -96,6 +96,9 @@ static inline void cpu_physical_memory_write(hwaddr addr,
 {
     cpu_physical_memory_rw(addr, (void *)buf, len, true);
 }
+
+uint64_t cpu_physical_memory_pte_mask_debug(void);
+
 void *cpu_physical_memory_map(hwaddr addr,
                               hwaddr *plen,
                               bool is_write);
include/exec/memory.h +1 -0
index 74f2dcec00..ebe8ffc1eb 100644
@@ -2428,6 +2428,7 @@ typedef struct MemoryDebugOps {
     MemTxResult (*write)(AddressSpace *as, hwaddr phys_addr,
                          MemTxAttrs attrs, const void *buf,
                          hwaddr len);
+    uint64_t (*pte_mask)(void);
 } MemoryDebugOps;
 
 void address_space_set_debug_ops(const MemoryDebugOps *ops);
softmmu/physmem.c +12 -1
index 6945bd5efe..fc6b5588fc 100644
@@ -166,9 +166,15 @@ struct DirtyBitmapSnapshot {
     unsigned long dirty[];
 };
 
+static uint64_t address_space_pte_mask(void)
+{
+    return ~0;
+}
+
 static const MemoryDebugOps default_debug_ops = {
     .read = address_space_read,
-    .write = address_space_write_rom
+    .write = address_space_write_rom,
+    .pte_mask = address_space_pte_mask
 };
 
 static const MemoryDebugOps *debug_ops = &default_debug_ops;
@@ -3401,6 +3407,11 @@ void cpu_physical_memory_rw_debug(hwaddr addr, uint8_t *buf,
 
 }
 
+uint64_t cpu_physical_memory_pte_mask_debug(void)
+{
+    return debug_ops->pte_mask();
+}
+
 int64_t address_space_cache_init(MemoryRegionCache *cache,
                                  AddressSpace *as,
                                  hwaddr addr,
target/i386/monitor.c +45 -25

212 lines not on this page · raw message

target/i386/sev.c +2 -1
index b942593bc8..97726a639f 100644
@@ -1003,7 +1003,8 @@ MemTxResult sev_address_space_read_debug(AddressSpace *as, hwaddr addr,
 
 static const MemoryDebugOps sev_debug_ops = {
     .read = sev_address_space_read_debug,
-    .write = sev_address_space_write_rom_debug
+    .write = sev_address_space_write_rom_debug,
+    .pte_mask = sev_get_me_mask,
 };
 
 void
Signature · 1 line
2.17.1
#18 lore raw

Re: [PATCH 05/11] exec: add debug version of physical memory read and write API

In reply to Ashish Kalra
Quoted 16 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Adds the following new APIs
- cpu_physical_memory_read_debug
- cpu_physical_memory_write_debug
- cpu_physical_memory_rw_debug
- ldl_phys_debug
- ldq_phys_debug

The subsequent patch will make use of the API introduced, to ensure that the page table walks are handled correctly when debugging an SEV guest.

---

[...]

Quoted 25 lines
softmmu/physmem.c +17 -0
index 2c08624ca8..6945bd5efe 100644
@@ -3354,6 +3354,53 @@ inline MemTxResult address_space_write_rom_debug(AddressSpace *as,
      return MEMTX_OK;
  }
 
+uint32_t ldl_phys_debug(CPUState *cpu, hwaddr addr)
+{
+    MemTxAttrs attrs;
+    int asidx = cpu_asidx_from_attrs(cpu, attrs);
+    uint32_t val;
+
+    /* set debug attrs to indicate memory access is from the debugger */
+    attrs.debug = 1;
+
+    debug_ops->read(cpu->cpu_ases[asidx].as, addr, attrs,
+                    (void *) &val, 4);
+
+    return tswap32(val);
+}
+
+uint64_t ldq_phys_debug(CPUState *cpu, hwaddr addr)
+{

softmmu/physmem.c inline MemTxResult address_space_write_rom_debug(AddressSpace *as, · line 3383

Quoted 10 lines
softmmu/physmem.c +10 -0
+    MemTxAttrs attrs;
+    int asidx = cpu_asidx_from_attrs(cpu, attrs);
+    uint64_t val;
+
+    /* set debug attrs to indicate memory access is from the debugger */
+    attrs.debug = 1;
+
+    debug_ops->read(cpu->cpu_ases[asidx].as, addr, attrs,
+                    (void *) &val, 8);
+    return val;

You probably want tswap64(val) here like in ldl_phys_debug (even though I assume it's a noop in the SEV case).

softmmu/physmem.c inline MemTxResult address_space_write_rom_debug(AddressSpace *as, · line 3392

Quoted 9 lines
softmmu/physmem.c +9 -0
+}
+
+void cpu_physical_memory_rw_debug(hwaddr addr, uint8_t *buf,
+                                  int len, int is_write)
+{
+    MemTxAttrs attrs;
+
+    /* set debug attrs to indicate memory access is from the debugger */
+    attrs.debug = 1;

Maybe:

    MemTxAttrs attrs = { .debug = 1 };

(Also in the functions above.)

Quoted 15 lines
softmmu/physmem.c +11 -0
+
+    if (is_write) {
+                debug_ops->write(&address_space_memory, addr,
+                                 attrs, buf, len);
+        } else {
+                debug_ops->read(&address_space_memory, addr,
+                                attrs, buf, len);
+        }
+
+}
+
  int64_t address_space_cache_init(MemoryRegionCache *cache,
                                   AddressSpace *as,
                                   hwaddr addr,
#2 lore raw

Re: [PATCH 01/11] memattrs: add debug attribute

In reply to Ashish Kalra
Quoted 11 lines · * Ashish Kalra ([email protected]) wrote:

From: Brijesh Singh <[email protected]>

From: Brijesh Singh <[email protected]>

Extend the MemTxAttrs to include a 'debug' flag. The flag can be used as general indicator that operation was triggered by the debugger.

A subsequent patch will set the debug=1 when issuing a memory access from the gdbstub or HMP commands. This is a prerequisite to support debugging an encrypted guest. When a request with debug=1 is seen, the encryption APIs will be used to access the guest memory.

Is this also the flag that would be used for memory dumping?

Quoted 11 lines
---
 include/exec/memattrs.h | 2 ++
 1 file changed, 2 insertions(+)
include/exec/memattrs.h +0 -0
index 95f2d20d55..c8b56389d6 100644

include/exec/memattrs.h typedef struct MemTxAttrs { · line 53

Quoted 5 lines
include/exec/memattrs.h +2 -0
@@ -49,6 +49,8 @@ typedef struct MemTxAttrs {
     unsigned int target_tlb_bit0 : 1;
     unsigned int target_tlb_bit1 : 1;
     unsigned int target_tlb_bit2 : 1;
+    /* Memory access request from the debugger */
+    unsigned int debug:1;

It might be good to clarify that this is for QEMU debug features, not guest side debug features (e.g. CPU debug facilities/registers)

Dave

Quoted 6 lines
include/exec/memattrs.h +0 -0
 } MemTxAttrs;
 
 /* Bus masters which don't specify any attributes will get this,
Signature · 1 line
2.17.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Reviewed-by on v1 2/11
#8 lore raw

Re: [PATCH 02/11] exec: Add new MemoryDebugOps.

In reply to Ashish Kalra
Quoted 9 lines · * Ashish Kalra ([email protected]) wrote:

From: Ashish Kalra <[email protected]>

Introduce new MemoryDebugOps which hook into guest virtual and physical memory debug interfaces such as cpu_memory_rw_debug, to allow vendor specific assist/hooks for debugging and delegating accessing the guest memory. This is required for example in case of AMD SEV platform where the guest memory is encrypted and a SEV specific debug assist/hook will be required to access the guest memory.

Quoted 4 lines

The MemoryDebugOps are used by cpu_memory_rw_debug() and default to address_space_read and address_space_write_rom.

Quoted 80 lines
---
 include/exec/memory.h | 11 +++++++++++
 softmmu/physmem.c     | 24 ++++++++++++++++++++----
 2 files changed, 31 insertions(+), 4 deletions(-)
include/exec/memory.h +11 -0
index aff6ef7605..73deb4b456 100644
@@ -2394,6 +2394,17 @@ MemTxResult address_space_write_cached_slow(MemoryRegionCache *cache,
                                             hwaddr addr, const void *buf,
                                             hwaddr len);
 
+typedef struct MemoryDebugOps {
+    MemTxResult (*read)(AddressSpace *as, hwaddr phys_addr,
+                        MemTxAttrs attrs, void *buf,
+                        hwaddr len);
+    MemTxResult (*write)(AddressSpace *as, hwaddr phys_addr,
+                         MemTxAttrs attrs, const void *buf,
+                         hwaddr len);
+} MemoryDebugOps;
+
+void address_space_set_debug_ops(const MemoryDebugOps *ops);
+
 static inline bool memory_access_is_direct(MemoryRegion *mr, bool is_write)
 {
     if (is_write) {
softmmu/physmem.c +20 -4
index a9adedb9f8..057d6d4ce1 100644
@@ -166,6 +166,18 @@ struct DirtyBitmapSnapshot {
     unsigned long dirty[];
 };
 
+static const MemoryDebugOps default_debug_ops = {
+    .read = address_space_read,
+    .write = address_space_write_rom
+};
+
+static const MemoryDebugOps *debug_ops = &default_debug_ops;
+
+void address_space_set_debug_ops(const MemoryDebugOps *ops)
+{
+    debug_ops = ops;
+}
+
 static void phys_map_node_reserve(PhysPageMap *map, unsigned nodes)
 {
     static unsigned alloc_hint = 16;
@@ -3407,6 +3419,10 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
         page = addr & TARGET_PAGE_MASK;
         phys_addr = cpu_get_phys_page_attrs_debug(cpu, page, &attrs);
         asidx = cpu_asidx_from_attrs(cpu, attrs);
+
+        /* set debug attrs to indicate memory access is from the debugger */
+        attrs.debug = 1;
+
         /* if no physical page mapped, return an error */
         if (phys_addr == -1)
             return -1;
@@ -3415,11 +3431,11 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
             l = len;
         phys_addr += (addr & ~TARGET_PAGE_MASK);
         if (is_write) {
-            res = address_space_write_rom(cpu->cpu_ases[asidx].as, phys_addr,
-                                          attrs, buf, l);
+            res = debug_ops->write(cpu->cpu_ases[asidx].as, phys_addr,
+                                   attrs, buf, l);
         } else {
-            res = address_space_read(cpu->cpu_ases[asidx].as, phys_addr,
-                                     attrs, buf, l);
+            res = debug_ops->read(cpu->cpu_ases[asidx].as, phys_addr,
+                                  attrs, buf, l);
         }
         if (res != MEMTX_OK) {
             return -1;
Signature · 1 line
2.17.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#3 lore raw

Re: [PATCH 01/11] memattrs: add debug attribute

In reply to Ashish Kalra
Quoted 5 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

From: Brijesh Singh <[email protected]>

Quoted 7 lines

Extend the MemTxAttrs to include a 'debug' flag. The flag can be used as general indicator that operation was triggered by the debugger.

A subsequent patch will set the debug=1 when issuing a memory access from the gdbstub or HMP commands. This is a prerequisite to support debugging an encrypted guest. When a request with debug=1 is seen, the encryption APIs will be used to access the guest memory.

So, what counts as "debug" here, and why are debug requests special? If "debug=1" means "can actually get at the guest memory", why wouldn't every device model want to use it?

thanks
-- PMM

#9 lore raw

Re: [PATCH 02/11] exec: Add new MemoryDebugOps.

In reply to Ashish Kalra
Quoted 10 lines · Ashish Kalra

From: Ashish Kalra <[email protected]>

Introduce new MemoryDebugOps which hook into guest virtual and physical memory debug interfaces such as cpu_memory_rw_debug, to allow vendor specific assist/hooks for debugging and delegating accessing the guest memory. This is required for example in case of AMD SEV platform where the guest memory is encrypted and a SEV specific debug assist/hook will be required to access the guest memory.

Quoted 2 lines

The MemoryDebugOps are used by cpu_memory_rw_debug() and default to address_space_read and address_space_write_rom.

This seems like a weird place to insert these hooks. Not all debug related accesses are going to go via cpu_memory_rw_debug(). For instance when the gdb stub is in "PhyMemMode" and all addresses from the debugger are treated as physical rather than virtual, gdbstub.c will call cpu_physical_memory_write()/_read().

I would have expected the "oh, this is a debug access, do something special" to be at a lower level, so that any address_space_* access to the guest memory with the debug attribute gets the magic treatment, whether that was done as a direct "read this physaddr" or via cpu_memory_rw_debug() doing the virt-to-phys conversion first.

thanks
-- PMM

#4 lore raw

Re: [PATCH 01/11] memattrs: add debug attribute

In reply to Peter Maydell
Quoted 9 lines · * Peter Maydell ([email protected]) wrote:
Quoted 8 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

From: Brijesh Singh <[email protected]>

Extend the MemTxAttrs to include a 'debug' flag. The flag can be used as general indicator that operation was triggered by the debugger.

Quoted 8 lines
Quoted 4 lines

A subsequent patch will set the debug=1 when issuing a memory access from the gdbstub or HMP commands. This is a prerequisite to support debugging an encrypted guest. When a request with debug=1 is seen, the encryption APIs will be used to access the guest memory.

So, what counts as "debug" here, and why are debug requests special? If "debug=1" means "can actually get at the guest memory", why wouldn't every device model want to use it?

SEV has a flag that the guest-owner can set on a VM to enable debug; it's rare for it to be enabled; so it's not suitable for use by normal devices. It's only there for debug if the guest owner allows you to.

Dave

Quoted 3 lines

thanks
-- PMM

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#20 lore raw

Re: [PATCH 06/11] monitor/i386: use debug APIs when accessing guest memory

In reply to Ashish Kalra
Quoted 19 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Update the HMP commands to use the debug version of APIs when accessing guest memory.

---
 monitor/misc.c        |  4 ++--
 softmmu/cpus.c        |  2 +-
 target/i386/monitor.c | 54 ++++++++++++++++++++++++-------------------
 3 files changed, 33 insertions(+), 27 deletions(-)
monitor/misc.c +0 -0
index 32e6a8c13d..7eba3a6fce 100644

monitor/misc.c static void hmp_sum(Monitor *mon, const QDict *qdict) · line 828

Quoted 7 lines
monitor/misc.c +2 -2
@@ -824,8 +824,8 @@ static void hmp_sum(Monitor *mon, const QDict *qdict)
 
     sum = 0;
     for(addr = start; addr < (start + size); addr++) {
-        uint8_t val = address_space_ldub(&address_space_memory, addr,
-                                         MEMTXATTRS_UNSPECIFIED, NULL);
+        uint8_t val;
+        cpu_physical_memory_read_debug(addr, &val, 1);

Don't introduce new uses of cpu_* memory read/write functions, please. They're an old API that has some flaws, like not being able to report read/write access errors.

If debug accesses are accesses with a MemTxAttrs that says debug=1, then you should just provide the right MemTxAttrs here.

thanks
-- PMM

#5 lore raw

Re: [PATCH 01/11] memattrs: add debug attribute

In reply to Dr. David Alan Gilbert
Quoted 16 lines · Dr. David Alan Gilbert
Quoted 14 lines · * Peter Maydell ([email protected]) wrote:
Quoted 12 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

From: Brijesh Singh <[email protected]>

Extend the MemTxAttrs to include a 'debug' flag. The flag can be used as general indicator that operation was triggered by the debugger.

A subsequent patch will set the debug=1 when issuing a memory access from the gdbstub or HMP commands. This is a prerequisite to support debugging an encrypted guest. When a request with debug=1 is seen, the encryption APIs will be used to access the guest memory.

Quoted 7 lines
Quoted 3 lines

So, what counts as "debug" here, and why are debug requests special? If "debug=1" means "can actually get at the guest memory", why wouldn't every device model want to use it?

SEV has a flag that the guest-owner can set on a VM to enable debug; it's rare for it to be enabled; so it's not suitable for use by normal devices. It's only there for debug if the guest owner allows you to.

So if I do a memory transaction with debug=1 then I should expect that it might come back with a failure status (meaning "this VM doesn't permit debug") and I should handle that error ?

thanks
-- PMM

#21 lore raw

Re: [PATCH 06/11] monitor/i386: use debug APIs when accessing guest memory

In reply to Ashish Kalra
Quoted 19 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Update the HMP commands to use the debug version of APIs when accessing guest memory.

---
 monitor/misc.c        |  4 ++--
 softmmu/cpus.c        |  2 +-
 target/i386/monitor.c | 54 ++++++++++++++++++++++++-------------------
 3 files changed, 33 insertions(+), 27 deletions(-)
monitor/misc.c +0 -0
index 32e6a8c13d..7eba3a6fce 100644

monitor/misc.c static void hmp_sum(Monitor *mon, const QDict *qdict) · line 831

Quoted 10 lines
monitor/misc.c +2 -2
@@ -824,8 +824,8 @@ static void hmp_sum(Monitor *mon, const QDict *qdict)
 
     sum = 0;
     for(addr = start; addr < (start + size); addr++) {
-        uint8_t val = address_space_ldub(&address_space_memory, addr,
-                                         MEMTXATTRS_UNSPECIFIED, NULL);
+        uint8_t val;
+        cpu_physical_memory_read_debug(addr, &val, 1);
         /* BSD sum algorithm ('sum' Unix command) */
         sum = (sum >> 1) | (sum << 15);
         sum += val;

Side note -- are byte-by-byte accesses to the encrypted guest memory noticeably higher overhead than if we asked for a larger buffer to be decrypted at once? If so and if anybody cares about hmp_sum performance we might consider having it work on a larger buffer at a time rather than byte-by-byte...

thanks
-- PMM

#14 lore raw

Re: [PATCH 03/11] exec: add ram_debug_ops support

In reply to Ashish Kalra
Quoted 9 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

From: Brijesh Singh <[email protected]>

Currently, guest memory access for debugging purposes is performed using memcpy(). Extend the 'struct MemoryRegion' to include new callbacks that can be used to override the use of memcpy() with something else.

Quoted 7 lines

The new callbacks can be used to display the guest memory of an SEV guest by registering callbacks to the SEV memory encryption/decryption APIs.

Typical usage:

mem_read(uint8_t *dst, uint8_t *src, uint32_t len, MemTxAttrs *attrs); mem_write(uint8_t *dst, uint8_t *src, uint32_t len, MemTxAttrs *attrs);

We already have a function prototype for "I need to call a function to do this read or write":

    MemTxResult (*read_with_attrs)(void *opaque,
                                   hwaddr addr,
                                   uint64_t *data,
                                   unsigned size,
                                   MemTxAttrs attrs);
    MemTxResult (*write_with_attrs)(void *opaque,
                                    hwaddr addr,
                                    uint64_t data,
                                    unsigned size,
                                    MemTxAttrs attrs);

Do the prototypes for accessing guest RAM that needs decryption really need to be different from that?

thanks
-- PMM

#10 lore raw

Re: [PATCH 02/11] exec: Add new MemoryDebugOps.

In reply to Peter Maydell
Quoted 21 lines · Peter Maydell
Quoted 10 lines · Ashish Kalra

From: Ashish Kalra <[email protected]>

Introduce new MemoryDebugOps which hook into guest virtual and physical memory debug interfaces such as cpu_memory_rw_debug, to allow vendor specific assist/hooks for debugging and delegating accessing the guest memory. This is required for example in case of AMD SEV platform where the guest memory is encrypted and a SEV specific debug assist/hook will be required to access the guest memory.

Quoted 2 lines

The MemoryDebugOps are used by cpu_memory_rw_debug() and default to address_space_read and address_space_write_rom.

This seems like a weird place to insert these hooks. Not all debug related accesses are going to go via cpu_memory_rw_debug(). For instance when the gdb stub is in "PhyMemMode" and all addresses from the debugger are treated as physical rather than virtual, gdbstub.c will call cpu_physical_memory_write()/_read().

Quoted 7 lines

I would have expected the "oh, this is a debug access, do something special" to be at a lower level, so that any address_space_* access to the guest memory with the debug attribute gets the magic treatment, whether that was done as a direct "read this physaddr" or via cpu_memory_rw_debug() doing the virt-to-phys conversion first.

Actually, the earlier patch-set used to do this at a lower level, i.e., at the address_space level, but then Paolo's feedback on that was that we don't want to add debug specific hooks into generic code such as address_space_* interfaces, hence, these hooks are introduced at a higher level so that we can do this "debug" abstraction at cpu_memory_rw_debug() and adding new interfaces for physical memory read/write debugging such as cpu_physical_memory_rw_debug().

This seems logical too as cpu_memory_rw_debug() is invoked via the debugger, i.e., either gdbstub or qemu monitor, so this interface seems to be the right place to add these hooks.

Thanks,
Ashish

#11 lore raw

Re: [PATCH 02/11] exec: Add new MemoryDebugOps.

Quoted 16 lines · Ashish Kalra
Quoted 7 lines · Peter Maydell

This seems like a weird place to insert these hooks. Not all debug related accesses are going to go via cpu_memory_rw_debug(). For instance when the gdb stub is in "PhyMemMode" and all addresses from the debugger are treated as physical rather than virtual, gdbstub.c will call cpu_physical_memory_write()/_read().

Quoted 7 lines

I would have expected the "oh, this is a debug access, do something special" to be at a lower level, so that any address_space_* access to the guest memory with the debug attribute gets the magic treatment, whether that was done as a direct "read this physaddr" or via cpu_memory_rw_debug() doing the virt-to-phys conversion first.

Quoted 7 lines

Actually, the earlier patch-set used to do this at a lower level, i.e., at the address_space level, but then Paolo's feedback on that was that we don't want to add debug specific hooks into generic code such as address_space_* interfaces, hence, these hooks are introduced at a higher level so that we can do this "debug" abstraction at cpu_memory_rw_debug() and adding new interfaces for physical memory read/write debugging such as cpu_physical_memory_rw_debug().

This seems to be mixing two separate designs, then. Either you want to try to provide separate "debug" functions like this, or you want to have a MemTxAttrs "debug" attribute, but you don't need both. Personally I prefer the MemTxAttrs approach (and disagree with Paolo :-)), because otherwise you're going to end up duplicating a lot of functions, and the handling of "this memory is encrypted and needs special handling" ends up being dealt with in various layers of the code rather than being only in one place where the lowest layer says "oh, debug access to encrypted memory, this is how to do that".

Quoted 3 lines

This seems logical too as cpu_memory_rw_debug() is invoked via the debugger, i.e., either gdbstub or qemu monitor, so this interface seems to be the right place to add these hooks.

Except that as noted, although all uses of cpu_memory_rw_debug() are debug related, not all debug related accesses are to cpu_memory_rw_debug()... The interesting characteristics of cpu_memory_rw_debug() are (1) it takes a virtual address rather than physical (2) it writes to ROMs (3) it refuses to write to devices.

thanks
-- PMM

#15 lore raw

Re: [PATCH 03/11] exec: add ram_debug_ops support

In reply to Peter Maydell
Quoted 31 lines · Peter Maydell
Quoted 9 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

From: Brijesh Singh <[email protected]>

Currently, guest memory access for debugging purposes is performed using memcpy(). Extend the 'struct MemoryRegion' to include new callbacks that can be used to override the use of memcpy() with something else.

Quoted 7 lines

The new callbacks can be used to display the guest memory of an SEV guest by registering callbacks to the SEV memory encryption/decryption APIs.

Typical usage:

mem_read(uint8_t *dst, uint8_t *src, uint32_t len, MemTxAttrs *attrs); mem_write(uint8_t *dst, uint8_t *src, uint32_t len, MemTxAttrs *attrs);

We already have a function prototype for "I need to call a function to do this read or write":

    MemTxResult (*read_with_attrs)(void *opaque,
                                   hwaddr addr,
                                   uint64_t *data,
                                   unsigned size,
                                   MemTxAttrs attrs);
    MemTxResult (*write_with_attrs)(void *opaque,
                                    hwaddr addr,
                                    uint64_t data,
                                    unsigned size,
                                    MemTxAttrs attrs);
Quoted 3 lines

Do the prototypes for accessing guest RAM that needs decryption really need to be different from that?

This again falls back to the same thought process, to keep the debug specific code separate from the generic code. If the above MemoryRegionOps are used, then again we are using generic code to invoke debug specific stuff.

Thanks,
Ashish

#12 lore raw

Re: [PATCH 02/11] exec: Add new MemoryDebugOps.

In reply to Peter Maydell
Quoted 25 lines · Peter Maydell
Quoted 16 lines · Ashish Kalra
Quoted 7 lines · Peter Maydell

This seems like a weird place to insert these hooks. Not all debug related accesses are going to go via cpu_memory_rw_debug(). For instance when the gdb stub is in "PhyMemMode" and all addresses from the debugger are treated as physical rather than virtual, gdbstub.c will call cpu_physical_memory_write()/_read().

Quoted 7 lines

I would have expected the "oh, this is a debug access, do something special" to be at a lower level, so that any address_space_* access to the guest memory with the debug attribute gets the magic treatment, whether that was done as a direct "read this physaddr" or via cpu_memory_rw_debug() doing the virt-to-phys conversion first.

Quoted 7 lines

Actually, the earlier patch-set used to do this at a lower level, i.e., at the address_space level, but then Paolo's feedback on that was that we don't want to add debug specific hooks into generic code such as address_space_* interfaces, hence, these hooks are introduced at a higher level so that we can do this "debug" abstraction at cpu_memory_rw_debug() and adding new interfaces for physical memory read/write debugging such as cpu_physical_memory_rw_debug().

Quoted 11 lines

This seems to be mixing two separate designs, then. Either you want to try to provide separate "debug" functions like this, or you want to have a MemTxAttrs "debug" attribute, but you don't need both. Personally I prefer the MemTxAttrs approach (and disagree with Paolo :-)), because otherwise you're going to end up duplicating a lot of functions, and the handling of "this memory is encrypted and needs special handling" ends up being dealt with in various layers of the code rather than being only in one place where the lowest layer says "oh, debug access to encrypted memory, this is how to do that".

I agree that we end up duplicating a lot of functions, but doesn't that keep this whole debugging stuff separate and clean and also isolated from generic code ?

Thanks,
Ashish

Quoted 11 lines
Quoted 3 lines

This seems logical too as cpu_memory_rw_debug() is invoked via the debugger, i.e., either gdbstub or qemu monitor, so this interface seems to be the right place to add these hooks.

Except that as noted, although all uses of cpu_memory_rw_debug() are debug related, not all debug related accesses are to cpu_memory_rw_debug()... The interesting characteristics of cpu_memory_rw_debug() are (1) it takes a virtual address rather than physical (2) it writes to ROMs (3) it refuses to write to devices.

#6 lore raw

Re: [PATCH 01/11] memattrs: add debug attribute

In reply to Peter Maydell
Quoted 21 lines · * Peter Maydell ([email protected]) wrote:
Quoted 20 lines · Dr. David Alan Gilbert
Quoted 17 lines · * Peter Maydell ([email protected]) wrote:
Quoted 5 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

From: Brijesh Singh <[email protected]>

Quoted 7 lines

Extend the MemTxAttrs to include a 'debug' flag. The flag can be used as general indicator that operation was triggered by the debugger.

A subsequent patch will set the debug=1 when issuing a memory access from the gdbstub or HMP commands. This is a prerequisite to support debugging an encrypted guest. When a request with debug=1 is seen, the encryption APIs will be used to access the guest memory.

So, what counts as "debug" here, and why are debug requests special? If "debug=1" means "can actually get at the guest memory", why wouldn't every device model want to use it?

Quoted 7 lines
Quoted 3 lines

SEV has a flag that the guest-owner can set on a VM to enable debug; it's rare for it to be enabled; so it's not suitable for use by normal devices. It's only there for debug if the guest owner allows you to.

So if I do a memory transaction with debug=1 then I should expect that it might come back with a failure status (meaning "this VM doesn't permit debug") and I should handle that error ?

I think that's probably true.

Dave

Quoted 3 lines

thanks
-- PMM

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK