Skip to contentSkip to conversation

softmmu/memory_mapping: optimize dump/tpm for virtio-mem

PATCH Superseded David Hildenbrand 1 participant 6 messages Mar 1, 2021

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover softmmu/memory_mapping: optimize dump/tpm for virtio-mem

Minor fixes and cleanups, followed by an optimization for virtio-mem regarding guest dumps and tpm.

virtio-mem logically plugs/unplugs memory within a sparse memory region and notifies via the RamDiscardMgr interface when parts become plugged (populated) or unplugged (discarded).

Currently, guest_phys_blocks_append() appends the whole (sparse) virtio-mem managed region and therefore tpm code might zero the hole region and dump code will dump the whole region. Let's only add logically plugged (populated) parts of that region, skipping over logically unplugged (discarded) parts by reusing the RamDiscardMgr infrastructure introduced to handle virtio-mem + VFIO properly.

Based-on: [email protected]

v1 -> v2:
- "softmmu/memory_mapping: factor out adding physical memory ranges" -- Simplify based on RamDiscardManager changes: add using a MemoryRegionSection
- "softmmu/memory_mapping: optimize for RamDiscardManager sections" -- Simplify based on RamDiscardManager changes

David Hildenbrand (5):
tpm: mark correct memory region range dirty when clearing RAM softmmu/memory_mapping: reuse qemu_get_guest_simple_memory_mapping() softmmu/memory_mapping: never merge ranges accross memory regions softmmu/memory_mapping: factor out adding physical memory ranges softmmu/memory_mapping: optimize for RamDiscardManager sections

 hw/tpm/tpm_ppi.c         |  4 ++-
 softmmu/memory_mapping.c | 72 ++++++++++++++++++++++++----------------
 2 files changed, 46 insertions(+), 30 deletions(-)
Signature · 1 line
2.29.2
#1 lore raw

PATCH v2 1/5 tpm: mark correct memory region range dirty when clearing RAM

We might not start at the beginning of the memory region. We could also calculate via the difference in the host address; however, memory_region_set_dirty() also relies on memory_region_get_ram_addr() internally, so let's just use that.

Fixes ffab1be70692 ("tpm: clear RAM when "memory overwrite" requested")
Cc Marc-André Lureau <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>
Cc Alex Williamson <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Claudio Fontana <[email protected]>
Cc Thomas Huth <[email protected]>
Cc "Alex Bennée" <[email protected]>
Cc Peter Xu <[email protected]>
Cc Laurent Vivier <[email protected]>
Cc Stefan Berger <[email protected]>
---
 hw/tpm/tpm_ppi.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
hw/tpm/tpm_ppi.c +3 -1
index 72d7a3d926..e0e2d2c8e1 100644
@@ -30,11 +30,13 @@ void tpm_ppi_reset(TPMPPI *tpmppi)
         guest_phys_blocks_init(&guest_phys_blocks);
         guest_phys_blocks_append(&guest_phys_blocks);
         QTAILQ_FOREACH(block, &guest_phys_blocks.head, next) {
+            ram_addr_t mr_start = memory_region_get_ram_addr(block->mr);
+
             trace_tpm_ppi_memset(block->host_addr,
                                  block->target_end - block->target_start);
             memset(block->host_addr, 0,
                    block->target_end - block->target_start);
-            memory_region_set_dirty(block->mr, 0,
+            memory_region_set_dirty(block->mr, block->target_start - mr_start,
                                     block->target_end - block->target_start);
         }
         guest_phys_blocks_free(&guest_phys_blocks);
Signature · 1 line
2.29.2
#2 lore raw

PATCH v2 2/5 softmmu/memory_mapping: reuse qemu_get_guest_simple_memory_mapping()

In reply to David Hildenbrand

Let's reuse qemu_get_guest_simple_memory_mapping(), which does exactly what we want.

Cc Marc-André Lureau <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>
Cc Alex Williamson <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Claudio Fontana <[email protected]>
Cc Thomas Huth <[email protected]>
Cc "Alex Bennée" <[email protected]>
Cc Peter Xu <[email protected]>
Cc Laurent Vivier <[email protected]>
Cc Stefan Berger <[email protected]>
---
 softmmu/memory_mapping.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
softmmu/memory_mapping.c +1 -7
index 18d0b8067c..2677392de7 100644
@@ -289,8 +289,6 @@ void qemu_get_guest_memory_mapping(MemoryMappingList *list,
                                    Error **errp)
 {
     CPUState *cpu, *first_paging_enabled_cpu;
-    GuestPhysBlock *block;
-    ram_addr_t offset, length;
 
     first_paging_enabled_cpu = find_paging_enabled_cpu(first_cpu);
     if (first_paging_enabled_cpu) {
@@ -310,11 +308,7 @@ void qemu_get_guest_memory_mapping(MemoryMappingList *list,
      * If the guest doesn't use paging, the virtual address is equal to physical
      * address.
      */
-    QTAILQ_FOREACH(block, &guest_phys_blocks->head, next) {
-        offset = block->target_start;
-        length = block->target_end - block->target_start;
-        create_new_memory_mapping(list, offset, offset, length);
-    }
+    qemu_get_guest_simple_memory_mapping(list, guest_phys_blocks);
 }
 
 void qemu_get_guest_simple_memory_mapping(MemoryMappingList *list,
Signature · 1 line
2.29.2
#3 lore raw

PATCH v2 3/5 softmmu/memory_mapping: never merge ranges accross memory regions

In reply to David Hildenbrand

Let's make sure to not merge when different memory regions are involved. Unlikely, but theoretically possible.

Cc Marc-André Lureau <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>
Cc Alex Williamson <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Claudio Fontana <[email protected]>
Cc Thomas Huth <[email protected]>
Cc "Alex Bennée" <[email protected]>
Cc Peter Xu <[email protected]>
Cc Laurent Vivier <[email protected]>
Cc Stefan Berger <[email protected]>
---
 softmmu/memory_mapping.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
softmmu/memory_mapping.c +2 -1
index 2677392de7..ad4911427a 100644
@@ -230,7 +230,8 @@ static void guest_phys_blocks_region_add(MemoryListener *listener,
 
         /* we want continuity in both guest-physical and host-virtual memory */
         if (predecessor->target_end < target_start ||
-            predecessor->host_addr + predecessor_size != host_addr) {
+            predecessor->host_addr + predecessor_size != host_addr ||
+            predecessor->mr != section->mr) {
             predecessor = NULL;
         }
     }
Signature · 1 line
2.29.2
#4 lore raw

PATCH v2 4/5 softmmu/memory_mapping: factor out adding physical memory ranges

In reply to David Hildenbrand

Let's factor out adding a MemoryRegionSection to the list, to be reused in RamDiscardManager context next.

Cc Marc-André Lureau <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>
Cc Alex Williamson <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Claudio Fontana <[email protected]>
Cc Thomas Huth <[email protected]>
Cc "Alex Bennée" <[email protected]>
Cc Peter Xu <[email protected]>
Cc Laurent Vivier <[email protected]>
Cc Stefan Berger <[email protected]>
---
 softmmu/memory_mapping.c | 41 ++++++++++++++++++++--------------------
 1 file changed, 20 insertions(+), 21 deletions(-)
softmmu/memory_mapping.c +20 -21
index ad4911427a..28d28cc702 100644
@@ -194,29 +194,14 @@ typedef struct GuestPhysListener {
     MemoryListener listener;
 } GuestPhysListener;
 
-static void guest_phys_blocks_region_add(MemoryListener *listener,
+static void guest_phys_block_add_section(GuestPhysListener *g,
                                          MemoryRegionSection *section)
 {
-    GuestPhysListener *g;
-    uint64_t section_size;
-    hwaddr target_start, target_end;
-    uint8_t *host_addr;
-    GuestPhysBlock *predecessor;
-
-    /* we only care about RAM */
-    if (!memory_region_is_ram(section->mr) ||
-        memory_region_is_ram_device(section->mr) ||
-        memory_region_is_nonvolatile(section->mr)) {
-        return;
-    }
-
-    g            = container_of(listener, GuestPhysListener, listener);
-    section_size = int128_get64(section->size);
-    target_start = section->offset_within_address_space;
-    target_end   = target_start + section_size;
-    host_addr    = memory_region_get_ram_ptr(section->mr) +
-                   section->offset_within_region;
-    predecessor  = NULL;
+    const hwaddr target_start = section->offset_within_address_space;
+    const hwaddr target_end = target_start + int128_get64(section->size);
+    uint8_t *host_addr = memory_region_get_ram_ptr(section->mr) +
+                         section->offset_within_region;
+    GuestPhysBlock *predecessor = NULL;
 
     /* find continuity in guest physical address space */
     if (!QTAILQ_EMPTY(&g->list->head)) {
@@ -262,6 +247,20 @@ static void guest_phys_blocks_region_add(MemoryListener *listener,
 #endif
 }
 
+static void guest_phys_blocks_region_add(MemoryListener *listener,
+                                         MemoryRegionSection *section)
+{
+    GuestPhysListener *g = container_of(listener, GuestPhysListener, listener);
+
+    /* we only care about RAM */
+    if (!memory_region_is_ram(section->mr) ||
+        memory_region_is_ram_device(section->mr) ||
+        memory_region_is_nonvolatile(section->mr)) {
+        return;
+    }
+    guest_phys_block_add_section(g, section);
+}
+
 void guest_phys_blocks_append(GuestPhysBlockList *list)
 {
     GuestPhysListener g = { 0 };
Signature · 1 line
2.29.2
#5 lore raw

PATCH v2 5/5 softmmu/memory_mapping: optimize for RamDiscardManager sections

In reply to David Hildenbrand

virtio-mem logically plugs/unplugs memory within a sparse memory region and notifies via the RamDiscardManager interface when parts become plugged (populated) or unplugged (discarded).

Currently, we end up (via the two users)
1) zeroing all logically unplugged/discarded memory during TPM resets.
2) reading all logically unplugged/discarded memory when dumping, to figure out the content is zero.

1) is always bad, because we assume unplugged memory stays discarded (and is already implicitly zero).
2) isn't that bad with anonymous memory, we end up reading the zero page (slow and unnecessary, though). However, once we use some file-backed memory (future use case), even reading will populate memory.

Let's cut out all parts marked as not-populated (discarded) via the RamDiscardManager. As virtio-mem is the single user, this now means that logically unplugged memory ranges will no longer be included in the dump, which results in smaller dump files and faster dumping.

virtio-mem has a minimum granularity of 1 MiB (and the default is usually 2 MiB). Theoretically, we can see quite some fragmentation, in practice we won't have it completely fragmented in 1 MiB pieces. Still, we might end up with many physical ranges.

Both, the ELF format and kdump seem to be ready to support many individual ranges (e.g., for ELF it seems to be UINT32_MAX, kdump has a linear bitmap).

Cc Marc-André Lureau <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>
Cc Alex Williamson <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Claudio Fontana <[email protected]>
Cc Thomas Huth <[email protected]>
Cc "Alex Bennée" <[email protected]>
Cc Peter Xu <[email protected]>
Cc Laurent Vivier <[email protected]>
Cc Stefan Berger <[email protected]>
---
 softmmu/memory_mapping.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
softmmu/memory_mapping.c +20 -0
index 28d28cc702..30a12d4321 100644
@@ -247,6 +247,15 @@ static void guest_phys_block_add_section(GuestPhysListener *g,
 #endif
 }
 
+static int guest_phys_ram_populate_cb(MemoryRegionSection *section,
+                                      void *opaque)
+{
+    GuestPhysListener *g = opaque;
+
+    guest_phys_block_add_section(g, section);
+    return 0;
+}
+
 static void guest_phys_blocks_region_add(MemoryListener *listener,
                                          MemoryRegionSection *section)
 {
@@ -258,6 +267,17 @@ static void guest_phys_blocks_region_add(MemoryListener *listener,
         memory_region_is_nonvolatile(section->mr)) {
         return;
     }
+
+    /* for special sparse regions, only add populated parts */
+    if (memory_region_has_ram_discard_manager(section->mr)) {
+        RamDiscardManager *rdm;
+
+        rdm = memory_region_get_ram_discard_manager(section->mr);
+        ram_discard_manager_replay_populated(rdm, section,
+                                             guest_phys_ram_populate_cb, g);
+        return;
+    }
+
     guest_phys_block_add_section(g, section);
 }
 
Signature · 1 line
2.29.2