Skip to contentSkip to conversation

target/{arm,i386}/hvf: Fix HVF migration

PATCH In review Scott J. Goldman 3 participants 5 messages Apr 27 – May 15

lore
Share & tools
lore
#0 lore raw

cover target/{arm,i386}/hvf: Fix HVF migration

From: "Scott J. Goldman" <[email protected]>

I've included two small HVF fixes that allow migration to work again. I've only tested on ARM, but beforehand, starting the migration would segfault immediately and now it works.

Scott J. Goldman (2):
target/arm/hvf, target/i386/hvf: Pass MR-relative offset to
memory_region_set_dirty()
target/arm/hvf: Stop pre-allocating cpreg_vmstate arrays

 target/arm/hvf/hvf.c  | 9 +--------
 target/i386/hvf/hvf.c | 2 +-
 2 files changed, 2 insertions(+), 9 deletions(-)
Signature · 1 line
2.50.1 (Apple Git-155)
#2 lore raw

PATCH 2/2 target/arm/hvf: Stop pre-allocating cpreg_vmstate arrays

Commit ab2ddc7b66 ("target/arm/machine: Use VMSTATE_VARRAY_INT32_ALLOC for cpreg arrays") moved cpreg_vmstate_indexes / cpreg_vmstate_values to be allocated by VMSTATE_VARRAY_INT32_ALLOC and added an assertion in cpu_pre_load() that they are NULL on entry. The same commit dropped the redundant g_renew()/array_len assignments from the kvm, whpx and helper.c cpu init paths, but the hvf cpu init path still pre-allocates them.

The result is that loading a snapshot or migration stream into an HVF guest immediately aborts:

    ERROR:target/arm/machine.c:1043:cpu_pre_load:
        assertion failed: (!cpu->cpreg_vmstate_indexes)

Drop the leftover cpreg_vmstate_indexes / cpreg_vmstate_values allocations and the cpreg_vmstate_array_len assignment from hvf_arch_init_vcpu(), matching what was already done for the other arm accelerators.

---
 target/arm/hvf/hvf.c | 7 -------
 1 file changed, 7 deletions(-)
target/arm/hvf/hvf.c +0 -7
index a36c634423..fe7dc9448d 100644
@@ -1284,12 +1284,6 @@ int hvf_arch_init_vcpu(CPUState *cpu)
                                      sregs_match_len);
     arm_cpu->cpreg_values = g_renew(uint64_t, arm_cpu->cpreg_values,
                                     sregs_match_len);
-    arm_cpu->cpreg_vmstate_indexes = g_renew(uint64_t,
-                                             arm_cpu->cpreg_vmstate_indexes,
-                                             sregs_match_len);
-    arm_cpu->cpreg_vmstate_values = g_renew(uint64_t,
-                                            arm_cpu->cpreg_vmstate_values,
-                                            sregs_match_len);
 
     memset(arm_cpu->cpreg_values, 0, sregs_match_len * sizeof(uint64_t));
 
@@ -1326,7 +1320,6 @@ int hvf_arch_init_vcpu(CPUState *cpu)
         }
     }
     arm_cpu->cpreg_array_len = sregs_cnt;
-    arm_cpu->cpreg_vmstate_array_len = sregs_cnt;
 
     /* cpreg tuples must be in strictly ascending order */
     qsort(arm_cpu->cpreg_indexes, sregs_cnt, sizeof(uint64_t), compare_u64);
Signature · 1 line
2.50.1 (Apple Git-155)
#1 lore raw

PATCH 1/2 target/arm/hvf, target/i386/hvf: Pass MR-relative offset to memory_region_set_dirty()

In reply to Scott J. Goldman

Both the arm and i386 hvf accelerators have the same bug in their dirty-page logging path: the address fed to memory_region_set_dirty() is computed as "<ipa,gpa>_page + xlat", but memory_region_set_dirty() expects an offset relative to the start of the resolved MemoryRegion. address_space_translate() already returns that offset in xlat, while ipa_page / gpa_page is the guest-physical (system address space) address.

Adding the two together produces a bogus offset that for any non- trivial RAM size walks well past the end of the MemoryRegion's dirty bitmap. With dirty logging active (e.g. live migration on a guest with several GB of RAM), this triggers an out-of-bounds atomic write inside bitmap_set_atomic() and crashes the source QEMU as soon as the guest writes to RAM:

Thread .. 'CPU N/HVF', stop reason = EXC_BAD_ACCESS ...
bitmap_set_atomic at bitmap.c:213 physical_memory_set_dirty_range at physmem.c:1038 memory_region_set_dirty at memory.c:2191 hvf_handle_exception at hvf.c

Fix it by passing only the MR-relative offset xlat. ipa_page / gpa_page is still the right argument to hvf_unprotect_dirty_range(), which works on the guest-physical address space.

---
 target/arm/hvf/hvf.c  | 2 +-
 target/i386/hvf/hvf.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
target/arm/hvf/hvf.c +1 -1
index a19d7a5e1f..a36c634423 100644
@@ -2243,7 +2243,7 @@ static int hvf_handle_exception(CPUState *cpu, hv_vcpu_exit_exception_t *excp)
                 assert(!mr->readonly);
 
                 if (memory_region_get_dirty_log_mask(mr)) {
-                    memory_region_set_dirty(mr, ipa_page + xlat, page_size);
+                    memory_region_set_dirty(mr, xlat, page_size);
                     hvf_unprotect_dirty_range(ipa_page, page_size);
                 }
 
target/i386/hvf/hvf.c +1 -1
index c0d028b147..cdc8bd1950 100644
@@ -146,7 +146,7 @@ static bool ept_emulation_fault(CPUState *cs, uint64_t gpa, uint64_t ept_qual)
     if (write && memory_region_get_dirty_log_mask(mr)) {
         uintptr_t page_size = qemu_real_host_page_size();
 
-        memory_region_set_dirty(mr, gpa_page + xlat, page_size);
+        memory_region_set_dirty(mr, xlat, page_size);
         hvf_unprotect_dirty_range(gpa_page, page_size);
     }
 
Signature · 1 line
2.50.1 (Apple Git-155)
Reviewed-by on v1 2/2
#3 lore raw

Re: [PATCH 2/2] target/arm/hvf: Stop pre-allocating cpreg_vmstate arrays

In reply to Scott J. Goldman
Quoted 8 lines · Scott J. Goldman

Commit ab2ddc7b66 ("target/arm/machine: Use VMSTATE_VARRAY_INT32_ALLOC for cpreg arrays") moved cpreg_vmstate_indexes / cpreg_vmstate_values to be allocated by VMSTATE_VARRAY_INT32_ALLOC and added an assertion in cpu_pre_load() that they are NULL on entry. The same commit dropped the redundant g_renew()/array_len assignments from the kvm, whpx and helper.c cpu init paths, but the hvf cpu init path still pre-allocates them.

Quoted 5 lines

The result is that loading a snapshot or migration stream into an HVF guest immediately aborts:

     ERROR:target/arm/machine.c:1043:cpu_pre_load:
         assertion failed: (!cpu->cpreg_vmstate_indexes)

Ah! Thanks :)

Quoted 9 lines

Drop the leftover cpreg_vmstate_indexes / cpreg_vmstate_values allocations and the cpreg_vmstate_array_len assignment from hvf_arch_init_vcpu(), matching what was already done for the other arm accelerators.

---
  target/arm/hvf/hvf.c | 7 -------
  1 file changed, 7 deletions(-)
#4 lore raw
In reply to Scott J. Goldman
Quoted 11 lines · Scott J. Goldman

From: "Scott J. Goldman" <[email protected]>

I've included two small HVF fixes that allow migration to work again. I've only tested on ARM, but beforehand, starting the migration would segfault immediately and now it works.

Scott J. Goldman (2):
target/arm/hvf, target/i386/hvf: Pass MR-relative offset to
memory_region_set_dirty()
target/arm/hvf: Stop pre-allocating cpreg_vmstate arrays

It looks like patch 1 has already been picked up into git; I'm taking patch 2 into target-arm.next.

thanks
-- PMM