target/{arm,i386}/hvf: Fix HVF migration
cover target/{arm,i386}/hvf: Fix HVF migration
From: "Scott J. Goldman" <[email protected]>
I've included two small HVF fixes that allow migration to work again. I've only tested on ARM, but beforehand, starting the migration would segfault immediately and now it works.
Scott J. Goldman (2):
target/arm/hvf, target/i386/hvf: Pass MR-relative offset to
memory_region_set_dirty()
target/arm/hvf: Stop pre-allocating cpreg_vmstate arrays
target/arm/hvf/hvf.c | 9 +-------- target/i386/hvf/hvf.c | 2 +- 2 files changed, 2 insertions(+), 9 deletions(-)
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH 1/2 target/arm/hvf, target/i386/hvf: Pass MR-relative offset to memory_region_set_dirty()
Both the arm and i386 hvf accelerators have the same bug in their dirty-page logging path: the address fed to memory_region_set_dirty() is computed as "<ipa,gpa>_page + xlat", but memory_region_set_dirty() expects an offset relative to the start of the resolved MemoryRegion. address_space_translate() already returns that offset in xlat, while ipa_page / gpa_page is the guest-physical (system address space) address.
Adding the two together produces a bogus offset that for any non- trivial RAM size walks well past the end of the MemoryRegion's dirty bitmap. With dirty logging active (e.g. live migration on a guest with several GB of RAM), this triggers an out-of-bounds atomic write inside bitmap_set_atomic() and crashes the source QEMU as soon as the guest writes to RAM:
Thread .. 'CPU N/HVF', stop reason = EXC_BAD_ACCESS ...
bitmap_set_atomic at bitmap.c:213 physical_memory_set_dirty_range at physmem.c:1038 memory_region_set_dirty at memory.c:2191 hvf_handle_exception at hvf.c
Fix it by passing only the MR-relative offset xlat. ipa_page / gpa_page is still the right argument to hvf_unprotect_dirty_range(), which works on the guest-physical address space.
--- target/arm/hvf/hvf.c | 2 +- target/i386/hvf/hvf.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
target/arm/hvf/hvf.c +1 -1
@@ -2243,7 +2243,7 @@ static int hvf_handle_exception(CPUState *cpu, hv_vcpu_exit_exception_t *excp)assert (!mr->readonly);if (memory_region_get_dirty_log_mask (mr)) { -memory_region_set_dirty (mr, ipa_page + xlat, page_size); +memory_region_set_dirty (mr, xlat, page_size);hvf_unprotect_dirty_range (ipa_page, page_size); }
target/i386/hvf/hvf.c +1 -1
@@ -146,7 +146,7 @@ static bool ept_emulation_fault(CPUState *cs, uint64_t gpa, uint64_t ept_qual)if (write &&memory_region_get_dirty_log_mask (mr)) {uintptr_t page_size =qemu_real_host_page_size (); -memory_region_set_dirty (mr, gpa_page + xlat, page_size); +memory_region_set_dirty (mr, xlat, page_size);hvf_unprotect_dirty_range (gpa_page, page_size); }
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH 2/2 target/arm/hvf: Stop pre-allocating cpreg_vmstate arrays
In reply to Scott J. GoldmanCommit ab2ddc7b66 ("target/arm/machine: Use VMSTATE_VARRAY_INT32_ALLOC for cpreg arrays") moved cpreg_vmstate_indexes / cpreg_vmstate_values to be allocated by VMSTATE_VARRAY_INT32_ALLOC and added an assertion in cpu_pre_load() that they are NULL on entry. The same commit dropped the redundant g_renew()/array_len assignments from the kvm, whpx and helper.c cpu init paths, but the hvf cpu init path still pre-allocates them.
The result is that loading a snapshot or migration stream into an HVF guest immediately aborts:
ERROR:target/arm/machine.c:1043:cpu_pre_load:
assertion failed: (!cpu->cpreg_vmstate_indexes)Drop the leftover cpreg_vmstate_indexes / cpreg_vmstate_values allocations and the cpreg_vmstate_array_len assignment from hvf_arch_init_vcpu(), matching what was already done for the other arm accelerators.
--- target/arm/hvf/hvf.c | 7 ------- 1 file changed, 7 deletions(-)
target/arm/hvf/hvf.c +0 -7
@@ -1284,12 +1284,6 @@ int hvf_arch_init_vcpu(CPUState *cpu) sregs_match_len); arm_cpu->cpreg_values =g_renew (uint64_t, arm_cpu->cpreg_values, sregs_match_len); - arm_cpu->cpreg_vmstate_indexes =g_renew (uint64_t, - arm_cpu->cpreg_vmstate_indexes, - sregs_match_len); - arm_cpu->cpreg_vmstate_values =g_renew (uint64_t, - arm_cpu->cpreg_vmstate_values, - sregs_match_len);memset (arm_cpu->cpreg_values,0 , sregs_match_len *sizeof (uint64_t ));
@@ -1326,7 +1320,6 @@ int hvf_arch_init_vcpu(CPUState *cpu) } } arm_cpu->cpreg_array_len = sregs_cnt; - arm_cpu->cpreg_vmstate_array_len = sregs_cnt;/* cpreg tuples must be in strictly ascending order */ qsort (arm_cpu->cpreg_indexes, sregs_cnt,sizeof (uint64_t ), compare_u64);
Signature · 1 line
2.50.1 (Apple Git-155)
Re: [PATCH 2/2] target/arm/hvf: Stop pre-allocating cpreg_vmstate arrays
Quoted 8 lines · Scott J. Goldman
Commit
ab2ddc7b66("target/arm/machine: UseVMSTATE_VARRAY_INT32_ALLOCfor cpreg arrays") movedcpreg_vmstate_indexes/cpreg_vmstate_valuesto be allocated byVMSTATE_VARRAY_INT32_ALLOCand added an assertion incpu_pre_load()that they are NULL on entry. The same commit dropped the redundant g_renew()/array_len assignments from the kvm, whpx andhelper.ccpu init paths, but the hvf cpu init path still pre-allocates them.
Quoted 5 lines
The result is that loading a snapshot or migration stream into an HVF guest immediately aborts:
ERROR:target/arm/machine.c:1043:cpu_pre_load: assertion failed: (!cpu->cpreg_vmstate_indexes)
Ah! Thanks :)
Quoted 9 lines
Drop the leftover
cpreg_vmstate_indexes/cpreg_vmstate_valuesallocations and thecpreg_vmstate_array_lenassignment fromhvf_arch_init_vcpu(), matching what was already done for the other arm accelerators.Signed-off-by Scott J. Goldman <[email protected]>--- target/arm/hvf/hvf.c | 7 ------- 1 file changed, 7 deletions(-)
Quoted 11 lines · Scott J. Goldman
From: "Scott J. Goldman" <[email protected]>
I've included two small HVF fixes that allow migration to work again. I've only tested on ARM, but beforehand, starting the migration would segfault immediately and now it works.
Scott J. Goldman (2):
target/arm/hvf, target/i386/hvf: Pass MR-relative offset to
memory_region_set_dirty()
target/arm/hvf: Stop pre-allocatingcpreg_vmstatearrays
It looks like patch 1 has already been picked up into git; I'm taking patch 2 into target-arm.next.
thanks
-- PMM