Skip to contentSkip to conversation

Mitigation of "failed to load cpu:cpreg_vmstate_array_len" migration failures

PATCH Superseded Eric Auger 2 participants 14 messages Jan 21 – Jan 26

v5 of 10 Superseded Latest is v10 →

lore
Share & tools
lore

Eric Auger

author
#0 lore raw

v5 cover Mitigation of "failed to load cpu:cpreg_vmstate_array_len" migration failures

When migrating ARM guests accross same machines with different host kernels we are likely to encounter failures such as:

"failed to load cpu:cpreg_vmstate_array_len"

This is due to the fact KVM exposes a different number of registers to qemu on source and destination. When trying to migrate a bigger register set to a smaller one, qemu cannot save the CPU state.

For example, recently we faced such kind of situations with:
- unconditionnal exposure of KVM_REG_ARM_VENDOR_HYP_BMAP_2 FW pseudo register from v6.16 onwards. Causes backward migration failure.
- removal of unconditionnal exposure of TCR2_EL1, PIRE0_EL1, PIR_EL1 from v6.13 onwards. Causes forward migration failure.

This situation is really problematic for distributions which want to guarantee forward and backward migration of a given machine type between different releases.

While the series mainly targets KVM acceleration, this problem also exists with TCG. For instance some registers may be exposed while they shouldn't. Then it is tricky to fix that situation without breaking forward migration. An example was provided by
Peter: 4f2b82f60 ("target/arm: Reinstate bogus AArch32 DBGDTRTX register for migration compat).

This series introduces 2 CPU array properties that list
- the CPU registers to hide from the exposes sysregs (aims at removing registers from the destination)
- The CPU registers that may not exist but which can be found in the incoming migration stream (aims at ignoring extra registers in the incoming state)

An example is given to illustrate how those props
could be used to apply compats for machine types supposed to "see" the same register set accross various host kernels.

Mitigation of DBGDTRTX issue would be achieved by setting x-mig-safe-missing-regs=0x40200000200e0298 which matches AArch32 DBGDTRTX register index.

The first patch improves the tracing so that we can quickly detect which registers do not match between the incoming stream and the exposed sysregs

---

Available at:
https://github.com/eauger/qemu/tree/mitig-v5

---

Tests:
- migration with 10.2 machine with old qemu featuring DBGDTRTX and this one where it is removed. Forward migration works. backward doesn't because the register is not present in the input migration stream and write_list_to_cpustate() fails while write_raw_cp_reg and reading it back. write_raw_cp_reg() seems to read an unintialized values from cpu->cpreg_values[i]. write has no effect since type is ARM_CP_CONST but read_raw_cp_reg returns ri->resetvalue which differs from uninitialized value. I would have expected the initial cpu->cpreg_values[i] to match reset value which is obviously not the case. Laso the comment hints that it should be. So maybe another issue? Nevertheless I am not totally sure supporting backward migration for TCG is a must. This may be fixed separately if it is confirmed this is a bug.

- migration with accel=kvm back and forth old host/qemu where host does not feature fixes for TCR2_EL1, PIRE0_EL1, PIR_EL1 and recent KVM_REG_ARM_VENDOR_HYP_BMAP_2 FW and more recent kernel/this qemu that feature them. Migration works forward and backward with 10.1 machine type.

History:

v4 -> v5:
- Fixed issue reported by Sebastian about aggregated array props. This lead to the introduction of hw/arm/virt: Introduce framework to aggregate hidden-regs and safe-missing-regs
- Collected additional hacks from Connie

v3 -> v4:
- Collected Connie's & Sebastian's R-bs
- Squashed patches 3 and 5
- various typos and rewording

v2 -> v3:
- revert target/arm: Reinstate bogus AArch32 DBGDTRTX register for migration compat
- fix some typos and rework target/arm/cpu.h hidden_regs comment (Connie)
- Even for TCG we use KVM index

v1 -> v2:
- fixed typos (Connie)
- Make it less KVM specific (tentative hidding of TCG regs, not tested)
- Tested DBGDTRTX TCG case reported by Peter
- No change to the property format yet. Ran out of idea. However I changed the name of the property with x-mig prefix
- Changed the terminology, kept hidding but remove fake which was confusing
- Simplified the logic for regs missing in the incoming stream and do not check anymore they are exposed on dest

Eric Auger (11):
hw/arm/virt: Rename arm_virt_compat into arm_virt_compat_defaults target/arm/machine: Improve traces on register mismatch during
migration
target/arm/cpu: Allow registers to be hidden target/arm/machine: Allow extra regs in the incoming stream
kvm-all: Enforce hidden regs are never accessed target/arm/cpu: Implement hide_reg callback() target/arm/cpu: Expose x-mig-hidden-regs and x-mig-safe-missing-regs
properties
hw/arm/virt: Declare AArch32 DBGDTRTX as safe to ignore in incoming
stream
Revert "target/arm: Reinstate bogus AArch32 DBGDTRTX register for
migration compat"
hw/arm/virt: Introduce framework to aggregate hidden-regs and
safe-missing-regs
hw/arm/virt: [DO NOT UPSTREAM] Enforce compatibility with older
kernels

 include/hw/arm/virt.h     | 23 +++++++++++
 include/hw/core/cpu.h     |  2 +
 target/arm/cpu.h          | 48 ++++++++++++++++++++++
 accel/kvm/kvm-all.c       | 12 ++++++
 hw/arm/virt.c             | 86 ++++++++++++++++++++++++++++++++++++---
 target/arm/cpu.c          | 11 +++++
 target/arm/debug_helper.c | 29 -------------
 target/arm/helper.c       | 12 +++++-
 target/arm/kvm.c          | 35 +++++++++++++++-
 target/arm/machine.c      | 70 ++++++++++++++++++++++++++++---
 target/arm/trace-events   | 10 +++++
 11 files changed, 295 insertions(+), 43 deletions(-)
Signature · 1 line
2.52.0
author
#1 lore raw

PATCH v5 1/11 hw/arm/virt: Rename arm_virt_compat into arm_virt_compat_defaults

Renaming arm_virt_compat into arm_virt_compat_defaults makes more obvious that those compats apply to all machine types by default, if not overriden for specific ones. This also matches the terminology used for pc-q35.

---
 hw/arm/virt.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
hw/arm/virt.c +7 -6
index 4badc1a7348..baa4e31aac1 100644
@@ -94,20 +94,21 @@
 #include "hw/cxl/cxl_host.h"
 #include "qemu/guest-random.h"
 
-static GlobalProperty arm_virt_compat[] = {
+static GlobalProperty arm_virt_compat_defaults[] = {
     { TYPE_VIRTIO_IOMMU_PCI, "aw-bits", "48" },
 };
-static const size_t arm_virt_compat_len = G_N_ELEMENTS(arm_virt_compat);
+static const size_t arm_virt_compat_defaults_len =
+    G_N_ELEMENTS(arm_virt_compat_defaults);
 
 /*
  * This cannot be called from the virt_machine_class_init() because
  * TYPE_VIRT_MACHINE is abstract and mc->compat_props g_ptr_array_new()
  * only is called on virt non abstract class init.
  */
-static void arm_virt_compat_set(MachineClass *mc)
+static void arm_virt_compat_default_set(MachineClass *mc)
 {
-    compat_props_add(mc->compat_props, arm_virt_compat,
-                     arm_virt_compat_len);
+    compat_props_add(mc->compat_props, arm_virt_compat_defaults,
+                     arm_virt_compat_defaults_len);
 }
 
 #define DEFINE_VIRT_MACHINE_IMPL(latest, ...) \
@@ -116,7 +117,7 @@ static void arm_virt_compat_set(MachineClass *mc)
         const void *data) \
     { \
         MachineClass *mc = MACHINE_CLASS(oc); \
-        arm_virt_compat_set(mc); \
+        arm_virt_compat_default_set(mc); \
         MACHINE_VER_SYM(options, virt, __VA_ARGS__)(mc); \
         mc->desc = "QEMU " MACHINE_VER_STR(__VA_ARGS__) " ARM Virtual Machine"; \
         MACHINE_VER_DEPRECATION(__VA_ARGS__); \
Signature · 1 line
2.52.0
author
#2 lore raw

PATCH v5 2/11 target/arm/machine: Improve traces on register mismatch during migration

In reply to Eric Auger

Currently when the number of KVM registers exposed by the source is larger than the one exposed on the destination, the migration fails
with: "failed to load cpu:cpreg_vmstate_array_len"

This gives no information about which registers are causing the trouble.

This patch reworks the target/arm/machine code so that it becomes able to handle an input stream with a larger set of registers than the destination and print useful information about which registers are causing the trouble. The migration outcome is unchanged:
- unexpected registers still will fail the migration
- missing ones are printed but will not fail the migration, as done today.

The input stream can contain MAX_CPREG_VMSTATE_ANOMALIES(10) extra registers compared to what exists on the target.

If there are more registers we will still hit the previous "load cpu:cpreg_vmstate_array_len" error.

At most, MAX_CPREG_VMSTATE_ANOMALIES missing registers and MAX_CPREG_VMSTATE_ANOMALIES unexpected registers are printed.

Example:

qemu-system-aarch64: kvm_arm_cpu_post_load Missing register in input stream: 0 0x6030000000160003 fw feat reg 3
qemu-system-aarch64: kvm_arm_cpu_post_load Unexpected register in input stream: 0 0x603000000013c103 op0:3 op1:0 crn:2 crm:0 op2:3
qemu-system-aarch64: kvm_arm_cpu_post_load Unexpected register in input stream: 1 0x603000000013c512 op0:3 op1:0 crn:10 crm:2 op2:2
qemu-system-aarch64: kvm_arm_cpu_post_load Unexpected register in input stream: 2 0x603000000013c513 op0:3 op1:0 crn:10 crm:2 op2:3
qemu-system-aarch64: error while loading state for instance 0x0 of device 'cpu'
qemu-system-aarch64: load of migration failed: Operation not permitted

With TCG there is no user friendly formatting of the faulting register indexes as with KVM. However the 2 added trace points help to identify the culprit indexes.

---

v2 -> v3:
- some extra typos (Connie)
- collected Connie's R-b

v1 -> v2:
- fixed some type in the commit msg

---
 target/arm/cpu.h        |  6 +++++
 target/arm/kvm.c        | 23 ++++++++++++++++
 target/arm/machine.c    | 58 ++++++++++++++++++++++++++++++++++++-----
 target/arm/trace-events |  7 +++++
 4 files changed, 88 insertions(+), 6 deletions(-)
target/arm/cpu.h +6 -0
index 2f124d1b157..7f67c42cfd7 100644
@@ -939,6 +939,12 @@ struct ArchCPU {
     uint64_t *cpreg_vmstate_values;
     int32_t cpreg_vmstate_array_len;
 
+    #define MAX_CPREG_VMSTATE_ANOMALIES 10
+    uint64_t cpreg_vmstate_missing_indexes[MAX_CPREG_VMSTATE_ANOMALIES];
+    int32_t cpreg_vmstate_missing_indexes_array_len;
+    uint64_t cpreg_vmstate_unexpected_indexes[MAX_CPREG_VMSTATE_ANOMALIES];
+    int32_t cpreg_vmstate_unexpected_indexes_array_len;
+
     DynamicGDBFeatureInfo dyn_sysreg_feature;
     DynamicGDBFeatureInfo dyn_svereg_feature;
     DynamicGDBFeatureInfo dyn_smereg_feature;
target/arm/kvm.c +23 -0
index 48f853fff80..c6f0d0fc4e1 100644
@@ -1024,6 +1024,29 @@ void kvm_arm_cpu_pre_save(ARMCPU *cpu)
 
 bool kvm_arm_cpu_post_load(ARMCPU *cpu)
 {
+    int i;
+
+    for (i = 0; i < cpu->cpreg_vmstate_missing_indexes_array_len; i++) {
+        gchar *name;
+
+        name = kvm_print_register_name(cpu->cpreg_vmstate_missing_indexes[i]);
+        trace_kvm_arm_cpu_post_load_missing_reg(name);
+        g_free(name);
+    }
+
+    for (i = 0; i < cpu->cpreg_vmstate_unexpected_indexes_array_len; i++) {
+        gchar *name;
+
+        name = kvm_print_register_name(cpu->cpreg_vmstate_unexpected_indexes[i]);
+        error_report("%s Unexpected register in input stream: %i 0x%"PRIx64" %s",
+                     __func__, i, cpu->cpreg_vmstate_unexpected_indexes[i], name);
+        g_free(name);
+    }
+    /* Fail the migration if we detect unexpected registers */
+    if (cpu->cpreg_vmstate_unexpected_indexes_array_len) {
+        return false;
+    }
+
     if (!write_list_to_kvmstate(cpu, KVM_PUT_FULL_STATE)) {
         return false;
     }
target/arm/machine.c +52 -6

99 lines not on this page · raw message

target/arm/trace-events +7 -0
index 676d29fe516..0a5ed3e69d5 100644
@@ -13,6 +13,7 @@ arm_gt_update_irq(int timer, int irqstate) "gt_update_irq: timer %d irqstate %d"
 
 # kvm.c
 kvm_arm_fixup_msi_route(uint64_t iova, uint64_t gpa) "MSI iova = 0x%"PRIx64" is translated into 0x%"PRIx64
+kvm_arm_cpu_post_load_missing_reg(char *name) "Missing register in input stream: %s"
 
 # cpu.c
 arm_cpu_reset(uint64_t mp_aff) "cpu %" PRIu64
@@ -26,3 +27,9 @@ arm_powerctl_reset_cpu(uint64_t mp_aff) "cpu %" PRIu64
 
 # tcg/psci.c and hvf/hvf.c
 arm_psci_call(uint64_t x0, uint64_t x1, uint64_t x2, uint64_t x3, uint32_t cpuid) "PSCI Call x0=0x%016"PRIx64" x1=0x%016"PRIx64" x2=0x%016"PRIx64" x3=0x%016"PRIx64" cpuid=0x%x"
+
+# machine.c
+cpu_post_load_len(int cpreg_array_len, int cpreg_vmstate_array_len) "cpreg_array_len=%d cpreg_vmstate_array_len=%d"
+cpu_post_load(int i, int v, uint64_t regidx) "i=%d v=%d regidx=0x%"PRIx64
+cpu_post_load_missing(int i, uint64_t regidx, int v) "missing register in input stream: i=%d index=0x%"PRIx64" (v=%d)"
+cpu_post_load_unexpected(int v, uint64_t regidx, int i) "unexpected register in input stream: v=%d index=0x%"PRIx64" (i=%d)"
Signature · 1 line
2.52.0
author
#3 lore raw

PATCH v5 3/11 target/arm/cpu: Allow registers to be hidden

In reply to Eric Auger

More recent kernels sometimes expose new registers in an unconditionnal manner. This situation breaks backward migration as qemu notices there are more registers in the input stream than supported on the destination host. This leads to a "failed to load cpu:cpreg_vmstate_array_len" error.

A good example is the introduction of KVM_REG_ARM_VENDOR_HYP_BMAP_2 pseudo FW register in v6.16 by commit C0000e58c74e (“KVM: arm64:
Introduce KVM_REG_ARM_VENDOR_HYP_BMAP_2”). Trying to do backward migration from a host kernel that features the commit to a destination host that doesn't, fail with above error.

Currently QEMU is not using that feature so ignoring this latter is not a problem. An easy way to fix the migration issue is to teach qemu we don't care about that register and we can simply ignore it when syncing its state during migration.

This patch introduces an array of such hidden registers. Soon it will be settable through an array property.

If hidden, the register is moved out of the array of cpreg which is built in kvm_arm_init_cpreg_list(). That way their state won't be synced.

To extend that functionality to TCG, do the same in add_cpreg_to_list() and count_cpreg().

---
v2 -> v3:
- use kvm_regidx

v1 -> v2:
- Move the property in a separate patch
- improve the commit msg
- change the trace point to just print info in
kvm_arm_init_cpreg_list()
- improve comment in cpu.h (Connie)

target/arm/helper: Skip hidden registers

In case a cpreg is hidden, skip it when initialing the cpreg list.

---
 target/arm/cpu.h        | 20 ++++++++++++++++++++
 target/arm/helper.c     | 12 +++++++++++-
 target/arm/kvm.c        | 12 +++++++++++-
 target/arm/trace-events |  2 ++
 4 files changed, 44 insertions(+), 2 deletions(-)
target/arm/cpu.h +20 -0
index 7f67c42cfd7..b80f77d3743 100644
@@ -1045,6 +1045,15 @@ struct ArchCPU {
     /* KVM steal time */
     OnOffAuto kvm_steal_time;
 
+    /*
+     * Array of register indexes that need to be hidden to allow migration
+     * in certain cases, i.e. when a register is exposed in KVM or defined
+     * in TCG but not actually used in QEMU. Indexes are described in Linux
+     * Documentation/virt/kvm/api.rst for both KVM and TCG.
+     */
+    uint64_t *hidden_regs;
+    uint32_t nr_hidden_regs;
+
     /* Uniprocessor system with MP extensions */
     bool mp_is_up;
 
@@ -1185,6 +1194,17 @@ struct ARMCPUClass {
     ResettablePhases parent_phases;
 };
 
+static inline bool
+arm_cpu_hidden_reg(ARMCPU *cpu, uint64_t regidx)
+{
+    for (int i = 0; i < cpu->nr_hidden_regs; i++) {
+        if (cpu->hidden_regs[i] == regidx) {
+            return true;
+        }
+    }
+    return false;
+}
+
 /* Callback functions for the generic timer's timers. */
 void arm_gt_ptimer_cb(void *opaque);
 void arm_gt_vtimer_cb(void *opaque);
target/arm/helper.c +11 -1
index dce648b4824..8217517150b 100644
@@ -235,9 +235,13 @@ static void add_cpreg_to_list(gpointer key, gpointer value, gpointer opaque)
     ARMCPU *cpu = opaque;
     uint32_t regidx = (uintptr_t)key;
     const ARMCPRegInfo *ri = value;
+    uint64_t kvm_regidx = cpreg_to_kvm_id(regidx);
 
+    if (arm_cpu_hidden_reg(cpu, kvm_regidx)) {
+        return;
+    }
     if (!(ri->type & (ARM_CP_NO_RAW | ARM_CP_ALIAS))) {
-        cpu->cpreg_indexes[cpu->cpreg_array_len] = cpreg_to_kvm_id(regidx);
+        cpu->cpreg_indexes[cpu->cpreg_array_len] = kvm_regidx;
         /* The value array need not be initialized at this point */
         cpu->cpreg_array_len++;
     }
@@ -247,6 +251,12 @@ static void count_cpreg(gpointer key, gpointer value, gpointer opaque)
 {
     ARMCPU *cpu = opaque;
     const ARMCPRegInfo *ri = value;
+    uint32_t regidx = (uintptr_t)key;
+    uint64_t kvm_regidx = cpreg_to_kvm_id(regidx);
+
+    if (arm_cpu_hidden_reg(cpu, kvm_regidx)) {
+        return;
+    }
 
     if (!(ri->type & (ARM_CP_NO_RAW | ARM_CP_ALIAS))) {
         cpu->cpreg_array_len++;
target/arm/kvm.c +11 -1
index c6f0d0fc4e1..7e0a3680748 100644
@@ -789,7 +789,10 @@ static int kvm_arm_init_cpreg_list(ARMCPU *cpu)
     qsort(&rlp->reg, rlp->n, sizeof(rlp->reg[0]), compare_u64);
 
     for (i = 0, arraylen = 0; i < rlp->n; i++) {
-        if (!kvm_arm_reg_syncs_via_cpreg_list(rlp->reg[i])) {
+        uint64_t regidx = rlp->reg[i];
+
+        if (!kvm_arm_reg_syncs_via_cpreg_list(regidx) ||
+            arm_cpu_hidden_reg(cpu, regidx)) {
             continue;
         }
         switch (rlp->reg[i] & KVM_REG_SIZE_MASK) {
@@ -805,6 +808,8 @@ static int kvm_arm_init_cpreg_list(ARMCPU *cpu)
         arraylen++;
     }
 
+    trace_kvm_arm_init_cpreg_list_arraylen(arraylen);
+
     cpu->cpreg_indexes = g_renew(uint64_t, cpu->cpreg_indexes, arraylen);
     cpu->cpreg_values = g_renew(uint64_t, cpu->cpreg_values, arraylen);
     cpu->cpreg_vmstate_indexes = g_renew(uint64_t, cpu->cpreg_vmstate_indexes,
@@ -816,9 +821,14 @@ static int kvm_arm_init_cpreg_list(ARMCPU *cpu)
 
     for (i = 0, arraylen = 0; i < rlp->n; i++) {
         uint64_t regidx = rlp->reg[i];
+
         if (!kvm_arm_reg_syncs_via_cpreg_list(regidx)) {
             continue;
         }
+        if (arm_cpu_hidden_reg(cpu, regidx)) {
+            trace_kvm_arm_init_cpreg_list_skip_hidden_reg(rlp->reg[i]);
+            continue;
+        }
         cpu->cpreg_indexes[arraylen] = regidx;
         arraylen++;
     }
target/arm/trace-events +2 -0
index 0a5ed3e69d5..20f4b4f2cd0 100644
@@ -14,6 +14,8 @@ arm_gt_update_irq(int timer, int irqstate) "gt_update_irq: timer %d irqstate %d"
 # kvm.c
 kvm_arm_fixup_msi_route(uint64_t iova, uint64_t gpa) "MSI iova = 0x%"PRIx64" is translated into 0x%"PRIx64
 kvm_arm_cpu_post_load_missing_reg(char *name) "Missing register in input stream: %s"
+kvm_arm_init_cpreg_list_arraylen(uint32_t arraylen) "arraylen=%d"
+kvm_arm_init_cpreg_list_skip_hidden_reg(uint64_t regidx) "hidden 0x%"PRIx64" is skipped"
 
 # cpu.c
 arm_cpu_reset(uint64_t mp_aff) "cpu %" PRIu64
Signature · 1 line
2.52.0
author
#4 lore raw

PATCH v5 4/11 target/arm/machine: Allow extra regs in the incoming stream

In reply to Eric Auger

Newer kernels may revoke exposure of KVM regs to userspace. This can happen when one notices that some registers were unconditionnally exposed whether they shall be conditionnally exposed for example.

An example of such situation is: TCR2_EL1, PIRE0_EL1, PIR_EL1. Associated kernel commits were:
0fcb4eea5345  KVM: arm64: Hide TCR2_EL1 from userspace when disabled for guests a68cddbe47ef  KVM: arm64: Hide S1PIE registers from userspace when disabled for guests

Those commits were actual fixes but the cons is that is breaks forward migration on some HW. Indeed when migrating from an old kernel that does not feature those commits to a more recent one, destination qemu detects there are more KVM regs in the input migration stream than exposed by the destination host and the migration fails with:
"failed to load cpu:cpreg_vmstate_array_len"

This patchs adds the capability to define an array of register indexes that may exist in the migration incoming stream but may be not exposed by KVM on the destination.

We provision for extra space in cpreg_vmstate_* arrays during the preload phase to allow the state to be saved without overflow, in case the registers only are in the inbound data.

On postload we make sure to ignore them when analyzing potential mismatch between registers. The actual cpreg array is never altered meaning those registers are never accessed nor saved.

The array will be populated with a dedicated array property.

---

v2 -> v3:
- add a missing_as_expected trace point

v1 -> v2:
- get rid of the enforced/fake terminology
- remove the useless array of fake regs. Only the number of missing regs is needed

RFC -> v1:
- improve comment in target/arm/cpu.h (Connie)

---
 target/arm/cpu.h        | 22 ++++++++++++++++++++++
 target/arm/machine.c    | 30 +++++++++++++++++++++---------
 target/arm/trace-events |  1 +
 3 files changed, 44 insertions(+), 9 deletions(-)
target/arm/cpu.h +22 -0
index b80f77d3743..985d74b2ff6 100644
@@ -1054,6 +1054,15 @@ struct ArchCPU {
     uint64_t *hidden_regs;
     uint32_t nr_hidden_regs;
 
+    /*
+     * Registers that are likely to be part of the migration
+     * incoming stream but not exposed on destination. If
+     * their indexes are stored in this array, it is OK to
+     * ignore those registers in the inbound data.
+     */
+    uint64_t *mig_safe_missing_regs;
+    uint32_t nr_mig_safe_missing_regs;
+
     /* Uniprocessor system with MP extensions */
     bool mp_is_up;
 
@@ -1205,6 +1214,19 @@ arm_cpu_hidden_reg(ARMCPU *cpu, uint64_t regidx)
     return false;
 }
 
+
+static inline bool
+arm_cpu_safe_missing_reg(ARMCPU *cpu, uint64_t regidx)
+{
+    for (int i = 0; i < cpu->nr_mig_safe_missing_regs; i++) {
+        if (regidx == cpu->mig_safe_missing_regs[i]) {
+            return true;
+        }
+    }
+    return false;
+}
+
+
 /* Callback functions for the generic timer's timers. */
 void arm_gt_ptimer_cb(void *opaque);
 void arm_gt_vtimer_cb(void *opaque);
target/arm/machine.c +21 -9
index f06a920aba1..98851afc615 100644
@@ -991,7 +991,8 @@ static int cpu_pre_load(void *opaque)
 {
     ARMCPU *cpu = opaque;
     CPUARMState *env = &cpu->env;
-    int arraylen = cpu->cpreg_vmstate_array_len + MAX_CPREG_VMSTATE_ANOMALIES;
+    int arraylen = cpu->cpreg_vmstate_array_len +
+                   cpu->nr_mig_safe_missing_regs + MAX_CPREG_VMSTATE_ANOMALIES;
 
     cpu->cpreg_vmstate_indexes = g_renew(uint64_t, cpu->cpreg_vmstate_indexes,
                                          arraylen);
@@ -1058,6 +1059,10 @@ static int cpu_post_load(void *opaque, int version_id)
      * entries with the right slots in our own values array.
      */
 
+    /*
+     * at this point cpu->cpreg_vmstate_array_len was migrated with the
+     * actual length saved on source
+     */
     trace_cpu_post_load_len(cpu->cpreg_array_len, cpu->cpreg_vmstate_array_len);
     for (; i < cpu->cpreg_array_len && v < cpu->cpreg_vmstate_array_len;) {
         trace_cpu_post_load(i, v , cpu->cpreg_indexes[i]);
@@ -1072,10 +1077,15 @@ static int cpu_post_load(void *opaque, int version_id)
         }
         if (cpu->cpreg_vmstate_indexes[v] < cpu->cpreg_indexes[i]) {
             /* register in their list but not ours: those will fail migration */
-            trace_cpu_post_load_unexpected(v, cpu->cpreg_vmstate_indexes[v], i);
-            if (k < MAX_CPREG_VMSTATE_ANOMALIES) {
-                cpu->cpreg_vmstate_unexpected_indexes[k++] =
-                    cpu->cpreg_vmstate_indexes[v];
+            if (!arm_cpu_safe_missing_reg(cpu, cpu->cpreg_vmstate_indexes[v])) {
+                trace_cpu_post_load_unexpected(v, cpu->cpreg_vmstate_indexes[v], i);
+                if (k < MAX_CPREG_VMSTATE_ANOMALIES) {
+                    cpu->cpreg_vmstate_unexpected_indexes[k++] =
+                        cpu->cpreg_vmstate_indexes[v];
+                }
+            } else {
+                trace_cpu_post_load_missing_as_expected(v, cpu->cpreg_vmstate_indexes[v],
+                                                        i);
             }
             v++;
             continue;
@@ -1101,10 +1111,12 @@ static int cpu_post_load(void *opaque, int version_id)
      * still regs in the input stream, continue parsing the vmstate array
      */
     for ( ; v < cpu->cpreg_vmstate_array_len; v++) {
-        if (k < MAX_CPREG_VMSTATE_ANOMALIES) {
-            trace_cpu_post_load_unexpected(v, cpu->cpreg_vmstate_indexes[v], i);
-            cpu->cpreg_vmstate_unexpected_indexes[k++] =
-                cpu->cpreg_vmstate_indexes[v];
+        if (!arm_cpu_safe_missing_reg(cpu, cpu->cpreg_vmstate_indexes[v])) {
+            if (k < MAX_CPREG_VMSTATE_ANOMALIES) {
+                trace_cpu_post_load_unexpected(v, cpu->cpreg_vmstate_indexes[v], i);
+                cpu->cpreg_vmstate_unexpected_indexes[k++] =
+                    cpu->cpreg_vmstate_indexes[v];
+            }
         }
     }
 
target/arm/trace-events +1 -0
index 20f4b4f2cd0..2e22012c692 100644
@@ -35,3 +35,4 @@ cpu_post_load_len(int cpreg_array_len, int cpreg_vmstate_array_len) "cpreg_array
 cpu_post_load(int i, int v, uint64_t regidx) "i=%d v=%d regidx=0x%"PRIx64
 cpu_post_load_missing(int i, uint64_t regidx, int v) "missing register in input stream: i=%d index=0x%"PRIx64" (v=%d)"
 cpu_post_load_unexpected(int v, uint64_t regidx, int i) "unexpected register in input stream: v=%d index=0x%"PRIx64" (i=%d)"
+cpu_post_load_missing_as_expected(int v, uint64_t regidx, int i) "register missing as expected in input stream: v=%d index=0x%"PRIx64" (i=%d)"
Signature · 1 line
2.52.0
author
#5 lore raw

PATCH v5 5/11 kvm-all: Enforce hidden regs are never accessed

In reply to Eric Auger

On ARM we want to be able to hide some registers which are exposed by KVM. To mitigate some migration failures that occur when a new register is exposed and does not exist on the destination, some registers are tagged "hidden" and their state won't be saved. As the state is not saved and they are expected not to be used, we want to enforce they aren't. So let's check this. The new CPUClass hide_reg() callback is optional and will be implemented on ARM in a subsequent patch.

---

v3 -> v4:
- don't use blacklist terminology (Connie & Sebastian) and reword the commit title to something clearer

---
 include/hw/core/cpu.h |  2 ++
 accel/kvm/kvm-all.c   | 12 ++++++++++++
 2 files changed, 14 insertions(+)
include/hw/core/cpu.h +2 -0
index f6f17df9e64..3e30e1cbddf 100644
@@ -142,6 +142,7 @@ struct SysemuCPUOps;
  * the caller will not g_free() it.
  * @disas_set_info: Setup architecture specific components of disassembly info
  * @adjust_watchpoint_address: Perform a target-specific adjustment to an
+ * @hide_reg: Check if a register must be hidden (optional)
  * address before attempting to match it against watchpoints.
  * @deprecation_note: If this CPUClass is deprecated, this field provides
  *                    related information.
@@ -167,6 +168,7 @@ struct CPUClass {
     int (*gdb_read_register)(CPUState *cpu, GByteArray *buf, int reg);
     int (*gdb_write_register)(CPUState *cpu, uint8_t *buf, int reg);
     vaddr (*gdb_adjust_breakpoint)(CPUState *cpu, vaddr addr);
+    bool (*hide_reg)(CPUState *cpu, uint64_t regidex);
 
     const char *gdb_core_xml_file;
     const char * (*gdb_arch_name)(CPUState *cpu);
accel/kvm/kvm-all.c +12 -0
index f85eb42d78b..bcbf45cee8d 100644
@@ -3784,9 +3784,15 @@ bool kvm_device_supported(int vmfd, uint64_t type)
 
 int kvm_set_one_reg(CPUState *cs, uint64_t id, void *source)
 {
+    CPUClass *cc = CPU_GET_CLASS(cs);
     struct kvm_one_reg reg;
     int r;
 
+    if (cc->hide_reg && cc->hide_reg(cs, id)) {
+        error_report("%s reg 0x%"PRIx64" is hidden and shall never been accessed",
+                     __func__, id);
+        g_assert_not_reached();
+    }
     reg.id = id;
     reg.addr = (uintptr_t) source;
     r = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &reg);
@@ -3798,9 +3804,15 @@ int kvm_set_one_reg(CPUState *cs, uint64_t id, void *source)
 
 int kvm_get_one_reg(CPUState *cs, uint64_t id, void *target)
 {
+    CPUClass *cc = CPU_GET_CLASS(cs);
     struct kvm_one_reg reg;
     int r;
 
+    if (cc->hide_reg && cc->hide_reg(cs, id)) {
+        error_report("%s reg 0x%"PRIx64" is hidden and shall never been accessed",
+                     __func__, id);
+        g_assert_not_reached();
+    }
     reg.id = id;
     reg.addr = (uintptr_t) target;
     r = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &reg);
Signature · 1 line
2.52.0
author
#6 lore raw

PATCH v5 6/11 target/arm/cpu: Implement hide_reg callback()

In reply to Eric Auger

Check if the register is hidden.

---
 target/arm/cpu.c | 6 ++++++
 1 file changed, 6 insertions(+)
target/arm/cpu.c +6 -0
index 05fa3339b1a..04a73254420 100644
@@ -2370,6 +2370,11 @@ static const TCGCPUOps arm_tcg_ops = {
 };
 #endif /* CONFIG_TCG */
 
+static inline bool arm_cpu_hide_reg(CPUState *s, uint64_t regidx)
+{
+    return arm_cpu_hidden_reg(ARM_CPU(s), regidx);
+}
+
 static void arm_cpu_class_init(ObjectClass *oc, const void *data)
 {
     ARMCPUClass *acc = ARM_CPU_CLASS(oc);
@@ -2398,6 +2403,7 @@ static void arm_cpu_class_init(ObjectClass *oc, const void *data)
     cc->gdb_get_core_xml_file = arm_gdb_get_core_xml_file;
     cc->gdb_stop_before_watchpoint = true;
     cc->disas_set_info = arm_disas_set_info;
+    cc->hide_reg = arm_cpu_hide_reg;
 
 #ifdef CONFIG_TCG
     cc->tcg_ops = &arm_tcg_ops;
Signature · 1 line
2.52.0
author
#7 lore raw

PATCH v5 7/11 target/arm/cpu: Expose x-mig-hidden-regs and x-mig-safe-missing-regs properties

In reply to Eric Auger

Allows to set both array properties along with arm cpus. Their "x-" prefix reminds that those shall be used carefully for distro specific use cases to enable cross kernel migration.

This will allow to define such compat machine props like:

    static GlobalProperty arm_virt_kernel_compat_10_1[] = {
        /* KVM_REG_ARM_VENDOR_HYP_BMAP_2 */
        { TYPE_ARM_CPU, "x-mig-hidden-regs", "0x6030000000160003" },
        { TYPE_ARM_CPU, "x-mig-safe-missing-regs",
          /* TCR_EL1, PIRE0_EL1, PIR_EL1 */
          "0x603000000013c103, 0x603000000013c512, 0x603000000013c513" },
    }

The first one means KVM_REG_ARM_VENDOR_HYP_BMAP_2 shall always be hidden for machine types older than 10.1. The second one means that along with 10.1 machine type we may receive in the incoming migration stream, 3 registers that are unknown on destination.

Obviously, using the reg index as defined in linux/Documentation/virt/kvm/api.rst is not user friendly. However These options are supposed to be used to enable specific, rare cases, and in general, by people trying to configure distribution defaults familiar with those specific cases.

---

v3 -> v4:
- typo and rewording in the commit description (Connie)

---
 target/arm/cpu.c | 5 +++++
 1 file changed, 5 insertions(+)
target/arm/cpu.c +5 -0
index 04a73254420..88a22453795 100644
@@ -2249,6 +2249,11 @@ static const Property arm_cpu_properties[] = {
     DEFINE_PROP_BOOL("backcompat-cntfrq", ARMCPU, backcompat_cntfrq, false),
     DEFINE_PROP_BOOL("backcompat-pauth-default-use-qarma5", ARMCPU,
                       backcompat_pauth_default_use_qarma5, false),
+    DEFINE_PROP_ARRAY("x-mig-hidden-regs", ARMCPU,
+                      nr_hidden_regs, hidden_regs, qdev_prop_uint64, uint64_t),
+    DEFINE_PROP_ARRAY("x-mig-safe-missing-regs", ARMCPU,
+                      nr_mig_safe_missing_regs, mig_safe_missing_regs,
+                      qdev_prop_uint64, uint64_t),
 };
 
 static const gchar *arm_gdb_arch_name(CPUState *cs)
Signature · 1 line
2.52.0
author
#8 lore raw

PATCH v5 8/11 hw/arm/virt: Declare AArch32 DBGDTRTX as safe to ignore in incoming stream

In reply to Eric Auger

With the new infrastructure in place it is now feasible to teach qemu that it is safe to ignore a sysreg in the incoming migration stream. So with the plan to revert commit 4f2b82f60431 ("target/arm:
Reinstate bogus AArch32 DBGDTRTX register for migration compat") from qemu 11.0 onwards, let's add a compat in 10.2 machine options stating that this reg is safe to ignore. from 11.0 onwards we will not need that register anymore.

---

v4 -> v5:
- rebased on top of latest machine types (Connie)

v3 -> v4:
- add a comment related to DBGDTRTX (Connie)

---
 hw/arm/virt.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
hw/arm/virt.c +10 -0
index baa4e31aac1..03d5af18f26 100644
@@ -100,6 +100,15 @@ static GlobalProperty arm_virt_compat_defaults[] = {
 static const size_t arm_virt_compat_defaults_len =
     G_N_ELEMENTS(arm_virt_compat_defaults);
 
+/* Register erronously exposed on 10.2 and earlier */
+#define DBGDTRTX 0x40200000200e0298
+
+static GlobalProperty arm_virt_compat_10_2[] = {
+    { TYPE_ARM_CPU, "x-mig-safe-missing-regs", stringify(DBGDTRTX)},
+};
+static const size_t arm_virt_compat_10_2_len =
+    G_N_ELEMENTS(arm_virt_compat_10_2);
+
 /*
  * This cannot be called from the virt_machine_class_init() because
  * TYPE_VIRT_MACHINE is abstract and mc->compat_props g_ptr_array_new()
@@ -3552,6 +3561,7 @@ static void virt_machine_10_2_options(MachineClass *mc)
 {
     virt_machine_11_0_options(mc);
     compat_props_add(mc->compat_props, hw_compat_10_2, hw_compat_10_2_len);
+    compat_props_add(mc->compat_props, arm_virt_compat_10_2, arm_virt_compat_10_2_len);
 }
 DEFINE_VIRT_MACHINE(10, 2)
 
Signature · 1 line
2.52.0
author
#9 lore raw

PATCH v5 9/11 Revert "target/arm: Reinstate bogus AArch32 DBGDTRTX register for migration compat"

In reply to Eric Auger

This reverts commit 4f2b82f60431e4792ecfd86a4d6b824248ee4c21. We don't need that commit anymore as the AArch32 DBGDTRTX register is declared to be safe to ignore in the incoming migration stream using a compat in arm virt machine.

---
 target/arm/debug_helper.c | 29 -----------------------------
 1 file changed, 29 deletions(-)
target/arm/debug_helper.c +0 -29
index 579516e1541..aee06d4d426 100644
@@ -940,13 +940,6 @@ static void dbgclaimclr_write(CPUARMState *env, const ARMCPRegInfo *ri,
     env->cp15.dbgclaim &= ~(value & 0xFF);
 }
 
-static CPAccessResult access_bogus(CPUARMState *env, const ARMCPRegInfo *ri,
-                                   bool isread)
-{
-    /* Always UNDEF, as if this cpreg didn't exist */
-    return CP_ACCESS_UNDEFINED;
-}
-
 static const ARMCPRegInfo debug_cp_reginfo[] = {
     /*
      * DBGDRAR, DBGDSAR: always RAZ since we don't implement memory mapped
@@ -1009,28 +1002,6 @@ static const ARMCPRegInfo debug_cp_reginfo[] = {
       .opc0 = 2, .opc1 = 3, .crn = 0, .crm = 4, .opc2 = 0,
       .access = PL0_RW, .accessfn = access_tdcc,
       .type = ARM_CP_CONST, .resetvalue = 0 },
-    /*
-     * This is not a real AArch32 register. We used to incorrectly expose
-     * this due to a QEMU bug; to avoid breaking migration compatibility we
-     * need to continue to provide it so that we don't fail the inbound
-     * migration when it tells us about a sysreg that we don't have.
-     * We set an always-fails .accessfn, which means that the guest doesn't
-     * actually see this register (it will always UNDEF, identically to if
-     * there were no cpreg definition for it other than that we won't print
-     * a LOG_UNIMP message about it), and we set the ARM_CP_NO_GDB flag so the
-     * gdbstub won't see it either.
-     * (We can't just set .access = 0, because add_cpreg_to_hashtable()
-     * helpfully ignores cpregs which aren't accessible to the highest
-     * implemented EL.)
-     *
-     * TODO: implement a system for being able to describe "this register
-     * can be ignored if it appears in the inbound stream"; then we can
-     * remove this temporary hack.
-     */
-    { .name = "BOGUS_DBGDTR_EL0", .state = ARM_CP_STATE_AA32,
-      .cp = 14, .opc1 = 3, .crn = 0, .crm = 5, .opc2 = 0,
-      .access = PL0_RW, .accessfn = access_bogus,
-      .type = ARM_CP_CONST | ARM_CP_NO_GDB, .resetvalue = 0 },
     /*
      * OSECCR_EL1 provides a mechanism for an operating system
      * to access the contents of EDECCR. EDECCR is not implemented though,
Signature · 1 line
2.52.0
author
#10 lore raw

PATCH v5 10/11 hw/arm/virt: Introduce framework to aggregate hidden-regs and safe-missing-regs

In reply to Eric Auger

Currently if a virt_machine_<n>_options() sets a TYPE_ARM_CPU x-mig-hidden-regs or x-mig-safe-missing-regs array property, another one cannot overwrite it or extend it. We end up with a core dump:

qemu-system-aarch64: can't apply global arm-cpu.x-mig-safe-missing-regs=0x603000000013c103, 0x603000000013c512, 0x603000000013c513: array size property x-mig-safe-missing-regs may not be set more than once Aborted (core dumped)

In practice we would like an easy way to register regs that belong to either of those categories and allow aggregation of those.

We introduce arm_virt_compat_register_safe_missing_reg() and arm_virt_compat_register_hidden_reg() which populate GLists of int64_t. After all virt_machine_<n>_options have been called and have registered their regs, the GList are converted into the associated array property value and the GlobalProperties are set.

---
 include/hw/arm/virt.h | 23 ++++++++++++++
 hw/arm/virt.c         | 70 ++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 86 insertions(+), 7 deletions(-)
include/hw/arm/virt.h +23 -0
index 5907d41dbb2..d83e6f00068 100644
@@ -41,6 +41,7 @@
 #include "system/kvm.h"
 #include "hw/intc/arm_gicv3_common.h"
 #include "qom/object.h"
+#include "qobject/qlist.h"
 
 #define NUM_GICV2M_SPIS       64
 #define NUM_VIRTIO_TRANSPORTS 32
@@ -131,6 +132,8 @@ struct VirtMachineClass {
     bool no_tcg_lpa2;
     bool no_ns_el2_virt_timer_irq;
     bool no_nested_smmu;
+    QList *safe_missing_regs;
+    QList *hidden_regs;
 };
 
 struct VirtMachineState {
@@ -216,4 +219,24 @@ static inline int virt_gicv3_redist_region_count(VirtMachineState *vms)
             vms->highmem_redists) ? 2 : 1;
 }
 
+static inline void arm_virt_class_init(MachineClass *mc)
+{
+    VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
+
+    vmc->safe_missing_regs = qlist_new();
+    vmc->hidden_regs = qlist_new();
+}
+
+static inline void
+arm_virt_compat_register_safe_missing_reg(VirtMachineClass *vmc, int64_t regidx)
+{
+    qlist_append_int(vmc->safe_missing_regs, regidx);
+}
+
+static inline void
+arm_virt_compat_register_hidden_reg(VirtMachineClass *vmc, int64_t regidx)
+{
+    qlist_append_int(vmc->hidden_regs, regidx);
+}
+
 #endif /* QEMU_ARM_VIRT_H */
hw/arm/virt.c +63 -7

107 lines not on this page · raw message

Signature · 1 line
2.52.0
Reviewed-by on v5 10/11
#11 lore raw

Re: [PATCH v5 10/11] hw/arm/virt: Introduce framework to aggregate hidden-regs and safe-missing-regs

Quoted 16 lines · Eric Auger

Currently if a virt_machine_<n>_options() sets a TYPE_ARM_CPU x-mig-hidden-regs or x-mig-safe-missing-regs array property, another one cannot overwrite it or extend it. We end up with a core dump:

qemu-system-aarch64: can't apply global arm-cpu.x-mig-safe-missing-regs=0x603000000013c103, 0x603000000013c512, 0x603000000013c513: array size property x-mig-safe-missing-regs may not be set more than once Aborted (core dumped)

In practice we would like an easy way to register regs that belong to either of those categories and allow aggregation of those.

We introduce arm_virt_compat_register_safe_missing_reg() and arm_virt_compat_register_hidden_reg() which populate GLists of int64_t. After all virt_machine_<n>_options have been called and have registered their regs, the GList are converted into the associated array property value and the GlobalProperties are set.

[...]

hw/arm/virt.c static void arm_virt_compat_default_set(MachineClass *mc) · line 136

Quoted 10 lines
hw/arm/virt.c +10 -0
+static char *get_prop_value_from_reg_qlist(QList *l)
+{
+    size_t size = qlist_size(l);
+    GString *s = g_string_new("");
+    QListEntry *item;
+    int i = 0;
+    QNum *qi;
+
+    if (!size) {
+        return NULL;

Do we need to free s here (or allocate later) or is there some RAII magic involed?

hw/arm/virt.c static void arm_virt_compat_default_set(MachineClass *mc) · line 149

Quoted 13 lines
hw/arm/virt.c +13 -0
+    }
+    QLIST_FOREACH_ENTRY(l, item) {
+        qi = qobject_to(QNum, qlist_entry_obj(item));
+        int64_t regidx;
+
+        qnum_get_try_int(qi, &regidx);
+        if (i++ > 0) {
+            g_string_append(s, ", ");
+        }
+        g_string_append_printf(s, "%" G_GINT64_FORMAT, regidx);
+    }
+    return g_string_free(s, false);
+}
author
#12 lore raw

Re: [PATCH v5 10/11] hw/arm/virt: Introduce framework to aggregate hidden-regs and safe-missing-regs

Hi Sebastian,

Quoted 21 lines · Sebastian Ott
Quoted 19 lines · Eric Auger

Currently if a virt_machine_<n>_options() sets a TYPE_ARM_CPU x-mig-hidden-regs or x-mig-safe-missing-regs array property, another one cannot overwrite it or extend it. We end up with a core dump:

qemu-system-aarch64: can't apply global arm-cpu.x-mig-safe-missing-regs=0x603000000013c103, 0x603000000013c512, 0x603000000013c513: array size property x-mig-safe-missing-regs may not be set more than once Aborted (core dumped)

In practice we would like an easy way to register regs that belong to either of those categories and allow aggregation of those.

We introduce arm_virt_compat_register_safe_missing_reg() and arm_virt_compat_register_hidden_reg() which populate GLists of int64_t. After all virt_machine_<n>_options have been called and have registered their regs, the GList are converted into the associated array property value and the GlobalProperties are set.

[...]

Quoted 13 lines

hw/arm/virt.c static void arm_virt_compat_default_set(MachineClass *mc) · line 136

Quoted 10 lines
hw/arm/virt.c +10 -0
+static char *get_prop_value_from_reg_qlist(QList *l)
+{
+    size_t size = qlist_size(l);
+    GString *s = g_string_new("");
+    QListEntry *item;
+    int i = 0;
+    QNum *qi;
+
+    if (!size) {
+        return NULL;

Do we need to free s here (or allocate later) or is there some RAII magic involed?

yes I can move the init below the above check

Quoted 16 lines
Quoted 5 lines

+    }
+    QLIST_FOREACH_ENTRY(l, item) {
+        qi = qobject_to(QNum, qlist_entry_obj(item)); +        int64_t regidx;

hw/arm/virt.c static void arm_virt_compat_default_set(MachineClass *mc) · line 149

Quoted 9 lines
hw/arm/virt.c +9 -0
+
+        qnum_get_try_int(qi, &regidx);
+        if (i++ > 0) {
+            g_string_append(s, ", ");
+        }
+        g_string_append_printf(s, "%" G_GINT64_FORMAT, regidx);
+    }
+    return g_string_free(s, false);
+}

Thanks!

Eric

author
#13 lore raw

PATCH v5 11/11 hw/arm/virt: [DO NOT UPSTREAM] Enforce compatibility with older kernels

In reply to Eric Auger

This is an example on how to use the new CPU options. This catters to distributions who want machines to be migratable (forward and backward) accross different host kernel versions in case KVM registers exposed to qemu vary accross kernels. This patch is not meant to be upstreamed as it is really kernel dependent. The goal is to illustrate how this would be used.

In this example, For 10_1 machines types and older we apply the following host kernel related compats:

1) Make sure the KVM_REG_ARM_VENDOR_HYP_BMAP_2 exposed from v6.15 onwards is ignored/hidden.
2) Make sure TCR_EL1, PIRE0_EL1, PIR_EL1 are always seen by qemu although not exposed by KVM. They were unconditionnally exposed before v6.13 while from v6.13 they are only exposed if supported by the guest.

This will allow 10_1 machines types and older machines to migrate forward and backward from old downstream kernels that do not feature those changes to newer kernels (>= v6.15).

---
 hw/arm/virt.c | 7 +++++++
 1 file changed, 7 insertions(+)
hw/arm/virt.c +7 -0
index 6c57e436053..860bc8a787a 100644
@@ -3623,9 +3623,16 @@ DEFINE_VIRT_MACHINE(10, 2)
 
 static void virt_machine_10_1_options(MachineClass *mc)
 {
+    VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
+
     virt_machine_10_2_options(mc);
     mc->smbios_memory_device_size = 2047 * TiB;
     compat_props_add(mc->compat_props, hw_compat_10_1, hw_compat_10_1_len);
+    /* KVM_REG_ARM_VENDOR_HYP_BMAP_2 */
+    arm_virt_compat_register_hidden_reg(vmc, 0x6030000000160003);
+    arm_virt_compat_register_safe_missing_reg(vmc, 0x603000000013c103 /* TCR_EL1 */);
+    arm_virt_compat_register_safe_missing_reg(vmc, 0x603000000013c512 /* PIRE0_EL1 */);
+    arm_virt_compat_register_safe_missing_reg(vmc, 0x603000000013c513 /* PIR_EL1 */);
 }
 DEFINE_VIRT_MACHINE(10, 1)
 
Signature · 1 line
2.52.0