Skip to contentSkip to conversation

arm: enable MTE for QEMU + kvm

PATCH Superseded Cornelia Huck 4 participants 22 messages Feb 3, 2023 – Feb 27, 2023 KVM

v5 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v5 cover arm: enable MTE for QEMU + kvm

Respin of my kvm mte series; tested via check + check-tcg and on FVP.

Changes v4->v5:
- new patch: "arm/virt: don't try to spell out the accelerator"
- some refactoring in the kvm enablement code
- fixes suggested by various people
- rebase to current master

Cornelia Huck (3):
arm/virt: don't try to spell out the accelerator arm/kvm: add support for MTE
qtests/arm: add some mte tests

 docs/system/arm/cpu-features.rst |  21 ++++++
 hw/arm/virt.c                    |   8 +--
 target/arm/cpu.c                 |  18 ++---
 target/arm/cpu.h                 |   1 +
 target/arm/cpu64.c               | 114 +++++++++++++++++++++++++++++++
 target/arm/internals.h           |   1 +
 target/arm/kvm.c                 |  29 ++++++++
 target/arm/kvm64.c               |   5 ++
 target/arm/kvm_arm.h             |  19 ++++++
 target/arm/monitor.c             |   1 +
 tests/qtest/arm-cpu-features.c   |  75 ++++++++++++++++++++
 11 files changed, 276 insertions(+), 16 deletions(-)
Signature · 1 line
2.39.1
#1 lore raw

PATCH v5 1/3 arm/virt: don't try to spell out the accelerator

Just use current_accel_name() directly.

---
 hw/arm/virt.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
hw/arm/virt.c +3 -3
index ea2413a0bad7..bdc297a4570c 100644
@@ -2123,21 +2123,21 @@ static void machvirt_init(MachineState *machine)
     if (vms->secure && (kvm_enabled() || hvf_enabled())) {
         error_report("mach-virt: %s does not support providing "
                      "Security extensions (TrustZone) to the guest CPU",
-                     kvm_enabled() ? "KVM" : "HVF");
+                     current_accel_name());
         exit(1);
     }
 
     if (vms->virt && (kvm_enabled() || hvf_enabled())) {
         error_report("mach-virt: %s does not support providing "
                      "Virtualization extensions to the guest CPU",
-                     kvm_enabled() ? "KVM" : "HVF");
+                     current_accel_name());
         exit(1);
     }
 
     if (vms->mte && (kvm_enabled() || hvf_enabled())) {
         error_report("mach-virt: %s does not support providing "
                      "MTE to the guest CPU",
-                     kvm_enabled() ? "KVM" : "HVF");
+                     current_accel_name());
         exit(1);
     }
 
Signature · 1 line
2.39.1
Reviewed-by on v5 1/3
#3 lore raw

Re: [PATCH v5 1/3] arm/virt: don't try to spell out the accelerator

In reply to Cornelia Huck

Hi Connie,

Quoted 3 lines · Cornelia Huck

Just use current_accel_name() directly.

Thanks

Eric

Quoted 33 lines
---
 hw/arm/virt.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
hw/arm/virt.c +3 -3
index ea2413a0bad7..bdc297a4570c 100644
@@ -2123,21 +2123,21 @@ static void machvirt_init(MachineState *machine)
     if (vms->secure && (kvm_enabled() || hvf_enabled())) {
         error_report("mach-virt: %s does not support providing "
                      "Security extensions (TrustZone) to the guest CPU",
-                     kvm_enabled() ? "KVM" : "HVF");
+                     current_accel_name());
         exit(1);
     }
 
     if (vms->virt && (kvm_enabled() || hvf_enabled())) {
         error_report("mach-virt: %s does not support providing "
                      "Virtualization extensions to the guest CPU",
-                     kvm_enabled() ? "KVM" : "HVF");
+                     current_accel_name());
         exit(1);
     }
 
     if (vms->mte && (kvm_enabled() || hvf_enabled())) {
         error_report("mach-virt: %s does not support providing "
                      "MTE to the guest CPU",
-                     kvm_enabled() ? "KVM" : "HVF");
+                     current_accel_name());
         exit(1);
     }
 
#4 lore raw

PATCH v5 2/3 arm/kvm: add support for MTE

In reply to Cornelia Huck

Introduce a new cpu feature flag to control MTE support. To preserve backwards compatibility for tcg, MTE will continue to be enabled as long as tag memory has been provided.

If MTE has been enabled, we need to disable migration, as we do not yet have a way to migrate the tags as well. Therefore, MTE will stay off with KVM unless requested explicitly.

---
 docs/system/arm/cpu-features.rst |  21 ++++++
 hw/arm/virt.c                    |   2 +-
 target/arm/cpu.c                 |  18 ++---
 target/arm/cpu.h                 |   1 +
 target/arm/cpu64.c               | 114 +++++++++++++++++++++++++++++++
 target/arm/internals.h           |   1 +
 target/arm/kvm.c                 |  29 ++++++++
 target/arm/kvm64.c               |   5 ++
 target/arm/kvm_arm.h             |  19 ++++++
 target/arm/monitor.c             |   1 +
 10 files changed, 198 insertions(+), 13 deletions(-)
docs/system/arm/cpu-features.rst +21 -0
index 00c444042ff5..f8b0f339d32d 100644
@@ -443,3 +443,24 @@ As with ``sve-default-vector-length``, if the default length is larger
 than the maximum vector length enabled, the actual vector length will
 be reduced.  If this property is set to ``-1`` then the default vector
 length is set to the maximum possible length.
+
+MTE CPU Property
+================
+
+The ``mte`` property controls the Memory Tagging Extension. For TCG, it requires
+presence of tag memory (which can be turned on for the ``virt`` machine via
+``mte=on``). For KVM, it requires the ``KVM_CAP_ARM_MTE`` capability; until
+proper migration support is implemented, enabling MTE will install a migration
+blocker.
+
+If not specified explicitly via ``on`` or ``off``, MTE will be available
+according to the following rules:
+
+* When TCG is used, MTE will be available if and only if tag memory is available;
+  i.e. it preserves the behaviour prior to the introduction of the feature.
+
+* When KVM is used, MTE will default to off, so that migration will not
+  unintentionally be blocked. This might change in a future QEMU version.
+
+* Other accelerators currently don't support MTE.
+
hw/arm/virt.c +1 -1
index bdc297a4570c..3aff0b8425f4 100644
@@ -2136,7 +2136,7 @@ static void machvirt_init(MachineState *machine)
 
     if (vms->mte && (kvm_enabled() || hvf_enabled())) {
         error_report("mach-virt: %s does not support providing "
-                     "MTE to the guest CPU",
+                     "emulated MTE to the guest CPU (tag memory not supported)",
                      current_accel_name());
         exit(1);
     }
target/arm/cpu.c +6 -12
index 5f63316dbf22..decab743d0d5 100644
@@ -1529,6 +1529,11 @@ void arm_cpu_finalize_features(ARMCPU *cpu, Error **errp)
             error_propagate(errp, local_err);
             return;
         }
+        arm_cpu_mte_finalize(cpu, &local_err);
+        if (local_err != NULL) {
+            error_propagate(errp, local_err);
+            return;
+        }
     }
 #endif
 
@@ -1605,7 +1610,7 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
         }
         if (cpu->tag_memory) {
             error_setg(errp,
-                       "Cannot enable %s when guest CPUs has MTE enabled",
+                       "Cannot enable %s when guest CPUs has tag memory enabled",
                        current_accel_name());
             return;
         }
@@ -1984,17 +1989,6 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
                                        ID_PFR1, VIRTUALIZATION, 0);
     }
 
-#ifndef CONFIG_USER_ONLY
-    if (cpu->tag_memory == NULL && cpu_isar_feature(aa64_mte, cpu)) {
-        /*
-         * Disable the MTE feature bits if we do not have tag-memory
-         * provided by the machine.
-         */
-        cpu->isar.id_aa64pfr1 =
-            FIELD_DP64(cpu->isar.id_aa64pfr1, ID_AA64PFR1, MTE, 0);
-    }
-#endif
-
     if (tcg_enabled()) {
         /*
          * Don't report the Statistical Profiling Extension in the ID
target/arm/cpu.h +1 -0
index 8cf70693be41..6cfaba30e30f 100644
@@ -1039,6 +1039,7 @@ struct ArchCPU {
     bool prop_pauth;
     bool prop_pauth_impdef;
     bool prop_lpa2;
+    OnOffAuto prop_mte;
 
     /* DCZ blocksize, in log_2(words), ie low 4 bits of DCZID_EL0 */
     uint32_t dcz_blocksize;
target/arm/cpu64.c +114 -0

143 lines not on this page · raw message

target/arm/internals.h +1 -0
index d9555309df0f..4dc6d19be42b 100644
@@ -1348,6 +1348,7 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp);
 void arm_cpu_sme_finalize(ARMCPU *cpu, Error **errp);
 void arm_cpu_pauth_finalize(ARMCPU *cpu, Error **errp);
 void arm_cpu_lpa2_finalize(ARMCPU *cpu, Error **errp);
+void arm_cpu_mte_finalize(ARMCPU *cpu, Error **errp);
 #endif
 
 #ifdef CONFIG_USER_ONLY
target/arm/kvm.c +29 -0
index f022c644d2ff..e6f2cb807bde 100644
@@ -31,6 +31,7 @@
 #include "hw/boards.h"
 #include "hw/irq.h"
 #include "qemu/log.h"
+#include "migration/blocker.h"
 
 const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
     KVM_CAP_LAST_INFO
@@ -1062,3 +1063,31 @@ bool kvm_arch_cpu_check_are_resettable(void)
 void kvm_arch_accel_class_init(ObjectClass *oc)
 {
 }
+
+void kvm_arm_enable_mte(Error **errp)
+{
+    static bool tried_to_enable = false;
+    Error *mte_migration_blocker = NULL;
+    int ret;
+
+    if (tried_to_enable) {
+        /*
+         * MTE on KVM is enabled on a per-VM basis (and retrying doesn't make
+         * sense), and we only want a single migration blocker as well.
+         */
+        return;
+    }
+    tried_to_enable = true;
+
+    if ((ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_MTE, 0))) {
+        error_setg_errno(errp, -ret, "Failed to enable KVM_CAP_ARM_MTE");
+        return;
+    }
+
+    /* TODO: add proper migration support with MTE enabled */
+    error_setg(&mte_migration_blocker,
+               "Live migration disabled due to MTE enabled");
+    if (migrate_add_blocker(mte_migration_blocker, errp)) {
+        error_free(mte_migration_blocker);
+    }
+}
target/arm/kvm64.c +5 -0
index 1197253d12f7..b777bd0a11d2 100644
@@ -764,6 +764,11 @@ bool kvm_arm_steal_time_supported(void)
     return kvm_check_extension(kvm_state, KVM_CAP_STEAL_TIME);
 }
 
+bool kvm_arm_mte_supported(void)
+{
+    return kvm_check_extension(kvm_state, KVM_CAP_ARM_MTE);
+}
+
 QEMU_BUILD_BUG_ON(KVM_ARM64_SVE_VQ_MIN != 1);
 
 uint32_t kvm_arm_sve_get_vls(CPUState *cs)
target/arm/kvm_arm.h +19 -0
index 99017b635ce4..9f88b0722293 100644
@@ -305,6 +305,13 @@ bool kvm_arm_pmu_supported(void);
  */
 bool kvm_arm_sve_supported(void);
 
+/**
+ * kvm_arm_mte_supported:
+ *
+ * Returns: true if KVM can enable MTE, and false otherwise.
+ */
+bool kvm_arm_mte_supported(void);
+
 /**
  * kvm_arm_get_max_vm_ipa_size:
  * @ms: Machine state handle
@@ -369,6 +376,8 @@ void kvm_arm_pvtime_init(CPUState *cs, uint64_t ipa);
 
 int kvm_arm_set_irq(int cpu, int irqtype, int irq, int level);
 
+void kvm_arm_enable_mte(Error **errp);
+
 #else
 
 /*
@@ -395,6 +404,11 @@ static inline bool kvm_arm_steal_time_supported(void)
     return false;
 }
 
+static inline bool kvm_arm_mte_supported(void)
+{
+    return false;
+}
+
 /*
  * These functions should never actually be called without KVM support.
  */
@@ -443,6 +457,11 @@ static inline uint32_t kvm_arm_sve_get_vls(CPUState *cs)
     g_assert_not_reached();
 }
 
+static inline void kvm_arm_enable_mte(Error **errp)
+{
+    g_assert_not_reached();
+}
+
 #endif
 
 static inline const char *gic_class_name(void)
target/arm/monitor.c +1 -0
index ecdd5ee81742..c419c81612ed 100644
@@ -96,6 +96,7 @@ static const char *cpu_model_advertised_features[] = {
     "sve1408", "sve1536", "sve1664", "sve1792", "sve1920", "sve2048",
     "kvm-no-adjvtime", "kvm-steal-time",
     "pauth", "pauth-impdef",
+    "mte",
     NULL
 };
 
Signature · 1 line
2.39.1
#5 lore raw

Re: [PATCH v5 2/3] arm/kvm: add support for MTE

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1111

Quoted 8 lines · Cornelia Huck
target/arm/cpu64.c +8 -0
+static void aarch64_cpu_get_mte(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+    OnOffAuto mte = cpu->prop_mte;
+
+    visit_type_OnOffAuto(v, name, &mte, errp);
+}

You don't need to copy to a local variable here.

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1119

Quoted 8 lines
target/arm/cpu64.c +8 -0
+
+static void aarch64_cpu_set_mte(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+
+    visit_type_OnOffAuto(v, name, &cpu->prop_mte, errp);
+}

... which makes get and set functions identical.
No need for both.

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1141

Quoted 8 lines
target/arm/cpu64.c +8 -0
+static inline bool arm_machine_has_tag_memory(void)
+{
+#ifndef CONFIG_USER_ONLY
+    Object *obj = object_dynamic_cast(qdev_get_machine(), TYPE_VIRT_MACHINE);
+
+    /* so far, only the virt machine has support for tag memory */
+    if (obj) {
+        VirtMachineState *vms = VIRT_MACHINE(obj);

VIRT_MACHINE() does object_dynamic_cast_assert, and we've just done that.

As this is startup, it's not the speed that matters. But it does look unfortunate. Not for this patch set, but perhaps we ought to add TRY_OBJ_NAME to DECLARE_INSTANCE_CHECKER?

Quoted 16 lines
target/arm/cpu64.c +16 -0
+void arm_cpu_mte_finalize(ARMCPU *cpu, Error **errp)
+{
+    bool enable_mte;
+
+    switch (cpu->prop_mte) {
+    case ON_OFF_AUTO_OFF:
+        enable_mte = false;
+        break;
+    case ON_OFF_AUTO_ON:
+        if (tcg_enabled()) {
+            if (cpu_isar_feature(aa64_mte, cpu)) {
+                if (!arm_machine_has_tag_memory()) {
+                    error_setg(errp, "mte=on requires tag memory");
+                    return;
+                }
+            } else {

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1175

Quoted 10 lines
target/arm/cpu64.c +10 -0
+                error_setg(errp, "mte not supported by this CPU type");
+                return;
+            }
+        }
+        if (kvm_enabled() && !kvm_arm_mte_supported()) {
+            error_setg(errp, "mte not supported by kvm");
+            return;
+        }
+        enable_mte = true;
+        break;

What's here is not wrong, but maybe better structured as

	enable_mte = true;
        if (qtest_enabled()) {
            break;
        }
        if (tcg_enabled()) {
            if (arm_machine_tag_mem) {
                break;
            }
            error;
            return;
        }
        if (kvm_enabled() && kvm_arm_mte_supported) {
            break;
        }
        error("mte not supported by %s", current_accel_type());
        return;

We only add the property for tcg via -cpu max, so the isar check is redundant.

r~

#6 lore raw

Re: [PATCH v5 2/3] arm/kvm: add support for MTE

Hi,

Quoted 13 lines · Richard Henderson

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1111

Quoted 9 lines · Cornelia Huck

+static void aarch64_cpu_get_mte(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)

target/arm/cpu64.c +6 -0
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+    OnOffAuto mte = cpu->prop_mte;
+
+    visit_type_OnOffAuto(v, name, &mte, errp);
+}

You don't need to copy to a local variable here.

Quoted 12 lines

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1119

Quoted 9 lines
(quoted diff) +2 -0
+
+static void aarch64_cpu_set_mte(Object *obj, Visitor *v, const char

*name,
+                                void *opaque, Error **errp)

target/arm/cpu64.c +5 -0
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+
+    visit_type_OnOffAuto(v, name, &cpu->prop_mte, errp);
+}

... which makes get and set functions identical.
No need for both.

This looks like a common pattern though. virt_get_acpi/set_acpi in virt.c or pc_machine_get_vmport/set_vmport in i386/pc.c and many other places (microvm ...). Do those other callers also need some simplifications?

Eric

Quoted 69 lines
Quoted 10 lines
target/arm/cpu64.c +4 -0
+static inline bool arm_machine_has_tag_memory(void)
+{
+#ifndef CONFIG_USER_ONLY
+    Object *obj = object_dynamic_cast(qdev_get_machine(),

TYPE_VIRT_MACHINE);
+
+    /* so far, only the virt machine has support for tag memory */ +    if (obj) {
+        VirtMachineState *vms = VIRT_MACHINE(obj);

VIRT_MACHINE() does object_dynamic_cast_assert, and we've just done that.

As this is startup, it's not the speed that matters.  But it does look unfortunate.  Not for this patch set, but perhaps we ought to add TRY_OBJ_NAME to DECLARE_INSTANCE_CHECKER?

Quoted 16 lines
target/arm/cpu64.c +16 -0
+void arm_cpu_mte_finalize(ARMCPU *cpu, Error **errp)
+{
+    bool enable_mte;
+
+    switch (cpu->prop_mte) {
+    case ON_OFF_AUTO_OFF:
+        enable_mte = false;
+        break;
+    case ON_OFF_AUTO_ON:
+        if (tcg_enabled()) {
+            if (cpu_isar_feature(aa64_mte, cpu)) {
+                if (!arm_machine_has_tag_memory()) {
+                    error_setg(errp, "mte=on requires tag memory");
+                    return;
+                }
+            } else {

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1175

Quoted 10 lines
target/arm/cpu64.c +10 -0
+                error_setg(errp, "mte not supported by this CPU type");
+                return;
+            }
+        }
+        if (kvm_enabled() && !kvm_arm_mte_supported()) {
+            error_setg(errp, "mte not supported by kvm");
+            return;
+        }
+        enable_mte = true;
+        break;

What's here is not wrong, but maybe better structured as

    enable_mte = true;
        if (qtest_enabled()) {
            break;
        }
        if (tcg_enabled()) {
            if (arm_machine_tag_mem) {
                break;
            }
            error;
            return;
        }
        if (kvm_enabled() && kvm_arm_mte_supported) { break;
        }
        error("mte not supported by %s", current_accel_type()); return;

We only add the property for tcg via -cpu max, so the isar check is redundant.

r~

#7 lore raw

Re: [PATCH v5 2/3] arm/kvm: add support for MTE

Quoted 28 lines · Eric Auger

Hi,

Quoted 13 lines · Richard Henderson

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1111

Quoted 9 lines · Cornelia Huck

+static void aarch64_cpu_get_mte(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)

target/arm/cpu64.c +6 -0
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+    OnOffAuto mte = cpu->prop_mte;
+
+    visit_type_OnOffAuto(v, name, &mte, errp);
+}

You don't need to copy to a local variable here.

Quoted 12 lines

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1119

Quoted 9 lines
(quoted diff) +2 -0
+
+static void aarch64_cpu_set_mte(Object *obj, Visitor *v, const char

*name,
+                                void *opaque, Error **errp)

target/arm/cpu64.c +5 -0
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+
+    visit_type_OnOffAuto(v, name, &cpu->prop_mte, errp);
+}

... which makes get and set functions identical.
No need for both.

Quoted 3 lines

This looks like a common pattern though. virt_get_acpi/set_acpi in virt.c or pc_machine_get_vmport/set_vmport in i386/pc.c and many other places (microvm ...). Do those other callers also need some simplifications?

Indeed, I'm pretty sure that I copied + adapted it from somewhere :)

Should we clean up all instances in one go instead? (Probably on top of this series, in order to minimize conflicts with other changes.)

#8 lore raw

Re: [PATCH v5 2/3] arm/kvm: add support for MTE

Quoted 30 lines · Cornelia Huck
Quoted 28 lines · Eric Auger

Hi,

Quoted 25 lines · Richard Henderson

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1111

Quoted 9 lines · Cornelia Huck

+static void aarch64_cpu_get_mte(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)

target/arm/cpu64.c +6 -0
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+    OnOffAuto mte = cpu->prop_mte;
+
+    visit_type_OnOffAuto(v, name, &mte, errp);
+}

You don't need to copy to a local variable here.

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1119

Quoted 9 lines
(quoted diff) +2 -0
+
+static void aarch64_cpu_set_mte(Object *obj, Visitor *v, const char

*name,
+                                void *opaque, Error **errp)

target/arm/cpu64.c +5 -0
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+
+    visit_type_OnOffAuto(v, name, &cpu->prop_mte, errp);
+}

... which makes get and set functions identical.
No need for both.

Quoted 8 lines
Quoted 3 lines

This looks like a common pattern though. virt_get_acpi/set_acpi in virt.c or pc_machine_get_vmport/set_vmport in i386/pc.c and many other places (microvm ...). Do those other callers also need some simplifications?

Indeed, I'm pretty sure that I copied + adapted it from somewhere :)

Should we clean up all instances in one go instead? (Probably on top of this series, in order to minimize conflicts with other changes.)

Any objections to going with the code above and just doing a general cleanup on top?

#9 lore raw

Re: [PATCH v5 2/3] arm/kvm: add support for MTE

In reply to Richard Henderson
Quoted 12 lines · Richard Henderson

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1141

Quoted 8 lines · Cornelia Huck
target/arm/cpu64.c +8 -0
+static inline bool arm_machine_has_tag_memory(void)
+{
+#ifndef CONFIG_USER_ONLY
+    Object *obj = object_dynamic_cast(qdev_get_machine(), TYPE_VIRT_MACHINE);
+
+    /* so far, only the virt machine has support for tag memory */
+    if (obj) {
+        VirtMachineState *vms = VIRT_MACHINE(obj);

VIRT_MACHINE() does object_dynamic_cast_assert, and we've just done that.

Quoted 2 lines

As this is startup, it's not the speed that matters. But it does look unfortunate. Not for this patch set, but perhaps we ought to add TRY_OBJ_NAME to DECLARE_INSTANCE_CHECKER?

Instead of the pattern above, we could also do

VirtMachineState *vms = (VirtMachineState *) object_dynamic_cast(...); if (vms) {
(...)

Quoted 30 lines
Quoted 17 lines
target/arm/cpu64.c +16 -0
+void arm_cpu_mte_finalize(ARMCPU *cpu, Error **errp)
+{
+    bool enable_mte;
+
+    switch (cpu->prop_mte) {
+    case ON_OFF_AUTO_OFF:
+        enable_mte = false;
+        break;
+    case ON_OFF_AUTO_ON:
+        if (tcg_enabled()) {
+            if (cpu_isar_feature(aa64_mte, cpu)) {
+                if (!arm_machine_has_tag_memory()) {
+                    error_setg(errp, "mte=on requires tag memory");
+                    return;
+                }
+            } else {

target/arm/cpu64.c static void aarch64_neoverse_n1_initfn(Object *obj) · line 1175

Quoted 10 lines
target/arm/cpu64.c +10 -0
+                error_setg(errp, "mte not supported by this CPU type");
+                return;
+            }
+        }
+        if (kvm_enabled() && !kvm_arm_mte_supported()) {
+            error_setg(errp, "mte not supported by kvm");
+            return;
+        }
+        enable_mte = true;
+        break;

What's here is not wrong, but maybe better structured as

Quoted 16 lines
	enable_mte = true;
         if (qtest_enabled()) {
             break;
         }
         if (tcg_enabled()) {
             if (arm_machine_tag_mem) {
                 break;
             }
             error;
             return;
         }
         if (kvm_enabled() && kvm_arm_mte_supported) {
             break;
         }
         error("mte not supported by %s", current_accel_type());
         return;

That's indeed better, as we also see what's going on for the different accelarators.

Quoted 1 line

We only add the property for tcg via -cpu max, so the isar check is redundant.

#10 lore raw

Re: [PATCH v5 2/3] arm/kvm: add support for MTE

In reply to Cornelia Huck

Hi Connie,

Quoted 295 lines · Cornelia Huck

Introduce a new cpu feature flag to control MTE support. To preserve backwards compatibility for tcg, MTE will continue to be enabled as long as tag memory has been provided.

If MTE has been enabled, we need to disable migration, as we do not yet have a way to migrate the tags as well. Therefore, MTE will stay off with KVM unless requested explicitly.

---
 docs/system/arm/cpu-features.rst |  21 ++++++
 hw/arm/virt.c                    |   2 +-
 target/arm/cpu.c                 |  18 ++---
 target/arm/cpu.h                 |   1 +
 target/arm/cpu64.c               | 114 +++++++++++++++++++++++++++++++
 target/arm/internals.h           |   1 +
 target/arm/kvm.c                 |  29 ++++++++
 target/arm/kvm64.c               |   5 ++
 target/arm/kvm_arm.h             |  19 ++++++
 target/arm/monitor.c             |   1 +
 10 files changed, 198 insertions(+), 13 deletions(-)
docs/system/arm/cpu-features.rst +21 -0
index 00c444042ff5..f8b0f339d32d 100644
@@ -443,3 +443,24 @@ As with ``sve-default-vector-length``, if the default length is larger
 than the maximum vector length enabled, the actual vector length will
 be reduced.  If this property is set to ``-1`` then the default vector
 length is set to the maximum possible length.
+
+MTE CPU Property
+================
+
+The ``mte`` property controls the Memory Tagging Extension. For TCG, it requires
+presence of tag memory (which can be turned on for the ``virt`` machine via
+``mte=on``). For KVM, it requires the ``KVM_CAP_ARM_MTE`` capability; until
+proper migration support is implemented, enabling MTE will install a migration
+blocker.
+
+If not specified explicitly via ``on`` or ``off``, MTE will be available
+according to the following rules:
+
+* When TCG is used, MTE will be available if and only if tag memory is available;
+  i.e. it preserves the behaviour prior to the introduction of the feature.
+
+* When KVM is used, MTE will default to off, so that migration will not
+  unintentionally be blocked. This might change in a future QEMU version.
+
+* Other accelerators currently don't support MTE.
+
hw/arm/virt.c +1 -1
index bdc297a4570c..3aff0b8425f4 100644
@@ -2136,7 +2136,7 @@ static void machvirt_init(MachineState *machine)
 
     if (vms->mte && (kvm_enabled() || hvf_enabled())) {
         error_report("mach-virt: %s does not support providing "
-                     "MTE to the guest CPU",
+                     "emulated MTE to the guest CPU (tag memory not supported)",
                      current_accel_name());
         exit(1);
     }
target/arm/cpu.c +6 -12
index 5f63316dbf22..decab743d0d5 100644
@@ -1529,6 +1529,11 @@ void arm_cpu_finalize_features(ARMCPU *cpu, Error **errp)
             error_propagate(errp, local_err);
             return;
         }
+        arm_cpu_mte_finalize(cpu, &local_err);
+        if (local_err != NULL) {
+            error_propagate(errp, local_err);
+            return;
+        }
     }
 #endif
 
@@ -1605,7 +1610,7 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
         }
         if (cpu->tag_memory) {
             error_setg(errp,
-                       "Cannot enable %s when guest CPUs has MTE enabled",
+                       "Cannot enable %s when guest CPUs has tag memory enabled",
                        current_accel_name());
             return;
         }
@@ -1984,17 +1989,6 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
                                        ID_PFR1, VIRTUALIZATION, 0);
     }
 
-#ifndef CONFIG_USER_ONLY
-    if (cpu->tag_memory == NULL && cpu_isar_feature(aa64_mte, cpu)) {
-        /*
-         * Disable the MTE feature bits if we do not have tag-memory
-         * provided by the machine.
-         */
-        cpu->isar.id_aa64pfr1 =
-            FIELD_DP64(cpu->isar.id_aa64pfr1, ID_AA64PFR1, MTE, 0);
-    }
-#endif
-
     if (tcg_enabled()) {
         /*
          * Don't report the Statistical Profiling Extension in the ID
target/arm/cpu.h +1 -0
index 8cf70693be41..6cfaba30e30f 100644
@@ -1039,6 +1039,7 @@ struct ArchCPU {
     bool prop_pauth;
     bool prop_pauth_impdef;
     bool prop_lpa2;
+    OnOffAuto prop_mte;
 
     /* DCZ blocksize, in log_2(words), ie low 4 bits of DCZID_EL0 */
     uint32_t dcz_blocksize;
target/arm/cpu64.c +114 -0
index 0e021960fb5b..20c18b607b0b 100644
@@ -24,11 +24,16 @@
 #include "qemu/module.h"
 #include "sysemu/kvm.h"
 #include "sysemu/hvf.h"
+#include "sysemu/tcg.h"
 #include "kvm_arm.h"
 #include "hvf_arm.h"
 #include "qapi/visitor.h"
 #include "hw/qdev-properties.h"
 #include "internals.h"
+#include "qapi/qapi-visit-common.h"
+#if !defined(CONFIG_USER_ONLY)
+#include "hw/arm/virt.h"
+#endif
 
 static void aarch64_a35_initfn(Object *obj)
 {
@@ -1096,6 +1101,113 @@ static void aarch64_neoverse_n1_initfn(Object *obj)
     cpu->isar.reset_pmcr_el0 = 0x410c3000;
 }
 
+static void aarch64_cpu_get_mte(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+    OnOffAuto mte = cpu->prop_mte;
+
+    visit_type_OnOffAuto(v, name, &mte, errp);
+}
+
+static void aarch64_cpu_set_mte(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+
+    visit_type_OnOffAuto(v, name, &cpu->prop_mte, errp);
+}
+
+static void aarch64_add_mte_properties(Object *obj)
+{
+    /*
+     * For tcg, "AUTO" means turn on mte if tag memory has been provided, and
+     * turn it off (without error) if not.
+     * For kvm, "AUTO" currently means mte off, as migration is not supported
+     * yet.
+     * For all others, "AUTO" means mte off.
+     */
+    object_property_add(obj, "mte", "OnOffAuto", aarch64_cpu_get_mte,
+                        aarch64_cpu_set_mte, NULL, NULL);
+}
+
+static inline bool arm_machine_has_tag_memory(void)
+{
+#ifndef CONFIG_USER_ONLY
+    Object *obj = object_dynamic_cast(qdev_get_machine(), TYPE_VIRT_MACHINE);
+
+    /* so far, only the virt machine has support for tag memory */
+    if (obj) {
+        VirtMachineState *vms = VIRT_MACHINE(obj);
+
+        return vms->mte;
+    }
+    return false;
+#endif
+    return true;
+}
+
+void arm_cpu_mte_finalize(ARMCPU *cpu, Error **errp)
+{
+    bool enable_mte;
+
+    switch (cpu->prop_mte) {
+    case ON_OFF_AUTO_OFF:
+        enable_mte = false;
+        break;
+    case ON_OFF_AUTO_ON:
+        if (tcg_enabled()) {
+            if (cpu_isar_feature(aa64_mte, cpu)) {
+                if (!arm_machine_has_tag_memory()) {
+                    error_setg(errp, "mte=on requires tag memory");
+                    return;
+                }
+            } else {
+                error_setg(errp, "mte not supported by this CPU type");
+                return;
+            }
+        }
+        if (kvm_enabled() && !kvm_arm_mte_supported()) {
+            error_setg(errp, "mte not supported by kvm");
+            return;
+        }
+        enable_mte = true;
+        break;
+    default: /* AUTO */
+        if (tcg_enabled()) {
+            if (cpu_isar_feature(aa64_mte, cpu)) {
+                /*
+                 * Tie mte enablement to presence of tag memory, in order to
+                 * preserve pre-existing behaviour.
+                 */
+                enable_mte = arm_machine_has_tag_memory();
+            } else {
+                enable_mte = false;
+            }
+            break;
+        } else {
+            /*
+             * This cannot yet be
+             * enable_mte = kvm_arm_mte_supported();
+             * as we don't support migration yet.
+             */
+            enable_mte = false;
+        }
+    }
+
+    if (!enable_mte) {
+        /* Disable MTE feature bits. */
+        cpu->isar.id_aa64pfr1 =
+            FIELD_DP64(cpu->isar.id_aa64pfr1, ID_AA64PFR1, MTE, 0);
+        return;
+    }
+
+    /* accelerator-specific enablement */
+    if (kvm_enabled()) {
+        kvm_arm_enable_mte(errp);
+    }
+}
+
 static void aarch64_host_initfn(Object *obj)
 {
 #if defined(CONFIG_KVM)
@@ -1104,6 +1216,7 @@ static void aarch64_host_initfn(Object *obj)
     if (arm_feature(&cpu->env, ARM_FEATURE_AARCH64)) {
         aarch64_add_sve_properties(obj);
         aarch64_add_pauth_properties(obj);
+        aarch64_add_mte_properties(obj);
     }
 #elif defined(CONFIG_HVF)
     ARMCPU *cpu = ARM_CPU(obj);
@@ -1300,6 +1413,7 @@ static void aarch64_max_initfn(Object *obj)
     object_property_add(obj, "sve-max-vq", "uint32", cpu_max_get_sve_max_vq,
                         cpu_max_set_sve_max_vq, NULL, NULL);
     qdev_property_add_static(DEVICE(obj), &arm_cpu_lpa2_property);
+    aarch64_add_mte_properties(obj);
 }
 
 static const ARMCPUInfo aarch64_cpus[] = {
target/arm/internals.h +1 -0
index d9555309df0f..4dc6d19be42b 100644
@@ -1348,6 +1348,7 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp);
 void arm_cpu_sme_finalize(ARMCPU *cpu, Error **errp);
 void arm_cpu_pauth_finalize(ARMCPU *cpu, Error **errp);
 void arm_cpu_lpa2_finalize(ARMCPU *cpu, Error **errp);
+void arm_cpu_mte_finalize(ARMCPU *cpu, Error **errp);
 #endif
 
 #ifdef CONFIG_USER_ONLY
target/arm/kvm.c +1 -0
index f022c644d2ff..e6f2cb807bde 100644
@@ -31,6 +31,7 @@
 #include "hw/boards.h"
 #include "hw/irq.h"
 #include "qemu/log.h"
+#include "migration/blocker.h"
 
 const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
     KVM_CAP_LAST_INFO

target/arm/kvm.c bool kvm_arch_cpu_check_are_resettable(void) · line 1070

Quoted 8 lines
target/arm/kvm.c +5 -0
@@ -1062,3 +1063,31 @@ bool kvm_arch_cpu_check_are_resettable(void)
 void kvm_arch_accel_class_init(ObjectClass *oc)
 {
 }
+
+void kvm_arm_enable_mte(Error **errp)
+{
+    static bool tried_to_enable = false;
+    Error *mte_migration_blocker = NULL;

can't you make the mte_migration_blocker static instead?

Thanks

Eric

Quoted 102 lines
target/arm/kvm.c +23 -0
+    int ret;
+
+    if (tried_to_enable) {
+        /*
+         * MTE on KVM is enabled on a per-VM basis (and retrying doesn't make
+         * sense), and we only want a single migration blocker as well.
+         */
+        return;
+    }
+    tried_to_enable = true;
+
+    if ((ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_MTE, 0))) {
+        error_setg_errno(errp, -ret, "Failed to enable KVM_CAP_ARM_MTE");
+        return;
+    }
+
+    /* TODO: add proper migration support with MTE enabled */
+    error_setg(&mte_migration_blocker,
+               "Live migration disabled due to MTE enabled");
+    if (migrate_add_blocker(mte_migration_blocker, errp)) {
+        error_free(mte_migration_blocker);
+    }
+}
target/arm/kvm64.c +5 -0
index 1197253d12f7..b777bd0a11d2 100644
@@ -764,6 +764,11 @@ bool kvm_arm_steal_time_supported(void)
     return kvm_check_extension(kvm_state, KVM_CAP_STEAL_TIME);
 }
 
+bool kvm_arm_mte_supported(void)
+{
+    return kvm_check_extension(kvm_state, KVM_CAP_ARM_MTE);
+}
+
 QEMU_BUILD_BUG_ON(KVM_ARM64_SVE_VQ_MIN != 1);
 
 uint32_t kvm_arm_sve_get_vls(CPUState *cs)
target/arm/kvm_arm.h +19 -0
index 99017b635ce4..9f88b0722293 100644
@@ -305,6 +305,13 @@ bool kvm_arm_pmu_supported(void);
  */
 bool kvm_arm_sve_supported(void);
 
+/**
+ * kvm_arm_mte_supported:
+ *
+ * Returns: true if KVM can enable MTE, and false otherwise.
+ */
+bool kvm_arm_mte_supported(void);
+
 /**
  * kvm_arm_get_max_vm_ipa_size:
  * @ms: Machine state handle
@@ -369,6 +376,8 @@ void kvm_arm_pvtime_init(CPUState *cs, uint64_t ipa);
 
 int kvm_arm_set_irq(int cpu, int irqtype, int irq, int level);
 
+void kvm_arm_enable_mte(Error **errp);
+
 #else
 
 /*
@@ -395,6 +404,11 @@ static inline bool kvm_arm_steal_time_supported(void)
     return false;
 }
 
+static inline bool kvm_arm_mte_supported(void)
+{
+    return false;
+}
+
 /*
  * These functions should never actually be called without KVM support.
  */
@@ -443,6 +457,11 @@ static inline uint32_t kvm_arm_sve_get_vls(CPUState *cs)
     g_assert_not_reached();
 }
 
+static inline void kvm_arm_enable_mte(Error **errp)
+{
+    g_assert_not_reached();
+}
+
 #endif
 
 static inline const char *gic_class_name(void)
target/arm/monitor.c +1 -0
index ecdd5ee81742..c419c81612ed 100644
@@ -96,6 +96,7 @@ static const char *cpu_model_advertised_features[] = {
     "sve1408", "sve1536", "sve1664", "sve1792", "sve1920", "sve2048",
     "kvm-no-adjvtime", "kvm-steal-time",
     "pauth", "pauth-impdef",
+    "mte",
     NULL
 };
 
#11 lore raw

Re: [PATCH v5 2/3] arm/kvm: add support for MTE

Quoted 17 lines · Eric Auger

target/arm/kvm.c bool kvm_arch_cpu_check_are_resettable(void) · line 1070

Quoted 4 lines
target/arm/kvm.c +4 -0
+void kvm_arm_enable_mte(Error **errp)
+{
+    static bool tried_to_enable = false;
+    Error *mte_migration_blocker = NULL;

can't you make the mte_migration_blocker static instead?

Quoted 11 lines
target/arm/kvm.c +11 -0
+    int ret;
+
+    if (tried_to_enable) {
+        /*
+         * MTE on KVM is enabled on a per-VM basis (and retrying doesn't make
+         * sense), and we only want a single migration blocker as well.
+         */
+        return;
+    }
+    tried_to_enable = true;
+
Quoted 8 lines

target/arm/kvm.c bool kvm_arch_cpu_check_are_resettable(void) · line 1089

Quoted 8 lines
target/arm/kvm.c +8 -0
+    if ((ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_MTE, 0))) {
+        error_setg_errno(errp, -ret, "Failed to enable KVM_CAP_ARM_MTE");
+        return;
+    }
+
+    /* TODO: add proper migration support with MTE enabled */
+    error_setg(&mte_migration_blocker,
+               "Live migration disabled due to MTE enabled");

Making the blocker static wouldn't stop multiple errors from kvm_vm_enable_cap.

r~

#12 lore raw

Re: [PATCH v5 2/3] arm/kvm: add support for MTE

Hi Richard,

Quoted 23 lines · Richard Henderson
Quoted 22 lines · Eric Auger

target/arm/kvm.c bool kvm_arch_cpu_check_are_resettable(void) · line 1070

Quoted 4 lines
target/arm/kvm.c +4 -0
+void kvm_arm_enable_mte(Error **errp)
+{
+    static bool tried_to_enable = false;
+    Error *mte_migration_blocker = NULL;

can't you make the mte_migration_blocker static instead?

Quoted 16 lines

+    int ret;
+
+    if (tried_to_enable) {
+        /*
+         * MTE on KVM is enabled on a per-VM basis (and retrying doesn't make
+         * sense), and we only want a single migration blocker as well. +         */
+        return;
+    }
+    tried_to_enable = true;
+
+    if ((ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_MTE, 0))) { +        error_setg_errno(errp, -ret, "Failed to enable KVM_CAP_ARM_MTE");
+        return;

Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

+    }
+
+    /* TODO: add proper migration support with MTE enabled */ +    error_setg(&mte_migration_blocker,
+               "Live migration disabled due to MTE enabled");

Making the blocker static wouldn't stop multiple errors from kvm_vm_enable_cap.

Sorry I don't get what you mean. instead of checking tried_to_enable why can't we check !mte_migration_blocker?

Eric

Quoted 4 lines

r~

#13 lore raw

Re: [PATCH v5 2/3] arm/kvm: add support for MTE

Quoted 33 lines · Eric Auger

Hi Richard,

Quoted 23 lines · Richard Henderson
Quoted 22 lines · Eric Auger

target/arm/kvm.c bool kvm_arch_cpu_check_are_resettable(void) · line 1070

Quoted 4 lines
target/arm/kvm.c +4 -0
+void kvm_arm_enable_mte(Error **errp)
+{
+    static bool tried_to_enable = false;
+    Error *mte_migration_blocker = NULL;

can't you make the mte_migration_blocker static instead?

Quoted 16 lines

+    int ret;
+
+    if (tried_to_enable) {
+        /*
+         * MTE on KVM is enabled on a per-VM basis (and retrying doesn't make
+         * sense), and we only want a single migration blocker as well. +         */
+        return;
+    }
+    tried_to_enable = true;
+
+    if ((ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_MTE, 0))) { +        error_setg_errno(errp, -ret, "Failed to enable KVM_CAP_ARM_MTE");
+        return;

Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

+    }
+
+    /* TODO: add proper migration support with MTE enabled */ +    error_setg(&mte_migration_blocker,
+               "Live migration disabled due to MTE enabled");

Making the blocker static wouldn't stop multiple errors from kvm_vm_enable_cap.

Quoted 2 lines

Sorry I don't get what you mean. instead of checking tried_to_enable why can't we check !mte_migration_blocker?

[missed this one]

Do you mean

if (mte_migration_blocker) {
    return;
}

error_setg(&mte_migration_blocker, ...);

if ((ret = kvm_vm_enable_cap(...))) {
    return;
}
if (migrate_add_blocker(...)) {
    error_free(mte_migration_blocker);
    // is mte_migration_blocker guaranteed != NULL here?
}
#14 lore raw

PATCH v5 3/3 qtests/arm: add some mte tests

In reply to Cornelia Huck
---
 tests/qtest/arm-cpu-features.c | 75 ++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)
tests/qtest/arm-cpu-features.c +75 -0
index 8691802950ca..c5dbf66e938a 100644
@@ -22,6 +22,7 @@
 
 #define MACHINE     "-machine virt,gic-version=max -accel tcg "
 #define MACHINE_KVM "-machine virt,gic-version=max -accel kvm -accel tcg "
+#define MACHINE_MTE "-machine virt,gic-version=max,mte=on -accel tcg "
 #define QUERY_HEAD  "{ 'execute': 'query-cpu-model-expansion', " \
                     "  'arguments': { 'type': 'full', "
 #define QUERY_TAIL  "}}"
@@ -156,6 +157,18 @@ static bool resp_get_feature(QDict *resp, const char *feature)
     g_assert(qdict_get_bool(_props, feature) == (expected_value));     \
 })
 
+#define resp_assert_feature_str(resp, feature, expected_value)         \
+({                                                                     \
+    QDict *_props;                                                     \
+                                                                       \
+    g_assert(_resp);                                                   \
+    g_assert(resp_has_props(_resp));                                   \
+    _props = resp_get_props(_resp);                                    \
+    g_assert(qdict_get(_props, feature));                              \
+    g_assert_cmpstr(qdict_get_try_str(_props, feature), ==,            \
+                    expected_value);                                   \
+})
+
 #define assert_feature(qts, cpu_type, feature, expected_value)         \
 ({                                                                     \
     QDict *_resp;                                                      \
@@ -166,6 +179,16 @@ static bool resp_get_feature(QDict *resp, const char *feature)
     qobject_unref(_resp);                                              \
 })
 
+#define assert_feature_str(qts, cpu_type, feature, expected_value)     \
+({                                                                     \
+    QDict *_resp;                                                      \
+                                                                       \
+    _resp = do_query_no_props(qts, cpu_type);                          \
+    g_assert(_resp);                                                   \
+    resp_assert_feature_str(_resp, feature, expected_value);           \
+    qobject_unref(_resp);                                              \
+})
+
 #define assert_set_feature(qts, cpu_type, feature, value)              \
 ({                                                                     \
     const char *_fmt = (value) ? "{ %s: true }" : "{ %s: false }";     \
@@ -177,6 +200,17 @@ static bool resp_get_feature(QDict *resp, const char *feature)
     qobject_unref(_resp);                                              \
 })
 
+#define assert_set_feature_str(qts, cpu_type, feature, value, _fmt)    \
+({                                                                     \
+    const char *__fmt = _fmt;                                          \
+    QDict *_resp;                                                      \
+                                                                       \
+    _resp = do_query(qts, cpu_type, __fmt, feature);                   \
+    g_assert(_resp);                                                   \
+    resp_assert_feature_str(_resp, feature, value);                    \
+    qobject_unref(_resp);                                              \
+})
+
 #define assert_has_feature_enabled(qts, cpu_type, feature)             \
     assert_feature(qts, cpu_type, feature, true)
 
@@ -413,6 +447,24 @@ static void sve_tests_sve_off_kvm(const void *data)
     qtest_quit(qts);
 }
 
+static void mte_tests_tag_memory_on(const void *data)
+{
+    QTestState *qts;
+
+    qts = qtest_init(MACHINE_MTE "-cpu max");
+
+    /*
+     * With tag memory, "mte" should default to on, and explicitly specifying
+     * either on or off should be fine.
+     */
+    assert_has_feature(qts, "max", "mte");
+
+    assert_set_feature_str(qts, "max", "mte", "off", "{ 'mte': 'off' }");
+    assert_set_feature_str(qts, "max", "mte", "on", "{ 'mte': 'on' }");
+
+    qtest_quit(qts);
+}
+
 static void pauth_tests_default(QTestState *qts, const char *cpu_type)
 {
     assert_has_feature_enabled(qts, cpu_type, "pauth");
@@ -425,6 +477,19 @@ static void pauth_tests_default(QTestState *qts, const char *cpu_type)
                  "{ 'pauth': false, 'pauth-impdef': true }");
 }
 
+static void mte_tests_default(QTestState *qts, const char *cpu_type)
+{
+    assert_has_feature(qts, cpu_type, "mte");
+
+    /*
+     * Without tag memory, mte will be off under tcg.
+     * Explicitly enabling it yields an error.
+     */
+    assert_set_feature_str(qts, "max", "mte", "off", "{ 'mte': 'off' }");
+    assert_error(qts, cpu_type, "mte=on requires tag memory",
+                 "{ 'mte': 'on' }");
+}
+
 static void test_query_cpu_model_expansion(const void *data)
 {
     QTestState *qts;
@@ -474,6 +539,7 @@ static void test_query_cpu_model_expansion(const void *data)
 
         sve_tests_default(qts, "max");
         pauth_tests_default(qts, "max");
+        mte_tests_default(qts, "max");
 
         /* Test that features that depend on KVM generate errors without. */
         assert_error(qts, "max",
@@ -517,6 +583,13 @@ static void test_query_cpu_model_expansion_kvm(const void *data)
         assert_set_feature(qts, "host", "pmu", false);
         assert_set_feature(qts, "host", "pmu", true);
 
+        /*
+         * Unfortunately, there's no easy way to test whether this instance
+         * of KVM supports MTE. So we can only assert that the feature
+         * is present, but not whether it can be toggled.
+         */
+        assert_has_feature(qts, "host", "mte");
+
         /*
          * Some features would be enabled by default, but they're disabled
          * because this instance of KVM doesn't support them. Test that the
@@ -631,6 +704,8 @@ int main(int argc, char **argv)
                             NULL, sve_tests_sve_off);
         qtest_add_data_func("/arm/kvm/query-cpu-model-expansion/sve-off",
                             NULL, sve_tests_sve_off_kvm);
+        qtest_add_data_func("/arm/max/query-cpu-model-expansion/tag-memory",
+                            NULL, mte_tests_tag_memory_on);
     }
 
     return g_test_run();
Signature · 1 line
2.39.1
#15 lore raw

Re: [PATCH v5 3/3] qtests/arm: add some mte tests

Hi,

Quoted 3 lines · Cornelia Huck

Still as you need to respin I think adding a short commit msg wouldn't hurt ;-) Add new cpu MTE feature tests with TCG+virt tag memory and TCG-no tag memory (default) attempting to set cpu mte option on/off. No real test for KVM because ../..

Quoted 53 lines
---
 tests/qtest/arm-cpu-features.c | 75 ++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)
tests/qtest/arm-cpu-features.c +23 -0
index 8691802950ca..c5dbf66e938a 100644
@@ -22,6 +22,7 @@
 
 #define MACHINE     "-machine virt,gic-version=max -accel tcg "
 #define MACHINE_KVM "-machine virt,gic-version=max -accel kvm -accel tcg "
+#define MACHINE_MTE "-machine virt,gic-version=max,mte=on -accel tcg "
 #define QUERY_HEAD  "{ 'execute': 'query-cpu-model-expansion', " \
                     "  'arguments': { 'type': 'full', "
 #define QUERY_TAIL  "}}"
@@ -156,6 +157,18 @@ static bool resp_get_feature(QDict *resp, const char *feature)
     g_assert(qdict_get_bool(_props, feature) == (expected_value));     \
 })
 
+#define resp_assert_feature_str(resp, feature, expected_value)         \
+({                                                                     \
+    QDict *_props;                                                     \
+                                                                       \
+    g_assert(_resp);                                                   \
+    g_assert(resp_has_props(_resp));                                   \
+    _props = resp_get_props(_resp);                                    \
+    g_assert(qdict_get(_props, feature));                              \
+    g_assert_cmpstr(qdict_get_try_str(_props, feature), ==,            \
+                    expected_value);                                   \
+})
+
 #define assert_feature(qts, cpu_type, feature, expected_value)         \
 ({                                                                     \
     QDict *_resp;                                                      \
@@ -166,6 +179,16 @@ static bool resp_get_feature(QDict *resp, const char *feature)
     qobject_unref(_resp);                                              \
 })
 
+#define assert_feature_str(qts, cpu_type, feature, expected_value)     \
+({                                                                     \
+    QDict *_resp;                                                      \
+                                                                       \
+    _resp = do_query_no_props(qts, cpu_type);                          \
+    g_assert(_resp);                                                   \
+    resp_assert_feature_str(_resp, feature, expected_value);           \
+    qobject_unref(_resp);                                              \
+})
+
 #define assert_set_feature(qts, cpu_type, feature, value)              \
 ({                                                                     \
     const char *_fmt = (value) ? "{ %s: true }" : "{ %s: false }";     \

tests/qtest/arm-cpu-features.c static bool resp_get_feature(QDict *resp, const char *feature) · line 202

Quoted 3 lines
tests/qtest/arm-cpu-features.c +0 -0
@@ -177,6 +200,17 @@ static bool resp_get_feature(QDict *resp, const char *feature)
     qobject_unref(_resp);                                              \
 })
 

Not really related to your series but those macros become increasingly difficult to follow. Especially the feature param versus format that are partly redundant look weird: "mte", "off", "{ 'mte': 'off' }

Starting adding comments may help the reading.

Quoted 18 lines
tests/qtest/arm-cpu-features.c +11 -0
+#define assert_set_feature_str(qts, cpu_type, feature, value, _fmt)    \
+({                                                                     \
+    const char *__fmt = _fmt;                                          \
+    QDict *_resp;                                                      \
+                                                                       \
+    _resp = do_query(qts, cpu_type, __fmt, feature);                   \
+    g_assert(_resp);                                                   \
+    resp_assert_feature_str(_resp, feature, value);                    \
+    qobject_unref(_resp);                                              \
+})
+
 #define assert_has_feature_enabled(qts, cpu_type, feature)             \
     assert_feature(qts, cpu_type, feature, true)
 
@@ -413,6 +447,24 @@ static void sve_tests_sve_off_kvm(const void *data)
     qtest_quit(qts);
 }
 

tests/qtest/arm-cpu-features.c static void sve_tests_sve_off_kvm(const void *data) · line 459

Quoted 10 lines
tests/qtest/arm-cpu-features.c +10 -0
+static void mte_tests_tag_memory_on(const void *data)
+{
+    QTestState *qts;
+
+    qts = qtest_init(MACHINE_MTE "-cpu max");
+
+    /*
+     * With tag memory, "mte" should default to on, and explicitly specifying
+     * either on or off should be fine.
+     */

the above comment rather applies to assert_set_feature_str's, right?

Quoted 16 lines
tests/qtest/arm-cpu-features.c +9 -0
+    assert_has_feature(qts, "max", "mte");
+
+    assert_set_feature_str(qts, "max", "mte", "off", "{ 'mte': 'off' }");
+    assert_set_feature_str(qts, "max", "mte", "on", "{ 'mte': 'on' }");
+
+    qtest_quit(qts);
+}
+
 static void pauth_tests_default(QTestState *qts, const char *cpu_type)
 {
     assert_has_feature_enabled(qts, cpu_type, "pauth");
@@ -425,6 +477,19 @@ static void pauth_tests_default(QTestState *qts, const char *cpu_type)
                  "{ 'pauth': false, 'pauth-impdef': true }");
 }
 
+static void mte_tests_default(QTestState *qts, const char *cpu_type)

tests/qtest/arm-cpu-features.c static void pauth_tests_default(QTestState *qts, const char *cpu_type) · line 490

Quoted 10 lines
tests/qtest/arm-cpu-features.c +10 -0
+{
+    assert_has_feature(qts, cpu_type, "mte");
+
+    /*
+     * Without tag memory, mte will be off under tcg.
+     * Explicitly enabling it yields an error.
+     */
+    assert_set_feature_str(qts, "max", "mte", "off", "{ 'mte': 'off' }");
+    assert_error(qts, cpu_type, "mte=on requires tag memory",
+                 "{ 'mte': 'on' }");

Sorry in v4 I reported I preferred the pauth msg, clarifying now:

assert_error(qts, cpu_type, "cannot enable pauth-impdef without pauth",
"{ 'pauth': false, 'pauth-impdef': true }");

Here would translate into cannot enable mte without tag memory.

Quoted 14 lines
tests/qtest/arm-cpu-features.c +3 -0
+}
+
 static void test_query_cpu_model_expansion(const void *data)
 {
     QTestState *qts;
@@ -474,6 +539,7 @@ static void test_query_cpu_model_expansion(const void *data)
 
         sve_tests_default(qts, "max");
         pauth_tests_default(qts, "max");
+        mte_tests_default(qts, "max");
 
         /* Test that features that depend on KVM generate errors without. */
         assert_error(qts, "max",

tests/qtest/arm-cpu-features.c static void test_query_cpu_model_expansion_kvm(const void *data) · line 591

Quoted 9 lines
tests/qtest/arm-cpu-features.c +6 -0
@@ -517,6 +583,13 @@ static void test_query_cpu_model_expansion_kvm(const void *data)
         assert_set_feature(qts, "host", "pmu", false);
         assert_set_feature(qts, "host", "pmu", true);
 
+        /*
+         * Unfortunately, there's no easy way to test whether this instance
+         * of KVM supports MTE. So we can only assert that the feature
+         * is present, but not whether it can be toggled.
+         */
+        assert_has_feature(qts, "host", "mte");

I know you replied in v4 but I am still confused:
What does

      (QEMU) query-cpu-model-expansion type=full model={"name":"host"}

return on a MTE capable host and and on a non MTE capable host?

If I remember correctly qmp_query_cpu_model_expansion loops over the advertised features and try to set them explicitly so if the host does not support it this should fail and the result should be different from the case where the host supports it (even if it is off by default)

Does assert_has_feature_enabled() returns false?

Thanks

Eric

Quoted 13 lines
tests/qtest/arm-cpu-features.c +3 -0
+
         /*
          * Some features would be enabled by default, but they're disabled
          * because this instance of KVM doesn't support them. Test that the
@@ -631,6 +704,8 @@ int main(int argc, char **argv)
                             NULL, sve_tests_sve_off);
         qtest_add_data_func("/arm/kvm/query-cpu-model-expansion/sve-off",
                             NULL, sve_tests_sve_off_kvm);
+        qtest_add_data_func("/arm/max/query-cpu-model-expansion/tag-memory",
+                            NULL, mte_tests_tag_memory_on);
     }
 
     return g_test_run();
#16 lore raw

Re: [PATCH v5 3/3] qtests/arm: add some mte tests

Quoted 11 lines · Eric Auger

Hi,

Quoted 3 lines · Cornelia Huck

Still as you need to respin I think adding a short commit msg wouldn't hurt ;-) Add new cpu MTE feature tests with TCG+virt tag memory and TCG-no tag memory (default) attempting to set cpu mte option on/off. No real test for KVM because ../..

Ok, I'll add some lines :)

Quoted 3 lines
Quoted 3 lines
---
 tests/qtest/arm-cpu-features.c | 75 ++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)

(...)

Quoted 13 lines

tests/qtest/arm-cpu-features.c static void pauth_tests_default(QTestState *qts, const char *cpu_type) · line 490

Quoted 11 lines
tests/qtest/arm-cpu-features.c +11 -0
+static void mte_tests_default(QTestState *qts, const char *cpu_type)
+{
+    assert_has_feature(qts, cpu_type, "mte");
+
+    /*
+     * Without tag memory, mte will be off under tcg.
+     * Explicitly enabling it yields an error.
+     */
+    assert_set_feature_str(qts, "max", "mte", "off", "{ 'mte': 'off' }");
+    assert_error(qts, cpu_type, "mte=on requires tag memory",
+                 "{ 'mte': 'on' }");

Sorry in v4 I reported I preferred the pauth msg, clarifying now:

Quoted 4 lines

assert_error(qts, cpu_type, "cannot enable pauth-impdef without pauth",
"{ 'pauth': false, 'pauth-impdef': true }");

Here would translate into cannot enable mte without tag memory.

Oh, so you mean that I should adapt the message generated by the code?

[did not get around to the rest of it this week, will try again next week]

#17 lore raw

Re: [PATCH v5 3/3] qtests/arm: add some mte tests

Quoted 18 lines · Cornelia Huck
Quoted 16 lines · Eric Auger

Hi,

tests/qtest/arm-cpu-features.c static void pauth_tests_default(QTestState *qts, const char *cpu_type) · line 490

Quoted 11 lines · Cornelia Huck
tests/qtest/arm-cpu-features.c +11 -0
+static void mte_tests_default(QTestState *qts, const char *cpu_type)
+{
+    assert_has_feature(qts, cpu_type, "mte");
+
+    /*
+     * Without tag memory, mte will be off under tcg.
+     * Explicitly enabling it yields an error.
+     */
+    assert_set_feature_str(qts, "max", "mte", "off", "{ 'mte': 'off' }");
+    assert_error(qts, cpu_type, "mte=on requires tag memory",
+                 "{ 'mte': 'on' }");

Sorry in v4 I reported I preferred the pauth msg, clarifying now:

Quoted 6 lines
Quoted 4 lines

assert_error(qts, cpu_type, "cannot enable pauth-impdef without pauth",
"{ 'pauth': false, 'pauth-impdef': true }");

Here would translate into cannot enable mte without tag memory.

Oh, so you mean that I should adapt the message generated by the code?

Friendly ping :) Did you mean to adapt the error messages in cpu64.c?

#18 lore raw

Re: [PATCH v5 3/3] qtests/arm: add some mte tests

In reply to Eric Auger
Quoted 15 lines · Eric Auger

Hi,

tests/qtest/arm-cpu-features.c static void test_query_cpu_model_expansion_kvm(const void *data) · line 591

Quoted 10 lines · Cornelia Huck
tests/qtest/arm-cpu-features.c +6 -0
@@ -517,6 +583,13 @@ static void test_query_cpu_model_expansion_kvm(const void *data)
         assert_set_feature(qts, "host", "pmu", false);
         assert_set_feature(qts, "host", "pmu", true);
 
+        /*
+         * Unfortunately, there's no easy way to test whether this instance
+         * of KVM supports MTE. So we can only assert that the feature
+         * is present, but not whether it can be toggled.
+         */
+        assert_has_feature(qts, "host", "mte");

I know you replied in v4 but I am still confused:
What does

Quoted 2 lines
      (QEMU) query-cpu-model-expansion type=full model={"name":"host"}

return on a MTE capable host and and on a non MTE capable host?

FWIW, it's "auto" in both cases, but the main problem is actually something else...

Quoted 7 lines

If I remember correctly qmp_query_cpu_model_expansion loops over the advertised features and try to set them explicitly so if the host does not support it this should fail and the result should be different from the case where the host supports it (even if it is off by default)

Does assert_has_feature_enabled() returns false?

I poked around a bit with qmp on a system (well, model) with MTE where starting a guest with MTE works just fine. I used the minimal setup described in docs/devel/writing-monitor-commands.rst, and trying to do a cpu model expansion with mte=on fails because the KVM ioctl fails with -EINVAL (as we haven't set up proper memory mappings). The qtest setup doesn't do any proper setup either AFAICS, so enabling MTE won't work even if KVM and the host support it. (Trying to enable MTE on a host that doesn't support it would also report an error, but a different one, as KVM would not support the MTE cap at all.) We don't really know beforehand what to expect ("auto" is not yet expanded, see above), so I'm not sure how to test this in a meaningful way, even if we did set up memory mappings (which seems like overkill for a feature test.)

The comment describing this could be improved, though :)

#19 lore raw

Re: [PATCH v5 3/3] qtests/arm: add some mte tests

Hi Connie,

Quoted 44 lines · Cornelia Huck
Quoted 15 lines · Eric Auger

Hi,

tests/qtest/arm-cpu-features.c static void test_query_cpu_model_expansion_kvm(const void *data) · line 591

Quoted 10 lines · Cornelia Huck
tests/qtest/arm-cpu-features.c +6 -0
@@ -517,6 +583,13 @@ static void test_query_cpu_model_expansion_kvm(const void *data)
         assert_set_feature(qts, "host", "pmu", false);
         assert_set_feature(qts, "host", "pmu", true);
 
+        /*
+         * Unfortunately, there's no easy way to test whether this instance
+         * of KVM supports MTE. So we can only assert that the feature
+         * is present, but not whether it can be toggled.
+         */
+        assert_has_feature(qts, "host", "mte");

I know you replied in v4 but I am still confused:
What does

Quoted 2 lines
      (QEMU) query-cpu-model-expansion type=full model={"name":"host"}

return on a MTE capable host and and on a non MTE capable host?

FWIW, it's "auto" in both cases, but the main problem is actually something else...

Quoted 7 lines

If I remember correctly qmp_query_cpu_model_expansion loops over the advertised features and try to set them explicitly so if the host does not support it this should fail and the result should be different from the case where the host supports it (even if it is off by default)

Does assert_has_feature_enabled() returns false?

I poked around a bit with qmp on a system (well, model) with MTE where starting a guest with MTE works just fine. I used the minimal setup described in docs/devel/writing-monitor-commands.rst, and trying to do a cpu model expansion with mte=on fails because the KVM ioctl fails with -EINVAL (as we haven't set up proper memory mappings). The qtest setup doesn't do any proper setup either AFAICS, so enabling MTE won't work even if KVM and the host support it. (Trying to enable MTE on a host that doesn't support it would also report an error, but a different one, as KVM would not support the MTE cap at all.) We don't really know beforehand what to expect ("auto" is not yet expanded, see above), so I'm not sure how to test this in a meaningful way, even if we did set up memory mappings (which seems like overkill for a feature test.)

Quoted 2 lines

The comment describing this could be improved, though :)

OK fair enough, don't make it a blocking issue for the series and simply update the comment up to your knowledge.

Thanks

Eric

#20 lore raw
In reply to Cornelia Huck
Quoted 2 lines · Cornelia Huck

Respin of my kvm mte series; tested via check + check-tcg and on FVP.

I've taken patch 1 into target-arm.next since it's a simple cleanup.

thanks
-- PMM

#21 lore raw
Quoted 6 lines · Peter Maydell
Quoted 2 lines · Cornelia Huck

Respin of my kvm mte series; tested via check + check-tcg and on FVP.

I've taken patch 1 into target-arm.next since it's a simple cleanup.

Thanks!

(I plan to send a respin of the remainder once we've agreed on some of the feedback.)