Skip to contentSkip to conversation

arm: Fixes and preparatory cleanups for split-accel

PATCH Superseded Philippe Mathieu-Daudé 5 participants 49 messages Jun 23, 2025 – Jul 1, 2025

Subsystems (3)

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v3 cover arm: Fixes and preparatory cleanups for split-accel

Only the last patch is missing review (#26)

Since v2:
- Addressed thuth review comments

Since v1:
- Addressed rth's review comments

Omnibus series of ARM-related patches (noticed during the "split accel" PoC work).

- Usual prototypes cleanups
- Check TCG for EL2/EL3 features (and not !KVM or !HVF)
- Improve HVF debugging
- Correct HVF 'dtb_compatible' value for Linux
- Fix HVF GTimer frequency (My M1 hardware has 24 MHz) (this implies accel/ rework w.r.t. QDev vCPU REALIZE)
- Expand functional tests w.r.t. HVF

Regards,

Phil.

Philippe Mathieu-Daudé (26):
target/arm: Remove arm_handle_psci_call() stub
target/arm: Reduce arm_cpu_post_init() declaration scope target/arm: Unify gen_exception_internal()
target/arm/hvf: Simplify GIC hvf_arch_init_vcpu() target/arm/hvf: Directly re-lock BQL after hv_vcpu_run() target/arm/hvf: Trace hv_vcpu_run() failures
accel/hvf: Trace VM memory mapping
target/arm/hvf: Log $pc in hvf_unknown_hvc() trace event target/arm: Correct KVM & HVF dtb_compatible value accel/hvf: Model PhysTimer register
target/arm/hvf: Pass @target_el argument to hvf_raise_exception() target/arm: Restrict system register properties to system binary target/arm: Create GTimers *after* features finalized / accel realized
accel: Keep reference to AccelOpsClass in AccelClass
accel: Introduce AccelOpsClass::cpu_target_realize() hook accel/hvf: Add hvf_arch_cpu_realize() stubs target/arm/hvf: Really set Generic Timer counter frequency

  hw/arm/virt: Only require TCG || QTest to use TrustZone
  hw/arm/virt: Only require TCG || QTest to use virtualization extension

hw/arm/virt: Rename cpu_post_init() -> post_cpus_gic_realized() hw/arm/sbsa-ref: Tidy up use of RAMLIMIT_GB definition tests/functional: Set sbsa-ref machine type in each test function tests/functional: Restrict nested Aarch64 Xen test to TCG tests/functional: Require TCG to run Aarch64 imx8mp-evk test tests/functional: Add hvf_available() helper tests/functional: Expand Aarch64 SMMU tests to run on HVF accelerator

 meson.build                                   |  1 +
 accel/hvf/trace.h                             |  2 +
 include/qemu/accel.h                          |  3 +
 include/system/accel-ops.h                    |  4 +-
 include/system/hvf.h                          |  3 +
 target/arm/cpu.h                              |  2 -
 target/arm/internals.h                        |  6 +-
 target/arm/tcg/translate.h                    |  1 +
 accel/accel-common.c                          |  4 +
 accel/accel-system.c                          |  3 +-
 accel/hvf/hvf-accel-ops.c                     |  8 ++
 accel/tcg/tcg-accel-ops.c                     |  4 +-
 hw/arm/sbsa-ref.c                             |  8 +-
 hw/arm/virt.c                                 |  9 +-
 target/arm/cpu.c                              | 78 +++++++++--------
 target/arm/hvf/hvf.c                          | 86 ++++++++++++-------
 target/arm/kvm.c                              |  2 +-
 target/arm/tcg/translate-a64.c                |  6 --
 target/arm/tcg/translate.c                    |  2 +-
 target/i386/hvf/hvf.c                         |  5 ++
 accel/hvf/trace-events                        |  7 ++
 python/qemu/utils/__init__.py                 |  2 +-
 python/qemu/utils/accel.py                    |  8 ++
 target/arm/hvf/trace-events                   |  5 +-
 tests/functional/qemu_test/testcase.py        |  6 +-
 tests/functional/test_aarch64_imx8mp_evk.py   |  1 +
 tests/functional/test_aarch64_sbsaref.py      |  5 +-
 .../functional/test_aarch64_sbsaref_alpine.py |  3 +-
 .../test_aarch64_sbsaref_freebsd.py           |  3 +-
 tests/functional/test_aarch64_smmu.py         | 12 ++-
 tests/functional/test_aarch64_xen.py          |  1 +
 31 files changed, 186 insertions(+), 104 deletions(-)
 create mode 100644 accel/hvf/trace.h
 create mode 100644 accel/hvf/trace-events
Signature · 1 line
2.49.0
#1 lore raw

PATCH v3 1/26 target/arm: Remove arm_handle_psci_call() stub

Since commit 0c1aaa66c24 ("target/arm: wrap psci call with tcg_enabled") the arm_handle_psci_call() call is elided when TCG is disabled.

---
 target/arm/internals.h | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
target/arm/internals.h +1 -5
index 3360de9150f..5ed25d33208 100644
@@ -650,16 +650,12 @@ static inline bool arm_is_psci_call(ARMCPU *cpu, int excp_type)
 {
     return false;
 }
-static inline void arm_handle_psci_call(ARMCPU *cpu)
-{
-    g_assert_not_reached();
-}
 #else
 /* Return true if the r0/x0 value indicates that this SMC/HVC is a PSCI call. */
 bool arm_is_psci_call(ARMCPU *cpu, int excp_type);
+#endif
 /* Actually handle a PSCI call */
 void arm_handle_psci_call(ARMCPU *cpu);
-#endif
 
 /**
  * arm_clear_exclusive: clear the exclusive monitor
Signature · 1 line
2.49.0
#2 lore raw

PATCH v3 2/26 target/arm: Reduce arm_cpu_post_init() declaration scope

In reply to Philippe Mathieu-Daudé

arm_cpu_post_init() is only used within the same file unit.

---
 target/arm/cpu.h | 2 --
 target/arm/cpu.c | 2 +-
 2 files changed, 1 insertion(+), 3 deletions(-)
target/arm/cpu.h +0 -2
index 302c24e2324..c31f69912b8 100644
@@ -1150,8 +1150,6 @@ void arm_gt_sel2vtimer_cb(void *opaque);
 unsigned int gt_cntfrq_period_ns(ARMCPU *cpu);
 void gt_rme_post_el_change(ARMCPU *cpu, void *opaque);
 
-void arm_cpu_post_init(Object *obj);
-
 #define ARM_AFF0_SHIFT 0
 #define ARM_AFF0_MASK  (0xFFULL << ARM_AFF0_SHIFT)
 #define ARM_AFF1_SHIFT 8
target/arm/cpu.c +1 -1
index e025e241eda..eb0639de719 100644
@@ -1713,7 +1713,7 @@ static void arm_cpu_propagate_feature_implications(ARMCPU *cpu)
     }
 }
 
-void arm_cpu_post_init(Object *obj)
+static void arm_cpu_post_init(Object *obj)
 {
     ARMCPU *cpu = ARM_CPU(obj);
 
Signature · 1 line
2.49.0
#3 lore raw

PATCH v3 3/26 target/arm: Unify gen_exception_internal()

In reply to Philippe Mathieu-Daudé

Same code, use the generic variant.

---
 target/arm/tcg/translate.h     | 1 +
 target/arm/tcg/translate-a64.c | 6 ------
 target/arm/tcg/translate.c     | 2 +-
 3 files changed, 2 insertions(+), 7 deletions(-)
target/arm/tcg/translate.h +1 -0
index 1bfdb0fb9bb..0004a97219b 100644
@@ -347,6 +347,7 @@ void arm_jump_cc(DisasCompare *cmp, TCGLabel *label);
 void arm_gen_test_cc(int cc, TCGLabel *label);
 MemOp pow2_align(unsigned i);
 void unallocated_encoding(DisasContext *s);
+void gen_exception_internal(int excp);
 void gen_exception_insn_el(DisasContext *s, target_long pc_diff, int excp,
                            uint32_t syn, uint32_t target_el);
 void gen_exception_insn(DisasContext *s, target_long pc_diff,
target/arm/tcg/translate-a64.c +0 -6
index ac80f572a2d..7c79b8c4401 100644
@@ -433,12 +433,6 @@ static void gen_rebuild_hflags(DisasContext *s)
     gen_helper_rebuild_hflags_a64(tcg_env, tcg_constant_i32(s->current_el));
 }
 
-static void gen_exception_internal(int excp)
-{
-    assert(excp_is_internal(excp));
-    gen_helper_exception_internal(tcg_env, tcg_constant_i32(excp));
-}
-
 static void gen_exception_internal_insn(DisasContext *s, int excp)
 {
     gen_a64_update_pc(s, 0);
target/arm/tcg/translate.c +1 -1
index 9962f43b1d0..f7d6d8ce196 100644
@@ -372,7 +372,7 @@ static void gen_rebuild_hflags(DisasContext *s, bool new_el)
     }
 }
 
-static void gen_exception_internal(int excp)
+void gen_exception_internal(int excp)
 {
     assert(excp_is_internal(excp));
     gen_helper_exception_internal(tcg_env, tcg_constant_i32(excp));
Signature · 1 line
2.49.0
#4 lore raw

PATCH v3 4/26 target/arm/hvf: Simplify GIC hvf_arch_init_vcpu()

In reply to Philippe Mathieu-Daudé

Only update the ID_AA64PFR0_EL1 register when a GIC is provided.

---
 target/arm/hvf/hvf.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
target/arm/hvf/hvf.c +9 -5
index 42258cc2d88..c1ed8b510db 100644
@@ -1057,11 +1057,15 @@ int hvf_arch_init_vcpu(CPUState *cpu)
                               arm_cpu->mp_affinity);
     assert_hvf_ok(ret);
 
-    ret = hv_vcpu_get_sys_reg(cpu->accel->fd, HV_SYS_REG_ID_AA64PFR0_EL1, &pfr);
-    assert_hvf_ok(ret);
-    pfr |= env->gicv3state ? (1 << 24) : 0;
-    ret = hv_vcpu_set_sys_reg(cpu->accel->fd, HV_SYS_REG_ID_AA64PFR0_EL1, pfr);
-    assert_hvf_ok(ret);
+    if (env->gicv3state) {
+        ret = hv_vcpu_get_sys_reg(cpu->accel->fd,
+                                  HV_SYS_REG_ID_AA64PFR0_EL1, &pfr);
+        assert_hvf_ok(ret);
+        pfr = FIELD_DP64(pfr, ID_AA64PFR0, GIC, 1);
+        ret = hv_vcpu_set_sys_reg(cpu->accel->fd,
+                                  HV_SYS_REG_ID_AA64PFR0_EL1, pfr);
+        assert_hvf_ok(ret);
+    }
 
     /* We're limited to underlying hardware caps, override internal versions */
     ret = hv_vcpu_get_sys_reg(cpu->accel->fd, HV_SYS_REG_ID_AA64MMFR0_EL1,
Signature · 1 line
2.49.0
#5 lore raw

Re: [PATCH v3 04/26] target/arm/hvf: Simplify GIC hvf_arch_init_vcpu()

Quoted 21 lines · Philippe Mathieu-Daudé

Only update the ID_AA64PFR0_EL1 register when a GIC is provided.

---
 target/arm/hvf/hvf.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
target/arm/hvf/hvf.c +0 -4
index 42258cc2d88..c1ed8b510db 100644
@@ -1057,11 +1057,15 @@ int hvf_arch_init_vcpu(CPUState *cpu)
                               arm_cpu->mp_affinity);
     assert_hvf_ok(ret);
 
-    ret = hv_vcpu_get_sys_reg(cpu->accel->fd, HV_SYS_REG_ID_AA64PFR0_EL1, &pfr);
-    assert_hvf_ok(ret);
-    pfr |= env->gicv3state ? (1 << 24) : 0;
-    ret = hv_vcpu_set_sys_reg(cpu->accel->fd, HV_SYS_REG_ID_AA64PFR0_EL1, pfr);

target/arm/hvf/hvf.c int hvf_arch_init_vcpu(CPUState *cpu) · line 1068

Quoted 10 lines
target/arm/hvf/hvf.c +9 -1
-    assert_hvf_ok(ret);
+    if (env->gicv3state) {
+        ret = hv_vcpu_get_sys_reg(cpu->accel->fd,
+                                  HV_SYS_REG_ID_AA64PFR0_EL1, &pfr);
+        assert_hvf_ok(ret);
+        pfr = FIELD_DP64(pfr, ID_AA64PFR0, GIC, 1);
+        ret = hv_vcpu_set_sys_reg(cpu->accel->fd,
+                                  HV_SYS_REG_ID_AA64PFR0_EL1, pfr);
+        assert_hvf_ok(ret);
+    }

This doesn't seem like a simplification to me...

Looking at the code, I suspect what we should really be doing is setting the GIC field to either 0 or 1 depending on whether env->gicv3state. Currently if hvf hands us an initial value with the GIC field set to 1 but we don't have a gicv3state we won't correctly clear it to 0. i.e. we should change the current
pfr |= env->gicv3state ? (1 << 24) : 0;
to
pfr = FIELD_DP64(pfr, ID_AA64PFR0, GIC, env->gicv3state ? 1 : 0);

thanks
-- PMM

#6 lore raw

Re: [PATCH v3 04/26] target/arm/hvf: Simplify GIC hvf_arch_init_vcpu()

Quoted 35 lines · Peter Maydell
Quoted 21 lines · Philippe Mathieu-Daudé

Only update the ID_AA64PFR0_EL1 register when a GIC is provided.

---
  target/arm/hvf/hvf.c | 14 +++++++++-----
  1 file changed, 9 insertions(+), 5 deletions(-)
target/arm/hvf/hvf.c +0 -4
index 42258cc2d88..c1ed8b510db 100644
@@ -1057,11 +1057,15 @@ int hvf_arch_init_vcpu(CPUState *cpu)
                                arm_cpu->mp_affinity);
      assert_hvf_ok(ret);
 
-    ret = hv_vcpu_get_sys_reg(cpu->accel->fd, HV_SYS_REG_ID_AA64PFR0_EL1, &pfr);
-    assert_hvf_ok(ret);
-    pfr |= env->gicv3state ? (1 << 24) : 0;
-    ret = hv_vcpu_set_sys_reg(cpu->accel->fd, HV_SYS_REG_ID_AA64PFR0_EL1, pfr);

target/arm/hvf/hvf.c int hvf_arch_init_vcpu(CPUState *cpu) · line 1068

Quoted 10 lines
target/arm/hvf/hvf.c +9 -1
-    assert_hvf_ok(ret);
+    if (env->gicv3state) {
+        ret = hv_vcpu_get_sys_reg(cpu->accel->fd,
+                                  HV_SYS_REG_ID_AA64PFR0_EL1, &pfr);
+        assert_hvf_ok(ret);
+        pfr = FIELD_DP64(pfr, ID_AA64PFR0, GIC, 1);
+        ret = hv_vcpu_set_sys_reg(cpu->accel->fd,
+                                  HV_SYS_REG_ID_AA64PFR0_EL1, pfr);
+        assert_hvf_ok(ret);
+    }

This doesn't seem like a simplification to me...

Quoted 8 lines

Looking at the code, I suspect what we should really be doing is setting the GIC field to either 0 or 1 depending on whether env->gicv3state. Currently if hvf hands us an initial value with the GIC field set to 1 but we don't have a gicv3state we won't correctly clear it to 0. i.e. we should change the current
pfr |= env->gicv3state ? (1 << 24) : 0;
to
pfr = FIELD_DP64(pfr, ID_AA64PFR0, GIC, env->gicv3state ? 1 : 0);

Good idea.

#7 lore raw

PATCH v3 5/26 target/arm/hvf: Directly re-lock BQL after hv_vcpu_run()

In reply to Philippe Mathieu-Daudé

Keep bql_unlock() / bql_lock() close.

---
 target/arm/hvf/hvf.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
target/arm/hvf/hvf.c +3 -2
index c1ed8b510db..ef76dcd28de 100644
@@ -1914,7 +1914,9 @@ int hvf_vcpu_exec(CPUState *cpu)
     flush_cpu_state(cpu);
 
     bql_unlock();
-    assert_hvf_ok(hv_vcpu_run(cpu->accel->fd));
+    r = hv_vcpu_run(cpu->accel->fd);
+    bql_lock();
+    assert_hvf_ok(r);
 
     /* handle VMEXIT */
     uint64_t exit_reason = hvf_exit->reason;
@@ -1922,7 +1924,6 @@ int hvf_vcpu_exec(CPUState *cpu)
     uint32_t ec = syn_get_ec(syndrome);
 
     ret = 0;
-    bql_lock();
     switch (exit_reason) {
     case HV_EXIT_REASON_EXCEPTION:
         /* This is the main one, handle below. */
Signature · 1 line
2.49.0
Reviewed-by on v3 5/26
#8 lore raw

Re: [PATCH v3 05/26] target/arm/hvf: Directly re-lock BQL after hv_vcpu_run()

Quoted 24 lines · Philippe Mathieu-Daudé

Keep bql_unlock() / bql_lock() close.

---
  target/arm/hvf/hvf.c | 5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)
target/arm/hvf/hvf.c +3 -1
index c1ed8b510db..ef76dcd28de 100644
@@ -1914,7 +1914,9 @@ int hvf_vcpu_exec(CPUState *cpu)
      flush_cpu_state(cpu);
 
      bql_unlock();
-    assert_hvf_ok(hv_vcpu_run(cpu->accel->fd));
+    r = hv_vcpu_run(cpu->accel->fd);
+    bql_lock();
+    assert_hvf_ok(r);
 
      /* handle VMEXIT */
      uint64_t exit_reason = hvf_exit->reason;

target/arm/hvf/hvf.c int hvf_vcpu_exec(CPUState *cpu) · line 1929

Quoted 7 lines
target/arm/hvf/hvf.c +0 -1
@@ -1922,7 +1924,6 @@ int hvf_vcpu_exec(CPUState *cpu)
      uint32_t ec = syn_get_ec(syndrome);
 
      ret = 0;
-    bql_lock();
      switch (exit_reason) {
      case HV_EXIT_REASON_EXCEPTION:
          /* This is the main one, handle below. */

Just moves the lock a few lines up, does not impact what is protected.

#9 lore raw

PATCH v3 6/26 target/arm/hvf: Trace hv_vcpu_run() failures

In reply to Philippe Mathieu-Daudé

Allow distinguishing HV_ILLEGAL_GUEST_STATE in trace events.

---
 target/arm/hvf/hvf.c        | 10 +++++++++-
 target/arm/hvf/trace-events |  1 +
 2 files changed, 10 insertions(+), 1 deletion(-)
target/arm/hvf/hvf.c +9 -1
index ef76dcd28de..cc5bbc155d2 100644
@@ -1916,7 +1916,15 @@ int hvf_vcpu_exec(CPUState *cpu)
     bql_unlock();
     r = hv_vcpu_run(cpu->accel->fd);
     bql_lock();
-    assert_hvf_ok(r);
+    switch (r) {
+    case HV_SUCCESS:
+        break;
+    case HV_ILLEGAL_GUEST_STATE:
+        trace_hvf_illegal_guest_state();
+        /* fall through */
+    default:
+        g_assert_not_reached();
+    }
 
     /* handle VMEXIT */
     uint64_t exit_reason = hvf_exit->reason;
target/arm/hvf/trace-events +1 -0
index 4fbbe4b45ec..a4870e0a5c4 100644
@@ -11,3 +11,4 @@ hvf_exit(uint64_t syndrome, uint32_t ec, uint64_t pc) "exit: 0x%"PRIx64" [ec=0x%
 hvf_psci_call(uint64_t x0, uint64_t x1, uint64_t x2, uint64_t x3, uint32_t cpuid) "PSCI Call x0=0x%016"PRIx64" x1=0x%016"PRIx64" x2=0x%016"PRIx64" x3=0x%016"PRIx64" cpu=0x%x"
 hvf_vgic_write(const char *name, uint64_t val) "vgic write to %s [val=0x%016"PRIx64"]"
 hvf_vgic_read(const char *name, uint64_t val) "vgic read from %s [val=0x%016"PRIx64"]"
+hvf_illegal_guest_state(void) "HV_ILLEGAL_GUEST_STATE"
Signature · 1 line
2.49.0
#10 lore raw

Re: [PATCH v3 06/26] target/arm/hvf: Trace hv_vcpu_run() failures

Quoted 17 lines · Philippe Mathieu-Daudé

Allow distinguishing HV_ILLEGAL_GUEST_STATE in trace events.

---
 target/arm/hvf/hvf.c        | 10 +++++++++-
 target/arm/hvf/trace-events |  1 +
 2 files changed, 10 insertions(+), 1 deletion(-)
target/arm/hvf/hvf.c +0 -0
index ef76dcd28de..cc5bbc155d2 100644
@@ -1916,7 +1916,15 @@ int hvf_vcpu_exec(CPUState *cpu)
     bql_unlock();
     r = hv_vcpu_run(cpu->accel->fd);

target/arm/hvf/hvf.c int hvf_vcpu_exec(CPUState *cpu) · line 1926

Quoted 10 lines
target/arm/hvf/hvf.c +8 -1
     bql_lock();
-    assert_hvf_ok(r);
+    switch (r) {
+    case HV_SUCCESS:
+        break;
+    case HV_ILLEGAL_GUEST_STATE:
+        trace_hvf_illegal_guest_state();
+        /* fall through */
+    default:
+        g_assert_not_reached();

This seems kind of odd.

If it can happen, we shouldn't g_assert_not_reached(). If it can't happen, we shouldn't trace it.

But the hvf code already has a lot of "assert success rather than handling possible-but-fatal errors more gracefully", so I guess it's OK.

-- PMM

#11 lore raw

Re: [PATCH v3 06/26] target/arm/hvf: Trace hv_vcpu_run() failures

Quoted 31 lines · Peter Maydell
Quoted 17 lines · Philippe Mathieu-Daudé

Allow distinguishing HV_ILLEGAL_GUEST_STATE in trace events.

---
  target/arm/hvf/hvf.c        | 10 +++++++++-
  target/arm/hvf/trace-events |  1 +
  2 files changed, 10 insertions(+), 1 deletion(-)
target/arm/hvf/hvf.c +0 -0
index ef76dcd28de..cc5bbc155d2 100644
@@ -1916,7 +1916,15 @@ int hvf_vcpu_exec(CPUState *cpu)
      bql_unlock();
      r = hv_vcpu_run(cpu->accel->fd);

target/arm/hvf/hvf.c int hvf_vcpu_exec(CPUState *cpu) · line 1926

Quoted 10 lines
target/arm/hvf/hvf.c +8 -1
      bql_lock();
-    assert_hvf_ok(r);
+    switch (r) {
+    case HV_SUCCESS:
+        break;
+    case HV_ILLEGAL_GUEST_STATE:
+        trace_hvf_illegal_guest_state();
+        /* fall through */
+    default:
+        g_assert_not_reached();

This seems kind of odd.

Quoted 6 lines

If it can happen, we shouldn't g_assert_not_reached(). If it can't happen, we shouldn't trace it.

But the hvf code already has a lot of "assert success rather than handling possible-but-fatal errors more gracefully", so I guess it's OK.

OK, you can drop this patch: I will replace with error("unrecoverable:
...") && exit(1); to avoid such oddity.

#12 lore raw

PATCH v3 7/26 accel/hvf: Trace VM memory mapping

In reply to Philippe Mathieu-Daudé

Trace memory mapped / unmapped in the guest.

---
 meson.build               | 1 +
 accel/hvf/trace.h         | 2 ++
 accel/hvf/hvf-accel-ops.c | 6 ++++++
 accel/hvf/trace-events    | 7 +++++++
 4 files changed, 16 insertions(+)
 create mode 100644 accel/hvf/trace.h
 create mode 100644 accel/hvf/trace-events
meson.build +1 -0
index 34729c2a3dd..5004678a26b 100644
@@ -3633,6 +3633,7 @@ if have_block
 endif
 if have_system
   trace_events_subdirs += [
+    'accel/hvf',
     'accel/kvm',
     'audio',
     'backends',
accel/hvf/trace.h new file +2 -0
new file mode 100644
index 00000000000..83a1883343a
@@ -0,0 +1,2 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#include "trace/trace-accel_hvf.h"
accel/hvf/hvf-accel-ops.c +6 -0
index d60446b85b8..b38977207d2 100644
@@ -59,6 +59,7 @@
 #include "system/hvf_int.h"
 #include "system/runstate.h"
 #include "qemu/guest-random.h"
+#include "trace.h"
 
 HVFState *hvf_state;
 
@@ -97,6 +98,7 @@ static int do_hvf_set_memory(hvf_slot *slot, hv_memory_flags_t flags)
     if (macslot->present) {
         if (macslot->size != slot->size) {
             macslot->present = 0;
+            trace_hvf_vm_unmap(macslot->gpa_start, macslot->size);
             ret = hv_vm_unmap(macslot->gpa_start, macslot->size);
             assert_hvf_ok(ret);
         }
@@ -109,6 +111,10 @@ static int do_hvf_set_memory(hvf_slot *slot, hv_memory_flags_t flags)
     macslot->present = 1;
     macslot->gpa_start = slot->start;
     macslot->size = slot->size;
+    trace_hvf_vm_map(slot->start, slot->size, slot->mem, flags,
+                     flags & HV_MEMORY_READ ?  'R' : '-',
+                     flags & HV_MEMORY_WRITE ? 'W' : '-',
+                     flags & HV_MEMORY_EXEC ?  'E' : '-');
     ret = hv_vm_map(slot->mem, slot->start, slot->size, flags);
     assert_hvf_ok(ret);
     return 0;
accel/hvf/trace-events new file +7 -0
new file mode 100644
index 00000000000..2fd3e127c74
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# See docs/devel/tracing.rst for syntax documentation.
+
+# hvf-accel-ops.c
+hvf_vm_map(uint64_t paddr, uint64_t size, void *vaddr, uint8_t flags, const char r, const char w, const char e) "paddr:0x%016"PRIx64" size:0x%08"PRIx64" vaddr:%p flags:0x%02x/%c%c%c"
+hvf_vm_unmap(uint64_t paddr, uint64_t size) "paddr:0x%016"PRIx64" size:0x%08"PRIx64
Signature · 1 line
2.49.0
#13 lore raw

PATCH v3 8/26 target/arm/hvf: Log $pc in hvf_unknown_hvc() trace event

In reply to Philippe Mathieu-Daudé

Tracing $PC for unknown HVC instructions to not have to look at the disassembled flow of instructions.

---
 target/arm/hvf/hvf.c        | 4 ++--
 target/arm/hvf/trace-events | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
target/arm/hvf/hvf.c +2 -2
index cc5bbc155d2..d4c58516e8b 100644
@@ -2071,12 +2071,12 @@ int hvf_vcpu_exec(CPUState *cpu)
         cpu_synchronize_state(cpu);
         if (arm_cpu->psci_conduit == QEMU_PSCI_CONDUIT_HVC) {
             if (!hvf_handle_psci_call(cpu)) {
-                trace_hvf_unknown_hvc(env->xregs[0]);
+                trace_hvf_unknown_hvc(env->pc, env->xregs[0]);
                 /* SMCCC 1.3 section 5.2 says every unknown SMCCC call returns -1 */
                 env->xregs[0] = -1;
             }
         } else {
-            trace_hvf_unknown_hvc(env->xregs[0]);
+            trace_hvf_unknown_hvc(env->pc, env->xregs[0]);
             hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized());
         }
         break;
target/arm/hvf/trace-events +2 -2
index a4870e0a5c4..b49746f28d1 100644
@@ -5,10 +5,10 @@ hvf_inject_irq(void) "injecting IRQ"
 hvf_data_abort(uint64_t pc, uint64_t va, uint64_t pa, bool isv, bool iswrite, bool s1ptw, uint32_t len, uint32_t srt) "data abort: [pc=0x%"PRIx64" va=0x%016"PRIx64" pa=0x%016"PRIx64" isv=%d iswrite=%d s1ptw=%d len=%d srt=%d]"
 hvf_sysreg_read(uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_t crm, uint32_t op2, uint64_t val) "sysreg read 0x%08x (op0=%d op1=%d crn=%d crm=%d op2=%d) = 0x%016"PRIx64
 hvf_sysreg_write(uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_t crm, uint32_t op2, uint64_t val) "sysreg write 0x%08x (op0=%d op1=%d crn=%d crm=%d op2=%d, val=0x%016"PRIx64")"
-hvf_unknown_hvc(uint64_t x0) "unknown HVC! 0x%016"PRIx64
+hvf_unknown_hvc(uint64_t pc, uint64_t x0) "pc=0x%"PRIx64" unknown HVC! 0x%016"PRIx64
 hvf_unknown_smc(uint64_t x0) "unknown SMC! 0x%016"PRIx64
 hvf_exit(uint64_t syndrome, uint32_t ec, uint64_t pc) "exit: 0x%"PRIx64" [ec=0x%x pc=0x%"PRIx64"]"
-hvf_psci_call(uint64_t x0, uint64_t x1, uint64_t x2, uint64_t x3, uint32_t cpuid) "PSCI Call x0=0x%016"PRIx64" x1=0x%016"PRIx64" x2=0x%016"PRIx64" x3=0x%016"PRIx64" cpu=0x%x"
+hvf_psci_call(uint64_t x0, uint64_t x1, uint64_t x2, uint64_t x3, uint32_t cpuid) "PSCI Call x0=0x%016"PRIx64" x1=0x%016"PRIx64" x2=0x%016"PRIx64" x3=0x%016"PRIx64" cpuid=0x%x"
 hvf_vgic_write(const char *name, uint64_t val) "vgic write to %s [val=0x%016"PRIx64"]"
 hvf_vgic_read(const char *name, uint64_t val) "vgic read from %s [val=0x%016"PRIx64"]"
 hvf_illegal_guest_state(void) "HV_ILLEGAL_GUEST_STATE"
Signature · 1 line
2.49.0
#14 lore raw

PATCH v3 9/26 target/arm: Correct KVM & HVF dtb_compatible value

In reply to Philippe Mathieu-Daudé

Linux kernel knows how to parse "arm,armv8", not "arm,arm-v8".

See arch/arm64/boot/dts/foundation-v8.dts:

https://github.com/torvalds/linux/commit/90556ca1ebdd

Fixes 26861c7ce06 ("target-arm: Add minimal KVM AArch64 support")
Fixes 585df85efea ("hvf: arm: Implement -cpu host")
---
 target/arm/hvf/hvf.c | 2 +-
 target/arm/kvm.c     | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
target/arm/hvf/hvf.c +1 -1
index d4c58516e8b..bf59b17dcb9 100644
@@ -879,7 +879,7 @@ static bool hvf_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
     hv_vcpu_exit_t *exit;
     int i;
 
-    ahcf->dtb_compatible = "arm,arm-v8";
+    ahcf->dtb_compatible = "arm,armv8";
     ahcf->features = (1ULL << ARM_FEATURE_V8) |
                      (1ULL << ARM_FEATURE_NEON) |
                      (1ULL << ARM_FEATURE_AARCH64) |
target/arm/kvm.c +1 -1
index 74fda8b8090..9a1b031556a 100644
@@ -266,7 +266,7 @@ static bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
     }
 
     ahcf->target = init.target;
-    ahcf->dtb_compatible = "arm,arm-v8";
+    ahcf->dtb_compatible = "arm,armv8";
 
     err = read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64pfr0,
                          ARM64_SYS_REG(3, 0, 0, 4, 0));
Signature · 1 line
2.49.0
#15 lore raw

PATCH v3 10/26 accel/hvf: Model PhysTimer register

In reply to Philippe Mathieu-Daudé

Emulate PhysTimer dispatching to TCG, like we do with GIC registers.

---
 target/arm/hvf/hvf.c | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)
target/arm/hvf/hvf.c +8 -11
index bf59b17dcb9..5169bf6e23c 100644
@@ -187,6 +187,7 @@ void hvf_arm_init_debug(void)
 #define SYSREG_OSDLR_EL1      SYSREG(2, 0, 1, 3, 4)
 #define SYSREG_CNTPCT_EL0     SYSREG(3, 3, 14, 0, 1)
 #define SYSREG_CNTP_CTL_EL0   SYSREG(3, 3, 14, 2, 1)
+#define SYSREG_CNTP_CVAL_EL0  SYSREG(3, 3, 14, 2, 2)
 #define SYSREG_PMCR_EL0       SYSREG(3, 3, 9, 12, 0)
 #define SYSREG_PMUSERENR_EL0  SYSREG(3, 3, 9, 14, 0)
 #define SYSREG_PMCNTENSET_EL0 SYSREG(3, 3, 9, 12, 1)
@@ -198,6 +199,7 @@ void hvf_arm_init_debug(void)
 #define SYSREG_PMCEID0_EL0    SYSREG(3, 3, 9, 12, 6)
 #define SYSREG_PMCEID1_EL0    SYSREG(3, 3, 9, 12, 7)
 #define SYSREG_PMCCNTR_EL0    SYSREG(3, 3, 9, 13, 0)
+#define SYSREG_CNTP_TVAL_EL0  SYSREG(3, 3, 14, 2, 0)
 #define SYSREG_PMCCFILTR_EL0  SYSREG(3, 3, 14, 15, 7)
 
 #define SYSREG_ICC_AP0R0_EL1     SYSREG(3, 0, 12, 8, 4)
@@ -1326,16 +1328,15 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint64_t *val)
     }
 
     switch (reg) {
-    case SYSREG_CNTPCT_EL0:
-        *val = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) /
-              gt_cntfrq_period_ns(arm_cpu);
-        return 0;
     case SYSREG_OSLSR_EL1:
         *val = env->cp15.oslsr_el1;
         return 0;
     case SYSREG_OSDLR_EL1:
         /* Dummy register */
         return 0;
+    case SYSREG_CNTP_CTL_EL0:
+    case SYSREG_CNTP_TVAL_EL0:
+    case SYSREG_CNTPCT_EL0:
     case SYSREG_ICC_AP0R0_EL1:
     case SYSREG_ICC_AP0R1_EL1:
     case SYSREG_ICC_AP0R2_EL1:
@@ -1639,16 +1640,12 @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val)
     case SYSREG_OSLAR_EL1:
         env->cp15.oslsr_el1 = val & 1;
         return 0;
-    case SYSREG_CNTP_CTL_EL0:
-        /*
-         * Guests should not rely on the physical counter, but macOS emits
-         * disable writes to it. Let it do so, but ignore the requests.
-         */
-        qemu_log_mask(LOG_UNIMP, "Unsupported write to CNTP_CTL_EL0\n");
-        return 0;
     case SYSREG_OSDLR_EL1:
         /* Dummy register */
         return 0;
+    case SYSREG_CNTP_CTL_EL0:
+    case SYSREG_CNTP_CVAL_EL0:
+    case SYSREG_CNTP_TVAL_EL0:
     case SYSREG_ICC_AP0R0_EL1:
     case SYSREG_ICC_AP0R1_EL1:
     case SYSREG_ICC_AP0R2_EL1:
Signature · 1 line
2.49.0
#16 lore raw

Re: [PATCH v3 10/26] accel/hvf: Model PhysTimer register

Quoted 20 lines · Philippe Mathieu-Daudé

Emulate PhysTimer dispatching to TCG, like we do with GIC registers.

---
 target/arm/hvf/hvf.c | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)
target/arm/hvf/hvf.c +0 -6
@@ -1639,16 +1640,12 @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val)
     case SYSREG_OSLAR_EL1:
         env->cp15.oslsr_el1 = val & 1;
         return 0;
-    case SYSREG_CNTP_CTL_EL0:
-        /*
-         * Guests should not rely on the physical counter, but macOS emits
-         * disable writes to it. Let it do so, but ignore the requests.
-         */
-        qemu_log_mask(LOG_UNIMP, "Unsupported write to CNTP_CTL_EL0\n");

target/arm/hvf/hvf.c static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val) · line 1651

Quoted 10 lines
target/arm/hvf/hvf.c +3 -1
-        return 0;
     case SYSREG_OSDLR_EL1:
         /* Dummy register */
         return 0;
+    case SYSREG_CNTP_CTL_EL0:
+    case SYSREG_CNTP_CVAL_EL0:
+    case SYSREG_CNTP_TVAL_EL0:
     case SYSREG_ICC_AP0R0_EL1:
     case SYSREG_ICC_AP0R1_EL1:
     case SYSREG_ICC_AP0R2_EL1:

This adds three registers which aren't GICv3 registers to a code path which has a comment

        /* Call the TCG sysreg handler. This is only safe for GICv3 regs. */

In general the TCG sysreg implementations aren't expecting to be called for non-TCG accelerators. I would like to see some analysis of why this is OK for the timer, please. (Compare commit a2260983c65 which notes the limitations under which this works for the GIC registers and had to make some minor changes to the GIC code both noting that it might be called for non-TCG accelerators and adjusting things so that would work.)

thanks
-- PMM

#17 lore raw

PATCH v3 11/26 target/arm/hvf: Pass @target_el argument to hvf_raise_exception()

In reply to Philippe Mathieu-Daudé

In preparation of raising exceptions at EL2, add the 'target_el' argument to hvf_raise_exception().

---
 target/arm/hvf/hvf.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
target/arm/hvf/hvf.c +7 -7
index 5169bf6e23c..b932134a833 100644
@@ -1089,13 +1089,13 @@ void hvf_kick_vcpu_thread(CPUState *cpu)
 }
 
 static void hvf_raise_exception(CPUState *cpu, uint32_t excp,
-                                uint32_t syndrome)
+                                uint32_t syndrome, int target_el)
 {
     ARMCPU *arm_cpu = ARM_CPU(cpu);
     CPUARMState *env = &arm_cpu->env;
 
     cpu->exception_index = excp;
-    env->exception.target_el = 1;
+    env->exception.target_el = target_el;
     env->exception.syndrome = syndrome;
 
     arm_cpu_do_interrupt(cpu);
@@ -1454,7 +1454,7 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint64_t *val)
                                     SYSREG_CRN(reg),
                                     SYSREG_CRM(reg),
                                     SYSREG_OP2(reg));
-    hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized());
+    hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized(), 1);
     return 1;
 }
 
@@ -1760,7 +1760,7 @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val)
                                      SYSREG_CRN(reg),
                                      SYSREG_CRM(reg),
                                      SYSREG_OP2(reg));
-    hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized());
+    hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized(), 1);
     return 1;
 }
 
@@ -1963,7 +1963,7 @@ int hvf_vcpu_exec(CPUState *cpu)
         if (!hvf_find_sw_breakpoint(cpu, env->pc)) {
             /* Re-inject into the guest */
             ret = 0;
-            hvf_raise_exception(cpu, EXCP_BKPT, syn_aa64_bkpt(0));
+            hvf_raise_exception(cpu, EXCP_BKPT, syn_aa64_bkpt(0), 1);
         }
         break;
     }
@@ -2074,7 +2074,7 @@ int hvf_vcpu_exec(CPUState *cpu)
             }
         } else {
             trace_hvf_unknown_hvc(env->pc, env->xregs[0]);
-            hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized());
+            hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized(), 1);
         }
         break;
     case EC_AA64_SMC:
@@ -2089,7 +2089,7 @@ int hvf_vcpu_exec(CPUState *cpu)
             }
         } else {
             trace_hvf_unknown_smc(env->xregs[0]);
-            hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized());
+            hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized(), 1);
         }
         break;
     default:
Signature · 1 line
2.49.0
#18 lore raw

Re: [PATCH v3 11/26] target/arm/hvf: Pass @target_el argument to hvf_raise_exception()

Quoted 7 lines · Philippe Mathieu-Daudé

In preparation of raising exceptions at EL2, add the 'target_el' argument to hvf_raise_exception().

---

I'm taking this patch, but note that caution is required here:
part of the analysis that says we're OK to call into the TCG GICv3 implementation includes "we will only do this for EL0 or EL1, not EL2 or EL3".

-- PMM

#19 lore raw

PATCH v3 12/26 target/arm: Restrict system register properties to system binary

In reply to Philippe Mathieu-Daudé

Do not expose the following system-specific properties on user-mode binaries:

- psci-conduit
- cntfrq (ARM_FEATURE_GENERIC_TIMER)
- rvbar (ARM_FEATURE_V8)
- has-mpu (ARM_FEATURE_PMSA)
- pmsav7-dregion (ARM_FEATURE_PMSA)
- reset-cbar (ARM_FEATURE_CBAR)
- reset-hivecs (ARM_FEATURE_M)
- init-nsvtor (ARM_FEATURE_M)
- init-svtor (ARM_FEATURE_M_SECURITY)
- idau (ARM_FEATURE_M_SECURITY)

---
 target/arm/cpu.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
target/arm/cpu.c +6 -5
index eb0639de719..e5b70f5de81 100644
@@ -1500,6 +1500,7 @@ static void arm_cpu_initfn(Object *obj)
  * 0 means "unset, use the default value". That default might vary depending
  * on the CPU type, and is set in the realize fn.
  */
+#ifndef CONFIG_USER_ONLY
 static const Property arm_cpu_gt_cntfrq_property =
             DEFINE_PROP_UINT64("cntfrq", ARMCPU, gt_cntfrq_hz, 0);
 
@@ -1509,7 +1510,6 @@ static const Property arm_cpu_reset_cbar_property =
 static const Property arm_cpu_reset_hivecs_property =
             DEFINE_PROP_BOOL("reset-hivecs", ARMCPU, reset_hivecs, false);
 
-#ifndef CONFIG_USER_ONLY
 static const Property arm_cpu_has_el2_property =
             DEFINE_PROP_BOOL("has_el2", ARMCPU, has_el2, true);
 
@@ -1532,6 +1532,7 @@ static const Property arm_cpu_has_neon_property =
 static const Property arm_cpu_has_dsp_property =
             DEFINE_PROP_BOOL("dsp", ARMCPU, has_dsp, true);
 
+#ifndef CONFIG_USER_ONLY
 static const Property arm_cpu_has_mpu_property =
             DEFINE_PROP_BOOL("has-mpu", ARMCPU, has_mpu, true);
 
@@ -1544,6 +1545,7 @@ static const Property arm_cpu_pmsav7_dregion_property =
             DEFINE_PROP_UNSIGNED_NODEFAULT("pmsav7-dregion", ARMCPU,
                                            pmsav7_dregion,
                                            qdev_prop_uint32, uint32_t);
+#endif
 
 static bool arm_get_pmu(Object *obj, Error **errp)
 {
@@ -1731,6 +1733,7 @@ static void arm_cpu_post_init(Object *obj)
                                         "Set on/off to enable/disable aarch64 "
                                         "execution state ");
     }
+#ifndef CONFIG_USER_ONLY
     if (arm_feature(&cpu->env, ARM_FEATURE_CBAR) ||
         arm_feature(&cpu->env, ARM_FEATURE_CBAR_RO)) {
         qdev_property_add_static(DEVICE(obj), &arm_cpu_reset_cbar_property);
@@ -1746,7 +1749,6 @@ static void arm_cpu_post_init(Object *obj)
                                        OBJ_PROP_FLAG_READWRITE);
     }
 
-#ifndef CONFIG_USER_ONLY
     if (arm_feature(&cpu->env, ARM_FEATURE_EL3)) {
         /* Add the has_el3 state CPU property only if EL3 is allowed.  This will
          * prevent "has_el3" from existing on CPUs which cannot support EL3.
@@ -1818,6 +1820,7 @@ static void arm_cpu_post_init(Object *obj)
         qdev_property_add_static(DEVICE(obj), &arm_cpu_has_dsp_property);
     }
 
+#ifndef CONFIG_USER_ONLY
     if (arm_feature(&cpu->env, ARM_FEATURE_PMSA)) {
         qdev_property_add_static(DEVICE(obj), &arm_cpu_has_mpu_property);
         if (arm_feature(&cpu->env, ARM_FEATURE_V7)) {
@@ -1854,8 +1857,6 @@ static void arm_cpu_post_init(Object *obj)
                                    &cpu->psci_conduit,
                                    OBJ_PROP_FLAG_READWRITE);
 
-    qdev_property_add_static(DEVICE(obj), &arm_cpu_cfgend_property);
-
     if (arm_feature(&cpu->env, ARM_FEATURE_GENERIC_TIMER)) {
         qdev_property_add_static(DEVICE(cpu), &arm_cpu_gt_cntfrq_property);
     }
@@ -1864,7 +1865,6 @@ static void arm_cpu_post_init(Object *obj)
         kvm_arm_add_vcpu_properties(cpu);
     }
 
-#ifndef CONFIG_USER_ONLY
     if (arm_feature(&cpu->env, ARM_FEATURE_AARCH64) &&
         cpu_isar_feature(aa64_mte, cpu)) {
         object_property_add_link(obj, "tag-memory",
@@ -1882,6 +1882,7 @@ static void arm_cpu_post_init(Object *obj)
         }
     }
 #endif
+    qdev_property_add_static(DEVICE(obj), &arm_cpu_cfgend_property);
 }
 
 static void arm_cpu_finalizefn(Object *obj)
Signature · 1 line
2.49.0
#20 lore raw

Re: [PATCH v3 12/26] target/arm: Restrict system register properties to system binary

Quoted 4 lines · Philippe Mathieu-Daudé

Do not expose the following system-specific properties on user-mode binaries:

Quoted 10 lines

- psci-conduit
- cntfrq (ARM_FEATURE_GENERIC_TIMER)
- rvbar (ARM_FEATURE_V8)
- has-mpu (ARM_FEATURE_PMSA)
- pmsav7-dregion (ARM_FEATURE_PMSA)
- reset-cbar (ARM_FEATURE_CBAR)
- reset-hivecs (ARM_FEATURE_M)
- init-nsvtor (ARM_FEATURE_M)
- init-svtor (ARM_FEATURE_M_SECURITY)
- idau (ARM_FEATURE_M_SECURITY)

I guess these are user-accessible via "qemu-arm -cpu max,cntfrq= ..." syntax? Makes sense to not expose them, they won't do anything sensible.

thanks
-- PMM

#21 lore raw

Re: [PATCH v3 12/26] target/arm: Restrict system register properties to system binary

Quoted 16 lines · Peter Maydell
Quoted 4 lines · Philippe Mathieu-Daudé

Do not expose the following system-specific properties on user-mode binaries:

Quoted 10 lines

- psci-conduit
- cntfrq (ARM_FEATURE_GENERIC_TIMER)
- rvbar (ARM_FEATURE_V8)
- has-mpu (ARM_FEATURE_PMSA)
- pmsav7-dregion (ARM_FEATURE_PMSA)
- reset-cbar (ARM_FEATURE_CBAR)
- reset-hivecs (ARM_FEATURE_M)
- init-nsvtor (ARM_FEATURE_M)
- init-svtor (ARM_FEATURE_M_SECURITY)
- idau (ARM_FEATURE_M_SECURITY)

Quoted 3 lines

I guess these are user-accessible via "qemu-arm -cpu max,cntfrq= ..." syntax? Makes sense to not expose them, they won't do anything sensible.

Indeed, which could be confusing for users (set this property but no behavior change), which is why I prefer to not expose them.

#22 lore raw

PATCH v3 13/26 target/arm: Create GTimers *after* features finalized / accel realized

In reply to Philippe Mathieu-Daudé

Call generic (including accelerator) cpu_realize() handlers *before* setting @gt_cntfrq_hz default

---
 target/arm/cpu.c | 65 ++++++++++++++++++++++++------------------------
 1 file changed, 33 insertions(+), 32 deletions(-)
target/arm/cpu.c +33 -32
index e5b70f5de81..ab5fbd9b40b 100644
@@ -1985,26 +1985,6 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
         return;
     }
 
-    if (!cpu->gt_cntfrq_hz) {
-        /*
-         * 0 means "the board didn't set a value, use the default". (We also
-         * get here for the CONFIG_USER_ONLY case.)
-         * ARMv8.6 and later CPUs architecturally must use a 1GHz timer; before
-         * that it was an IMPDEF choice, and QEMU initially picked 62.5MHz,
-         * which gives a 16ns tick period.
-         *
-         * We will use the back-compat value:
-         *  - for QEMU CPU types added before we standardized on 1GHz
-         *  - for versioned machine types with a version of 9.0 or earlier
-         */
-        if (arm_feature(env, ARM_FEATURE_BACKCOMPAT_CNTFRQ) ||
-            cpu->backcompat_cntfrq) {
-            cpu->gt_cntfrq_hz = GTIMER_BACKCOMPAT_HZ;
-        } else {
-            cpu->gt_cntfrq_hz = GTIMER_DEFAULT_HZ;
-        }
-    }
-
 #ifndef CONFIG_USER_ONLY
     /* The NVIC and M-profile CPU are two halves of a single piece of
      * hardware; trying to use one without the other is a command line
@@ -2051,7 +2031,40 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
             return;
         }
     }
+#endif
 
+    cpu_exec_realizefn(cs, &local_err);
+    if (local_err != NULL) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    arm_cpu_finalize_features(cpu, &local_err);
+    if (local_err != NULL) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+#ifndef CONFIG_USER_ONLY
+    if (!cpu->gt_cntfrq_hz) {
+        /*
+         * 0 means "the board didn't set a value, use the default". (We also
+         * get here for the CONFIG_USER_ONLY case.)
+         * ARMv8.6 and later CPUs architecturally must use a 1GHz timer; before
+         * that it was an IMPDEF choice, and QEMU initially picked 62.5MHz,
+         * which gives a 16ns tick period.
+         *
+         * We will use the back-compat value:
+         *  - for QEMU CPU types added before we standardized on 1GHz
+         *  - for versioned machine types with a version of 9.0 or earlier
+         */
+        if (arm_feature(env, ARM_FEATURE_BACKCOMPAT_CNTFRQ) ||
+            cpu->backcompat_cntfrq) {
+            cpu->gt_cntfrq_hz = GTIMER_BACKCOMPAT_HZ;
+        } else {
+            cpu->gt_cntfrq_hz = GTIMER_DEFAULT_HZ;
+        }
+    }
     {
         uint64_t scale = gt_cntfrq_period_ns(cpu);
 
@@ -2072,18 +2085,6 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
     }
 #endif
 
-    cpu_exec_realizefn(cs, &local_err);
-    if (local_err != NULL) {
-        error_propagate(errp, local_err);
-        return;
-    }
-
-    arm_cpu_finalize_features(cpu, &local_err);
-    if (local_err != NULL) {
-        error_propagate(errp, local_err);
-        return;
-    }
-
 #ifdef CONFIG_USER_ONLY
     /*
      * User mode relies on IC IVAU instructions to catch modification of
Signature · 1 line
2.49.0
#23 lore raw

PATCH v3 14/26 accel: Keep reference to AccelOpsClass in AccelClass

In reply to Philippe Mathieu-Daudé

Allow dereferencing AccelOpsClass outside of accel/accel-system.c.

---
 include/qemu/accel.h       | 3 +++
 include/system/accel-ops.h | 3 ++-
 accel/accel-common.c       | 1 +
 accel/accel-system.c       | 3 ++-
 accel/tcg/tcg-accel-ops.c  | 4 +++-
 5 files changed, 11 insertions(+), 3 deletions(-)
include/qemu/accel.h +3 -0
index fbd3d897fef..9dea3145429 100644
@@ -37,6 +37,9 @@ typedef struct AccelClass {
     /*< public >*/
 
     const char *name;
+    /* Cached by accel_init_ops_interfaces() when created */
+    AccelOpsClass *ops;
+
     int (*init_machine)(MachineState *ms);
     bool (*cpu_common_realize)(CPUState *cpu, Error **errp);
     void (*cpu_common_unrealize)(CPUState *cpu);
include/system/accel-ops.h +2 -1
index 4c99d25aeff..44b37592d02 100644
@@ -10,6 +10,7 @@
 #ifndef ACCEL_OPS_H
 #define ACCEL_OPS_H
 
+#include "qemu/accel.h"
 #include "exec/vaddr.h"
 #include "qom/object.h"
 
@@ -31,7 +32,7 @@ struct AccelOpsClass {
     /*< public >*/
 
     /* initialization function called when accel is chosen */
-    void (*ops_init)(AccelOpsClass *ops);
+    void (*ops_init)(AccelClass *ac);
 
     bool (*cpus_are_resettable)(void);
     void (*cpu_reset_hold)(CPUState *cpu);
accel/accel-common.c +1 -0
index 4894b98d64a..56d88940f92 100644
@@ -10,6 +10,7 @@
 #include "qemu/osdep.h"
 #include "qemu/accel.h"
 #include "qemu/target-info.h"
+#include "system/accel-ops.h"
 #include "accel/accel-cpu.h"
 #include "accel-internal.h"
 
accel/accel-system.c +2 -1
index a0f562ae9ff..64bc991b1ce 100644
@@ -85,8 +85,9 @@ void accel_init_ops_interfaces(AccelClass *ac)
      * non-NULL create_vcpu_thread operation.
      */
     ops = ACCEL_OPS_CLASS(oc);
+    ac->ops = ops;
     if (ops->ops_init) {
-        ops->ops_init(ops);
+        ops->ops_init(ac);
     }
     cpus_register_accel(ops);
 }
accel/tcg/tcg-accel-ops.c +3 -1
index b24d6a75625..da2e22a7dff 100644
@@ -198,8 +198,10 @@ static inline void tcg_remove_all_breakpoints(CPUState *cpu)
     cpu_watchpoint_remove_all(cpu, BP_GDB);
 }
 
-static void tcg_accel_ops_init(AccelOpsClass *ops)
+static void tcg_accel_ops_init(AccelClass *ac)
 {
+    AccelOpsClass *ops = ac->ops;
+
     if (qemu_tcg_mttcg_enabled()) {
         ops->create_vcpu_thread = mttcg_start_vcpu_thread;
         ops->kick_vcpu_thread = mttcg_kick_vcpu_thread;
Signature · 1 line
2.49.0
#24 lore raw

PATCH v3 15/26 accel: Introduce AccelOpsClass::cpu_target_realize() hook

In reply to Philippe Mathieu-Daudé

Allow accelerators to set vCPU properties before its realization.

---
 include/system/accel-ops.h | 1 +
 accel/accel-common.c       | 3 +++
 2 files changed, 4 insertions(+)
include/system/accel-ops.h +1 -0
index 44b37592d02..a863fe59388 100644
@@ -35,6 +35,7 @@ struct AccelOpsClass {
     void (*ops_init)(AccelClass *ac);
 
     bool (*cpus_are_resettable)(void);
+    bool (*cpu_target_realize)(CPUState *cpu, Error **errp);
     void (*cpu_reset_hold)(CPUState *cpu);
 
     void (*create_vcpu_thread)(CPUState *cpu); /* MANDATORY NON-NULL */
accel/accel-common.c +3 -0
index 56d88940f92..55d21b63a48 100644
@@ -105,6 +105,9 @@ bool accel_cpu_common_realize(CPUState *cpu, Error **errp)
     if (acc->cpu_common_realize && !acc->cpu_common_realize(cpu, errp)) {
         return false;
     }
+    if (acc->ops->cpu_target_realize && !acc->ops->cpu_target_realize(cpu, errp)) {
+        return false;
+    }
 
     return true;
 }
Signature · 1 line
2.49.0
#25 lore raw

PATCH v3 16/26 accel/hvf: Add hvf_arch_cpu_realize() stubs

In reply to Philippe Mathieu-Daudé

Implement HVF AccelOpsClass::cpu_target_realize() hook as empty stubs. Target implementations will come separately.

---
 include/system/hvf.h      | 3 +++
 accel/hvf/hvf-accel-ops.c | 2 ++
 target/arm/hvf/hvf.c      | 5 +++++
 target/i386/hvf/hvf.c     | 5 +++++
 4 files changed, 15 insertions(+)
include/system/hvf.h +3 -0
index a9a502f0c8f..8c4409a13f1 100644
@@ -72,6 +72,9 @@ void hvf_arch_update_guest_debug(CPUState *cpu);
  * Return whether the guest supports debugging.
  */
 bool hvf_arch_supports_guest_debug(void);
+
+bool hvf_arch_cpu_realize(CPUState *cpu, Error **errp);
+
 #endif /* COMPILING_PER_TARGET */
 
 #endif
accel/hvf/hvf-accel-ops.c +2 -0
index b38977207d2..b9511103a75 100644
@@ -588,6 +588,8 @@ static void hvf_accel_ops_class_init(ObjectClass *oc, const void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
+    ops->cpu_target_realize = hvf_arch_cpu_realize;
+
     ops->create_vcpu_thread = hvf_start_vcpu_thread;
     ops->kick_vcpu_thread = hvf_kick_vcpu_thread;
 
target/arm/hvf/hvf.c +5 -0
index b932134a833..fd493f45af1 100644
@@ -1082,6 +1082,11 @@ int hvf_arch_init_vcpu(CPUState *cpu)
     return 0;
 }
 
+bool hvf_arch_cpu_realize(CPUState *cs, Error **errp)
+{
+    return true;
+}
+
 void hvf_kick_vcpu_thread(CPUState *cpu)
 {
     cpus_kick_thread(cpu);
target/i386/hvf/hvf.c +5 -0
index 99e37a33e50..28484496710 100644
@@ -367,6 +367,11 @@ int hvf_arch_init_vcpu(CPUState *cpu)
     return 0;
 }
 
+bool hvf_arch_cpu_realize(CPUState *cs, Error **errp)
+{
+    return true;
+}
+
 static void hvf_store_events(CPUState *cpu, uint32_t ins_len, uint64_t idtvec_info)
 {
     X86CPU *x86_cpu = X86_CPU(cpu);
Signature · 1 line
2.49.0
#26 lore raw

PATCH v3 17/26 target/arm/hvf: Really set Generic Timer counter frequency

In reply to Philippe Mathieu-Daudé

Setting ARMCPU::gt_cntfrq_hz in hvf_arch_init_vcpu() is not correct because the timers have already be initialized with the default frequency.

Set it earlier in the AccelOpsClass::cpu_target_realize() handler instead, and assert the value is correct when reaching hvf_arch_init_vcpu().

Fixes a1477da3dde ("hvf: Add Apple Silicon support")
---
 target/arm/hvf/hvf.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
target/arm/hvf/hvf.c +14 -1
index fd493f45af1..52199c4ff9d 100644
@@ -1004,6 +1004,13 @@ cleanup:
     return ret;
 }
 
+static uint64_t get_cntfrq_el0(void)
+{
+    uint64_t freq_hz = 0;
+    asm volatile("mrs %0, cntfrq_el0" : "=r"(freq_hz));
+    return freq_hz;
+}
+
 int hvf_arch_init_vcpu(CPUState *cpu)
 {
     ARMCPU *arm_cpu = ARM_CPU(cpu);
@@ -1015,7 +1022,9 @@ int hvf_arch_init_vcpu(CPUState *cpu)
     int i;
 
     env->aarch64 = true;
-    asm volatile("mrs %0, cntfrq_el0" : "=r"(arm_cpu->gt_cntfrq_hz));
+
+    /* system count frequency sanity check */
+    assert(arm_cpu->gt_cntfrq_hz == get_cntfrq_el0());
 
     /* Allocate enough space for our sysreg sync */
     arm_cpu->cpreg_indexes = g_renew(uint64_t, arm_cpu->cpreg_indexes,
@@ -1084,6 +1093,10 @@ int hvf_arch_init_vcpu(CPUState *cpu)
 
 bool hvf_arch_cpu_realize(CPUState *cs, Error **errp)
 {
+    ARMCPU *cpu = ARM_CPU(cs);
+
+    cpu->gt_cntfrq_hz = get_cntfrq_el0();
+
     return true;
 }
 
Signature · 1 line
2.49.0
#27 lore raw

PATCH v3 18/26 hw/arm/virt: Only require TCG || QTest to use TrustZone

In reply to Philippe Mathieu-Daudé

We only need TCG (or QTest) to use TrustZone, whether KVM or HVF are used is not relevant.

---
 hw/arm/virt.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
hw/arm/virt.c +1 -1
index 99fde5836c9..b49d8579161 100644
@@ -2203,7 +2203,7 @@ static void machvirt_init(MachineState *machine)
         exit(1);
     }
 
-    if (vms->secure && (kvm_enabled() || hvf_enabled())) {
+    if (vms->secure && !tcg_enabled() && !qtest_enabled()) {
         error_report("mach-virt: %s does not support providing "
                      "Security extensions (TrustZone) to the guest CPU",
                      current_accel_name());
Signature · 1 line
2.49.0
#28 lore raw

Re: [PATCH v3 18/26] hw/arm/virt: Only require TCG || QTest to use TrustZone

Quoted 16 lines · Philippe Mathieu-Daudé

We only need TCG (or QTest) to use TrustZone, whether KVM or HVF are used is not relevant.

---
 hw/arm/virt.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
hw/arm/virt.c +0 -0
index 99fde5836c9..b49d8579161 100644

hw/arm/virt.c static void machvirt_init(MachineState *machine) · line 2209

Quoted 8 lines
hw/arm/virt.c +1 -1
@@ -2203,7 +2203,7 @@ static void machvirt_init(MachineState *machine)
         exit(1);
     }
 
-    if (vms->secure && (kvm_enabled() || hvf_enabled())) {
+    if (vms->secure && !tcg_enabled() && !qtest_enabled()) {
         error_report("mach-virt: %s does not support providing "
                      "Security extensions (TrustZone) to the guest CPU",
                      current_accel_name());

The change is fine, but the commit message is odd. You only get to pick one accelerator. The reason for preferring "fail unless accelerator A or B" over "fail if accelerator C or D" is that if/when we add a new accelerator type E we want the default to be "fail". Then the person implementing the new accelerator can add E to the accept-list if they implement support for an EL3 guest.

For the not-yet-implemented case of a hybrid hvf+TCG accelerator, it's not clear what to do: in some cases where we check the accelerator type you'll want it to act like TCG, and sometimes like hvf.

I'll take these patches, with an updated commit message.

-- PMM

#29 lore raw

Re: [PATCH v3 18/26] hw/arm/virt: Only require TCG || QTest to use TrustZone

Quoted 34 lines · Peter Maydell
Quoted 16 lines · Philippe Mathieu-Daudé

We only need TCG (or QTest) to use TrustZone, whether KVM or HVF are used is not relevant.

---
  hw/arm/virt.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
hw/arm/virt.c +0 -0
index 99fde5836c9..b49d8579161 100644

hw/arm/virt.c static void machvirt_init(MachineState *machine) · line 2209

Quoted 8 lines
hw/arm/virt.c +1 -1
@@ -2203,7 +2203,7 @@ static void machvirt_init(MachineState *machine)
          exit(1);
      }
 
-    if (vms->secure && (kvm_enabled() || hvf_enabled())) {
+    if (vms->secure && !tcg_enabled() && !qtest_enabled()) {
          error_report("mach-virt: %s does not support providing "
                       "Security extensions (TrustZone) to the guest CPU",
                       current_accel_name());

The change is fine, but the commit message is odd. You only get to pick one accelerator. The reason for preferring "fail unless accelerator A or B" over "fail if accelerator C or D" is that if/when we add a new accelerator type E we want the default to be "fail". Then the person implementing the new accelerator can add E to the accept-list if they implement support for an EL3 guest.

Quoted 4 lines

For the not-yet-implemented case of a hybrid hvf+TCG accelerator, it's not clear what to do: in some cases where we check the accelerator type you'll want it to act like TCG, and sometimes like hvf.

In that case we want to defer to the accelerator, not block from the machine init.

BTW hybrid hw/sw accelerators *is* implemented, but not yet ready to be merged:
https://lore.kernel.org/qemu-devel/[email protected]/

Quoted 2 lines

I'll take these patches, with an updated commit message.

Thank you!

#30 lore raw

PATCH v3 19/26 hw/arm/virt: Only require TCG || QTest to use virtualization extension

In reply to Philippe Mathieu-Daudé

We can not start in EL2 / EL3 with anything but TCG (or QTest); whether KVM or HVF are used is not relevant.

---
 hw/arm/virt.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
hw/arm/virt.c +1 -1
index b49d8579161..a9099570faa 100644
@@ -2210,7 +2210,7 @@ static void machvirt_init(MachineState *machine)
         exit(1);
     }
 
-    if (vms->virt && (kvm_enabled() || hvf_enabled())) {
+    if (vms->virt && !tcg_enabled() && !qtest_enabled()) {
         error_report("mach-virt: %s does not support providing "
                      "Virtualization extensions to the guest CPU",
                      current_accel_name());
Signature · 1 line
2.49.0
#31 lore raw

PATCH v3 20/26 hw/arm/virt: Rename cpu_post_init() -> post_cpus_gic_realized()

In reply to Philippe Mathieu-Daudé

QDev uses _post_init() during instance creation, before being realized. Since here both vCPUs and GIC are REALIZED, rename as virt_post_cpus_gic_realized() for clarity.

---
 hw/arm/virt.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
hw/arm/virt.c +3 -2
index a9099570faa..da453768cce 100644
@@ -2032,7 +2032,8 @@ static void finalize_gic_version(VirtMachineState *vms)
  * virt_cpu_post_init() must be called after the CPUs have
  * been realized and the GIC has been created.
  */
-static void virt_cpu_post_init(VirtMachineState *vms, MemoryRegion *sysmem)
+static void virt_post_cpus_gic_realized(VirtMachineState *vms,
+                                        MemoryRegion *sysmem)
 {
     int max_cpus = MACHINE(vms)->smp.max_cpus;
     bool aarch64, pmu, steal_time;
@@ -2349,7 +2350,7 @@ static void machvirt_init(MachineState *machine)
 
     create_gic(vms, sysmem);
 
-    virt_cpu_post_init(vms, sysmem);
+    virt_post_cpus_gic_realized(vms, sysmem);
 
     fdt_add_pmu_nodes(vms);
 
Signature · 1 line
2.49.0
#32 lore raw

Re: [PATCH v3 20/26] hw/arm/virt: Rename cpu_post_init() -> post_cpus_gic_realized()

Quoted 16 lines · Philippe Mathieu-Daudé

QDev uses _post_init() during instance creation, before being realized. Since here both vCPUs and GIC are REALIZED, rename as virt_post_cpus_gic_realized() for clarity.

---
 hw/arm/virt.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
hw/arm/virt.c +0 -0
index a9099570faa..da453768cce 100644

hw/arm/virt.c static void finalize_gic_version(VirtMachineState *vms) · line 2032

Quoted 1 line
hw/arm/virt.c +0 -0
@@ -2032,7 +2032,8 @@ static void finalize_gic_version(VirtMachineState *vms)
  * virt_cpu_post_init() must be called after the CPUs have

You forgot to update the function name in this comment :-)

hw/arm/virt.c static void finalize_gic_version(VirtMachineState *vms) · line 2036

Quoted 5 lines
hw/arm/virt.c +2 -1
  * been realized and the GIC has been created.
  */
-static void virt_cpu_post_init(VirtMachineState *vms, MemoryRegion *sysmem)
+static void virt_post_cpus_gic_realized(VirtMachineState *vms,
+                                        MemoryRegion *sysmem)

thanks
-- PMM

#33 lore raw

PATCH v3 21/26 hw/arm/sbsa-ref: Tidy up use of RAMLIMIT_GB definition

In reply to Philippe Mathieu-Daudé

Define RAMLIMIT_BYTES using the TiB definition and display the error parsed with size_to_str():

  $ qemu-system-aarch64-unsigned -M sbsa-ref -m 9T
  qemu-system-aarch64-unsigned: sbsa-ref: cannot model more than 8 TiB of RAM
---
 hw/arm/sbsa-ref.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
hw/arm/sbsa-ref.c +5 -3
index deae5cf9861..15c1ff4b140 100644
@@ -19,6 +19,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/cutils.h"
 #include "qemu/datadir.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
@@ -53,8 +54,7 @@
 #include "target/arm/cpu-qom.h"
 #include "target/arm/gtimer.h"
 
-#define RAMLIMIT_GB 8192
-#define RAMLIMIT_BYTES (RAMLIMIT_GB * GiB)
+#define RAMLIMIT_BYTES (8 * TiB)
 
 #define NUM_IRQS        256
 #define NUM_SMMU_IRQS   4
@@ -756,7 +756,9 @@ static void sbsa_ref_init(MachineState *machine)
     sms->smp_cpus = smp_cpus;
 
     if (machine->ram_size > sbsa_ref_memmap[SBSA_MEM].size) {
-        error_report("sbsa-ref: cannot model more than %dGB RAM", RAMLIMIT_GB);
+        char *size_str = size_to_str(RAMLIMIT_BYTES);
+
+        error_report("sbsa-ref: cannot model more than %s of RAM", size_str);
         exit(1);
     }
 
Signature · 1 line
2.49.0
#34 lore raw

PATCH v3 22/26 tests/functional: Set sbsa-ref machine type in each test function

In reply to Philippe Mathieu-Daudé

fetch_firmware() is only about fetching firmware.
Set the machine type and its default console in
test_sbsaref_edk2_firmware().

---
 tests/functional/test_aarch64_sbsaref.py         | 5 +++--
 tests/functional/test_aarch64_sbsaref_alpine.py  | 3 ++-
 tests/functional/test_aarch64_sbsaref_freebsd.py | 3 ++-
 3 files changed, 7 insertions(+), 4 deletions(-)
tests/functional/test_aarch64_sbsaref.py +3 -2
index e6a55aecfac..d3402f5080a 100755
@@ -40,8 +40,6 @@ def fetch_firmware(test):
         with open(path, "ab+") as fd:
             fd.truncate(256 << 20)  # Expand volumes to 256MiB
 
-    test.set_machine('sbsa-ref')
-    test.vm.set_console()
     test.vm.add_args(
         "-drive", f"if=pflash,file={fs0_path},format=raw",
         "-drive", f"if=pflash,file={fs1_path},format=raw",
@@ -68,8 +66,11 @@ class Aarch64SbsarefMachine(QemuSystemTest):
 
     def test_sbsaref_edk2_firmware(self):
 
+        self.set_machine('sbsa-ref')
+
         fetch_firmware(self)
 
+        self.vm.set_console()
         self.vm.add_args('-cpu', 'cortex-a57')
         self.vm.launch()
 
tests/functional/test_aarch64_sbsaref_alpine.py +2 -1
index 6108ec65a54..87769993831 100755
@@ -26,8 +26,9 @@ class Aarch64SbsarefAlpine(QemuSystemTest):
     # We only boot a whole OS for the current top level CPU and GIC
     # Other test profiles should use more minimal boots
     def boot_alpine_linux(self, cpu=None):
-        fetch_firmware(self)
+        self.set_machine('sbsa-ref')
 
+        fetch_firmware(self)
         iso_path = self.ASSET_ALPINE_ISO.fetch()
 
         self.vm.set_console()
tests/functional/test_aarch64_sbsaref_freebsd.py +2 -1
index 26dfc5878bb..3cddc082f3b 100755
@@ -26,8 +26,9 @@ class Aarch64SbsarefFreeBSD(QemuSystemTest):
     # We only boot a whole OS for the current top level CPU and GIC
     # Other test profiles should use more minimal boots
     def boot_freebsd14(self, cpu=None):
-        fetch_firmware(self)
+        self.set_machine('sbsa-ref')
 
+        fetch_firmware(self)
         img_path = self.ASSET_FREEBSD_ISO.fetch()
 
         self.vm.set_console()
Signature · 1 line
2.49.0
#35 lore raw

PATCH v3 23/26 tests/functional: Restrict nested Aarch64 Xen test to TCG

In reply to Philippe Mathieu-Daudé

Currently QEMU only support accelerating EL0 and EL1, so features requiring EL2 (like virtualization) or EL3 must be emulated with TCG.

On macOS this test fails:

qemu-system-aarch64: mach-virt: HVF does not support providing Virtualization extensions to the guest CPU

---
 tests/functional/test_aarch64_xen.py | 1 +
 1 file changed, 1 insertion(+)
tests/functional/test_aarch64_xen.py +1 -0
index 339904221b0..261d796540d 100755
@@ -33,6 +33,7 @@ def launch_xen(self, xen_path):
         """
         Launch Xen with a dom0 guest kernel
         """
+        self.require_accelerator("tcg") # virtualization=on
         self.set_machine('virt')
         self.cpu = "cortex-a57"
         self.kernel_path = self.ASSET_KERNEL.fetch()
Signature · 1 line
2.49.0
#36 lore raw

PATCH v3 24/26 tests/functional: Require TCG to run Aarch64 imx8mp-evk test

In reply to Philippe Mathieu-Daudé

The imx8mp-evk machine can only run with the TCG accelerator.

---
 tests/functional/test_aarch64_imx8mp_evk.py | 1 +
 1 file changed, 1 insertion(+)
tests/functional/test_aarch64_imx8mp_evk.py +1 -0
index 638bf9e1310..99ddcdef835 100755
@@ -49,6 +49,7 @@ def setUp(self):
                      self.DTB_OFFSET, self.DTB_SIZE)
 
     def test_aarch64_imx8mp_evk_usdhc(self):
+        self.require_accelerator("tcg")
         self.set_machine('imx8mp-evk')
         self.vm.set_console(console_index=1)
         self.vm.add_args('-m', '2G',
Signature · 1 line
2.49.0
#37 lore raw

PATCH v3 25/26 tests/functional: Add hvf_available() helper

In reply to Philippe Mathieu-Daudé
---
 python/qemu/utils/__init__.py          | 2 +-
 python/qemu/utils/accel.py             | 8 ++++++++
 tests/functional/qemu_test/testcase.py | 6 ++++--
 3 files changed, 13 insertions(+), 3 deletions(-)
python/qemu/utils/__init__.py +1 -1
index 017cfdcda75..d2fe5db223c 100644
@@ -23,7 +23,7 @@
 from typing import Optional
 
 # pylint: disable=import-error
-from .accel import kvm_available, list_accel, tcg_available
+from .accel import hvf_available, kvm_available, list_accel, tcg_available
 
 
 __all__ = (
python/qemu/utils/accel.py +8 -0
index 386ff640ca8..376d1e30005 100644
@@ -82,3 +82,11 @@ def tcg_available(qemu_bin: str) -> bool:
     @param qemu_bin (str): path to the QEMU binary
     """
     return 'tcg' in list_accel(qemu_bin)
+
+def hvf_available(qemu_bin: str) -> bool:
+    """
+    Check if HVF is available.
+
+    @param qemu_bin (str): path to the QEMU binary
+    """
+    return 'hvf' in list_accel(qemu_bin)
tests/functional/qemu_test/testcase.py +4 -2
index 50c401b8c3c..2082c6fce43 100644
@@ -23,7 +23,7 @@
 import uuid
 
 from qemu.machine import QEMUMachine
-from qemu.utils import kvm_available, tcg_available
+from qemu.utils import hvf_available, kvm_available, tcg_available
 
 from .archive import archive_extract
 from .asset import Asset
@@ -317,7 +317,9 @@ def require_accelerator(self, accelerator):
         :type accelerator: str
         """
         checker = {'tcg': tcg_available,
-                   'kvm': kvm_available}.get(accelerator)
+                   'kvm': kvm_available,
+                   'hvf': hvf_available,
+                  }.get(accelerator)
         if checker is None:
             self.skipTest("Don't know how to check for the presence "
                           "of accelerator %s" % accelerator)
Signature · 1 line
2.49.0
#38 lore raw

Re: [PATCH v3 25/26] tests/functional: Add hvf_available() helper

Quoted 9 lines · Philippe Mathieu-Daudé
---
 python/qemu/utils/__init__.py          | 2 +-
 python/qemu/utils/accel.py             | 8 ++++++++
 tests/functional/qemu_test/testcase.py | 6 ++++--
 3 files changed, 13 insertions(+), 3 deletions(-)

This seems to trigger errors in the check-python-minreqs job:
https://gitlab.com/pm215/qemu/-/jobs/10529051338

Log file "stdout" content for test "01-tests/flake8.sh" (FAIL):
qemu/utils/__init__.py:26:1: F401 '.accel.hvf_available' imported but unused qemu/utils/accel.py:86:1: E302 expected 2 blank lines, found 1 Log file "stderr" content for test "01-tests/flake8.sh" (FAIL):
Log file "stdout" content for test "04-tests/isort.sh" (FAIL):
ERROR: /builds/pm215/qemu/python/qemu/utils/__init__.py Imports are incorrectly sorted and/or formatted.

I'll see if I can fix this up locally. (The missing blank line is easy; I think probably hvf_available needs to be in the __all__ = () list in __init__.py like kvm_available and tcg_available. Not sure about the incorrectly-sorted warning.)

-- PMM

#39 lore raw

Re: [PATCH v3 25/26] tests/functional: Add hvf_available() helper

Quoted 23 lines · Peter Maydell
Quoted 9 lines · Philippe Mathieu-Daudé
---
 python/qemu/utils/__init__.py          | 2 +-
 python/qemu/utils/accel.py             | 8 ++++++++
 tests/functional/qemu_test/testcase.py | 6 ++++--
 3 files changed, 13 insertions(+), 3 deletions(-)

This seems to trigger errors in the check-python-minreqs job:
https://gitlab.com/pm215/qemu/-/jobs/10529051338

Log file "stdout" content for test "01-tests/flake8.sh" (FAIL):
qemu/utils/__init__.py:26:1: F401 '.accel.hvf_available' imported but unused qemu/utils/accel.py:86:1: E302 expected 2 blank lines, found 1 Log file "stderr" content for test "01-tests/flake8.sh" (FAIL):
Log file "stdout" content for test "04-tests/isort.sh" (FAIL):
ERROR: /builds/pm215/qemu/python/qemu/utils/__init__.py Imports are incorrectly sorted and/or formatted.

Quoted 4 lines

I'll see if I can fix this up locally. (The missing blank line is easy; I think probably hvf_available needs to be in the __all__ = () list in __init__.py like kvm_available and tcg_available. Not sure about the incorrectly-sorted warning.)

Squashing this in fixed things. I guess that going from three imports to four makes the linter want you to list them one per line...

python/qemu/utils/__init__.py +7 -1
index d2fe5db223c..be5daa83634 100644
@@ -23,13 +23,19 @@
 from typing import Optional
 
 # pylint: disable=import-error
-from .accel import hvf_available, kvm_available, list_accel, tcg_available
+from .accel import (
+    hvf_available,
+    kvm_available,
+    list_accel,
+    tcg_available,
+)
 
 
 __all__ = (
     'VerboseProcessError',
     'add_visual_margin',
     'get_info_usernet_hostfwd_port',
+    'hvf_available',
     'kvm_available',
     'list_accel',
     'tcg_available',
python/qemu/utils/accel.py +1 -0
index 376d1e30005..f915b646692 100644
@@ -83,6 +83,7 @@ def tcg_available(qemu_bin: str) -> bool:
     """
     return 'tcg' in list_accel(qemu_bin)
 
+
 def hvf_available(qemu_bin: str) -> bool:
     """
     Check if HVF is available.

-- PMM

#40 lore raw

Re: [PATCH v3 25/26] tests/functional: Add hvf_available() helper

In reply to Peter Maydell
Quoted 22 lines · Peter Maydell
Quoted 9 lines · Philippe Mathieu-Daudé
---
  python/qemu/utils/__init__.py          | 2 +-
  python/qemu/utils/accel.py             | 8 ++++++++
  tests/functional/qemu_test/testcase.py | 6 ++++--
  3 files changed, 13 insertions(+), 3 deletions(-)

This seems to trigger errors in the check-python-minreqs job:
https://gitlab.com/pm215/qemu/-/jobs/10529051338

Log file "stdout" content for test "01-tests/flake8.sh" (FAIL):
qemu/utils/__init__.py:26:1: F401 '.accel.hvf_available' imported but unused qemu/utils/accel.py:86:1: E302 expected 2 blank lines, found 1 Log file "stderr" content for test "01-tests/flake8.sh" (FAIL):
Log file "stdout" content for test "04-tests/isort.sh" (FAIL):
ERROR: /builds/pm215/qemu/python/qemu/utils/__init__.py Imports are incorrectly sorted and/or formatted.

Quoted 4 lines

I'll see if I can fix this up locally. (The missing blank line is easy; I think probably hvf_available needs to be in the __all__ = () list in __init__.py like kvm_available and tcg_available. Not sure about the incorrectly-sorted warning.)

I neglected to rebuild the Python package, sorry...

Don't worry dropping this patch and the following.

#41 lore raw

PATCH v3 26/26 tests/functional: Expand Aarch64 SMMU tests to run on HVF accelerator

In reply to Philippe Mathieu-Daudé
---
 tests/functional/test_aarch64_smmu.py | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
tests/functional/test_aarch64_smmu.py +9 -3
index c65d0f28178..e0f4a922176 100755
@@ -17,7 +17,7 @@
 
 from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pattern
 from qemu_test import BUILD_DIR
-from qemu.utils import kvm_available
+from qemu.utils import kvm_available, hvf_available
 
 
 class SMMU(LinuxKernelTest):
@@ -45,11 +45,17 @@ def set_up_boot(self, path):
         self.vm.add_args('-device', 'virtio-net,netdev=n1' + self.IOMMU_ADDON)
 
     def common_vm_setup(self, kernel, initrd, disk):
-        self.require_accelerator("kvm")
+        if hvf_available(self.qemu_bin):
+            accel = "hvf"
+        elif kvm_available(self.qemu_bin):
+            accel = "kvm"
+        else:
+            self.skipTest("Neither HVF nor KVM accelerator is available")
+        self.require_accelerator(accel)
         self.require_netdev('user')
         self.set_machine("virt")
         self.vm.add_args('-m', '1G')
-        self.vm.add_args("-accel", "kvm")
+        self.vm.add_args("-accel", accel)
         self.vm.add_args("-cpu", "host")
         self.vm.add_args("-machine", "iommu=smmuv3")
         self.vm.add_args("-d", "guest_errors")
Signature · 1 line
2.49.0
Reviewed-by on v3 26/26
#42 lore raw

Re: [PATCH v3 26/26] tests/functional: Expand Aarch64 SMMU tests to run on HVF accelerator

Quoted 28 lines · Philippe Mathieu-Daudé
---
  tests/functional/test_aarch64_smmu.py | 12 +++++++++---
  1 file changed, 9 insertions(+), 3 deletions(-)
tests/functional/test_aarch64_smmu.py +6 -2
index c65d0f28178..e0f4a922176 100755
@@ -17,7 +17,7 @@
 
  from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pattern
  from qemu_test import BUILD_DIR
-from qemu.utils import kvm_available
+from qemu.utils import kvm_available, hvf_available
 
 
  class SMMU(LinuxKernelTest):
@@ -45,11 +45,17 @@ def set_up_boot(self, path):
          self.vm.add_args('-device', 'virtio-net,netdev=n1' + self.IOMMU_ADDON)
 
      def common_vm_setup(self, kernel, initrd, disk):
-        self.require_accelerator("kvm")
+        if hvf_available(self.qemu_bin):
+            accel = "hvf"
+        elif kvm_available(self.qemu_bin):
+            accel = "kvm"
+        else:

tests/functional/test_aarch64_smmu.py def set_up_boot(self, path): · line 61

Quoted 10 lines
tests/functional/test_aarch64_smmu.py +3 -1
+            self.skipTest("Neither HVF nor KVM accelerator is available")
+        self.require_accelerator(accel)
          self.require_netdev('user')
          self.set_machine("virt")
          self.vm.add_args('-m', '1G')
-        self.vm.add_args("-accel", "kvm")
+        self.vm.add_args("-accel", accel)
          self.vm.add_args("-cpu", "host")
          self.vm.add_args("-machine", "iommu=smmuv3")
          self.vm.add_args("-d", "guest_errors")
Reviewed-by on v3 26/26
#43 lore raw

Re: [PATCH v3 26/26] tests/functional: Expand Aarch64 SMMU tests to run on HVF accelerator

In reply to Philippe Mathieu-Daudé
Quoted 20 lines · Philippe Mathieu-Daudé
---
 tests/functional/test_aarch64_smmu.py | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
tests/functional/test_aarch64_smmu.py +1 -1
index c65d0f28178..e0f4a922176 100755
@@ -17,7 +17,7 @@
 
 from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pattern
 from qemu_test import BUILD_DIR
-from qemu.utils import kvm_available
+from qemu.utils import kvm_available, hvf_available
 
 
 class SMMU(LinuxKernelTest):
@@ -45,11 +45,17 @@ def set_up_boot(self, path):
         self.vm.add_args('-device', 'virtio-net,netdev=n1' + self.IOMMU_ADDON)

tests/functional/test_aarch64_smmu.py def set_up_boot(self, path): · line 54

Quoted 10 lines
tests/functional/test_aarch64_smmu.py +7 -1
 
     def common_vm_setup(self, kernel, initrd, disk):
-        self.require_accelerator("kvm")
+        if hvf_available(self.qemu_bin):
+            accel = "hvf"
+        elif kvm_available(self.qemu_bin):
+            accel = "kvm"
+        else:
+            self.skipTest("Neither HVF nor KVM accelerator is available")
+        self.require_accelerator(accel)

I think this is fine so:

However I wonder if something like:

hwaccel = self.require_hw_accelerator()

Could fetch the appropriate platform accelerator for use in -accel bellow?

Quoted 8 lines
tests/functional/test_aarch64_smmu.py +1 -1
         self.require_netdev('user')
         self.set_machine("virt")
         self.vm.add_args('-m', '1G')
-        self.vm.add_args("-accel", "kvm")
+        self.vm.add_args("-accel", accel)
         self.vm.add_args("-cpu", "host")
         self.vm.add_args("-machine", "iommu=smmuv3")
         self.vm.add_args("-d", "guest_errors")
Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
#44 lore raw

Re: [PATCH v3 26/26] tests/functional: Expand Aarch64 SMMU tests to run on HVF accelerator

Quoted 35 lines · Alex Bennée
Quoted 20 lines · Philippe Mathieu-Daudé
---
  tests/functional/test_aarch64_smmu.py | 12 +++++++++---
  1 file changed, 9 insertions(+), 3 deletions(-)
tests/functional/test_aarch64_smmu.py +1 -1
index c65d0f28178..e0f4a922176 100755
@@ -17,7 +17,7 @@
 
  from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pattern
  from qemu_test import BUILD_DIR
-from qemu.utils import kvm_available
+from qemu.utils import kvm_available, hvf_available
 
 
  class SMMU(LinuxKernelTest):
@@ -45,11 +45,17 @@ def set_up_boot(self, path):
          self.vm.add_args('-device', 'virtio-net,netdev=n1' + self.IOMMU_ADDON)

tests/functional/test_aarch64_smmu.py def set_up_boot(self, path): · line 54

Quoted 10 lines
tests/functional/test_aarch64_smmu.py +7 -1
 
      def common_vm_setup(self, kernel, initrd, disk):
-        self.require_accelerator("kvm")
+        if hvf_available(self.qemu_bin):
+            accel = "hvf"
+        elif kvm_available(self.qemu_bin):
+            accel = "kvm"
+        else:
+            self.skipTest("Neither HVF nor KVM accelerator is available")
+        self.require_accelerator(accel)

I think this is fine so:

Quoted 1 line

Thanks.

Quoted 5 lines

However I wonder if something like:

hwaccel = self.require_hw_accelerator()

Could fetch the appropriate platform accelerator for use in -accel bellow?

Then we'd need to make it per-host arch, and I'm pretty sure hw accelerators don't support the same features. So I'd expect a rather painful experience. WDYT?

#45 lore raw

Re: [PATCH v3 26/26] tests/functional: Expand Aarch64 SMMU tests to run on HVF accelerator

Quoted 35 lines · Philippe Mathieu-Daudé
Quoted 30 lines · Alex Bennée
Quoted 17 lines · Philippe Mathieu-Daudé
---
  tests/functional/test_aarch64_smmu.py | 12 +++++++++---
  1 file changed, 9 insertions(+), 3 deletions(-)
tests/functional/test_aarch64_smmu.py +0 -0
index c65d0f28178..e0f4a922176 100755
@@ -17,7 +17,7 @@
    from qemu_test import LinuxKernelTest, Asset,

exec_command_and_wait_for_pattern
from qemu_test import BUILD_DIR

tests/functional/test_aarch64_smmu.py +1 -1
-from qemu.utils import kvm_available
+from qemu.utils import kvm_available, hvf_available
      class SMMU(LinuxKernelTest):

tests/functional/test_aarch64_smmu.py def set_up_boot(self, path): · line 53

Quoted 10 lines
tests/functional/test_aarch64_smmu.py +7 -1
@@ -45,11 +45,17 @@ def set_up_boot(self, path):
          self.vm.add_args('-device', 'virtio-net,netdev=n1' + self.IOMMU_ADDON)
        def common_vm_setup(self, kernel, initrd, disk):
-        self.require_accelerator("kvm")
+        if hvf_available(self.qemu_bin):
+            accel = "hvf"
+        elif kvm_available(self.qemu_bin):
+            accel = "kvm"
+        else:
+            self.skipTest("Neither HVF nor KVM accelerator is available")
+        self.require_accelerator(accel)

I think this is fine so:

Quoted 1 line

Thanks.

Quoted 8 lines
Quoted 4 lines

However I wonder if something like:
hwaccel = self.require_hw_accelerator()
Could fetch the appropriate platform accelerator for use in -accel bellow?

Then we'd need to make it per-host arch, and I'm pretty sure hw accelerators don't support the same features. So I'd expect a rather painful experience. WDYT?

Aren't the features a function of the machine type rather than the host? Shouldn't an -M virt machine look the same on TCG, KVM and HVF regardless of the underlying accelerator?

I guess there are cases like split-irqchip which affect the implementation but hopefully not the guest view of things.

Do you have an example?

Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
#46 lore raw
In reply to Philippe Mathieu-Daudé

Ping? (series fully reviewed)

Quoted 27 lines · Philippe Mathieu-Daudé

Philippe Mathieu-Daudé (26):
target/arm: Remove arm_handle_psci_call() stub
target/arm: Reduce arm_cpu_post_init() declaration scope target/arm: Unify gen_exception_internal()
target/arm/hvf: Simplify GIC hvf_arch_init_vcpu() target/arm/hvf: Directly re-lock BQL after hv_vcpu_run() target/arm/hvf: Trace hv_vcpu_run() failures
accel/hvf: Trace VM memory mapping
target/arm/hvf: Log $pc in hvf_unknown_hvc() trace event target/arm: Correct KVM & HVF dtb_compatible value accel/hvf: Model PhysTimer register
target/arm/hvf: Pass @target_el argument to hvf_raise_exception() target/arm: Restrict system register properties to system binary target/arm: Create GTimers *after* features finalized / accel realized
accel: Keep reference to AccelOpsClass in AccelClass
accel: Introduce AccelOpsClass::cpu_target_realize() hook accel/hvf: Add hvf_arch_cpu_realize() stubs target/arm/hvf: Really set Generic Timer counter frequency

   hw/arm/virt: Only require TCG || QTest to use TrustZone
   hw/arm/virt: Only require TCG || QTest to use virtualization extension

hw/arm/virt: Rename cpu_post_init() -> post_cpus_gic_realized() hw/arm/sbsa-ref: Tidy up use of RAMLIMIT_GB definition tests/functional: Set sbsa-ref machine type in each test function tests/functional: Restrict nested Aarch64 Xen test to TCG tests/functional: Require TCG to run Aarch64 imx8mp-evk test tests/functional: Add hvf_available() helper tests/functional: Expand Aarch64 SMMU tests to run on HVF accelerator

#47 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 20 lines · Philippe Mathieu-Daudé

Only the last patch is missing review (#26)

Since v2:
- Addressed thuth review comments

Since v1:
- Addressed rth's review comments

Omnibus series of ARM-related patches (noticed during the "split accel" PoC work).

- Usual prototypes cleanups
- Check TCG for EL2/EL3 features (and not !KVM or !HVF)
- Improve HVF debugging
- Correct HVF 'dtb_compatible' value for Linux
- Fix HVF GTimer frequency (My M1 hardware has 24 MHz) (this implies accel/ rework w.r.t. QDev vCPU REALIZE)
- Expand functional tests w.r.t. HVF

Quoted 3 lines

Regards,

Phil.

Hi; I've applied these to target-arm.next:

Quoted 13 lines

Philippe Mathieu-Daudé (26):
target/arm: Remove arm_handle_psci_call() stub target/arm: Reduce arm_cpu_post_init() declaration scope target/arm: Unify gen_exception_internal() target/arm/hvf: Directly re-lock BQL after hv_vcpu_run() target/arm/hvf: Trace hv_vcpu_run() failures accel/hvf: Trace VM memory mapping
target/arm/hvf: Log $pc in hvf_unknown_hvc() trace event target/arm: Correct KVM & HVF dtb_compatible value target/arm/hvf: Pass @target_el argument to hvf_raise_exception() target/arm: Restrict system register properties to system binary

  hw/arm/virt: Only require TCG || QTest to use TrustZone
  hw/arm/virt: Only require TCG || QTest to use virtualization extension
Quoted 7 lines

hw/arm/virt: Rename cpu_post_init() -> post_cpus_gic_realized() hw/arm/sbsa-ref: Tidy up use of RAMLIMIT_GB definition tests/functional: Set sbsa-ref machine type in each test function tests/functional: Restrict nested Aarch64 Xen test to TCG tests/functional: Require TCG to run Aarch64 imx8mp-evk test tests/functional: Add hvf_available() helper tests/functional: Expand Aarch64 SMMU tests to run on HVF accelerator

Where I haven't picked up a patch it doesn't mean I'm rejecting it, just that I don't have time to think through the more complicated ones this week, and I wanted to at least take the easy patches to reduce the size of your patchset.

-- PMM

#48 lore raw
Quoted 23 lines · Peter Maydell

Hi; I've applied these to target-arm.next:

Quoted 13 lines

Philippe Mathieu-Daudé (26):
target/arm: Remove arm_handle_psci_call() stub target/arm: Reduce arm_cpu_post_init() declaration scope target/arm: Unify gen_exception_internal() target/arm/hvf: Directly re-lock BQL after hv_vcpu_run() target/arm/hvf: Trace hv_vcpu_run() failures accel/hvf: Trace VM memory mapping
target/arm/hvf: Log $pc in hvf_unknown_hvc() trace event target/arm: Correct KVM & HVF dtb_compatible value target/arm/hvf: Pass @target_el argument to hvf_raise_exception() target/arm: Restrict system register properties to system binary

   hw/arm/virt: Only require TCG || QTest to use TrustZone
   hw/arm/virt: Only require TCG || QTest to use virtualization extension
Quoted 7 lines

hw/arm/virt: Rename cpu_post_init() -> post_cpus_gic_realized() hw/arm/sbsa-ref: Tidy up use of RAMLIMIT_GB definition tests/functional: Set sbsa-ref machine type in each test function tests/functional: Restrict nested Aarch64 Xen test to TCG tests/functional: Require TCG to run Aarch64 imx8mp-evk test tests/functional: Add hvf_available() helper tests/functional: Expand Aarch64 SMMU tests to run on HVF accelerator

Quoted 4 lines

Where I haven't picked up a patch it doesn't mean I'm rejecting it, just that I don't have time to think through the more complicated ones this week, and I wanted to at least take the easy patches to reduce the size of your patchset.

Thank you, appreciated!