Skip to contentSkip to conversation

accel/split/arm: Run EL2 using TCG and EL1/EL0 in hardware with HVF

RFC PATCH In review Philippe Mathieu-Daudé 2 participants 83 messages Jun 20, 2025 – Sep 2, 2025 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

cover accel/split/arm: Run EL2 using TCG and EL1/EL0 in hardware with HVF

Hi,

This RFC is a proof-of-concept we can have QEMU run both software emulator (TCG) and hardware accelerator (here HVF). Unfortunately I'm exhausted so I'll let Alex explain what this is about, otherwise I'll post the real cover letter next Monday after getting some rest.

Full work available in the split-accel-rfc-v5 tag:
https://gitlab.com/philmd/qemu/-/commits/split-accel-rfc-v5

Regards,

Phil.

Julian Armistead (1):
accel/split: Minimal stubs for split accelerator

Philippe Mathieu-Daudé (41):
accel/split: Define SplitAccelState
accel/split: Implement cpus_are_resettable() accel/split: Implement accel_init_machine() accel/split: Expose 'hw' and 'sw' properties accel/split: Empty setup_post()
accel/split: Implement supports_guest_debug() accel/split: Implement gdbstub_supported_sstep_flags() accel/split: Add cpu_thread_routine() stub
accel/split: Define and allocate AccelCPUState accel/split: Register MTTCG
accel/split: Have thread routine ready to dispatch over HW/SW accel/split: Implement cpu_reset_hold()
accel/split: Implement synchronize_post_init() accel/split: Implement synchronize_pre_resume() accel/split: Implement synchronize_state()
accel/split: Implement synchronize_post_reset() accel/split: Implement synchronize_pre_loadvm() accel/split: Implement kick_vcpu_thread()
accel/split: Implement cpu_common_realize() accel/split: Set use_hw in cpu_thread_routine() and switch over accel/split: Add few trace events in cpu_thread_routine handler target/arm: Implement SysemuCPUOps::can_accelerate() handler accel/split: Implement handle_interrupt()
accel/split: Empty ops_init()
accel/split: Empty set/get_virtual_clock()
accel/split: Empty get_elapsed_ticks()
accel/split: Empty cpu_thread_is_idle()
accel/split: Kludge qemu_tcg_mttcg_enabled() accel/split: Implement remove_all_breakpoints() accel/split: Implement remove_breakpoint()
accel/split: Implement insert_breakpoint()
accel/split: Implement update_guest_debug() accel/split: Implement get_[vcpu]_stats()
target/arm: Emulate EL2 under TCG
target/arm: Have ERET switch to hw accel for EL0/EL1 accel/hvf: Emulate HVC at EL2
accel/tcg: Introduce TCGCPUOps::rebuild_tb_hflags handler target/arm: Implement TCGCPUOps::rebuild_tb_hflags() accel/split: Call TCGCPUOps::rebuild_tb_hflags() tests/functional: Add split_available() helper tests/functional: Test Aarch64 virt machine with split-accelerator

 meson.build                                 |   1 +
 accel/split/split-accel.h                   |  54 +++
 accel/split/trace.h                         |   2 +
 accel/tcg/internal-common.h                 |   1 +
 include/accel/tcg/cpu-ops.h                 |   2 +
 include/exec/cpu-common.h                   |   1 +
 include/system/accel-ops.h                  |   8 +
 target/arm/cpu.h                            |   2 +
 accel/split/split-accel-ops.c               | 381 ++++++++++++++++++++
 accel/split/split-all.c                     | 196 ++++++++++
 accel/tcg/cpu-exec.c                        |   9 +
 accel/tcg/tcg-accel-ops.c                   |   1 +
 accel/tcg/tcg-all.c                         |   1 +
 system/vl.c                                 |   4 +
 target/arm/cpu.c                            |   9 +
 target/arm/cpu64.c                          |   7 +-
 target/arm/hvf/hvf.c                        |   6 +
 target/arm/tcg/cpu-v7m.c                    |   1 +
 target/arm/tcg/helper-a64.c                 |   9 +
 target/arm/tcg/hflags.c                     |   5 +
 accel/Kconfig                               |   6 +
 accel/meson.build                           |   1 +
 accel/split/meson.build                     |   9 +
 accel/split/trace-events                    |   9 +
 python/qemu/utils/__init__.py               |   3 +-
 python/qemu/utils/accel.py                  |  10 +
 tests/functional/meson.build                |   1 +
 tests/functional/qemu_test/testcase.py      |   4 +-
 tests/functional/test_aarch64_virt_split.py |  69 ++++
 29 files changed, 808 insertions(+), 4 deletions(-)
 create mode 100644 accel/split/split-accel.h
 create mode 100644 accel/split/trace.h
 create mode 100644 accel/split/split-accel-ops.c
 create mode 100644 accel/split/split-all.c
 create mode 100644 accel/split/meson.build
 create mode 100644 accel/split/trace-events
 create mode 100644 tests/functional/test_aarch64_virt_split.py
Signature · 1 line
2.49.0
#1 lore raw

RFC PATCH 1/42 accel/split: Minimal stubs for split accelerator

From: Julian Armistead <[email protected]>

---
 accel/split/split-accel-ops.c | 161 ++++++++++++++++++++++++++++++++++
 accel/split/split-all.c       |  77 ++++++++++++++++
 system/vl.c                   |   4 +
 accel/Kconfig                 |   6 ++
 accel/meson.build             |   1 +
 accel/split/meson.build       |   9 ++
 6 files changed, 258 insertions(+)
 create mode 100644 accel/split/split-accel-ops.c
 create mode 100644 accel/split/split-all.c
 create mode 100644 accel/split/meson.build
accel/split/split-accel-ops.c new file +161 -0
new file mode 100644
index 00000000000..e5c1d51d426
@@ -0,0 +1,161 @@
+/*
+ * QEMU "split" accelerator (HW + SW) ops
+ *
+ * Copyright (c) 2025 Linaro Ltd.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "system/accel-ops.h"
+
+static void *split_cpu_thread_routine(void *arg)
+{
+    g_assert_not_reached();
+}
+
+static void split_ops_init(AccelClass *ac)
+{
+    g_assert_not_reached();
+}
+
+static bool split_cpu_common_realize(CPUState *cpu, Error **errp)
+{
+    g_assert_not_reached();
+}
+
+static void split_cpu_common_unrealize(CPUState *cpu)
+{
+    g_assert_not_reached();
+}
+
+static void split_cpu_reset_hold(CPUState *cpu)
+{
+    g_assert_not_reached();
+}
+
+static void split_kick_vcpu_thread(CPUState *cpu)
+{
+    g_assert_not_reached();
+}
+
+static bool split_cpu_thread_is_idle(CPUState *cpu)
+{
+    g_assert_not_reached();
+}
+
+static void split_synchronize_post_reset(CPUState *cpu)
+{
+    g_assert_not_reached();
+}
+
+static void split_synchronize_post_init(CPUState *cpu)
+{
+    g_assert_not_reached();
+}
+
+static void split_synchronize_state(CPUState *cpu)
+{
+    g_assert_not_reached();
+}
+
+static void split_synchronize_pre_loadvm(CPUState *cpu)
+{
+    g_assert_not_reached();
+}
+
+static void split_synchronize_pre_resume(bool step_pending)
+{
+    g_assert_not_reached();
+}
+
+static void split_handle_interrupt(CPUState *cpu, int mask)
+{
+    g_assert_not_reached();
+}
+
+static int64_t split_get_virtual_clock(void)
+{
+    g_assert_not_reached();
+}
+
+static void split_set_virtual_clock(int64_t time)
+{
+    g_assert_not_reached();
+}
+
+static int64_t split_get_elapsed_ticks(void)
+{
+    g_assert_not_reached();
+}
+
+static int split_update_guest_debug(CPUState *cpu)
+{
+    g_assert_not_reached();
+}
+
+static int split_insert_breakpoint(CPUState *cpu, int type,
+                                   vaddr addr, vaddr len)
+{
+    g_assert_not_reached();
+}
+
+static int split_remove_breakpoint(CPUState *cpu, int type,
+                                   vaddr addr, vaddr len)
+{
+    g_assert_not_reached();
+}
+
+static void split_remove_all_breakpoints(CPUState *cpu)
+{
+    g_assert_not_reached();
+}
+
+static void split_get_vcpu_stats(CPUState *cpu, GString *buf)
+{
+    g_assert_not_reached();
+}
+
+static void split_accel_ops_class_init(ObjectClass *oc, const void *data)
+{
+    AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
+
+    ops->ops_init = split_ops_init;
+    ops->cpu_common_realize = split_cpu_common_realize;
+    ops->cpu_common_unrealize = split_cpu_common_unrealize;
+    ops->cpu_reset_hold = split_cpu_reset_hold;
+    ops->cpu_thread_routine = split_cpu_thread_routine;
+    ops->kick_vcpu_thread = split_kick_vcpu_thread;
+    ops->cpu_thread_is_idle = split_cpu_thread_is_idle;
+
+    ops->synchronize_post_reset = split_synchronize_post_reset;
+    ops->synchronize_post_init = split_synchronize_post_init;
+    ops->synchronize_state = split_synchronize_state;
+    ops->synchronize_pre_loadvm = split_synchronize_pre_loadvm;
+    ops->synchronize_pre_resume = split_synchronize_pre_resume;
+
+    ops->handle_interrupt = split_handle_interrupt;
+    ops->get_vcpu_stats = split_get_vcpu_stats;
+
+    ops->get_virtual_clock = split_get_virtual_clock;
+    ops->set_virtual_clock = split_set_virtual_clock;
+    ops->get_elapsed_ticks = split_get_elapsed_ticks;
+
+    ops->update_guest_debug = split_update_guest_debug;
+    ops->insert_breakpoint = split_insert_breakpoint;
+    ops->remove_breakpoint = split_remove_breakpoint;
+    ops->remove_all_breakpoints = split_remove_all_breakpoints;
+}
+
+static const TypeInfo split_accel_ops_type = {
+    .name = ACCEL_OPS_NAME("split"),
+    .parent = TYPE_ACCEL_OPS,
+    .class_init = split_accel_ops_class_init,
+    .abstract = true,
+};
+
+static void split_accel_ops_register_types(void)
+{
+    type_register_static(&split_accel_ops_type);
+}
+type_init(split_accel_ops_register_types);
accel/split/split-all.c new file +77 -0

77 lines not on this page · raw message

system/vl.c +4 -0
index 3b7057e6c66..c7111d743ab 100644
@@ -2433,6 +2433,7 @@ static void configure_accelerators(const char *progname)
             bool have_tcg = accel_find("tcg");
             bool have_kvm = accel_find("kvm");
             bool have_hvf = accel_find("hvf");
+            bool have_split = accel_find("split");
 
             if (have_tcg && have_kvm) {
                 if (g_str_has_suffix(progname, "kvm")) {
@@ -2447,6 +2448,9 @@ static void configure_accelerators(const char *progname)
                 accelerators = "tcg";
             } else if (have_hvf) {
                 accelerators = "hvf";
+            } else if (have_split) {
+                assert(have_tcg);
+                accelerators = "split";
             } else {
                 error_report("No accelerator selected and"
                              " no default accelerator available");
accel/Kconfig +6 -0
index 4263cab7227..122ca9945ad 100644
@@ -10,6 +10,12 @@ config HVF
 config TCG
     bool
 
+config SPLIT
+    bool
+    default y
+    depends on TCG
+    depends on HVF
+
 config KVM
     bool
 
accel/meson.build +1 -0
index 52909314bfa..b6d4740725f 100644
@@ -9,6 +9,7 @@ if have_system
   subdir('qtest')
   subdir('kvm')
   subdir('xen')
+  subdir('split')
   subdir('stubs')
 endif
 
accel/split/meson.build new file +9 -0
new file mode 100644
index 00000000000..fbc68593180
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+split_ss = ss.source_set()
+split_ss.add(files(
+  'split-all.c',
+  'split-accel-ops.c',
+))
+
+specific_ss.add_all(when: 'CONFIG_SPLIT', if_true: split_ss)
Signature · 1 line
2.49.0
Reviewed-by on v1 1/42
#2 lore raw

Re: [RFC PATCH RESEND 01/42] accel/split: Minimal stubs for split accelerator

Quoted 12 lines · Philippe Mathieu-Daudé

From: Julian Armistead<[email protected]>

---
  accel/split/split-accel-ops.c | 161 ++++++++++++++++++++++++++++++++++
  accel/split/split-all.c       |  77 ++++++++++++++++
  system/vl.c                   |   4 +
  accel/Kconfig                 |   6 ++
  accel/meson.build             |   1 +
  accel/split/meson.build       |   9 ++
  6 files changed, 258 insertions(+)
Quoted 3 lines

create mode 100644 accel/split/split-accel-ops.c create mode 100644 accel/split/split-all.c create mode 100644 accel/split/meson.build

r~

#3 lore raw

RFC PATCH 2/42 accel/split: Define SplitAccelState

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel.h | 38 ++++++++++++++++++++++++++++++++++++++
 accel/split/split-all.c   |  5 ++++-
 2 files changed, 42 insertions(+), 1 deletion(-)
 create mode 100644 accel/split/split-accel.h
accel/split/split-accel.h new file +38 -0
new file mode 100644
index 00000000000..cc825a92a6d
@@ -0,0 +1,38 @@
+/*
+ * QEMU "split" accelerator (HW + SW) state
+ *
+ * Copyright (c) 2025 Linaro Ltd.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef SPLIT_ACCEL_H
+#define SPLIT_ACCEL_H
+
+#include "qemu/accel.h"
+#include "qom/object.h"
+
+#define TYPE_SPLIT_ACCEL ACCEL_CLASS_NAME("split")
+
+typedef struct SplitAccelClass SplitAccelClass;
+DECLARE_CLASS_CHECKERS(SplitAccelClass, SPLIT_ACCEL, TYPE_SPLIT_ACCEL)
+
+typedef struct SplitAccelState SplitAccelState;
+DECLARE_INSTANCE_CHECKER(SplitAccelState, SPLIT_ACCEL,
+                         TYPE_SPLIT_ACCEL)
+
+struct SplitAccelClass {
+    AccelClass parent_obj;
+
+    AccelClass *hw;
+    AccelClass *sw;
+};
+
+struct SplitAccelState {
+    AccelState parent_obj;
+
+    AccelState *hw;
+    AccelState *sw;
+};
+
+#endif /* SPLIT_ACCEL_H */
accel/split/split-all.c +4 -1
index 7e308f3c1e7..6bc95c7a7c8 100644
@@ -8,6 +8,7 @@
 
 #include "qemu/osdep.h"
 #include "qemu/accel.h"
+#include "split-accel.h"
 
 bool split_allowed;
 
@@ -64,8 +65,10 @@ static void split_accel_class_init(ObjectClass *oc, const void *data)
 }
 
 static const TypeInfo split_accel_type = {
-    .name = ACCEL_CLASS_NAME("split"),
+    .name = TYPE_SPLIT_ACCEL,
     .parent = TYPE_ACCEL,
+    .instance_size = sizeof(SplitAccelState),
+    .class_size = sizeof(SplitAccelClass),
     .class_init = split_accel_class_init,
 };
 
Signature · 1 line
2.49.0
#5 lore raw

RFC PATCH 3/42 accel/split: Implement cpus_are_resettable()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-all.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
accel/split/split-all.c +12 -1
index 6bc95c7a7c8..c86d0e8583a 100644
@@ -30,7 +30,18 @@ static bool split_has_memory(MachineState *ms, AddressSpace *as,
 
 static bool split_cpus_are_resettable(AccelState *as)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    bool rv = false;
+
+    if (hwc->cpus_are_resettable) {
+        rv |= hwc->cpus_are_resettable(sas->hw);
+    }
+    if (swc->cpus_are_resettable) {
+        rv |= swc->cpus_are_resettable(sas->sw);
+    }
+    return rv;
 }
 
 static bool split_supports_guest_debug(AccelState *as)
Signature · 1 line
2.49.0
Reviewed-by on v1 3/42
#6 lore raw

Re: [RFC PATCH RESEND 03/42] accel/split: Implement cpus_are_resettable()

Quoted 19 lines · Philippe Mathieu-Daudé
---
  accel/split/split-all.c | 13 ++++++++++++-
  1 file changed, 12 insertions(+), 1 deletion(-)
accel/split/split-all.c +5 -1
index 6bc95c7a7c8..c86d0e8583a 100644
@@ -30,7 +30,18 @@ static bool split_has_memory(MachineState *ms, AddressSpace *as,
 
  static bool split_cpus_are_resettable(AccelState *as)
  {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    bool rv = false;
+

accel/split/split-all.c static bool split_has_memory(MachineState *ms, AddressSpace *as, · line 47

Quoted 10 lines
accel/split/split-all.c +7 -0
+    if (hwc->cpus_are_resettable) {
+        rv |= hwc->cpus_are_resettable(sas->hw);
+    }
+    if (swc->cpus_are_resettable) {
+        rv |= swc->cpus_are_resettable(sas->sw);
+    }
+    return rv;
  }
 
  static bool split_supports_guest_debug(AccelState *as)

r~

#7 lore raw

RFC PATCH 4/42 accel/split: Implement accel_init_machine()

In reply to Philippe Mathieu-Daudé

Force to TCG + HVF for now.

---
 accel/split/split-accel.h |  3 +++
 accel/split/split-all.c   | 30 +++++++++++++++++++++++++++++-
 2 files changed, 32 insertions(+), 1 deletion(-)
accel/split/split-accel.h +3 -0
index cc825a92a6d..19fb530f207 100644
@@ -33,6 +33,9 @@ struct SplitAccelState {
 
     AccelState *hw;
     AccelState *sw;
+
+    char *hw_name;
+    char *sw_name;
 };
 
 #endif /* SPLIT_ACCEL_H */
accel/split/split-all.c +29 -1
index c86d0e8583a..7cbe32ea768 100644
@@ -8,13 +8,32 @@
 
 #include "qemu/osdep.h"
 #include "qemu/accel.h"
+#include "hw/boards.h"
+#include "accel/accel-internal.h"
 #include "split-accel.h"
 
 bool split_allowed;
 
 static int split_accel_init_machine(MachineState *ms, AccelState *as)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    AccelClass *hwc;
+    AccelClass *swc;
+
+    swc = accel_find(sas->sw_name);
+    accel_init_ops_interfaces(swc);
+    hwc = accel_find(sas->hw_name);
+    accel_init_ops_interfaces(hwc);
+
+    sas->sw = ACCEL(object_new_with_class(OBJECT_CLASS(swc)));
+    sas->hw = ACCEL(object_new_with_class(OBJECT_CLASS(hwc)));
+
+    accel_init_machine(sas->sw, ms);
+    accel_init_machine(sas->hw, ms);
+
+    ms->accelerator = as;
+
+    return 0;
 }
 
 static void split_setup_post(MachineState *ms, AccelState *accel)
@@ -59,6 +78,14 @@ static void split_get_stats(AccelState *as, GString *buf)
     g_assert_not_reached();
 }
 
+static void split_accel_instance_init(Object *obj)
+{
+    SplitAccelState *sas = SPLIT_ACCEL(obj);
+
+    sas->sw_name = g_strdup("tcg");
+    sas->hw_name = g_strdup("hvf");
+}
+
 static void split_accel_class_init(ObjectClass *oc, const void *data)
 {
     AccelClass *ac = ACCEL_CLASS(oc);
@@ -79,6 +106,7 @@ static const TypeInfo split_accel_type = {
     .name = TYPE_SPLIT_ACCEL,
     .parent = TYPE_ACCEL,
     .instance_size = sizeof(SplitAccelState),
+    .instance_init  = split_accel_instance_init,
     .class_size = sizeof(SplitAccelClass),
     .class_init = split_accel_class_init,
 };
Signature · 1 line
2.49.0
#9 lore raw

Re: [RFC PATCH RESEND 04/42] accel/split: Implement accel_init_machine()

In reply to Philippe Mathieu-Daudé

accel/split/split-all.c static void split_get_stats(AccelState *as, GString *buf) · line 86

Quoted 6 lines · Philippe Mathieu-Daudé
accel/split/split-all.c +6 -0
+static void split_accel_instance_init(Object *obj)
+{
+    SplitAccelState *sas = SPLIT_ACCEL(obj);
+
+    sas->sw_name = g_strdup("tcg");
+    sas->hw_name = g_strdup("hvf");

There's no real need to strdup.

r~

#10 lore raw

RFC PATCH 5/42 accel/split: Expose 'hw' and 'sw' properties

In reply to Philippe Mathieu-Daudé

In preparation of other accelerator (or potential emulator), expose the "hw" and "sw" keys. Only HVF and TCG allowed so far.

---
 accel/split/split-all.c | 46 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)
accel/split/split-all.c +46 -0
index 7cbe32ea768..28f626d0ff4 100644
@@ -8,6 +8,7 @@
 
 #include "qemu/osdep.h"
 #include "qemu/accel.h"
+#include "qapi/error.h"
 #include "hw/boards.h"
 #include "accel/accel-internal.h"
 #include "split-accel.h"
@@ -78,6 +79,42 @@ static void split_get_stats(AccelState *as, GString *buf)
     g_assert_not_reached();
 }
 
+static char *split_get_hw(Object *obj, Error **errp)
+{
+    SplitAccelState *s = SPLIT_ACCEL(obj);
+
+    return g_strdup(s->hw_name);
+}
+
+static void split_set_hw(Object *obj, const char *value, Error **errp)
+{
+    SplitAccelState *s = SPLIT_ACCEL(obj);
+
+    if (strcmp(value, "hvf") == 0) {
+        s->hw_name = g_strdup(value);
+    } else {
+        error_setg(errp, "'%s' accelerator no supported", value);
+    }
+}
+
+static char *split_get_sw(Object *obj, Error **errp)
+{
+    SplitAccelState *s = SPLIT_ACCEL(obj);
+
+    return g_strdup(s->sw_name);
+}
+
+static void split_set_sw(Object *obj, const char *value, Error **errp)
+{
+    SplitAccelState *s = SPLIT_ACCEL(obj);
+
+    if (strcmp(value, "tcg") == 0) {
+        s->hw_name = g_strdup(value);
+    } else {
+        error_setg(errp, "'%s' emulator no supported", value);
+    }
+}
+
 static void split_accel_instance_init(Object *obj)
 {
     SplitAccelState *sas = SPLIT_ACCEL(obj);
@@ -100,6 +137,15 @@ static void split_accel_class_init(ObjectClass *oc, const void *data)
     ac->get_stats = split_get_stats;
     ac->allowed = &split_allowed;
     ac->compat_props = NULL;
+
+    object_class_property_add_str(oc, "hw",
+                                  split_get_hw,
+                                  split_set_hw);
+    object_class_property_set_description(oc, "hw", "Hardware accelerator");
+    object_class_property_add_str(oc, "sw",
+                                  split_get_sw,
+                                  split_set_sw);
+    object_class_property_set_description(oc, "sw", "Software emulator");
 }
 
 static const TypeInfo split_accel_type = {
Signature · 1 line
2.49.0
#11 lore raw

Re: [RFC PATCH RESEND 05/42] accel/split: Expose 'hw' and 'sw' properties

Quoted 27 lines · Philippe Mathieu-Daudé

In preparation of other accelerator (or potential emulator), expose the "hw" and "sw" keys. Only HVF and TCG allowed so far.

---
  accel/split/split-all.c | 46 +++++++++++++++++++++++++++++++++++++++++
  1 file changed, 46 insertions(+)
accel/split/split-all.c +4 -0
index 7cbe32ea768..28f626d0ff4 100644
@@ -8,6 +8,7 @@
 
  #include "qemu/osdep.h"
  #include "qemu/accel.h"
+#include "qapi/error.h"
  #include "hw/boards.h"
  #include "accel/accel-internal.h"
  #include "split-accel.h"
@@ -78,6 +79,42 @@ static void split_get_stats(AccelState *as, GString *buf)
      g_assert_not_reached();
  }
 
+static char *split_get_hw(Object *obj, Error **errp)
+{
+    SplitAccelState *s = SPLIT_ACCEL(obj);

accel/split/split-all.c static void split_get_stats(AccelState *as, GString *buf) · line 94

Quoted 10 lines
accel/split/split-all.c +10 -0
+
+    return g_strdup(s->hw_name);
+}
+
+static void split_set_hw(Object *obj, const char *value, Error **errp)
+{
+    SplitAccelState *s = SPLIT_ACCEL(obj);
+
+    if (strcmp(value, "hvf") == 0) {
+        s->hw_name = g_strdup(value);

Since you've compared, you could assign the literal and not strdup. I.e.

    if (strcmp(value, "hvf") == 0) {
        s->hw_name = "hvf";

accel/split/split-all.c static void split_get_stats(AccelState *as, GString *buf) · line 96

Quoted 2 lines
accel/split/split-all.c +2 -0
+    } else {
+        error_setg(errp, "'%s' accelerator no supported", value);

not

Quoted 7 lines
accel/split/split-all.c +7 -0
+static char *split_get_sw(Object *obj, Error **errp)
+{
+    SplitAccelState *s = SPLIT_ACCEL(obj);
+
+    return g_strdup(s->sw_name);
+}
+

accel/split/split-all.c static void split_get_stats(AccelState *as, GString *buf) · line 116

Quoted 10 lines
accel/split/split-all.c +10 -0
+static void split_set_sw(Object *obj, const char *value, Error **errp)
+{
+    SplitAccelState *s = SPLIT_ACCEL(obj);
+
+    if (strcmp(value, "tcg") == 0) {
+        s->hw_name = g_strdup(value);
+    } else {
+        error_setg(errp, "'%s' emulator no supported", value);
+    }
+}

Maybe just skip this until such time as there really is a second sw emulator.

r~

#12 lore raw

RFC PATCH 6/42 accel/split: Empty setup_post()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-all.c | 6 ------
 1 file changed, 6 deletions(-)
accel/split/split-all.c +0 -6
index 28f626d0ff4..8b9f8ff77a4 100644
@@ -37,11 +37,6 @@ static int split_accel_init_machine(MachineState *ms, AccelState *as)
     return 0;
 }
 
-static void split_setup_post(MachineState *ms, AccelState *accel)
-{
-    g_assert_not_reached();
-}
-
 static bool split_has_memory(MachineState *ms, AddressSpace *as,
                              hwaddr start_addr, hwaddr size)
 {
@@ -129,7 +124,6 @@ static void split_accel_class_init(ObjectClass *oc, const void *data)
 
     ac->name = "split";
     ac->init_machine = split_accel_init_machine;
-    ac->setup_post = split_setup_post;
     ac->has_memory = split_has_memory;
     ac->cpus_are_resettable = split_cpus_are_resettable;
     ac->supports_guest_debug = split_supports_guest_debug;
Signature · 1 line
2.49.0
Reviewed-by on v1 6/42
#13 lore raw

Re: [RFC PATCH RESEND 06/42] accel/split: Empty setup_post()

Quoted 22 lines · Philippe Mathieu-Daudé
---
  accel/split/split-all.c | 6 ------
  1 file changed, 6 deletions(-)
accel/split/split-all.c +0 -5
index 28f626d0ff4..8b9f8ff77a4 100644
@@ -37,11 +37,6 @@ static int split_accel_init_machine(MachineState *ms, AccelState *as)
      return 0;
  }
 
-static void split_setup_post(MachineState *ms, AccelState *accel)
-{
-    g_assert_not_reached();
-}
-
  static bool split_has_memory(MachineState *ms, AddressSpace *as,
                               hwaddr start_addr, hwaddr size)
  {

accel/split/split-all.c static void split_accel_class_init(ObjectClass *oc, const void *data) · line 129

Quoted 7 lines
accel/split/split-all.c +0 -1
@@ -129,7 +124,6 @@ static void split_accel_class_init(ObjectClass *oc, const void *data)
 
      ac->name = "split";
      ac->init_machine = split_accel_init_machine;
-    ac->setup_post = split_setup_post;
      ac->has_memory = split_has_memory;
      ac->cpus_are_resettable = split_cpus_are_resettable;
      ac->supports_guest_debug = split_supports_guest_debug;

r~

#14 lore raw

RFC PATCH 7/42 accel/split: Implement supports_guest_debug()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-all.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
accel/split/split-all.c +12 -1
index 8b9f8ff77a4..cb910300931 100644
@@ -61,7 +61,18 @@ static bool split_cpus_are_resettable(AccelState *as)
 
 static bool split_supports_guest_debug(AccelState *as)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    bool supported = false;
+
+    if (hwc->supports_guest_debug) {
+        supported |= hwc->supports_guest_debug(sas->hw);
+    }
+    if (swc->supports_guest_debug) {
+        supported |= swc->supports_guest_debug(sas->sw);
+    }
+    return supported;
 }
 
 static int split_gdbstub_supported_sstep_flags(AccelState *as)
Signature · 1 line
2.49.0
#15 lore raw

Re: [RFC PATCH RESEND 07/42] accel/split: Implement supports_guest_debug()

Quoted 19 lines · Philippe Mathieu-Daudé
---
  accel/split/split-all.c | 13 ++++++++++++-
  1 file changed, 12 insertions(+), 1 deletion(-)
accel/split/split-all.c +5 -1
index 8b9f8ff77a4..cb910300931 100644
@@ -61,7 +61,18 @@ static bool split_cpus_are_resettable(AccelState *as)
 
  static bool split_supports_guest_debug(AccelState *as)
  {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    bool supported = false;
+

accel/split/split-all.c static bool split_cpus_are_resettable(AccelState *as) · line 78

Quoted 10 lines
accel/split/split-all.c +7 -0
+    if (hwc->supports_guest_debug) {
+        supported |= hwc->supports_guest_debug(sas->hw);
+    }
+    if (swc->supports_guest_debug) {
+        supported |= swc->supports_guest_debug(sas->sw);
+    }
+    return supported;
  }
 
  static int split_gdbstub_supported_sstep_flags(AccelState *as)

Require both to support debug? Otherwise support depends on state.

r~

#16 lore raw

RFC PATCH 8/42 accel/split: Implement gdbstub_supported_sstep_flags()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-all.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
accel/split/split-all.c +13 -1
index cb910300931..413954af96c 100644
@@ -77,7 +77,19 @@ static bool split_supports_guest_debug(AccelState *as)
 
 static int split_gdbstub_supported_sstep_flags(AccelState *as)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    int flags = -1;
+
+    if (hwc->gdbstub_supported_sstep_flags) {
+        flags &= hwc->gdbstub_supported_sstep_flags(sas->hw);
+    }
+    if (swc->gdbstub_supported_sstep_flags) {
+        flags &= swc->gdbstub_supported_sstep_flags(sas->sw);
+    }
+
+    return flags;
 }
 
 static void split_get_stats(AccelState *as, GString *buf)
Signature · 1 line
2.49.0
#17 lore raw

Re: [RFC PATCH RESEND 08/42] accel/split: Implement gdbstub_supported_sstep_flags()

Quoted 10 lines · Philippe Mathieu-Daudé
---
  accel/split/split-all.c | 14 +++++++++++++-
  1 file changed, 13 insertions(+), 1 deletion(-)
accel/split/split-all.c +0 -0
index cb910300931..413954af96c 100644

accel/split/split-all.c static bool split_supports_guest_debug(AccelState *as) · line 83

Quoted 8 lines
accel/split/split-all.c +4 -1
@@ -77,7 +77,19 @@ static bool split_supports_guest_debug(AccelState *as)
 
  static int split_gdbstub_supported_sstep_flags(AccelState *as)
  {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    int flags = -1;

This only works if one of hw/sw implements the hook.

r~

Quoted 12 lines
accel/split/split-all.c +9 -0
+
+    if (hwc->gdbstub_supported_sstep_flags) {
+        flags &= hwc->gdbstub_supported_sstep_flags(sas->hw);
+    }
+    if (swc->gdbstub_supported_sstep_flags) {
+        flags &= swc->gdbstub_supported_sstep_flags(sas->sw);
+    }
+
+    return flags;
  }
 
  static void split_get_stats(AccelState *as, GString *buf)
#18 lore raw

RFC PATCH 9/42 accel/split: Add cpu_thread_routine() stub

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 70 ++++++++++++++++++++++++++++++++++-
 1 file changed, 69 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +69 -1
index e5c1d51d426..294ea79420e 100644
@@ -7,11 +7,79 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+#include "qemu/guest-random.h"
+#include "exec/cpu-common.h"
 #include "system/accel-ops.h"
+#include "system/cpus.h"
 
 static void *split_cpu_thread_routine(void *arg)
 {
-    g_assert_not_reached();
+    CPUState *cpu = arg;
+    int r;
+
+    /* TODO: check accel allowed */
+
+    rcu_register_thread();
+
+    bql_lock();
+    qemu_thread_get_self(cpu->thread);
+
+    cpu->thread_id = qemu_get_thread_id();
+    current_cpu = cpu;
+
+    /* TODO: init_vcpu_thread() */
+    /* TODO: allocate AccelCPUState */
+
+    /* signal CPU creation */
+    cpu_thread_signal_created(cpu);
+    qemu_guest_random_seed_thread_part2(cpu->random_seed);
+
+    /* process any pending work */
+    cpu->exit_request = 1;
+
+    do {
+        r = 0;
+
+        if (cpu_can_run(cpu)) {
+            r = 0; /* TODO: exec_vcpu_thread() */
+            switch (r) {
+            case 0:
+                break;
+            case EXCP_INTERRUPT:
+                break;
+            case EXCP_YIELD:
+                break;
+            case EXCP_DEBUG:
+                cpu_handle_guest_debug(cpu);
+                break;
+            case EXCP_HALTED:
+                /*
+                 * Usually cpu->halted is set, but may have already been
+                 * reset by another thread by the time we arrive here.
+                 */
+                break;
+            case EXCP_ATOMIC:
+                bql_unlock();
+                cpu_exec_step_atomic(cpu);
+                bql_lock();
+                break;
+            default:
+                /* Ignore everything else? */
+                break;
+            }
+        }
+        qemu_wait_io_event(cpu);
+    } while (!cpu->unplug || cpu_can_run(cpu));
+
+    /* TODO: destroy_vcpu_thread() */
+
+    cpu_thread_signal_destroyed(cpu);
+    bql_unlock();
+
+    rcu_unregister_thread();
+
+    return NULL;
 }
 
 static void split_ops_init(AccelClass *ac)
Signature · 1 line
2.49.0
Reviewed-by on v1 9/42
#19 lore raw

Re: [RFC PATCH RESEND 09/42] accel/split: Add cpu_thread_routine() stub

Quoted 4 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 70 ++++++++++++++++++++++++++++++++++-
  1 file changed, 69 insertions(+), 1 deletion(-)

r~

Quoted 86 lines
accel/split/split-accel-ops.c +69 -1
index e5c1d51d426..294ea79420e 100644
@@ -7,11 +7,79 @@
   */
 
  #include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+#include "qemu/guest-random.h"
+#include "exec/cpu-common.h"
  #include "system/accel-ops.h"
+#include "system/cpus.h"
 
  static void *split_cpu_thread_routine(void *arg)
  {
-    g_assert_not_reached();
+    CPUState *cpu = arg;
+    int r;
+
+    /* TODO: check accel allowed */
+
+    rcu_register_thread();
+
+    bql_lock();
+    qemu_thread_get_self(cpu->thread);
+
+    cpu->thread_id = qemu_get_thread_id();
+    current_cpu = cpu;
+
+    /* TODO: init_vcpu_thread() */
+    /* TODO: allocate AccelCPUState */
+
+    /* signal CPU creation */
+    cpu_thread_signal_created(cpu);
+    qemu_guest_random_seed_thread_part2(cpu->random_seed);
+
+    /* process any pending work */
+    cpu->exit_request = 1;
+
+    do {
+        r = 0;
+
+        if (cpu_can_run(cpu)) {
+            r = 0; /* TODO: exec_vcpu_thread() */
+            switch (r) {
+            case 0:
+                break;
+            case EXCP_INTERRUPT:
+                break;
+            case EXCP_YIELD:
+                break;
+            case EXCP_DEBUG:
+                cpu_handle_guest_debug(cpu);
+                break;
+            case EXCP_HALTED:
+                /*
+                 * Usually cpu->halted is set, but may have already been
+                 * reset by another thread by the time we arrive here.
+                 */
+                break;
+            case EXCP_ATOMIC:
+                bql_unlock();
+                cpu_exec_step_atomic(cpu);
+                bql_lock();
+                break;
+            default:
+                /* Ignore everything else? */
+                break;
+            }
+        }
+        qemu_wait_io_event(cpu);
+    } while (!cpu->unplug || cpu_can_run(cpu));
+
+    /* TODO: destroy_vcpu_thread() */
+
+    cpu_thread_signal_destroyed(cpu);
+    bql_unlock();
+
+    rcu_unregister_thread();
+
+    return NULL;
  }
 
  static void split_ops_init(AccelClass *ac)
#20 lore raw

RFC PATCH 10/42 accel/split: Define and allocate AccelCPUState

In reply to Philippe Mathieu-Daudé

AccelCPUState is where we store per-vCPU accelerator related information.

FIXME: structures might clash so add padding (ok with TCG+HVF)

---
 accel/split/split-accel.h     | 7 +++++++
 accel/split/split-accel-ops.c | 9 ++++++++-
 2 files changed, 15 insertions(+), 1 deletion(-)
accel/split/split-accel.h +7 -0
index 19fb530f207..87a08454ce6 100644
@@ -38,4 +38,11 @@ struct SplitAccelState {
     char *sw_name;
 };
 
+struct AccelCPUState {
+    char pad[128];
+
+    AccelState *accel;
+    bool use_hw;
+};
+
 #endif /* SPLIT_ACCEL_H */
accel/split/split-accel-ops.c +8 -1
index 294ea79420e..efef0d60fde 100644
@@ -12,10 +12,13 @@
 #include "exec/cpu-common.h"
 #include "system/accel-ops.h"
 #include "system/cpus.h"
+#include "split-accel.h"
 
 static void *split_cpu_thread_routine(void *arg)
 {
+    AccelState *as = current_accel();
     CPUState *cpu = arg;
+    AccelCPUState *acs;
     int r;
 
     /* TODO: check accel allowed */
@@ -29,7 +32,10 @@ static void *split_cpu_thread_routine(void *arg)
     current_cpu = cpu;
 
     /* TODO: init_vcpu_thread() */
-    /* TODO: allocate AccelCPUState */
+    cpu->accel = g_renew(AccelCPUState, cpu->accel, 1); /* XXX only with current TCG */
+    acs = cpu->accel;
+    acs->accel = as;
+    acs->use_hw = false;
 
     /* signal CPU creation */
     cpu_thread_signal_created(cpu);
@@ -60,6 +66,7 @@ static void *split_cpu_thread_routine(void *arg)
                  */
                 break;
             case EXCP_ATOMIC:
+                assert(!acs->use_hw);
                 bql_unlock();
                 cpu_exec_step_atomic(cpu);
                 bql_lock();
Signature · 1 line
2.49.0
#21 lore raw

Re: [RFC PATCH RESEND 10/42] accel/split: Define and allocate AccelCPUState

Quoted 16 lines · Philippe Mathieu-Daudé

AccelCPUState is where we store per-vCPU accelerator related information.

FIXME: structures might clash so add padding (ok with TCG+HVF)

---
  accel/split/split-accel.h     | 7 +++++++
  accel/split/split-accel-ops.c | 9 ++++++++-
  2 files changed, 15 insertions(+), 1 deletion(-)
accel/split/split-accel.h +0 -0
index 19fb530f207..87a08454ce6 100644

accel/split/split-accel.h struct SplitAccelState { · line 42

Quoted 5 lines
accel/split/split-accel.h +2 -0
@@ -38,4 +38,11 @@ struct SplitAccelState {
      char *sw_name;
  };
 
+struct AccelCPUState {
+    char pad[128];

That's a pretty big fixme.
Is this a problem of name overloading?

r~

Quoted 44 lines
accel/split/split-accel.h +5 -0
+
+    AccelState *accel;
+    bool use_hw;
+};
+
  #endif /* SPLIT_ACCEL_H */
accel/split/split-accel-ops.c +8 -1
index 294ea79420e..efef0d60fde 100644
@@ -12,10 +12,13 @@
  #include "exec/cpu-common.h"
  #include "system/accel-ops.h"
  #include "system/cpus.h"
+#include "split-accel.h"
 
  static void *split_cpu_thread_routine(void *arg)
  {
+    AccelState *as = current_accel();
      CPUState *cpu = arg;
+    AccelCPUState *acs;
      int r;
 
      /* TODO: check accel allowed */
@@ -29,7 +32,10 @@ static void *split_cpu_thread_routine(void *arg)
      current_cpu = cpu;
 
      /* TODO: init_vcpu_thread() */
-    /* TODO: allocate AccelCPUState */
+    cpu->accel = g_renew(AccelCPUState, cpu->accel, 1); /* XXX only with current TCG */
+    acs = cpu->accel;
+    acs->accel = as;
+    acs->use_hw = false;
 
      /* signal CPU creation */
      cpu_thread_signal_created(cpu);
@@ -60,6 +66,7 @@ static void *split_cpu_thread_routine(void *arg)
                   */
                  break;
              case EXCP_ATOMIC:
+                assert(!acs->use_hw);
                  bql_unlock();
                  cpu_exec_step_atomic(cpu);
                  bql_lock();
#22 lore raw

RFC PATCH 11/42 accel/split: Register MTTCG

In reply to Philippe Mathieu-Daudé

FIXME: Use sw-hooks?

---
 accel/split/split-accel-ops.c | 5 +++++
 1 file changed, 5 insertions(+)
accel/split/split-accel-ops.c +5 -0
index efef0d60fde..2c7945b6331 100644
@@ -14,9 +14,12 @@
 #include "system/cpus.h"
 #include "split-accel.h"
 
+#include "accel/tcg/tcg-accel-ops-mttcg.h"
+
 static void *split_cpu_thread_routine(void *arg)
 {
     AccelState *as = current_accel();
+    void *sw_force_rcu;
     CPUState *cpu = arg;
     AccelCPUState *acs;
     int r;
@@ -24,6 +27,7 @@ static void *split_cpu_thread_routine(void *arg)
     /* TODO: check accel allowed */
 
     rcu_register_thread();
+    sw_force_rcu = mttcg_vcpu_register(cpu);
 
     bql_lock();
     qemu_thread_get_self(cpu->thread);
@@ -84,6 +88,7 @@ static void *split_cpu_thread_routine(void *arg)
     cpu_thread_signal_destroyed(cpu);
     bql_unlock();
 
+    mttcg_vcpu_unregister(cpu, sw_force_rcu);
     rcu_unregister_thread();
 
     return NULL;
Signature · 1 line
2.49.0
#24 lore raw

RFC PATCH 12/42 accel/split: Have thread routine ready to dispatch over HW/SW

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)
accel/split/split-accel-ops.c +16 -6
index 2c7945b6331..39495fdff14 100644
@@ -19,12 +19,18 @@
 static void *split_cpu_thread_routine(void *arg)
 {
     AccelState *as = current_accel();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    AccelOpsClass *hwops = hwc->ops;
+    AccelOpsClass *swops = swc->ops;
     void *sw_force_rcu;
     CPUState *cpu = arg;
     AccelCPUState *acs;
     int r;
 
-    /* TODO: check accel allowed */
+    assert(swc->allowed);
+    assert(hwc->allowed);
 
     rcu_register_thread();
     sw_force_rcu = mttcg_vcpu_register(cpu);
@@ -35,7 +41,8 @@ static void *split_cpu_thread_routine(void *arg)
     cpu->thread_id = qemu_get_thread_id();
     current_cpu = cpu;
 
-    /* TODO: init_vcpu_thread() */
+    hwops->init_vcpu_thread(cpu);
+    swops->init_vcpu_thread(cpu);
     cpu->accel = g_renew(AccelCPUState, cpu->accel, 1); /* XXX only with current TCG */
     acs = cpu->accel;
     acs->accel = as;
@@ -49,10 +56,12 @@ static void *split_cpu_thread_routine(void *arg)
     cpu->exit_request = 1;
 
     do {
-        r = 0;
-
         if (cpu_can_run(cpu)) {
-            r = 0; /* TODO: exec_vcpu_thread() */
+            if (acs->use_hw) {
+                r = hwops->exec_vcpu_thread(cpu);
+            } else {
+                r = swops->exec_vcpu_thread(cpu);
+            }
             switch (r) {
             case 0:
                 break;
@@ -83,7 +92,8 @@ static void *split_cpu_thread_routine(void *arg)
         qemu_wait_io_event(cpu);
     } while (!cpu->unplug || cpu_can_run(cpu));
 
-    /* TODO: destroy_vcpu_thread() */
+    hwops->destroy_vcpu_thread(cpu);
+    swops->destroy_vcpu_thread(cpu);
 
     cpu_thread_signal_destroyed(cpu);
     bql_unlock();
Signature · 1 line
2.49.0
Reviewed-by on v1 12/42
#25 lore raw

Re: [RFC PATCH RESEND 12/42] accel/split: Have thread routine ready to dispatch over HW/SW

accel/split/split-accel-ops.c static void *split_cpu_thread_routine(void *arg) · line 64

Quoted 13 lines · Philippe Mathieu-Daudé
accel/split/split-accel-ops.c +5 -3
@@ -49,10 +56,12 @@ static void *split_cpu_thread_routine(void *arg)
      cpu->exit_request = 1;
 
      do {
-        r = 0;
-
          if (cpu_can_run(cpu)) {
-            r = 0; /* TODO: exec_vcpu_thread() */
+            if (acs->use_hw) {
+                r = hwops->exec_vcpu_thread(cpu);
+            } else {
+                r = swops->exec_vcpu_thread(cpu);
+            }

Maybe

  AccelOpsClass *curops = acs->use_hw ? hwops : swops;
  r = curops->exec_vcpu_thread(cpu);

Anyway,

r~

#26 lore raw

RFC PATCH 13/42 accel/split: Implement cpu_reset_hold()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +11 -1
index 39495fdff14..d4fa07c2dec 100644
@@ -10,6 +10,7 @@
 #include "qemu/main-loop.h"
 #include "qemu/guest-random.h"
 #include "exec/cpu-common.h"
+#include "hw/core/cpu.h"
 #include "system/accel-ops.h"
 #include "system/cpus.h"
 #include "split-accel.h"
@@ -121,7 +122,16 @@ static void split_cpu_common_unrealize(CPUState *cpu)
 
 static void split_cpu_reset_hold(CPUState *cpu)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+
+    if (hwc->ops->cpu_reset_hold) {
+        hwc->ops->cpu_reset_hold(cpu);
+    }
+    if (swc->ops->cpu_reset_hold) {
+        swc->ops->cpu_reset_hold(cpu);
+    }
 }
 
 static void split_kick_vcpu_thread(CPUState *cpu)
Signature · 1 line
2.49.0
Reviewed-by on v1 13/42
#27 lore raw

Re: [RFC PATCH RESEND 13/42] accel/split: Implement cpu_reset_hold()

Quoted 4 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 12 +++++++++++-
  1 file changed, 11 insertions(+), 1 deletion(-)

r~

Quoted 31 lines
accel/split/split-accel-ops.c +11 -1
index 39495fdff14..d4fa07c2dec 100644
@@ -10,6 +10,7 @@
  #include "qemu/main-loop.h"
  #include "qemu/guest-random.h"
  #include "exec/cpu-common.h"
+#include "hw/core/cpu.h"
  #include "system/accel-ops.h"
  #include "system/cpus.h"
  #include "split-accel.h"
@@ -121,7 +122,16 @@ static void split_cpu_common_unrealize(CPUState *cpu)
 
  static void split_cpu_reset_hold(CPUState *cpu)
  {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+
+    if (hwc->ops->cpu_reset_hold) {
+        hwc->ops->cpu_reset_hold(cpu);
+    }
+    if (swc->ops->cpu_reset_hold) {
+        swc->ops->cpu_reset_hold(cpu);
+    }
  }
 
  static void split_kick_vcpu_thread(CPUState *cpu)
#28 lore raw

RFC PATCH 14/42 accel/split: Implement synchronize_post_init()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +10 -1
index d4fa07c2dec..6b3ddf21a17 100644
@@ -151,7 +151,16 @@ static void split_synchronize_post_reset(CPUState *cpu)
 
 static void split_synchronize_post_init(CPUState *cpu)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+
+    if (hwc->ops->synchronize_post_init) {
+        hwc->ops->synchronize_post_init(cpu);
+    }
+    if (swc->ops->synchronize_post_init) {
+        swc->ops->synchronize_post_init(cpu);
+    }
 }
 
 static void split_synchronize_state(CPUState *cpu)
Signature · 1 line
2.49.0
Reviewed-by on v1 14/42
#29 lore raw

Re: [RFC PATCH RESEND 14/42] accel/split: Implement synchronize_post_init()

Quoted 17 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 11 ++++++++++-
  1 file changed, 10 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +3 -1
index d4fa07c2dec..6b3ddf21a17 100644
@@ -151,7 +151,16 @@ static void split_synchronize_post_reset(CPUState *cpu)
 
  static void split_synchronize_post_init(CPUState *cpu)
  {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);

accel/split/split-accel-ops.c static void split_synchronize_post_reset(CPUState *cpu) · line 166

Quoted 10 lines
accel/split/split-accel-ops.c +7 -0
+
+    if (hwc->ops->synchronize_post_init) {
+        hwc->ops->synchronize_post_init(cpu);
+    }
+    if (swc->ops->synchronize_post_init) {
+        swc->ops->synchronize_post_init(cpu);
+    }
  }
 
  static void split_synchronize_state(CPUState *cpu)

r~

#30 lore raw

RFC PATCH 15/42 accel/split: Implement synchronize_pre_resume()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +11 -1
index 6b3ddf21a17..65feb929404 100644
@@ -175,7 +175,17 @@ static void split_synchronize_pre_loadvm(CPUState *cpu)
 
 static void split_synchronize_pre_resume(bool step_pending)
 {
-    g_assert_not_reached();
+    AccelState *as = current_accel();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+
+    if (hwc->ops->synchronize_pre_resume) {
+        hwc->ops->synchronize_pre_resume(step_pending);
+    }
+    if (swc->ops->synchronize_pre_resume) {
+        swc->ops->synchronize_pre_resume(step_pending);
+    }
 }
 
 static void split_handle_interrupt(CPUState *cpu, int mask)
Signature · 1 line
2.49.0
Reviewed-by on v1 15/42
#31 lore raw

Re: [RFC PATCH RESEND 15/42] accel/split: Implement synchronize_pre_resume()

Quoted 18 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 12 +++++++++++-
  1 file changed, 11 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +4 -1
index 6b3ddf21a17..65feb929404 100644
@@ -175,7 +175,17 @@ static void split_synchronize_pre_loadvm(CPUState *cpu)
 
  static void split_synchronize_pre_resume(bool step_pending)
  {
-    g_assert_not_reached();
+    AccelState *as = current_accel();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);

accel/split/split-accel-ops.c static void split_synchronize_pre_loadvm(CPUState *cpu) · line 191

Quoted 10 lines
accel/split/split-accel-ops.c +7 -0
+
+    if (hwc->ops->synchronize_pre_resume) {
+        hwc->ops->synchronize_pre_resume(step_pending);
+    }
+    if (swc->ops->synchronize_pre_resume) {
+        swc->ops->synchronize_pre_resume(step_pending);
+    }
  }
 
  static void split_handle_interrupt(CPUState *cpu, int mask)

r~

#32 lore raw

RFC PATCH 16/42 accel/split: Implement synchronize_state()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +10 -1
index 65feb929404..80efbd82091 100644
@@ -165,7 +165,16 @@ static void split_synchronize_post_init(CPUState *cpu)
 
 static void split_synchronize_state(CPUState *cpu)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+
+    if (hwc->ops->synchronize_state) {
+        hwc->ops->synchronize_state(cpu);
+    }
+    if (swc->ops->synchronize_state) {
+        swc->ops->synchronize_state(cpu);
+    }
 }
 
 static void split_synchronize_pre_loadvm(CPUState *cpu)
Signature · 1 line
2.49.0
#33 lore raw

Re: [RFC PATCH RESEND 16/42] accel/split: Implement synchronize_state()

Quoted 15 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 11 ++++++++++-
  1 file changed, 10 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +1 -1
index 65feb929404..80efbd82091 100644
@@ -165,7 +165,16 @@ static void split_synchronize_post_init(CPUState *cpu)
 
  static void split_synchronize_state(CPUState *cpu)
  {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);

accel/split/split-accel-ops.c static void split_synchronize_post_init(CPUState *cpu) · line 178

Quoted 10 lines
accel/split/split-accel-ops.c +9 -0
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+
+    if (hwc->ops->synchronize_state) {
+        hwc->ops->synchronize_state(cpu);
+    }
+    if (swc->ops->synchronize_state) {
+        swc->ops->synchronize_state(cpu);
+    }
  }

I suppose in the short-term this is ok, but it does more work than it should.

Let's make the simplifying assumption that even if there is another sw emulator, it'll still use CPUArchState. So the "sync" to sw is pointless.

But syncing state to hw is only applicable when use_hw is true. I'm not sure how to do the pointer chasing to find that from here.

r~

#34 lore raw

Re: [RFC PATCH RESEND 16/42] accel/split: Implement synchronize_state()

Quoted 28 lines · Richard Henderson
Quoted 14 lines · Philippe Mathieu-Daudé

---
  accel/split/split-accel-ops.c | 11 ++++++++++-

  1 file changed, 10 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +1 -1
index 65feb929404..80efbd82091 100644
@@ -165,7 +165,16 @@ static void split_synchronize_post_init(CPUState *cpu)
   static void split_synchronize_state(CPUState *cpu)
   {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);

accel/split/split-accel-ops.c static void split_synchronize_post_init(CPUState *cpu) · line 177

Quoted 10 lines
accel/split/split-accel-ops.c +9 -0
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+
+    if (hwc->ops->synchronize_state) {
+        hwc->ops->synchronize_state(cpu);
+    }
+    if (swc->ops->synchronize_state) {
+        swc->ops->synchronize_state(cpu);
+    }
   }

I suppose in the short-term this is ok, but it does more work than it should.

Quoted 4 lines

Let's make the simplifying assumption that even if there is another sw emulator, it'll still use CPUArchState.  So the "sync" to sw is pointless.

But syncing state to hw is only applicable when use_hw is true. I'm not sure how to do the pointer chasing to find that from here.

The same applies to patches 14-18.

r~

#35 lore raw

RFC PATCH 17/42 accel/split: Implement synchronize_post_reset()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +10 -1
index 80efbd82091..91ed0af2746 100644
@@ -146,7 +146,16 @@ static bool split_cpu_thread_is_idle(CPUState *cpu)
 
 static void split_synchronize_post_reset(CPUState *cpu)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+
+    if (hwc->ops->synchronize_post_reset) {
+        hwc->ops->synchronize_post_reset(cpu);
+    }
+    if (swc->ops->synchronize_post_reset) {
+        swc->ops->synchronize_post_reset(cpu);
+    }
 }
 
 static void split_synchronize_post_init(CPUState *cpu)
Signature · 1 line
2.49.0
#36 lore raw

RFC PATCH 18/42 accel/split: Implement synchronize_pre_loadvm()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +10 -1
index 91ed0af2746..746a6a2782d 100644
@@ -188,7 +188,16 @@ static void split_synchronize_state(CPUState *cpu)
 
 static void split_synchronize_pre_loadvm(CPUState *cpu)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+
+    if (hwc->ops->synchronize_pre_loadvm) {
+        hwc->ops->synchronize_pre_loadvm(cpu);
+    }
+    if (swc->ops->synchronize_pre_loadvm) {
+        swc->ops->synchronize_pre_loadvm(cpu);
+    }
 }
 
 static void split_synchronize_pre_resume(bool step_pending)
Signature · 1 line
2.49.0
#37 lore raw

RFC PATCH 19/42 accel/split: Implement kick_vcpu_thread()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +7 -1
index 746a6a2782d..63b6217f3fa 100644
@@ -136,7 +136,13 @@ static void split_cpu_reset_hold(CPUState *cpu)
 
 static void split_kick_vcpu_thread(CPUState *cpu)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *ac = cpu->accel->use_hw ? ACCEL_GET_CLASS(sas->hw)
+                                        : ACCEL_GET_CLASS(sas->sw);
+
+    if (ac->ops->kick_vcpu_thread) {
+        ac->ops->kick_vcpu_thread(cpu);
+    }
 }
 
 static bool split_cpu_thread_is_idle(CPUState *cpu)
Signature · 1 line
2.49.0
Reviewed-by on v1 19/42
#38 lore raw

Re: [RFC PATCH RESEND 19/42] accel/split: Implement kick_vcpu_thread()

Quoted 10 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 8 +++++++-
  1 file changed, 7 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +0 -0
index 746a6a2782d..63b6217f3fa 100644

accel/split/split-accel-ops.c static void split_cpu_reset_hold(CPUState *cpu) · line 141

Quoted 7 lines
accel/split/split-accel-ops.c +3 -1
@@ -136,7 +136,13 @@ static void split_cpu_reset_hold(CPUState *cpu)
 
  static void split_kick_vcpu_thread(CPUState *cpu)
  {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *ac = cpu->accel->use_hw ? ACCEL_GET_CLASS(sas->hw)
+                                        : ACCEL_GET_CLASS(sas->sw);

Move the ?: inside the ACCEL_GET_CLASS?

Anyway,

r~

Quoted 7 lines
accel/split/split-accel-ops.c +4 -0
+
+    if (ac->ops->kick_vcpu_thread) {
+        ac->ops->kick_vcpu_thread(cpu);
+    }
  }
 
  static bool split_cpu_thread_is_idle(CPUState *cpu)
#39 lore raw

RFC PATCH 20/42 accel/split: Implement cpu_common_realize()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +9 -1
index 63b6217f3fa..3278e01f18a 100644
@@ -15,6 +15,7 @@
 #include "system/cpus.h"
 #include "split-accel.h"
 
+#include "accel/accel-internal.h"
 #include "accel/tcg/tcg-accel-ops-mttcg.h"
 
 static void *split_cpu_thread_routine(void *arg)
@@ -112,7 +113,14 @@ static void split_ops_init(AccelClass *ac)
 
 static bool split_cpu_common_realize(CPUState *cpu, Error **errp)
 {
-    g_assert_not_reached();
+    AccelState *as = current_accel();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    bool success = false;
+
+    success |= accel_cpu_realize(sas->sw, cpu, errp);
+    success |= accel_cpu_realize(sas->hw, cpu, errp);
+
+    return success;
 }
 
 static void split_cpu_common_unrealize(CPUState *cpu)
Signature · 1 line
2.49.0
#40 lore raw

Re: [RFC PATCH RESEND 20/42] accel/split: Implement cpu_common_realize()

Quoted 18 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 10 +++++++++-
  1 file changed, 9 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +1 -0
index 63b6217f3fa..3278e01f18a 100644
@@ -15,6 +15,7 @@
  #include "system/cpus.h"
  #include "split-accel.h"
 
+#include "accel/accel-internal.h"
  #include "accel/tcg/tcg-accel-ops-mttcg.h"
 
  static void *split_cpu_thread_routine(void *arg)

accel/split/split-accel-ops.c static void split_ops_init(AccelClass *ac) · line 121

Quoted 10 lines
accel/split/split-accel-ops.c +6 -1
@@ -112,7 +113,14 @@ static void split_ops_init(AccelClass *ac)
 
  static bool split_cpu_common_realize(CPUState *cpu, Error **errp)
  {
-    g_assert_not_reached();
+    AccelState *as = current_accel();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    bool success = false;
+
+    success |= accel_cpu_realize(sas->sw, cpu, errp);
+    success |= accel_cpu_realize(sas->hw, cpu, errp);

&&

r~

#41 lore raw

RFC PATCH 21/42 accel/split: Set use_hw in cpu_thread_routine() and switch over

In reply to Philippe Mathieu-Daudé

Introduce the EXCP_HWACCEL definition to switch to hardware accelerator.

---
 accel/split/split-accel.h     |  1 +
 include/exec/cpu-common.h     |  1 +
 accel/split/split-accel-ops.c | 11 +++++++++++
 3 files changed, 13 insertions(+)
accel/split/split-accel.h +1 -0
index 87a08454ce6..8a2888507f3 100644
@@ -42,6 +42,7 @@ struct AccelCPUState {
     char pad[128];
 
     AccelState *accel;
+
     bool use_hw;
 };
 
include/exec/cpu-common.h +1 -0
index a68485547d5..6216be5f5e3 100644
@@ -20,6 +20,7 @@
 #define EXCP_HALTED     0x10003 /* cpu is halted (waiting for external event) */
 #define EXCP_YIELD      0x10004 /* cpu wants to yield timeslice to another */
 #define EXCP_ATOMIC     0x10005 /* stop-the-world and emulate atomic */
+#define EXCP_HWACCEL    0x10006 /* use hardware accelerator */
 
 void cpu_exec_init_all(void);
 void cpu_exec_step_atomic(CPUState *cpu);
accel/split/split-accel-ops.c +11 -0
index 3278e01f18a..1766834d537 100644
@@ -10,9 +10,12 @@
 #include "qemu/main-loop.h"
 #include "qemu/guest-random.h"
 #include "exec/cpu-common.h"
+#include "exec/cpu-interrupt.h"
 #include "hw/core/cpu.h"
 #include "system/accel-ops.h"
 #include "system/cpus.h"
+#include "system/hw_accel.h"
+#include "system/tcg.h"
 #include "split-accel.h"
 
 #include "accel/accel-internal.h"
@@ -66,10 +69,14 @@ static void *split_cpu_thread_routine(void *arg)
             }
             switch (r) {
             case 0:
+                if (acs->use_hw) {
+                    acs->use_hw = cpu_acceleratable(cpu);
+                }
                 break;
             case EXCP_INTERRUPT:
                 break;
             case EXCP_YIELD:
+                assert(!acs->use_hw);
                 break;
             case EXCP_DEBUG:
                 cpu_handle_guest_debug(cpu);
@@ -86,6 +93,10 @@ static void *split_cpu_thread_routine(void *arg)
                 cpu_exec_step_atomic(cpu);
                 bql_lock();
                 break;
+            case EXCP_HWACCEL:
+                assert(!acs->use_hw);
+                acs->use_hw = true;
+                break;
             default:
                 /* Ignore everything else? */
                 break;
Signature · 1 line
2.49.0
#42 lore raw

Re: [RFC PATCH RESEND 21/42] accel/split: Set use_hw in cpu_thread_routine() and switch over

Quoted 15 lines · Philippe Mathieu-Daudé

Introduce the EXCP_HWACCEL definition to switch to hardware accelerator.

---
  accel/split/split-accel.h     |  1 +
  include/exec/cpu-common.h     |  1 +
  accel/split/split-accel-ops.c | 11 +++++++++++
  3 files changed, 13 insertions(+)
accel/split/split-accel.h +0 -0
index 87a08454ce6..8a2888507f3 100644

accel/split/split-accel.h struct AccelCPUState { · line 46

Quoted 5 lines
accel/split/split-accel.h +1 -0
@@ -42,6 +42,7 @@ struct AccelCPUState {
      char pad[128];
 
      AccelState *accel;
+
      bool use_hw;

Stray.

accel/split/split-accel.h static void *split_cpu_thread_routine(void *arg) · line 75

Quoted 8 lines
accel/split/split-accel.h +3 -0
@@ -66,10 +69,14 @@ static void *split_cpu_thread_routine(void *arg)
              }
              switch (r) {
              case 0:
+                if (acs->use_hw) {
+                    acs->use_hw = cpu_acceleratable(cpu);
+                }
                  break;

Why the conditional? Why can't we enable use_hw here?

Quoted 8 lines
accel/split/split-accel.h +1 -0
              case EXCP_INTERRUPT:
                  break;
              case EXCP_YIELD:
+                assert(!acs->use_hw);
                  break;
              case EXCP_DEBUG:
                  cpu_handle_guest_debug(cpu);

accel/split/split-accel.h static void *split_cpu_thread_routine(void *arg) · line 98

Quoted 6 lines
accel/split/split-accel.h +3 -0
@@ -86,6 +93,10 @@ static void *split_cpu_thread_routine(void *arg)
                  cpu_exec_step_atomic(cpu);
                  bql_lock();
                  break;
+            case EXCP_HWACCEL:
+                assert(!acs->use_hw);
+                acs->use_hw = true;

Why the unconditional? Why isn't it more correct to check cpu_acceleratable?

r~

#43 lore raw

RFC PATCH 22/42 accel/split: Add few trace events in cpu_thread_routine handler

In reply to Philippe Mathieu-Daudé
---
 meson.build                   | 1 +
 accel/split/trace.h           | 2 ++
 accel/split/split-accel-ops.c | 7 +++++++
 accel/split/trace-events      | 9 +++++++++
 4 files changed, 19 insertions(+)
 create mode 100644 accel/split/trace.h
 create mode 100644 accel/split/trace-events
meson.build +1 -0
index 5004678a26b..06c32e0e958 100644
@@ -3635,6 +3635,7 @@ if have_system
   trace_events_subdirs += [
     'accel/hvf',
     'accel/kvm',
+    'accel/split',
     'audio',
     'backends',
     'backends/tpm',
accel/split/trace.h new file +2 -0
new file mode 100644
index 00000000000..46b45388930
@@ -0,0 +1,2 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#include "trace/trace-accel_split.h"
accel/split/split-accel-ops.c +7 -0
index 1766834d537..399bf71a9ec 100644
@@ -17,6 +17,7 @@
 #include "system/hw_accel.h"
 #include "system/tcg.h"
 #include "split-accel.h"
+#include "trace.h"
 
 #include "accel/accel-internal.h"
 #include "accel/tcg/tcg-accel-ops-mttcg.h"
@@ -64,13 +65,18 @@ static void *split_cpu_thread_routine(void *arg)
         if (cpu_can_run(cpu)) {
             if (acs->use_hw) {
                 r = hwops->exec_vcpu_thread(cpu);
+                trace_accel_split_exec_vcpu_thread_hw(r);
             } else {
                 r = swops->exec_vcpu_thread(cpu);
+                trace_accel_split_exec_vcpu_thread_sw(r);
             }
             switch (r) {
             case 0:
                 if (acs->use_hw) {
                     acs->use_hw = cpu_acceleratable(cpu);
+                    if (!acs->use_hw) {
+                        trace_accel_split_sw_emulate();
+                    }
                 }
                 break;
             case EXCP_INTERRUPT:
@@ -95,6 +101,7 @@ static void *split_cpu_thread_routine(void *arg)
                 break;
             case EXCP_HWACCEL:
                 assert(!acs->use_hw);
+                trace_accel_split_hw_accelerate();
                 acs->use_hw = true;
                 break;
             default:
accel/split/trace-events new file +9 -0
new file mode 100644
index 00000000000..c0b9fd97d1d
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# See docs/devel/tracing.rst for syntax documentation.
+
+# split-accel-ops.c
+accel_split_exec_vcpu_thread_sw(int retval) "sw: rv:0x%x"
+accel_split_exec_vcpu_thread_hw(int retval) "hw: rv:0x%x"
+accel_split_hw_accelerate(void) ""
+accel_split_sw_emulate(void) ""
Signature · 1 line
2.49.0
#44 lore raw

RFC PATCH 23/42 target/arm: Implement SysemuCPUOps::can_accelerate() handler

In reply to Philippe Mathieu-Daudé

ARM hardware can only accelerate EL0 and EL1.

---
 target/arm/cpu.c | 8 ++++++++
 1 file changed, 8 insertions(+)
target/arm/cpu.c +8 -0
index ab5fbd9b40b..1a19e5cfb45 100644
@@ -149,6 +149,13 @@ static bool arm_cpu_has_work(CPUState *cs)
          | CPU_INTERRUPT_VFIQ | CPU_INTERRUPT_VIRQ | CPU_INTERRUPT_VSERR
          | CPU_INTERRUPT_EXITTB);
 }
+
+static bool arm_cpu_can_accelerate(CPUState *cs)
+{
+    CPUARMState *env = cpu_env(cs);
+
+    return arm_current_el(env) < 2;
+}
 #endif /* !CONFIG_USER_ONLY */
 
 void arm_register_pre_el_change_hook(ARMCPU *cpu, ARMELChangeHookFn *hook,
@@ -2695,6 +2702,7 @@ static vaddr aarch64_untagged_addr(CPUState *cs, vaddr x)
 
 static const struct SysemuCPUOps arm_sysemu_ops = {
     .has_work = arm_cpu_has_work,
+    .can_accelerate = arm_cpu_can_accelerate,
     .get_phys_page_attrs_debug = arm_cpu_get_phys_page_attrs_debug,
     .asidx_from_attrs = arm_asidx_from_attrs,
     .write_elf32_note = arm_cpu_write_elf32_note,
Signature · 1 line
2.49.0
#46 lore raw

RFC PATCH 24/42 accel/split: Implement handle_interrupt()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +9 -1
index 399bf71a9ec..5a36e22f205 100644
@@ -249,7 +249,15 @@ static void split_synchronize_pre_resume(bool step_pending)
 
 static void split_handle_interrupt(CPUState *cpu, int mask)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *ac = cpu->accel->use_hw ? ACCEL_GET_CLASS(sas->hw)
+                                        : ACCEL_GET_CLASS(sas->sw);
+
+    if (ac->ops->handle_interrupt) {
+        ac->ops->handle_interrupt(cpu, mask);
+    } else {
+        generic_handle_interrupt(cpu, mask);
+    }
 }
 
 static int64_t split_get_virtual_clock(void)
Signature · 1 line
2.49.0
#47 lore raw

Re: [RFC PATCH RESEND 24/42] accel/split: Implement handle_interrupt()

Quoted 13 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 10 +++++++++-
  1 file changed, 9 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +0 -0
index 399bf71a9ec..5a36e22f205 100644
@@ -249,7 +249,15 @@ static void split_synchronize_pre_resume(bool step_pending)
 
  static void split_handle_interrupt(CPUState *cpu, int mask)
  {

accel/split/split-accel-ops.c static void split_synchronize_pre_resume(bool step_pending) · line 260

Quoted 10 lines
accel/split/split-accel-ops.c +9 -1
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *ac = cpu->accel->use_hw ? ACCEL_GET_CLASS(sas->hw)
+                                        : ACCEL_GET_CLASS(sas->sw);
+
+    if (ac->ops->handle_interrupt) {
+        ac->ops->handle_interrupt(cpu, mask);
+    } else {
+        generic_handle_interrupt(cpu, mask);
+    }

Better to have handle_interrupt always filled-in.

r~

#48 lore raw

RFC PATCH 25/42 accel/split: Empty ops_init()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 6 ------
 1 file changed, 6 deletions(-)
accel/split/split-accel-ops.c +0 -6
index 5a36e22f205..375821f1064 100644
@@ -124,11 +124,6 @@ static void *split_cpu_thread_routine(void *arg)
     return NULL;
 }
 
-static void split_ops_init(AccelClass *ac)
-{
-    g_assert_not_reached();
-}
-
 static bool split_cpu_common_realize(CPUState *cpu, Error **errp)
 {
     AccelState *as = current_accel();
@@ -306,7 +301,6 @@ static void split_accel_ops_class_init(ObjectClass *oc, const void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
-    ops->ops_init = split_ops_init;
     ops->cpu_common_realize = split_cpu_common_realize;
     ops->cpu_common_unrealize = split_cpu_common_unrealize;
     ops->cpu_reset_hold = split_cpu_reset_hold;
Signature · 1 line
2.49.0
Reviewed-by on v1 25/42
#49 lore raw

Re: [RFC PATCH RESEND 25/42] accel/split: Empty ops_init()

Quoted 22 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 6 ------
  1 file changed, 6 deletions(-)
accel/split/split-accel-ops.c +0 -5
index 5a36e22f205..375821f1064 100644
@@ -124,11 +124,6 @@ static void *split_cpu_thread_routine(void *arg)
      return NULL;
  }
 
-static void split_ops_init(AccelClass *ac)
-{
-    g_assert_not_reached();
-}
-
  static bool split_cpu_common_realize(CPUState *cpu, Error **errp)
  {
      AccelState *as = current_accel();

accel/split/split-accel-ops.c static void split_accel_ops_class_init(ObjectClass *oc, const void *data) · line 306

Quoted 7 lines
accel/split/split-accel-ops.c +0 -1
@@ -306,7 +301,6 @@ static void split_accel_ops_class_init(ObjectClass *oc, const void *data)
  {
      AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
-    ops->ops_init = split_ops_init;
      ops->cpu_common_realize = split_cpu_common_realize;
      ops->cpu_common_unrealize = split_cpu_common_unrealize;
      ops->cpu_reset_hold = split_cpu_reset_hold;

r~

#50 lore raw

RFC PATCH 26/42 accel/split: Empty set/get_virtual_clock()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 12 ------------
 1 file changed, 12 deletions(-)
accel/split/split-accel-ops.c +0 -12
index 375821f1064..d19b4641a2e 100644
@@ -255,16 +255,6 @@ static void split_handle_interrupt(CPUState *cpu, int mask)
     }
 }
 
-static int64_t split_get_virtual_clock(void)
-{
-    g_assert_not_reached();
-}
-
-static void split_set_virtual_clock(int64_t time)
-{
-    g_assert_not_reached();
-}
-
 static int64_t split_get_elapsed_ticks(void)
 {
     g_assert_not_reached();
@@ -317,8 +307,6 @@ static void split_accel_ops_class_init(ObjectClass *oc, const void *data)
     ops->handle_interrupt = split_handle_interrupt;
     ops->get_vcpu_stats = split_get_vcpu_stats;
 
-    ops->get_virtual_clock = split_get_virtual_clock;
-    ops->set_virtual_clock = split_set_virtual_clock;
     ops->get_elapsed_ticks = split_get_elapsed_ticks;
 
     ops->update_guest_debug = split_update_guest_debug;
Signature · 1 line
2.49.0
#52 lore raw

RFC PATCH 27/42 accel/split: Empty get_elapsed_ticks()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 7 -------
 1 file changed, 7 deletions(-)
accel/split/split-accel-ops.c +0 -7
index d19b4641a2e..034b5ebc96c 100644
@@ -255,11 +255,6 @@ static void split_handle_interrupt(CPUState *cpu, int mask)
     }
 }
 
-static int64_t split_get_elapsed_ticks(void)
-{
-    g_assert_not_reached();
-}
-
 static int split_update_guest_debug(CPUState *cpu)
 {
     g_assert_not_reached();
@@ -307,8 +302,6 @@ static void split_accel_ops_class_init(ObjectClass *oc, const void *data)
     ops->handle_interrupt = split_handle_interrupt;
     ops->get_vcpu_stats = split_get_vcpu_stats;
 
-    ops->get_elapsed_ticks = split_get_elapsed_ticks;
-
     ops->update_guest_debug = split_update_guest_debug;
     ops->insert_breakpoint = split_insert_breakpoint;
     ops->remove_breakpoint = split_remove_breakpoint;
Signature · 1 line
2.49.0
Reviewed-by on v1 27/42
#53 lore raw

Re: [RFC PATCH RESEND 27/42] accel/split: Empty get_elapsed_ticks()

Quoted 22 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 7 -------
  1 file changed, 7 deletions(-)
accel/split/split-accel-ops.c +0 -5
index d19b4641a2e..034b5ebc96c 100644
@@ -255,11 +255,6 @@ static void split_handle_interrupt(CPUState *cpu, int mask)
      }
  }
 
-static int64_t split_get_elapsed_ticks(void)
-{
-    g_assert_not_reached();
-}
-
  static int split_update_guest_debug(CPUState *cpu)
  {
      g_assert_not_reached();

accel/split/split-accel-ops.c static void split_accel_ops_class_init(ObjectClass *oc, const void *data) · line 307

Quoted 8 lines
accel/split/split-accel-ops.c +0 -2
@@ -307,8 +302,6 @@ static void split_accel_ops_class_init(ObjectClass *oc, const void *data)
      ops->handle_interrupt = split_handle_interrupt;
      ops->get_vcpu_stats = split_get_vcpu_stats;
 
-    ops->get_elapsed_ticks = split_get_elapsed_ticks;
-
      ops->update_guest_debug = split_update_guest_debug;
      ops->insert_breakpoint = split_insert_breakpoint;
      ops->remove_breakpoint = split_remove_breakpoint;

r~

#54 lore raw

RFC PATCH 28/42 accel/split: Empty cpu_thread_is_idle()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 6 ------
 1 file changed, 6 deletions(-)
accel/split/split-accel-ops.c +0 -6
index 034b5ebc96c..e1d91ace2fa 100644
@@ -166,11 +166,6 @@ static void split_kick_vcpu_thread(CPUState *cpu)
     }
 }
 
-static bool split_cpu_thread_is_idle(CPUState *cpu)
-{
-    g_assert_not_reached();
-}
-
 static void split_synchronize_post_reset(CPUState *cpu)
 {
     SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
@@ -291,7 +286,6 @@ static void split_accel_ops_class_init(ObjectClass *oc, const void *data)
     ops->cpu_reset_hold = split_cpu_reset_hold;
     ops->cpu_thread_routine = split_cpu_thread_routine;
     ops->kick_vcpu_thread = split_kick_vcpu_thread;
-    ops->cpu_thread_is_idle = split_cpu_thread_is_idle;
 
     ops->synchronize_post_reset = split_synchronize_post_reset;
     ops->synchronize_post_init = split_synchronize_post_init;
Signature · 1 line
2.49.0
#55 lore raw

Re: [RFC PATCH RESEND 28/42] accel/split: Empty cpu_thread_is_idle()

Quoted 4 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 6 ------
  1 file changed, 6 deletions(-)

Why does this not defer to the current sub-accel?

r~

Quoted 25 lines
accel/split/split-accel-ops.c +0 -6
index 034b5ebc96c..e1d91ace2fa 100644
@@ -166,11 +166,6 @@ static void split_kick_vcpu_thread(CPUState *cpu)
      }
  }
 
-static bool split_cpu_thread_is_idle(CPUState *cpu)
-{
-    g_assert_not_reached();
-}
-
  static void split_synchronize_post_reset(CPUState *cpu)
  {
      SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
@@ -291,7 +286,6 @@ static void split_accel_ops_class_init(ObjectClass *oc, const void *data)
      ops->cpu_reset_hold = split_cpu_reset_hold;
      ops->cpu_thread_routine = split_cpu_thread_routine;
      ops->kick_vcpu_thread = split_kick_vcpu_thread;
-    ops->cpu_thread_is_idle = split_cpu_thread_is_idle;
 
      ops->synchronize_post_reset = split_synchronize_post_reset;
      ops->synchronize_post_init = split_synchronize_post_init;
#56 lore raw

RFC PATCH 29/42 accel/split: Kludge qemu_tcg_mttcg_enabled()

In reply to Philippe Mathieu-Daudé

../../accel/tcg/tcg-all.c:59:TCG_STATE: Object 0x600000c42740 is not an instance of type tcg-accel

---
 accel/tcg/tcg-all.c | 1 +
 1 file changed, 1 insertion(+)
accel/tcg/tcg-all.c +1 -0
index f5920b5796e..96104915eef 100644
@@ -61,6 +61,7 @@ DECLARE_INSTANCE_CHECKER(TCGState, TCG_STATE,
 #ifndef CONFIG_USER_ONLY
 bool qemu_tcg_mttcg_enabled(void)
 {
+    return true;
     TCGState *s = TCG_STATE(current_accel());
     return s->mttcg_enabled == ON_OFF_AUTO_ON;
 }
Signature · 1 line
2.49.0
#57 lore raw

RFC PATCH 30/42 accel/split: Implement remove_all_breakpoints()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +10 -1
index e1d91ace2fa..0f3d48fc68d 100644
@@ -269,7 +269,16 @@ static int split_remove_breakpoint(CPUState *cpu, int type,
 
 static void split_remove_all_breakpoints(CPUState *cpu)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+
+    if (hwc->ops->remove_all_breakpoints) {
+        hwc->ops->remove_all_breakpoints(cpu);
+    }
+    if (swc->ops->remove_all_breakpoints) {
+        swc->ops->remove_all_breakpoints(cpu);
+    }
 }
 
 static void split_get_vcpu_stats(CPUState *cpu, GString *buf)
Signature · 1 line
2.49.0
Reviewed-by on v1 30/42
#58 lore raw

Re: [RFC PATCH RESEND 30/42] accel/split: Implement remove_all_breakpoints()

Quoted 17 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 11 ++++++++++-
  1 file changed, 10 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +3 -1
index e1d91ace2fa..0f3d48fc68d 100644
@@ -269,7 +269,16 @@ static int split_remove_breakpoint(CPUState *cpu, int type,
 
  static void split_remove_all_breakpoints(CPUState *cpu)
  {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);

accel/split/split-accel-ops.c static int split_remove_breakpoint(CPUState *cpu, int type, · line 284

Quoted 10 lines
accel/split/split-accel-ops.c +7 -0
+
+    if (hwc->ops->remove_all_breakpoints) {
+        hwc->ops->remove_all_breakpoints(cpu);
+    }
+    if (swc->ops->remove_all_breakpoints) {
+        swc->ops->remove_all_breakpoints(cpu);
+    }
  }
 
  static void split_get_vcpu_stats(CPUState *cpu, GString *buf)

r~

#59 lore raw

RFC PATCH 31/42 accel/split: Implement remove_breakpoint()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +12 -1
index 0f3d48fc68d..40cd39aea5c 100644
@@ -264,7 +264,18 @@ static int split_insert_breakpoint(CPUState *cpu, int type,
 static int split_remove_breakpoint(CPUState *cpu, int type,
                                    vaddr addr, vaddr len)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    int err = 0;
+
+    if (hwc->ops->remove_breakpoint) {
+        err |= hwc->ops->remove_breakpoint(cpu, type, addr, len);
+    }
+    if (swc->ops->remove_breakpoint) {
+        err |= swc->ops->remove_breakpoint(cpu, type, addr, len);
+    }
+    return err;
 }
 
 static void split_remove_all_breakpoints(CPUState *cpu)
Signature · 1 line
2.49.0
Reviewed-by on v1 31/42
#60 lore raw

Re: [RFC PATCH RESEND 31/42] accel/split: Implement remove_breakpoint()

Quoted 19 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 13 ++++++++++++-
  1 file changed, 12 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +5 -1
index 0f3d48fc68d..40cd39aea5c 100644
@@ -264,7 +264,18 @@ static int split_insert_breakpoint(CPUState *cpu, int type,
  static int split_remove_breakpoint(CPUState *cpu, int type,
                                     vaddr addr, vaddr len)
  {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    int err = 0;
+

accel/split/split-accel-ops.c static int split_insert_breakpoint(CPUState *cpu, int type, · line 281

Quoted 10 lines
accel/split/split-accel-ops.c +7 -0
+    if (hwc->ops->remove_breakpoint) {
+        err |= hwc->ops->remove_breakpoint(cpu, type, addr, len);
+    }
+    if (swc->ops->remove_breakpoint) {
+        err |= swc->ops->remove_breakpoint(cpu, type, addr, len);
+    }
+    return err;
  }
 
  static void split_remove_all_breakpoints(CPUState *cpu)

r~

#61 lore raw

RFC PATCH 32/42 accel/split: Implement insert_breakpoint()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +12 -1
index 40cd39aea5c..eb52d690c7c 100644
@@ -258,7 +258,18 @@ static int split_update_guest_debug(CPUState *cpu)
 static int split_insert_breakpoint(CPUState *cpu, int type,
                                    vaddr addr, vaddr len)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    int err = 0;
+
+    if (hwc->ops->insert_breakpoint) {
+        err |= hwc->ops->insert_breakpoint(cpu, type, addr, len);
+    }
+    if (swc->ops->insert_breakpoint) {
+        err |= swc->ops->insert_breakpoint(cpu, type, addr, len);
+    }
+    return err;
 }
 
 static int split_remove_breakpoint(CPUState *cpu, int type,
Signature · 1 line
2.49.0
Reviewed-by on v1 32/42
#62 lore raw

Re: [RFC PATCH RESEND 32/42] accel/split: Implement insert_breakpoint()

Quoted 19 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 13 ++++++++++++-
  1 file changed, 12 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +5 -1
index 40cd39aea5c..eb52d690c7c 100644
@@ -258,7 +258,18 @@ static int split_update_guest_debug(CPUState *cpu)
  static int split_insert_breakpoint(CPUState *cpu, int type,
                                     vaddr addr, vaddr len)
  {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    int err = 0;
+

accel/split/split-accel-ops.c static int split_update_guest_debug(CPUState *cpu) · line 275

Quoted 10 lines
accel/split/split-accel-ops.c +7 -0
+    if (hwc->ops->insert_breakpoint) {
+        err |= hwc->ops->insert_breakpoint(cpu, type, addr, len);
+    }
+    if (swc->ops->insert_breakpoint) {
+        err |= swc->ops->insert_breakpoint(cpu, type, addr, len);
+    }
+    return err;
  }
 
  static int split_remove_breakpoint(CPUState *cpu, int type,

r~

#63 lore raw

RFC PATCH 33/42 accel/split: Implement update_guest_debug()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel-ops.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +10 -1
index eb52d690c7c..d59e70e0d9b 100644
@@ -252,7 +252,16 @@ static void split_handle_interrupt(CPUState *cpu, int mask)
 
 static int split_update_guest_debug(CPUState *cpu)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *ac = cpu->accel->use_hw ? ACCEL_GET_CLASS(sas->hw)
+                                        : ACCEL_GET_CLASS(sas->sw);
+    int ret = 0;
+
+    if (ac->ops->update_guest_debug) {
+        ret = ac->ops->update_guest_debug(cpu);
+    }
+
+    return ret;
 }
 
 static int split_insert_breakpoint(CPUState *cpu, int type,
Signature · 1 line
2.49.0
Reviewed-by on v1 33/42
#64 lore raw

Re: [RFC PATCH RESEND 33/42] accel/split: Implement update_guest_debug()

Quoted 17 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel-ops.c | 11 ++++++++++-
  1 file changed, 10 insertions(+), 1 deletion(-)
accel/split/split-accel-ops.c +3 -1
index eb52d690c7c..d59e70e0d9b 100644
@@ -252,7 +252,16 @@ static void split_handle_interrupt(CPUState *cpu, int mask)
 
  static int split_update_guest_debug(CPUState *cpu)
  {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *ac = cpu->accel->use_hw ? ACCEL_GET_CLASS(sas->hw)
+                                        : ACCEL_GET_CLASS(sas->sw);

accel/split/split-accel-ops.c static void split_handle_interrupt(CPUState *cpu, int mask) · line 267

Quoted 10 lines
accel/split/split-accel-ops.c +7 -0
+    int ret = 0;
+
+    if (ac->ops->update_guest_debug) {
+        ret = ac->ops->update_guest_debug(cpu);
+    }
+
+    return ret;
  }
 
  static int split_insert_breakpoint(CPUState *cpu, int type,

r~

#65 lore raw

RFC PATCH 34/42 accel/split: Implement get_[vcpu]_stats()

In reply to Philippe Mathieu-Daudé
---
 accel/split/split-accel.h     |  5 +++++
 accel/split/split-accel-ops.c | 24 +++++++++++++++++++++++-
 accel/split/split-all.c       | 16 +++++++++++++++-
 3 files changed, 43 insertions(+), 2 deletions(-)
accel/split/split-accel.h +5 -0
index 8a2888507f3..80a19bbb1e5 100644
@@ -36,6 +36,11 @@ struct SplitAccelState {
 
     char *hw_name;
     char *sw_name;
+
+    struct {
+        unsigned transitions; /* number of HW <-> SW transitions */
+        uint64_t exec_count[2]; /* SW:0 HW:1 */
+    } stats;
 };
 
 struct AccelCPUState {
accel/split/split-accel-ops.c +23 -1
index d59e70e0d9b..615faf1d96b 100644
@@ -9,6 +9,7 @@
 #include "qemu/osdep.h"
 #include "qemu/main-loop.h"
 #include "qemu/guest-random.h"
+#include "qapi/type-helpers.h"
 #include "exec/cpu-common.h"
 #include "exec/cpu-interrupt.h"
 #include "hw/core/cpu.h"
@@ -16,6 +17,7 @@
 #include "system/cpus.h"
 #include "system/hw_accel.h"
 #include "system/tcg.h"
+#include "monitor/monitor.h"
 #include "split-accel.h"
 #include "trace.h"
 
@@ -70,6 +72,7 @@ static void *split_cpu_thread_routine(void *arg)
                 r = swops->exec_vcpu_thread(cpu);
                 trace_accel_split_exec_vcpu_thread_sw(r);
             }
+            sas->stats.exec_count[acs->use_hw]++;
             switch (r) {
             case 0:
                 if (acs->use_hw) {
@@ -103,6 +106,7 @@ static void *split_cpu_thread_routine(void *arg)
                 assert(!acs->use_hw);
                 trace_accel_split_hw_accelerate();
                 acs->use_hw = true;
+                sas->stats.transitions++;
                 break;
             default:
                 /* Ignore everything else? */
@@ -314,7 +318,25 @@ static void split_remove_all_breakpoints(CPUState *cpu)
 
 static void split_get_vcpu_stats(CPUState *cpu, GString *buf)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(cpu->accel->accel);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+    g_autofree char *cpu_model =
+        cpu_model_from_type(object_get_typename(OBJECT(cpu)));
+
+    if (cpu->cluster_index != -1) {
+        g_string_append_printf(buf, "CPU#%d@%d: %s\n",
+                               cpu->cpu_index, cpu->cluster_index, cpu_model);
+    } else {
+        g_string_append_printf(buf, "CPU#%d: %s\n",
+                               cpu->cpu_index, cpu_model);
+    }
+    if (hwc->ops->get_vcpu_stats) {
+        hwc->ops->get_vcpu_stats(cpu, buf);
+    }
+    if (swc->ops->get_vcpu_stats) {
+        swc->ops->get_vcpu_stats(cpu, buf);
+    }
 }
 
 static void split_accel_ops_class_init(ObjectClass *oc, const void *data)
accel/split/split-all.c +15 -1
index 413954af96c..9e0848db971 100644
@@ -94,7 +94,21 @@ static int split_gdbstub_supported_sstep_flags(AccelState *as)
 
 static void split_get_stats(AccelState *as, GString *buf)
 {
-    g_assert_not_reached();
+    SplitAccelState *sas = SPLIT_ACCEL(as);
+    AccelClass *hwc = ACCEL_GET_CLASS(sas->hw);
+    AccelClass *swc = ACCEL_GET_CLASS(sas->sw);
+
+    g_string_append_printf(buf,
+                           "%u transitions, %llu SW and %llu HW executions\n",
+                           sas->stats.transitions,
+                           sas->stats.exec_count[0],
+                           sas->stats.exec_count[1]);
+    if (hwc->get_stats) {
+        hwc->get_stats(as, buf);
+    }
+    if (swc->get_stats) {
+        swc->get_stats(as, buf);
+    }
 }
 
 static char *split_get_hw(Object *obj, Error **errp)
Signature · 1 line
2.49.0
Reviewed-by on v1 34/42
#66 lore raw

Re: [RFC PATCH RESEND 34/42] accel/split: Implement get_[vcpu]_stats()

Quoted 52 lines · Philippe Mathieu-Daudé
---
  accel/split/split-accel.h     |  5 +++++
  accel/split/split-accel-ops.c | 24 +++++++++++++++++++++++-
  accel/split/split-all.c       | 16 +++++++++++++++-
  3 files changed, 43 insertions(+), 2 deletions(-)
accel/split/split-accel.h +5 -0
index 8a2888507f3..80a19bbb1e5 100644
@@ -36,6 +36,11 @@ struct SplitAccelState {
 
      char *hw_name;
      char *sw_name;
+
+    struct {
+        unsigned transitions; /* number of HW <-> SW transitions */
+        uint64_t exec_count[2]; /* SW:0 HW:1 */
+    } stats;
  };
 
  struct AccelCPUState {
accel/split/split-accel-ops.c +3 -0
index d59e70e0d9b..615faf1d96b 100644
@@ -9,6 +9,7 @@
  #include "qemu/osdep.h"
  #include "qemu/main-loop.h"
  #include "qemu/guest-random.h"
+#include "qapi/type-helpers.h"
  #include "exec/cpu-common.h"
  #include "exec/cpu-interrupt.h"
  #include "hw/core/cpu.h"
@@ -16,6 +17,7 @@
  #include "system/cpus.h"
  #include "system/hw_accel.h"
  #include "system/tcg.h"
+#include "monitor/monitor.h"
  #include "split-accel.h"
  #include "trace.h"
 
@@ -70,6 +72,7 @@ static void *split_cpu_thread_routine(void *arg)
                  r = swops->exec_vcpu_thread(cpu);
                  trace_accel_split_exec_vcpu_thread_sw(r);
              }
+            sas->stats.exec_count[acs->use_hw]++;
              switch (r) {
              case 0:
                  if (acs->use_hw) {

accel/split/split-accel-ops.c static void *split_cpu_thread_routine(void *arg) · line 109

Quoted 4 lines
accel/split/split-accel-ops.c +1 -0
@@ -103,6 +106,7 @@ static void *split_cpu_thread_routine(void *arg)
                  assert(!acs->use_hw);
                  trace_accel_split_hw_accelerate();
                  acs->use_hw = true;
+                sas->stats.transitions++;

There's also the transitions in case 0.
Otherwise,

r~

#67 lore raw

RFC PATCH 35/42 target/arm: Emulate EL2 under TCG

In reply to Philippe Mathieu-Daudé

When using hardware acceleration and TCG is available, expose EL2 and EL3 features as available (they will be emulated).

---
 target/arm/cpu64.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
target/arm/cpu64.c +5 -2
index 200da1c489b..b5c2796f3a1 100644
@@ -738,20 +738,23 @@ static void aarch64_a53_initfn(Object *obj)
 
 static void aarch64_host_initfn(Object *obj)
 {
-#if defined(CONFIG_KVM)
     ARMCPU *cpu = ARM_CPU(obj);
+#if defined(CONFIG_KVM)
     kvm_arm_set_cpu_features_from_host(cpu);
     if (arm_feature(&cpu->env, ARM_FEATURE_AARCH64)) {
         aarch64_add_sve_properties(obj);
         aarch64_add_pauth_properties(obj);
     }
 #elif defined(CONFIG_HVF)
-    ARMCPU *cpu = ARM_CPU(obj);
     hvf_arm_set_cpu_features_from_host(cpu);
     aarch64_add_pauth_properties(obj);
 #else
     g_assert_not_reached();
 #endif
+    if (tcg_enabled()) {
+        set_feature(&cpu->env, ARM_FEATURE_EL2);
+        set_feature(&cpu->env, ARM_FEATURE_EL3);
+    }
 }
 
 static void aarch64_max_initfn(Object *obj)
Signature · 1 line
2.49.0
Reviewed-by on v1 35/42
#68 lore raw

Re: [RFC PATCH RESEND 35/42] target/arm: Emulate EL2 under TCG

Quoted 29 lines · Philippe Mathieu-Daudé

When using hardware acceleration and TCG is available, expose EL2 and EL3 features as available (they will be emulated).

---
  target/arm/cpu64.c | 7 +++++--
  1 file changed, 5 insertions(+), 2 deletions(-)
target/arm/cpu64.c +1 -2
index 200da1c489b..b5c2796f3a1 100644
@@ -738,20 +738,23 @@ static void aarch64_a53_initfn(Object *obj)
 
  static void aarch64_host_initfn(Object *obj)
  {
-#if defined(CONFIG_KVM)
      ARMCPU *cpu = ARM_CPU(obj);
+#if defined(CONFIG_KVM)
      kvm_arm_set_cpu_features_from_host(cpu);
      if (arm_feature(&cpu->env, ARM_FEATURE_AARCH64)) {
          aarch64_add_sve_properties(obj);
          aarch64_add_pauth_properties(obj);
      }
  #elif defined(CONFIG_HVF)
-    ARMCPU *cpu = ARM_CPU(obj);
      hvf_arm_set_cpu_features_from_host(cpu);
      aarch64_add_pauth_properties(obj);

target/arm/cpu64.c static void aarch64_a53_initfn(Object *obj) · line 760

Quoted 10 lines
target/arm/cpu64.c +4 -0
  #else
      g_assert_not_reached();
  #endif
+    if (tcg_enabled()) {
+        set_feature(&cpu->env, ARM_FEATURE_EL2);
+        set_feature(&cpu->env, ARM_FEATURE_EL3);
+    }
  }
 
  static void aarch64_max_initfn(Object *obj)

r~

#69 lore raw

RFC PATCH 36/42 target/arm: Have ERET switch to hw accel for EL0/EL1

In reply to Philippe Mathieu-Daudé

When hardware accelerator available, use it to
directly run EL0 and EL1.

---
 target/arm/tcg/helper-a64.c | 9 +++++++++
 1 file changed, 9 insertions(+)
target/arm/tcg/helper-a64.c +9 -0
index 4f618ae390e..5962cff4e7a 100644
@@ -29,6 +29,7 @@
 #include "internals.h"
 #include "qemu/crc32c.h"
 #include "exec/cpu-common.h"
+#include "system/hw_accel.h"
 #include "accel/tcg/cpu-ldst.h"
 #include "accel/tcg/helper-retaddr.h"
 #include "accel/tcg/probe.h"
@@ -761,6 +762,14 @@ void HELPER(exception_return)(CPUARMState *env, uint64_t new_pc)
     arm_call_el_change_hook(cpu);
     bql_unlock();
 
+    if (new_el < 2 && hwaccel_enabled()) {
+        CPUState *cs = env_cpu(env);
+
+        cs->exception_index = EXCP_HWACCEL;
+        cs->exit_request = 1;
+        cpu_loop_exit(cs);
+        /* never returns */
+    }
     return;
 
 illegal_return:
Signature · 1 line
2.49.0
#70 lore raw

Re: [RFC PATCH RESEND 36/42] target/arm: Have ERET switch to hw accel for EL0/EL1

Quoted 21 lines · Philippe Mathieu-Daudé

When hardware accelerator available, use it to
directly run EL0 and EL1.

---
  target/arm/tcg/helper-a64.c | 9 +++++++++
  1 file changed, 9 insertions(+)
target/arm/tcg/helper-a64.c +1 -0
index 4f618ae390e..5962cff4e7a 100644
@@ -29,6 +29,7 @@
  #include "internals.h"
  #include "qemu/crc32c.h"
  #include "exec/cpu-common.h"
+#include "system/hw_accel.h"
  #include "accel/tcg/cpu-ldst.h"
  #include "accel/tcg/helper-retaddr.h"
  #include "accel/tcg/probe.h"

target/arm/tcg/helper-a64.c void HELPER(exception_return)(CPUARMState *env, uint64_t new_pc) · line 765

Quoted 4 lines
target/arm/tcg/helper-a64.c +1 -0
@@ -761,6 +762,14 @@ void HELPER(exception_return)(CPUARMState *env, uint64_t new_pc)
      arm_call_el_change_hook(cpu);
      bql_unlock();
 
+    if (new_el < 2 && hwaccel_enabled()) {

Because you're raising EXCP_HWACCEL, I really think this should be

  if (split_enabled() && arm_cpu_can_accelerate(cs))

r~

Quoted 10 lines
target/arm/tcg/helper-a64.c +7 -0
+        CPUState *cs = env_cpu(env);
+
+        cs->exception_index = EXCP_HWACCEL;
+        cs->exit_request = 1;
+        cpu_loop_exit(cs);
+        /* never returns */
+    }
      return;
 
  illegal_return:
#71 lore raw

RFC PATCH 37/42 accel/hvf: Emulate HVC at EL2

In reply to Philippe Mathieu-Daudé
---
 target/arm/hvf/hvf.c | 6 ++++++
 1 file changed, 6 insertions(+)
target/arm/hvf/hvf.c +6 -0
index 3907ea8791e..a4f823f834d 100644
@@ -17,6 +17,7 @@
 #include "system/hvf.h"
 #include "system/hvf_int.h"
 #include "system/hw_accel.h"
+#include "system/tcg.h"
 #include "hvf_arm.h"
 #include "cpregs.h"
 
@@ -1117,6 +1118,8 @@ static void hvf_raise_exception(CPUState *cpu, uint32_t excp,
     env->exception.syndrome = syndrome;
 
     arm_cpu_do_interrupt(cpu);
+    cpu->interrupt_request &= ~CPU_INTERRUPT_EXITTB;
+    cpu->exception_index = -1;
 }
 
 static void hvf_psci_cpu_off(ARMCPU *arm_cpu)
@@ -2090,6 +2093,9 @@ int hvf_vcpu_exec(CPUState *cpu)
                 /* SMCCC 1.3 section 5.2 says every unknown SMCCC call returns -1 */
                 env->xregs[0] = -1;
             }
+        } else if (tcg_enabled()) {
+            /* FIXME check we are called from EL1 (so EL0 -> UDEF) ? */
+            hvf_raise_exception(cpu, EXCP_HVC, syn_aa64_hvc(0), 2);
         } else {
             trace_hvf_unknown_hvc(env->pc, env->xregs[0]);
             hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized(), 1);
Signature · 1 line
2.49.0
#72 lore raw

Re: [RFC PATCH RESEND 37/42] accel/hvf: Emulate HVC at EL2

Quoted 18 lines · Philippe Mathieu-Daudé
---
  target/arm/hvf/hvf.c | 6 ++++++
  1 file changed, 6 insertions(+)
target/arm/hvf/hvf.c +1 -0
index 3907ea8791e..a4f823f834d 100644
@@ -17,6 +17,7 @@
  #include "system/hvf.h"
  #include "system/hvf_int.h"
  #include "system/hw_accel.h"
+#include "system/tcg.h"
  #include "hvf_arm.h"
  #include "cpregs.h"
 

target/arm/hvf/hvf.c static void hvf_raise_exception(CPUState *cpu, uint32_t excp, · line 1121

Quoted 4 lines
target/arm/hvf/hvf.c +1 -0
@@ -1117,6 +1118,8 @@ static void hvf_raise_exception(CPUState *cpu, uint32_t excp,
      env->exception.syndrome = syndrome;
 
      arm_cpu_do_interrupt(cpu);
+    cpu->interrupt_request &= ~CPU_INTERRUPT_EXITTB;

Why is this needed?

r~

Quoted 14 lines
target/arm/hvf/hvf.c +4 -0
+    cpu->exception_index = -1;
  }
 
  static void hvf_psci_cpu_off(ARMCPU *arm_cpu)
@@ -2090,6 +2093,9 @@ int hvf_vcpu_exec(CPUState *cpu)
                  /* SMCCC 1.3 section 5.2 says every unknown SMCCC call returns -1 */
                  env->xregs[0] = -1;
              }
+        } else if (tcg_enabled()) {
+            /* FIXME check we are called from EL1 (so EL0 -> UDEF) ? */
+            hvf_raise_exception(cpu, EXCP_HVC, syn_aa64_hvc(0), 2);
          } else {
              trace_hvf_unknown_hvc(env->pc, env->xregs[0]);
              hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized(), 1);
#73 lore raw

Re: [RFC PATCH RESEND 37/42] accel/hvf: Emulate HVC at EL2

Quoted 16 lines · Richard Henderson
Quoted 15 lines · Philippe Mathieu-Daudé

---
  target/arm/hvf/hvf.c | 6 ++++++

  1 file changed, 6 insertions(+)
target/arm/hvf/hvf.c +1 -0
index 3907ea8791e..a4f823f834d 100644
@@ -17,6 +17,7 @@
   #include "system/hvf.h"
   #include "system/hvf_int.h"
   #include "system/hw_accel.h"
+#include "system/tcg.h"
   #include "hvf_arm.h"
Quoted 8 lines

target/arm/hvf/hvf.c static void hvf_raise_exception(CPUState *cpu, · line 1121

Quoted 6 lines
target/arm/hvf/hvf.c +1 -0
   #include "cpregs.h"
@@ -1117,6 +1118,8 @@ static void hvf_raise_exception(CPUState *cpu, 
 uint32_t excp,
       env->exception.syndrome = syndrome;
       arm_cpu_do_interrupt(cpu);
+    cpu->interrupt_request &= ~CPU_INTERRUPT_EXITTB;

Why is this needed?

Left-over from previous intents, indeed unneeded.

Quoted 11 lines

r~

target/arm/hvf/hvf.c int hvf_vcpu_exec(CPUState *cpu) · line 2096

Quoted 8 lines
target/arm/hvf/hvf.c +2 -0
+    cpu->exception_index = -1;
   }
   static void hvf_psci_cpu_off(ARMCPU *arm_cpu)
@@ -2090,6 +2093,9 @@ int hvf_vcpu_exec(CPUState *cpu)
                   /* SMCCC 1.3 section 5.2 says every unknown SMCCC call returns -1 */
                   env->xregs[0] = -1;
               }
+        } else if (tcg_enabled()) {

As you noted you last night during an epic debugging session, since HVF doesn't execute the instructions but we get here being trapped, if we are going to resume under TCG we need to skip the HVC call (otherwise we keep looping over it) with:

                env->pc += 4;

(Same applies to "Emulate SMC at EL3" patch).

Quoted 6 lines
Quoted 5 lines
target/arm/hvf/hvf.c +2 -0
+            /* FIXME check we are called from EL1 (so EL0 -> UDEF) ? */
+            hvf_raise_exception(cpu, EXCP_HVC, syn_aa64_hvc(0), 2);
           } else {
               trace_hvf_unknown_hvc(env->pc, env->xregs[0]);
               hvf_raise_exception(cpu, EXCP_UDEF, syn_uncategorized(), 1);
#74 lore raw

RFC PATCH 38/42 accel/tcg: Introduce TCGCPUOps::rebuild_tb_hflags handler

In reply to Philippe Mathieu-Daudé

In order to allow rebuilding target specific TB flags, introduce tcg_rebuild_tb_flags() which dispatches to a TCGCPUOps handler.

---
 accel/tcg/internal-common.h | 1 +
 include/accel/tcg/cpu-ops.h | 2 ++
 include/system/accel-ops.h  | 8 ++++++++
 accel/tcg/cpu-exec.c        | 9 +++++++++
 accel/tcg/tcg-accel-ops.c   | 1 +
 5 files changed, 21 insertions(+)
accel/tcg/internal-common.h +1 -0
index fb265d0cefa..a3828e34145 100644
@@ -54,6 +54,7 @@ void tb_reset_jump(TranslationBlock *tb, int n);
 TranslationBlock *tb_link_page(TranslationBlock *tb);
 void cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
                                uintptr_t host_pc);
+void tcg_rebuild_tb_flags(CPUState *cpu);
 
 /**
  * tlb_init - initialize a CPU's TLB
include/accel/tcg/cpu-ops.h +2 -0
index dd8ea300168..bb047461973 100644
@@ -67,6 +67,8 @@ struct TCGCPUOps {
      * Fill in all data required to select or compile a TranslationBlock.
      */
     TCGTBCPUState (*get_tb_cpu_state)(CPUState *cs);
+    /** @rebuild_tb_hflags: Callback to rebuild TB hflags.  */
+    void (*rebuild_tb_hflags)(CPUState *cpu);
     /**
      * @synchronize_from_tb: Synchronize state from a TCG #TranslationBlock
      *
include/system/accel-ops.h +8 -0
index 29ebcf45928..20999033c89 100644
@@ -71,6 +71,14 @@ struct AccelOpsClass {
     void (*synchronize_pre_loadvm)(CPUState *cpu);
     void (*synchronize_pre_resume)(bool step_pending);
 
+    /**
+     * rebuild_tcg_tb_flags:
+     *
+     * Used to rebuild TCG TB flags when a hardware accelerator transitions
+     * to TCG, prior to calling TCG %exec_vcpu_thread() handler.
+     */
+    void (*rebuild_tcg_tb_flags)(CPUState *cpu);
+
     void (*handle_interrupt)(CPUState *cpu, int mask);
 
     void (*get_vcpu_stats)(CPUState *cpu, GString *buf);
accel/tcg/cpu-exec.c +9 -0
index 713bdb20564..7ded765889c 100644
@@ -1028,6 +1028,15 @@ int cpu_exec(CPUState *cpu)
     return ret;
 }
 
+void tcg_rebuild_tb_flags(CPUState *cpu)
+{
+    const TCGCPUOps *tcg_ops = cpu->cc->tcg_ops;
+
+    if (tcg_ops->rebuild_tb_hflags) {
+        tcg_ops->rebuild_tb_hflags(cpu);
+    }
+}
+
 bool tcg_exec_realizefn(CPUState *cpu, Error **errp)
 {
     static bool tcg_target_initialized;
accel/tcg/tcg-accel-ops.c +1 -0
index 0e4ef548f99..a141c4702e4 100644
@@ -242,6 +242,7 @@ static void tcg_accel_ops_init(AccelClass *ac)
     ops->insert_breakpoint = tcg_insert_breakpoint;
     ops->remove_breakpoint = tcg_remove_breakpoint;
     ops->remove_all_breakpoints = tcg_remove_all_breakpoints;
+    ops->rebuild_tcg_tb_flags = tcg_rebuild_tb_flags;
 }
 
 static void tcg_accel_ops_class_init(ObjectClass *oc, const void *data)
Signature · 1 line
2.49.0
#75 lore raw

Re: [RFC PATCH RESEND 38/42] accel/tcg: Introduce TCGCPUOps::rebuild_tb_hflags handler

Quoted 4 lines · Philippe Mathieu-Daudé

index 29ebcf45928..20999033c89 100644

include/system/accel-ops.h struct AccelOpsClass { · line 80

Quoted 10 lines
include/system/accel-ops.h +7 -0
@@ -71,6 +71,14 @@ struct AccelOpsClass {
      void (*synchronize_pre_loadvm)(CPUState *cpu);
      void (*synchronize_pre_resume)(bool step_pending);
 
+    /**
+     * rebuild_tcg_tb_flags:
+     *
+     * Used to rebuild TCG TB flags when a hardware accelerator transitions
+     * to TCG, prior to calling TCG %exec_vcpu_thread() handler.
+     */
+    void (*rebuild_tcg_tb_flags)(CPUState *cpu);

I think this isn't right. This should be part of synchronize between hw/sw.

r~

#76 lore raw

RFC PATCH 39/42 target/arm: Implement TCGCPUOps::rebuild_tb_hflags()

In reply to Philippe Mathieu-Daudé
---
 target/arm/cpu.h         | 2 ++
 target/arm/cpu.c         | 1 +
 target/arm/tcg/cpu-v7m.c | 1 +
 target/arm/tcg/hflags.c  | 5 +++++
 4 files changed, 9 insertions(+)
target/arm/cpu.h +2 -0
index c31f69912b8..b703ec7edc9 100644
@@ -3160,9 +3160,11 @@ void arm_register_el_change_hook(ARMCPU *cpu, ARMELChangeHookFn *hook, void
 
 /**
  * arm_rebuild_hflags:
+ * arm_cpu_rebuild_hflags:
  * Rebuild the cached TBFLAGS for arbitrary changed processor state.
  */
 void arm_rebuild_hflags(CPUARMState *env);
+void arm_cpu_rebuild_hflags(CPUState *cpu);
 
 /**
  * aa32_vfp_dreg:
target/arm/cpu.c +1 -0
index 1a19e5cfb45..32a2e6c75e1 100644
@@ -2743,6 +2743,7 @@ static const TCGCPUOps arm_tcg_ops = {
 
     .initialize = arm_translate_init,
     .translate_code = arm_translate_code,
+    .rebuild_tb_hflags = arm_cpu_rebuild_hflags,
     .get_tb_cpu_state = arm_get_tb_cpu_state,
     .synchronize_from_tb = arm_cpu_synchronize_from_tb,
     .debug_excp_handler = arm_debug_excp_handler,
target/arm/tcg/cpu-v7m.c +1 -0
index 8e1a083b911..fa17029c65f 100644
@@ -238,6 +238,7 @@ static const TCGCPUOps arm_v7m_tcg_ops = {
 
     .initialize = arm_translate_init,
     .translate_code = arm_translate_code,
+    .rebuild_tb_hflags = arm_cpu_rebuild_hflags,
     .get_tb_cpu_state = arm_get_tb_cpu_state,
     .synchronize_from_tb = arm_cpu_synchronize_from_tb,
     .debug_excp_handler = arm_debug_excp_handler,
target/arm/tcg/hflags.c +5 -0
index 1ccec63bbd4..ea1174c661c 100644
@@ -453,6 +453,11 @@ void arm_rebuild_hflags(CPUARMState *env)
     env->hflags = rebuild_hflags_internal(env);
 }
 
+void arm_cpu_rebuild_hflags(CPUState *cpu)
+{
+    arm_rebuild_hflags(cpu_env(cpu));
+}
+
 /*
  * If we have triggered a EL state change we can't rely on the
  * translator having passed it to us, we need to recompute.
Signature · 1 line
2.49.0
#77 lore raw

RFC PATCH 40/42 accel/split: Call TCGCPUOps::rebuild_tb_hflags()

In reply to Philippe Mathieu-Daudé

Call TCG rebuild_tb_hflags() when transitioning from hardware accelerator to TCG.

---
 accel/split/split-accel-ops.c | 3 +++
 1 file changed, 3 insertions(+)
accel/split/split-accel-ops.c +3 -0
index 615faf1d96b..4b058034252 100644
@@ -40,6 +40,8 @@ static void *split_cpu_thread_routine(void *arg)
     assert(swc->allowed);
     assert(hwc->allowed);
 
+    assert(swops->rebuild_tcg_tb_flags);
+
     rcu_register_thread();
     sw_force_rcu = mttcg_vcpu_register(cpu);
 
@@ -69,6 +71,7 @@ static void *split_cpu_thread_routine(void *arg)
                 r = hwops->exec_vcpu_thread(cpu);
                 trace_accel_split_exec_vcpu_thread_hw(r);
             } else {
+                swops->rebuild_tcg_tb_flags(cpu);
                 r = swops->exec_vcpu_thread(cpu);
                 trace_accel_split_exec_vcpu_thread_sw(r);
             }
Signature · 1 line
2.49.0
#78 lore raw

Re: [RFC PATCH RESEND 40/42] accel/split: Call TCGCPUOps::rebuild_tb_hflags()

Quoted 22 lines · Philippe Mathieu-Daudé

Call TCG rebuild_tb_hflags() when transitioning from hardware accelerator to TCG.

---
  accel/split/split-accel-ops.c | 3 +++
  1 file changed, 3 insertions(+)
accel/split/split-accel-ops.c +2 -0
index 615faf1d96b..4b058034252 100644
@@ -40,6 +40,8 @@ static void *split_cpu_thread_routine(void *arg)
      assert(swc->allowed);
      assert(hwc->allowed);
 
+    assert(swops->rebuild_tcg_tb_flags);
+
      rcu_register_thread();
      sw_force_rcu = mttcg_vcpu_register(cpu);
 

accel/split/split-accel-ops.c static void *split_cpu_thread_routine(void *arg) · line 75

Quoted 5 lines
accel/split/split-accel-ops.c +1 -0
@@ -69,6 +71,7 @@ static void *split_cpu_thread_routine(void *arg)
                  r = hwops->exec_vcpu_thread(cpu);
                  trace_accel_split_exec_vcpu_thread_hw(r);
              } else {
+                swops->rebuild_tcg_tb_flags(cpu);
                  r = swops->exec_vcpu_thread(cpu)

You should not need to do this before every exec, only on transitions.

r~

#79 lore raw

RFC PATCH 41/42 tests/functional: Add split_available() helper

In reply to Philippe Mathieu-Daudé
---
 python/qemu/utils/__init__.py          |  3 ++-
 python/qemu/utils/accel.py             | 10 ++++++++++
 tests/functional/qemu_test/testcase.py |  4 +++-
 3 files changed, 15 insertions(+), 2 deletions(-)
python/qemu/utils/__init__.py +2 -1
index d2fe5db223c..74218b9f2c6 100644
@@ -23,7 +23,8 @@
 from typing import Optional
 
 # pylint: disable=import-error
-from .accel import hvf_available, kvm_available, list_accel, tcg_available
+from .accel import hvf_available, kvm_available, list_accel, tcg_available, \
+                   split_available
 
 
 __all__ = (
python/qemu/utils/accel.py +10 -0
index 376d1e30005..0f58998b2f7 100644
@@ -90,3 +90,13 @@ def hvf_available(qemu_bin: str) -> bool:
     @param qemu_bin (str): path to the QEMU binary
     """
     return 'hvf' in list_accel(qemu_bin)
+
+def split_available(qemu_bin: str) -> bool:
+    """
+    Check if split accelerator is available.
+
+    @param qemu_bin (str): path to the QEMU binary
+    """
+    return 'split' in list_accel(qemu_bin) \
+           and tcg_available(qemu_bin) \
+           and hvf_available(qemu_bin)
tests/functional/qemu_test/testcase.py +3 -1
index 2082c6fce43..b6a534a29f0 100644
@@ -23,7 +23,8 @@
 import uuid
 
 from qemu.machine import QEMUMachine
-from qemu.utils import hvf_available, kvm_available, tcg_available
+from qemu.utils import hvf_available, kvm_available, tcg_available, \
+                       split_available
 
 from .archive import archive_extract
 from .asset import Asset
@@ -319,6 +320,7 @@ def require_accelerator(self, accelerator):
         checker = {'tcg': tcg_available,
                    'kvm': kvm_available,
                    'hvf': hvf_available,
+                   'split': split_available,
                   }.get(accelerator)
         if checker is None:
             self.skipTest("Don't know how to check for the presence "
Signature · 1 line
2.49.0
#80 lore raw

Re: [RFC PATCH RESEND 41/42] tests/functional: Add split_available() helper

python/qemu/utils/accel.py def hvf_available(qemu_bin: str) -> bool: · line 102

Quoted 3 lines · Philippe Mathieu-Daudé
python/qemu/utils/accel.py +3 -0
+    return 'split' in list_accel(qemu_bin) \
+           and tcg_available(qemu_bin) \
+           and hvf_available(qemu_bin)

Do you really need 3 checks? Kconfig gates on this already.

r~

#81 lore raw

RFC PATCH 42/42 tests/functional: Test Aarch64 virt machine with split-accelerator

In reply to Philippe Mathieu-Daudé
---
 tests/functional/meson.build                |  1 +
 tests/functional/test_aarch64_virt_split.py | 69 +++++++++++++++++++++
 2 files changed, 70 insertions(+)
 create mode 100644 tests/functional/test_aarch64_virt_split.py
tests/functional/meson.build +1 -0
index e9f19d54a27..20d18eb8f5d 100644
@@ -98,6 +98,7 @@ tests_aarch64_system_thorough = [
   'aarch64_tuxrun',
   'aarch64_virt',
   'aarch64_virt_gpu',
+  'aarch64_virt_split',
   'aarch64_xen',
   'aarch64_xlnx_versal',
   'multiprocess',
tests/functional/test_aarch64_virt_split.py new file +69 -0

69 lines not on this page · raw message

Signature · 1 line
2.49.0
#82 lore raw
In reply to Philippe Mathieu-Daudé

Cc'ing Salil

Quoted 99 lines · Philippe Mathieu-Daudé

Hi,

This RFC is a proof-of-concept we can have QEMU run both software emulator (TCG) and hardware accelerator (here HVF). Unfortunately I'm exhausted so I'll let Alex explain what this is about, otherwise I'll post the real cover letter next Monday after getting some rest.

Full work available in the split-accel-rfc-v5 tag:
https://gitlab.com/philmd/qemu/-/commits/split-accel-rfc-v5

Regards,

Phil.

Julian Armistead (1):
accel/split: Minimal stubs for split accelerator

Philippe Mathieu-Daudé (41):
accel/split: Define SplitAccelState
accel/split: Implement cpus_are_resettable() accel/split: Implement accel_init_machine() accel/split: Expose 'hw' and 'sw' properties accel/split: Empty setup_post()
accel/split: Implement supports_guest_debug() accel/split: Implement gdbstub_supported_sstep_flags() accel/split: Add cpu_thread_routine() stub
accel/split: Define and allocate AccelCPUState accel/split: Register MTTCG
accel/split: Have thread routine ready to dispatch over HW/SW accel/split: Implement cpu_reset_hold()
accel/split: Implement synchronize_post_init() accel/split: Implement synchronize_pre_resume() accel/split: Implement synchronize_state()
accel/split: Implement synchronize_post_reset() accel/split: Implement synchronize_pre_loadvm() accel/split: Implement kick_vcpu_thread()
accel/split: Implement cpu_common_realize() accel/split: Set use_hw in cpu_thread_routine() and switch over accel/split: Add few trace events in cpu_thread_routine handler target/arm: Implement SysemuCPUOps::can_accelerate() handler accel/split: Implement handle_interrupt()
accel/split: Empty ops_init()
accel/split: Empty set/get_virtual_clock()
accel/split: Empty get_elapsed_ticks()
accel/split: Empty cpu_thread_is_idle()
accel/split: Kludge qemu_tcg_mttcg_enabled() accel/split: Implement remove_all_breakpoints() accel/split: Implement remove_breakpoint()
accel/split: Implement insert_breakpoint()
accel/split: Implement update_guest_debug() accel/split: Implement get_[vcpu]_stats()
target/arm: Emulate EL2 under TCG
target/arm: Have ERET switch to hw accel for EL0/EL1 accel/hvf: Emulate HVC at EL2
accel/tcg: Introduce TCGCPUOps::rebuild_tb_hflags handler target/arm: Implement TCGCPUOps::rebuild_tb_hflags() accel/split: Call TCGCPUOps::rebuild_tb_hflags() tests/functional: Add split_available() helper tests/functional: Test Aarch64 virt machine with split-accelerator

  meson.build                                 |   1 +
  accel/split/split-accel.h                   |  54 +++
  accel/split/trace.h                         |   2 +
  accel/tcg/internal-common.h                 |   1 +
  include/accel/tcg/cpu-ops.h                 |   2 +
  include/exec/cpu-common.h                   |   1 +
  include/system/accel-ops.h                  |   8 +
  target/arm/cpu.h                            |   2 +
  accel/split/split-accel-ops.c               | 381 ++++++++++++++++++++
  accel/split/split-all.c                     | 196 ++++++++++
  accel/tcg/cpu-exec.c                        |   9 +
  accel/tcg/tcg-accel-ops.c                   |   1 +
  accel/tcg/tcg-all.c                         |   1 +
  system/vl.c                                 |   4 +
  target/arm/cpu.c                            |   9 +
  target/arm/cpu64.c                          |   7 +-
  target/arm/hvf/hvf.c                        |   6 +
  target/arm/tcg/cpu-v7m.c                    |   1 +
  target/arm/tcg/helper-a64.c                 |   9 +
  target/arm/tcg/hflags.c                     |   5 +
  accel/Kconfig                               |   6 +
  accel/meson.build                           |   1 +
  accel/split/meson.build                     |   9 +
  accel/split/trace-events                    |   9 +
  python/qemu/utils/__init__.py               |   3 +-
  python/qemu/utils/accel.py                  |  10 +
  tests/functional/meson.build                |   1 +
  tests/functional/qemu_test/testcase.py      |   4 +-
  tests/functional/test_aarch64_virt_split.py |  69 ++++
  29 files changed, 808 insertions(+), 4 deletions(-)

create mode 100644 accel/split/split-accel.h
create mode 100644 accel/split/trace.h
create mode 100644 accel/split/split-accel-ops.c create mode 100644 accel/split/split-all.c
create mode 100644 accel/split/meson.build
create mode 100644 accel/split/trace-events
create mode 100644 tests/functional/test_aarch64_virt_split.py