QEMU cpus.c refactoring part2
v2 cover QEMU cpus.c refactoring part2
Motivation and higher level steps:
https://lists.gnu.org/archive/html/qemu-devel/2020-05/msg04628.html
This is based on and requires "QEMU cpus.c refactoring part1":
https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg09401.html
OPEN ITEMS:
* needs review
* requires HVF patches before this can be committed (Roman).
RFC v1 -> v2:
* split the cpus.c accelerator refactoring into 6 patches.
* other minor changes to be able to proceed step by step.
----
* Rebased on commit 255ae6e2158c743717bed76c9a2365ee4bcd326e, "replay: notify the main loop when there are no instructions"
[SPLIT into part1 and part2]
----
v6 -> v7:
* rebased changes on top of Pavel Dovgalyuk changes to dma-helpers.c "icount: make dma reads deterministic"
----
v5 -> v6:
* rebased changes on top of Emilio G. Cota changes to cpus.c "cpu: convert queued work to a QSIMPLEQ"
* keep a pointer in cpus.c instead of a copy of CpusAccel (Alex)
----
v4 -> v5: rebase on latest master
* rebased changes on top of roman series to remove one of the extra states for hvf. (Is the result now functional for HVF?)
* rebased changes on top of icount changes and fixes to icount_configure and the new shift vmstate. (Markus)
v3 -> v4:
* overall: added copyright headers to all files that were missing them (used copyright and license of the module the stuff was extracted from). For the new interface files, added SUSE LLC.
* 1/4 (move softmmu only files from root):
MAINTAINERS: moved softmmu/cpus.c to its final location (from patch 2)
* 2/4 (cpu-throttle):
MAINTAINERS (to patch 1),
copyright Fabrice Bellard and license from cpus.c
* 3/4 (cpu-timers, icount):
- MAINTAINERS: add cpu-timers.c and icount.c to Paolo
- break very long lines (patchew)
- add copyright SUSE LLC, GPLv2 to cpu-timers.h
- add copyright Fabrice Bellard and license from cpus.c to timers-state.h as it is lifted from cpus.c
- vl.c: in configure_accelerators bail out if icount_enabled() and !tcg_enabled() as qtest does not enable icount anymore.
* 4/4 (accel stuff to accel):
- add copyright SUSE LLC to files that mostly only consist of the new interface. Add whatever copyright was in the accelerator code if instead they mostly consist of accelerator code.
- change a comment to mention the result of the AccelClass experiment
- moved qtest accelerator into accel/qtest/ , make it like the others.
- rename xxx-cpus-interface to xxx-cpus (remove "interface" from names)
- rename accel_int to cpus_accel
- rename CpusAccel functions from cpu_synchronize_* to synchronize_*
--------
v2 -> v3:
* turned into a 4 patch series, adding a first patch moving softmmu code currently in top_srcdir to softmmu/
* cpu-throttle: moved to softmmu/
* cpu-timers, icount:
- moved to softmmu/
- fixed assumption of qtest_enabled() => icount_enabled() causing the failure of check-qtest-arm goal, in test-arm-mptimer.c
Fix is in hw/core/ptimer.c,
where the artificial timeout rate limit should not be applied under qtest_enabled(), in a similar way to how it is not applied for icount_enabled().
* CpuAccelInterface: no change.
--------
v1 -> v2:
* 1/3 (cpu-throttle): provide a description in the commit message
* 2/3 (cpu-timers, icount): in this v2 separate icount from cpu-timers, as icount is actually TCG-specific. Only build it under CONFIG_TCG.
To do this, qtest had to be detached from icount. To this end, a trivial global counter for qtest has been introduced.
* 3/3 (CpuAccelInterface): provided a description.
This is point 8) in that plan. The idea is to extract the unrelated parts in cpus, and register interfaces from each single accelerator to the main cpus module (cpus.c).
While doing this RFC, I noticed some assumptions about Windows being either TCG or HAX (not considering WHPX) that might need to be revisited. I added a comment there.
The thing builds successfully based on Linux cross-compilations for windows/hax, windows/whpx, and I got a good build on Darwin/hvf.
Tests run successully for tcg and kvm configurations, but did not test on windows or darwin.
Welcome your feedback and help on this,
Claudio
Claudio Fontana (6):
cpus: extract out TCG-specific code to accel/tcg
cpus: extract out qtest-specific code to accel/qtest
cpus: extract out kvm-specific code to accel/kvm
cpus: extract out hax-specific code to target/i386/
cpus: extract out whpx-specific code to target/i386/
cpus: extract out hvf-specific code to target/i386/hvf/
MAINTAINERS | 3 +-
accel/Makefile.objs | 2 +-
accel/kvm/Makefile.objs | 2 +
accel/kvm/kvm-all.c | 14 +-
accel/kvm/kvm-cpus.c | 94 +++++
accel/kvm/kvm-cpus.h | 17 +
accel/qtest/Makefile.objs | 2 +
accel/qtest/qtest-cpus.c | 104 +++++
accel/qtest/qtest-cpus.h | 17 +
accel/{ => qtest}/qtest.c | 7 +
accel/stubs/kvm-stub.c | 3 +-
accel/tcg/Makefile.objs | 1 +
accel/tcg/tcg-all.c | 12 +-
accel/tcg/tcg-cpus.c | 532 ++++++++++++++++++++++++
accel/tcg/tcg-cpus.h | 17 +
hw/core/cpu.c | 1 +
include/sysemu/cpus.h | 33 ++
include/sysemu/hw_accel.h | 57 +--
include/sysemu/kvm.h | 2 +-
softmmu/cpus.c | 917 ++++--------------------------------------
stubs/Makefile.objs | 1 +
stubs/cpu-synchronize-state.c | 15 +
target/i386/Makefile.objs | 7 +-
target/i386/hax-all.c | 6 +-
target/i386/hax-cpus.c | 85 ++++
target/i386/hax-cpus.h | 17 +
target/i386/hax-i386.h | 2 +
target/i386/hax-posix.c | 12 +
target/i386/hax-windows.c | 20 +
target/i386/hvf/Makefile.objs | 2 +-
target/i386/hvf/hvf-cpus.c | 141 +++++++
target/i386/hvf/hvf-cpus.h | 17 +
target/i386/hvf/hvf.c | 3 +
target/i386/whpx-all.c | 3 +
target/i386/whpx-cpus.c | 96 +++++
target/i386/whpx-cpus.h | 17 +
36 files changed, 1368 insertions(+), 913 deletions(-)
create mode 100644 accel/kvm/kvm-cpus.c
create mode 100644 accel/kvm/kvm-cpus.h
create mode 100644 accel/qtest/Makefile.objs
create mode 100644 accel/qtest/qtest-cpus.c
create mode 100644 accel/qtest/qtest-cpus.h
rename accel/{ => qtest}/qtest.c (86%)
create mode 100644 accel/tcg/tcg-cpus.c
create mode 100644 accel/tcg/tcg-cpus.h
create mode 100644 stubs/cpu-synchronize-state.c
create mode 100644 target/i386/hax-cpus.c
create mode 100644 target/i386/hax-cpus.h
create mode 100644 target/i386/hvf/hvf-cpus.c
create mode 100644 target/i386/hvf/hvf-cpus.h
create mode 100644 target/i386/whpx-cpus.c
create mode 100644 target/i386/whpx-cpus.hSignature · 1 line
2.16.4
RFC v2 1/6 cpus: extract out TCG-specific code to accel/tcg
TCG is the first accelerator to register a "CpusAccel" interface on initialization, providing functions for starting a vcpu, kicking a vcpu, and sychronizing state.
--- accel/tcg/Makefile.objs | 1 + accel/tcg/tcg-all.c | 12 +- accel/tcg/tcg-cpus.c | 532 ++++++++++++++++++++++++++++++++++ accel/tcg/tcg-cpus.h | 17 ++ hw/core/cpu.c | 1 + include/sysemu/cpus.h | 33 +++ include/sysemu/hw_accel.h | 57 +--- softmmu/cpus.c | 643 ++++++++---------------------------------- stubs/Makefile.objs | 1 + stubs/cpu-synchronize-state.c | 15 + 10 files changed, 732 insertions(+), 580 deletions(-) create mode 100644 accel/tcg/tcg-cpus.c create mode 100644 accel/tcg/tcg-cpus.h create mode 100644 stubs/cpu-synchronize-state.c
accel/tcg/Makefile.objs +1 -0
@@ -1,5 +1,6 @@ obj-$(CONFIG_SOFTMMU) += tcg-all.o obj-$(CONFIG_SOFTMMU) += cputlb.o +obj-$(CONFIG_SOFTMMU) += tcg-cpus.o obj-y += tcg-runtime.o tcg-runtime-gvec.o obj-y += cpu-exec.o cpu-exec-common.o translate-all.o obj-y += translator.o
accel/tcg/tcg-all.c +7 -5
@@ -24,19 +24,17 @@*/ #include "qemu/osdep.h" -#include "sysemu/accel.h" +#include "qemu-common.h" #include "sysemu/tcg.h" -#include "qom/object.h" -#include "cpu.h" -#include "sysemu/cpus.h" #include "sysemu/cpu-timers.h" -#include "qemu/main-loop.h" #include "tcg/tcg.h" #include "qapi/error.h" #include "qemu/error-report.h" #include "hw/boards.h" #include "qapi/qapi-builtin-visit.h" +#include "tcg-cpus.h" + typedef struct TCGState { AccelState parent_obj;
@@ -123,6 +121,8 @@ static void tcg_accel_instance_init(Object *obj) s->mttcg_enabled =default_mttcg_enabled (); } +bool mttcg_enabled; +static int tcg_init (MachineState *ms) {TCGState *s =TCG_STATE (current_accel ());
@@ -130,6 +130,8 @@ static int tcg_init(MachineState *ms)tcg_exec_init (s->tb_size *1024 *1024 ); cpu_interrupt_handler = tcg_handle_interrupt; mttcg_enabled = s->mttcg_enabled; +cpus_register_accel (&tcg_cpus); +return 0 ; }
accel/tcg/tcg-cpus.c new file +532 -0
532 lines not on this page · raw message
accel/tcg/tcg-cpus.h new file +17 -0
@@ -0,0 +1,17 @@ +/* +* Accelerator CPUS Interface +* +* Copyright 2020 SUSE LLC +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*/ + +#ifndef TCG_CPUS_H +#define TCG_CPUS_H + +#include "sysemu/cpus.h" + +extern CpusAccel tcg_cpus; + +#endif /* TCG_CPUS_H */
hw/core/cpu.c +1 -0
@@ -33,6 +33,7 @@#include "hw/qdev-properties.h" #include "trace-root.h" #include "qemu/plugin.h" +#include "sysemu/hw_accel.h" CPUInterruptHandler cpu_interrupt_handler;
include/sysemu/cpus.h +33 -0
@@ -4,7 +4,40 @@#include "qemu/timer.h" /* cpus.c */ + +/* CPU execution threads */ + +typedef struct CpusAccel { +void (*create_vcpu_thread)(CPUState *cpu); +void (*kick_vcpu_thread)(CPUState *cpu); + +void (*synchronize_post_reset)(CPUState *cpu); +void (*synchronize_post_init)(CPUState *cpu); +void (*synchronize_state)(CPUState *cpu); +void (*synchronize_pre_loadvm)(CPUState *cpu); +}CpusAccel ; + +/* register accel-specific cpus interface implementation */ +void cpus_register_accel (CpusAccel *i); + +/* interface available for cpus accelerator threads */ + +/* For temporary buffers for forming a name */ +#define VCPU_THREAD_NAME_SIZE 16 + +void cpus_kick_thread (CPUState *cpu); +bool cpu_work_list_empty (CPUState *cpu); +bool cpu_thread_is_idle (CPUState *cpu);bool all_cpu_threads_idle (void ); +bool cpu_can_run (CPUState *cpu); +void qemu_wait_io_event_common (CPUState *cpu); +void qemu_wait_io_event (CPUState *cpu); +void cpu_thread_signal_created (CPUState *cpu); +void cpu_thread_signal_destroyed (CPUState *cpu); +void cpu_handle_guest_debug (CPUState *cpu); + +/* end interface for cpus accelerator threads */ +bool qemu_in_vcpu_thread (void );void qemu_init_cpu_loop (void );void resume_all_vcpus (void );
include/sysemu/hw_accel.h +5 -52
66 lines not on this page · raw message
softmmu/cpus.c +120 -523
837 lines not on this page · raw message
stubs/Makefile.objs +1 -0
@@ -26,6 +26,7 @@ stub-obj-y += tpm.o stub-obj-y += trace-control.o stub-obj-y += vmstate.o stub-obj-$(CONFIG_SOFTMMU) += win32-kbd-hook.o +stub-obj-y += cpu-synchronize-state.o ####################################################################### # code used by both qemu system emulation and qemu-img
stubs/cpu-synchronize-state.c new file +15 -0
@@ -0,0 +1,15 @@ +#include "qemu/osdep.h" +#include "sysemu/hw_accel.h" + +void cpu_synchronize_state (CPUState *cpu) +{ +} +void cpu_synchronize_post_reset (CPUState *cpu) +{ +} +void cpu_synchronize_post_init (CPUState *cpu) +{ +} +void cpu_synchronize_pre_loadvm (CPUState *cpu) +{ +}
Signature · 1 line
2.16.4
Re: [RFC v2 1/6] cpus: extract out TCG-specific code to accel/tcg
accel/tcg/tcg-cpus.c line 532
Quoted 10 lines · Claudio Fontana
accel/tcg/tcg-cpus.c new file +10
-0+ +CpusAccel tcg_cpus = { + .create_vcpu_thread = tcg_start_vcpu_thread, + .kick_vcpu_thread = tcg_kick_vcpu_thread, + + .synchronize_post_reset = tcg_cpu_synchronize_noop, + .synchronize_post_init = tcg_cpu_synchronize_noop, + .synchronize_state = tcg_cpu_synchronize_noop, + .synchronize_pre_loadvm = tcg_cpu_synchronize_noop, +};
Could this struct reside in AccelClass instead, so that there's no need to register the operations explicitly? We could still cache it in a global variable in accel_init_machine, in order to avoid pointer chasing.
Thanks,
Paolo
Re: [RFC v2 1/6] cpus: extract out TCG-specific code to accel/tcg
Quoted 14 lines · Paolo Bonzini
accel/tcg/tcg-cpus.c line 532
Quoted 10 lines · Claudio Fontana
accel/tcg/tcg-cpus.c new file +10
-0+ +CpusAccel tcg_cpus = { + .create_vcpu_thread = tcg_start_vcpu_thread, + .kick_vcpu_thread = tcg_kick_vcpu_thread, + + .synchronize_post_reset = tcg_cpu_synchronize_noop, + .synchronize_post_init = tcg_cpu_synchronize_noop, + .synchronize_state = tcg_cpu_synchronize_noop, + .synchronize_pre_loadvm = tcg_cpu_synchronize_noop, +};Could this struct reside in AccelClass instead, so that there's no need to register the operations explicitly?
This I tried a few times before, you can dig the history a bit of the comments.
The result of my attempts is that you end up replacing an "explicit registration", with an explicit internal copy from a structure defined in kvm-cpus.c (for example) to the accelerator state, so you just switch one registration for another.
The result is a pile of boilerplate of no use that is just confusing for the reader.
Quoted 2 lines
We could still cache it in a
global variable inaccel_init_machine, in order to avoid pointer chasing.
Indeed, the "cache" though ends up being the only useful thing,
and it can be even more confusing for the reader to see the whole machinery in AccelClass.
Quoted 5 lines
Thanks,
Paolo
Still if you are interested to see what the result ends up being I can share it, will need some fiddling..
Ciao,
Claudio
RFC v2 2/6 cpus: extract out qtest-specific code to accel/qtest
In reply to Claudio Fontanaregister a "CpusAccel" interface for qtest as well.
---
MAINTAINERS | 2 +-
accel/Makefile.objs | 2 +-
accel/qtest/Makefile.objs | 2 +
accel/qtest/qtest-cpus.c | 104 ++++++++++++++++++++++++++++++++++++++++++++++
accel/qtest/qtest-cpus.h | 17 ++++++++
accel/{ => qtest}/qtest.c | 7 ++++
softmmu/cpus.c | 61 +--------------------------
7 files changed, 133 insertions(+), 62 deletions(-)
create mode 100644 accel/qtest/Makefile.objs
create mode 100644 accel/qtest/qtest-cpus.c
create mode 100644 accel/qtest/qtest-cpus.h
rename accel/{ => qtest}/qtest.c (86%)MAINTAINERS +1 -1
@@ -2380,7 +2380,7 @@ M: Laurent Vivier <[email protected]> R: Paolo Bonzini <[email protected]> S: Maintained F: softmmu/qtest.c -F: accel/qtest.c +F: accel/qtest/ F: tests/qtest/ X: tests/qtest/bios-tables-test-allowed-diff.h
accel/Makefile.objs +1 -1
@@ -1,5 +1,5 @@ common-obj-$(CONFIG_SOFTMMU) += accel.o -obj-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_POSIX)) += qtest.o +obj-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_POSIX)) += qtest/ obj-$(CONFIG_KVM) += kvm/ obj-$(CONFIG_TCG) += tcg/ obj-$(CONFIG_XEN) += xen/
accel/qtest/Makefile.objs new file +2 -0
@@ -0,0 +1,2 @@ +obj-y += qtest.o +obj-y += qtest-cpus.o
accel/qtest/qtest-cpus.c new file +104 -0
104 lines not on this page · raw message
accel/qtest/qtest-cpus.h new file +17 -0
@@ -0,0 +1,17 @@ +/* +* Accelerator CPUS Interface +* +* Copyright 2020 SUSE LLC +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*/ + +#ifndef QTEST_CPUS_H +#define QTEST_CPUS_H + +#include "sysemu/cpus.h" + +extern CpusAccel qtest_cpus; + +#endif /* QTEST_CPUS_H */
accel/qtest/qtest.c renamed from accel/qtest.c +7 -0
@@ -12,6 +12,7 @@*/ #include "qemu/osdep.h" +#include "qemu/rcu.h" #include "qapi/error.h" #include "qemu/module.h" #include "qemu/option.h"
@@ -20,9 +21,15 @@#include "sysemu/qtest.h" #include "sysemu/cpus.h" #include "sysemu/cpu-timers.h" +#include "qemu/guest-random.h" +#include "qemu/main-loop.h" +#include "hw/core/cpu.h" + +#include "qtest-cpus.h" static int qtest_init_accel (MachineState *ms) { +cpus_register_accel (&qtest_cpus);return 0 ; }
softmmu/cpus.c +1 -60
85 lines not on this page · raw message
Signature · 1 line
2.16.4
RFC v2 3/6 cpus: extract out kvm-specific code to accel/kvm
In reply to Claudio Fontanaregister a "CpusAccel" interface for KVM as well.
--- accel/kvm/Makefile.objs | 2 ++ accel/kvm/kvm-all.c | 14 +++++++- accel/kvm/kvm-cpus.c | 94 +++++++++++++++++++++++++++++++++++++++++++++++++ accel/kvm/kvm-cpus.h | 17 +++++++++ accel/stubs/kvm-stub.c | 3 +- include/sysemu/kvm.h | 2 +- softmmu/cpus.c | 77 ---------------------------------------- 7 files changed, 128 insertions(+), 81 deletions(-) create mode 100644 accel/kvm/kvm-cpus.c create mode 100644 accel/kvm/kvm-cpus.h
accel/kvm/Makefile.objs +2 -0
@@ -1,2 +1,4 @@ obj-y += kvm-all.o +obj-y += kvm-cpus.o + obj-$(call lnot,$(CONFIG_SEV)) += sev-stub.o
accel/kvm/kvm-all.c +13 -1
@@ -45,6 +45,9 @@#include "qapi/qapi-types-common.h" #include "qapi/qapi-visit-common.h" #include "sysemu/reset.h" +#include "qemu/guest-random.h" +#include "sysemu/hw_accel.h" +#include "kvm-cpus.h" #include "hw/boards.h"
@@ -379,7 +382,7 @@ err:return ret; } -int kvm_destroy_vcpu (CPUState *cpu) +static int do_kvm_destroy_vcpu (CPUState *cpu) {KVMState *s = kvm_state;long mmap_size;
@@ -413,6 +416,14 @@ err:return ret; } +void kvm_destroy_vcpu (CPUState *cpu) +{ +if (do_kvm_destroy_vcpu (cpu) <0 ) { +error_report ("kvm_destroy_vcpu failed" ); +exit (EXIT_FAILURE ); + } +} +static int kvm_get_vcpu (KVMState *s,unsigned long vcpu_id) {struct KVMParkedVcpu *cpu;
@@ -2232,6 +2243,7 @@ static int kvm_init(MachineState *ms)qemu_balloon_inhibit (true); } +cpus_register_accel (&kvm_cpus);return 0 ; err:
accel/kvm/kvm-cpus.c new file +94 -0
94 lines not on this page · raw message
accel/kvm/kvm-cpus.h new file +17 -0
@@ -0,0 +1,17 @@ +/* +* Accelerator CPUS Interface +* +* Copyright 2020 SUSE LLC +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*/ + +#ifndef KVM_CPUS_H +#define KVM_CPUS_H + +#include "sysemu/cpus.h" + +extern CpusAccel kvm_cpus; + +#endif /* KVM_CPUS_H */
accel/stubs/kvm-stub.c +1 -2
@@ -32,9 +32,8 @@ bool kvm_readonly_mem_allowed;bool kvm_ioeventfd_any_length_allowed;bool kvm_msi_use_devid; -int kvm_destroy_vcpu (CPUState *cpu) +void kvm_destroy_vcpu (CPUState *cpu) { -return -ENOSYS ; }int kvm_init_vcpu (CPUState *cpu)
include/sysemu/kvm.h +1 -1
@@ -218,7 +218,7 @@ int kvm_has_intx_set_mask(void);int kvm_init_vcpu (CPUState *cpu);int kvm_cpu_exec (CPUState *cpu); -int kvm_destroy_vcpu (CPUState *cpu); +void kvm_destroy_vcpu (CPUState *cpu);/** * kvm_arm_supports_user_irq
softmmu/cpus.c +0 -77
125 lines not on this page · raw message
Signature · 1 line
2.16.4
Re: [RFC v2 3/6] cpus: extract out kvm-specific code to accel/kvm
accel/kvm/kvm-cpus.c line 30
Quoted 5 lines · Claudio Fontana
accel/kvm/kvm-cpus.c new file +5
-0+static void kvm_kick_vcpu_thread (CPUState *cpu) +{ +cpus_kick_thread (cpu); +} +
I would just use cpus_kick_thread instead of wrapping it (and likewise would provide a global function for noop synchronization).
start_vcpu_thread is also a candidate for abstraction, so that kvm_start_vcpu_thread(CPUState *cpu) would be just
qemu_start_vcpu_thread(cpu, "KVM", kvm_vcpu_thread_fn);
Paolo
Re: [RFC v2 3/6] cpus: extract out kvm-specific code to accel/kvm
Ciao Paolo,
Quoted 8 lines · Paolo Bonzini
accel/kvm/kvm-cpus.c line 30
Quoted 5 lines · Claudio Fontana
accel/kvm/kvm-cpus.c new file +5
-0+static void kvm_kick_vcpu_thread (CPUState *cpu) +{ +cpus_kick_thread (cpu); +} +I would just use
cpus_kick_threadinstead of wrapping it (and likewise
Here I left the common code in cpus_kick_thread, which is just used "as is" for kvm, but for hax f.e. we have:
void hax_kick_vcpu_thread(CPUState *cpu)
{
/*
* FIXME: race condition with the exit_request check in
* hax_vcpu_hax_exec
*/
cpu->exit_request = 1;
cpus_kick_thread(cpu);
}Maybe we will need additional code that specializes the kick also for HVF?
I cannot run cpus_kick_thread for _all_ accels, because it is not good for Windows (whpx and hax-windows).
Quoted 1 line
would provide a global function for noop synchronization).
I will look into providing global noops, thanks!
Quoted 2 lines
start_vcpu_threadis also a candidate for abstraction, so that
Thanks, will look into this!
The accel that I expect will resist this will be tcg, but will try to fit it.
Quoted 7 lines
kvm_start_vcpu_thread(CPUState *cpu)would be justqemu_start_vcpu_thread(cpu, "KVM", kvm_vcpu_thread_fn);Paolo
Ciao,
Claudio
Re: [RFC v2 3/6] cpus: extract out kvm-specific code to accel/kvm
Quoted 12 lines · Claudio Fontana
Ciao Paolo,
Quoted 8 lines · Paolo Bonzini
accel/kvm/kvm-cpus.c line 30
Quoted 5 lines · Claudio Fontana
accel/kvm/kvm-cpus.c new file +5
-0+static void kvm_kick_vcpu_thread (CPUState *cpu) +{ +cpus_kick_thread (cpu); +} +I would just use
cpus_kick_threadinstead of wrapping it (and likewise
Quoted 8 lines
Here I left the common code in
cpus_kick_thread, which is just used "as is" for kvm, but for hax f.e. we have:[...]
Maybe we will need additional code that specializes the kick also for HVF?
I cannot run
cpus_kick_threadfor _all_ accels, because it is not good for Windows (whpx and hax-windows).
Yeah, I meant just initialize the function pointer with cpus_kick_thread, instead of having the wrapper, similar to the global noops.
Quoted 8 lines
Quoted 1 line
would provide a global function for noop synchronization).
I will look into providing global noops, thanks!
Quoted 1 line
start_vcpu_threadis also a candidate for abstraction, so thatThanks, will look into this!
The accel that I expect will resist this will be tcg, but will try to fit it.
Yes, no problem if some of them are different.
Paolo
Quoted 13 lines
Quoted 8 lines
kvm_start_vcpu_thread(CPUState *cpu)would be justqemu_start_vcpu_thread(cpu, "KVM", kvm_vcpu_thread_fn);Paolo
Ciao,
Claudio
RFC v2 4/6 cpus: extract out hax-specific code to target/i386/
In reply to Claudio Fontanaregister a "CpusAccel" interface for HAX as well.
--- softmmu/cpus.c | 78 ------------------------------------------- target/i386/Makefile.objs | 5 +-- target/i386/hax-all.c | 6 +++- target/i386/hax-cpus.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++ target/i386/hax-cpus.h | 17 ++++++++++ target/i386/hax-i386.h | 2 ++ target/i386/hax-posix.c | 12 +++++++ target/i386/hax-windows.c | 20 +++++++++++ 8 files changed, 144 insertions(+), 81 deletions(-) create mode 100644 target/i386/hax-cpus.c create mode 100644 target/i386/hax-cpus.h
softmmu/cpus.c +0 -78
@@ -33,7 +33,6 @@#include "exec/gdbstub.h" #include "sysemu/hw_accel.h" #include "sysemu/kvm.h" -#include "sysemu/hax.h" #include "sysemu/hvf.h" #include "sysemu/whpx.h" #include "exec/exec-all.h"
@@ -192,9 +191,6 @@ void cpu_synchronize_state(CPUState *cpu)/* accelerator already implements the CpusAccel interface */ cpus_accel->synchronize_state (cpu); } -if (hax_enabled ()) { -hax_cpu_synchronize_state (cpu); - }if (whpx_enabled ()) {whpx_cpu_synchronize_state (cpu); }
@@ -206,9 +202,6 @@ void cpu_synchronize_post_reset(CPUState *cpu)/* accelerator already implements the CpusAccel interface */ cpus_accel->synchronize_post_reset (cpu); } -if (hax_enabled ()) { -hax_cpu_synchronize_post_reset (cpu); - }if (whpx_enabled ()) {whpx_cpu_synchronize_post_reset (cpu); }
@@ -220,9 +213,6 @@ void cpu_synchronize_post_init(CPUState *cpu)/* accelerator already implements the CpusAccel interface */ cpus_accel->synchronize_post_init (cpu); } -if (hax_enabled ()) { -hax_cpu_synchronize_post_init (cpu); - }if (whpx_enabled ()) {whpx_cpu_synchronize_post_init (cpu); }
@@ -234,9 +224,6 @@ void cpu_synchronize_pre_loadvm(CPUState *cpu)/* accelerator already implements the CpusAccel interface */ cpus_accel->synchronize_pre_loadvm (cpu); } -if (hax_enabled ()) { -hax_cpu_synchronize_pre_loadvm (cpu); - }if (whpx_enabled ()) {whpx_cpu_synchronize_pre_loadvm (cpu); }
@@ -409,35 +396,6 @@ void qemu_wait_io_event(CPUState *cpu)qemu_wait_io_event_common (cpu); } -static void *qemu_hax_cpu_thread_fn (void *arg) -{ -CPUState *cpu = arg; -int r; - -rcu_register_thread (); -qemu_mutex_lock_iothread (); -qemu_thread_get_self (cpu->thread); - - cpu->thread_id =qemu_get_thread_id (); - current_cpu = cpu; -hax_init_vcpu (cpu); -cpu_thread_signal_created (cpu); -qemu_guest_random_seed_thread_part2 (cpu->random_seed); - -do { -if (cpu_can_run (cpu)) { - r =hax_smp_cpu_exec (cpu); -if (r ==EXCP_DEBUG ) { -cpu_handle_guest_debug (cpu); - } - } - -qemu_wait_io_event (cpu); - }while (!cpu->unplug ||cpu_can_run (cpu)); -rcu_unregister_thread (); -return NULL ; -} -/* The HVF-specific vCPU thread function. This one should only run when the host * CPU supports the VMX "unrestricted guest" feature. */ static void *qemu_hvf_cpu_thread_fn (void *arg)
@@ -522,12 +480,6 @@ static void *qemu_whpx_cpu_thread_fn(void *arg)return NULL ; } -#ifdef _WIN32 -static void CALLBACK dummy_apc_func(ULONG_PTR unused) -{ -} -#endif -void cpus_kick_thread (CPUState *cpu) {#ifndef _WIN32
@@ -546,10 +498,6 @@ void cpus_kick_thread(CPUState *cpu)if (!qemu_cpu_is_self (cpu)) {if (whpx_enabled ()) {whpx_vcpu_kick (cpu); - }else if (!QueueUserAPC (dummy_apc_func, cpu->hThread,0 )) { -fprintf (stderr,"%s: QueueUserAPC failed with error %lu\n" , - __func__,GetLastError ()); -exit (1 ); } }#endif
@@ -562,13 +510,6 @@ void qemu_cpu_kick(CPUState *cpu)/* accelerator already implements the CpusAccel interface */ cpus_accel->kick_vcpu_thread (cpu); }else { -if (hax_enabled ()) { -/* -* FIXME: race condition with the exit_request check in -* hax_vcpu_hax_exec -*/ - cpu->exit_request =1 ; - }cpus_kick_thread (cpu); } }
@@ -716,23 +657,6 @@ void cpu_remove_sync(CPUState *cpu)qemu_mutex_lock_iothread (); } -static void qemu_hax_start_vcpu (CPUState *cpu) -{ -char thread_name[VCPU_THREAD_NAME_SIZE ]; - - cpu->thread =g_malloc0 (sizeof (QemuThread)); - cpu->halt_cond =g_malloc0 (sizeof (QemuCond)); -qemu_cond_init (cpu->halt_cond); - -snprintf (thread_name,VCPU_THREAD_NAME_SIZE ,"CPU %d/HAX" , - cpu->cpu_index); -qemu_thread_create (cpu->thread, thread_name, qemu_hax_cpu_thread_fn, - cpu,QEMU_THREAD_JOINABLE ); -#ifdef _WIN32 - cpu->hThread =qemu_thread_get_handle (cpu->thread); -#endif -} -static void qemu_hvf_start_vcpu (CPUState *cpu) {char thread_name[VCPU_THREAD_NAME_SIZE ];
@@ -793,8 +717,6 @@ void qemu_init_vcpu(CPUState *cpu)if (cpus_accel) {/* accelerator already implements the CpusAccel interface */ cpus_accel->create_vcpu_thread (cpu); - }else if (hax_enabled ()) { -qemu_hax_start_vcpu (cpu); }else if (hvf_enabled ()) {qemu_hvf_start_vcpu (cpu); }else if (whpx_enabled ()) {
target/i386/Makefile.objs +3 -2
@@ -9,11 +9,12 @@ obj-y += machine.o arch_memory_mapping.o arch_dump.o monitor.o obj-$(CONFIG_KVM) += kvm.o obj-$(CONFIG_HYPERV) += hyperv.o obj-$(call lnot,$(CONFIG_HYPERV)) += hyperv-stub.o +obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-cpus.o ifeq ($(CONFIG_WIN32),y) -obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-windows.o +obj-$(CONFIG_HAX) += hax-windows.o endif ifeq ($(CONFIG_POSIX),y) -obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-posix.o +obj-$(CONFIG_HAX) += hax-posix.o endif obj-$(CONFIG_HVF) += hvf/ obj-$(CONFIG_WHPX) += whpx-all.o
target/i386/hax-all.c +5 -1
@@ -32,9 +32,10 @@#include "sysemu/accel.h" #include "sysemu/reset.h" #include "sysemu/runstate.h" -#include "qemu/main-loop.h" #include "hw/boards.h" +#include "hax-cpus.h" +#define DEBUG_HAX 0#define DPRINTF (fmt, ...) \
@@ -374,6 +375,9 @@ static int hax_accel_init(MachineState *ms) !ret ?"working" :"not working" , !ret ? "fast virt" : "emulation"); } +if (ret ==0 ) { +cpus_register_accel (&hax_cpus); + }return ret; }
target/i386/hax-cpus.c new file +85 -0
85 lines not on this page · raw message
target/i386/hax-cpus.h new file +17 -0
@@ -0,0 +1,17 @@ +/* +* Accelerator CPUS Interface +* +* Copyright 2020 SUSE LLC +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*/ + +#ifndef HAX_CPUS_H +#define HAX_CPUS_H + +#include "sysemu/cpus.h" + +extern CpusAccel hax_cpus; + +#endif /* HAX_CPUS_H */
target/i386/hax-i386.h +2 -0
@@ -60,6 +60,8 @@ int hax_inject_interrupt(CPUArchState *env, int vector);struct hax_vm *hax_vm_create (struct hax_state *hax,int max_cpus);int hax_vcpu_run (struct hax_vcpu_state *vcpu);int hax_vcpu_create (int id); +void hax_kick_vcpu_thread (CPUState *cpu); +int hax_sync_vcpu_state (CPUArchState *env,struct vcpu_state_t *state,int set);int hax_sync_msr (CPUArchState *env,struct hax_msr_data *msrs,int set);
target/i386/hax-posix.c +12 -0
@@ -16,6 +16,8 @@#include "target/i386/hax-i386.h" +#include "sysemu/cpus.h" +hax_fd hax_mod_open (void ) {int fd =open ("/dev/HAX" ,O_RDWR );
@@ -292,3 +294,13 @@ int hax_inject_interrupt(CPUArchState *env, int vector)return ioctl (fd,HAX_VCPU_IOCTL_INTERRUPT , &vector); } + +void hax_kick_vcpu_thread (CPUState *cpu) +{ +/* +* FIXME: race condition with the exit_request check in +* hax_vcpu_hax_exec +*/ + cpu->exit_request =1 ; +cpus_kick_thread (cpu); +}
target/i386/hax-windows.c +20 -0
@@ -463,3 +463,23 @@ int hax_inject_interrupt(CPUArchState *env, int vector)return 0 ; } } + +static void CALLBACK dummy_apc_func(ULONG_PTR unused) +{ +} + +void hax_kick_vcpu_thread (CPUState *cpu) +{ +/* +* FIXME: race condition with the exit_request check in +* hax_vcpu_hax_exec +*/ + cpu->exit_request =1 ; +if (!qemu_cpu_is_self (cpu)) { +if (!QueueUserAPC (dummy_apc_func, cpu->hThread,0 )) { +fprintf (stderr,"%s: QueueUserAPC failed with error %lu\n" , + __func__,GetLastError ()); +exit (1 ); + } + } +}
Signature · 1 line
2.16.4
RFC v2 5/6 cpus: extract out whpx-specific code to target/i386/
In reply to Claudio Fontanaregister a "CpusAccel" interface for WHPX as well.
--- MAINTAINERS | 1 + softmmu/cpus.c | 81 --------------------------------------- target/i386/Makefile.objs | 2 +- target/i386/whpx-all.c | 3 ++ target/i386/whpx-cpus.c | 96 +++++++++++++++++++++++++++++++++++++++++++++++ target/i386/whpx-cpus.h | 17 +++++++++ 6 files changed, 118 insertions(+), 82 deletions(-) create mode 100644 target/i386/whpx-cpus.c create mode 100644 target/i386/whpx-cpus.h
MAINTAINERS +1 -0
@@ -427,6 +427,7 @@ WHPX CPUs M: Sunil Muthuswamy <[email protected]> S: Supported F: target/i386/whpx-all.c +F: target/i386/whpx-cpus.c F: target/i386/whp-dispatch.h F: accel/stubs/whpx-stub.c F: include/sysemu/whpx.h
softmmu/cpus.c +0 -81
141 lines not on this page · raw message
target/i386/Makefile.objs +1 -1
@@ -17,7 +17,7 @@ ifeq ($(CONFIG_POSIX),y) obj-$(CONFIG_HAX) += hax-posix.o endif obj-$(CONFIG_HVF) += hvf/ -obj-$(CONFIG_WHPX) += whpx-all.o +obj-$(CONFIG_WHPX) += whpx-all.o whpx-cpus.o endif obj-$(CONFIG_SEV) += sev.o obj-$(call lnot,$(CONFIG_SEV)) += sev-stub.o
target/i386/whpx-all.c +3 -0
@@ -24,6 +24,8 @@#include "migration/blocker.h" #include "whp-dispatch.h" +#include "whpx-cpus.h" +#include <WinHvPlatform.h> #include <WinHvEmulation.h>
@@ -1575,6 +1577,7 @@ static int whpx_accel_init(MachineState *ms)whpx_memory_init (); cpu_interrupt_handler = whpx_handle_interrupt; +cpus_register_accel (&whpx_cpus);printf ("Windows Hypervisor Platform accelerator is operational\n" );return 0 ;
target/i386/whpx-cpus.c new file +96 -0
96 lines not on this page · raw message
target/i386/whpx-cpus.h new file +17 -0
@@ -0,0 +1,17 @@ +/* +* Accelerator CPUS Interface +* +* Copyright 2020 SUSE LLC +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*/ + +#ifndef WHPX_CPUS_H +#define WHPX_CPUS_H + +#include "sysemu/cpus.h" + +extern CpusAccel whpx_cpus; + +#endif /* WHPX_CPUS_H */
Signature · 1 line
2.16.4
RFC v2 6/6 cpus: extract out hvf-specific code to target/i386/hvf/
In reply to Claudio Fontanaregister a "CpusAccel" interface for HVF as well.
--- softmmu/cpus.c | 75 ---------------------- target/i386/hvf/Makefile.objs | 2 +- target/i386/hvf/hvf-cpus.c | 141 ++++++++++++++++++++++++++++++++++++++++++ target/i386/hvf/hvf-cpus.h | 17 +++++ target/i386/hvf/hvf.c | 3 + 5 files changed, 162 insertions(+), 76 deletions(-) create mode 100644 target/i386/hvf/hvf-cpus.c create mode 100644 target/i386/hvf/hvf-cpus.h
softmmu/cpus.c +0 -75
@@ -33,7 +33,6 @@#include "exec/gdbstub.h" #include "sysemu/hw_accel.h" #include "sysemu/kvm.h" -#include "sysemu/hvf.h" #include "exec/exec-all.h" #include "qemu/thread.h" #include "qemu/plugin.h"
@@ -142,10 +141,6 @@ void cpu_synchronize_all_states(void)CPU_FOREACH (cpu) {cpu_synchronize_state (cpu); -/* TODO: move to cpu_synchronize_state() */ -if (hvf_enabled ()) { -hvf_cpu_synchronize_state (cpu); - } } }
@@ -155,10 +150,6 @@ void cpu_synchronize_all_post_reset(void)CPU_FOREACH (cpu) {cpu_synchronize_post_reset (cpu); -/* TODO: move to cpu_synchronize_post_reset() */ -if (hvf_enabled ()) { -hvf_cpu_synchronize_post_reset (cpu); - } } }
@@ -168,10 +159,6 @@ void cpu_synchronize_all_post_init(void)CPU_FOREACH (cpu) {cpu_synchronize_post_init (cpu); -/* TODO: move to cpu_synchronize_post_init() */ -if (hvf_enabled ()) { -hvf_cpu_synchronize_post_init (cpu); - } } }
@@ -383,48 +370,6 @@ void qemu_wait_io_event(CPUState *cpu)qemu_wait_io_event_common (cpu); } -/* The HVF-specific vCPU thread function. This one should only run when the host -* CPU supports the VMX "unrestricted guest" feature. */ -static void *qemu_hvf_cpu_thread_fn (void *arg) -{ -CPUState *cpu = arg; - -int r; - -assert (hvf_enabled ()); - -rcu_register_thread (); - -qemu_mutex_lock_iothread (); -qemu_thread_get_self (cpu->thread); - - cpu->thread_id =qemu_get_thread_id (); - cpu->can_do_io =1 ; - current_cpu = cpu; - -hvf_init_vcpu (cpu); - -/* signal CPU creation */ -cpu_thread_signal_created (cpu); -qemu_guest_random_seed_thread_part2 (cpu->random_seed); - -do { -if (cpu_can_run (cpu)) { - r =hvf_vcpu_exec (cpu); -if (r ==EXCP_DEBUG ) { -cpu_handle_guest_debug (cpu); - } - } -qemu_wait_io_event (cpu); - }while (!cpu->unplug ||cpu_can_run (cpu)); - -hvf_vcpu_destroy (cpu); -cpu_thread_signal_destroyed (cpu); -qemu_mutex_unlock_iothread (); -rcu_unregister_thread (); -return NULL ; -} -void cpus_kick_thread (CPUState *cpu) {#ifndef _WIN32
@@ -594,24 +539,6 @@ void cpu_remove_sync(CPUState *cpu)qemu_mutex_lock_iothread (); } -static void qemu_hvf_start_vcpu (CPUState *cpu) -{ -char thread_name[VCPU_THREAD_NAME_SIZE ]; - -/* HVF currently does not support TCG, and only runs in -* unrestricted-guest mode. */ -assert (hvf_enabled ()); - - cpu->thread =g_malloc0 (sizeof (QemuThread)); - cpu->halt_cond =g_malloc0 (sizeof (QemuCond)); -qemu_cond_init (cpu->halt_cond); - -snprintf (thread_name,VCPU_THREAD_NAME_SIZE ,"CPU %d/HVF" , - cpu->cpu_index); -qemu_thread_create (cpu->thread, thread_name, qemu_hvf_cpu_thread_fn, - cpu,QEMU_THREAD_JOINABLE ); -} -void cpus_register_accel (CpusAccel *ca) {assert (ca !=NULL );
@@ -638,8 +565,6 @@ void qemu_init_vcpu(CPUState *cpu)if (cpus_accel) {/* accelerator already implements the CpusAccel interface */ cpus_accel->create_vcpu_thread (cpu); - }else if (hvf_enabled ()) { -qemu_hvf_start_vcpu (cpu); }else {assert (0 ); }
target/i386/hvf/Makefile.objs +1 -1
@@ -1,2 +1,2 @@ -obj-y += hvf.o +obj-y += hvf.o hvf-cpus.o obj-y += x86.o x86_cpuid.o x86_decode.o x86_descr.o x86_emu.o x86_flags.o x86_mmu.o x86hvf.o x86_task.o
target/i386/hvf/hvf-cpus.c new file +141 -0
141 lines not on this page · raw message
target/i386/hvf/hvf-cpus.h new file +17 -0
@@ -0,0 +1,17 @@ +/* +* Accelerator CPUS Interface +* +* Copyright 2020 SUSE LLC +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*/ + +#ifndef HVF_CPUS_H +#define HVF_CPUS_H + +#include "sysemu/cpus.h" + +extern CpusAccel hvf_cpus; + +#endif /* HVF_CPUS_H */
target/i386/hvf/hvf.c +3 -0
@@ -72,6 +72,8 @@#include "sysemu/accel.h" #include "target/i386/cpu.h" +#include "hvf-cpus.h" +HVFState *hvf_state;static void assert_hvf_ok (hv_return_t ret)
@@ -977,6 +979,7 @@ static int hvf_accel_init(MachineState *ms) hvf_state = s; cpu_interrupt_handler = hvf_handle_interrupt;memory_listener_register (&hvf_memory_listener, &address_space_memory); +cpus_register_accel (&hvf_cpus);return 0 ; }
Signature · 1 line
2.16.4
Re: [RFC v2 6/6] cpus: extract out hvf-specific code to target/i386/hvf/
Quoted 9 lines · Claudio Fontana
register a "CpusAccel" interface for HVF as well.
Signed-off-by Claudio Fontana <[email protected]>--- softmmu/cpus.c | 75 ---------------------- target/i386/hvf/Makefile.objs | 2 +- target/i386/hvf/hvf-cpus.c | 141 ++++++++++++++++++++++++++++++++++++++++++ target/i386/hvf/hvf-cpus.h | 17 +++++ target/i386/hvf/hvf.c | 3 +
Given the recent Apple announcement we might want to consider making HVF more architecture agnostic - or at least the main loop bits of it.
Signature · 1 line
Alex Bennée