hvf: Implement Apple Silicon Support
v3 cover hvf: Implement Apple Silicon Support
Now that Apple Silicon is widely available, people are obviously excited to try and run virtualized workloads on them, such as Linux and Windows.
This patch set implements a fully functional version to get the ball going on that. With this applied, I can successfully run both Linux and Windows as guests. I am not aware of any limitations specific to Hypervisor.framework apart from:
- Live migration / savevm
- gdbstub debugging (SP register)
Enjoy!
Alex
v1 -> v2:
- New patch: hvf: Actually set SIG_IPI mask
- New patch: hvf: Introduce hvf vcpu struct
- New patch: hvf: arm: Mark CPU as dirty on reset
- Removed patch: hw/arm/virt: Disable highmem when on hypervisor.framework
- Removed patch: arm: Synchronize CPU on PSCI on
- Fix build on 32bit arm
- Merge vcpu kick function patch into ARM enablement
- Implement WFI handling (allows vCPUs to sleep)
- Synchronize system registers (fixes OVMF crashes and reboot)
- Don't always call cpu_synchronize_state()
- Use more fine grained iothread locking
- Populate aa64mmfr0 from hardware
- Make safe to ctrl-C entitlement application
v2 -> v3:
- Removed patch: hvf: Actually set SIG_IPI mask
- New patch: hvf: arm: Add support for GICv3
- New patch: hvf: arm: Implement -cpu host
- Advance PC on SMC
- Use cp list interface for sysreg syncs
- Do not set current_cpu
- Fix sysreg isread mask
- Move sysreg handling to functions
- Remove WFI logic again
- Revert to global iothread locking
Alexander Graf (9):
hvf: Add hypervisor entitlement to output binaries
hvf: Move common code out
hvf: Introduce hvf vcpu struct
arm: Set PSCI to 0.2 for HVF
hvf: arm: Mark CPU as dirty on reset
hvf: Add Apple Silicon support
arm: Add Hypervisor.framework build target
hvf: arm: Add support for GICv3
hvf: arm: Implement -cpu host
Peter Collingbourne (1):
arm/hvf: Add a WFI handler
MAINTAINERS | 14 +- accel/hvf/entitlements.plist | 8 + accel/hvf/hvf-all.c | 56 +++ accel/hvf/hvf-cpus.c | 481 ++++++++++++++++++++ accel/hvf/meson.build | 7 + accel/meson.build | 1 + include/hw/core/cpu.h | 3 +- include/sysemu/hvf.h | 2 + include/sysemu/hvf_int.h | 77 ++++ meson.build | 41 +- scripts/entitlement.sh | 13 + target/arm/arm-powerctl.c | 1 + target/arm/cpu.c | 15 +- target/arm/cpu.h | 2 + target/arm/hvf/hvf.c | 857 +++++++++++++++++++++++++++++++++++ target/arm/hvf/meson.build | 3 + target/arm/kvm_arm.h | 2 - target/arm/meson.build | 2 + target/i386/hvf/hvf-cpus.c | 131 ------ target/i386/hvf/hvf-cpus.h | 25 - target/i386/hvf/hvf-i386.h | 48 +- target/i386/hvf/hvf.c | 462 +++---------------- target/i386/hvf/meson.build | 1 - target/i386/hvf/vmx.h | 24 +- target/i386/hvf/x86.c | 28 +- target/i386/hvf/x86_descr.c | 26 +- target/i386/hvf/x86_emu.c | 62 +-- target/i386/hvf/x86_mmu.c | 4 +- target/i386/hvf/x86_task.c | 12 +- target/i386/hvf/x86hvf.c | 221 ++++----- target/i386/hvf/x86hvf.h | 2 - 31 files changed, 1818 insertions(+), 813 deletions(-) create mode 100644 accel/hvf/entitlements.plist create mode 100644 accel/hvf/hvf-all.c create mode 100644 accel/hvf/hvf-cpus.c create mode 100644 accel/hvf/meson.build create mode 100644 include/sysemu/hvf_int.h create mode 100755 scripts/entitlement.sh create mode 100644 target/arm/hvf/hvf.c create mode 100644 target/arm/hvf/meson.build delete mode 100644 target/i386/hvf/hvf-cpus.c delete mode 100644 target/i386/hvf/hvf-cpus.h
Signature · 1 line
2.24.3 (Apple Git-128)
PATCH v3 1/10 hvf: Add hypervisor entitlement to output binaries
In macOS 11, QEMU only gets access to Hypervisor.framework if it has the respective entitlement. Add an entitlement template and automatically self sign and apply the entitlement in the build.
---
v1 -> v2:
- Make safe to ctrl-C
--- accel/hvf/entitlements.plist | 8 ++++++++ meson.build | 30 ++++++++++++++++++++++++++---- scripts/entitlement.sh | 13 +++++++++++++ 3 files changed, 47 insertions(+), 4 deletions(-) create mode 100644 accel/hvf/entitlements.plist create mode 100755 scripts/entitlement.sh
accel/hvf/entitlements.plist new file +8 -0
@@ -0,0 +1,8 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> +<plist version="1.0"> +<dict> + <key>com.apple.security.hypervisor</key> + <true/> +</dict> +</plist>
meson.build +26 -4
@@ -1844,9 +1844,14 @@ foreach target : target_dirs }] endif foreach exe: execs - emulators += {exe['name']: - executable(exe['name'], exe['sources'], - install: true, + exe_name = exe['name'] + exe_sign = 'CONFIG_HVF' in config_target + if exe_sign + exe_name += '-unsigned' + endif + + emulator = executable(exe_name, exe['sources'], + install: not exe_sign, c_args: c_args, dependencies: arch_deps + deps + exe['dependencies'], objects: lib.extract_all_objects(recursive: true),
@@ -1854,7 +1859,24 @@ foreach target : target_dirs link_depends: [block_syms, qemu_syms] + exe.get('link_depends', []), link_args: link_args, gui_app: exe['gui']) - } + + if exe_sign + exe_full = meson.current_build_dir() / exe['name'] + emulators += {exe['name'] : custom_target(exe['name'], + install: true, + install_dir: get_option('bindir'), + depends: emulator, + output: exe['name'], + command: [ + meson.current_source_dir() / 'scripts/entitlement.sh', + meson.current_build_dir() / exe['name'] + '-unsigned', + meson.current_build_dir() / exe['name'], + meson.current_source_dir() / 'accel/hvf/entitlements.plist' + ]) + } + else + emulators += {exe['name']: emulator} + endif if 'CONFIG_TRACE_SYSTEMTAP' in config_host foreach stp: [
scripts/entitlement.sh new file +13 -0
@@ -0,0 +1,13 @@ +#!/bin/sh -e +# +# Helper script for the build process to apply entitlements + +SRC="$1" +DST="$2" +ENTITLEMENT="$3" + +trap 'rm "$DST.tmp"' exit +cp -af "$SRC" "$DST.tmp" +codesign --entitlements "$ENTITLEMENT" --force -s - "$DST.tmp" +mv "$DST.tmp" "$DST" +trap '' exit
Signature · 1 line
2.24.3 (Apple Git-128)
Re: [PATCH v3 01/10] hvf: Add hypervisor entitlement to output binaries
Quoted 56 lines · Alexander Graf
In macOS 11, QEMU only gets access to Hypervisor.framework if it has the respective entitlement. Add an entitlement template and automatically self sign and apply the entitlement in the build.
Signed-off-by Alexander Graf <[email protected]>---
v1 -> v2:
- Make safe to ctrl-C
--- accel/hvf/entitlements.plist | 8 ++++++++ meson.build | 30 ++++++++++++++++++++++++++---- scripts/entitlement.sh | 13 +++++++++++++ 3 files changed, 47 insertions(+), 4 deletions(-) create mode 100644 accel/hvf/entitlements.plist create mode 100755 scripts/entitlement.shaccel/hvf/entitlements.plist new file +8
-0@@ -0,0 +1,8 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> +<plist version="1.0"> +<dict> + <key>com.apple.security.hypervisor</key> + <true/> +</dict> +</plist>meson.build +8
-3@@ -1844,9 +1844,14 @@ foreach target : target_dirs }] endif foreach exe: execs - emulators += {exe['name']: - executable(exe['name'], exe['sources'], - install: true, + exe_name = exe['name'] + exe_sign = 'CONFIG_HVF' in config_target + if exe_sign + exe_name += '-unsigned' + endif + + emulator = executable(exe_name, exe['sources'], + install: not exe_sign, c_args: c_args, dependencies: arch_deps + deps + exe['dependencies'], objects: lib.extract_all_objects(recursive: true),
meson.build foreach target : target_dirs · line 1864
Quoted 7 lines
meson.build +3
-1@@ -1854,7 +1859,24 @@ foreach target : target_dirs link_depends: [block_syms, qemu_syms] + exe.get('link_depends', []), link_args: link_args, gui_app: exe['gui']) - } + + if exe_sign + exe_full = meson.current_build_dir() / exe['name']
It's defined but not used.
meson.build foreach target : target_dirs · line 1872
Quoted 8 lines
meson.build +8
-0+ emulators += {exe['name'] : custom_target(exe['name'], + install: true, + install_dir: get_option('bindir'), + depends: emulator, + output: exe['name'], + command: [ + meson.current_source_dir() / 'scripts/entitlement.sh', + meson.current_build_dir() / exe['name'] + '-unsigned',
exe_name might be used instead of:
exe['name'] + '-unsigned'
Thanks,
Roman
Quoted 32 lines
meson.build +7
-0+ meson.current_build_dir() / exe['name'], + meson.current_source_dir() / 'accel/hvf/entitlements.plist' + ]) + } + else + emulators += {exe['name']: emulator} + endif if 'CONFIG_TRACE_SYSTEMTAP' in config_host foreach stp: [scripts/entitlement.sh new file +13
-0@@ -0,0 +1,13 @@ +#!/bin/sh -e +# +# Helper script for the build process to apply entitlements + +SRC="$1" +DST="$2" +ENTITLEMENT="$3" + +trap 'rm "$DST.tmp"' exit +cp -af "$SRC" "$DST.tmp" +codesign --entitlements "$ENTITLEMENT" --force -s - "$DST.tmp" +mv "$DST.tmp" "$DST" +trap '' exitSignature · 1 line
2.24.3 (Apple Git-128)
PATCH v3 2/10 hvf: Move common code out
In reply to Alexander GrafUntil now, Hypervisor.framework has only been available on x86_64 systems. With Apple Silicon shipping now, it extends its reach to aarch64. To prepare for support for multiple architectures, let's move common code out into its own accel directory.
--- MAINTAINERS | 9 +- accel/hvf/hvf-all.c | 56 +++++ accel/hvf/hvf-cpus.c | 464 ++++++++++++++++++++++++++++++++++++ accel/hvf/meson.build | 7 + accel/meson.build | 1 + include/sysemu/hvf_int.h | 69 ++++++ target/i386/hvf/hvf-cpus.c | 131 ---------- target/i386/hvf/hvf-cpus.h | 25 -- target/i386/hvf/hvf-i386.h | 48 +--- target/i386/hvf/hvf.c | 360 +--------------------------- target/i386/hvf/meson.build | 1 - target/i386/hvf/x86hvf.c | 11 +- target/i386/hvf/x86hvf.h | 2 - 13 files changed, 615 insertions(+), 569 deletions(-) create mode 100644 accel/hvf/hvf-all.c create mode 100644 accel/hvf/hvf-cpus.c create mode 100644 accel/hvf/meson.build create mode 100644 include/sysemu/hvf_int.h delete mode 100644 target/i386/hvf/hvf-cpus.c delete mode 100644 target/i386/hvf/hvf-cpus.h
MAINTAINERS +8 -1
@@ -444,9 +444,16 @@ M: Cameron Esfahani <[email protected]> M: Roman Bolshakov <[email protected]> W: https://wiki.qemu.org/Features/HVF S: Maintained -F: accel/stubs/hvf-stub.c F: target/i386/hvf/ + +HVF +M: Cameron Esfahani <[email protected]> +M: Roman Bolshakov <[email protected]> +W: https://wiki.qemu.org/Features/HVF +S: Maintained +F: accel/hvf/ F: include/sysemu/hvf.h +F: include/sysemu/hvf_int.h WHPX CPUs M: Sunil Muthuswamy <[email protected]>
accel/hvf/hvf-all.c new file +56 -0
56 lines not on this page · raw message
accel/hvf/hvf-cpus.c new file +464 -0
464 lines not on this page · raw message
accel/hvf/meson.build new file +7 -0
@@ -0,0 +1,7 @@ +hvf_ss = ss.source_set() +hvf_ss.add(files( + 'hvf-all.c', + 'hvf-cpus.c', +)) + +specific_ss.add_all(when: 'CONFIG_HVF', if_true: hvf_ss)
accel/meson.build +1 -0
@@ -1,5 +1,6 @@ softmmu_ss.add(files('accel.c')) +subdir('hvf') subdir('qtest') subdir('kvm') subdir('tcg')
include/sysemu/hvf_int.h new file +69 -0
69 lines not on this page · raw message
target/i386/hvf/hvf-cpus.c deleted +0 -131
131 lines not on this page · raw message
target/i386/hvf/hvf-cpus.h deleted +0 -25
@@ -1,25 +0,0 @@ -/* -* Accelerator CPUS Interface -* -* Copyright 2020 SUSE LLC -* -* This work is licensed under the terms of the GNU GPL, version 2 or later. -* See the COPYING file in the top-level directory. -*/ - -#ifndef HVF_CPUS_H -#define HVF_CPUS_H - -#include "sysemu/cpus.h" - -extern const CpusAccel hvf_cpus; - -int hvf_init_vcpu (CPUState *); -int hvf_vcpu_exec (CPUState *); -void hvf_cpu_synchronize_state (CPUState *); -void hvf_cpu_synchronize_post_reset (CPUState *); -void hvf_cpu_synchronize_post_init (CPUState *); -void hvf_cpu_synchronize_pre_loadvm (CPUState *); -void hvf_vcpu_destroy (CPUState *); - -#endif /* HVF_CPUS_H */
target/i386/hvf/hvf-i386.h +1 -47
58 lines not on this page · raw message
target/i386/hvf/hvf.c +3 -357
404 lines not on this page · raw message
target/i386/hvf/meson.build +0 -1
@@ -1,6 +1,5 @@ i386_softmmu_ss.add(when: [hvf, 'CONFIG_HVF'], if_true: files( 'hvf.c', - 'hvf-cpus.c', 'x86.c', 'x86_cpuid.c', 'x86_decode.c',
target/i386/hvf/x86hvf.c +6 -5
@@ -20,6 +20,9 @@#include "qemu/osdep.h" #include "qemu-common.h" +#include "sysemu/hvf.h" +#include "sysemu/hvf_int.h" +#include "sysemu/hw_accel.h" #include "x86hvf.h" #include "vmx.h" #include "vmcs.h"
@@ -32,8 +35,6 @@#include <Hypervisor/hv.h> #include <Hypervisor/hv_vmx.h> -#include "hvf-cpus.h" -void hvf_set_segment (struct CPUState *cpu,struct vmx_segment *vmx_seg,SegmentCache *qseg,bool is_tr) {
@@ -437,7 +438,7 @@ int hvf_process_events(CPUState *cpu_state) env->eflags =rreg (cpu_state->hvf_fd,HV_X86_RFLAGS );if (cpu_state->interrupt_request &CPU_INTERRUPT_INIT ) { -hvf_cpu_synchronize_state (cpu_state); +cpu_synchronize_state (cpu_state);do_cpu_init (cpu); }
@@ -451,12 +452,12 @@ int hvf_process_events(CPUState *cpu_state) cpu_state->halted =0 ; }if (cpu_state->interrupt_request &CPU_INTERRUPT_SIPI ) { -hvf_cpu_synchronize_state (cpu_state); +cpu_synchronize_state (cpu_state);do_cpu_sipi (cpu); }if (cpu_state->interrupt_request &CPU_INTERRUPT_TPR ) { cpu_state->interrupt_request &= ~CPU_INTERRUPT_TPR ; -hvf_cpu_synchronize_state (cpu_state); +cpu_synchronize_state (cpu_state);apic_handle_tpr_access_report (cpu->apic_state, env->eip, env->tpr_access_type); }
target/i386/hvf/x86hvf.h +0 -2
@@ -21,8 +21,6 @@#include "x86_descr.h" int hvf_process_events (CPUState *); -int hvf_put_registers (CPUState *); -int hvf_get_registers (CPUState *);bool hvf_inject_interrupts (CPUState *);void hvf_set_segment (struct CPUState *cpu,struct vmx_segment *vmx_seg,SegmentCache *qseg,bool is_tr);
Signature · 1 line
2.24.3 (Apple Git-128)
Re: [PATCH v3 02/10] hvf: Move common code out
Quoted 17 lines · Alexander Graf
Until now, Hypervisor.framework has only been available on
x86_64systems. With Apple Silicon shipping now, it extends its reach to aarch64. To prepare for support for multiple architectures, let's move common code out into its own accel directory.[...]
include/sysemu/hvf_int.h new file +4
-0@@ -0,0 +1,69 @@ +/* +* QEMU Hypervisor.framework (HVF) support > + * +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory.
include/sysemu/hvf_int.h line 14
Quoted 10 lines
include/sysemu/hvf_int.h new file +10
-0+* +*/ + +/* header to be included in HVF-specific code */ + +#ifndef HVF_INT_H +#define HVF_INT_H + +#include <Hypervisor/Hypervisor.h> +
From here
Quoted 5 lines
include/sysemu/hvf_int.h new file +5
-0+#define HVF_MAX_VCPU 0x10 + +extern struct hvf_state hvf_global; + +struct hvf_vm {
include/sysemu/hvf_int.h line 29
Quoted 10 lines
include/sysemu/hvf_int.h new file +10
-0+int id; +struct hvf_vcpu_state *vcpus[HVF_MAX_VCPU ]; +}; + +struct hvf_state { +uint32_t version; +struct hvf_vm *vm; +uint64_t mem_quota; +}; +
and down to here, it doesn't seem to be a common code, rather unused definitions. They can be safely dropped:
target/i386/hvf/hvf-i386.h +0 -16
@@ -21,21 +21,6 @@#include "cpu.h" #include "x86.h" -#define HVF_MAX_VCPU 0x10 - -extern struct hvf_state hvf_global; - -struct hvf_vm { -int id; -struct hvf_vcpu_state *vcpus[HVF_MAX_VCPU ]; -}; - -struct hvf_state { -uint32_t version; -struct hvf_vm *vm; -uint64_t mem_quota; -}; -/* hvf_slot flags */ #define HVF_SLOT_LOG (1 << 0)
@@ -75,7 +60,6 @@ hvf_slot *hvf_find_overlap_slot(uint64_t, uint64_t);/* Host specific functions */ int hvf_inject_interrupt (CPUArchState *env,int vector); -int hvf_vcpu_run (struct hvf_vcpu_state *vcpu);#endif #endif
PATCH v3 3/10 hvf: Introduce hvf vcpu struct
In reply to Alexander GrafWe will need more than a single field for hvf going forward. To keep the global vcpu struct uncluttered, let's allocate a special hvf vcpu struct, similar to how hax does it.
--- accel/hvf/hvf-cpus.c | 8 +- include/hw/core/cpu.h | 3 +- include/sysemu/hvf_int.h | 4 + target/i386/hvf/hvf.c | 102 +++++++++--------- target/i386/hvf/vmx.h | 24 +++-- target/i386/hvf/x86.c | 28 ++--- target/i386/hvf/x86_descr.c | 26 ++--- target/i386/hvf/x86_emu.c | 62 +++++------ target/i386/hvf/x86_mmu.c | 4 +- target/i386/hvf/x86_task.c | 12 +-- target/i386/hvf/x86hvf.c | 210 ++++++++++++++++++------------------ 11 files changed, 247 insertions(+), 236 deletions(-)
accel/hvf/hvf-cpus.c +6 -2
@@ -314,10 +314,12 @@ static void hvf_cpu_synchronize_pre_loadvm(CPUState *cpu)static void hvf_vcpu_destroy (CPUState *cpu) { -hv_return_t ret =hv_vcpu_destroy (cpu->hvf_fd); +hv_return_t ret =hv_vcpu_destroy (cpu->hvf->fd);assert_hvf_ok (ret);hvf_arch_vcpu_destroy (cpu); +free (cpu->hvf); + cpu->hvf =NULL ; }static void dummy_signal (int sig)
@@ -328,6 +330,8 @@ static int hvf_init_vcpu(CPUState *cpu) {int r; + cpu->hvf =g_malloc0 (sizeof (*cpu->hvf)); +/* init cpu signals */ sigset_t set;struct sigaction sigact;
@@ -339,7 +343,7 @@ static int hvf_init_vcpu(CPUState *cpu)pthread_sigmask (SIG_BLOCK ,NULL , &set);sigdelset (&set,SIG_IPI ); - r =hv_vcpu_create ((hv_vcpuid_t *)&cpu->hvf_fd,HV_VCPU_DEFAULT ); + r =hv_vcpu_create ((hv_vcpuid_t *)&cpu->hvf->fd,HV_VCPU_DEFAULT ); cpu->vcpu_dirty =1 ;assert_hvf_ok (r);
include/hw/core/cpu.h +2 -1
@@ -280,6 +280,7 @@ struct KVMState;struct kvm_run ;struct hax_vcpu_state ; +struct hvf_vcpu_state ;#define TB_JMP_CACHE_BITS 12#define TB_JMP_CACHE_SIZE (1 << TB_JMP_CACHE_BITS)
@@ -463,7 +464,7 @@ struct CPUState {struct hax_vcpu_state *hax_vcpu; -int hvf_fd; +struct hvf_vcpu_state *hvf;/* track IOMMUs whose translations we've cached in the TCG TLB */ GArray *iommu_notifiers;
include/sysemu/hvf_int.h +4 -0
@@ -58,6 +58,10 @@ struct HVFState { };extern HVFState *hvf_state; +struct hvf_vcpu_state { +int fd; +}; +void assert_hvf_ok (hv_return_t ret);int hvf_get_registers (CPUState *cpu);int hvf_put_registers (CPUState *cpu);
target/i386/hvf/hvf.c +51 -51
221 lines not on this page · raw message
target/i386/hvf/vmx.h +13 -11
@@ -30,6 +30,8 @@#include "vmcs.h" #include "cpu.h" #include "x86.h" +#include "sysemu/hvf.h" +#include "sysemu/hvf_int.h" #include "exec/address-spaces.h"
@@ -179,15 +181,15 @@ static inline void macvm_set_rip(CPUState *cpu, uint64_t rip)uint64_t val;/* BUG, should take considering overlap.. */ -wreg (cpu->hvf_fd,HV_X86_RIP , rip); +wreg (cpu->hvf->fd,HV_X86_RIP , rip); env->eip = rip;/* after moving forward in rip, we need to clean INTERRUPTABILITY */ - val =rvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY ); + val =rvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY );if (val & (VMCS_INTERRUPTIBILITY_STI_BLOCKING |VMCS_INTERRUPTIBILITY_MOVSS_BLOCKING )) { env->hflags &= ~HF_INHIBIT_IRQ_MASK ; -wvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY , +wvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY , val & ~(VMCS_INTERRUPTIBILITY_STI_BLOCKING |VMCS_INTERRUPTIBILITY_MOVSS_BLOCKING )); }
@@ -199,9 +201,9 @@ static inline void vmx_clear_nmi_blocking(CPUState *cpu)CPUX86State *env = &x86_cpu->env; env->hflags2 &= ~HF2_NMI_MASK ; -uint32_t gi = (uint32_t )rvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY ); +uint32_t gi = (uint32_t )rvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY ); gi &= ~VMCS_INTERRUPTIBILITY_NMI_BLOCKING ; -wvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY , gi); +wvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY , gi); }static inline void vmx_set_nmi_blocking (CPUState *cpu)
@@ -210,16 +212,16 @@ static inline void vmx_set_nmi_blocking(CPUState *cpu)CPUX86State *env = &x86_cpu->env; env->hflags2 |=HF2_NMI_MASK ; -uint32_t gi = (uint32_t )rvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY ); +uint32_t gi = (uint32_t )rvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY ); gi |=VMCS_INTERRUPTIBILITY_NMI_BLOCKING ; -wvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY , gi); +wvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY , gi); }static inline void vmx_set_nmi_window_exiting (CPUState *cpu) {uint64_t val; - val =rvmcs (cpu->hvf_fd,VMCS_PRI_PROC_BASED_CTLS ); -wvmcs (cpu->hvf_fd,VMCS_PRI_PROC_BASED_CTLS , val | + val =rvmcs (cpu->hvf->fd,VMCS_PRI_PROC_BASED_CTLS ); +wvmcs (cpu->hvf->fd,VMCS_PRI_PROC_BASED_CTLS , val |VMCS_PRI_PROC_BASED_CTLS_NMI_WINDOW_EXITING ); }
@@ -228,8 +230,8 @@ static inline void vmx_clear_nmi_window_exiting(CPUState *cpu) {uint64_t val; - val =rvmcs (cpu->hvf_fd,VMCS_PRI_PROC_BASED_CTLS ); -wvmcs (cpu->hvf_fd,VMCS_PRI_PROC_BASED_CTLS , val & + val =rvmcs (cpu->hvf->fd,VMCS_PRI_PROC_BASED_CTLS ); +wvmcs (cpu->hvf->fd,VMCS_PRI_PROC_BASED_CTLS , val & ~VMCS_PRI_PROC_BASED_CTLS_NMI_WINDOW_EXITING ); }
target/i386/hvf/x86.c +14 -14
@@ -62,11 +62,11 @@ bool x86_read_segment_descriptor(struct CPUState *cpu, }if (GDT_SEL == sel.ti) { - base =rvmcs (cpu->hvf_fd,VMCS_GUEST_GDTR_BASE ); - limit =rvmcs (cpu->hvf_fd,VMCS_GUEST_GDTR_LIMIT ); + base =rvmcs (cpu->hvf->fd,VMCS_GUEST_GDTR_BASE ); + limit =rvmcs (cpu->hvf->fd,VMCS_GUEST_GDTR_LIMIT ); }else { - base =rvmcs (cpu->hvf_fd,VMCS_GUEST_LDTR_BASE ); - limit =rvmcs (cpu->hvf_fd,VMCS_GUEST_LDTR_LIMIT ); + base =rvmcs (cpu->hvf->fd,VMCS_GUEST_LDTR_BASE ); + limit =rvmcs (cpu->hvf->fd,VMCS_GUEST_LDTR_LIMIT ); }if (sel.index *8 >= limit) {
@@ -85,11 +85,11 @@ bool x86_write_segment_descriptor(struct CPUState *cpu,uint32_t limit;if (GDT_SEL == sel.ti) { - base =rvmcs (cpu->hvf_fd,VMCS_GUEST_GDTR_BASE ); - limit =rvmcs (cpu->hvf_fd,VMCS_GUEST_GDTR_LIMIT ); + base =rvmcs (cpu->hvf->fd,VMCS_GUEST_GDTR_BASE ); + limit =rvmcs (cpu->hvf->fd,VMCS_GUEST_GDTR_LIMIT ); }else { - base =rvmcs (cpu->hvf_fd,VMCS_GUEST_LDTR_BASE ); - limit =rvmcs (cpu->hvf_fd,VMCS_GUEST_LDTR_LIMIT ); + base =rvmcs (cpu->hvf->fd,VMCS_GUEST_LDTR_BASE ); + limit =rvmcs (cpu->hvf->fd,VMCS_GUEST_LDTR_LIMIT ); }if (sel.index *8 >= limit) {
@@ -103,8 +103,8 @@ bool x86_write_segment_descriptor(struct CPUState *cpu,bool x86_read_call_gate (struct CPUState *cpu,struct x86_call_gate *idt_desc,int gate) { -target_ulong base =rvmcs (cpu->hvf_fd,VMCS_GUEST_IDTR_BASE ); -uint32_t limit =rvmcs (cpu->hvf_fd,VMCS_GUEST_IDTR_LIMIT ); +target_ulong base =rvmcs (cpu->hvf->fd,VMCS_GUEST_IDTR_BASE ); +uint32_t limit =rvmcs (cpu->hvf->fd,VMCS_GUEST_IDTR_LIMIT );memset (idt_desc,0 ,sizeof (*idt_desc));if (gate *8 >= limit) {
@@ -118,7 +118,7 @@ bool x86_read_call_gate(struct CPUState *cpu, struct x86_call_gate *idt_desc,bool x86_is_protected (struct CPUState *cpu) { -uint64_t cr0 =rvmcs (cpu->hvf_fd,VMCS_GUEST_CR0 ); +uint64_t cr0 =rvmcs (cpu->hvf->fd,VMCS_GUEST_CR0 );return cr0 &CR0_PE ; }
@@ -136,7 +136,7 @@ bool x86_is_v8086(struct CPUState *cpu)bool x86_is_long_mode (struct CPUState *cpu) { -return rvmcs (cpu->hvf_fd,VMCS_GUEST_IA32_EFER ) &MSR_EFER_LMA ; +return rvmcs (cpu->hvf->fd,VMCS_GUEST_IA32_EFER ) &MSR_EFER_LMA ; }bool x86_is_long64_mode (struct CPUState *cpu)
@@ -149,13 +149,13 @@ bool x86_is_long64_mode(struct CPUState *cpu)bool x86_is_paging_mode (struct CPUState *cpu) { -uint64_t cr0 =rvmcs (cpu->hvf_fd,VMCS_GUEST_CR0 ); +uint64_t cr0 =rvmcs (cpu->hvf->fd,VMCS_GUEST_CR0 );return cr0 &CR0_PG ; }bool x86_is_pae_enabled (struct CPUState *cpu) { -uint64_t cr4 =rvmcs (cpu->hvf_fd,VMCS_GUEST_CR4 ); +uint64_t cr4 =rvmcs (cpu->hvf->fd,VMCS_GUEST_CR4 );return cr4 &CR4_PAE ; }
target/i386/hvf/x86_descr.c +13 -13
@@ -48,47 +48,47 @@ static const struct vmx_segment_field {uint32_t vmx_read_segment_limit (CPUState *cpu,X86Seg seg) { -return (uint32_t )rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].limit); +return (uint32_t )rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].limit); }uint32_t vmx_read_segment_ar (CPUState *cpu,X86Seg seg) { -return (uint32_t )rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].ar_bytes); +return (uint32_t )rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].ar_bytes); }uint64_t vmx_read_segment_base (CPUState *cpu,X86Seg seg) { -return rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].base); +return rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].base); }x68_segment_selector vmx_read_segment_selector (CPUState *cpu,X86Seg seg) {x68_segment_selector sel; - sel.sel =rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].selector); + sel.sel =rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].selector);return sel; }void vmx_write_segment_selector (struct CPUState *cpu,x68_segment_selector selector,X86Seg seg) { -wvmcs (cpu->hvf_fd, vmx_segment_fields[seg].selector, selector.sel); +wvmcs (cpu->hvf->fd, vmx_segment_fields[seg].selector, selector.sel); }void vmx_read_segment_descriptor (struct CPUState *cpu,struct vmx_segment *desc,X86Seg seg) { - desc->sel =rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].selector); - desc->base =rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].base); - desc->limit =rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].limit); - desc->ar =rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].ar_bytes); + desc->sel =rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].selector); + desc->base =rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].base); + desc->limit =rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].limit); + desc->ar =rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].ar_bytes); }void vmx_write_segment_descriptor (CPUState *cpu,struct vmx_segment *desc,X86Seg seg) {const struct vmx_segment_field *sf = &vmx_segment_fields[seg]; -wvmcs (cpu->hvf_fd, sf->base, desc->base); -wvmcs (cpu->hvf_fd, sf->limit, desc->limit); -wvmcs (cpu->hvf_fd, sf->selector, desc->sel); -wvmcs (cpu->hvf_fd, sf->ar_bytes, desc->ar); +wvmcs (cpu->hvf->fd, sf->base, desc->base); +wvmcs (cpu->hvf->fd, sf->limit, desc->limit); +wvmcs (cpu->hvf->fd, sf->selector, desc->sel); +wvmcs (cpu->hvf->fd, sf->ar_bytes, desc->ar); }void x86_segment_descriptor_to_vmx (struct CPUState *cpu,x68_segment_selector selector,struct x86_segment_descriptor *desc,struct vmx_segment *vmx_desc)
target/i386/hvf/x86_emu.c +31 -31
117 lines not on this page · raw message
target/i386/hvf/x86_mmu.c +2 -2
@@ -128,7 +128,7 @@ static bool test_pt_entry(struct CPUState *cpu, struct gpt_translation *pt, pt->err_code |=MMU_PAGE_PT ; } -uint32_t cr0 =rvmcs (cpu->hvf_fd,VMCS_GUEST_CR0 ); +uint32_t cr0 =rvmcs (cpu->hvf->fd,VMCS_GUEST_CR0 );/* check protection */ if (cr0 &CR0_WP ) {if (pt->write_access && !pte_write_access (pte)) {
@@ -173,7 +173,7 @@ static bool walk_gpt(struct CPUState *cpu, target_ulong addr, int err_code, {int top_level, level;bool is_large = false; -target_ulong cr3 =rvmcs (cpu->hvf_fd,VMCS_GUEST_CR3 ); +target_ulong cr3 =rvmcs (cpu->hvf->fd,VMCS_GUEST_CR3 );uint64_t page_mask = pae ?PAE_PTE_PAGE_MASK :LEGACY_PTE_PAGE_MASK ;memset (pt,0 ,sizeof (*pt));
target/i386/hvf/x86_task.c +6 -6
@@ -62,7 +62,7 @@ static void load_state_from_tss32(CPUState *cpu, struct x86_tss_segment32 *tss)X86CPU *x86_cpu =X86_CPU (cpu);CPUX86State *env = &x86_cpu->env; -wvmcs (cpu->hvf_fd,VMCS_GUEST_CR3 , tss->cr3); +wvmcs (cpu->hvf->fd,VMCS_GUEST_CR3 , tss->cr3); env->eip = tss->eip; env->eflags = tss->eflags |2 ;
@@ -111,11 +111,11 @@ static int task_switch_32(CPUState *cpu, x68_segment_selector tss_sel, x68_segmevoid vmx_handle_task_switch (CPUState *cpu,x68_segment_selector tss_sel,int reason,bool gate_valid,uint8_t gate,uint64_t gate_type) { -uint64_t rip =rreg (cpu->hvf_fd,HV_X86_RIP ); +uint64_t rip =rreg (cpu->hvf->fd,HV_X86_RIP );if (!gate_valid || (gate_type !=VMCS_INTR_T_HWEXCEPTION && gate_type !=VMCS_INTR_T_HWINTR && gate_type !=VMCS_INTR_T_NMI )) { -int ins_len =rvmcs (cpu->hvf_fd,VMCS_EXIT_INSTRUCTION_LENGTH ); +int ins_len =rvmcs (cpu->hvf->fd,VMCS_EXIT_INSTRUCTION_LENGTH );macvm_set_rip (cpu, rip + ins_len);return ; }
@@ -174,12 +174,12 @@ void vmx_handle_task_switch(CPUState *cpu, x68_segment_selector tss_sel, int rea//ret = task_switch_16(cpu, tss_sel, old_tss_sel, old_tss_base, &next_tss_desc); VM_PANIC ("task_switch_16" ); -macvm_set_cr0 (cpu->hvf_fd,rvmcs (cpu->hvf_fd,VMCS_GUEST_CR0 ) |CR0_TS ); +macvm_set_cr0 (cpu->hvf->fd,rvmcs (cpu->hvf->fd,VMCS_GUEST_CR0 ) |CR0_TS );x86_segment_descriptor_to_vmx (cpu, tss_sel, &next_tss_desc, &vmx_seg);vmx_write_segment_descriptor (cpu, &vmx_seg,R_TR );store_regs (cpu); -hv_vcpu_invalidate_tlb (cpu->hvf_fd); -hv_vcpu_flush (cpu->hvf_fd); +hv_vcpu_invalidate_tlb (cpu->hvf->fd); +hv_vcpu_flush (cpu->hvf->fd); }
target/i386/hvf/x86hvf.c +105 -105
354 lines not on this page · raw message
Signature · 1 line
2.24.3 (Apple Git-128)
Re: [PATCH v3 03/10] hvf: Introduce hvf vcpu struct
Quoted 4 lines · Alexander Graf
We will need more than a single field for hvf going forward. To keep the global vcpu struct uncluttered, let's allocate a special hvf vcpu struct, similar to how hax does it.
Thanks,
Roman
PATCH v3 4/10 arm: Set PSCI to 0.2 for HVF
In reply to Alexander GrafIn Hypervisor.framework, we just pass PSCI calls straight on to the QEMU emulation of it. That means, if TCG is compatible with PSCI 0.2, so are we. Let's transpose that fact in code too.
--- target/arm/cpu.c | 4 ++++ 1 file changed, 4 insertions(+)
target/arm/cpu.c +4 -0
@@ -1062,6 +1062,10 @@ static void arm_cpu_initfn(Object *obj)if (tcg_enabled ()) { cpu->psci_version =2 ;/* TCG implements PSCI 0.2 */ } + +if (hvf_enabled ()) { + cpu->psci_version =2 ;/* HVF uses TCG's PSCI */ + } } static Property arm_cpu_gt_cntfrq_property =
Signature · 1 line
2.24.3 (Apple Git-128)
Re: [PATCH v3 04/10] arm: Set PSCI to 0.2 for HVF
Quoted 14 lines · Alexander Graf
In Hypervisor.framework, we just pass PSCI calls straight on to the QEMU emulation of it. That means, if TCG is compatible with PSCI 0.2, so are we. Let's transpose that fact in code too.
Signed-off-by Alexander Graf <[email protected]>--- target/arm/cpu.c | 4 ++++ 1 file changed, 4 insertions(+)target/arm/cpu.c +0
-0
target/arm/cpu.c static void arm_cpu_initfn(Object *obj) · line 1068
Quoted 7 lines
target/arm/cpu.c +4
-0@@ -1062,6 +1062,10 @@ static void arm_cpu_initfn(Object *obj)if (tcg_enabled ()) { cpu->psci_version =2 ;/* TCG implements PSCI 0.2 */ } + +if (hvf_enabled ()) { + cpu->psci_version =2 ;/* HVF uses TCG's PSCI */ + }
If HVF is piggybacking on TCG and they're both the same versions would it be better if:
target/arm/cpu.c static void arm_cpu_initfn(Object *obj) · line 1071
Quoted 5 lines
target/arm/cpu.c +2
-2-if (tcg_enabled ()) { - cpu->psci_version =2 ;/* TCG implements PSCI 0.2 */ +if (tcg_enabled () ||hvf_enabled ()) { + cpu->psci_version =2 ;/* TCG implements PSCI 0.2 */ }
That'd avoid a case where they get out of sync.
Thanks,
Roman
Quoted 6 lines
}
static Property
arm_cpu_gt_cntfrq_property=Signature · 1 line
2.24.3 (Apple Git-128)
PATCH v3 5/10 hvf: arm: Mark CPU as dirty on reset
In reply to Alexander GrafWhen clearing internal state of a CPU, we should also make sure that HVF knows about it and can push the new values down to vcpu state.
Make sure that with HVF enabled, we tell it that it should synchronize CPU state on next entry after a reset.
This fixes PSCI handling, because now newly pushed state such as X0 and PC on remote CPU enablement also get pushed into HVF.
--- target/arm/arm-powerctl.c | 1 + target/arm/cpu.c | 2 ++ 2 files changed, 3 insertions(+)
target/arm/arm-powerctl.c +1 -0
@@ -15,6 +15,7 @@#include "arm-powerctl.h" #include "qemu/log.h" #include "qemu/main-loop.h" +#include "sysemu/hw_accel.h" #ifndef DEBUG_ARM_POWERCTL #define DEBUG_ARM_POWERCTL 0
target/arm/cpu.c +2 -0
@@ -411,6 +411,8 @@ static void arm_cpu_reset(DeviceState *dev)#ifndef CONFIG_USER_ONLY if (kvm_enabled ()) {kvm_arm_reset_vcpu (cpu); + }else if (hvf_enabled ()) { + s->vcpu_dirty = true; }#endif
Signature · 1 line
2.24.3 (Apple Git-128)
Re: [PATCH v3 05/10] hvf: arm: Mark CPU as dirty on reset
Quoted 3 lines · Alexander Graf
When clearing internal state of a CPU, we should also make sure that HVF knows about it and can push the new values down to vcpu state.
I'm sorry if I'm asking something dumb. But isn't cpu_synchronize_all_post_reset() is supposed to push QEMU state into HVF (or any other accel) after reset?
For x86 it used to work:
static void do_hvf_cpu_synchronize_post_reset(CPUState *cpu,
run_on_cpu_data arg)
{
hvf_put_registers(cpu); cpu->vcpu_dirty = false;
}Thanks,
Roman
Quoted 40 lines
Make sure that with HVF enabled, we tell it that it should synchronize CPU state on next entry after a reset.
This fixes PSCI handling, because now newly pushed state such as X0 and PC on remote CPU enablement also get pushed into HVF.
Signed-off-by Alexander Graf <[email protected]>--- target/arm/arm-powerctl.c | 1 + target/arm/cpu.c | 2 ++ 2 files changed, 3 insertions(+)target/arm/arm-powerctl.c +1
-0@@ -15,6 +15,7 @@#include "arm-powerctl.h" #include "qemu/log.h" #include "qemu/main-loop.h" +#include "sysemu/hw_accel.h" #ifndef DEBUG_ARM_POWERCTL #define DEBUG_ARM_POWERCTL 0target/arm/cpu.c +2
-0@@ -411,6 +411,8 @@ static void arm_cpu_reset(DeviceState *dev)#ifndef CONFIG_USER_ONLY if (kvm_enabled ()) {kvm_arm_reset_vcpu (cpu); + }else if (hvf_enabled ()) { + s->vcpu_dirty = true; }#endif Signature · 1 line
2.24.3 (Apple Git-128)
Re: [PATCH v3 05/10] hvf: arm: Mark CPU as dirty on reset
Quoted 8 lines · Roman Bolshakov
Quoted 3 lines · Alexander Graf
When clearing internal state of a CPU, we should also make sure that HVF knows about it and can push the new values down to vcpu state.
I'm sorry if I'm asking something dumb. But isn't
cpu_synchronize_all_post_reset()is supposed to push QEMU state into HVF (or any other accel) after reset?
Quoted 7 lines
For x86 it used to work:
static void do_hvf_cpu_synchronize_post_reset(CPUState *cpu, run_on_cpu_data arg) { hvf_put_registers(cpu); cpu->vcpu_dirty = false; }
Yes, it works because after the reset is done, there is no other register modification happening. With the PSCI emulation code in QEMU, we still do modify CPU state after reset though.
Different question though: Why do we need the post_reset() call at all here to push state? We would just push it on the next run anyways, right? So if we don't set vcpu_dirty to false then, we wouldn't need this patch here I think.
Alex
Re: [PATCH v3 05/10] hvf: arm: Mark CPU as dirty on reset
Quoted 20 lines · Alexander Graf
Quoted 8 lines · Roman Bolshakov
Quoted 3 lines · Alexander Graf
When clearing internal state of a CPU, we should also make sure that HVF knows about it and can push the new values down to vcpu state.
I'm sorry if I'm asking something dumb. But isn't
cpu_synchronize_all_post_reset()is supposed to push QEMU state into HVF (or any other accel) after reset?Quoted 8 lines
For x86 it used to work:
static void do_hvf_cpu_synchronize_post_reset(CPUState *cpu, run_on_cpu_data arg) { hvf_put_registers(cpu); cpu->vcpu_dirty = false; }
Quoted 4 lines
Yes, it works because after the reset is done, there is no other register modification happening. With the PSCI emulation code in QEMU, we still do modify CPU state after reset though.
Maybe I miss something but that doesn't seem correct. Why PSCI reset is split from machine reset?
Quoted 2 lines
Different question though: Why do we need the
post_reset()call at all here to push state?
My understanding that post_reset is akin to a commit of the CPU state after all reset actions have been done to QEMU CPU Arch env state. i.e. arch/machine reset modifies env state and then the env is pushed to accel. cpu->vcpu_dirty is cleared because env is in-sync with vcpu.
Quoted 1 line
We would just push it on the next run anyways, right?
That's correct (at least for x86 HVF).
Quoted 3 lines
So if we don't set
vcpu_dirtyto false then, we wouldn't need this patch here I think.
That's right but the same post-reset approach is used for all accels, including KVM. But I haven't found this for TCG.
Regards,
Roman
Re: [PATCH v3 05/10] hvf: arm: Mark CPU as dirty on reset
Quoted 19 lines · Roman Bolshakov
Quoted 18 lines · Alexander Graf
Quoted 16 lines · Roman Bolshakov
Quoted 3 lines · Alexander Graf
When clearing internal state of a CPU, we should also make sure that HVF knows about it and can push the new values down to vcpu state.
I'm sorry if I'm asking something dumb. But isn't
cpu_synchronize_all_post_reset()is supposed to push QEMU state into HVF (or any other accel) after reset?For x86 it used to work:
static void do_hvf_cpu_synchronize_post_reset(CPUState *cpu, run_on_cpu_data arg) { hvf_put_registers(cpu); cpu->vcpu_dirty = false; }
Quoted 6 lines
Quoted 4 lines
Yes, it works because after the reset is done, there is no other register modification happening. With the PSCI emulation code in QEMU, we still do modify CPU state after reset though.
Maybe I miss something but that doesn't seem correct. Why PSCI reset is split from machine reset?
Because with PSCI, you can online/offline individual CPUs, not just the full system.
Quoted 7 lines
Quoted 3 lines
Different question though: Why do we need the
post_reset()call at all here to push state?My understanding that
post_resetis akin to a commit of the CPU state after all reset actions have been done to QEMU CPU Arch env state. i.e.arch/machinereset modifies env state and then the env is pushed to accel.cpu->vcpu_dirtyis cleared because env is in-sync with vcpu.
I think that's only half the truth. What it semantically means is "QEMU's env structure is what holds the current state." Which basically translates to cpu->vcpu_dirty = true.
So all of these callbacks could literally just be that, no?
Alex
PATCH v3 6/10 hvf: Add Apple Silicon support
In reply to Alexander GrafWith Apple Silicon available to the masses, it's a good time to add support for driving its virtualization extensions from QEMU.
This patch adds all necessary architecture specific code to get basic VMs working. It's still pretty raw, but definitely functional.
Known limitations:
- Vtimer acknowledgement is hacky
- Should implement more sysregs and fault on invalid ones then
- WFI handling is missing, need to marry it with vtimer
---
v1 -> v2:
- Merge vcpu kick function patch
- Implement WFI handling (allows vCPUs to sleep)
- Synchronize system registers (fixes OVMF crashes and reboot)
- Don't always call cpu_synchronize_state()
- Use more fine grained iothread locking
- Populate aa64mmfr0 from hardware
v2 -> v3:
- Advance PC on SMC
- Use cp list interface for sysreg syncs
- Do not set current_cpu
- Fix sysreg isread mask
- Move sysreg handling to functions
- Remove WFI logic again
- Revert to global iothread locking
--- MAINTAINERS | 5 + accel/hvf/hvf-cpus.c | 14 + include/sysemu/hvf_int.h | 5 +- target/arm/hvf/hvf.c | 620 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 643 insertions(+), 1 deletion(-) create mode 100644 target/arm/hvf/hvf.c
MAINTAINERS +5 -0
@@ -439,6 +439,11 @@ F: accel/accel.c F: accel/Makefile.objs F: accel/stubs/Makefile.objs +Apple Silicon HVF CPUs +M: Alexander Graf <[email protected]> +S: Maintained +F: target/arm/hvf/ + X86 HVF CPUs M: Cameron Esfahani <[email protected]> M: Roman Bolshakov <[email protected]>
accel/hvf/hvf-cpus.c +14 -0
@@ -60,6 +60,10 @@#include <Hypervisor/Hypervisor.h> +#ifdef __aarch64__ +#define HV_VM_DEFAULT NULL +#endif +/* Memory slots */ struct mac_slot {
@@ -343,7 +347,11 @@ static int hvf_init_vcpu(CPUState *cpu)pthread_sigmask (SIG_BLOCK ,NULL , &set);sigdelset (&set,SIG_IPI ); +#ifdef __aarch64__ + r =hv_vcpu_create (&cpu->hvf->fd, (hv_vcpu_exit_t **)&cpu->hvf->exit,NULL ); +#else r =hv_vcpu_create ((hv_vcpuid_t *)&cpu->hvf->fd,HV_VCPU_DEFAULT ); +#endif cpu->vcpu_dirty =1 ;assert_hvf_ok (r);
@@ -414,8 +422,14 @@ static void hvf_start_vcpu_thread(CPUState *cpu) cpu, QEMU_THREAD_JOINABLE); } +__attribute__((weak))void hvf_kick_vcpu_thread (CPUState *cpu) +{ +cpus_kick_thread (cpu); +} +static const CpusAccel hvf_cpus = { .create_vcpu_thread = hvf_start_vcpu_thread, + .kick_vcpu_thread = hvf_kick_vcpu_thread, .synchronize_post_reset = hvf_cpu_synchronize_post_reset, .synchronize_post_init = hvf_cpu_synchronize_post_init,
include/sysemu/hvf_int.h +4 -1
@@ -11,6 +11,7 @@#ifndef HVF_INT_H #define HVF_INT_H +#include "qemu/osdep.h" #include <Hypervisor/Hypervisor.h> #define HVF_MAX_VCPU 0x10
@@ -59,7 +60,8 @@ struct HVFState {extern HVFState *hvf_state;struct hvf_vcpu_state { -int fd; +uint64_t fd; +void *exit; };void assert_hvf_ok (hv_return_t ret);
@@ -69,5 +71,6 @@ int hvf_arch_init_vcpu(CPUState *cpu);void hvf_arch_vcpu_destroy (CPUState *cpu);int hvf_vcpu_exec (CPUState *cpu);hvf_slot *hvf_find_overlap_slot (uint64_t ,uint64_t ); +void hvf_kick_vcpu_thread (CPUState *cpu);#endif
target/arm/hvf/hvf.c new file +620 -0
620 lines not on this page · raw message
Signature · 1 line
2.24.3 (Apple Git-128)
Re: [PATCH v3 06/10] hvf: Add Apple Silicon support
Quoted 6 lines · Alexander Graf
With Apple Silicon available to the masses, it's a good time to add support for driving its virtualization extensions from QEMU.
This patch adds all necessary architecture specific code to get basic VMs working. It's still pretty raw, but definitely functional.
That's very cool, Alex!
accel/hvf/hvf-cpus.c line 62
Quoted 10 lines
[...]
accel/hvf/hvf-cpus.c +0
-0@@ -60,6 +60,10 @@#include <Hypervisor/Hypervisor.h>
On an older laptop with 10.15 I've noticed this causes a build failure. Here's layout of Hypervisor.framework on 10.15:
Hypervisor.framework find .
.
./Versions
./Versions/A
./Versions/A/Hypervisor.tbd
./Versions/A/Headers
./Versions/A/Headers/hv_arch_vmx.h
./Versions/A/Headers/hv_error.h
./Versions/A/Headers/hv_types.h
./Versions/A/Headers/hv.h
./Versions/A/Headers/hv_arch_x86.h
./Versions/A/Headers/hv_vmx.h
./Versions/Current
./module.map
./Hypervisor.tbd
./Headers
The issue also exists in another patch in the series:
"hvf: Move common code out"
accel/hvf/hvf-cpus.c line 66
Quoted 4 lines
accel/hvf/hvf-cpus.c +4
-0+#ifdef __aarch64__ +#define HV_VM_DEFAULT NULL +#endif +
I don't see if it's used anywhere.
accel/hvf/hvf-cpus.c line 69
Quoted 5 lines
accel/hvf/hvf-cpus.c +0
-0/* Memory slots */ struct mac_slot {
Side question. I have very little knowledge of ARM but it seems much leaner compared to x86 trap/emulation layer. Is it a consequence of load/store architecture and it's expected to be that small on ARM?
I have only noticed MMIO, system registers (access to them apparently leads to a trap), kick and PSCI traps (which sounds somewhat similar to Intel MPSpec/APIC) and no system instruction traps (except WFI in the next patch).
Thanks,
Roman
Re: [PATCH v3 06/10] hvf: Add Apple Silicon support
Quoted 38 lines · Roman Bolshakov
Quoted 6 lines · Alexander Graf
With Apple Silicon available to the masses, it's a good time to add support for driving its virtualization extensions from QEMU.
This patch adds all necessary architecture specific code to get basic VMs working. It's still pretty raw, but definitely functional.
That's very cool, Alex!
accel/hvf/hvf-cpus.c line 62
Quoted 9 lines
[...]
accel/hvf/hvf-cpus.c +0
-0@@ -60,6 +60,10 @@#include <Hypervisor/Hypervisor.h> On an older laptop with 10.15 I've noticed this causes a build failure. Here's layout of Hypervisor.framework on 10.15:
Hypervisor.framework find .
.
./Versions
./Versions/A
./Versions/A/Hypervisor.tbd
./Versions/A/Headers
./Versions/A/Headers/hv_arch_vmx.h
./Versions/A/Headers/hv_error.h
./Versions/A/Headers/hv_types.h
./Versions/A/Headers/hv.h
./Versions/A/Headers/hv_arch_x86.h
./Versions/A/Headers/hv_vmx.h
./Versions/Current
./module.map
./Hypervisor.tbd
./Headers
Quoted 2 lines
The issue also exists in another patch in the series:
"hvf: Move common code out"
Ugh, I'll try and see if I can dig out a 10.15 machine somewhere.
Quoted 6 lines
accel/hvf/hvf-cpus.c line 66
Quoted 5 lines
accel/hvf/hvf-cpus.c +4
-0+#ifdef __aarch64__ +#define HV_VM_DEFAULT NULL +#endif +I don't see if it's used anywhere.
It's used in hv_vm_create() as argument. The enum that contains it is hidden behind an #ifdef __x86_64__ in HVF though.
Quoted 9 lines
accel/hvf/hvf-cpus.c line 69
Quoted 6 lines
accel/hvf/hvf-cpus.c +0
-0/* Memory slots */ struct mac_slot {Side question. I have very little knowledge of ARM but it seems much leaner compared to x86 trap/emulation layer. Is it a consequence of load/store architecture and it's expected to be that small on ARM?
It's multiple things coming together. Early in the virtualization days of KVM on ARM, we decided that KVM would only support MMIO for instructions that hardware predecodes on exception. That seems to have trickled down into basically all relevant OSs these days, so you can run Linux, *BSD and Windows just fine without handling any instructions that are not already predecoded for you.
This in turn means you don't need an instruction emulator, which is most of the complexity on the x86 hvf code.
Quoted 4 lines
I have only noticed MMIO, system registers (access to them apparently leads to a trap), kick and PSCI traps (which sounds somewhat similar to Intel MPSpec/APIC) and no system instruction traps (except WFI in the next patch).
System Registers are a bit tricky. Some of them lead to traps (as you've seen), others do not and instead read/write shadow registers directly. For those, we need to do the register sync.
But yes, there is little to handle for an ARM guest. I was positively surprised as well. To be fair though, most of the complexity in KVM ARM code comes from vGIC (not available on M1), debug registers (not handled here yet), PMU multiplexing (not handled) and stage2 page table maintenance (done by HVF). We just wiggle ourselves out of those.
Alex
PATCH v3 7/10 arm: Add Hypervisor.framework build target
In reply to Alexander GrafNow that we have all logic in place that we need to handle Hypervisor.framework on Apple Silicon systems, let's add CONFIG_HVF for aarch64 as well so that we can build it.
---
v1 -> v2:
- Fix build on 32bit arm
--- meson.build | 11 ++++++++++- target/arm/hvf/meson.build | 3 +++ target/arm/meson.build | 2 ++ 3 files changed, 15 insertions(+), 1 deletion(-) create mode 100644 target/arm/hvf/meson.build
meson.build +10 -1
@@ -74,16 +74,25 @@ else endif accelerator_targets = { 'CONFIG_KVM': kvm_targets } + +if cpu in ['x86', 'x86_64'] + hvf_targets = ['i386-softmmu', 'x86_64-softmmu'] +elif cpu in ['aarch64'] + hvf_targets = ['aarch64-softmmu'] +else + hvf_targets = [] +endif + if cpu in ['x86', 'x86_64', 'arm', 'aarch64'] # i368 emulator provides xenpv machine type for multiple architectures accelerator_targets += { 'CONFIG_XEN': ['i386-softmmu', 'x86_64-softmmu'], + 'CONFIG_HVF': hvf_targets, } endif if cpu in ['x86', 'x86_64'] accelerator_targets += { 'CONFIG_HAX': ['i386-softmmu', 'x86_64-softmmu'], - 'CONFIG_HVF': ['x86_64-softmmu'], 'CONFIG_WHPX': ['i386-softmmu', 'x86_64-softmmu'], } endif
target/arm/hvf/meson.build new file +3 -0
@@ -0,0 +1,3 @@ +arm_softmmu_ss.add(when: [hvf, 'CONFIG_HVF'], if_true: files( + 'hvf.c', +))
target/arm/meson.build +2 -0
@@ -56,5 +56,7 @@ arm_softmmu_ss.add(files( 'psci.c', )) +subdir('hvf') + target_arch += {'arm': arm_ss} target_softmmu_arch += {'arm': arm_softmmu_ss}
Signature · 1 line
2.24.3 (Apple Git-128)
Re: [PATCH v3 07/10] arm: Add Hypervisor.framework build target
Quoted 23 lines · Alexander Graf
Now that we have all logic in place that we need to handle Hypervisor.framework on Apple Silicon systems, let's add
CONFIG_HVFfor aarch64 as well so that we can build it.Signed-off-by Alexander Graf <[email protected]>---
v1 -> v2:
- Fix build on 32bit arm
--- meson.build | 11 ++++++++++- target/arm/hvf/meson.build | 3 +++ target/arm/meson.build | 2 ++ 3 files changed, 15 insertions(+), 1 deletion(-) create mode 100644 target/arm/hvf/meson.buildmeson.build +0
-0
meson.build else · line 79
Quoted 6 lines
meson.build +3
-0@@ -74,16 +74,25 @@ else endif accelerator_targets = { 'CONFIG_KVM': kvm_targets } + +if cpu in ['x86', 'x86_64'] + hvf_targets = ['i386-softmmu', 'x86_64-softmmu']
i386-softmmu had issues with hvf. Perhaps better to leave it disabled for the target as it was before.
Thanks,
Roman
Quoted 44 lines
meson.build +7
-1+elif cpu in ['aarch64'] + hvf_targets = ['aarch64-softmmu'] +else + hvf_targets = [] +endif + if cpu in ['x86', 'x86_64', 'arm', 'aarch64'] # i368 emulator provides xenpv machine type for multiple architectures accelerator_targets += { 'CONFIG_XEN': ['i386-softmmu', 'x86_64-softmmu'], + 'CONFIG_HVF': hvf_targets, } endif if cpu in ['x86', 'x86_64'] accelerator_targets += { 'CONFIG_HAX': ['i386-softmmu', 'x86_64-softmmu'], - 'CONFIG_HVF': ['x86_64-softmmu'], 'CONFIG_WHPX': ['i386-softmmu', 'x86_64-softmmu'], } endiftarget/arm/hvf/meson.build new file +3
-0@@ -0,0 +1,3 @@ +arm_softmmu_ss.add(when: [hvf, 'CONFIG_HVF'], if_true: files( + 'hvf.c', +))target/arm/meson.build +2
-0@@ -56,5 +56,7 @@ arm_softmmu_ss.add(files( 'psci.c', )) +subdir('hvf') + target_arch += {'arm': arm_ss} target_softmmu_arch += {'arm': arm_softmmu_ss}Signature · 1 line
2.24.3 (Apple Git-128)
PATCH v3 8/10 arm/hvf: Add a WFI handler
In reply to Alexander GrafFrom: Peter Collingbourne <[email protected]>
Sleep on WFI until the VTIMER is due but allow ourselves to be woken up on IPI.
--- accel/hvf/hvf-cpus.c | 5 ++-- include/sysemu/hvf_int.h | 1 + target/arm/hvf/hvf.c | 55 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+), 3 deletions(-)
accel/hvf/hvf-cpus.c +2 -3
@@ -337,15 +337,14 @@ static int hvf_init_vcpu(CPUState *cpu) cpu->hvf =g_malloc0 (sizeof (*cpu->hvf));/* init cpu signals */ -sigset_t set;struct sigaction sigact;memset (&sigact,0 ,sizeof (sigact)); sigact.sa_handler = dummy_signal;sigaction (SIG_IPI , &sigact,NULL ); -pthread_sigmask (SIG_BLOCK ,NULL , &set); -sigdelset (&set,SIG_IPI ); +pthread_sigmask (SIG_BLOCK ,NULL , &cpu->hvf->unblock_ipi_mask); +sigdelset (&cpu->hvf->unblock_ipi_mask,SIG_IPI );#ifdef __aarch64__ r =hv_vcpu_create (&cpu->hvf->fd, (hv_vcpu_exit_t **)&cpu->hvf->exit,NULL );
include/sysemu/hvf_int.h +1 -0
@@ -62,6 +62,7 @@ extern HVFState *hvf_state;struct hvf_vcpu_state {uint64_t fd;void *exit; +sigset_t unblock_ipi_mask; };void assert_hvf_ok (hv_return_t ret);
target/arm/hvf/hvf.c +55 -0
85 lines not on this page · raw message
Signature · 1 line
2.24.3 (Apple Git-128)
Re: [PATCH v3 08/10] arm/hvf: Add a WFI handler
Quoted 78 lines · Alexander Graf
From: Peter Collingbourne <[email protected]>
Sleep on WFI until the VTIMER is due but allow ourselves to be woken up on IPI.
Signed-off-by Peter Collingbourne <[email protected]>[agraf: Remove unused 'set' variable, always advance PC on WFX trap]Signed-off-by Alexander Graf <[email protected]>--- accel/hvf/hvf-cpus.c | 5 ++-- include/sysemu/hvf_int.h | 1 + target/arm/hvf/hvf.c | 55 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+), 3 deletions(-)accel/hvf/hvf-cpus.c +2
-3@@ -337,15 +337,14 @@ static int hvf_init_vcpu(CPUState *cpu) cpu->hvf =g_malloc0 (sizeof (*cpu->hvf));/* init cpu signals */ -sigset_t set;struct sigaction sigact;memset (&sigact,0 ,sizeof (sigact)); sigact.sa_handler = dummy_signal;sigaction (SIG_IPI , &sigact,NULL ); -pthread_sigmask (SIG_BLOCK ,NULL , &set); -sigdelset (&set,SIG_IPI ); +pthread_sigmask (SIG_BLOCK ,NULL , &cpu->hvf->unblock_ipi_mask); +sigdelset (&cpu->hvf->unblock_ipi_mask,SIG_IPI );#ifdef __aarch64__ r =hv_vcpu_create (&cpu->hvf->fd, (hv_vcpu_exit_t **)&cpu->hvf->exit,NULL );include/sysemu/hvf_int.h +1
-0@@ -62,6 +62,7 @@ extern HVFState *hvf_state;struct hvf_vcpu_state {uint64_t fd;void *exit; +sigset_t unblock_ipi_mask; };void assert_hvf_ok (hv_return_t ret);target/arm/hvf/hvf.c +3
-0@@ -2,6 +2,7 @@* QEMU Hypervisor.framework support for Apple Silicon * Copyright 2020 Alexander Graf <[email protected]> +* Copyright 2020 Google LLC * * This work is licensed under the terms of the GNU GPL, version 2 or later. * See the COPYING file in the top-level directory. @@ -18,6 +19,7 @@#include "sysemu/hw_accel.h" #include <Hypervisor/Hypervisor.h> +#include <mach/mach_time.h> #include "exec/address-spaces.h" #include "hw/irq.h" @@ -413,6 +415,7 @@ int hvf_arch_init_vcpu(CPUState *cpu)void hvf_kick_vcpu_thread (CPUState *cpu) { +cpus_kick_thread (cpu);hv_vcpus_exit (&cpu->hvf->fd,1 ); }@@ -468,6 +471,18 @@ static int hvf_inject_interrupts(CPUState *cpu)return 0 ;
target/arm/hvf/hvf.c static int hvf_inject_interrupts(CPUState *cpu) · line 481
Quoted 10 lines
target/arm/hvf/hvf.c +8
-0} +static void hvf_wait_for_ipi (CPUState *cpu,struct timespec *ts) +{ +/* +* Use pselect to sleep so that other threads can IPI us while we're +* sleeping. +*/ +qatomic_mb_set (&cpu->thread_kicked, false); +qemu_mutex_unlock_iothread ();
I raised a concern earlier, but I don't for sure if a kick could be lost right here. On x86 it could be lost.
Quoted 21 lines
target/arm/hvf/hvf.c +14
-0+pselect (0 ,0 ,0 ,0 , ts, &cpu->hvf->unblock_ipi_mask); +qemu_mutex_lock_iothread (); +} +int hvf_vcpu_exec (CPUState *cpu) {ARMCPU *arm_cpu =ARM_CPU (cpu);@@ -579,6 +594,46 @@ int hvf_vcpu_exec(CPUState *cpu) }case EC_WFX_TRAP : advance_pc = true; +if (!(syndrome &WFX_IS_WFE ) && !(cpu->interrupt_request & + (CPU_INTERRUPT_HARD |CPU_INTERRUPT_FIQ ))) { + +uint64_t ctl; + r =hv_vcpu_get_sys_reg (cpu->hvf->fd,HV_SYS_REG_CNTV_CTL_EL0 , + &ctl); +assert_hvf_ok (r); + +if (!(ctl &1 ) || (ctl &2 )) { +/* Timer disabled or masked, just wait for an IPI. */
target/arm/hvf/hvf.c int hvf_vcpu_exec(CPUState *cpu) · line 616
Quoted 10 lines
target/arm/hvf/hvf.c +10
-0+hvf_wait_for_ipi (cpu,NULL ); +break ; + } + +uint64_t cval; + r =hv_vcpu_get_sys_reg (cpu->hvf->fd,HV_SYS_REG_CNTV_CVAL_EL0 , + &cval); +assert_hvf_ok (r); + +int64_t ticks_to_sleep = cval -mach_absolute_time ();
Apple reference recommends to use [1]:
clock_gettime_nsec_np(CLOCK_UPTIME_RAW)
It, internally in Libc, invokes mach_absolute_time() [2].
1. https://developer.apple.com/documentation/kernel/1462446-mach_absolute_time
2. https://opensource.apple.com/source/Libc/Libc-1158.1.2/gen/clock_gettime.c.auto.html
Thanks,
Roman
Quoted 26 lines
target/arm/hvf/hvf.c +20
-0+if (ticks_to_sleep <0 ) { +break ; + } + +uint64_t seconds = ticks_to_sleep / arm_cpu->gt_cntfrq_hz; +uint64_t nanos = + (ticks_to_sleep - arm_cpu->gt_cntfrq_hz * seconds) * +1000000000 / arm_cpu->gt_cntfrq_hz; + +/* +* Don't sleep for less than 2ms. This is believed to improve +* latency of message passing workloads. +*/ +if (!seconds && nanos <2000000 ) { +break ; + } + +struct timespec ts = { seconds, nanos }; +hvf_wait_for_ipi (cpu, &ts); + }break ;case EC_AA64_HVC :cpu_synchronize_state (cpu);Signature · 1 line
2.24.3 (Apple Git-128)
Re: [PATCH v3 08/10] arm/hvf: Add a WFI handler
Quoted 82 lines · Roman Bolshakov
Quoted 80 lines · Alexander Graf
From: Peter Collingbourne <[email protected]>
Sleep on WFI until the VTIMER is due but allow ourselves to be woken up on IPI.
Signed-off-by Peter Collingbourne <[email protected]>[agraf: Remove unused 'set' variable, always advance PC on WFX trap]Signed-off-by Alexander Graf <[email protected]>--- accel/hvf/hvf-cpus.c | 5 ++-- include/sysemu/hvf_int.h | 1 + target/arm/hvf/hvf.c | 55 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+), 3 deletions(-)accel/hvf/hvf-cpus.c +2
-3@@ -337,15 +337,14 @@ static int hvf_init_vcpu(CPUState *cpu) cpu->hvf =g_malloc0 (sizeof (*cpu->hvf));/* init cpu signals */ -sigset_t set;struct sigaction sigact;memset (&sigact,0 ,sizeof (sigact)); sigact.sa_handler = dummy_signal;sigaction (SIG_IPI , &sigact,NULL ); -pthread_sigmask (SIG_BLOCK ,NULL , &set); -sigdelset (&set,SIG_IPI ); +pthread_sigmask (SIG_BLOCK ,NULL , &cpu->hvf->unblock_ipi_mask); +sigdelset (&cpu->hvf->unblock_ipi_mask,SIG_IPI );#ifdef __aarch64__ r =hv_vcpu_create (&cpu->hvf->fd, (hv_vcpu_exit_t **)&cpu->hvf->exit,NULL );include/sysemu/hvf_int.h +1
-0@@ -62,6 +62,7 @@ extern HVFState *hvf_state;struct hvf_vcpu_state {uint64_t fd;void *exit; +sigset_t unblock_ipi_mask; };void assert_hvf_ok (hv_return_t ret);target/arm/hvf/hvf.c +3
-0@@ -2,6 +2,7 @@* QEMU Hypervisor.framework support for Apple Silicon * Copyright 2020 Alexander Graf <[email protected]> +* Copyright 2020 Google LLC * * This work is licensed under the terms of the GNU GPL, version 2 or later. * See the COPYING file in the top-level directory. @@ -18,6 +19,7 @@#include "sysemu/hw_accel.h" #include <Hypervisor/Hypervisor.h> +#include <mach/mach_time.h> #include "exec/address-spaces.h" #include "hw/irq.h" @@ -413,6 +415,7 @@ int hvf_arch_init_vcpu(CPUState *cpu)void hvf_kick_vcpu_thread (CPUState *cpu) { +cpus_kick_thread (cpu);hv_vcpus_exit (&cpu->hvf->fd,1 ); }@@ -468,6 +471,18 @@ static int hvf_inject_interrupts(CPUState *cpu)return 0 ; }
Quoted 11 lines
target/arm/hvf/hvf.c static int hvf_inject_interrupts(CPUState *cpu) · line 481
Quoted 8 lines
target/arm/hvf/hvf.c +8
-0+static void hvf_wait_for_ipi (CPUState *cpu,struct timespec *ts) +{ +/* +* Use pselect to sleep so that other threads can IPI us while we're +* sleeping. +*/ +qatomic_mb_set (&cpu->thread_kicked, false); +qemu_mutex_unlock_iothread ();I raised a concern earlier, but I don't for sure if a kick could be lost right here. On x86 it could be lost.
If the signal is sent right before the pselect() it will be blocked i.e. left pending. With the pselect() we get an atomic unblock of SIG_IPI at the same time as we begin sleeping, which means that we will receive the signal and leave the pselect() immediately.
I think at some point macOS had an incorrect implementation of pselect() where the signal mask was non-atomically set in userspace which could lead to the signal being missed but I checked the latest XNU sources and it looks like the pselect() implementation has been moved to the kernel.
Quoted 35 lines
Quoted 21 lines
target/arm/hvf/hvf.c +14
-0+pselect (0 ,0 ,0 ,0 , ts, &cpu->hvf->unblock_ipi_mask); +qemu_mutex_lock_iothread (); +} +int hvf_vcpu_exec (CPUState *cpu) {ARMCPU *arm_cpu =ARM_CPU (cpu);@@ -579,6 +594,46 @@ int hvf_vcpu_exec(CPUState *cpu) }case EC_WFX_TRAP : advance_pc = true; +if (!(syndrome &WFX_IS_WFE ) && !(cpu->interrupt_request & + (CPU_INTERRUPT_HARD |CPU_INTERRUPT_FIQ ))) { + +uint64_t ctl; + r =hv_vcpu_get_sys_reg (cpu->hvf->fd,HV_SYS_REG_CNTV_CTL_EL0 , + &ctl); +assert_hvf_ok (r); + +if (!(ctl &1 ) || (ctl &2 )) { +/* Timer disabled or masked, just wait for an IPI. */ target/arm/hvf/hvf.c int hvf_vcpu_exec(CPUState *cpu) · line 616
Quoted 10 lines
target/arm/hvf/hvf.c +10
-0+hvf_wait_for_ipi (cpu,NULL ); +break ; + } + +uint64_t cval; + r =hv_vcpu_get_sys_reg (cpu->hvf->fd,HV_SYS_REG_CNTV_CVAL_EL0 , + &cval); +assert_hvf_ok (r); + +int64_t ticks_to_sleep = cval -mach_absolute_time ();Apple reference recommends to use [1]:
Quoted 6 lines
clock_gettime_nsec_np(CLOCK_UPTIME_RAW)It, internally in Libc, invokes
mach_absolute_time()[2].1. https://developer.apple.com/documentation/kernel/1462446-mach_absolute_time
2. https://opensource.apple.com/source/Libc/Libc-1158.1.2/gen/clock_gettime.c.auto.html
I think that recommendation is because most people want to deal with seconds, not ticks. In our case we specifically want ticks because we're comparing against a ticks value from the guest, so I don't see the benefit of converting from ticks to seconds and back again.
Peter
Quoted 30 lines
Thanks,
RomanQuoted 26 lines
target/arm/hvf/hvf.c +20
-1+if (ticks_to_sleep <0 ) { +break ; + } + +uint64_t seconds = ticks_to_sleep / arm_cpu->gt_cntfrq_hz; +uint64_t nanos = + (ticks_to_sleep - arm_cpu->gt_cntfrq_hz * seconds) * +1000000000 / arm_cpu->gt_cntfrq_hz; + +/* +* Don't sleep for less than 2ms. This is believed to improve +* latency of message passing workloads. +*/ +if (!seconds && nanos <2000000 ) { +break ; + } + +struct timespec ts = { seconds, nanos }; +hvf_wait_for_ipi (cpu, &ts); + }break ;case EC_AA64_HVC :cpu_synchronize_state (cpu); --2.24.3 (Apple Git-128)
Re: [PATCH v3 08/10] arm/hvf: Add a WFI handler
Quoted 29 lines · Peter Collingbourne
Quoted 7 lines · Roman Bolshakov
Quoted 5 lines · Alexander Graf
From: Peter Collingbourne <[email protected]>
Sleep on WFI until the VTIMER is due but allow ourselves to be woken up on IPI.
Quoted 15 lines
target/arm/hvf/hvf.c static int hvf_inject_interrupts(CPUState *cpu) · line 481
Quoted 12 lines
Signed-off-by Peter Collingbourne <[email protected]>[agraf: Remove unused 'set' variable, always advance PC on WFX trap]Signed-off-by Alexander Graf <[email protected]>---
target/arm/hvf/hvf.c +8
-0+static void hvf_wait_for_ipi (CPUState *cpu,struct timespec *ts) +{ +/* +* Use pselect to sleep so that other threads can IPI us while we're +* sleeping. +*/ +qatomic_mb_set (&cpu->thread_kicked, false); +qemu_mutex_unlock_iothread ();I raised a concern earlier, but I don't for sure if a kick could be lost right here. On x86 it could be lost.
If the signal is sent right before the
pselect()it will be blocked i.e. left pending. With thepselect()we get an atomic unblock ofSIG_IPIat the same time as we begin sleeping, which means that we will receive the signal and leave thepselect()immediately.
Quoted 6 lines
I think at some point macOS had an incorrect implementation of
pselect()where the signal mask was non-atomically set in userspace which could lead to the signal being missed but I checked the latest XNU sources and it looks like thepselect()implementation has been moved to the kernel.
Yeah, you're right here.
Quoted 39 lines
Quoted 39 lines
Quoted 21 lines
target/arm/hvf/hvf.c +14
-0+pselect (0 ,0 ,0 ,0 , ts, &cpu->hvf->unblock_ipi_mask); +qemu_mutex_lock_iothread (); +} +int hvf_vcpu_exec (CPUState *cpu) {ARMCPU *arm_cpu =ARM_CPU (cpu);@@ -579,6 +594,46 @@ int hvf_vcpu_exec(CPUState *cpu) }case EC_WFX_TRAP : advance_pc = true; +if (!(syndrome &WFX_IS_WFE ) && !(cpu->interrupt_request & + (CPU_INTERRUPT_HARD |CPU_INTERRUPT_FIQ ))) { + +uint64_t ctl; + r =hv_vcpu_get_sys_reg (cpu->hvf->fd,HV_SYS_REG_CNTV_CTL_EL0 , + &ctl); +assert_hvf_ok (r); + +if (!(ctl &1 ) || (ctl &2 )) { +/* Timer disabled or masked, just wait for an IPI. */ target/arm/hvf/hvf.c int hvf_vcpu_exec(CPUState *cpu) · line 616
Quoted 10 lines
target/arm/hvf/hvf.c +10
-0+hvf_wait_for_ipi (cpu,NULL ); +break ; + } + +uint64_t cval; + r =hv_vcpu_get_sys_reg (cpu->hvf->fd,HV_SYS_REG_CNTV_CVAL_EL0 , + &cval); +assert_hvf_ok (r); + +int64_t ticks_to_sleep = cval -mach_absolute_time ();Apple reference recommends to use [1]:
clock_gettime_nsec_np(CLOCK_UPTIME_RAW)It, internally in Libc, invokes
mach_absolute_time()[2].
Quoted 8 lines
Quoted 2 lines
1. https://developer.apple.com/documentation/kernel/1462446-mach_absolute_time
2. https://opensource.apple.com/source/Libc/Libc-1158.1.2/gen/clock_gettime.c.auto.htmlI think that recommendation is because most people want to deal with seconds, not ticks. In our case we specifically want ticks because we're comparing against a ticks value from the guest, so I don't see the benefit of converting from ticks to seconds and back again.
Thanks for the clarifications, Peter.
Regards,
Roman
PATCH v3 9/10 hvf: arm: Add support for GICv3
In reply to Alexander GrafWe currently only support GICv2 emulation. To also support GICv3, we will need to pass a few system registers into their respective handler functions.
This patch adds handling for all of the required system registers, so that we can run with more than 8 vCPUs.
--- target/arm/hvf/hvf.c | 141 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+)
target/arm/hvf/hvf.c +141 -0
@@ -23,6 +23,7 @@#include "exec/address-spaces.h" #include "hw/irq.h" +#include "hw/intc/gicv3_internal.h" #include "qemu/main-loop.h" #include "sysemu/accel.h" #include "sysemu/cpus.h"
@@ -47,6 +48,33 @@#define SYSREG_CNTPCT_EL0 SYSREG(3, 3, 1, 14, 0)#define SYSREG_PMCCNTR_EL0 SYSREG(3, 3, 0, 9, 13) +#define SYSREG_ICC_AP0R0_EL1 SYSREG(3, 0, 4, 12, 8) +#define SYSREG_ICC_AP0R1_EL1 SYSREG(3, 0, 5, 12, 8) +#define SYSREG_ICC_AP0R2_EL1 SYSREG(3, 0, 6, 12, 8) +#define SYSREG_ICC_AP0R3_EL1 SYSREG(3, 0, 7, 12, 8) +#define SYSREG_ICC_AP1R0_EL1 SYSREG(3, 0, 0, 12, 9) +#define SYSREG_ICC_AP1R1_EL1 SYSREG(3, 0, 1, 12, 9) +#define SYSREG_ICC_AP1R2_EL1 SYSREG(3, 0, 2, 12, 9) +#define SYSREG_ICC_AP1R3_EL1 SYSREG(3, 0, 3, 12, 9) +#define SYSREG_ICC_ASGI1R_EL1 SYSREG(3, 0, 6, 12, 11) +#define SYSREG_ICC_BPR0_EL1 SYSREG(3, 0, 3, 12, 8) +#define SYSREG_ICC_BPR1_EL1 SYSREG(3, 0, 3, 12, 12) +#define SYSREG_ICC_CTLR_EL1 SYSREG(3, 0, 4, 12, 12) +#define SYSREG_ICC_DIR_EL1 SYSREG(3, 0, 1, 12, 11) +#define SYSREG_ICC_EOIR0_EL1 SYSREG(3, 0, 1, 12, 8) +#define SYSREG_ICC_EOIR1_EL1 SYSREG(3, 0, 1, 12, 12) +#define SYSREG_ICC_HPPIR0_EL1 SYSREG(3, 0, 2, 12, 8) +#define SYSREG_ICC_HPPIR1_EL1 SYSREG(3, 0, 2, 12, 12) +#define SYSREG_ICC_IAR0_EL1 SYSREG(3, 0, 0, 12, 8) +#define SYSREG_ICC_IAR1_EL1 SYSREG(3, 0, 0, 12, 12) +#define SYSREG_ICC_IGRPEN0_EL1 SYSREG(3, 0, 6, 12, 12) +#define SYSREG_ICC_IGRPEN1_EL1 SYSREG(3, 0, 7, 12, 12) +#define SYSREG_ICC_PMR_EL1 SYSREG(3, 0, 0, 4, 6) +#define SYSREG_ICC_RPR_EL1 SYSREG(3, 0, 3, 12, 11) +#define SYSREG_ICC_SGI0R_EL1 SYSREG(3, 0, 7, 12, 11) +#define SYSREG_ICC_SGI1R_EL1 SYSREG(3, 0, 5, 12, 11) +#define SYSREG_ICC_SRE_EL1 SYSREG(3, 0, 5, 12, 12) +#define WFX_IS_WFE (1 << 0)struct hvf_reg_match {
@@ -419,6 +447,38 @@ void hvf_kick_vcpu_thread(CPUState *cpu)hv_vcpus_exit (&cpu->hvf->fd,1 ); } +static uint32_t hvf_reg2cp_reg (uint32_t reg) +{ +return ENCODE_AA64_CP_REG (CP_REG_ARM64_SYSREG_CP , + (reg >>10 ) &0xf , + (reg >>1 ) &0xf , + (reg >>20 ) &0x3 , + (reg >>14 ) &0x7 , + (reg >>17 ) &0x7 ); +} + +static uint64_t hvf_sysreg_read_cp (CPUState *cpu,uint32_t reg) +{ +ARMCPU *arm_cpu =ARM_CPU (cpu); +CPUARMState *env = &arm_cpu->env; +const ARMCPRegInfo *ri; +uint64_t val =0 ; + + ri =get_arm_cp_reginfo (arm_cpu->cp_regs,hvf_reg2cp_reg (reg)); +if (ri) { +if (ri->type &ARM_CP_CONST ) { + val = ri->resetvalue; + }else if (ri->readfn) { + val = ri->readfn (env, ri); + }else { + val =CPREG_FIELD64 (env, ri); + } +DPRINTF ("vgic read from %s [val=%016llx]" , ri->name, val); + } + +return val; +} +static uint64_t hvf_sysreg_read (CPUState *cpu,uint32_t reg) {ARMCPU *arm_cpu =ARM_CPU (cpu);
@@ -432,6 +492,39 @@ static uint64_t hvf_sysreg_read(CPUState *cpu, uint32_t reg)case SYSREG_PMCCNTR_EL0 : val =qemu_clock_get_ns (QEMU_CLOCK_VIRTUAL );break ; +case SYSREG_ICC_AP0R0_EL1 : +case SYSREG_ICC_AP0R1_EL1 : +case SYSREG_ICC_AP0R2_EL1 : +case SYSREG_ICC_AP0R3_EL1 : +case SYSREG_ICC_AP1R0_EL1 : +case SYSREG_ICC_AP1R1_EL1 : +case SYSREG_ICC_AP1R2_EL1 : +case SYSREG_ICC_AP1R3_EL1 : +case SYSREG_ICC_ASGI1R_EL1 : +case SYSREG_ICC_BPR0_EL1 : +case SYSREG_ICC_BPR1_EL1 : +case SYSREG_ICC_DIR_EL1 : +case SYSREG_ICC_EOIR0_EL1 : +case SYSREG_ICC_EOIR1_EL1 : +case SYSREG_ICC_HPPIR0_EL1 : +case SYSREG_ICC_HPPIR1_EL1 : +case SYSREG_ICC_IAR0_EL1 : +case SYSREG_ICC_IAR1_EL1 : +case SYSREG_ICC_IGRPEN0_EL1 : +case SYSREG_ICC_IGRPEN1_EL1 : +case SYSREG_ICC_PMR_EL1 : +case SYSREG_ICC_SGI0R_EL1 : +case SYSREG_ICC_SGI1R_EL1 : +case SYSREG_ICC_SRE_EL1 : + val =hvf_sysreg_read_cp (cpu, reg); +break ; +case SYSREG_ICC_CTLR_EL1 : + val =hvf_sysreg_read_cp (cpu, reg); + +/* AP0R registers above 0 don't trap, expose less PRIs to fit */ + val &= ~ICC_CTLR_EL1_PRIBITS_MASK ; + val |=4 <<ICC_CTLR_EL1_PRIBITS_SHIFT ; +break ;default : DPRINTF("unhandled sysreg read %08x (op0=%d op1=%d op2=%d " "crn=%d crm=%d)", reg, (reg >> 20) & 0x3,
@@ -443,6 +536,24 @@ static uint64_t hvf_sysreg_read(CPUState *cpu, uint32_t reg)return val; } +static void hvf_sysreg_write_cp (CPUState *cpu,uint32_t reg,uint64_t val) +{ +ARMCPU *arm_cpu =ARM_CPU (cpu); +CPUARMState *env = &arm_cpu->env; +const ARMCPRegInfo *ri; + + ri =get_arm_cp_reginfo (arm_cpu->cp_regs,hvf_reg2cp_reg (reg)); + +if (ri) { +if (ri->writefn) { + ri->writefn (env, ri, val); + }else { +CPREG_FIELD64 (env, ri) = val; + } +DPRINTF ("vgic write to %s [val=%016llx]" , ri->name, val); + } +} +static void hvf_sysreg_write (CPUState *cpu,uint32_t reg,uint64_t val) {ARMCPU *arm_cpu =ARM_CPU (cpu);
@@ -450,6 +561,36 @@ static void hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val)switch (reg) {case SYSREG_CNTPCT_EL0 :break ; +case SYSREG_ICC_AP0R0_EL1 : +case SYSREG_ICC_AP0R1_EL1 : +case SYSREG_ICC_AP0R2_EL1 : +case SYSREG_ICC_AP0R3_EL1 : +case SYSREG_ICC_AP1R0_EL1 : +case SYSREG_ICC_AP1R1_EL1 : +case SYSREG_ICC_AP1R2_EL1 : +case SYSREG_ICC_AP1R3_EL1 : +case SYSREG_ICC_ASGI1R_EL1 : +case SYSREG_ICC_BPR0_EL1 : +case SYSREG_ICC_BPR1_EL1 : +case SYSREG_ICC_CTLR_EL1 : +case SYSREG_ICC_DIR_EL1 : +case SYSREG_ICC_HPPIR0_EL1 : +case SYSREG_ICC_HPPIR1_EL1 : +case SYSREG_ICC_IAR0_EL1 : +case SYSREG_ICC_IAR1_EL1 : +case SYSREG_ICC_IGRPEN0_EL1 : +case SYSREG_ICC_IGRPEN1_EL1 : +case SYSREG_ICC_PMR_EL1 : +case SYSREG_ICC_SGI0R_EL1 : +case SYSREG_ICC_SGI1R_EL1 : +case SYSREG_ICC_SRE_EL1 : +hvf_sysreg_write_cp (cpu, reg, val); +break ; +case SYSREG_ICC_EOIR0_EL1 : +case SYSREG_ICC_EOIR1_EL1 : +hvf_sysreg_write_cp (cpu, reg, val); +qemu_set_irq (arm_cpu->gt_timer_outputs[GTIMER_VIRT ],0 ); +hv_vcpu_set_vtimer_mask (cpu->hvf->fd, false);default :DPRINTF ("unhandled sysreg write %08x" , reg);break ;
Signature · 1 line
2.24.3 (Apple Git-128)
PATCH v3 10/10 hvf: arm: Implement -cpu host
In reply to Alexander GrafNow that we have working system register sync, we push more target CPU properties into the virtual machine. That might be useful in some situations, but is not the typical case that users want.
So let's add a -cpu host option that allows them to explicitly pass all CPU capabilities of their host CPU into the guest.
--- include/sysemu/hvf.h | 2 ++ target/arm/cpu.c | 9 ++++++--- target/arm/cpu.h | 2 ++ target/arm/hvf/hvf.c | 41 +++++++++++++++++++++++++++++++++++++++++ target/arm/kvm_arm.h | 2 -- 5 files changed, 51 insertions(+), 5 deletions(-)
include/sysemu/hvf.h +2 -0
@@ -19,6 +19,8 @@#ifdef CONFIG_HVF uint32_t hvf_get_supported_cpuid (uint32_t func,uint32_t idx,int reg); +struct ARMCPU ; +void hvf_arm_set_cpu_features_from_host (struct ARMCPU *cpu);extern bool hvf_allowed;#define hvf_enabled () (hvf_allowed)#else /* !CONFIG_HVF */
target/arm/cpu.c +6 -3
@@ -2279,12 +2279,16 @@ static void arm_cpu_class_init(ObjectClass *oc, void *data)#endif } -#ifdef CONFIG_KVM +#if defined(CONFIG_KVM ) || defined(CONFIG_HVF )static void arm_host_initfn (Object *obj) {ARMCPU *cpu =ARM_CPU (obj); +#ifdef CONFIG_KVM kvm_arm_set_cpu_features_from_host (cpu); +#else +hvf_arm_set_cpu_features_from_host (cpu); +#endif if (arm_feature (&cpu->env,ARM_FEATURE_AARCH64 )) {aarch64_add_sve_properties (obj); }
@@ -2296,7 +2300,6 @@ static const TypeInfo host_arm_cpu_type_info = { .parent = TYPE_AARCH64_CPU, .instance_init = arm_host_initfn, }; -#endif static void arm_cpu_instance_init (Object *obj)
@@ -2355,7 +2358,7 @@ static void arm_cpu_register_types(void)type_register_static (&arm_cpu_type_info); -#ifdef CONFIG_KVM +#if defined(CONFIG_KVM ) || defined(CONFIG_HVF )type_register_static (&host_arm_cpu_type_info);#endif
target/arm/cpu.h +2 -0
@@ -2823,6 +2823,8 @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync);#define ARM_CPU_TYPE_NAME (name) (name ARM_CPU_TYPE_SUFFIX)#define CPU_RESOLVING_TYPE TYPE_ARM_CPU +#define TYPE_ARM_HOST_CPU "host-" TYPE_ARM_CPU +#define cpu_signal_handler cpu_arm_signal_handler#define cpu_list arm_cpu_list
target/arm/hvf/hvf.c +41 -0
47 lines not on this page · raw message
target/arm/kvm_arm.h +0 -2
@@ -214,8 +214,6 @@ bool kvm_arm_create_scratch_host_vcpu(const uint32_t *cpus_to_try,*/ void kvm_arm_destroy_scratch_host_vcpu (int *fdarray); -#define TYPE_ARM_HOST_CPU "host-" TYPE_ARM_CPU -/** * ARMHostCPUFeatures: information about the host CPU (identified * by asking the host kernel)
Signature · 1 line
2.24.3 (Apple Git-128)
Patchew URL: https://patchew.org/QEMU/[email protected]/
Patchew URL: https://patchew.org/QEMU/[email protected]/
Hi,
This series seems to have some coding style problems. See output below for more information:
Type: series
Message-id: [email protected]
Subject: [PATCH v3 00/10] hvf: Implement Apple Silicon Support
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 From https://github.com/patchew-project/qemu
* [new tag] patchew/[email protected] -> patchew/[email protected]
Switched to a new branch 'test'
4d827f3 hvf: arm: Implement -cpu host
bde0110 hvf: arm: Add support for GICv3
5c824d7 arm/hvf: Add a WFI handler
68f28c6 arm: Add Hypervisor.framework build target ae48800 hvf: Add Apple Silicon support
ced03a5 hvf: arm: Mark CPU as dirty on reset 9830bf6 arm: Set PSCI to 0.2 for HVF
b2218df hvf: Introduce hvf vcpu struct
0d5f075 hvf: Move common code out
6ae373a hvf: Add hypervisor entitlement to output binaries
=== OUTPUT BEGIN ===
1/10 Checking commit 6ae373af35d9 (hvf: Add hypervisor entitlement to output binaries)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #16:
new file mode 100644
total: 0 errors, 1 warnings, 63 lines checked
Patch 1/10 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
2/10 Checking commit 0d5f07559d56 (hvf: Move common code out)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #38:
new file mode 100644
total: 0 errors, 1 warnings, 1088 lines checked
Patch 2/10 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
3/10 Checking commit b2218df9dabc (hvf: Introduce hvf vcpu struct)
WARNING: line over 80 characters
#138: FILE: target/i386/hvf/hvf.c:213:
+ wvmcs(cpu->hvf->fd, VMCS_ENTRY_CTLS, cap2ctrl(hvf_state->hvf_caps->vmx_cap_entry,
ERROR: "(foo*)" should be "(foo *)"
#746: FILE: target/i386/hvf/x86hvf.c:85:
+ if (hv_vcpu_write_fpstate(cpu_state->hvf->fd, (void*)xsave, 4096)) {
ERROR: "(foo*)" should be "(foo *)"
#827: FILE: target/i386/hvf/x86hvf.c:167:
+ if (hv_vcpu_read_fpstate(cpu_state->hvf->fd, (void*)xsave, 4096)) {
total: 2 errors, 1 warnings, 996 lines checked
Patch 3/10 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
4/10 Checking commit 9830bf633313 (arm: Set PSCI to 0.2 for HVF) 5/10 Checking commit ced03a5fa1eb (hvf: arm: Mark CPU as dirty on reset) 6/10 Checking commit ae4880007b12 (hvf: Add Apple Silicon support)
WARNING: architecture specific defines should be avoided #47: FILE: accel/hvf/hvf-cpus.c:63:
+#ifdef __aarch64__
WARNING: architecture specific defines should be avoided #58: FILE: accel/hvf/hvf-cpus.c:350:
+#ifdef __aarch64__
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #111:
new file mode 100644
WARNING: line over 80 characters
#575: FILE: target/arm/hvf/hvf.c:460:
+ hv_vcpu_set_pending_interrupt(cpu->hvf->fd, HV_INTERRUPT_TYPE_FIQ, true);
WARNING: line over 80 characters
#580: FILE: target/arm/hvf/hvf.c:465:
+ hv_vcpu_set_pending_interrupt(cpu->hvf->fd, HV_INTERRUPT_TYPE_IRQ, true);
total: 0 errors, 5 warnings, 688 lines checked
Patch 6/10 has style problems, please review. If any of these errors are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/10 Checking commit 68f28c62f682 (arm: Add Hypervisor.framework build target)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #47:
new file mode 100644
total: 0 errors, 1 warnings, 36 lines checked
Patch 7/10 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
8/10 Checking commit 5c824d7a4a5e (arm/hvf: Add a WFI handler) 9/10 Checking commit bde0110d9163 (hvf: arm: Add support for GICv3) 10/10 Checking commit 4d827f39b205 (hvf: arm: Implement -cpu host) === OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]