hvf: Implement Apple Silicon Support
v2 cover hvf: Implement Apple Silicon Support
Now that Apple Silicon is widely available, people are obviously excited to try and run virtualized workloads on them, such as Linux and Windows.
This patch set implements a fully functional version to get the ball going on that. With this applied, I can successfully run both Linux and Windows as guests. I am not aware of any limitations specific to Hypervisor.framework apart from:
- Live migration / savevm
- gdbstub debugging (SP register)
Enjoy!
Alex
v1 -> v2:
- New patch: hvf: Actually set SIG_IPI mask
- New patch: hvf: Introduce hvf vcpu struct
- New patch: hvf: arm: Mark CPU as dirty on reset
- Removed patch: hw/arm/virt: Disable highmem when on hypervisor.framework
- Removed patch: arm: Synchronize CPU on PSCI on
- Fix build on 32bit arm
- Merge vcpu kick function patch into ARM enablement
- Implement WFI handling (allows vCPUs to sleep)
- Synchronize system registers (fixes OVMF crashes and reboot)
- Don't always call cpu_synchronize_state()
- Use more fine grained iothread locking
- Populate aa64mmfr0 from hardware
- Make safe to ctrl-C entitlement application
Alexander Graf (8):
hvf: Add hypervisor entitlement to output binaries
hvf: Move common code out
hvf: Actually set SIG_IPI mask
hvf: Introduce hvf vcpu struct
arm: Set PSCI to 0.2 for HVF
hvf: arm: Mark CPU as dirty on reset
hvf: Add Apple Silicon support
arm: Add Hypervisor.framework build target
MAINTAINERS | 14 +- accel/hvf/entitlements.plist | 8 + accel/hvf/hvf-all.c | 56 ++++ accel/hvf/hvf-cpus.c | 483 ++++++++++++++++++++++++++++ accel/hvf/meson.build | 7 + accel/meson.build | 1 + include/hw/core/cpu.h | 3 +- include/sysemu/hvf_int.h | 78 +++++ meson.build | 41 ++- scripts/entitlement.sh | 13 + target/arm/arm-powerctl.c | 1 + target/arm/cpu.c | 6 + target/arm/hvf/hvf.c | 603 +++++++++++++++++++++++++++++++++++ target/arm/hvf/meson.build | 3 + target/arm/meson.build | 2 + target/i386/hvf/hvf-cpus.c | 131 -------- target/i386/hvf/hvf-cpus.h | 25 -- target/i386/hvf/hvf-i386.h | 48 +-- target/i386/hvf/hvf.c | 462 ++++----------------------- target/i386/hvf/meson.build | 1 - target/i386/hvf/vmx.h | 24 +- target/i386/hvf/x86.c | 28 +- target/i386/hvf/x86_descr.c | 26 +- target/i386/hvf/x86_emu.c | 62 ++-- target/i386/hvf/x86_mmu.c | 4 +- target/i386/hvf/x86_task.c | 12 +- target/i386/hvf/x86hvf.c | 221 ++++++------- target/i386/hvf/x86hvf.h | 2 - 28 files changed, 1557 insertions(+), 808 deletions(-) create mode 100644 accel/hvf/entitlements.plist create mode 100644 accel/hvf/hvf-all.c create mode 100644 accel/hvf/hvf-cpus.c create mode 100644 accel/hvf/meson.build create mode 100644 include/sysemu/hvf_int.h create mode 100755 scripts/entitlement.sh create mode 100644 target/arm/hvf/hvf.c create mode 100644 target/arm/hvf/meson.build delete mode 100644 target/i386/hvf/hvf-cpus.c delete mode 100644 target/i386/hvf/hvf-cpus.h
Signature · 1 line
2.24.3 (Apple Git-128)
PATCH v2 1/8 hvf: Add hypervisor entitlement to output binaries
In macOS 11, QEMU only gets access to Hypervisor.framework if it has the respective entitlement. Add an entitlement template and automatically self sign and apply the entitlement in the build.
---
v1 -> v2:
- Make safe to ctrl-C
--- accel/hvf/entitlements.plist | 8 ++++++++ meson.build | 30 ++++++++++++++++++++++++++---- scripts/entitlement.sh | 13 +++++++++++++ 3 files changed, 47 insertions(+), 4 deletions(-) create mode 100644 accel/hvf/entitlements.plist create mode 100755 scripts/entitlement.sh
accel/hvf/entitlements.plist new file +8 -0
@@ -0,0 +1,8 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> +<plist version="1.0"> +<dict> + <key>com.apple.security.hypervisor</key> + <true/> +</dict> +</plist>
meson.build +26 -4
@@ -1844,9 +1844,14 @@ foreach target : target_dirs }] endif foreach exe: execs - emulators += {exe['name']: - executable(exe['name'], exe['sources'], - install: true, + exe_name = exe['name'] + exe_sign = 'CONFIG_HVF' in config_target + if exe_sign + exe_name += '-unsigned' + endif + + emulator = executable(exe_name, exe['sources'], + install: not exe_sign, c_args: c_args, dependencies: arch_deps + deps + exe['dependencies'], objects: lib.extract_all_objects(recursive: true),
@@ -1854,7 +1859,24 @@ foreach target : target_dirs link_depends: [block_syms, qemu_syms] + exe.get('link_depends', []), link_args: link_args, gui_app: exe['gui']) - } + + if exe_sign + exe_full = meson.current_build_dir() / exe['name'] + emulators += {exe['name'] : custom_target(exe['name'], + install: true, + install_dir: get_option('bindir'), + depends: emulator, + output: exe['name'], + command: [ + meson.current_source_dir() / 'scripts/entitlement.sh', + meson.current_build_dir() / exe['name'] + '-unsigned', + meson.current_build_dir() / exe['name'], + meson.current_source_dir() / 'accel/hvf/entitlements.plist' + ]) + } + else + emulators += {exe['name']: emulator} + endif if 'CONFIG_TRACE_SYSTEMTAP' in config_host foreach stp: [
scripts/entitlement.sh new file +13 -0
@@ -0,0 +1,13 @@ +#!/bin/sh -e +# +# Helper script for the build process to apply entitlements + +SRC="$1" +DST="$2" +ENTITLEMENT="$3" + +trap 'rm "$DST.tmp"' exit +cp -af "$SRC" "$DST.tmp" +codesign --entitlements "$ENTITLEMENT" --force -s - "$DST.tmp" +mv "$DST.tmp" "$DST" +trap '' exit
Signature · 1 line
2.24.3 (Apple Git-128)
PATCH v2 2/8 hvf: Move common code out
In reply to Alexander GrafUntil now, Hypervisor.framework has only been available on x86_64 systems. With Apple Silicon shipping now, it extends its reach to aarch64. To prepare for support for multiple architectures, let's move common code out into its own accel directory.
--- MAINTAINERS | 9 +- accel/hvf/hvf-all.c | 56 +++++ accel/hvf/hvf-cpus.c | 464 ++++++++++++++++++++++++++++++++++++ accel/hvf/meson.build | 7 + accel/meson.build | 1 + include/sysemu/hvf_int.h | 69 ++++++ target/i386/hvf/hvf-cpus.c | 131 ---------- target/i386/hvf/hvf-cpus.h | 25 -- target/i386/hvf/hvf-i386.h | 48 +--- target/i386/hvf/hvf.c | 360 +--------------------------- target/i386/hvf/meson.build | 1 - target/i386/hvf/x86hvf.c | 11 +- target/i386/hvf/x86hvf.h | 2 - 13 files changed, 615 insertions(+), 569 deletions(-) create mode 100644 accel/hvf/hvf-all.c create mode 100644 accel/hvf/hvf-cpus.c create mode 100644 accel/hvf/meson.build create mode 100644 include/sysemu/hvf_int.h delete mode 100644 target/i386/hvf/hvf-cpus.c delete mode 100644 target/i386/hvf/hvf-cpus.h
MAINTAINERS +8 -1
@@ -444,9 +444,16 @@ M: Cameron Esfahani <[email protected]> M: Roman Bolshakov <[email protected]> W: https://wiki.qemu.org/Features/HVF S: Maintained -F: accel/stubs/hvf-stub.c F: target/i386/hvf/ + +HVF +M: Cameron Esfahani <[email protected]> +M: Roman Bolshakov <[email protected]> +W: https://wiki.qemu.org/Features/HVF +S: Maintained +F: accel/hvf/ F: include/sysemu/hvf.h +F: include/sysemu/hvf_int.h WHPX CPUs M: Sunil Muthuswamy <[email protected]>
accel/hvf/hvf-all.c new file +56 -0
56 lines not on this page · raw message
accel/hvf/hvf-cpus.c new file +464 -0
464 lines not on this page · raw message
accel/hvf/meson.build new file +7 -0
@@ -0,0 +1,7 @@ +hvf_ss = ss.source_set() +hvf_ss.add(files( + 'hvf-all.c', + 'hvf-cpus.c', +)) + +specific_ss.add_all(when: 'CONFIG_HVF', if_true: hvf_ss)
accel/meson.build +1 -0
@@ -1,5 +1,6 @@ softmmu_ss.add(files('accel.c')) +subdir('hvf') subdir('qtest') subdir('kvm') subdir('tcg')
include/sysemu/hvf_int.h new file +69 -0
69 lines not on this page · raw message
target/i386/hvf/hvf-cpus.c deleted +0 -131
131 lines not on this page · raw message
target/i386/hvf/hvf-cpus.h deleted +0 -25
@@ -1,25 +0,0 @@ -/* -* Accelerator CPUS Interface -* -* Copyright 2020 SUSE LLC -* -* This work is licensed under the terms of the GNU GPL, version 2 or later. -* See the COPYING file in the top-level directory. -*/ - -#ifndef HVF_CPUS_H -#define HVF_CPUS_H - -#include "sysemu/cpus.h" - -extern const CpusAccel hvf_cpus; - -int hvf_init_vcpu (CPUState *); -int hvf_vcpu_exec (CPUState *); -void hvf_cpu_synchronize_state (CPUState *); -void hvf_cpu_synchronize_post_reset (CPUState *); -void hvf_cpu_synchronize_post_init (CPUState *); -void hvf_cpu_synchronize_pre_loadvm (CPUState *); -void hvf_vcpu_destroy (CPUState *); - -#endif /* HVF_CPUS_H */
target/i386/hvf/hvf-i386.h +1 -47
58 lines not on this page · raw message
target/i386/hvf/hvf.c +3 -357
404 lines not on this page · raw message
target/i386/hvf/meson.build +0 -1
@@ -1,6 +1,5 @@ i386_softmmu_ss.add(when: [hvf, 'CONFIG_HVF'], if_true: files( 'hvf.c', - 'hvf-cpus.c', 'x86.c', 'x86_cpuid.c', 'x86_decode.c',
target/i386/hvf/x86hvf.c +6 -5
@@ -20,6 +20,9 @@#include "qemu/osdep.h" #include "qemu-common.h" +#include "sysemu/hvf.h" +#include "sysemu/hvf_int.h" +#include "sysemu/hw_accel.h" #include "x86hvf.h" #include "vmx.h" #include "vmcs.h"
@@ -32,8 +35,6 @@#include <Hypervisor/hv.h> #include <Hypervisor/hv_vmx.h> -#include "hvf-cpus.h" -void hvf_set_segment (struct CPUState *cpu,struct vmx_segment *vmx_seg,SegmentCache *qseg,bool is_tr) {
@@ -437,7 +438,7 @@ int hvf_process_events(CPUState *cpu_state) env->eflags =rreg (cpu_state->hvf_fd,HV_X86_RFLAGS );if (cpu_state->interrupt_request &CPU_INTERRUPT_INIT ) { -hvf_cpu_synchronize_state (cpu_state); +cpu_synchronize_state (cpu_state);do_cpu_init (cpu); }
@@ -451,12 +452,12 @@ int hvf_process_events(CPUState *cpu_state) cpu_state->halted =0 ; }if (cpu_state->interrupt_request &CPU_INTERRUPT_SIPI ) { -hvf_cpu_synchronize_state (cpu_state); +cpu_synchronize_state (cpu_state);do_cpu_sipi (cpu); }if (cpu_state->interrupt_request &CPU_INTERRUPT_TPR ) { cpu_state->interrupt_request &= ~CPU_INTERRUPT_TPR ; -hvf_cpu_synchronize_state (cpu_state); +cpu_synchronize_state (cpu_state);apic_handle_tpr_access_report (cpu->apic_state, env->eip, env->tpr_access_type); }
target/i386/hvf/x86hvf.h +0 -2
@@ -21,8 +21,6 @@#include "x86_descr.h" int hvf_process_events (CPUState *); -int hvf_put_registers (CPUState *); -int hvf_get_registers (CPUState *);bool hvf_inject_interrupts (CPUState *);void hvf_set_segment (struct CPUState *cpu,struct vmx_segment *vmx_seg,SegmentCache *qseg,bool is_tr);
Signature · 1 line
2.24.3 (Apple Git-128)
PATCH v2 3/8 hvf: Actually set SIG_IPI mask
In reply to Alexander GrafIn the hvf init function, we prepare a signal mask for SIG_IPI, but then fail to set it. This seems to work by chance on some systems, where SIGUSR2 is already unmasked by default. It fails however on ARM Big Sur.
So let's set the signal mask as intended.
--- accel/hvf/hvf-cpus.c | 1 + 1 file changed, 1 insertion(+)
accel/hvf/hvf-cpus.c +1 -0
@@ -338,6 +338,7 @@ static int hvf_init_vcpu(CPUState *cpu)pthread_sigmask (SIG_BLOCK ,NULL , &set);sigdelset (&set,SIG_IPI ); +pthread_sigmask (SIG_SETMASK , &set,NULL ); r =hv_vcpu_create ((hv_vcpuid_t *)&cpu->hvf_fd,HV_VCPU_DEFAULT ); cpu->vcpu_dirty =1 ;
Signature · 1 line
2.24.3 (Apple Git-128)
PATCH v2 4/8 hvf: Introduce hvf vcpu struct
In reply to Alexander GrafWe will need more than a single field for hvf going forward. To keep the global vcpu struct uncluttered, let's allocate a special hvf vcpu struct, similar to how hax does it.
--- accel/hvf/hvf-cpus.c | 8 +- include/hw/core/cpu.h | 3 +- include/sysemu/hvf_int.h | 4 + target/i386/hvf/hvf.c | 102 +++++++++--------- target/i386/hvf/vmx.h | 24 +++-- target/i386/hvf/x86.c | 28 ++--- target/i386/hvf/x86_descr.c | 26 ++--- target/i386/hvf/x86_emu.c | 62 +++++------ target/i386/hvf/x86_mmu.c | 4 +- target/i386/hvf/x86_task.c | 12 +-- target/i386/hvf/x86hvf.c | 210 ++++++++++++++++++------------------ 11 files changed, 247 insertions(+), 236 deletions(-)
accel/hvf/hvf-cpus.c +6 -2
@@ -314,10 +314,12 @@ static void hvf_cpu_synchronize_pre_loadvm(CPUState *cpu)static void hvf_vcpu_destroy (CPUState *cpu) { -hv_return_t ret =hv_vcpu_destroy (cpu->hvf_fd); +hv_return_t ret =hv_vcpu_destroy (cpu->hvf->fd);assert_hvf_ok (ret);hvf_arch_vcpu_destroy (cpu); +free (cpu->hvf); + cpu->hvf =NULL ; }static void dummy_signal (int sig)
@@ -328,6 +330,8 @@ static int hvf_init_vcpu(CPUState *cpu) {int r; + cpu->hvf =g_malloc0 (sizeof (*cpu->hvf)); +/* init cpu signals */ sigset_t set;struct sigaction sigact;
@@ -340,7 +344,7 @@ static int hvf_init_vcpu(CPUState *cpu)sigdelset (&set,SIG_IPI );pthread_sigmask (SIG_SETMASK , &set,NULL ); - r =hv_vcpu_create ((hv_vcpuid_t *)&cpu->hvf_fd,HV_VCPU_DEFAULT ); + r =hv_vcpu_create ((hv_vcpuid_t *)&cpu->hvf->fd,HV_VCPU_DEFAULT ); cpu->vcpu_dirty =1 ;assert_hvf_ok (r);
include/hw/core/cpu.h +2 -1
@@ -280,6 +280,7 @@ struct KVMState;struct kvm_run ;struct hax_vcpu_state ; +struct hvf_vcpu_state ;#define TB_JMP_CACHE_BITS 12#define TB_JMP_CACHE_SIZE (1 << TB_JMP_CACHE_BITS)
@@ -463,7 +464,7 @@ struct CPUState {struct hax_vcpu_state *hax_vcpu; -int hvf_fd; +struct hvf_vcpu_state *hvf;/* track IOMMUs whose translations we've cached in the TCG TLB */ GArray *iommu_notifiers;
include/sysemu/hvf_int.h +4 -0
@@ -58,6 +58,10 @@ struct HVFState { };extern HVFState *hvf_state; +struct hvf_vcpu_state { +int fd; +}; +void assert_hvf_ok (hv_return_t ret);int hvf_get_registers (CPUState *cpu);int hvf_put_registers (CPUState *cpu);
target/i386/hvf/hvf.c +51 -51
221 lines not on this page · raw message
target/i386/hvf/vmx.h +13 -11
@@ -30,6 +30,8 @@#include "vmcs.h" #include "cpu.h" #include "x86.h" +#include "sysemu/hvf.h" +#include "sysemu/hvf_int.h" #include "exec/address-spaces.h"
@@ -179,15 +181,15 @@ static inline void macvm_set_rip(CPUState *cpu, uint64_t rip)uint64_t val;/* BUG, should take considering overlap.. */ -wreg (cpu->hvf_fd,HV_X86_RIP , rip); +wreg (cpu->hvf->fd,HV_X86_RIP , rip); env->eip = rip;/* after moving forward in rip, we need to clean INTERRUPTABILITY */ - val =rvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY ); + val =rvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY );if (val & (VMCS_INTERRUPTIBILITY_STI_BLOCKING |VMCS_INTERRUPTIBILITY_MOVSS_BLOCKING )) { env->hflags &= ~HF_INHIBIT_IRQ_MASK ; -wvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY , +wvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY , val & ~(VMCS_INTERRUPTIBILITY_STI_BLOCKING |VMCS_INTERRUPTIBILITY_MOVSS_BLOCKING )); }
@@ -199,9 +201,9 @@ static inline void vmx_clear_nmi_blocking(CPUState *cpu)CPUX86State *env = &x86_cpu->env; env->hflags2 &= ~HF2_NMI_MASK ; -uint32_t gi = (uint32_t )rvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY ); +uint32_t gi = (uint32_t )rvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY ); gi &= ~VMCS_INTERRUPTIBILITY_NMI_BLOCKING ; -wvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY , gi); +wvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY , gi); }static inline void vmx_set_nmi_blocking (CPUState *cpu)
@@ -210,16 +212,16 @@ static inline void vmx_set_nmi_blocking(CPUState *cpu)CPUX86State *env = &x86_cpu->env; env->hflags2 |=HF2_NMI_MASK ; -uint32_t gi = (uint32_t )rvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY ); +uint32_t gi = (uint32_t )rvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY ); gi |=VMCS_INTERRUPTIBILITY_NMI_BLOCKING ; -wvmcs (cpu->hvf_fd,VMCS_GUEST_INTERRUPTIBILITY , gi); +wvmcs (cpu->hvf->fd,VMCS_GUEST_INTERRUPTIBILITY , gi); }static inline void vmx_set_nmi_window_exiting (CPUState *cpu) {uint64_t val; - val =rvmcs (cpu->hvf_fd,VMCS_PRI_PROC_BASED_CTLS ); -wvmcs (cpu->hvf_fd,VMCS_PRI_PROC_BASED_CTLS , val | + val =rvmcs (cpu->hvf->fd,VMCS_PRI_PROC_BASED_CTLS ); +wvmcs (cpu->hvf->fd,VMCS_PRI_PROC_BASED_CTLS , val |VMCS_PRI_PROC_BASED_CTLS_NMI_WINDOW_EXITING ); }
@@ -228,8 +230,8 @@ static inline void vmx_clear_nmi_window_exiting(CPUState *cpu) {uint64_t val; - val =rvmcs (cpu->hvf_fd,VMCS_PRI_PROC_BASED_CTLS ); -wvmcs (cpu->hvf_fd,VMCS_PRI_PROC_BASED_CTLS , val & + val =rvmcs (cpu->hvf->fd,VMCS_PRI_PROC_BASED_CTLS ); +wvmcs (cpu->hvf->fd,VMCS_PRI_PROC_BASED_CTLS , val & ~VMCS_PRI_PROC_BASED_CTLS_NMI_WINDOW_EXITING ); }
target/i386/hvf/x86.c +14 -14
@@ -62,11 +62,11 @@ bool x86_read_segment_descriptor(struct CPUState *cpu, }if (GDT_SEL == sel.ti) { - base =rvmcs (cpu->hvf_fd,VMCS_GUEST_GDTR_BASE ); - limit =rvmcs (cpu->hvf_fd,VMCS_GUEST_GDTR_LIMIT ); + base =rvmcs (cpu->hvf->fd,VMCS_GUEST_GDTR_BASE ); + limit =rvmcs (cpu->hvf->fd,VMCS_GUEST_GDTR_LIMIT ); }else { - base =rvmcs (cpu->hvf_fd,VMCS_GUEST_LDTR_BASE ); - limit =rvmcs (cpu->hvf_fd,VMCS_GUEST_LDTR_LIMIT ); + base =rvmcs (cpu->hvf->fd,VMCS_GUEST_LDTR_BASE ); + limit =rvmcs (cpu->hvf->fd,VMCS_GUEST_LDTR_LIMIT ); }if (sel.index *8 >= limit) {
@@ -85,11 +85,11 @@ bool x86_write_segment_descriptor(struct CPUState *cpu,uint32_t limit;if (GDT_SEL == sel.ti) { - base =rvmcs (cpu->hvf_fd,VMCS_GUEST_GDTR_BASE ); - limit =rvmcs (cpu->hvf_fd,VMCS_GUEST_GDTR_LIMIT ); + base =rvmcs (cpu->hvf->fd,VMCS_GUEST_GDTR_BASE ); + limit =rvmcs (cpu->hvf->fd,VMCS_GUEST_GDTR_LIMIT ); }else { - base =rvmcs (cpu->hvf_fd,VMCS_GUEST_LDTR_BASE ); - limit =rvmcs (cpu->hvf_fd,VMCS_GUEST_LDTR_LIMIT ); + base =rvmcs (cpu->hvf->fd,VMCS_GUEST_LDTR_BASE ); + limit =rvmcs (cpu->hvf->fd,VMCS_GUEST_LDTR_LIMIT ); }if (sel.index *8 >= limit) {
@@ -103,8 +103,8 @@ bool x86_write_segment_descriptor(struct CPUState *cpu,bool x86_read_call_gate (struct CPUState *cpu,struct x86_call_gate *idt_desc,int gate) { -target_ulong base =rvmcs (cpu->hvf_fd,VMCS_GUEST_IDTR_BASE ); -uint32_t limit =rvmcs (cpu->hvf_fd,VMCS_GUEST_IDTR_LIMIT ); +target_ulong base =rvmcs (cpu->hvf->fd,VMCS_GUEST_IDTR_BASE ); +uint32_t limit =rvmcs (cpu->hvf->fd,VMCS_GUEST_IDTR_LIMIT );memset (idt_desc,0 ,sizeof (*idt_desc));if (gate *8 >= limit) {
@@ -118,7 +118,7 @@ bool x86_read_call_gate(struct CPUState *cpu, struct x86_call_gate *idt_desc,bool x86_is_protected (struct CPUState *cpu) { -uint64_t cr0 =rvmcs (cpu->hvf_fd,VMCS_GUEST_CR0 ); +uint64_t cr0 =rvmcs (cpu->hvf->fd,VMCS_GUEST_CR0 );return cr0 &CR0_PE ; }
@@ -136,7 +136,7 @@ bool x86_is_v8086(struct CPUState *cpu)bool x86_is_long_mode (struct CPUState *cpu) { -return rvmcs (cpu->hvf_fd,VMCS_GUEST_IA32_EFER ) &MSR_EFER_LMA ; +return rvmcs (cpu->hvf->fd,VMCS_GUEST_IA32_EFER ) &MSR_EFER_LMA ; }bool x86_is_long64_mode (struct CPUState *cpu)
@@ -149,13 +149,13 @@ bool x86_is_long64_mode(struct CPUState *cpu)bool x86_is_paging_mode (struct CPUState *cpu) { -uint64_t cr0 =rvmcs (cpu->hvf_fd,VMCS_GUEST_CR0 ); +uint64_t cr0 =rvmcs (cpu->hvf->fd,VMCS_GUEST_CR0 );return cr0 &CR0_PG ; }bool x86_is_pae_enabled (struct CPUState *cpu) { -uint64_t cr4 =rvmcs (cpu->hvf_fd,VMCS_GUEST_CR4 ); +uint64_t cr4 =rvmcs (cpu->hvf->fd,VMCS_GUEST_CR4 );return cr4 &CR4_PAE ; }
target/i386/hvf/x86_descr.c +13 -13
@@ -48,47 +48,47 @@ static const struct vmx_segment_field {uint32_t vmx_read_segment_limit (CPUState *cpu,X86Seg seg) { -return (uint32_t )rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].limit); +return (uint32_t )rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].limit); }uint32_t vmx_read_segment_ar (CPUState *cpu,X86Seg seg) { -return (uint32_t )rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].ar_bytes); +return (uint32_t )rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].ar_bytes); }uint64_t vmx_read_segment_base (CPUState *cpu,X86Seg seg) { -return rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].base); +return rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].base); }x68_segment_selector vmx_read_segment_selector (CPUState *cpu,X86Seg seg) {x68_segment_selector sel; - sel.sel =rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].selector); + sel.sel =rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].selector);return sel; }void vmx_write_segment_selector (struct CPUState *cpu,x68_segment_selector selector,X86Seg seg) { -wvmcs (cpu->hvf_fd, vmx_segment_fields[seg].selector, selector.sel); +wvmcs (cpu->hvf->fd, vmx_segment_fields[seg].selector, selector.sel); }void vmx_read_segment_descriptor (struct CPUState *cpu,struct vmx_segment *desc,X86Seg seg) { - desc->sel =rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].selector); - desc->base =rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].base); - desc->limit =rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].limit); - desc->ar =rvmcs (cpu->hvf_fd, vmx_segment_fields[seg].ar_bytes); + desc->sel =rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].selector); + desc->base =rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].base); + desc->limit =rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].limit); + desc->ar =rvmcs (cpu->hvf->fd, vmx_segment_fields[seg].ar_bytes); }void vmx_write_segment_descriptor (CPUState *cpu,struct vmx_segment *desc,X86Seg seg) {const struct vmx_segment_field *sf = &vmx_segment_fields[seg]; -wvmcs (cpu->hvf_fd, sf->base, desc->base); -wvmcs (cpu->hvf_fd, sf->limit, desc->limit); -wvmcs (cpu->hvf_fd, sf->selector, desc->sel); -wvmcs (cpu->hvf_fd, sf->ar_bytes, desc->ar); +wvmcs (cpu->hvf->fd, sf->base, desc->base); +wvmcs (cpu->hvf->fd, sf->limit, desc->limit); +wvmcs (cpu->hvf->fd, sf->selector, desc->sel); +wvmcs (cpu->hvf->fd, sf->ar_bytes, desc->ar); }void x86_segment_descriptor_to_vmx (struct CPUState *cpu,x68_segment_selector selector,struct x86_segment_descriptor *desc,struct vmx_segment *vmx_desc)
target/i386/hvf/x86_emu.c +31 -31
117 lines not on this page · raw message
target/i386/hvf/x86_mmu.c +2 -2
@@ -128,7 +128,7 @@ static bool test_pt_entry(struct CPUState *cpu, struct gpt_translation *pt, pt->err_code |=MMU_PAGE_PT ; } -uint32_t cr0 =rvmcs (cpu->hvf_fd,VMCS_GUEST_CR0 ); +uint32_t cr0 =rvmcs (cpu->hvf->fd,VMCS_GUEST_CR0 );/* check protection */ if (cr0 &CR0_WP ) {if (pt->write_access && !pte_write_access (pte)) {
@@ -173,7 +173,7 @@ static bool walk_gpt(struct CPUState *cpu, target_ulong addr, int err_code, {int top_level, level;bool is_large = false; -target_ulong cr3 =rvmcs (cpu->hvf_fd,VMCS_GUEST_CR3 ); +target_ulong cr3 =rvmcs (cpu->hvf->fd,VMCS_GUEST_CR3 );uint64_t page_mask = pae ?PAE_PTE_PAGE_MASK :LEGACY_PTE_PAGE_MASK ;memset (pt,0 ,sizeof (*pt));
target/i386/hvf/x86_task.c +6 -6
@@ -62,7 +62,7 @@ static void load_state_from_tss32(CPUState *cpu, struct x86_tss_segment32 *tss)X86CPU *x86_cpu =X86_CPU (cpu);CPUX86State *env = &x86_cpu->env; -wvmcs (cpu->hvf_fd,VMCS_GUEST_CR3 , tss->cr3); +wvmcs (cpu->hvf->fd,VMCS_GUEST_CR3 , tss->cr3); env->eip = tss->eip; env->eflags = tss->eflags |2 ;
@@ -111,11 +111,11 @@ static int task_switch_32(CPUState *cpu, x68_segment_selector tss_sel, x68_segmevoid vmx_handle_task_switch (CPUState *cpu,x68_segment_selector tss_sel,int reason,bool gate_valid,uint8_t gate,uint64_t gate_type) { -uint64_t rip =rreg (cpu->hvf_fd,HV_X86_RIP ); +uint64_t rip =rreg (cpu->hvf->fd,HV_X86_RIP );if (!gate_valid || (gate_type !=VMCS_INTR_T_HWEXCEPTION && gate_type !=VMCS_INTR_T_HWINTR && gate_type !=VMCS_INTR_T_NMI )) { -int ins_len =rvmcs (cpu->hvf_fd,VMCS_EXIT_INSTRUCTION_LENGTH ); +int ins_len =rvmcs (cpu->hvf->fd,VMCS_EXIT_INSTRUCTION_LENGTH );macvm_set_rip (cpu, rip + ins_len);return ; }
@@ -174,12 +174,12 @@ void vmx_handle_task_switch(CPUState *cpu, x68_segment_selector tss_sel, int rea//ret = task_switch_16(cpu, tss_sel, old_tss_sel, old_tss_base, &next_tss_desc); VM_PANIC ("task_switch_16" ); -macvm_set_cr0 (cpu->hvf_fd,rvmcs (cpu->hvf_fd,VMCS_GUEST_CR0 ) |CR0_TS ); +macvm_set_cr0 (cpu->hvf->fd,rvmcs (cpu->hvf->fd,VMCS_GUEST_CR0 ) |CR0_TS );x86_segment_descriptor_to_vmx (cpu, tss_sel, &next_tss_desc, &vmx_seg);vmx_write_segment_descriptor (cpu, &vmx_seg,R_TR );store_regs (cpu); -hv_vcpu_invalidate_tlb (cpu->hvf_fd); -hv_vcpu_flush (cpu->hvf_fd); +hv_vcpu_invalidate_tlb (cpu->hvf->fd); +hv_vcpu_flush (cpu->hvf->fd); }
target/i386/hvf/x86hvf.c +105 -105
354 lines not on this page · raw message
Signature · 1 line
2.24.3 (Apple Git-128)
PATCH v2 5/8 arm: Set PSCI to 0.2 for HVF
In reply to Alexander GrafIn Hypervisor.framework, we just pass PSCI calls straight on to the QEMU emulation of it. That means, if TCG is compatible with PSCI 0.2, so are we. Let's transpose that fact in code too.
--- target/arm/cpu.c | 4 ++++ 1 file changed, 4 insertions(+)
target/arm/cpu.c +4 -0
@@ -1062,6 +1062,10 @@ static void arm_cpu_initfn(Object *obj)if (tcg_enabled ()) { cpu->psci_version =2 ;/* TCG implements PSCI 0.2 */ } + +if (hvf_enabled ()) { + cpu->psci_version =2 ;/* HVF uses TCG's PSCI */ + } } static Property arm_cpu_gt_cntfrq_property =
Signature · 1 line
2.24.3 (Apple Git-128)
PATCH v2 6/8 hvf: arm: Mark CPU as dirty on reset
In reply to Alexander GrafWhen clearing internal state of a CPU, we should also make sure that HVF knows about it and can push the new values down to vcpu state.
Make sure that with HVF enabled, we tell it that it should synchronize CPU state on next entry after a reset.
This fixes PSCI handling, because now newly pushed state such as X0 and PC on remote CPU enablement also get pushed into HVF.
--- target/arm/arm-powerctl.c | 1 + target/arm/cpu.c | 2 ++ 2 files changed, 3 insertions(+)
target/arm/arm-powerctl.c +1 -0
@@ -15,6 +15,7 @@#include "arm-powerctl.h" #include "qemu/log.h" #include "qemu/main-loop.h" +#include "sysemu/hw_accel.h" #ifndef DEBUG_ARM_POWERCTL #define DEBUG_ARM_POWERCTL 0
target/arm/cpu.c +2 -0
@@ -411,6 +411,8 @@ static void arm_cpu_reset(DeviceState *dev)#ifndef CONFIG_USER_ONLY if (kvm_enabled ()) {kvm_arm_reset_vcpu (cpu); + }else if (hvf_enabled ()) { + s->vcpu_dirty = true; }#endif
Signature · 1 line
2.24.3 (Apple Git-128)
PATCH v2 7/8 hvf: Add Apple Silicon support
In reply to Alexander GrafWith Apple Silicon available to the masses, it's a good time to add support for driving its virtualization extensions from QEMU.
This patch adds all necessary architecture specific code to get basic VMs working. It's still pretty raw, but definitely functional.
Known limitations:
- Vtimer acknowledgement is hacky
- Should implement more sysregs and fault on invalid ones then
- WFI handling is missing, need to marry it with vtimer
---
v1 -> v2:
- Merge vcpu kick function patch
- Implement WFI handling (allows vCPUs to sleep)
- Synchronize system registers (fixes OVMF crashes and reboot)
- Don't always call cpu_synchronize_state()
- Use more fine grained iothread locking
- Populate aa64mmfr0 from hardware
--- MAINTAINERS | 5 + accel/hvf/hvf-cpus.c | 14 + include/sysemu/hvf_int.h | 7 +- target/arm/hvf/hvf.c | 603 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 628 insertions(+), 1 deletion(-) create mode 100644 target/arm/hvf/hvf.c
MAINTAINERS +5 -0
@@ -439,6 +439,11 @@ F: accel/accel.c F: accel/Makefile.objs F: accel/stubs/Makefile.objs +Apple Silicon HVF CPUs +M: Alexander Graf <[email protected]> +S: Maintained +F: target/arm/hvf/ + X86 HVF CPUs M: Cameron Esfahani <[email protected]> M: Roman Bolshakov <[email protected]>
accel/hvf/hvf-cpus.c +14 -0
@@ -60,6 +60,10 @@#include <Hypervisor/Hypervisor.h> +#ifdef __aarch64__ +#define HV_VM_DEFAULT NULL +#endif +/* Memory slots */ struct mac_slot {
@@ -344,7 +348,11 @@ static int hvf_init_vcpu(CPUState *cpu)sigdelset (&set,SIG_IPI );pthread_sigmask (SIG_SETMASK , &set,NULL ); +#ifdef __aarch64__ + r =hv_vcpu_create (&cpu->hvf->fd, (hv_vcpu_exit_t **)&cpu->hvf->exit,NULL ); +#else r =hv_vcpu_create ((hv_vcpuid_t *)&cpu->hvf->fd,HV_VCPU_DEFAULT ); +#endif cpu->vcpu_dirty =1 ;assert_hvf_ok (r);
@@ -415,8 +423,14 @@ static void hvf_start_vcpu_thread(CPUState *cpu) cpu, QEMU_THREAD_JOINABLE); } +__attribute__((weak))void hvf_kick_vcpu_thread (CPUState *cpu) +{ +cpus_kick_thread (cpu); +} +static const CpusAccel hvf_cpus = { .create_vcpu_thread = hvf_start_vcpu_thread, + .kick_vcpu_thread = hvf_kick_vcpu_thread, .synchronize_post_reset = hvf_cpu_synchronize_post_reset, .synchronize_post_init = hvf_cpu_synchronize_post_init,
include/sysemu/hvf_int.h +6 -1
@@ -11,6 +11,7 @@#ifndef HVF_INT_H #define HVF_INT_H +#include "qemu/osdep.h" #include <Hypervisor/Hypervisor.h> #define HVF_MAX_VCPU 0x10
@@ -59,7 +60,10 @@ struct HVFState {extern HVFState *hvf_state;struct hvf_vcpu_state { -int fd; +uint64_t fd; +void *exit; +struct timespec ts; +bool sleeping; };void assert_hvf_ok (hv_return_t ret);
@@ -69,5 +73,6 @@ int hvf_arch_init_vcpu(CPUState *cpu);void hvf_arch_vcpu_destroy (CPUState *cpu);int hvf_vcpu_exec (CPUState *cpu);hvf_slot *hvf_find_overlap_slot (uint64_t ,uint64_t ); +void hvf_kick_vcpu_thread (CPUState *cpu);#endif
target/arm/hvf/hvf.c new file +603 -0
603 lines not on this page · raw message
Signature · 1 line
2.24.3 (Apple Git-128)
Re: [PATCH v2 7/8] hvf: Add Apple Silicon support
Quoted 25 lines · Alexander Graf
With Apple Silicon available to the masses, it's a good time to add support for driving its virtualization extensions from QEMU.
This patch adds all necessary architecture specific code to get basic VMs working. It's still pretty raw, but definitely functional.
Known limitations:
- Vtimer acknowledgement is hacky
- Should implement more sysregs and fault on invalid ones then
- WFI handling is missing, need to marry it with vtimerSigned-off-by Alexander Graf <[email protected]>(quoted diff) +9
-0+static const struct hvf_reg_match hvf_sreg_match[] = { + { HV_SYS_REG_DBGBVR0_EL1, offsetof(CPUARMState, cp15.dbgbvr[0]) }, + { HV_SYS_REG_DBGBCR0_EL1, offsetof(CPUARMState, cp15.dbgbcr[0]) }, + { HV_SYS_REG_DBGWVR0_EL1, offsetof(CPUARMState, cp15.dbgwvr[0]) }, +}; + +int hvf_get_registers(CPUState *cpu) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu);
Review
Quoted 10 lines
(quoted diff) +10
-0+ CPUARMState *env = &arm_cpu->env; + hv_return_t ret; + uint64_t val; + int i; + + for (i = 0; i < ARRAY_SIZE(hvf_reg_match); i++) { + ret = hv_vcpu_get_reg(cpu->hvf->fd, hvf_reg_match[i].reg, &val); + *(uint64_t *)((void *)env + hvf_reg_match[i].offset) = val; + assert_hvf_ok(ret); + }
Please don't just blat straight into the CPU struct like this. Compare KVM's handling of the cpreg list (data structure set up in kvm_arm_init_cpreg_list() and the various write_*_to_* functions that use it), where we look up the cpreg by its encoding and then that gives us the field offset if necessary and also various flags including ones that say "don't sync this". You'll need to set up the cpreg list for migration anyway.
thanks
-- PMM
PATCH v2 8/8 arm: Add Hypervisor.framework build target
In reply to Alexander GrafNow that we have all logic in place that we need to handle Hypervisor.framework on Apple Silicon systems, let's add CONFIG_HVF for aarch64 as well so that we can build it.
---
v1 -> v2:
- Fix build on 32bit arm
--- meson.build | 11 ++++++++++- target/arm/hvf/meson.build | 3 +++ target/arm/meson.build | 2 ++ 3 files changed, 15 insertions(+), 1 deletion(-) create mode 100644 target/arm/hvf/meson.build
meson.build +10 -1
@@ -74,16 +74,25 @@ else endif accelerator_targets = { 'CONFIG_KVM': kvm_targets } + +if cpu in ['x86', 'x86_64'] + hvf_targets = ['i386-softmmu', 'x86_64-softmmu'] +elif cpu in ['aarch64'] + hvf_targets = ['aarch64-softmmu'] +else + hvf_targets = [] +endif + if cpu in ['x86', 'x86_64', 'arm', 'aarch64'] # i368 emulator provides xenpv machine type for multiple architectures accelerator_targets += { 'CONFIG_XEN': ['i386-softmmu', 'x86_64-softmmu'], + 'CONFIG_HVF': hvf_targets, } endif if cpu in ['x86', 'x86_64'] accelerator_targets += { 'CONFIG_HAX': ['i386-softmmu', 'x86_64-softmmu'], - 'CONFIG_HVF': ['x86_64-softmmu'], 'CONFIG_WHPX': ['i386-softmmu', 'x86_64-softmmu'], } endif
target/arm/hvf/meson.build new file +3 -0
@@ -0,0 +1,3 @@ +arm_softmmu_ss.add(when: [hvf, 'CONFIG_HVF'], if_true: files( + 'hvf.c', +))
target/arm/meson.build +2 -0
@@ -56,5 +56,7 @@ arm_softmmu_ss.add(files( 'psci.c', )) +subdir('hvf') + target_arch += {'arm': arm_ss} target_softmmu_arch += {'arm': arm_softmmu_ss}
Signature · 1 line
2.24.3 (Apple Git-128)
Patchew URL: https://patchew.org/QEMU/[email protected]/
Patchew URL: https://patchew.org/QEMU/[email protected]/
Hi,
This series seems to have some coding style problems. See output below for more information:
Type: series
Message-id: [email protected]
Subject: [PATCH v2 0/8] hvf: Implement Apple Silicon Support
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 From https://github.com/patchew-project/qemu
* [new tag] patchew/[email protected] -> patchew/[email protected]
Switched to a new branch 'test'
ae13163 arm: Add Hypervisor.framework build target 76a368a hvf: Add Apple Silicon support
ed57a9a hvf: arm: Mark CPU as dirty on reset 6d357c1 arm: Set PSCI to 0.2 for HVF
47e1fa7 hvf: Introduce hvf vcpu struct
d10d6e1 hvf: Actually set SIG_IPI mask
5bf37ca hvf: Move common code out
7b8f95a hvf: Add hypervisor entitlement to output binaries
=== OUTPUT BEGIN ===
1/8 Checking commit 7b8f95aad117 (hvf: Add hypervisor entitlement to output binaries)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #16:
new file mode 100644
total: 0 errors, 1 warnings, 63 lines checked
Patch 1/8 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
2/8 Checking commit 5bf37ca7b0a0 (hvf: Move common code out)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #38:
new file mode 100644
total: 0 errors, 1 warnings, 1088 lines checked
Patch 2/8 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
3/8 Checking commit d10d6e1d8ec6 (hvf: Actually set SIG_IPI mask) 4/8 Checking commit 47e1fa754e85 (hvf: Introduce hvf vcpu struct)
WARNING: line over 80 characters
#138: FILE: target/i386/hvf/hvf.c:213:
+ wvmcs(cpu->hvf->fd, VMCS_ENTRY_CTLS, cap2ctrl(hvf_state->hvf_caps->vmx_cap_entry,
ERROR: "(foo*)" should be "(foo *)"
#746: FILE: target/i386/hvf/x86hvf.c:85:
+ if (hv_vcpu_write_fpstate(cpu_state->hvf->fd, (void*)xsave, 4096)) {
ERROR: "(foo*)" should be "(foo *)"
#827: FILE: target/i386/hvf/x86hvf.c:167:
+ if (hv_vcpu_read_fpstate(cpu_state->hvf->fd, (void*)xsave, 4096)) {
total: 2 errors, 1 warnings, 996 lines checked
Patch 4/8 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
5/8 Checking commit 6d357c130379 (arm: Set PSCI to 0.2 for HVF) 6/8 Checking commit ed57a9a142df (hvf: arm: Mark CPU as dirty on reset) 7/8 Checking commit 76a368a42427 (hvf: Add Apple Silicon support)
WARNING: architecture specific defines should be avoided #47: FILE: accel/hvf/hvf-cpus.c:63:
+#ifdef __aarch64__
WARNING: architecture specific defines should be avoided #58: FILE: accel/hvf/hvf-cpus.c:351:
+#ifdef __aarch64__
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #113:
new file mode 100644
WARNING: line over 80 characters
#299: FILE: target/arm/hvf/hvf.c:182:
+ { HV_SYS_REG_CNTV_CTL_EL0, offsetof(CPUARMState, cp15.c14_timer[GTIMER_VIRT].ctl) },
WARNING: line over 80 characters
#300: FILE: target/arm/hvf/hvf.c:183:
+ { HV_SYS_REG_CNTV_CVAL_EL0, offsetof(CPUARMState, cp15.c14_timer[GTIMER_VIRT].cval) },
WARNING: line over 80 characters
#458: FILE: target/arm/hvf/hvf.c:341:
+ hv_vcpu_set_pending_interrupt(cpu->hvf->fd, HV_INTERRUPT_TYPE_FIQ, true);
WARNING: line over 80 characters
#463: FILE: target/arm/hvf/hvf.c:346:
+ hv_vcpu_set_pending_interrupt(cpu->hvf->fd, HV_INTERRUPT_TYPE_IRQ, true);
WARNING: line over 80 characters
#616: FILE: target/arm/hvf/hvf.c:499:
+ r = hv_vcpu_get_sys_reg(cpu->hvf->fd, HV_SYS_REG_CNTV_CTL_EL0, &ctl);
WARNING: line over 80 characters
#618: FILE: target/arm/hvf/hvf.c:501:
+ r = hv_vcpu_get_sys_reg(cpu->hvf->fd, HV_SYS_REG_CNTV_CVAL_EL0, &cval);
WARNING: line over 80 characters
#651: FILE: target/arm/hvf/hvf.c:534:
+ /* Set cpu->hvf->sleeping so that we get a SIG_IPI signal. */
ERROR: memory barrier without comment
#653: FILE: target/arm/hvf/hvf.c:536:
+ smp_mb();
total: 1 errors, 10 warnings, 673 lines checked
Patch 7/8 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
8/8 Checking commit ae13163aef39 (arm: Add Hypervisor.framework build target)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #47:
new file mode 100644
total: 0 errors, 1 warnings, 36 lines checked
Patch 8/8 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]