hvf: Implement Apple Silicon Support
Subsystems (2)
v12 cover hvf: Implement Apple Silicon Support
Now that Apple Silicon is widely available, people are obviously excited to try and run virtualized workloads on them, such as Linux and Windows.
This patch set implements a fully functional version to get the ball going on that. With this applied, I can successfully run both Linux and Windows as guests. I am not aware of any limitations specific to Hypervisor.framework apart from:
- gdbstub debugging (breakpoints)
- missing GICv3 support
- Windows will not work due to UDEF SMC implementation
To use hvf support, please make sure to run -M virt,highmem=off to fit in M1's physical address space limits and use -cpu host.
Enjoy!
Alex
v1 -> v2:
- New patch: hvf: Actually set SIG_IPI mask
- New patch: hvf: Introduce hvf vcpu struct
- New patch: hvf: arm: Mark CPU as dirty on reset
- Removed patch: hw/arm/virt: Disable highmem when on hypervisor.framework
- Removed patch: arm: Synchronize CPU on PSCI on
- Fix build on 32bit arm
- Merge vcpu kick function patch into ARM enablement
- Implement WFI handling (allows vCPUs to sleep)
- Synchronize system registers (fixes OVMF crashes and reboot)
- Don't always call cpu_synchronize_state()
- Use more fine grained iothread locking
- Populate aa64mmfr0 from hardware
- Make safe to ctrl-C entitlement application
v2 -> v3:
- Removed patch: hvf: Actually set SIG_IPI mask
- New patch: hvf: arm: Add support for GICv3
- New patch: hvf: arm: Implement -cpu host
- Advance PC on SMC
- Use cp list interface for sysreg syncs
- Do not set current_cpu
- Fix sysreg isread mask
- Move sysreg handling to functions
- Remove WFI logic again
- Revert to global iothread locking
v3 -> v4:
- Removed patch: hvf: arm: Mark CPU as dirty on reset
- New patch: hvf: Simplify post reset/init/loadvm hooks
- Remove i386-softmmu target (meson.build for hvf target)
- Combine both if statements (PSCI)
- Use hv.h instead of Hypervisor.h for 10.15 compat
- Remove manual inclusion of Hypervisor.h in common .c files
- No longer include Hypervisor.h in arm hvf .c files
- Remove unused exe_full variable
- Reuse exe_name variable
v4 -> v5:
- Use g_free() on destroy
v5 -> v6:
- Switch SYSREG() macro order to the same as asm intrinsics
v6 -> v7:
- Already merged: hvf: Add hypervisor entitlement to output binaries
- Already merged: hvf: x86: Remove unused definitions
- Patch split: hvf: Move common code out
-> hvf: Move assert_hvf_ok() into common directory
-> hvf: Move vcpu thread functions into common directory
-> hvf: Move cpu functions into common directory
-> hvf: Move hvf internal definitions into common header
-> hvf: Make hvf_set_phys_mem() static
-> hvf: Remove use of hv_uvaddr_t and hv_gpaddr_t
-> hvf: Split out common code on vcpu init and destroy
-> hvf: Use cpu_synchronize_state()
-> hvf: Make synchronize functions static
-> hvf: Remove hvf-accel-ops.h - New patch: hvf: arm: Implement PSCI handling
- New patch: arm: Enable Windows 10 trusted SMCCC boot call
- New patch: hvf: arm: Handle Windows 10 SMC call
- Removed patch: "arm: Set PSCI to 0.2 for HVF" (included above)
- Removed patch: "hvf: arm: Add support for GICv3" (deferred to later)
- Remove osdep.h include from hvf_int.h
- Synchronize SIMD registers as well
- Prepend 0x for hex values
- Convert DPRINTF to trace points
- Use main event loop (fixes gdbstub issues)
- Remove PSCI support, inject UDEF on HVC/SMC
- Change vtimer logic to look at ctl.istatus for vtimer mask sync
- Add kick callback again (fixes remote CPU notification)
- Move function define to own header
- Do not propagate SVE features for HVF
- Remove stray whitespace change
- Verify that EL0 and EL1 do not allow AArch32 mode
- Only probe host CPU features once
- Move WFI into function
- Improve comment wording
- Simplify HVF matching logic in meson build file
v7 -> v8:
- checkpatch fixes
- Do not advance for HVC, PC is already updated by hvf (fixes Linux boot)
v8 -> v9:
- [Merged] hvf: Move assert_hvf_ok() into common directory
- [Merged] hvf: Move vcpu thread functions into common directory
- [Merged] hvf: Move cpu functions into common directory
- [Merged] hvf: Move hvf internal definitions into common header
- [Merged] hvf: Make hvf_set_phys_mem() static
- [Merged] hvf: Remove use of hv_uvaddr_t and hv_gpaddr_t
- [Merged] hvf: Split out common code on vcpu init and destroy
- [Merged] hvf: Use cpu_synchronize_state()
- [Merged] hvf: Make synchronize functions static
- [Merged] hvf: Remove hvf-accel-ops.h
- [Merged] hvf: Introduce hvf vcpu struct
- [Merged] hvf: Simplify post reset/init/loadvm hooks
- [Dropped] arm: Enable Windows 10 trusted SMCCC boot call
- [Dropped] hvf: arm: Handle Windows 10 SMC call
- [New] arm: Move PMC register definitions to cpu.h
- [New] hvf: Add execute to dirty log permission bitmap
- [New] hvf: Introduce hvf_arch_init() callback
- [New] hvf: arm: Implement PSCI handling
- [New] hvf: arm: Add rudimentary PMC support
- [New] arm: tcg: Adhere to SMCCC 1.3 section 5.2
- [New] hvf: arm: Adhere to SMCCC 1.3 section 5.2
- Make kick function non-weak
- Use arm_cpu_do_interrupt()
- Remove CNTPCT_EL0 write case
- Inject UDEF on invalid sysreg access
- Add support for OS locking sysregs
- Remove PMCCNTR_EL0 handling
- Print PC on unhandled sysreg trace
- Sync SP (x31) based on SP_EL0/SP_EL1
- Fix SPSR_EL1 mapping
- Only sync known sysregs, assert when syncing fails
- Improve error message on unhandled ec
- Move vtimer sync to post-exit (fixes disable corner case from kvm-unit-tests)
- Add vtimer offset, migration and pause logic
- Flush registers only after EXCP checkers (fixes PSCI on race)
- Remove Windows specifics and just comply with SMCCC spec
- Zero-initialize host_isar
- Use M1 SCTLR reset value
- Add support for cntv offsets
- Improve code readability
- Use new hvf_raise_exception() prototype
- Make cpu_off function void
- Add comment about return value, use -1 for "not found"
- Remove cpu_synchronize_state() when halted
v9 -> v10:
- [Dropped] hvf: arm: Adhere to SMCCC 1.3 section 5.2
- Only handle PSCI calls for the current conduit
- Return true/false
- Return -1 in x0 on unknown SMC/HVC calls
- Move to target/arm/internals.h
- Fail -cpu host class creation gracefully
- Adjust error message on -cpu host realize failure
- Extend SCTLR comment that hvf returns 0 as default value
- Return true/false
- Report errors lazily
- Fix comment
v10 -> v11:
- Treat SMC as UDEF. A follow-up patch set will try to change behavior consistently in TCG as well as HVF.
v11 -> v12:
- Structure SMC/HVC handling code similarly
- Inject UDEF on HVC with SMC conduit
- s/hvf_unknown_hvf/hvf_unknown_hvc/g
- s/hvf_unknown_hvf/hvf_unknown_hvc/g
- Use muldiv64
- Declare ts at start of block
- Improve SPAN comment
Alexander Graf (9):
arm: Move PMC register definitions to internals.h
hvf: Add execute to dirty log permission bitmap
hvf: Introduce hvf_arch_init() callback
hvf: Add Apple Silicon support
hvf: arm: Implement -cpu host
hvf: arm: Implement PSCI handling
arm: Add Hypervisor.framework build target
hvf: arm: Add rudimentary PMC support
arm: tcg: Adhere to SMCCC 1.3 section 5.2
Peter Collingbourne (1):
arm/hvf: Add a WFI handler
MAINTAINERS | 5 + accel/hvf/hvf-accel-ops.c | 21 +- include/sysemu/hvf_int.h | 12 +- meson.build | 8 + target/arm/cpu.c | 17 +- target/arm/cpu.h | 2 + target/arm/helper.c | 44 -- target/arm/hvf/hvf.c | 1274 +++++++++++++++++++++++++++++++++++ target/arm/hvf/meson.build | 3 + target/arm/hvf/trace-events | 11 + target/arm/hvf_arm.h | 19 + target/arm/internals.h | 44 ++ target/arm/kvm_arm.h | 2 - target/arm/meson.build | 2 + target/arm/psci.c | 35 +- target/i386/hvf/hvf.c | 10 + 16 files changed, 1421 insertions(+), 88 deletions(-) create mode 100644 target/arm/hvf/hvf.c create mode 100644 target/arm/hvf/meson.build create mode 100644 target/arm/hvf/trace-events create mode 100644 target/arm/hvf_arm.h
Signature · 1 line
2.30.1 (Apple Git-130)
PATCH v12 1/10 arm: Move PMC register definitions to internals.h
We will need PMC register definitions in accel specific code later. Move all constant definitions to common arm headers so we can reuse them.
---
v9 -> v10:
- Move to target/arm/internals.h
v10 -> v11:
- Fix subject line
--- target/arm/helper.c | 44 ------------------------------------------ target/arm/internals.h | 44 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 44 deletions(-)
target/arm/helper.c +0 -44
@@ -1114,50 +1114,6 @@ static const ARMCPRegInfo v6_cp_reginfo[] = {REGINFO_SENTINEL }; -/* Definitions for the PMU registers */ -#define PMCRN_MASK 0xf800 -#define PMCRN_SHIFT 11 -#define PMCRLC 0x40 -#define PMCRDP 0x20 -#define PMCRX 0x10 -#define PMCRD 0x8 -#define PMCRC 0x4 -#define PMCRP 0x2 -#define PMCRE 0x1 -/* -* Mask of PMCR bits writeable by guest (not including WO bits like C, P, -* which can be written as 1 to trigger behaviour but which stay RAZ). -*/ -#define PMCR_WRITEABLE_MASK (PMCRLC | PMCRDP | PMCRX | PMCRD | PMCRE) - -#define PMXEVTYPER_P 0x80000000 -#define PMXEVTYPER_U 0x40000000 -#define PMXEVTYPER_NSK 0x20000000 -#define PMXEVTYPER_NSU 0x10000000 -#define PMXEVTYPER_NSH 0x08000000 -#define PMXEVTYPER_M 0x04000000 -#define PMXEVTYPER_MT 0x02000000 -#define PMXEVTYPER_EVTCOUNT 0x0000ffff -#define PMXEVTYPER_MASK (PMXEVTYPER_P | PMXEVTYPER_U | PMXEVTYPER_NSK | \ - PMXEVTYPER_NSU | PMXEVTYPER_NSH | \ - PMXEVTYPER_M | PMXEVTYPER_MT | \ - PMXEVTYPER_EVTCOUNT) - -#define PMCCFILTR 0xf8000000 -#define PMCCFILTR_M PMXEVTYPER_M -#define PMCCFILTR_EL0 (PMCCFILTR | PMCCFILTR_M) - -static inline uint32_t pmu_num_counters (CPUARMState *env) -{ -return (env->cp15.c9_pmcr &PMCRN_MASK ) >>PMCRN_SHIFT ; -} - -/* Bits allowed to be set/cleared for PMCNTEN* and PMINTEN* */ -static inline uint64_t pmu_counter_mask (CPUARMState *env) -{ -return (1 <<31 ) | ((1 <<pmu_num_counters (env)) -1 ); -} - typedef struct pm_event { uint16_t number; /* PMEVTYPER.evtCount is 16 bits wide */ /* If the event is supported on this CPU (used to generate PMCEID[01]) */
target/arm/internals.h +44 -0
48 lines not on this page · raw message
Signature · 1 line
2.30.1 (Apple Git-130)
PATCH v12 2/10 hvf: Add execute to dirty log permission bitmap
In reply to Alexander GrafHvf's permission bitmap during and after dirty logging does not include the HV_MEMORY_EXEC permission. At least on Apple Silicon, this leads to instruction faults once dirty logging was enabled.
Add the bit to make it work properly.
--- accel/hvf/hvf-accel-ops.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
accel/hvf/hvf-accel-ops.c +2 -2
@@ -239,12 +239,12 @@ static void hvf_set_dirty_tracking(MemoryRegionSection *section, bool on)if (on) { slot->flags |=HVF_SLOT_LOG ;hv_vm_protect ((uintptr_t )slot->start, (size_t )slot->size, -HV_MEMORY_READ ); +HV_MEMORY_READ |HV_MEMORY_EXEC );/* stop tracking region*/ }else { slot->flags &= ~HVF_SLOT_LOG ;hv_vm_protect ((uintptr_t )slot->start, (size_t )slot->size, -HV_MEMORY_READ |HV_MEMORY_WRITE ); +HV_MEMORY_READ |HV_MEMORY_WRITE |HV_MEMORY_EXEC ); } }
Signature · 1 line
2.30.1 (Apple Git-130)
PATCH v12 3/10 hvf: Introduce hvf_arch_init() callback
In reply to Alexander GrafWe will need to install a migration helper for the ARM hvf backend. Let's introduce an arch callback for the overall hvf init chain to do so.
--- accel/hvf/hvf-accel-ops.c | 3 ++- include/sysemu/hvf_int.h | 1 + target/i386/hvf/hvf.c | 5 +++++ 3 files changed, 8 insertions(+), 1 deletion(-)
accel/hvf/hvf-accel-ops.c +2 -1
@@ -324,7 +324,8 @@ static int hvf_accel_init(MachineState *ms) hvf_state = s;memory_listener_register (&hvf_memory_listener, &address_space_memory); -return 0 ; + +return hvf_arch_init (); }static void hvf_accel_class_init (ObjectClass *oc,void *data)
include/sysemu/hvf_int.h +1 -0
@@ -48,6 +48,7 @@ struct hvf_vcpu_state { };void assert_hvf_ok (hv_return_t ret); +int hvf_arch_init (void );int hvf_arch_init_vcpu (CPUState *cpu);void hvf_arch_vcpu_destroy (CPUState *cpu);int hvf_vcpu_exec (CPUState *);
target/i386/hvf/hvf.c +5 -0
@@ -206,6 +206,11 @@ static inline bool apic_bus_freq_is_known(CPUX86State *env)return env->apic_bus_freq !=0 ; } +int hvf_arch_init (void ) +{ +return 0 ; +} +int hvf_arch_init_vcpu (CPUState *cpu) {X86CPU *x86cpu =X86_CPU (cpu);
Signature · 1 line
2.30.1 (Apple Git-130)
PATCH v12 4/10 hvf: Add Apple Silicon support
In reply to Alexander GrafWith Apple Silicon available to the masses, it's a good time to add support for driving its virtualization extensions from QEMU.
This patch adds all necessary architecture specific code to get basic VMs working, including save/restore.
Known limitations:
- WFI handling is missing (follows in later patch)
- No watchpoint/breakpoint support
---
v1 -> v2:
- Merge vcpu kick function patch
- Implement WFI handling (allows vCPUs to sleep)
- Synchronize system registers (fixes OVMF crashes and reboot)
- Don't always call cpu_synchronize_state()
- Use more fine grained iothread locking
- Populate aa64mmfr0 from hardware
v2 -> v3:
- Advance PC on SMC
- Use cp list interface for sysreg syncs
- Do not set current_cpu
- Fix sysreg isread mask
- Move sysreg handling to functions
- Remove WFI logic again
- Revert to global iothread locking
- Use Hypervisor.h on arm, hv.h does not contain aarch64 definitions
v3 -> v4:
- No longer include Hypervisor.h
v5 -> v6:
- Swap sysreg definition order. This way we're in line with asm outputs.
v6 -> v7:
- Remove osdep.h include from hvf_int.h
- Synchronize SIMD registers as well
- Prepend 0x for hex values
- Convert DPRINTF to trace points
- Use main event loop (fixes gdbstub issues)
- Remove PSCI support, inject UDEF on HVC/SMC
- Change vtimer logic to look at ctl.istatus for vtimer mask sync
- Add kick callback again (fixes remote CPU notification)
v7 -> v8:
- Fix checkpatch errors
v8 -> v9:
- Make kick function non-weak
- Use arm_cpu_do_interrupt()
- Remove CNTPCT_EL0 write case
- Inject UDEF on invalid sysreg access
- Add support for OS locking sysregs
- Remove PMCCNTR_EL0 handling
- Print PC on unhandled sysreg trace
- Sync SP (x31) based on SP_EL0/SP_EL1
- Fix SPSR_EL1 mapping
- Only sync known sysregs, assert when syncing fails
- Improve error message on unhandled ec
- Move vtimer sync to post-exit (fixes disable corner case from kvm-unit-tests)
- Add vtimer offset, migration and pause logic
- Flush registers only after EXCP checkers (fixes PSCI on race)
v9 -> v10:
- Return -1 in x0 on unknown SMC/HVC calls
v10 -> v11:
- Treat SMC as UDEF. A follow-up patch set will try to change behavior consistently in TCG as well as HVF.
v11 -> v12:
- s/hvf_unknown_hvf/hvf_unknown_hvc/g
--- MAINTAINERS | 5 + accel/hvf/hvf-accel-ops.c | 9 + include/sysemu/hvf_int.h | 10 +- meson.build | 1 + target/arm/hvf/hvf.c | 794 ++++++++++++++++++++++++++++++++++++ target/arm/hvf/trace-events | 10 + target/i386/hvf/hvf.c | 5 + 7 files changed, 833 insertions(+), 1 deletion(-) create mode 100644 target/arm/hvf/hvf.c create mode 100644 target/arm/hvf/trace-events
MAINTAINERS +5 -0
@@ -433,6 +433,11 @@ F: accel/accel-*.c F: accel/Makefile.objs F: accel/stubs/Makefile.objs +Apple Silicon HVF CPUs +M: Alexander Graf <[email protected]> +S: Maintained +F: target/arm/hvf/ + X86 HVF CPUs M: Cameron Esfahani <[email protected]> M: Roman Bolshakov <[email protected]>
accel/hvf/hvf-accel-ops.c +9 -0
@@ -60,6 +60,10 @@HVFState *hvf_state; +#ifdef __aarch64__ +#define HV_VM_DEFAULT NULL +#endif +/* Memory slots */ hvf_slot *hvf_find_overlap_slot (uint64_t start,uint64_t size)
@@ -376,7 +380,11 @@ static int hvf_init_vcpu(CPUState *cpu)pthread_sigmask (SIG_BLOCK ,NULL , &set);sigdelset (&set,SIG_IPI ); +#ifdef __aarch64__ + r =hv_vcpu_create (&cpu->hvf->fd, (hv_vcpu_exit_t **)&cpu->hvf->exit,NULL ); +#else r =hv_vcpu_create ((hv_vcpuid_t *)&cpu->hvf->fd,HV_VCPU_DEFAULT ); +#endif cpu->vcpu_dirty =1 ;assert_hvf_ok (r);
@@ -452,6 +460,7 @@ static void hvf_accel_ops_class_init(ObjectClass *oc, void *data)AccelOpsClass *ops =ACCEL_OPS_CLASS (oc); ops->create_vcpu_thread = hvf_start_vcpu_thread; + ops->kick_vcpu_thread = hvf_kick_vcpu_thread; ops->synchronize_post_reset = hvf_cpu_synchronize_post_reset; ops->synchronize_post_init = hvf_cpu_synchronize_post_init;
include/sysemu/hvf_int.h +9 -1
@@ -11,7 +11,11 @@#ifndef HVF_INT_H #define HVF_INT_H +#ifdef __aarch64__ +#include <Hypervisor/Hypervisor.h> +#else #include <Hypervisor/hv.h> +#endif /* hvf_slot flags */ #define HVF_SLOT_LOG (1 << 0)
@@ -40,11 +44,14 @@ struct HVFState {int num_slots;hvf_vcpu_caps *hvf_caps; +uint64_t vtimer_offset; };extern HVFState *hvf_state;struct hvf_vcpu_state { -int fd; +uint64_t fd; +void *exit; +bool vtimer_masked; };void assert_hvf_ok (hv_return_t ret);
@@ -55,5 +62,6 @@ int hvf_vcpu_exec(CPUState *);hvf_slot *hvf_find_overlap_slot (uint64_t ,uint64_t );int hvf_put_registers (CPUState *);int hvf_get_registers (CPUState *); +void hvf_kick_vcpu_thread (CPUState *cpu);#endif
meson.build +1 -0
@@ -2169,6 +2169,7 @@ if have_system or have_user 'accel/tcg', 'hw/core', 'target/arm', + 'target/arm/hvf', 'target/hppa', 'target/i386', 'target/i386/kvm',
target/arm/hvf/hvf.c new file +794 -0
794 lines not on this page · raw message
target/arm/hvf/trace-events new file +10 -0
@@ -0,0 +1,10 @@ +hvf_unhandled_sysreg_read(uint64_t pc, uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_t crm, uint32_t op2) "unhandled sysreg read at pc=0x%"PRIx64": 0x%08x (op0=%d op1=%d crn=%d crm=%d op2=%d)" +hvf_unhandled_sysreg_write(uint64_t pc, uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_t crm, uint32_t op2) "unhandled sysreg write at pc=0x%"PRIx64": 0x%08x (op0=%d op1=%d crn=%d crm=%d op2=%d)" +hvf_inject_fiq(void) "injecting FIQ" +hvf_inject_irq(void) "injecting IRQ" +hvf_data_abort(uint64_t pc, uint64_t va, uint64_t pa, bool isv, bool iswrite, bool s1ptw, uint32_t len, uint32_t srt) "data abort: [pc=0x%"PRIx64" va=0x%016"PRIx64" pa=0x%016"PRIx64" isv=%d iswrite=%d s1ptw=%d len=%d srt=%d]" +hvf_sysreg_read(uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_t crm, uint32_t op2, uint64_t val) "sysreg read 0x%08x (op0=%d op1=%d crn=%d crm=%d op2=%d) = 0x%016"PRIx64 +hvf_sysreg_write(uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_t crm, uint32_t op2, uint64_t val) "sysreg write 0x%08x (op0=%d op1=%d crn=%d crm=%d op2=%d, val=0x%016"PRIx64")" +hvf_unknown_hvc(uint64_t x0) "unknown HVC! 0x%016"PRIx64 +hvf_unknown_smc(uint64_t x0) "unknown SMC! 0x%016"PRIx64 +hvf_exit(uint64_t syndrome, uint32_t ec, uint64_t pc) "exit: 0x%"PRIx64" [ec=0x%x pc=0x%"PRIx64"]"
target/i386/hvf/hvf.c +5 -0
@@ -206,6 +206,11 @@ static inline bool apic_bus_freq_is_known(CPUX86State *env)return env->apic_bus_freq !=0 ; } +void hvf_kick_vcpu_thread (CPUState *cpu) +{ +cpus_kick_thread (cpu); +} +int hvf_arch_init (void ) {return 0 ;
Signature · 1 line
2.30.1 (Apple Git-130)
Re: [PATCH v12 04/10] hvf: Add Apple Silicon support
Quoted 9 lines · Alexander Graf
With Apple Silicon available to the masses, it's a good time to add support for driving its virtualization extensions from QEMU.
This patch adds all necessary architecture specific code to get basic VMs working, including save/restore.
Known limitations:
Quoted 7 lines
- WFI handling is missing (follows in later patch)
- No watchpoint/breakpoint supportSigned-off-by Alexander Graf <[email protected]>Reviewed-by Roman Bolshakov <[email protected]>Reviewed-by Sergio Lopez <[email protected]>Reviewed-by Peter Maydell <[email protected]>
This broke compilation on x86 osx:
target/i386/hvf/hvf.c static inline bool apic_bus_freq_is_known(CPUX86State *env) · line 212
Quoted 10 lines
target/i386/hvf/hvf.c +4
-0@@ -206,6 +206,11 @@ static inline bool apic_bus_freq_is_known(CPUX86State *env)return env->apic_bus_freq !=0 ; } +void hvf_kick_vcpu_thread (CPUState *cpu) +{ +cpus_kick_thread (cpu); +}
This won't build without an extra #include "sysemu/cpus.h". I've folded in the fix.
-- PMM
Re: [PATCH v12 04/10] hvf: Add Apple Silicon support
Quoted 24 lines · Peter Maydell
Quoted 8 lines · Alexander Graf
With Apple Silicon available to the masses, it's a good time to add support for driving its virtualization extensions from QEMU.
This patch adds all necessary architecture specific code to get basic VMs working, including save/restore.
Known limitations:
Quoted 7 lines
- WFI handling is missing (follows in later patch)
- No watchpoint/breakpoint supportSigned-off-by Alexander Graf <[email protected]>Reviewed-by Roman Bolshakov <[email protected]>Reviewed-by Sergio Lopez <[email protected]>Reviewed-by Peter Maydell <[email protected]>This broke compilation on x86 osx:
Quoted 6 lines
target/i386/hvf/hvf.c +0
-0@@ -206,6 +206,11 @@ static inline bool apic_bus_freq_is_known(CPUX86State *env)return env->apic_bus_freq !=0 ; }
Quoted 6 lines
target/i386/hvf/hvf.c static inline bool apic_bus_freq_is_known(CPUX86State *env) · line 212
Quoted 4 lines
target/i386/hvf/hvf.c +4
-0+void hvf_kick_vcpu_thread (CPUState *cpu) +{ +cpus_kick_thread (cpu); +}This won't build without an extra #include "
sysemu/cpus.h". I've folded in the fix.
Thanks a lot! I test compiled all the earlier versions of the patch set on x86 as well, but no longer have easy access to an x86 macOS system :(. They're just so insanely slow in comparison!
Alex
Re: [PATCH v12 04/10] hvf: Add Apple Silicon support
In reply to Alexander GrafHi,
Quoted 57 lines · Alexander Graf
With Apple Silicon available to the masses, it's a good time to add support for driving its virtualization extensions from QEMU.
This patch adds all necessary architecture specific code to get basic VMs working, including save/restore.
Known limitations:
- WFI handling is missing (follows in later patch)
- No watchpoint/breakpoint supportSigned-off-by Alexander Graf <[email protected]>Reviewed-by Roman Bolshakov <[email protected]>Reviewed-by Sergio Lopez <[email protected]>Reviewed-by Peter Maydell <[email protected]>--- MAINTAINERS | 5 + accel/hvf/hvf-accel-ops.c | 9 + include/sysemu/hvf_int.h | 10 +- meson.build | 1 + target/arm/hvf/hvf.c | 794 ++++++++++++++++++++++++++++++++++++ target/arm/hvf/trace-events | 10 + target/i386/hvf/hvf.c | 5 + 7 files changed, 833 insertions(+), 1 deletion(-)create mode 100644
target/arm/hvf/hvf.c
create mode 100644 target/arm/hvf/trace-eventstarget/arm/hvf/hvf.c new file +27
-0+int hvf_arch_init_vcpu (CPUState *cpu) +{ +ARMCPU *arm_cpu =ARM_CPU (cpu); +CPUARMState *env = &arm_cpu->env; +uint32_t sregs_match_len =ARRAY_SIZE (hvf_sreg_match); +uint32_t sregs_cnt =0 ; +uint64_t pfr; +hv_return_t ret; +int i; + + env->aarch64 =1 ; + asmvolatile ("mrs %0, cntfrq_el0" :"=r" (arm_cpu->gt_cntfrq_hz)); + +/* Allocate enough space for our sysreg sync */ + arm_cpu->cpreg_indexes =g_renew (uint64_t, arm_cpu->cpreg_indexes, + sregs_match_len); + arm_cpu->cpreg_values =g_renew (uint64_t, arm_cpu->cpreg_values, + sregs_match_len); + arm_cpu->cpreg_vmstate_indexes =g_renew (uint64_t, + arm_cpu->cpreg_vmstate_indexes, + sregs_match_len); + arm_cpu->cpreg_vmstate_values =g_renew (uint64_t, + arm_cpu->cpreg_vmstate_values, + sregs_match_len); + +memset (arm_cpu->cpreg_values,0 , sregs_match_len *sizeof (uint64_t )); +
target/arm/hvf/hvf.c line 458
Quoted 10 lines
target/arm/hvf/hvf.c new file +10
-0+/* Populate cp list for all known sysregs */ +for (i =0 ; i < sregs_match_len; i++) { +const ARMCPRegInfo *ri; +uint32_t key = hvf_sreg_match[i].key; + + ri =get_arm_cp_reginfo (arm_cpu->cp_regs, key); +if (ri) { +assert (!(ri->type &ARM_CP_NO_RAW )); + hvf_sreg_match[i].cp_idx = sregs_cnt; + arm_cpu->cpreg_indexes[sregs_cnt++] =cpreg_to_kvm_id (key);
So this hash ...:
/*
* Convert a truncated 32 bit hashtable key into the full
* 64 bit KVM register ID.
*/
static uint64_t cpreg_to_kvm_id(uint32_t cpregid)
{
uint64_t kvmid;
if (cpregid & CP_REG_AA64_MASK) {
kvmid = cpregid & ~CP_REG_AA64_MASK;
kvmid |= CP_REG_SIZE_U64 | CP_REG_ARM64;
} else {
kvmid = cpregid & ~(1 << 15);
if (cpregid & (1 << 15)) {
kvmid |= CP_REG_SIZE_U64 | CP_REG_ARM;
} else {
kvmid |= CP_REG_SIZE_U32 | CP_REG_ARM;
}
}
return kvmid;
}... just happens to work the same way for HVF?
Quoted 31 lines
target/arm/hvf/hvf.c new file +31
-0+ }else { + hvf_sreg_match[i].cp_idx =-1 ; + } + } + arm_cpu->cpreg_array_len = sregs_cnt; + arm_cpu->cpreg_vmstate_array_len = sregs_cnt; + +assert (write_cpustate_to_list (arm_cpu, false)); + +/* Set CP_NO_RAW system registers on init */ + ret =hv_vcpu_set_sys_reg (cpu->hvf->fd,HV_SYS_REG_MIDR_EL1 , + arm_cpu->midr); +assert_hvf_ok (ret); + + ret =hv_vcpu_set_sys_reg (cpu->hvf->fd,HV_SYS_REG_MPIDR_EL1 , + arm_cpu->mp_affinity); +assert_hvf_ok (ret); + + ret =hv_vcpu_get_sys_reg (cpu->hvf->fd,HV_SYS_REG_ID_AA64PFR0_EL1 , &pfr); +assert_hvf_ok (ret); + pfr |= env->gicv3state ? (1 <<24 ) :0 ; + ret =hv_vcpu_set_sys_reg (cpu->hvf->fd,HV_SYS_REG_ID_AA64PFR0_EL1 , pfr); +assert_hvf_ok (ret); + +/* We're limited to underlying hardware caps, override internal versions */ + ret =hv_vcpu_get_sys_reg (cpu->hvf->fd,HV_SYS_REG_ID_AA64MMFR0_EL1 , + &arm_cpu->isar.id_aa64mmfr0); +assert_hvf_ok (ret); + +return 0 ; +}
Re: [PATCH v12 04/10] hvf: Add Apple Silicon support
Quoted 96 lines · Philippe Mathieu-Daudé
Hi,
Quoted 58 lines · Alexander Graf
With Apple Silicon available to the masses, it's a good time to add support for driving its virtualization extensions from QEMU.
This patch adds all necessary architecture specific code to get basic VMs working, including save/restore.
Known limitations:
- WFI handling is missing (follows in later patch)
- No watchpoint/breakpoint supportSigned-off-by Alexander Graf <[email protected]>Reviewed-by Roman Bolshakov <[email protected]>Reviewed-by Sergio Lopez <[email protected]>Reviewed-by Peter Maydell <[email protected]>---
MAINTAINERS | 5 + accel/hvf/hvf-accel-ops.c | 9 + include/sysemu/hvf_int.h | 10 +-
meson.build | 1 +
target/arm/hvf/hvf.c | 794 ++++++++++++++++++++++++++++++++++++ target/arm/hvf/trace-events | 10 +
target/i386/hvf/hvf.c | 5 +7 files changed, 833 insertions(+), 1 deletion(-)create mode 100644
target/arm/hvf/hvf.c
create mode 100644 target/arm/hvf/trace-eventstarget/arm/hvf/hvf.c new file +26
-0+int hvf_arch_init_vcpu (CPUState *cpu) +{ + ARMCPU *arm_cpu = ARM_CPU(cpu); + CPUARMState *env = &arm_cpu->env; + uint32_t sregs_match_len = ARRAY_SIZE(hvf_sreg_match); + uint32_t sregs_cnt = 0; + uint64_t pfr; + hv_return_t ret; + int i; + + env->aarch64 = 1; + asm volatile("mrs %0, cntfrq_el0" : "=r"(arm_cpu->gt_cntfrq_hz)); + + /* Allocate enough space for our sysreg sync */ + arm_cpu->cpreg_indexes = g_renew(uint64_t, arm_cpu->cpreg_indexes, + sregs_match_len); + arm_cpu->cpreg_values = g_renew(uint64_t, arm_cpu->cpreg_values, + sregs_match_len); + arm_cpu->cpreg_vmstate_indexes = g_renew(uint64_t, + arm_cpu->cpreg_vmstate_indexes, + sregs_match_len); + arm_cpu->cpreg_vmstate_values = g_renew(uint64_t, + arm_cpu->cpreg_vmstate_values, + sregs_match_len); + + memset(arm_cpu->cpreg_values, 0, sregs_match_len *
sizeof(uint64_t));target/arm/hvf/hvf.c new file +1
-0+target/arm/hvf/hvf.c line 458
Quoted 10 lines
target/arm/hvf/hvf.c new file +10
-0+ /* Populate cp list for all known sysregs */ + for (i = 0; i < sregs_match_len; i++) { + const ARMCPRegInfo *ri; + uint32_t key = hvf_sreg_match[i].key; + + ri = get_arm_cp_reginfo(arm_cpu->cp_regs, key); + if (ri) { + assert(!(ri->type & ARM_CP_NO_RAW)); + hvf_sreg_match[i].cp_idx = sregs_cnt; + arm_cpu->cpreg_indexes[sregs_cnt++] = cpreg_to_kvm_id(key);So this hash ...:
/*
* Convert a truncated 32 bit hashtable key into the full
* 64 bit KVM register ID.
*/
staticuint64_tcpreg_to_kvm_id(uint32_t cpregid){
uint64_tkvmid;if (cpregid &
CP_REG_AA64_MASK) { kvmid = cpregid & ~CP_REG_AA64_MASK; kvmid |=CP_REG_SIZE_U64|CP_REG_ARM64; } else {
kvmid = cpregid & ~(1 << 15);
if (cpregid & (1 << 15)) {
kvmid |=CP_REG_SIZE_U64|CP_REG_ARM; } else {
kvmid |=CP_REG_SIZE_U32|CP_REG_ARM; }
}
return kvmid;
}
Quoted 1 line
... just happens to work the same way for HVF?
It never feeds into HVF - we only use these values as unique identifiers inside QEMU, no? See write_cpustate_to_list() and write_list_to_cpustate() for reference.
Alex
Re: [PATCH v12 04/10] hvf: Add Apple Silicon support
In reply to Alexander GrafHi Alexander,
Quoted 48 lines · Alexander Graf
target/arm/hvf/hvf.c new file +48
-0+int hvf_arch_init_vcpu (CPUState *cpu) +{ +ARMCPU *arm_cpu =ARM_CPU (cpu); +CPUARMState *env = &arm_cpu->env; +uint32_t sregs_match_len =ARRAY_SIZE (hvf_sreg_match); +uint32_t sregs_cnt =0 ; +uint64_t pfr; +hv_return_t ret; +int i; + + env->aarch64 =1 ; + asmvolatile ("mrs %0, cntfrq_el0" :"=r" (arm_cpu->gt_cntfrq_hz)); + +/* Allocate enough space for our sysreg sync */ + arm_cpu->cpreg_indexes =g_renew (uint64_t, arm_cpu->cpreg_indexes, + sregs_match_len); + arm_cpu->cpreg_values =g_renew (uint64_t, arm_cpu->cpreg_values, + sregs_match_len); + arm_cpu->cpreg_vmstate_indexes =g_renew (uint64_t, + arm_cpu->cpreg_vmstate_indexes, + sregs_match_len); + arm_cpu->cpreg_vmstate_values =g_renew (uint64_t, + arm_cpu->cpreg_vmstate_values, + sregs_match_len); + +memset (arm_cpu->cpreg_values,0 , sregs_match_len *sizeof (uint64_t )); + +/* Populate cp list for all known sysregs */ +for (i =0 ; i < sregs_match_len; i++) { +const ARMCPRegInfo *ri; +uint32_t key = hvf_sreg_match[i].key; + + ri =get_arm_cp_reginfo (arm_cpu->cp_regs, key); +if (ri) { +assert (!(ri->type &ARM_CP_NO_RAW )); + hvf_sreg_match[i].cp_idx = sregs_cnt; + arm_cpu->cpreg_indexes[sregs_cnt++] =cpreg_to_kvm_id (key); + }else { + hvf_sreg_match[i].cp_idx =-1 ; + } + } + arm_cpu->cpreg_array_len = sregs_cnt; + arm_cpu->cpreg_vmstate_array_len = sregs_cnt; + +assert (write_cpustate_to_list (arm_cpu, false)); + +/* Set CP_NO_RAW system registers on init */ + ret = hv_vcpu_set_sys_reg(cpu->hvf->fd, HV_SYS_REG_MIDR_EL1,
target/arm/hvf/hvf.c line 479
Quoted 10 lines
target/arm/hvf/hvf.c new file +10
-0+ arm_cpu->midr); +assert_hvf_ok (ret); + + ret =hv_vcpu_set_sys_reg (cpu->hvf->fd,HV_SYS_REG_MPIDR_EL1 , + arm_cpu->mp_affinity); +assert_hvf_ok (ret); + + ret =hv_vcpu_get_sys_reg (cpu->hvf->fd,HV_SYS_REG_ID_AA64PFR0_EL1 , &pfr); +assert_hvf_ok (ret); + pfr |= env->gicv3state ? (1 <<24 ) :0 ;
It seems that this happens before arm_gicv3_common_realize()/gicv3_set_gicv3state() so we always get a *NULL* env->gicv3state.
Noticed by starting a guest with '-machine gic-version=3' but the guest visible ID_AA64PFR0_EL1.GIC is 0.
target/arm/hvf/hvf.c line 489
Quoted 10 lines
target/arm/hvf/hvf.c new file +10
-0+ ret =hv_vcpu_set_sys_reg (cpu->hvf->fd,HV_SYS_REG_ID_AA64PFR0_EL1 , pfr); +assert_hvf_ok (ret); + +/* We're limited to underlying hardware caps, override internal versions */ + ret =hv_vcpu_get_sys_reg (cpu->hvf->fd,HV_SYS_REG_ID_AA64MMFR0_EL1 , + &arm_cpu->isar.id_aa64mmfr0); +assert_hvf_ok (ret); + +return 0 ; +}
Thanks,
Zenghui
PATCH v12 5/10 arm/hvf: Add a WFI handler
In reply to Alexander GrafFrom: Peter Collingbourne <[email protected]>
Sleep on WFI until the VTIMER is due but allow ourselves to be woken up on IPI.
In this implementation IPI is blocked on the CPU thread at startup and pselect() is used to atomically unblock the signal and begin sleeping. The signal is sent unconditionally so there's no need to worry about races between actually sleeping and the "we think we're sleeping" state. It may lead to an extra wakeup but that's better than missing it entirely.
[agraf: Remove unused 'set' variable, always advance PC on WFX trap,
support vm stop / continue operations and cntv offsets]
---
v6 -> v7:
- Move WFI into function
- Improve comment wording
v8 -> v9:
- Add support for cntv offsets
- Improve code readability
v11 -> v12:
- Use muldiv64
- Declare ts at start of block
--- accel/hvf/hvf-accel-ops.c | 5 +-- include/sysemu/hvf_int.h | 1 + target/arm/hvf/hvf.c | 79 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 82 insertions(+), 3 deletions(-)
accel/hvf/hvf-accel-ops.c +2 -3
@@ -370,15 +370,14 @@ static int hvf_init_vcpu(CPUState *cpu) cpu->hvf =g_malloc0 (sizeof (*cpu->hvf));/* init cpu signals */ -sigset_t set;struct sigaction sigact;memset (&sigact,0 ,sizeof (sigact)); sigact.sa_handler = dummy_signal;sigaction (SIG_IPI , &sigact,NULL ); -pthread_sigmask (SIG_BLOCK ,NULL , &set); -sigdelset (&set,SIG_IPI ); +pthread_sigmask (SIG_BLOCK ,NULL , &cpu->hvf->unblock_ipi_mask); +sigdelset (&cpu->hvf->unblock_ipi_mask,SIG_IPI );#ifdef __aarch64__ r =hv_vcpu_create (&cpu->hvf->fd, (hv_vcpu_exit_t **)&cpu->hvf->exit,NULL );
include/sysemu/hvf_int.h +1 -0
@@ -52,6 +52,7 @@ struct hvf_vcpu_state {uint64_t fd;void *exit;bool vtimer_masked; +sigset_t unblock_ipi_mask; };void assert_hvf_ok (hv_return_t ret);
target/arm/hvf/hvf.c +79 -0
103 lines not on this page · raw message
Signature · 1 line
2.30.1 (Apple Git-130)
PATCH v12 6/10 hvf: arm: Implement -cpu host
In reply to Alexander GrafNow that we have working system register sync, we push more target CPU properties into the virtual machine. That might be useful in some situations, but is not the typical case that users want.
So let's add a -cpu host option that allows them to explicitly pass all CPU capabilities of their host CPU into the guest.
---
v6 -> v7:
- Move function define to own header
- Do not propagate SVE features for HVF
- Remove stray whitespace change
- Verify that EL0 and EL1 do not allow AArch32 mode
- Only probe host CPU features once
v8 -> v9:
- Zero-initialize host_isar
- Use M1 SCTLR reset value
v9 -> v10:
- Fail -cpu host class creation gracefully
- Adjust error message on -cpu host realize failure
- Extend SCTLR comment that hvf returns 0 as default value
- Return true/false
- Report errors lazily
v11 -> v12:
- Improve SPAN comment
--- target/arm/cpu.c | 13 ++++-- target/arm/cpu.h | 2 + target/arm/hvf/hvf.c | 95 ++++++++++++++++++++++++++++++++++++++++++++ target/arm/hvf_arm.h | 19 +++++++++ target/arm/kvm_arm.h | 2 - 5 files changed, 125 insertions(+), 6 deletions(-) create mode 100644 target/arm/hvf_arm.h
target/arm/cpu.c +9 -4
@@ -39,6 +39,7 @@#include "sysemu/tcg.h" #include "sysemu/hw_accel.h" #include "kvm_arm.h" +#include "hvf_arm.h" #include "disas/capstone.h" #include "fpu/softfloat.h"
@@ -1400,8 +1401,8 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)* this is the first point where we can report it. */ if (cpu->host_cpu_probe_failed) { -if (!kvm_enabled ()) { -error_setg (errp,"The 'host' CPU type can only be used with KVM" ); +if (!kvm_enabled () && !hvf_enabled ()) { +error_setg (errp,"The 'host' CPU type can only be used with KVM or HVF" ); }else {error_setg (errp,"Failed to retrieve host CPU features" ); }
@@ -2061,15 +2062,19 @@ static void arm_cpu_class_init(ObjectClass *oc, void *data)#endif /* CONFIG_TCG */ } -#ifdef CONFIG_KVM +#if defined(CONFIG_KVM ) || defined(CONFIG_HVF )static void arm_host_initfn (Object *obj) {ARMCPU *cpu =ARM_CPU (obj); +#ifdef CONFIG_KVM kvm_arm_set_cpu_features_from_host (cpu);if (arm_feature (&cpu->env,ARM_FEATURE_AARCH64 )) {aarch64_add_sve_properties (obj); } +#else +hvf_arm_set_cpu_features_from_host (cpu); +#endif arm_cpu_post_init (obj); }
@@ -2129,7 +2134,7 @@ static void arm_cpu_register_types(void) {type_register_static (&arm_cpu_type_info); -#ifdef CONFIG_KVM +#if defined(CONFIG_KVM ) || defined(CONFIG_HVF )type_register_static (&host_arm_cpu_type_info);#endif }
target/arm/cpu.h +2 -0
@@ -3015,6 +3015,8 @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync);#define ARM_CPU_TYPE_NAME (name) (name ARM_CPU_TYPE_SUFFIX)#define CPU_RESOLVING_TYPE TYPE_ARM_CPU +#define TYPE_ARM_HOST_CPU "host-" TYPE_ARM_CPU +#define cpu_signal_handler cpu_arm_signal_handler#define cpu_list arm_cpu_list
target/arm/hvf/hvf.c +95 -0
113 lines not on this page · raw message
target/arm/hvf_arm.h new file +19 -0
@@ -0,0 +1,19 @@ +/* +* QEMU Hypervisor.framework (HVF) support -- ARM specifics +* +* Copyright (c) 2021 Alexander Graf +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#ifndef QEMU_HVF_ARM_H +#define QEMU_HVF_ARM_H + +#include "qemu/accel.h" +#include "cpu.h" + +void hvf_arm_set_cpu_features_from_host (struct ARMCPU *cpu); + +#endif
target/arm/kvm_arm.h +0 -2
@@ -214,8 +214,6 @@ bool kvm_arm_create_scratch_host_vcpu(const uint32_t *cpus_to_try,*/ void kvm_arm_destroy_scratch_host_vcpu (int *fdarray); -#define TYPE_ARM_HOST_CPU "host-" TYPE_ARM_CPU -/** * ARMHostCPUFeatures: information about the host CPU (identified * by asking the host kernel)
Signature · 1 line
2.30.1 (Apple Git-130)
Re: [PATCH v12 06/10] hvf: arm: Implement -cpu host
Quoted 24 lines · Alexander Graf
Now that we have working system register sync, we push more target CPU properties into the virtual machine. That might be useful in some situations, but is not the typical case that users want.
So let's add a -cpu host option that allows them to explicitly pass all CPU capabilities of their host CPU into the guest.
Signed-off-by Alexander Graf <[email protected]>Acked-by Roman Bolshakov <[email protected]>Reviewed-by Sergio Lopez <[email protected]>Reviewed-by Peter Maydell <[email protected]>---
target/arm/hvf_arm.h new file +4
-0@@ -0,0 +1,19 @@ +/* +* QEMU Hypervisor.framework (HVF) support -- ARM specifics +* +* Copyright (c) 2021 Alexander Graf
target/arm/hvf_arm.h line 14
Quoted 10 lines
target/arm/hvf_arm.h new file +10
-0+* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#ifndef QEMU_HVF_ARM_H +#define QEMU_HVF_ARM_H + +#include "qemu/accel.h"
This include shouldn't be necessary.
Quoted 5 lines
target/arm/hvf_arm.h new file +5
-0+#include "cpu.h" + +void hvf_arm_set_cpu_features_from_host (struct ARMCPU *cpu); + +#endif
Re: [PATCH v12 06/10] hvf: arm: Implement -cpu host
Quoted 30 lines · Philippe Mathieu-Daudé
Quoted 28 lines · Alexander Graf
Now that we have working system register sync, we push more target CPU properties into the virtual machine. That might be useful in some situations, but is not the typical case that users want.
So let's add a -cpu host option that allows them to explicitly pass all CPU capabilities of their host CPU into the guest.
Signed-off-by Alexander Graf <[email protected]>Acked-by Roman Bolshakov <[email protected]>Reviewed-by Sergio Lopez <[email protected]>Reviewed-by Peter Maydell <[email protected]>---
target/arm/hvf_arm.h new file +8
-0@@ -0,0 +1,19 @@ +/* +* QEMU Hypervisor.framework (HVF) support -- ARM specifics +* +* Copyright (c) 2021 Alexander Graf +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*
Quoted 8 lines
target/arm/hvf_arm.h line 14
Quoted 6 lines
target/arm/hvf_arm.h new file +6
-0+*/ + +#ifndef QEMU_HVF_ARM_H +#define QEMU_HVF_ARM_H + +#include "qemu/accel.h" This include shouldn't be necessary.
Yep, and we don't need to add it to any .c files, they all seem to compile OK (on osx and Linux) without explicitly including this anywhere. Since this is the only change needed to an otherwise fully-reviewed patchset I'll just fold this change in locally.
-- PMM
PATCH v12 7/10 hvf: arm: Implement PSCI handling
In reply to Alexander GrafWe need to handle PSCI calls. Most of the TCG code works for us, but we can simplify it to only handle aa64 mode and we need to handle SUSPEND differently.
This patch takes the TCG code as template and duplicates it in HVF.
To tell the guest that we support PSCI 0.2 now, update the check in arm_cpu_initfn() as well.
---
v6 -> v7:
- This patch integrates "arm: Set PSCI to 0.2 for HVF"
v7 -> v8:
- Do not advance for HVC, PC is already updated by hvf
- Fix checkpatch error
v8 -> v9:
- Use new hvf_raise_exception() prototype
- Make cpu_off function void
- Add comment about return value, use -1 for "not found"
- Remove cpu_synchronize_state() when halted
v9 -> v10:
- Only handle PSCI calls for the current conduit
- Return true/false
v10 -> v11:
- Inject UDEF on SMC. Will be changed in TCG+HVF in a later patch set.
v11 -> v12:
- Structure SMC/HVC handling code similarly
- Inject UDEF on HVC with SMC conduit
- s/hvf_unknown_hvf/hvf_unknown_hvc/g
--- target/arm/cpu.c | 4 +- target/arm/hvf/hvf.c | 141 ++++++++++++++++++++++++++++++++++-- target/arm/hvf/trace-events | 1 + 3 files changed, 139 insertions(+), 7 deletions(-)
target/arm/cpu.c +2 -2
@@ -1096,8 +1096,8 @@ static void arm_cpu_initfn(Object *obj) cpu->psci_version =1 ;/* By default assume PSCI v0.1 */ cpu->kvm_target =QEMU_KVM_ARM_TARGET_NONE ; -if (tcg_enabled ()) { - cpu->psci_version =2 ;/* TCG implements PSCI 0.2 */ +if (tcg_enabled () ||hvf_enabled ()) { + cpu->psci_version =2 ;/* TCG and HVF implement PSCI 0.2 */ } }
target/arm/hvf/hvf.c +136 -5
168 lines not on this page · raw message
target/arm/hvf/trace-events +1 -0
@@ -8,3 +8,4 @@ hvf_sysreg_write(uint32_t reg, uint32_t op0, uint32_t op1, uint32_t crn, uint32_ hvf_unknown_hvc(uint64_t x0) "unknown HVC! 0x%016"PRIx64 hvf_unknown_smc(uint64_t x0) "unknown SMC! 0x%016"PRIx64 hvf_exit(uint64_t syndrome, uint32_t ec, uint64_t pc) "exit: 0x%"PRIx64" [ec=0x%x pc=0x%"PRIx64"]" +hvf_psci_call(uint64_t x0, uint64_t x1, uint64_t x2, uint64_t x3, uint32_t cpuid) "PSCI Call x0=0x%016"PRIx64" x1=0x%016"PRIx64" x2=0x%016"PRIx64" x3=0x%016"PRIx64" cpu=0x%x"
Signature · 1 line
2.30.1 (Apple Git-130)
PATCH v12 8/10 arm: Add Hypervisor.framework build target
In reply to Alexander GrafNow that we have all logic in place that we need to handle Hypervisor.framework on Apple Silicon systems, let's add CONFIG_HVF for aarch64 as well so that we can build it.
---
v1 -> v2:
- Fix build on 32bit arm
v3 -> v4:
- Remove i386-softmmu target
v6 -> v7:
- Simplify HVF matching logic in meson build file
--- meson.build | 7 +++++++ target/arm/hvf/meson.build | 3 +++ target/arm/meson.build | 2 ++ 3 files changed, 12 insertions(+) create mode 100644 target/arm/hvf/meson.build
meson.build +7 -0
@@ -77,6 +77,13 @@ else endif accelerator_targets = { 'CONFIG_KVM': kvm_targets } + +if cpu in ['aarch64'] + accelerator_targets += { + 'CONFIG_HVF': ['aarch64-softmmu'] + } +endif + if cpu in ['x86', 'x86_64', 'arm', 'aarch64'] # i386 emulator provides xenpv machine type for multiple architectures accelerator_targets += {
target/arm/hvf/meson.build new file +3 -0
@@ -0,0 +1,3 @@ +arm_softmmu_ss.add(when: [hvf, 'CONFIG_HVF'], if_true: files( + 'hvf.c', +))
target/arm/meson.build +2 -0
@@ -60,5 +60,7 @@ arm_softmmu_ss.add(files( 'psci.c', )) +subdir('hvf') + target_arch += {'arm': arm_ss} target_softmmu_arch += {'arm': arm_softmmu_ss}
Signature · 1 line
2.30.1 (Apple Git-130)
PATCH v12 9/10 hvf: arm: Add rudimentary PMC support
In reply to Alexander GrafWe can expose cycle counters on the PMU easily. To be as compatible as possible, let's do so, but make sure we don't expose any other architectural counters that we can not model yet.
This allows OSs to work that require PMU support.
--- target/arm/hvf/hvf.c | 179 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 179 insertions(+)
target/arm/hvf/hvf.c +179 -0
@@ -42,6 +42,18 @@#define SYSREG_OSLSR_EL1 SYSREG(2, 0, 1, 1, 4)#define SYSREG_OSDLR_EL1 SYSREG(2, 0, 1, 3, 4)#define SYSREG_CNTPCT_EL0 SYSREG(3, 3, 14, 0, 1) +#define SYSREG_PMCR_EL0 SYSREG(3, 3, 9, 12, 0) +#define SYSREG_PMUSERENR_EL0 SYSREG(3, 3, 9, 14, 0) +#define SYSREG_PMCNTENSET_EL0 SYSREG(3, 3, 9, 12, 1) +#define SYSREG_PMCNTENCLR_EL0 SYSREG(3, 3, 9, 12, 2) +#define SYSREG_PMINTENCLR_EL1 SYSREG(3, 0, 9, 14, 2) +#define SYSREG_PMOVSCLR_EL0 SYSREG(3, 3, 9, 12, 3) +#define SYSREG_PMSWINC_EL0 SYSREG(3, 3, 9, 12, 4) +#define SYSREG_PMSELR_EL0 SYSREG(3, 3, 9, 12, 5) +#define SYSREG_PMCEID0_EL0 SYSREG(3, 3, 9, 12, 6) +#define SYSREG_PMCEID1_EL0 SYSREG(3, 3, 9, 12, 7) +#define SYSREG_PMCCNTR_EL0 SYSREG(3, 3, 9, 13, 0) +#define SYSREG_PMCCFILTR_EL0 SYSREG(3, 3, 14, 15, 7)#define WFX_IS_WFE (1 << 0)
@@ -728,6 +740,40 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint32_t rt) val =qemu_clock_get_ns (QEMU_CLOCK_VIRTUAL ) /gt_cntfrq_period_ns (arm_cpu);break ; +case SYSREG_PMCR_EL0 : + val = env->cp15.c9_pmcr; +break ; +case SYSREG_PMCCNTR_EL0 : +pmu_op_start (env); + val = env->cp15.c15_ccnt; +pmu_op_finish (env); +break ; +case SYSREG_PMCNTENCLR_EL0 : + val = env->cp15.c9_pmcnten; +break ; +case SYSREG_PMOVSCLR_EL0 : + val = env->cp15.c9_pmovsr; +break ; +case SYSREG_PMSELR_EL0 : + val = env->cp15.c9_pmselr; +break ; +case SYSREG_PMINTENCLR_EL1 : + val = env->cp15.c9_pminten; +break ; +case SYSREG_PMCCFILTR_EL0 : + val = env->cp15.pmccfiltr_el0; +break ; +case SYSREG_PMCNTENSET_EL0 : + val = env->cp15.c9_pmcnten; +break ; +case SYSREG_PMUSERENR_EL0 : + val = env->cp15.c9_pmuserenr; +break ; +case SYSREG_PMCEID0_EL0 : +case SYSREG_PMCEID1_EL0 : +/* We can't really count anything yet, declare all events invalid */ + val =0 ; +break ;case SYSREG_OSLSR_EL1 : val = env->cp15.oslsr_el1;break ;
@@ -758,6 +804,82 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint32_t rt)return 0 ; } +static void pmu_update_irq (CPUARMState *env) +{ +ARMCPU *cpu =env_archcpu (env); +qemu_set_irq (cpu->pmu_interrupt, (env->cp15.c9_pmcr &PMCRE ) && + (env->cp15.c9_pminten & env->cp15.c9_pmovsr)); +} + +static bool pmu_event_supported (uint16_t number) +{ +return false; +} + +/* Returns true if the counter (pass 31 for PMCCNTR) should count events using +* the current EL, security state, and register configuration. +*/ +static bool pmu_counter_enabled (CPUARMState *env,uint8_t counter) +{ +uint64_t filter; +bool enabled, filtered = true; +int el =arm_current_el (env); + + enabled = (env->cp15.c9_pmcr &PMCRE ) && + (env->cp15.c9_pmcnten & (1 << counter)); + +if (counter ==31 ) { + filter = env->cp15.pmccfiltr_el0; + }else { + filter = env->cp15.c14_pmevtyper[counter]; + } + +if (el ==0 ) { + filtered = filter &PMXEVTYPER_U ; + }else if (el ==1 ) { + filtered = filter &PMXEVTYPER_P ; + } + +if (counter !=31 ) { +/* +* If not checking PMCCNTR, ensure the counter is setup to an event we +* support +*/ +uint16_t event = filter &PMXEVTYPER_EVTCOUNT ; +if (!pmu_event_supported (event)) { +return false; + } + } + +return enabled && !filtered; +} + +static void pmswinc_write (CPUARMState *env,uint64_t value) +{ +unsigned int i; +for (i =0 ; i <pmu_num_counters (env); i++) { +/* Increment a counter's count iff: */ +if ((value & (1 << i)) &&/* counter's bit is set */ +/* counter is enabled and not filtered */ +pmu_counter_enabled (env, i) && +/* counter is SW_INCR */ + (env->cp15.c14_pmevtyper[i] &PMXEVTYPER_EVTCOUNT ) ==0x0 ) { +/* +* Detect if this write causes an overflow since we can't predict +* PMSWINC overflows like we can for other events +*/ +uint32_t new_pmswinc = env->cp15.c14_pmevcntr[i] +1 ; + +if (env->cp15.c14_pmevcntr[i] & ~new_pmswinc &INT32_MIN ) { + env->cp15.c9_pmovsr |= (1 << i); +pmu_update_irq (env); + } + + env->cp15.c14_pmevcntr[i] = new_pmswinc; + } + } +} +static int hvf_sysreg_write (CPUState *cpu,uint32_t reg,uint64_t val) {ARMCPU *arm_cpu =ARM_CPU (cpu);
@@ -772,6 +894,63 @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val) val);switch (reg) { +case SYSREG_PMCCNTR_EL0 : +pmu_op_start (env); + env->cp15.c15_ccnt = val; +pmu_op_finish (env); +break ; +case SYSREG_PMCR_EL0 : +pmu_op_start (env); + +if (val &PMCRC ) { +/* The counter has been reset */ + env->cp15.c15_ccnt =0 ; + } + +if (val &PMCRP ) { +unsigned int i; +for (i =0 ; i <pmu_num_counters (env); i++) { + env->cp15.c14_pmevcntr[i] =0 ; + } + } + + env->cp15.c9_pmcr &= ~PMCR_WRITEABLE_MASK ; + env->cp15.c9_pmcr |= (val &PMCR_WRITEABLE_MASK ); + +pmu_op_finish (env); +break ; +case SYSREG_PMUSERENR_EL0 : + env->cp15.c9_pmuserenr = val &0xf ; +break ; +case SYSREG_PMCNTENSET_EL0 : + env->cp15.c9_pmcnten |= (val &pmu_counter_mask (env)); +break ; +case SYSREG_PMCNTENCLR_EL0 : + env->cp15.c9_pmcnten &= ~(val &pmu_counter_mask (env)); +break ; +case SYSREG_PMINTENCLR_EL1 : +pmu_op_start (env); + env->cp15.c9_pminten |= val; +pmu_op_finish (env); +break ; +case SYSREG_PMOVSCLR_EL0 : +pmu_op_start (env); + env->cp15.c9_pmovsr &= ~val; +pmu_op_finish (env); +break ; +case SYSREG_PMSWINC_EL0 : +pmu_op_start (env); +pmswinc_write (env, val); +pmu_op_finish (env); +break ; +case SYSREG_PMSELR_EL0 : + env->cp15.c9_pmselr = val &0x1f ; +break ; +case SYSREG_PMCCFILTR_EL0 : +pmu_op_start (env); + env->cp15.pmccfiltr_el0 = val &PMCCFILTR_EL0 ; +pmu_op_finish (env); +break ;case SYSREG_OSLAR_EL1 : env->cp15.oslsr_el1 = val &1 ;break ;
Signature · 1 line
2.30.1 (Apple Git-130)
PATCH v12 10/10 arm: tcg: Adhere to SMCCC 1.3 section 5.2
In reply to Alexander GrafThe SMCCC 1.3 spec section 5.2 says
The Unknown SMC Function Identifier is a sign-extended value of (-1) that is returned in the R0, W0 or X0 registers. An implementation must return this error code when it receives:
* An SMC or HVC call with an unknown Function Identifier
* An SMC or HVC call for a removed Function Identifier
* An SMC64/HVC64 call from AArch32 state
To comply with these statements, let's always return -1 when we encounter an unknown HVC or SMC call.
---
v8 -> v9:
- Remove Windows specifics and just comply with SMCCC spec
v9 -> v10:
- Fix comment
--- target/arm/psci.c | 35 ++++++----------------------------- 1 file changed, 6 insertions(+), 29 deletions(-)
target/arm/psci.c +6 -29
@@ -27,15 +27,13 @@bool arm_is_psci_call (ARMCPU *cpu,int excp_type) { -/* Return true if the r0/x0 value indicates a PSCI call and -* the exception type matches the configured PSCI conduit. This is -* called before the SMC/HVC instruction is executed, to decide whether -* we should treat it as a PSCI call or with the architecturally +/* +* Return true if the exception type matches the configured PSCI conduit. +* This is called before the SMC/HVC instruction is executed, to decide +* whether we should treat it as a PSCI call or with the architecturally * defined behaviour for an SMC or HVC (which might be UNDEF or trap * to EL2 or to EL3). */ -CPUARMState *env = &cpu->env; -uint64_t param =is_a64 (env) ? env->xregs[0 ] : env->regs[0 ];switch (excp_type) {case EXCP_HVC :
@@ -52,27 +50,7 @@ bool arm_is_psci_call(ARMCPU *cpu, int excp_type)return false; } -switch (param) { -case QEMU_PSCI_0_2_FN_PSCI_VERSION : -case QEMU_PSCI_0_2_FN_MIGRATE_INFO_TYPE : -case QEMU_PSCI_0_2_FN_AFFINITY_INFO : -case QEMU_PSCI_0_2_FN64_AFFINITY_INFO : -case QEMU_PSCI_0_2_FN_SYSTEM_RESET : -case QEMU_PSCI_0_2_FN_SYSTEM_OFF : -case QEMU_PSCI_0_1_FN_CPU_ON : -case QEMU_PSCI_0_2_FN_CPU_ON : -case QEMU_PSCI_0_2_FN64_CPU_ON : -case QEMU_PSCI_0_1_FN_CPU_OFF : -case QEMU_PSCI_0_2_FN_CPU_OFF : -case QEMU_PSCI_0_1_FN_CPU_SUSPEND : -case QEMU_PSCI_0_2_FN_CPU_SUSPEND : -case QEMU_PSCI_0_2_FN64_CPU_SUSPEND : -case QEMU_PSCI_0_1_FN_MIGRATE : -case QEMU_PSCI_0_2_FN_MIGRATE : -return true; -default : -return false; - } +return true; }void arm_handle_psci_call (ARMCPU *cpu)
@@ -194,10 +172,9 @@ void arm_handle_psci_call(ARMCPU *cpu)break ;case QEMU_PSCI_0_1_FN_MIGRATE :case QEMU_PSCI_0_2_FN_MIGRATE : +default : ret =QEMU_PSCI_RET_NOT_SUPPORTED ;break ; -default : -g_assert_not_reached (); } err:
Signature · 1 line
2.30.1 (Apple Git-130)
Peter Maydell
AppliedRe: [PATCH v12 10/10] arm: tcg: Adhere to SMCCC 1.3 section 5.2
Quoted 11 lines · Alexander Graf
The SMCCC 1.3 spec section 5.2 says
The Unknown SMC Function Identifier is a sign-extended value of (-1) that is returned in the R0, W0 or X0 registers. An implementation must return this error code when it receives:
* An SMC or HVC call with an unknown Function Identifier
* An SMC or HVC call for a removed Function Identifier
* An SMC64/HVC64 call from AArch32 state
Quoted 5 lines
To comply with these statements, let's always return -1 when we encounter an unknown HVC or SMC call.
Signed-off-by Alexander Graf <[email protected]>Reviewed-by Peter Maydell <[email protected]>
Applied this final patch to target-arm.next now we've sorted out the problem with the orangepi, thanks.
-- PMM
Peter Maydell
AppliedQuoted 9 lines · Alexander Graf
Now that Apple Silicon is widely available, people are obviously excited to try and run virtualized workloads on them, such as Linux and Windows.
This patch set implements a fully functional version to get the ball going on that. With this applied, I can successfully run both Linux and Windows as guests. I am not aware of any limitations specific to Hypervisor.framework apart from:
Quoted 6 lines
- gdbstub debugging (breakpoints)
- missing GICv3 support
- Windows will not work due to UDEF SMC implementationTo use hvf support, please make sure to run -M virt,highmem=off to fit in M1's physical address space limits and use -cpu host.
Applied to target-arm.next, thanks (with the unnecessary #include in patch 6 removed).
-- PMM
Quoted 15 lines · Peter Maydell
Quoted 13 lines · Alexander Graf
Now that Apple Silicon is widely available, people are obviously excited to try and run virtualized workloads on them, such as Linux and Windows.
This patch set implements a fully functional version to get the ball going on that. With this applied, I can successfully run both Linux and Windows as guests. I am not aware of any limitations specific to Hypervisor.framework apart from:
- gdbstub debugging (breakpoints)
- missing GICv3 support
- Windows will not work due to UDEF SMC implementation
Quoted 5 lines
Quoted 2 lines
To use hvf support, please make sure to run -M virt,highmem=off to fit in M1's physical address space limits and use -cpu host.
Applied to target-arm.next, thanks (with the unnecessary #include in patch 6 removed).
Turns out that the final patch breaks "make check-acceptance". All the orangepi boot tests timeout:
(15/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'15-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi', 'logdir': '/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tests/... (90.24 s)
(16/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'16-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd', 'logdir': '/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang... (90.24 s)
(17/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'17-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd', 'logdir': '/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tes... (90.24 s)
So I'm going to drop that one; we need to work out what's going wrong with the orangepi-pc machine.
thanks
-- PMM
Quoted 34 lines · Peter Maydell
Quoted 14 lines · Peter Maydell
Quoted 13 lines · Alexander Graf
Now that Apple Silicon is widely available, people are obviously excited to try and run virtualized workloads on them, such as Linux and Windows.
This patch set implements a fully functional version to get the ball going on that. With this applied, I can successfully run both Linux and Windows as guests. I am not aware of any limitations specific to Hypervisor.framework apart from:
- gdbstub debugging (breakpoints)
- missing GICv3 support
- Windows will not work due to UDEF SMC implementationQuoted 5 lines
Quoted 2 lines
To use hvf support, please make sure to run -M virt,highmem=off to fit in M1's physical address space limits and use -cpu host.
Applied to target-arm.next, thanks (with the unnecessary #include in patch 6 removed).
Turns out that the final patch breaks "make check-acceptance". All the orangepi boot tests timeout:
(15/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'15-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi', 'logdir': '/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tests/... (90.24 s)
(16/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'16-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd',
Quoted 8 lines
'logdir': '
/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang... (90.24 s)
(17/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'17-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd', 'logdir': '/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tes... (90.24 s)
Works for me on x86_64 Fedora 34 built with
--enable-trace-backends=log --enable-debug:
$ ./tests/venv/bin/avocado run tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08 Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9 Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9 JOB ID : b19f151f7320def3a432255f3a99c0dde3da95c0 JOB LOG : /home/phil/avocado/job-results/job-2021-09-20T18.12-b19f151/job.log (1/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi: PASS (6.29 s) (2/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd: PASS (51.23 s) (3/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd: PASS (76.53 s) (4/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08: SKIP: storage limited (5/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9: SKIP: storage limited RESULTS : PASS 3 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 135.18 s
Quoted 74 lines · Philippe Mathieu-Daudé
Quoted 31 lines · Peter Maydell
Quoted 9 lines · Peter Maydell
Quoted 8 lines · Alexander Graf
Now that Apple Silicon is widely available, people are obviously excited to try and run virtualized workloads on them, such as Linux and Windows.
This patch set implements a fully functional version to get the ball going on that. With this applied, I can successfully run both Linux and Windows as guests. I am not aware of any limitations specific to Hypervisor.framework apart from:
Quoted 8 lines
Quoted 6 lines
- gdbstub debugging (breakpoints)
- missing GICv3 support
- Windows will not work due to UDEF SMC implementationTo use hvf support, please make sure to run -M virt,highmem=off to fit in M1's physical address space limits and use -cpu host.
Applied to target-arm.next, thanks (with the unnecessary #include in patch 6 removed).
Turns out that the final patch breaks "make check-acceptance". All the orangepi boot tests timeout:
(15/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'15-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi', 'logdir': '/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tests/... (90.24 s)
(16/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'16-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd',Quoted 8 lines
'logdir': '
/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang... (90.24 s)
(17/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'17-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd', 'logdir': '/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tes... (90.24 s)Works for me on
x86_64Fedora 34 built with
--enable-trace-backends=log --enable-debug:$ ./tests/venv/bin/avocado run tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08 Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9 Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9 JOB ID : b19f151f7320def3a432255f3a99c0dde3da95c0 JOB LOG : /home/phil/avocado/job-results/job-2021-09-20T18.12-b19f151/job.log (1/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi: PASS (6.29 s) (2/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd: PASS (51.23 s) (3/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd: PASS (76.53 s) (4/5)
Quoted 9 lines
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08: SKIP: storage limited (5/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9: SKIP: storage limited RESULTS : PASS 3 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 135.18 s
The OrangePi kernel goes into an endless loop here:
The reason is simple: It tries to install its own HYP code using the old "install HYP handler, then invoke it" trick based on SPSR's indication that HYP is available, but fails to do so because HYP calls get handled by QEMU instead because the PSCI conduit is configured to HVC.
The patch below seems to fix it for me. Please advise how you want to proceed.
Alex
hw/arm/allwinner-h3.c +0 -0
@@ -237,7 +237,7 @@ static void allwinner_h3_realize(DeviceState *dev,
Error **errp)
/* Provide Power State Coordination Interface */ qdev_prop_set_int32(DEVICE(&s->cpus[i]), "psci-conduit", - QEMU_PSCI_CONDUIT_HVC);
+ QEMU_PSCI_CONDUIT_SMC);
/* Disable secondary CPUs */ qdev_prop_set_bit(DEVICE(&s->cpus[i]), "start-powered-off",
Quoted 85 lines · Alexander Graf
Quoted 74 lines · Philippe Mathieu-Daudé
Quoted 31 lines · Peter Maydell
Quoted 9 lines · Peter Maydell
Quoted 8 lines · Alexander Graf
Now that Apple Silicon is widely available, people are obviously excited to try and run virtualized workloads on them, such as Linux and Windows.
This patch set implements a fully functional version to get the ball going on that. With this applied, I can successfully run both Linux and Windows as guests. I am not aware of any limitations specific to Hypervisor.framework apart from:
Quoted 8 lines
Quoted 6 lines
- gdbstub debugging (breakpoints)
- missing GICv3 support
- Windows will not work due to UDEF SMC implementationTo use hvf support, please make sure to run -M virt,highmem=off to fit in M1's physical address space limits and use -cpu host.
Applied to target-arm.next, thanks (with the unnecessary #include in patch 6 removed).
Turns out that the final patch breaks "make check-acceptance". All the orangepi boot tests timeout:
(15/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'15-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi', 'logdir': '/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tests/... (90.24 s)
(16/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'16-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd',Quoted 8 lines
'logdir': '
/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang... (90.24 s)
(17/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'17-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd', 'logdir': '/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tes... (90.24 s)Works for me on
x86_64Fedora 34 built with
--enable-trace-backends=log --enable-debug:$ ./tests/venv/bin/avocado run tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08 Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9 Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9 JOB ID : b19f151f7320def3a432255f3a99c0dde3da95c0 JOB LOG : /home/phil/avocado/job-results/job-2021-09-20T18.12-b19f151/job.log (1/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi: PASS (6.29 s) (2/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd: PASS (51.23 s) (3/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd: PASS (76.53 s) (4/5)Quoted 9 lines
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08: SKIP: storage limited (5/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9: SKIP: storage limited RESULTS : PASS 3 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 135.18 s
Quoted 4 lines
The OrangePi kernel goes into an endless loop here:
As you said, this is a Linux specific behavior, ...
Quoted 17 lines
The reason is simple: It tries to install its own HYP code using the old "install HYP handler, then invoke it" trick based on SPSR's indication that HYP is available, but fails to do so because HYP calls get handled by QEMU instead because the PSCI conduit is configured to HVC.
The patch below seems to fix it for me. Please advise how you want to proceed.
Alex
hw/arm/allwinner-h3.c +0
-0
hw/arm/allwinner-h3.c static void allwinner_h3_realize(DeviceState *dev, · line 241
Quoted 6 lines
hw/arm/allwinner-h3.c +1
-1@@ -237,7 +237,7 @@ static void allwinner_h3_realize(DeviceState *dev, Error **errp) /* Provide Power State Coordination Interface */ qdev_prop_set_int32(DEVICE(&s->cpus[i]), "psci-conduit", - QEMU_PSCI_CONDUIT_HVC); + QEMU_PSCI_CONDUIT_SMC);
... so I'd rather follow the approach taken on the Versal board, see versal_virt_init() in commit 6f16da53ffe ("hw/arm: versal:
Add a virtual Xilinx Versal board"):
1/ add "psci-conduit" property in TYPE_AW_H3,
forward this property to each core in allwinner_h3_realize()
2/ set property in orangepi_init():
object_property_set_int(OBJECT(h3), "psci-conduit",
machine->kernel_filename
? QEMU_PSCI_CONDUIT_SMC
: QEMU_PSCI_CONDUIT_HVC,
&error_abort);That way we don't break non-Linux firmwares.
Quoted 116 lines · Philippe Mathieu-Daudé
Quoted 115 lines · Alexander Graf
Quoted 113 lines · Philippe Mathieu-Daudé
Quoted 51 lines · Peter Maydell
Quoted 17 lines · Peter Maydell
Quoted 15 lines · Alexander Graf
Now that Apple Silicon is widely available, people are obviously excited
to try and run virtualized workloads on them, such as Linux and Windows.This patch set implements a fully functional version to get the ball going on that. With this applied, I can successfully run both Linux and
Windows as guests. I am not aware of any limitations specific to Hypervisor.framework apart from:- gdbstub debugging (breakpoints)
- missing GICv3 support
- Windows will not work due to UDEF SMC implementationQuoted 5 lines
Quoted 3 lines
To use hvf support, please make sure to run -M virt,highmem=off to fit
in M1's physical address space limits and use -cpu host.Applied to target-arm.next, thanks (with the unnecessary #include in patch 6 removed).
Turns out that the final patch breaks "make check-acceptance". All the orangepi boot tests timeout:
(15/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'15-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi','logdir':
'/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tests/... (90.24 s)
(16/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'16-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd','logdir':
'/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang...
(90.24 s)
(17/58) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'17-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd',Quoted 3 lines
'logdir':
'/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tes... (90.24 s)Works for me on
x86_64Fedora 34 built with
--enable-trace-backends=log --enable-debug:$ ./tests/venv/bin/avocado run tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepiFetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9
JOB ID :
b19f151f7320def3a432255f3a99c0dde3da95c0
JOB LOG :/home/phil/avocado/job-results/job-2021-09-20T18.12-b19f151/job.log(1/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi:PASS (6.29 s)
(2/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd:PASS (51.23 s)
(3/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd:PASS (76.53 s)
(4/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08:SKIP: storage limited
(5/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9:SKIP: storage limited
RESULTS : PASS 3 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 |
CANCEL 0
JOB TIME : 135.18 s
Quoted 6 lines
Quoted 4 lines
The OrangePi kernel goes into an endless loop here:
As you said, this is a Linux specific behavior, ...
It's not really Linux specific behavior. Linux just includes preloader code that moves it into HYP mode when booted in SVC mode.
Quoted 34 lines
Quoted 18 lines
The reason is simple: It tries to install its own HYP code using the old "install HYP handler, then invoke it" trick based on SPSR's indication that HYP is available, but fails to do so because HYP calls get handled by QEMU instead because the PSCI conduit is configured to HVC.
The patch below seems to fix it for me. Please advise how you want to proceed.
Alex
hw/arm/allwinner-h3.c +0
-0hw/arm/allwinner-h3.c static void allwinner_h3_realize(DeviceState *dev, · line 241
Quoted 6 lines
hw/arm/allwinner-h3.c +1
-1@@ -237,7 +237,7 @@ static void allwinner_h3_realize(DeviceState *dev, Error **errp) /* Provide Power State Coordination Interface */ qdev_prop_set_int32(DEVICE(&s->cpus[i]), "psci-conduit", - QEMU_PSCI_CONDUIT_HVC); + QEMU_PSCI_CONDUIT_SMC);... so I'd rather follow the approach taken on the Versal board, see
versal_virt_init()in commit6f16da53ffe("hw/arm: versal:
Add a virtual Xilinx Versal board"):1/ add "psci-conduit" property in
TYPE_AW_H3,
forward this property to each core inallwinner_h3_realize()2/ set property in
orangepi_init():
Quoted 7 lines
object_property_set_int(OBJECT(h3), "psci-conduit",
machine->kernel_filename?QEMU_PSCI_CONDUIT_SMC:QEMU_PSCI_CONDUIT_HVC,&error_abort);That way we don't break non-Linux firmwares.
This doesn't make sense to me. We need to decide what we want from the OrangePi emulation:
1) Guest owns USR, SVC. HYP's PSCI is emulated by QEMU
2) Guest owns USR, SVC, HYP. MON's PSCI is emulated by QEMU
3) Guest owns USR, SVC, HYP, MON. PSCI is handled 100% inside the guest.
We currently advertise to the guest that it can use HYP, but don't allow it to run anything in MON (option 2), but then at the same time override parts of HYP with QEMU code. That's just plain wrong - we're breaking hypervisors running on OrangePi :).
So if the goal of that board is to run OrangePi ATF (does it do ATF?) code or any other "full firmware", then we would need to move QEMU completely out of the conduit picture, implement option 3 and ship pre-HYP firmware with QEMU.
If the goal is to just make HYP and below work, the fix above is the best path forward IMHO.
Option 1 would require us to tell the guest that it can not make use of HYP. That's possible, but why should we?
I don't really see why we should even consider option 1. 2 and 3 make sense. 3 is a lot of work. 2 is a one line fix that makes the board at least do what it's supposed to do :).
Alex
Quoted 192 lines · Alexander Graf
Quoted 116 lines · Philippe Mathieu-Daudé
Quoted 115 lines · Alexander Graf
Quoted 113 lines · Philippe Mathieu-Daudé
Quoted 51 lines · Peter Maydell
Quoted 17 lines · Peter Maydell
Quoted 15 lines · Alexander Graf
Now that Apple Silicon is widely available, people are obviously excited
to try and run virtualized workloads on them, such as Linux and Windows.This patch set implements a fully functional version to get the ball going on that. With this applied, I can successfully run both Linux and
Windows as guests. I am not aware of any limitations specific to Hypervisor.framework apart from:- gdbstub debugging (breakpoints)
- missing GICv3 support
- Windows will not work due to UDEF SMC implementationQuoted 5 lines
Quoted 3 lines
To use hvf support, please make sure to run -M virt,highmem=off to fit
in M1's physical address space limits and use -cpu host.Applied to target-arm.next, thanks (with the unnecessary #include in patch 6 removed).
Turns out that the final patch breaks "make check-acceptance". All the orangepi boot tests timeout:
(15/58)
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'15-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi','logdir':
'/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tests/... (90.24 s)
(16/58)
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'16-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd','logdir':
'/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang...
(90.24 s)
(17/58)
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd:
INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred:
Timeout reached\nOriginal status: ERROR\n{'name':
'17-tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd',Quoted 3 lines
'logdir':
'/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tes... (90.24 s)Works for me on
x86_64Fedora 34 built with
--enable-trace-backends=log --enable-debug:$ ./tests/venv/bin/avocado run tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepiFetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9
Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9
JOB ID :
b19f151f7320def3a432255f3a99c0dde3da95c0
JOB LOG :/home/phil/avocado/job-results/job-2021-09-20T18.12-b19f151/job.log
(1/5)
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi:PASS (6.29 s)
(2/5)
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd:PASS (51.23 s)
(3/5)
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd:PASS (76.53 s)
(4/5)
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08:SKIP: storage limited
(5/5)
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9:SKIP: storage limited
RESULTS : PASS 3 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 |
CANCEL 0
JOB TIME : 135.18 sQuoted 6 lines
Quoted 4 lines
The OrangePi kernel goes into an endless loop here:
As you said, this is a Linux specific behavior, ...
It's not really Linux specific behavior. Linux just includes preloader code that moves it into HYP mode when booted in SVC mode.
Quoted 34 lines
Quoted 18 lines
The reason is simple: It tries to install its own HYP code using the old "install HYP handler, then invoke it" trick based on SPSR's indication that HYP is available, but fails to do so because HYP calls get handled by QEMU instead because the PSCI conduit is configured to HVC.
The patch below seems to fix it for me. Please advise how you want to proceed.
Alex
hw/arm/allwinner-h3.c +0
-0hw/arm/allwinner-h3.c static void allwinner_h3_realize(DeviceState *dev, · line 241
Quoted 6 lines
hw/arm/allwinner-h3.c +1
-1@@ -237,7 +237,7 @@ static void allwinner_h3_realize(DeviceState *dev, Error **errp) /* Provide Power State Coordination Interface */ qdev_prop_set_int32(DEVICE(&s->cpus[i]), "psci-conduit", - QEMU_PSCI_CONDUIT_HVC); + QEMU_PSCI_CONDUIT_SMC);... so I'd rather follow the approach taken on the Versal board, see
versal_virt_init()in commit6f16da53ffe("hw/arm: versal:
Add a virtual Xilinx Versal board"):1/ add "psci-conduit" property in
TYPE_AW_H3,
forward this property to each core inallwinner_h3_realize()2/ set property in
orangepi_init():Quoted 7 lines
object_property_set_int(OBJECT(h3), "psci-conduit",
machine->kernel_filename?QEMU_PSCI_CONDUIT_SMC:QEMU_PSCI_CONDUIT_HVC,&error_abort);That way we don't break non-Linux firmwares.
This doesn't make sense to me. We need to decide what we want from the OrangePi emulation:
1) Guest owns USR, SVC. HYP's PSCI is emulated by QEMU
2) Guest owns USR, SVC, HYP. MON's PSCI is emulated by QEMU
3) Guest owns USR, SVC, HYP, MON. PSCI is handled 100% inside the guest.We currently advertise to the guest that it can use HYP, but don't allow it to run anything in MON (option 2), but then at the same time override parts of HYP with QEMU code. That's just plain wrong - we're breaking hypervisors running on OrangePi :).
So if the goal of that board is to run OrangePi ATF (does it do ATF?) code or any other "full firmware", then we would need to move QEMU completely out of the conduit picture, implement option 3 and ship pre-HYP firmware with QEMU.
If the goal is to just make HYP and below work, the fix above is the best path forward IMHO.
Quoted 6 lines
Option 1 would require us to tell the guest that it can not make use of HYP. That's possible, but why should we?
I don't really see why we should even consider option 1. 2 and 3 make sense. 3 is a lot of work. 2 is a one line fix that makes the board at least do what it's supposed to do :).
So this part of Xilinx Versal:
* When loading an OS, we turn on QEMU's PSCI implementation with SMC
* as the PSCI conduit. When there's no -kernel, we assume the user
* provides EL3 firmware to handle PSCI.
*/
if (machine->kernel_filename) {
psci_conduit = QEMU_PSCI_CONDUIT_SMC;}
abuses the fact that -kernel command line expect a *Linux* kernel able to read the provided dtb which describe SMC. This won't work if we manually provide a crafted dtb with another conduit, or if we pass any other binary (not Linux, not particularly ELF, since -kernel can load many formats).
I guess I understand better now.
Thanks for the detailed explanation,
Phil.
Quoted 10 lines · Philippe Mathieu-Daudé
So this part of Xilinx Versal:
* When loading an OS, we turn on QEMU's PSCI implementation with SMC * as the PSCI conduit. When there's no -kernel, we assume the user * provides EL3 firmware to handle PSCI. */ if (machine->kernel_filename) { psci_conduit = QEMU_PSCI_CONDUIT_SMC; }
Quoted 3 lines
abuses the fact that -kernel command line expect a *Linux* kernel able to read the provided dtb which describe SMC. >This won't work if we manually provide a crafted dtb with another conduit
...which is not a supported thing to do (it won't work with the 'virt' board either)...
Quoted 3 lines
, or if we pass any
other binary (not Linux, not particularly ELF, since -kernel can load many formats).
-kernel means "assume I am a Linux kernel and boot me accordingly". Sometimes it works for other things, but there are no guarantees. Among other things it always means "start me at EL2 or EL1, not EL3", so PSCI handling via SMC doesn't get in anybody's way even if they're not using it. (People who want a pure "load this ELF file" should use the generic-loader.)
The basic distinction the versal-virt board is making here is "we have EL3 firmware to run in the guest" vs "we don't"; for the same reason that the virt board does, in the former case we disable QEMU's internal PSCI implementation, and in the latter we enable it. In theory the orangepi board code should do the same, because if we're really running guest code at EL3 it probably is going to assume it has SMC, and QEMU taking control instead is going to confuse it.
-- PMM