In-kernel PSCI v0.2 emulation for KVM ARM/ARM64
Subsystems (6)
v11 cover In-kernel PSCI v0.2 emulation for KVM ARM/ARM64
Currently, KVM ARM/ARM64 only provides in-kernel emulation of Power State and Coordination Interface (PSCI) v0.1.
This patchset aims at providing newer PSCI v0.2 for KVM ARM/ARM64 VCPUs such that it does not break current KVM ARM/ARM64 ABI.
The user space tools (i.e. QEMU or KVMTOOL) will have to explicitly enable KVM_ARM_VCPU_PSCI_0_2 feature using KVM_ARM_VCPU_INIT ioctl for providing PSCI v0.2 to VCPUs.
Changlog:
V11:
- Added more comments to uapi/linux/psci.h
- Added comment about why we store INTERNAL_FAILURE in r0 (or x0) for SYSTEM_OFF and SYSTEM_RESET emulation
V10:
- Updated PSCI_VERSION_xxx defines in uapi/linux/psci.h
- Added PSCI_0_2_AFFINITY_LEVEL_xxxx defines in uapi/linux/psci.h
- Removed PSCI v0.1 related defines from uapi/linux/psci.h
- Inject undefined exception for all types of errors in PSCI emulation (i.e kvm_psci_call(vcpu) < 0)
- Removed "inline" attribute of kvm_prepare_system_event()
- Store INTERNAL_FAILURE in r0 (or x0) before exiting to userspace
- Use MPIDR_LEVEL_BITS in AFFINITY_MASK define
- Updated comment in kvm_psci_vcpu_suspend() as-per Marc's suggestion
V9:
- Rename undefined PSCI_VER_xxx defines to PSCI_VERSION_xxx defines
V8:
- Add #define for possible values of migrate type in uapi/linux/psci.h
- Simplified psci_affinity_mask() in psci.c
- Update comments in kvm_psci_vcpu_suspend() to indicate that for KVM wakeup events are interrupts.
- Unconditionally update r0 (or x0) in kvm_psci_vcpu_on()
V7:
- Make uapi/linux/psci.h inline with Ashwin's patch
http://www.spinics.net/lists/arm-kernel/msg319090.html
- Incorporate Rob's suggestions for uapi/linux/psci.h
- Treat CPU_SUSPEND power-down request to be same as standby request. This further simplifies CPU_SUSPEND emulation.
V6:
- Introduce uapi/linux/psci.h for sharing PSCI defines between ARM kernel, ARM64 kernel, KVM ARM/ARM64 and user space
- Make CPU_SUSPEND emulation similar to WFI emulation
V5:
- Have separate last patch to advertise KVM_CAP_ARM_PSCI_0_2
- Use kvm_psci_version() in kvm_psci_vcpu_on()
- Return ALREADY_ON for PSCI v0.2 CPU_ON if VCPU is not paused
- Remove per-VCPU suspend context
- As-per PSCI v0.2 spec, only current CPU can suspend itself
V4:
- Implement all mandatory functions required by PSCI v0.2
V3:
- Make KVM_ARM_VCPU_PSCI_0_2 feature experiementatl for now so that it fails for user space till all mandatory PSCI v0.2 functions are emulated by KVM ARM/ARM64
- Have separate patch for making KVM_ARM_VCPU_PSCI_0_2 feature available to user space. This patch can be defferred for now
V2:
- Don't rename PSCI return values KVM_PSCI_RET_NI and KVM_PSCI_RET_INVAL
- Added kvm_psci_version() to get PSCI version available to VCPU
- Fixed grammer in Documentation/virtual/kvm/api.txt
V1:
- Initial RFC PATCH
Anup Patel (12):
KVM: Add capability to advertise PSCI v0.2 support ARM/ARM64: KVM: Add common header for PSCI related defines ARM/ARM64: KVM: Add base for PSCI v0.2 emulation
KVM: Documentation: Add info regarding KVM_ARM_VCPU_PSCI_0_2 feature ARM/ARM64: KVM: Make kvm_psci_call() return convention more flexible
KVM: Add KVM_EXIT_SYSTEM_EVENT to user space API header ARM/ARM64: KVM: Emulate PSCI v0.2 SYSTEM_OFF and SYSTEM_RESET ARM/ARM64: KVM: Emulate PSCI v0.2 AFFINITY_INFO ARM/ARM64: KVM: Emulate PSCI v0.2 MIGRATE_INFO_TYPE and related
functions
ARM/ARM64: KVM: Fix CPU_ON emulation for PSCI v0.2 ARM/ARM64: KVM: Emulate PSCI v0.2 CPU_SUSPEND
ARM/ARM64: KVM: Advertise KVM_CAP_ARM_PSCI_0_2 to user space
Documentation/virtual/kvm/api.txt | 17 +++ arch/arm/include/asm/kvm_host.h | 2 +- arch/arm/include/asm/kvm_psci.h | 6 +- arch/arm/include/uapi/asm/kvm.h | 10 +- arch/arm/kvm/arm.c | 1 + arch/arm/kvm/handle_exit.c | 10 +- arch/arm/kvm/psci.c | 235 ++++++++++++++++++++++++++++++++++--- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/include/asm/kvm_psci.h | 6 +- arch/arm64/include/uapi/asm/kvm.h | 10 +- arch/arm64/kvm/handle_exit.c | 10 +- include/uapi/linux/Kbuild | 1 + include/uapi/linux/kvm.h | 9 ++ include/uapi/linux/psci.h | 90 ++++++++++++++ 14 files changed, 372 insertions(+), 37 deletions(-) create mode 100644 include/uapi/linux/psci.h
Signature · 1 line
1.7.9.5
PATCH v11 1/12 KVM: Add capability to advertise PSCI v0.2 support
User space (i.e. QEMU or KVMTOOL) should be able to check whether KVM ARM/ARM64 supports in-kernel PSCI v0.2 emulation. For this purpose, we define KVM_CAP_ARM_PSCI_0_2 in KVM user space interface header.
--- include/uapi/linux/kvm.h | 1 + 1 file changed, 1 insertion(+)
include/uapi/linux/kvm.h +1 -0
@@ -743,6 +743,7 @@ struct kvm_ppc_smmu_info {#define KVM_CAP_IOAPIC_POLARITY_IGNORED 97#define KVM_CAP_ENABLE_CAP_VM 98#define KVM_CAP_S390_IRQCHIP 99 +#define KVM_CAP_ARM_PSCI_0_2 100#ifdef KVM_CAP_IRQ_ROUTING
Signature · 1 line
1.7.9.5
PATCH v11 2/12 ARM/ARM64: KVM: Add common header for PSCI related defines
In reply to Anup PatelWe need a common place to share PSCI related defines among ARM kernel, ARM64 kernel, KVM ARM/ARM64 PSCI emulation, and user space.
We introduce uapi/linux/psci.h for this purpose. This newly added header will be first used by KVM ARM/ARM64 in-kernel PSCI emulation and user space (i.e. QEMU or KVMTOOL).
--- include/uapi/linux/Kbuild | 1 + include/uapi/linux/psci.h | 90 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+) create mode 100644 include/uapi/linux/psci.h
include/uapi/linux/Kbuild +1 -0
@@ -317,6 +317,7 @@ header-y += ppp-ioctl.h header-y += ppp_defs.h header-y += pps.h header-y += prctl.h +header-y += psci.h header-y += ptp_clock.h header-y += ptrace.h header-y += qnx4_fs.h
include/uapi/linux/psci.h new file +90 -0
90 lines not on this page · raw message
Signature · 1 line
1.7.9.5
Re: [PATCH v11 02/12] ARM/ARM64: KVM: Add common header for PSCI related defines
Quoted 11 lines · Anup Patel
We need a common place to share PSCI related defines among ARM kernel, ARM64 kernel, KVM ARM/ARM64 PSCI emulation, and user space.
We introduce
uapi/linux/psci.hfor this purpose. This newly added header will be first used by KVM ARM/ARM64 in-kernel PSCI emulation and user space (i.e. QEMU or KVMTOOL).Signed-off-by Anup Patel <[email protected]>Signed-off-by Pranavkumar Sawargaonkar <[email protected]>Signed-off-by Ashwin Chaugule <[email protected]>---
PATCH v11 3/12 ARM/ARM64: KVM: Add base for PSCI v0.2 emulation
In reply to Anup PatelCurrently, the in-kernel PSCI emulation provides PSCI v0.1 interface to VCPUs. This patch extends current in-kernel PSCI emulation to provide PSCI v0.2 interface to VCPUs.
By default, ARM/ARM64 KVM will always provide PSCI v0.1 interface for keeping the ABI backward-compatible.
To select PSCI v0.2 interface for VCPUs, the user space (i.e. QEMU or KVMTOOL) will have to set KVM_ARM_VCPU_PSCI_0_2 feature when doing VCPU init using KVM_ARM_VCPU_INIT ioctl.
--- arch/arm/include/asm/kvm_host.h | 2 +- arch/arm/include/asm/kvm_psci.h | 4 ++ arch/arm/include/uapi/asm/kvm.h | 10 ++-- arch/arm/kvm/psci.c | 93 ++++++++++++++++++++++++++++++------- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/include/asm/kvm_psci.h | 4 ++ arch/arm64/include/uapi/asm/kvm.h | 10 ++-- 7 files changed, 99 insertions(+), 26 deletions(-)
arch/arm/include/asm/kvm_host.h +1 -1
@@ -36,7 +36,7 @@#define KVM_COALESCED_MMIO_PAGE_OFFSET 1#define KVM_HAVE_ONE_REG -#define KVM_VCPU_MAX_FEATURES 1 +#define KVM_VCPU_MAX_FEATURES 2#include <kvm/arm_vgic.h>
arch/arm/include/asm/kvm_psci.h +4 -0
@@ -18,6 +18,10 @@#ifndef __ARM_KVM_PSCI_H__#define __ARM_KVM_PSCI_H__ +#define KVM_ARM_PSCI_0_1 1 +#define KVM_ARM_PSCI_0_2 2 + +int kvm_psci_version (struct kvm_vcpu *vcpu);bool kvm_psci_call (struct kvm_vcpu *vcpu);#endif /* __ARM_KVM_PSCI_H__ */
arch/arm/include/uapi/asm/kvm.h +6 -4
@@ -20,6 +20,7 @@#define __ARM_KVM_H__#include <linux/types.h> +#include <linux/psci.h> #include <asm/ptrace.h> #define __KVM_HAVE_GUEST_DEBUG
@@ -83,6 +84,7 @@ struct kvm_regs {#define KVM_VGIC_V2_CPU_SIZE 0x2000#define KVM_ARM_VCPU_POWER_OFF 0/* CPU is started in OFF state */ +#define KVM_ARM_VCPU_PSCI_0_2 1/* CPU uses PSCI v0.2 */ struct kvm_vcpu_init {__u32 target;
@@ -201,9 +203,9 @@ struct kvm_arch_memory_slot {#define KVM_PSCI_FN_CPU_ON KVM_PSCI_FN(2)#define KVM_PSCI_FN_MIGRATE KVM_PSCI_FN(3) -#define KVM_PSCI_RET_SUCCESS 0 -#define KVM_PSCI_RET_NI ((unsigned long)-1) -#define KVM_PSCI_RET_INVAL ((unsigned long)-2) -#define KVM_PSCI_RET_DENIED ((unsigned long)-3) +#define KVM_PSCI_RET_SUCCESS PSCI_RET_SUCCESS +#define KVM_PSCI_RET_NI PSCI_RET_NOT_SUPPORTED +#define KVM_PSCI_RET_INVAL PSCI_RET_INVALID_PARAMS +#define KVM_PSCI_RET_DENIED PSCI_RET_DENIED#endif /* __ARM_KVM_H__ */
arch/arm/kvm/psci.c +77 -16
123 lines not on this page · raw message
arch/arm64/include/asm/kvm_host.h +1 -1
@@ -39,7 +39,7 @@#include <kvm/arm_vgic.h> #include <kvm/arm_arch_timer.h> -#define KVM_VCPU_MAX_FEATURES 2 +#define KVM_VCPU_MAX_FEATURES 3struct kvm_vcpu ;int kvm_target_cpu (void );
arch/arm64/include/asm/kvm_psci.h +4 -0
@@ -18,6 +18,10 @@#ifndef __ARM64_KVM_PSCI_H__#define __ARM64_KVM_PSCI_H__ +#define KVM_ARM_PSCI_0_1 1 +#define KVM_ARM_PSCI_0_2 2 + +int kvm_psci_version (struct kvm_vcpu *vcpu);bool kvm_psci_call (struct kvm_vcpu *vcpu);#endif /* __ARM64_KVM_PSCI_H__ */
arch/arm64/include/uapi/asm/kvm.h +6 -4
@@ -31,6 +31,7 @@#define KVM_NR_SPSR 5#ifndef __ASSEMBLY__ +#include <linux/psci.h> #include <asm/types.h> #include <asm/ptrace.h>
@@ -77,6 +78,7 @@ struct kvm_regs {#define KVM_ARM_VCPU_POWER_OFF 0/* CPU is started in OFF state */ #define KVM_ARM_VCPU_EL1_32BIT 1/* CPU running a 32bit VM */ +#define KVM_ARM_VCPU_PSCI_0_2 2/* CPU uses PSCI v0.2 */ struct kvm_vcpu_init {__u32 target;
@@ -186,10 +188,10 @@ struct kvm_arch_memory_slot {#define KVM_PSCI_FN_CPU_ON KVM_PSCI_FN(2)#define KVM_PSCI_FN_MIGRATE KVM_PSCI_FN(3) -#define KVM_PSCI_RET_SUCCESS 0 -#define KVM_PSCI_RET_NI ((unsigned long)-1) -#define KVM_PSCI_RET_INVAL ((unsigned long)-2) -#define KVM_PSCI_RET_DENIED ((unsigned long)-3) +#define KVM_PSCI_RET_SUCCESS PSCI_RET_SUCCESS +#define KVM_PSCI_RET_NI PSCI_RET_NOT_SUPPORTED +#define KVM_PSCI_RET_INVAL PSCI_RET_INVALID_PARAMS +#define KVM_PSCI_RET_DENIED PSCI_RET_DENIED#endif
Signature · 1 line
1.7.9.5
PATCH v11 4/12 KVM: Documentation: Add info regarding KVM_ARM_VCPU_PSCI_0_2 feature
In reply to Anup PatelWe have in-kernel emulation of PSCI v0.2 in KVM ARM/ARM64. To provide PSCI v0.2 interface to VCPUs, we have to enable KVM_ARM_VCPU_PSCI_0_2 feature when doing KVM_ARM_VCPU_INIT ioctl.
The patch updates documentation of KVM_ARM_VCPU_INIT ioctl to provide info regarding KVM_ARM_VCPU_PSCI_0_2 feature.
--- Documentation/virtual/kvm/api.txt | 2 ++ 1 file changed, 2 insertions(+)
Documentation/virtual/kvm/api.txt +2 -0
@@ -2376,6 +2376,8 @@ Possible features: Depends on KVM_CAP_ARM_PSCI. - KVM_ARM_VCPU_EL1_32BIT: Starts the CPU in a 32bit mode. Depends on KVM_CAP_ARM_EL1_32BIT (arm64 only). + - KVM_ARM_VCPU_PSCI_0_2: Emulate PSCI v0.2 for the CPU. + Depends on KVM_CAP_ARM_PSCI_0_2. 4.83 KVM_ARM_PREFERRED_TARGET
Signature · 1 line
1.7.9.5
PATCH v11 5/12 ARM/ARM64: KVM: Make kvm_psci_call() return convention more flexible
In reply to Anup PatelCurrently, the kvm_psci_call() returns 'true' or 'false' based on whether the PSCI function call was handled successfully or not. This does not help us emulate system-level PSCI functions where the actual emulation work will be done by user space (QEMU or KVMTOOL). Examples of such system-level PSCI functions are: PSCI v0.2 SYSTEM_OFF and SYSTEM_RESET.
This patch updates kvm_psci_call() to return three types of values:
1) > 0 (success)
2) = 0 (success but exit to user space)
3) < 0 (errors)
--- arch/arm/include/asm/kvm_psci.h | 2 +- arch/arm/kvm/handle_exit.c | 10 +++++++--- arch/arm/kvm/psci.c | 28 ++++++++++++++++------------ arch/arm64/include/asm/kvm_psci.h | 2 +- arch/arm64/kvm/handle_exit.c | 10 +++++++--- 5 files changed, 32 insertions(+), 20 deletions(-)
arch/arm/include/asm/kvm_psci.h +1 -1
@@ -22,6 +22,6 @@#define KVM_ARM_PSCI_0_2 2int kvm_psci_version (struct kvm_vcpu *vcpu); -bool kvm_psci_call (struct kvm_vcpu *vcpu); +int kvm_psci_call (struct kvm_vcpu *vcpu);#endif /* __ARM_KVM_PSCI_H__ */
arch/arm/kvm/handle_exit.c +7 -3
@@ -38,14 +38,18 @@ static int handle_svc_hyp(struct kvm_vcpu *vcpu, struct kvm_run *run)static int handle_hvc (struct kvm_vcpu *vcpu,struct kvm_run *run) { +int ret; +trace_kvm_hvc (*vcpu_pc (vcpu), *vcpu_reg (vcpu,0 ),kvm_vcpu_hvc_get_imm (vcpu)); -if (kvm_psci_call (vcpu)) + ret =kvm_psci_call (vcpu); +if (ret <0 ) { +kvm_inject_undefined (vcpu);return 1 ; + } -kvm_inject_undefined (vcpu); -return 1 ; +return ret; }static int handle_smc (struct kvm_vcpu *vcpu,struct kvm_run *run)
arch/arm/kvm/psci.c +16 -12
@@ -93,7 +93,7 @@ int kvm_psci_version(struct kvm_vcpu *vcpu)return KVM_ARM_PSCI_0_1 ; } -static bool kvm_psci_0_2_call (struct kvm_vcpu *vcpu) +static int kvm_psci_0_2_call (struct kvm_vcpu *vcpu) {unsigned long psci_fn = *vcpu_reg (vcpu,0 ) & ~((u32 )0 );unsigned long val;
@@ -128,14 +128,14 @@ static bool kvm_psci_0_2_call(struct kvm_vcpu *vcpu) val =PSCI_RET_NOT_SUPPORTED ;break ;default : -return false; +return -EINVAL ; } *vcpu_reg (vcpu,0 ) = val; -return true; +return 1 ; } -static bool kvm_psci_0_1_call (struct kvm_vcpu *vcpu) +static int kvm_psci_0_1_call (struct kvm_vcpu *vcpu) {unsigned long psci_fn = *vcpu_reg (vcpu,0 ) & ~((u32 )0 );unsigned long val;
@@ -153,11 +153,11 @@ static bool kvm_psci_0_1_call(struct kvm_vcpu *vcpu) val =PSCI_RET_NOT_SUPPORTED ;break ;default : -return false; +return -EINVAL ; } *vcpu_reg (vcpu,0 ) = val; -return true; +return 1 ; }/**
@@ -165,12 +165,16 @@ static bool kvm_psci_0_1_call(struct kvm_vcpu *vcpu)* @vcpu: Pointer to the VCPU struct * * Handle PSCI calls from guests through traps from HVC instructions. -* The calling convention is similar to SMC calls to the secure world where -* the function number is placed in r0 and this function returns true if the -* function number specified in r0 is withing the PSCI range, and false -* otherwise. +* The calling convention is similar to SMC calls to the secure world +* where the function number is placed in r0. +* +* This function returns: > 0 (success), 0 (success but exit to user +* space), and < 0 (errors) +* +* Errors: +* -EINVAL: Unrecognized PSCI function */ -bool kvm_psci_call (struct kvm_vcpu *vcpu) +int kvm_psci_call (struct kvm_vcpu *vcpu) {switch (kvm_psci_version (vcpu)) {case KVM_ARM_PSCI_0_2 :
@@ -178,6 +182,6 @@ bool kvm_psci_call(struct kvm_vcpu *vcpu)case KVM_ARM_PSCI_0_1 :return kvm_psci_0_1_call (vcpu);default : -return false; +return -EINVAL ; }; }
arch/arm64/include/asm/kvm_psci.h +1 -1
@@ -22,6 +22,6 @@#define KVM_ARM_PSCI_0_2 2int kvm_psci_version (struct kvm_vcpu *vcpu); -bool kvm_psci_call (struct kvm_vcpu *vcpu); +int kvm_psci_call (struct kvm_vcpu *vcpu);#endif /* __ARM64_KVM_PSCI_H__ */
arch/arm64/kvm/handle_exit.c +7 -3
@@ -30,11 +30,15 @@ typedef int (*exit_handle_fn)(struct kvm_vcpu *, struct kvm_run *);static int handle_hvc (struct kvm_vcpu *vcpu,struct kvm_run *run) { -if (kvm_psci_call (vcpu)) +int ret; + + ret =kvm_psci_call (vcpu); +if (ret <0 ) { +kvm_inject_undefined (vcpu);return 1 ; + } -kvm_inject_undefined (vcpu); -return 1 ; +return ret; }static int handle_smc (struct kvm_vcpu *vcpu,struct kvm_run *run)
Signature · 1 line
1.7.9.5
PATCH v11 6/12 KVM: Add KVM_EXIT_SYSTEM_EVENT to user space API header
In reply to Anup PatelCurrently, we don't have an exit reason to notify user space about a system-level event (for e.g. system reset or shutdown) triggered by the VCPU. This patch adds exit reason KVM_EXIT_SYSTEM_EVENT for this purpose. We can also inform user space about the 'type' and architecture specific 'flags' of a system-level event using the kvm_run structure.
This newly added KVM_EXIT_SYSTEM_EVENT will be used by KVM ARM/ARM64 in-kernel PSCI v0.2 support to reset/shutdown VMs.
--- Documentation/virtual/kvm/api.txt | 15 +++++++++++++++ include/uapi/linux/kvm.h | 8 ++++++++ 2 files changed, 23 insertions(+)
Documentation/virtual/kvm/api.txt +15 -0
@@ -2740,6 +2740,21 @@ It gets triggered whenever both KVM_CAP_PPC_EPR are enabled and an external interrupt has just been delivered into the guest. User space should put the acknowledged interrupt vector into the 'epr' field. + /* KVM_EXIT_SYSTEM_EVENT */ + struct { +#define KVM_SYSTEM_EVENT_SHUTDOWN 1 +#define KVM_SYSTEM_EVENT_RESET 2 + __u32 type; + __u64 flags; + } system_event; + +If exit_reason is KVM_EXIT_SYSTEM_EVENT then the vcpu has triggered +a system-level event using some architecture specific mechanism (hypercall +or some special instruction). In case of ARM/ARM64, this is triggered using +HVC instruction based PSCI call from the vcpu. The 'type' field describes +the system-level event type. The 'flags' field describes architecture +specific flags for the system-level event. + /* Fix the size of the union. */ char padding[256]; };
include/uapi/linux/kvm.h +8 -0
@@ -171,6 +171,7 @@ struct kvm_pit_config {#define KVM_EXIT_WATCHDOG 21#define KVM_EXIT_S390_TSCH 22#define KVM_EXIT_EPR 23 +#define KVM_EXIT_SYSTEM_EVENT 24/* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */
@@ -301,6 +302,13 @@ struct kvm_run {struct {__u32 epr; } epr; +/* KVM_EXIT_SYSTEM_EVENT */ +struct { +#define KVM_SYSTEM_EVENT_SHUTDOWN 1 +#define KVM_SYSTEM_EVENT_RESET 2 +__u32 type; +__u64 flags; + } system_event;/* Fix the size of the union. */ char padding[256 ]; };
Signature · 1 line
1.7.9.5
PATCH v11 7/12 ARM/ARM64: KVM: Emulate PSCI v0.2 SYSTEM_OFF and SYSTEM_RESET
In reply to Anup PatelThe PSCI v0.2 SYSTEM_OFF and SYSTEM_RESET functions are system-level functions hence cannot be fully emulated by in-kernel PSCI emulation code.
To tackle this, we forward PSCI v0.2 SYSTEM_OFF and SYSTEM_RESET function calls from vcpu to user space (i.e. QEMU or KVMTOOL) via kvm_run structure using KVM_EXIT_SYSTEM_EVENT exit reasons.
--- arch/arm/kvm/psci.c | 46 +++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 43 insertions(+), 3 deletions(-)
arch/arm/kvm/psci.c +43 -3
@@ -85,6 +85,23 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)return PSCI_RET_SUCCESS ; } +static void kvm_prepare_system_event (struct kvm_vcpu *vcpu,u32 type) +{ +memset (&vcpu->run->system_event,0 ,sizeof (vcpu->run->system_event)); + vcpu->run->system_event.type = type; + vcpu->run->exit_reason =KVM_EXIT_SYSTEM_EVENT ; +} + +static void kvm_psci_system_off (struct kvm_vcpu *vcpu) +{ +kvm_prepare_system_event (vcpu,KVM_SYSTEM_EVENT_SHUTDOWN ); +} + +static void kvm_psci_system_reset (struct kvm_vcpu *vcpu) +{ +kvm_prepare_system_event (vcpu,KVM_SYSTEM_EVENT_RESET ); +} +int kvm_psci_version (struct kvm_vcpu *vcpu) {if (test_bit (KVM_ARM_VCPU_PSCI_0_2 , vcpu->arch.features))
@@ -95,6 +112,7 @@ int kvm_psci_version(struct kvm_vcpu *vcpu)static int kvm_psci_0_2_call (struct kvm_vcpu *vcpu) { +int ret =1 ;unsigned long psci_fn = *vcpu_reg (vcpu,0 ) & ~((u32 )0 );unsigned long val;
@@ -114,13 +132,35 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu)case PSCI_0_2_FN64_CPU_ON : val =kvm_psci_vcpu_on (vcpu);break ; +case PSCI_0_2_FN_SYSTEM_OFF : +kvm_psci_system_off (vcpu); +/* +* We should'nt be going back to guest VCPU after +* receiving SYSTEM_OFF request. +* +* If user space accidently/deliberately resumes +* guest VCPU after SYSTEM_OFF request then guest +* VCPU should see internal failure from PSCI return +* value. To achieve this, we preload r0 (or x0) with +* PSCI return value INTERNAL_FAILURE. +*/ + val =PSCI_RET_INTERNAL_FAILURE ; + ret =0 ; +break ; +case PSCI_0_2_FN_SYSTEM_RESET : +kvm_psci_system_reset (vcpu); +/* +* Same reason as SYSTEM_OFF for preloading r0 (or x0) +* with PSCI return value INTERNAL_FAILURE. +*/ + val =PSCI_RET_INTERNAL_FAILURE ; + ret =0 ; +break ;case PSCI_0_2_FN_CPU_SUSPEND :case PSCI_0_2_FN_AFFINITY_INFO :case PSCI_0_2_FN_MIGRATE :case PSCI_0_2_FN_MIGRATE_INFO_TYPE :case PSCI_0_2_FN_MIGRATE_INFO_UP_CPU : -case PSCI_0_2_FN_SYSTEM_OFF : -case PSCI_0_2_FN_SYSTEM_RESET :case PSCI_0_2_FN64_CPU_SUSPEND :case PSCI_0_2_FN64_AFFINITY_INFO :case PSCI_0_2_FN64_MIGRATE :
@@ -132,7 +172,7 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) } *vcpu_reg (vcpu,0 ) = val; -return 1 ; +return ret; }static int kvm_psci_0_1_call (struct kvm_vcpu *vcpu)
Signature · 1 line
1.7.9.5
PATCH v11 8/12 ARM/ARM64: KVM: Emulate PSCI v0.2 AFFINITY_INFO
In reply to Anup PatelThis patch adds emulation of PSCI v0.2 AFFINITY_INFO function call for KVM ARM/ARM64. This is a VCPU-level function call which will be used to determine current state of given affinity level.
--- arch/arm/kvm/psci.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 50 insertions(+), 2 deletions(-)
arch/arm/kvm/psci.c +50 -2
@@ -27,6 +27,16 @@* as described in ARM document number ARM DEN 0022A. */ +#define AFFINITY_MASK (level) ~((0x1UL << ((level) * MPIDR_LEVEL_BITS)) - 1) + +static unsigned long psci_affinity_mask (unsigned long affinity_level) +{ +if (affinity_level <=3 ) +return MPIDR_HWID_BITMASK &AFFINITY_MASK (affinity_level); + +return 0 ; +} +static void kvm_psci_vcpu_off (struct kvm_vcpu *vcpu) { vcpu->arch.pause = true;
@@ -85,6 +95,42 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)return PSCI_RET_SUCCESS ; } +static unsigned long kvm_psci_vcpu_affinity_info (struct kvm_vcpu *vcpu) +{ +int i; +unsigned long mpidr; +unsigned long target_affinity; +unsigned long target_affinity_mask; +unsigned long lowest_affinity_level; +struct kvm *kvm = vcpu->kvm; +struct kvm_vcpu *tmp; + + target_affinity = *vcpu_reg (vcpu,1 ); + lowest_affinity_level = *vcpu_reg (vcpu,2 ); + +/* Determine target affinity mask */ + target_affinity_mask =psci_affinity_mask (lowest_affinity_level); +if (!target_affinity_mask) +return PSCI_RET_INVALID_PARAMS ; + +/* Ignore other bits of target affinity */ + target_affinity &= target_affinity_mask; + +/* +* If one or more VCPU matching target affinity are running +* then ON else OFF +*/ +kvm_for_each_vcpu (i, tmp, kvm) { + mpidr =kvm_vcpu_get_mpidr (tmp); +if (((mpidr & target_affinity_mask) == target_affinity) && + !tmp->arch.pause) { +return PSCI_0_2_AFFINITY_LEVEL_ON ; + } + } + +return PSCI_0_2_AFFINITY_LEVEL_OFF ; +} +static void kvm_prepare_system_event (struct kvm_vcpu *vcpu,u32 type) {memset (&vcpu->run->system_event,0 ,sizeof (vcpu->run->system_event));
@@ -132,6 +178,10 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu)case PSCI_0_2_FN64_CPU_ON : val =kvm_psci_vcpu_on (vcpu);break ; +case PSCI_0_2_FN_AFFINITY_INFO : +case PSCI_0_2_FN64_AFFINITY_INFO : + val =kvm_psci_vcpu_affinity_info (vcpu); +break ;case PSCI_0_2_FN_SYSTEM_OFF :kvm_psci_system_off (vcpu);/*
@@ -157,12 +207,10 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) ret =0 ;break ;case PSCI_0_2_FN_CPU_SUSPEND : -case PSCI_0_2_FN_AFFINITY_INFO :case PSCI_0_2_FN_MIGRATE :case PSCI_0_2_FN_MIGRATE_INFO_TYPE :case PSCI_0_2_FN_MIGRATE_INFO_UP_CPU :case PSCI_0_2_FN64_CPU_SUSPEND : -case PSCI_0_2_FN64_AFFINITY_INFO :case PSCI_0_2_FN64_MIGRATE :case PSCI_0_2_FN64_MIGRATE_INFO_UP_CPU : val =PSCI_RET_NOT_SUPPORTED ;
Signature · 1 line
1.7.9.5
PATCH v11 9/12 ARM/ARM64: KVM: Emulate PSCI v0.2 MIGRATE_INFO_TYPE and related functions
In reply to Anup PatelThis patch adds emulation of PSCI v0.2 MIGRATE, MIGRATE_INFO_TYPE, and MIGRATE_INFO_UP_CPU function calls for KVM ARM/ARM64.
KVM ARM/ARM64 being a hypervisor (and not a Trusted OS), we cannot provide this functions hence we emulate these functions in following way:
1. MIGRATE - Returns "Not Supported"
2. MIGRATE_INFO_TYPE - Return 2 i.e. Trusted OS is not present
3. MIGRATE_INFO_UP_CPU - Returns "Not Supported"
--- arch/arm/kvm/psci.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)
arch/arm/kvm/psci.c +16 -5
@@ -182,6 +182,22 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu)case PSCI_0_2_FN64_AFFINITY_INFO : val =kvm_psci_vcpu_affinity_info (vcpu);break ; +case PSCI_0_2_FN_MIGRATE : +case PSCI_0_2_FN64_MIGRATE : + val =PSCI_RET_NOT_SUPPORTED ; +break ; +case PSCI_0_2_FN_MIGRATE_INFO_TYPE : +/* +* Trusted OS is MP hence does not require migration +* or +* Trusted OS is not present +*/ + val =PSCI_0_2_TOS_MP ; +break ; +case PSCI_0_2_FN_MIGRATE_INFO_UP_CPU : +case PSCI_0_2_FN64_MIGRATE_INFO_UP_CPU : + val =PSCI_RET_NOT_SUPPORTED ; +break ;case PSCI_0_2_FN_SYSTEM_OFF :kvm_psci_system_off (vcpu);/*
@@ -207,12 +223,7 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) ret =0 ;break ;case PSCI_0_2_FN_CPU_SUSPEND : -case PSCI_0_2_FN_MIGRATE : -case PSCI_0_2_FN_MIGRATE_INFO_TYPE : -case PSCI_0_2_FN_MIGRATE_INFO_UP_CPU :case PSCI_0_2_FN64_CPU_SUSPEND : -case PSCI_0_2_FN64_MIGRATE : -case PSCI_0_2_FN64_MIGRATE_INFO_UP_CPU : val =PSCI_RET_NOT_SUPPORTED ;break ;default :
Signature · 1 line
1.7.9.5
PATCH v11 10/12 ARM/ARM64: KVM: Fix CPU_ON emulation for PSCI v0.2
In reply to Anup PatelAs-per PSCI v0.2, the source CPU provides physical address of "entry point" and "context id" for starting a target CPU. Also, if target CPU is already running then we should return ALREADY_ON.
Current emulation of CPU_ON function does not consider physical address of "context id" and returns INVALID_PARAMETERS if target CPU is already running.
This patch updates kvm_psci_vcpu_on() such that it works for both PSCI v0.1 and PSCI v0.2.
--- arch/arm/kvm/psci.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
arch/arm/kvm/psci.c +14 -1
@@ -48,6 +48,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)struct kvm_vcpu *vcpu =NULL , *tmp;wait_queue_head_t *wq;unsigned long cpu_id; +unsigned long context_id;unsigned long mpidr;phys_addr_t target_pc;int i;
@@ -68,10 +69,17 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)* Make sure the caller requested a valid CPU and that the CPU is * turned off. */ -if (!vcpu || !vcpu->arch.pause) +if (!vcpu)return PSCI_RET_INVALID_PARAMS ; +if (!vcpu->arch.pause) { +if (kvm_psci_version (source_vcpu) !=KVM_ARM_PSCI_0_1 ) +return PSCI_RET_ALREADY_ON ; +else +return PSCI_RET_INVALID_PARAMS ; + } target_pc = *vcpu_reg (source_vcpu,2 ); + context_id = *vcpu_reg (source_vcpu,3 );kvm_reset_vcpu (vcpu);
@@ -86,6 +94,11 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)kvm_vcpu_set_be (vcpu); *vcpu_pc (vcpu) = target_pc; +/* +* NOTE: We always update r0 (or x0) because for PSCI v0.1 +* the general puspose registers are undefined upon CPU_ON. +*/ + *vcpu_reg (vcpu,0 ) = context_id; vcpu->arch.pause = false;smp_mb ();/* Make sure the above is visible */
Signature · 1 line
1.7.9.5
PATCH v11 11/12 ARM/ARM64: KVM: Emulate PSCI v0.2 CPU_SUSPEND
In reply to Anup PatelThis patch adds emulation of PSCI v0.2 CPU_SUSPEND function call for KVM ARM/ARM64. This is a CPU-level function call which can suspend current CPU or current CPU cluster. We don't have VCPU clusters in KVM so we only suspend the current VCPU.
The CPU_SUSPEND emulation is not tested much because currently there is no CPUIDLE driver in Linux kernel that uses PSCI CPU_SUSPEND. The PSCI CPU_SUSPEND implementation in ARM64 kernel was tested using a Simple CPUIDLE driver which is not published due to unstable DT-bindings for PSCI.
(For more info, http://lwn.net/Articles/574950/)
For simplicity, we implement CPU_SUSPEND emulation similar to WFI (Wait-for-interrupt) emulation and we also treat power-down request to be same as stand-by request. This is consistent with section 5.4.1 and section 5.4.2 of PSCI v0.2 specification.
--- arch/arm/kvm/psci.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-)
arch/arm/kvm/psci.c +24 -4
@@ -37,6 +37,26 @@ static unsigned long psci_affinity_mask(unsigned long affinity_level)return 0 ; } +static unsigned long kvm_psci_vcpu_suspend (struct kvm_vcpu *vcpu) +{ +/* +* NOTE: For simplicity, we make VCPU suspend emulation to be +* same-as WFI (Wait-for-interrupt) emulation. +* +* This means for KVM the wakeup events are interrupts and +* this is consistent with intended use of StateID as described +* in section 5.4.1 of PSCI v0.2 specification (ARM DEN 0022A). +* +* Further, we also treat power-down request to be same as +* stand-by request as-per section 5.4.2 clause 3 of PSCI v0.2 +* specification (ARM DEN 0022A). This means all suspend states +* for KVM will preserve the register state. +*/ +kvm_vcpu_block (vcpu); + +return PSCI_RET_SUCCESS ; +} +static void kvm_psci_vcpu_off (struct kvm_vcpu *vcpu) { vcpu->arch.pause = true;
@@ -183,6 +203,10 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu)*/ val =2 ;break ; +case PSCI_0_2_FN_CPU_SUSPEND : +case PSCI_0_2_FN64_CPU_SUSPEND : + val =kvm_psci_vcpu_suspend (vcpu); +break ;case PSCI_0_2_FN_CPU_OFF :kvm_psci_vcpu_off (vcpu); val =PSCI_RET_SUCCESS ;
@@ -235,10 +259,6 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) val =PSCI_RET_INTERNAL_FAILURE ; ret =0 ;break ; -case PSCI_0_2_FN_CPU_SUSPEND : -case PSCI_0_2_FN64_CPU_SUSPEND : - val =PSCI_RET_NOT_SUPPORTED ; -break ;default :return -EINVAL ; }
Signature · 1 line
1.7.9.5
PATCH v11 12/12 ARM/ARM64: KVM: Advertise KVM_CAP_ARM_PSCI_0_2 to user space
In reply to Anup PatelWe have PSCI v0.2 emulation available in KVM ARM/ARM64 hence advertise this to user space (i.e. QEMU or KVMTOOL) via KVM_CHECK_EXTENSION ioctl.
--- arch/arm/kvm/arm.c | 1 + 1 file changed, 1 insertion(+)
arch/arm/kvm/arm.c +1 -0
@@ -197,6 +197,7 @@ int kvm_dev_ioctl_check_extension(long ext)case KVM_CAP_DESTROY_MEMORY_REGION_WORKS :case KVM_CAP_ONE_REG :case KVM_CAP_ARM_PSCI : +case KVM_CAP_ARM_PSCI_0_2 : r =1 ;break ;case KVM_CAP_COALESCED_MMIO :
Signature · 1 line
1.7.9.5
Quoted 10 lines · Anup Patel
Currently, KVM ARM/ARM64 only provides in-kernel emulation of Power State and Coordination Interface (PSCI) v0.1.
This patchset aims at providing newer PSCI v0.2 for KVM ARM/ARM64 VCPUs such that it does not break current KVM ARM/ARM64 ABI.
The user space tools (i.e. QEMU or KVMTOOL) will have to explicitly enable
KVM_ARM_VCPU_PSCI_0_2feature usingKVM_ARM_VCPU_INITioctl for providing PSCI v0.2 to VCPUs.
I feel like this is ready to queue for the next merge window. Last chance to object.
-Christoffer
Quoted 7 lines · Christoffer Dall
Quoted 6 lines · Anup Patel
Currently, KVM ARM/ARM64 only provides in-kernel emulation of Power State and Coordination Interface (PSCI) v0.1.
This patchset aims at providing newer PSCI v0.2 for KVM ARM/ARM64 VCPUs such that it does not break current KVM ARM/ARM64 ABI.
Quoted 7 lines
Quoted 4 lines
The user space tools (i.e. QEMU or KVMTOOL) will have to explicitly enable
KVM_ARM_VCPU_PSCI_0_2feature usingKVM_ARM_VCPU_INITioctl for providing PSCI v0.2 to VCPUs.I feel like this is ready to queue for the next merge window. Last chance to object.
Looks OK to me.
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Christoffer Dall
AppliedQuoted 13 lines · Marc Zyngier
Quoted 12 lines · Christoffer Dall
Quoted 10 lines · Anup Patel
Currently, KVM ARM/ARM64 only provides in-kernel emulation of Power State and Coordination Interface (PSCI) v0.1.
This patchset aims at providing newer PSCI v0.2 for KVM ARM/ARM64 VCPUs such that it does not break current KVM ARM/ARM64 ABI.
The user space tools (i.e. QEMU or KVMTOOL) will have to explicitly enable
KVM_ARM_VCPU_PSCI_0_2feature usingKVM_ARM_VCPU_INITioctl for providing PSCI v0.2 to VCPUs.
Quoted 5 lines
Quoted 2 lines
I feel like this is ready to queue for the next merge window. Last chance to object.
Looks OK to me.
Applied to queue. Note that KVM_CAP_ARM_PSCI_0_2 is now defined to 102 after rebasing the patch set onto kvm/next.
Thanks,
-Christoffer