Skip to contentSkip to conversation

In-kernel PSCI v0.2 emulation for KVM ARM/ARM64

PATCH Superseded Anup Patel 2 participants 6 messages Feb 4, 2014

Subsystems (5)
Documentation Arm (32-bit) KVM arm64 +1 more subsystems

v3 of 11 Superseded Latest is v11 →

lore
Share & tools
lore

Anup Patel

author
#0 lore raw

v3 cover In-kernel PSCI v0.2 emulation for KVM ARM/ARM64

Currently, KVM ARM/ARM64 only provides in-kernel emulation of Power State and Coordination Interface (PSCI) v0.1.

This patchset aims at providing newer PSCI v0.2 for KVM ARM/ARM64 VCPUs such that it does not break current KVM ARM/ARM64 ABI. Also, the patchset provides emulation of only few PSCI v0.2 functions such as PSCI_VERSION, CPU_ON, and CPU_OFF. Emulation of other PSCI v0.2 functions will be added later.

The user space tools (i.e. QEMU or KVMTOOL) will have to explicitly enable KVM_ARM_VCPU_PSCI_0_2 feature using KVM_ARM_VCPU_INIT ioctl for providing PSCI v0.2 to VCPUs.

Changlog:

V3:
- Make KVM_ARM_VCPU_PSCI_0_2 feature experiementatl for now so that it fails for user space till all mandatory PSCI v0.2 functions are emulated by KVM ARM/ARM64
- Have separate patch for making KVM_ARM_VCPU_PSCI_0_2 feature available to user space. This patch can be defferred for now.

V2:
- Don't rename PSCI return values KVM_PSCI_RET_NI and KVM_PSCI_RET_INVAL
- Added kvm_psci_version() to get PSCI version available to VCPU
- Fixed grammer in Documentation/virtual/kvm/api.txt

V1:
- Initial RFC PATCH

Anup Patel (4):
ARM/ARM64: KVM: Add support for PSCI v0.2 emulation
KVM: Add capability to advertise PSCI v0.2 support
KVM: Documentation: Add info regarding KVM_ARM_VCPU_PSCI_0_2 feature ARM/ARM64: KVM: Allow KVM_ARM_VCPU_PSCI_0_2 feature for user space

 Documentation/virtual/kvm/api.txt |    2 +
 arch/arm/include/asm/kvm_host.h   |    2 +-
 arch/arm/include/asm/kvm_psci.h   |    4 ++
 arch/arm/include/uapi/asm/kvm.h   |   35 ++++++++++++++-
 arch/arm/kvm/arm.c                |    1 +
 arch/arm/kvm/psci.c               |   85 +++++++++++++++++++++++++++++++------
 arch/arm64/include/asm/kvm_host.h |    2 +-
 arch/arm64/include/asm/kvm_psci.h |    4 ++
 arch/arm64/include/uapi/asm/kvm.h |   35 ++++++++++++++-
 include/uapi/linux/kvm.h          |    1 +
 10 files changed, 155 insertions(+), 16 deletions(-)
Signature · 1 line
1.7.9.5
author
#1 lore raw

PATCH v3 1/4 ARM/ARM64: KVM: Add support for PSCI v0.2 emulation

Currently, the in-kernel PSCI emulation provides PSCI v0.1 interface to VCPUs. This patch extends current in-kernel PSCI emulation to provide PSCI v0.2 interface to VCPUs.

By default, ARM/ARM64 KVM will always provide PSCI v0.1 interface for keeping the ABI backward-compatible.

To select PSCI v0.2 interface for VCPUs, the user space (i.e. QEMU or KVMTOOL) will have to set KVM_ARM_VCPU_PSCI_0_2 feature when doing VCPU init using KVM_ARM_VCPU_INIT ioctl.

Please note that we do not provide all mandatory functions required by PSCI v0.2 which makes this implemenation as a base for extension by subsequent patches. This also means KVM_ARM_VCPU_PSCI_0_2 feature is experimental and will not be available to user space untill all PSCI v0.2 mandatory functions are provided by KVM ARM/ARM64.

Signed-off-by Anup Patel <[email protected]>
Signed-off-by Pranavkumar Sawargaonkar <[email protected]>
---
 arch/arm/include/asm/kvm_host.h   |    2 +-
 arch/arm/include/asm/kvm_psci.h   |    4 ++
 arch/arm/include/uapi/asm/kvm.h   |   35 ++++++++++++++-
 arch/arm/kvm/guest.c              |    3 ++
 arch/arm/kvm/psci.c               |   85 +++++++++++++++++++++++++++++++------
 arch/arm64/include/asm/kvm_host.h |    2 +-
 arch/arm64/include/asm/kvm_psci.h |    4 ++
 arch/arm64/include/uapi/asm/kvm.h |   35 ++++++++++++++-
 arch/arm64/kvm/guest.c            |    3 ++
 9 files changed, 157 insertions(+), 16 deletions(-)
arch/arm/include/asm/kvm_host.h +1 -1
index 09af149..193ceaf 100644
@@ -36,7 +36,7 @@
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
 #define KVM_HAVE_ONE_REG
 
-#define KVM_VCPU_MAX_FEATURES 1
+#define KVM_VCPU_MAX_FEATURES 2
 
 #include <kvm/arm_vgic.h>
arch/arm/include/asm/kvm_psci.h +4 -0
index 9a83d98..4c0e3e1 100644
@@ -18,6 +18,10 @@
 #ifndef __ARM_KVM_PSCI_H__
 #define __ARM_KVM_PSCI_H__
 
+#define KVM_ARM_PSCI_0_1	1
+#define KVM_ARM_PSCI_0_2	2
+
+int kvm_psci_version(struct kvm_vcpu *vcpu);
 bool kvm_psci_call(struct kvm_vcpu *vcpu);
 
 #endif /* __ARM_KVM_PSCI_H__ */
arch/arm/include/uapi/asm/kvm.h +34 -1
index ef0c878..9c922d9 100644
@@ -83,6 +83,7 @@ struct kvm_regs {
 #define KVM_VGIC_V2_CPU_SIZE		0x2000
 
 #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
+#define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
 
 struct kvm_vcpu_init {
 	__u32 target;

@@ -192,7 +193,7 @@ struct kvm_arch_memory_slot {
 /* Highest supported SPI, from VGIC_NR_IRQS */
 #define KVM_ARM_IRQ_GIC_MAX		127
 
-/* PSCI interface */
+/* PSCI v0.1 interface */
 #define KVM_PSCI_FN_BASE		0x95c1ba5e
 #define KVM_PSCI_FN(n)			(KVM_PSCI_FN_BASE + (n))
 

@@ -201,9 +202,41 @@ struct kvm_arch_memory_slot {
 #define KVM_PSCI_FN_CPU_ON		KVM_PSCI_FN(2)
 #define KVM_PSCI_FN_MIGRATE		KVM_PSCI_FN(3)
 
+/* PSCI v0.2 interface */
+#define KVM_PSCI_0_2_FN_BASE		0x84000000
+#define KVM_PSCI_0_2_FN(n)		(KVM_PSCI_0_2_FN_BASE + (n))
+#define KVM_PSCI_0_2_FN64_BASE		0xC4000000
+#define KVM_PSCI_0_2_FN64(n)		(KVM_PSCI_0_2_FN64_BASE + (n))
+
+#define KVM_PSCI_0_2_FN_PSCI_VERSION	KVM_PSCI_0_2_FN(0)
+#define KVM_PSCI_0_2_FN_CPU_SUSPEND	KVM_PSCI_0_2_FN(1)
+#define KVM_PSCI_0_2_FN_CPU_OFF		KVM_PSCI_0_2_FN(2)
+#define KVM_PSCI_0_2_FN_CPU_ON		KVM_PSCI_0_2_FN(3)
+#define KVM_PSCI_0_2_FN_AFFINITY_INFO	KVM_PSCI_0_2_FN(4)
+#define KVM_PSCI_0_2_FN_MIGRATE		KVM_PSCI_0_2_FN(5)
+#define KVM_PSCI_0_2_FN_MIGRATE_INFO_TYPE \
+					KVM_PSCI_0_2_FN(6)
+#define KVM_PSCI_0_2_FN_MIGRATE_INFO_UP_CPU \
+					KVM_PSCI_0_2_FN(7)
+#define KVM_PSCI_0_2_FN_SYSTEM_OFF	KVM_PSCI_0_2_FN(8)
+#define KVM_PSCI_0_2_FN_SYSTEM_RESET	KVM_PSCI_0_2_FN(9)
+
+#define KVM_PSCI_0_2_FN64_CPU_SUSPEND	KVM_PSCI_0_2_FN64(1)
+#define KVM_PSCI_0_2_FN64_CPU_ON	KVM_PSCI_0_2_FN64(3)
+#define KVM_PSCI_0_2_FN64_AFFINITY_INFO	KVM_PSCI_0_2_FN64(4)
+#define KVM_PSCI_0_2_FN64_MIGRATE	KVM_PSCI_0_2_FN64(5)
+#define KVM_PSCI_0_2_FN64_MIGRATE_INFO_UP_CPU \
+					KVM_PSCI_0_2_FN64(7)
+
+/* PSCI return values */
 #define KVM_PSCI_RET_SUCCESS		0
 #define KVM_PSCI_RET_NI			((unsigned long)-1)
 #define KVM_PSCI_RET_INVAL		((unsigned long)-2)
 #define KVM_PSCI_RET_DENIED		((unsigned long)-3)
+#define KVM_PSCI_RET_ALREADY_ON		((unsigned long)-4)
+#define KVM_PSCI_RET_ON_PENDING		((unsigned long)-5)
+#define KVM_PSCI_RET_INTERNAL_FAILURE	((unsigned long)-6)
+#define KVM_PSCI_RET_NOT_PRESENT	((unsigned long)-7)
+#define KVM_PSCI_RET_DISABLED		((unsigned long)-8)
 
 #endif /* __ARM_KVM_H__ */
arch/arm/kvm/guest.c +3 -0
index b23a59c..89929b6 100644
@@ -307,6 +307,9 @@ int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
 		if (test_bit(i, (void *)init->features)) {
 			if (i >= KVM_VCPU_MAX_FEATURES)
 				return -ENOENT;
+			/* Don't allow setting experimental features */
+			if (i == KVM_ARM_VCPU_PSCI_0_2)
+				return -ENOENT;
 			set_bit(i, vcpu->arch.features);
 		}
 	}
arch/arm/kvm/psci.c +73 -12

100 lines not on this page · raw message

arch/arm64/include/asm/kvm_host.h +1 -1
index 0a1d697..92242ce 100644
@@ -39,7 +39,7 @@
 #include <kvm/arm_vgic.h>
 #include <kvm/arm_arch_timer.h>
 
-#define KVM_VCPU_MAX_FEATURES 2
+#define KVM_VCPU_MAX_FEATURES 3
 
 struct kvm_vcpu;
 int kvm_target_cpu(void);
arch/arm64/include/asm/kvm_psci.h +4 -0
index e301a48..e25c658 100644
@@ -18,6 +18,10 @@
 #ifndef __ARM64_KVM_PSCI_H__
 #define __ARM64_KVM_PSCI_H__
 
+#define KVM_ARM_PSCI_0_1	1
+#define KVM_ARM_PSCI_0_2	2
+
+int kvm_psci_version(struct kvm_vcpu *vcpu);
 bool kvm_psci_call(struct kvm_vcpu *vcpu);
 
 #endif /* __ARM64_KVM_PSCI_H__ */
arch/arm64/include/uapi/asm/kvm.h +34 -1
index 495ab6f..31c2f54 100644
@@ -77,6 +77,7 @@ struct kvm_regs {
 
 #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
 #define KVM_ARM_VCPU_EL1_32BIT		1 /* CPU running a 32bit VM */
+#define KVM_ARM_VCPU_PSCI_0_2		2 /* CPU uses PSCI v0.2 */
 
 struct kvm_vcpu_init {
 	__u32 target;

@@ -168,7 +169,7 @@ struct kvm_arch_memory_slot {
 /* Highest supported SPI, from VGIC_NR_IRQS */
 #define KVM_ARM_IRQ_GIC_MAX		127
 
-/* PSCI interface */
+/* PSCI v0.1 interface */
 #define KVM_PSCI_FN_BASE		0x95c1ba5e
 #define KVM_PSCI_FN(n)			(KVM_PSCI_FN_BASE + (n))
 

@@ -177,10 +178,42 @@ struct kvm_arch_memory_slot {
 #define KVM_PSCI_FN_CPU_ON		KVM_PSCI_FN(2)
 #define KVM_PSCI_FN_MIGRATE		KVM_PSCI_FN(3)
 
+/* PSCI v0.2 interface */
+#define KVM_PSCI_0_2_FN_BASE		0x84000000
+#define KVM_PSCI_0_2_FN(n)		(KVM_PSCI_0_2_FN_BASE + (n))
+#define KVM_PSCI_0_2_FN64_BASE		0xC4000000
+#define KVM_PSCI_0_2_FN64(n)		(KVM_PSCI_0_2_FN64_BASE + (n))
+
+#define KVM_PSCI_0_2_FN_PSCI_VERSION	KVM_PSCI_0_2_FN(0)
+#define KVM_PSCI_0_2_FN_CPU_SUSPEND	KVM_PSCI_0_2_FN(1)
+#define KVM_PSCI_0_2_FN_CPU_OFF		KVM_PSCI_0_2_FN(2)
+#define KVM_PSCI_0_2_FN_CPU_ON		KVM_PSCI_0_2_FN(3)
+#define KVM_PSCI_0_2_FN_AFFINITY_INFO	KVM_PSCI_0_2_FN(4)
+#define KVM_PSCI_0_2_FN_MIGRATE		KVM_PSCI_0_2_FN(5)
+#define KVM_PSCI_0_2_FN_MIGRATE_INFO_TYPE \
+					KVM_PSCI_0_2_FN(6)
+#define KVM_PSCI_0_2_FN_MIGRATE_INFO_UP_CPU \
+					KVM_PSCI_0_2_FN(7)
+#define KVM_PSCI_0_2_FN_SYSTEM_OFF	KVM_PSCI_0_2_FN(8)
+#define KVM_PSCI_0_2_FN_SYSTEM_RESET	KVM_PSCI_0_2_FN(9)
+
+#define KVM_PSCI_0_2_FN64_CPU_SUSPEND	KVM_PSCI_0_2_FN64(1)
+#define KVM_PSCI_0_2_FN64_CPU_ON	KVM_PSCI_0_2_FN64(3)
+#define KVM_PSCI_0_2_FN64_AFFINITY_INFO	KVM_PSCI_0_2_FN64(4)
+#define KVM_PSCI_0_2_FN64_MIGRATE	KVM_PSCI_0_2_FN64(5)
+#define KVM_PSCI_0_2_FN64_MIGRATE_INFO_UP_CPU \
+					KVM_PSCI_0_2_FN64(7)
+
+/* PSCI return values */
 #define KVM_PSCI_RET_SUCCESS		0
 #define KVM_PSCI_RET_NI			((unsigned long)-1)
 #define KVM_PSCI_RET_INVAL		((unsigned long)-2)
 #define KVM_PSCI_RET_DENIED		((unsigned long)-3)
+#define KVM_PSCI_RET_ALREADY_ON		((unsigned long)-4)
+#define KVM_PSCI_RET_ON_PENDING		((unsigned long)-5)
+#define KVM_PSCI_RET_INTERNAL_FAILURE	((unsigned long)-6)
+#define KVM_PSCI_RET_NOT_PRESENT	((unsigned long)-7)
+#define KVM_PSCI_RET_DISABLED		((unsigned long)-8)
 
 #endif
arch/arm64/kvm/guest.c +3 -0
index 0874557..b27877c 100644
@@ -246,6 +246,9 @@ int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
 		if (init->features[i / 32] & (1 << (i % 32))) {
 			if (i >= KVM_VCPU_MAX_FEATURES)
 				return -ENOENT;
+			/* Don't allow setting experimental features */
+			if (i == KVM_ARM_VCPU_PSCI_0_2)
+				return -ENOENT;
 			set_bit(i, vcpu->arch.features);
 		}
 	}
Signature · 1 line
1.7.9.5
author
#2 lore raw

PATCH v3 2/4 KVM: Add capability to advertise PSCI v0.2 support

In reply to Anup Patel

User space (i.e. QEMU or KVMTOOL) should be able to check whether KVM ARM/ARM64 supports in-kernel PSCI v0.2 emulation. For this purpose, we define KVM_CAP_ARM_PSCI_0_2 in KVM user space interface header.

Signed-off-by Anup Patel <[email protected]>
Signed-off-by Pranavkumar Sawargaonkar <[email protected]>
---
 arch/arm/kvm/arm.c       |    1 +
 include/uapi/linux/kvm.h |    1 +
 2 files changed, 2 insertions(+)
arch/arm/kvm/arm.c +1 -0
index 1d8248e..c8a71df 100644
@@ -197,6 +197,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 	case KVM_CAP_DESTROY_MEMORY_REGION_WORKS:
 	case KVM_CAP_ONE_REG:
 	case KVM_CAP_ARM_PSCI:
+	case KVM_CAP_ARM_PSCI_0_2:
 		r = 1;
 		break;
 	case KVM_CAP_COALESCED_MMIO:
include/uapi/linux/kvm.h +1 -0
index 932d7f2..fb3c3f3 100644
@@ -675,6 +675,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_SPAPR_MULTITCE 94
 #define KVM_CAP_EXT_EMUL_CPUID 95
 #define KVM_CAP_HYPERV_TIME 96
+#define KVM_CAP_ARM_PSCI_0_2 97
 
 #ifdef KVM_CAP_IRQ_ROUTING
Signature · 1 line
1.7.9.5
author
#3 lore raw

PATCH v3 3/4 KVM: Documentation: Add info regarding KVM_ARM_VCPU_PSCI_0_2 feature

In reply to Anup Patel

We have in-kernel emulation of PSCI v0.2 in KVM ARM/ARM64. To provide PSCI v0.2 interface to VCPUs, we have to enable KVM_ARM_VCPU_PSCI_0_2 feature when doing KVM_ARM_VCPU_INIT ioctl.

The patch updates documentation of KVM_ARM_VCPU_INIT ioctl to provide info regarding KVM_ARM_VCPU_PSCI_0_2 feature.

Signed-off-by Anup Patel <[email protected]>
Signed-off-by Pranavkumar Sawargaonkar <[email protected]>
---
 Documentation/virtual/kvm/api.txt |    2 ++
 1 file changed, 2 insertions(+)
Documentation/virtual/kvm/api.txt +2 -0
index 6cd63a9..73cb211 100644
@@ -2347,6 +2347,8 @@ Possible features:
 	  Depends on KVM_CAP_ARM_PSCI.
 	- KVM_ARM_VCPU_EL1_32BIT: Starts the CPU in a 32bit mode.
 	  Depends on KVM_CAP_ARM_EL1_32BIT (arm64 only).
+	- KVM_ARM_VCPU_PSCI_0_2: Emulate PSCI v0.2 for the CPU.
+	  Depends on KVM_CAP_ARM_PSCI_0_2.
 
 
 4.83 KVM_ARM_PREFERRED_TARGET
Signature · 1 line
1.7.9.5
author
#4 lore raw

PATCH v3 4/4 ARM/ARM64: KVM: Allow KVM_ARM_VCPU_PSCI_0_2 feature for user space

In reply to Anup Patel

Allow user space to set KVM_ARM_VCPU_PSCI_0_2 feature because all mandatory PSCI v0.2 functions (i.e. complete PSCI v0.2) are available.

Signed-off-by Anup Patel <[email protected]>
Signed-off-by Pranavkumar Sawargaonkar <[email protected]>
---
 arch/arm/kvm/guest.c   |    3 ---
 arch/arm64/kvm/guest.c |    3 ---
 2 files changed, 6 deletions(-)
arch/arm/kvm/guest.c +0 -3
index 89929b6..b23a59c 100644
@@ -307,9 +307,6 @@ int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
 		if (test_bit(i, (void *)init->features)) {
 			if (i >= KVM_VCPU_MAX_FEATURES)
 				return -ENOENT;
-			/* Don't allow setting experimental features */
-			if (i == KVM_ARM_VCPU_PSCI_0_2)
-				return -ENOENT;
 			set_bit(i, vcpu->arch.features);
 		}
 	}
arch/arm64/kvm/guest.c +0 -3
index b27877c..0874557 100644
@@ -246,9 +246,6 @@ int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
 		if (init->features[i / 32] & (1 << (i % 32))) {
 			if (i >= KVM_VCPU_MAX_FEATURES)
 				return -ENOENT;
-			/* Don't allow setting experimental features */
-			if (i == KVM_ARM_VCPU_PSCI_0_2)
-				return -ENOENT;
 			set_bit(i, vcpu->arch.features);
 		}
 	}
Signature · 1 line
1.7.9.5
#5 lore raw

v3 cover In-kernel PSCI v0.2 emulation for KVM ARM/ARM64

In reply to Anup Patel
Quoted 13 lines · Anup Patel

Currently, KVM ARM/ARM64 only provides in-kernel emulation of Power State and Coordination Interface (PSCI) v0.1.

This patchset aims at providing newer PSCI v0.2 for KVM ARM/ARM64 VCPUs such that it does not break current KVM ARM/ARM64 ABI. Also, the patchset provides emulation of only few PSCI v0.2 functions such as PSCI_VERSION, CPU_ON, and CPU_OFF. Emulation of other PSCI v0.2 functions will be added later.

The user space tools (i.e. QEMU or KVMTOOL) will have to explicitly enable KVM_ARM_VCPU_PSCI_0_2 feature using KVM_ARM_VCPU_INIT ioctl for providing PSCI v0.2 to VCPUs.

Quoted 9 lines

Changlog:

V3:
- Make KVM_ARM_VCPU_PSCI_0_2 feature experiementatl for now so that it fails for user space till all mandatory PSCI v0.2 functions are emulated by KVM ARM/ARM64
- Have separate patch for making KVM_ARM_VCPU_PSCI_0_2 feature available to user space. This patch can be defferred for now.

I think we just want the feature bit when there's proper PSCI 0.2 support as per Mark's comments, so I'll hold off with reviewing until we have a more complete patch set.

Thanks,
-Christoffer