Skip to contentSkip to conversation

Target CPU=Host implementation for KVM ARM/ARM64

RFC PATCH New Anup Patel 7 participants 21 messages Sep 5, 2013 – Sep 6, 2013

Subsystems (4)
lore
Share & tools
lore

Anup Patel

author
#0 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

It will be very useful for user space if it has a method for specifying KVM ARM/ARM64 to give a VCPU with target type suitable to underlying host but with particular set of features.

In other words, user space might not be interested in having a particular target type for VCPU but it will certainely want particular set of features in the VCPU.

The patch tries to implement above described method of specifying VCPU target CPU=Host from user space by extending the KVM_ARM_VCPU_INIT ioctl and having a dummy target KVM_ARM_TARGET_HOST which means Target CPU same as (or suitable to) underlying host.

This change in KVM_ARM_VCPU_INIT ioctl is backward compatible to its current semantics hence does not break user space. The only difference is that it will now update the struct kvm_vcpu_init instance (passed by user space) with targer type and features of the VCPU that will be emulated by KVM ARM/ARM64.

Anup Patel (3):
ARM: KVM: Implement target CPU=Host
ARM64: KVM: Implement target CPU=Host
KVM: ARM: Update documentation for KVM_ARM_VCPU_INIT ioctl

 Documentation/virtual/kvm/api.txt |    5 ++++-
 arch/arm/include/asm/kvm_host.h   |    2 +-
 arch/arm/include/uapi/asm/kvm.h   |    1 +
 arch/arm/kvm/arm.c                |    9 ++++++++-
 arch/arm/kvm/guest.c              |   11 +++++++++--
 arch/arm64/include/asm/kvm_host.h |    2 +-
 arch/arm64/include/uapi/asm/kvm.h |    2 +-
 arch/arm64/kvm/guest.c            |   10 ++++++++--
 8 files changed, 33 insertions(+), 9 deletions(-)
Signature · 1 line
1.7.9.5
author
#1 lore raw

RFC PATCH 1/3 ARM: KVM: Implement target CPU=Host

This patch implements KVM_ARM_TARGET_HOST for KVM ARM.

If user space provides KVM_ARM_TARGET_HOST as target type in KVM_ARM_VCPU_INIT ioctl then we find out appropriate target type based on underlying host and return that to user space via struct kvm_vcpu_init.

---
 arch/arm/include/asm/kvm_host.h |    2 +-
 arch/arm/include/uapi/asm/kvm.h |    1 +
 arch/arm/kvm/arm.c              |    9 ++++++++-
 arch/arm/kvm/guest.c            |   11 +++++++++--
 4 files changed, 19 insertions(+), 4 deletions(-)
arch/arm/include/asm/kvm_host.h +1 -1
index 7d22517..3bb6c2b 100644
@@ -153,7 +153,7 @@ struct kvm_vcpu_stat {
 
 struct kvm_vcpu_init;
 int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
-			const struct kvm_vcpu_init *init);
+			struct kvm_vcpu_init *init);
 unsigned long kvm_arm_num_regs(struct kvm_vcpu *vcpu);
 int kvm_arm_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices);
 struct kvm_one_reg;
arch/arm/include/uapi/asm/kvm.h +1 -0
index c1ee007..644012e 100644
@@ -63,6 +63,7 @@ struct kvm_regs {
 
 /* Supported Processor Types */
 #define KVM_ARM_TARGET_CORTEX_A15	0
+#define KVM_ARM_TARGET_HOST		0x0FFFFFFF
 #define KVM_ARM_NUM_TARGETS		1
 
 /* KVM_ARM_SET_DEVICE_ADDR ioctl id encoding */
arch/arm/kvm/arm.c +8 -1
index 741f66a..d8e494e 100644
@@ -698,6 +698,7 @@ int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_level,
 long kvm_arch_vcpu_ioctl(struct file *filp,
 			 unsigned int ioctl, unsigned long arg)
 {
+	int err;
 	struct kvm_vcpu *vcpu = filp->private_data;
 	void __user *argp = (void __user *)arg;
 
@@ -708,8 +709,14 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 		if (copy_from_user(&init, argp, sizeof(init)))
 			return -EFAULT;
 
-		return kvm_vcpu_set_target(vcpu, &init);
+		err = kvm_vcpu_set_target(vcpu, &init);
+		if (err)
+			return err;
+
+		if (copy_to_user(argp, &init, sizeof(init)))
+			return -EFAULT;
 
+		return 0;
 	}
 	case KVM_SET_ONE_REG:
 	case KVM_GET_ONE_REG: {
arch/arm/kvm/guest.c +9 -2
index 152d036..050df63 100644
@@ -198,12 +198,19 @@ int __attribute_const__ kvm_target_cpu(void)
 }
 
 int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
-			const struct kvm_vcpu_init *init)
+			struct kvm_vcpu_init *init)
 {
 	unsigned int i;
+	int phys_target = kvm_target_cpu();
+
+	if (phys_target < 0) {
+		return phys_target;
+	}
 
 	/* We can only do a cortex A15 for now. */
-	if (init->target != kvm_target_cpu())
+	if (init->target == KVM_ARM_TARGET_HOST)
+		init->target = phys_target;
+	else if (init->target != phys_target)
 		return -EINVAL;
 
 	vcpu->arch.target = init->target;
Signature · 1 line
1.7.9.5
#2 lore raw

RFC PATCH 1/3 ARM: KVM: Implement target CPU=Host

Quoted 86 lines · Anup Patel

This patch implements KVM_ARM_TARGET_HOST for KVM ARM.

If user space provides KVM_ARM_TARGET_HOST as target type in KVM_ARM_VCPU_INIT ioctl then we find out appropriate target type based on underlying host and return that to user space via struct kvm_vcpu_init.

---
 arch/arm/include/asm/kvm_host.h |    2 +-
 arch/arm/include/uapi/asm/kvm.h |    1 +
 arch/arm/kvm/arm.c              |    9 ++++++++-
 arch/arm/kvm/guest.c            |   11 +++++++++--
 4 files changed, 19 insertions(+), 4 deletions(-)
arch/arm/include/asm/kvm_host.h +1 -1
index 7d22517..3bb6c2b 100644
@@ -153,7 +153,7 @@ struct kvm_vcpu_stat {
 
 struct kvm_vcpu_init;
 int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
-			const struct kvm_vcpu_init *init);
+			struct kvm_vcpu_init *init);
 unsigned long kvm_arm_num_regs(struct kvm_vcpu *vcpu);
 int kvm_arm_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices);
 struct kvm_one_reg;
arch/arm/include/uapi/asm/kvm.h +1 -0
index c1ee007..644012e 100644
@@ -63,6 +63,7 @@ struct kvm_regs {
 
 /* Supported Processor Types */
 #define KVM_ARM_TARGET_CORTEX_A15	0
+#define KVM_ARM_TARGET_HOST		0x0FFFFFFF
 #define KVM_ARM_NUM_TARGETS		1
 
 /* KVM_ARM_SET_DEVICE_ADDR ioctl id encoding */
arch/arm/kvm/arm.c +8 -1
index 741f66a..d8e494e 100644
@@ -698,6 +698,7 @@ int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_level,
 long kvm_arch_vcpu_ioctl(struct file *filp,
 			 unsigned int ioctl, unsigned long arg)
 {
+	int err;
 	struct kvm_vcpu *vcpu = filp->private_data;
 	void __user *argp = (void __user *)arg;
 
@@ -708,8 +709,14 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 		if (copy_from_user(&init, argp, sizeof(init)))
 			return -EFAULT;
 
-		return kvm_vcpu_set_target(vcpu, &init);
+		err = kvm_vcpu_set_target(vcpu, &init);
+		if (err)
+			return err;
+
+		if (copy_to_user(argp, &init, sizeof(init)))
+			return -EFAULT;
 
+		return 0;
 	}
 	case KVM_SET_ONE_REG:
 	case KVM_GET_ONE_REG: {
arch/arm/kvm/guest.c +5 -1
index 152d036..050df63 100644
@@ -198,12 +198,19 @@ int __attribute_const__ kvm_target_cpu(void)
 }
 
 int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
-			const struct kvm_vcpu_init *init)
+			struct kvm_vcpu_init *init)
 {
 	unsigned int i;
+	int phys_target = kvm_target_cpu();
+
+	if (phys_target < 0) {
+		return phys_target;

arch/arm/kvm/guest.c int __attribute_const__ kvm_target_cpu(void) · line 216

Quoted 10 lines
arch/arm/kvm/guest.c +4 -1
+	}
 
 	/* We can only do a cortex A15 for now. */
-	if (init->target != kvm_target_cpu())
+	if (init->target == KVM_ARM_TARGET_HOST)
+		init->target = phys_target;
+	else if (init->target != phys_target)
 		return -EINVAL;
 
 	vcpu->arch.target = init->target;
author
#3 lore raw

RFC PATCH 2/3 ARM64: KVM: Implement target CPU=Host

In reply to Anup Patel

This patch implements KVM_ARM_TARGET_HOST for KVM ARM64.

If user space provides KVM_ARM_TARGET_HOST as target type in KVM_ARM_VCPU_INIT ioctl then we find out appropriate target type based on underlying host and return that to user space via struct kvm_vcpu_init.

---
 arch/arm64/include/asm/kvm_host.h |    2 +-
 arch/arm64/include/uapi/asm/kvm.h |    2 +-
 arch/arm64/kvm/guest.c            |   10 ++++++++--
 3 files changed, 10 insertions(+), 4 deletions(-)
arch/arm64/include/asm/kvm_host.h +1 -1
index 0859a4d..9ee431c 100644
@@ -150,7 +150,7 @@ struct kvm_vcpu_stat {
 
 struct kvm_vcpu_init;
 int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
-			const struct kvm_vcpu_init *init);
+			struct kvm_vcpu_init *init);
 unsigned long kvm_arm_num_regs(struct kvm_vcpu *vcpu);
 int kvm_arm_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices);
 struct kvm_one_reg;
arch/arm64/include/uapi/asm/kvm.h +1 -1
index d9f026b..eeb5726 100644
@@ -56,7 +56,7 @@ struct kvm_regs {
 #define KVM_ARM_TARGET_FOUNDATION_V8	1
 #define KVM_ARM_TARGET_CORTEX_A57	2
 #define KVM_ARM_TARGET_XGENE_POTENZA	3
-
+#define KVM_ARM_TARGET_HOST		0x0FFFFFFF
 #define KVM_ARM_NUM_TARGETS		4
 
 /* KVM_ARM_SET_DEVICE_ADDR ioctl id encoding */
arch/arm64/kvm/guest.c +8 -2
index d7bf7d6..261e836 100644
@@ -230,12 +230,18 @@ int __attribute_const__ kvm_target_cpu(void)
 }
 
 int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
-			const struct kvm_vcpu_init *init)
+			struct kvm_vcpu_init *init)
 {
 	unsigned int i;
 	int phys_target = kvm_target_cpu();
 
-	if (init->target != phys_target)
+	if (phys_target < 0) {
+		return phys_target;
+	}
+
+	if (init->target == KVM_ARM_TARGET_HOST)
+		init->target = phys_target;
+	else if (init->target != phys_target)
 		return -EINVAL;
 
 	vcpu->arch.target = phys_target;
Signature · 1 line
1.7.9.5
#4 lore raw

RFC PATCH 2/3 ARM64: KVM: Implement target CPU=Host

Quoted 57 lines · Anup Patel

This patch implements KVM_ARM_TARGET_HOST for KVM ARM64.

If user space provides KVM_ARM_TARGET_HOST as target type in KVM_ARM_VCPU_INIT ioctl then we find out appropriate target type based on underlying host and return that to user space via struct kvm_vcpu_init.

---
 arch/arm64/include/asm/kvm_host.h |    2 +-
 arch/arm64/include/uapi/asm/kvm.h |    2 +-
 arch/arm64/kvm/guest.c            |   10 ++++++++--
 3 files changed, 10 insertions(+), 4 deletions(-)
arch/arm64/include/asm/kvm_host.h +1 -1
index 0859a4d..9ee431c 100644
@@ -150,7 +150,7 @@ struct kvm_vcpu_stat {
 
 struct kvm_vcpu_init;
 int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
-			const struct kvm_vcpu_init *init);
+			struct kvm_vcpu_init *init);
 unsigned long kvm_arm_num_regs(struct kvm_vcpu *vcpu);
 int kvm_arm_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices);
 struct kvm_one_reg;
arch/arm64/include/uapi/asm/kvm.h +1 -1
index d9f026b..eeb5726 100644
@@ -56,7 +56,7 @@ struct kvm_regs {
 #define KVM_ARM_TARGET_FOUNDATION_V8	1
 #define KVM_ARM_TARGET_CORTEX_A57	2
 #define KVM_ARM_TARGET_XGENE_POTENZA	3
-
+#define KVM_ARM_TARGET_HOST		0x0FFFFFFF
 #define KVM_ARM_NUM_TARGETS		4
 
 /* KVM_ARM_SET_DEVICE_ADDR ioctl id encoding */
arch/arm64/kvm/guest.c +1 -2
index d7bf7d6..261e836 100644
@@ -230,12 +230,18 @@ int __attribute_const__ kvm_target_cpu(void)
 }
 
 int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
-			const struct kvm_vcpu_init *init)
+			struct kvm_vcpu_init *init)
 {
 	unsigned int i;
 	int phys_target = kvm_target_cpu();
 
-	if (init->target != phys_target)

arch/arm64/kvm/guest.c int __attribute_const__ kvm_target_cpu(void) · line 247

Quoted 10 lines
arch/arm64/kvm/guest.c +7 -0
+	if (phys_target < 0) {
+		return phys_target;
+	}
+
+	if (init->target == KVM_ARM_TARGET_HOST)
+		init->target = phys_target;
+	else if (init->target != phys_target)
 		return -EINVAL;
 
 	vcpu->arch.target = phys_target;
author
#5 lore raw

RFC PATCH 3/3 KVM: ARM: Update documentation for KVM_ARM_VCPU_INIT ioctl

In reply to Anup Patel

To implement target type KVM_ARM_TARGET_HOST we make the KVM_ARM_VCPU_INIT ioctl bi-direction so that KVM ARM/ARM64 can return appropriate target type to user space via struct kvm_vcpu_init.

---
 Documentation/virtual/kvm/api.txt |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
Documentation/virtual/kvm/api.txt +4 -1
index ef925ea..c0a3a05 100644
@@ -2283,7 +2283,7 @@ current state.  "addr" is ignored.
 Capability: basic
 Architectures: arm, arm64
 Type: vcpu ioctl
-Parameters: struct struct kvm_vcpu_init (in)
+Parameters: struct kvm_vcpu_init (in/out)
 Returns: 0 on success; -1 on error
 Errors:
  ?EINVAL: ???the target is unknown, or the combination of features is invalid.
@@ -2303,6 +2303,9 @@ Possible features:
 	- KVM_ARM_VCPU_EL1_32BIT: Starts the CPU in a 32bit mode.
 	  Depends on KVM_CAP_ARM_EL1_32BIT (arm64 only).
 
+This ioctl returns updated struct kvm_vcpu_init showing VCPU target 
+type and VCPU features that will be available.
+
 
 4.83 KVM_GET_REG_LIST
 
Signature · 1 line
1.7.9.5
#6 lore raw

RFC PATCH 3/3 KVM: ARM: Update documentation for KVM_ARM_VCPU_INIT ioctl

Hi Anup,

I am no native English speaker, but this is my attempt at improving the descriptions:

Quoted 4 lines · Anup Patel

To implement target type KVM_ARM_TARGET_HOST we make the KVM_ARM_VCPU_INIT ioctl bi-direction so that KVM ARM/ARM64 can return appropriate target type to user space via struct kvm_vcpu_init.

If I understood correctly:

To implement target type KVM_ARM_TARGET_HOST we change the "struct kvm_cpu_init" input parameter into an in/out parameter, so that KVM ARM/ARM64 can inform user space about the chosen target type.

Quoted 20 lines
---
 Documentation/virtual/kvm/api.txt |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
Documentation/virtual/kvm/api.txt +1 -1
index ef925ea..c0a3a05 100644
@@ -2283,7 +2283,7 @@ current state.  "addr" is ignored.
 Capability: basic
 Architectures: arm, arm64
 Type: vcpu ioctl
-Parameters: struct struct kvm_vcpu_init (in)
+Parameters: struct kvm_vcpu_init (in/out)
 Returns: 0 on success; -1 on error
 Errors:
   EINVAL:    the target is unknown, or the combination of features is invalid.

Documentation/virtual/kvm/api.txt Possible features: · line 2308

Quoted 6 lines
Documentation/virtual/kvm/api.txt +3 -0
@@ -2303,6 +2303,9 @@ Possible features:
 	- KVM_ARM_VCPU_EL1_32BIT: Starts the CPU in a 32bit mode.
 	  Depends on KVM_CAP_ARM_EL1_32BIT (arm64 only).
 
+This ioctl returns updated struct kvm_vcpu_init showing VCPU target 
+type and VCPU features that will be available.
+

This ioctl returns an updated struct kvm_vcpu_init informing about the chosen VCPU target type and available VCPU features.

Documentation/virtual/kvm/api.txt Possible features:

Quoted 4 lines
Documentation/virtual/kvm/api.txt +0 -0
 4.83 KVM_GET_REG_LIST
 
 

Ciao,

Claudio

#7 lore raw

RFC PATCH 3/3 KVM: ARM: Update documentation for KVM_ARM_VCPU_INIT ioctl

Quoted 10 lines · Claudio Fontana

Hi Anup,

I am no native English speaker, but this is my attempt at improving the descriptions:

Quoted 4 lines · Anup Patel

To implement target type KVM_ARM_TARGET_HOST we make the KVM_ARM_VCPU_INIT ioctl bi-direction so that KVM ARM/ARM64 can return appropriate target type to user space via struct kvm_vcpu_init.

Quoted 6 lines

If I understood correctly:

To implement target type KVM_ARM_TARGET_HOST we change the "struct kvm_cpu_init" input parameter into an in/out parameter, so that KVM ARM/ARM64 can inform user space about the chosen target type.

Yes, you got it right.

Actually, KVM will return back both VCPU target type and VCPU features that KVM will be emulating for the VCPU.

Quoted 38 lines
Quoted 21 lines
---
 Documentation/virtual/kvm/api.txt |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
Documentation/virtual/kvm/api.txt +1 -1
index ef925ea..c0a3a05 100644
@@ -2283,7 +2283,7 @@ current state.  "addr" is ignored.
 Capability: basic
 Architectures: arm, arm64
 Type: vcpu ioctl
-Parameters: struct struct kvm_vcpu_init (in)
+Parameters: struct kvm_vcpu_init (in/out)
 Returns: 0 on success; -1 on error
 Errors:
   EINVAL:    the target is unknown, or the combination of features is invalid.

Documentation/virtual/kvm/api.txt Possible features: · line 2308

Quoted 6 lines
Documentation/virtual/kvm/api.txt +3 -0
@@ -2303,6 +2303,9 @@ Possible features:
      - KVM_ARM_VCPU_EL1_32BIT: Starts the CPU in a 32bit mode.
        Depends on KVM_CAP_ARM_EL1_32BIT (arm64 only).
 
+This ioctl returns updated struct kvm_vcpu_init showing VCPU target
+type and VCPU features that will be available.
+

This ioctl returns an updated struct kvm_vcpu_init informing about the chosen VCPU target type and available VCPU features.

Documentation/virtual/kvm/api.txt Possible features:

Quoted 4 lines
Documentation/virtual/kvm/api.txt +0 -0
 4.83 KVM_GET_REG_LIST
 
 

Ciao,

Quoted 8 lines

Claudio

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Regards,
Anup

#8 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

In reply to Anup Patel
Quoted 4 lines · Anup Patel

It will be very useful for user space if it has a method for specifying KVM ARM/ARM64 to give a VCPU with target type suitable to underlying host but with particular set of features.

Quoted 8 lines

In other words, user space might not be interested in having a particular target type for VCPU but it will certainely want particular set of features in the VCPU.

The patch tries to implement above described method of specifying VCPU target CPU=Host from user space by extending the KVM_ARM_VCPU_INIT ioctl and having a dummy target KVM_ARM_TARGET_HOST which means Target CPU same as (or suitable to) underlying host.

I thought the consensus on the call last week was
to have an ioctl for "get best CPU for this host"
and then for userspace to pass that value to
VCPU_INIT ?

-- PMM

#9 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

Quoted 14 lines · Peter Maydell
Quoted 4 lines · Anup Patel

It will be very useful for user space if it has a method for specifying KVM ARM/ARM64 to give a VCPU with target type suitable to underlying host but with particular set of features.

Quoted 8 lines

In other words, user space might not be interested in having a particular target type for VCPU but it will certainely want particular set of features in the VCPU.

The patch tries to implement above described method of specifying VCPU target CPU=Host from user space by extending the KVM_ARM_VCPU_INIT ioctl and having a dummy target KVM_ARM_TARGET_HOST which means Target CPU same as (or suitable to) underlying host.

Quoted 4 lines

I thought the consensus on the call last week was
to have an ioctl for "get best CPU for this host"
and then for userspace to pass that value to
VCPU_INIT ?

I had considered having a separate ioctl for "get best CPU for this host" (as per KVM call minutes) but the same thing is possible with existing KVM_ARM_VCPU_INIT so why not extend this ioctl. Also, I was finding it hard to come up with a use case where having a separate ioctl for "get best CPU for this host" would be better for user space.

Further, the approach proposed by this patch also makes the KVM_ARM_VCPU_INIT ioctl inline with what we do for KVM x86 that is provide only a set of features user space wish and KVM x86 will try to provide most of those features with a virtual CPU suitable to host.

Another advantage I saw in extending KVM_ARM_VCPU_INIT ioctl is backward compatibility with current semantics. In other words, this patch does not break current KVMTOOL/QEMU and they can implement "-cpu host" whenever they wish without using any additional ioctl.

PFA, KVMTOOL patch which I used to test KVM_ARM_TARGET_HOST.

Quoted 6 lines

-- PMM
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/cucslists/listinfo/kvmarm

--Anup
-------------- next part --------------
A non-text attachment was scrubbed...
Name: kvmtool_cpu_host.patch
Type: application/octet-stream
Size: 1377 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130906/da5336f0/attachment.obj>

#10 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

Quoted 15 lines · Anup Patel
Quoted 14 lines · Peter Maydell
Quoted 12 lines · Anup Patel

It will be very useful for user space if it has a method for specifying KVM ARM/ARM64 to give a VCPU with target type suitable to underlying host but with particular set of features.

In other words, user space might not be interested in having a particular target type for VCPU but it will certainely want particular set of features in the VCPU.

The patch tries to implement above described method of specifying VCPU target CPU=Host from user space by extending the KVM_ARM_VCPU_INIT ioctl and having a dummy target KVM_ARM_TARGET_HOST which means Target CPU same as (or suitable to) underlying host.

Quoted 8 lines
Quoted 4 lines

I thought the consensus on the call last week was
to have an ioctl for "get best CPU for this host"
and then for userspace to pass that value to
VCPU_INIT ?

I had considered having a separate ioctl for "get best CPU for this host" (as per KVM call minutes) but the same thing is possible with existing KVM_ARM_VCPU_INIT so why not extend this ioctl. Also, I was finding

Because it means that everything as of the INIT call is reproducible by user space. There is no way the kernel can do magic behind our back to create a vcpu that user space couldn't create the same way on a different host when you want to live migrate.

Quoted 2 lines

it hard to come up with a use case where having a separate ioctl for "get best CPU for this host" would be better for user space.

It can store it and migrate it as part of its migration protocol (probably hard for QEMU's current work flow, but that's QEMU's issue).

Quoted 4 lines

Further, the approach proposed by this patch also makes the KVM_ARM_VCPU_INIT ioctl inline with what we do for KVM x86 that is provide only a set of features user space wish and KVM x86 will try to provide most of those features with a virtual CPU suitable to host.

I don't think you fully grasped how the x86 feature negotiation thing works :).

Quoted 4 lines

Another advantage I saw in extending KVM_ARM_VCPU_INIT ioctl is backward compatibility with current semantics. In other words, this patch does not break current KVMTOOL/QEMU and they can implement "-cpu host" whenever they wish without using any additional ioctl.

It's the opposite actually. By making the ioctl parameter in/out direction you change the ioctl number, breaking the ABI, no?

Alex

#11 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

Quoted 27 lines · Alexander Graf
Quoted 15 lines · Anup Patel
Quoted 14 lines · Peter Maydell
Quoted 12 lines · Anup Patel

It will be very useful for user space if it has a method for specifying KVM ARM/ARM64 to give a VCPU with target type suitable to underlying host but with particular set of features.

In other words, user space might not be interested in having a particular target type for VCPU but it will certainely want particular set of features in the VCPU.

The patch tries to implement above described method of specifying VCPU target CPU=Host from user space by extending the KVM_ARM_VCPU_INIT ioctl and having a dummy target KVM_ARM_TARGET_HOST which means Target CPU same as (or suitable to) underlying host.

Quoted 8 lines
Quoted 4 lines

I thought the consensus on the call last week was
to have an ioctl for "get best CPU for this host"
and then for userspace to pass that value to
VCPU_INIT ?

I had considered having a separate ioctl for "get best CPU for this host" (as per KVM call minutes) but the same thing is possible with existing KVM_ARM_VCPU_INIT so why not extend this ioctl. Also, I was finding

Quoted 6 lines

Because it means that everything as of the INIT call is reproducible by user space. There is no way the kernel can do magic behind our back to create a vcpu that user space couldn't create the same way on a different host when you want to live migrate.

Quoted 2 lines

it hard to come up with a use case where having a separate ioctl for "get best CPU for this host" would be better for user space.

It can store it and migrate it as part of its migration protocol (probably hard for QEMU's current work flow, but that's QEMU's issue).

It's also better from the point of view of devicetree generation. For example, kvmtool needs to generate the correct architected timer interrupt numbers for the target CPU, so munging this into KVM_ARM_VCPU_INIT means we have to go off and figure out the host CPU separately anyway.

Will

#12 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

Quoted 33 lines · Will Deacon
Quoted 32 lines · Alexander Graf
Quoted 15 lines · Anup Patel
Quoted 14 lines · Peter Maydell
Quoted 12 lines · Anup Patel

It will be very useful for user space if it has a method for specifying KVM ARM/ARM64 to give a VCPU with target type suitable to underlying host but with particular set of features.

In other words, user space might not be interested in having a particular target type for VCPU but it will certainely want particular set of features in the VCPU.

The patch tries to implement above described method of specifying VCPU target CPU=Host from user space by extending the KVM_ARM_VCPU_INIT ioctl and having a dummy target KVM_ARM_TARGET_HOST which means Target CPU same as (or suitable to) underlying host.

Quoted 8 lines
Quoted 4 lines

I thought the consensus on the call last week was
to have an ioctl for "get best CPU for this host"
and then for userspace to pass that value to
VCPU_INIT ?

I had considered having a separate ioctl for "get best CPU for this host" (as per KVM call minutes) but the same thing is possible with existing KVM_ARM_VCPU_INIT so why not extend this ioctl. Also, I was finding

Because it means that everything as of the INIT call is reproducible by user space. There is no way the kernel can do magic behind our back to create a vcpu that user space couldn't create the same way on a different host when you want to live migrate.

Quoted 2 lines

it hard to come up with a use case where having a separate ioctl for "get best CPU for this host" would be better for user space.

Quoted 6 lines
Quoted 1 line

It can store it and migrate it as part of its migration protocol (probably hard for QEMU's current work flow, but that's QEMU's issue).

It's also better from the point of view of devicetree generation. For example, kvmtool needs to generate the correct architected timer interrupt numbers for the target CPU, so munging this into KVM_ARM_VCPU_INIT means we have to go off and figure out the host CPU separately anyway.

Please look at the patch carefully we are returning VCPU target type back to user space so, you can generate correct architected timer interrupt numbers for the target CPU.

In fact, you will be able to generate complete DTB for the Guest based on the VCPU target type returned by KVM.

For your reference also look at the KVMTOOL patch that I have attached in my previous reply.

Quoted 2 lines

Will

--Anup

#13 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

Quoted 8 lines · Anup Patel
Quoted 7 lines · Will Deacon
Quoted 5 lines · Alexander Graf
Quoted 2 lines · Anup Patel

it hard to come up with a use case where having a separate ioctl for "get best CPU for this host" would be better for user space.

It can store it and migrate it as part of its migration protocol (probably hard for QEMU's current work flow, but that's QEMU's issue).

Quoted 8 lines
Quoted 4 lines

It's also better from the point of view of devicetree generation. For example, kvmtool needs to generate the correct architected timer interrupt numbers for the target CPU, so munging this into KVM_ARM_VCPU_INIT means we have to go off and figure out the host CPU separately anyway.

Please look at the patch carefully we are returning VCPU target type back to user space so, you can generate correct architected timer interrupt numbers for the target CPU.

I did look at the patch, but I also looked at the overriding consensus in the feedback saying that you can't change the direction of the ioctl, so that would require what I said above.

Will

#14 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

Quoted 18 lines · Will Deacon
Quoted 8 lines · Anup Patel
Quoted 7 lines · Will Deacon
Quoted 5 lines · Alexander Graf
Quoted 2 lines · Anup Patel

it hard to come up with a use case where having a separate ioctl for "get best CPU for this host" would be better for user space.

It can store it and migrate it as part of its migration protocol (probably hard for QEMU's current work flow, but that's QEMU's issue).

Quoted 8 lines
Quoted 4 lines

It's also better from the point of view of devicetree generation. For example, kvmtool needs to generate the correct architected timer interrupt numbers for the target CPU, so munging this into KVM_ARM_VCPU_INIT means we have to go off and figure out the host CPU separately anyway.

Please look at the patch carefully we are returning VCPU target type back to user space so, you can generate correct architected timer interrupt numbers for the target CPU.

Quoted 5 lines

I did look at the patch, but I also looked at the overriding consensus in the feedback saying that you can't change the direction of the ioctl, so that would require what I said above.

Will

Instead of arguing more, I'll save everyone's time and revise this patch as needed by everyone.

--Anup

#15 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

Quoted 19 lines · Anup Patel
Quoted 18 lines · Will Deacon
Quoted 16 lines · Anup Patel
Quoted 11 lines · Will Deacon
Quoted 5 lines · Alexander Graf
Quoted 2 lines · Anup Patel

it hard to come up with a use case where having a separate ioctl for "get best CPU for this host" would be better for user space.

It can store it and migrate it as part of its migration protocol (probably hard for QEMU's current work flow, but that's QEMU's issue).

It's also better from the point of view of devicetree generation. For example, kvmtool needs to generate the correct architected timer interrupt numbers for the target CPU, so munging this into KVM_ARM_VCPU_INIT means we have to go off and figure out the host CPU separately anyway.

Please look at the patch carefully we are returning VCPU target type back to user space so, you can generate correct architected timer interrupt numbers for the target CPU.

Quoted 8 lines
Quoted 5 lines

I did look at the patch, but I also looked at the overriding consensus in the feedback saying that you can't change the direction of the ioctl, so that would require what I said above.

Will

Instead of arguing more, I'll save everyone's time and revise this patch as needed by everyone.

Cheers :)

Will

#16 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

In reply to Alexander Graf
Quoted 23 lines · Alexander Graf
Quoted 20 lines · Anup Patel
Quoted 18 lines · Peter Maydell
Quoted 4 lines · Anup Patel

It will be very useful for user space if it has a method for specifying KVM ARM/ARM64 to give a VCPU with target type suitable to underlying host but with particular set of features.

Quoted 8 lines

In other words, user space might not be interested in having a particular target type for VCPU but it will certainely want particular set of features in the VCPU.

The patch tries to implement above described method of specifying VCPU target CPU=Host from user space by extending the KVM_ARM_VCPU_INIT ioctl and having a dummy target KVM_ARM_TARGET_HOST which means Target CPU same as (or suitable to) underlying host.

I thought the consensus on the call last week was
to have an ioctl for "get best CPU for this host"
and then for userspace to pass that value to
VCPU_INIT ?

Quoted 5 lines
Quoted 3 lines

I had considered having a separate ioctl for "get best CPU for this host" (as per KVM call minutes) but the same thing is possible with existing KVM_ARM_VCPU_INIT so why not extend this ioctl. Also, I was finding

Because it means that everything as of the INIT call is reproducible by user space. There is no way the kernel can do magic behind our back to create a vcpu that user space couldn't create the same way on a different host when you want to live migrate.

Yes, whatever we do in INIT call is still reproducible.

Also, the final VCPU target type and VCPU features are returned back to the user space.

Quoted 5 lines
Quoted 3 lines

it hard to come up with a use case where having a separate ioctl for "get best CPU for this host" would be better for user space.

It can store it and migrate it as part of its migration protocol (probably hard for QEMU's current work flow, but that's QEMU's issue).

Please look at the patch carefully.

The patch makes KVM_ARM_VCPU_INIT ioctl bi-directional this means we return the target of VCPU and features of VCPU back to user space.

Quoted 7 lines
Quoted 5 lines

Further, the approach proposed by this patch also makes the KVM_ARM_VCPU_INIT ioctl inline with what we do for KVM x86 that is provide only a set of features user space wish and KVM x86 will try to provide most of those features with a virtual CPU suitable to host.

I don't think you fully grasped how the x86 feature negotiation thing works :).

The feature negotiation implemented by this patch is as follows:
1. user space proposes a VCPU target type (or VCPU same as host) and VCPU features
2. KVM does the checking and determines appropriate VCPU target type and VCPU features
3. KVM returns VCPU target type and VCPU features back to user space

Now I am not x86 expert but the above scheme of feature negotiation is generic enough.

Quoted 7 lines
Quoted 5 lines

Another advantage I saw in extending KVM_ARM_VCPU_INIT ioctl is backward compatibility with current semantics. In other words, this patch does not break current KVMTOOL/QEMU and they can implement "-cpu host" whenever they wish without using any additional ioctl.

It's the opposite actually. By making the ioctl parameter in/out direction you change the ioctl number, breaking the ABI, no?

Originally the ioctl was only "in" and so we are preserving the "in" semantics. Thats why it is semantically backward compatible.

I have tested this patch with unmodified QEMU and KVMTOOL.

Quoted 4 lines

Alex

--Anup

#17 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

Quoted 28 lines · Anup Patel
Quoted 27 lines · Alexander Graf
Quoted 23 lines · Anup Patel
Quoted 14 lines · Peter Maydell
Quoted 4 lines · Anup Patel

It will be very useful for user space if it has a method for specifying KVM ARM/ARM64 to give a VCPU with target type suitable to underlying host but with particular set of features.

Quoted 8 lines

In other words, user space might not be interested in having a particular target type for VCPU but it will certainely want particular set of features in the VCPU.

The patch tries to implement above described method of specifying VCPU target CPU=Host from user space by extending the KVM_ARM_VCPU_INIT ioctl and having a dummy target KVM_ARM_TARGET_HOST which means Target CPU same as (or suitable to) underlying host.

Quoted 4 lines

I thought the consensus on the call last week was
to have an ioctl for "get best CPU for this host"
and then for userspace to pass that value to
VCPU_INIT ?

I had considered having a separate ioctl for "get best CPU for this host" (as per KVM call minutes) but the same thing is possible with existing KVM_ARM_VCPU_INIT so why not extend this ioctl. Also, I was finding

Quoted 6 lines
Quoted 1 line

Because it means that everything as of the INIT call is reproducible by user space. There is no way the kernel can do magic behind our back to create a vcpu that user space couldn't create the same way on a different host when you want to live migrate.

Yes, whatever we do in INIT call is still reproducible.

Also, the final VCPU target type and VCPU features are returned back to the user space.

But you're combining two semantically separate things into a single operation. So now there is no way for the machine model to ask KVM what CPU it prefers without creating a CPU. So you have to first create a CPU, then somehow route the type information back into the machine model so that the machine model can create the appropriate timer.

This is messy without any good reason.

Quoted 27 lines
Quoted 6 lines
Quoted 4 lines

it hard to come up with a use case where having a separate ioctl for "get best CPU for this host" would be better for user space.

It can store it and migrate it as part of its migration protocol (probably hard for QEMU's current work flow, but that's QEMU's issue).

Please look at the patch carefully.

The patch makes KVM_ARM_VCPU_INIT ioctl bi-directional this means we return the target of VCPU and features of VCPU back to user space.

Quoted 7 lines
Quoted 5 lines

Further, the approach proposed by this patch also makes the KVM_ARM_VCPU_INIT ioctl inline with what we do for KVM x86 that is provide only a set of features user space wish and KVM x86 will try to provide most of those features with a virtual CPU suitable to host.

I don't think you fully grasped how the x86 feature negotiation thing works :).

The feature negotiation implemented by this patch is as follows:
1. user space proposes a VCPU target type (or VCPU same as host) and VCPU features
2. KVM does the checking and determines appropriate VCPU target type and VCPU features
3. KVM returns VCPU target type and VCPU features back to user space

Quoted 2 lines

Now I am not x86 expert but the above scheme of feature negotiation is generic enough.

It's orders of magnitude less flexible than what x86 allows you to do. I'm not criticizing the intent here, but I'm saying that it's not even remotely close to anything x86 does. So referring to it here is pointless :).

Quoted 11 lines
Quoted 8 lines
Quoted 6 lines

Another advantage I saw in extending KVM_ARM_VCPU_INIT ioctl is backward compatibility with current semantics. In other words, this patch does not break current KVMTOOL/QEMU and they can implement "-cpu host" whenever they wish without using any additional ioctl.

It's the opposite actually. By making the ioctl parameter in/out direction you change the ioctl number, breaking the ABI, no?

Originally the ioctl was only "in" and so we are preserving the "in" semantics. Thats why it is semantically backward compatible.

Great. So now we have an ioctl that says it's "in" in its ioctl descriptor, but really it's in/out. This really only works by accident because nobody is filtering the direction today.

Nack.

Alex

#18 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

Quoted 6 lines · Alexander Graf
Quoted 4 lines · Anup Patel
Quoted 2 lines · Alexander Graf

On 06.09.2013, at 09:44, Anup Patel wrote:

[...]

Quoted 12 lines
Quoted 11 lines
Quoted 8 lines
Quoted 5 lines

Another advantage I saw in extending KVM_ARM_VCPU_INIT ioctl is backward compatibility with current semantics. In other words, this patch
does not break current KVMTOOL/QEMU and they can implement "-cpu host" whenever they wish without using any additional ioctl.

It's the opposite actually. By making the ioctl parameter in/out direction you change the ioctl number, breaking the ABI, no?

Originally the ioctl was only "in" and so we are preserving the "in" semantics. Thats why it is semantically backward compatible.

Quoted 6 lines

Great. So now we have an ioctl that says it's "in" in its ioctl descriptor, but really it's in/out. This really only works by accident
because nobody is filtering the direction today.

Nack.

Agreed. We don't break the ABI, we don't try to fool the kernel. Please.

There's been previous suggestions on how to implement this feature, please consider them.

         M.
Signature · 1 line
Fast, cheap, reliable. Pick two.
#19 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

Quoted 28 lines · Marc Zyngier
Quoted 7 lines · Alexander Graf
Quoted 4 lines · Anup Patel
Quoted 3 lines · Alexander Graf

On 06.09.2013, at 09:44, Anup Patel wrote:

[...]

Quoted 15 lines
Quoted 13 lines
Quoted 9 lines
Quoted 5 lines

Another advantage I saw in extending KVM_ARM_VCPU_INIT ioctl is backward compatibility with current semantics. In other words, this patch
does not break current KVMTOOL/QEMU and they can implement "-cpu host" whenever they wish without using any additional ioctl.

It's the opposite actually. By making the ioctl parameter in/out direction you change the ioctl number, breaking the ABI, no?

Originally the ioctl was only "in" and so we are preserving the "in" semantics. Thats why it is semantically backward compatible.

Quoted 8 lines
Quoted 5 lines

Great. So now we have an ioctl that says it's "in" in its ioctl descriptor, but really it's in/out. This really only works by accident because nobody is filtering the direction today.

Nack.

Agreed. We don't break the ABI, we don't try to fool the kernel. Please.

We are not breaking the ABI here and also not trying to fool the kernel.

Quoted 3 lines

There's been previous suggestions on how to implement this feature, please consider them.

I am not convinced about how is this approach not better.

Quoted 4 lines
        M.

--
Fast, cheap, reliable. Pick two.

--Anup

#20 lore raw

cover Target CPU=Host implementation for KVM ARM/ARM64

In reply to Alexander Graf
Quoted 33 lines · Alexander Graf
Quoted 30 lines · Anup Patel
Quoted 28 lines · Alexander Graf
Quoted 15 lines · Anup Patel
Quoted 14 lines · Peter Maydell
Quoted 12 lines · Anup Patel

It will be very useful for user space if it has a method for specifying KVM ARM/ARM64 to give a VCPU with target type suitable to underlying host but with particular set of features.

In other words, user space might not be interested in having a particular target type for VCPU but it will certainely want particular set of features in the VCPU.

The patch tries to implement above described method of specifying VCPU target CPU=Host from user space by extending the KVM_ARM_VCPU_INIT ioctl and having a dummy target KVM_ARM_TARGET_HOST which means Target CPU same as (or suitable to) underlying host.

Quoted 8 lines
Quoted 4 lines

I thought the consensus on the call last week was
to have an ioctl for "get best CPU for this host"
and then for userspace to pass that value to
VCPU_INIT ?

I had considered having a separate ioctl for "get best CPU for this host" (as per KVM call minutes) but the same thing is possible with existing KVM_ARM_VCPU_INIT so why not extend this ioctl. Also, I was finding

Because it means that everything as of the INIT call is reproducible by user space. There is no way the kernel can do magic behind our back to create a vcpu that user space couldn't create the same way on a different host when you want to live migrate.

Quoted 8 lines
Quoted 4 lines

Yes, whatever we do in INIT call is still reproducible.

Also, the final VCPU target type and VCPU features are returned back to the user space.

But you're combining two semantically separate things into a single operation. So now there is no way for the machine model to ask KVM what CPU it prefers without creating a CPU. So you have to first create a CPU, then somehow route the type information back into the machine model so that the machine model can create the appropriate timer.

This is messy without any good reason.

Machine model resembling real world machines will always ask for fixed VCPU target type whereas generic machines will ask for VCPU target type same as host.

For eg. VExpress-A15 machine model will ask for KVM_ARM_TARGET_CORTEX-A15 whereas mach-virt machine model will ask for KVM_ARM_TARGET_HOST.

I don't see how this will be messy for QEMU.

Quoted 40 lines
Quoted 28 lines
Quoted 7 lines
Quoted 5 lines

it hard to come up with a use case where having a separate ioctl for "get best CPU for this host" would be better for user space.

It can store it and migrate it as part of its migration protocol (probably hard for QEMU's current work flow, but that's QEMU's issue).

Please look at the patch carefully.

The patch makes KVM_ARM_VCPU_INIT ioctl bi-directional this means we return the target of VCPU and features of VCPU back to user space.

Quoted 7 lines
Quoted 5 lines

Further, the approach proposed by this patch also makes the KVM_ARM_VCPU_INIT ioctl inline with what we do for KVM x86 that is provide only a set of features user space wish and KVM x86 will try to provide most of those features with a virtual CPU suitable to host.

I don't think you fully grasped how the x86 feature negotiation thing works :).

The feature negotiation implemented by this patch is as follows:
1. user space proposes a VCPU target type (or VCPU same as host) and VCPU features
2. KVM does the checking and determines appropriate VCPU target type and VCPU features
3. KVM returns VCPU target type and VCPU features back to user space

Quoted 2 lines

Now I am not x86 expert but the above scheme of feature negotiation is generic enough.

It's orders of magnitude less flexible than what x86 allows you to do. I'm not criticizing the intent here, but I'm saying that it's not even remotely close to anything x86 does. So referring to it here is pointless :).

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

Another advantage I saw in extending KVM_ARM_VCPU_INIT ioctl is backward compatibility with current semantics. In other words, this patch does not break current KVMTOOL/QEMU and they can implement "-cpu host" whenever they wish without using any additional ioctl.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

It's the opposite actually. By making the ioctl parameter in/out direction you change the ioctl number, breaking the ABI, no?

Originally the ioctl was only "in" and so we are preserving the "in" semantics. Thats why it is semantically backward compatible.

Great. So now we have an ioctl that says it's "in" in its ioctl descriptor, but really it's in/out. This really only works by accident because nobody is filtering the direction today.

The patch series also updates documentations.

Quoted 6 lines

Nack.

Alex

--Anup