KVM: arm64: pvtime: Fixes and a new cap

Don't confuse the guest by saying steal-time is supported when it hasn't been configured by userspace and won't work.

Signed-off-by Andrew Jones <[email protected]>

---
 arch/arm64/kvm/pvtime.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

arch/arm64/kvm/pvtime.c +4 -1

index f7b52ce1557e..2b22214909be 100644
@@ -42,9 +42,12 @@ long kvm_hypercall_pv_features(struct kvm_vcpu 
 *vcpu)
 
 	switch (feature) {
 	case ARM_SMCCC_HV_PV_TIME_FEATURES:
-	case ARM_SMCCC_HV_PV_TIME_ST:
 		val = SMCCC_RET_SUCCESS;
 		break;
+	case ARM_SMCCC_HV_PV_TIME_ST:
+		if (vcpu->arch.steal.base != GPA_INVALID)
+			val = SMCCC_RET_SUCCESS;
+		break;
 	}
 

return val;

Hi Andrew,

Quoted 16 lines · Andrew Jones

Don't confuse the guest by saying steal-time is supported when it hasn't been configured by userspace and won't work.

Signed-off-by Andrew Jones <[email protected]>

---
 arch/arm64/kvm/pvtime.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

arch/arm64/kvm/pvtime.c +0 -0

index f7b52ce1557e..2b22214909be 100644
@@ -42,9 +42,12 @@ long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu)
 
 	switch (feature) {
 	case ARM_SMCCC_HV_PV_TIME_FEATURES:

arch/arm64/kvm/pvtime.c long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu) · line 53

Quoted 10 lines

arch/arm64/kvm/pvtime.c +4 -1

-	case ARM_SMCCC_HV_PV_TIME_ST:
 		val = SMCCC_RET_SUCCESS;
 		break;
+	case ARM_SMCCC_HV_PV_TIME_ST:
+		if (vcpu->arch.steal.base != GPA_INVALID)
+			val = SMCCC_RET_SUCCESS;
+		break;
 	}
 
 	return val;

I'm not so sure about this. I have always considered the discovery interface to be "do you know about this SMCCC function". And if you look at the spec, it says (4.2, PV_TIME_FEATURES):

<quote>
If PV_call_id identifies PV_TIME_FEATURES, this call returns:
• NOT_SUPPORTED (-1) to indicate that all paravirtualized time functions in this specification are not supported.
• SUCCESS (0) to indicate that all the paravirtualized time functions in this specification are supported. </quote>

So the way I understand it, you cannot return "supported" for PV_TIME_FEATURES, and yet return NOT_SUPPORTED for PV_TIME_ST. It applies to *all* features.

Yes, this is very bizarre. But I don't think we can deviate from it.

Quoted 44 lines · Marc Zyngier

Hi Andrew,

Quoted 16 lines · Andrew Jones

Don't confuse the guest by saying steal-time is supported when it hasn't been configured by userspace and won't work.

Signed-off-by Andrew Jones <[email protected]>

---
 arch/arm64/kvm/pvtime.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

arch/arm64/kvm/pvtime.c +0 -0

index f7b52ce1557e..2b22214909be 100644
@@ -42,9 +42,12 @@ long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu)
 
 	switch (feature) {
 	case ARM_SMCCC_HV_PV_TIME_FEATURES:

arch/arm64/kvm/pvtime.c long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu) · line 53

Quoted 10 lines

arch/arm64/kvm/pvtime.c +4 -1

-	case ARM_SMCCC_HV_PV_TIME_ST:
 		val = SMCCC_RET_SUCCESS;
 		break;
+	case ARM_SMCCC_HV_PV_TIME_ST:
+		if (vcpu->arch.steal.base != GPA_INVALID)
+			val = SMCCC_RET_SUCCESS;
+		break;
 	}
 
 	return val;

I'm not so sure about this. I have always considered the discovery interface to be "do you know about this SMCCC function". And if you look at the spec, it says (4.2, PV_TIME_FEATURES):

<quote>
If PV_call_id identifies PV_TIME_FEATURES, this call returns:
• NOT_SUPPORTED (-1) to indicate that all paravirtualized time functions in this specification are not supported.
• SUCCESS (0) to indicate that all the paravirtualized time functions in this specification are supported. </quote>

Quoted 6 lines

So the way I understand it, you cannot return "supported" for PV_TIME_FEATURES, and yet return NOT_SUPPORTED for PV_TIME_ST. It applies to *all* features.

Yes, this is very bizarre. But I don't think we can deviate from it.

Ah, I see your point. But I wonder if we should drop this patch or if we should change the return of ARM_SMCCC_HV_PV_TIME_FEATURES to be dependant on all the pv calls?

Discovery would look like this

IF (SMCCC_ARCH_FEATURES, PV_TIME_FEATURES) == 0; THEN
IF (PV_TIME_FEATURES, PV_TIME_FEATURES) == 0; THEN
PV_TIME_ST is supported, as well as all other PV calls
ELIF (PV_TIME_FEATURES, PV_TIME_ST) == 0; THEN
PV_TIME_ST is supported
ELIF (PV_TIME_FEATURES, <another-pv-call>) == 0; THEN
<another-pv-call> is supported
...
ENDIF
ELSE
No PV calls are supported
ENDIF

I believe the above implements a reasonable interpretation of the specification, but the all feature (PV_TIME_FEATURES, PV_TIME_FEATURES) thing is indeed bizarre no matter how you look at it.

Quoted 71 lines · Andrew Jones

Quoted 44 lines · Marc Zyngier

Hi Andrew,

Quoted 16 lines · Andrew Jones

Don't confuse the guest by saying steal-time is supported when it hasn't been configured by userspace and won't work.

Signed-off-by Andrew Jones <[email protected]>

---
 arch/arm64/kvm/pvtime.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

arch/arm64/kvm/pvtime.c +0 -0

index f7b52ce1557e..2b22214909be 100644
@@ -42,9 +42,12 @@ long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu)
 
 	switch (feature) {
 	case ARM_SMCCC_HV_PV_TIME_FEATURES:

arch/arm64/kvm/pvtime.c long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu) · line 53

Quoted 10 lines

arch/arm64/kvm/pvtime.c +4 -1

-	case ARM_SMCCC_HV_PV_TIME_ST:
 		val = SMCCC_RET_SUCCESS;
 		break;
+	case ARM_SMCCC_HV_PV_TIME_ST:
+		if (vcpu->arch.steal.base != GPA_INVALID)
+			val = SMCCC_RET_SUCCESS;
+		break;
 	}
 
 	return val;

I'm not so sure about this. I have always considered the discovery interface to be "do you know about this SMCCC function". And if you look at the spec, it says (4.2, PV_TIME_FEATURES):

<quote>
If PV_call_id identifies PV_TIME_FEATURES, this call returns:
• NOT_SUPPORTED (-1) to indicate that all paravirtualized time functions in this specification are not supported.
• SUCCESS (0) to indicate that all the paravirtualized time functions in this specification are supported. </quote>

Quoted 6 lines

So the way I understand it, you cannot return "supported" for PV_TIME_FEATURES, and yet return NOT_SUPPORTED for PV_TIME_ST. It applies to *all* features.

Yes, this is very bizarre. But I don't think we can deviate from it.

Ah, I see your point. But I wonder if we should drop this patch or if we should change the return of ARM_SMCCC_HV_PV_TIME_FEATURES to be dependant on all the pv calls?

Discovery would look like this

IF (SMCCC_ARCH_FEATURES, PV_TIME_FEATURES) == 0; THEN
IF (PV_TIME_FEATURES, PV_TIME_FEATURES) == 0; THEN
PV_TIME_ST is supported, as well as all other PV calls
ELIF (PV_TIME_FEATURES, PV_TIME_ST) == 0; THEN
PV_TIME_ST is supported
ELIF (PV_TIME_FEATURES, <another-pv-call>) == 0; THEN
<another-pv-call> is supported
...
ENDIF
ELSE
No PV calls are supported
ENDIF

Quoted 3 lines

I believe the above implements a reasonable interpretation of the specification, but the all feature (PV_TIME_FEATURES, PV_TIME_FEATURES) thing is indeed bizarre no matter how you look at it.

It it indeed true to the spec. Thankfully we only support PV_TIME as a feature for now, so we are (sort of) immune to the braindead aspect of the discovery protocol.

I think returning a failure when PV_TIME isn't setup is a valid thing to do, as long as it applies to all functions (i.e. something like the below patch).

Thanks,

        M.

arch/arm64/kvm/pvtime.c +0 -0

index f7b52ce1557e..c3ef4ebd6846 100644

We should only check current->sched_info.run_delay once when updating stolen time. Otherwise there's a chance there could be a change between checks that we miss (preemption disabling comes after vcpu request checks).

Signed-off-by Andrew Jones <[email protected]>

---
 arch/arm64/kvm/pvtime.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

arch/arm64/kvm/pvtime.c +1 -0

index 2b22214909be..db5ef097a166 100644
@@ -13,6 +13,7 @@
 void kvm_update_stolen_time(struct kvm_vcpu *vcpu)
 {
 	struct kvm *kvm = vcpu->kvm;
+	u64 last_steal = vcpu->arch.steal.last_steal;
 	u64 steal;
 	__le64 steal_le;
 	u64 offset;

When updating the stolen time we should always read the current stolen time from the user provided memory, not from a kernel cache. If we use a cache then we'll end up resetting stolen time to zero on the first update after migration.

Signed-off-by Andrew Jones <[email protected]>

---
 arch/arm64/include/asm/kvm_host.h |  1 -
 arch/arm64/kvm/pvtime.c           | 23 +++++++++--------------
 include/linux/kvm_host.h          | 19 +++++++++++++++++++
 3 files changed, 28 insertions(+), 15 deletions(-)

arch/arm64/include/asm/kvm_host.h +0 -1

index c3e6fcc664b1..b01f52b61572 100644
@@ -343,7 +343,6 @@ struct kvm_vcpu_arch {
 
 	/* Guest PV state */
 	struct {
-		u64 steal;
 		u64 last_steal;
 		gpa_t base;
 	} steal;

arch/arm64/kvm/pvtime.c +9 -14

index db5ef097a166..025b5f3a97ef 100644
@@ -13,26 +13,22 @@
 void kvm_update_stolen_time(struct kvm_vcpu *vcpu)
 {
 	struct kvm *kvm = vcpu->kvm;
+	u64 base = vcpu->arch.steal.base;
 	u64 last_steal = vcpu->arch.steal.last_steal;
-	u64 steal;
-	__le64 steal_le;
-	u64 offset;
+	u64 offset = offsetof(struct pvclock_vcpu_stolen_time, stolen_time);
+	u64 steal = 0;
 	int idx;
-	u64 base = vcpu->arch.steal.base;
 
 	if (base == GPA_INVALID)
 		return;
 
-	/* Let's do the local bookkeeping */
-	steal = vcpu->arch.steal.steal;
-	vcpu->arch.steal.last_steal = current->sched_info.run_delay;
-	steal += vcpu->arch.steal.last_steal - last_steal;
-	vcpu->arch.steal.steal = steal;
-
-	steal_le = cpu_to_le64(steal);
 	idx = srcu_read_lock(&kvm->srcu);
-	offset = offsetof(struct pvclock_vcpu_stolen_time, stolen_time);
-	kvm_put_guest(kvm, base + offset, steal_le, u64);
+	if (!kvm_get_guest(kvm, base + offset, steal, u64)) {
+		steal = le64_to_cpu(steal);
+		vcpu->arch.steal.last_steal = current->sched_info.run_delay;
+		steal += vcpu->arch.steal.last_steal - last_steal;
+		kvm_put_guest(kvm, base + offset, cpu_to_le64(steal), u64);
+	}
 	srcu_read_unlock(&kvm->srcu, idx);
 }
 

@@ -68,7 +64,6 @@ gpa_t kvm_init_stolen_time(struct kvm_vcpu *vcpu)
 	 * Start counting stolen time from the time the guest requests
 	 * the feature enabled.
 	 */
-	vcpu->arch.steal.steal = 0;
 	vcpu->arch.steal.last_steal = current->sched_info.run_delay;
 
 	idx = srcu_read_lock(&kvm->srcu);

include/linux/kvm_host.h +0 -0

index d564855243d8..e2fc655f0b5b 100644

include/linux/kvm_host.h +15 -0

+	int __ret = -EFAULT;						\
+									\
+	if (!kvm_is_error_hva(__addr))					\
+		__ret = get_user(x, __uaddr);				\
+	__ret;								\
+})
+
+#define kvm_get_guest(kvm, gpa, x, type)				\
+({									\
+	gpa_t __gpa = gpa;						\
+	struct kvm *__kvm = kvm;					\
+	__kvm_get_guest(__kvm, __gpa >> PAGE_SHIFT,			\
+			offset_in_page(__gpa), x, type);		\
+})
+
 #define __kvm_put_guest(kvm, gfn, offset, value, type)			\

({ \
unsigned long __addr = gfn_to_hva(kvm, gfn); \

Reviewed-by Steven Price <[email protected]>

Signed-off-by Andrew Jones <[email protected]>

---
 Documentation/virt/kvm/api.rst | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

Documentation/virt/kvm/api.rst +0 -0

index 320788f81a05..3bd96c1a3962 100644
@@ -6122,7 +6122,7 @@ HvCallSendSyntheticClusterIpi,

HvCallSendSyntheticClusterIpiEx.
8.21 KVM_CAP_HYPERV_DIRECT_TLBFLUSH
-----------------------------------

-:Architecture: x86
+:Architectures: x86

This capability indicates that KVM running on top of Hyper-V
hypervisor
enables Direct TLB flush for its guests meaning that TLB flush

Documentation/virt/kvm/api.rst +0 -0

@@ -6135,16 +6135,17 @@ in CPUID and only exposes Hyper-V

identification. In this case, guest
thinks it's running on Hyper-V and only use Hyper-V hypercalls.

8.22 KVM_CAP_S390_VCPU_RESETS
+-----------------------------

Documentation/virt/kvm/api.rst +1 -1

-Architectures: s390
+:Architectures: s390
 
 This capability indicates that the KVM_S390_NORMAL_RESET and
 KVM_S390_CLEAR_RESET ioctls are available.

arm64 requires a vcpu fd (KVM_HAS_DEVICE_ATTR vcpu ioctl) to probe support for steal-time. However this is unnecessary, as only a KVM fd is required, and it complicates userspace (userspace may prefer delaying vcpu creation until after feature probing). Introduce a cap that can be checked instead. While x86 can already probe steal-time support with a kvm fd (KVM_GET_SUPPORTED_CPUID), we add the cap there too for consistency.

Reviewed-by Steven Price <[email protected]>

Signed-off-by Andrew Jones <[email protected]>

---
 Documentation/virt/kvm/api.rst    | 11 +++++++++++
 arch/arm64/include/asm/kvm_host.h |  1 +
 arch/arm64/kvm/arm.c              |  3 +++
 arch/arm64/kvm/pvtime.c           |  2 +-
 arch/x86/kvm/x86.c                |  3 +++
 include/uapi/linux/kvm.h          |  1 +
 6 files changed, 20 insertions(+), 1 deletion(-)

Documentation/virt/kvm/api.rst +1 -0

index 3bd96c1a3962..3716d1e29771 100644
@@ -6152,3 +6152,14 @@ KVM can therefore start protected VMs.
 This capability governs the KVM_S390_PV_COMMAND ioctl and the
 KVM_MP_STATE_LOAD MP_STATE. KVM_SET_MP_STATE can fail for protected
 guests when the state change is invalid.
+

arch/arm64/include/asm/kvm_host.h +1 -0

index b01f52b61572..8909c840ea91 100644
@@ -500,6 +500,7 @@ long kvm_hypercall_pv_features(struct kvm_vcpu 
 *vcpu);
 gpa_t kvm_init_stolen_time(struct kvm_vcpu *vcpu);
 void kvm_update_stolen_time(struct kvm_vcpu *vcpu);
 
+bool kvm_arm_pvtime_supported(void);
 int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
 			    struct kvm_device_attr *attr);

int kvm_arm_pvtime_get_attr(struct kvm_vcpu *vcpu,

arch/arm64/kvm/arm.c +3 -0

index 90cb90561446..0f7f8cd2f341 100644
@@ -222,6 +222,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, 
 long ext)
 		 */
 		r = 1;
 		break;
+	case KVM_CAP_STEAL_TIME:
+		r = kvm_arm_pvtime_supported();
+		break;
 	default:
 		r = kvm_arch_vm_ioctl_check_extension(kvm, ext);

arch/arm64/kvm/arm.c +0 -0

 		break;

arch/arm64/kvm/pvtime.c +1 -1

index 025b5f3a97ef..468bd1ef9c7e 100644
@@ -73,7 +73,7 @@ gpa_t kvm_init_stolen_time(struct kvm_vcpu *vcpu)
 	return base;
 }
 
-static bool kvm_arm_pvtime_supported(void)
+bool kvm_arm_pvtime_supported(void)
 {
 	return !!sched_info_on();
 }

arch/x86/kvm/x86.c +3 -0

index 88c593f83b28..27fc86bb28c4 100644
@@ -3538,6 +3538,9 @@ int kvm_vm_ioctl_check_extension(struct kvm
 *kvm, long ext)
 	case KVM_CAP_HYPERV_ENLIGHTENED_VMCS:
 		r = kvm_x86_ops.nested_ops->enable_evmcs != NULL;
 		break;
+	case KVM_CAP_STEAL_TIME:
+		r = sched_info_on();
+		break;
 	default:
 		break;

arch/x86/kvm/x86.c +0 -0

 	}

include/uapi/linux/kvm.h +0 -0

index 4fdf30316582..121fb29ac004 100644

The first three patches in the series are fixes that come from testing and reviewing pvtime code while writing the QEMU support (I'll reply to this mail with a link to the QEMU patches after posting - which I'll do shortly). The last patch is only a convenience for userspace, and I wouldn't be heartbroken if it wasn't deemed worth it. The QEMU patches I'll be posting are currently written without the cap. However, if the cap is accepted, then I'll change the QEMU code to use it.

Thanks,
drew

Andrew Jones (5):
KVM: arm64: pvtime: steal-time is only supported when configured
KVM: arm64: pvtime: Fix potential loss of stolen time
KVM: arm64: pvtime: Fix stolen time accounting across migration
KVM: Documentation minor fixups
arm64/x86: KVM: Introduce steal-time cap

 Documentation/virt/kvm/api.rst    | 20 ++++++++++++++++----
 arch/arm64/include/asm/kvm_host.h |  2 +-
 arch/arm64/kvm/arm.c              |  3 +++
 arch/arm64/kvm/pvtime.c           | 31 +++++++++++++++----------------
 arch/x86/kvm/x86.c                |  3 +++
 include/linux/kvm_host.h          | 19 +++++++++++++++++++
 include/uapi/linux/kvm.h          |  1 +
 7 files changed, 58 insertions(+), 21 deletions(-)

Signature · 1 line

2.25.4

The first three patches in the series are fixes that come from testing and reviewing pvtime code while writing the QEMU support (I'll reply to this mail with a link to the QEMU patches after posting - which I'll do shortly). The last patch is only a convenience for userspace, and I wouldn't be heartbroken if it wasn't deemed worth it. The QEMU patches I'll be posting are currently written without the cap. However, if the cap is accepted, then I'll change the QEMU code to use it.

Thanks,
drew

Andrew Jones (5):
KVM: arm64: pvtime: steal-time is only supported when configured
KVM: arm64: pvtime: Fix potential loss of stolen time
KVM: arm64: pvtime: Fix stolen time accounting across migration
KVM: Documentation minor fixups
arm64/x86: KVM: Introduce steal-time cap

  Documentation/virt/kvm/api.rst    | 20 ++++++++++++++++----
  arch/arm64/include/asm/kvm_host.h |  2 +-
  arch/arm64/kvm/arm.c              |  3 +++
  arch/arm64/kvm/pvtime.c           | 31 +++++++++++++++----------------
  arch/x86/kvm/x86.c                |  3 +++
  include/linux/kvm_host.h          | 19 +++++++++++++++++++
  include/uapi/linux/kvm.h          |  1 +
  7 files changed, 58 insertions(+), 21 deletions(-)

The first three patches in the series are fixes that come from testing and reviewing pvtime code while writing the QEMU support (I'll reply to this mail with a link to the QEMU patches after posting - which I'll do shortly). The last patch is only a convenience for userspace, and I wouldn't be heartbroken if it wasn't deemed worth it. The QEMU patches I'll be posting are currently written without the cap. However, if the cap is accepted, then I'll change the QEMU code to use it.

Thanks,
drew

Andrew Jones (5):
KVM: arm64: pvtime: steal-time is only supported when configured
KVM: arm64: pvtime: Fix potential loss of stolen time
KVM: arm64: pvtime: Fix stolen time accounting across migration
KVM: Documentation minor fixups
arm64/x86: KVM: Introduce steal-time cap

 Documentation/virt/kvm/api.rst    | 20 ++++++++++++++++----
 arch/arm64/include/asm/kvm_host.h |  2 +-
 arch/arm64/kvm/arm.c              |  3 +++
 arch/arm64/kvm/pvtime.c           | 31 +++++++++++++++----------------
 arch/x86/kvm/x86.c                |  3 +++
 include/linux/kvm_host.h          | 19 +++++++++++++++++++
 include/uapi/linux/kvm.h          |  1 +
 7 files changed, 58 insertions(+), 21 deletions(-)

Signature · 1 line

2.25.4

Hi Marc,

Ping?

The first three patches in the series are fixes that come from testing and reviewing pvtime code while writing the QEMU support (I'll reply to this mail with a link to the QEMU patches after posting - which I'll do shortly). The last patch is only a convenience for userspace, and I wouldn't be heartbroken if it wasn't deemed worth it. The QEMU patches I'll be posting are currently written without the cap. However, if the cap is accepted, then I'll change the QEMU code to use it.

Thanks,
drew

Andrew Jones (5):
KVM: arm64: pvtime: steal-time is only supported when configured
KVM: arm64: pvtime: Fix potential loss of stolen time
KVM: arm64: pvtime: Fix stolen time accounting across migration
KVM: Documentation minor fixups
arm64/x86: KVM: Introduce steal-time cap

 Documentation/virt/kvm/api.rst    | 20 ++++++++++++++++----
 arch/arm64/include/asm/kvm_host.h |  2 +-
 arch/arm64/kvm/arm.c              |  3 +++
 arch/arm64/kvm/pvtime.c           | 31 +++++++++++++++----------------
 arch/x86/kvm/x86.c                |  3 +++
 include/linux/kvm_host.h          | 19 +++++++++++++++++++
 include/uapi/linux/kvm.h          |  1 +
 7 files changed, 58 insertions(+), 21 deletions(-)

Quoted 18 lines · Andrew Jones

The first three patches in the series are fixes that come from testing and reviewing pvtime code while writing the QEMU support (I'll reply to this mail with a link to the QEMU patches after posting - which I'll do shortly). The last patch is only a convenience for userspace, and I wouldn't be heartbroken if it wasn't deemed worth it. The QEMU patches I'll be posting are currently written without the cap. However, if the cap is accepted, then I'll change the QEMU code to use it.

Thanks,
drew

Andrew Jones (5):
KVM: arm64: pvtime: steal-time is only supported when configured
KVM: arm64: pvtime: Fix potential loss of stolen time
KVM: arm64: pvtime: Fix stolen time accounting across migration
KVM: Documentation minor fixups
arm64/x86: KVM: Introduce steal-time cap

Quoted 8 lines

 Documentation/virt/kvm/api.rst    | 20 ++++++++++++++++----
 arch/arm64/include/asm/kvm_host.h |  2 +-
 arch/arm64/kvm/arm.c              |  3 +++
 arch/arm64/kvm/pvtime.c           | 31 +++++++++++++++----------------
 arch/x86/kvm/x86.c                |  3 +++
 include/linux/kvm_host.h          | 19 +++++++++++++++++++
 include/uapi/linux/kvm.h          |  1 +
 7 files changed, 58 insertions(+), 21 deletions(-)

Hi Andrew,

Sorry about the time it took to get to this series.

Although I had a number of comments, they are all easy to address, and you will hopefully be able to respin it quickly

(assuming we agree that patch #1 is unnecessary).