Skip to contentSkip to conversation

KVM: arm64: Make KVM_CAP_MAX_VCPUS compatible with the selected GIC version

PATCH In review Marc Zyngier 2 participants 2 messages Apr 27, 2020 – Apr 28, 2020

Subsystems (2)
lore
Share & tools
lore

Marc Zyngier

author maintainer
#0 lore raw

PATCH 1/1 KVM: arm64: Make KVM_CAP_MAX_VCPUS compatible with the selected GIC version

KVM_CAP_MAX_VCPUS always return the maximum possible number of VCPUs, irrespective of the selected interrupt controller. This is pretty misleading for userspace that selects a GICv2 on a GICv3 system that supports v2 compat: It always gets a maximum of 512 VCPUs, even if the effective limit is 8. The 9th VCPU will fail to be created, which is unexpected as far as userspace is concerned.

Fortunately, we already have the right information stashed in the kvm structure, and we can return it as requested.

---
 virt/kvm/arm/arm.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
virt/kvm/arm/arm.c +10 -5
index 48d0ec44ad77..f9b0528f7305 100644
@@ -95,6 +95,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 	return r;
 }
 
+static int kvm_arm_default_max_vcpus(void)
+{
+	return vgic_present ? kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
+}
+
 /**
  * kvm_arch_init_vm - initializes a VM data structure
  * @kvm:	pointer to the KVM struct
@@ -128,8 +133,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	kvm->arch.vmid.vmid_gen = 0;
 
 	/* The maximum number of VCPUs is limited by the host's GIC model */
-	kvm->arch.max_vcpus = vgic_present ?
-				kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
+	kvm->arch.max_vcpus = kvm_arm_default_max_vcpus();
 
 	return ret;
 out_free_stage2_pgd:
@@ -204,10 +208,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		r = num_online_cpus();
 		break;
 	case KVM_CAP_MAX_VCPUS:
-		r = KVM_MAX_VCPUS;
-		break;
 	case KVM_CAP_MAX_VCPU_ID:
-		r = KVM_MAX_VCPU_ID;
+		if (kvm)
+			r = kvm->arch.max_vcpus;
+		else
+			r = kvm_arm_default_max_vcpus();
 		break;
 	case KVM_CAP_MSI_DEVID:
 		if (!kvm)
Signature · 1 line
2.26.2
Reviewed-by Tested-by on v1 1/1
#1 lore raw

Hi,

Quoted 1 line · Marc Zyngier

KVM_CAP_MAX_VCPUS always return the maximum possible number of

s/return/returns?

Quoted 32 lines

VCPUs, irrespective of the selected interrupt controller. This is pretty misleading for userspace that selects a GICv2 on a GICv3 system that supports v2 compat: It always gets a maximum of 512 VCPUs, even if the effective limit is 8. The 9th VCPU will fail to be created, which is unexpected as far as userspace is concerned.

Fortunately, we already have the right information stashed in the kvm structure, and we can return it as requested.

---
 virt/kvm/arm/arm.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
virt/kvm/arm/arm.c +5 -0
index 48d0ec44ad77..f9b0528f7305 100644
@@ -95,6 +95,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 	return r;
 }
 
+static int kvm_arm_default_max_vcpus(void)
+{
+	return vgic_present ? kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
+}
+
 /**
  * kvm_arch_init_vm - initializes a VM data structure
  * @kvm:	pointer to the KVM struct

virt/kvm/arm/arm.c int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) · line 136

Quoted 6 lines
virt/kvm/arm/arm.c +1 -2
@@ -128,8 +133,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	kvm->arch.vmid.vmid_gen = 0;
 
 	/* The maximum number of VCPUs is limited by the host's GIC model */
-	kvm->arch.max_vcpus = vgic_present ?
-				kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
+	kvm->arch.max_vcpus = kvm_arm_default_max_vcpus();

Nitpicking, but the comment is not 100% true because the maximum number of vcpus is limited based on the requested vgic type, even before this patch.

virt/kvm/arm/arm.c int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · line 208

Quoted 5 lines
virt/kvm/arm/arm.c +0 -0
 	return ret;
 out_free_stage2_pgd:
@@ -204,10 +208,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		r = num_online_cpus();

Not relevant to this patch. If the host has a GICv3, and userspace requests a GICv2, it is possible that KVM_CAP_NR_VCPUS > KVM_CAP_MAX_VCPUS. I am curious, I don't see anything in the KVM API documentation about this case, so I suppose it's perfectly legal, right?

virt/kvm/arm/arm.c int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · line 215

Quoted 10 lines
virt/kvm/arm/arm.c +4 -3
 		break;
 	case KVM_CAP_MAX_VCPUS:
-		r = KVM_MAX_VCPUS;
-		break;
 	case KVM_CAP_MAX_VCPU_ID:
-		r = KVM_MAX_VCPU_ID;
+		if (kvm)
+			r = kvm->arch.max_vcpus;
+		else
+			r = kvm_arm_default_max_vcpus();

This works as expected - when KVM_CHECK_EXTENSION is called on the kvm fd, struct kvm is NULL.

virt/kvm/arm/arm.c int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · line 218

Quoted 3 lines
virt/kvm/arm/arm.c +0 -0
 		break;
 	case KVM_CAP_MSI_DEVID:
 		if (!kvm)

The patch looks fine to me:

Tested it on a rockpro64, which has a GICv3 that can also emulate a GICv2. When the vgic is a GICv3, before and after instantiating the device, the ioctl returns 512 on both /dev/kvm and the vm fd, as you would expect. When the vgic is a GICv2, the ioctl return 512 on /dev/kvm and the vm fd before instantiating the vgic; afterward it returns 512 on /dev/kvm and 8 on the vm fd:

Tested-by Alexandru Elisei