Skip to contentSkip to conversation

KVM/ARM: Some vgic fixes and init sequence KVM selftests

PATCH Superseded Eric Auger 3 participants 33 messages Dec 12, 2020 – Mar 12, 2021

Subsystems (5)
Documentation KVM Kselftest arm64 +1 more subsystems

v1 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Eric Auger

author
#0 lore raw

cover KVM/ARM: Some vgic fixes and init sequence KVM selftests

While writting vgic v3 init sequence KVM selftests I noticed some relatively minor issues. This was also the opportunity to try to fix the issue laterly reported by Zenghui, related to the RDIST_TYPER last bit emulation. The final patch is a first batch of VGIC init sequence selftests. Of course they can be augmented with a lot more register access tests, but let's try to move forward incrementally ...

Best Regards

Eric

This series can be found at:
https://github.com/eauger/linux/tree/vgic-selftests-and-fixes-v1

Eric Auger (9):
KVM: arm64: vgic-v3: Fix some error codes when setting RDIST base
KVM: arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION read
KVM: arm64: vgic-v3: Fix error handling in vgic_v3_set_redist_base()
KVM: arm/arm64: vgic: Reset base address on kvm_vgic_dist_destroy()
KVM: arm: move has_run_once after the map_resources
docs: kvm: devices/arm-vgic-v3: enhance KVM_DEV_ARM_VGIC_CTRL_INIT doc
KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]
KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace
KVM: selftests: aarch64/vgic-v3 init sequence tests

 .../virt/kvm/devices/arm-vgic-v3.rst          |   2 +-
 arch/arm64/kvm/arm.c                          |   4 +-
 arch/arm64/kvm/vgic/vgic-init.c               |   7 +-
 arch/arm64/kvm/vgic/vgic-kvm-device.c         |   3 +
 arch/arm64/kvm/vgic/vgic-mmio-v3.c            |  24 +-
 arch/arm64/kvm/vgic/vgic-mmio.c               |  10 +-
 include/kvm/arm_vgic.h                        |   1 +
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../testing/selftests/kvm/aarch64/vgic_init.c | 453 ++++++++++++++++++
 .../testing/selftests/kvm/include/kvm_util.h  |   5 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |  51 ++
 11 files changed, 546 insertions(+), 15 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/aarch64/vgic_init.c
Signature · 6 lines
2.21.3

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#1 lore raw

PATCH 1/9 KVM: arm64: vgic-v3: Fix some error codes when setting RDIST base

KVM_DEV_ARM_VGIC_GRP_ADDR group doc says we should return -EEXIST in case the base address of the redist is already set. We currently return -EINVAL.

However we need to return -EINVAL in case a legacy REDIST address is attempted to be set while REDIST_REGIONS were set. This case is discriminated by looking at the count field.

---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +7 -2
index 15a6c98ee92f..8e8a862def76 100644
@@ -792,8 +792,13 @@ static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index,
 	int ret;
 
 	/* single rdist region already set ?*/
-	if (!count && !list_empty(rd_regions))
-		return -EINVAL;
+	if (!count && !list_empty(rd_regions)) {
+		rdreg = list_last_entry(rd_regions,
+				       struct vgic_redist_region, list);
+		if (rdreg->count)
+			return -EINVAL; /* Mixing REDIST and REDIST_REGION API */
+		return -EEXIST;
+	}
 
 	/* cross the end of memory ? */
 	if (base + size < base)
Signature · 6 lines
2.21.3

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#4 lore raw

PATCH 2/9 KVM: arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION read

In reply to Eric Auger

The doc says:
"The characteristics of a specific redistributor region can
be read by presetting the index field in the attr data. Only valid for KVM_DEV_TYPE_ARM_VGIC_V3"

Unfortunately the existing code fails to read the input attr data and thus the index always is 0.

Fixes 04c110932225 ("KVM: arm/arm64: Implement KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION")
---
 arch/arm64/kvm/vgic/vgic-kvm-device.c | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/kvm/vgic/vgic-kvm-device.c +3 -0
index 44419679f91a..2f66cf247282 100644
@@ -226,6 +226,9 @@ static int vgic_get_common_attr(struct kvm_device *dev,
 		u64 addr;
 		unsigned long type = (unsigned long)attr->attr;
 
+		if (copy_from_user(&addr, uaddr, sizeof(addr)))
+			return -EFAULT;
+
 		r = kvm_vgic_addr(dev->kvm, type, &addr, false);
 		if (r)
 			return (r == -ENODEV) ? -ENXIO : r;
Signature · 6 lines
2.21.3

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#7 lore raw

PATCH 3/9 KVM: arm64: vgic-v3: Fix error handling in vgic_v3_set_redist_base()

In reply to Eric Auger

vgic_register_all_redist_iodevs may succeed while vgic_register_all_redist_iodevs fails. For example this can happen while adding a redistributor region overlapping a dist region. The failure only is detected on vgic_register_all_redist_iodevs when vgic_v3_check_base() gets called.

In such a case, remove the newly added redistributor region and free it.

---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +7 -1
index 8e8a862def76..581f0f490000 100644
@@ -866,8 +866,14 @@ int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count)
 	 * afterwards will register the iodevs when needed.
 	 */
 	ret = vgic_register_all_redist_iodevs(kvm);
-	if (ret)
+	if (ret) {
+		struct vgic_redist_region *rdreg =
+			vgic_v3_rdist_region_from_index(kvm, index);
+
+		list_del(&rdreg->list);
+		kfree(rdreg);
 		return ret;
+	}
 
 	return 0;
 }
Signature · 6 lines
2.21.3

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#10 lore raw

PATCH 4/9 KVM: arm/arm64: vgic: Reset base address on kvm_vgic_dist_destroy()

In reply to Eric Auger

On vgic_dist_destroy(), the addresses are not reset. However for kvm selftest purpose this would allow to continue the test execution even after a failure when running KVM_RUN. So let's reset the base addresses.

---
 arch/arm64/kvm/vgic/vgic-init.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
arch/arm64/kvm/vgic/vgic-init.c +5 -2
index 32e32d67a127..6147bed56b1b 100644
@@ -335,14 +335,16 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
 	kfree(dist->spis);
 	dist->spis = NULL;
 	dist->nr_spis = 0;
+	dist->vgic_dist_base = VGIC_ADDR_UNDEF;
 
-	if (kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
+	if (dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
 		list_for_each_entry_safe(rdreg, next, &dist->rd_regions, list) {
 			list_del(&rdreg->list);
 			kfree(rdreg);
 		}
 		INIT_LIST_HEAD(&dist->rd_regions);
-	}
+	} else
+		kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;
 
 	if (vgic_has_its(kvm))
 		vgic_lpi_translation_cache_destroy(kvm);
@@ -362,6 +364,7 @@ void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
 	vgic_flush_pending_lpis(vcpu);
 
 	INIT_LIST_HEAD(&vgic_cpu->ap_list_head);
+	vgic_cpu->rd_iodev.base_addr = VGIC_ADDR_UNDEF;
 }
 
 /* To be called with kvm->lock held */
Signature · 6 lines
2.21.3

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#13 lore raw

PATCH 5/9 KVM: arm: move has_run_once after the map_resources

In reply to Eric Auger

has_run_once is set to true at the beginning of kvm_vcpu_first_run_init(). This generally is not an issue except when exercising the code with KVM selftests. Indeed, if kvm_vgic_map_resources() fails due to erroneous user settings, has_run_once is set and this prevents from continuing executing the test. This patch moves the assignment after the kvm_vgic_map_resources().

---
 arch/arm64/kvm/arm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/kvm/arm.c +2 -2
index c0ffb019ca8b..331fae6bff31 100644
@@ -540,8 +540,6 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 	if (!kvm_arm_vcpu_is_finalized(vcpu))
 		return -EPERM;
 
-	vcpu->arch.has_run_once = true;
-
 	if (likely(irqchip_in_kernel(kvm))) {
 		/*
 		 * Map the VGIC hardware resources before running a vcpu the
@@ -560,6 +558,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 		static_branch_inc(&userspace_irqchip_in_use);
 	}
 
+	vcpu->arch.has_run_once = true;
+
 	ret = kvm_timer_enable(vcpu);
 	if (ret)
 		return ret;
Signature · 6 lines
2.21.3

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#18 lore raw

PATCH 6/9 docs: kvm: devices/arm-vgic-v3: enhance KVM_DEV_ARM_VGIC_CTRL_INIT doc

In reply to Eric Auger

kvm_arch_vcpu_precreate() returns -EBUSY if the vgic is already initialized. So let's document that KVM_DEV_ARM_VGIC_CTRL_INIT must be called after all vcpu creations.

---
 Documentation/virt/kvm/devices/arm-vgic-v3.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/virt/kvm/devices/arm-vgic-v3.rst +1 -1
index 5dd3bff51978..322de6aebdec 100644
@@ -228,7 +228,7 @@ Groups:
 
     KVM_DEV_ARM_VGIC_CTRL_INIT
       request the initialization of the VGIC, no additional parameter in
-      kvm_device_attr.addr.
+      kvm_device_attr.addr. Must be called after all vcpu creations.
     KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES
       save all LPI pending bits into guest RAM pending tables.
 
Signature · 6 lines
2.21.3

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#21 lore raw

PATCH 7/9 KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]

In reply to Eric Auger

Instead of converting the vgic_io_device handle to a kvm_io_device handled and then do the oppositive, pass a vgic_io_device pointer all along the call chain.

---
 arch/arm64/kvm/vgic/vgic-mmio.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
arch/arm64/kvm/vgic/vgic-mmio.c +4 -6
index b2d73fc0d1ef..48c6067fc5ec 100644
@@ -938,10 +938,9 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 	return region;
 }
 
-static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 			     gpa_t addr, u32 *val)
 {
-	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
 	const struct vgic_register_region *region;
 	struct kvm_vcpu *r_vcpu;
 
@@ -960,10 +959,9 @@ static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 	return 0;
 }
 
-static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 			      gpa_t addr, const u32 *val)
 {
-	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
 	const struct vgic_register_region *region;
 	struct kvm_vcpu *r_vcpu;
 
@@ -986,9 +984,9 @@ int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
 		 bool is_write, int offset, u32 *val)
 {
 	if (is_write)
-		return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
+		return vgic_uaccess_write(vcpu, dev, offset, val);
 	else
-		return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
+		return vgic_uaccess_read(vcpu, dev, offset, val);
 }
 
 static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
Signature · 6 lines
2.21.3

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#25 lore raw

PATCH 8/9 KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace

In reply to Eric Auger

Commit 23bde34771f1 ("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access the GICR_TYPER register before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.

---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++-
 include/kvm/arm_vgic.h             | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +6 -1
index 581f0f490000..2f9ef6058f6e 100644
@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 						 gpa_t addr, unsigned int len)
 {
 	unsigned long mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
+	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+	struct vgic_redist_region *rdreg = vgic_cpu->rdreg;
 	int target_vcpu_id = vcpu->vcpu_id;
 	u64 value;
 
@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 	if (vgic_has_its(vcpu->kvm))
 		value |= GICR_TYPER_PLPIS;
 
-	/* reporting of the Last bit is not supported for userspace */
+	if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index - 1)))
+		value |= GICR_TYPER_LAST;
+
 	return extract_bytes(value, addr & 7, len);
 }
 
@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 
 	vgic_cpu->rdreg = rdreg;
+	vgic_cpu->rdreg_index = rdreg->free_index;
 
 	rd_base = rdreg->base + rdreg->free_index * KVM_VGIC_V3_REDIST_SIZE;
 
include/kvm/arm_vgic.h +1 -0
index a8d8fdcd3723..596c069263a7 100644
@@ -322,6 +322,7 @@ struct vgic_cpu {
 	 */
 	struct vgic_io_device	rd_iodev;
 	struct vgic_redist_region *rdreg;
+	u32 rdreg_index;
 
 	/* Contains the attributes and gpa of the LPI pending tables. */
 	u64 pendbaser;
Signature · 6 lines
2.21.3

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#32 lore raw

PATCH 9/9 KVM: selftests: aarch64/vgic-v3 init sequence tests

In reply to Eric Auger

The tests exercise the VGIC_V3 device creation including the associated KVM_DEV_ARM_VGIC_GRP_ADDR group attributes:

- KVM_VGIC_V3_ADDR_TYPE_DIST/REDIST (vcpu_first and vgic_first)
- KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION (redist_regions).

Another test dedicates to KVM_DEV_ARM_VGIC_GRP_REDIST_REGS group and especially the GICR_TYPER read. The goal was to test the case recently fixed by commit 23bde34771f1
("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace").

The API under test can be found at
Documentation/virt/kvm/devices/arm-vgic-v3.rst

---
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../testing/selftests/kvm/aarch64/vgic_init.c | 453 ++++++++++++++++++
 .../testing/selftests/kvm/include/kvm_util.h  |   5 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |  51 ++
 4 files changed, 510 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/aarch64/vgic_init.c
tools/testing/selftests/kvm/Makefile +1 -0
index 3d14ef77755e..dc2fd33bd580 100644
@@ -68,6 +68,7 @@ TEST_GEN_PROGS_x86_64 += steal_time
 
 TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
 TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list-sve
+TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
 TEST_GEN_PROGS_aarch64 += demand_paging_test
 TEST_GEN_PROGS_aarch64 += dirty_log_test
 TEST_GEN_PROGS_aarch64 += kvm_create_max_vcpus
tools/testing/selftests/kvm/aarch64/vgic_init.c new file +453 -0

453 lines not on this page · raw message

tools/testing/selftests/kvm/include/kvm_util.h +5 -0
index 7d29aa786959..0fecea13a570 100644
@@ -200,6 +200,11 @@ int vcpu_nested_state_set(struct kvm_vm *vm, uint32_t vcpuid,
 			  struct kvm_nested_state *state, bool ignore_error);
 #endif
 
+int kvm_device_check_attr(int dev_fd, uint32_t group, uint64_t attr);
+int kvm_create_device(struct kvm_vm *vm, uint64_t type, bool test);
+int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,
+		      void *val, bool write);
+
 const char *exit_reason_str(unsigned int exit_reason);
 
 void virt_pgd_alloc(struct kvm_vm *vm, uint32_t pgd_memslot);
tools/testing/selftests/kvm/lib/kvm_util.c +51 -0

57 lines not on this page · raw message

Signature · 6 lines
2.21.3

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
maintainer
#8 lore raw

Re: [PATCH 3/9] KVM: arm64: vgic-v3: Fix error handling in vgic_v3_set_redist_base()

In reply to Eric Auger

Hi Eric,

Quoted 3 lines · Eric Auger

vgic_register_all_redist_iodevs may succeed while vgic_register_all_redist_iodevs fails. For example this can happen

The same function cannot both fail and succeed ;-) Can you shed some light on what you had in mind?

Quoted 17 lines

while adding a redistributor region overlapping a dist region. The failure only is detected on vgic_register_all_redist_iodevs when vgic_v3_check_base() gets called.

In such a case, remove the newly added redistributor region and free it.

---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0 -0
index 8e8a862def76..581f0f490000 100644

arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count) · line 872

Quoted 8 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +4 -1
@@ -866,8 +866,14 @@ int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count)
 	 * afterwards will register the iodevs when needed.
 	 */
 	ret = vgic_register_all_redist_iodevs(kvm);
-	if (ret)
+	if (ret) {
+		struct vgic_redist_region *rdreg =
+			vgic_v3_rdist_region_from_index(kvm, index);
+

nit: consider splitting declaration and assignment so that we avoid the line split if you insist on the 80 character limit.

Quoted 11 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +3 -0
+		list_del(&rdreg->list);
+		kfree(rdreg);
 		return ret;
+	}
 
 	return 0;
 }
Signature · 1 line
2.21.3

Thanks,

	M.
Signature · 5 lines
Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
maintainer
#11 lore raw

Re: [PATCH 4/9] KVM: arm/arm64: vgic: Reset base address on kvm_vgic_dist_destroy()

In reply to Eric Auger
Quoted 21 lines · Eric Auger

On vgic_dist_destroy(), the addresses are not reset. However for kvm selftest purpose this would allow to continue the test execution even after a failure when running KVM_RUN. So let's reset the base addresses.

---
 arch/arm64/kvm/vgic/vgic-init.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
arch/arm64/kvm/vgic/vgic-init.c +1 -0
index 32e32d67a127..6147bed56b1b 100644
@@ -335,14 +335,16 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
 	kfree(dist->spis);
 	dist->spis = NULL;
 	dist->nr_spis = 0;
+	dist->vgic_dist_base = VGIC_ADDR_UNDEF;
 

arch/arm64/kvm/vgic/vgic-init.c static void kvm_vgic_dist_destroy(struct kvm *kvm) · line 347

Quoted 10 lines
arch/arm64/kvm/vgic/vgic-init.c +3 -2
-	if (kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
+	if (dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
 		list_for_each_entry_safe(rdreg, next, &dist->rd_regions, list) {
 			list_del(&rdreg->list);
 			kfree(rdreg);
 		}
 		INIT_LIST_HEAD(&dist->rd_regions);
-	}
+	} else
+		kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;

Since you have converted the hunk above to use dist->, you could do the same thing here. And the coding style dictates that you need {} on the else side as well.

Quoted 15 lines
arch/arm64/kvm/vgic/vgic-init.c +1 -0
 	if (vgic_has_its(kvm))
 		vgic_lpi_translation_cache_destroy(kvm);
@@ -362,6 +364,7 @@ void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
 	vgic_flush_pending_lpis(vcpu);
 
 	INIT_LIST_HEAD(&vgic_cpu->ap_list_head);
+	vgic_cpu->rd_iodev.base_addr = VGIC_ADDR_UNDEF;
 }
 
 /* To be called with kvm->lock held */
Signature · 1 line
2.21.3

Thanks,

	M.
Signature · 5 lines
Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
#2 lore raw

Re: [PATCH 1/9] KVM: arm64: vgic-v3: Fix some error codes when setting RDIST base

In reply to Eric Auger

Hi Eric,

Quoted 20 lines · Eric Auger

KVM_DEV_ARM_VGIC_GRP_ADDR group doc says we should return -EEXIST in case the base address of the redist is already set. We currently return -EINVAL.

However we need to return -EINVAL in case a legacy REDIST address is attempted to be set while REDIST_REGIONS were set. This case is discriminated by looking at the count field.

---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0 -0
index 15a6c98ee92f..8e8a862def76 100644
@@ -792,8 +792,13 @@ static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index,
 	int ret;
 

arch/arm64/kvm/vgic/vgic-mmio-v3.c static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index, · line 801

Quoted 10 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +7 -2
 	/* single rdist region already set ?*/
-	if (!count && !list_empty(rd_regions))
-		return -EINVAL;
+	if (!count && !list_empty(rd_regions)) {
+		rdreg = list_last_entry(rd_regions,
+				       struct vgic_redist_region, list);
+		if (rdreg->count)
+			return -EINVAL; /* Mixing REDIST and REDIST_REGION API */
+		return -EEXIST;
+	}

A few instructions below:

    if (list_empty(rd_regions)) {
        [..]
    } else {
        rdreg = list_last_entry(rd_regions, struct vgic_redist_region, list); [..]

        /* Cannot add an explicitly sized regions after legacy region */ if (!rdreg->count)
            return -EINVAL;
    }

Isn't this testing for the same thing, but using the opposite condition? Or am I misunderstanding the code (quite likely)?

Looks to me like KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST{,_REGION}) used to return -EEXIST (from vgic_check_ioaddr()) before commit ccc27bf5be7b7 ("KVM: arm/arm64: Helper to register a new redistributor region") which added the vgic_v3_insert_redist_region() function, so bringing back the -EEXIST return code looks the right thing to me.

Thanks,
Alex

arch/arm64/kvm/vgic/vgic-mmio-v3.c static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index, · line 804

Quoted 3 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0 -0
 	/* cross the end of memory ? */
 	if (base + size < base)

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Reviewed-by on v1 2/9
#5 lore raw

Re: [PATCH 2/9] KVM: arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION read

In reply to Eric Auger

Hi Eric,

The patch looks correct to me. kvm_vgic_addr() masks out all the bits except index from addr, so we don't need to do it in vgic_get_common_attr():

One nitpick below.

Quoted 7 lines · Eric Auger

The doc says:
"The characteristics of a specific redistributor region can
be read by presetting the index field in the attr data. Only valid for KVM_DEV_TYPE_ARM_VGIC_V3"

Unfortunately the existing code fails to read the input attr data and thus the index always is 0.

addr is allocated on the stack, I don't think it will always be 0.

Thanks,
Alex

Quoted 13 lines
Fixes 04c110932225 ("KVM: arm/arm64: Implement KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION")
---
 arch/arm64/kvm/vgic/vgic-kvm-device.c | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/kvm/vgic/vgic-kvm-device.c +0 -0
index 44419679f91a..2f66cf247282 100644

arch/arm64/kvm/vgic/vgic-kvm-device.c static int vgic_get_common_attr(struct kvm_device *dev, · line 234

Quoted 9 lines
arch/arm64/kvm/vgic/vgic-kvm-device.c +3 -0
@@ -226,6 +226,9 @@ static int vgic_get_common_attr(struct kvm_device *dev,
 		u64 addr;
 		unsigned long type = (unsigned long)attr->attr;
 
+		if (copy_from_user(&addr, uaddr, sizeof(addr)))
+			return -EFAULT;
+
 		r = kvm_vgic_addr(dev->kvm, type, &addr, false);
 		if (r)
 			return (r == -ENODEV) ? -ENXIO : r;

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

#14 lore raw

Re: [PATCH 5/9] KVM: arm: move has_run_once after the map_resources

In reply to Eric Auger

Hi Eric,

Quoted 27 lines · Eric Auger

has_run_once is set to true at the beginning of kvm_vcpu_first_run_init(). This generally is not an issue except when exercising the code with KVM selftests. Indeed, if kvm_vgic_map_resources() fails due to erroneous user settings, has_run_once is set and this prevents from continuing executing the test. This patch moves the assignment after the kvm_vgic_map_resources().

---
 arch/arm64/kvm/arm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/kvm/arm.c +0 -2
index c0ffb019ca8b..331fae6bff31 100644
@@ -540,8 +540,6 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 	if (!kvm_arm_vcpu_is_finalized(vcpu))
 		return -EPERM;
 
-	vcpu->arch.has_run_once = true;
-
 	if (likely(irqchip_in_kernel(kvm))) {
 		/*
 		 * Map the VGIC hardware resources before running a vcpu the

arch/arm64/kvm/arm.c static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu) · line 561

Quoted 4 lines
arch/arm64/kvm/arm.c +1 -0
@@ -560,6 +558,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 		static_branch_inc(&userspace_irqchip_in_use);
 	}
 
+	vcpu->arch.has_run_once = true;

I have a few concerns regarding this:

1. Moving has_run_once = true here seems very arbitrary to me - kvm_timer_enable() and kvm_arm_pmu_v3_enable(), below it, can both fail because of erroneous user values. If there's a reason why the assignment cannot be moved at the end of the function, I think it should be clearly stated in a comment for the people who might be tempted to write similar tests for the timer or pmu.

2. There are many ways that kvm_vgic_map_resources() can fail, other than incorrect user settings. I started digging into how kvm_vgic_map_resources()->vgic_v2_map_resources() can fail for a VGIC V2 and this is what I managed to find before I gave up:

* vgic_init() can fail in:
    - kvm_vgic_dist_init()
    - vgic_v3_init()
    - kvm_vgic_setup_default_irq_routing()
* vgic_register_dist_iodev() can fail in:
    - vgic_v3_init_dist_iodev()
    - kvm_io_bus_register_dev()(*)
* kvm_phys_addr_ioremap() can fail in:
    - kvm_mmu_topup_memory_cache()
    - kvm_pgtable_stage2_map()

So if any of the functions below fail, are we 100% sure it is safe to allow the user to execute kvm_vgic_map_resources() again?

(*) It looks to me like kvm_io_bus_register_dev() doesn't take into account a caller that tries to register the same device address range and it will create another identical range. Is this intentional? Is it a bug that should be fixed? Or am I misunderstanding the function?

Thanks,
Alex

arch/arm64/kvm/arm.c static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu) · line 565

Quoted 4 lines
arch/arm64/kvm/arm.c +1 -0
+
 	ret = kvm_timer_enable(vcpu);
 	if (ret)
 		return ret;

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

#19 lore raw

Re: [PATCH 6/9] docs: kvm: devices/arm-vgic-v3: enhance KVM_DEV_ARM_VGIC_CTRL_INIT doc

In reply to Eric Auger

Hi Eric,

Quoted 3 lines · Eric Auger

kvm_arch_vcpu_precreate() returns -EBUSY if the vgic is already initialized. So let's document that KVM_DEV_ARM_VGIC_CTRL_INIT must be called after all vcpu creations.

Checked and this is indeed the case, kvm_vm_ioctl_create_vcpu()->kvm_arch_vcpu_precreate() returns -EBUSY is vgic_initialized() is true.

Quoted 11 lines
---
 Documentation/virt/kvm/devices/arm-vgic-v3.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/virt/kvm/devices/arm-vgic-v3.rst +0 -0
index 5dd3bff51978..322de6aebdec 100644

Documentation/virt/kvm/devices/arm-vgic-v3.rst Groups: · line 231

Quoted 5 lines
Documentation/virt/kvm/devices/arm-vgic-v3.rst +1 -1
@@ -228,7 +228,7 @@ Groups:
 
     KVM_DEV_ARM_VGIC_CTRL_INIT
       request the initialization of the VGIC, no additional parameter in
-      kvm_device_attr.addr.
+      kvm_device_attr.addr. Must be called after all vcpu creations.

Nitpick here: the document writes VCPU with all caps. This also sounds a bit weird, I think something like "Must be called after all VCPUs have been created" is clearer.

Thanks,
Alex

Documentation/virt/kvm/devices/arm-vgic-v3.rst Groups: · line 234

Quoted 3 lines
Documentation/virt/kvm/devices/arm-vgic-v3.rst +0 -0
     KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES
       save all LPI pending bits into guest RAM pending tables.
 

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

#22 lore raw

Re: [PATCH 7/9] KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]

In reply to Eric Auger

Hi Eric,

Quoted 3 lines · Eric Auger

Instead of converting the vgic_io_device handle to a kvm_io_device handled and then do the oppositive, pass a vgic_io_device pointer all along the call chain.

To me, it looks like the commit message describes what the patch does instead of why it does it.

What are "vgic_io_device handle" and "kvm_io_device handled"?

Thanks,
Alex

Quoted 36 lines
---
 arch/arm64/kvm/vgic/vgic-mmio.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
arch/arm64/kvm/vgic/vgic-mmio.c +2 -4
index b2d73fc0d1ef..48c6067fc5ec 100644
@@ -938,10 +938,9 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 	return region;
 }
 
-static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 			     gpa_t addr, u32 *val)
 {
-	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
 	const struct vgic_register_region *region;
 	struct kvm_vcpu *r_vcpu;
 
@@ -960,10 +959,9 @@ static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 	return 0;
 }
 
-static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 			      gpa_t addr, const u32 *val)
 {
-	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
 	const struct vgic_register_region *region;
 	struct kvm_vcpu *r_vcpu;
 
@@ -986,9 +984,9 @@ int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
 		 bool is_write, int offset, u32 *val)

arch/arm64/kvm/vgic/vgic-mmio.c int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, · line 992

Quoted 10 lines
arch/arm64/kvm/vgic/vgic-mmio.c +2 -2
 {
 	if (is_write)
-		return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
+		return vgic_uaccess_write(vcpu, dev, offset, val);
 	else
-		return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
+		return vgic_uaccess_read(vcpu, dev, offset, val);
 }
 
 static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

#23 lore raw

Re: [PATCH 7/9] KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]

Hi Eric,

Quoted 10 lines · Alexandru Elisei

Hi Eric,

Quoted 3 lines · Eric Auger

Instead of converting the vgic_io_device handle to a kvm_io_device handled and then do the oppositive, pass a vgic_io_device pointer all along the call chain.

To me, it looks like the commit message describes what the patch does instead of why it does it.

What are "vgic_io_device handle" and "kvm_io_device handled"?

Sorry, I think I got it now. You were referring to the argument types struct vgic_io_device and struct kvm_io_device. The patch looks like a very good cleanup.

How changing to commit message to sound something like this (feel free to ignore/change it if you think of something else):

vgic_uaccess() takes a struct vgic_io_device argument, converts it to a struct kvm_io_device and passes it to the read/write accessor functions, which convert it back to a struct vgic_io_device. Avoid the indirection by passing the struct vgic_io_device argument directly to vgic_uaccess_{read,write).

Thanks,
Alex

Quoted 47 lines

Thanks,
Alex

Quoted 44 lines
---
 arch/arm64/kvm/vgic/vgic-mmio.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
arch/arm64/kvm/vgic/vgic-mmio.c +4 -6
index b2d73fc0d1ef..48c6067fc5ec 100644
@@ -938,10 +938,9 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 	return region;
 }
 
-static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 			     gpa_t addr, u32 *val)
 {
-	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
 	const struct vgic_register_region *region;
 	struct kvm_vcpu *r_vcpu;
 
@@ -960,10 +959,9 @@ static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 	return 0;
 }
 
-static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 			      gpa_t addr, const u32 *val)
 {
-	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
 	const struct vgic_register_region *region;
 	struct kvm_vcpu *r_vcpu;
 
@@ -986,9 +984,9 @@ int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
 		 bool is_write, int offset, u32 *val)
 {
 	if (is_write)
-		return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
+		return vgic_uaccess_write(vcpu, dev, offset, val);
 	else
-		return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
+		return vgic_uaccess_read(vcpu, dev, offset, val);
 }
 
Quoted 5 lines

arch/arm64/kvm/vgic/vgic-mmio.c int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, · line 992

Quoted 1 line
arch/arm64/kvm/vgic/vgic-mmio.c +0 -0
 static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

#26 lore raw

Re: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace

In reply to Eric Auger

Hi Eric,

Quoted 7 lines · Eric Auger

Commit 23bde34771f1 ("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access the GICR_TYPER register before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.

I suppose the reason for emulating GICR_TYPER.Last is for architecture compliance, right? I think that should be in the commit message.

Quoted 32 lines
---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++-
 include/kvm/arm_vgic.h             | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +5 -1
index 581f0f490000..2f9ef6058f6e 100644
@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 						 gpa_t addr, unsigned int len)
 {
 	unsigned long mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
+	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+	struct vgic_redist_region *rdreg = vgic_cpu->rdreg;
 	int target_vcpu_id = vcpu->vcpu_id;
 	u64 value;
 
@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 	if (vgic_has_its(vcpu->kvm))
 		value |= GICR_TYPER_PLPIS;
 
-	/* reporting of the Last bit is not supported for userspace */
+	if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index - 1)))
+		value |= GICR_TYPER_LAST;
+
 	return extract_bytes(value, addr & 7, len);
 }
 

arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_register_redist_iodev(struct kvm_vcpu *vcpu) · line 721

Quoted 4 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +1 -0
@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 
 	vgic_cpu->rdreg = rdreg;
+	vgic_cpu->rdreg_index = rdreg->free_index;

What happens if the next redistributor region we register has the base address adjacent to this one?

I'm really not familiar with the code, but is it not possible to create two Redistributor regions (via KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST)) where the second Redistributor region start address is immediately after the last Redistributor in the preceding region?

Thanks,
Alex

Quoted 8 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0 -0
 	rd_base = rdreg->base + rdreg->free_index * KVM_VGIC_V3_REDIST_SIZE;
 
include/kvm/arm_vgic.h +0 -0
index a8d8fdcd3723..596c069263a7 100644

include/kvm/arm_vgic.h struct vgic_cpu { · line 328

Quoted 7 lines
include/kvm/arm_vgic.h +1 -0
@@ -322,6 +322,7 @@ struct vgic_cpu {
 	 */
 	struct vgic_io_device	rd_iodev;
 	struct vgic_redist_region *rdreg;
+	u32 rdreg_index;
 
 	/* Contains the attributes and gpa of the LPI pending tables. */
 	u64 pendbaser;

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

#30 lore raw

Re: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace

In reply to Eric Auger

Hi Eric,

Quoted 7 lines · Eric Auger

Commit 23bde34771f1 ("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access the GICR_TYPER register before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.

If I understand your patch correctly, it is possible for the GICR_TYPER.Last bit to be transiently 1 if the register is accessed before all the redistributors regions have been configured.

Arm IHI 0069F states that accesses to the GICR_TYPER register are RO. I haven't found exactly what RO means (please point me to the definition if you find it in the architecture!), but I assume it means read-only and I'm not sure how correct (from an architectural point of view) it is for two subsequent reads of this register to return different values. Maybe Marc can shed some light on this.

Thanks,
Alex

Quoted 44 lines
---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++-
 include/kvm/arm_vgic.h             | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +6 -1
index 581f0f490000..2f9ef6058f6e 100644
@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 						 gpa_t addr, unsigned int len)
 {
 	unsigned long mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
+	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+	struct vgic_redist_region *rdreg = vgic_cpu->rdreg;
 	int target_vcpu_id = vcpu->vcpu_id;
 	u64 value;
 
@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 	if (vgic_has_its(vcpu->kvm))
 		value |= GICR_TYPER_PLPIS;
 
-	/* reporting of the Last bit is not supported for userspace */
+	if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index - 1)))
+		value |= GICR_TYPER_LAST;
+
 	return extract_bytes(value, addr & 7, len);
 }
 
@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 
 	vgic_cpu->rdreg = rdreg;
+	vgic_cpu->rdreg_index = rdreg->free_index;
 
 	rd_base = rdreg->base + rdreg->free_index * KVM_VGIC_V3_REDIST_SIZE;
 
include/kvm/arm_vgic.h +0 -0
index a8d8fdcd3723..596c069263a7 100644

include/kvm/arm_vgic.h struct vgic_cpu { · line 328

Quoted 7 lines
include/kvm/arm_vgic.h +1 -0
@@ -322,6 +322,7 @@ struct vgic_cpu {
 	 */
 	struct vgic_io_device	rd_iodev;
 	struct vgic_redist_region *rdreg;
+	u32 rdreg_index;
 
 	/* Contains the attributes and gpa of the LPI pending tables. */
 	u64 pendbaser;

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

maintainer
#31 lore raw

Re: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace

Quoted 15 lines · Alexandru Elisei

Hi Eric,

Quoted 7 lines · Eric Auger

Commit 23bde34771f1 ("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access the GICR_TYPER register before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.

If I understand your patch correctly, it is possible for the GICR_TYPER.Last bit to be transiently 1 if the register is accessed before all the redistributors regions have been configured.

Quoted 5 lines

Arm IHI 0069F states that accesses to the GICR_TYPER register are RO. I haven't found exactly what RO means (please point me to the definition if you find it in the architecture!), but I assume it means read-only and I'm not sure how correct (from an architectural point of view) it is for two subsequent reads of this register to return different values. Maybe Marc can shed some light on this.

RO = Read-Only indeed. Not sure that's documented anywhere in the architecture, but this is enough of a well known acronym that even the ARM ARM doesn't feel the need to invent a new definition for it.

As for your concern, I don't think it is a problem to return different values if the HW has changed in between. This may need to be documented though.

Thanks,

       M.
Signature · 5 lines
Jazz is not dead. It just smells funny...
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author
#24 lore raw

Re: [PATCH 7/9] KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]

In reply to Alexandru Elisei

Hi Alexandru,

Quoted 13 lines · Alexandru Elisei

Hi Eric,

Quoted 10 lines · Alexandru Elisei

Hi Eric,

Quoted 3 lines · Eric Auger

Instead of converting the vgic_io_device handle to a kvm_io_device handled and then do the oppositive, pass a vgic_io_device pointer all along the call chain.

To me, it looks like the commit message describes what the patch does instead of why it does it.

What are "vgic_io_device handle" and "kvm_io_device handled"?

Yes unfortunate typo, sorry.

Quoted 11 lines

Sorry, I think I got it now. You were referring to the argument types struct vgic_io_device and struct kvm_io_device. The patch looks like a very good cleanup.

How changing to commit message to sound something like this (feel free to ignore/change it if you think of something else):

vgic_uaccess() takes a struct vgic_io_device argument, converts it to a struct kvm_io_device and passes it to the read/write accessor functions, which convert it back to a struct vgic_io_device. Avoid the indirection by passing the struct vgic_io_device argument directly to vgic_uaccess_{read,write).

I reworded the commit message as you suggested.

Thanks

Eric

Quoted 56 lines

Thanks,
Alex

Quoted 52 lines

Thanks,
Alex

Quoted 35 lines
---
 arch/arm64/kvm/vgic/vgic-mmio.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
arch/arm64/kvm/vgic/vgic-mmio.c +2 -4
index b2d73fc0d1ef..48c6067fc5ec 100644
@@ -938,10 +938,9 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 	return region;
 }
 
-static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 			     gpa_t addr, u32 *val)
 {
-	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
 	const struct vgic_register_region *region;
 	struct kvm_vcpu *r_vcpu;
 
@@ -960,10 +959,9 @@ static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 	return 0;
 }
 
-static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,
 			      gpa_t addr, const u32 *val)
 {
-	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
 	const struct vgic_register_region *region;
 	struct kvm_vcpu *r_vcpu;
 
@@ -986,9 +984,9 @@ int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
 		 bool is_write, int offset, u32 *val)

arch/arm64/kvm/vgic/vgic-mmio.c int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, · line 992

Quoted 10 lines
arch/arm64/kvm/vgic/vgic-mmio.c +2 -2
 {
 	if (is_write)
-		return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
+		return vgic_uaccess_write(vcpu, dev, offset, val);
 	else
-		return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
+		return vgic_uaccess_read(vcpu, dev, offset, val);
 }
 
 static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author
#20 lore raw

Re: [PATCH 6/9] docs: kvm: devices/arm-vgic-v3: enhance KVM_DEV_ARM_VGIC_CTRL_INIT doc

In reply to Alexandru Elisei

Hi Alexandru,

Quoted 10 lines · Alexandru Elisei

Hi Eric,

Quoted 3 lines · Eric Auger

kvm_arch_vcpu_precreate() returns -EBUSY if the vgic is already initialized. So let's document that KVM_DEV_ARM_VGIC_CTRL_INIT must be called after all vcpu creations.

Checked and this is indeed the case, kvm_vm_ioctl_create_vcpu()->kvm_arch_vcpu_precreate() returns -EBUSY is vgic_initialized() is true.

thanks!

Quoted 13 lines
Quoted 13 lines
---
 Documentation/virt/kvm/devices/arm-vgic-v3.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/virt/kvm/devices/arm-vgic-v3.rst +0 -0
index 5dd3bff51978..322de6aebdec 100644
@@ -228,7 +228,7 @@ Groups:
 
Quoted 8 lines

Documentation/virt/kvm/devices/arm-vgic-v3.rst Groups: · line 231

Quoted 4 lines
Documentation/virt/kvm/devices/arm-vgic-v3.rst +1 -1
     KVM_DEV_ARM_VGIC_CTRL_INIT
       request the initialization of the VGIC, no additional parameter in
-      kvm_device_attr.addr.
+      kvm_device_attr.addr. Must be called after all vcpu creations.

Nitpick here: the document writes VCPU with all caps. This also sounds a bit weird, I think something like "Must be called after all VCPUs have been created" is clearer.

I took your suggestion.

Thanks

Eric

Quoted 7 lines

Thanks,
Alex

Quoted 3 lines
Documentation/virt/kvm/devices/arm-vgic-v3.rst +0 -0
     KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES
       save all LPI pending bits into guest RAM pending tables.
 

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author
#6 lore raw

Re: [PATCH 2/9] KVM: arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION read

In reply to Alexandru Elisei

Hi Alexandru,

Quoted 15 lines · Alexandru Elisei

Hi Eric,

The patch looks correct to me. kvm_vgic_addr() masks out all the bits except index from addr, so we don't need to do it in vgic_get_common_attr():

One nitpick below.

Quoted 5 lines · Eric Auger

The doc says:
"The characteristics of a specific redistributor region can
be read by presetting the index field in the attr data. Only valid for KVM_DEV_TYPE_ARM_VGIC_V3"

Quoted 4 lines
Quoted 2 lines

Unfortunately the existing code fails to read the input attr data and thus the index always is 0.

addr is allocated on the stack, I don't think it will always be 0.

I removed this statement in the commit message. Thanks!

Eric

Quoted 19 lines

Thanks,
Alex

Quoted 16 lines
Fixes 04c110932225 ("KVM: arm/arm64: Implement KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION")
---
 arch/arm64/kvm/vgic/vgic-kvm-device.c | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/kvm/vgic/vgic-kvm-device.c +0 -0
index 44419679f91a..2f66cf247282 100644
@@ -226,6 +226,9 @@ static int vgic_get_common_attr(struct kvm_device *dev,
 		u64 addr;
 		unsigned long type = (unsigned long)attr->attr;
 
Quoted 7 lines

arch/arm64/kvm/vgic/vgic-kvm-device.c static int vgic_get_common_attr(struct kvm_device *dev, · line 234

Quoted 6 lines
arch/arm64/kvm/vgic/vgic-kvm-device.c +3 -0
+		if (copy_from_user(&addr, uaddr, sizeof(addr)))
+			return -EFAULT;
+
 		r = kvm_vgic_addr(dev->kvm, type, &addr, false);
 		if (r)
 			return (r == -ENODEV) ? -ENXIO : r;

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author
#12 lore raw

Re: [PATCH 4/9] KVM: arm/arm64: vgic: Reset base address on kvm_vgic_dist_destroy()

In reply to Marc Zyngier

Hi Marc,

Quoted 23 lines · Marc Zyngier
Quoted 21 lines · Eric Auger

On vgic_dist_destroy(), the addresses are not reset. However for kvm selftest purpose this would allow to continue the test execution even after a failure when running KVM_RUN. So let's reset the base addresses.

---
 arch/arm64/kvm/vgic/vgic-init.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
arch/arm64/kvm/vgic/vgic-init.c +1 -0
index 32e32d67a127..6147bed56b1b 100644
@@ -335,14 +335,16 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
 	kfree(dist->spis);
 	dist->spis = NULL;
 	dist->nr_spis = 0;
+	dist->vgic_dist_base = VGIC_ADDR_UNDEF;
 
Quoted 14 lines

arch/arm64/kvm/vgic/vgic-init.c static void kvm_vgic_dist_destroy(struct kvm *kvm) · line 347

Quoted 10 lines
arch/arm64/kvm/vgic/vgic-init.c +3 -2
-	if (kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
+	if (dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
 		list_for_each_entry_safe(rdreg, next, &dist->rd_regions, list) {
 			list_del(&rdreg->list);
 			kfree(rdreg);
 		}
 		INIT_LIST_HEAD(&dist->rd_regions);
-	}
+	} else
+		kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;

Since you have converted the hunk above to use dist->, you could do the same thing here. And the coding style dictates that you need {} on the else side as well.

sure

Thanks

Eric

Quoted 19 lines
Quoted 16 lines
arch/arm64/kvm/vgic/vgic-init.c +1 -0
 	if (vgic_has_its(kvm))
 		vgic_lpi_translation_cache_destroy(kvm);
@@ -362,6 +364,7 @@ void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
 	vgic_flush_pending_lpis(vcpu);
 
 	INIT_LIST_HEAD(&vgic_cpu->ap_list_head);
+	vgic_cpu->rd_iodev.base_addr = VGIC_ADDR_UNDEF;
 }
 
 /* To be called with kvm->lock held */
Signature · 1 line
2.21.3

Thanks,

Quoted 2 lines
	M.

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author
#9 lore raw

Re: [PATCH 3/9] KVM: arm64: vgic-v3: Fix error handling in vgic_v3_set_redist_base()

In reply to Marc Zyngier

Hi Marc,

Quoted 10 lines · Marc Zyngier

Hi Eric,

Quoted 3 lines · Eric Auger

vgic_register_all_redist_iodevs may succeed while vgic_register_all_redist_iodevs fails. For example this can happen

The same function cannot both fail and succeed ;-) Can you shed some light on what you had in mind?

Damn, I meant vgic_v3_insert_redist_region() can be successful and then vgic_register_all_redist_iodevs() fails due to detection of overlap.

Quoted 13 lines
Quoted 13 lines

while adding a redistributor region overlapping a dist region. The failure only is detected on vgic_register_all_redist_iodevs when vgic_v3_check_base() gets called.

In such a case, remove the newly added redistributor region and free it.

---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
Quoted 16 lines

arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count) · line 872

Quoted 13 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +4 -1
index 8e8a862def76..581f0f490000 100644
@@ -866,8 +866,14 @@ int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count)
 	 * afterwards will register the iodevs when needed.
 	 */
 	ret = vgic_register_all_redist_iodevs(kvm);
-	if (ret)
+	if (ret) {
+		struct vgic_redist_region *rdreg =
+			vgic_v3_rdist_region_from_index(kvm, index);
+

nit: consider splitting declaration and assignment so that we avoid the line split if you insist on the 80 character limit.

Sure

Thanks

Eric

Quoted 15 lines
Quoted 12 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +3 -0
+		list_del(&rdreg->list);
+		kfree(rdreg);
 		return ret;
+	}
 
 	return 0;
 }
Signature · 1 line
2.21.3

Thanks,

Quoted 2 lines
	M.

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author
#15 lore raw

Re: [PATCH 5/9] KVM: arm: move has_run_once after the map_resources

In reply to Alexandru Elisei

Hi Alexandru,

Quoted 37 lines · Alexandru Elisei

Hi Eric,

Quoted 27 lines · Eric Auger

has_run_once is set to true at the beginning of kvm_vcpu_first_run_init(). This generally is not an issue except when exercising the code with KVM selftests. Indeed, if kvm_vgic_map_resources() fails due to erroneous user settings, has_run_once is set and this prevents from continuing executing the test. This patch moves the assignment after the kvm_vgic_map_resources().

---
 arch/arm64/kvm/arm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/kvm/arm.c +0 -2
index c0ffb019ca8b..331fae6bff31 100644
@@ -540,8 +540,6 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 	if (!kvm_arm_vcpu_is_finalized(vcpu))
 		return -EPERM;
 
-	vcpu->arch.has_run_once = true;
-
 	if (likely(irqchip_in_kernel(kvm))) {
 		/*
 		 * Map the VGIC hardware resources before running a vcpu the

arch/arm64/kvm/arm.c static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu) · line 561

Quoted 4 lines
arch/arm64/kvm/arm.c +1 -0
@@ -560,6 +558,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 		static_branch_inc(&userspace_irqchip_in_use);
 	}
 
+	vcpu->arch.has_run_once = true;

I have a few concerns regarding this:

Quoted 5 lines

1. Moving has_run_once = true here seems very arbitrary to me - kvm_timer_enable() and kvm_arm_pmu_v3_enable(), below it, can both fail because of erroneous user values. If there's a reason why the assignment cannot be moved at the end of the function, I think it should be clearly stated in a comment for the people who might be tempted to write similar tests for the timer or pmu.

Setting has_run_once = true at the entry of the function looks to me even more arbitrary. I agree with you that eventually has_run_once may be moved at the very end but maybe this can be done later once timer, pmu tests haven ben written

Quoted 6 lines

2. There are many ways that kvm_vgic_map_resources() can fail, other than incorrect user settings. I started digging into how kvm_vgic_map_resources()->vgic_v2_map_resources() can fail for a VGIC V2 and this is what I managed to find before I gave up:

Quoted 10 lines

* vgic_init() can fail in:
    - kvm_vgic_dist_init()
    - vgic_v3_init()
    - kvm_vgic_setup_default_irq_routing()
* vgic_register_dist_iodev() can fail in:
    - vgic_v3_init_dist_iodev()
    - kvm_io_bus_register_dev()(*)
* kvm_phys_addr_ioremap() can fail in:
    - kvm_mmu_topup_memory_cache()
    - kvm_pgtable_stage2_map()

I changed the commit msg so that "incorrect user settings" sounds as an example.

Quoted 3 lines

So if any of the functions below fail, are we 100% sure it is safe to allow the user to execute kvm_vgic_map_resources() again?

I think additional tests will confirm this. However at the moment, moving the assignment, which does not look wrong to me, allows to greatly simplify the tests so I would tend to say that it is worth.

Quoted 5 lines

(*) It looks to me like kvm_io_bus_register_dev() doesn't take into account a caller that tries to register the same device address range and it will create another identical range. Is this intentional? Is it a bug that should be fixed? Or am I misunderstanding the function?

doesn't kvm_io_bus_cmp() do the check?

Thanks

Eric

Quoted 8 lines

Thanks,
Alex

Quoted 4 lines
arch/arm64/kvm/arm.c +1 -0
+
 	ret = kvm_timer_enable(vcpu);
 	if (ret)
 		return ret;

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author
#3 lore raw

Re: [PATCH 1/9] KVM: arm64: vgic-v3: Fix some error codes when setting RDIST base

In reply to Alexandru Elisei

Hi Alexandru,

Quoted 43 lines · Alexandru Elisei

Hi Eric,

Quoted 20 lines · Eric Auger

KVM_DEV_ARM_VGIC_GRP_ADDR group doc says we should return -EEXIST in case the base address of the redist is already set. We currently return -EINVAL.

However we need to return -EINVAL in case a legacy REDIST address is attempted to be set while REDIST_REGIONS were set. This case is discriminated by looking at the count field.

---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0 -0
index 15a6c98ee92f..8e8a862def76 100644
@@ -792,8 +792,13 @@ static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index,
 	int ret;
 

arch/arm64/kvm/vgic/vgic-mmio-v3.c static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index, · line 801

Quoted 10 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +7 -2
 	/* single rdist region already set ?*/
-	if (!count && !list_empty(rd_regions))
-		return -EINVAL;
+	if (!count && !list_empty(rd_regions)) {
+		rdreg = list_last_entry(rd_regions,
+				       struct vgic_redist_region, list);
+		if (rdreg->count)
+			return -EINVAL; /* Mixing REDIST and REDIST_REGION API */
+		return -EEXIST;
+	}

A few instructions below:

    if (list_empty(rd_regions)) {
        [..]
    } else {
        rdreg = list_last_entry(rd_regions, struct vgic_redist_region, list); [..]

Quoted 7 lines

        /* Cannot add an explicitly sized regions after legacy region */ if (!rdreg->count)
            return -EINVAL;
    }

Isn't this testing for the same thing, but using the opposite condition? Or am I misunderstanding the code (quite likely)?

the 1st test sequence handles the case where the legacy KVM_VGIC_V3_ADDR_TYPE_REDIST is used (!count) while the second handles the case where the REDIST_REGION is used. Nevertheless I think this can be simplified into:

        if (list_empty(rd_regions)) {
                if (index != 0)
                        return -EINVAL;
        } else {
                rdreg = list_last_entry(rd_regions,
                                        struct vgic_redist_region, list);

                if ((!count) != (!rdreg->count))
                        return -EINVAL; /* Mix REDIST and REDIST_REGION */

                if (!count)
                        return -EEXIST;

                if (index != rdreg->index + 1)
                        return -EINVAL;
        }
Quoted 6 lines

Looks to me like KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST{,_REGION}) used to return -EEXIST (from vgic_check_ioaddr()) before commit ccc27bf5be7b7 ("KVM: arm/arm64: Helper to register a new redistributor region") which added the vgic_v3_insert_redist_region() function, so bringing back the -EEXIST return code looks the right thing to me.

OK thank you for the detailed study.

Eric

Quoted 7 lines

Thanks,
Alex

Quoted 3 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0 -0
 	/* cross the end of memory ? */
 	if (base + size < base)

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author
#27 lore raw

Re: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace

In reply to Alexandru Elisei

Hi Alexandru,

Quoted 11 lines · Alexandru Elisei

Hi Eric,

Quoted 7 lines · Eric Auger

Commit 23bde34771f1 ("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access the GICR_TYPER register before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.

Quoted 2 lines

I suppose the reason for emulating GICR_TYPER.Last is for architecture compliance, right? I think that should be in the commit message.

OK added this in the commit msg.

Quoted 41 lines
Quoted 33 lines
---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++-
 include/kvm/arm_vgic.h             | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +5 -1
index 581f0f490000..2f9ef6058f6e 100644
@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 						 gpa_t addr, unsigned int len)
 {
 	unsigned long mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
+	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+	struct vgic_redist_region *rdreg = vgic_cpu->rdreg;
 	int target_vcpu_id = vcpu->vcpu_id;
 	u64 value;
 
@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 	if (vgic_has_its(vcpu->kvm))
 		value |= GICR_TYPER_PLPIS;
 
-	/* reporting of the Last bit is not supported for userspace */
+	if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index - 1)))
+		value |= GICR_TYPER_LAST;
+
 	return extract_bytes(value, addr & 7, len);
 }
 

arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_register_redist_iodev(struct kvm_vcpu *vcpu) · line 721

Quoted 4 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +1 -0
@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 
 	vgic_cpu->rdreg = rdreg;
+	vgic_cpu->rdreg_index = rdreg->free_index;

What happens if the next redistributor region we register has the base address adjacent to this one?

Quoted 5 lines

I'm really not familiar with the code, but is it not possible to create two Redistributor regions (via KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST)) where the second Redistributor region start address is immediately after the last Redistributor in the preceding region?

KVM_VGIC_V3_ADDR_TYPE_REDIST only allows to create a single rdist region. Only KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION allows to register several of them.

with KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION, it is possible to register adjacent rdist regions. vgic_v3_rdist_free_slot() previously returned the 1st rdist region where enough space remains for inserting the new reg. We put the rdist at the free index there.

But maybe I misunderstood your question?

Thanks

Eric

Quoted 16 lines

Thanks,
Alex

Quoted 13 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0 -0
 	rd_base = rdreg->base + rdreg->free_index * KVM_VGIC_V3_REDIST_SIZE;
 
include/kvm/arm_vgic.h +1 -0
index a8d8fdcd3723..596c069263a7 100644
@@ -322,6 +322,7 @@ struct vgic_cpu {
 	 */
 	struct vgic_io_device	rd_iodev;
 	struct vgic_redist_region *rdreg;
+	u32 rdreg_index;
 
Quoted 3 lines

include/kvm/arm_vgic.h struct vgic_cpu { · line 328

Quoted 2 lines
include/kvm/arm_vgic.h +0 -0
 	/* Contains the attributes and gpa of the LPI pending tables. */
 	u64 pendbaser;

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

#16 lore raw

Re: [PATCH 5/9] KVM: arm: move has_run_once after the map_resources

In reply to Auger Eric

Hi Eric,

Quoted 39 lines · Auger Eric

Hi Alexandru,

Quoted 36 lines · Alexandru Elisei

Hi Eric,

Quoted 27 lines · Eric Auger

has_run_once is set to true at the beginning of kvm_vcpu_first_run_init(). This generally is not an issue except when exercising the code with KVM selftests. Indeed, if kvm_vgic_map_resources() fails due to erroneous user settings, has_run_once is set and this prevents from continuing executing the test. This patch moves the assignment after the kvm_vgic_map_resources().

---
 arch/arm64/kvm/arm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/kvm/arm.c +0 -2
index c0ffb019ca8b..331fae6bff31 100644
@@ -540,8 +540,6 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 	if (!kvm_arm_vcpu_is_finalized(vcpu))
 		return -EPERM;
 
-	vcpu->arch.has_run_once = true;
-
 	if (likely(irqchip_in_kernel(kvm))) {
 		/*
 		 * Map the VGIC hardware resources before running a vcpu the

arch/arm64/kvm/arm.c static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu) · line 561

Quoted 4 lines
arch/arm64/kvm/arm.c +1 -0
@@ -560,6 +558,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 		static_branch_inc(&userspace_irqchip_in_use);
 	}
 
+	vcpu->arch.has_run_once = true;

I have a few concerns regarding this:

Quoted 7 lines
Quoted 5 lines

1. Moving has_run_once = true here seems very arbitrary to me - kvm_timer_enable() and kvm_arm_pmu_v3_enable(), below it, can both fail because of erroneous user values. If there's a reason why the assignment cannot be moved at the end of the function, I think it should be clearly stated in a comment for the people who might be tempted to write similar tests for the timer or pmu.

Setting has_run_once = true at the entry of the function looks to me even more arbitrary. I agree with you that eventually has_run_once may

Or it could be it's there to prevent the user from calling kvm_vgic_map_resources() a second time after it failed. This is what I'm concerned about and I think deserves more investigation.

Thanks,
Alex

Quoted 28 lines

be moved at the very end but maybe this can be done later once timer, pmu tests haven ben written

Quoted 5 lines

2. There are many ways that kvm_vgic_map_resources() can fail, other than incorrect user settings. I started digging into how kvm_vgic_map_resources()->vgic_v2_map_resources() can fail for a VGIC V2 and this is what I managed to find before I gave up:

Quoted 10 lines

* vgic_init() can fail in:
    - kvm_vgic_dist_init()
    - vgic_v3_init()
    - kvm_vgic_setup_default_irq_routing()
* vgic_register_dist_iodev() can fail in:
    - vgic_v3_init_dist_iodev()
    - kvm_io_bus_register_dev()(*)
* kvm_phys_addr_ioremap() can fail in:
    - kvm_mmu_topup_memory_cache()
    - kvm_pgtable_stage2_map()

I changed the commit msg so that "incorrect user settings" sounds as an example.

Quoted 2 lines

So if any of the functions below fail, are we 100% sure it is safe to allow the user to execute kvm_vgic_map_resources() again?

I think additional tests will confirm this. However at the moment, moving the assignment, which does not look wrong to me, allows to greatly simplify the tests so I would tend to say that it is worth.

Quoted 4 lines

(*) It looks to me like kvm_io_bus_register_dev() doesn't take into account a caller that tries to register the same device address range and it will create another identical range. Is this intentional? Is it a bug that should be fixed? Or am I misunderstanding the function?

Quoted 11 lines

doesn't kvm_io_bus_cmp() do the check?

Thanks

Eric

Quoted 6 lines

Thanks,
Alex

Quoted 4 lines
arch/arm64/kvm/arm.c +1 -0
+
 	ret = kvm_timer_enable(vcpu);
 	if (ret)
 		return ret;

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

#28 lore raw

Re: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace

In reply to Auger Eric

Hi Eric,

Quoted 63 lines · Auger Eric

Hi Alexandru,

Quoted 10 lines · Alexandru Elisei

Hi Eric,

Quoted 7 lines · Eric Auger

Commit 23bde34771f1 ("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access the GICR_TYPER register before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.

Quoted 2 lines

I suppose the reason for emulating GICR_TYPER.Last is for architecture compliance, right? I think that should be in the commit message.

OK added this in the commit msg.

Quoted 38 lines
Quoted 31 lines
---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++-
 include/kvm/arm_vgic.h             | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +5 -1
index 581f0f490000..2f9ef6058f6e 100644
@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 						 gpa_t addr, unsigned int len)
 {
 	unsigned long mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
+	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+	struct vgic_redist_region *rdreg = vgic_cpu->rdreg;
 	int target_vcpu_id = vcpu->vcpu_id;
 	u64 value;
 
@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 	if (vgic_has_its(vcpu->kvm))
 		value |= GICR_TYPER_PLPIS;
 
-	/* reporting of the Last bit is not supported for userspace */
+	if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index - 1)))
+		value |= GICR_TYPER_LAST;
+
 	return extract_bytes(value, addr & 7, len);
 }
 

arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_register_redist_iodev(struct kvm_vcpu *vcpu) · line 721

Quoted 4 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +1 -0
@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 
 	vgic_cpu->rdreg = rdreg;
+	vgic_cpu->rdreg_index = rdreg->free_index;

What happens if the next redistributor region we register has the base address adjacent to this one?

Quoted 5 lines

I'm really not familiar with the code, but is it not possible to create two Redistributor regions (via KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST)) where the second Redistributor region start address is immediately after the last Redistributor in the preceding region?

KVM_VGIC_V3_ADDR_TYPE_REDIST only allows to create a single rdist region. Only KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION allows to register several of them.

Quoted 6 lines

with KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION, it is possible to register adjacent rdist regions. vgic_v3_rdist_free_slot() previously returned the 1st rdist region where enough space remains for inserting the new reg. We put the rdist at the free index there.

But maybe I misunderstood your question?

Yes, I think you did a good job at answering my poorly worded question.

This is the case I am concerned about:

1. Userspace sets first redistributor base address to 0x0 via KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION(count = 1, base = 0x0, flags = 0, index = 0).

2. Userspace sets first redistributor base address to 0x0 + 128K, immediately following the previous Redistributor.

In that case the two Redistributors will be represented by two separate struct vgic_redist_region, but they are adjacent to one another and represent one contiguous memory region.

From what I understand from your patch, GICR_TYPER.Last will be set for both Redistributors, when it should be set only for the second Redistributor. Does any of that make sense?

Thanks,
Alex

Quoted 19 lines

Thanks

Eric

Quoted 15 lines

Thanks,
Alex

Quoted 13 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0 -0
 	rd_base = rdreg->base + rdreg->free_index * KVM_VGIC_V3_REDIST_SIZE;
 
include/kvm/arm_vgic.h +1 -0
index a8d8fdcd3723..596c069263a7 100644
@@ -322,6 +322,7 @@ struct vgic_cpu {
 	 */
 	struct vgic_io_device	rd_iodev;
 	struct vgic_redist_region *rdreg;
+	u32 rdreg_index;
 
Quoted 2 lines
Quoted 2 lines

include/kvm/arm_vgic.h struct vgic_cpu { · line 328

Quoted 2 lines
include/kvm/arm_vgic.h +0 -0
 	/* Contains the attributes and gpa of the LPI pending tables. */
 	u64 pendbaser;

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author
#29 lore raw

Re: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace

Hi Alexandru,

Quoted 83 lines · Alexandru Elisei

Hi Eric,

Quoted 63 lines · Auger Eric

Hi Alexandru,

Quoted 10 lines · Alexandru Elisei

Hi Eric,

Quoted 7 lines · Eric Auger

Commit 23bde34771f1 ("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access the GICR_TYPER register before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.

Quoted 2 lines

I suppose the reason for emulating GICR_TYPER.Last is for architecture compliance, right? I think that should be in the commit message.

OK added this in the commit msg.

Quoted 38 lines
Quoted 31 lines
---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++-
 include/kvm/arm_vgic.h             | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +5 -1
index 581f0f490000..2f9ef6058f6e 100644
@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 						 gpa_t addr, unsigned int len)
 {
 	unsigned long mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
+	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+	struct vgic_redist_region *rdreg = vgic_cpu->rdreg;
 	int target_vcpu_id = vcpu->vcpu_id;
 	u64 value;
 
@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
 	if (vgic_has_its(vcpu->kvm))
 		value |= GICR_TYPER_PLPIS;
 
-	/* reporting of the Last bit is not supported for userspace */
+	if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index - 1)))
+		value |= GICR_TYPER_LAST;
+
 	return extract_bytes(value, addr & 7, len);
 }
 

arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_register_redist_iodev(struct kvm_vcpu *vcpu) · line 721

Quoted 4 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +1 -0
@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 
 	vgic_cpu->rdreg = rdreg;
+	vgic_cpu->rdreg_index = rdreg->free_index;

What happens if the next redistributor region we register has the base address adjacent to this one?

Quoted 5 lines

I'm really not familiar with the code, but is it not possible to create two Redistributor regions (via KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST)) where the second Redistributor region start address is immediately after the last Redistributor in the preceding region?

KVM_VGIC_V3_ADDR_TYPE_REDIST only allows to create a single rdist region. Only KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION allows to register several of them.

Quoted 6 lines

with KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION, it is possible to register adjacent rdist regions. vgic_v3_rdist_free_slot() previously returned the 1st rdist region where enough space remains for inserting the new reg. We put the rdist at the free index there.

But maybe I misunderstood your question?

Yes, I think you did a good job at answering my poorly worded question.

This is the case I am concerned about:

1. Userspace sets first redistributor base address to 0x0 via KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION(count = 1, base = 0x0, flags = 0, index = 0).

2. Userspace sets first redistributor base address to 0x0 + 128K, immediately following the previous Redistributor.

Quoted 7 lines

In that case the two Redistributors will be represented by two separate struct vgic_redist_region, but they are adjacent to one another and represent one contiguous memory region.

From what I understand from your patch, GICR_TYPER.Last will be set for both Redistributors, when it should be set only for the second Redistributor. Does any of that make sense?

Please forgive me for not having replied before on this thread.

This is a valid concern. Nothing prevents the redistributor regions from being contiguous although this is not the goal. Also nothing prevents vcpu rdists to be laid out within a redist region in non ascending order. Also redist regions with ascending indices may not have increasing base addresses.

So this becomes a gas factory for emulating a single bit but I have reworked this in v3 ;-)

Thanks

Eric

Quoted 22 lines

Thanks,
Alex

Quoted 19 lines

Thanks

Eric

Quoted 15 lines

Thanks,
Alex

Quoted 13 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0 -0
 	rd_base = rdreg->base + rdreg->free_index * KVM_VGIC_V3_REDIST_SIZE;
 
include/kvm/arm_vgic.h +1 -0
index a8d8fdcd3723..596c069263a7 100644
@@ -322,6 +322,7 @@ struct vgic_cpu {
 	 */
 	struct vgic_io_device	rd_iodev;
 	struct vgic_redist_region *rdreg;
+	u32 rdreg_index;
 
Quoted 3 lines
Quoted 2 lines
Quoted 2 lines

include/kvm/arm_vgic.h struct vgic_cpu { · line 328

Quoted 2 lines
include/kvm/arm_vgic.h +0 -0
 	/* Contains the attributes and gpa of the LPI pending tables. */
 	u64 pendbaser;

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author
#17 lore raw

Re: [PATCH 5/9] KVM: arm: move has_run_once after the map_resources

In reply to Alexandru Elisei

Hi Alexandru,

Quoted 47 lines · Alexandru Elisei

Hi Eric,

Quoted 44 lines · Auger Eric

Hi Alexandru,

Quoted 41 lines · Alexandru Elisei

Hi Eric,

Quoted 27 lines · Eric Auger

has_run_once is set to true at the beginning of kvm_vcpu_first_run_init(). This generally is not an issue except when exercising the code with KVM selftests. Indeed, if kvm_vgic_map_resources() fails due to erroneous user settings, has_run_once is set and this prevents from continuing executing the test. This patch moves the assignment after the kvm_vgic_map_resources().

---
 arch/arm64/kvm/arm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/kvm/arm.c +0 -2
index c0ffb019ca8b..331fae6bff31 100644
@@ -540,8 +540,6 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 	if (!kvm_arm_vcpu_is_finalized(vcpu))
 		return -EPERM;
 
-	vcpu->arch.has_run_once = true;
-
 	if (likely(irqchip_in_kernel(kvm))) {
 		/*
 		 * Map the VGIC hardware resources before running a vcpu the

arch/arm64/kvm/arm.c static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu) · line 561

Quoted 4 lines
arch/arm64/kvm/arm.c +1 -0
@@ -560,6 +558,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
 		static_branch_inc(&userspace_irqchip_in_use);
 	}
 
+	vcpu->arch.has_run_once = true;

I have a few concerns regarding this:

1. Moving has_run_once = true here seems very arbitrary to me - kvm_timer_enable() and kvm_arm_pmu_v3_enable(), below it, can both fail because of erroneous user values. If there's a reason why the assignment cannot be moved at the end of the function, I think it should be clearly stated in a comment for the people who might be tempted to write similar tests for the timer or pmu.

Quoted 6 lines
Quoted 2 lines

Setting has_run_once = true at the entry of the function looks to me even more arbitrary. I agree with you that eventually has_run_once may

Or it could be it's there to prevent the user from calling kvm_vgic_map_resources() a second time after it failed. This is what I'm concerned about and I think deserves more investigation.

I have reworked my kvm selftests to live without that change.

Thanks

Eric

Quoted 43 lines

Thanks,
Alex

Quoted 39 lines

be moved at the very end but maybe this can be done later once timer, pmu tests haven ben written

Quoted 5 lines

2. There are many ways that kvm_vgic_map_resources() can fail, other than incorrect user settings. I started digging into how kvm_vgic_map_resources()->vgic_v2_map_resources() can fail for a VGIC V2 and this is what I managed to find before I gave up:

Quoted 10 lines

* vgic_init() can fail in:
    - kvm_vgic_dist_init()
    - vgic_v3_init()
    - kvm_vgic_setup_default_irq_routing()
* vgic_register_dist_iodev() can fail in:
    - vgic_v3_init_dist_iodev()
    - kvm_io_bus_register_dev()(*)
* kvm_phys_addr_ioremap() can fail in:
    - kvm_mmu_topup_memory_cache()
    - kvm_pgtable_stage2_map()

I changed the commit msg so that "incorrect user settings" sounds as an example.

Quoted 2 lines

So if any of the functions below fail, are we 100% sure it is safe to allow the user to execute kvm_vgic_map_resources() again?

I think additional tests will confirm this. However at the moment, moving the assignment, which does not look wrong to me, allows to greatly simplify the tests so I would tend to say that it is worth.

Quoted 4 lines

(*) It looks to me like kvm_io_bus_register_dev() doesn't take into account a caller that tries to register the same device address range and it will create another identical range. Is this intentional? Is it a bug that should be fixed? Or am I misunderstanding the function?

doesn't kvm_io_bus_cmp() do the check?

Thanks

Eric

Quoted 6 lines

Thanks,
Alex

Quoted 4 lines
arch/arm64/kvm/arm.c +1 -0
+
 	ret = kvm_timer_enable(vcpu);
 	if (ret)
 		return ret;

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm