KVM/ARM: Some vgic fixes and init sequence KVM selftests
Subsystems (5)
v2 cover KVM/ARM: Some vgic fixes and init sequence KVM selftests
While writting vgic v3 init sequence KVM selftests I noticed some relatively minor issues. This was also the opportunity to try to fix the issue laterly reported by Zenghui, related to the RDIST_TYPER last bit emulation. The final patch is a first batch of VGIC init sequence selftests. Of course they can be augmented with a lot more register access tests, but let's try to move forward incrementally ...
Best Regards
Eric
This series can be found at:
https://github.com/eauger/linux/tree/vgic_kvmselftests_v2
History:
- Took into account all comments from Marc and Alexandru's except the has_run_once still after the map_resources (this would oblige me to revisit in depth the selftests)
Eric Auger (9):
KVM: arm64: vgic-v3: Fix some error codes when setting RDIST base
KVM: arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION read
KVM: arm64: vgic-v3: Fix error handling in vgic_v3_set_redist_base()
KVM: arm/arm64: vgic: Reset base address on kvm_vgic_dist_destroy()
KVM: arm: move has_run_once after the map_resources
docs: kvm: devices/arm-vgic-v3: enhance KVM_DEV_ARM_VGIC_CTRL_INIT doc
KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]
KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace
KVM: selftests: aarch64/vgic-v3 init sequence tests
.../virt/kvm/devices/arm-vgic-v3.rst | 2 +- arch/arm64/kvm/arm.c | 4 +- arch/arm64/kvm/vgic/vgic-init.c | 6 +- arch/arm64/kvm/vgic/vgic-kvm-device.c | 3 + arch/arm64/kvm/vgic/vgic-mmio-v3.c | 30 +- arch/arm64/kvm/vgic/vgic-mmio.c | 10 +- include/kvm/arm_vgic.h | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../testing/selftests/kvm/aarch64/vgic_init.c | 453 ++++++++++++++++++ .../testing/selftests/kvm/include/kvm_util.h | 5 + tools/testing/selftests/kvm/lib/kvm_util.c | 51 ++ 11 files changed, 546 insertions(+), 20 deletions(-) create mode 100644 tools/testing/selftests/kvm/aarch64/vgic_init.c
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 1/9 KVM: arm64: vgic-v3: Fix some error codes when setting RDIST base
KVM_DEV_ARM_VGIC_GRP_ADDR group doc says we should return -EEXIST in case the base address of the redist is already set. We currently return -EINVAL.
However we need to return -EINVAL in case a legacy REDIST address is attempted to be set while REDIST_REGIONS were set. This case is discriminated by looking at the count field.
---
v1 -> v2:
- simplify the check sequence
--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +7 -8
@@ -791,10 +791,6 @@ static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index,size_t size = count *KVM_VGIC_V3_REDIST_SIZE ;int ret; -/* single rdist region already set ?*/ -if (!count && !list_empty (rd_regions)) -return -EINVAL ; -/* cross the end of memory ? */ if (base + size < base)return -EINVAL ;
@@ -805,11 +801,14 @@ static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index, }else { rdreg =list_last_entry (rd_regions, struct vgic_redist_region, list); -if (index != rdreg->index +1 ) -return -EINVAL ; -/* Cannot add an explicitly sized regions after legacy region */ -if (!rdreg->count) +if ((!count) != (!rdreg->count)) +return -EINVAL ;/* Mix REDIST and REDIST_REGION */ + +if (!count) +return -EEXIST ; + +if (index != rdreg->index +1 )return -EINVAL ; }
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 2/9 KVM: arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION read
In reply to Eric AugerThe doc says:
"The characteristics of a specific redistributor region can
be read by presetting the index field in the attr data. Only valid for KVM_DEV_TYPE_ARM_VGIC_V3"
Unfortunately the existing code fails to read the input attr data.
---
v1 -> v2:
- in the commit message, remove the statement that the index always is 0
- add Alexandru's R-b
--- arch/arm64/kvm/vgic/vgic-kvm-device.c | 3 +++ 1 file changed, 3 insertions(+)
arch/arm64/kvm/vgic/vgic-kvm-device.c +3 -0
@@ -226,6 +226,9 @@ static int vgic_get_common_attr(struct kvm_device *dev,u64 addr;unsigned long type = (unsigned long )attr->attr; +if (copy_from_user (&addr, uaddr,sizeof (addr))) +return -EFAULT ; + r =kvm_vgic_addr (dev->kvm, type, &addr, false);if (r)return (r == -ENODEV ) ? -ENXIO : r;
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 3/9 KVM: arm64: vgic-v3: Fix error handling in vgic_v3_set_redist_base()
In reply to Eric Augervgic_v3_insert_redist_region() may succeed while vgic_register_all_redist_iodevs fails. For example this happens while adding a redistributor region overlapping a dist region. The failure only is detected on vgic_register_all_redist_iodevs when vgic_v3_check_base() gets called in vgic_register_redist_iodev().
In such a case, remove the newly added redistributor region and free it.
---
v1 -> v2:
- fix the commit message and split declaration/assignment of rdreg
--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +7 -1
@@ -860,8 +860,14 @@ int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count)* afterwards will register the iodevs when needed. */ ret =vgic_register_all_redist_iodevs (kvm); -if (ret) +if (ret) { +struct vgic_redist_region *rdreg; + + rdreg =vgic_v3_rdist_region_from_index (kvm, index); +list_del (&rdreg->list); +kfree (rdreg);return ret; + }return 0 ; }
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 4/9 KVM: arm/arm64: vgic: Reset base address on kvm_vgic_dist_destroy()
In reply to Eric AugerOn vgic_dist_destroy(), the addresses are not reset. However for kvm selftest purpose this would allow to continue the test execution even after a failure when running KVM_RUN. So let's reset the base addresses.
---
v1 -> v2:
- use dist-> in the else and add braces
--- arch/arm64/kvm/vgic/vgic-init.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-init.c +5 -1
@@ -335,13 +335,16 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)kfree (dist->spis); dist->spis =NULL ; dist->nr_spis =0 ; + dist->vgic_dist_base =VGIC_ADDR_UNDEF ; -if (kvm->arch.vgic.vgic_model ==KVM_DEV_TYPE_ARM_VGIC_V3 ) { +if (dist->vgic_model ==KVM_DEV_TYPE_ARM_VGIC_V3 ) {list_for_each_entry_safe (rdreg, next, &dist->rd_regions, list) {list_del (&rdreg->list);kfree (rdreg); }INIT_LIST_HEAD (&dist->rd_regions); + }else { + dist->vgic_cpu_base =VGIC_ADDR_UNDEF ; }if (vgic_has_its (kvm))
@@ -362,6 +365,7 @@ void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)vgic_flush_pending_lpis (vcpu);INIT_LIST_HEAD (&vgic_cpu->ap_list_head); + vgic_cpu->rd_iodev.base_addr =VGIC_ADDR_UNDEF ; }/* To be called with kvm->lock held */
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 5/9 KVM: arm: move has_run_once after the map_resources
In reply to Eric Augerhas_run_once is set to true at the beginning of kvm_vcpu_first_run_init(). This generally is not an issue except when exercising the code with KVM selftests. for instance, if kvm_vgic_map_resources() fails due to erroneous user settings, has_run_once is set and this prevents from continuing executing the test. This patch moves the assignment after the kvm_vgic_map_resources().
---
v1 -> v2:
- slight reword of the commit msg (for instance)
--- arch/arm64/kvm/arm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/kvm/arm.c +2 -2
@@ -573,8 +573,6 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)if (!kvm_arm_vcpu_is_finalized (vcpu))return -EPERM ; - vcpu->arch.has_run_once = true; -if (likely (irqchip_in_kernel (kvm))) {/* * Map the VGIC hardware resources before running a vcpu the
@@ -591,6 +589,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)static_branch_inc (&userspace_irqchip_in_use); } + vcpu->arch.has_run_once = true; + ret =kvm_timer_enable (vcpu);if (ret)return ret;
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 6/9 docs: kvm: devices/arm-vgic-v3: enhance KVM_DEV_ARM_VGIC_CTRL_INIT doc
In reply to Eric Augerkvm_arch_vcpu_precreate() returns -EBUSY if the vgic is already initialized. So let's document that KVM_DEV_ARM_VGIC_CTRL_INIT must be called after all vcpu creations.
---
v1 -> v2:
- Must be called after all vcpu creations -> Must be called after all VCPUs have been created as per Alexandru's suggestion
--- Documentation/virt/kvm/devices/arm-vgic-v3.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/virt/kvm/devices/arm-vgic-v3.rst +1 -1
@@ -228,7 +228,7 @@ Groups: KVM_DEV_ARM_VGIC_CTRL_INIT request the initialization of the VGIC, no additional parameter in - kvm_device_attr.addr. + kvm_device_attr.addr. Must be called after all VCPUs have been created. KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES save all LPI pending bits into guest RAM pending tables.
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 7/9 KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]
In reply to Eric Augervgic_uaccess() takes a struct vgic_io_device argument, converts it to a struct kvm_io_device and passes it to the read/write accessor functions, which convert it back to a struct vgic_io_device. Avoid the indirection by passing the struct vgic_io_device argument directly to vgic_uaccess_{read,write}.
---
v1 -> v2:
- reworded the commit message as suggested by Alexandru
--- arch/arm64/kvm/vgic/vgic-mmio.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
arch/arm64/kvm/vgic/vgic-mmio.c +4 -6
@@ -938,10 +938,9 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,return region; } -static int vgic_uaccess_read (struct kvm_vcpu *vcpu,struct kvm_io_device *dev, +static int vgic_uaccess_read (struct kvm_vcpu *vcpu,struct vgic_io_device *iodev,gpa_t addr,u32 *val) { -struct vgic_io_device *iodev =kvm_to_vgic_iodev (dev);const struct vgic_register_region *region;struct kvm_vcpu *r_vcpu;
@@ -960,10 +959,9 @@ static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,return 0 ; } -static int vgic_uaccess_write (struct kvm_vcpu *vcpu,struct kvm_io_device *dev, +static int vgic_uaccess_write (struct kvm_vcpu *vcpu,struct vgic_io_device *iodev,gpa_t addr,const u32 *val) { -struct vgic_io_device *iodev =kvm_to_vgic_iodev (dev);const struct vgic_register_region *region;struct kvm_vcpu *r_vcpu;
@@ -986,9 +984,9 @@ int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, bool is_write, int offset, u32 *val) { if (is_write) - return vgic_uaccess_write(vcpu, &dev->dev, offset, val); + return vgic_uaccess_write(vcpu, dev, offset, val); else - return vgic_uaccess_read(vcpu, &dev->dev, offset, val); + return vgic_uaccess_read(vcpu, dev, offset, val); } static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 8/9 KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace
In reply to Eric AugerCommit 23bde34771f1 ("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access the GICR_TYPER register before the redistributor region setting but dropped the support of the LAST bit. Emulating the GICR_TYPER.Last bit still makes sense for architecture compliance.
This patch restores its support (if the redistributor region was set) while keeping the code safe.
--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++- include/kvm/arm_vgic.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +6 -1
@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu, gpa_t addr, unsigned int len) {unsigned long mpidr =kvm_vcpu_get_mpidr_aff (vcpu); +struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; +struct vgic_redist_region *rdreg = vgic_cpu->rdreg;int target_vcpu_id = vcpu->vcpu_id;u64 value;
@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,if (vgic_has_its (vcpu->kvm)) value |=GICR_TYPER_PLPIS ; -/* reporting of the Last bit is not supported for userspace */ +if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index -1 ))) + value |=GICR_TYPER_LAST ; +return extract_bytes (value, addr &7 , len); }
@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)return -EINVAL ; vgic_cpu->rdreg = rdreg; + vgic_cpu->rdreg_index = rdreg->free_index; rd_base = rdreg->base + rdreg->free_index *KVM_VGIC_V3_REDIST_SIZE ;
include/kvm/arm_vgic.h +1 -0
@@ -322,6 +322,7 @@ struct vgic_cpu {*/ struct vgic_io_device rd_iodev;struct vgic_redist_region *rdreg; +u32 rdreg_index;/* Contains the attributes and gpa of the LPI pending tables. */ u64 pendbaser;
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 9/9 KVM: selftests: aarch64/vgic-v3 init sequence tests
In reply to Eric AugerThe tests exercise the VGIC_V3 device creation including the associated KVM_DEV_ARM_VGIC_GRP_ADDR group attributes:
- KVM_VGIC_V3_ADDR_TYPE_DIST/REDIST (vcpu_first and vgic_first)
- KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION (redist_regions).
Another test dedicates to KVM_DEV_ARM_VGIC_GRP_REDIST_REGS group and especially the GICR_TYPER read. The goal was to test the case recently fixed by commit 23bde34771f1
("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace").
The API under test can be found atDocumentation/virt/kvm/devices/arm-vgic-v3.rst
--- tools/testing/selftests/kvm/Makefile | 1 + .../testing/selftests/kvm/aarch64/vgic_init.c | 453 ++++++++++++++++++ .../testing/selftests/kvm/include/kvm_util.h | 5 + tools/testing/selftests/kvm/lib/kvm_util.c | 51 ++ 4 files changed, 510 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/vgic_init.c
tools/testing/selftests/kvm/Makefile +1 -0
@@ -68,6 +68,7 @@ TEST_GEN_PROGS_x86_64 += steal_time TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list-sve +TEST_GEN_PROGS_aarch64 += aarch64/vgic_init TEST_GEN_PROGS_aarch64 += demand_paging_test TEST_GEN_PROGS_aarch64 += dirty_log_test TEST_GEN_PROGS_aarch64 += dirty_log_perf_test
tools/testing/selftests/kvm/aarch64/vgic_init.c new file +453 -0
453 lines not on this page · raw message
tools/testing/selftests/kvm/include/kvm_util.h +5 -0
@@ -229,6 +229,11 @@ int vcpu_nested_state_set(struct kvm_vm *vm, uint32_t vcpuid,#endif void *vcpu_map_dirty_ring (struct kvm_vm *vm,uint32_t vcpuid); +int kvm_device_check_attr (int dev_fd,uint32_t group,uint64_t attr); +int kvm_create_device (struct kvm_vm *vm,uint64_t type,bool test); +int kvm_device_access (int dev_fd,uint32_t group,uint64_t attr, +void *val,bool write); +const char *exit_reason_str (unsigned int exit_reason);void virt_pgd_alloc (struct kvm_vm *vm,uint32_t pgd_memslot);
tools/testing/selftests/kvm/lib/kvm_util.c +51 -0
57 lines not on this page · raw message
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm