KVM/ARM: Some vgic fixes and init sequence KVM selftests
Subsystems (5)
cover KVM/ARM: Some vgic fixes and init sequence KVM selftests
While writting vgic v3 init sequence KVM selftests I noticed some relatively minor issues. This was also the opportunity to try to fix the issue laterly reported by Zenghui, related to the RDIST_TYPER last bit emulation. The final patch is a first batch of VGIC init sequence selftests. Of course they can be augmented with a lot more register access tests, but let's try to move forward incrementally ...
Best Regards
Eric
This series can be found at:
https://github.com/eauger/linux/tree/vgic-selftests-and-fixes-v1
Eric Auger (9):
KVM: arm64: vgic-v3: Fix some error codes when setting RDIST base
KVM: arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION read
KVM: arm64: vgic-v3: Fix error handling in vgic_v3_set_redist_base()
KVM: arm/arm64: vgic: Reset base address on kvm_vgic_dist_destroy()
KVM: arm: move has_run_once after the map_resources
docs: kvm: devices/arm-vgic-v3: enhance KVM_DEV_ARM_VGIC_CTRL_INIT doc
KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]
KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace
KVM: selftests: aarch64/vgic-v3 init sequence tests
.../virt/kvm/devices/arm-vgic-v3.rst | 2 +- arch/arm64/kvm/arm.c | 4 +- arch/arm64/kvm/vgic/vgic-init.c | 7 +- arch/arm64/kvm/vgic/vgic-kvm-device.c | 3 + arch/arm64/kvm/vgic/vgic-mmio-v3.c | 24 +- arch/arm64/kvm/vgic/vgic-mmio.c | 10 +- include/kvm/arm_vgic.h | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../testing/selftests/kvm/aarch64/vgic_init.c | 453 ++++++++++++++++++ .../testing/selftests/kvm/include/kvm_util.h | 5 + tools/testing/selftests/kvm/lib/kvm_util.c | 51 ++ 11 files changed, 546 insertions(+), 15 deletions(-) create mode 100644 tools/testing/selftests/kvm/aarch64/vgic_init.c
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH 1/9 KVM: arm64: vgic-v3: Fix some error codes when setting RDIST base
KVM_DEV_ARM_VGIC_GRP_ADDR group doc says we should return -EEXIST in case the base address of the redist is already set. We currently return -EINVAL.
However we need to return -EINVAL in case a legacy REDIST address is attempted to be set while REDIST_REGIONS were set. This case is discriminated by looking at the count field.
--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +7 -2
@@ -792,8 +792,13 @@ static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index,int ret;/* single rdist region already set ?*/ -if (!count && !list_empty (rd_regions)) -return -EINVAL ; +if (!count && !list_empty (rd_regions)) { + rdreg =list_last_entry (rd_regions, + struct vgic_redist_region, list); +if (rdreg->count) +return -EINVAL ;/* Mixing REDIST and REDIST_REGION API */ +return -EEXIST ; + }/* cross the end of memory ? */ if (base + size < base)
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH 2/9 KVM: arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION read
In reply to Eric AugerThe doc says:
"The characteristics of a specific redistributor region can
be read by presetting the index field in the attr data. Only valid for KVM_DEV_TYPE_ARM_VGIC_V3"
Unfortunately the existing code fails to read the input attr data and thus the index always is 0.
--- arch/arm64/kvm/vgic/vgic-kvm-device.c | 3 +++ 1 file changed, 3 insertions(+)
arch/arm64/kvm/vgic/vgic-kvm-device.c +3 -0
@@ -226,6 +226,9 @@ static int vgic_get_common_attr(struct kvm_device *dev,u64 addr;unsigned long type = (unsigned long )attr->attr; +if (copy_from_user (&addr, uaddr,sizeof (addr))) +return -EFAULT ; + r =kvm_vgic_addr (dev->kvm, type, &addr, false);if (r)return (r == -ENODEV ) ? -ENXIO : r;
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH 3/9 KVM: arm64: vgic-v3: Fix error handling in vgic_v3_set_redist_base()
In reply to Eric Augervgic_register_all_redist_iodevs may succeed while vgic_register_all_redist_iodevs fails. For example this can happen while adding a redistributor region overlapping a dist region. The failure only is detected on vgic_register_all_redist_iodevs when vgic_v3_check_base() gets called.
In such a case, remove the newly added redistributor region and free it.
--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +7 -1
@@ -866,8 +866,14 @@ int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count)* afterwards will register the iodevs when needed. */ ret =vgic_register_all_redist_iodevs (kvm); -if (ret) +if (ret) { +struct vgic_redist_region *rdreg = +vgic_v3_rdist_region_from_index (kvm, index); + +list_del (&rdreg->list); +kfree (rdreg);return ret; + }return 0 ; }
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH 4/9 KVM: arm/arm64: vgic: Reset base address on kvm_vgic_dist_destroy()
In reply to Eric AugerOn vgic_dist_destroy(), the addresses are not reset. However for kvm selftest purpose this would allow to continue the test execution even after a failure when running KVM_RUN. So let's reset the base addresses.
--- arch/arm64/kvm/vgic/vgic-init.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
arch/arm64/kvm/vgic/vgic-init.c +5 -2
@@ -335,14 +335,16 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)kfree (dist->spis); dist->spis =NULL ; dist->nr_spis =0 ; + dist->vgic_dist_base =VGIC_ADDR_UNDEF ; -if (kvm->arch.vgic.vgic_model ==KVM_DEV_TYPE_ARM_VGIC_V3 ) { +if (dist->vgic_model ==KVM_DEV_TYPE_ARM_VGIC_V3 ) {list_for_each_entry_safe (rdreg, next, &dist->rd_regions, list) {list_del (&rdreg->list);kfree (rdreg); }INIT_LIST_HEAD (&dist->rd_regions); - } + }else + kvm->arch.vgic.vgic_cpu_base =VGIC_ADDR_UNDEF ;if (vgic_has_its (kvm))vgic_lpi_translation_cache_destroy (kvm);
@@ -362,6 +364,7 @@ void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)vgic_flush_pending_lpis (vcpu);INIT_LIST_HEAD (&vgic_cpu->ap_list_head); + vgic_cpu->rd_iodev.base_addr =VGIC_ADDR_UNDEF ; }/* To be called with kvm->lock held */
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH 5/9 KVM: arm: move has_run_once after the map_resources
In reply to Eric Augerhas_run_once is set to true at the beginning of kvm_vcpu_first_run_init(). This generally is not an issue except when exercising the code with KVM selftests. Indeed, if kvm_vgic_map_resources() fails due to erroneous user settings, has_run_once is set and this prevents from continuing executing the test. This patch moves the assignment after the kvm_vgic_map_resources().
--- arch/arm64/kvm/arm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/kvm/arm.c +2 -2
@@ -540,8 +540,6 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)if (!kvm_arm_vcpu_is_finalized (vcpu))return -EPERM ; - vcpu->arch.has_run_once = true; -if (likely (irqchip_in_kernel (kvm))) {/* * Map the VGIC hardware resources before running a vcpu the
@@ -560,6 +558,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)static_branch_inc (&userspace_irqchip_in_use); } + vcpu->arch.has_run_once = true; + ret =kvm_timer_enable (vcpu);if (ret)return ret;
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH 6/9 docs: kvm: devices/arm-vgic-v3: enhance KVM_DEV_ARM_VGIC_CTRL_INIT doc
In reply to Eric Augerkvm_arch_vcpu_precreate() returns -EBUSY if the vgic is already initialized. So let's document that KVM_DEV_ARM_VGIC_CTRL_INIT must be called after all vcpu creations.
--- Documentation/virt/kvm/devices/arm-vgic-v3.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/virt/kvm/devices/arm-vgic-v3.rst +1 -1
@@ -228,7 +228,7 @@ Groups: KVM_DEV_ARM_VGIC_CTRL_INIT request the initialization of the VGIC, no additional parameter in - kvm_device_attr.addr. + kvm_device_attr.addr. Must be called after all vcpu creations. KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES save all LPI pending bits into guest RAM pending tables.
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH 7/9 KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]
In reply to Eric AugerInstead of converting the vgic_io_device handle to a kvm_io_device handled and then do the oppositive, pass a vgic_io_device pointer all along the call chain.
--- arch/arm64/kvm/vgic/vgic-mmio.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
arch/arm64/kvm/vgic/vgic-mmio.c +4 -6
@@ -938,10 +938,9 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,return region; } -static int vgic_uaccess_read (struct kvm_vcpu *vcpu,struct kvm_io_device *dev, +static int vgic_uaccess_read (struct kvm_vcpu *vcpu,struct vgic_io_device *iodev,gpa_t addr,u32 *val) { -struct vgic_io_device *iodev =kvm_to_vgic_iodev (dev);const struct vgic_register_region *region;struct kvm_vcpu *r_vcpu;
@@ -960,10 +959,9 @@ static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,return 0 ; } -static int vgic_uaccess_write (struct kvm_vcpu *vcpu,struct kvm_io_device *dev, +static int vgic_uaccess_write (struct kvm_vcpu *vcpu,struct vgic_io_device *iodev,gpa_t addr,const u32 *val) { -struct vgic_io_device *iodev =kvm_to_vgic_iodev (dev);const struct vgic_register_region *region;struct kvm_vcpu *r_vcpu;
@@ -986,9 +984,9 @@ int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, bool is_write, int offset, u32 *val) { if (is_write) - return vgic_uaccess_write(vcpu, &dev->dev, offset, val); + return vgic_uaccess_write(vcpu, dev, offset, val); else - return vgic_uaccess_read(vcpu, &dev->dev, offset, val); + return vgic_uaccess_read(vcpu, dev, offset, val); } static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH 8/9 KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace
In reply to Eric AugerCommit 23bde34771f1 ("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access the GICR_TYPER register before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.
--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++- include/kvm/arm_vgic.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-mmio-v3.c +6 -1
@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu, gpa_t addr, unsigned int len) {unsigned long mpidr =kvm_vcpu_get_mpidr_aff (vcpu); +struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; +struct vgic_redist_region *rdreg = vgic_cpu->rdreg;int target_vcpu_id = vcpu->vcpu_id;u64 value;
@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,if (vgic_has_its (vcpu->kvm)) value |=GICR_TYPER_PLPIS ; -/* reporting of the Last bit is not supported for userspace */ +if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index -1 ))) + value |=GICR_TYPER_LAST ; +return extract_bytes (value, addr &7 , len); }
@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)return -EINVAL ; vgic_cpu->rdreg = rdreg; + vgic_cpu->rdreg_index = rdreg->free_index; rd_base = rdreg->base + rdreg->free_index *KVM_VGIC_V3_REDIST_SIZE ;
include/kvm/arm_vgic.h +1 -0
@@ -322,6 +322,7 @@ struct vgic_cpu {*/ struct vgic_io_device rd_iodev;struct vgic_redist_region *rdreg; +u32 rdreg_index;/* Contains the attributes and gpa of the LPI pending tables. */ u64 pendbaser;
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH 9/9 KVM: selftests: aarch64/vgic-v3 init sequence tests
In reply to Eric AugerThe tests exercise the VGIC_V3 device creation including the associated KVM_DEV_ARM_VGIC_GRP_ADDR group attributes:
- KVM_VGIC_V3_ADDR_TYPE_DIST/REDIST (vcpu_first and vgic_first)
- KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION (redist_regions).
Another test dedicates to KVM_DEV_ARM_VGIC_GRP_REDIST_REGS group and especially the GICR_TYPER read. The goal was to test the case recently fixed by commit 23bde34771f1
("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace").
The API under test can be found atDocumentation/virt/kvm/devices/arm-vgic-v3.rst
--- tools/testing/selftests/kvm/Makefile | 1 + .../testing/selftests/kvm/aarch64/vgic_init.c | 453 ++++++++++++++++++ .../testing/selftests/kvm/include/kvm_util.h | 5 + tools/testing/selftests/kvm/lib/kvm_util.c | 51 ++ 4 files changed, 510 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/vgic_init.c
tools/testing/selftests/kvm/Makefile +1 -0
@@ -68,6 +68,7 @@ TEST_GEN_PROGS_x86_64 += steal_time TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list-sve +TEST_GEN_PROGS_aarch64 += aarch64/vgic_init TEST_GEN_PROGS_aarch64 += demand_paging_test TEST_GEN_PROGS_aarch64 += dirty_log_test TEST_GEN_PROGS_aarch64 += kvm_create_max_vcpus
tools/testing/selftests/kvm/aarch64/vgic_init.c new file +453 -0
453 lines not on this page · raw message
tools/testing/selftests/kvm/include/kvm_util.h +5 -0
@@ -200,6 +200,11 @@ int vcpu_nested_state_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_nested_state *state, bool ignore_error);#endif +int kvm_device_check_attr (int dev_fd,uint32_t group,uint64_t attr); +int kvm_create_device (struct kvm_vm *vm,uint64_t type,bool test); +int kvm_device_access (int dev_fd,uint32_t group,uint64_t attr, +void *val,bool write); +const char *exit_reason_str (unsigned int exit_reason);void virt_pgd_alloc (struct kvm_vm *vm,uint32_t pgd_memslot);
tools/testing/selftests/kvm/lib/kvm_util.c +51 -0
57 lines not on this page · raw message
Signature · 6 lines
2.21.3 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH 3/9] KVM: arm64: vgic-v3: Fix error handling in vgic_v3_set_redist_base()
In reply to Eric AugerHi Eric,
Quoted 3 lines · Eric Auger
vgic_register_all_redist_iodevsmay succeed whilevgic_register_all_redist_iodevsfails. For example this can happen
The same function cannot both fail and succeed ;-) Can you shed some light on what you had in mind?
Quoted 17 lines
while adding a redistributor region overlapping a dist region. The failure only is detected on
vgic_register_all_redist_iodevswhenvgic_v3_check_base()gets called.In such a case, remove the newly added redistributor region and free it.
Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)arch/arm64/kvm/vgic/vgic-mmio-v3.c +0
-0
arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count) · line 872
Quoted 8 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +4
-1@@ -866,8 +866,14 @@ int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count)* afterwards will register the iodevs when needed. */ ret =vgic_register_all_redist_iodevs (kvm); -if (ret) +if (ret) { +struct vgic_redist_region *rdreg = +vgic_v3_rdist_region_from_index (kvm, index); +
nit: consider splitting declaration and assignment so that we avoid the line split if you insist on the 80 character limit.
Quoted 11 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +3
-0+list_del (&rdreg->list); +kfree (rdreg);return ret; + }return 0 ; }Signature · 1 line
2.21.3
Thanks,
M.
Signature · 5 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH 4/9] KVM: arm/arm64: vgic: Reset base address on kvm_vgic_dist_destroy()
In reply to Eric AugerQuoted 21 lines · Eric Auger
On
vgic_dist_destroy(), the addresses are not reset. However for kvm selftest purpose this would allow to continue the test execution even after a failure when runningKVM_RUN. So let's reset the base addresses.Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-init.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)arch/arm64/kvm/vgic/vgic-init.c +1
-0@@ -335,14 +335,16 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)kfree (dist->spis); dist->spis =NULL ; dist->nr_spis =0 ; + dist->vgic_dist_base =VGIC_ADDR_UNDEF ;
arch/arm64/kvm/vgic/vgic-init.c static void kvm_vgic_dist_destroy(struct kvm *kvm) · line 347
Quoted 10 lines
arch/arm64/kvm/vgic/vgic-init.c +3
-2-if (kvm->arch.vgic.vgic_model ==KVM_DEV_TYPE_ARM_VGIC_V3 ) { +if (dist->vgic_model ==KVM_DEV_TYPE_ARM_VGIC_V3 ) {list_for_each_entry_safe (rdreg, next, &dist->rd_regions, list) {list_del (&rdreg->list);kfree (rdreg); }INIT_LIST_HEAD (&dist->rd_regions); - } + }else + kvm->arch.vgic.vgic_cpu_base =VGIC_ADDR_UNDEF ;
Since you have converted the hunk above to use dist->, you could do the same thing here. And the coding style dictates that you need {} on the else side as well.
Quoted 15 lines
arch/arm64/kvm/vgic/vgic-init.c +1
-0if (vgic_has_its (kvm))vgic_lpi_translation_cache_destroy (kvm);@@ -362,6 +364,7 @@ void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)vgic_flush_pending_lpis (vcpu);INIT_LIST_HEAD (&vgic_cpu->ap_list_head); + vgic_cpu->rd_iodev.base_addr =VGIC_ADDR_UNDEF ; }/* To be called with kvm->lock held */ Signature · 1 line
2.21.3
Thanks,
M.
Signature · 5 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 1/9] KVM: arm64: vgic-v3: Fix some error codes when setting RDIST base
In reply to Eric AugerHi Eric,
Quoted 20 lines · Eric Auger
KVM_DEV_ARM_VGIC_GRP_ADDRgroup doc says we should return-EEXISTin case the base address of the redist is already set. We currently return-EINVAL.However we need to return
-EINVALin case a legacy REDIST address is attempted to be set whileREDIST_REGIONSwere set. This case is discriminated by looking at the count field.Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)arch/arm64/kvm/vgic/vgic-mmio-v3.c +0
-0@@ -792,8 +792,13 @@ static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index,int ret;
arch/arm64/kvm/vgic/vgic-mmio-v3.c static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index, · line 801
Quoted 10 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +7
-2/* single rdist region already set ?*/ -if (!count && !list_empty (rd_regions)) -return -EINVAL ; +if (!count && !list_empty (rd_regions)) { + rdreg =list_last_entry (rd_regions, + struct vgic_redist_region, list); +if (rdreg->count) +return -EINVAL ;/* Mixing REDIST and REDIST_REGION API */ +return -EEXIST ; + }
A few instructions below:
if (list_empty(rd_regions)) {
[..]
} else {
rdreg = list_last_entry(rd_regions, struct vgic_redist_region, list); [..]
/* Cannot add an explicitly sized regions after legacy region */ if (!rdreg->count)
return -EINVAL;
}
Isn't this testing for the same thing, but using the opposite condition? Or am I misunderstanding the code (quite likely)?
Looks to me like KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST{,_REGION}) used to return -EEXIST (from vgic_check_ioaddr()) before commit ccc27bf5be7b7 ("KVM: arm/arm64: Helper to register a new redistributor region") which added the vgic_v3_insert_redist_region() function, so bringing back the -EEXIST return code looks the right thing to me.
Thanks,
Alex
arch/arm64/kvm/vgic/vgic-mmio-v3.c static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index, · line 804
Quoted 3 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0
-0/* cross the end of memory ? */ if (base + size < base)
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 2/9] KVM: arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION read
In reply to Eric AugerHi Eric,
The patch looks correct to me. kvm_vgic_addr() masks out all the bits except index from addr, so we don't need to do it in vgic_get_common_attr():
One nitpick below.
Quoted 7 lines · Eric Auger
The doc says:
"The characteristics of a specific redistributor region can
be read by presetting the index field in the attr data. Only valid forKVM_DEV_TYPE_ARM_VGIC_V3"Unfortunately the existing code fails to read the input attr data and thus the index always is 0.
addr is allocated on the stack, I don't think it will always be 0.
Thanks,
Alex
Quoted 13 lines
Fixes 04c110932225 ("KVM: arm/arm64: Implement KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION")Cc [email protected]#v4.17+Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-kvm-device.c | 3 +++ 1 file changed, 3 insertions(+)arch/arm64/kvm/vgic/vgic-kvm-device.c +0
-0
arch/arm64/kvm/vgic/vgic-kvm-device.c static int vgic_get_common_attr(struct kvm_device *dev, · line 234
Quoted 9 lines
arch/arm64/kvm/vgic/vgic-kvm-device.c +3
-0@@ -226,6 +226,9 @@ static int vgic_get_common_attr(struct kvm_device *dev,u64 addr;unsigned long type = (unsigned long )attr->attr; +if (copy_from_user (&addr, uaddr,sizeof (addr))) +return -EFAULT ; + r =kvm_vgic_addr (dev->kvm, type, &addr, false);if (r)return (r == -ENODEV ) ? -ENXIO : r;
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 5/9] KVM: arm: move has_run_once after the map_resources
In reply to Eric AugerHi Eric,
Quoted 27 lines · Eric Auger
has_run_onceis set to true at the beginning ofkvm_vcpu_first_run_init(). This generally is not an issue except when exercising the code with KVM selftests. Indeed, ifkvm_vgic_map_resources()fails due to erroneous user settings,has_run_onceis set and this prevents from continuing executing the test. This patch moves the assignment after thekvm_vgic_map_resources().Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/arm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/arm64/kvm/arm.c +0
-2@@ -540,8 +540,6 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)if (!kvm_arm_vcpu_is_finalized (vcpu))return -EPERM ; - vcpu->arch.has_run_once = true; -if (likely (irqchip_in_kernel (kvm))) {/* * Map the VGIC hardware resources before running a vcpu the
arch/arm64/kvm/arm.c static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu) · line 561
Quoted 4 lines
arch/arm64/kvm/arm.c +1
-0@@ -560,6 +558,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)static_branch_inc (&userspace_irqchip_in_use); } + vcpu->arch.has_run_once = true;
I have a few concerns regarding this:
1. Moving has_run_once = true here seems very arbitrary to me - kvm_timer_enable() and kvm_arm_pmu_v3_enable(), below it, can both fail because of erroneous user values. If there's a reason why the assignment cannot be moved at the end of the function, I think it should be clearly stated in a comment for the people who might be tempted to write similar tests for the timer or pmu.
2. There are many ways that kvm_vgic_map_resources() can fail, other than incorrect user settings. I started digging into how kvm_vgic_map_resources()->vgic_v2_map_resources() can fail for a VGIC V2 and this is what I managed to find before I gave up:
* vgic_init() can fail in:
- kvm_vgic_dist_init()
- vgic_v3_init()
- kvm_vgic_setup_default_irq_routing()
* vgic_register_dist_iodev() can fail in:
- vgic_v3_init_dist_iodev()
- kvm_io_bus_register_dev()(*)
* kvm_phys_addr_ioremap() can fail in:
- kvm_mmu_topup_memory_cache()
- kvm_pgtable_stage2_map()
So if any of the functions below fail, are we 100% sure it is safe to allow the user to execute kvm_vgic_map_resources() again?
(*) It looks to me like kvm_io_bus_register_dev() doesn't take into account a caller that tries to register the same device address range and it will create another identical range. Is this intentional? Is it a bug that should be fixed? Or am I misunderstanding the function?
Thanks,
Alex
arch/arm64/kvm/arm.c static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu) · line 565
Quoted 4 lines
arch/arm64/kvm/arm.c +1
-0+ ret =kvm_timer_enable (vcpu);if (ret)return ret;
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 6/9] docs: kvm: devices/arm-vgic-v3: enhance KVM_DEV_ARM_VGIC_CTRL_INIT doc
In reply to Eric AugerHi Eric,
Quoted 3 lines · Eric Auger
kvm_arch_vcpu_precreate()returns-EBUSYif the vgic is already initialized. So let's document thatKVM_DEV_ARM_VGIC_CTRL_INITmust be called after all vcpu creations.
Checked and this is indeed the case, kvm_vm_ioctl_create_vcpu()->kvm_arch_vcpu_precreate() returns -EBUSY is vgic_initialized() is true.
Quoted 11 lines
Signed-off-by Eric Auger <[email protected]>--- Documentation/virt/kvm/devices/arm-vgic-v3.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Documentation/virt/kvm/devices/arm-vgic-v3.rst +0
-0
Documentation/virt/kvm/devices/arm-vgic-v3.rst Groups: · line 231
Quoted 5 lines
Documentation/virt/kvm/devices/arm-vgic-v3.rst +1
-1@@ -228,7 +228,7 @@ Groups: KVM_DEV_ARM_VGIC_CTRL_INIT request the initialization of the VGIC, no additional parameter in - kvm_device_attr.addr. + kvm_device_attr.addr. Must be called after all vcpu creations.
Nitpick here: the document writes VCPU with all caps. This also sounds a bit weird, I think something like "Must be called after all VCPUs have been created" is clearer.
Thanks,
Alex
Documentation/virt/kvm/devices/arm-vgic-v3.rst Groups: · line 234
Quoted 3 lines
Documentation/virt/kvm/devices/arm-vgic-v3.rst +0
-0KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES save all LPI pending bits into guest RAM pending tables.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 7/9] KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]
In reply to Eric AugerHi Eric,
Quoted 3 lines · Eric Auger
Instead of converting the
vgic_io_devicehandle to akvm_io_devicehandled and then do the oppositive, pass avgic_io_devicepointer all along the call chain.
To me, it looks like the commit message describes what the patch does instead of why it does it.
What are "vgic_io_device handle" and "kvm_io_device handled"?
Thanks,
Alex
Quoted 36 lines
Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)arch/arm64/kvm/vgic/vgic-mmio.c +2
-4@@ -938,10 +938,9 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,return region; } -static int vgic_uaccess_read (struct kvm_vcpu *vcpu,struct kvm_io_device *dev, +static int vgic_uaccess_read (struct kvm_vcpu *vcpu,struct vgic_io_device *iodev,gpa_t addr,u32 *val) { -struct vgic_io_device *iodev =kvm_to_vgic_iodev (dev);const struct vgic_register_region *region;struct kvm_vcpu *r_vcpu;@@ -960,10 +959,9 @@ static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,return 0 ; } -static int vgic_uaccess_write (struct kvm_vcpu *vcpu,struct kvm_io_device *dev, +static int vgic_uaccess_write (struct kvm_vcpu *vcpu,struct vgic_io_device *iodev,gpa_t addr,const u32 *val) { -struct vgic_io_device *iodev =kvm_to_vgic_iodev (dev);const struct vgic_register_region *region;struct kvm_vcpu *r_vcpu;@@ -986,9 +984,9 @@ int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, bool is_write, int offset, u32 *val)
arch/arm64/kvm/vgic/vgic-mmio.c int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, · line 992
Quoted 10 lines
arch/arm64/kvm/vgic/vgic-mmio.c +2
-2{if (is_write) -return vgic_uaccess_write (vcpu, &dev->dev, offset, val); +return vgic_uaccess_write (vcpu, dev, offset, val);else -return vgic_uaccess_read (vcpu, &dev->dev, offset, val); +return vgic_uaccess_read (vcpu, dev, offset, val); } static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 7/9] KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]
Hi Eric,
Quoted 10 lines · Alexandru Elisei
Hi Eric,
Quoted 3 lines · Eric Auger
Instead of converting the
vgic_io_devicehandle to akvm_io_devicehandled and then do the oppositive, pass avgic_io_devicepointer all along the call chain.To me, it looks like the commit message describes what the patch does instead of why it does it.
What are "
vgic_io_devicehandle" and "kvm_io_devicehandled"?
Sorry, I think I got it now. You were referring to the argument types struct vgic_io_device and struct kvm_io_device. The patch looks like a very good cleanup.
How changing to commit message to sound something like this (feel free to ignore/change it if you think of something else):
vgic_uaccess() takes a struct vgic_io_device argument, converts it to a struct kvm_io_device and passes it to the read/write accessor functions, which convert it back to a struct vgic_io_device. Avoid the indirection by passing the struct vgic_io_device argument directly to vgic_uaccess_{read,write).
Thanks,
Alex
Quoted 47 lines
Thanks,
AlexQuoted 44 lines
Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)arch/arm64/kvm/vgic/vgic-mmio.c +4
-6@@ -938,10 +938,9 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,return region; } -static int vgic_uaccess_read (struct kvm_vcpu *vcpu,struct kvm_io_device *dev, +static int vgic_uaccess_read (struct kvm_vcpu *vcpu,struct vgic_io_device *iodev,gpa_t addr,u32 *val) { -struct vgic_io_device *iodev =kvm_to_vgic_iodev (dev);const struct vgic_register_region *region;struct kvm_vcpu *r_vcpu;@@ -960,10 +959,9 @@ static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,return 0 ; } -static int vgic_uaccess_write (struct kvm_vcpu *vcpu,struct kvm_io_device *dev, +static int vgic_uaccess_write (struct kvm_vcpu *vcpu,struct vgic_io_device *iodev,gpa_t addr,const u32 *val) { -struct vgic_io_device *iodev =kvm_to_vgic_iodev (dev);const struct vgic_register_region *region;struct kvm_vcpu *r_vcpu;@@ -986,9 +984,9 @@ int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, bool is_write, int offset, u32 *val) { if (is_write) - return vgic_uaccess_write(vcpu, &dev->dev, offset, val); + return vgic_uaccess_write(vcpu, dev, offset, val); else - return vgic_uaccess_read(vcpu, &dev->dev, offset, val); + return vgic_uaccess_read(vcpu, dev, offset, val); }
Quoted 5 lines
arch/arm64/kvm/vgic/vgic-mmio.c int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, · line 992
Quoted 1 line
arch/arm64/kvm/vgic/vgic-mmio.c +0
-0static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace
In reply to Eric AugerHi Eric,
Quoted 7 lines · Eric Auger
Commit
23bde34771f1("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access theGICR_TYPERregister before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.
I suppose the reason for emulating GICR_TYPER.Last is for architecture compliance, right? I think that should be in the commit message.
Quoted 32 lines
Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++- include/kvm/arm_vgic.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-)arch/arm64/kvm/vgic/vgic-mmio-v3.c +5
-1@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu, gpa_t addr, unsigned int len) {unsigned long mpidr =kvm_vcpu_get_mpidr_aff (vcpu); +struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; +struct vgic_redist_region *rdreg = vgic_cpu->rdreg;int target_vcpu_id = vcpu->vcpu_id;u64 value;@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,if (vgic_has_its (vcpu->kvm)) value |=GICR_TYPER_PLPIS ; -/* reporting of the Last bit is not supported for userspace */ +if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index -1 ))) + value |=GICR_TYPER_LAST ; +return extract_bytes (value, addr &7 , len); }
arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_register_redist_iodev(struct kvm_vcpu *vcpu) · line 721
Quoted 4 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +1
-0@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)return -EINVAL ; vgic_cpu->rdreg = rdreg; + vgic_cpu->rdreg_index = rdreg->free_index;
What happens if the next redistributor region we register has the base address adjacent to this one?
I'm really not familiar with the code, but is it not possible to create two Redistributor regions (via KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST)) where the second Redistributor region start address is immediately after the last Redistributor in the preceding region?
Thanks,
Alex
Quoted 8 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0
-0rd_base = rdreg->base + rdreg->free_index *KVM_VGIC_V3_REDIST_SIZE ;include/kvm/arm_vgic.h +0
-0
include/kvm/arm_vgic.h struct vgic_cpu { · line 328
Quoted 7 lines
include/kvm/arm_vgic.h +1
-0@@ -322,6 +322,7 @@ struct vgic_cpu {*/ struct vgic_io_device rd_iodev;struct vgic_redist_region *rdreg; +u32 rdreg_index;/* Contains the attributes and gpa of the LPI pending tables. */ u64 pendbaser;
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace
In reply to Eric AugerHi Eric,
Quoted 7 lines · Eric Auger
Commit
23bde34771f1("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access theGICR_TYPERregister before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.
If I understand your patch correctly, it is possible for the GICR_TYPER.Last bit to be transiently 1 if the register is accessed before all the redistributors regions have been configured.
Arm IHI 0069F states that accesses to the GICR_TYPER register are RO. I haven't found exactly what RO means (please point me to the definition if you find it in the architecture!), but I assume it means read-only and I'm not sure how correct (from an architectural point of view) it is for two subsequent reads of this register to return different values. Maybe Marc can shed some light on this.
Thanks,
Alex
Quoted 44 lines
Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++- include/kvm/arm_vgic.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-)arch/arm64/kvm/vgic/vgic-mmio-v3.c +6
-1@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu, gpa_t addr, unsigned int len) {unsigned long mpidr =kvm_vcpu_get_mpidr_aff (vcpu); +struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; +struct vgic_redist_region *rdreg = vgic_cpu->rdreg;int target_vcpu_id = vcpu->vcpu_id;u64 value;@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,if (vgic_has_its (vcpu->kvm)) value |=GICR_TYPER_PLPIS ; -/* reporting of the Last bit is not supported for userspace */ +if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index -1 ))) + value |=GICR_TYPER_LAST ; +return extract_bytes (value, addr &7 , len); }@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)return -EINVAL ; vgic_cpu->rdreg = rdreg; + vgic_cpu->rdreg_index = rdreg->free_index; rd_base = rdreg->base + rdreg->free_index *KVM_VGIC_V3_REDIST_SIZE ;include/kvm/arm_vgic.h +0
-0
include/kvm/arm_vgic.h struct vgic_cpu { · line 328
Quoted 7 lines
include/kvm/arm_vgic.h +1
-0@@ -322,6 +322,7 @@ struct vgic_cpu {*/ struct vgic_io_device rd_iodev;struct vgic_redist_region *rdreg; +u32 rdreg_index;/* Contains the attributes and gpa of the LPI pending tables. */ u64 pendbaser;
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace
Quoted 15 lines · Alexandru Elisei
Hi Eric,
Quoted 7 lines · Eric Auger
Commit
23bde34771f1("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access theGICR_TYPERregister before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.If I understand your patch correctly, it is possible for the GICR_TYPER.Last bit to be transiently 1 if the register is accessed before all the redistributors regions have been configured.
Quoted 5 lines
Arm IHI 0069F states that accesses to the
GICR_TYPERregister are RO. I haven't found exactly what RO means (please point me to the definition if you find it in the architecture!), but I assume it means read-only and I'm not sure how correct (from an architectural point of view) it is for two subsequent reads of this register to return different values. Maybe Marc can shed some light on this.
RO = Read-Only indeed. Not sure that's documented anywhere in the architecture, but this is enough of a well known acronym that even the ARM ARM doesn't feel the need to invent a new definition for it.
As for your concern, I don't think it is a problem to return different values if the HW has changed in between. This may need to be documented though.
Thanks,
M.
Signature · 5 lines
Jazz is not dead. It just smells funny... _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 7/9] KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write]
In reply to Alexandru EliseiHi Alexandru,
Quoted 13 lines · Alexandru Elisei
Hi Eric,
Quoted 10 lines · Alexandru Elisei
Hi Eric,
Quoted 3 lines · Eric Auger
Instead of converting the
vgic_io_devicehandle to akvm_io_devicehandled and then do the oppositive, pass avgic_io_devicepointer all along the call chain.To me, it looks like the commit message describes what the patch does instead of why it does it.
What are "
vgic_io_devicehandle" and "kvm_io_devicehandled"?
Yes unfortunate typo, sorry.
Quoted 11 lines
Sorry, I think I got it now. You were referring to the argument types struct
vgic_io_deviceand structkvm_io_device. The patch looks like a very good cleanup.How changing to commit message to sound something like this (feel free to ignore/change it if you think of something else):
vgic_uaccess()takes a structvgic_io_deviceargument, converts it to a structkvm_io_deviceand passes it to the read/write accessor functions, which convert it back to a structvgic_io_device. Avoid the indirection by passing the structvgic_io_deviceargument directly to vgic_uaccess_{read,write).
I reworded the commit message as you suggested.
Thanks
Eric
Quoted 56 lines
Thanks,
AlexQuoted 52 lines
Thanks,
AlexQuoted 35 lines
Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)arch/arm64/kvm/vgic/vgic-mmio.c +2
-4@@ -938,10 +938,9 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,return region; } -static int vgic_uaccess_read (struct kvm_vcpu *vcpu,struct kvm_io_device *dev, +static int vgic_uaccess_read (struct kvm_vcpu *vcpu,struct vgic_io_device *iodev,gpa_t addr,u32 *val) { -struct vgic_io_device *iodev =kvm_to_vgic_iodev (dev);const struct vgic_register_region *region;struct kvm_vcpu *r_vcpu;@@ -960,10 +959,9 @@ static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,return 0 ; } -static int vgic_uaccess_write (struct kvm_vcpu *vcpu,struct kvm_io_device *dev, +static int vgic_uaccess_write (struct kvm_vcpu *vcpu,struct vgic_io_device *iodev,gpa_t addr,const u32 *val) { -struct vgic_io_device *iodev =kvm_to_vgic_iodev (dev);const struct vgic_register_region *region;struct kvm_vcpu *r_vcpu;@@ -986,9 +984,9 @@ int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, bool is_write, int offset, u32 *val)arch/arm64/kvm/vgic/vgic-mmio.c int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, · line 992
Quoted 10 lines
arch/arm64/kvm/vgic/vgic-mmio.c +2
-2{if (is_write) -return vgic_uaccess_write (vcpu, &dev->dev, offset, val); +return vgic_uaccess_write (vcpu, dev, offset, val);else -return vgic_uaccess_read (vcpu, &dev->dev, offset, val); +return vgic_uaccess_read (vcpu, dev, offset, val); } static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 6/9] docs: kvm: devices/arm-vgic-v3: enhance KVM_DEV_ARM_VGIC_CTRL_INIT doc
In reply to Alexandru EliseiHi Alexandru,
Quoted 10 lines · Alexandru Elisei
Hi Eric,
Quoted 3 lines · Eric Auger
kvm_arch_vcpu_precreate()returns-EBUSYif the vgic is already initialized. So let's document thatKVM_DEV_ARM_VGIC_CTRL_INITmust be called after all vcpu creations.Checked and this is indeed the case, kvm_vm_ioctl_create_vcpu()->kvm_arch_vcpu_precreate() returns
-EBUSYisvgic_initialized()is true.
thanks!
Quoted 13 lines
Quoted 13 lines
Signed-off-by Eric Auger <[email protected]>--- Documentation/virt/kvm/devices/arm-vgic-v3.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Documentation/virt/kvm/devices/arm-vgic-v3.rst +0
-0@@ -228,7 +228,7 @@ Groups:
Quoted 8 lines
Documentation/virt/kvm/devices/arm-vgic-v3.rst Groups: · line 231
Quoted 4 lines
Documentation/virt/kvm/devices/arm-vgic-v3.rst +1
-1KVM_DEV_ARM_VGIC_CTRL_INIT request the initialization of the VGIC, no additional parameter in - kvm_device_attr.addr. + kvm_device_attr.addr. Must be called after all vcpu creations.Nitpick here: the document writes VCPU with all caps. This also sounds a bit weird, I think something like "Must be called after all VCPUs have been created" is clearer.
I took your suggestion.
Thanks
Eric
Quoted 7 lines
Thanks,
AlexQuoted 3 lines
Documentation/virt/kvm/devices/arm-vgic-v3.rst +0
-0KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES save all LPI pending bits into guest RAM pending tables.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 2/9] KVM: arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION read
In reply to Alexandru EliseiHi Alexandru,
Quoted 15 lines · Alexandru Elisei
Hi Eric,
The patch looks correct to me.
kvm_vgic_addr()masks out all the bits except index from addr, so we don't need to do it invgic_get_common_attr():Reviewed-by Alexandru Elisei <[email protected]>One nitpick below.
Quoted 5 lines · Eric Auger
The doc says:
"The characteristics of a specific redistributor region can
be read by presetting the index field in the attr data. Only valid forKVM_DEV_TYPE_ARM_VGIC_V3"
Quoted 4 lines
Quoted 2 lines
Unfortunately the existing code fails to read the input attr data and thus the index always is 0.
addr is allocated on the stack, I don't think it will always be 0.
I removed this statement in the commit message. Thanks!
Eric
Quoted 19 lines
Thanks,
AlexQuoted 16 lines
Fixes 04c110932225 ("KVM: arm/arm64: Implement KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION")Cc [email protected]#v4.17+Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-kvm-device.c | 3 +++ 1 file changed, 3 insertions(+)arch/arm64/kvm/vgic/vgic-kvm-device.c +0
-0@@ -226,6 +226,9 @@ static int vgic_get_common_attr(struct kvm_device *dev,u64 addr;unsigned long type = (unsigned long )attr->attr;
Quoted 7 lines
arch/arm64/kvm/vgic/vgic-kvm-device.c static int vgic_get_common_attr(struct kvm_device *dev, · line 234
Quoted 6 lines
arch/arm64/kvm/vgic/vgic-kvm-device.c +3
-0+if (copy_from_user (&addr, uaddr,sizeof (addr))) +return -EFAULT ; + r =kvm_vgic_addr (dev->kvm, type, &addr, false);if (r)return (r == -ENODEV ) ? -ENXIO : r;
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 4/9] KVM: arm/arm64: vgic: Reset base address on kvm_vgic_dist_destroy()
In reply to Marc ZyngierHi Marc,
Quoted 23 lines · Marc Zyngier
Quoted 21 lines · Eric Auger
On
vgic_dist_destroy(), the addresses are not reset. However for kvm selftest purpose this would allow to continue the test execution even after a failure when runningKVM_RUN. So let's reset the base addresses.Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-init.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)arch/arm64/kvm/vgic/vgic-init.c +1
-0@@ -335,14 +335,16 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)kfree (dist->spis); dist->spis =NULL ; dist->nr_spis =0 ; + dist->vgic_dist_base =VGIC_ADDR_UNDEF ;
Quoted 14 lines
arch/arm64/kvm/vgic/vgic-init.c static void kvm_vgic_dist_destroy(struct kvm *kvm) · line 347
Quoted 10 lines
arch/arm64/kvm/vgic/vgic-init.c +3
-2-if (kvm->arch.vgic.vgic_model ==KVM_DEV_TYPE_ARM_VGIC_V3 ) { +if (dist->vgic_model ==KVM_DEV_TYPE_ARM_VGIC_V3 ) {list_for_each_entry_safe (rdreg, next, &dist->rd_regions, list) {list_del (&rdreg->list);kfree (rdreg); }INIT_LIST_HEAD (&dist->rd_regions); - } + }else + kvm->arch.vgic.vgic_cpu_base =VGIC_ADDR_UNDEF ;Since you have converted the hunk above to use dist->, you could do the same thing here. And the coding style dictates that you need {} on the else side as well.
sure
Thanks
Eric
Quoted 19 lines
Quoted 16 lines
arch/arm64/kvm/vgic/vgic-init.c +1
-0if (vgic_has_its (kvm))vgic_lpi_translation_cache_destroy (kvm);@@ -362,6 +364,7 @@ void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)vgic_flush_pending_lpis (vcpu);INIT_LIST_HEAD (&vgic_cpu->ap_list_head); + vgic_cpu->rd_iodev.base_addr =VGIC_ADDR_UNDEF ; }/* To be called with kvm->lock held */ Signature · 1 line
2.21.3Thanks,
Quoted 2 lines
M.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 3/9] KVM: arm64: vgic-v3: Fix error handling in vgic_v3_set_redist_base()
In reply to Marc ZyngierHi Marc,
Quoted 10 lines · Marc Zyngier
Hi Eric,
Quoted 3 lines · Eric Auger
vgic_register_all_redist_iodevsmay succeed whilevgic_register_all_redist_iodevsfails. For example this can happenThe same function cannot both fail and succeed ;-) Can you shed some light on what you had in mind?
Damn, I meant vgic_v3_insert_redist_region() can be successful and then vgic_register_all_redist_iodevs() fails due to detection of overlap.
Quoted 13 lines
Quoted 13 lines
while adding a redistributor region overlapping a dist region. The failure only is detected on
vgic_register_all_redist_iodevswhenvgic_v3_check_base()gets called.In such a case, remove the newly added redistributor region and free it.
Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
Quoted 16 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count) · line 872
Quoted 13 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +4
-1@@ -866,8 +866,14 @@ int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count)* afterwards will register the iodevs when needed. */ ret =vgic_register_all_redist_iodevs (kvm); -if (ret) +if (ret) { +struct vgic_redist_region *rdreg = +vgic_v3_rdist_region_from_index (kvm, index); +nit: consider splitting declaration and assignment so that we avoid the line split if you insist on the 80 character limit.
Sure
Thanks
Eric
Quoted 15 lines
Quoted 12 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +3
-0+list_del (&rdreg->list); +kfree (rdreg);return ret; + }return 0 ; }Signature · 1 line
2.21.3Thanks,
Quoted 2 lines
M.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 5/9] KVM: arm: move has_run_once after the map_resources
In reply to Alexandru EliseiHi Alexandru,
Quoted 37 lines · Alexandru Elisei
Hi Eric,
Quoted 27 lines · Eric Auger
has_run_onceis set to true at the beginning ofkvm_vcpu_first_run_init(). This generally is not an issue except when exercising the code with KVM selftests. Indeed, ifkvm_vgic_map_resources()fails due to erroneous user settings,has_run_onceis set and this prevents from continuing executing the test. This patch moves the assignment after thekvm_vgic_map_resources().Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/arm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/arm64/kvm/arm.c +0
-2@@ -540,8 +540,6 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)if (!kvm_arm_vcpu_is_finalized (vcpu))return -EPERM ; - vcpu->arch.has_run_once = true; -if (likely (irqchip_in_kernel (kvm))) {/* * Map the VGIC hardware resources before running a vcpu the arch/arm64/kvm/arm.c static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu) · line 561
Quoted 4 lines
arch/arm64/kvm/arm.c +1
-0@@ -560,6 +558,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)static_branch_inc (&userspace_irqchip_in_use); } + vcpu->arch.has_run_once = true;I have a few concerns regarding this:
Quoted 5 lines
1. Moving
has_run_once= true here seems very arbitrary to me -kvm_timer_enable()andkvm_arm_pmu_v3_enable(), below it, can both fail because of erroneous user values. If there's a reason why the assignment cannot be moved at the end of the function, I think it should be clearly stated in a comment for the people who might be tempted to write similar tests for the timer or pmu.
Setting has_run_once = true at the entry of the function looks to me even more arbitrary. I agree with you that eventually has_run_once may be moved at the very end but maybe this can be done later once timer, pmu tests haven ben written
Quoted 6 lines
2. There are many ways that
kvm_vgic_map_resources()can fail, other than incorrect user settings. I started digging into how kvm_vgic_map_resources()->vgic_v2_map_resources() can fail for a VGIC V2 and this is what I managed to find before I gave up:
Quoted 10 lines
*
vgic_init()can fail in:
-kvm_vgic_dist_init()
-vgic_v3_init()
-kvm_vgic_setup_default_irq_routing()
*vgic_register_dist_iodev()can fail in:
-vgic_v3_init_dist_iodev()
- kvm_io_bus_register_dev()(*)
*kvm_phys_addr_ioremap()can fail in:
-kvm_mmu_topup_memory_cache()
-kvm_pgtable_stage2_map()
I changed the commit msg so that "incorrect user settings" sounds as an example.
Quoted 3 lines
So if any of the functions below fail, are we 100% sure it is safe to allow the user to execute
kvm_vgic_map_resources()again?
I think additional tests will confirm this. However at the moment, moving the assignment, which does not look wrong to me, allows to greatly simplify the tests so I would tend to say that it is worth.
Quoted 5 lines
(*) It looks to me like
kvm_io_bus_register_dev()doesn't take into account a caller that tries to register the same device address range and it will create another identical range. Is this intentional? Is it a bug that should be fixed? Or am I misunderstanding the function?
doesn't kvm_io_bus_cmp() do the check?
Thanks
Eric
Quoted 8 lines
Thanks,
AlexQuoted 4 lines
arch/arm64/kvm/arm.c +1
-0+ ret =kvm_timer_enable (vcpu);if (ret)return ret;
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 1/9] KVM: arm64: vgic-v3: Fix some error codes when setting RDIST base
In reply to Alexandru EliseiHi Alexandru,
Quoted 43 lines · Alexandru Elisei
Hi Eric,
Quoted 20 lines · Eric Auger
KVM_DEV_ARM_VGIC_GRP_ADDRgroup doc says we should return-EEXISTin case the base address of the redist is already set. We currently return-EINVAL.However we need to return
-EINVALin case a legacy REDIST address is attempted to be set whileREDIST_REGIONSwere set. This case is discriminated by looking at the count field.Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)arch/arm64/kvm/vgic/vgic-mmio-v3.c +0
-0@@ -792,8 +792,13 @@ static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index,int ret;arch/arm64/kvm/vgic/vgic-mmio-v3.c static int vgic_v3_insert_redist_region(struct kvm *kvm, uint32_t index, · line 801
Quoted 10 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +7
-2/* single rdist region already set ?*/ -if (!count && !list_empty (rd_regions)) -return -EINVAL ; +if (!count && !list_empty (rd_regions)) { + rdreg =list_last_entry (rd_regions, + struct vgic_redist_region, list); +if (rdreg->count) +return -EINVAL ;/* Mixing REDIST and REDIST_REGION API */ +return -EEXIST ; + }A few instructions below:
if (
list_empty(rd_regions)) {
[..]
} else {
rdreg =list_last_entry(rd_regions, struct vgic_redist_region, list); [..]
Quoted 7 lines
/* Cannot add an explicitly sized regions after legacy region */ if (!rdreg->count)
return-EINVAL;
}Isn't this testing for the same thing, but using the opposite condition? Or am I misunderstanding the code (quite likely)?
the 1st test sequence handles the case where the legacy KVM_VGIC_V3_ADDR_TYPE_REDIST is used (!count) while the second handles the case where the REDIST_REGION is used. Nevertheless I think this can be simplified into:
if (list_empty(rd_regions)) {
if (index != 0)
return -EINVAL;
} else {
rdreg = list_last_entry(rd_regions,
struct vgic_redist_region, list);
if ((!count) != (!rdreg->count))
return -EINVAL; /* Mix REDIST and REDIST_REGION */
if (!count)
return -EEXIST;
if (index != rdreg->index + 1)
return -EINVAL;
}Quoted 6 lines
Looks to me like
KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST{,_REGION})used to return-EEXIST(fromvgic_check_ioaddr()) before commitccc27bf5be7b7("KVM: arm/arm64: Helper to register a new redistributor region") which added thevgic_v3_insert_redist_region()function, so bringing back the-EEXISTreturn code looks the right thing to me.
OK thank you for the detailed study.
Eric
Quoted 7 lines
Thanks,
AlexQuoted 3 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0
-0/* cross the end of memory ? */ if (base + size < base)
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace
In reply to Alexandru EliseiHi Alexandru,
Quoted 11 lines · Alexandru Elisei
Hi Eric,
Quoted 7 lines · Eric Auger
Commit
23bde34771f1("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access theGICR_TYPERregister before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.
Quoted 2 lines
I suppose the reason for emulating GICR_TYPER.Last is for architecture compliance, right? I think that should be in the commit message.
OK added this in the commit msg.
Quoted 41 lines
Quoted 33 lines
Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++- include/kvm/arm_vgic.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-)arch/arm64/kvm/vgic/vgic-mmio-v3.c +5
-1@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu, gpa_t addr, unsigned int len) {unsigned long mpidr =kvm_vcpu_get_mpidr_aff (vcpu); +struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; +struct vgic_redist_region *rdreg = vgic_cpu->rdreg;int target_vcpu_id = vcpu->vcpu_id;u64 value;@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,if (vgic_has_its (vcpu->kvm)) value |=GICR_TYPER_PLPIS ; -/* reporting of the Last bit is not supported for userspace */ +if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index -1 ))) + value |=GICR_TYPER_LAST ; +return extract_bytes (value, addr &7 , len); }arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_register_redist_iodev(struct kvm_vcpu *vcpu) · line 721
Quoted 4 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +1
-0@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)return -EINVAL ; vgic_cpu->rdreg = rdreg; + vgic_cpu->rdreg_index = rdreg->free_index;What happens if the next redistributor region we register has the base address adjacent to this one?
Quoted 5 lines
I'm really not familiar with the code, but is it not possible to create two Redistributor regions (via
KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST)) where the second Redistributor region start address is immediately after the last Redistributor in the preceding region?
KVM_VGIC_V3_ADDR_TYPE_REDIST only allows to create a single rdist region. Only KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION allows to register several of them.
with KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION, it is possible to register adjacent rdist regions. vgic_v3_rdist_free_slot() previously returned the 1st rdist region where enough space remains for inserting the new reg. We put the rdist at the free index there.
But maybe I misunderstood your question?
Thanks
Eric
Quoted 16 lines
Thanks,
AlexQuoted 13 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0
-0rd_base = rdreg->base + rdreg->free_index *KVM_VGIC_V3_REDIST_SIZE ;include/kvm/arm_vgic.h +1
-0@@ -322,6 +322,7 @@ struct vgic_cpu {*/ struct vgic_io_device rd_iodev;struct vgic_redist_region *rdreg; +u32 rdreg_index;
Quoted 3 lines
include/kvm/arm_vgic.h struct vgic_cpu { · line 328
Quoted 2 lines
include/kvm/arm_vgic.h +0
-0/* Contains the attributes and gpa of the LPI pending tables. */ u64 pendbaser;
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 5/9] KVM: arm: move has_run_once after the map_resources
In reply to Auger EricHi Eric,
Quoted 39 lines · Auger Eric
Hi Alexandru,
Quoted 36 lines · Alexandru Elisei
Hi Eric,
Quoted 27 lines · Eric Auger
has_run_onceis set to true at the beginning ofkvm_vcpu_first_run_init(). This generally is not an issue except when exercising the code with KVM selftests. Indeed, ifkvm_vgic_map_resources()fails due to erroneous user settings,has_run_onceis set and this prevents from continuing executing the test. This patch moves the assignment after thekvm_vgic_map_resources().Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/arm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/arm64/kvm/arm.c +0
-2@@ -540,8 +540,6 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)if (!kvm_arm_vcpu_is_finalized (vcpu))return -EPERM ; - vcpu->arch.has_run_once = true; -if (likely (irqchip_in_kernel (kvm))) {/* * Map the VGIC hardware resources before running a vcpu the arch/arm64/kvm/arm.c static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu) · line 561
Quoted 4 lines
arch/arm64/kvm/arm.c +1
-0@@ -560,6 +558,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)static_branch_inc (&userspace_irqchip_in_use); } + vcpu->arch.has_run_once = true;I have a few concerns regarding this:
Quoted 7 lines
Quoted 5 lines
1. Moving
has_run_once= true here seems very arbitrary to me -kvm_timer_enable()andkvm_arm_pmu_v3_enable(), below it, can both fail because of erroneous user values. If there's a reason why the assignment cannot be moved at the end of the function, I think it should be clearly stated in a comment for the people who might be tempted to write similar tests for the timer or pmu.Setting
has_run_once= true at the entry of the function looks to me even more arbitrary. I agree with you that eventuallyhas_run_oncemay
Or it could be it's there to prevent the user from calling kvm_vgic_map_resources() a second time after it failed. This is what I'm concerned about and I think deserves more investigation.
Thanks,
Alex
Quoted 28 lines
be moved at the very end but maybe this can be done later once timer, pmu tests haven ben written
Quoted 5 lines
2. There are many ways that
kvm_vgic_map_resources()can fail, other than incorrect user settings. I started digging into how kvm_vgic_map_resources()->vgic_v2_map_resources() can fail for a VGIC V2 and this is what I managed to find before I gave up:Quoted 10 lines
*
vgic_init()can fail in:
-kvm_vgic_dist_init()
-vgic_v3_init()
-kvm_vgic_setup_default_irq_routing()
*vgic_register_dist_iodev()can fail in:
-vgic_v3_init_dist_iodev()
- kvm_io_bus_register_dev()(*)
*kvm_phys_addr_ioremap()can fail in:
-kvm_mmu_topup_memory_cache()
-kvm_pgtable_stage2_map()I changed the commit msg so that "incorrect user settings" sounds as an example.
Quoted 2 lines
So if any of the functions below fail, are we 100% sure it is safe to allow the user to execute
kvm_vgic_map_resources()again?I think additional tests will confirm this. However at the moment, moving the assignment, which does not look wrong to me, allows to greatly simplify the tests so I would tend to say that it is worth.
Quoted 4 lines
(*) It looks to me like
kvm_io_bus_register_dev()doesn't take into account a caller that tries to register the same device address range and it will create another identical range. Is this intentional? Is it a bug that should be fixed? Or am I misunderstanding the function?
Quoted 11 lines
doesn't
kvm_io_bus_cmp()do the check?Thanks
Eric
Quoted 6 lines
Thanks,
AlexQuoted 4 lines
arch/arm64/kvm/arm.c +1
-0+ ret =kvm_timer_enable (vcpu);if (ret)return ret;
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace
In reply to Auger EricHi Eric,
Quoted 63 lines · Auger Eric
Hi Alexandru,
Quoted 10 lines · Alexandru Elisei
Hi Eric,
Quoted 7 lines · Eric Auger
Commit
23bde34771f1("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access theGICR_TYPERregister before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.Quoted 2 lines
I suppose the reason for emulating GICR_TYPER.Last is for architecture compliance, right? I think that should be in the commit message.
OK added this in the commit msg.
Quoted 38 lines
Quoted 31 lines
Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++- include/kvm/arm_vgic.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-)arch/arm64/kvm/vgic/vgic-mmio-v3.c +5
-1@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu, gpa_t addr, unsigned int len) {unsigned long mpidr =kvm_vcpu_get_mpidr_aff (vcpu); +struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; +struct vgic_redist_region *rdreg = vgic_cpu->rdreg;int target_vcpu_id = vcpu->vcpu_id;u64 value;@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,if (vgic_has_its (vcpu->kvm)) value |=GICR_TYPER_PLPIS ; -/* reporting of the Last bit is not supported for userspace */ +if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index -1 ))) + value |=GICR_TYPER_LAST ; +return extract_bytes (value, addr &7 , len); }arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_register_redist_iodev(struct kvm_vcpu *vcpu) · line 721
Quoted 4 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +1
-0@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)return -EINVAL ; vgic_cpu->rdreg = rdreg; + vgic_cpu->rdreg_index = rdreg->free_index;What happens if the next redistributor region we register has the base address adjacent to this one?
Quoted 5 lines
I'm really not familiar with the code, but is it not possible to create two Redistributor regions (via
KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST)) where the second Redistributor region start address is immediately after the last Redistributor in the preceding region?
KVM_VGIC_V3_ADDR_TYPE_REDISTonly allows to create a single rdist region. OnlyKVM_VGIC_V3_ADDR_TYPE_REDIST_REGIONallows to register several of them.
Quoted 6 lines
with
KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION, it is possible to register adjacent rdist regions.vgic_v3_rdist_free_slot()previously returned the 1st rdist region where enough space remains for inserting the new reg. We put the rdist at the free index there.But maybe I misunderstood your question?
Yes, I think you did a good job at answering my poorly worded question.
This is the case I am concerned about:
1. Userspace sets first redistributor base address to 0x0 via KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION(count = 1, base = 0x0, flags = 0, index = 0).
2. Userspace sets first redistributor base address to 0x0 + 128K, immediately following the previous Redistributor.
In that case the two Redistributors will be represented by two separate struct vgic_redist_region, but they are adjacent to one another and represent one contiguous memory region.
From what I understand from your patch, GICR_TYPER.Last will be set for both Redistributors, when it should be set only for the second Redistributor. Does any of that make sense?
Thanks,
Alex
Quoted 19 lines
Thanks
Eric
Quoted 15 lines
Thanks,
AlexQuoted 13 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0
-0rd_base = rdreg->base + rdreg->free_index *KVM_VGIC_V3_REDIST_SIZE ;include/kvm/arm_vgic.h +1
-0@@ -322,6 +322,7 @@ struct vgic_cpu {*/ struct vgic_io_device rd_iodev;struct vgic_redist_region *rdreg; +u32 rdreg_index;
Quoted 2 lines
Quoted 2 lines
include/kvm/arm_vgic.h struct vgic_cpu { · line 328
Quoted 2 lines
include/kvm/arm_vgic.h +0
-0/* Contains the attributes and gpa of the LPI pending tables. */ u64 pendbaser;
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 8/9] KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace
Hi Alexandru,
Quoted 83 lines · Alexandru Elisei
Hi Eric,
Quoted 63 lines · Auger Eric
Hi Alexandru,
Quoted 10 lines · Alexandru Elisei
Hi Eric,
Quoted 7 lines · Eric Auger
Commit
23bde34771f1("KVM: arm64: vgic-v3: Drop the reporting of GICR_TYPER.Last for userspace") temporarily fixed a bug identified when attempting to access theGICR_TYPERregister before the redistributor region setting but dropped the support of the LAST bit. This patch restores its support (if the redistributor region was set) while keeping the code safe.Quoted 2 lines
I suppose the reason for emulating GICR_TYPER.Last is for architecture compliance, right? I think that should be in the commit message.
OK added this in the commit msg.
Quoted 38 lines
Quoted 31 lines
Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 7 ++++++- include/kvm/arm_vgic.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-)arch/arm64/kvm/vgic/vgic-mmio-v3.c +5
-1@@ -277,6 +277,8 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu, gpa_t addr, unsigned int len) {unsigned long mpidr =kvm_vcpu_get_mpidr_aff (vcpu); +struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; +struct vgic_redist_region *rdreg = vgic_cpu->rdreg;int target_vcpu_id = vcpu->vcpu_id;u64 value;@@ -286,7 +288,9 @@ static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,if (vgic_has_its (vcpu->kvm)) value |=GICR_TYPER_PLPIS ; -/* reporting of the Last bit is not supported for userspace */ +if (rdreg && (vgic_cpu->rdreg_index == (rdreg->free_index -1 ))) + value |=GICR_TYPER_LAST ; +return extract_bytes (value, addr &7 , len); }arch/arm64/kvm/vgic/vgic-mmio-v3.c int vgic_register_redist_iodev(struct kvm_vcpu *vcpu) · line 721
Quoted 4 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +1
-0@@ -714,6 +718,7 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)return -EINVAL ; vgic_cpu->rdreg = rdreg; + vgic_cpu->rdreg_index = rdreg->free_index;What happens if the next redistributor region we register has the base address adjacent to this one?
Quoted 5 lines
I'm really not familiar with the code, but is it not possible to create two Redistributor regions (via
KVM_DEV_ARM_VGIC_GRP_ADDR(KVM_VGIC_V3_ADDR_TYPE_REDIST)) where the second Redistributor region start address is immediately after the last Redistributor in the preceding region?
KVM_VGIC_V3_ADDR_TYPE_REDISTonly allows to create a single rdist region. OnlyKVM_VGIC_V3_ADDR_TYPE_REDIST_REGIONallows to register several of them.Quoted 6 lines
with
KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION, it is possible to register adjacent rdist regions.vgic_v3_rdist_free_slot()previously returned the 1st rdist region where enough space remains for inserting the new reg. We put the rdist at the free index there.But maybe I misunderstood your question?
Yes, I think you did a good job at answering my poorly worded question.
This is the case I am concerned about:
1. Userspace sets first redistributor base address to
0x0viaKVM_VGIC_V3_ADDR_TYPE_REDIST_REGION(count = 1, base = 0x0, flags = 0, index = 0).2. Userspace sets first redistributor base address to
0x0+ 128K, immediately following the previous Redistributor.
Quoted 7 lines
In that case the two Redistributors will be represented by two separate struct
vgic_redist_region, but they are adjacent to one another and represent one contiguous memory region.From what I understand from your patch, GICR_TYPER.Last will be set for both Redistributors, when it should be set only for the second Redistributor. Does any of that make sense?
Please forgive me for not having replied before on this thread.
This is a valid concern. Nothing prevents the redistributor regions from being contiguous although this is not the goal. Also nothing prevents vcpu rdists to be laid out within a redist region in non ascending order. Also redist regions with ascending indices may not have increasing base addresses.
So this becomes a gas factory for emulating a single bit but I have reworked this in v3 ;-)
Thanks
Eric
Quoted 22 lines
Thanks,
AlexQuoted 19 lines
Thanks
Eric
Quoted 15 lines
Thanks,
AlexQuoted 13 lines
arch/arm64/kvm/vgic/vgic-mmio-v3.c +0
-0rd_base = rdreg->base + rdreg->free_index *KVM_VGIC_V3_REDIST_SIZE ;include/kvm/arm_vgic.h +1
-0@@ -322,6 +322,7 @@ struct vgic_cpu {*/ struct vgic_io_device rd_iodev;struct vgic_redist_region *rdreg; +u32 rdreg_index;
Quoted 3 lines
Quoted 2 lines
Quoted 2 lines
include/kvm/arm_vgic.h struct vgic_cpu { · line 328
Quoted 2 lines
include/kvm/arm_vgic.h +0
-0/* Contains the attributes and gpa of the LPI pending tables. */ u64 pendbaser;
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 5/9] KVM: arm: move has_run_once after the map_resources
In reply to Alexandru EliseiHi Alexandru,
Quoted 47 lines · Alexandru Elisei
Hi Eric,
Quoted 44 lines · Auger Eric
Hi Alexandru,
Quoted 41 lines · Alexandru Elisei
Hi Eric,
Quoted 27 lines · Eric Auger
has_run_onceis set to true at the beginning ofkvm_vcpu_first_run_init(). This generally is not an issue except when exercising the code with KVM selftests. Indeed, ifkvm_vgic_map_resources()fails due to erroneous user settings,has_run_onceis set and this prevents from continuing executing the test. This patch moves the assignment after thekvm_vgic_map_resources().Signed-off-by Eric Auger <[email protected]>--- arch/arm64/kvm/arm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/arm64/kvm/arm.c +0
-2@@ -540,8 +540,6 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)if (!kvm_arm_vcpu_is_finalized (vcpu))return -EPERM ; - vcpu->arch.has_run_once = true; -if (likely (irqchip_in_kernel (kvm))) {/* * Map the VGIC hardware resources before running a vcpu the arch/arm64/kvm/arm.c static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu) · line 561
Quoted 4 lines
arch/arm64/kvm/arm.c +1
-0@@ -560,6 +558,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)static_branch_inc (&userspace_irqchip_in_use); } + vcpu->arch.has_run_once = true;I have a few concerns regarding this:
1. Moving
has_run_once= true here seems very arbitrary to me -kvm_timer_enable()andkvm_arm_pmu_v3_enable(), below it, can both fail because of erroneous user values. If there's a reason why the assignment cannot be moved at the end of the function, I think it should be clearly stated in a comment for the people who might be tempted to write similar tests for the timer or pmu.
Quoted 6 lines
Quoted 2 lines
Setting
has_run_once= true at the entry of the function looks to me even more arbitrary. I agree with you that eventuallyhas_run_oncemayOr it could be it's there to prevent the user from calling
kvm_vgic_map_resources()a second time after it failed. This is what I'm concerned about and I think deserves more investigation.
I have reworked my kvm selftests to live without that change.
Thanks
Eric
Quoted 43 lines
Thanks,
AlexQuoted 39 lines
be moved at the very end but maybe this can be done later once timer, pmu tests haven ben written
Quoted 5 lines
2. There are many ways that
kvm_vgic_map_resources()can fail, other than incorrect user settings. I started digging into how kvm_vgic_map_resources()->vgic_v2_map_resources() can fail for a VGIC V2 and this is what I managed to find before I gave up:Quoted 10 lines
*
vgic_init()can fail in:
-kvm_vgic_dist_init()
-vgic_v3_init()
-kvm_vgic_setup_default_irq_routing()
*vgic_register_dist_iodev()can fail in:
-vgic_v3_init_dist_iodev()
- kvm_io_bus_register_dev()(*)
*kvm_phys_addr_ioremap()can fail in:
-kvm_mmu_topup_memory_cache()
-kvm_pgtable_stage2_map()I changed the commit msg so that "incorrect user settings" sounds as an example.
Quoted 2 lines
So if any of the functions below fail, are we 100% sure it is safe to allow the user to execute
kvm_vgic_map_resources()again?I think additional tests will confirm this. However at the moment, moving the assignment, which does not look wrong to me, allows to greatly simplify the tests so I would tend to say that it is worth.
Quoted 4 lines
(*) It looks to me like
kvm_io_bus_register_dev()doesn't take into account a caller that tries to register the same device address range and it will create another identical range. Is this intentional? Is it a bug that should be fixed? Or am I misunderstanding the function?doesn't
kvm_io_bus_cmp()do the check?Thanks
Eric
Quoted 6 lines
Thanks,
AlexQuoted 4 lines
arch/arm64/kvm/arm.c +1
-0+ ret =kvm_timer_enable (vcpu);if (ret)return ret;
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm