KVM: selftests: Fixes for ucall pool + page_fault_test
v2 cover KVM: selftests: Fixes for ucall pool + page_fault_test
The combination of the pool-based ucall implementation + page_fault_test resulted in some 'fun' bugs. As has always been the case, KVM selftests is a house of cards.
Small series to fix up the issues on kvm/queue. Patches 1-2 can probably be squashed into Paolo's merge resolution, if desired.
Tested on Ampere Altra and a Skylake box, since there was a decent amount of munging in architecture-generic code.
v1 -> v2:
- Collect R-b from Sean (thanks!)
- Use a common routine for split and contiguous VA spaces, with commentary on why arm64 is different since we all get to look at it now. (Sean)
- Don't identity map the ucall MMIO hole
- Fix an off-by-one issue in the accounting of virtual memory, discovered in fighting with #2
- Fix an infinite loop in ucall_alloc(), discovered fighting with the ucall_init() v. kvm_vm_elf_load() ordering issue
Mark Brown (1):
KVM: selftests: Fix build due to ucall_uninit() removal
Oliver Upton (6):
KVM: selftests: Setup ucall after loading program into guest memory
KVM: selftests: Mark correct page as mapped in virt_map()
KVM: selftests: Correctly initialize the VA space for TTBR0_EL1
KVM: arm64: selftests: Don't identity map the ucall MMIO hole
KVM: selftests: Allocate ucall pool from MEM_REGION_DATA
KVM: selftests: Avoid infinite loop if ucall_alloc() fails
.../selftests/kvm/aarch64/page_fault_test.c | 9 +++- .../selftests/kvm/include/kvm_util_base.h | 1 + .../testing/selftests/kvm/lib/aarch64/ucall.c | 6 ++- tools/testing/selftests/kvm/lib/kvm_util.c | 53 ++++++++++++++++--- .../testing/selftests/kvm/lib/ucall_common.c | 14 +++-- 5 files changed, 68 insertions(+), 15 deletions(-)
base-commit: 89b2395859651113375101bb07cd6340b1ba3637
Signature · 6 lines
2.39.0.rc1.256.g54fd8350bd-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 1/7 KVM: selftests: Fix build due to ucall_uninit() removal
From: Mark Brown <[email protected]>
Today's -next fails to build on arm64 due to:
In file included from include/kvm_util.h:11,
from aarch64/page_fault_test.c:15:
include/ucall_common.h:36:47: note: expected ‘vm_paddr_t’ {aka ‘long unsigned int’} but argument is of type ‘void *’
36 | void ucall_init(struct kvm_vm *vm, vm_paddr_t mmio_gpa);
| ~~~~~~~~~~~^~~~~~~~
aarch64/page_fault_test.c:725:2: warning: implicit declaration of function ‘ucall_uninit’; did you mean ‘ucall_init’? [-Wimplicit-function-declaration]
725 | ucall_uninit(vm);
| ^~~~~~~~~~~~
| ucall_initwhich is caused by commit
interacting poorly with commit
28a65567acb5 ("KVM: selftests: Drop now-unnecessary ucall_uninit()")
As is done for other ucall_uninit() users remove the call in the newly added page_fault_test.c.
--- tools/testing/selftests/kvm/aarch64/page_fault_test.c | 1 - 1 file changed, 1 deletion(-)
tools/testing/selftests/kvm/aarch64/page_fault_test.c +0 -1
@@ -722,7 +722,6 @@ static void run_test(enum vm_guest_mode mode, void *arg)vcpu_run_loop (vm, vcpu, test); -ucall_uninit (vm);kvm_vm_free (vm);free_uffd (test, pt_uffd, data_uffd);
Signature · 6 lines
2.39.0.rc1.256.g54fd8350bd-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 2/7 KVM: selftests: Setup ucall after loading program into guest memory
In reply to Oliver UptonThe new ucall infrastructure needs to update a couple of guest globals to pass through the ucall MMIO addr and pool of ucall structs. A precondition of writing to the guest's program image is to have it already loaded into guest memory.
Call ucall_init() after kvm_vm_elf_load(). Continue to park the ucall MMIO addr after MEM_REGION_TEST_DATA.
--- tools/testing/selftests/kvm/aarch64/page_fault_test.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/aarch64/page_fault_test.c +7 -1
@@ -609,8 +609,13 @@ static void setup_memslots(struct kvm_vm *vm, struct test_params *p) data_size / guest_page_size, p->test_desc->data_memslot_flags); vm->memslots[MEM_REGION_TEST_DATA ] =TEST_DATA_MEMSLOT ; +} + +static void setup_ucall (struct kvm_vm *vm) +{ +struct userspace_mem_region *region =vm_get_mem_region (vm,MEM_REGION_TEST_DATA ); -ucall_init (vm, data_gpa + data_size); +ucall_init (vm, region->region.guest_phys_addr + region->region.memory_size); }static void setup_default_handlers (struct test_desc *test)
@@ -702,6 +707,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) vm =____vm_create (mode);setup_memslots (vm, p);kvm_vm_elf_load (vm, program_invocation_name); +setup_ucall (vm); vcpu =vm_vcpu_add (vm,0 , guest_code);setup_gva_maps (vm);
Signature · 6 lines
2.39.0.rc1.256.g54fd8350bd-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 3/7 KVM: selftests: Mark correct page as mapped in virt_map()
In reply to Oliver UptonThe loop marks vaddr as mapped after incrementing it by page size, thereby marking the *next* page as mapped. Set the bit in vpages_mapped first instead.
--- tools/testing/selftests/kvm/lib/kvm_util.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/lib/kvm_util.c +2 -2
@@ -1411,10 +1411,10 @@ void virt_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,while (npages--) {virt_pg_map (vm, vaddr, paddr); +sparsebit_set (vm->vpages_mapped, vaddr >> vm->page_shift); + vaddr += page_size; paddr += page_size; - -sparsebit_set (vm->vpages_mapped, vaddr >> vm->page_shift); } }
Signature · 6 lines
2.39.0.rc1.256.g54fd8350bd-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 4/7 KVM: selftests: Correctly initialize the VA space for TTBR0_EL1
In reply to Oliver UptonAn interesting feature of the Arm architecture is that the stage-1 MMU supports two distinct VA regions, controlled by TTBR{0,1}_EL1. As KVM selftests on arm64 only uses TTBR0_EL1, the VA space is constrained to [0, 2^(va_bits)). This is different from other architectures that allow for addressing low and high regions of the VA space from a single page table.
KVM selftests' VA space allocator presumes the valid address range is split between low and high memory based the MSB, which of course is a poor match for arm64's TTBR0 region.
Add a helper that correctly handles both addressing schemes with a comment describing each.
--- .../selftests/kvm/include/kvm_util_base.h | 1 + tools/testing/selftests/kvm/lib/kvm_util.c | 49 ++++++++++++++++--- 2 files changed, 44 insertions(+), 6 deletions(-)
tools/testing/selftests/kvm/include/kvm_util_base.h +1 -0
@@ -103,6 +103,7 @@ struct kvm_vm {struct sparsebit *vpages_mapped;bool has_irqchip;bool pgd_created; +bool has_split_va_space;vm_paddr_t ucall_mmio_addr;vm_paddr_t pgd;vm_vaddr_t gdt;
tools/testing/selftests/kvm/lib/kvm_util.c +43 -6
66 lines not on this page · raw message
Signature · 6 lines
2.39.0.rc1.256.g54fd8350bd-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 5/7 KVM: arm64: selftests: Don't identity map the ucall MMIO hole
In reply to Oliver UptonCurrently the ucall MMIO hole is placed immediately after slot0, which is a relatively safe address in the PA space. However, it is possible that the same address has already been used for something else (like the guest program image) in the VA space. At least in my own testing, building the vgic_irq test with clang leads to the MMIO hole appearing underneath gicv3_ops.
Stop identity mapping the MMIO hole and instead find an unused VA to map to it. Yet another subtle detail of the KVM selftests library is that virt_pg_map() does not update vm->vpages_mapped. Switch over to virt_map() instead to guarantee that the chosen VA isn't to something else.
--- tools/testing/selftests/kvm/lib/aarch64/ucall.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/lib/aarch64/ucall.c +4 -2
@@ -14,11 +14,13 @@ static vm_vaddr_t *ucall_exit_mmio_addr;void ucall_arch_init (struct kvm_vm *vm,vm_paddr_t mmio_gpa) { -virt_pg_map (vm, mmio_gpa, mmio_gpa); +vm_vaddr_t mmio_gva =vm_vaddr_unused_gap (vm, vm->page_size,KVM_UTIL_MIN_VADDR ); + +virt_map (vm, mmio_gva, mmio_gpa,1 ); vm->ucall_mmio_addr = mmio_gpa; -write_guest_global (vm, ucall_exit_mmio_addr, (vm_vaddr_t *)mmio_gpa); +write_guest_global (vm, ucall_exit_mmio_addr, (vm_vaddr_t *)mmio_gva); }void ucall_arch_do_ucall (vm_vaddr_t uc)
Signature · 6 lines
2.39.0.rc1.256.g54fd8350bd-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 6/7 KVM: selftests: Allocate ucall pool from MEM_REGION_DATA
In reply to Oliver UptonMEM_REGION_TEST_DATA is meant to hold data explicitly used by a selftest, not implicit allocations due to the selftests infrastructure. Allocate the ucall pool from MEM_REGION_DATA much like the rest of the selftests library allocations.
--- tools/testing/selftests/kvm/lib/ucall_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/kvm/lib/ucall_common.c +1 -1
@@ -22,7 +22,7 @@ void ucall_init(struct kvm_vm *vm, vm_paddr_t mmio_gpa)vm_vaddr_t vaddr;int i; - vaddr =vm_vaddr_alloc (vm,sizeof (*hdr),KVM_UTIL_MIN_VADDR ); + vaddr =__vm_vaddr_alloc (vm,sizeof (*hdr),KVM_UTIL_MIN_VADDR ,MEM_REGION_DATA ); hdr = (struct ucall_header *)addr_gva2hva (vm, vaddr);memset (hdr,0 ,sizeof (*hdr));
Signature · 6 lines
2.39.0.rc1.256.g54fd8350bd-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 7/7 KVM: selftests: Avoid infinite loop if ucall_alloc() fails
In reply to Oliver UptonGuest assertions depend on successfully allocating a ucall structure. As such, the use of guest assertions when ucall_alloc() fails simply leads to an infinite loop in guest code.
Use GUEST_UCALL_NONE() to indicate failure instead. Though not technically necessary, use a goto to have a single callsite and an associated comment about why assertions don't work here. It isn't perfect, at least the poor developer gets some signal out of the guest...
--- tools/testing/selftests/kvm/lib/ucall_common.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/lib/ucall_common.c +10 -2
@@ -41,7 +41,8 @@ static struct ucall *ucall_alloc(void)struct ucall *uc;int i; -GUEST_ASSERT (ucall_pool); +if (!ucall_pool) + goto out;for (i =0 ; i <KVM_MAX_VCPUS ; ++i) {if (!test_and_set_bit (i, ucall_pool->in_use)) {
@@ -51,7 +52,14 @@ static struct ucall *ucall_alloc(void) } } -GUEST_ASSERT (0 ); +out: +/* +* If the guest cannot grab a ucall structure from the pool then the +* only option to get out to userspace is a bare ucall. This is probably +* a good time to mention that guest assertions depend on ucalls with +* arguments too. +*/ +GUEST_UCALL_NONE ();return NULL ; }
Signature · 6 lines
2.39.0.rc1.256.g54fd8350bd-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Quoted 40 lines · Oliver Upton
The combination of the pool-based ucall implementation +
page_fault_testresulted in some 'fun' bugs. As has always been the case, KVM selftests is a house of cards.Small series to fix up the issues on kvm/queue. Patches 1-2 can probably be squashed into Paolo's merge resolution, if desired.
Tested on Ampere Altra and a Skylake box, since there was a decent amount of munging in architecture-generic code.
v1 -> v2:
- Collect R-b from Sean (thanks!)
- Use a common routine for split and contiguous VA spaces, with commentary on why arm64 is different since we all get to look at it now. (Sean)
- Don't identity map the ucall MMIO hole
- Fix an off-by-one issue in the accounting of virtual memory, discovered in fighting with #2
- Fix an infinite loop inucall_alloc(), discovered fighting with theucall_init()v.kvm_vm_elf_load()ordering issueMark Brown (1):
KVM: selftests: Fix build due toucall_uninit()removalOliver Upton (6):
KVM: selftests: Setup ucall after loading program into guest memory
KVM: selftests: Mark correct page as mapped invirt_map()
KVM: selftests: Correctly initialize the VA space forTTBR0_EL1
KVM: arm64: selftests: Don't identity map the ucall MMIO hole
KVM: selftests: Allocate ucall pool fromMEM_REGION_DATA
KVM: selftests: Avoid infinite loop ifucall_alloc()fails.../selftests/kvm/aarch64/page_fault_test.c | 9 +++- .../selftests/kvm/include/kvm_util_base.h | 1 + .../testing/selftests/kvm/lib/aarch64/ucall.c | 6 ++- tools/testing/selftests/kvm/lib/kvm_util.c | 53 ++++++++++++++++--- .../testing/selftests/kvm/lib/ucall_common.c | 14 +++-- 5 files changed, 68 insertions(+), 15 deletions(-)
Quoted 1 line
base-commit:
89b2395859651113375101bb07cd6340b1ba3637
This commit doesn't seem to exist linux-next or kvm/queue, but the patch context seems to match up with linux-next pretty well. Anyway,
For the series
Thanks,
drew
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Quoted 5 lines · Andrew Jones
Quoted 1 line · Oliver Upton
base-commit:
89b2395859651113375101bb07cd6340b1ba3637This commit doesn't seem to exist linux-next or kvm/queue, but the patch context seems to match up with linux-next pretty well. Anyway,
Ah, a force push to kvm/queue likely explains it :) I believe Paolo has taken the first two patches in his merge resolution now on kvm/next.
Quoted 3 lines
For the series
Reviewed-by Andrew Jones <[email protected]>
Thanks!
--
Best,
Oliver
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v2 4/7] KVM: selftests: Correctly initialize the VA space for TTBR0_EL1
In reply to Oliver UptonQuoted 11 lines · Oliver Upton
An interesting feature of the Arm architecture is that the stage-1 MMU supports two distinct VA regions, controlled by TTBR{0,1}_EL1. As KVM selftests on arm64 only uses
TTBR0_EL1, the VA space is constrained to [0, 2^(va_bits)). This is different from other architectures that allow for addressing low and high regions of the VA space from a single page table.KVM selftests' VA space allocator presumes the valid address range is split between low and high memory based the MSB, which of course is a poor match for arm64's TTBR0 region.
Quoted 5 lines
Add a helper that correctly handles both addressing schemes with a comment describing each.
Signed-off-by Oliver Upton <[email protected]>---
Thanks much! Looks awesome, especially the comment!
Quoted 25 lines
.../selftests/kvm/include/kvm_util_base.h | 1 + tools/testing/selftests/kvm/lib/kvm_util.c | 49 ++++++++++++++++--- 2 files changed, 44 insertions(+), 6 deletions(-)tools/testing/selftests/kvm/include/kvm_util_base.h +1
-0@@ -103,6 +103,7 @@ struct kvm_vm {struct sparsebit *vpages_mapped;bool has_irqchip;bool pgd_created; +bool has_split_va_space;vm_paddr_t ucall_mmio_addr;vm_paddr_t pgd;vm_vaddr_t gdt;tools/testing/selftests/kvm/lib/kvm_util.c +1
-0@@ -186,6 +186,43 @@ const struct vm_guest_mode_params vm_guest_mode_params[] = {_Static_assert (sizeof (vm_guest_mode_params)/sizeof (struct vm_guest_mode_params ) ==NUM_VM_MODES ,"Missing new mode params?" ); +/*
tools/testing/selftests/kvm/lib/kvm_util.c const struct vm_guest_mode_params vm_guest_mode_params[] = { · line 199
Quoted 10 lines
tools/testing/selftests/kvm/lib/kvm_util.c +10
-0+* Initializes vm->vpages_valid to match the canonical VA space of the +* architecture. +* +* Most architectures split the range addressed by a single page table into a +* low and high region based on the MSB of the VA. On architectures with this +* behavior the VA region spans [0, 2^(va_bits - 1)), [-(2^(va_bits - 1), -1]. +* +* arm64 is a bit different from the rest of the crowd, as the low and high +* regions of the VA space are addressed by distinct paging structures +* (TTBR{0,1}_EL1).
Oooh, they're different CR3s in x86 terminology?
Sean Christopherson
reviewerRe: [PATCH v2 7/7] KVM: selftests: Avoid infinite loop if ucall_alloc() fails
In reply to Oliver UptonQuoted 33 lines · Oliver Upton
Guest assertions depend on successfully allocating a ucall structure. As such, the use of guest assertions when
ucall_alloc()fails simply leads to an infinite loop in guest code.Use
GUEST_UCALL_NONE()to indicate failure instead. Though not technically necessary, use a goto to have a single callsite and an associated comment about why assertions don't work here. It isn't perfect, at least the poor developer gets some signal out of the guest...Fixes 426729b2cf2e ("KVM: selftests: Add ucall pool based implementation")Signed-off-by Oliver Upton <[email protected]>--- tools/testing/selftests/kvm/lib/ucall_common.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)tools/testing/selftests/kvm/lib/ucall_common.c +2
-1@@ -41,7 +41,8 @@ static struct ucall *ucall_alloc(void)struct ucall *uc;int i; -GUEST_ASSERT (ucall_pool); +if (!ucall_pool) + goto out;for (i =0 ; i <KVM_MAX_VCPUS ; ++i) {if (!test_and_set_bit (i, ucall_pool->in_use)) {@@ -51,7 +52,14 @@ static struct ucall *ucall_alloc(void) } }
tools/testing/selftests/kvm/lib/ucall_common.c static struct ucall *ucall_alloc(void) · line 62
Quoted 10 lines
tools/testing/selftests/kvm/lib/ucall_common.c +8
-1-GUEST_ASSERT (0 ); +out: +/* +* If the guest cannot grab a ucall structure from the pool then the +* only option to get out to userspace is a bare ucall. This is probably +* a good time to mention that guest assertions depend on ucalls with +* arguments too. +*/ +GUEST_UCALL_NONE ();
UCALL_NONE isn't much better than infinite stack recursion, e.g. a test might end up passing by dumb luck, or go in the wrong direction because it sometimes handles UCALL_NONE.
How about this?
From: Sean Christopherson <[email protected]>
Date: Fri, 9 Dec 2022 12:55:44 -0800
Subject: [PATCH] KVM: selftests: Use magic value to signal ucall_alloc()
failure
Use a magic value to signal a ucall_alloc() failure instead of simply doing GUEST_ASSERT(). GUEST_ASSERT() relies on ucall_alloc() and so a failure puts the guest into an infinite loop.
Use -1 as the magic value, as a real ucall struct should never wrap.
--- tools/testing/selftests/kvm/lib/ucall_common.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/lib/ucall_common.c +14 -2
@@ -4,6 +4,8 @@#include "linux/bitmap.h" #include "linux/atomic.h" +#define GUEST_UCALL_FAILED -1 +struct ucall_header { DECLARE_BITMAP(in_use, KVM_MAX_VCPUS);struct ucall ucalls[KVM_MAX_VCPUS ];
@@ -41,7 +43,8 @@ static struct ucall *ucall_alloc(void)struct ucall *uc;int i; -GUEST_ASSERT (ucall_pool); +if (!ucall_pool) + goto ucall_failed;for (i =0 ; i <KVM_MAX_VCPUS ; ++i) {if (!test_and_set_bit (i, ucall_pool->in_use)) {
@@ -51,7 +54,13 @@ static struct ucall *ucall_alloc(void) } } -GUEST_ASSERT (0 ); +ucall_failed: +/* +* If the vCPU cannot grab a ucall structure, make a bare ucall with a +* magic value to signal to get_ucall() that things went sideways. +* GUEST_ASSERT() depends on ucall_alloc() and so cannot be used here. +*/ +ucall_arch_do_ucall (GUEST_UCALL_FAILED );return NULL ; }
@@ -93,6 +102,9 @@ uint64_t get_ucall(struct kvm_vcpu *vcpu, struct ucall *uc) addr =ucall_arch_get_ucall (vcpu);if (addr) { +TEST_ASSERT (addr != (void *)GUEST_UCALL_FAILED , +"Guest failed to allocate ucall struct" ); +memcpy (uc, addr,sizeof (*uc));vcpu_run_complete_io (vcpu); }else {
base-commit: dc2efbe4813e0dc4368779bc36c5f0e636cb8eb2
Re: [PATCH v2 4/7] KVM: selftests: Correctly initialize the VA space for TTBR0_EL1
In reply to Sean ChristophersonQuoted 15 lines · Sean Christopherson
Quoted 14 lines · Oliver Upton
An interesting feature of the Arm architecture is that the stage-1 MMU supports two distinct VA regions, controlled by TTBR{0,1}_EL1. As KVM selftests on arm64 only uses
TTBR0_EL1, the VA space is constrained to [0, 2^(va_bits)). This is different from other architectures that allow for addressing low and high regions of the VA space from a single page table.KVM selftests' VA space allocator presumes the valid address range is split between low and high memory based the MSB, which of course is a poor match for arm64's TTBR0 region.
Add a helper that correctly handles both addressing schemes with a comment describing each.
Quoted 6 lines
Quoted 2 lines
Signed-off-by Oliver Upton <[email protected]>---
Thanks much! Looks awesome, especially the comment!
Reviewed-by Sean Christopherson <[email protected]>
ty!
Quoted 24 lines
Quoted 24 lines
.../selftests/kvm/include/kvm_util_base.h | 1 + tools/testing/selftests/kvm/lib/kvm_util.c | 49 ++++++++++++++++--- 2 files changed, 44 insertions(+), 6 deletions(-)tools/testing/selftests/kvm/include/kvm_util_base.h +1
-0@@ -103,6 +103,7 @@ struct kvm_vm {struct sparsebit *vpages_mapped;bool has_irqchip;bool pgd_created; +bool has_split_va_space;vm_paddr_t ucall_mmio_addr;vm_paddr_t pgd;vm_vaddr_t gdt;tools/testing/selftests/kvm/lib/kvm_util.c +0
-0@@ -186,6 +186,43 @@ const struct vm_guest_mode_params vm_guest_mode_params[] = {_Static_assert (sizeof (vm_guest_mode_params)/sizeof (struct vm_guest_mode_params ) ==NUM_VM_MODES ,"Missing new mode params?" );
Quoted 13 lines
tools/testing/selftests/kvm/lib/kvm_util.c const struct vm_guest_mode_params vm_guest_mode_params[] = {
Quoted 11 lines
tools/testing/selftests/kvm/lib/kvm_util.c +11
-0+/* +* Initializes vm->vpages_valid to match the canonical VA space of the +* architecture. +* +* Most architectures split the range addressed by a single page table into a +* low and high region based on the MSB of the VA. On architectures with this +* behavior the VA region spans [0, 2^(va_bits - 1)), [-(2^(va_bits - 1), -1]. +* +* arm64 is a bit different from the rest of the crowd, as the low and high +* regions of the VA space are addressed by distinct paging structures +* (TTBR{0,1}_EL1). Oooh, they're different CR3s in x86 terminology?
Right, we can have two active table roots at any given time, each mapping their own portion of the address space.
--
Thanks,
Oliver
Re: [PATCH v2 7/7] KVM: selftests: Avoid infinite loop if ucall_alloc() fails
In reply to Sean ChristophersonOn Fri, Dec 09, 2022 at 09:03:45PM +0000, Sean Christopherson wrote:
[...]
Quoted 5 lines
Quoted 5 lines
(quoted diff) +4
-1- GUEST_ASSERT(0); +out: + /* + * If the guest cannot grab a ucall structure from the pool then the + * only option to get out to userspace is a bare ucall. This is probably
Quoted 8 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ * a good time to mention that guest assertions depend on ucalls with + * arguments too. + */ + GUEST_UCALL_NONE();
UCALL_NONEisn't much better than infinite stack recursion, e.g. a test might end up passing by dumb luck, or go in the wrong direction because it sometimes handlesUCALL_NONE.
Oh, I was just seeking an end to my misery. Yeah, we can use a magic value to signal this instead.
Quoted 1 line
How about this?
LGTM.
--
Thanks,
Oliver
Quoted 69 lines
From: Sean Christopherson <[email protected]>
Date: Fri, 9 Dec 2022 12:55:44 -0800
Subject: [PATCH] KVM: selftests: Use magic value to signalucall_alloc()
failureUse a magic value to signal a
ucall_alloc()failure instead of simply doingGUEST_ASSERT().GUEST_ASSERT()relies onucall_alloc()and so a failure puts the guest into an infinite loop.Use -1 as the magic value, as a real ucall struct should never wrap.
Reported-by Oliver Upton <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- tools/testing/selftests/kvm/lib/ucall_common.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)tools/testing/selftests/kvm/lib/ucall_common.c +14
-2@@ -4,6 +4,8 @@#include "linux/bitmap.h" #include "linux/atomic.h" +#define GUEST_UCALL_FAILED -1 +struct ucall_header { DECLARE_BITMAP(in_use, KVM_MAX_VCPUS);struct ucall ucalls[KVM_MAX_VCPUS ];@@ -41,7 +43,8 @@ static struct ucall *ucall_alloc(void)struct ucall *uc;int i; -GUEST_ASSERT (ucall_pool); +if (!ucall_pool) + goto ucall_failed;for (i =0 ; i <KVM_MAX_VCPUS ; ++i) {if (!test_and_set_bit (i, ucall_pool->in_use)) {@@ -51,7 +54,13 @@ static struct ucall *ucall_alloc(void) } } -GUEST_ASSERT (0 ); +ucall_failed: +/* +* If the vCPU cannot grab a ucall structure, make a bare ucall with a +* magic value to signal to get_ucall() that things went sideways. +* GUEST_ASSERT() depends on ucall_alloc() and so cannot be used here. +*/ +ucall_arch_do_ucall (GUEST_UCALL_FAILED );return NULL ; }@@ -93,6 +102,9 @@ uint64_t get_ucall(struct kvm_vcpu *vcpu, struct ucall *uc) addr =ucall_arch_get_ucall (vcpu);if (addr) { +TEST_ASSERT (addr != (void *)GUEST_UCALL_FAILED , +"Guest failed to allocate ucall struct" ); +memcpy (uc, addr,sizeof (*uc));vcpu_run_complete_io (vcpu); }else {base-commit:
dc2efbe4813e0dc4368779bc36c5f0e636cb8eb2
Paolo Bonzini
maintainerRe: [PATCH v2 4/7] KVM: selftests: Correctly initialize the VA space for TTBR0_EL1
In reply to Oliver UptonReview struct kvm_vm *____vm_create(enum vm_guest_mode mode) · line 314
Quoted 11 lines · Oliver Upton
(quoted diff) +5
-0@@ -268,17 +305,17 @@ struct kvm_vm *____vm_create(enum vm_guest_mode mode) #ifdef __aarch64__ if (vm->pa_bits != 40) vm->type = KVM_VM_TYPE_ARM_IPA_SIZE(vm->pa_bits); + + /* selftests use TTBR0 only, meaning there is a single VA region. */ + vm->has_split_va_space = false; +#else + vm->has_split_va_space = true; #endif
Even though there happens to be already a suitable #ifdef, I don't really like them and don't think there should be a new bool unless something else uses it.
However, the new comment is very useful, so I added it to kvm_util.c as follows:
/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index 759a45540108..56d5ea949cbb 100644
tools/testing/selftests/kvm/lib/kvm_util.c +9 -0
@@ -186,6 +186,15 @@ const struct vm_guest_mode_params vm_guest_mode_params[] = {_Static_assert (sizeof (vm_guest_mode_params)/sizeof (struct vm_guest_mode_params ) ==NUM_VM_MODES ,"Missing new mode params?" ); +/* +* Initializes vm->vpages_valid to match the canonical VA space of the +* architecture. +* +* The default implementation is valid for architectures which split the +* range addressed by a single page table into a low and high region +* based on the MSB of the VA. On architectures with this behavior +* the VA region spans [0, 2^(va_bits - 1)), [-(2^(va_bits - 1), -1]. +*/ __weak void vm_vaddr_populate_bitmap (struct kvm_vm *vm) { sparsebit_set_num(vm->vpages_valid,
Quoted 10 lines · Oliver Upton
The combination of the pool-based ucall implementation +
page_fault_testresulted in some 'fun' bugs. As has always been the case, KVM selftests is a house of cards.Small series to fix up the issues on kvm/queue. Patches 1-2 can probably be squashed into Paolo's merge resolution, if desired.
Tested on Ampere Altra and a Skylake box, since there was a decent amount of munging in architecture-generic code.
Quoted 10 lines
v1 -> v2:
- Collect R-b from Sean (thanks!)
- Use a common routine for split and contiguous VA spaces, with commentary on why arm64 is different since we all get to look at it now. (Sean)
- Don't identity map the ucall MMIO hole
- Fix an off-by-one issue in the accounting of virtual memory, discovered in fighting with #2
- Fix an infinite loop inucall_alloc(), discovered fighting with theucall_init()v.kvm_vm_elf_load()ordering issue
Queued 3+5, thanks.
Paolo
Re: [PATCH v2 7/7] KVM: selftests: Avoid infinite loop if ucall_alloc() fails
In reply to Sean ChristophersonQuoted 66 lines · Sean Christopherson
From: Sean Christopherson<[email protected]>
Date: Fri, 9 Dec 2022 12:55:44 -0800
Subject: [PATCH] KVM: selftests: Use magic value to signalucall_alloc()
failureUse a magic value to signal a
ucall_alloc()failure instead of simply doingGUEST_ASSERT().GUEST_ASSERT()relies onucall_alloc()and so a failure puts the guest into an infinite loop.Use -1 as the magic value, as a real ucall struct should never wrap.
Reported-by Oliver Upton<[email protected]>Signed-off-by Sean Christopherson<[email protected]>--- tools/testing/selftests/kvm/lib/ucall_common.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)tools/testing/selftests/kvm/lib/ucall_common.c +14
-2@@ -4,6 +4,8 @@#include "linux/bitmap.h" #include "linux/atomic.h" +#define GUEST_UCALL_FAILED -1 +struct ucall_header { DECLARE_BITMAP(in_use, KVM_MAX_VCPUS);struct ucall ucalls[KVM_MAX_VCPUS ];@@ -41,7 +43,8 @@ static struct ucall *ucall_alloc(void)struct ucall *uc;int i; -GUEST_ASSERT (ucall_pool); +if (!ucall_pool) + goto ucall_failed;for (i =0 ; i <KVM_MAX_VCPUS ; ++i) {if (!test_and_set_bit (i, ucall_pool->in_use)) {@@ -51,7 +54,13 @@ static struct ucall *ucall_alloc(void) } } -GUEST_ASSERT (0 ); +ucall_failed: +/* +* If the vCPU cannot grab a ucall structure, make a bare ucall with a +* magic value to signal to get_ucall() that things went sideways. +* GUEST_ASSERT() depends on ucall_alloc() and so cannot be used here. +*/ +ucall_arch_do_ucall (GUEST_UCALL_FAILED );return NULL ; }@@ -93,6 +102,9 @@ uint64_t get_ucall(struct kvm_vcpu *vcpu, struct ucall *uc) addr =ucall_arch_get_ucall (vcpu);if (addr) { +TEST_ASSERT (addr != (void *)GUEST_UCALL_FAILED , +"Guest failed to allocate ucall struct" ); +memcpy (uc, addr,sizeof (*uc));vcpu_run_complete_io (vcpu); }else {
Quoted 2 lines
base-commit:
dc2efbe4813e0dc4368779bc36c5f0e636cb8eb2
Queued, thanks.
Paolo