KVM: selftests: Add aarch64/page_fault_test
v10 cover KVM: selftests: Add aarch64/page_fault_test
This series adds a new aarch64 selftest for testing stage 2 fault handling for various combinations of guest accesses (e.g., write, S1PTW), backing sources (e.g., anon), and types of faults (e.g., read on hugetlbfs with a hole, write on a readonly memslot). Each test tries a different combination and then checks that the access results in the right behavior (e.g., uffd faults with the right address and write/read flag). Some interesting combinations are:
- loading an instruction leads to a stage 1 page-table-walk that misses on stage 2 because the backing memslot for the page table it not in host memory (a hole was punched right there) and the fault is handled using userfaultfd. The expected behavior is that this leads to a userfaultfd fault marked as a write. See commit c4ad98e4b72c ("KVM: arm64: Assume write fault on S1PTW permission fault on instruction fetch") for why that's a write.
- a cas (compare-and-swap) on a readonly memslot leads to a failed vcpu run.
- write-faulting on a memslot that's marked for userfaultfd handling and dirty logging should result in a uffd fault and having the respective bit set in the dirty log.
The first 8 commits of this series add library support. The first one adds a new userfaultfd library. Commits 2-5 add some misc library changes that will be used by the new test, like a library function to get the GPA of a PTE. Commits 6-8 breaks the implicit assumption that code and page tables live in memslot memslots should allocators use. This is then used by the new test to place the page tables in a specific memslot. The last 5 commits add the new selftest, one type of test at a time. It first adds core tests, then uffd, then dirty logging, then readonly memslots tests, and finally combinations of the previous ones (like uffd and dirty logging at the same time).
v9 -> v10: https://lore.kernel.org/kvmarm/[email protected]/
- collected r-b's from Andrew
- fixed indentation in several places (mainly alignment of params) [Sean]
- renamed args in uffd_setup [Sean]
v8 -> v9: https://lore.kernel.org/kvmarm/[email protected]/
- removed check before trying madvise(MADV_DONTNEED) on anon hugetlb. [Sean]
- renamed punch_hole_in_memslot() [Sean]
- changed the comment describing the accesses on "holes" [Sean]
- collectd r-b's from Sean
v7 -> v8: https://lore.kernel.org/kvmarm/[email protected]/
- applied Sean's suggestion of adding a fourth region: DATA, and renamed the old DATA one to DATA_TEST. [Sean]
- removed some unneeded code in page_fault_test.c. [Andrew]
- removed r-b's from Andrew and Oliver in commit "KVM: selftests: Use the right memslot for code, page-tables, and data allocations", as the commit changed quite a bit (again). Thanks for the reviews! would it have been OK to keep it? not sure how strict we all are about collecting r-b's on a commit that changed.
Ricardo Koller (14):
KVM: selftests: Add a userfaultfd library
KVM: selftests: aarch64: Add virt_get_pte_hva() library function
KVM: selftests: Add missing close and munmap in
__vm_mem_region_delete()
KVM: selftests: aarch64: Construct DEFAULT_MAIR_EL1 using sysreg.h
macros
tools: Copy bitfield.h from the kernel sources
KVM: selftests: Stash backing_src_type in struct userspace_mem_region
KVM: selftests: Add vm->memslots[] and enum kvm_mem_region_type
KVM: selftests: Fix alignment in virt_arch_pgd_alloc() and
vm_vaddr_alloc()
KVM: selftests: Use the right memslot for code, page-tables, and data
allocations
KVM: selftests: aarch64: Add aarch64/page_fault_test
KVM: selftests: aarch64: Add userfaultfd tests into page_fault_test
KVM: selftests: aarch64: Add dirty logging tests into page_fault_test
KVM: selftests: aarch64: Add readonly memslot tests into
page_fault_test
KVM: selftests: aarch64: Add mix of tests into page_fault_test
tools/include/linux/bitfield.h | 176 +++ tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 2 + .../selftests/kvm/aarch64/page_fault_test.c | 1112 +++++++++++++++++ .../selftests/kvm/demand_paging_test.c | 228 +--- .../selftests/kvm/include/aarch64/processor.h | 35 +- .../selftests/kvm/include/kvm_util_base.h | 31 +- .../selftests/kvm/include/userfaultfd_util.h | 45 + .../selftests/kvm/lib/aarch64/processor.c | 48 +- tools/testing/selftests/kvm/lib/elf.c | 3 +- tools/testing/selftests/kvm/lib/kvm_util.c | 82 +- .../selftests/kvm/lib/riscv/processor.c | 29 +- .../selftests/kvm/lib/s390x/processor.c | 8 +- .../selftests/kvm/lib/userfaultfd_util.c | 186 +++ .../selftests/kvm/lib/x86_64/processor.c | 13 +- 15 files changed, 1723 insertions(+), 276 deletions(-) create mode 100644 tools/include/linux/bitfield.h create mode 100644 tools/testing/selftests/kvm/aarch64/page_fault_test.c create mode 100644 tools/testing/selftests/kvm/include/userfaultfd_util.h create mode 100644 tools/testing/selftests/kvm/lib/userfaultfd_util.c
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 1/14 KVM: selftests: Add a userfaultfd library
Move the generic userfaultfd code out of demand_paging_test.c into a common library, userfaultfd_util. This library consists of a setup and a stop function. The setup function starts a thread for handling page faults using the handler callback function. This setup returns a uffd_desc object which is then used in the stop function (to wait and destroy the threads).
--- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/demand_paging_test.c | 228 +++--------------- .../selftests/kvm/include/userfaultfd_util.h | 45 ++++ .../selftests/kvm/lib/userfaultfd_util.c | 186 ++++++++++++++ 4 files changed, 262 insertions(+), 198 deletions(-) create mode 100644 tools/testing/selftests/kvm/include/userfaultfd_util.h create mode 100644 tools/testing/selftests/kvm/lib/userfaultfd_util.c
tools/testing/selftests/kvm/Makefile +1 -0
@@ -47,6 +47,7 @@ LIBKVM += lib/perf_test_util.c LIBKVM += lib/rbtree.c LIBKVM += lib/sparsebit.c LIBKVM += lib/test_util.c +LIBKVM += lib/userfaultfd_util.c LIBKVM_STRING += lib/string_override.c
tools/testing/selftests/kvm/demand_paging_test.c +30 -198
296 lines not on this page · raw message
tools/testing/selftests/kvm/include/userfaultfd_util.h new file +45 -0
45 lines not on this page · raw message
tools/testing/selftests/kvm/lib/userfaultfd_util.c new file +186 -0
186 lines not on this page · raw message
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
Sean Christopherson
reviewerRe: [PATCH v10 01/14] KVM: selftests: Add a userfaultfd library
Quoted 31 lines · Ricardo Koller
tools/testing/selftests/kvm/demand_paging_test.c +2
-12@@ -22,23 +22,13 @@#include "test_util.h" #include "perf_test_util.h" #include "guest_modes.h" +#include "userfaultfd_util.h" #ifdef __NR_userfaultfd -#ifdef PRINT_PER_PAGE_UPDATES -#define PER_PAGE_DEBUG (...) printf(__VA_ARGS__) -#else -#define PER_PAGE_DEBUG (...) _no_printf(__VA_ARGS__) -#endif - -#ifdef PRINT_PER_VCPU_UPDATES -#define PER_VCPU_DEBUG (...) printf(__VA_ARGS__) -#else -#define PER_VCPU_DEBUG (...) _no_printf(__VA_ARGS__) -#endif -static int nr_vcpus =1 ;static uint64_t guest_percpu_mem_size =DEFAULT_PER_VCPU_MEM_SIZE ; +static size_t demand_paging_size;static char *guest_data_prototype;
tools/testing/selftests/kvm/demand_paging_test.c static void vcpu_worker(struct perf_test_vcpu_args *vcpu_args) · line 61
Quoted 6 lines
tools/testing/selftests/kvm/demand_paging_test.c +2
-1@@ -67,9 +57,11 @@ static void vcpu_worker(struct perf_test_vcpu_args *vcpu_args) ts_diff.tv_sec, ts_diff.tv_nsec); } -static int handle_uffd_page_request (int uffd_mode,int uffd,uint64_t addr) +static int handle_uffd_page_request (int uffd_mode,int uffd, +struct uffd_msg *msg)
Heh, one last alignment goof.
PATCH v10 2/14 KVM: selftests: aarch64: Add virt_get_pte_hva() library function
In reply to Ricardo KollerAdd a library function to get the PTE (a host virtual address) of a given GVA. This will be used in a future commit by a test to clear and check the access flag of a particular page.
--- .../selftests/kvm/include/aarch64/processor.h | 2 ++ tools/testing/selftests/kvm/lib/aarch64/processor.c | 13 ++++++++++--- 2 files changed, 12 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/include/aarch64/processor.h +2 -0
@@ -109,6 +109,8 @@ void vm_install_exception_handler(struct kvm_vm *vm,void vm_install_sync_handler (struct kvm_vm *vm,int vector,int ec,handler_fn handler); +uint64_t *virt_get_pte_hva (struct kvm_vm *vm,vm_vaddr_t gva); +static inline void cpu_relax (void ) { asmvolatile ("yield" :::"memory" );
tools/testing/selftests/kvm/lib/aarch64/processor.c +10 -3
@@ -138,7 +138,7 @@ void virt_arch_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr)_virt_pg_map (vm, vaddr, paddr, attr_idx); } -vm_paddr_t addr_arch_gva2gpa (struct kvm_vm *vm,vm_vaddr_t gva) +uint64_t *virt_get_pte_hva (struct kvm_vm *vm,vm_vaddr_t gva) {uint64_t *ptep;
@@ -169,11 +169,18 @@ vm_paddr_t addr_arch_gva2gpa(struct kvm_vm *vm, vm_vaddr_t gva)TEST_FAIL ("Page table levels must be 2, 3, or 4" ); } -return pte_addr (vm, *ptep) + (gva & (vm->page_size -1 )); +return ptep; unmapped_gva:TEST_FAIL ("No mapping for vm virtual address, gva: 0x%lx" , gva); -exit (1 ); +exit (EXIT_FAILURE ); +} + +vm_paddr_t addr_arch_gva2gpa (struct kvm_vm *vm,vm_vaddr_t gva) +{ +uint64_t *ptep =virt_get_pte_hva (vm, gva); + +return pte_addr (vm, *ptep) + (gva & (vm->page_size -1 )); }static void pte_dump (FILE *stream,struct kvm_vm *vm,uint8_t indent,uint64_t page,int level)
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 3/14 KVM: selftests: Add missing close and munmap in __vm_mem_region_delete()
In reply to Ricardo KollerDeleting a memslot (when freeing a VM) is not closing the backing fd, nor it's unmapping the alias mapping. Fix by adding the missing close and munmap.
--- tools/testing/selftests/kvm/lib/kvm_util.c | 6 ++++++ 1 file changed, 6 insertions(+)
tools/testing/selftests/kvm/lib/kvm_util.c +6 -0
@@ -586,6 +586,12 @@ static void __vm_mem_region_delete(struct kvm_vm *vm,sparsebit_free (®ion->unused_phy_pages); ret =munmap (region->mmap_start, region->mmap_size);TEST_ASSERT (!ret,__KVM_SYSCALL_ERROR ("munmap()" , ret)); +if (region->fd >=0 ) { +/* There's an extra map when using shared memory. */ + ret =munmap (region->mmap_alias, region->mmap_size); +TEST_ASSERT (!ret,__KVM_SYSCALL_ERROR ("munmap()" , ret)); +close (region->fd); + }free (region); }
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 4/14 KVM: selftests: aarch64: Construct DEFAULT_MAIR_EL1 using sysreg.h macros
In reply to Ricardo KollerDefine macros for memory type indexes and construct DEFAULT_MAIR_EL1 with macros from asm/sysreg.h. The index macros can then be used when constructing PTEs (instead of using raw numbers).
--- .../selftests/kvm/include/aarch64/processor.h | 25 ++++++++++++++----- .../selftests/kvm/lib/aarch64/processor.c | 2 +- 2 files changed, 20 insertions(+), 7 deletions(-)
tools/testing/selftests/kvm/include/aarch64/processor.h +19 -6
@@ -38,12 +38,25 @@* NORMAL 4 1111:1111 * NORMAL_WT 5 1011:1011 */ -#define DEFAULT_MAIR_EL1 ((0x00ul << (0 * 8)) | \ - (0x04ul << (1 * 8)) | \ - (0x0cul << (2 * 8)) | \ - (0x44ul << (3 * 8)) | \ - (0xfful << (4 * 8)) | \ - (0xbbul << (5 * 8))) + +/* Linux doesn't use these memory types, so let's define them. */ +#define MAIR_ATTR_DEVICE_GRE UL(0x0c) +#define MAIR_ATTR_NORMAL_WT UL(0xbb) + +#define MT_DEVICE_nGnRnE 0 +#define MT_DEVICE_nGnRE 1 +#define MT_DEVICE_GRE 2 +#define MT_NORMAL_NC 3 +#define MT_NORMAL 4 +#define MT_NORMAL_WT 5 + +#define DEFAULT_MAIR_EL1 \ + (MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRnE) | \ + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRE, MT_DEVICE_nGnRE) | \ + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_GRE, MT_DEVICE_GRE) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_NC, MT_NORMAL_NC) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT))#define MPIDR_HWID_BITMASK (0xff00fffffful)
tools/testing/selftests/kvm/lib/aarch64/processor.c +1 -1
@@ -133,7 +133,7 @@ static void _virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,void virt_arch_pg_map (struct kvm_vm *vm,uint64_t vaddr,uint64_t paddr) { -uint64_t attr_idx =4 ;/* NORMAL (See DEFAULT_MAIR_EL1) */ +uint64_t attr_idx =MT_NORMAL ;_virt_pg_map (vm, vaddr, paddr, attr_idx); }
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 5/14 tools: Copy bitfield.h from the kernel sources
In reply to Ricardo KollerCopy bitfield.h from include/linux/bitfield.h. A subsequent change will make use of some FIELD_{GET,PREP} macros defined in this header.
The header was copied as-is, no changes needed.
--- tools/include/linux/bitfield.h | 176 +++++++++++++++++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100644 tools/include/linux/bitfield.h
tools/include/linux/bitfield.h new file +176 -0
@@ -0,0 +1,176 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* Copyright (C) 2014 Felix Fietkau <[email protected]> +* Copyright (C) 2004 - 2009 Ivo van Doorn <[email protected]> +*/ + +#ifndef _LINUX_BITFIELD_H +#define _LINUX_BITFIELD_H + +#include <linux/build_bug.h> +#include <asm/byteorder.h> + +/* +* Bitfield access macros +* +* FIELD_{GET,PREP} macros take as first parameter shifted mask +* from which they extract the base mask and shift amount. +* Mask must be a compilation time constant. +* +* Example: +* +* #define REG_FIELD_A GENMASK(6, 0) +* #define REG_FIELD_B BIT(7) +* #define REG_FIELD_C GENMASK(15, 8) +* #define REG_FIELD_D GENMASK(31, 16) +* +* Get: +* a = FIELD_GET(REG_FIELD_A, reg); +* b = FIELD_GET(REG_FIELD_B, reg); +* +* Set: +* reg = FIELD_PREP(REG_FIELD_A, 1) | +* FIELD_PREP(REG_FIELD_B, 0) | +* FIELD_PREP(REG_FIELD_C, c) | +* FIELD_PREP(REG_FIELD_D, 0x40); +* +* Modify: +* reg &= ~REG_FIELD_C; +* reg |= FIELD_PREP(REG_FIELD_C, c); +*/ + +#define __bf_shf (x) (__builtin_ffsll(x) - 1) + +#define __scalar_type_to_unsigned_cases (type) \ + unsigned type: (unsigned type)0, \ + signed type: (unsigned type)0 + +#define __unsigned_scalar_typeof (x) typeof( \ + _Generic((x), \ + char: (unsigned char)0, \ + __scalar_type_to_unsigned_cases(char), \ + __scalar_type_to_unsigned_cases(short), \ + __scalar_type_to_unsigned_cases(int), \ + __scalar_type_to_unsigned_cases(long), \ + __scalar_type_to_unsigned_cases(long long), \ + default: (x))) + +#define __bf_cast_unsigned (type, x) ((__unsigned_scalar_typeof(type))(x)) + +#define __BF_FIELD_CHECK (_mask, _reg, _val, _pfx) \ + ({ \ + BUILD_BUG_ON_MSG(!__builtin_constant_p(_mask), \ + _pfx "mask is not constant"); \ + BUILD_BUG_ON_MSG((_mask) == 0, _pfx "mask is zero"); \ + BUILD_BUG_ON_MSG(__builtin_constant_p(_val) ? \ + ~((_mask) >> __bf_shf(_mask)) & (_val) : 0, \ + _pfx "value too large for the field"); \ + BUILD_BUG_ON_MSG(__bf_cast_unsigned(_mask, _mask) > \ + __bf_cast_unsigned(_reg, ~0ull), \ + _pfx "type of reg too small for mask"); \ + __BUILD_BUG_ON_NOT_POWER_OF_2((_mask) + \ + (1ULL << __bf_shf(_mask))); \ + }) + +/** +* FIELD_MAX() - produce the maximum value representable by a field +* @_mask: shifted mask defining the field's length and position +* +* FIELD_MAX() returns the maximum value that can be held in the field +* specified by @_mask. +*/ +#define FIELD_MAX (_mask) \ + ({ \ + __BF_FIELD_CHECK(_mask, 0ULL, 0ULL, "FIELD_MAX: "); \ + (typeof(_mask))((_mask) >> __bf_shf(_mask)); \ + }) + +/** +* FIELD_FIT() - check if value fits in the field +* @_mask: shifted mask defining the field's length and position +* @_val: value to test against the field +* +* Return: true if @_val can fit inside @_mask, false if @_val is too big. +*/ +#define FIELD_FIT (_mask, _val) \ + ({ \ + __BF_FIELD_CHECK(_mask, 0ULL, 0ULL, "FIELD_FIT: "); \ + !((((typeof(_mask))_val) << __bf_shf(_mask)) & ~(_mask)); \ + }) + +/** +* FIELD_PREP() - prepare a bitfield element +* @_mask: shifted mask defining the field's length and position +* @_val: value to put in the field +* +* FIELD_PREP() masks and shifts up the value. The result should +* be combined with other fields of the bitfield using logical OR. +*/ +#define FIELD_PREP (_mask, _val) \ + ({ \ + __BF_FIELD_CHECK(_mask, 0ULL, _val, "FIELD_PREP: "); \ + ((typeof(_mask))(_val) << __bf_shf(_mask)) & (_mask); \ + }) + +/** +* FIELD_GET() - extract a bitfield element +* @_mask: shifted mask defining the field's length and position +* @_reg: value of entire bitfield +* +* FIELD_GET() extracts the field specified by @_mask from the +* bitfield passed in as @_reg by masking and shifting it down. +*/ +#define FIELD_GET (_mask, _reg) \ + ({ \ + __BF_FIELD_CHECK(_mask, _reg, 0U, "FIELD_GET: "); \ + (typeof(_mask))(((_reg) & (_mask)) >> __bf_shf(_mask)); \ + }) + +extern void __compiletime_error("value doesn't fit into mask") +__field_overflow(void); +extern void __compiletime_error("bad bitfield mask") +__bad_mask(void); +static __always_inline u64 field_multiplier(u64 field) +{ +if ((field | (field -1 )) & ((field | (field -1 )) +1 )) +__bad_mask (); +return field & -field; +} +static __always_inline u64 field_mask(u64 field) +{ +return field /field_multiplier (field); +} +#define field_max (field) ((typeof(field))field_mask(field)) +#define ____MAKE_OP (type,base,to,from) \ +static __always_inline __##type type##_encode_bits(base v, base field) \ +{ \ + if (__builtin_constant_p(v) && (v & ~field_mask(field))) \ + __field_overflow(); \ + return to((v & field_mask(field)) * field_multiplier(field)); \ +} \ +static __always_inline __##type type##_replace_bits(__##type old, \ + base val, base field) \ +{ \ + return (old & ~to(field)) | type##_encode_bits(val, field); \ +} \ +static __always_inline void type##p_replace_bits(__##type *p, \ + base val, base field) \ +{ \ + *p = (*p & ~to(field)) | type##_encode_bits(val, field); \ +} \ +static __always_inline base type##_get_bits(__##type v, base field) \ +{ \ + return (from(v) & field)/field_multiplier(field); \ +} +#define __MAKE_OP (size) \ + ____MAKE_OP(le##size,u##size,cpu_to_le##size,le##size##_to_cpu) \ + ____MAKE_OP(be##size,u##size,cpu_to_be##size,be##size##_to_cpu) \ + ____MAKE_OP(u##size,u##size,,) +____MAKE_OP(u8,u8,,) +__MAKE_OP(16) +__MAKE_OP(32) +__MAKE_OP(64) +#undef __MAKE_OP +#undef ____MAKE_OP + +#endif
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 6/14 KVM: selftests: Stash backing_src_type in struct userspace_mem_region
In reply to Ricardo KollerAdd the backing_src_type into struct userspace_mem_region. This struct already stores a lot of info about memory regions, except the backing source type. This info will be used by a future commit in order to determine the method for punching a hole.
--- tools/testing/selftests/kvm/include/kvm_util_base.h | 1 + tools/testing/selftests/kvm/lib/kvm_util.c | 1 + 2 files changed, 2 insertions(+)
tools/testing/selftests/kvm/include/kvm_util_base.h +1 -0
@@ -34,6 +34,7 @@ struct userspace_mem_region {struct sparsebit *unused_phy_pages;int fd;off_t offset; +enum vm_mem_backing_src_type backing_src_type;void *host_mem;void *host_alias;void *mmap_start;
tools/testing/selftests/kvm/lib/kvm_util.c +1 -0
@@ -929,6 +929,7 @@ void vm_userspace_mem_region_add(struct kvm_vm *vm, vm_mem_backing_src_alias(src_type)->name); } + region->backing_src_type = src_type; region->unused_phy_pages =sparsebit_alloc ();sparsebit_set_num (region->unused_phy_pages, guest_paddr >> vm->page_shift, npages);
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 7/14 KVM: selftests: Add vm->memslots[] and enum kvm_mem_region_type
In reply to Ricardo KollerThe vm_create() helpers are hardcoded to place most page types (code, page-tables, stacks, etc) in the same memslot #0, and always backed with anonymous 4K. There are a couple of issues with that. First, tests willing to differ a bit, like placing page-tables in a different backing source type must replicate much of what's already done by the vm_create() functions. Second, the hardcoded assumption of memslot #0 holding most things is spread everywhere; this makes it very hard to change.
Fix the above issues by having selftests specify how they want memory to be laid out. Start by changing ____vm_create() to not create memslot #0; a test (to come) will specify all memslots used by the VM. Then, add the vm->memslots[] array to specify the right memslot for different memory allocators, e.g.,: lib/elf should use the vm->[MEM_REGION_CODE] memslot. This will be used as a way to specify the page-tables memslots (to be backed by huge pages for example).
There is no functional change intended. The current commit lays out memory exactly as before. A future commit will change the allocators to get the region they should be using, e.g.,: like the page table allocators using the pt memslot.
--- .../selftests/kvm/include/kvm_util_base.h | 26 +++++++++++++++++-- tools/testing/selftests/kvm/lib/kvm_util.c | 18 +++++++------ 2 files changed, 34 insertions(+), 10 deletions(-)
tools/testing/selftests/kvm/include/kvm_util_base.h +24 -2
@@ -65,6 +65,14 @@ struct userspace_mem_regions {DECLARE_HASHTABLE (slot_hash,9 ); }; +enum kvm_mem_region_type { +MEM_REGION_CODE , +MEM_REGION_DATA , +MEM_REGION_PT , +MEM_REGION_TEST_DATA , +NR_MEM_REGIONS , +}; +struct kvm_vm {int mode;unsigned long type;
@@ -93,6 +101,13 @@ struct kvm_vm {int stats_fd;struct kvm_stats_header stats_header;struct kvm_stats_desc *stats_desc; + +/* +* KVM region slots. These are the default memslots used by page +* allocators, e.g., lib/elf uses the memslots[MEM_REGION_CODE] +* memslot. +*/ +uint32_t memslots[NR_MEM_REGIONS ]; };
@@ -105,6 +120,13 @@ struct kvm_vm {struct userspace_mem_region *memslot2region (struct kvm_vm *vm,uint32_t memslot); +static inline struct userspace_mem_region *vm_get_mem_region (struct kvm_vm *vm, +enum kvm_mem_region_type type) +{ +assert (type <NR_MEM_REGIONS ); +return memslot2region (vm, vm->memslots[type]); +} +/* Minimum allocated guest virtual and physical addresses */ #define KVM_UTIL_MIN_VADDR 0x2000#define KVM_GUEST_PAGE_TABLE_MIN_PADDR 0x180000
@@ -647,13 +669,13 @@ vm_paddr_t vm_alloc_page_table(struct kvm_vm *vm);* __vm_create() does NOT create vCPUs, @nr_runnable_vcpus is used purely to * calculate the amount of memory needed for per-vCPU data, e.g. stacks. */ -struct kvm_vm *____vm_create (enum vm_guest_mode mode,uint64_t nr_pages); +struct kvm_vm *____vm_create (enum vm_guest_mode mode);struct kvm_vm *__vm_create (enum vm_guest_mode mode,uint32_t nr_runnable_vcpus,uint64_t nr_extra_pages);static inline struct kvm_vm *vm_create_barebones (void ) { -return ____vm_create (VM_MODE_DEFAULT ,0 ); +return ____vm_create (VM_MODE_DEFAULT ); }static inline struct kvm_vm *vm_create (uint32_t nr_runnable_vcpus)
tools/testing/selftests/kvm/lib/kvm_util.c +10 -8
@@ -185,13 +185,10 @@ const struct vm_guest_mode_params vm_guest_mode_params[] = {_Static_assert (sizeof (vm_guest_mode_params)/sizeof (struct vm_guest_mode_params ) ==NUM_VM_MODES ,"Missing new mode params?" ); -struct kvm_vm *____vm_create (enum vm_guest_mode mode,uint64_t nr_pages) +struct kvm_vm *____vm_create (enum vm_guest_mode mode) {struct kvm_vm *vm; -pr_debug ("%s: mode='%s' pages='%ld'\n" , __func__, -vm_guest_mode_string (mode), nr_pages); - vm =calloc (1 ,sizeof (*vm));TEST_ASSERT (vm !=NULL ,"Insufficient Memory" );
@@ -287,9 +284,6 @@ struct kvm_vm *____vm_create(enum vm_guest_mode mode, uint64_t nr_pages)/* Allocate and setup memory for guest. */ vm->vpages_mapped =sparsebit_alloc (); -if (nr_pages !=0 ) -vm_userspace_mem_region_add (vm,VM_MEM_SRC_ANONYMOUS , -0 ,0 , nr_pages,0 );return vm; }
@@ -335,8 +329,16 @@ struct kvm_vm *__vm_create(enum vm_guest_mode mode, uint32_t nr_runnable_vcpus,uint64_t nr_pages =vm_nr_pages_required (mode, nr_runnable_vcpus, nr_extra_pages);struct kvm_vm *vm; +int i; + +pr_debug ("%s: mode='%s' pages='%ld'\n" , __func__, +vm_guest_mode_string (mode), nr_pages); + + vm =____vm_create (mode); - vm =____vm_create (mode, nr_pages); +vm_userspace_mem_region_add (vm,VM_MEM_SRC_ANONYMOUS ,0 ,0 , nr_pages,0 ); +for (i =0 ; i <NR_MEM_REGIONS ; i++) + vm->memslots[i] =0 ;kvm_vm_elf_load (vm, program_invocation_name);
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 8/14 KVM: selftests: Fix alignment in virt_arch_pgd_alloc() and vm_vaddr_alloc()
In reply to Ricardo KollerRefactor virt_arch_pgd_alloc() and vm_vaddr_alloc() in both RISC-V and aarch64 to fix the alignment of parameters in a couple of calls. This will make it easier to fix the alignment in a future commit that adds an extra parameter (that happens to be very long).
No functional change intended.
--- .../selftests/kvm/lib/aarch64/processor.c | 27 ++++++++++--------- .../selftests/kvm/lib/riscv/processor.c | 27 ++++++++++--------- 2 files changed, 30 insertions(+), 24 deletions(-)
tools/testing/selftests/kvm/lib/aarch64/processor.c +15 -12
@@ -76,13 +76,14 @@ static uint64_t __maybe_unused ptrs_per_pte(struct kvm_vm *vm)void virt_arch_pgd_alloc (struct kvm_vm *vm) { -if (!vm->pgd_created) { -vm_paddr_t paddr =vm_phy_pages_alloc (vm, -page_align (vm,ptrs_per_pgd (vm) *8 ) / vm->page_size, -KVM_GUEST_PAGE_TABLE_MIN_PADDR ,0 ); - vm->pgd = paddr; - vm->pgd_created = true; - } +size_t nr_pages =page_align (vm,ptrs_per_pgd (vm) *8 ) / vm->page_size; + +if (vm->pgd_created) +return ; + + vm->pgd =vm_phy_pages_alloc (vm, nr_pages, +KVM_GUEST_PAGE_TABLE_MIN_PADDR ,0 ); + vm->pgd_created = true; } static void _virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,
@@ -325,13 +326,15 @@ void vcpu_arch_dump(FILE *stream, struct kvm_vcpu *vcpu, uint8_t indent)struct kvm_vcpu *aarch64_vcpu_add (struct kvm_vm *vm,uint32_t vcpu_id,struct kvm_vcpu_init *init,void *guest_code) { -size_t stack_size = vm->page_size ==4096 ? -DEFAULT_STACK_PGS * vm->page_size : - vm->page_size; -uint64_t stack_vaddr =vm_vaddr_alloc (vm, stack_size, -DEFAULT_ARM64_GUEST_STACK_VADDR_MIN ); +size_t stack_size; +uint64_t stack_vaddr;struct kvm_vcpu *vcpu =__vm_vcpu_add (vm, vcpu_id); + stack_size = vm->page_size ==4096 ?DEFAULT_STACK_PGS * vm->page_size : + vm->page_size; + stack_vaddr =vm_vaddr_alloc (vm, stack_size, +DEFAULT_ARM64_GUEST_STACK_VADDR_MIN ); +aarch64_vcpu_setup (vcpu, init);vcpu_set_reg (vcpu,ARM64_CORE_REG (sp_el1), stack_vaddr + stack_size);
tools/testing/selftests/kvm/lib/riscv/processor.c +15 -12
@@ -55,13 +55,14 @@ static uint64_t pte_index(struct kvm_vm *vm, vm_vaddr_t gva, int level)void virt_arch_pgd_alloc (struct kvm_vm *vm) { -if (!vm->pgd_created) { -vm_paddr_t paddr =vm_phy_pages_alloc (vm, -page_align (vm,ptrs_per_pte (vm) *8 ) / vm->page_size, -KVM_GUEST_PAGE_TABLE_MIN_PADDR ,0 ); - vm->pgd = paddr; - vm->pgd_created = true; - } +size_t nr_pages =page_align (vm,ptrs_per_pte (vm) *8 ) / vm->page_size; + +if (vm->pgd_created) +return ; + + vm->pgd =vm_phy_pages_alloc (vm, nr_pages, +KVM_GUEST_PAGE_TABLE_MIN_PADDR ,0 ); + vm->pgd_created = true; }void virt_arch_pg_map (struct kvm_vm *vm,uint64_t vaddr,uint64_t paddr)
@@ -279,15 +280,17 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id, void *guest_code) {int r; -size_t stack_size = vm->page_size ==4096 ? -DEFAULT_STACK_PGS * vm->page_size : - vm->page_size; -unsigned long stack_vaddr =vm_vaddr_alloc (vm, stack_size, -DEFAULT_RISCV_GUEST_STACK_VADDR_MIN ); +size_t stack_size; +unsigned long stack_vaddr;unsigned long current_gp =0 ;struct kvm_mp_state mps;struct kvm_vcpu *vcpu; + stack_size = vm->page_size ==4096 ?DEFAULT_STACK_PGS * vm->page_size : + vm->page_size; + stack_vaddr =vm_vaddr_alloc (vm, stack_size, +DEFAULT_RISCV_GUEST_STACK_VADDR_MIN ); + vcpu =__vm_vcpu_add (vm, vcpu_id);riscv_vcpu_mmu_setup (vcpu);
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 9/14 KVM: selftests: Use the right memslot for code, page-tables, and data allocations
In reply to Ricardo KollerNow that kvm_vm allows specifying different memslots for code, page tables, and data, use the appropriate memslot when making allocations in common/libraty code. Change them accordingly:
- code (allocated by lib/elf) use the CODE memslot
- stacks, exception tables, and other core data pages (like the TSS in x86) use the DATA memslot
- page tables and the PGD use the PT memslot
- test data (anything allocated with vm_vaddr_alloc()) uses the TEST_DATA memslot
No functional change intended. All allocators keep using memslot #0.
--- .../selftests/kvm/include/kvm_util_base.h | 4 ++ .../selftests/kvm/lib/aarch64/processor.c | 12 ++-- tools/testing/selftests/kvm/lib/elf.c | 3 +- tools/testing/selftests/kvm/lib/kvm_util.c | 57 ++++++++++++------- .../selftests/kvm/lib/riscv/processor.c | 8 ++- .../selftests/kvm/lib/s390x/processor.c | 8 ++- .../selftests/kvm/lib/x86_64/processor.c | 13 +++-- 7 files changed, 65 insertions(+), 40 deletions(-)
tools/testing/selftests/kvm/include/kvm_util_base.h +4 -0
@@ -407,7 +407,11 @@ void vm_mem_region_move(struct kvm_vm *vm, uint32_t slot, uint64_t new_gpa);void vm_mem_region_delete (struct kvm_vm *vm,uint32_t slot);struct kvm_vcpu *__vm_vcpu_add (struct kvm_vm *vm,uint32_t vcpu_id);vm_vaddr_t vm_vaddr_alloc (struct kvm_vm *vm,size_t sz,vm_vaddr_t vaddr_min); +vm_vaddr_t __vm_vaddr_alloc (struct kvm_vm *vm,size_t sz,vm_vaddr_t vaddr_min, +enum kvm_mem_region_type type);vm_vaddr_t vm_vaddr_alloc_pages (struct kvm_vm *vm,int nr_pages); +vm_vaddr_t __vm_vaddr_alloc_page (struct kvm_vm *vm, +enum kvm_mem_region_type type);vm_vaddr_t vm_vaddr_alloc_page (struct kvm_vm *vm); void virt_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,
tools/testing/selftests/kvm/lib/aarch64/processor.c +7 -5
@@ -82,7 +82,8 @@ void virt_arch_pgd_alloc(struct kvm_vm *vm)return ; vm->pgd =vm_phy_pages_alloc (vm, nr_pages, -KVM_GUEST_PAGE_TABLE_MIN_PADDR ,0 ); +KVM_GUEST_PAGE_TABLE_MIN_PADDR , + vm->memslots[MEM_REGION_PT ]); vm->pgd_created = true; }
@@ -332,8 +333,9 @@ struct kvm_vcpu *aarch64_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id, stack_size = vm->page_size ==4096 ?DEFAULT_STACK_PGS * vm->page_size : vm->page_size; - stack_vaddr =vm_vaddr_alloc (vm, stack_size, -DEFAULT_ARM64_GUEST_STACK_VADDR_MIN ); + stack_vaddr =__vm_vaddr_alloc (vm, stack_size, +DEFAULT_ARM64_GUEST_STACK_VADDR_MIN , +MEM_REGION_DATA );aarch64_vcpu_setup (vcpu, init);
@@ -438,8 +440,8 @@ void route_exception(struct ex_regs *regs, int vector)void vm_init_descriptor_tables (struct kvm_vm *vm) { - vm->handlers =vm_vaddr_alloc (vm,sizeof (struct handlers ), - vm->page_size); + vm->handlers =__vm_vaddr_alloc (vm,sizeof (struct handlers ), + vm->page_size,MEM_REGION_DATA ); *(vm_vaddr_t *)addr_gva2hva (vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers; }
tools/testing/selftests/kvm/lib/elf.c +2 -1
@@ -161,7 +161,8 @@ void kvm_vm_elf_load(struct kvm_vm *vm, const char *filename) seg_vend |= vm->page_size -1 ;size_t seg_size = seg_vend - seg_vstart +1 ; -vm_vaddr_t vaddr =vm_vaddr_alloc (vm, seg_size, seg_vstart); +vm_vaddr_t vaddr =__vm_vaddr_alloc (vm, seg_size, seg_vstart, +MEM_REGION_CODE );TEST_ASSERT (vaddr == seg_vstart,"Unable to allocate " "virtual memory for segment at requested min addr,\n" " segment idx: %u\n"
tools/testing/selftests/kvm/lib/kvm_util.c +35 -22
86 lines not on this page · raw message
tools/testing/selftests/kvm/lib/riscv/processor.c +5 -3
@@ -61,7 +61,8 @@ void virt_arch_pgd_alloc(struct kvm_vm *vm)return ; vm->pgd =vm_phy_pages_alloc (vm, nr_pages, -KVM_GUEST_PAGE_TABLE_MIN_PADDR ,0 ); +KVM_GUEST_PAGE_TABLE_MIN_PADDR , + vm->memslots[MEM_REGION_PT ]); vm->pgd_created = true; }
@@ -288,8 +289,9 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id, stack_size = vm->page_size ==4096 ?DEFAULT_STACK_PGS * vm->page_size : vm->page_size; - stack_vaddr =vm_vaddr_alloc (vm, stack_size, -DEFAULT_RISCV_GUEST_STACK_VADDR_MIN ); + stack_vaddr =__vm_vaddr_alloc (vm, stack_size, +DEFAULT_RISCV_GUEST_STACK_VADDR_MIN , +MEM_REGION_DATA ); vcpu =__vm_vcpu_add (vm, vcpu_id);riscv_vcpu_mmu_setup (vcpu);
tools/testing/selftests/kvm/lib/s390x/processor.c +5 -3
@@ -21,7 +21,8 @@ void virt_arch_pgd_alloc(struct kvm_vm *vm)return ; paddr =vm_phy_pages_alloc (vm,PAGES_PER_REGION , -KVM_GUEST_PAGE_TABLE_MIN_PADDR ,0 ); +KVM_GUEST_PAGE_TABLE_MIN_PADDR , + vm->memslots[MEM_REGION_PT ]);memset (addr_gpa2hva (vm, paddr),0xff ,PAGES_PER_REGION * vm->page_size); vm->pgd = paddr;
@@ -167,8 +168,9 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id,TEST_ASSERT (vm->page_size ==4096 ,"Unsupported page size: 0x%x" , vm->page_size); - stack_vaddr =vm_vaddr_alloc (vm, stack_size, -DEFAULT_GUEST_STACK_VADDR_MIN ); + stack_vaddr =__vm_vaddr_alloc (vm, stack_size, +DEFAULT_GUEST_STACK_VADDR_MIN , +MEM_REGION_DATA ); vcpu =__vm_vcpu_add (vm, vcpu_id);
tools/testing/selftests/kvm/lib/x86_64/processor.c +7 -6
@@ -552,7 +552,7 @@ vm_paddr_t addr_arch_gva2gpa(struct kvm_vm *vm, vm_vaddr_t gva)static void kvm_setup_gdt (struct kvm_vm *vm,struct kvm_dtable *dt) {if (!vm->gdt) - vm->gdt =vm_vaddr_alloc_page (vm); + vm->gdt =__vm_vaddr_alloc_page (vm,MEM_REGION_DATA ); dt->base = vm->gdt; dt->limit =getpagesize ();
@@ -562,7 +562,7 @@ static void kvm_setup_tss_64bit(struct kvm_vm *vm, struct kvm_segment *segp, int selector) {if (!vm->tss) - vm->tss =vm_vaddr_alloc_page (vm); + vm->tss =__vm_vaddr_alloc_page (vm,MEM_REGION_DATA );memset (segp,0 ,sizeof (*segp)); segp->base = vm->tss;
@@ -647,8 +647,9 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id,vm_vaddr_t stack_vaddr;struct kvm_vcpu *vcpu; - stack_vaddr =vm_vaddr_alloc (vm,DEFAULT_STACK_PGS *getpagesize (), -DEFAULT_GUEST_STACK_VADDR_MIN ); + stack_vaddr =__vm_vaddr_alloc (vm,DEFAULT_STACK_PGS *getpagesize (), +DEFAULT_GUEST_STACK_VADDR_MIN , +MEM_REGION_DATA ); vcpu =__vm_vcpu_add (vm, vcpu_id);vcpu_init_cpuid (vcpu,kvm_get_supported_cpuid ());
@@ -1145,8 +1146,8 @@ void vm_init_descriptor_tables(struct kvm_vm *vm)extern void *idt_handlers;int i; - vm->idt =vm_vaddr_alloc_page (vm); - vm->handlers =vm_vaddr_alloc_page (vm); + vm->idt =__vm_vaddr_alloc_page (vm,MEM_REGION_DATA ); + vm->handlers =__vm_vaddr_alloc_page (vm,MEM_REGION_DATA );/* Handlers have the same address in both address spaces.*/ for (i =0 ; i <NUM_INTERRUPTS ; i++) set_idt_entry(vm, i, (unsigned long)(&idt_handlers)[i], 0,
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 10/14 KVM: selftests: aarch64: Add aarch64/page_fault_test
In reply to Ricardo KollerAdd a new test for stage 2 faults when using different combinations of guest accesses (e.g., write, S1PTW), backing source type (e.g., anon) and types of faults (e.g., read on hugetlbfs with a hole). The next commits will add different handling methods and more faults (e.g., uffd and dirty logging). This first commit starts by adding two sanity checks for all types of accesses: AF setting by the hw, and accessing memslots with holes.
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/aarch64/page_fault_test.c | 594 ++++++++++++++++++ .../selftests/kvm/include/aarch64/processor.h | 8 + 4 files changed, 604 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/page_fault_test.c
tools/testing/selftests/kvm/.gitignore +1 -0
@@ -4,6 +4,7 @@ /aarch64/debug-exceptions /aarch64/get-reg-list /aarch64/hypercalls +/aarch64/page_fault_test /aarch64/psci_test /aarch64/vcpu_width_config /aarch64/vgic_init
tools/testing/selftests/kvm/Makefile +1 -0
@@ -153,6 +153,7 @@ TEST_GEN_PROGS_aarch64 += aarch64/arch_timer TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list TEST_GEN_PROGS_aarch64 += aarch64/hypercalls +TEST_GEN_PROGS_aarch64 += aarch64/page_fault_test TEST_GEN_PROGS_aarch64 += aarch64/psci_test TEST_GEN_PROGS_aarch64 += aarch64/vcpu_width_config TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
tools/testing/selftests/kvm/aarch64/page_fault_test.c new file +594 -0
594 lines not on this page · raw message
tools/testing/selftests/kvm/include/aarch64/processor.h +8 -0
@@ -105,11 +105,19 @@ enum {#define ESR_EC_MASK (ESR_EC_NUM - 1)#define ESR_EC_SVC64 0x15 +#define ESR_EC_IABT 0x21 +#define ESR_EC_DABT 0x25#define ESR_EC_HW_BP_CURRENT 0x31#define ESR_EC_SSTEP_CURRENT 0x33#define ESR_EC_WP_CURRENT 0x35#define ESR_EC_BRK_INS 0x3c +/* Access flag */ +#define PTE_AF (1ULL << 10) + +/* Access flag update enable/disable */ +#define TCR_EL1_HA (1ULL << 39) +void aarch64_get_supported_page_sizes (uint32_t ipa,bool *ps4k,bool *ps16k,bool *ps64k);
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 11/14 KVM: selftests: aarch64: Add userfaultfd tests into page_fault_test
In reply to Ricardo KollerAdd some userfaultfd tests into page_fault_test. Punch holes into the data and/or page-table memslots, perform some accesses, and check that the faults are taken (or not taken) when expected.
--- .../selftests/kvm/aarch64/page_fault_test.c | 187 ++++++++++++++++++ 1 file changed, 187 insertions(+)
tools/testing/selftests/kvm/aarch64/page_fault_test.c +187 -0
@@ -35,6 +35,12 @@ static uint64_t *guest_test_memory = (uint64_t *)TEST_GVA;#define PREPARE_FN_NR 10#define CHECK_FN_NR 10 +static struct event_cnt { +int uffd_faults; +/* uffd_faults is incremented from multiple threads. */ +pthread_mutex_t uffd_faults_mutex; +} events; +struct test_desc {const char *name;uint64_t mem_mark_cmd;
@@ -42,11 +48,14 @@ struct test_desc {bool (*guest_prepare[PREPARE_FN_NR ])(void );void (*guest_test)(void );void (*guest_test_check[CHECK_FN_NR ])(void ); +uffd_handler_t uffd_pt_handler; +uffd_handler_t uffd_data_handler;void (*dabt_handler)(struct ex_regs *regs);void (*iabt_handler)(struct ex_regs *regs);uint32_t pt_memslot_flags;uint32_t data_memslot_flags;bool skip; +struct event_cnt expected_events; };struct test_params {
@@ -263,7 +272,110 @@ static void no_iabt_handler(struct ex_regs *regs)GUEST_ASSERT_1 (false, regs->pc); } +static struct uffd_args { +char *copy; +void *hva; +uint64_t paging_size; +} pt_args, data_args; +/* Returns true to continue the test, and false if it should be skipped. */ +static int uffd_generic_handler (int uffd_mode,int uffd,struct uffd_msg *msg, +struct uffd_args *args,bool expect_write) +{ +uint64_t addr = msg->arg.pagefault.address; +uint64_t flags = msg->arg.pagefault.flags; +struct uffdio_copy copy; +int ret; + +TEST_ASSERT (uffd_mode ==UFFDIO_REGISTER_MODE_MISSING , +"The only expected UFFD mode is MISSING" ); +ASSERT_EQ (!!(flags &UFFD_PAGEFAULT_FLAG_WRITE ), expect_write); +ASSERT_EQ (addr, (uint64_t )args->hva); + +pr_debug ("uffd fault: addr=%p write=%d\n" , + (void *)addr, !!(flags &UFFD_PAGEFAULT_FLAG_WRITE )); + + copy.src = (uint64_t )args->copy; + copy.dst = addr; + copy.len = args->paging_size; + copy.mode =0 ; + + ret =ioctl (uffd,UFFDIO_COPY , ©); +if (ret ==-1 ) { +pr_info ("Failed UFFDIO_COPY in 0x%lx with errno: %d\n" , + addr, errno); +return ret; + } + +pthread_mutex_lock (&events.uffd_faults_mutex); + events.uffd_faults +=1 ; +pthread_mutex_unlock (&events.uffd_faults_mutex); +return 0 ; +} + +static int uffd_pt_write_handler (int mode,int uffd,struct uffd_msg *msg) +{ +return uffd_generic_handler (mode, uffd, msg, &pt_args, true); +} + +static int uffd_data_write_handler (int mode,int uffd,struct uffd_msg *msg) +{ +return uffd_generic_handler (mode, uffd, msg, &data_args, true); +} + +static int uffd_data_read_handler (int mode,int uffd,struct uffd_msg *msg) +{ +return uffd_generic_handler (mode, uffd, msg, &data_args, false); +} + +static void setup_uffd_args (struct userspace_mem_region *region, +struct uffd_args *args) +{ + args->hva = (void *)region->region.userspace_addr; + args->paging_size = region->region.memory_size; + + args->copy =malloc (args->paging_size); +TEST_ASSERT (args->copy,"Failed to allocate data copy." ); +memcpy (args->copy, args->hva, args->paging_size); +} + +static void setup_uffd (struct kvm_vm *vm,struct test_params *p, +struct uffd_desc **pt_uffd,struct uffd_desc **data_uffd) +{ +struct test_desc *test = p->test_desc; +int uffd_mode =UFFDIO_REGISTER_MODE_MISSING ; + +setup_uffd_args (vm_get_mem_region (vm,MEM_REGION_PT ), &pt_args); +setup_uffd_args (vm_get_mem_region (vm,MEM_REGION_TEST_DATA ), &data_args); + + *pt_uffd =NULL ; +if (test->uffd_pt_handler) + *pt_uffd =uffd_setup_demand_paging (uffd_mode,0 , + pt_args.hva, + pt_args.paging_size, + test->uffd_pt_handler); + + *data_uffd =NULL ; +if (test->uffd_data_handler) + *data_uffd =uffd_setup_demand_paging (uffd_mode,0 , + data_args.hva, + data_args.paging_size, + test->uffd_data_handler); +} + +static void free_uffd (struct test_desc *test,struct uffd_desc *pt_uffd, +struct uffd_desc *data_uffd) +{ +if (test->uffd_pt_handler) +uffd_stop_demand_paging (pt_uffd); +if (test->uffd_data_handler) +uffd_stop_demand_paging (data_uffd); + +free (pt_args.copy); +free (data_args.copy); +} + +/* Returns false if the test should be skipped. */ static bool punch_hole_in_backing_store (struct kvm_vm *vm,struct userspace_mem_region *region) {
@@ -404,6 +516,11 @@ static void setup_memslots(struct kvm_vm *vm, struct test_params *p) vm->memslots[MEM_REGION_TEST_DATA ] =TEST_DATA_MEMSLOT ; } +static void check_event_counts (struct test_desc *test) +{ +ASSERT_EQ (test->expected_events.uffd_faults, events.uffd_faults); +} +static void print_test_banner (enum vm_guest_mode mode,struct test_params *p) {struct test_desc *test = p->test_desc;
@@ -414,6 +531,11 @@ static void print_test_banner(enum vm_guest_mode mode, struct test_params *p) vm_mem_backing_src_alias(p->src_type)->name); } +static void reset_event_counts (void ) +{ +memset (&events,0 ,sizeof (events)); +} +/* * This function either succeeds, skips the test (after setting test->skip), or * fails with a TEST_FAIL that aborts all tests.
@@ -453,6 +575,7 @@ static void run_test(enum vm_guest_mode mode, void *arg)struct test_desc *test = p->test_desc;struct kvm_vm *vm;struct kvm_vcpu *vcpu; +struct uffd_desc *pt_uffd, *data_uffd;print_test_banner (mode, p);
@@ -465,7 +588,16 @@ static void run_test(enum vm_guest_mode mode, void *arg)ucall_init (vm,NULL ); +reset_event_counts (); + +/* +* Set some code in the data memslot for the guest to execute (only +* applicable to the EXEC tests). This has to be done before +* setup_uffd() as that function copies the memslot data for the uffd +* handler. +*/ load_exec_code_for_test (vm); +setup_uffd (vm, p, &pt_uffd, &data_uffd);setup_abort_handlers (vm, vcpu, test);vcpu_args_set (vcpu,1 , test);
@@ -473,6 +605,14 @@ static void run_test(enum vm_guest_mode mode, void *arg)ucall_uninit (vm);kvm_vm_free (vm); +free_uffd (test, pt_uffd, data_uffd); + +/* +* Make sure we check the events after the uffd threads have exited, +* which means they updated their respective event counters. +*/ +if (!test->skip) +check_event_counts (test); }static void help (char *name)
@@ -488,6 +628,7 @@ static void help(char *name)#define SNAME (s) #s#define SCAT2 (a, b) SNAME(a ## _ ## b)#define SCAT3 (a, b, c) SCAT2(a, SCAT2(b, c)) +#define SCAT4 (a, b, c, d) SCAT2(a, SCAT3(b, c, d))#define _CHECK (_test) _CHECK_##_test#define _PREPARE (_test) _PREPARE_##_test
@@ -515,6 +656,21 @@ static void help(char *name) .mem_mark_cmd = _mark_cmd, \ .guest_test = _access, \ .guest_test_check = { _CHECK(_with_af) }, \ + .expected_events = { 0 }, \ +} + +#define TEST_UFFD (_access, _with_af, _mark_cmd, \ + _uffd_data_handler, _uffd_pt_handler, _uffd_faults) \ +{ \ + .name = SCAT4(uffd, _access, _with_af, #_mark_cmd), \ + .guest_prepare = { _PREPARE(_with_af), \ + _PREPARE(_access) }, \ + .guest_test = _access, \ + .mem_mark_cmd = _mark_cmd, \ + .guest_test_check = { _CHECK(_with_af) }, \ + .uffd_data_handler = _uffd_data_handler, \ + .uffd_pt_handler = _uffd_pt_handler, \ + .expected_events = { .uffd_faults = _uffd_faults, }, \ } static struct test_desc tests[] = {
@@ -545,6 +701,37 @@ static struct test_desc tests[] = {TEST_ACCESS (guest_at, no_af,CMD_HOLE_DATA ),TEST_ACCESS (guest_dc_zva, no_af,CMD_HOLE_DATA ), +/* +* Punch holes in the data and PT backing stores and mark them for +* userfaultfd handling. This should result in 2 faults: the access +* on the data backing store, and its respective S1 page table walk +* (S1PTW). +*/ +TEST_UFFD (guest_read64, with_af,CMD_HOLE_DATA |CMD_HOLE_PT , + uffd_data_read_handler, uffd_pt_write_handler,2 ), +/* no_af should also lead to a PT write. */ +TEST_UFFD (guest_read64, no_af,CMD_HOLE_DATA |CMD_HOLE_PT , + uffd_data_read_handler, uffd_pt_write_handler,2 ), +/* Note how that cas invokes the read handler. */ +TEST_UFFD (guest_cas, with_af,CMD_HOLE_DATA |CMD_HOLE_PT , + uffd_data_read_handler, uffd_pt_write_handler,2 ), +/* +* Can't test guest_at with_af as it's IMPDEF whether the AF is set. +* The S1PTW fault should still be marked as a write. +*/ +TEST_UFFD (guest_at, no_af,CMD_HOLE_DATA |CMD_HOLE_PT , + uffd_data_read_handler, uffd_pt_write_handler,1 ), +TEST_UFFD (guest_ld_preidx, with_af,CMD_HOLE_DATA |CMD_HOLE_PT , + uffd_data_read_handler, uffd_pt_write_handler,2 ), +TEST_UFFD (guest_write64, with_af,CMD_HOLE_DATA |CMD_HOLE_PT , + uffd_data_write_handler, uffd_pt_write_handler,2 ), +TEST_UFFD (guest_dc_zva, with_af,CMD_HOLE_DATA |CMD_HOLE_PT , + uffd_data_write_handler, uffd_pt_write_handler,2 ), +TEST_UFFD (guest_st_preidx, with_af,CMD_HOLE_DATA |CMD_HOLE_PT , + uffd_data_write_handler, uffd_pt_write_handler,2 ), +TEST_UFFD (guest_exec, with_af,CMD_HOLE_DATA |CMD_HOLE_PT , + uffd_data_read_handler, uffd_pt_write_handler,2 ), + { 0 } };
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 12/14 KVM: selftests: aarch64: Add dirty logging tests into page_fault_test
In reply to Ricardo KollerAdd some dirty logging tests into page_fault_test. Mark the data and/or page-table memory regions for dirty logging, perform some accesses, and check that the dirty log bits are set or clean when expected.
--- .../selftests/kvm/aarch64/page_fault_test.c | 76 +++++++++++++++++++ 1 file changed, 76 insertions(+)
tools/testing/selftests/kvm/aarch64/page_fault_test.c +76 -0
@@ -31,6 +31,11 @@ static uint64_t *guest_test_memory = (uint64_t *)TEST_GVA;#define CMD_SKIP_TEST (1ULL << 1)#define CMD_HOLE_PT (1ULL << 2)#define CMD_HOLE_DATA (1ULL << 3) +#define CMD_CHECK_WRITE_IN_DIRTY_LOG (1ULL << 4) +#define CMD_CHECK_S1PTW_WR_IN_DIRTY_LOG (1ULL << 5) +#define CMD_CHECK_NO_WRITE_IN_DIRTY_LOG (1ULL << 6) +#define CMD_CHECK_NO_S1PTW_WR_IN_DIRTY_LOG (1ULL << 7) +#define CMD_SET_PTE_AF (1ULL << 8)#define PREPARE_FN_NR 10#define CHECK_FN_NR 10
@@ -213,6 +218,21 @@ static void guest_check_pte_af(void)GUEST_ASSERT_EQ (*((uint64_t *)TEST_PTE_GVA ) &PTE_AF ,PTE_AF ); } +static void guest_check_write_in_dirty_log (void ) +{ +GUEST_SYNC (CMD_CHECK_WRITE_IN_DIRTY_LOG ); +} + +static void guest_check_no_write_in_dirty_log (void ) +{ +GUEST_SYNC (CMD_CHECK_NO_WRITE_IN_DIRTY_LOG ); +} + +static void guest_check_s1ptw_wr_in_dirty_log (void ) +{ +GUEST_SYNC (CMD_CHECK_S1PTW_WR_IN_DIRTY_LOG ); +} +static void guest_exec (void ) {int (*code)(void ) = (int (*)(void ))TEST_EXEC_GVA ;
@@ -395,6 +415,22 @@ static bool punch_hole_in_backing_store(struct kvm_vm *vm,return true; } +static bool check_write_in_dirty_log (struct kvm_vm *vm, +struct userspace_mem_region *region, +uint64_t host_pg_nr) +{ +unsigned long *bmap; +bool first_page_dirty; +uint64_t size = region->region.memory_size; + +/* getpage_size() is not always equal to vm->page_size */ + bmap =bitmap_zalloc (size /getpagesize ()); +kvm_vm_get_dirty_log (vm, region->region.slot, bmap); + first_page_dirty =test_bit (host_pg_nr, bmap); +free (bmap); +return first_page_dirty; +} +/* Returns true to continue the test, and false if it should be skipped. */ static bool handle_cmd (struct kvm_vm *vm,int cmd) {
@@ -411,6 +447,18 @@ static bool handle_cmd(struct kvm_vm *vm, int cmd) continue_test =punch_hole_in_backing_store (vm, pt_region);if (cmd &CMD_HOLE_DATA ) continue_test =punch_hole_in_backing_store (vm, data_region); +if (cmd &CMD_CHECK_WRITE_IN_DIRTY_LOG ) +TEST_ASSERT (check_write_in_dirty_log (vm, data_region,0 ), +"Missing write in dirty log" ); +if (cmd &CMD_CHECK_S1PTW_WR_IN_DIRTY_LOG ) +TEST_ASSERT (check_write_in_dirty_log (vm, pt_region,0 ), +"Missing s1ptw write in dirty log" ); +if (cmd &CMD_CHECK_NO_WRITE_IN_DIRTY_LOG ) +TEST_ASSERT (!check_write_in_dirty_log (vm, data_region,0 ), +"Unexpected write in dirty log" ); +if (cmd &CMD_CHECK_NO_S1PTW_WR_IN_DIRTY_LOG ) +TEST_ASSERT (!check_write_in_dirty_log (vm, pt_region,0 ), +"Unexpected s1ptw write in dirty log" );return continue_test; }
@@ -673,6 +721,19 @@ static void help(char *name) .expected_events = { .uffd_faults = _uffd_faults, }, \ } +#define TEST_DIRTY_LOG (_access, _with_af, _test_check) \ +{ \ + .name = SCAT3(dirty_log, _access, _with_af), \ + .data_memslot_flags = KVM_MEM_LOG_DIRTY_PAGES, \ + .pt_memslot_flags = KVM_MEM_LOG_DIRTY_PAGES, \ + .guest_prepare = { _PREPARE(_with_af), \ + _PREPARE(_access) }, \ + .guest_test = _access, \ + .guest_test_check = { _CHECK(_with_af), _test_check, \ + guest_check_s1ptw_wr_in_dirty_log}, \ + .expected_events = { 0 }, \ +} +static struct test_desc tests[] = {/* Check that HW is setting the Access Flag (AF) (sanity checks). */
@@ -732,6 +793,21 @@ static struct test_desc tests[] = {TEST_UFFD (guest_exec, with_af,CMD_HOLE_DATA |CMD_HOLE_PT , uffd_data_read_handler, uffd_pt_write_handler,2 ), +/* +* Try accesses when the data and PT memory regions are both +* tracked for dirty logging. +*/ +TEST_DIRTY_LOG (guest_read64, with_af, guest_check_no_write_in_dirty_log), +/* no_af should also lead to a PT write. */ +TEST_DIRTY_LOG (guest_read64, no_af, guest_check_no_write_in_dirty_log), +TEST_DIRTY_LOG (guest_ld_preidx, with_af, guest_check_no_write_in_dirty_log), +TEST_DIRTY_LOG (guest_at, no_af, guest_check_no_write_in_dirty_log), +TEST_DIRTY_LOG (guest_exec, with_af, guest_check_no_write_in_dirty_log), +TEST_DIRTY_LOG (guest_write64, with_af, guest_check_write_in_dirty_log), +TEST_DIRTY_LOG (guest_cas, with_af, guest_check_write_in_dirty_log), +TEST_DIRTY_LOG (guest_dc_zva, with_af, guest_check_write_in_dirty_log), +TEST_DIRTY_LOG (guest_st_preidx, with_af, guest_check_write_in_dirty_log), + { 0 } };
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 13/14 KVM: selftests: aarch64: Add readonly memslot tests into page_fault_test
In reply to Ricardo KollerAdd some readonly memslot tests into page_fault_test. Mark the data and/or page-table memory regions as readonly, perform some accesses, and check that the right fault is triggered when expected (e.g., a store with no write-back should lead to an mmio exit).
--- .../selftests/kvm/aarch64/page_fault_test.c | 102 +++++++++++++++++- 1 file changed, 101 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/aarch64/page_fault_test.c +101 -1
@@ -41,6 +41,8 @@ static uint64_t *guest_test_memory = (uint64_t *)TEST_GVA;#define CHECK_FN_NR 10 static struct event_cnt { + int mmio_exits; + int fail_vcpu_runs; int uffd_faults; /* uffd_faults is incremented from multiple threads. */ pthread_mutex_t uffd_faults_mutex;
@@ -57,6 +59,8 @@ struct test_desc {uffd_handler_t uffd_data_handler;void (*dabt_handler)(struct ex_regs *regs);void (*iabt_handler)(struct ex_regs *regs); +void (*mmio_handler)(struct kvm_vm *vm,struct kvm_run *run); +void (*fail_vcpu_run_handler)(int ret);uint32_t pt_memslot_flags;uint32_t data_memslot_flags;bool skip;
@@ -415,6 +419,31 @@ static bool punch_hole_in_backing_store(struct kvm_vm *vm,return true; } +static void mmio_on_test_gpa_handler (struct kvm_vm *vm,struct kvm_run *run) +{ +struct userspace_mem_region *region; +void *hva; + + region =vm_get_mem_region (vm,MEM_REGION_TEST_DATA ); + hva = (void *)region->region.userspace_addr; + +ASSERT_EQ (run->mmio.phys_addr, region->region.guest_phys_addr); + +memcpy (hva, run->mmio.data, run->mmio.len); + events.mmio_exits +=1 ; +} + +static void mmio_no_handler (struct kvm_vm *vm,struct kvm_run *run) +{ +uint64_t data; + +memcpy (&data, run->mmio.data,sizeof (data)); +pr_debug ("addr=%lld len=%d w=%d data=%lx\n" , + run->mmio.phys_addr, run->mmio.len, + run->mmio.is_write, data); +TEST_FAIL ("There was no MMIO exit expected." ); +} +static bool check_write_in_dirty_log (struct kvm_vm *vm,struct userspace_mem_region *region,uint64_t host_pg_nr)
@@ -463,6 +492,18 @@ static bool handle_cmd(struct kvm_vm *vm, int cmd)return continue_test; } +void fail_vcpu_run_no_handler (int ret) +{ +TEST_FAIL ("Unexpected vcpu run failure\n" ); +} + +void fail_vcpu_run_mmio_no_syndrome_handler (int ret) +{ +TEST_ASSERT (errno ==ENOSYS , +"The mmio handler should have returned not implemented." ); + events.fail_vcpu_runs +=1 ; +} +typedef uint32_t aarch64_insn_t ;extern aarch64_insn_t __exec_test[2 ];
@@ -564,9 +605,20 @@ static void setup_memslots(struct kvm_vm *vm, struct test_params *p) vm->memslots[MEM_REGION_TEST_DATA ] =TEST_DATA_MEMSLOT ; } +static void setup_default_handlers (struct test_desc *test) +{ +if (!test->mmio_handler) + test->mmio_handler = mmio_no_handler; + +if (!test->fail_vcpu_run_handler) + test->fail_vcpu_run_handler = fail_vcpu_run_no_handler; +} +static void check_event_counts (struct test_desc *test) {ASSERT_EQ (test->expected_events.uffd_faults, events.uffd_faults); +ASSERT_EQ (test->expected_events.mmio_exits, events.mmio_exits); +ASSERT_EQ (test->expected_events.fail_vcpu_runs, events.fail_vcpu_runs); }static void print_test_banner (enum vm_guest_mode mode,struct test_params *p)
@@ -591,10 +643,18 @@ static void reset_event_counts(void)static void vcpu_run_loop (struct kvm_vm *vm,struct kvm_vcpu *vcpu,struct test_desc *test) { +struct kvm_run *run;struct ucall uc; +int ret; + + run = vcpu->run;for (;;) { -vcpu_run (vcpu); + ret =_vcpu_run (vcpu); +if (ret) { + test->fail_vcpu_run_handler (ret); + goto done; + }switch (get_ucall (vcpu, &uc)) {case UCALL_SYNC :
@@ -608,6 +668,10 @@ static void vcpu_run_loop(struct kvm_vm *vm, struct kvm_vcpu *vcpu,break ;case UCALL_DONE : goto done; +case UCALL_NONE : +if (run->exit_reason ==KVM_EXIT_MMIO ) + test->mmio_handler (vm, run); +break ;default :TEST_FAIL ("Unknown ucall %lu" , uc.cmd); }
@@ -647,6 +711,7 @@ static void run_test(enum vm_guest_mode mode, void *arg)load_exec_code_for_test (vm);setup_uffd (vm, p, &pt_uffd, &data_uffd);setup_abort_handlers (vm, vcpu, test); +setup_default_handlers (test);vcpu_args_set (vcpu,1 , test);vcpu_run_loop (vm, vcpu, test);
@@ -734,6 +799,25 @@ static void help(char *name) .expected_events = { 0 }, \ } +#define TEST_RO_MEMSLOT (_access, _mmio_handler, _mmio_exits) \ +{ \ + .name = SCAT3(ro_memslot, _access, _with_af), \ + .data_memslot_flags = KVM_MEM_READONLY, \ + .guest_prepare = { _PREPARE(_access) }, \ + .guest_test = _access, \ + .mmio_handler = _mmio_handler, \ + .expected_events = { .mmio_exits = _mmio_exits }, \ +} + +#define TEST_RO_MEMSLOT_NO_SYNDROME (_access) \ +{ \ + .name = SCAT2(ro_memslot_no_syndrome, _access), \ + .data_memslot_flags = KVM_MEM_READONLY, \ + .guest_test = _access, \ + .fail_vcpu_run_handler = fail_vcpu_run_mmio_no_syndrome_handler, \ + .expected_events = { .fail_vcpu_runs = 1 }, \ +} +static struct test_desc tests[] = {/* Check that HW is setting the Access Flag (AF) (sanity checks). */
@@ -808,6 +892,22 @@ static struct test_desc tests[] = {TEST_DIRTY_LOG (guest_dc_zva, with_af, guest_check_write_in_dirty_log),TEST_DIRTY_LOG (guest_st_preidx, with_af, guest_check_write_in_dirty_log), +/* +* Try accesses when the data memory region is marked read-only +* (with KVM_MEM_READONLY). Writes with a syndrome result in an +* MMIO exit, writes with no syndrome (e.g., CAS) result in a +* failed vcpu run, and reads/execs with and without syndroms do +* not fault. +*/ +TEST_RO_MEMSLOT (guest_read64,0 ,0 ), +TEST_RO_MEMSLOT (guest_ld_preidx,0 ,0 ), +TEST_RO_MEMSLOT (guest_at,0 ,0 ), +TEST_RO_MEMSLOT (guest_exec,0 ,0 ), +TEST_RO_MEMSLOT (guest_write64, mmio_on_test_gpa_handler,1 ), +TEST_RO_MEMSLOT_NO_SYNDROME (guest_dc_zva), +TEST_RO_MEMSLOT_NO_SYNDROME (guest_cas), +TEST_RO_MEMSLOT_NO_SYNDROME (guest_st_preidx), + { 0 } };
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
PATCH v10 14/14 KVM: selftests: aarch64: Add mix of tests into page_fault_test
In reply to Ricardo KollerAdd some mix of tests into page_fault_test: memory regions with all the pairwise combinations of read-only, userfaultfd, and dirty-logging. For example, writing into a read-only region which has a hole handled with userfaultfd.
--- .../selftests/kvm/aarch64/page_fault_test.c | 155 ++++++++++++++++++ 1 file changed, 155 insertions(+)
tools/testing/selftests/kvm/aarch64/page_fault_test.c +155 -0
@@ -399,6 +399,12 @@ static void free_uffd(struct test_desc *test, struct uffd_desc *pt_uffd,free (data_args.copy); } +static int uffd_no_handler (int mode,int uffd,struct uffd_msg *msg) +{ +TEST_FAIL ("There was no UFFD fault expected." ); +return -1 ; +} +/* Returns false if the test should be skipped. */ static bool punch_hole_in_backing_store (struct kvm_vm *vm,struct userspace_mem_region *region)
@@ -799,6 +805,22 @@ static void help(char *name) .expected_events = { 0 }, \ } +#define TEST_UFFD_AND_DIRTY_LOG (_access, _with_af, _uffd_data_handler, \ + _uffd_faults, _test_check) \ +{ \ + .name = SCAT3(uffd_and_dirty_log, _access, _with_af), \ + .data_memslot_flags = KVM_MEM_LOG_DIRTY_PAGES, \ + .pt_memslot_flags = KVM_MEM_LOG_DIRTY_PAGES, \ + .guest_prepare = { _PREPARE(_with_af), \ + _PREPARE(_access) }, \ + .guest_test = _access, \ + .mem_mark_cmd = CMD_HOLE_DATA | CMD_HOLE_PT, \ + .guest_test_check = { _CHECK(_with_af), _test_check }, \ + .uffd_data_handler = _uffd_data_handler, \ + .uffd_pt_handler = uffd_pt_write_handler, \ + .expected_events = { .uffd_faults = _uffd_faults, }, \ +} +#define TEST_RO_MEMSLOT (_access, _mmio_handler, _mmio_exits) \ { \ .name = SCAT3(ro_memslot, _access, _with_af), \
@@ -818,6 +840,59 @@ static void help(char *name) .expected_events = { .fail_vcpu_runs = 1 }, \ } +#define TEST_RO_MEMSLOT_AND_DIRTY_LOG (_access, _mmio_handler, _mmio_exits, \ + _test_check) \ +{ \ + .name = SCAT3(ro_memslot, _access, _with_af), \ + .data_memslot_flags = KVM_MEM_READONLY | KVM_MEM_LOG_DIRTY_PAGES, \ + .pt_memslot_flags = KVM_MEM_LOG_DIRTY_PAGES, \ + .guest_prepare = { _PREPARE(_access) }, \ + .guest_test = _access, \ + .guest_test_check = { _test_check }, \ + .mmio_handler = _mmio_handler, \ + .expected_events = { .mmio_exits = _mmio_exits}, \ +} + +#define TEST_RO_MEMSLOT_NO_SYNDROME_AND_DIRTY_LOG (_access, _test_check) \ +{ \ + .name = SCAT2(ro_memslot_no_syn_and_dlog, _access), \ + .data_memslot_flags = KVM_MEM_READONLY | KVM_MEM_LOG_DIRTY_PAGES, \ + .pt_memslot_flags = KVM_MEM_LOG_DIRTY_PAGES, \ + .guest_test = _access, \ + .guest_test_check = { _test_check }, \ + .fail_vcpu_run_handler = fail_vcpu_run_mmio_no_syndrome_handler, \ + .expected_events = { .fail_vcpu_runs = 1 }, \ +} + +#define TEST_RO_MEMSLOT_AND_UFFD (_access, _mmio_handler, _mmio_exits, \ + _uffd_data_handler, _uffd_faults) \ +{ \ + .name = SCAT2(ro_memslot_uffd, _access), \ + .data_memslot_flags = KVM_MEM_READONLY, \ + .mem_mark_cmd = CMD_HOLE_DATA | CMD_HOLE_PT, \ + .guest_prepare = { _PREPARE(_access) }, \ + .guest_test = _access, \ + .uffd_data_handler = _uffd_data_handler, \ + .uffd_pt_handler = uffd_pt_write_handler, \ + .mmio_handler = _mmio_handler, \ + .expected_events = { .mmio_exits = _mmio_exits, \ + .uffd_faults = _uffd_faults }, \ +} + +#define TEST_RO_MEMSLOT_NO_SYNDROME_AND_UFFD (_access, _uffd_data_handler, \ + _uffd_faults) \ +{ \ + .name = SCAT2(ro_memslot_no_syndrome, _access), \ + .data_memslot_flags = KVM_MEM_READONLY, \ + .mem_mark_cmd = CMD_HOLE_DATA | CMD_HOLE_PT, \ + .guest_test = _access, \ + .uffd_data_handler = _uffd_data_handler, \ + .uffd_pt_handler = uffd_pt_write_handler, \ + .fail_vcpu_run_handler = fail_vcpu_run_mmio_no_syndrome_handler, \ + .expected_events = { .fail_vcpu_runs = 1, \ + .uffd_faults = _uffd_faults }, \ +} +static struct test_desc tests[] = {/* Check that HW is setting the Access Flag (AF) (sanity checks). */
@@ -892,6 +967,35 @@ static struct test_desc tests[] = {TEST_DIRTY_LOG (guest_dc_zva, with_af, guest_check_write_in_dirty_log),TEST_DIRTY_LOG (guest_st_preidx, with_af, guest_check_write_in_dirty_log), +/* +* Access when the data and PT memory regions are both marked for +* dirty logging and UFFD at the same time. The expected result is +* that writes should mark the dirty log and trigger a userfaultfd +* write fault. Reads/execs should result in a read userfaultfd +* fault, and nothing in the dirty log. Any S1PTW should result in +* a write in the dirty log and a userfaultfd write. +*/ +TEST_UFFD_AND_DIRTY_LOG (guest_read64, with_af, uffd_data_read_handler,2 , + guest_check_no_write_in_dirty_log), +/* no_af should also lead to a PT write. */ +TEST_UFFD_AND_DIRTY_LOG (guest_read64, no_af, uffd_data_read_handler,2 , + guest_check_no_write_in_dirty_log), +TEST_UFFD_AND_DIRTY_LOG (guest_ld_preidx, with_af, uffd_data_read_handler, +2 , guest_check_no_write_in_dirty_log), +TEST_UFFD_AND_DIRTY_LOG (guest_at, with_af,0 ,1 , + guest_check_no_write_in_dirty_log), +TEST_UFFD_AND_DIRTY_LOG (guest_exec, with_af, uffd_data_read_handler,2 , + guest_check_no_write_in_dirty_log), +TEST_UFFD_AND_DIRTY_LOG (guest_write64, with_af, uffd_data_write_handler, +2 , guest_check_write_in_dirty_log), +TEST_UFFD_AND_DIRTY_LOG (guest_cas, with_af, uffd_data_read_handler,2 , + guest_check_write_in_dirty_log), +TEST_UFFD_AND_DIRTY_LOG (guest_dc_zva, with_af, uffd_data_write_handler, +2 , guest_check_write_in_dirty_log), +TEST_UFFD_AND_DIRTY_LOG (guest_st_preidx, with_af, + uffd_data_write_handler,2 , + guest_check_write_in_dirty_log), +/* * Try accesses when the data memory region is marked read-only * (with KVM_MEM_READONLY). Writes with a syndrome result in an
@@ -908,6 +1012,57 @@ static struct test_desc tests[] = {TEST_RO_MEMSLOT_NO_SYNDROME (guest_cas),TEST_RO_MEMSLOT_NO_SYNDROME (guest_st_preidx), +/* +* Access when both the data region is both read-only and marked +* for dirty logging at the same time. The expected result is that +* for writes there should be no write in the dirty log. The +* readonly handling is the same as if the memslot was not marked +* for dirty logging: writes with a syndrome result in an MMIO +* exit, and writes with no syndrome result in a failed vcpu run. +*/ +TEST_RO_MEMSLOT_AND_DIRTY_LOG (guest_read64,0 ,0 , + guest_check_no_write_in_dirty_log), +TEST_RO_MEMSLOT_AND_DIRTY_LOG (guest_ld_preidx,0 ,0 , + guest_check_no_write_in_dirty_log), +TEST_RO_MEMSLOT_AND_DIRTY_LOG (guest_at,0 ,0 , + guest_check_no_write_in_dirty_log), +TEST_RO_MEMSLOT_AND_DIRTY_LOG (guest_exec,0 ,0 , + guest_check_no_write_in_dirty_log), +TEST_RO_MEMSLOT_AND_DIRTY_LOG (guest_write64, mmio_on_test_gpa_handler, +1 , guest_check_no_write_in_dirty_log), +TEST_RO_MEMSLOT_NO_SYNDROME_AND_DIRTY_LOG (guest_dc_zva, + guest_check_no_write_in_dirty_log), +TEST_RO_MEMSLOT_NO_SYNDROME_AND_DIRTY_LOG (guest_cas, + guest_check_no_write_in_dirty_log), +TEST_RO_MEMSLOT_NO_SYNDROME_AND_DIRTY_LOG (guest_st_preidx, + guest_check_no_write_in_dirty_log), + +/* +* Access when the data region is both read-only and punched with +* holes tracked with userfaultfd. The expected result is the +* union of both userfaultfd and read-only behaviors. For example, +* write accesses result in a userfaultfd write fault and an MMIO +* exit. Writes with no syndrome result in a failed vcpu run and +* no userfaultfd write fault. Reads result in userfaultfd getting +* triggered. +*/ +TEST_RO_MEMSLOT_AND_UFFD (guest_read64,0 ,0 , + uffd_data_read_handler,2 ), +TEST_RO_MEMSLOT_AND_UFFD (guest_ld_preidx,0 ,0 , + uffd_data_read_handler,2 ), +TEST_RO_MEMSLOT_AND_UFFD (guest_at,0 ,0 , + uffd_no_handler,1 ), +TEST_RO_MEMSLOT_AND_UFFD (guest_exec,0 ,0 , + uffd_data_read_handler,2 ), +TEST_RO_MEMSLOT_AND_UFFD (guest_write64, mmio_on_test_gpa_handler,1 , + uffd_data_write_handler,2 ), +TEST_RO_MEMSLOT_NO_SYNDROME_AND_UFFD (guest_cas, + uffd_data_read_handler,2 ), +TEST_RO_MEMSLOT_NO_SYNDROME_AND_UFFD (guest_dc_zva, + uffd_no_handler,1 ), +TEST_RO_MEMSLOT_NO_SYNDROME_AND_UFFD (guest_st_preidx, + uffd_no_handler,1 ), + { 0 } };
Signature · 1 line
2.38.0.413.g74048e4d9e-goog
Sean Christopherson
reviewerQuoted 7 lines · Ricardo Koller
This series adds a new aarch64 selftest for testing stage 2 fault handling for various combinations of guest accesses (e.g., write, S1PTW), backing sources (e.g., anon), and types of faults (e.g., read on hugetlbfs with a hole, write on a readonly memslot). Each test tries a different combination and then checks that the access results in the right behavior (e.g., uffd faults with the right address and write/read flag). Some interesting combinations are:
...
Quoted 5 lines
Ricardo Koller (14):
KVM: selftests: Add a userfaultfd library
KVM: selftests: aarch64: Addvirt_get_pte_hva()library function
KVM: selftests: Add missing close and munmap in__vm_mem_region_delete()
Quoted 15 lines
KVM: selftests: aarch64: Construct
DEFAULT_MAIR_EL1usingsysreg.h
macros
tools: Copybitfield.hfrom the kernel sources
KVM: selftests: Stashbacking_src_typein structuserspace_mem_region
KVM: selftests: Add vm->memslots[] and enumkvm_mem_region_type
KVM: selftests: Fix alignment invirt_arch_pgd_alloc()and
vm_vaddr_alloc()
KVM: selftests: Use the right memslot for code, page-tables, and data
allocations
KVM: selftests: aarch64: Add aarch64/page_fault_test
KVM: selftests: aarch64: Add userfaultfd tests intopage_fault_test
KVM: selftests: aarch64: Add dirty logging tests intopage_fault_test
KVM: selftests: aarch64: Add readonly memslot tests into
page_fault_test
KVM: selftests: aarch64: Add mix of tests intopage_fault_test
One alignment nit in the first patch, but definitely not worthy of a v11. If it gets fixed up when applying, yay. If not, no big deal.
Quoted 9 lines · Ricardo Koller
This series adds a new aarch64 selftest for testing stage 2 fault handling for various combinations of guest accesses (e.g., write, S1PTW), backing sources (e.g., anon), and types of faults (e.g., read on hugetlbfs with a hole, write on a readonly memslot). Each test tries a different combination and then checks that the access results in the right behavior (e.g., uffd faults with the right address and write/read flag). Some interesting combinations are:
[...]
Applied to next, thanks!
[01/14] KVM: selftests: Add a userfaultfd library
commit: a93871d0ea9fd59fb5eb783619334183d7f07f51
[02/14] KVM: selftests: aarch64: Add virt_get_pte_hva() library function
commit: 228f324dc718f702e8777164c4e2e7426824fb13
[03/14] KVM: selftests: Add missing close and munmap in __vm_mem_region_delete()
commit: b6b03b86c0250a80b671313dbc0d7bcdbab78f41
[04/14] KVM: selftests: aarch64: Construct DEFAULT_MAIR_EL1 using sysreg.h macros
commit: 41f5189ea9c08f7fc28340a7aefc93d0d2dcb769
[05/14] tools: Copy bitfield.h from the kernel sources
commit: 590b949597b1e811d35df2f32021dd17d8e47f8c
[06/14] KVM: selftests: Stash backing_src_type in struct userspace_mem_region
commit: bd3ed7e1a47eb7b3838ca09439f1eb289ec3be1f
[07/14] KVM: selftests: Add vm->memslots[] and enum kvm_mem_region_type
commit: 290c5b54012b7f05e9c51af32d557574bf69a654
[08/14] KVM: selftests: Fix alignment in virt_arch_pgd_alloc() and vm_vaddr_alloc()
commit: 5485e822e31a75dfac3713d94b6b22025d4895da
[09/14] KVM: selftests: Use the right memslot for code, page-tables, and data allocations
commit: 1446e331432d7f24ed56b870ad605a4345fee43f
[10/14] KVM: selftests: aarch64: Add aarch64/page_fault_test
commit: 35c5810157124cb71aaa939cd2d5508192714877
[11/14] KVM: selftests: aarch64: Add userfaultfd tests into page_fault_test
commit: 3b1d915659c64dce079f4926a648f2271faea008
[12/14] KVM: selftests: aarch64: Add dirty logging tests into page_fault_test
commit: a4edf25b3e25656c69cbc768d1c704868e4a616f
[13/14] KVM: selftests: aarch64: Add readonly memslot tests into page_fault_test
commit: 45acde40f538a30e759f3b3f4aa5089edf097b2f
[14/14] KVM: selftests: aarch64: Add mix of tests into page_fault_test
commit: ff2b5509e1d252cd18bb1430b5461d5044701559Cheers,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 7 lines · Ricardo Koller
This series adds a new aarch64 selftest for testing stage 2 fault handling for various combinations of guest accesses (e.g., write, S1PTW), backing sources (e.g., anon), and types of faults (e.g., read on hugetlbfs with a hole, write on a readonly memslot). Each test tries a different combination and then checks that the access results in the right behavior (e.g., uffd faults with the right address and write/read flag). Some interesting combinations are:
I'm seeing issues with the page_fault_test tests in both -next and mainline all the way back to v6.1 when they were introduced running on both the fast model and hardware. With -next the reports come back as:
# selftests: kvm: page_fault_test
# ==== Test Assertion Failure ====
# aarch64/page_fault_test.c:316: __a == __b
# pid=851 tid=860 errno=0 - Success
# 1 0x0000000000402253: uffd_generic_handler at page_fault_test.c:316 # 2 0x000000000040be07: uffd_handler_thread_fn at userfaultfd_util.c:97 # 3 0x0000ffff8b39edd7: ?? ??:0 # 4 0x0000ffff8b407e9b: ?? ??:0
# ASSERT_EQ(!!(flags & UFFD_PAGEFAULT_FLAG_WRITE), expect_write) failed.
# !!(flags & UFFD_PAGEFAULT_FLAG_WRITE) is 0
# expect_write is 0x1
not ok 6 selftests: kvm: page_fault_test # exit=254
(addr2line seemed to be not doing much, I've not poked too hard at that). I've been unable to find any case where the program passes. Is this expected?
Some random full runs on hardware:
4xA53: https://lava.sirena.org.uk/scheduler/job/244678
4xA72: https://lkft.validation.linaro.org/scheduler/job/6114427
Hi Mark,
Quoted 27 lines · Mark Brown
Quoted 7 lines · Ricardo Koller
This series adds a new aarch64 selftest for testing stage 2 fault handling for various combinations of guest accesses (e.g., write, S1PTW), backing sources (e.g., anon), and types of faults (e.g., read on hugetlbfs with a hole, write on a readonly memslot). Each test tries a different combination and then checks that the access results in the right behavior (e.g., uffd faults with the right address and write/read flag). Some interesting combinations are:
I'm seeing issues with the
page_fault_testtests in both -next and mainline all the way back to v6.1 when they were introduced running on both the fast model and hardware. With -next the reports come back as:# selftests: kvm:
page_fault_test
# ==== Test Assertion Failure ====
#aarch64/page_fault_test.c:316:__a==__b
# pid=851 tid=860 errno=0 - Success# 1 0x0000000000402253: uffd_generic_handler at page_fault_test.c:316 # 2 0x000000000040be07: uffd_handler_thread_fn at userfaultfd_util.c:97 # 3 0x0000ffff8b39edd7: ?? ??:0 # 4 0x0000ffff8b407e9b: ?? ??:0#
ASSERT_EQ(!!(flags & UFFD_PAGEFAULT_FLAG_WRITE), expect_write)failed.
# !!(flags &UFFD_PAGEFAULT_FLAG_WRITE) is 0
#expect_writeis0x1
not ok 6 selftests: kvm:page_fault_test# exit=254
Quoted 5 lines
(addr2line seemed to be not doing much, I've not poked too hard at that). I've been unable to find any case where the program passes. Is this expected?
Some random full runs on hardware:
That failure was fixed with this series:
"KVM: selftests: aarch64: page_fault_test S1PTW related fixes"
https://lore.kernel.org/kvmarm/[email protected]/
which made it into kvmarm/fixes and should get into 6.2:
https://lore.kernel.org/kvmarm/[email protected]/
Note that the failing assert does not exist after the mentioned series:
Quoted 5 lines
#
ASSERT_EQ(!!(flags & UFFD_PAGEFAULT_FLAG_WRITE), expect_write)failed.4xA53: https://lava.sirena.org.uk/scheduler/job/244678
4xA72: https://lkft.validation.linaro.org/scheduler/job/6114427
Thanks,
Ricardo
Quoted 11 lines · Ricardo Koller
Quoted 9 lines · Mark Brown
On Mon, Oct 17, 2022 at 07:58:20PM +0000, Ricardo Koller wrote:
# ==== Test Assertion Failure ====
#aarch64/page_fault_test.c:316:__a==__b
# pid=851 tid=860 errno=0 - Success#
ASSERT_EQ(!!(flags & UFFD_PAGEFAULT_FLAG_WRITE), expect_write)failed.
# !!(flags &UFFD_PAGEFAULT_FLAG_WRITE) is 0
#expect_writeis0x1
Quoted 9 lines
That failure was fixed with this series:
"KVM: selftests: aarch64:page_fault_testS1PTW related fixes"
https://lore.kernel.org/kvmarm/[email protected]/which made it into kvmarm/fixes and should get into 6.2:
https://lore.kernel.org/kvmarm/[email protected]/Note that the failing assert does not exist after the mentioned series:
Quoted 1 line
#
ASSERT_EQ(!!(flags & UFFD_PAGEFAULT_FLAG_WRITE), expect_write)failed.
Ah, good. That's not made it into -next yet unfortunately so far as I can see but hopefully it'll turn up shortly and everything will start passing. Thanks!