selftests/mm: Split / Refactor userfault test
Subsystems (7)
Peter Xu
reviewerv2 cover selftests/mm: Split / Refactor userfault test
v2:
- Added r-bs
- Leverage kselftests.h helpers [David]
- Fixed up zeropage test to detect uffdio_register.ioctls to decide whether to test UFFDIO_ZEROPAGE or not [MikeK, Axel, David]
- renamed uffd_register_detect_zp() to uffd_register_detect_zeropage [MikeR]
- Added patch "selftests/mm: Add uffdio register ioctls test" to test all combinations of uffdio_register.ioctls
- Added patch "selftests/mm: Rename uffd_stats to uffd_args" [MikeR]
- Cosmetic changes here and there
v1: https://lore.kernel.org/r/[email protected]
This patchset splits userfaultfd.c into two tests:
- uffd-stress: the "vanilla", old and powerful stress test
- uffd-unit-tests: all the unit tests will be moved here
This is on my todo list for a long time but I never did it for real. The uffd test is growing into a small and cute monster. I start to notice it's going harder to maintain such a test and make it useful.
A few issues I found when looking at userfaultfd test:
- We have a bunch of unit tests in userfaultfd.c, but they always need to be run only after a stress type. No way to not do it.
- We can only run an unit test for one memory type only, if we want to do a quick smoke test to check regressions, there's no good way. The best to come currently is "bash ./run_vmtests.sh -t userfaultfd" thanks to the most recent changes to run_vmtests.sh on tagging. Still, that needs to run the stress tests always and hard to see what's wrong.
- It's hard to add a new unit test to userfaultfd.c, we don't really know what's happening, not until we mostly read the whole file.
- We did a bunch of useless tests, e.g. we run twice the whole suite of stress test just to verify both syscall and /dev/userfaultfd. They're all using userfaultfd_new() to create the handle, everything should really be the same underneath. One simple unit test should cover that!
- We have tens of global variables in one file but shared with all the tests. Some of them are not suitable to be a global var from maintainance pov. It enforces every unit test to consider how these vars affects the stress test and vice versa, but that's logically not necessary.
- Userfaultfd test is not friendly to old kernels. Mostly it only works on the latest kernel tree. It's preferrable to be run on all kernels and properly report what's missing.
I'll stop here, I feel like I can still list some..
This patchset should resolve all issues above, and actually we can do even more on top. I stopped doing that until I found I already got 29 patches and 2000+ LOC changes. That's already a patchset terrible enough so we should move in small steps.
After the whole set applied, "./run_vmtests.sh -t userfaultfd" looks like this:
===8<===
vm.nr_hugepages = 1024
-------------------------
running ./uffd-unit-tests
-------------------------
Testing UFFDIO_API (with syscall)... done
Testing UFFDIO_API (with /dev/userfaultfd)... done
Testing register-ioctls on anon... done
Testing register-ioctls on shmem... done
Testing register-ioctls on shmem-private... done
Testing register-ioctls on hugetlb... done
Testing register-ioctls on hugetlb-private... done
Testing zeropage on anon... done
Testing zeropage on shmem... done
Testing zeropage on shmem-private... done
Testing zeropage on hugetlb... done
Testing zeropage on hugetlb-private... done
Testing pagemap on anon... done
Testing wp-unpopulated on anon... done
Testing minor on shmem... done
Testing minor on hugetlb... done
Testing minor-wp on shmem... done
Testing minor-wp on hugetlb... done
Testing minor-collapse on shmem... done
Testing sigbus on anon... done
Testing sigbus on shmem... done
Testing sigbus on shmem-private... done
Testing sigbus on hugetlb... done
Testing sigbus on hugetlb-private... done
Testing sigbus-wp on anon... done
Testing sigbus-wp on shmem... done
Testing sigbus-wp on shmem-private... done
Testing sigbus-wp on hugetlb... done
Testing sigbus-wp on hugetlb-private... done
Testing events on anon... done
Testing events on shmem... done
Testing events on shmem-private... done
Testing events on hugetlb... done
Testing events on hugetlb-private... done
Testing events-wp on anon... done
Testing events-wp on shmem... done
Testing events-wp on shmem-private... done
Testing events-wp on hugetlb... done
Testing events-wp on hugetlb-private... done
Userfaults unit tests: pass=39, skip=0, fail=0 (total=39)
[PASS]
--------------------------------
running ./uffd-stress anon 20 16
--------------------------------nr_pages: 5120, nr_pages_per_cpu: 640
bounces: 15, mode: rnd racing ver poll, userfaults: 345 missing (26+48+61+102+30+12+59+7) 1596 wp (120+139+317+346+215+67+306+86) [...]
[PASS]
------------------------------------
running ./uffd-stress hugetlb 128 32
------------------------------------nr_pages: 64, nr_pages_per_cpu: 8
bounces: 31, mode: rnd racing ver poll, userfaults: 29 missing (6+6+6+5+4+2+0+0) 104 wp (20+19+22+18+7+12+5+1) [...]
[PASS]
--------------------------------------------
running ./uffd-stress hugetlb-private 128 32
--------------------------------------------nr_pages: 64, nr_pages_per_cpu: 8
bounces: 31, mode: rnd racing ver poll, userfaults: 33 missing (12+9+7+0+5+0+0+0) 111 wp (24+25+14+14+11+17+5+1) [...]
[PASS]
---------------------------------
running ./uffd-stress shmem 20 16
---------------------------------nr_pages: 5120, nr_pages_per_cpu: 640
bounces: 15, mode: rnd racing ver poll, userfaults: 247 missing (15+17+34+60+81+37+3+0) 2038 wp (180+114+276+400+381+318+165+204) [...]
[PASS]
-----------------------------------------
running ./uffd-stress shmem-private 20 16
-----------------------------------------nr_pages: 5120, nr_pages_per_cpu: 640
bounces: 15, mode: rnd racing ver poll, userfaults: 235 missing (52+29+55+56+13+9+16+5) 2849 wp (218+406+461+531+328+284+430+191) [...]
[PASS]
SUMMARY: PASS=6 SKIP=0 FAIL=0
===8<===
The output may be different if we miss some features (e.g., hugetlb not allocated, old kernel, less privilege of uffd handle), but they should show up with good reasons. E.g., I tried to run the unit test on my Fedora kernel and it gives me:
===8<===UFFDIO_API (with syscall)... failed [reason: UFFDIO_API should fail with wrong api but didn't] UFFDIO_API (with /dev/userfaultfd)... skipped [reason: cannot open userfaultfd handle] zeropage on anon... done
zeropage on shmem... done
zeropage on shmem-private... done
zeropage-hugetlb on hugetlb... done
zeropage-hugetlb on hugetlb-private... done
pagemap on anon... pagemap on anon... pagemap on anon... done
wp-unpopulated on anon... skipped [reason: feature missing]
minor on shmem... done
minor on hugetlb... done
minor-wp on shmem... skipped [reason: feature missing]
minor-wp on hugetlb... skipped [reason: feature missing]
minor-collapse on shmem... done
sigbus on anon... skipped [reason: possible lack of priviledge]
sigbus on shmem... skipped [reason: possible lack of priviledge]
sigbus on shmem-private... skipped [reason: possible lack of priviledge]
sigbus on hugetlb... skipped [reason: possible lack of priviledge]
sigbus on hugetlb-private... skipped [reason: possible lack of priviledge] sigbus-wp on anon... skipped [reason: possible lack of priviledge]
sigbus-wp on shmem... skipped [reason: possible lack of priviledge]
sigbus-wp on shmem-private... skipped [reason: possible lack of priviledge] sigbus-wp on hugetlb... skipped [reason: possible lack of priviledge]
sigbus-wp on hugetlb-private... skipped [reason: possible lack of priviledge] events on anon... skipped [reason: possible lack of priviledge]
events on shmem... skipped [reason: possible lack of priviledge]
events on shmem-private... skipped [reason: possible lack of priviledge]
events on hugetlb... skipped [reason: possible lack of priviledge]
events on hugetlb-private... skipped [reason: possible lack of priviledge] events-wp on anon... skipped [reason: possible lack of priviledge]
events-wp on shmem... skipped [reason: possible lack of priviledge]
events-wp on shmem-private... skipped [reason: possible lack of priviledge] events-wp on hugetlb... skipped [reason: possible lack of priviledge]
events-wp on hugetlb-private... skipped [reason: possible lack of priviledge] Userfaults unit tests: pass=9, skip=24, fail=1 (total=34)
===8<===
Patch layout:
- Revert "userfaultfd: don't fail on unrecognized features"
Something I found when I got the UFFDIO_API test below. Axel, I still propose to revert it as a whole, but feel free to continue the discussion from the original patch thread.
- selftests/mm: Update .gitignore with two missing tests
- selftests/mm: Dump a summary in run_vmtests.sh
- selftests/mm: Merge util.h into vm_util.h
- selftests/mm: Use TEST_GEN_PROGS where proper
- selftests/mm: Link vm_util.c always
- selftests/mm: Merge default_huge_page_size() into one
- selftests/mm: Use PM_* macros in vm_utils.h
- selftests/mm: Reuse pagemap_get_entry() in vm_util.h
- selftests/mm: Test UFFDIO_ZEROPAGE only when !hugetlb
- selftests/mm: Drop test_uffdio_zeropage_eexist
Until here, all cleanups here and there. I wanted to keep going, but I found that maybe it'll take a few more days to split the test. Hence I did a split starting from the next one, so we have a working thing first.
- selftests/mm: Create uffd-common.[ch]
- selftests/mm: Split uffd tests into uffd-stress and uffd-unit-tests
This did the major brute force split of common codes into uffd-common.[ch]. That'll be the so far common base for stress and unit tests. Then a new unit test is created.
- selftests/mm: uffd_[un]register()
- selftests/mm: uffd_open_{dev|sys}()
- selftests/mm: UFFDIO_API test
This patch hides here to start writting the 1st unit test with UFFDIO_API, also detection of userfaultfd privileges.
- selftests/mm: Drop global mem_fd in uffd tests
- selftests/mm: Drop global hpage_size in uffd tests
- selftests/mm: Rename uffd_stats to uffd_args
- selftests/mm: Let uffd_handle_page_fault() takes wp parameter
- selftests/mm: Allow allocate_area() to fail properly
Some further cleanup that I noticed otherwise hard to move the tests.
- selftests/mm: Add framework for uffd-unit-test
The major patch provides the framework for most of the rest unit tests.
- selftests/mm: Move uffd pagemap test to unit test
- selftests/mm: Move uffd minor test to unit test
- selftests/mm: Move uffd sig/events tests into uffd unit tests
- selftests/mm: Move zeropage test into uffd unit tests
Move unit tests and suite them into the new file.
- selftests/mm: Workaround no way to detect uffd-minor + wp
- selftests/mm: Allow uffd test to skip properly with no privilege
- selftests/mm: Drop sys/dev test in uffd-stress test
- selftests/mm: Add shmem-private test to uffd-stress
A bunch of changes to do better on error reportings, and add shmem-private to the stress test which was long missing.
- selftests/mm: Add uffdio register ioctls test
One more patch to test uffdio_register.ioctls.
Please have a look, any comment welcomed.
Thanks,
Peter Xu (31):
Revert "userfaultfd: don't fail on unrecognized features" selftests/mm: Update .gitignore with two missing tests selftests/mm: Dump a summary in run_vmtests.sh selftests/mm: Merge util.h into vm_util.h
selftests/mm: Use TEST_GEN_PROGS where proper selftests/mm: Link vm_util.c always
selftests/mm: Merge default_huge_page_size() into one selftests/mm: Use PM_* macros in vm_utils.h selftests/mm: Reuse pagemap_get_entry() in vm_util.h selftests/mm: Test UFFDIO_ZEROPAGE only when !hugetlb selftests/mm: Drop test_uffdio_zeropage_eexist selftests/mm: Create uffd-common.[ch]
selftests/mm: Split uffd tests into uffd-stress and uffd-unit-tests selftests/mm: uffd_[un]register()
selftests/mm: uffd_open_{dev|sys}()
selftests/mm: UFFDIO_API test
selftests/mm: Drop global mem_fd in uffd tests selftests/mm: Drop global hpage_size in uffd tests selftests/mm: Rename uffd_stats to uffd_args selftests/mm: Let uffd_handle_page_fault() take wp parameter selftests/mm: Allow allocate_area() to fail properly selftests/mm: Add framework for uffd-unit-test selftests/mm: Move uffd pagemap test to unit test selftests/mm: Move uffd minor test to unit test selftests/mm: Move uffd sig/events tests into uffd unit tests selftests/mm: Move zeropage test into uffd unit tests selftests/mm: Workaround no way to detect uffd-minor + wp selftests/mm: Allow uffd test to skip properly with no privilege selftests/mm: Drop sys/dev test in uffd-stress test selftests/mm: Add shmem-private test to uffd-stress selftests/mm: Add uffdio register ioctls test
fs/userfaultfd.c | 6 +- tools/testing/selftests/mm/.gitignore | 5 +- tools/testing/selftests/mm/Makefile | 78 +- tools/testing/selftests/mm/gup_test.c | 5 +- tools/testing/selftests/mm/hugepage-mremap.c | 7 +- tools/testing/selftests/mm/hugetlb-madvise.c | 25 +- .../selftests/mm/ksm_functional_tests.c | 6 +- tools/testing/selftests/mm/ksm_tests.c | 2 +- tools/testing/selftests/mm/mrelease_test.c | 11 +- tools/testing/selftests/mm/run_vmtests.sh | 26 +- tools/testing/selftests/mm/thuge-gen.c | 19 +- tools/testing/selftests/mm/transhuge-stress.c | 12 +- tools/testing/selftests/mm/uffd-common.c | 618 ++++++ tools/testing/selftests/mm/uffd-common.h | 117 + tools/testing/selftests/mm/uffd-stress.c | 481 +++++ tools/testing/selftests/mm/uffd-unit-tests.c | 970 +++++++++ tools/testing/selftests/mm/userfaultfd.c | 1903 ----------------- tools/testing/selftests/mm/util.h | 69 - tools/testing/selftests/mm/vm_util.c | 176 +- tools/testing/selftests/mm/vm_util.h | 50 + 20 files changed, 2481 insertions(+), 2105 deletions(-) create mode 100644 tools/testing/selftests/mm/uffd-common.c create mode 100644 tools/testing/selftests/mm/uffd-common.h create mode 100644 tools/testing/selftests/mm/uffd-stress.c create mode 100644 tools/testing/selftests/mm/uffd-unit-tests.c delete mode 100644 tools/testing/selftests/mm/userfaultfd.c delete mode 100644 tools/testing/selftests/mm/util.h
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 1/31 Revert "userfaultfd: don't fail on unrecognized features"
This is a proposal to revert commit 914eedcb9ba0ff53c33808.
I found this when writting a simple UFFDIO_API test to be the first unit test in this set. Two things breaks with the commit:
- UFFDIO_API check was lost and missing. According to man page, the kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa. This check is needed if the api version will be extended in the future, or user app won't be able to identify which is a new kernel.
- Feature flags checks were removed, which means UFFDIO_API with a feature that does not exist will also succeed. According to the man page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if unknown features passed in.
--- fs/userfaultfd.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/userfaultfd.c +4 -2
@@ -1977,8 +1977,10 @@ static int userfaultfd_api(struct userfaultfd_ctx *ctx, ret = -EFAULT ;if (copy_from_user (&uffdio_api, buf,sizeof (uffdio_api))) goto out; -/* Ignore unsupported features (userspace built against newer kernel) */ - features = uffdio_api.features &UFFD_API_FEATURES ; + features = uffdio_api.features; + ret = -EINVAL ; +if (uffdio_api.api !=UFFD_API || (features & ~UFFD_API_FEATURES )) + goto err_out; ret = -EPERM ;if ((features &UFFD_FEATURE_EVENT_FORK ) && !capable (CAP_SYS_PTRACE )) goto err_out;
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 2/31 selftests/mm: Update .gitignore with two missing tests
In reply to Peter Xu--- tools/testing/selftests/mm/.gitignore | 2 ++ 1 file changed, 2 insertions(+)
tools/testing/selftests/mm/.gitignore +2 -0
@@ -36,3 +36,5 @@ split_huge_page_test ksm_tests local_config.h local_config.mk +ksm_functional_tests +mdwe_test
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 3/31 selftests/mm: Dump a summary in run_vmtests.sh
In reply to Peter XuDump a summary after running whatever test specified. Useful for human runners to identify any kind of failures (besides exit code).
--- tools/testing/selftests/mm/run_vmtests.sh | 8 ++++++++ 1 file changed, 8 insertions(+)
tools/testing/selftests/mm/run_vmtests.sh +8 -0
@@ -5,6 +5,9 @@ # Kselftest framework requirement - SKIP code is 4. ksft_skip=4 +count_pass=0 +count_fail=0 +count_skip=0 exitcode=0 usage() {
@@ -149,11 +152,14 @@ run_test() { "$@" local ret=$? if [ $ret -eq 0 ]; then + count_pass=$(( count_pass + 1 )) echo "[PASS]" elif [ $ret -eq $ksft_skip ]; then + count_skip=$(( count_skip + 1 )) echo "[SKIP]" exitcode=$ksft_skip else + count_fail=$(( count_fail + 1 )) echo "[FAIL]" exitcode=1 fi
@@ -279,4 +285,6 @@ CATEGORY="soft_dirty" run_test ./soft-dirty # COW tests CATEGORY="cow" run_test ./cow +echo "SUMMARY: PASS=${count_pass} SKIP=${count_skip} FAIL=${count_fail}" + exit $exitcode
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 4/31 selftests/mm: Merge util.h into vm_util.h
In reply to Peter XuThere're two util headers under mm/ kselftest. Merge one with another. It turns out util.h is the easy one to move.
When merging, drop PAGE_SIZE / PAGE_SHIFT because they're unnecessary wrappers to page_size() / page_shift(), meanwhile rename them to psize() and pshift() so as to not conflict with some existing definitions in some test files that includes vm_util.h.
--- tools/testing/selftests/mm/Makefile | 4 ++ tools/testing/selftests/mm/gup_test.c | 5 +- tools/testing/selftests/mm/ksm_tests.c | 2 +- tools/testing/selftests/mm/mrelease_test.c | 11 ++- tools/testing/selftests/mm/transhuge-stress.c | 12 ++-- tools/testing/selftests/mm/util.h | 69 ------------------- tools/testing/selftests/mm/vm_util.c | 31 +++++++++ tools/testing/selftests/mm/vm_util.h | 31 +++++++++ 8 files changed, 80 insertions(+), 85 deletions(-) delete mode 100644 tools/testing/selftests/mm/util.h
tools/testing/selftests/mm/Makefile +4 -0
@@ -113,6 +113,10 @@ $(OUTPUT)/mkdirty: vm_util.c $(OUTPUT)/soft-dirty: vm_util.c $(OUTPUT)/split_huge_page_test: vm_util.c $(OUTPUT)/userfaultfd: vm_util.c +$(OUTPUT)/gup_test: vm_util.c +$(OUTPUT)/mrelease_test: vm_util.c +$(OUTPUT)/transhuge-stress: vm_util.c +$(OUTPUT)/ksm_tests: vm_util.c ifeq ($(MACHINE),x86_64) BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
tools/testing/selftests/mm/gup_test.c +2 -3
@@ -12,8 +12,7 @@#include <assert.h> #include <mm/gup_test.h> #include "../kselftest.h" - -#include "util.h" +#include "vm_util.h" #define MB (1UL << 20)
@@ -251,7 +250,7 @@ int main(int argc, char **argv)if (touch) { gup.gup_flags |=FOLL_TOUCH ; }else { -for (; (unsigned long )p < gup.addr + size; p +=PAGE_SIZE ) +for (; (unsigned long )p < gup.addr + size; p +=psize ()) p[0 ] =0 ; }
tools/testing/selftests/mm/ksm_tests.c +1 -1
@@ -12,7 +12,7 @@#include "../kselftest.h" #include <include/vdso/time64.h> -#include "util.h" +#include "vm_util.h" #define KSM_SYSFS_PATH "/sys/kernel/mm/ksm/"#define KSM_FP (s) (KSM_SYSFS_PATH s)
tools/testing/selftests/mm/mrelease_test.c +5 -6
@@ -9,8 +9,7 @@#include <stdlib.h> #include <sys/wait.h> #include <unistd.h> - -#include "util.h" +#include "vm_util.h" #include "../kselftest.h"
@@ -32,7 +31,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)unsigned long i;char *buf; - buf = (char *)mmap (NULL , nr_pages *PAGE_SIZE ,PROT_READ |PROT_WRITE , + buf = (char *)mmap (NULL , nr_pages *psize (),PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANON ,0 ,0 );if (buf ==MAP_FAILED ) {perror ("mmap failed, halting the test" );
@@ -40,7 +39,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd) }for (i =0 ; i < nr_pages; i++) - *((unsigned long *)(buf + (i *PAGE_SIZE ))) = i; + *((unsigned long *)(buf + (i *psize ()))) = i;/* Signal the parent that the child is ready */ if (write (pipefd,"" ,1 ) <0 ) {
@@ -54,7 +53,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd) timeout--; } -munmap (buf, nr_pages *PAGE_SIZE ); +munmap (buf, nr_pages *psize ());return (timeout >0 ) ?KSFT_PASS :KSFT_FAIL ; }
@@ -87,7 +86,7 @@ static int child_main(int pipefd[], size_t size)/* Allocate and fault-in memory and wait to be killed */ close (pipefd[0 ]); - res =alloc_noexit (MB (size) /PAGE_SIZE , pipefd[1 ]); + res =alloc_noexit (MB (size) /psize (), pipefd[1 ]);close (pipefd[1 ]);return res; }
tools/testing/selftests/mm/transhuge-stress.c +6 -6
@@ -15,7 +15,7 @@#include <fcntl.h> #include <string.h> #include <sys/mman.h> -#include "util.h" +#include "vm_util.h" int backing_fd =-1 ;int mmap_flags =MAP_ANONYMOUS |MAP_NORESERVE |MAP_PRIVATE ;
@@ -34,10 +34,10 @@ int main(int argc, char **argv)int pagemap_fd; ram =sysconf (_SC_PHYS_PAGES); -if (ram >SIZE_MAX /sysconf (_SC_PAGESIZE) /4 ) +if (ram >SIZE_MAX /psize () /4 ) ram =SIZE_MAX /4 ;else - ram *=sysconf (_SC_PAGESIZE); + ram *=psize (); len = ram;while (++i < argc) {
@@ -58,7 +58,7 @@ int main(int argc, char **argv)warnx ("allocate %zd transhuge pages, using %zd MiB virtual memory" " and %zd MiB of ram" , len >>HPAGE_SHIFT , len >>20 , - ram >> (20 +HPAGE_SHIFT -PAGE_SHIFT -1 )); + ram >> (20 +HPAGE_SHIFT -pshift () -1 )); pagemap_fd =open ("/proc/self/pagemap" ,O_RDONLY );if (pagemap_fd <0 )
@@ -92,7 +92,7 @@ int main(int argc, char **argv)if (pfn <0 ) { nr_failed++; }else { -size_t idx = pfn >> (HPAGE_SHIFT -PAGE_SHIFT ); +size_t idx = pfn >> (HPAGE_SHIFT -pshift ()); nr_succeed++;if (idx >= map_len) {
@@ -108,7 +108,7 @@ int main(int argc, char **argv) }/* split transhuge page, keep last page */ -if (madvise (p,HPAGE_SIZE -PAGE_SIZE ,MADV_DONTNEED )) +if (madvise (p,HPAGE_SIZE -psize (),MADV_DONTNEED ))err (2 ,"MADV_DONTNEED" ); }clock_gettime (CLOCK_MONOTONIC , &b);
tools/testing/selftests/mm/util.h deleted +0 -69
69 lines not on this page · raw message
tools/testing/selftests/mm/vm_util.c +31 -0
@@ -8,6 +8,9 @@#define SMAP_FILE_PATH "/proc/self/smaps"#define MAX_LINE_LENGTH 500 +unsigned int __page_size; +unsigned int __page_shift; +uint64_t pagemap_get_entry (int fd,char *start) {const unsigned long pfn = (unsigned long )start /getpagesize ();
@@ -149,3 +152,31 @@ bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size) {return __check_huge (addr,"ShmemPmdMapped:" , nr_hpages, hpage_size); } + +int64_t allocate_transhuge (void *ptr,int pagemap_fd) +{ +uint64_t ent[2 ]; + +/* drop pmd */ +if (mmap (ptr,HPAGE_SIZE ,PROT_READ |PROT_WRITE , +MAP_FIXED |MAP_ANONYMOUS | +MAP_NORESERVE |MAP_PRIVATE ,-1 ,0 ) != ptr) +errx (2 ,"mmap transhuge" ); + +if (madvise (ptr,HPAGE_SIZE ,MADV_HUGEPAGE )) +err (2 ,"MADV_HUGEPAGE" ); + +/* allocate transparent huge page */ + *(volatile void **)ptr = ptr; + +if (pread (pagemap_fd, ent,sizeof (ent), + (uintptr_t )ptr >> (pshift () -3 )) !=sizeof (ent)) +err (2 ,"read pagemap" ); + +if (PAGEMAP_PRESENT (ent[0 ]) &&PAGEMAP_PRESENT (ent[1 ]) && +PAGEMAP_PFN (ent[0 ]) +1 ==PAGEMAP_PFN (ent[1 ]) && + !(PAGEMAP_PFN (ent[0 ]) & ((1 << (HPAGE_SHIFT -pshift ())) -1 ))) +return PAGEMAP_PFN (ent[0 ]); + +return -1 ; +}
tools/testing/selftests/mm/vm_util.h +31 -0
@@ -1,6 +1,27 @@/* SPDX-License-Identifier: GPL-2.0 */ #include <stdint.h> #include <stdbool.h> +#include <sys/mman.h> +#include <err.h> +#include <string.h> /* ffsl() */ +#include <unistd.h> /* _SC_PAGESIZE */ + +extern unsigned int __page_size; +extern unsigned int __page_shift; + +static inline unsigned int psize (void ) +{ +if (!__page_size) + __page_size =sysconf (_SC_PAGESIZE); +return __page_size; +} + +static inline unsigned int pshift (void ) +{ +if (!__page_shift) + __page_shift = (ffsl (psize ()) -1 ); +return __page_shift; +}uint64_t pagemap_get_entry (int fd,char *start);bool pagemap_is_softdirty (int fd,char *start);
@@ -13,3 +34,13 @@ uint64_t read_pmd_pagesize(void);bool check_huge_anon (void *addr,int nr_hpages,uint64_t hpage_size);bool check_huge_file (void *addr,int nr_hpages,uint64_t hpage_size);bool check_huge_shmem (void *addr,int nr_hpages,uint64_t hpage_size); +int64_t allocate_transhuge (void *ptr,int pagemap_fd); + +/* +* On ppc64 this will only work with radix 2M hugepage size +*/ +#define HPAGE_SHIFT 21 +#define HPAGE_SIZE (1 << HPAGE_SHIFT) + +#define PAGEMAP_PRESENT (ent) (((ent) & (1ull << 63)) != 0) +#define PAGEMAP_PFN (ent) ((ent) & ((1ull << 55) - 1))
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 5/31 selftests/mm: Use TEST_GEN_PROGS where proper
In reply to Peter XuTEST_GEN_PROGS and TEST_GEN_FILES are used randomly in the mm/Makefile to specify programs that need to build. Logically all these binaries should all fall into TEST_GEN_PROGS.
Replace those TEST_GEN_FILES with TEST_GEN_PROGS, so that we can reference all the tests easily later.
--- tools/testing/selftests/mm/Makefile | 63 +++++++++++++++-------------- 1 file changed, 32 insertions(+), 31 deletions(-)
tools/testing/selftests/mm/Makefile +32 -31
@@ -31,35 +31,36 @@ MAKEFLAGS += --no-builtin-rules CFLAGS = -Wall -I $(top_srcdir) $(EXTRA_CFLAGS) $(KHDR_INCLUDES) LDLIBS = -lrt -lpthread -TEST_GEN_FILES = cow -TEST_GEN_FILES += compaction_test -TEST_GEN_FILES += gup_test -TEST_GEN_FILES += hmm-tests -TEST_GEN_FILES += hugetlb-madvise -TEST_GEN_FILES += hugepage-mmap -TEST_GEN_FILES += hugepage-mremap -TEST_GEN_FILES += hugepage-shm -TEST_GEN_FILES += hugepage-vmemmap -TEST_GEN_FILES += khugepaged + +TEST_GEN_PROGS = cow +TEST_GEN_PROGS += compaction_test +TEST_GEN_PROGS += gup_test +TEST_GEN_PROGS += hmm-tests +TEST_GEN_PROGS += hugetlb-madvise +TEST_GEN_PROGS += hugepage-mmap +TEST_GEN_PROGS += hugepage-mremap +TEST_GEN_PROGS += hugepage-shm +TEST_GEN_PROGS += hugepage-vmemmap +TEST_GEN_PROGS += khugepaged TEST_GEN_PROGS = madv_populate -TEST_GEN_FILES += map_fixed_noreplace -TEST_GEN_FILES += map_hugetlb -TEST_GEN_FILES += map_populate -TEST_GEN_FILES += memfd_secret -TEST_GEN_FILES += migration +TEST_GEN_PROGS += map_fixed_noreplace +TEST_GEN_PROGS += map_hugetlb +TEST_GEN_PROGS += map_populate +TEST_GEN_PROGS += memfd_secret +TEST_GEN_PROGS += migration TEST_GEN_PROGS += mkdirty -TEST_GEN_FILES += mlock-random-test -TEST_GEN_FILES += mlock2-tests -TEST_GEN_FILES += mrelease_test -TEST_GEN_FILES += mremap_dontunmap -TEST_GEN_FILES += mremap_test -TEST_GEN_FILES += on-fault-limit -TEST_GEN_FILES += thuge-gen -TEST_GEN_FILES += transhuge-stress -TEST_GEN_FILES += userfaultfd +TEST_GEN_PROGS += mlock-random-test +TEST_GEN_PROGS += mlock2-tests +TEST_GEN_PROGS += mrelease_test +TEST_GEN_PROGS += mremap_dontunmap +TEST_GEN_PROGS += mremap_test +TEST_GEN_PROGS += on-fault-limit +TEST_GEN_PROGS += thuge-gen +TEST_GEN_PROGS += transhuge-stress +TEST_GEN_PROGS += userfaultfd TEST_GEN_PROGS += soft-dirty TEST_GEN_PROGS += split_huge_page_test -TEST_GEN_FILES += ksm_tests +TEST_GEN_PROGS += ksm_tests TEST_GEN_PROGS += ksm_functional_tests TEST_GEN_PROGS += mdwe_test
@@ -77,24 +78,24 @@ CFLAGS += -no-pie endif ifeq ($(CAN_BUILD_I386),1) -TEST_GEN_FILES += $(BINARIES_32) +TEST_GEN_PROGS += $(BINARIES_32) endif ifeq ($(CAN_BUILD_X86_64),1) -TEST_GEN_FILES += $(BINARIES_64) +TEST_GEN_PROGS += $(BINARIES_64) endif else ifneq (,$(findstring $(MACHINE),ppc64)) -TEST_GEN_FILES += protection_keys +TEST_GEN_PROGS += protection_keys endif endif ifneq (,$(filter $(MACHINE),arm64 ia64 mips64 parisc64 ppc64 riscv64 s390x sh64 sparc64 x86_64)) -TEST_GEN_FILES += va_128TBswitch -TEST_GEN_FILES += virtual_address_range -TEST_GEN_FILES += write_to_hugetlbfs +TEST_GEN_PROGS += va_128TBswitch +TEST_GEN_PROGS += virtual_address_range +TEST_GEN_PROGS += write_to_hugetlbfs endif TEST_PROGS := run_vmtests.sh
Signature · 1 line
2.39.1
Re: [PATCH v2 05/31] selftests/mm: Use TEST_GEN_PROGS where proper
Quoted 34 lines · Peter Xu
TEST_GEN_PROGSandTEST_GEN_FILESare used randomly in the mm/Makefile to specify programs that need to build. Logically all these binaries should all fall intoTEST_GEN_PROGS.Replace those
TEST_GEN_FILESwithTEST_GEN_PROGS, so that we can reference all the tests easily later.Reviewed-by David Hildenbrand <[email protected]>Reviewed-by Mike Rapoport (IBM) <[email protected]>Signed-off-by Peter Xu <[email protected]>--- tools/testing/selftests/mm/Makefile | 63 +++++++++++++++-------------- 1 file changed, 32 insertions(+), 31 deletions(-)tools/testing/selftests/mm/Makefile +2
-10@@ -31,35 +31,36 @@ MAKEFLAGS += --no-builtin-rules CFLAGS = -Wall -I $(top_srcdir) $(EXTRA_CFLAGS) $(KHDR_INCLUDES) LDLIBS = -lrt -lpthread -TEST_GEN_FILES = cow -TEST_GEN_FILES += compaction_test -TEST_GEN_FILES += gup_test -TEST_GEN_FILES += hmm-tests -TEST_GEN_FILES += hugetlb-madvise -TEST_GEN_FILES += hugepage-mmap -TEST_GEN_FILES += hugepage-mremap -TEST_GEN_FILES += hugepage-shm -TEST_GEN_FILES += hugepage-vmemmap -TEST_GEN_FILES += khugepaged + +TEST_GEN_PROGS = cow
tools/testing/selftests/mm/Makefile MAKEFLAGS += --no-builtin-rules · line 45
Quoted 10 lines
tools/testing/selftests/mm/Makefile +9
-0+TEST_GEN_PROGS += compaction_test +TEST_GEN_PROGS += gup_test +TEST_GEN_PROGS += hmm-tests +TEST_GEN_PROGS += hugetlb-madvise +TEST_GEN_PROGS += hugepage-mmap +TEST_GEN_PROGS += hugepage-mremap +TEST_GEN_PROGS += hugepage-shm +TEST_GEN_PROGS += hugepage-vmemmap +TEST_GEN_PROGS += khugepaged TEST_GEN_PROGS = madv_populate
This now results in TEST_GEN_PROGS being set to "cow compaction_test gup_test..." then here that is discarded and we start again from madv_populate meaning none of the prior tests get built.
I notice this today when trying to build the gup_test :)
I think this should be a += not an = (which made sense before as there were two different variables being assigned then).
Quoted 70 lines
tools/testing/selftests/mm/Makefile +21
-22-TEST_GEN_FILES += map_fixed_noreplace -TEST_GEN_FILES += map_hugetlb -TEST_GEN_FILES += map_populate -TEST_GEN_FILES += memfd_secret -TEST_GEN_FILES += migration +TEST_GEN_PROGS += map_fixed_noreplace +TEST_GEN_PROGS += map_hugetlb +TEST_GEN_PROGS += map_populate +TEST_GEN_PROGS += memfd_secret +TEST_GEN_PROGS += migration TEST_GEN_PROGS += mkdirty -TEST_GEN_FILES += mlock-random-test -TEST_GEN_FILES += mlock2-tests -TEST_GEN_FILES += mrelease_test -TEST_GEN_FILES += mremap_dontunmap -TEST_GEN_FILES += mremap_test -TEST_GEN_FILES += on-fault-limit -TEST_GEN_FILES += thuge-gen -TEST_GEN_FILES += transhuge-stress -TEST_GEN_FILES += userfaultfd +TEST_GEN_PROGS += mlock-random-test +TEST_GEN_PROGS += mlock2-tests +TEST_GEN_PROGS += mrelease_test +TEST_GEN_PROGS += mremap_dontunmap +TEST_GEN_PROGS += mremap_test +TEST_GEN_PROGS += on-fault-limit +TEST_GEN_PROGS += thuge-gen +TEST_GEN_PROGS += transhuge-stress +TEST_GEN_PROGS += userfaultfd TEST_GEN_PROGS += soft-dirty TEST_GEN_PROGS += split_huge_page_test -TEST_GEN_FILES += ksm_tests +TEST_GEN_PROGS += ksm_tests TEST_GEN_PROGS += ksm_functional_tests TEST_GEN_PROGS += mdwe_test@@ -77,24 +78,24 @@ CFLAGS += -no-pie endif ifeq ($(CAN_BUILD_I386),1) -TEST_GEN_FILES += $(BINARIES_32) +TEST_GEN_PROGS += $(BINARIES_32) endif ifeq ($(CAN_BUILD_X86_64),1) -TEST_GEN_FILES += $(BINARIES_64) +TEST_GEN_PROGS += $(BINARIES_64) endif else ifneq (,$(findstring $(MACHINE),ppc64)) -TEST_GEN_FILES += protection_keys +TEST_GEN_PROGS += protection_keys endif endif ifneq (,$(filter $(MACHINE),arm64 ia64 mips64 parisc64 ppc64 riscv64 s390x sh64 sparc64 x86_64)) -TEST_GEN_FILES += va_128TBswitch -TEST_GEN_FILES += virtual_address_range -TEST_GEN_FILES += write_to_hugetlbfs +TEST_GEN_PROGS += va_128TBswitch +TEST_GEN_PROGS += virtual_address_range +TEST_GEN_PROGS += write_to_hugetlbfs endif TEST_PROGS := run_vmtests.sh --2.39.1
Peter Xu
reviewerRe: [PATCH v2 05/31] selftests/mm: Use TEST_GEN_PROGS where proper
Quoted 10 lines · Lorenzo Stoakes
Quoted 1 line
TEST_GEN_PROGS=madv_populateThis now results in
TEST_GEN_PROGSbeing set to "cowcompaction_testgup_test..." then here that is discarded and we start again frommadv_populatemeaning none of the prior tests get built.I notice this today when trying to build the
gup_test:)I think this should be a += not an = (which made sense before as there were two different variables being assigned then).
Right, thanks for reporting.
This patch will need a fixup. After we start to build all tests for real I found that the other patch "selftests/mm: uffd_[un]register()" actually broke build for testcase hugepage-mremap.c on missing headers.
I've attached two fixups and attached, one for this patch, one for the other one. I hope we can just queue these two fixups in Andrew's tree, or I'll see what I should do.
Thanks,
Signature · 1 line
Peter Xu
From 3ae6d1a0af0aff93734c9b07ff75dc89d3feafaf Mon Sep 17 00:00:00 2001
From: Peter Xu <[email protected]>
Date: Sun, 16 Apr 2023 17:35:02 -0400
Subject: [PATCH 1/2] fixup! selftests/mm: use TEST_GEN_PROGS where proper
--- tools/testing/selftests/mm/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/mm/Makefile +1 -1
@@ -42,7 +42,7 @@ TEST_GEN_PROGS += hugepage-mremap TEST_GEN_PROGS += hugepage-shm TEST_GEN_PROGS += hugepage-vmemmap TEST_GEN_PROGS += khugepaged -TEST_GEN_PROGS = madv_populate +TEST_GEN_PROGS += madv_populate TEST_GEN_PROGS += map_fixed_noreplace TEST_GEN_PROGS += map_hugetlb TEST_GEN_PROGS += map_populate
Signature · 1 line
2.39.1
From b301e2301120d70477f3ae539704cfc0fe031119 Mon Sep 17 00:00:00 2001
From: Peter Xu <[email protected]>
Date: Sun, 16 Apr 2023 17:36:37 -0400
Subject: [PATCH 2/2] fixup! selftests/mm: uffd_[un]register()
--- tools/testing/selftests/mm/hugepage-mremap.c | 2 ++ 1 file changed, 2 insertions(+)
tools/testing/selftests/mm/hugepage-mremap.c +2 -0
@@ -23,6 +23,8 @@#include <linux/userfaultfd.h> #include <sys/ioctl.h> #include <string.h> +#include <stdbool.h> +#include "vm_util.h" #define DEFAULT_LENGTH_MB 10UL#define MB_TO_BYTES (x) (x * 1024 * 1024)
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 6/31 selftests/mm: Link vm_util.c always
In reply to Peter XuWe do have plenty of files that want to link against vm_util.c. Just make it simple by linking it always.
--- tools/testing/selftests/mm/Makefile | 13 +------------ 1 file changed, 1 insertion(+), 12 deletions(-)
tools/testing/selftests/mm/Makefile +1 -12
@@ -106,18 +106,7 @@ TEST_FILES += va_128TBswitch.sh include ../lib.mk -$(OUTPUT)/cow: vm_util.c -$(OUTPUT)/khugepaged: vm_util.c -$(OUTPUT)/ksm_functional_tests: vm_util.c -$(OUTPUT)/madv_populate: vm_util.c -$(OUTPUT)/mkdirty: vm_util.c -$(OUTPUT)/soft-dirty: vm_util.c -$(OUTPUT)/split_huge_page_test: vm_util.c -$(OUTPUT)/userfaultfd: vm_util.c -$(OUTPUT)/gup_test: vm_util.c -$(OUTPUT)/mrelease_test: vm_util.c -$(OUTPUT)/transhuge-stress: vm_util.c -$(OUTPUT)/ksm_tests: vm_util.c +$(TEST_GEN_PROGS): vm_util.c ifeq ($(MACHINE),x86_64) BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 7/31 selftests/mm: Merge default_huge_page_size() into one
In reply to Peter XuThere're already 3 same definitions of the three functions. Move it into vm_util.[ch].
--- tools/testing/selftests/mm/hugetlb-madvise.c | 25 +------------------- tools/testing/selftests/mm/thuge-gen.c | 19 +-------------- tools/testing/selftests/mm/userfaultfd.c | 24 ------------------- tools/testing/selftests/mm/vm_util.c | 21 ++++++++++++++++ tools/testing/selftests/mm/vm_util.h | 1 + 5 files changed, 24 insertions(+), 66 deletions(-)
tools/testing/selftests/mm/hugetlb-madvise.c +1 -24
@@ -18,6 +18,7 @@#include <unistd.h> #include <sys/mman.h> #include <fcntl.h> +#include "vm_util.h" #define MIN_FREE_PAGES 20#define NR_HUGE_PAGES 10/* common number of pages to map/allocate */
@@ -35,30 +36,6 @@unsigned long huge_page_size;unsigned long base_page_size; -/* -* default_huge_page_size copied from mlock2-tests.c -*/ -unsigned long default_huge_page_size (void ) -{ -unsigned long hps =0 ; -char *line =NULL ; -size_t linelen =0 ; -FILE *f =fopen ("/proc/meminfo" ,"r" ); - -if (!f) -return 0 ; -while (getline (&line, &linelen, f) >0 ) { -if (sscanf (line,"Hugepagesize: %lu kB" , &hps) ==1 ) { - hps <<=10 ; -break ; - } - } - -free (line); -fclose (f); -return hps; -} -unsigned long get_free_hugepages (void ) {unsigned long fhp =0 ;
tools/testing/selftests/mm/thuge-gen.c +1 -18
@@ -24,6 +24,7 @@#include <unistd.h> #include <stdarg.h> #include <string.h> +#include "vm_util.h" #define err (x) perror(x), exit(1)
@@ -74,24 +75,6 @@ void find_pagesizes(void)globfree (&g); } -unsigned long default_huge_page_size (void ) -{ -unsigned long hps =0 ; -char *line =NULL ; -size_t linelen =0 ; -FILE *f =fopen ("/proc/meminfo" ,"r" ); -if (!f) -return 0 ; -while (getline (&line, &linelen, f) >0 ) { -if (sscanf (line,"Hugepagesize: %lu kB" , &hps) ==1 ) { - hps <<=10 ; -break ; - } - } -free (line); -return hps; -} -void show (unsigned long ps) {char buf[100 ];
tools/testing/selftests/mm/userfaultfd.c +0 -24
@@ -1703,30 +1703,6 @@ static int userfaultfd_stress(void) || userfaultfd_events_test() || userfaultfd_minor_test(); } -/* -* Copied from mlock2-tests.c -*/ -unsigned long default_huge_page_size (void ) -{ -unsigned long hps =0 ; -char *line =NULL ; -size_t linelen =0 ; -FILE *f =fopen ("/proc/meminfo" ,"r" ); - -if (!f) -return 0 ; -while (getline (&line, &linelen, f) >0 ) { -if (sscanf (line,"Hugepagesize: %lu kB" , &hps) ==1 ) { - hps <<=10 ; -break ; - } - } - -free (line); -fclose (f); -return hps; -} -static void set_test_type (const char *type) {if (!strcmp (type,"anon" )) {
tools/testing/selftests/mm/vm_util.c +21 -0
@@ -180,3 +180,24 @@ int64_t allocate_transhuge(void *ptr, int pagemap_fd)return -1 ; } + +unsigned long default_huge_page_size (void ) +{ +unsigned long hps =0 ; +char *line =NULL ; +size_t linelen =0 ; +FILE *f =fopen ("/proc/meminfo" ,"r" ); + +if (!f) +return 0 ; +while (getline (&line, &linelen, f) >0 ) { +if (sscanf (line,"Hugepagesize: %lu kB" , &hps) ==1 ) { + hps <<=10 ; +break ; + } + } + +free (line); +fclose (f); +return hps; +}
tools/testing/selftests/mm/vm_util.h +1 -0
@@ -35,6 +35,7 @@ bool check_huge_anon(void *addr, int nr_hpages, uint64_t hpage_size);bool check_huge_file (void *addr,int nr_hpages,uint64_t hpage_size);bool check_huge_shmem (void *addr,int nr_hpages,uint64_t hpage_size);int64_t allocate_transhuge (void *ptr,int pagemap_fd); +unsigned long default_huge_page_size (void );/* * On ppc64 this will only work with radix 2M hugepage size
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 8/31 selftests/mm: Use PM_* macros in vm_utils.h
In reply to Peter XuWe've got the macros in uffd-stress.c, move it over and use it in vm_util.h.
--- tools/testing/selftests/mm/userfaultfd.c | 8 -------- tools/testing/selftests/mm/vm_util.c | 16 ++++------------ tools/testing/selftests/mm/vm_util.h | 8 ++++++++ 3 files changed, 12 insertions(+), 20 deletions(-)
tools/testing/selftests/mm/userfaultfd.c +0 -8
@@ -1389,14 +1389,6 @@ static int userfaultfd_minor_test(void)return stats.missing_faults !=0 || stats.minor_faults != nr_pages; } -#define BIT_ULL (nr) (1ULL << (nr)) -#define PM_SOFT_DIRTY BIT_ULL(55) -#define PM_MMAP_EXCLUSIVE BIT_ULL(56) -#define PM_UFFD_WP BIT_ULL(57) -#define PM_FILE BIT_ULL(61) -#define PM_SWAP BIT_ULL(62) -#define PM_PRESENT BIT_ULL(63) -static int pagemap_open (void ) {int fd =open ("/proc/self/pagemap" ,O_RDONLY );
tools/testing/selftests/mm/vm_util.c +4 -12
@@ -25,25 +25,17 @@ uint64_t pagemap_get_entry(int fd, char *start)bool pagemap_is_softdirty (int fd,char *start) { -uint64_t entry =pagemap_get_entry (fd, start); - -// Check if dirty bit (55th bit) is set -return entry &0x0080000000000000ull ; +return pagemap_get_entry (fd, start) &PM_SOFT_DIRTY ; }bool pagemap_is_swapped (int fd,char *start) { -uint64_t entry =pagemap_get_entry (fd, start); - -return entry &0x4000000000000000ull ; +return pagemap_get_entry (fd, start) &PM_SWAP ; }bool pagemap_is_populated (int fd,char *start) { -uint64_t entry =pagemap_get_entry (fd, start); - -/* Present or swapped. */ -return entry &0xc000000000000000ull ; +return pagemap_get_entry (fd, start) & (PM_PRESENT |PM_SWAP ); }unsigned long pagemap_get_pfn (int fd,char *start)
@@ -51,7 +43,7 @@ unsigned long pagemap_get_pfn(int fd, char *start)uint64_t entry =pagemap_get_entry (fd, start);/* If present (63th bit), PFN is at bit 0 -- 54. */ -if (entry &0x8000000000000000ull ) +if (entry &PM_PRESENT )return entry &0x007fffffffffffffull ;return -1ul ; }
tools/testing/selftests/mm/vm_util.h +8 -0
@@ -6,6 +6,14 @@#include <string.h> /* ffsl() */ #include <unistd.h> /* _SC_PAGESIZE */ +#define BIT_ULL (nr) (1ULL << (nr)) +#define PM_SOFT_DIRTY BIT_ULL(55) +#define PM_MMAP_EXCLUSIVE BIT_ULL(56) +#define PM_UFFD_WP BIT_ULL(57) +#define PM_FILE BIT_ULL(61) +#define PM_SWAP BIT_ULL(62) +#define PM_PRESENT BIT_ULL(63) +extern unsigned int __page_size;extern unsigned int __page_shift;
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 9/31 selftests/mm: Reuse pagemap_get_entry() in vm_util.h
In reply to Peter XuMeanwhile drop pagemap_read_vaddr().
--- tools/testing/selftests/mm/userfaultfd.c | 31 +++++++----------------- 1 file changed, 9 insertions(+), 22 deletions(-)
tools/testing/selftests/mm/userfaultfd.c +9 -22
@@ -1399,19 +1399,6 @@ static int pagemap_open(void)return fd; } -static uint64_t pagemap_read_vaddr (int fd,void *vaddr) -{ -uint64_t value; -int ret; - - ret =pread (fd, &value,sizeof (uint64_t ), - ((uint64_t )vaddr >>12 ) *sizeof (uint64_t )); -if (ret !=sizeof (uint64_t )) -err ("pread() on pagemap failed" ); - -return value; -} -/* This macro let __LINE__ works in err() */ #define pagemap_check_wp (value, wp) do { \ if (!!(value & PM_UFFD_WP) != wp) \
@@ -1427,7 +1414,7 @@ static int pagemap_test_fork(bool present)if (!child) {/* Open the pagemap fd of the child itself */ fd =pagemap_open (); - value =pagemap_read_vaddr (fd, area_dst); + value =pagemap_get_entry (fd, area_dst);/* * After fork() uffd-wp bit should be gone as long as we're * without UFFD_FEATURE_EVENT_FORK
@@ -1446,24 +1433,24 @@ static void userfaultfd_wp_unpopulated_test(int pagemap_fd)/* Test applying pte marker to anon unpopulated */ wp_range (uffd, (uint64_t )area_dst, page_size, true); - value =pagemap_read_vaddr (pagemap_fd, area_dst); + value =pagemap_get_entry (pagemap_fd, area_dst);pagemap_check_wp (value, true);/* Test unprotect on anon pte marker */ wp_range (uffd, (uint64_t )area_dst, page_size, false); - value =pagemap_read_vaddr (pagemap_fd, area_dst); + value =pagemap_get_entry (pagemap_fd, area_dst);pagemap_check_wp (value, false);/* Test zap on anon marker */ wp_range (uffd, (uint64_t )area_dst, page_size, true);if (madvise (area_dst, page_size,MADV_DONTNEED ))err ("madvise(MADV_DONTNEED) failed" ); - value =pagemap_read_vaddr (pagemap_fd, area_dst); + value =pagemap_get_entry (pagemap_fd, area_dst);pagemap_check_wp (value, false);/* Test fault in after marker removed */ *area_dst =1 ; - value =pagemap_read_vaddr (pagemap_fd, area_dst); + value =pagemap_get_entry (pagemap_fd, area_dst);pagemap_check_wp (value, false);/* Drop it to make pte none again */ if (madvise (area_dst, page_size,MADV_DONTNEED ))
@@ -1522,7 +1509,7 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)/* Touch the page */ *area_dst =1 ;wp_range (uffd, (uint64_t )area_dst, test_pgsize, true); - value =pagemap_read_vaddr (pagemap_fd, area_dst); + value =pagemap_get_entry (pagemap_fd, area_dst);pagemap_check_wp (value, true);/* Make sure uffd-wp bit dropped when fork */ if (pagemap_test_fork (true))
@@ -1536,7 +1523,7 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)err ("madvise(MADV_PAGEOUT) failed" );/* Uffd-wp should persist even swapped out */ - value =pagemap_read_vaddr (pagemap_fd, area_dst); + value =pagemap_get_entry (pagemap_fd, area_dst);pagemap_check_wp (value, true);/* Make sure uffd-wp bit dropped when fork */ if (pagemap_test_fork (false))
@@ -1544,12 +1531,12 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)/* Unprotect; this tests swap pte modifications */ wp_range (uffd, (uint64_t )area_dst, page_size, false); - value =pagemap_read_vaddr (pagemap_fd, area_dst); + value =pagemap_get_entry (pagemap_fd, area_dst);pagemap_check_wp (value, false);/* Fault in the page from disk */ *area_dst =2 ; - value =pagemap_read_vaddr (pagemap_fd, area_dst); + value =pagemap_get_entry (pagemap_fd, area_dst);pagemap_check_wp (value, false);close (pagemap_fd);
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 10/31 selftests/mm: Test UFFDIO_ZEROPAGE only when !hugetlb
In reply to Peter XuMake the check as simple as "test_type == TEST_HUGETLB" because that's the only mem that doesn't support ZEROPAGE.
--- tools/testing/selftests/mm/userfaultfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/mm/userfaultfd.c +1 -1
@@ -1118,7 +1118,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry) {struct uffdio_zeropage uffdio_zeropage;int ret; -bool has_zeropage =get_expected_ioctls (0 ) & (1 << _UFFDIO_ZEROPAGE); +bool has_zeropage = !(test_type ==TEST_HUGETLB );__s64 res;if (offset >= nr_pages * page_size)
Signature · 1 line
2.39.1
Re: [PATCH v2 10/31] selftests/mm: Test UFFDIO_ZEROPAGE only when !hugetlb
Quoted 5 lines · Peter Xu
Make the check as simple as "
test_type==TEST_HUGETLB" because that's the only mem that doesn't support ZEROPAGE.Signed-off-by Peter Xu <[email protected]>
The end state we get to in patch 26 is what I was hoping for - we check the ioctls reported by UFFDIO_REGISTER to decide if we rest this or not. So then this intermediate state used to get rid of get_expected_ioctls() is fine.
Quoted 20 lines
--- tools/testing/selftests/mm/userfaultfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/testing/selftests/mm/userfaultfd.c +1
-2@@ -1118,7 +1118,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry) {struct uffdio_zeropage uffdio_zeropage;int ret; -bool has_zeropage =get_expected_ioctls (0 ) & (1 << _UFFDIO_ZEROPAGE); +bool has_zeropage = !(test_type ==TEST_HUGETLB );__s64 res;if (offset >= nr_pages * page_size) --2.39.1
Peter Xu
reviewerPATCH v2 11/31 selftests/mm: Drop test_uffdio_zeropage_eexist
In reply to Peter XuThe idea was trying to flip this var in the alarm handler from time to time to test -EEXIST of UFFDIO_ZEROPAGE, but firstly it's only used in the zeropage test so probably only used once, meanwhile we passed "retry==false" so it'll never got tested anyway.
Drop both sides so we always test UFFDIO_ZEROPAGE retries if has_zeropage is set (!hugetlb).
One more thing to do is doing UFFDIO_REGISTER for the alias buffer too, because otherwise the test won't even pass! We were just lucky that this test never really got ran at all.
--- tools/testing/selftests/mm/userfaultfd.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-)
tools/testing/selftests/mm/userfaultfd.c +11 -9
@@ -88,7 +88,6 @@ static bool test_dev_userfaultfd;/* exercise the test_uffdio_*_eexist every ALARM_INTERVAL_SECS */ #define ALARM_INTERVAL_SECS 10static volatile bool test_uffdio_copy_eexist = true; -static volatile bool test_uffdio_zeropage_eexist = true;/* Whether to test uffd write-protection */ static bool test_uffdio_wp = true;/* Whether to test uffd minor faults */
@@ -1114,7 +1113,7 @@ static void retry_uffdio_zeropage(int ufd, } } -static int __uffdio_zeropage (int ufd,unsigned long offset,bool retry) +static int __uffdio_zeropage (int ufd,unsigned long offset) {struct uffdio_zeropage uffdio_zeropage;int ret;
@@ -1138,11 +1137,8 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)if (res != page_size) {err ("UFFDIO_ZEROPAGE unexpected size" ); }else { -if (test_uffdio_zeropage_eexist && retry) { - test_uffdio_zeropage_eexist = false; -retry_uffdio_zeropage (ufd, &uffdio_zeropage, - offset); - } +retry_uffdio_zeropage (ufd, &uffdio_zeropage, + offset);return 1 ; } }else
@@ -1153,7 +1149,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)static int uffdio_zeropage (int ufd,unsigned long offset) { -return __uffdio_zeropage (ufd, offset, false); +return __uffdio_zeropage (ufd, offset); }/* exercise UFFDIO_ZEROPAGE */
@@ -1177,6 +1173,13 @@ static int userfaultfd_zeropage_test(void)assert_expected_ioctls_present ( uffdio_register.mode, uffdio_register.ioctls); +if (area_dst_alias) { +/* Needed this to test zeropage-retry on shared memory */ + uffdio_register.range.start = (unsigned long ) area_dst_alias; +if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register)) +err ("register failure" ); + } +if (uffdio_zeropage (uffd,0 ))if (my_bcmp (area_dst, zeropage, page_size))err ("zeropage is not zero" );
@@ -1763,7 +1766,6 @@ static void sigalrm(int sig)if (sig !=SIGALRM )abort (); test_uffdio_copy_eexist = true; - test_uffdio_zeropage_eexist = true;alarm (ALARM_INTERVAL_SECS ); }
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 12/31 selftests/mm: Create uffd-common.[ch]
In reply to Peter XuMove common utility functions into uffd-common.[ch] files from the original userfaultfd.c. This prepares for a split of userfaultfd.c into two tests:
one to only cover the old but powerful stress test, the other one covers all the functional tests.
This movement is kind of a brute-force effort for now, with light touch-ups but nothing should really change. There's chances to optimize more, but let's leave that for later.
--- tools/testing/selftests/mm/Makefile | 2 + tools/testing/selftests/mm/uffd-common.c | 611 ++++++++++++++++++++ tools/testing/selftests/mm/uffd-common.h | 117 ++++ tools/testing/selftests/mm/userfaultfd.c | 694 +---------------------- 4 files changed, 731 insertions(+), 693 deletions(-) create mode 100644 tools/testing/selftests/mm/uffd-common.c create mode 100644 tools/testing/selftests/mm/uffd-common.h
tools/testing/selftests/mm/Makefile +2 -0
@@ -108,6 +108,8 @@ include ../lib.mk $(TEST_GEN_PROGS): vm_util.c +$(OUTPUT)/userfaultfd: uffd-common.c + ifeq ($(MACHINE),x86_64) BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32)) BINARIES_64 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_64))
tools/testing/selftests/mm/uffd-common.c new file +611 -0
611 lines not on this page · raw message
tools/testing/selftests/mm/uffd-common.h new file +117 -0
117 lines not on this page · raw message
tools/testing/selftests/mm/userfaultfd.c +1 -693
748 lines not on this page · raw message
Signature · 1 line
2.39.1
Re: [PATCH v2 12/31] selftests/mm: Create uffd-common.[ch]
Quoted 6 lines · Peter Xu
Move common utility functions into uffd-common.[ch] files from the original
userfaultfd.c. This prepares for a split ofuserfaultfd.cinto two tests:
one to only cover the old but powerful stress test, the other one covers all the functional tests.
Quoted 6 lines
This movement is kind of a brute-force effort for now, with light touch-ups but nothing should really change. There's chances to optimize more, but let's leave that for later.
Reviewed-by Mike Rapoport (IBM) <[email protected]>Signed-off-by Peter Xu <[email protected]>
Quoted 1,523 lines
--- tools/testing/selftests/mm/Makefile | 2 + tools/testing/selftests/mm/uffd-common.c | 611 ++++++++++++++++++++ tools/testing/selftests/mm/uffd-common.h | 117 ++++ tools/testing/selftests/mm/userfaultfd.c | 694 +---------------------- 4 files changed, 731 insertions(+), 693 deletions(-) create mode 100644 tools/testing/selftests/mm/uffd-common.c create mode 100644 tools/testing/selftests/mm/uffd-common.htools/testing/selftests/mm/Makefile +2
-0@@ -108,6 +108,8 @@ include ../lib.mk $(TEST_GEN_PROGS): vm_util.c +$(OUTPUT)/userfaultfd: uffd-common.c + ifeq ($(MACHINE),x86_64) BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32)) BINARIES_64 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_64))tools/testing/selftests/mm/uffd-common.c new file +611
-0@@ -0,0 +1,611 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Userfaultfd tests util functions +* +* Copyright (C) 2015-2023 Red Hat, Inc. +*/ + +#include "uffd-common.h" + +#define BASE_PMD_ADDR ((void *)(1UL << 30)) + +volatile bool test_uffdio_copy_eexist = true; +unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size; +char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap; +int mem_fd, uffd =-1 , uffd_flags, finished, *pipefd, test_type; +bool map_shared, test_collapse, test_dev_userfaultfd; +bool test_uffdio_wp = true, test_uffdio_minor = false; +unsigned long long *count_verify; +uffd_test_ops_t *uffd_test_ops; + +static void anon_release_pages (char *rel_area) +{ +if (madvise (rel_area, nr_pages * page_size,MADV_DONTNEED )) +err ("madvise(MADV_DONTNEED) failed" ); +} + +static void anon_allocate_area (void **alloc_area,bool is_src) +{ + *alloc_area =mmap (NULL , nr_pages * page_size,PROT_READ |PROT_WRITE , +MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 ); +} + +static void noop_alias_mapping (__u64 *start,size_t len,unsigned long offset) +{ +} + +static void hugetlb_release_pages (char *rel_area) +{ +if (!map_shared) { +if (madvise (rel_area, nr_pages * page_size,MADV_DONTNEED )) +err ("madvise(MADV_DONTNEED) failed" ); + }else { +if (madvise (rel_area, nr_pages * page_size,MADV_REMOVE )) +err ("madvise(MADV_REMOVE) failed" ); + } +} + +static void hugetlb_allocate_area (void **alloc_area,bool is_src) +{ +off_t size = nr_pages * page_size; +off_t offset = is_src ?0 : size; +void *area_alias =NULL ; +char **alloc_area_alias; + + *alloc_area =mmap (NULL , size,PROT_READ |PROT_WRITE , + (map_shared ?MAP_SHARED :MAP_PRIVATE ) | + (is_src ?0 :MAP_NORESERVE ), + mem_fd, offset); +if (*alloc_area ==MAP_FAILED ) +err ("mmap of hugetlbfs file failed" ); + +if (map_shared) { + area_alias =mmap (NULL , size,PROT_READ |PROT_WRITE , +MAP_SHARED , mem_fd, offset); +if (area_alias ==MAP_FAILED ) +err ("mmap of hugetlb file alias failed" ); + } + +if (is_src) { + alloc_area_alias = &area_src_alias; + }else { + alloc_area_alias = &area_dst_alias; + } +if (area_alias) + *alloc_area_alias = area_alias; +} + +static void hugetlb_alias_mapping (__u64 *start,size_t len,unsigned long offset) +{ +if (!map_shared) +return ; + + *start = (unsigned long ) area_dst_alias + offset; +} + +static void shmem_release_pages (char *rel_area) +{ +if (madvise (rel_area, nr_pages * page_size,MADV_REMOVE )) +err ("madvise(MADV_REMOVE) failed" ); +} + +static void shmem_allocate_area (void **alloc_area,bool is_src) +{ +void *area_alias =NULL ; +size_t bytes = nr_pages * page_size; +unsigned long offset = is_src ?0 : bytes; +char *p =NULL , *p_alias =NULL ; + +if (test_collapse) { + p =BASE_PMD_ADDR ; +if (!is_src) +/* src map + alias + interleaved hpages */ + p +=2 * (bytes + hpage_size); + p_alias = p; + p_alias += bytes; + p_alias += hpage_size;/* Prevent src/dst VMA merge */ + } + + *alloc_area =mmap (p, bytes,PROT_READ |PROT_WRITE ,MAP_SHARED , + mem_fd, offset); +if (*alloc_area ==MAP_FAILED ) +err ("mmap of memfd failed" ); +if (test_collapse && *alloc_area != p) +err ("mmap of memfd failed at %p" , p); + + area_alias =mmap (p_alias, bytes,PROT_READ |PROT_WRITE ,MAP_SHARED , + mem_fd, offset); +if (area_alias ==MAP_FAILED ) +err ("mmap of memfd alias failed" ); +if (test_collapse && area_alias != p_alias) +err ("mmap of anonymous memory failed at %p" , p_alias); + +if (is_src) + area_src_alias = area_alias; +else + area_dst_alias = area_alias; +} + +static void shmem_alias_mapping (__u64 *start,size_t len,unsigned long offset) +{ + *start = (unsigned long )area_dst_alias + offset; +} + +static void shmem_check_pmd_mapping (void *p,int expect_nr_hpages) +{ +if (!check_huge_shmem (area_dst_alias, expect_nr_hpages, hpage_size)) +err ("Did not find expected %d number of hugepages" , + expect_nr_hpages); +} + +struct uffd_test_ops anon_uffd_test_ops = { + .allocate_area = anon_allocate_area, + .release_pages = anon_release_pages, + .alias_mapping = noop_alias_mapping, + .check_pmd_mapping =NULL , +}; + +struct uffd_test_ops shmem_uffd_test_ops = { + .allocate_area = shmem_allocate_area, + .release_pages = shmem_release_pages, + .alias_mapping = shmem_alias_mapping, + .check_pmd_mapping = shmem_check_pmd_mapping, +}; + +struct uffd_test_ops hugetlb_uffd_test_ops = { + .allocate_area = hugetlb_allocate_area, + .release_pages = hugetlb_release_pages, + .alias_mapping = hugetlb_alias_mapping, + .check_pmd_mapping =NULL , +}; + +void uffd_stats_report (struct uffd_stats *stats,int n_cpus) +{ +int i; +unsigned long long miss_total =0 , wp_total =0 , minor_total =0 ; + +for (i =0 ; i < n_cpus; i++) { + miss_total += stats[i].missing_faults; + wp_total += stats[i].wp_faults; + minor_total += stats[i].minor_faults; + } + +printf ("userfaults: " ); +if (miss_total) { +printf ("%llu missing (" , miss_total); +for (i =0 ; i < n_cpus; i++) +printf ("%lu+" , stats[i].missing_faults); +printf ("\b) " ); + } +if (wp_total) { +printf ("%llu wp (" , wp_total); +for (i =0 ; i < n_cpus; i++) +printf ("%lu+" , stats[i].wp_faults); +printf ("\b) " ); + } +if (minor_total) { +printf ("%llu minor (" , minor_total); +for (i =0 ; i < n_cpus; i++) +printf ("%lu+" , stats[i].minor_faults); +printf ("\b)" ); + } +printf ("\n" ); +} + +static int __userfaultfd_open_dev (void ) +{ +int fd, _uffd; + + fd =open ("/dev/userfaultfd" ,O_RDWR |O_CLOEXEC ); +if (fd <0 ) +errexit (KSFT_SKIP ,"opening /dev/userfaultfd failed" ); + + _uffd =ioctl (fd,USERFAULTFD_IOC_NEW ,UFFD_FLAGS ); +if (_uffd <0 ) +errexit (errno ==ENOTTY ?KSFT_SKIP :1 , +"creating userfaultfd failed" ); +close (fd); +return _uffd; +} + +void userfaultfd_open (uint64_t *features) +{ +struct uffdio_api uffdio_api; + +if (test_dev_userfaultfd) + uffd =__userfaultfd_open_dev (); +else { + uffd =syscall (__NR_userfaultfd,UFFD_FLAGS ); +if (uffd <0 ) +errexit (errno ==ENOSYS ?KSFT_SKIP :1 , +"creating userfaultfd failed" ); + } + uffd_flags =fcntl (uffd,F_GETFD ,NULL ); + + uffdio_api.api =UFFD_API ; + uffdio_api.features = *features; +if (ioctl (uffd,UFFDIO_API , &uffdio_api)) +err ("UFFDIO_API failed.\nPlease make sure to " +"run with either root or ptrace capability." ); +if (uffdio_api.api !=UFFD_API ) +err ("UFFDIO_API error: %" PRIu64, (uint64_t )uffdio_api.api); + + *features = uffdio_api.features; +} + +static inline void munmap_area (void **area) +{ +if (*area) +if (munmap (*area, nr_pages * page_size)) +err ("munmap" ); + + *area =NULL ; +} + +static void uffd_test_ctx_clear (void ) +{ +size_t i; + +if (pipefd) { +for (i =0 ; i < nr_cpus *2 ; ++i) { +if (close (pipefd[i])) +err ("close pipefd" ); + } +free (pipefd); + pipefd =NULL ; + } + +if (count_verify) { +free (count_verify); + count_verify =NULL ; + } + +if (uffd !=-1 ) { +if (close (uffd)) +err ("close uffd" ); + uffd =-1 ; + } + +munmap_area ((void **)&area_src); +munmap_area ((void **)&area_src_alias); +munmap_area ((void **)&area_dst); +munmap_area ((void **)&area_dst_alias); +munmap_area ((void **)&area_remap); +} + +void uffd_test_ctx_init (uint64_t features) +{ +unsigned long nr, cpu; + +uffd_test_ctx_clear (); + + uffd_test_ops->allocate_area ((void **)&area_src, true); + uffd_test_ops->allocate_area ((void **)&area_dst, false); + +userfaultfd_open (&features); + + count_verify =malloc (nr_pages *sizeof (unsigned long long )); +if (!count_verify) +err ("count_verify" ); + +for (nr =0 ; nr < nr_pages; nr++) { + *area_mutex (area_src, nr) = + (pthread_mutex_t )PTHREAD_MUTEX_INITIALIZER ; + count_verify[nr] = *area_count (area_src, nr) =1 ; +/* +* In the transition between 255 to 256, powerpc will +* read out of order in my_bcmp and see both bytes as +* zero, so leave a placeholder below always non-zero +* after the count, to avoid my_bcmp to trigger false +* positives. +*/ + *(area_count (area_src, nr) +1 ) =1 ; + } + +/* +* After initialization of area_src, we must explicitly release pages +* for area_dst to make sure it's fully empty. Otherwise we could have +* some area_dst pages be errornously initialized with zero pages, +* hence we could hit memory corruption later in the test. +* +* One example is when THP is globally enabled, above allocate_area() +* calls could have the two areas merged into a single VMA (as they +* will have the same VMA flags so they're mergeable). When we +* initialize the area_src above, it's possible that some part of +* area_dst could have been faulted in via one huge THP that will be +* shared between area_src and area_dst. It could cause some of the +* area_dst won't be trapped by missing userfaults. +* +* This release_pages() will guarantee even if that happened, we'll +* proactively split the thp and drop any accidentally initialized +* pages within area_dst. +*/ + uffd_test_ops->release_pages (area_dst); + + pipefd =malloc (sizeof (int ) * nr_cpus *2 ); +if (!pipefd) +err ("pipefd" ); +for (cpu =0 ; cpu < nr_cpus; cpu++) +if (pipe2 (&pipefd[cpu *2 ],O_CLOEXEC |O_NONBLOCK )) +err ("pipe" ); +} + +uint64_t get_expected_ioctls (uint64_t mode) +{ +uint64_t ioctls =UFFD_API_RANGE_IOCTLS ; + +if (test_type ==TEST_HUGETLB ) + ioctls &= ~(1 << _UFFDIO_ZEROPAGE); + +if (!((mode &UFFDIO_REGISTER_MODE_WP ) && test_uffdio_wp)) + ioctls &= ~(1 << _UFFDIO_WRITEPROTECT); + +if (!((mode &UFFDIO_REGISTER_MODE_MINOR ) && test_uffdio_minor)) + ioctls &= ~(1 << _UFFDIO_CONTINUE); + +return ioctls; +} + +void assert_expected_ioctls_present (uint64_t mode,uint64_t ioctls) +{ +uint64_t expected =get_expected_ioctls (mode); +uint64_t actual = ioctls & expected; + +if (actual != expected) { +err ("missing ioctl(s): expected %" PRIx64" actual: %" PRIx64, + expected, actual); + } +} + +void wp_range (int ufd,__u64 start,__u64 len,bool wp) +{ +struct uffdio_writeprotect prms; + +/* Write protection page faults */ + prms.range.start = start; + prms.range.len = len; +/* Undo write-protect, do wakeup after that */ + prms.mode = wp ?UFFDIO_WRITEPROTECT_MODE_WP :0 ; + +if (ioctl (ufd,UFFDIO_WRITEPROTECT , &prms)) +err ("clear WP failed: address=0x%" PRIx64, (uint64_t )start); +} + +static void continue_range (int ufd,__u64 start,__u64 len) +{ +struct uffdio_continue req; +int ret; + + req.range.start = start; + req.range.len = len; + req.mode =0 ; +if (test_uffdio_wp) + req.mode |=UFFDIO_CONTINUE_MODE_WP ; + +if (ioctl (ufd,UFFDIO_CONTINUE , &req)) +err ("UFFDIO_CONTINUE failed for address 0x%" PRIx64, + (uint64_t )start); + +/* +* Error handling within the kernel for continue is subtly different +* from copy or zeropage, so it may be a source of bugs. Trigger an +* error (-EEXIST) on purpose, to verify doing so doesn't cause a BUG. +*/ + req.mapped =0 ; + ret =ioctl (ufd,UFFDIO_CONTINUE , &req); +if (ret >=0 || req.mapped != -EEXIST ) +err ("failed to exercise UFFDIO_CONTINUE error handling, ret=%d, mapped=%" PRId64, + ret, (int64_t ) req.mapped); +} + +int uffd_read_msg (int ufd,struct uffd_msg *msg) +{ +int ret =read (uffd, msg,sizeof (*msg)); + +if (ret !=sizeof (*msg)) { +if (ret <0 ) { +if (errno ==EAGAIN || errno ==EINTR ) +return 1 ; +err ("blocking read error" ); + }else { +err ("short read" ); + } + } + +return 0 ; +} + +void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_stats *stats) +{ +unsigned long offset; + +if (msg->event !=UFFD_EVENT_PAGEFAULT ) +err ("unexpected msg event %u" , msg->event); + +if (msg->arg.pagefault.flags &UFFD_PAGEFAULT_FLAG_WP ) { +/* Write protect page faults */ +wp_range (uffd, msg->arg.pagefault.address, page_size, false); + stats->wp_faults++; + }else if (msg->arg.pagefault.flags &UFFD_PAGEFAULT_FLAG_MINOR ) { +uint8_t *area; +int b; + +/* +* Minor page faults +* +* To prove we can modify the original range for testing +* purposes, we're going to bit flip this range before +* continuing. +* +* Note that this requires all minor page fault tests operate on +* area_dst (non-UFFD-registered) and area_dst_alias +* (UFFD-registered). +*/ + + area = (uint8_t *)(area_dst + + ((char *)msg->arg.pagefault.address - + area_dst_alias)); +for (b =0 ; b < page_size; ++b) + area[b] = ~area[b]; +continue_range (uffd, msg->arg.pagefault.address, page_size); + stats->minor_faults++; + }else { +/* +* Missing page faults. +* +* Here we force a write check for each of the missing mode +* faults. It's guaranteed because the only threads that +* will trigger uffd faults are the locking threads, and +* their first instruction to touch the missing page will +* always be pthread_mutex_lock(). +* +* Note that here we relied on an NPTL glibc impl detail to +* always read the lock type at the entry of the lock op +* (pthread_mutex_t.__data.__type, offset 0x10) before +* doing any locking operations to guarantee that. It's +* actually not good to rely on this impl detail because +* logically a pthread-compatible lib can implement the +* locks without types and we can fail when linking with +* them. However since we used to find bugs with this +* strict check we still keep it around. Hopefully this +* could be a good hint when it fails again. If one day +* it'll break on some other impl of glibc we'll revisit. +*/ +if (msg->arg.pagefault.flags &UFFD_PAGEFAULT_FLAG_WRITE ) +err ("unexpected write fault" ); + + offset = (char *)(unsigned long )msg->arg.pagefault.address - area_dst; + offset &= ~(page_size-1 ); + +if (copy_page (uffd, offset)) + stats->missing_faults++; + } +} + +void *uffd_poll_thread (void *arg) +{ +struct uffd_stats *stats = (struct uffd_stats *)arg; +unsigned long cpu = stats->cpu; +struct pollfd pollfd[2 ]; +struct uffd_msg msg; +struct uffdio_register uffd_reg; +int ret; +char tmp_chr; + + pollfd[0 ].fd = uffd; + pollfd[0 ].events =POLLIN ; + pollfd[1 ].fd = pipefd[cpu*2 ]; + pollfd[1 ].events =POLLIN ; + +for (;;) { + ret =poll (pollfd,2 ,-1 ); +if (ret <=0 ) { +if (errno ==EINTR || errno ==EAGAIN ) +continue ; +err ("poll error: %d" , ret); + } +if (pollfd[1 ].revents) { +if (!(pollfd[1 ].revents &POLLIN )) +err ("pollfd[1].revents %d" , pollfd[1 ].revents); +if (read (pollfd[1 ].fd, &tmp_chr,1 ) !=1 ) +err ("read pipefd error" ); +break ; + } +if (!(pollfd[0 ].revents &POLLIN )) +err ("pollfd[0].revents %d" , pollfd[0 ].revents); +if (uffd_read_msg (uffd, &msg)) +continue ; +switch (msg.event) { +default : +err ("unexpected msg event %u\n" , msg.event); +break ; +case UFFD_EVENT_PAGEFAULT : +uffd_handle_page_fault (&msg, stats); +break ; +case UFFD_EVENT_FORK : +close (uffd); + uffd = msg.arg.fork.ufd; + pollfd[0 ].fd = uffd; +break ; +case UFFD_EVENT_REMOVE : + uffd_reg.range.start = msg.arg.remove.start; + uffd_reg.range.len = msg.arg.remove.end - + msg.arg.remove.start; +if (ioctl (uffd,UFFDIO_UNREGISTER , &uffd_reg.range)) +err ("remove failure" ); +break ; +case UFFD_EVENT_REMAP : + area_remap = area_dst;/* save for later unmap */ + area_dst = (char *)(unsigned long )msg.arg.remap.to; +break ; + } + } + +return NULL ; +} + +static void retry_copy_page (int ufd,struct uffdio_copy *uffdio_copy, +unsigned long offset) +{ + uffd_test_ops->alias_mapping (&uffdio_copy->dst, + uffdio_copy->len, + offset); +if (ioctl (ufd,UFFDIO_COPY , uffdio_copy)) { +/* real retval in ufdio_copy.copy */ +if (uffdio_copy->copy != -EEXIST ) +err ("UFFDIO_COPY retry error: %" PRId64, + (int64_t )uffdio_copy->copy); + }else { +err ("UFFDIO_COPY retry unexpected: %" PRId64, + (int64_t )uffdio_copy->copy); + } +} + +static void wake_range (int ufd,unsigned long addr,unsigned long len) +{ +struct uffdio_range uffdio_wake; + + uffdio_wake.start = addr; + uffdio_wake.len = len; + +if (ioctl (ufd,UFFDIO_WAKE , &uffdio_wake)) +fprintf (stderr,"error waking %lu\n" , + addr),exit (1 ); +} + +int __copy_page (int ufd,unsigned long offset,bool retry) +{ +struct uffdio_copy uffdio_copy; + +if (offset >= nr_pages * page_size) +err ("unexpected offset %lu\n" , offset); + uffdio_copy.dst = (unsigned long ) area_dst + offset; + uffdio_copy.src = (unsigned long ) area_src + offset; + uffdio_copy.len = page_size; +if (test_uffdio_wp) + uffdio_copy.mode =UFFDIO_COPY_MODE_WP ; +else + uffdio_copy.mode =0 ; + uffdio_copy.copy =0 ; +if (ioctl (ufd,UFFDIO_COPY , &uffdio_copy)) { +/* real retval in ufdio_copy.copy */ +if (uffdio_copy.copy != -EEXIST ) +err ("UFFDIO_COPY error: %" PRId64, + (int64_t )uffdio_copy.copy); +wake_range (ufd, uffdio_copy.dst, page_size); + }else if (uffdio_copy.copy != page_size) { +err ("UFFDIO_COPY error: %" PRId64, (int64_t )uffdio_copy.copy); + }else { +if (test_uffdio_copy_eexist && retry) { + test_uffdio_copy_eexist = false; +retry_copy_page (ufd, &uffdio_copy, offset); + } +return 1 ; + } +return 0 ; +} + +int copy_page (int ufd,unsigned long offset) +{ +return __copy_page (ufd, offset, false); +}tools/testing/selftests/mm/uffd-common.h new file +117
-0@@ -0,0 +1,117 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Userfaultfd tests common header +* +* Copyright (C) 2015-2023 Red Hat, Inc. +*/ +#ifndef __UFFD_COMMON_H__ +#define __UFFD_COMMON_H__ + +#define _GNU_SOURCE +#include <stdio.h> +#include <errno.h> +#include <unistd.h> +#include <stdlib.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <time.h> +#include <signal.h> +#include <poll.h> +#include <string.h> +#include <linux/mman.h> +#include <sys/mman.h> +#include <sys/syscall.h> +#include <sys/ioctl.h> +#include <sys/wait.h> +#include <pthread.h> +#include <linux/userfaultfd.h> +#include <setjmp.h> +#include <stdbool.h> +#include <assert.h> +#include <inttypes.h> +#include <stdint.h> +#include <sys/random.h> + +#include "../kselftest.h" +#include "vm_util.h" + +#define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY) + +#define _err (fmt, ...) \ + do { \ + int ret = errno; \ + fprintf(stderr, "ERROR: " fmt, ##__VA_ARGS__); \ + fprintf(stderr, " (errno=%d, @%s:%d)\n", \ + ret, __FILE__, __LINE__); \ + } while (0) + +#define errexit (exitcode, fmt, ...) \ + do { \ + _err(fmt, ##__VA_ARGS__); \ + exit(exitcode); \ + } while (0) + +#define err (fmt, ...) errexit(1, fmt, ##__VA_ARGS__) + +/* pthread_mutex_t starts at page offset 0 */ +#define area_mutex (___area, ___nr) \ + ((pthread_mutex_t *) ((___area) + (___nr)*page_size)) +/* +* count is placed in the page after pthread_mutex_t naturally aligned +* to avoid non alignment faults on non-x86 archs. +*/ +#define area_count (___area, ___nr) \ + ((volatile unsigned long long *) ((unsigned long) \ + ((___area) + (___nr)*page_size + \ + sizeof(pthread_mutex_t) + \ + sizeof(unsigned long long) - 1) & \ + ~(unsigned long)(sizeof(unsigned long long) \ + - 1))) + +/* Userfaultfd test statistics */ +struct uffd_stats { +int cpu; +unsigned long missing_faults; +unsigned long wp_faults; +unsigned long minor_faults; +}; + +struct uffd_test_ops { +void (*allocate_area)(void **alloc_area,bool is_src); +void (*release_pages)(char *rel_area); +void (*alias_mapping)(__u64 *start,size_t len,unsigned long offset); +void (*check_pmd_mapping)(void *p,int expect_nr_hpages); +}; +typedef struct uffd_test_ops uffd_test_ops_t ; + +extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size; +extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap; +extern int mem_fd, uffd, uffd_flags, finished, *pipefd, test_type; +extern bool map_shared, test_collapse, test_dev_userfaultfd; +extern bool test_uffdio_wp, test_uffdio_minor; +extern unsigned long long *count_verify; +extern volatile bool test_uffdio_copy_eexist; + +extern uffd_test_ops_t anon_uffd_test_ops; +extern uffd_test_ops_t shmem_uffd_test_ops; +extern uffd_test_ops_t hugetlb_uffd_test_ops; +extern uffd_test_ops_t *uffd_test_ops; + +void uffd_stats_report (struct uffd_stats *stats,int n_cpus); +void uffd_test_ctx_init (uint64_t features); +void userfaultfd_open (uint64_t *features); +uint64_t get_expected_ioctls (uint64_t mode); +void assert_expected_ioctls_present (uint64_t mode,uint64_t ioctls); +int uffd_read_msg (int ufd,struct uffd_msg *msg); +void wp_range (int ufd,__u64 start,__u64 len,bool wp); +void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_stats *stats); +int __copy_page (int ufd,unsigned long offset,bool retry); +int copy_page (int ufd,unsigned long offset); +void *uffd_poll_thread (void *arg); + +#define TEST_ANON 1 +#define TEST_HUGETLB 2 +#define TEST_SHMEM 3 + +#endif tools/testing/selftests/mm/userfaultfd.c +1
-694@@ -34,96 +34,20 @@* transfer (UFFDIO_COPY). */ -#define _GNU_SOURCE -#include <stdio.h> -#include <errno.h> -#include <unistd.h> -#include <stdlib.h> -#include <sys/types.h> -#include <sys/stat.h> -#include <fcntl.h> -#include <time.h> -#include <signal.h> -#include <poll.h> -#include <string.h> -#include <linux/mman.h> -#include <sys/mman.h> -#include <sys/syscall.h> -#include <sys/ioctl.h> -#include <sys/wait.h> -#include <pthread.h> -#include <linux/userfaultfd.h> -#include <setjmp.h> -#include <stdbool.h> -#include <assert.h> -#include <inttypes.h> -#include <stdint.h> -#include <sys/random.h> - -#include "../kselftest.h" -#include "vm_util.h" +#include "uffd-common.h" #ifdef __NR_userfaultfd -static unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size; -#define BOUNCE_RANDOM (1<<0)#define BOUNCE_RACINGFAULTS (1<<1)#define BOUNCE_VERIFY (1<<2)#define BOUNCE_POLL (1<<3)static int bounces; -#define TEST_ANON 1 -#define TEST_HUGETLB 2 -#define TEST_SHMEM 3 -static int test_type; - -#define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY) - -#define BASE_PMD_ADDR ((void *)(1UL << 30)) - -/* test using /dev/userfaultfd, instead of userfaultfd(2) */ -static bool test_dev_userfaultfd; -/* exercise the test_uffdio_*_eexist every ALARM_INTERVAL_SECS */ #define ALARM_INTERVAL_SECS 10 -static volatile bool test_uffdio_copy_eexist = true; -/* Whether to test uffd write-protection */ -static bool test_uffdio_wp = true; -/* Whether to test uffd minor faults */ -static bool test_uffdio_minor = false; -static bool map_shared; -static int mem_fd; -static unsigned long long *count_verify; -static int uffd =-1 ; -static int uffd_flags, finished, *pipefd; -static char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;static char *zeropage;pthread_attr_t attr; -static bool test_collapse; - -/* Userfaultfd test statistics */ -struct uffd_stats { -int cpu; -unsigned long missing_faults; -unsigned long wp_faults; -unsigned long minor_faults; -}; - -/* pthread_mutex_t starts at page offset 0 */ -#define area_mutex (___area, ___nr) \ - ((pthread_mutex_t *) ((___area) + (___nr)*page_size)) -/* -* count is placed in the page after pthread_mutex_t naturally aligned -* to avoid non alignment faults on non-x86 archs. -*/ -#define area_count (___area, ___nr) \ - ((volatile unsigned long long *) ((unsigned long) \ - ((___area) + (___nr)*page_size + \ - sizeof(pthread_mutex_t) + \ - sizeof(unsigned long long) - 1) & \ - ~(unsigned long)(sizeof(unsigned long long) \ - - 1)))#define swap (a, b) \ do { typeof(a) __tmp = (a); (a) = (b); (b) = __tmp; } while (0)@@ -166,22 +90,6 @@ static void usage(void)exit (1 ); } -#define _err (fmt, ...) \ - do { \ - int ret = errno; \ - fprintf(stderr, "ERROR: " fmt, ##__VA_ARGS__); \ - fprintf(stderr, " (errno=%d, line=%d)\n", \ - ret, __LINE__); \ - } while (0) - -#define errexit (exitcode, fmt, ...) \ - do { \ - _err(fmt, ##__VA_ARGS__); \ - exit(exitcode); \ - } while (0) - -#define err (fmt, ...) errexit(1, fmt, ##__VA_ARGS__) -static void uffd_stats_reset (struct uffd_stats *uffd_stats,unsigned long n_cpus) {@@ -195,189 +103,6 @@ static void uffd_stats_reset(struct uffd_stats *uffd_stats, } } -static void uffd_stats_report (struct uffd_stats *stats,int n_cpus) -{ -int i; -unsigned long long miss_total =0 , wp_total =0 , minor_total =0 ; - -for (i =0 ; i < n_cpus; i++) { - miss_total += stats[i].missing_faults; - wp_total += stats[i].wp_faults; - minor_total += stats[i].minor_faults; - } - -printf ("userfaults: " ); -if (miss_total) { -printf ("%llu missing (" , miss_total); -for (i =0 ; i < n_cpus; i++) -printf ("%lu+" , stats[i].missing_faults); -printf ("\b) " ); - } -if (wp_total) { -printf ("%llu wp (" , wp_total); -for (i =0 ; i < n_cpus; i++) -printf ("%lu+" , stats[i].wp_faults); -printf ("\b) " ); - } -if (minor_total) { -printf ("%llu minor (" , minor_total); -for (i =0 ; i < n_cpus; i++) -printf ("%lu+" , stats[i].minor_faults); -printf ("\b)" ); - } -printf ("\n" ); -} - -static void anon_release_pages (char *rel_area) -{ -if (madvise (rel_area, nr_pages * page_size,MADV_DONTNEED )) -err ("madvise(MADV_DONTNEED) failed" ); -} - -static void anon_allocate_area (void **alloc_area,bool is_src) -{ - *alloc_area =mmap (NULL , nr_pages * page_size,PROT_READ |PROT_WRITE , -MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 ); -} - -static void noop_alias_mapping (__u64 *start,size_t len,unsigned long offset) -{ -} - -static void hugetlb_release_pages (char *rel_area) -{ -if (!map_shared) { -if (madvise (rel_area, nr_pages * page_size,MADV_DONTNEED )) -err ("madvise(MADV_DONTNEED) failed" ); - }else { -if (madvise (rel_area, nr_pages * page_size,MADV_REMOVE )) -err ("madvise(MADV_REMOVE) failed" ); - } -} - -static void hugetlb_allocate_area (void **alloc_area,bool is_src) -{ -off_t size = nr_pages * page_size; -off_t offset = is_src ?0 : size; -void *area_alias =NULL ; -char **alloc_area_alias; - - *alloc_area =mmap (NULL , size,PROT_READ |PROT_WRITE , - (map_shared ?MAP_SHARED :MAP_PRIVATE ) | - (is_src ?0 :MAP_NORESERVE ), - mem_fd, offset); -if (*alloc_area ==MAP_FAILED ) -err ("mmap of hugetlbfs file failed" ); - -if (map_shared) { - area_alias =mmap (NULL , size,PROT_READ |PROT_WRITE , -MAP_SHARED , mem_fd, offset); -if (area_alias ==MAP_FAILED ) -err ("mmap of hugetlb file alias failed" ); - } - -if (is_src) { - alloc_area_alias = &area_src_alias; - }else { - alloc_area_alias = &area_dst_alias; - } -if (area_alias) - *alloc_area_alias = area_alias; -} - -static void hugetlb_alias_mapping (__u64 *start,size_t len,unsigned long offset) -{ -if (!map_shared) -return ; - - *start = (unsigned long ) area_dst_alias + offset; -} - -static void shmem_release_pages (char *rel_area) -{ -if (madvise (rel_area, nr_pages * page_size,MADV_REMOVE )) -err ("madvise(MADV_REMOVE) failed" ); -} - -static void shmem_allocate_area (void **alloc_area,bool is_src) -{ -void *area_alias =NULL ; -size_t bytes = nr_pages * page_size; -unsigned long offset = is_src ?0 : bytes; -char *p =NULL , *p_alias =NULL ; - -if (test_collapse) { - p =BASE_PMD_ADDR ; -if (!is_src) -/* src map + alias + interleaved hpages */ - p +=2 * (bytes + hpage_size); - p_alias = p; - p_alias += bytes; - p_alias += hpage_size;/* Prevent src/dst VMA merge */ - } - - *alloc_area =mmap (p, bytes,PROT_READ |PROT_WRITE ,MAP_SHARED , - mem_fd, offset); -if (*alloc_area ==MAP_FAILED ) -err ("mmap of memfd failed" ); -if (test_collapse && *alloc_area != p) -err ("mmap of memfd failed at %p" , p); - - area_alias =mmap (p_alias, bytes,PROT_READ |PROT_WRITE ,MAP_SHARED , - mem_fd, offset); -if (area_alias ==MAP_FAILED ) -err ("mmap of memfd alias failed" ); -if (test_collapse && area_alias != p_alias) -err ("mmap of anonymous memory failed at %p" , p_alias); - -if (is_src) - area_src_alias = area_alias; -else - area_dst_alias = area_alias; -} - -static void shmem_alias_mapping (__u64 *start,size_t len,unsigned long offset) -{ - *start = (unsigned long )area_dst_alias + offset; -} - -static void shmem_check_pmd_mapping (void *p,int expect_nr_hpages) -{ -if (!check_huge_shmem (area_dst_alias, expect_nr_hpages, hpage_size)) -err ("Did not find expected %d number of hugepages" , - expect_nr_hpages); -} - -struct uffd_test_ops { -void (*allocate_area)(void **alloc_area,bool is_src); -void (*release_pages)(char *rel_area); -void (*alias_mapping)(__u64 *start,size_t len,unsigned long offset); -void (*check_pmd_mapping)(void *p,int expect_nr_hpages); -}; - -static struct uffd_test_ops anon_uffd_test_ops = { - .allocate_area = anon_allocate_area, - .release_pages = anon_release_pages, - .alias_mapping = noop_alias_mapping, - .check_pmd_mapping =NULL , -}; - -static struct uffd_test_ops shmem_uffd_test_ops = { - .allocate_area = shmem_allocate_area, - .release_pages = shmem_release_pages, - .alias_mapping = shmem_alias_mapping, - .check_pmd_mapping = shmem_check_pmd_mapping, -}; - -static struct uffd_test_ops hugetlb_uffd_test_ops = { - .allocate_area = hugetlb_allocate_area, - .release_pages = hugetlb_release_pages, - .alias_mapping = hugetlb_alias_mapping, - .check_pmd_mapping =NULL , -}; - -static struct uffd_test_ops *uffd_test_ops; -static inline uint64_t uffd_minor_feature (void ) {if (test_type ==TEST_HUGETLB && map_shared)@@ -388,171 +113,6 @@ static inline uint64_t uffd_minor_feature(void)return 0 ; } -static uint64_t get_expected_ioctls (uint64_t mode) -{ -uint64_t ioctls =UFFD_API_RANGE_IOCTLS ; - -if (test_type ==TEST_HUGETLB ) - ioctls &= ~(1 << _UFFDIO_ZEROPAGE); - -if (!((mode &UFFDIO_REGISTER_MODE_WP ) && test_uffdio_wp)) - ioctls &= ~(1 << _UFFDIO_WRITEPROTECT); - -if (!((mode &UFFDIO_REGISTER_MODE_MINOR ) && test_uffdio_minor)) - ioctls &= ~(1 << _UFFDIO_CONTINUE); - -return ioctls; -} - -static void assert_expected_ioctls_present (uint64_t mode,uint64_t ioctls) -{ -uint64_t expected =get_expected_ioctls (mode); -uint64_t actual = ioctls & expected; - -if (actual != expected) { -err ("missing ioctl(s): expected %" PRIx64" actual: %" PRIx64, - expected, actual); - } -} - -static int __userfaultfd_open_dev (void ) -{ -int fd, _uffd; - - fd =open ("/dev/userfaultfd" ,O_RDWR |O_CLOEXEC ); -if (fd <0 ) -errexit (KSFT_SKIP ,"opening /dev/userfaultfd failed" ); - - _uffd =ioctl (fd,USERFAULTFD_IOC_NEW ,UFFD_FLAGS ); -if (_uffd <0 ) -errexit (errno ==ENOTTY ?KSFT_SKIP :1 , -"creating userfaultfd failed" ); -close (fd); -return _uffd; -} - -static void userfaultfd_open (uint64_t *features) -{ -struct uffdio_api uffdio_api; - -if (test_dev_userfaultfd) - uffd =__userfaultfd_open_dev (); -else { - uffd =syscall (__NR_userfaultfd,UFFD_FLAGS ); -if (uffd <0 ) -errexit (errno ==ENOSYS ?KSFT_SKIP :1 , -"creating userfaultfd failed" ); - } - uffd_flags =fcntl (uffd,F_GETFD ,NULL ); - - uffdio_api.api =UFFD_API ; - uffdio_api.features = *features; -if (ioctl (uffd,UFFDIO_API , &uffdio_api)) -err ("UFFDIO_API failed.\nPlease make sure to " -"run with either root or ptrace capability." ); -if (uffdio_api.api !=UFFD_API ) -err ("UFFDIO_API error: %" PRIu64, (uint64_t )uffdio_api.api); - - *features = uffdio_api.features; -} - -static inline void munmap_area (void **area) -{ -if (*area) -if (munmap (*area, nr_pages * page_size)) -err ("munmap" ); - - *area =NULL ; -} - -static void uffd_test_ctx_clear (void ) -{ -size_t i; - -if (pipefd) { -for (i =0 ; i < nr_cpus *2 ; ++i) { -if (close (pipefd[i])) -err ("close pipefd" ); - } -free (pipefd); - pipefd =NULL ; - } - -if (count_verify) { -free (count_verify); - count_verify =NULL ; - } - -if (uffd !=-1 ) { -if (close (uffd)) -err ("close uffd" ); - uffd =-1 ; - } - -munmap_area ((void **)&area_src); -munmap_area ((void **)&area_src_alias); -munmap_area ((void **)&area_dst); -munmap_area ((void **)&area_dst_alias); -munmap_area ((void **)&area_remap); -} - -static void uffd_test_ctx_init (uint64_t features) -{ -unsigned long nr, cpu; - -uffd_test_ctx_clear (); - - uffd_test_ops->allocate_area ((void **)&area_src, true); - uffd_test_ops->allocate_area ((void **)&area_dst, false); - -userfaultfd_open (&features); - - count_verify =malloc (nr_pages *sizeof (unsigned long long )); -if (!count_verify) -err ("count_verify" ); - -for (nr =0 ; nr < nr_pages; nr++) { - *area_mutex (area_src, nr) = - (pthread_mutex_t )PTHREAD_MUTEX_INITIALIZER ; - count_verify[nr] = *area_count (area_src, nr) =1 ; -/* -* In the transition between 255 to 256, powerpc will -* read out of order in my_bcmp and see both bytes as -* zero, so leave a placeholder below always non-zero -* after the count, to avoid my_bcmp to trigger false -* positives. -*/ - *(area_count (area_src, nr) +1 ) =1 ; - } - -/* -* After initialization of area_src, we must explicitly release pages -* for area_dst to make sure it's fully empty. Otherwise we could have -* some area_dst pages be errornously initialized with zero pages, -* hence we could hit memory corruption later in the test. -* -* One example is when THP is globally enabled, above allocate_area() -* calls could have the two areas merged into a single VMA (as they -* will have the same VMA flags so they're mergeable). When we -* initialize the area_src above, it's possible that some part of -* area_dst could have been faulted in via one huge THP that will be -* shared between area_src and area_dst. It could cause some of the -* area_dst won't be trapped by missing userfaults. -* -* This release_pages() will guarantee even if that happened, we'll -* proactively split the thp and drop any accidentally initialized -* pages within area_dst. -*/ - uffd_test_ops->release_pages (area_dst); - - pipefd =malloc (sizeof (int ) * nr_cpus *2 ); -if (!pipefd) -err ("pipefd" ); -for (cpu =0 ; cpu < nr_cpus; cpu++) -if (pipe2 (&pipefd[cpu *2 ],O_CLOEXEC |O_NONBLOCK )) -err ("pipe" ); -} -static int my_bcmp (char *str1,char *str2,size_t n) {unsigned long i;@@ -562,47 +122,6 @@ static int my_bcmp(char *str1, char *str2, size_t n)return 0 ; } -static void wp_range (int ufd,__u64 start,__u64 len,bool wp) -{ -struct uffdio_writeprotect prms; - -/* Write protection page faults */ - prms.range.start = start; - prms.range.len = len; -/* Undo write-protect, do wakeup after that */ - prms.mode = wp ?UFFDIO_WRITEPROTECT_MODE_WP :0 ; - -if (ioctl (ufd,UFFDIO_WRITEPROTECT , &prms)) -err ("clear WP failed: address=0x%" PRIx64, (uint64_t )start); -} - -static void continue_range (int ufd,__u64 start,__u64 len) -{ -struct uffdio_continue req; -int ret; - - req.range.start = start; - req.range.len = len; - req.mode =0 ; -if (test_uffdio_wp) - req.mode |=UFFDIO_CONTINUE_MODE_WP ; - -if (ioctl (ufd,UFFDIO_CONTINUE , &req)) -err ("UFFDIO_CONTINUE failed for address 0x%" PRIx64, - (uint64_t )start); - -/* -* Error handling within the kernel for continue is subtly different -* from copy or zeropage, so it may be a source of bugs. Trigger an -* error (-EEXIST) on purpose, to verify doing so doesn't cause a BUG. -*/ - req.mapped =0 ; - ret =ioctl (ufd,UFFDIO_CONTINUE , &req); -if (ret >=0 || req.mapped != -EEXIST ) -err ("failed to exercise UFFDIO_CONTINUE error handling, ret=%d, mapped=%" PRId64, - ret, (int64_t ) req.mapped); -} -static void *locking_thread (void *arg) {unsigned long cpu = (unsigned long ) arg;@@ -635,222 +154,11 @@ static void *locking_thread(void *arg)return NULL ; } -static void retry_copy_page (int ufd,struct uffdio_copy *uffdio_copy, -unsigned long offset) -{ - uffd_test_ops->alias_mapping (&uffdio_copy->dst, - uffdio_copy->len, - offset); -if (ioctl (ufd,UFFDIO_COPY , uffdio_copy)) { -/* real retval in ufdio_copy.copy */ -if (uffdio_copy->copy != -EEXIST ) -err ("UFFDIO_COPY retry error: %" PRId64, - (int64_t )uffdio_copy->copy); - }else { -err ("UFFDIO_COPY retry unexpected: %" PRId64, - (int64_t )uffdio_copy->copy); - } -} - -static void wake_range (int ufd,unsigned long addr,unsigned long len) -{ -struct uffdio_range uffdio_wake; - - uffdio_wake.start = addr; - uffdio_wake.len = len; - -if (ioctl (ufd,UFFDIO_WAKE , &uffdio_wake)) -fprintf (stderr,"error waking %lu\n" , - addr),exit (1 ); -} - -static int __copy_page (int ufd,unsigned long offset,bool retry) -{ -struct uffdio_copy uffdio_copy; - -if (offset >= nr_pages * page_size) -err ("unexpected offset %lu\n" , offset); - uffdio_copy.dst = (unsigned long ) area_dst + offset; - uffdio_copy.src = (unsigned long ) area_src + offset; - uffdio_copy.len = page_size; -if (test_uffdio_wp) - uffdio_copy.mode =UFFDIO_COPY_MODE_WP ; -else - uffdio_copy.mode =0 ; - uffdio_copy.copy =0 ; -if (ioctl (ufd,UFFDIO_COPY , &uffdio_copy)) { -/* real retval in ufdio_copy.copy */ -if (uffdio_copy.copy != -EEXIST ) -err ("UFFDIO_COPY error: %" PRId64, - (int64_t )uffdio_copy.copy); -wake_range (ufd, uffdio_copy.dst, page_size); - }else if (uffdio_copy.copy != page_size) { -err ("UFFDIO_COPY error: %" PRId64, (int64_t )uffdio_copy.copy); - }else { -if (test_uffdio_copy_eexist && retry) { - test_uffdio_copy_eexist = false; -retry_copy_page (ufd, &uffdio_copy, offset); - } -return 1 ; - } -return 0 ; -} -static int copy_page_retry (int ufd,unsigned long offset) {return __copy_page (ufd, offset, true); } -static int copy_page (int ufd,unsigned long offset) -{ -return __copy_page (ufd, offset, false); -} - -static int uffd_read_msg (int ufd,struct uffd_msg *msg) -{ -int ret =read (uffd, msg,sizeof (*msg)); - -if (ret !=sizeof (*msg)) { -if (ret <0 ) { -if (errno ==EAGAIN || errno ==EINTR ) -return 1 ; -err ("blocking read error" ); - }else { -err ("short read" ); - } - } - -return 0 ; -} - -static void uffd_handle_page_fault (struct uffd_msg *msg, -struct uffd_stats *stats) -{ -unsigned long offset; - -if (msg->event !=UFFD_EVENT_PAGEFAULT ) -err ("unexpected msg event %u" , msg->event); - -if (msg->arg.pagefault.flags &UFFD_PAGEFAULT_FLAG_WP ) { -/* Write protect page faults */ -wp_range (uffd, msg->arg.pagefault.address, page_size, false); - stats->wp_faults++; - }else if (msg->arg.pagefault.flags &UFFD_PAGEFAULT_FLAG_MINOR ) { -uint8_t *area; -int b; - -/* -* Minor page faults -* -* To prove we can modify the original range for testing -* purposes, we're going to bit flip this range before -* continuing. -* -* Note that this requires all minor page fault tests operate on -* area_dst (non-UFFD-registered) and area_dst_alias -* (UFFD-registered). -*/ - - area = (uint8_t *)(area_dst + - ((char *)msg->arg.pagefault.address - - area_dst_alias)); -for (b =0 ; b < page_size; ++b) - area[b] = ~area[b]; -continue_range (uffd, msg->arg.pagefault.address, page_size); - stats->minor_faults++; - }else { -/* -* Missing page faults. -* -* Here we force a write check for each of the missing mode -* faults. It's guaranteed because the only threads that -* will trigger uffd faults are the locking threads, and -* their first instruction to touch the missing page will -* always be pthread_mutex_lock(). -* -* Note that here we relied on an NPTL glibc impl detail to -* always read the lock type at the entry of the lock op -* (pthread_mutex_t.__data.__type, offset 0x10) before -* doing any locking operations to guarantee that. It's -* actually not good to rely on this impl detail because -* logically a pthread-compatible lib can implement the -* locks without types and we can fail when linking with -* them. However since we used to find bugs with this -* strict check we still keep it around. Hopefully this -* could be a good hint when it fails again. If one day -* it'll break on some other impl of glibc we'll revisit. -*/ -if (msg->arg.pagefault.flags &UFFD_PAGEFAULT_FLAG_WRITE ) -err ("unexpected write fault" ); - - offset = (char *)(unsigned long )msg->arg.pagefault.address - area_dst; - offset &= ~(page_size-1 ); - -if (copy_page (uffd, offset)) - stats->missing_faults++; - } -} - -static void *uffd_poll_thread (void *arg) -{ -struct uffd_stats *stats = (struct uffd_stats *)arg; -unsigned long cpu = stats->cpu; -struct pollfd pollfd[2 ]; -struct uffd_msg msg; -struct uffdio_register uffd_reg; -int ret; -char tmp_chr; - - pollfd[0 ].fd = uffd; - pollfd[0 ].events =POLLIN ; - pollfd[1 ].fd = pipefd[cpu*2 ]; - pollfd[1 ].events =POLLIN ; - -for (;;) { - ret =poll (pollfd,2 ,-1 ); -if (ret <=0 ) { -if (errno ==EINTR || errno ==EAGAIN ) -continue ; -err ("poll error: %d" , ret); - } -if (pollfd[1 ].revents &POLLIN ) { -if (read (pollfd[1 ].fd, &tmp_chr,1 ) !=1 ) -err ("read pipefd error" ); -break ; - } -if (!(pollfd[0 ].revents &POLLIN )) -err ("pollfd[0].revents %d" , pollfd[0 ].revents); -if (uffd_read_msg (uffd, &msg)) -continue ; -switch (msg.event) { -default : -err ("unexpected msg event %u\n" , msg.event); -break ; -case UFFD_EVENT_PAGEFAULT : -uffd_handle_page_fault (&msg, stats); -break ; -case UFFD_EVENT_FORK : -close (uffd); - uffd = msg.arg.fork.ufd; - pollfd[0 ].fd = uffd; -break ; -case UFFD_EVENT_REMOVE : - uffd_reg.range.start = msg.arg.remove.start; - uffd_reg.range.len = msg.arg.remove.end - - msg.arg.remove.start; -if (ioctl (uffd,UFFDIO_UNREGISTER , &uffd_reg.range)) -err ("remove failure" ); -break ; -case UFFD_EVENT_REMAP : - area_remap = area_dst;/* save for later unmap */ - area_dst = (char *)(unsigned long )msg.arg.remap.to; -break ; - } - } - -return NULL ; -} -pthread_mutex_t uffd_read_mutex =PTHREAD_MUTEX_INITIALIZER ;static void *uffd_read_thread (void *arg) --2.39.1
Peter Xu
reviewerPATCH v2 13/31 selftests/mm: Split uffd tests into uffd-stress and uffd-unit-tests
In reply to Peter XuIn many ways it's weird and unwanted to keep all the tests in the same userfaultfd.c at least when still in the current way.
For example, it doesn't make much sense to run the stress test for each method we can create an userfaultfd handle (either via syscall or /dev/ node). It's a waste of time running this twice for the whole stress as the stress paths are the same, only the open path is different.
It's also just weird to need to manually specify different types of memory to run all unit tests for the userfaultfd interface. We should be able to just run a single program and that should go through all functional uffd tests without running the stress test at all. The stress test was more for torturing and finding race conditions. We don't want to wait for stress to finish just to regress test a functional test.
When we start to pile up more things on top of the same file and same functions, things start to go a bit chaos and the code is just harder to maintain too with tons of global variables.
This patch creates a new test uffd-unit-tests to keep userfaultfd unit tests in the future, currently empty.
Meanwhile rename the old userfaultfd.c test to uffd-stress.c.
---
tools/testing/selftests/mm/.gitignore | 3 ++-
tools/testing/selftests/mm/Makefile | 8 +++---
tools/testing/selftests/mm/run_vmtests.sh | 10 ++++---
.../mm/{userfaultfd.c => uffd-stress.c} | 0
tools/testing/selftests/mm/uffd-unit-tests.c | 27 +++++++++++++++++++
5 files changed, 40 insertions(+), 8 deletions(-)
rename tools/testing/selftests/mm/{userfaultfd.c => uffd-stress.c} (100%)
create mode 100644 tools/testing/selftests/mm/uffd-unit-tests.ctools/testing/selftests/mm/.gitignore +2 -1
@@ -21,7 +21,8 @@ protection_keys protection_keys_32 protection_keys_64 madv_populate -userfaultfd +uffd-stress +uffd-unit-tests mlock-intersect-test mlock-random-test virtual_address_range
tools/testing/selftests/mm/Makefile +5 -3
@@ -20,7 +20,7 @@ MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p # Avoid accidental wrong builds, due to built-in rules working just a little # bit too well--but not quite as well as required for our situation here. # -# In other words, "make userfaultfd" is supposed to fail to build at all, +# In other words, "make $SOME_TEST" is supposed to fail to build at all, # because this Makefile only supports either "make" (all), or "make /full/path". # However, the built-in rules, if not suppressed, will pick up CFLAGS and the # initial LDLIBS (but not the target-specific LDLIBS, because those are only
@@ -57,7 +57,8 @@ TEST_GEN_PROGS += mremap_test TEST_GEN_PROGS += on-fault-limit TEST_GEN_PROGS += thuge-gen TEST_GEN_PROGS += transhuge-stress -TEST_GEN_PROGS += userfaultfd +TEST_GEN_PROGS += uffd-stress +TEST_GEN_PROGS += uffd-unit-tests TEST_GEN_PROGS += soft-dirty TEST_GEN_PROGS += split_huge_page_test TEST_GEN_PROGS += ksm_tests
@@ -108,7 +109,8 @@ include ../lib.mk $(TEST_GEN_PROGS): vm_util.c -$(OUTPUT)/userfaultfd: uffd-common.c +$(OUTPUT)/uffd-stress: uffd-common.c +$(OUTPUT)/uffd-unit-tests: uffd-common.c ifeq ($(MACHINE),x86_64) BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
tools/testing/selftests/mm/run_vmtests.sh +6 -4
@@ -196,14 +196,16 @@ CATEGORY="gup_test" run_test ./gup_test -a # Dump pages 0, 19, and 4096, using pin_user_pages: CATEGORY="gup_test" run_test ./gup_test -ct -F 0x1 0 19 0x1000 +CATEGORY="userfaultfd" run_test ./uffd-unit-tests uffd_mods=("" ":dev") +uffd_stress_bin=./uffd-stress for mod in "${uffd_mods[@]}"; do - CATEGORY="userfaultfd" run_test ./userfaultfd anon${mod} 20 16 + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon${mod} 20 16 # Hugetlb tests require source and destination huge pages. Pass in half # the size ($half_ufd_size_MB), which is used for *each*. - CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb${mod} "$half_ufd_size_MB" 32 - CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb_shared${mod} "$half_ufd_size_MB" 32 - CATEGORY="userfaultfd" run_test ./userfaultfd shmem${mod} 20 16 + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb${mod} "$half_ufd_size_MB" 32 + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared${mod} "$half_ufd_size_MB" 32 + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem${mod} 20 16 done #cleanup
tools/testing/selftests/mm/uffd-stress.c renamed from tools/testing/selftests/mm/userfaultfd.c +0 -0
tools/testing/selftests/mm/uffd-unit-tests.c new file +27 -0
@@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Userfaultfd unit tests. +* +* Copyright (C) 2015-2023 Red Hat, Inc. +*/ + +#include "uffd-common.h" + +#ifdef __NR_userfaultfd + +int main (int argc,char *argv[]) +{ +return KSFT_PASS ; +} + +#else /* __NR_userfaultfd */ + +#warning "missing __NR_userfaultfd definition" + +int main (void ) +{ +printf ("Skipping %s (missing __NR_userfaultfd)\n" , __file__); +return KSFT_SKIP ; +} + +#endif /* __NR_userfaultfd */
Signature · 1 line
2.39.1
Re: [PATCH v2 13/31] selftests/mm: Split uffd tests into uffd-stress and uffd-unit-tests
Quoted 20 lines · Peter Xu
In many ways it's weird and unwanted to keep all the tests in the same
userfaultfd.cat least when still in the current way.For example, it doesn't make much sense to run the stress test for each method we can create an userfaultfd handle (either via syscall or
/dev/node). It's a waste of time running this twice for the whole stress as the stress paths are the same, only the open path is different.It's also just weird to need to manually specify different types of memory to run all unit tests for the userfaultfd interface. We should be able to just run a single program and that should go through all functional uffd tests without running the stress test at all. The stress test was more for torturing and finding race conditions. We don't want to wait for stress to finish just to regress test a functional test.
When we start to pile up more things on top of the same file and same functions, things start to go a bit chaos and the code is just harder to maintain too with tons of global variables.
Quoted 7 lines
This patch creates a new test uffd-unit-tests to keep userfaultfd unit tests in the future, currently empty.
Meanwhile rename the old
userfaultfd.ctest touffd-stress.c.Reviewed-by Mike Rapoport (IBM) <[email protected]>Signed-off-by Peter Xu <[email protected]>
Quoted 122 lines
--- tools/testing/selftests/mm/.gitignore | 3 ++- tools/testing/selftests/mm/Makefile | 8 +++--- tools/testing/selftests/mm/run_vmtests.sh | 10 ++++--- .../mm/{userfaultfd.c => uffd-stress.c} | 0 tools/testing/selftests/mm/uffd-unit-tests.c | 27 +++++++++++++++++++ 5 files changed, 40 insertions(+), 8 deletions(-) rename tools/testing/selftests/mm/{userfaultfd.c => uffd-stress.c} (100%) create mode 100644 tools/testing/selftests/mm/uffd-unit-tests.ctools/testing/selftests/mm/.gitignore +2
-1@@ -21,7 +21,8 @@ protection_keys protection_keys_32 protection_keys_64 madv_populate -userfaultfd +uffd-stress +uffd-unit-tests mlock-intersect-test mlock-random-test virtual_address_rangetools/testing/selftests/mm/Makefile +5
-3@@ -20,7 +20,7 @@ MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p # Avoid accidental wrong builds, due to built-in rules working just a little # bit too well--but not quite as well as required for our situation here. # -# In other words, "make userfaultfd" is supposed to fail to build at all, +# In other words, "make $SOME_TEST" is supposed to fail to build at all, # because this Makefile only supports either "make" (all), or "make /full/path". # However, the built-in rules, if not suppressed, will pick up CFLAGS and the # initial LDLIBS (but not the target-specific LDLIBS, because those are only@@ -57,7 +57,8 @@ TEST_GEN_PROGS += mremap_test TEST_GEN_PROGS += on-fault-limit TEST_GEN_PROGS += thuge-gen TEST_GEN_PROGS += transhuge-stress -TEST_GEN_PROGS += userfaultfd +TEST_GEN_PROGS += uffd-stress +TEST_GEN_PROGS += uffd-unit-tests TEST_GEN_PROGS += soft-dirty TEST_GEN_PROGS += split_huge_page_test TEST_GEN_PROGS += ksm_tests@@ -108,7 +109,8 @@ include ../lib.mk $(TEST_GEN_PROGS): vm_util.c -$(OUTPUT)/userfaultfd: uffd-common.c +$(OUTPUT)/uffd-stress: uffd-common.c +$(OUTPUT)/uffd-unit-tests: uffd-common.c ifeq ($(MACHINE),x86_64) BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))tools/testing/selftests/mm/run_vmtests.sh +6
-4@@ -196,14 +196,16 @@ CATEGORY="gup_test" run_test ./gup_test -a # Dump pages 0, 19, and 4096, using pin_user_pages: CATEGORY="gup_test" run_test ./gup_test -ct -F 0x1 0 19 0x1000 +CATEGORY="userfaultfd" run_test ./uffd-unit-tests uffd_mods=("" ":dev") +uffd_stress_bin=./uffd-stress for mod in "${uffd_mods[@]}"; do - CATEGORY="userfaultfd" run_test ./userfaultfd anon${mod} 20 16 + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon${mod} 20 16 # Hugetlb tests require source and destination huge pages. Pass in half # the size ($half_ufd_size_MB), which is used for *each*. - CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb${mod} "$half_ufd_size_MB" 32 - CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb_shared${mod} "$half_ufd_size_MB" 32 - CATEGORY="userfaultfd" run_test ./userfaultfd shmem${mod} 20 16 + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb${mod} "$half_ufd_size_MB" 32 + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared${mod} "$half_ufd_size_MB" 32 + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem${mod} 20 16 done #cleanuptools/testing/selftests/mm/uffd-stress.c renamed from tools/testing/selftests/mm/userfaultfd.c +0
-0tools/testing/selftests/mm/uffd-unit-tests.c new file +27
-1@@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Userfaultfd unit tests. +* +* Copyright (C) 2015-2023 Red Hat, Inc. +*/ + +#include "uffd-common.h" + +#ifdef __NR_userfaultfd + +int main (int argc,char *argv[]) +{ +return KSFT_PASS ; +} + +#else /* __NR_userfaultfd */ + +#warning "missing __NR_userfaultfd definition" + +int main (void ) +{ +printf ("Skipping %s (missing __NR_userfaultfd)\n" , __file__); +return KSFT_SKIP ; +} + +#endif /* __NR_userfaultfd */ --2.39.1
Peter Xu
reviewerPATCH v2 14/31 selftests/mm: uffd_[un]register()
In reply to Peter XuAdd two helpers to register/unregister to an uffd. Use them to drop duplicate codes.
This patch also drops assert_expected_ioctls_present() and get_expected_ioctls(). Reasons:
- It'll need a lot of effort to pass test_type==HUGETLB into it from the upper, so it's the simplest way to get rid of another global var
- The ioctls returned in UFFDIO_REGISTER is hardly useful at all, because any app can already detect kernel support on any ioctl via its corresponding UFFD_FEATURE_*. The check here is for sanity mostly but it's probably destined no user app will even use it.
- It's not friendly to one future goal of uffd to run on old kernels, the problem is get_expected_ioctls() compiles against UFFD_API_RANGE_IOCTLS, which is a value that can change depending on where the test is compiled, rather than reflecting what the kernel underneath has. It means it'll report false negatives on old kernels so it's against our will.
So let's make our live easier.
--- tools/testing/selftests/mm/hugepage-mremap.c | 7 +- .../selftests/mm/ksm_functional_tests.c | 6 +- tools/testing/selftests/mm/uffd-common.c | 27 ------ tools/testing/selftests/mm/uffd-common.h | 2 - tools/testing/selftests/mm/uffd-stress.c | 83 +++++-------------- tools/testing/selftests/mm/vm_util.c | 37 +++++++++ tools/testing/selftests/mm/vm_util.h | 4 + 7 files changed, 62 insertions(+), 104 deletions(-)
tools/testing/selftests/mm/hugepage-mremap.c +1 -6
@@ -60,7 +60,6 @@ static void register_region_with_uffd(char *addr, size_t len) {long uffd;/* userfaultfd file descriptor */ struct uffdio_api uffdio_api; -struct uffdio_register uffdio_register;/* Create and enable userfaultfd object. */
@@ -96,11 +95,7 @@ static void register_region_with_uffd(char *addr, size_t len)* handling by the userfaultfd object. In mode, we request to track * missing pages (i.e., pages that have not yet been faulted in). */ - - uffdio_register.range.start = (unsigned long )addr; - uffdio_register.range.len = len; - uffdio_register.mode =UFFDIO_REGISTER_MODE_MISSING ; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register) ==-1 ) { +if (uffd_register (uffd, addr, len, true, false, false)) {perror ("ioctl-UFFDIO_REGISTER" );exit (1 ); }
tools/testing/selftests/mm/ksm_functional_tests.c +1 -5
@@ -178,7 +178,6 @@ static void test_unmerge_discarded(void)static void test_unmerge_uffd_wp (void ) {struct uffdio_writeprotect uffd_writeprotect; -struct uffdio_register uffdio_register;const unsigned int size =2 * MiB;struct uffdio_api uffdio_api;char *map;
@@ -210,10 +209,7 @@ static void test_unmerge_uffd_wp(void) }/* Register UFFD-WP, no need for an actual handler. */ - uffdio_register.range.start = (unsigned long ) map; - uffdio_register.range.len = size; - uffdio_register.mode =UFFDIO_REGISTER_MODE_WP ; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register) <0 ) { +if (uffd_register (uffd, map, size, false, true, false)) {ksft_test_result_fail ("UFFDIO_REGISTER_MODE_WP failed\n" ); goto close_uffd; }
tools/testing/selftests/mm/uffd-common.c +0 -27
@@ -330,33 +330,6 @@ void uffd_test_ctx_init(uint64_t features)err ("pipe" ); } -uint64_t get_expected_ioctls (uint64_t mode) -{ -uint64_t ioctls =UFFD_API_RANGE_IOCTLS ; - -if (test_type ==TEST_HUGETLB ) - ioctls &= ~(1 << _UFFDIO_ZEROPAGE); - -if (!((mode &UFFDIO_REGISTER_MODE_WP ) && test_uffdio_wp)) - ioctls &= ~(1 << _UFFDIO_WRITEPROTECT); - -if (!((mode &UFFDIO_REGISTER_MODE_MINOR ) && test_uffdio_minor)) - ioctls &= ~(1 << _UFFDIO_CONTINUE); - -return ioctls; -} - -void assert_expected_ioctls_present (uint64_t mode,uint64_t ioctls) -{ -uint64_t expected =get_expected_ioctls (mode); -uint64_t actual = ioctls & expected; - -if (actual != expected) { -err ("missing ioctl(s): expected %" PRIx64" actual: %" PRIx64, - expected, actual); - } -} -void wp_range (int ufd,__u64 start,__u64 len,bool wp) {struct uffdio_writeprotect prms;
tools/testing/selftests/mm/uffd-common.h +0 -2
@@ -101,8 +101,6 @@ extern uffd_test_ops_t *uffd_test_ops;void uffd_stats_report (struct uffd_stats *stats,int n_cpus);void uffd_test_ctx_init (uint64_t features);void userfaultfd_open (uint64_t *features); -uint64_t get_expected_ioctls (uint64_t mode); -void assert_expected_ioctls_present (uint64_t mode,uint64_t ioctls);int uffd_read_msg (int ufd,struct uffd_msg *msg);void wp_range (int ufd,__u64 start,__u64 len,bool wp);void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_stats *stats);
tools/testing/selftests/mm/uffd-stress.c +19 -64
176 lines not on this page · raw message
tools/testing/selftests/mm/vm_util.c +37 -0
@@ -1,6 +1,8 @@// SPDX-License-Identifier: GPL-2.0 #include <string.h> #include <fcntl.h> +#include <sys/ioctl.h> +#include <linux/userfaultfd.h> #include "../kselftest.h" #include "vm_util.h"
@@ -193,3 +195,38 @@ unsigned long default_huge_page_size(void)fclose (f);return hps; } + +int uffd_register (int uffd,void *addr,uint64_t len, +bool miss,bool wp,bool minor) +{ +struct uffdio_register uffdio_register = {0 }; +uint64_t mode =0 ; +int ret =0 ; + +if (miss) + mode |=UFFDIO_REGISTER_MODE_MISSING ; +if (wp) + mode |=UFFDIO_REGISTER_MODE_WP ; +if (minor) + mode |=UFFDIO_REGISTER_MODE_MINOR ; + + uffdio_register.range.start = (unsigned long )addr; + uffdio_register.range.len = len; + uffdio_register.mode = mode; + +if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register) ==-1 ) + ret = -errno; + +return ret; +} + +int uffd_unregister (int uffd,void *addr,uint64_t len) +{ +struct uffdio_range range = { .start = (uintptr_t )addr, .len = len }; +int ret =0 ; + +if (ioctl (uffd,UFFDIO_UNREGISTER , &range) ==-1 ) + ret = -errno; + +return ret; +}
tools/testing/selftests/mm/vm_util.h +4 -0
@@ -45,6 +45,10 @@ bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);int64_t allocate_transhuge (void *ptr,int pagemap_fd);unsigned long default_huge_page_size (void ); +int uffd_register (int uffd,void *addr,uint64_t len, +bool miss,bool wp,bool minor); +int uffd_unregister (int uffd,void *addr,uint64_t len); +/* * On ppc64 this will only work with radix 2M hugepage size */
Signature · 1 line
2.39.1
Re: [PATCH v2 14/31] selftests/mm: uffd_[un]register()
Quoted 15 lines · Peter Xu
Add two helpers to register/unregister to an uffd. Use them to drop duplicate codes.
This patch also drops
assert_expected_ioctls_present()andget_expected_ioctls(). Reasons:- It'll need a lot of effort to pass test_type==HUGETLB into it from the upper, so it's the simplest way to get rid of another global var
- The ioctls returned in
UFFDIO_REGISTERis hardly useful at all, because any app can already detect kernel support on any ioctl via its corresponding UFFD_FEATURE_*. The check here is for sanity mostly but it's probably destined no user app will even use it.
Quoted 5 lines
- It's not friendly to one future goal of uffd to run on old kernels, the problem is
get_expected_ioctls()compiles againstUFFD_API_RANGE_IOCTLS, which is a value that can change depending on where the test is compiled, rather than reflecting what the kernel underneath has. It means it'll report false negatives on old kernels so it's against our will.
I do think there is some value in this test, but I see patch 31 adds a similar test back so it's fine to remove it in the meantime to clean up things like get_expected_ioctls().
Quoted 4 lines
So let's make our live easier.
Signed-off-by Peter Xu <[email protected]>
Quoted 23 lines
--- tools/testing/selftests/mm/hugepage-mremap.c | 7 +- .../selftests/mm/ksm_functional_tests.c | 6 +- tools/testing/selftests/mm/uffd-common.c | 27 ------ tools/testing/selftests/mm/uffd-common.h | 2 - tools/testing/selftests/mm/uffd-stress.c | 83 +++++-------------- tools/testing/selftests/mm/vm_util.c | 37 +++++++++ tools/testing/selftests/mm/vm_util.h | 4 + 7 files changed, 62 insertions(+), 104 deletions(-)tools/testing/selftests/mm/hugepage-mremap.c +0
-1@@ -60,7 +60,6 @@ static void register_region_with_uffd(char *addr, size_t len) {long uffd;/* userfaultfd file descriptor */ struct uffdio_api uffdio_api; -struct uffdio_register uffdio_register;/* Create and enable userfaultfd object. */
tools/testing/selftests/mm/hugepage-mremap.c static void register_region_with_uffd(char *addr, size_t len) · line 98
Quoted 9 lines
tools/testing/selftests/mm/hugepage-mremap.c +1
-5@@ -96,11 +95,7 @@ static void register_region_with_uffd(char *addr, size_t len)* handling by the userfaultfd object. In mode, we request to track * missing pages (i.e., pages that have not yet been faulted in). */ - - uffdio_register.range.start = (unsigned long )addr; - uffdio_register.range.len = len; - uffdio_register.mode =UFFDIO_REGISTER_MODE_MISSING ; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register) ==-1 ) { +if (uffd_register (uffd, addr, len, true, false, false)) {
For what it's worth, I agree with Mike that the booleans here are hard to read. It's not clear what "true, false, false" means without going to look at the header file, whereas "uffd_register(uffd, addr, len, MINOR)" would be immediately clear.
One solution I've seen outside the kernel is to comment the bools, like:
uffd_register(..., /*minor=*/true, /*wp=*/false, /*minor=*/false);
But, then I feel we lose most of the benefit we wanted from switching to bools anyway (code length). :)
I do agree the macro names are unwieldy, and lots of tests already use booleans so we'd have to convert from bool -> flag. If it were me, I would resolve that by:
- Define much shorter aliases for these macros in uffd-common.h
- Consider refactoring callers to not use bools either.
Then again, I also agree with Mike that it's not a deal breaker, if you like this way much more than that alternative we can leave it.
Quoted 340 lines
tools/testing/selftests/mm/hugepage-mremap.c +0
-0perror ("ioctl-UFFDIO_REGISTER" );exit (1 ); }tools/testing/selftests/mm/ksm_functional_tests.c +1
-5@@ -178,7 +178,6 @@ static void test_unmerge_discarded(void)static void test_unmerge_uffd_wp (void ) {struct uffdio_writeprotect uffd_writeprotect; -struct uffdio_register uffdio_register;const unsigned int size =2 * MiB;struct uffdio_api uffdio_api;char *map;@@ -210,10 +209,7 @@ static void test_unmerge_uffd_wp(void) }/* Register UFFD-WP, no need for an actual handler. */ - uffdio_register.range.start = (unsigned long ) map; - uffdio_register.range.len = size; - uffdio_register.mode =UFFDIO_REGISTER_MODE_WP ; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register) <0 ) { +if (uffd_register (uffd, map, size, false, true, false)) {ksft_test_result_fail ("UFFDIO_REGISTER_MODE_WP failed\n" ); goto close_uffd; }tools/testing/selftests/mm/uffd-common.c +0
-27@@ -330,33 +330,6 @@ void uffd_test_ctx_init(uint64_t features)err ("pipe" ); } -uint64_t get_expected_ioctls (uint64_t mode) -{ -uint64_t ioctls =UFFD_API_RANGE_IOCTLS ; - -if (test_type ==TEST_HUGETLB ) - ioctls &= ~(1 << _UFFDIO_ZEROPAGE); - -if (!((mode &UFFDIO_REGISTER_MODE_WP ) && test_uffdio_wp)) - ioctls &= ~(1 << _UFFDIO_WRITEPROTECT); - -if (!((mode &UFFDIO_REGISTER_MODE_MINOR ) && test_uffdio_minor)) - ioctls &= ~(1 << _UFFDIO_CONTINUE); - -return ioctls; -} - -void assert_expected_ioctls_present (uint64_t mode,uint64_t ioctls) -{ -uint64_t expected =get_expected_ioctls (mode); -uint64_t actual = ioctls & expected; - -if (actual != expected) { -err ("missing ioctl(s): expected %" PRIx64" actual: %" PRIx64, - expected, actual); - } -} -void wp_range (int ufd,__u64 start,__u64 len,bool wp) {struct uffdio_writeprotect prms;tools/testing/selftests/mm/uffd-common.h +0
-2@@ -101,8 +101,6 @@ extern uffd_test_ops_t *uffd_test_ops;void uffd_stats_report (struct uffd_stats *stats,int n_cpus);void uffd_test_ctx_init (uint64_t features);void userfaultfd_open (uint64_t *features); -uint64_t get_expected_ioctls (uint64_t mode); -void assert_expected_ioctls_present (uint64_t mode,uint64_t ioctls);int uffd_read_msg (int ufd,struct uffd_msg *msg);void wp_range (int ufd,__u64 start,__u64 len,bool wp);void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_stats *stats);tools/testing/selftests/mm/uffd-stress.c +19
-64@@ -463,28 +463,19 @@ static int uffdio_zeropage(int ufd, unsigned long offset)/* exercise UFFDIO_ZEROPAGE */ static int userfaultfd_zeropage_test (void ) { -struct uffdio_register uffdio_register; -printf ("testing UFFDIO_ZEROPAGE: " );fflush (stdout);uffd_test_ctx_init (0 ); - uffdio_register.range.start = (unsigned long ) area_dst; - uffdio_register.range.len = nr_pages * page_size; - uffdio_register.mode =UFFDIO_REGISTER_MODE_MISSING ; -if (test_uffdio_wp) - uffdio_register.mode |=UFFDIO_REGISTER_MODE_WP ; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register)) +if (uffd_register (uffd, area_dst, nr_pages * page_size, + true, test_uffdio_wp, false))err ("register failure" ); -assert_expected_ioctls_present ( - uffdio_register.mode, uffdio_register.ioctls); -if (area_dst_alias) {/* Needed this to test zeropage-retry on shared memory */ - uffdio_register.range.start = (unsigned long ) area_dst_alias; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register)) +if (uffd_register (uffd, area_dst_alias, nr_pages * page_size, + true, test_uffdio_wp, false))err ("register failure" ); }@@ -498,7 +489,6 @@ static int userfaultfd_zeropage_test(void)static int userfaultfd_events_test (void ) { -struct uffdio_register uffdio_register;pthread_t uffd_mon;int err, features;pid_t pid;@@ -514,17 +504,10 @@ static int userfaultfd_events_test(void)fcntl (uffd,F_SETFL , uffd_flags |O_NONBLOCK ); - uffdio_register.range.start = (unsigned long ) area_dst; - uffdio_register.range.len = nr_pages * page_size; - uffdio_register.mode =UFFDIO_REGISTER_MODE_MISSING ; -if (test_uffdio_wp) - uffdio_register.mode |=UFFDIO_REGISTER_MODE_WP ; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register)) +if (uffd_register (uffd, area_dst, nr_pages * page_size, + true, test_uffdio_wp, false))err ("register failure" ); -assert_expected_ioctls_present ( - uffdio_register.mode, uffdio_register.ioctls); -if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &stats))err ("uffd_poll_thread create" );@@ -550,7 +533,6 @@ static int userfaultfd_events_test(void)static int userfaultfd_sig_test (void ) { -struct uffdio_register uffdio_register;unsigned long userfaults;pthread_t uffd_mon;int err, features;@@ -566,17 +548,10 @@ static int userfaultfd_sig_test(void)fcntl (uffd,F_SETFL , uffd_flags |O_NONBLOCK ); - uffdio_register.range.start = (unsigned long ) area_dst; - uffdio_register.range.len = nr_pages * page_size; - uffdio_register.mode =UFFDIO_REGISTER_MODE_MISSING ; -if (test_uffdio_wp) - uffdio_register.mode |=UFFDIO_REGISTER_MODE_WP ; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register)) +if (uffd_register (uffd, area_dst, nr_pages * page_size, + true, test_uffdio_wp, false))err ("register failure" ); -assert_expected_ioctls_present ( - uffdio_register.mode, uffdio_register.ioctls); -if (faulting_process (1 ))err ("faulting process failed" );@@ -629,7 +604,6 @@ void check_memory_contents(char *p)static int userfaultfd_minor_test (void ) {unsigned long p; -struct uffdio_register uffdio_register;pthread_t uffd_mon;char c;struct uffd_stats stats = {0 };@@ -642,17 +616,10 @@ static int userfaultfd_minor_test(void)uffd_test_ctx_init (uffd_minor_feature ()); - uffdio_register.range.start = (unsigned long )area_dst_alias; - uffdio_register.range.len = nr_pages * page_size; - uffdio_register.mode =UFFDIO_REGISTER_MODE_MINOR ; -if (test_uffdio_wp) - uffdio_register.mode |=UFFDIO_REGISTER_MODE_WP ; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register)) +if (uffd_register (uffd, area_dst_alias, nr_pages * page_size, + false, test_uffdio_wp, true))err ("register failure" ); -assert_expected_ioctls_present ( - uffdio_register.mode, uffdio_register.ioctls); -/* * After registering with UFFD, populate the non-UFFD-registered side of * the shared mapping. This should *not* trigger any UFFD minor faults. @@ -777,7 +744,6 @@ static void userfaultfd_wp_unpopulated_test(int pagemap_fd)static void userfaultfd_pagemap_test (unsigned int test_pgsize) { -struct uffdio_register uffdio_register;int pagemap_fd;uint64_t value;@@ -805,10 +771,8 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)err ("madvise(MADV_NOHUGEPAGE) failed" ); } - uffdio_register.range.start = (unsigned long ) area_dst; - uffdio_register.range.len = nr_pages * page_size; - uffdio_register.mode =UFFDIO_REGISTER_MODE_WP ; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register)) +if (uffd_register (uffd, area_dst, nr_pages * page_size, + false, true, false))err ("register failed" ); pagemap_fd =pagemap_open ();@@ -858,8 +822,8 @@ static int userfaultfd_stress(void) {void *area;unsigned long nr; -struct uffdio_register uffdio_register;struct uffd_stats uffd_stats[nr_cpus]; +uint64_t mem_size = nr_pages * page_size;uffd_test_ctx_init (UFFD_FEATURE_WP_UNPOPULATED );@@ -894,20 +858,13 @@ static int userfaultfd_stress(void)fcntl (uffd,F_SETFL , uffd_flags & ~O_NONBLOCK );/* register */ - uffdio_register.range.start = (unsigned long ) area_dst; - uffdio_register.range.len = nr_pages * page_size; - uffdio_register.mode =UFFDIO_REGISTER_MODE_MISSING ; -if (test_uffdio_wp) - uffdio_register.mode |=UFFDIO_REGISTER_MODE_WP ; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register)) +if (uffd_register (uffd, area_dst, mem_size, + true, test_uffdio_wp, false))err ("register failure" ); -assert_expected_ioctls_present ( - uffdio_register.mode, uffdio_register.ioctls);if (area_dst_alias) { - uffdio_register.range.start = (unsigned long ) - area_dst_alias; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register)) +if (uffd_register (uffd, area_dst_alias, mem_size, + true, test_uffdio_wp, false))err ("register failure alias" ); }@@ -949,12 +906,10 @@ static int userfaultfd_stress(void) nr_pages * page_size, false);/* unregister */ -if (ioctl (uffd,UFFDIO_UNREGISTER , &uffdio_register.range)) +if (uffd_unregister (uffd, area_dst, mem_size))err ("unregister failure" );if (area_dst_alias) { - uffdio_register.range.start = (unsigned long ) area_dst; -if (ioctl (uffd,UFFDIO_UNREGISTER , - &uffdio_register.range)) +if (uffd_unregister (uffd, area_dst_alias, mem_size))err ("unregister failure alias" ); }tools/testing/selftests/mm/vm_util.c +37
-0@@ -1,6 +1,8 @@// SPDX-License-Identifier: GPL-2.0 #include <string.h> #include <fcntl.h> +#include <sys/ioctl.h> +#include <linux/userfaultfd.h> #include "../kselftest.h" #include "vm_util.h" @@ -193,3 +195,38 @@ unsigned long default_huge_page_size(void)fclose (f);return hps; } + +int uffd_register (int uffd,void *addr,uint64_t len, +bool miss,bool wp,bool minor) +{ +struct uffdio_register uffdio_register = {0 }; +uint64_t mode =0 ; +int ret =0 ; + +if (miss) + mode |=UFFDIO_REGISTER_MODE_MISSING ; +if (wp) + mode |=UFFDIO_REGISTER_MODE_WP ; +if (minor) + mode |=UFFDIO_REGISTER_MODE_MINOR ; + + uffdio_register.range.start = (unsigned long )addr; + uffdio_register.range.len = len; + uffdio_register.mode = mode; + +if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register) ==-1 ) + ret = -errno; + +return ret; +} + +int uffd_unregister (int uffd,void *addr,uint64_t len) +{ +struct uffdio_range range = { .start = (uintptr_t )addr, .len = len }; +int ret =0 ; + +if (ioctl (uffd,UFFDIO_UNREGISTER , &range) ==-1 ) + ret = -errno; + +return ret; +}tools/testing/selftests/mm/vm_util.h +4
-1@@ -45,6 +45,10 @@ bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);int64_t allocate_transhuge (void *ptr,int pagemap_fd);unsigned long default_huge_page_size (void ); +int uffd_register (int uffd,void *addr,uint64_t len, +bool miss,bool wp,bool minor); +int uffd_unregister (int uffd,void *addr,uint64_t len); +/* * On ppc64 this will only work with radix 2M hugepage size */ --2.39.1
Peter Xu
reviewerRe: [PATCH v2 14/31] selftests/mm: uffd_[un]register()
Quoted 22 lines · Axel Rasmussen
tools/testing/selftests/mm/hugepage-mremap.c static void register_region_with_uffd(char *addr, size_t len) · line 98
Quoted 5 lines
tools/testing/selftests/mm/hugepage-mremap.c +1
-4- uffdio_register.range.start = (unsigned long )addr; - uffdio_register.range.len = len; - uffdio_register.mode =UFFDIO_REGISTER_MODE_MISSING ; -if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register) ==-1 ) { +if (uffd_register (uffd, addr, len, true, false, false)) {For what it's worth, I agree with Mike that the booleans here are hard to read. It's not clear what "true, false, false" means without going to look at the header file, whereas "
uffd_register(uffd, addr, len, MINOR)" would be immediately clear.One solution I've seen outside the kernel is to comment the bools, like:
uffd_register(..., /*minor=*/true, /*wp=*/false, /*minor=*/false);But, then I feel we lose most of the benefit we wanted from switching to bools anyway (code length). :)
I do agree the macro names are unwieldy, and lots of tests already use booleans so we'd have to convert from bool -> flag. If it were me, I would resolve that by:
Quoted 5 lines
- Define much shorter aliases for these macros in
uffd-common.h
- Consider refactoring callers to not use bools either.Then again, I also agree with Mike that it's not a deal breaker, if you like this way much more than that alternative we can leave it.
No strong feeling here either. I kept it just to avoid code churns and rebases.
I'll see whether I'll need a repost, if so I can go back to use modes.
Thanks,
Signature · 1 line
Peter Xu
Peter Xu
reviewerPATCH v2 15/31 selftests/mm: uffd_open_{dev|sys}()
In reply to Peter XuProvide two helpers to open an uffd handle. Drop the error checks around SKIPs because it's inside an errexit() anyway, which IMHO doesn't really help much if the test will not continue.
--- tools/testing/selftests/mm/uffd-common.c | 28 +++++------------------- tools/testing/selftests/mm/vm_util.c | 24 ++++++++++++++++++++ tools/testing/selftests/mm/vm_util.h | 2 ++ 3 files changed, 31 insertions(+), 23 deletions(-)
tools/testing/selftests/mm/uffd-common.c +5 -23
@@ -192,34 +192,16 @@ void uffd_stats_report(struct uffd_stats *stats, int n_cpus)printf ("\n" ); } -static int __userfaultfd_open_dev (void ) -{ -int fd, _uffd; - - fd =open ("/dev/userfaultfd" ,O_RDWR |O_CLOEXEC ); -if (fd <0 ) -errexit (KSFT_SKIP ,"opening /dev/userfaultfd failed" ); - - _uffd =ioctl (fd,USERFAULTFD_IOC_NEW ,UFFD_FLAGS ); -if (_uffd <0 ) -errexit (errno ==ENOTTY ?KSFT_SKIP :1 , -"creating userfaultfd failed" ); -close (fd); -return _uffd; -} -void userfaultfd_open (uint64_t *features) {struct uffdio_api uffdio_api;if (test_dev_userfaultfd) - uffd =__userfaultfd_open_dev (); -else { - uffd =syscall (__NR_userfaultfd,UFFD_FLAGS ); -if (uffd <0 ) -errexit (errno ==ENOSYS ?KSFT_SKIP :1 , -"creating userfaultfd failed" ); - } + uffd =uffd_open_dev (UFFD_FLAGS ); +else + uffd =uffd_open_sys (UFFD_FLAGS ); +if (uffd <0 ) +err ("uffd open failed (dev=%d)" , test_dev_userfaultfd); uffd_flags =fcntl (uffd,F_GETFD ,NULL ); uffdio_api.api =UFFD_API ;
tools/testing/selftests/mm/vm_util.c +24 -0
@@ -3,6 +3,8 @@#include <fcntl.h> #include <sys/ioctl.h> #include <linux/userfaultfd.h> +#include <sys/syscall.h> +#include <unistd.h> #include "../kselftest.h" #include "vm_util.h"
@@ -230,3 +232,25 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)return ret; } + +int uffd_open_dev (unsigned int flags) +{ +int fd, uffd; + + fd =open ("/dev/userfaultfd" ,O_RDWR |O_CLOEXEC ); +if (fd <0 ) +return fd; + uffd =ioctl (fd,USERFAULTFD_IOC_NEW , flags); +close (fd); + +return uffd; +} + +int uffd_open_sys (unsigned int flags) +{ +#ifdef __NR_userfaultfd +return syscall (__NR_userfaultfd, flags); +#else +return -1 ; +#endif +}
tools/testing/selftests/mm/vm_util.h +2 -0
@@ -48,6 +48,8 @@ unsigned long default_huge_page_size(void);int uffd_register (int uffd,void *addr,uint64_t len,bool miss,bool wp,bool minor);int uffd_unregister (int uffd,void *addr,uint64_t len); +int uffd_open_dev (unsigned int flags); +int uffd_open_sys (unsigned int flags);/* * On ppc64 this will only work with radix 2M hugepage size
Signature · 1 line
2.39.1
Re: [PATCH v2 15/31] selftests/mm: uffd_open_{dev|sys}()
Quoted 8 lines · Peter Xu
Provide two helpers to open an uffd handle. Drop the error checks around SKIPs because it's inside an
errexit()anyway, which IMHO doesn't really help much if the test will not continue.Reviewed-by David Hildenbrand <[email protected]>Reviewed-by Mike Rapoport (IBM) <[email protected]>Signed-off-by Peter Xu <[email protected]>
Quoted 105 lines
--- tools/testing/selftests/mm/uffd-common.c | 28 +++++------------------- tools/testing/selftests/mm/vm_util.c | 24 ++++++++++++++++++++ tools/testing/selftests/mm/vm_util.h | 2 ++ 3 files changed, 31 insertions(+), 23 deletions(-)tools/testing/selftests/mm/uffd-common.c +5
-23@@ -192,34 +192,16 @@ void uffd_stats_report(struct uffd_stats *stats, int n_cpus)printf ("\n" ); } -static int __userfaultfd_open_dev (void ) -{ -int fd, _uffd; - - fd =open ("/dev/userfaultfd" ,O_RDWR |O_CLOEXEC ); -if (fd <0 ) -errexit (KSFT_SKIP ,"opening /dev/userfaultfd failed" ); - - _uffd =ioctl (fd,USERFAULTFD_IOC_NEW ,UFFD_FLAGS ); -if (_uffd <0 ) -errexit (errno ==ENOTTY ?KSFT_SKIP :1 , -"creating userfaultfd failed" ); -close (fd); -return _uffd; -} -void userfaultfd_open (uint64_t *features) {struct uffdio_api uffdio_api;if (test_dev_userfaultfd) - uffd =__userfaultfd_open_dev (); -else { - uffd =syscall (__NR_userfaultfd,UFFD_FLAGS ); -if (uffd <0 ) -errexit (errno ==ENOSYS ?KSFT_SKIP :1 , -"creating userfaultfd failed" ); - } + uffd =uffd_open_dev (UFFD_FLAGS ); +else + uffd =uffd_open_sys (UFFD_FLAGS ); +if (uffd <0 ) +err ("uffd open failed (dev=%d)" , test_dev_userfaultfd); uffd_flags =fcntl (uffd,F_GETFD ,NULL ); uffdio_api.api =UFFD_API ;tools/testing/selftests/mm/vm_util.c +24
-0@@ -3,6 +3,8 @@#include <fcntl.h> #include <sys/ioctl.h> #include <linux/userfaultfd.h> +#include <sys/syscall.h> +#include <unistd.h> #include "../kselftest.h" #include "vm_util.h" @@ -230,3 +232,25 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)return ret; } + +int uffd_open_dev (unsigned int flags) +{ +int fd, uffd; + + fd =open ("/dev/userfaultfd" ,O_RDWR |O_CLOEXEC ); +if (fd <0 ) +return fd; + uffd =ioctl (fd,USERFAULTFD_IOC_NEW , flags); +close (fd); + +return uffd; +} + +int uffd_open_sys (unsigned int flags) +{ +#ifdef __NR_userfaultfd +return syscall (__NR_userfaultfd, flags); +#else +return -1 ; +#endif +}tools/testing/selftests/mm/vm_util.h +2
-1@@ -48,6 +48,8 @@ unsigned long default_huge_page_size(void);int uffd_register (int uffd,void *addr,uint64_t len,bool miss,bool wp,bool minor);int uffd_unregister (int uffd,void *addr,uint64_t len); +int uffd_open_dev (unsigned int flags); +int uffd_open_sys (unsigned int flags);/* * On ppc64 this will only work with radix 2M hugepage size -- 2.39.1
Peter Xu
reviewerPATCH v2 16/31 selftests/mm: UFFDIO_API test
In reply to Peter XuAdd one simple test for UFFDIO_API. With that, I also added a bunch of small but handy helpers along the way.
--- tools/testing/selftests/mm/uffd-unit-tests.c | 109 ++++++++++++++++++- 1 file changed, 108 insertions(+), 1 deletion(-)
tools/testing/selftests/mm/uffd-unit-tests.c +108 -1
@@ -9,9 +9,116 @@#ifdef __NR_userfaultfd +static void uffd_test_report (void ) +{ +printf ("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n" , +ksft_get_pass_cnt (), +ksft_get_xskip_cnt (), +ksft_get_fail_cnt (), +ksft_test_num ()); +} + +static void uffd_test_pass (void ) +{ +printf ("done\n" ); +ksft_inc_pass_cnt (); +} + +#define uffd_test_start (...) do { \ + printf("Testing "); \ + printf(__VA_ARGS__); \ + printf("... "); \ + fflush(stdout); \ + } while (0) + +#define uffd_test_fail (...) do { \ + printf("failed [reason: "); \ + printf(__VA_ARGS__); \ + printf("]\n"); \ + ksft_inc_fail_cnt(); \ + } while (0) + +#define uffd_test_skip (...) do { \ + printf("skipped [reason: "); \ + printf(__VA_ARGS__); \ + printf("]\n"); \ + ksft_inc_xskip_cnt(); \ + } while (0) + +/* +* Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll +* return 1 even if some test failed as long as uffd supported, because in +* that case we still want to proceed with the rest uffd unit tests. +*/ +static int test_uffd_api (bool use_dev) +{ +struct uffdio_api uffdio_api; +int uffd; + +uffd_test_start ("UFFDIO_API (with %s)" , + use_dev ?"/dev/userfaultfd" :"syscall" ); + +if (use_dev) + uffd =uffd_open_dev (UFFD_FLAGS ); +else + uffd =uffd_open_sys (UFFD_FLAGS ); +if (uffd <0 ) { +uffd_test_skip ("cannot open userfaultfd handle" ); +return 0 ; + } + +/* Test wrong UFFD_API */ + uffdio_api.api =0xab ; + uffdio_api.features =0 ; +if (ioctl (uffd,UFFDIO_API , &uffdio_api) ==0 ) { +uffd_test_fail ("UFFDIO_API should fail with wrong api but didn't" ); + goto out; + } + +/* Test wrong feature bit */ + uffdio_api.api =UFFD_API ; + uffdio_api.features =BIT_ULL (63 ); +if (ioctl (uffd,UFFDIO_API , &uffdio_api) ==0 ) { +uffd_test_fail ("UFFDIO_API should fail with wrong feature but didn't" ); + goto out; + } + +/* Test normal UFFDIO_API */ + uffdio_api.api =UFFD_API ; + uffdio_api.features =0 ; +if (ioctl (uffd,UFFDIO_API , &uffdio_api)) { +uffd_test_fail ("UFFDIO_API should succeed but failed" ); + goto out; + } + +/* Test double requests of UFFDIO_API with a random feature set */ + uffdio_api.features =BIT_ULL (0 ); +if (ioctl (uffd,UFFDIO_API , &uffdio_api) ==0 ) { +uffd_test_fail ("UFFDIO_API should reject initialized uffd" ); + goto out; + } + +uffd_test_pass (); +out: +close (uffd); +/* We have a valid uffd handle */ +return 1 ; +} +int main (int argc,char *argv[]) { -return KSFT_PASS ; +int has_uffd; + + has_uffd =test_uffd_api (false); + has_uffd |=test_uffd_api (true); + +if (!has_uffd) { +printf ("Userfaultfd not supported or unprivileged, skip all tests\n" ); +exit (KSFT_SKIP ); + } +uffd_test_report (); + +return ksft_get_fail_cnt () ?KSFT_FAIL :KSFT_PASS ; }#else /* __NR_userfaultfd */
Signature · 1 line
2.39.1
Re: [PATCH v2 16/31] selftests/mm: UFFDIO_API test
Quoted 42 lines · Peter Xu
Add one simple test for
UFFDIO_API. With that, I also added a bunch of small but handy helpers along the way.Signed-off-by Peter Xu <[email protected]>--- tools/testing/selftests/mm/uffd-unit-tests.c | 109 ++++++++++++++++++- 1 file changed, 108 insertions(+), 1 deletion(-)tools/testing/selftests/mm/uffd-unit-tests.c +25
-0@@ -9,9 +9,116 @@#ifdef __NR_userfaultfd +static void uffd_test_report (void ) +{ +printf ("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n" , +ksft_get_pass_cnt (), +ksft_get_xskip_cnt (), +ksft_get_fail_cnt (), +ksft_test_num ()); +} + +static void uffd_test_pass (void ) +{ +printf ("done\n" ); +ksft_inc_pass_cnt (); +} + +#define uffd_test_start (...) do { \ + printf("Testing "); \ + printf(__VA_ARGS__); \ + printf("... "); \ + fflush(stdout); \ + } while (0) + +#define uffd_test_fail (...) do { \ + printf("failed [reason: "); \ + printf(__VA_ARGS__); \
tools/testing/selftests/mm/uffd-unit-tests.c line 46
Quoted 10 lines
tools/testing/selftests/mm/uffd-unit-tests.c +10
-0+printf ("]\n" ); \ +ksft_inc_fail_cnt (); \ + } while (0) + +#define uffd_test_skip (...) do { \ + printf("skipped [reason: "); \ + printf(__VA_ARGS__); \ + printf("]\n"); \ + ksft_inc_xskip_cnt(); \ + } while (0)
Some of this is duplicating what's in kselftest_harness.h - e.g. assertions, printing test summary, keeping track of pass/fail/skip counts, etc.
I wonder how you feel about using the standard harness? E.g. is the plan to do this (more straightforward?) refactor first, and then switch later? Or is there some reason it can't be used?
Quoted 82 lines
tools/testing/selftests/mm/uffd-unit-tests.c +73
-2+ +/* +* Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll +* return 1 even if some test failed as long as uffd supported, because in +* that case we still want to proceed with the rest uffd unit tests. +*/ +static int test_uffd_api (bool use_dev) +{ +struct uffdio_api uffdio_api; +int uffd; + +uffd_test_start ("UFFDIO_API (with %s)" , + use_dev ?"/dev/userfaultfd" :"syscall" ); + +if (use_dev) + uffd =uffd_open_dev (UFFD_FLAGS ); +else + uffd =uffd_open_sys (UFFD_FLAGS ); +if (uffd <0 ) { +uffd_test_skip ("cannot open userfaultfd handle" ); +return 0 ; + } + +/* Test wrong UFFD_API */ + uffdio_api.api =0xab ; + uffdio_api.features =0 ; +if (ioctl (uffd,UFFDIO_API , &uffdio_api) ==0 ) { +uffd_test_fail ("UFFDIO_API should fail with wrong api but didn't" ); + goto out; + } + +/* Test wrong feature bit */ + uffdio_api.api =UFFD_API ; + uffdio_api.features =BIT_ULL (63 ); +if (ioctl (uffd,UFFDIO_API , &uffdio_api) ==0 ) { +uffd_test_fail ("UFFDIO_API should fail with wrong feature but didn't" ); + goto out; + } + +/* Test normal UFFDIO_API */ + uffdio_api.api =UFFD_API ; + uffdio_api.features =0 ; +if (ioctl (uffd,UFFDIO_API , &uffdio_api)) { +uffd_test_fail ("UFFDIO_API should succeed but failed" ); + goto out; + } + +/* Test double requests of UFFDIO_API with a random feature set */ + uffdio_api.features =BIT_ULL (0 ); +if (ioctl (uffd,UFFDIO_API , &uffdio_api) ==0 ) { +uffd_test_fail ("UFFDIO_API should reject initialized uffd" ); + goto out; + } + +uffd_test_pass (); +out: +close (uffd); +/* We have a valid uffd handle */ +return 1 ; +} +int main (int argc,char *argv[]) { -return KSFT_PASS ; +int has_uffd; + + has_uffd =test_uffd_api (false); + has_uffd |=test_uffd_api (true); + +if (!has_uffd) { +printf ("Userfaultfd not supported or unprivileged, skip all tests\n" ); +exit (KSFT_SKIP ); + } +uffd_test_report (); + +return ksft_get_fail_cnt () ?KSFT_FAIL :KSFT_PASS ; }#else /* __NR_userfaultfd */ --2.39.1
Peter Xu
reviewerRe: [PATCH v2 16/31] selftests/mm: UFFDIO_API test
Quoted 18 lines · Axel Rasmussen
tools/testing/selftests/mm/uffd-unit-tests.c line 46
Quoted 13 lines
tools/testing/selftests/mm/uffd-unit-tests.c +13
-0+#define uffd_test_fail (...) do { \ + printf("failed [reason: "); \ + printf(__VA_ARGS__); \ + printf("]\n"); \ + ksft_inc_fail_cnt(); \ + } while (0) + +#define uffd_test_skip (...) do { \ + printf("skipped [reason: "); \ + printf(__VA_ARGS__); \ + printf("]\n"); \ + ksft_inc_xskip_cnt(); \ + } while (0)Some of this is duplicating what's in
kselftest_harness.h- e.g. assertions, printing test summary, keeping track of pass/fail/skip counts, etc.
Quoted 3 lines
I wonder how you feel about using the standard harness? E.g. is the plan to do this (more straightforward?) refactor first, and then switch later? Or is there some reason it can't be used?
David asked similar question. I wanted to arrange the output mostly for:
(1) one test per line, (2) dump test name before anything (even failing). The 2nd one is important for uffd tests since it uses err() which bails out immediately on unexpected failures (comparing to uffd_test_fail).
To achieve that I found I can use ksft_inc_xskip_cnt() etc. so I kept using the counters in the harness but leave the print format on my own. I assume that's also why these were exported in kselftests.h so when tests want to have customized output it can still use the counters.
Thanks,
Signature · 1 line
Peter Xu
Re: [PATCH v2 16/31] selftests/mm: UFFDIO_API test
In reply to Peter XuQuoted 4 lines · Peter Xu
Add one simple test for
UFFDIO_API. With that, I also added a bunch of small but handy helpers along the way.Signed-off-by Peter Xu <[email protected]>
Quoted 129 lines
--- tools/testing/selftests/mm/uffd-unit-tests.c | 109 ++++++++++++++++++- 1 file changed, 108 insertions(+), 1 deletion(-)tools/testing/selftests/mm/uffd-unit-tests.c +108
-1@@ -9,9 +9,116 @@#ifdef __NR_userfaultfd +static void uffd_test_report (void ) +{ +printf ("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n" , +ksft_get_pass_cnt (), +ksft_get_xskip_cnt (), +ksft_get_fail_cnt (), +ksft_test_num ()); +} + +static void uffd_test_pass (void ) +{ +printf ("done\n" ); +ksft_inc_pass_cnt (); +} + +#define uffd_test_start (...) do { \ + printf("Testing "); \ + printf(__VA_ARGS__); \ + printf("... "); \ + fflush(stdout); \ + } while (0) + +#define uffd_test_fail (...) do { \ + printf("failed [reason: "); \ + printf(__VA_ARGS__); \ + printf("]\n"); \ + ksft_inc_fail_cnt(); \ + } while (0) + +#define uffd_test_skip (...) do { \ + printf("skipped [reason: "); \ + printf(__VA_ARGS__); \ + printf("]\n"); \ + ksft_inc_xskip_cnt(); \ + } while (0) + +/* +* Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll +* return 1 even if some test failed as long as uffd supported, because in +* that case we still want to proceed with the rest uffd unit tests. +*/ +static int test_uffd_api (bool use_dev) +{ +struct uffdio_api uffdio_api; +int uffd; + +uffd_test_start ("UFFDIO_API (with %s)" , + use_dev ?"/dev/userfaultfd" :"syscall" ); + +if (use_dev) + uffd =uffd_open_dev (UFFD_FLAGS ); +else + uffd =uffd_open_sys (UFFD_FLAGS ); +if (uffd <0 ) { +uffd_test_skip ("cannot open userfaultfd handle" ); +return 0 ; + } + +/* Test wrong UFFD_API */ + uffdio_api.api =0xab ; + uffdio_api.features =0 ; +if (ioctl (uffd,UFFDIO_API , &uffdio_api) ==0 ) { +uffd_test_fail ("UFFDIO_API should fail with wrong api but didn't" ); + goto out; + } + +/* Test wrong feature bit */ + uffdio_api.api =UFFD_API ; + uffdio_api.features =BIT_ULL (63 ); +if (ioctl (uffd,UFFDIO_API , &uffdio_api) ==0 ) { +uffd_test_fail ("UFFDIO_API should fail with wrong feature but didn't" ); + goto out; + } + +/* Test normal UFFDIO_API */ + uffdio_api.api =UFFD_API ; + uffdio_api.features =0 ; +if (ioctl (uffd,UFFDIO_API , &uffdio_api)) { +uffd_test_fail ("UFFDIO_API should succeed but failed" ); + goto out; + } + +/* Test double requests of UFFDIO_API with a random feature set */ + uffdio_api.features =BIT_ULL (0 ); +if (ioctl (uffd,UFFDIO_API , &uffdio_api) ==0 ) { +uffd_test_fail ("UFFDIO_API should reject initialized uffd" ); + goto out; + } + +uffd_test_pass (); +out: +close (uffd); +/* We have a valid uffd handle */ +return 1 ; +} +int main (int argc,char *argv[]) { -return KSFT_PASS ; +int has_uffd; + + has_uffd =test_uffd_api (false); + has_uffd |=test_uffd_api (true); + +if (!has_uffd) { +printf ("Userfaultfd not supported or unprivileged, skip all tests\n" ); +exit (KSFT_SKIP ); + } +uffd_test_report (); + +return ksft_get_fail_cnt () ?KSFT_FAIL :KSFT_PASS ; }#else /* __NR_userfaultfd */ Signature · 1 line
2.39.1
Signature · 2 lines
Sincerely yours, Mike.
Peter Xu
reviewerPATCH v2 17/31 selftests/mm: Drop global mem_fd in uffd tests
In reply to Peter XuDrop it by creating the memfd dynamically in the tests.
--- tools/testing/selftests/mm/uffd-common.c | 28 +++++++++++++++++++++++- tools/testing/selftests/mm/uffd-common.h | 2 +- tools/testing/selftests/mm/uffd-stress.c | 15 ------------- 3 files changed, 28 insertions(+), 17 deletions(-)
tools/testing/selftests/mm/uffd-common.c +27 -1
@@ -12,12 +12,32 @@volatile bool test_uffdio_copy_eexist = true;unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap; -int mem_fd, uffd =-1 , uffd_flags, finished, *pipefd, test_type; +int uffd =-1 , uffd_flags, finished, *pipefd, test_type;bool map_shared, test_collapse, test_dev_userfaultfd;bool test_uffdio_wp = true, test_uffdio_minor = false;unsigned long long *count_verify;uffd_test_ops_t *uffd_test_ops; +static int uffd_mem_fd_create (off_t mem_size,bool hugetlb) +{ +unsigned int memfd_flags =0 ; +int mem_fd; + +if (hugetlb) + memfd_flags =MFD_HUGETLB ; + mem_fd =memfd_create ("uffd-test" , memfd_flags); +if (mem_fd <0 ) +err ("memfd_create" ); +if (ftruncate (mem_fd, mem_size)) +err ("ftruncate" ); +if (fallocate (mem_fd, +FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE ,0 , + mem_size)) +err ("fallocate" ); + +return mem_fd; +} +static void anon_release_pages (char *rel_area) {if (madvise (rel_area, nr_pages * page_size,MADV_DONTNEED ))
@@ -51,6 +71,7 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)off_t offset = is_src ?0 : size;void *area_alias =NULL ;char **alloc_area_alias; +int mem_fd =uffd_mem_fd_create (size *2 , true); *alloc_area = mmap(NULL, size, PROT_READ | PROT_WRITE, (map_shared ? MAP_SHARED : MAP_PRIVATE) |
@@ -73,6 +94,8 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src) }if (area_alias) *alloc_area_alias = area_alias; + +close (mem_fd); }static void hugetlb_alias_mapping (__u64 *start,size_t len,unsigned long offset)
@@ -95,6 +118,7 @@ static void shmem_allocate_area(void **alloc_area, bool is_src)size_t bytes = nr_pages * page_size;unsigned long offset = is_src ?0 : bytes;char *p =NULL , *p_alias =NULL ; +int mem_fd =uffd_mem_fd_create (bytes *2 , false);if (test_collapse) { p =BASE_PMD_ADDR ;
@@ -124,6 +148,8 @@ static void shmem_allocate_area(void **alloc_area, bool is_src) area_src_alias = area_alias; else area_dst_alias = area_alias; + +close (mem_fd); }static void shmem_alias_mapping (__u64 *start,size_t len,unsigned long offset)
tools/testing/selftests/mm/uffd-common.h +1 -1
@@ -87,7 +87,7 @@ typedef struct uffd_test_ops uffd_test_ops_t;extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap; -extern int mem_fd, uffd, uffd_flags, finished, *pipefd, test_type; +extern int uffd, uffd_flags, finished, *pipefd, test_type;extern bool map_shared, test_collapse, test_dev_userfaultfd;extern bool test_uffdio_wp, test_uffdio_minor;extern unsigned long long *count_verify;
tools/testing/selftests/mm/uffd-stress.c +0 -15
@@ -1090,21 +1090,6 @@ int main(int argc, char **argv) } nr_pages = nr_pages_per_cpu * nr_cpus; -if (test_type ==TEST_SHMEM || test_type ==TEST_HUGETLB ) { -unsigned int memfd_flags =0 ; - -if (test_type ==TEST_HUGETLB ) - memfd_flags =MFD_HUGETLB ; - mem_fd =memfd_create (argv[0 ], memfd_flags); -if (mem_fd <0 ) -err ("memfd_create" ); -if (ftruncate (mem_fd, nr_pages * page_size *2 )) -err ("ftruncate" ); -if (fallocate (mem_fd, -FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE ,0 , - nr_pages * page_size *2 )) -err ("fallocate" ); - }printf ("nr_pages: %lu, nr_pages_per_cpu: %lu\n" , nr_pages, nr_pages_per_cpu);return userfaultfd_stress ();
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 18/31 selftests/mm: Drop global hpage_size in uffd tests
In reply to Peter Xuhpage_size was wrongly used. Sometimes it means hugetlb default size, sometimes it was used as thp size.
Remove the global variable and use the right one at each place.
--- tools/testing/selftests/mm/uffd-common.c | 7 ++++--- tools/testing/selftests/mm/uffd-common.h | 2 +- tools/testing/selftests/mm/uffd-stress.c | 6 +++--- 3 files changed, 8 insertions(+), 7 deletions(-)
tools/testing/selftests/mm/uffd-common.c +4 -3
@@ -10,7 +10,7 @@#define BASE_PMD_ADDR ((void *)(1UL << 30))volatile bool test_uffdio_copy_eexist = true; -unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size; +unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;int uffd =-1 , uffd_flags, finished, *pipefd, test_type;bool map_shared, test_collapse, test_dev_userfaultfd;
@@ -115,7 +115,7 @@ static void shmem_release_pages(char *rel_area)static void shmem_allocate_area (void **alloc_area,bool is_src) {void *area_alias =NULL ; -size_t bytes = nr_pages * page_size; +size_t bytes = nr_pages * page_size, hpage_size =read_pmd_pagesize ();unsigned long offset = is_src ?0 : bytes;char *p =NULL , *p_alias =NULL ;int mem_fd =uffd_mem_fd_create (bytes *2 , false);
@@ -159,7 +159,8 @@ static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)static void shmem_check_pmd_mapping (void *p,int expect_nr_hpages) { -if (!check_huge_shmem (area_dst_alias, expect_nr_hpages, hpage_size)) +if (!check_huge_shmem (area_dst_alias, expect_nr_hpages, +read_pmd_pagesize ()))err ("Did not find expected %d number of hugepages" , expect_nr_hpages); }
tools/testing/selftests/mm/uffd-common.h +1 -1
@@ -85,7 +85,7 @@ struct uffd_test_ops { };typedef struct uffd_test_ops uffd_test_ops_t ; -extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size; +extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;extern int uffd, uffd_flags, finished, *pipefd, test_type;extern bool map_shared, test_collapse, test_dev_userfaultfd;
tools/testing/selftests/mm/uffd-stress.c +3 -3
@@ -655,7 +655,7 @@ static int userfaultfd_minor_test(void) uffd_test_ops->check_pmd_mapping (area_dst, nr_pages * page_size / - hpage_size); +read_pmd_pagesize ());/* * This won't cause uffd-fault - it purely just makes sure there * was no corruption.
@@ -997,7 +997,7 @@ static void parse_test_type_arg(const char *raw_type)err ("Unsupported test: %s" , raw_type);if (test_type ==TEST_HUGETLB ) - page_size = hpage_size; + page_size =default_huge_page_size ();else page_size =sysconf (_SC_PAGE_SIZE);
@@ -1035,6 +1035,7 @@ static void sigalrm(int sig)int main (int argc,char **argv) {size_t bytes; +size_t hpage_size =read_pmd_pagesize ();if (argc <4 )usage ();
@@ -1043,7 +1044,6 @@ int main(int argc, char **argv)err ("failed to arm SIGALRM" );alarm (ALARM_INTERVAL_SECS ); - hpage_size =default_huge_page_size ();parse_test_type_arg (argv[1 ]); bytes =atol (argv[2 ]) *1024 *1024 ;
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 19/31 selftests/mm: Rename uffd_stats to uffd_args
In reply to Peter XuPrepare for adding more fields into the struct.
--- tools/testing/selftests/mm/uffd-common.c | 28 ++++++------- tools/testing/selftests/mm/uffd-common.h | 6 +-- tools/testing/selftests/mm/uffd-stress.c | 51 ++++++++++++------------ 3 files changed, 42 insertions(+), 43 deletions(-)
tools/testing/selftests/mm/uffd-common.c +14 -14
@@ -186,34 +186,34 @@ struct uffd_test_ops hugetlb_uffd_test_ops = { .check_pmd_mapping = NULL, }; -void uffd_stats_report (struct uffd_stats *stats,int n_cpus) +void uffd_stats_report (struct uffd_args *args,int n_cpus) {int i;unsigned long long miss_total =0 , wp_total =0 , minor_total =0 ;for (i =0 ; i < n_cpus; i++) { - miss_total += stats[i].missing_faults; - wp_total += stats[i].wp_faults; - minor_total += stats[i].minor_faults; + miss_total += args[i].missing_faults; + wp_total += args[i].wp_faults; + minor_total += args[i].minor_faults; }printf ("userfaults: " );if (miss_total) {printf ("%llu missing (" , miss_total);for (i =0 ; i < n_cpus; i++) -printf ("%lu+" , stats[i].missing_faults); +printf ("%lu+" , args[i].missing_faults);printf ("\b) " ); }if (wp_total) {printf ("%llu wp (" , wp_total);for (i =0 ; i < n_cpus; i++) -printf ("%lu+" , stats[i].wp_faults); +printf ("%lu+" , args[i].wp_faults);printf ("\b) " ); }if (minor_total) {printf ("%llu minor (" , minor_total);for (i =0 ; i < n_cpus; i++) -printf ("%lu+" , stats[i].minor_faults); +printf ("%lu+" , args[i].minor_faults);printf ("\b)" ); }printf ("\n" );
@@ -397,7 +397,7 @@ int uffd_read_msg(int ufd, struct uffd_msg *msg)return 0 ; } -void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_stats *stats) +void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_args *args) {unsigned long offset;
@@ -407,7 +407,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)if (msg->arg.pagefault.flags &UFFD_PAGEFAULT_FLAG_WP ) {/* Write protect page faults */ wp_range (uffd, msg->arg.pagefault.address, page_size, false); - stats->wp_faults++; + args->wp_faults++; }else if (msg->arg.pagefault.flags &UFFD_PAGEFAULT_FLAG_MINOR ) {uint8_t *area;int b;
@@ -430,7 +430,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)for (b =0 ; b < page_size; ++b) area[b] = ~area[b];continue_range (uffd, msg->arg.pagefault.address, page_size); - stats->minor_faults++; + args->minor_faults++; }else {/* * Missing page faults.
@@ -460,14 +460,14 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats) offset &= ~(page_size-1 );if (copy_page (uffd, offset)) - stats->missing_faults++; + args->missing_faults++; } }void *uffd_poll_thread (void *arg) { -struct uffd_stats *stats = (struct uffd_stats *)arg; -unsigned long cpu = stats->cpu; +struct uffd_args *args = (struct uffd_args *)arg; +unsigned long cpu = args->cpu;struct pollfd pollfd[2 ];struct uffd_msg msg;struct uffdio_register uffd_reg;
@@ -502,7 +502,7 @@ void *uffd_poll_thread(void *arg)err ("unexpected msg event %u\n" , msg.event);break ;case UFFD_EVENT_PAGEFAULT : -uffd_handle_page_fault (&msg, stats); +uffd_handle_page_fault (&msg, args);break ;case UFFD_EVENT_FORK :close (uffd);
tools/testing/selftests/mm/uffd-common.h +3 -3
@@ -70,7 +70,7 @@ - 1)))/* Userfaultfd test statistics */ -struct uffd_stats { +struct uffd_args {int cpu;unsigned long missing_faults;unsigned long wp_faults;
@@ -98,12 +98,12 @@ extern uffd_test_ops_t shmem_uffd_test_ops;extern uffd_test_ops_t hugetlb_uffd_test_ops;extern uffd_test_ops_t *uffd_test_ops; -void uffd_stats_report (struct uffd_stats *stats,int n_cpus); +void uffd_stats_report (struct uffd_args *args,int n_cpus);void uffd_test_ctx_init (uint64_t features);void userfaultfd_open (uint64_t *features);int uffd_read_msg (int ufd,struct uffd_msg *msg);void wp_range (int ufd,__u64 start,__u64 len,bool wp); -void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_stats *stats); +void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_args *args);int __copy_page (int ufd,unsigned long offset,bool retry);int copy_page (int ufd,unsigned long offset);void *uffd_poll_thread (void *arg);
tools/testing/selftests/mm/uffd-stress.c +25 -26
170 lines not on this page · raw message
Signature · 1 line
2.39.1
Re: [PATCH v2 19/31] selftests/mm: Rename uffd_stats to uffd_args
Quoted 4 lines · Peter Xu
Prepare for adding more fields into the struct.
Suggested-by Mike Rapoport (IBM) <[email protected]>Signed-off-by Peter Xu <[email protected]>
Quoted 329 lines
--- tools/testing/selftests/mm/uffd-common.c | 28 ++++++------- tools/testing/selftests/mm/uffd-common.h | 6 +-- tools/testing/selftests/mm/uffd-stress.c | 51 ++++++++++++------------ 3 files changed, 42 insertions(+), 43 deletions(-)tools/testing/selftests/mm/uffd-common.c +14
-14@@ -186,34 +186,34 @@ struct uffd_test_ops hugetlb_uffd_test_ops = { .check_pmd_mapping = NULL, }; -void uffd_stats_report (struct uffd_stats *stats,int n_cpus) +void uffd_stats_report (struct uffd_args *args,int n_cpus) {int i;unsigned long long miss_total =0 , wp_total =0 , minor_total =0 ;for (i =0 ; i < n_cpus; i++) { - miss_total += stats[i].missing_faults; - wp_total += stats[i].wp_faults; - minor_total += stats[i].minor_faults; + miss_total += args[i].missing_faults; + wp_total += args[i].wp_faults; + minor_total += args[i].minor_faults; }printf ("userfaults: " );if (miss_total) {printf ("%llu missing (" , miss_total);for (i =0 ; i < n_cpus; i++) -printf ("%lu+" , stats[i].missing_faults); +printf ("%lu+" , args[i].missing_faults);printf ("\b) " ); }if (wp_total) {printf ("%llu wp (" , wp_total);for (i =0 ; i < n_cpus; i++) -printf ("%lu+" , stats[i].wp_faults); +printf ("%lu+" , args[i].wp_faults);printf ("\b) " ); }if (minor_total) {printf ("%llu minor (" , minor_total);for (i =0 ; i < n_cpus; i++) -printf ("%lu+" , stats[i].minor_faults); +printf ("%lu+" , args[i].minor_faults);printf ("\b)" ); }printf ("\n" );@@ -397,7 +397,7 @@ int uffd_read_msg(int ufd, struct uffd_msg *msg)return 0 ; } -void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_stats *stats) +void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_args *args) {unsigned long offset;@@ -407,7 +407,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)if (msg->arg.pagefault.flags &UFFD_PAGEFAULT_FLAG_WP ) {/* Write protect page faults */ wp_range (uffd, msg->arg.pagefault.address, page_size, false); - stats->wp_faults++; + args->wp_faults++; }else if (msg->arg.pagefault.flags &UFFD_PAGEFAULT_FLAG_MINOR ) {uint8_t *area;int b;@@ -430,7 +430,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)for (b =0 ; b < page_size; ++b) area[b] = ~area[b];continue_range (uffd, msg->arg.pagefault.address, page_size); - stats->minor_faults++; + args->minor_faults++; }else {/* * Missing page faults. @@ -460,14 +460,14 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats) offset &= ~(page_size-1 );if (copy_page (uffd, offset)) - stats->missing_faults++; + args->missing_faults++; } }void *uffd_poll_thread (void *arg) { -struct uffd_stats *stats = (struct uffd_stats *)arg; -unsigned long cpu = stats->cpu; +struct uffd_args *args = (struct uffd_args *)arg; +unsigned long cpu = args->cpu;struct pollfd pollfd[2 ];struct uffd_msg msg;struct uffdio_register uffd_reg;@@ -502,7 +502,7 @@ void *uffd_poll_thread(void *arg)err ("unexpected msg event %u\n" , msg.event);break ;case UFFD_EVENT_PAGEFAULT : -uffd_handle_page_fault (&msg, stats); +uffd_handle_page_fault (&msg, args);break ;case UFFD_EVENT_FORK :close (uffd);tools/testing/selftests/mm/uffd-common.h +3
-3@@ -70,7 +70,7 @@ - 1)))/* Userfaultfd test statistics */ -struct uffd_stats { +struct uffd_args {int cpu;unsigned long missing_faults;unsigned long wp_faults;@@ -98,12 +98,12 @@ extern uffd_test_ops_t shmem_uffd_test_ops;extern uffd_test_ops_t hugetlb_uffd_test_ops;extern uffd_test_ops_t *uffd_test_ops; -void uffd_stats_report (struct uffd_stats *stats,int n_cpus); +void uffd_stats_report (struct uffd_args *args,int n_cpus);void uffd_test_ctx_init (uint64_t features);void userfaultfd_open (uint64_t *features);int uffd_read_msg (int ufd,struct uffd_msg *msg);void wp_range (int ufd,__u64 start,__u64 len,bool wp); -void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_stats *stats); +void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_args *args);int __copy_page (int ufd,unsigned long offset,bool retry);int copy_page (int ufd,unsigned long offset);void *uffd_poll_thread (void *arg);tools/testing/selftests/mm/uffd-stress.c +25
-26@@ -90,16 +90,15 @@ static void usage(void)exit (1 ); } -static void uffd_stats_reset (struct uffd_stats *uffd_stats, -unsigned long n_cpus) +static void uffd_stats_reset (struct uffd_args *args,unsigned long n_cpus) {int i;for (i =0 ; i < n_cpus; i++) { - uffd_stats[i].cpu = i; - uffd_stats[i].missing_faults =0 ; - uffd_stats[i].wp_faults =0 ; - uffd_stats[i].minor_faults =0 ; + args[i].cpu = i; + args[i].missing_faults =0 ; + args[i].wp_faults =0 ; + args[i].minor_faults =0 ; } }@@ -163,7 +162,7 @@ pthread_mutex_t uffd_read_mutex = PTHREAD_MUTEX_INITIALIZER;static void *uffd_read_thread (void *arg) { -struct uffd_stats *stats = (struct uffd_stats *)arg; +struct uffd_args *args = (struct uffd_args *)arg;struct uffd_msg msg;pthread_mutex_unlock (&uffd_read_mutex);@@ -172,7 +171,7 @@ static void *uffd_read_thread(void *arg)for (;;) {if (uffd_read_msg (uffd, &msg))continue ; -uffd_handle_page_fault (&msg, stats); +uffd_handle_page_fault (&msg, args); }return NULL ;@@ -210,7 +209,7 @@ static void *background_thread(void *arg)return NULL ; } -static int stress (struct uffd_stats *uffd_stats) +static int stress (struct uffd_args *args) {unsigned long cpu;pthread_t locking_threads[nr_cpus];@@ -225,12 +224,12 @@ static int stress(struct uffd_stats *uffd_stats)if (bounces &BOUNCE_POLL ) {if (pthread_create (&uffd_threads[cpu], &attr, uffd_poll_thread, - (void *)&uffd_stats[cpu])) + (void *)&args[cpu]))return 1 ; }else {if (pthread_create (&uffd_threads[cpu], &attr, uffd_read_thread, - (void *)&uffd_stats[cpu])) + (void *)&args[cpu]))return 1 ;pthread_mutex_lock (&uffd_read_mutex); }@@ -264,7 +263,7 @@ static int stress(struct uffd_stats *uffd_stats)if (write (pipefd[cpu*2 +1 ], &c,1 ) !=1 )err ("pipefd write error" );if (pthread_join (uffd_threads[cpu], - (void *)&uffd_stats[cpu])) + (void *)&args[cpu]))return 1 ; }else {if (pthread_cancel (uffd_threads[cpu]))@@ -493,7 +492,7 @@ static int userfaultfd_events_test(void)int err, features;pid_t pid;char c; -struct uffd_stats stats = {0 }; +struct uffd_args args = {0 };printf ("testing events (fork, remap, remove): " );fflush (stdout);@@ -508,7 +507,7 @@ static int userfaultfd_events_test(void) true, test_uffdio_wp, false))err ("register failure" ); -if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &stats)) +if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &args))err ("uffd_poll_thread create" ); pid =fork ();@@ -526,9 +525,9 @@ static int userfaultfd_events_test(void)if (pthread_join (uffd_mon,NULL ))return 1 ; -uffd_stats_report (&stats,1 ); +uffd_stats_report (&args,1 ); -return stats.missing_faults != nr_pages; +return args.missing_faults != nr_pages; }static int userfaultfd_sig_test (void )@@ -538,7 +537,7 @@ static int userfaultfd_sig_test(void)int err, features;pid_t pid;char c; -struct uffd_stats stats = {0 }; +struct uffd_args args = {0 };printf ("testing signal delivery: " );fflush (stdout);@@ -557,7 +556,7 @@ static int userfaultfd_sig_test(void) uffd_test_ops->release_pages (area_dst); -if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &stats)) +if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &args))err ("uffd_poll_thread create" ); pid =fork ();@@ -606,7 +605,7 @@ static int userfaultfd_minor_test(void)unsigned long p;pthread_t uffd_mon;char c; -struct uffd_stats stats = {0 }; +struct uffd_args args = {0 };if (!test_uffdio_minor)return 0 ;@@ -629,7 +628,7 @@ static int userfaultfd_minor_test(void) page_size); } -if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &stats)) +if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &args))err ("uffd_poll_thread create" );/* @@ -645,7 +644,7 @@ static int userfaultfd_minor_test(void)if (pthread_join (uffd_mon,NULL ))return 1 ; -uffd_stats_report (&stats,1 ); +uffd_stats_report (&args,1 );if (test_collapse) {printf ("testing collapse of uffd memory into PMD-mapped THPs:" );@@ -664,7 +663,7 @@ static int userfaultfd_minor_test(void)printf (" done.\n" ); } -return stats.missing_faults !=0 || stats.minor_faults != nr_pages; +return args.missing_faults !=0 || args.minor_faults != nr_pages; }static int pagemap_open (void )@@ -822,7 +821,7 @@ static int userfaultfd_stress(void) {void *area;unsigned long nr; -struct uffd_stats uffd_stats[nr_cpus]; +struct uffd_args args[nr_cpus];uint64_t mem_size = nr_pages * page_size;uffd_test_ctx_init (UFFD_FEATURE_WP_UNPOPULATED );@@ -894,10 +893,10 @@ static int userfaultfd_stress(void)*/ uffd_test_ops->release_pages (area_dst); -uffd_stats_reset (uffd_stats, nr_cpus); +uffd_stats_reset (args, nr_cpus);/* bounce pass */ -if (stress (uffd_stats)) +if (stress (args))return 1 ;/* Clear all the write protections if there is any */ @@ -926,7 +925,7 @@ static int userfaultfd_stress(void)swap (area_src_alias, area_dst_alias); -uffd_stats_report (uffd_stats, nr_cpus); +uffd_stats_report (args, nr_cpus); }if (test_type ==TEST_ANON ) {Signature · 1 line
2.39.1
Signature · 2 lines
Sincerely yours, Mike.
Peter Xu
reviewerPATCH v2 20/31 selftests/mm: Let uffd_handle_page_fault() take wp parameter
In reply to Peter XuMake the handler optionally apply WP bit when resolving page faults for either missing or minor page faults. This moves towards removing global test_uffdio_wp outside of the common code.
--- tools/testing/selftests/mm/uffd-common.c | 17 +++++++++-------- tools/testing/selftests/mm/uffd-common.h | 6 ++++-- tools/testing/selftests/mm/uffd-stress.c | 16 ++++++++++------ 3 files changed, 23 insertions(+), 16 deletions(-)
tools/testing/selftests/mm/uffd-common.c +9 -8
@@ -353,7 +353,7 @@ void wp_range(int ufd, __u64 start, __u64 len, bool wp)err ("clear WP failed: address=0x%" PRIx64, (uint64_t )start); } -static void continue_range (int ufd,__u64 start,__u64 len) +static void continue_range (int ufd,__u64 start,__u64 len,bool wp) {struct uffdio_continue req;int ret;
@@ -361,7 +361,7 @@ static void continue_range(int ufd, __u64 start, __u64 len) req.range.start = start; req.range.len = len; req.mode =0 ; -if (test_uffdio_wp) +if (wp) req.mode |=UFFDIO_CONTINUE_MODE_WP ;if (ioctl (ufd,UFFDIO_CONTINUE , &req))
@@ -429,7 +429,8 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_args *args) area_dst_alias));for (b =0 ; b < page_size; ++b) area[b] = ~area[b]; -continue_range (uffd, msg->arg.pagefault.address, page_size); +continue_range (uffd, msg->arg.pagefault.address, page_size, + args->apply_wp); args->minor_faults++; }else {/*
@@ -459,7 +460,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_args *args) offset = (char *)(unsigned long )msg->arg.pagefault.address - area_dst; offset &= ~(page_size-1 ); -if (copy_page (uffd, offset)) +if (copy_page (uffd, offset, args->apply_wp)) args->missing_faults++; } }
@@ -555,7 +556,7 @@ static void wake_range(int ufd, unsigned long addr, unsigned long len) addr), exit(1); } -int __copy_page (int ufd,unsigned long offset,bool retry) +int __copy_page (int ufd,unsigned long offset,bool retry,bool wp) {struct uffdio_copy uffdio_copy;
@@ -564,7 +565,7 @@ int __copy_page(int ufd, unsigned long offset, bool retry) uffdio_copy.dst = (unsigned long ) area_dst + offset; uffdio_copy.src = (unsigned long ) area_src + offset; uffdio_copy.len = page_size; -if (test_uffdio_wp) +if (wp) uffdio_copy.mode =UFFDIO_COPY_MODE_WP ;else uffdio_copy.mode =0 ;
@@ -587,7 +588,7 @@ int __copy_page(int ufd, unsigned long offset, bool retry)return 0 ; } -int copy_page (int ufd,unsigned long offset) +int copy_page (int ufd,unsigned long offset,bool wp) { -return __copy_page (ufd, offset, false); +return __copy_page (ufd, offset, false, wp); }
tools/testing/selftests/mm/uffd-common.h +4 -2
@@ -72,6 +72,8 @@/* Userfaultfd test statistics */ struct uffd_args {int cpu; +/* Whether apply wr-protects when installing pages */ +bool apply_wp;unsigned long missing_faults;unsigned long wp_faults;unsigned long minor_faults;
@@ -104,8 +106,8 @@ void userfaultfd_open(uint64_t *features);int uffd_read_msg (int ufd,struct uffd_msg *msg);void wp_range (int ufd,__u64 start,__u64 len,bool wp);void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_args *args); -int __copy_page (int ufd,unsigned long offset,bool retry); -int copy_page (int ufd,unsigned long offset); +int __copy_page (int ufd,unsigned long offset,bool retry,bool wp); +int copy_page (int ufd,unsigned long offset,bool wp);void *uffd_poll_thread (void *arg);#define TEST_ANON 1
tools/testing/selftests/mm/uffd-stress.c +10 -6
@@ -96,6 +96,7 @@ static void uffd_stats_reset(struct uffd_args *args, unsigned long n_cpus)for (i =0 ; i < n_cpus; i++) { args[i].cpu = i; + args[i].apply_wp = test_uffdio_wp; args[i].missing_faults =0 ; args[i].wp_faults =0 ; args[i].minor_faults =0 ;
@@ -155,7 +156,7 @@ static void *locking_thread(void *arg)static int copy_page_retry (int ufd,unsigned long offset) { -return __copy_page (ufd, offset, true); +return __copy_page (ufd, offset, true, test_uffdio_wp); }pthread_mutex_t uffd_read_mutex =PTHREAD_MUTEX_INITIALIZER ;
@@ -308,7 +309,7 @@ static void sighndl(int sig, siginfo_t *siginfo, void *ptr)* This also tests UFFD_FEATURE_EVENT_FORK event along with the signal * feature. Using monitor thread, verify no userfault events are generated. */ -static int faulting_process (int signal_test) +static int faulting_process (int signal_test,bool wp) {unsigned long nr;unsigned long long count;
@@ -343,7 +344,7 @@ static int faulting_process(int signal_test)if (steps ==1 ) {/* This is a MISSING request */ steps++; -if (copy_page (uffd, offset)) +if (copy_page (uffd, offset, wp)) signalled++; }else {/* This is a WP request */
@@ -507,6 +508,7 @@ static int userfaultfd_events_test(void) true, test_uffdio_wp, false))err ("register failure" ); + args.apply_wp = test_uffdio_wp;if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &args))err ("uffd_poll_thread create" );
@@ -515,7 +517,7 @@ static int userfaultfd_events_test(void)err ("fork" );if (!pid) -exit (faulting_process (0 )); +exit (faulting_process (0 , test_uffdio_wp));waitpid (pid, &err,0 );if (err)
@@ -551,11 +553,12 @@ static int userfaultfd_sig_test(void) true, test_uffdio_wp, false))err ("register failure" ); -if (faulting_process (1 )) +if (faulting_process (1 , test_uffdio_wp))err ("faulting process failed" ); uffd_test_ops->release_pages (area_dst); + args.apply_wp = test_uffdio_wp;if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &args))err ("uffd_poll_thread create" );
@@ -564,7 +567,7 @@ static int userfaultfd_sig_test(void)err ("fork" );if (!pid) -exit (faulting_process (2 )); +exit (faulting_process (2 , test_uffdio_wp));waitpid (pid, &err,0 );if (err)
@@ -628,6 +631,7 @@ static int userfaultfd_minor_test(void) page_size); } + args.apply_wp = test_uffdio_wp;if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &args))err ("uffd_poll_thread create" );
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 21/31 selftests/mm: Allow allocate_area() to fail properly
In reply to Peter XuMostly to detect hugetlb allocation errors and skip hugetlb tests when pages are not allocated.
--- tools/testing/selftests/mm/uffd-common.c | 47 +++++++++++++++++------- tools/testing/selftests/mm/uffd-common.h | 4 +- 2 files changed, 36 insertions(+), 15 deletions(-)
tools/testing/selftests/mm/uffd-common.c +34 -13
@@ -44,10 +44,15 @@ static void anon_release_pages(char *rel_area)err ("madvise(MADV_DONTNEED) failed" ); } -static void anon_allocate_area (void **alloc_area,bool is_src) +static int anon_allocate_area (void **alloc_area,bool is_src) { *alloc_area =mmap (NULL , nr_pages * page_size,PROT_READ |PROT_WRITE ,MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 ); +if (*alloc_area ==MAP_FAILED ) { + *alloc_area =NULL ; +return -errno; + } +return 0 ; }static void noop_alias_mapping (__u64 *start,size_t len,unsigned long offset)
@@ -65,7 +70,7 @@ static void hugetlb_release_pages(char *rel_area) } } -static void hugetlb_allocate_area (void **alloc_area,bool is_src) +static int hugetlb_allocate_area (void **alloc_area,bool is_src) {off_t size = nr_pages * page_size;off_t offset = is_src ?0 : size;
@@ -77,14 +82,16 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src) (map_shared ?MAP_SHARED :MAP_PRIVATE ) | (is_src ?0 :MAP_NORESERVE ), mem_fd, offset); -if (*alloc_area ==MAP_FAILED ) -err ("mmap of hugetlbfs file failed" ); +if (*alloc_area ==MAP_FAILED ) { + *alloc_area =NULL ; +return -errno; + }if (map_shared) { area_alias =mmap (NULL , size,PROT_READ |PROT_WRITE ,MAP_SHARED , mem_fd, offset);if (area_alias ==MAP_FAILED ) -err ("mmap of hugetlb file alias failed" ); +return -errno; }if (is_src) {
@@ -96,6 +103,7 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src) *alloc_area_alias = area_alias;close (mem_fd); +return 0 ; }static void hugetlb_alias_mapping (__u64 *start,size_t len,unsigned long offset)
@@ -112,7 +120,7 @@ static void shmem_release_pages(char *rel_area)err ("madvise(MADV_REMOVE) failed" ); } -static void shmem_allocate_area (void **alloc_area,bool is_src) +static int shmem_allocate_area (void **alloc_area,bool is_src) {void *area_alias =NULL ;size_t bytes = nr_pages * page_size, hpage_size =read_pmd_pagesize ();
@@ -132,15 +140,20 @@ static void shmem_allocate_area(void **alloc_area, bool is_src) *alloc_area =mmap (p, bytes,PROT_READ |PROT_WRITE ,MAP_SHARED , mem_fd, offset); -if (*alloc_area ==MAP_FAILED ) -err ("mmap of memfd failed" ); +if (*alloc_area ==MAP_FAILED ) { + *alloc_area =NULL ; +return -errno; + }if (test_collapse && *alloc_area != p)err ("mmap of memfd failed at %p" , p); area_alias =mmap (p_alias, bytes,PROT_READ |PROT_WRITE ,MAP_SHARED , mem_fd, offset); -if (area_alias ==MAP_FAILED ) -err ("mmap of memfd alias failed" ); +if (area_alias ==MAP_FAILED ) { +munmap (*alloc_area, bytes); + *alloc_area =NULL ; +return -errno; + }if (test_collapse && area_alias != p_alias)err ("mmap of anonymous memory failed at %p" , p_alias);
@@ -150,6 +163,7 @@ static void shmem_allocate_area(void **alloc_area, bool is_src) area_dst_alias = area_alias;close (mem_fd); +return 0 ; }static void shmem_alias_mapping (__u64 *start,size_t len,unsigned long offset)
@@ -282,14 +296,19 @@ static void uffd_test_ctx_clear(void)munmap_area ((void **)&area_remap); } -void uffd_test_ctx_init (uint64_t features) +int uffd_test_ctx_init (uint64_t features) {unsigned long nr, cpu; +int ret;uffd_test_ctx_clear (); - uffd_test_ops->allocate_area ((void **)&area_src, true); - uffd_test_ops->allocate_area ((void **)&area_dst, false); + ret = uffd_test_ops->allocate_area ((void **)&area_src, true); +if (ret) +return ret; + ret = uffd_test_ops->allocate_area ((void **)&area_dst, false); +if (ret) +return ret;userfaultfd_open (&features);
@@ -337,6 +356,8 @@ void uffd_test_ctx_init(uint64_t features)for (cpu =0 ; cpu < nr_cpus; cpu++)if (pipe2 (&pipefd[cpu *2 ],O_CLOEXEC |O_NONBLOCK ))err ("pipe" ); + +return 0 ; }void wp_range (int ufd,__u64 start,__u64 len,bool wp)
tools/testing/selftests/mm/uffd-common.h +2 -2
@@ -80,7 +80,7 @@ struct uffd_args { };struct uffd_test_ops { -void (*allocate_area)(void **alloc_area,bool is_src); +int (*allocate_area)(void **alloc_area,bool is_src);void (*release_pages)(char *rel_area);void (*alias_mapping)(__u64 *start,size_t len,unsigned long offset);void (*check_pmd_mapping)(void *p,int expect_nr_hpages);
@@ -101,7 +101,7 @@ extern uffd_test_ops_t hugetlb_uffd_test_ops;extern uffd_test_ops_t *uffd_test_ops;void uffd_stats_report (struct uffd_args *args,int n_cpus); -void uffd_test_ctx_init (uint64_t features); +int uffd_test_ctx_init (uint64_t features);void userfaultfd_open (uint64_t *features);int uffd_read_msg (int ufd,struct uffd_msg *msg);void wp_range (int ufd,__u64 start,__u64 len,bool wp);
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 22/31 selftests/mm: Add framework for uffd-unit-test
In reply to Peter XuAdd a framework to be prepared to move unit tests from uffd-stress.c into uffd-unit-tests.c. The goal is to allow detection of uffd features for each test, and also loop over specified types of memory that a test support.
--- tools/testing/selftests/mm/uffd-unit-tests.c | 124 +++++++++++++++++++ tools/testing/selftests/mm/vm_util.c | 37 ++++++ tools/testing/selftests/mm/vm_util.h | 2 + 3 files changed, 163 insertions(+)
tools/testing/selftests/mm/uffd-unit-tests.c +124 -0
@@ -9,6 +9,66 @@#ifdef __NR_userfaultfd +/* The unit test doesn't need a large or random size, make it 32MB for now */ +#define UFFD_TEST_MEM_SIZE (32UL << 20) + +#define MEM_ANON BIT_ULL(0) +#define MEM_SHMEM BIT_ULL(1) +#define MEM_SHMEM_PRIVATE BIT_ULL(2) +#define MEM_HUGETLB BIT_ULL(3) +#define MEM_HUGETLB_PRIVATE BIT_ULL(4) + +struct mem_type { +const char *name; +unsigned int mem_flag; +uffd_test_ops_t *mem_ops; +bool shared; +}; +typedef struct mem_type mem_type_t ; + +mem_type_t mem_types[] = { + { + .name ="anon" , + .mem_flag =MEM_ANON , + .mem_ops = &anon_uffd_test_ops, + .shared = false, + }, + { + .name ="shmem" , + .mem_flag =MEM_SHMEM , + .mem_ops = &shmem_uffd_test_ops, + .shared = true, + }, + { + .name ="shmem-private" , + .mem_flag =MEM_SHMEM_PRIVATE , + .mem_ops = &shmem_uffd_test_ops, + .shared = false, + }, + { + .name ="hugetlb" , + .mem_flag =MEM_HUGETLB , + .mem_ops = &hugetlb_uffd_test_ops, + .shared = true, + }, + { + .name ="hugetlb-private" , + .mem_flag =MEM_HUGETLB_PRIVATE , + .mem_ops = &hugetlb_uffd_test_ops, + .shared = false, + }, +}; + +/* Returns: UFFD_TEST_* */ +typedef void (*uffd_test_fn )(void ); + +typedef struct { +const char *name; +uffd_test_fn uffd_fn; +unsigned int mem_targets; +uint64_t uffd_feature_required; +}uffd_test_case_t ; +static void uffd_test_report (void ) { printf("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n",
@@ -105,9 +165,50 @@ static int test_uffd_api(bool use_dev)return 1 ; } +/* +* This function initializes the global variables. TODO: remove global +* vars and then remove this. +*/ +static int uffd_setup_environment (uffd_test_case_t *test,mem_type_t *mem_type) +{ + map_shared = mem_type->shared; + uffd_test_ops = mem_type->mem_ops; + +if (mem_type->mem_flag & (MEM_HUGETLB_PRIVATE |MEM_HUGETLB )) + page_size =default_huge_page_size (); +else + page_size =psize (); + + nr_pages =UFFD_TEST_MEM_SIZE / page_size; +/* TODO: remove this global var.. it's so ugly */ + nr_cpus =1 ; + +return uffd_test_ctx_init (test->uffd_feature_required); +} + +static bool uffd_feature_supported (uffd_test_case_t *test) +{ +uint64_t features; + +if (uffd_get_features (&features)) +return false; + +return (features & test->uffd_feature_required) == + test->uffd_feature_required; +} + +uffd_test_case_t uffd_tests[] = { +}; +int main (int argc,char *argv[]) { +int n_tests =sizeof (uffd_tests) /sizeof (uffd_test_case_t); +int n_mems =sizeof (mem_types) /sizeof (mem_type_t); +uffd_test_case_t *test; +mem_type_t *mem_type; +char test_name[128 ];int has_uffd; +int i, j; has_uffd =test_uffd_api (false); has_uffd |=test_uffd_api (true);
@@ -116,6 +217,29 @@ int main(int argc, char *argv[])printf ("Userfaultfd not supported or unprivileged, skip all tests\n" );exit (KSFT_SKIP ); } + +for (i =0 ; i < n_tests; i++) { + test = &uffd_tests[i]; +for (j =0 ; j < n_mems; j++) { + mem_type = &mem_types[j]; +if (!(test->mem_targets & mem_type->mem_flag)) +continue ; +snprintf (test_name,sizeof (test_name), +"%s on %s" , test->name, mem_type->name); + +uffd_test_start (test_name); +if (!uffd_feature_supported (test)) { +uffd_test_skip ("feature missing" ); +continue ; + } +if (uffd_setup_environment (test, mem_type)) { +uffd_test_skip ("environment setup failed" ); +continue ; + } + test->uffd_fn (); + } + } +uffd_test_report ();return ksft_get_fail_cnt () ?KSFT_FAIL :KSFT_PASS ;
tools/testing/selftests/mm/vm_util.c +37 -0
@@ -254,3 +254,40 @@ int uffd_open_sys(unsigned int flags)return -1 ;#endif } + +int uffd_open (unsigned int flags) +{ +int uffd =uffd_open_sys (flags); + +if (uffd <0 ) + uffd =uffd_open_dev (flags); + +return uffd; +} + +int uffd_get_features (uint64_t *features) +{ +struct uffdio_api uffdio_api = { .api =UFFD_API , .features =0 }; +/* +* This should by default work in most kernels; the feature list +* will be the same no matter what we pass in here. +*/ +int fd =uffd_open (UFFD_USER_MODE_ONLY ); + +if (fd <0 ) +/* Maybe the kernel is older than user-only mode? */ + fd =uffd_open (0 ); + +if (fd <0 ) +return fd; + +if (ioctl (fd,UFFDIO_API , &uffdio_api)) { +close (fd); +return -errno; + } + + *features = uffdio_api.features; +close (fd); + +return 0 ; +}
tools/testing/selftests/mm/vm_util.h +2 -0
@@ -50,6 +50,8 @@ int uffd_register(int uffd, void *addr, uint64_t len,int uffd_unregister (int uffd,void *addr,uint64_t len);int uffd_open_dev (unsigned int flags);int uffd_open_sys (unsigned int flags); +int uffd_open (unsigned int flags); +int uffd_get_features (uint64_t *features);/* * On ppc64 this will only work with radix 2M hugepage size
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 23/31 selftests/mm: Move uffd pagemap test to unit test
In reply to Peter XuMove it over and make it split into two tests, one for pagemap and one for the new WP_UNPOPULATED (to be a separate one).
The thp pagemap test wasn't really working (with MADV_HUGEPAGE). Let's just drop it (since it never really worked anyway..) and leave that for later.
--- tools/testing/selftests/mm/uffd-stress.c | 166 ------------------- tools/testing/selftests/mm/uffd-unit-tests.c | 145 ++++++++++++++++ 2 files changed, 145 insertions(+), 166 deletions(-)
tools/testing/selftests/mm/uffd-stress.c +0 -166
@@ -670,157 +670,6 @@ static int userfaultfd_minor_test(void)return args.missing_faults !=0 || args.minor_faults != nr_pages; } -static int pagemap_open (void ) -{ -int fd =open ("/proc/self/pagemap" ,O_RDONLY ); - -if (fd <0 ) -err ("open pagemap" ); - -return fd; -} - -/* This macro let __LINE__ works in err() */ -#define pagemap_check_wp (value, wp) do { \ - if (!!(value & PM_UFFD_WP) != wp) \ - err("pagemap uffd-wp bit error: 0x%"PRIx64, value); \ - } while (0) - -static int pagemap_test_fork (bool present) -{ -pid_t child =fork (); -uint64_t value; -int fd, result; - -if (!child) { -/* Open the pagemap fd of the child itself */ - fd =pagemap_open (); - value =pagemap_get_entry (fd, area_dst); -/* -* After fork() uffd-wp bit should be gone as long as we're -* without UFFD_FEATURE_EVENT_FORK -*/ -pagemap_check_wp (value, false); -/* Succeed */ -exit (0 ); - } -waitpid (child, &result,0 ); -return result; -} - -static void userfaultfd_wp_unpopulated_test (int pagemap_fd) -{ -uint64_t value; - -/* Test applying pte marker to anon unpopulated */ -wp_range (uffd, (uint64_t )area_dst, page_size, true); - value =pagemap_get_entry (pagemap_fd, area_dst); -pagemap_check_wp (value, true); - -/* Test unprotect on anon pte marker */ -wp_range (uffd, (uint64_t )area_dst, page_size, false); - value =pagemap_get_entry (pagemap_fd, area_dst); -pagemap_check_wp (value, false); - -/* Test zap on anon marker */ -wp_range (uffd, (uint64_t )area_dst, page_size, true); -if (madvise (area_dst, page_size,MADV_DONTNEED )) -err ("madvise(MADV_DONTNEED) failed" ); - value =pagemap_get_entry (pagemap_fd, area_dst); -pagemap_check_wp (value, false); - -/* Test fault in after marker removed */ - *area_dst =1 ; - value =pagemap_get_entry (pagemap_fd, area_dst); -pagemap_check_wp (value, false); -/* Drop it to make pte none again */ -if (madvise (area_dst, page_size,MADV_DONTNEED )) -err ("madvise(MADV_DONTNEED) failed" ); - -/* Test read-zero-page upon pte marker */ -wp_range (uffd, (uint64_t )area_dst, page_size, true); - *(volatile char *)area_dst; -/* Drop it to make pte none again */ -if (madvise (area_dst, page_size,MADV_DONTNEED )) -err ("madvise(MADV_DONTNEED) failed" ); -} - -static void userfaultfd_pagemap_test (unsigned int test_pgsize) -{ -int pagemap_fd; -uint64_t value; - -/* Pagemap tests uffd-wp only */ -if (!test_uffdio_wp) -return ; - -/* Not enough memory to test this page size */ -if (test_pgsize > nr_pages * page_size) -return ; - -printf ("testing uffd-wp with pagemap (pgsize=%u): " , test_pgsize); -/* Flush so it doesn't flush twice in parent/child later */ -fflush (stdout); - -uffd_test_ctx_init (UFFD_FEATURE_WP_UNPOPULATED ); - -if (test_pgsize > page_size) { -/* This is a thp test */ -if (madvise (area_dst, nr_pages * page_size,MADV_HUGEPAGE )) -err ("madvise(MADV_HUGEPAGE) failed" ); - }else if (test_pgsize == page_size) { -/* This is normal page test; force no thp */ -if (madvise (area_dst, nr_pages * page_size,MADV_NOHUGEPAGE )) -err ("madvise(MADV_NOHUGEPAGE) failed" ); - } - -if (uffd_register (uffd, area_dst, nr_pages * page_size, - false, true, false)) -err ("register failed" ); - - pagemap_fd =pagemap_open (); - -/* Smoke test WP_UNPOPULATED first when it's still empty */ -if (test_pgsize == page_size) -userfaultfd_wp_unpopulated_test (pagemap_fd); - -/* Touch the page */ - *area_dst =1 ; -wp_range (uffd, (uint64_t )area_dst, test_pgsize, true); - value =pagemap_get_entry (pagemap_fd, area_dst); -pagemap_check_wp (value, true); -/* Make sure uffd-wp bit dropped when fork */ -if (pagemap_test_fork (true)) -err ("Detected stall uffd-wp bit in child" ); - -/* Exclusive required or PAGEOUT won't work */ -if (!(value &PM_MMAP_EXCLUSIVE )) -err ("multiple mapping detected: 0x%" PRIx64, value); - -if (madvise (area_dst, test_pgsize,MADV_PAGEOUT )) -err ("madvise(MADV_PAGEOUT) failed" ); - -/* Uffd-wp should persist even swapped out */ - value =pagemap_get_entry (pagemap_fd, area_dst); -pagemap_check_wp (value, true); -/* Make sure uffd-wp bit dropped when fork */ -if (pagemap_test_fork (false)) -err ("Detected stall uffd-wp bit in child" ); - -/* Unprotect; this tests swap pte modifications */ -wp_range (uffd, (uint64_t )area_dst, page_size, false); - value =pagemap_get_entry (pagemap_fd, area_dst); -pagemap_check_wp (value, false); - -/* Fault in the page from disk */ - *area_dst =2 ; - value =pagemap_get_entry (pagemap_fd, area_dst); -pagemap_check_wp (value, false); - -close (pagemap_fd); -printf ("done\n" ); -} -static int userfaultfd_stress (void ) {void *area;
@@ -932,21 +781,6 @@ static int userfaultfd_stress(void)uffd_stats_report (args, nr_cpus); } -if (test_type ==TEST_ANON ) { -/* -* shmem/hugetlb won't be able to run since they have different -* behavior on fork() (file-backed memory normally drops ptes -* directly when fork), meanwhile the pagemap test will verify -* pgtable entry of fork()ed child. -*/ -userfaultfd_pagemap_test (page_size); -/* -* Hard-code for x86_64 for now for 2M THP, as x86_64 is -* currently the only one that supports uffd-wp -*/ -userfaultfd_pagemap_test (page_size *512 ); - } -return userfaultfd_zeropage_test () ||userfaultfd_sig_test () ||userfaultfd_events_test () ||userfaultfd_minor_test (); }
tools/testing/selftests/mm/uffd-unit-tests.c +145 -0
152 lines not on this page · raw message
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 24/31 selftests/mm: Move uffd minor test to unit test
In reply to Peter XuThis moves the minor test to the new unit test.
Rewrite the content check with char* opeartions to avoid fiddling with my_bcmp().
Drop global vars test_uffdio_minor and test_collapse, just assume test them always in common code for now.
OTOH make this single test into five tests:
- minor test on [shmem, hugetlb] with wp=false
- minor test on [shmem, hugetlb] with wp=true
- minor test + collapse on shmem only
One thing to mention that we used to test COLLAPSE+WP but that doesn't sound right at all. It's possible it's silently broken but unnoticed because COLLAPSE is not part of the default test suite.
Make the MADV_COLLAPSE test fail-able (by skip it when failing), because it's not guaranteed to success anyway.
Drop a bunch of useless code after the move, because the unit test always use aligned num of pages and has nothing to do with n_cpus.
--- tools/testing/selftests/mm/uffd-common.c | 25 ++-- tools/testing/selftests/mm/uffd-common.h | 4 +- tools/testing/selftests/mm/uffd-stress.c | 131 +------------------ tools/testing/selftests/mm/uffd-unit-tests.c | 120 +++++++++++++++++ 4 files changed, 135 insertions(+), 145 deletions(-)
tools/testing/selftests/mm/uffd-common.c +12 -13
@@ -13,8 +13,8 @@ volatile bool test_uffdio_copy_eexist = true;unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;int uffd =-1 , uffd_flags, finished, *pipefd, test_type; -bool map_shared, test_collapse, test_dev_userfaultfd; -bool test_uffdio_wp = true, test_uffdio_minor = false; +bool map_shared, test_dev_userfaultfd; +bool test_uffdio_wp = true;unsigned long long *count_verify;uffd_test_ops_t *uffd_test_ops;
@@ -128,15 +128,14 @@ static int shmem_allocate_area(void **alloc_area, bool is_src)char *p =NULL , *p_alias =NULL ;int mem_fd =uffd_mem_fd_create (bytes *2 , false); -if (test_collapse) { - p =BASE_PMD_ADDR ; -if (!is_src) -/* src map + alias + interleaved hpages */ - p +=2 * (bytes + hpage_size); - p_alias = p; - p_alias += bytes; - p_alias += hpage_size;/* Prevent src/dst VMA merge */ - } +/* TODO: clean this up. Use a static addr is ugly */ + p =BASE_PMD_ADDR ; +if (!is_src) +/* src map + alias + interleaved hpages */ + p +=2 * (bytes + hpage_size); + p_alias = p; + p_alias += bytes; + p_alias += hpage_size;/* Prevent src/dst VMA merge */ *alloc_area =mmap (p, bytes,PROT_READ |PROT_WRITE ,MAP_SHARED , mem_fd, offset);
@@ -144,7 +143,7 @@ static int shmem_allocate_area(void **alloc_area, bool is_src) *alloc_area =NULL ;return -errno; } -if (test_collapse && *alloc_area != p) +if (*alloc_area != p)err ("mmap of memfd failed at %p" , p); area_alias = mmap(p_alias, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
@@ -154,7 +153,7 @@ static int shmem_allocate_area(void **alloc_area, bool is_src) *alloc_area =NULL ;return -errno; } -if (test_collapse && area_alias != p_alias) +if (area_alias != p_alias)err ("mmap of anonymous memory failed at %p" , p_alias);if (is_src)
tools/testing/selftests/mm/uffd-common.h +2 -2
@@ -90,8 +90,8 @@ typedef struct uffd_test_ops uffd_test_ops_t;extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;extern int uffd, uffd_flags, finished, *pipefd, test_type; -extern bool map_shared, test_collapse, test_dev_userfaultfd; -extern bool test_uffdio_wp, test_uffdio_minor; +extern bool map_shared, test_dev_userfaultfd; +extern bool test_uffdio_wp;extern unsigned long long *count_verify;extern volatile bool test_uffdio_copy_eexist;
tools/testing/selftests/mm/uffd-stress.c +1 -130
197 lines not on this page · raw message
tools/testing/selftests/mm/uffd-unit-tests.c +120 -0
132 lines not on this page · raw message
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 25/31 selftests/mm: Move uffd sig/events tests into uffd unit tests
In reply to Peter XuMove the two tests into the unit test, and convert it into 20 standalone tests:
- events test on all 5 mem types, with wp on/off
- signal test on all 5 mem types, with wp on/off
Testing sigbus on anon... done
Testing sigbus on shmem... done
Testing sigbus on shmem-private... done
Testing sigbus on hugetlb... done
Testing sigbus on hugetlb-private... done
Testing sigbus-wp on anon... done
Testing sigbus-wp on shmem... done
Testing sigbus-wp on shmem-private... done
Testing sigbus-wp on hugetlb... done
Testing sigbus-wp on hugetlb-private... done
Testing events on anon... done
Testing events on shmem... done
Testing events on shmem-private... done
Testing events on hugetlb... done
Testing events on hugetlb-private... done
Testing events-wp on anon... done
Testing events-wp on shmem... done
Testing events-wp on shmem-private... done
Testing events-wp on hugetlb... done
Testing events-wp on hugetlb-private... done
It'll also remove a lot of global references along the way, e.g. test_uffdio_wp will be replaced with the wp value passed over.
--- tools/testing/selftests/mm/uffd-stress.c | 227 +--------------- tools/testing/selftests/mm/uffd-unit-tests.c | 264 +++++++++++++++++++ 2 files changed, 265 insertions(+), 226 deletions(-)
tools/testing/selftests/mm/uffd-stress.c +1 -226
@@ -273,133 +273,6 @@ static int stress(struct uffd_args *args)return 0 ; } -sigjmp_buf jbuf, *sigbuf; - -static void sighndl (int sig,siginfo_t *siginfo,void *ptr) -{ -if (sig ==SIGBUS ) { -if (sigbuf) -siglongjmp (*sigbuf,1 ); -abort (); - } -} - -/* -* For non-cooperative userfaultfd test we fork() a process that will -* generate pagefaults, will mremap the area monitored by the -* userfaultfd and at last this process will release the monitored -* area. -* For the anonymous and shared memory the area is divided into two -* parts, the first part is accessed before mremap, and the second -* part is accessed after mremap. Since hugetlbfs does not support -* mremap, the entire monitored area is accessed in a single pass for -* HUGETLB_TEST. -* The release of the pages currently generates event for shmem and -* anonymous memory (UFFD_EVENT_REMOVE), hence it is not checked -* for hugetlb. -* For signal test(UFFD_FEATURE_SIGBUS), signal_test = 1, we register -* monitored area, generate pagefaults and test that signal is delivered. -* Use UFFDIO_COPY to allocate missing page and retry. For signal_test = 2 -* test robustness use case - we release monitored area, fork a process -* that will generate pagefaults and verify signal is generated. -* This also tests UFFD_FEATURE_EVENT_FORK event along with the signal -* feature. Using monitor thread, verify no userfault events are generated. -*/ -static int faulting_process (int signal_test,bool wp) -{ -unsigned long nr; -unsigned long long count; -unsigned long split_nr_pages; -unsigned long lastnr; -struct sigaction act; -volatile unsigned long signalled =0 ; - - split_nr_pages = (nr_pages +1 ) /2 ; - -if (signal_test) { - sigbuf = &jbuf; -memset (&act,0 ,sizeof (act)); - act.sa_sigaction = sighndl; - act.sa_flags =SA_SIGINFO ; -if (sigaction (SIGBUS , &act,0 )) -err ("sigaction" ); - lastnr = (unsigned long )-1 ; - } - -for (nr =0 ; nr < split_nr_pages; nr++) { -volatile int steps =1 ; -unsigned long offset = nr * page_size; - -if (signal_test) { -if (sigsetjmp (*sigbuf,1 ) !=0 ) { -if (steps ==1 && nr == lastnr) -err ("Signal repeated" ); - - lastnr = nr; -if (signal_test ==1 ) { -if (steps ==1 ) { -/* This is a MISSING request */ - steps++; -if (copy_page (uffd, offset, wp)) - signalled++; - }else { -/* This is a WP request */ -assert (steps ==2 ); -wp_range (uffd, - (__u64 )area_dst + - offset, - page_size, false); - } - }else { - signalled++; -continue ; - } - } - } - - count = *area_count (area_dst, nr); -if (count != count_verify[nr]) -err ("nr %lu memory corruption %llu %llu\n" , - nr, count, count_verify[nr]); -/* -* Trigger write protection if there is by writing -* the same value back. -*/ - *area_count (area_dst, nr) = count; - } - -if (signal_test) -return signalled != split_nr_pages; - - area_dst =mremap (area_dst, nr_pages * page_size, nr_pages * page_size, -MREMAP_MAYMOVE |MREMAP_FIXED , area_src); -if (area_dst ==MAP_FAILED ) -err ("mremap" ); -/* Reset area_src since we just clobbered it */ - area_src =NULL ; - -for (; nr < nr_pages; nr++) { - count = *area_count (area_dst, nr); -if (count != count_verify[nr]) { -err ("nr %lu memory corruption %llu %llu\n" , - nr, count, count_verify[nr]); - } -/* -* Trigger write protection if there is by writing -* the same value back. -*/ - *area_count (area_dst, nr) = count; - } - - uffd_test_ops->release_pages (area_dst); - -for (nr =0 ; nr < nr_pages; nr++) -if (my_bcmp (area_dst + nr * page_size, zeropage, page_size)) -err ("nr %lu is not zero" , nr); - -return 0 ; -} -static void retry_uffdio_zeropage (int ufd,struct uffdio_zeropage *uffdio_zeropage,unsigned long offset)
@@ -483,103 +356,6 @@ static int userfaultfd_zeropage_test(void)return 0 ; } -static int userfaultfd_events_test (void ) -{ -pthread_t uffd_mon; -int err, features; -pid_t pid; -char c; -struct uffd_args args = {0 }; - -printf ("testing events (fork, remap, remove): " ); -fflush (stdout); - - features =UFFD_FEATURE_EVENT_FORK |UFFD_FEATURE_EVENT_REMAP | -UFFD_FEATURE_EVENT_REMOVE ; -uffd_test_ctx_init (features); - -fcntl (uffd,F_SETFL , uffd_flags |O_NONBLOCK ); - -if (uffd_register (uffd, area_dst, nr_pages * page_size, - true, test_uffdio_wp, false)) -err ("register failure" ); - - args.apply_wp = test_uffdio_wp; -if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &args)) -err ("uffd_poll_thread create" ); - - pid =fork (); -if (pid <0 ) -err ("fork" ); - -if (!pid) -exit (faulting_process (0 , test_uffdio_wp)); - -waitpid (pid, &err,0 ); -if (err) -err ("faulting process failed" ); -if (write (pipefd[1 ], &c,sizeof (c)) !=sizeof (c)) -err ("pipe write" ); -if (pthread_join (uffd_mon,NULL )) -return 1 ; - -uffd_stats_report (&args,1 ); - -return args.missing_faults != nr_pages; -} - -static int userfaultfd_sig_test (void ) -{ -unsigned long userfaults; -pthread_t uffd_mon; -int err, features; -pid_t pid; -char c; -struct uffd_args args = {0 }; - -printf ("testing signal delivery: " ); -fflush (stdout); - - features =UFFD_FEATURE_EVENT_FORK |UFFD_FEATURE_SIGBUS ; -uffd_test_ctx_init (features); - -fcntl (uffd,F_SETFL , uffd_flags |O_NONBLOCK ); - -if (uffd_register (uffd, area_dst, nr_pages * page_size, - true, test_uffdio_wp, false)) -err ("register failure" ); - -if (faulting_process (1 , test_uffdio_wp)) -err ("faulting process failed" ); - - uffd_test_ops->release_pages (area_dst); - - args.apply_wp = test_uffdio_wp; -if (pthread_create (&uffd_mon, &attr, uffd_poll_thread, &args)) -err ("uffd_poll_thread create" ); - - pid =fork (); -if (pid <0 ) -err ("fork" ); - -if (!pid) -exit (faulting_process (2 , test_uffdio_wp)); - -waitpid (pid, &err,0 ); -if (err) -err ("faulting process failed" ); -if (write (pipefd[1 ], &c,sizeof (c)) !=sizeof (c)) -err ("pipe write" ); -if (pthread_join (uffd_mon, (void **)&userfaults)) -return 1 ; - -printf ("done.\n" ); -if (userfaults) -err ("Signal test failed, userfaults: %ld" , userfaults); - -return userfaults !=0 ; -} -static int userfaultfd_stress (void ) {void *area;
@@ -691,8 +467,7 @@ static int userfaultfd_stress(void)uffd_stats_report (args, nr_cpus); } -return userfaultfd_zeropage_test () ||userfaultfd_sig_test () - ||userfaultfd_events_test (); +return userfaultfd_zeropage_test (); }static void set_test_type (const char *type)
tools/testing/selftests/mm/uffd-unit-tests.c +264 -0
282 lines not on this page · raw message
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 26/31 selftests/mm: Move zeropage test into uffd unit tests
In reply to Peter XuSimplifies it a bit along the way, e.g., drop the never used offset field (which was always the 1st page so offset=0).
Introduce uffd_register_with_ioctls() out of uffd_register() to detect uffdio_register.ioctls got returned. Check that automatically when testing UFFDIO_ZEROPAGE on different types of memory (and kernel).
--- tools/testing/selftests/mm/uffd-stress.c | 94 +------------------- tools/testing/selftests/mm/uffd-unit-tests.c | 93 +++++++++++++++++++ tools/testing/selftests/mm/vm_util.c | 14 ++- tools/testing/selftests/mm/vm_util.h | 2 + 4 files changed, 108 insertions(+), 95 deletions(-)
tools/testing/selftests/mm/uffd-stress.c +1 -93
@@ -109,15 +109,6 @@ static inline uint64_t uffd_minor_feature(void)return 0 ; } -static int my_bcmp (char *str1,char *str2,size_t n) -{ -unsigned long i; -for (i =0 ; i < n; i++) -if (str1[i] != str2[i]) -return 1 ; -return 0 ; -} -static void *locking_thread (void *arg) {unsigned long cpu = (unsigned long ) arg;
@@ -273,89 +264,6 @@ static int stress(struct uffd_args *args)return 0 ; } -static void retry_uffdio_zeropage (int ufd, -struct uffdio_zeropage *uffdio_zeropage, -unsigned long offset) -{ - uffd_test_ops->alias_mapping (&uffdio_zeropage->range.start, - uffdio_zeropage->range.len, - offset); -if (ioctl (ufd,UFFDIO_ZEROPAGE , uffdio_zeropage)) { -if (uffdio_zeropage->zeropage != -EEXIST ) -err ("UFFDIO_ZEROPAGE error: %" PRId64, - (int64_t )uffdio_zeropage->zeropage); - }else { -err ("UFFDIO_ZEROPAGE error: %" PRId64, - (int64_t )uffdio_zeropage->zeropage); - } -} - -static int __uffdio_zeropage (int ufd,unsigned long offset) -{ -struct uffdio_zeropage uffdio_zeropage; -int ret; -bool has_zeropage = !(test_type ==TEST_HUGETLB ); -__s64 res; - -if (offset >= nr_pages * page_size) -err ("unexpected offset %lu" , offset); - uffdio_zeropage.range.start = (unsigned long ) area_dst + offset; - uffdio_zeropage.range.len = page_size; - uffdio_zeropage.mode =0 ; - ret =ioctl (ufd,UFFDIO_ZEROPAGE , &uffdio_zeropage); - res = uffdio_zeropage.zeropage; -if (ret) { -/* real retval in ufdio_zeropage.zeropage */ -if (has_zeropage) -err ("UFFDIO_ZEROPAGE error: %" PRId64, (int64_t )res); -else if (res != -EINVAL ) -err ("UFFDIO_ZEROPAGE not -EINVAL" ); - }else if (has_zeropage) { -if (res != page_size) { -err ("UFFDIO_ZEROPAGE unexpected size" ); - }else { -retry_uffdio_zeropage (ufd, &uffdio_zeropage, - offset); -return 1 ; - } - }else -err ("UFFDIO_ZEROPAGE succeeded" ); - -return 0 ; -} - -static int uffdio_zeropage (int ufd,unsigned long offset) -{ -return __uffdio_zeropage (ufd, offset); -} - -/* exercise UFFDIO_ZEROPAGE */ -static int userfaultfd_zeropage_test (void ) -{ -printf ("testing UFFDIO_ZEROPAGE: " ); -fflush (stdout); - -uffd_test_ctx_init (0 ); - -if (uffd_register (uffd, area_dst, nr_pages * page_size, - true, test_uffdio_wp, false)) -err ("register failure" ); - -if (area_dst_alias) { -/* Needed this to test zeropage-retry on shared memory */ -if (uffd_register (uffd, area_dst_alias, nr_pages * page_size, - true, test_uffdio_wp, false)) -err ("register failure" ); - } - -if (uffdio_zeropage (uffd,0 )) -if (my_bcmp (area_dst, zeropage, page_size)) -err ("zeropage is not zero" ); - -printf ("done.\n" ); -return 0 ; -} -static int userfaultfd_stress (void ) {void *area;
@@ -467,7 +375,7 @@ static int userfaultfd_stress(void)uffd_stats_report (args, nr_cpus); } -return userfaultfd_zeropage_test (); +return 0 ; }static void set_test_type (const char *type)
tools/testing/selftests/mm/uffd-unit-tests.c +93 -0
100 lines not on this page · raw message
tools/testing/selftests/mm/vm_util.c +12 -2
@@ -198,8 +198,9 @@ unsigned long default_huge_page_size(void)return hps; } -int uffd_register (int uffd,void *addr,uint64_t len, -bool miss,bool wp,bool minor) +/* If `ioctls' non-NULL, the allowed ioctls will be returned into the var */ +int uffd_register_with_ioctls (int uffd,void *addr,uint64_t len, +bool miss,bool wp,bool minor,uint64_t *ioctls) {struct uffdio_register uffdio_register = {0 };uint64_t mode =0 ;
@@ -218,10 +219,19 @@ int uffd_register(int uffd, void *addr, uint64_t len,if (ioctl (uffd,UFFDIO_REGISTER , &uffdio_register) ==-1 ) ret = -errno; +else if (ioctls) + *ioctls = uffdio_register.ioctls;return ret; } +int uffd_register (int uffd,void *addr,uint64_t len, +bool miss,bool wp,bool minor) +{ +return uffd_register_with_ioctls (uffd, addr, len, + miss, wp, minor,NULL ); +} +int uffd_unregister (int uffd,void *addr,uint64_t len) {struct uffdio_range range = { .start = (uintptr_t )addr, .len = len };
tools/testing/selftests/mm/vm_util.h +2 -0
@@ -52,6 +52,8 @@ int uffd_open_dev(unsigned int flags);int uffd_open_sys (unsigned int flags);int uffd_open (unsigned int flags);int uffd_get_features (uint64_t *features); +int uffd_register_with_ioctls (int uffd,void *addr,uint64_t len, +bool miss,bool wp,bool minor,uint64_t *ioctls);/* * On ppc64 this will only work with radix 2M hugepage size
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 27/31 selftests/mm: Workaround no way to detect uffd-minor + wp
In reply to Peter XuUserfaultfd minor+wp mode was very recently added. The test will fail on the old kernels at ioctl(UFFDIO_CONTINUE) which is misterious. Unfortunately there's no feature bit to detect for this support.
Add a hack to leverage WP_UNPOPULATED to detect whether that feature existed, since WP_UNPOPULATED was merged right after minor+wp.
--- tools/testing/selftests/mm/uffd-unit-tests.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
tools/testing/selftests/mm/uffd-unit-tests.c +7 -1
@@ -780,7 +780,13 @@ uffd_test_case_t uffd_tests[] = { .mem_targets = MEM_SHMEM | MEM_HUGETLB, .uffd_feature_required =UFFD_FEATURE_MINOR_HUGETLBFS |UFFD_FEATURE_MINOR_SHMEM | - UFFD_FEATURE_PAGEFAULT_FLAG_WP, +UFFD_FEATURE_PAGEFAULT_FLAG_WP | +/* +* HACK: here we leveraged WP_UNPOPULATED to detect whether +* minor mode supports wr-protect. There's no feature flag +* for it so this is the best we can test against. +*/ + UFFD_FEATURE_WP_UNPOPULATED, }, { .name = "minor-collapse",
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 28/31 selftests/mm: Allow uffd test to skip properly with no privilege
In reply to Peter XuAllow skip a unit test properly due to no privilege (e.g. sigbus and events tests).
--- tools/testing/selftests/mm/uffd-common.c | 27 ++++++++++++-------- tools/testing/selftests/mm/uffd-common.h | 4 +-- tools/testing/selftests/mm/uffd-stress.c | 6 +++-- tools/testing/selftests/mm/uffd-unit-tests.c | 10 +++++--- 4 files changed, 29 insertions(+), 18 deletions(-)
tools/testing/selftests/mm/uffd-common.c +17 -10
@@ -232,7 +232,7 @@ void uffd_stats_report(struct uffd_args *args, int n_cpus)printf ("\n" ); } -void userfaultfd_open (uint64_t *features) +int userfaultfd_open (uint64_t *features) {struct uffdio_api uffdio_api;
@@ -241,18 +241,19 @@ void userfaultfd_open(uint64_t *features)else uffd =uffd_open_sys (UFFD_FLAGS );if (uffd <0 ) -err ("uffd open failed (dev=%d)" , test_dev_userfaultfd); +return -1 ; uffd_flags =fcntl (uffd,F_GETFD ,NULL ); uffdio_api.api =UFFD_API ; uffdio_api.features = *features;if (ioctl (uffd,UFFDIO_API , &uffdio_api)) -err ("UFFDIO_API failed.\nPlease make sure to " -"run with either root or ptrace capability." ); +/* Probably lack of CAP_PTRACE? */ +return -1 ;if (uffdio_api.api !=UFFD_API )err ("UFFDIO_API error: %" PRIu64, (uint64_t )uffdio_api.api); *features = uffdio_api.features; +return 0 ; }static inline void munmap_area (void **area)
@@ -295,7 +296,7 @@ static void uffd_test_ctx_clear(void)munmap_area ((void **)&area_remap); } -int uffd_test_ctx_init (uint64_t features) +int uffd_test_ctx_init (uint64_t features,const char **errmsg) {unsigned long nr, cpu;int ret;
@@ -303,13 +304,19 @@ int uffd_test_ctx_init(uint64_t features)uffd_test_ctx_clear (); ret = uffd_test_ops->allocate_area ((void **)&area_src, true); -if (ret) -return ret; - ret = uffd_test_ops->allocate_area ((void **)&area_dst, false); -if (ret) + ret |= uffd_test_ops->allocate_area ((void **)&area_dst, false); +if (ret) { +if (errmsg) + *errmsg ="memory allocation failed" ;return ret; + } -userfaultfd_open (&features); + ret =userfaultfd_open (&features); +if (ret) { +if (errmsg) + *errmsg ="possible lack of priviledge" ; +return ret; + } count_verify =malloc (nr_pages *sizeof (unsigned long long ));if (!count_verify)
tools/testing/selftests/mm/uffd-common.h +2 -2
@@ -101,8 +101,8 @@ extern uffd_test_ops_t hugetlb_uffd_test_ops;extern uffd_test_ops_t *uffd_test_ops;void uffd_stats_report (struct uffd_args *args,int n_cpus); -int uffd_test_ctx_init (uint64_t features); -void userfaultfd_open (uint64_t *features); +int uffd_test_ctx_init (uint64_t features,const char **errmsg); +int userfaultfd_open (uint64_t *features);int uffd_read_msg (int ufd,struct uffd_msg *msg);void wp_range (int ufd,__u64 start,__u64 len,bool wp);void uffd_handle_page_fault (struct uffd_msg *msg,struct uffd_args *args);
tools/testing/selftests/mm/uffd-stress.c +4 -2
@@ -271,7 +271,8 @@ static int userfaultfd_stress(void)struct uffd_args args[nr_cpus];uint64_t mem_size = nr_pages * page_size; -uffd_test_ctx_init (UFFD_FEATURE_WP_UNPOPULATED ); +if (uffd_test_ctx_init (UFFD_FEATURE_WP_UNPOPULATED ,NULL )) +err ("context init failed" );if (posix_memalign (&area, page_size, page_size))err ("out of memory" );
@@ -435,7 +436,8 @@ static void parse_test_type_arg(const char *raw_type)* feature. */ -userfaultfd_open (&features); +if (userfaultfd_open (&features)) +err ("Userfaultfd open failed" ); test_uffdio_wp = test_uffdio_wp && (features &UFFD_FEATURE_PAGEFAULT_FLAG_WP );
tools/testing/selftests/mm/uffd-unit-tests.c +6 -4
@@ -172,7 +172,8 @@ static int test_uffd_api(bool use_dev)* This function initializes the global variables. TODO: remove global * vars and then remove this. */ -static int uffd_setup_environment (uffd_test_case_t *test,mem_type_t *mem_type) +static int uffd_setup_environment (uffd_test_case_t *test,mem_type_t *mem_type, +const char **errmsg) { map_shared = mem_type->shared; uffd_test_ops = mem_type->mem_ops;
@@ -186,7 +187,7 @@ static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type)/* TODO: remove this global var.. it's so ugly */ nr_cpus =1 ; -return uffd_test_ctx_init (test->uffd_feature_required); +return uffd_test_ctx_init (test->uffd_feature_required, errmsg); }static bool uffd_feature_supported (uffd_test_case_t *test)
@@ -835,6 +836,7 @@ int main(int argc, char *argv[])uffd_test_case_t *test;mem_type_t *mem_type;char test_name[128 ]; +const char *errmsg;int has_uffd;int i, j;
@@ -860,8 +862,8 @@ int main(int argc, char *argv[])uffd_test_skip ("feature missing" );continue ; } -if (uffd_setup_environment (test, mem_type)) { -uffd_test_skip ("environment setup failed" ); +if (uffd_setup_environment (test, mem_type, &errmsg)) { +uffd_test_skip (errmsg);continue ; } test->uffd_fn ();
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 29/31 selftests/mm: Drop sys/dev test in uffd-stress test
In reply to Peter XuWith the new uffd unit test covering the /dev/userfaultfd path and syscall path of uffd initializations, we can safely drop the devnode test in the old stress test.
One thing is to avoid duplication of running the stress test twice which is an overkill to only test the /dev/ interface in run_vmtests.sh.
The other benefit is now all uffd tests (that uses userfaultfd_open) can run automatically as long as any type of interface is enabled (either syscall or dev), so it's more likely to succeed rather than fail due to unprivilege.
With this patch lands, we can drop all the "mem_type:XXX" handlings too.
--- tools/testing/selftests/mm/run_vmtests.sh | 15 +++++-------- tools/testing/selftests/mm/uffd-common.c | 7 ++---- tools/testing/selftests/mm/uffd-common.h | 2 +- tools/testing/selftests/mm/uffd-stress.c | 27 ++--------------------- 4 files changed, 11 insertions(+), 40 deletions(-)
tools/testing/selftests/mm/run_vmtests.sh +6 -9
@@ -197,16 +197,13 @@ CATEGORY="gup_test" run_test ./gup_test -a CATEGORY="gup_test" run_test ./gup_test -ct -F 0x1 0 19 0x1000 CATEGORY="userfaultfd" run_test ./uffd-unit-tests -uffd_mods=("" ":dev") uffd_stress_bin=./uffd-stress -for mod in "${uffd_mods[@]}"; do - CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon${mod} 20 16 - # Hugetlb tests require source and destination huge pages. Pass in half - # the size ($half_ufd_size_MB), which is used for *each*. - CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb${mod} "$half_ufd_size_MB" 32 - CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared${mod} "$half_ufd_size_MB" 32 - CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem${mod} 20 16 -done +CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon 20 16 +# Hugetlb tests require source and destination huge pages. Pass in half +# the size ($half_ufd_size_MB), which is used for *each*. +CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb "$half_ufd_size_MB" 32 +CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared "$half_ufd_size_MB" 32 +CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem 20 16 #cleanup echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages
tools/testing/selftests/mm/uffd-common.c +2 -5
@@ -13,7 +13,7 @@ volatile bool test_uffdio_copy_eexist = true;unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;int uffd =-1 , uffd_flags, finished, *pipefd, test_type; -bool map_shared, test_dev_userfaultfd; +bool map_shared;bool test_uffdio_wp = true;unsigned long long *count_verify;uffd_test_ops_t *uffd_test_ops;
@@ -236,10 +236,7 @@ int userfaultfd_open(uint64_t *features) {struct uffdio_api uffdio_api; -if (test_dev_userfaultfd) - uffd =uffd_open_dev (UFFD_FLAGS ); -else - uffd =uffd_open_sys (UFFD_FLAGS ); + uffd =uffd_open (UFFD_FLAGS );if (uffd <0 )return -1 ; uffd_flags =fcntl (uffd,F_GETFD ,NULL );
tools/testing/selftests/mm/uffd-common.h +1 -1
@@ -90,7 +90,7 @@ typedef struct uffd_test_ops uffd_test_ops_t;extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;extern int uffd, uffd_flags, finished, *pipefd, test_type; -extern bool map_shared, test_dev_userfaultfd; +extern bool map_shared;extern bool test_uffdio_wp;extern unsigned long long *count_verify;extern volatile bool test_uffdio_copy_eexist;
tools/testing/selftests/mm/uffd-stress.c +2 -25
@@ -55,8 +55,6 @@ pthread_attr_t attr;const char *examples ="# Run anonymous memory test on 100MiB region with 99999 bounces:\n" "./userfaultfd anon 100 99999\n\n" -"# Run the same anonymous memory test, but using /dev/userfaultfd:\n" -"./userfaultfd anon:dev 100 99999\n\n" "# Run share memory test on 1GiB region with 99 bounces:\n" "./userfaultfd shmem 1000 99\n\n" "# Run hugetlb memory test on 256MiB region with 50 bounces:\n"
@@ -69,18 +67,9 @@ const char *examples =static void usage (void ) { -fprintf (stderr,"\nUsage: ./userfaultfd <test type> <MiB> <bounces> " -"[hugetlbfs_file]\n\n" ); +fprintf (stderr,"\nUsage: ./userfaultfd <test type> <MiB> <bounces>\n\n" );fprintf (stderr,"Supported <test type>: anon, hugetlb, " "hugetlb_shared, shmem\n\n" ); -fprintf (stderr,"'Test mods' can be joined to the test type string with a ':'. " -"Supported mods:\n" ); -fprintf (stderr,"\tsyscall - Use userfaultfd(2) (default)\n" ); -fprintf (stderr,"\tdev - Use /dev/userfaultfd instead of userfaultfd(2)\n" ); -fprintf (stderr,"\nExample test mod usage:\n" ); -fprintf (stderr,"# Run anonymous memory test with /dev/userfaultfd:\n" ); -fprintf (stderr,"./userfaultfd anon:dev 100 99999\n\n" ); -fprintf (stderr,"Examples:\n\n" );fprintf (stderr,"%s" , examples);exit (1 );
@@ -400,21 +389,9 @@ static void set_test_type(const char *type)static void parse_test_type_arg (const char *raw_type) { -char *buf =strdup (raw_type);uint64_t features =UFFD_API_FEATURES ; -while (buf) { -const char *token =strsep (&buf,":" ); - -if (!test_type) -set_test_type (token); -else if (!strcmp (token,"dev" )) - test_dev_userfaultfd = true; -else if (!strcmp (token,"syscall" )) - test_dev_userfaultfd = false; -else -err ("unrecognized test mod '%s'" , token); - } +set_test_type (raw_type);if (!test_type)err ("failed to parse test type argument: '%s'" , raw_type);
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 30/31 selftests/mm: Add shmem-private test to uffd-stress
In reply to Peter XuThe userfaultfd stress test never tested private shmem, which I think was overlooked long due. Add it so it matches with uffd unit test and it'll cover all memory supported with the three memory types.
Meanwhile, rename the memory types a bit. Considering shared mem is the major use case for both shmem / hugetlbfs, changing from:
anon, hugetlb, hugetlb_shared, shmem
To (with shmem-private added):
anon, hugetlb, hugetlb-private, shmem, shmem-private
Add the shmem-private to run_vmtests.sh too.
--- tools/testing/selftests/mm/run_vmtests.sh | 3 ++- tools/testing/selftests/mm/uffd-stress.c | 11 +++++++---- 2 files changed, 9 insertions(+), 5 deletions(-)
tools/testing/selftests/mm/run_vmtests.sh +2 -1
@@ -202,8 +202,9 @@ CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon 20 16 # Hugetlb tests require source and destination huge pages. Pass in half # the size ($half_ufd_size_MB), which is used for *each*. CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb "$half_ufd_size_MB" 32 -CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared "$half_ufd_size_MB" 32 +CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb-private "$half_ufd_size_MB" 32 CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem 20 16 +CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem-private 20 16 #cleanup echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages
tools/testing/selftests/mm/uffd-stress.c +7 -4
@@ -59,8 +59,8 @@ const char *examples ="./userfaultfd shmem 1000 99\n\n" "# Run hugetlb memory test on 256MiB region with 50 bounces:\n" "./userfaultfd hugetlb 256 50\n\n" -"# Run the same hugetlb test but using shared file:\n" -"./userfaultfd hugetlb_shared 256 50\n\n" +"# Run the same hugetlb test but using private file:\n" +"./userfaultfd hugetlb-private 256 50\n\n" "# 10MiB-~6GiB 999 bounces anonymous test, " "continue forever unless an error triggers\n" "while ./userfaultfd anon $[RANDOM % 6000 + 10] 999; do true; done\n\n" ;
@@ -69,7 +69,7 @@ static void usage(void) {fprintf (stderr,"\nUsage: ./userfaultfd <test type> <MiB> <bounces>\n\n" );fprintf (stderr,"Supported <test type>: anon, hugetlb, " -"hugetlb_shared, shmem\n\n" ); +"hugetlb-private, shmem, shmem-private\n\n" );fprintf (stderr,"Examples:\n\n" );fprintf (stderr,"%s" , examples);exit (1 );
@@ -376,14 +376,17 @@ static void set_test_type(const char *type) }else if (!strcmp (type,"hugetlb" )) { test_type =TEST_HUGETLB ; uffd_test_ops = &hugetlb_uffd_test_ops; - }else if (!strcmp (type,"hugetlb_shared" )) { map_shared = true; + }else if (!strcmp (type,"hugetlb-private" )) { test_type =TEST_HUGETLB ; uffd_test_ops = &hugetlb_uffd_test_ops; }else if (!strcmp (type,"shmem" )) { map_shared = true; test_type =TEST_SHMEM ; uffd_test_ops = &shmem_uffd_test_ops; + }else if (!strcmp (type,"shmem-private" )) { + test_type =TEST_SHMEM ; + uffd_test_ops = &shmem_uffd_test_ops; } }
Signature · 1 line
2.39.1
Peter Xu
reviewerPATCH v2 31/31 selftests/mm: Add uffdio register ioctls test
In reply to Peter XuThis new test tests against the returned ioctls from UFFDIO_REGISTER, where put into uffdio_register.ioctls.
This also tests the expected failure cases of UFFDIO_REGISTER, aka:
- Register with empty mode should fail with -EINVAL
- Register minor without page cache (anon) should fail with -EINVAL
--- tools/testing/selftests/mm/uffd-unit-tests.c | 112 ++++++++++++++++--- 1 file changed, 97 insertions(+), 15 deletions(-)
tools/testing/selftests/mm/uffd-unit-tests.c +97 -15
@@ -62,8 +62,14 @@ mem_type_t mem_types[] = { }, }; +/* Arguments to be passed over to each uffd unit test */ +struct uffd_test_args { +mem_type_t *mem_type; +}; +typedef struct uffd_test_args uffd_test_args_t ; +/* Returns: UFFD_TEST_* */ -typedef void (*uffd_test_fn )(void ); +typedef void (*uffd_test_fn )(uffd_test_args_t *); typedef struct { const char *name;
@@ -172,8 +178,9 @@ static int test_uffd_api(bool use_dev)* This function initializes the global variables. TODO: remove global * vars and then remove this. */ -static int uffd_setup_environment (uffd_test_case_t *test,mem_type_t *mem_type, -const char **errmsg) +static int +uffd_setup_environment (uffd_test_args_t *args,uffd_test_case_t *test, +mem_type_t *mem_type,const char **errmsg) { map_shared = mem_type->shared; uffd_test_ops = mem_type->mem_ops;
@@ -187,6 +194,9 @@ static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type,/* TODO: remove this global var.. it's so ugly */ nr_cpus =1 ; +/* Initialize test arguments */ + args->mem_type = mem_type; +return uffd_test_ctx_init (test->uffd_feature_required, errmsg); }
@@ -239,7 +249,7 @@ static int pagemap_test_fork(bool present)return result; } -static void uffd_wp_unpopulated_test (void ) +static void uffd_wp_unpopulated_test (uffd_test_args_t *args) {uint64_t value;int pagemap_fd;
@@ -285,7 +295,7 @@ static void uffd_wp_unpopulated_test(void)uffd_test_pass (); } -static void uffd_pagemap_test (void ) +static void uffd_pagemap_test (uffd_test_args_t *args) {int pagemap_fd;uint64_t value;
@@ -415,17 +425,17 @@ static void uffd_minor_test_common(bool test_collapse, bool test_wp)uffd_test_pass (); } -void uffd_minor_test (void ) +void uffd_minor_test (uffd_test_args_t *args) {uffd_minor_test_common (false, false); } -void uffd_minor_wp_test (void ) +void uffd_minor_wp_test (uffd_test_args_t *args) {uffd_minor_test_common (false, true); } -void uffd_minor_collapse_test (void ) +void uffd_minor_collapse_test (uffd_test_args_t *args) {uffd_minor_test_common (true, false); }
@@ -603,12 +613,12 @@ static void uffd_sigbus_test_common(bool wp)uffd_test_pass (); } -static void uffd_sigbus_test (void ) +static void uffd_sigbus_test (uffd_test_args_t *args) {uffd_sigbus_test_common (false); } -static void uffd_sigbus_wp_test (void ) +static void uffd_sigbus_wp_test (uffd_test_args_t *args) {uffd_sigbus_test_common (true); }
@@ -651,12 +661,12 @@ static void uffd_events_test_common(bool wp)uffd_test_pass (); } -static void uffd_events_test (void ) +static void uffd_events_test (uffd_test_args_t *args) {uffd_events_test_common (false); } -static void uffd_events_wp_test (void ) +static void uffd_events_wp_test (uffd_test_args_t *args) {uffd_events_test_common (true); }
@@ -724,7 +734,7 @@ uffd_register_detect_zeropage(int uffd, void *addr, uint64_t len) }/* exercise UFFDIO_ZEROPAGE */ -static void uffd_zeropage_test (void ) +static void uffd_zeropage_test (uffd_test_args_t *args) {bool has_zeropage;int i;
@@ -748,7 +758,77 @@ static void uffd_zeropage_test(void)uffd_test_pass (); } +/* +* Test the returned uffdio_register.ioctls with different register modes. +* Note that _UFFDIO_ZEROPAGE is tested separately in the zeropage test. +*/ +static void +do_register_ioctls_test (uffd_test_args_t *args,bool miss,bool wp,bool minor) +{ +uint64_t ioctls =0 , expected =BIT_ULL (_UFFDIO_WAKE); +mem_type_t *mem_type = args->mem_type; +int ret; + + ret =uffd_register_with_ioctls (uffd, area_dst, page_size, + miss, wp, minor, &ioctls); + +/* +* Handle special cases of UFFDIO_REGISTER here where it should +* just fail with -EINVAL first.. +* +* Case 1: register MINOR on anon +* Case 2: register with no mode selected +*/ +if ((minor && (mem_type->mem_flag ==MEM_ANON )) || + (!miss && !wp && !minor)) { +if (ret != -EINVAL ) +err ("register (miss=%d, wp=%d, minor=%d) failed " +"with wrong errno=%d" , miss, wp, minor, ret); +return ; + } + +/* UFFDIO_REGISTER should succeed, then check ioctls returned */ +if (miss) + expected |=BIT_ULL (_UFFDIO_COPY); +if (wp) + expected |=BIT_ULL (_UFFDIO_WRITEPROTECT); +if (minor) + expected |=BIT_ULL (_UFFDIO_CONTINUE); + +if ((ioctls & expected) != expected) +err ("unexpected uffdio_register.ioctls " +"(miss=%d, wp=%d, minor=%d): expected=0x%" PRIx64", " +"returned=0x%" PRIx64, miss, wp, minor, expected, ioctls); + +if (uffd_unregister (uffd, area_dst, page_size)) +err ("unregister" ); +} + +static void uffd_register_ioctls_test (uffd_test_args_t *args) +{ +int miss, wp, minor; + +for (miss =0 ; miss <=1 ; miss++) +for (wp =0 ; wp <=1 ; wp++) +for (minor =0 ; minor <=1 ; minor++) +do_register_ioctls_test (args, miss, wp, minor); + +uffd_test_pass (); +} +uffd_test_case_t uffd_tests[] = { + { +/* Test returned uffdio_register.ioctls. */ + .name ="register-ioctls" , + .uffd_fn = uffd_register_ioctls_test, + .mem_targets =MEM_ALL , + .uffd_feature_required =UFFD_FEATURE_MISSING_HUGETLBFS | +UFFD_FEATURE_MISSING_SHMEM | +UFFD_FEATURE_PAGEFAULT_FLAG_WP | +UFFD_FEATURE_WP_HUGETLBFS_SHMEM | +UFFD_FEATURE_MINOR_HUGETLBFS | +UFFD_FEATURE_MINOR_SHMEM , + }, { .name ="zeropage" , .uffd_fn = uffd_zeropage_test,
@@ -835,6 +915,7 @@ int main(int argc, char *argv[])int n_mems =sizeof (mem_types) /sizeof (mem_type_t);uffd_test_case_t *test;mem_type_t *mem_type; +uffd_test_args_t args;char test_name[128 ];const char *errmsg;int has_uffd;
@@ -862,11 +943,12 @@ int main(int argc, char *argv[])uffd_test_skip ("feature missing" );continue ; } -if (uffd_setup_environment (test, mem_type, &errmsg)) { +if (uffd_setup_environment (&args, test, mem_type, + &errmsg)) {uffd_test_skip (errmsg);continue ; } - test->uffd_fn (); + test->uffd_fn (&args); } }
Signature · 1 line
2.39.1