A minor flurry of selftest/mm fixes
Subsystems (6)
v2 cover A minor flurry of selftest/mm fixes
Hi,
This is v2 of a series that fixes up build errors and warnings for at least the 64-bit builds on x86 with clang.
There are lots of changes since v1 [1], thanks to reviews from Peter Xu, David Hildenbrand, and Muhammad Usama Anjum. These include:
* Using "make headers", and documenting that prerequisite as well.
* Better ways to avoid clang's Wformat-security warnings
* Added Cc's, ack-by's, reviewed-by's.
* Updated commit log messages.
The series also includes an optional "improvement" of moving some uffd code into uffd-common.[ch], which is proving to be somewhat controversial, and so if that doesn't get resolved, then patches 9 and 10 may just get dropped. They are not required in order to get a clean build, now that "make headers" is happening.
[1]: https://lore.kernel.org/all/[email protected]/
thanks,
John Hubbard
NVIDIA
John Hubbard (11):
selftests/mm: fix uffd-stress unused function warning selftests/mm: fix unused variable warnings in hugetlb-madvise.c,
migration.c
selftests/mm: fix "warning: expression which evaluates to zero..." in
mlock2-tests.c
selftests/mm: fix invocation of tests that are run via shell scripts selftests/mm: .gitignore: add mkdirty, va_high_addr_switch selftests/mm: fix two -Wformat-security warnings in uffd builds selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h selftests/mm: fix uffd-unit-tests.c build failure due to missing
MADV_COLLAPSE
selftests/mm: move psize(), pshift() into vm_utils.c selftests/mm: move uffd* routines from vm_util.c to uffd-common.c
Documentation: kselftest: "make headers" is a prerequisite
Documentation/dev-tools/kselftest.rst | 1 + tools/testing/selftests/mm/.gitignore | 2 + tools/testing/selftests/mm/Makefile | 7 +- tools/testing/selftests/mm/cow.c | 7 -- tools/testing/selftests/mm/hugepage-mremap.c | 2 +- tools/testing/selftests/mm/hugetlb-madvise.c | 8 +- tools/testing/selftests/mm/khugepaged.c | 10 -- .../selftests/mm/ksm_functional_tests.c | 2 +- tools/testing/selftests/mm/migration.c | 5 +- tools/testing/selftests/mm/mlock2-tests.c | 1 - tools/testing/selftests/mm/pkey-x86.h | 2 +- tools/testing/selftests/mm/run_vmtests.sh | 6 +- tools/testing/selftests/mm/uffd-common.c | 105 +++++++++++++++++ tools/testing/selftests/mm/uffd-common.h | 12 +- tools/testing/selftests/mm/uffd-stress.c | 10 -- tools/testing/selftests/mm/uffd-unit-tests.c | 16 +-- tools/testing/selftests/mm/vm_util.c | 106 ++---------------- tools/testing/selftests/mm/vm_util.h | 36 ++---- 18 files changed, 165 insertions(+), 173 deletions(-)
base-commit: 929ed21dfdb6ee94391db51c9eedb63314ef6847
Signature · 1 line
2.40.1
PATCH v2 1/11 selftests/mm: fix uffd-stress unused function warning
uffd_minor_feature() was unused. Remove it in order to fix the associated clang build warning.
--- tools/testing/selftests/mm/uffd-stress.c | 10 ---------- 1 file changed, 10 deletions(-)
tools/testing/selftests/mm/uffd-stress.c +0 -10
@@ -88,16 +88,6 @@ static void uffd_stats_reset(struct uffd_args *args, unsigned long n_cpus) } } -static inline uint64_t uffd_minor_feature (void ) -{ -if (test_type ==TEST_HUGETLB && map_shared) -return UFFD_FEATURE_MINOR_HUGETLBFS ; -else if (test_type ==TEST_SHMEM ) -return UFFD_FEATURE_MINOR_SHMEM ; -else -return 0 ; -} -static void *locking_thread (void *arg) {unsigned long cpu = (unsigned long ) arg;
Signature · 1 line
2.40.1
PATCH v2 2/11 selftests/mm: fix unused variable warnings in hugetlb-madvise.c, migration.c
In reply to John HubbardDummy variables are required in order to make these two (similar) routines work, so in both cases, declare the variables as volatile in order to avoid the clang compiler warning.
Furthermore, in order to ensure that each test actually does what is intended, add an asm volatile invocation (thanks to David Hildenbrand for the suggestion), with a clarifying comment so that it survives future maintenance.
--- tools/testing/selftests/mm/hugetlb-madvise.c | 8 ++++++-- tools/testing/selftests/mm/migration.c | 5 ++++- 2 files changed, 10 insertions(+), 3 deletions(-)
tools/testing/selftests/mm/hugetlb-madvise.c +6 -2
@@ -65,11 +65,15 @@ void write_fault_pages(void *addr, unsigned long nr_pages)void read_fault_pages (void *addr,unsigned long nr_pages) { -unsigned long dummy =0 ; +volatile unsigned long dummy =0 ;unsigned long i; -for (i =0 ; i < nr_pages; i++) +for (i =0 ; i < nr_pages; i++) { dummy += *((unsigned long *)(addr + (i * huge_page_size))); + +/* Prevent the compiler from optimizing out the entire loop: */ + asmvolatile ("" :"+r" (dummy)); + } }int main (int argc,char **argv)
tools/testing/selftests/mm/migration.c +4 -1
@@ -95,12 +95,15 @@ int migrate(uint64_t *ptr, int n1, int n2)void *access_mem (void *ptr) { -uint64_t y =0 ; +volatile uint64_t y =0 ;volatile uint64_t *x = ptr;while (1 ) {pthread_testcancel (); y += *x; + +/* Prevent the compiler from optimizing out the writes to y: */ + asmvolatile ("" :"+r" (y)); }return NULL ;
Signature · 1 line
2.40.1
Re: [PATCH v2 02/11] selftests/mm: fix unused variable warnings in hugetlb-madvise.c, migration.c
Quoted 50 lines · John Hubbard
Dummy variables are required in order to make these two (similar) routines work, so in both cases, declare the variables as volatile in order to avoid the clang compiler warning.
Furthermore, in order to ensure that each test actually does what is intended, add an asm volatile invocation (thanks to David Hildenbrand for the suggestion), with a clarifying comment so that it survives future maintenance.
Cc David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Signed-off-by John Hubbard <[email protected]>--- tools/testing/selftests/mm/hugetlb-madvise.c | 8 ++++++-- tools/testing/selftests/mm/migration.c | 5 ++++- 2 files changed, 10 insertions(+), 3 deletions(-)tools/testing/selftests/mm/hugetlb-madvise.c +6
-2@@ -65,11 +65,15 @@ void write_fault_pages(void *addr, unsigned long nr_pages)void read_fault_pages (void *addr,unsigned long nr_pages) { -unsigned long dummy =0 ; +volatile unsigned long dummy =0 ;unsigned long i; -for (i =0 ; i < nr_pages; i++) +for (i =0 ; i < nr_pages; i++) { dummy += *((unsigned long *)(addr + (i * huge_page_size))); + +/* Prevent the compiler from optimizing out the entire loop: */ + asmvolatile ("" :"+r" (dummy)); + } }int main (int argc,char **argv)tools/testing/selftests/mm/migration.c +1
-1@@ -95,12 +95,15 @@ int migrate(uint64_t *ptr, int n1, int n2)void *access_mem (void *ptr) { -uint64_t y =0 ; +volatile uint64_t y =0 ;volatile uint64_t *x = ptr;
tools/testing/selftests/mm/migration.c int migrate(uint64_t *ptr, int n1, int n2) · line 109
Quoted 10 lines
tools/testing/selftests/mm/migration.c +3
-0while (1 ) {pthread_testcancel (); y += *x; + +/* Prevent the compiler from optimizing out the writes to y: */ + asmvolatile ("" :"+r" (y)); }return NULL ;
With the asm, I think the "volatile" might be completely unnecessary. But it doesn't hurt.
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v2 02/11] selftests/mm: fix unused variable warnings in hugetlb-madvise.c, migration.c
In reply to John HubbardQuoted 4 lines · John Hubbard
Dummy variables are required in order to make these two (similar) routines work, so in both cases, declare the variables as volatile in order to avoid the clang compiler warning.
Quoted 8 lines
Furthermore, in order to ensure that each test actually does what is intended, add an asm volatile invocation (thanks to David Hildenbrand for the suggestion), with a clarifying comment so that it survives future maintenance.
Cc David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Signed-off-by John Hubbard <[email protected]>
Signature · 1 line
Peter Xu
PATCH v2 3/11 selftests/mm: fix "warning: expression which evaluates to zero..." in mlock2-tests.c
In reply to John HubbardThe stop variable is a char*, and the code was assigning a char value to it. This was generating a warning when compiling with clang.
However, as both David and Peter pointed out, stop is not even used after the problematic assignment to a char type. So just delete that line entirely.
--- tools/testing/selftests/mm/mlock2-tests.c | 1 - 1 file changed, 1 deletion(-)
tools/testing/selftests/mm/mlock2-tests.c +0 -1
@@ -50,7 +50,6 @@ static int get_vm_area(unsigned long addr, struct vm_boundaries *area)printf ("cannot parse /proc/self/maps\n" ); goto out; } - stop ='\0' ;sscanf (line,"%lx" , &start);sscanf (end_addr,"%lx" , &end);
Signature · 1 line
2.40.1
Re: [PATCH v2 03/11] selftests/mm: fix "warning: expression which evaluates to zero..." in mlock2-tests.c
Quoted 11 lines · John Hubbard
The stop variable is a char*, and the code was assigning a char value to it. This was generating a warning when compiling with clang.
However, as both David and Peter pointed out, stop is not even used after the problematic assignment to a char type. So just delete that line entirely.
Cc David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Signed-off-by John Hubbard <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
Peter Xu
reviewerRe: [PATCH v2 03/11] selftests/mm: fix "warning: expression which evaluates to zero..." in mlock2-tests.c
In reply to John HubbardQuoted 19 lines · John Hubbard
The stop variable is a char*, and the code was assigning a char value to it. This was generating a warning when compiling with clang.
However, as both David and Peter pointed out, stop is not even used after the problematic assignment to a char type. So just delete that line entirely.
Cc David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Signed-off-by John Hubbard <[email protected]>--- tools/testing/selftests/mm/mlock2-tests.c | 1 - 1 file changed, 1 deletion(-)tools/testing/selftests/mm/mlock2-tests.c +0
-0
tools/testing/selftests/mm/mlock2-tests.c static int get_vm_area(unsigned long addr, struct vm_boundaries *area) · line 55
Quoted 7 lines
tools/testing/selftests/mm/mlock2-tests.c +0
-1@@ -50,7 +50,6 @@ static int get_vm_area(unsigned long addr, struct vm_boundaries *area)printf ("cannot parse /proc/self/maps\n" ); goto out; } - stop ='\0' ;sscanf (line,"%lx" , &start);sscanf (end_addr,"%lx" , &end);
I'd rather simply make it "*stop = '\0'", or as David suggested dropping stop completely when we're it (assumes that scanf() will always work with number ending with space ' ').
No strong opinion here, though.
Signature · 1 line
Peter Xu
Re: [PATCH v2 03/11] selftests/mm: fix "warning: expression which evaluates to zero..." in mlock2-tests.c
Quoted 25 lines · Peter Xu
Quoted 24 lines · John Hubbard
The stop variable is a char*, and the code was assigning a char value to it. This was generating a warning when compiling with clang.
However, as both David and Peter pointed out, stop is not even used after the problematic assignment to a char type. So just delete that line entirely.
Cc David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Signed-off-by John Hubbard <[email protected]>--- tools/testing/selftests/mm/mlock2-tests.c | 1 - 1 file changed, 1 deletion(-)tools/testing/selftests/mm/mlock2-tests.c +0
-1@@ -50,7 +50,6 @@ static int get_vm_area(unsigned long addr, struct vm_boundaries *area)printf ("cannot parse /proc/self/maps\n" ); goto out; } - stop ='\0' ;
Quoted 6 lines
tools/testing/selftests/mm/mlock2-tests.c static int get_vm_area(unsigned long addr, struct vm_boundaries *area) · line 55
Quoted 2 lines
tools/testing/selftests/mm/mlock2-tests.c +0
-0sscanf (line,"%lx" , &start);sscanf (end_addr,"%lx" , &end);I'd rather simply make it "*stop = '\0'", or as David suggested dropping stop completely when we're it (assumes that
scanf()will always work with number ending with space ' ').
Actually it does not assume that. Rather, it follows the documented behavior of strchr(3), which is:
The strchr() and strrchr() functions return a pointer to the matched character or NULL if the character is not found. The terminating null byte is considered part of the string, so that if c is specified as '\0', these functions return a pointer to the terminator.
And we have this code now:
stop = strchr(end_addr, ' ');
if (!stop) {
printf("cannot parse /proc/self/maps\n");
goto out;
}So, either stop has a valid char* in it, or we goto out. There are no fragile assumptions in there, as far as I can see anyway.
Quoted 3 lines
No strong opinion here, though.
OK, I think it's kind of a flip of the coin whether to write this:
stop = strchr(end_addr, ' ');
if (!stop) {or this:
if (!strchr(end_addr, ' ')) {So I'll just leave it as the first one, which (depending on the day of the week) might read slightly clearer. :)
thanks,
Signature · 2 lines
John Hubbard NVIDIA
PATCH v2 4/11 selftests/mm: fix invocation of tests that are run via shell scripts
In reply to John HubbardWe cannot depend upon git to reliably retain the executable bit on shell scripts, or so I was told several years ago while working on this same run_vmtests.sh script. And sure enough, things such as test_hmm.sh are lately failing to run, due to lacking execute permissions.
Fix this by explicitly adding "bash" to each of the shell script invocations. Leave fixing the overall approach to another day.
--- tools/testing/selftests/mm/run_vmtests.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tools/testing/selftests/mm/run_vmtests.sh +3 -3
@@ -242,18 +242,18 @@ if [ $VADDR64 -ne 0 ]; then if [ "$ARCH" == "$ARCH_ARM64" ]; then echo 6 > /proc/sys/vm/nr_hugepages fi - CATEGORY="hugevm" run_test ./va_high_addr_switch.sh + CATEGORY="hugevm" run_test bash ./va_high_addr_switch.sh if [ "$ARCH" == "$ARCH_ARM64" ]; then echo $prev_nr_hugepages > /proc/sys/vm/nr_hugepages fi fi # VADDR64 # vmalloc stability smoke test -CATEGORY="vmalloc" run_test ./test_vmalloc.sh smoke +CATEGORY="vmalloc" run_test bash ./test_vmalloc.sh smoke CATEGORY="mremap" run_test ./mremap_dontunmap -CATEGORY="hmm" run_test ./test_hmm.sh smoke +CATEGORY="hmm" run_test bash ./test_hmm.sh smoke # MADV_POPULATE_READ and MADV_POPULATE_WRITE tests CATEGORY="madv_populate" run_test ./madv_populate
Signature · 1 line
2.40.1
PATCH v2 5/11 selftests/mm: .gitignore: add mkdirty, va_high_addr_switch
In reply to John HubbardThese new build products were left out of .gitignore, so add them now.
--- tools/testing/selftests/mm/.gitignore | 2 ++ 1 file changed, 2 insertions(+)
tools/testing/selftests/mm/.gitignore +2 -0
@@ -39,3 +39,5 @@ local_config.h local_config.mk ksm_functional_tests mdwe_test +mkdirty +va_high_addr_switch
Signature · 1 line
2.40.1
Re: [PATCH v2 05/11] selftests/mm: .gitignore: add mkdirty, va_high_addr_switch
Quoted 14 lines · John Hubbard
These new build products were left out of .gitignore, so add them now.
Reviewed-by David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Signed-off-by John Hubbard <[email protected]>--- tools/testing/selftests/mm/.gitignore | 2 ++ 1 file changed, 2 insertions(+)tools/testing/selftests/mm/.gitignore +0
-0
tools/testing/selftests/mm/.gitignore local_config.h · line 43
Quoted 6 lines
tools/testing/selftests/mm/.gitignore +2
-0@@ -39,3 +39,5 @@ local_config.h local_config.mk ksm_functional_tests mdwe_test +mkdirty +va_high_addr_switch
Maybe also the new gup_longterm?
Signature · 1 line
Peter Xu
Re: [PATCH v2 05/11] selftests/mm: .gitignore: add mkdirty, va_high_addr_switch
Quoted 10 lines · Peter Xu
Quoted 9 lines · John Hubbard
These new build products were left out of .gitignore, so add them now.
Reviewed-by David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Signed-off-by John Hubbard <[email protected]>--- tools/testing/selftests/mm/.gitignore | 2 ++ 1 file changed, 2 insertions(+)
Quoted 14 lines
tools/testing/selftests/mm/.gitignore local_config.h · line 43
Quoted 11 lines
tools/testing/selftests/mm/.gitignore +2
-0@@ -39,3 +39,5 @@ local_config.h local_config.mk ksm_functional_tests mdwe_test +mkdirty +va_high_addr_switchMaybe also the new
gup_longterm?
Yes I could add that. It might make slightly easier for Andrew because it would preemptively avoid a minor merge conflict.
Unless David is thinking of adding it to another version of that series?
Quoted 2 lines
Reviewed-by Peter Xu <[email protected]>
Thanks for this (and the other) reviews!
thanks,
Signature · 2 lines
John Hubbard NVIDIA
PATCH v2 6/11 selftests/mm: fix two -Wformat-security warnings in uffd builds
In reply to John HubbardThe uffd tests generate two compile time warnings from clang's -Wformat-security setting. These trigger at the call sites for uffd_test_start() and uffd_test_skip().
1) Fix the uffd_test_start() issue by removing the intermediate test_name variable (thanks to David Hildenbrand for showing how to do this).
2) Fix the uffd_test_skip() issue by observing that there is no need for a macro and a variable args approach, because all callers of uffd_test_skip() pass in a simple char* string, without any format specifiers. So just change uffd_test_skip() into a regular C function.
--- tools/testing/selftests/mm/uffd-unit-tests.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
tools/testing/selftests/mm/uffd-unit-tests.c +6 -10
@@ -109,12 +109,11 @@ static void uffd_test_pass(void)ksft_inc_fail_cnt (); \ }while (0 ) -#define uffd_test_skip (...) do { \ - printf("skipped [reason: "); \ - printf(__VA_ARGS__); \ - printf("]\n"); \ - ksft_inc_xskip_cnt(); \ - } while (0) +static void uffd_test_skip(const char *message) +{ +printf ("skipped [reason: %s]\n" , message); +ksft_inc_xskip_cnt (); +}/* * Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll
@@ -1149,7 +1148,6 @@ int main(int argc, char *argv[])uffd_test_case_t *test;mem_type_t *mem_type;uffd_test_args_t args; -char test_name[128 ];const char *errmsg;int has_uffd, opt;int i, j;
@@ -1192,10 +1190,8 @@ int main(int argc, char *argv[]) mem_type = &mem_types[j];if (!(test->mem_targets & mem_type->mem_flag))continue ; -snprintf (test_name,sizeof (test_name), -"%s on %s" , test->name, mem_type->name); -uffd_test_start (test_name); +uffd_test_start ("%s on %s" , test->name, mem_type->name);if (!uffd_feature_supported (test)) {uffd_test_skip ("feature missing" );continue ;
Signature · 1 line
2.40.1
Re: [PATCH v2 06/11] selftests/mm: fix two -Wformat-security warnings in uffd builds
Quoted 52 lines · John Hubbard
The uffd tests generate two compile time warnings from clang's -Wformat-security setting. These trigger at the call sites for
uffd_test_start()anduffd_test_skip().1) Fix the
uffd_test_start()issue by removing the intermediatetest_namevariable (thanks to David Hildenbrand for showing how to do this).2) Fix the
uffd_test_skip()issue by observing that there is no need for a macro and a variable args approach, because all callers ofuffd_test_skip()pass in a simple char* string, without any format specifiers. So just changeuffd_test_skip()into a regular C function.Cc David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Signed-off-by John Hubbard <[email protected]>--- tools/testing/selftests/mm/uffd-unit-tests.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)tools/testing/selftests/mm/uffd-unit-tests.c +5
-7@@ -109,12 +109,11 @@ static void uffd_test_pass(void)ksft_inc_fail_cnt (); \ }while (0 ) -#define uffd_test_skip (...) do { \ - printf("skipped [reason: "); \ - printf(__VA_ARGS__); \ - printf("]\n"); \ - ksft_inc_xskip_cnt(); \ - } while (0) +static void uffd_test_skip(const char *message) +{ +printf ("skipped [reason: %s]\n" , message); +ksft_inc_xskip_cnt (); +}/* * Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll @@ -1149,7 +1148,6 @@ int main(int argc, char *argv[])uffd_test_case_t *test;mem_type_t *mem_type;uffd_test_args_t args; -char test_name[128 ];const char *errmsg;int has_uffd, opt;int i, j;@@ -1192,10 +1190,8 @@ int main(int argc, char *argv[]) mem_type = &mem_types[j];
tools/testing/selftests/mm/uffd-unit-tests.c int main(int argc, char *argv[]) · line 1197
Quoted 10 lines
tools/testing/selftests/mm/uffd-unit-tests.c +1
-3if (!(test->mem_targets & mem_type->mem_flag))continue ; -snprintf (test_name,sizeof (test_name), -"%s on %s" , test->name, mem_type->name); -uffd_test_start (test_name); +uffd_test_start ("%s on %s" , test->name, mem_type->name);if (!uffd_feature_supported (test)) {uffd_test_skip ("feature missing" );continue ;
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v2 06/11] selftests/mm: fix two -Wformat-security warnings in uffd builds
In reply to John HubbardQuoted 8 lines · John Hubbard
The uffd tests generate two compile time warnings from clang's -Wformat-security setting. These trigger at the call sites for
uffd_test_start()anduffd_test_skip().1) Fix the
uffd_test_start()issue by removing the intermediatetest_namevariable (thanks to David Hildenbrand for showing how to do this).
Quoted 8 lines
2) Fix the
uffd_test_skip()issue by observing that there is no need for a macro and a variable args approach, because all callers ofuffd_test_skip()pass in a simple char* string, without any format specifiers. So just changeuffd_test_skip()into a regular C function.Cc David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Signed-off-by John Hubbard <[email protected]>
Thanks,
Signature · 1 line
Peter Xu
PATCH v2 7/11 selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h
In reply to John HubbardThis fixes a real bug, too, because xstate_size() was assuming that the stack variable xstate_size was initialized to zero. That's not guaranteed nor even especially likely.
--- tools/testing/selftests/mm/pkey-x86.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/mm/pkey-x86.h +1 -1
@@ -132,7 +132,7 @@ int pkey_reg_xstate_offset(void)unsigned int ecx;unsigned int edx;int xstate_offset; -int xstate_size; +int xstate_size =0 ;unsigned long XSTATE_CPUID =0xd ;int leaf;
Signature · 1 line
2.40.1
PATCH v2 8/11 selftests/mm: fix uffd-unit-tests.c build failure due to missing MADV_COLLAPSE
In reply to John HubbardMADV_PAGEOUT, MADV_POPULATE_READ, MADV_COLLAPSE are conditionally defined as necessary. However, that was being done in .c files, and a new build failure came up that would have been automatically avoided had these been in a common header file.
So consolidate and move them all to vm_util.h, which fixes the build failure.
An alternative approach from Muhammad Usama Anjum was: rely on "make headers" being required, and include asm-generic/mman-common.h. This works in the sense that it builds, but it still generates warnings about duplicate MADV_* symbols, and the goal here is to get a fully clean (no warnings) build here.
--- tools/testing/selftests/mm/cow.c | 7 ------- tools/testing/selftests/mm/khugepaged.c | 10 ---------- tools/testing/selftests/mm/vm_util.h | 10 ++++++++++ 3 files changed, 10 insertions(+), 17 deletions(-)
tools/testing/selftests/mm/cow.c +0 -7
@@ -30,13 +30,6 @@#include "../kselftest.h" #include "vm_util.h" -#ifndef MADV_PAGEOUT -#define MADV_PAGEOUT 21 -#endif -#ifndef MADV_COLLAPSE -#define MADV_COLLAPSE 25 -#endif -static size_t pagesize;static int pagemap_fd;static size_t thpsize;
tools/testing/selftests/mm/khugepaged.c +0 -10
@@ -22,16 +22,6 @@#include "vm_util.h" -#ifndef MADV_PAGEOUT -#define MADV_PAGEOUT 21 -#endif -#ifndef MADV_POPULATE_READ -#define MADV_POPULATE_READ 22 -#endif -#ifndef MADV_COLLAPSE -#define MADV_COLLAPSE 25 -#endif -#define BASE_ADDR ((void *)(1UL << 30))static unsigned long hpage_pmd_size;static unsigned long page_size;
tools/testing/selftests/mm/vm_util.h +10 -0
@@ -63,3 +63,13 @@ int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,#define PAGEMAP_PRESENT (ent) (((ent) & (1ull << 63)) != 0)#define PAGEMAP_PFN (ent) ((ent) & ((1ull << 55) - 1)) + +#ifndef MADV_PAGEOUT +#define MADV_PAGEOUT 21 +#endif +#ifndef MADV_POPULATE_READ +#define MADV_POPULATE_READ 22 +#endif +#ifndef MADV_COLLAPSE +#define MADV_COLLAPSE 25 +#endif
Signature · 1 line
2.40.1
Re: [PATCH v2 08/11] selftests/mm: fix uffd-unit-tests.c build failure due to missing MADV_COLLAPSE
Correction: due to refactoring things a bit in v2, this patch's subject line is no longer accurate. It should be changed to:
[PATCH v2 08/11] selftests/mm: fix build failures due to missing MADV_COLLAPSE
thanks,
Signature · 2 lines
John Hubbard NVIDIA
Quoted 81 lines · John Hubbard
MADV_PAGEOUT,MADV_POPULATE_READ,MADV_COLLAPSEare conditionally defined as necessary. However, that was being done in .c files, and a new build failure came up that would have been automatically avoided had these been in a common header file.So consolidate and move them all to
vm_util.h, which fixes the build failure.An alternative approach from Muhammad Usama Anjum was: rely on "make headers" being required, and include
asm-generic/mman-common.h. This works in the sense that it builds, but it still generates warnings about duplicate MADV_* symbols, and the goal here is to get a fully clean (no warnings) build here.Reviewed-by David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Cc Muhammad Usama Anjum <[email protected]>Signed-off-by John Hubbard <[email protected]>--- tools/testing/selftests/mm/cow.c | 7 ------- tools/testing/selftests/mm/khugepaged.c | 10 ---------- tools/testing/selftests/mm/vm_util.h | 10 ++++++++++ 3 files changed, 10 insertions(+), 17 deletions(-)tools/testing/selftests/mm/cow.c +0
-7@@ -30,13 +30,6 @@#include "../kselftest.h" #include "vm_util.h" -#ifndef MADV_PAGEOUT -#define MADV_PAGEOUT 21 -#endif -#ifndef MADV_COLLAPSE -#define MADV_COLLAPSE 25 -#endif -static size_t pagesize;static int pagemap_fd;static size_t thpsize;tools/testing/selftests/mm/khugepaged.c +0
-10@@ -22,16 +22,6 @@#include "vm_util.h" -#ifndef MADV_PAGEOUT -#define MADV_PAGEOUT 21 -#endif -#ifndef MADV_POPULATE_READ -#define MADV_POPULATE_READ 22 -#endif -#ifndef MADV_COLLAPSE -#define MADV_COLLAPSE 25 -#endif -#define BASE_ADDR ((void *)(1UL << 30))static unsigned long hpage_pmd_size;static unsigned long page_size;tools/testing/selftests/mm/vm_util.h +10
-0@@ -63,3 +63,13 @@ int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,#define PAGEMAP_PRESENT (ent) (((ent) & (1ull << 63)) != 0)#define PAGEMAP_PFN (ent) ((ent) & ((1ull << 55) - 1)) + +#ifndef MADV_PAGEOUT +#define MADV_PAGEOUT 21 +#endif +#ifndef MADV_POPULATE_READ +#define MADV_POPULATE_READ 22 +#endif +#ifndef MADV_COLLAPSE +#define MADV_COLLAPSE 25 +#endif
PATCH v2 9/11 selftests/mm: move psize(), pshift() into vm_utils.c
In reply to John HubbardThis is in preparation for linking test programs with both vm_utils.c and uffd-common.c. The static inline routines, while normally not a problem, in this case complicate an already fragile header file
situation: the header files including other header files leads to compilation failures in a subsequent patch that moves code around.
Anyway, there is no particular need for inlining here, so turn these into normal functions, as a workaround to avoid refactoring the header file includes for now.
--- tools/testing/selftests/mm/vm_util.c | 14 ++++++++++++++ tools/testing/selftests/mm/vm_util.h | 16 ++-------------- 2 files changed, 16 insertions(+), 14 deletions(-)
tools/testing/selftests/mm/vm_util.c +14 -0
@@ -301,3 +301,17 @@ int uffd_get_features(uint64_t *features)return 0 ; } + +unsigned int psize (void ) +{ +if (!__page_size) + __page_size =sysconf (_SC_PAGESIZE); +return __page_size; +} + +unsigned int pshift (void ) +{ +if (!__page_shift) + __page_shift = (ffsl (psize ()) -1 ); +return __page_shift; +}
tools/testing/selftests/mm/vm_util.h +2 -14
@@ -3,7 +3,6 @@#include <stdbool.h> #include <sys/mman.h> #include <err.h> -#include <string.h> /* ffsl() */ #include <unistd.h> /* _SC_PAGESIZE */ #define BIT_ULL (nr) (1ULL << (nr))
@@ -17,19 +16,8 @@extern unsigned int __page_size;extern unsigned int __page_shift; -static inline unsigned int psize (void ) -{ -if (!__page_size) - __page_size =sysconf (_SC_PAGESIZE); -return __page_size; -} - -static inline unsigned int pshift (void ) -{ -if (!__page_shift) - __page_shift = (ffsl (psize ()) -1 ); -return __page_shift; -} +unsigned int psize (void ); +unsigned int pshift (void );uint64_t pagemap_get_entry (int fd,char *start);bool pagemap_is_softdirty (int fd,char *start);
Signature · 1 line
2.40.1
PATCH v2 10/11 selftests/mm: move uffd* routines from vm_util.c to uffd-common.c
In reply to John HubbardMove the uffd*() routines to their natural home. Note that ksm_functional_tests.c also depend, intentionally (due to a recent commit [1]), upon uffd-common.[ch].
--- tools/testing/selftests/mm/Makefile | 7 +- tools/testing/selftests/mm/hugepage-mremap.c | 2 +- .../selftests/mm/ksm_functional_tests.c | 2 +- tools/testing/selftests/mm/uffd-common.c | 105 ++++++++++++++++++ tools/testing/selftests/mm/uffd-common.h | 12 +- tools/testing/selftests/mm/vm_util.c | 104 ----------------- tools/testing/selftests/mm/vm_util.h | 10 -- 7 files changed, 122 insertions(+), 120 deletions(-)
tools/testing/selftests/mm/Makefile +5 -2
@@ -109,8 +109,11 @@ include ../lib.mk $(TEST_GEN_PROGS): vm_util.c -$(OUTPUT)/uffd-stress: uffd-common.c -$(OUTPUT)/uffd-unit-tests: uffd-common.c +$(OUTPUT)/uffd-stress: uffd-common.c +$(OUTPUT)/uffd-unit-tests: uffd-common.c +$(OUTPUT)/hugepage-mremap: uffd-common.c +$(OUTPUT)/write_to_hugetlbfs: uffd-common.c +$(OUTPUT)/ksm_functional_tests: uffd-common.c ifeq ($(MACHINE),x86_64) BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
tools/testing/selftests/mm/hugepage-mremap.c +1 -1
@@ -24,7 +24,7 @@#include <sys/ioctl.h> #include <string.h> #include <stdbool.h> -#include "vm_util.h" +#include "uffd-common.h" #define DEFAULT_LENGTH_MB 10UL#define MB_TO_BYTES (x) (x * 1024 * 1024)
tools/testing/selftests/mm/ksm_functional_tests.c +1 -1
@@ -22,7 +22,7 @@#include <linux/userfaultfd.h> #include "../kselftest.h" -#include "vm_util.h" +#include "uffd-common.h" #define KiB 1024u#define MiB (1024 * KiB)
tools/testing/selftests/mm/uffd-common.c +105 -0
114 lines not on this page · raw message
tools/testing/selftests/mm/uffd-common.h +10 -2
@@ -19,8 +19,6 @@#include <signal.h> #include <poll.h> #include <string.h> -#include <linux/mman.h> -#include <sys/mman.h> #include <sys/syscall.h> #include <sys/ioctl.h> #include <sys/wait.h>
@@ -110,6 +108,16 @@ int __copy_page(int ufd, unsigned long offset, bool retry, bool wp);int copy_page (int ufd,unsigned long offset,bool wp);void *uffd_poll_thread (void *arg); +int uffd_register (int uffd,void *addr,uint64_t len, +bool miss,bool wp,bool minor); +int uffd_unregister (int uffd,void *addr,uint64_t len); +int uffd_open_dev (unsigned int flags); +int uffd_open_sys (unsigned int flags); +int uffd_open (unsigned int flags); +int uffd_get_features (uint64_t *features); +int uffd_register_with_ioctls (int uffd,void *addr,uint64_t len, +bool miss,bool wp,bool minor,uint64_t *ioctls); +#define TEST_ANON 1#define TEST_HUGETLB 2#define TEST_SHMEM 3
tools/testing/selftests/mm/vm_util.c +0 -104
110 lines not on this page · raw message
tools/testing/selftests/mm/vm_util.h +0 -10
@@ -33,16 +33,6 @@ bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);int64_t allocate_transhuge (void *ptr,int pagemap_fd);unsigned long default_huge_page_size (void ); -int uffd_register (int uffd,void *addr,uint64_t len, -bool miss,bool wp,bool minor); -int uffd_unregister (int uffd,void *addr,uint64_t len); -int uffd_open_dev (unsigned int flags); -int uffd_open_sys (unsigned int flags); -int uffd_open (unsigned int flags); -int uffd_get_features (uint64_t *features); -int uffd_register_with_ioctls (int uffd,void *addr,uint64_t len, -bool miss,bool wp,bool minor,uint64_t *ioctls); -/* * On ppc64 this will only work with radix 2M hugepage size */
Signature · 1 line
2.40.1
Re: [PATCH v2 10/11] selftests/mm: move uffd* routines from vm_util.c to uffd-common.c
Quoted 7 lines · John Hubbard
Move the uffd*() routines to their natural home. Note that
ksm_functional_tests.calso depend, intentionally (due to a recent commit [1]), upon uffd-common.[ch].Cc David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Signed-off-by John Hubbard <[email protected]>
Signature · 3 lines
Cheers, David / dhildenb
Peter Xu
reviewerRe: [PATCH v2 10/11] selftests/mm: move uffd* routines from vm_util.c to uffd-common.c
In reply to John HubbardQuoted 22 lines · John Hubbard
Move the uffd*() routines to their natural home. Note that
ksm_functional_tests.calso depend, intentionally (due to a recent commit [1]), upon uffd-common.[ch].Cc David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Signed-off-by John Hubbard <[email protected]>--- tools/testing/selftests/mm/Makefile | 7 +- tools/testing/selftests/mm/hugepage-mremap.c | 2 +- .../selftests/mm/ksm_functional_tests.c | 2 +- tools/testing/selftests/mm/uffd-common.c | 105 ++++++++++++++++++ tools/testing/selftests/mm/uffd-common.h | 12 +- tools/testing/selftests/mm/vm_util.c | 104 ----------------- tools/testing/selftests/mm/vm_util.h | 10 -- 7 files changed, 122 insertions(+), 120 deletions(-)tools/testing/selftests/mm/Makefile +0
-0
tools/testing/selftests/mm/Makefile include ../lib.mk · line 116
Quoted 10 lines
tools/testing/selftests/mm/Makefile +5
-2@@ -109,8 +109,11 @@ include ../lib.mk $(TEST_GEN_PROGS): vm_util.c -$(OUTPUT)/uffd-stress: uffd-common.c -$(OUTPUT)/uffd-unit-tests: uffd-common.c +$(OUTPUT)/uffd-stress: uffd-common.c +$(OUTPUT)/uffd-unit-tests: uffd-common.c +$(OUTPUT)/hugepage-mremap: uffd-common.c +$(OUTPUT)/write_to_hugetlbfs: uffd-common.c +$(OUTPUT)/ksm_functional_tests: uffd-common.c
Sorry, John, I still cannot follow..
As I said before uffd-common.[ch] was for uffd stress/unit tests. I confess my fault to not have named it uffd-test-common.[ch] already.
I think it's fine to keep uffd_*() helpers in vm_util.[ch] for now, until it grows. Just like if one day we'll have a pagemap.c test we don't necessary need to move pagemap_*() helpers from vm_utils.[ch] into pagemap.[ch]. It just keeps common test helpers.
Can we avoid linking those into other tests in whatever way? Maybe renaming it to uffd-test-common.[ch] may be cleaner?
Signature · 1 line
Peter Xu
Re: [PATCH v2 10/11] selftests/mm: move uffd* routines from vm_util.c to uffd-common.c
On 6/5/23 08:59, Peter Xu wrote:
...
Quoted 8 lines
Quoted 7 lines
-$(OUTPUT)/uffd-stress:
uffd-common.c
-$(OUTPUT)/uffd-unit-tests:uffd-common.c+$(OUTPUT)/uffd-stress: uffd-common.c +$(OUTPUT)/uffd-unit-tests: uffd-common.c +$(OUTPUT)/hugepage-mremap: uffd-common.c +$(OUTPUT)/write_to_hugetlbfs: uffd-common.c+$(OUTPUT)/ksm_functional_tests:
uffd-common.c
Quoted 4 lines
Sorry, John, I still cannot follow..
As I said before uffd-common.[ch] was for uffd stress/unit tests. I confess my fault to not have named it uffd-test-common.[ch] already.
Actually, given that there is nothing *except* test code in this directory, I think your original choice of file names is just right.
Quoted 9 lines
I think it's fine to keep uffd_*() helpers in vm_util.[ch] for now, until it grows. Just like if one day we'll have a
pagemap.ctest we don't necessary need to move pagemap_*() helpers from vm_utils.[ch] into pagemap.[ch]. It just keeps common test helpers.Can we avoid linking those into other tests in whatever way? Maybe renaming it to uffd-test-common.[ch] may be cleaner?
It sounds like you are suggesting this:
$(OUTPUT)/uffd-stress: uffd-common.c uffd-test-common.c $(OUTPUT)/uffd-unit-tests: uffd-common.c uffd-test-common.c $(OUTPUT)/hugepage-mremap: uffd-test-common.c $(OUTPUT)/write_to_hugetlbfs: uffd-test-common.c
$(OUTPUT)/ksm_functional_tests: uffd-test-common.c
...approximately. Do I have that correct? I can arrange it that way if you feel it's a better end result. (And it's better than leaving uffd*() helpers in vm_utils, imho.)
thanks,
Signature · 2 lines
John Hubbard NVIDIA
Peter Xu
reviewerRe: [PATCH v2 10/11] selftests/mm: move uffd* routines from vm_util.c to uffd-common.c
Quoted 34 lines · John Hubbard
On 6/5/23 08:59, Peter Xu wrote:
...Quoted 8 lines
Quoted 7 lines
-$(OUTPUT)/uffd-stress:
uffd-common.c
-$(OUTPUT)/uffd-unit-tests:uffd-common.c+$(OUTPUT)/uffd-stress: uffd-common.c +$(OUTPUT)/uffd-unit-tests: uffd-common.c +$(OUTPUT)/hugepage-mremap: uffd-common.c +$(OUTPUT)/write_to_hugetlbfs: uffd-common.c+$(OUTPUT)/ksm_functional_tests:
uffd-common.cQuoted 4 lines
Sorry, John, I still cannot follow..
As I said before uffd-common.[ch] was for uffd stress/unit tests. I confess my fault to not have named it uffd-test-common.[ch] already.
Actually, given that there is nothing *except* test code in this directory, I think your original choice of file names is just right.
Quoted 9 lines
I think it's fine to keep uffd_*() helpers in vm_util.[ch] for now, until it grows. Just like if one day we'll have a
pagemap.ctest we don't necessary need to move pagemap_*() helpers from vm_utils.[ch] into pagemap.[ch]. It just keeps common test helpers.Can we avoid linking those into other tests in whatever way? Maybe renaming it to uffd-test-common.[ch] may be cleaner?
It sounds like you are suggesting this:
$(OUTPUT)/uffd-stress: uffd-common.c uffd-test-common.c $(OUTPUT)/uffd-unit-tests: uffd-common.c uffd-test-common.c $(OUTPUT)/hugepage-mremap: uffd-test-common.c $(OUTPUT)/write_to_hugetlbfs: uffd-test-common.c
Quoted 5 lines
$(OUTPUT)/ksm_functional_tests:
uffd-test-common.c...approximately. Do I have that correct? I can arrange it that way if you feel it's a better end result. (And it's better than leaving uffd*() helpers in
vm_utils, imho.)
Yes, as long as we don't link (especially) the uffd test specific globals into non-uffd test programs I'll have no issue. Thanks.
Signature · 1 line
Peter Xu
Re: [PATCH v2 10/11] selftests/mm: move uffd* routines from vm_util.c to uffd-common.c
On 6/5/23 12:24, Peter Xu wrote:
...
Quoted 8 lines
Quoted 8 lines
It sounds like you are suggesting this:
$(OUTPUT)/uffd-stress: uffd-common.c uffd-test-common.c $(OUTPUT)/uffd-unit-tests: uffd-common.c uffd-test-common.c $(OUTPUT)/hugepage-mremap: uffd-test-common.c $(OUTPUT)/write_to_hugetlbfs: uffd-test-common.c$(OUTPUT)/ksm_functional_tests:
uffd-test-common.c
Quoted 7 lines
Quoted 3 lines
...approximately. Do I have that correct? I can arrange it that way if you feel it's a better end result. (And it's better than leaving uffd*() helpers in
vm_utils, imho.)Yes, as long as we don't link (especially) the uffd test specific globals into non-uffd test programs I'll have no issue. Thanks.
OK very good. I'll put that together.
thanks,
Signature · 2 lines
John Hubbard NVIDIA
PATCH v2 11/11 Documentation: kselftest: "make headers" is a prerequisite
In reply to John HubbardAs per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:
make headers && make -C tools/testing/selftests/mm
However, that's not yet documented anywhere. So add it to Documentation/dev-tools/kselftest.rst .
[1] https://lore.kernel.org/all/[email protected]/
--- Documentation/dev-tools/kselftest.rst | 1 + 1 file changed, 1 insertion(+)
Documentation/dev-tools/kselftest.rst +1 -0
@@ -36,6 +36,7 @@ Running the selftests (hotplug tests are run in limited mode) To build the tests:: + $ make headers $ make -C tools/testing/selftests To run the tests::
Signature · 1 line
2.40.1
Re: [PATCH v2 11/11] Documentation: kselftest: "make headers" is a prerequisite
Quoted 25 lines · John Hubbard
As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:
make headers && make -C tools/testing/selftests/mmHowever, that's not yet documented anywhere. So add it to
Documentation/dev-tools/kselftest.rst.[1] https://lore.kernel.org/all/[email protected]/
Cc David Hildenbrand <[email protected]>Cc Peter Xu <[email protected]>Cc Muhammad Usama Anjum <[email protected]>Cc Jonathan Corbet <[email protected]>Signed-off-by John Hubbard <[email protected]>--- Documentation/dev-tools/kselftest.rst | 1 + 1 file changed, 1 insertion(+)Documentation/dev-tools/kselftest.rst +0
-0
Documentation/dev-tools/kselftest.rst Running the selftests (hotplug tests are run in limited mode) · line 42
Quoted 7 lines
Documentation/dev-tools/kselftest.rst +1
-0@@ -36,6 +36,7 @@ Running the selftests (hotplug tests are run in limited mode) To build the tests:: + $ make headers $ make -C tools/testing/selftests To run the tests::
Signature · 3 lines
Cheers, David / dhildenb