make MM selftests more CI friendly
Subsystems (7)
Mike Rapoport
maintainerv3 cover make MM selftests more CI friendly
From: "Mike Rapoport (Microsoft)" <[email protected]>
Hi,
There's a lot of dancing around HugeTLB settings in run_vmtests.sh. Some test need just a few default huge pages, some require at least 256 MB, and some just skip lots of tests if huge pages of all supported sizes are not available.
The goal of this set is to make tests deal with HugeTLB setup and teardown.
There are already convenient helpers that allow easy reading and writing of /proc and /sysfs, so adding a few APIs that will detect and update HugeTLB settings shouldn't be a big deal. But these nice helpers use kselftest framework, and many of HugeTLB (and even THP) test don't, so as a result this patchset also includes a lot of churn for conversion of those tests to kselftest framework (patches 7-19).
The series break out:
patches 1-5: small fixes patch 6: merge of hugetlb mmap tests patch 7: renaming of hugepage-* to hugetlb-* patches 8-20: mechanical conversion to kselftest framework patches 21-27: extension of thp_settings to hugepage_settings to also include
HugeTLB helpers
patches 28-29: add helpers for setting up SHM limits in hugetlb-shm and
thuge-gen tests
patches 30-52: integrate the new APIs in all the tests that use HugeTLB patches 53-54: drop HugeTLB setup from run_vmtests.sh
v3 changes:
* add patches that fix cleanup of fork()ed processes in migration test
* drop MADV_REMOVE changes to khugepage test
* properly setup memfd for non-default hugepages in hugetlb-mmap test
* add forgotten hugetlb_save_settings() to hugetlb_setup()
* fixes for ksft reporting
* rebase on v7.1-rc1
* add more tags, thanks everyone!
v2: https://lore.kernel.org/all/[email protected]
* fix hugetlb mmap tests to actually test file mappings
* increase maximal tests number in khugepaged
* add tags, thanks everyone!
v1: https://lore.kernel.org/all/[email protected]
Mike Rapoport (Microsoft) (54):
selftests/mm: hugetlb-read-hwpoison: add SIGBUS handler selftests/mm: migration: don't assume hupe page is TWOMEG selftests/mm: migration: make nthreads represent number of working threads selftests/mm: migration: properly cleanup fork()ed processes selftests/mm: run_vmtests.sh: don't gate THP and KSM tests on HAVE_HUGEPAGES selftests/mm: merge map_hugetlb into hugepage-mmap
selftests/mm: rename hugepage-* tests to hugetlb-*
selftests/mm: hugetlb-shm: use kselftest framework
selftests/mm: hugetlb-vmemmap: use kselftest framework selftests/mm: hugetlb-madvise: use kselftest framework selftests/mm: hugetlb_madv_vs_map: use kselftest framework selftests/mm: hugetlb-read-hwpoison: use kselftest framework selftests/mm: khugepaged: group tests in an array
selftests/mm: khugepaged: use ksefltest framework
selftests/mm: ksm_tests: use kselftest framework
selftests/mm: protection_keys: use descriptive test names in TAP output selftests/mm: protection_keys: use kselftest framework selftests/mm: uffd-stress: use kselftest framework
selftests/mm: uffd-unit-tests: use kselftest framework selftests/mm: va_high_addr_switch: use kselftest framework selftests/mm: add atexit() and signal handlers to thp_settings selftests/mm: rename thp_settings.[ch] to hugepage_settings.[ch] selftests/mm: move HugeTLB helpers to hugepage_settings selftests/mm: hugepage_settings: use unsigned long in detect_hugetlb_page_size selftests/mm: hugepage_settings: add APIs to get and set nr_hugepages selftests/mm: hugepage_settings: rename get_free_hugepages() selftests/mm: hugepage_settings: add APIs for HugeTLB setup and teardown selftests/mm: move read_file(), read_num() and write_num() to vm_util selftests/mm: vm_util: add helpers to set and restore shm limits selftests/mm: compaction_test: use HugeTLB helpers ... selftests/mm: cow: add setup of HugeTLB pages
selftests/mm: gup_longterm: add setup of HugeTLB pages selftests/mm: gup_test: add setup of HugeTLB pages
selftests/mm: hmm-tests: add setup of HugeTLB pages
selftests/mm: hugepage_dio: add setup of HugeTLB pages selftests/mm: hugetlb_fault_after_madv: add setup of HugeTLB pages selftests/mm: hugetlb-madvise: add setup of HugeTLB pages selftests/mm: hugetlb_madv_vs_map: add setup of HugeTLB pages selftests/mm: hugetlb-mmap: add setup of HugeTLB pages selftests/mm: hugetlb-mremap: add setup of HugeTLB pages selftests/mm: hugetlb-shm: add setup of HugeTLB pages selftests/mm: hugetlb-soft-online: add setup of HugeTLB pages selftests/mm: hugetlb-vmemmap: add setup of HugeTLB pages selftests/mm: migration: add setup of HugeTLB pages
selftests/mm: pagemap_ioctl: add setup of HugeTLB pages selftests/mm: protection_keys: use library code for HugeTLB setup selftests/mm: thuge-gen: add setup of HugeTLB pages
selftests/mm: uffd-stress: use hugetlb_save and alloc huge pages selftests/mm: uffd-unit-tests: add setup of HugeTLB pages selftests/mm: uffd-wp-mremap: add setup of HugeTLB pages selftests/mm: va_high_addr_switch: add setup of HugeTLB pages selftests/mm: va_high_addr_switch.sh: drop huge pages setup selftests/mm: run_vmtests.sh: free memory if available memory is low selftests/mm: run_vmtests.sh: drop detection and setup of HugeTLB
tools/testing/selftests/mm/.gitignore | 4 +
tools/testing/selftests/mm/Makefile | 13 +-
tools/testing/selftests/mm/compaction_test.c | 113 +-----
tools/testing/selftests/mm/cow.c | 29 +-
.../selftests/mm/folio_split_race_test.c | 2 +-
tools/testing/selftests/mm/guard-regions.c | 2 +-
tools/testing/selftests/mm/gup_longterm.c | 3 +-
tools/testing/selftests/mm/gup_test.c | 12 +
tools/testing/selftests/mm/hmm-tests.c | 24 +-
tools/testing/selftests/mm/hugepage-mmap.c | 78 ----
.../{thp_settings.c => hugepage_settings.c} | 286 +++++++++++--
.../{thp_settings.h => hugepage_settings.h} | 75 +++-
tools/testing/selftests/mm/hugetlb-madvise.c | 209 ++++------
tools/testing/selftests/mm/hugetlb-mmap.c | 145 +++++++
.../{hugepage-mremap.c => hugetlb-mremap.c} | 13 +-
.../selftests/mm/hugetlb-read-hwpoison.c | 123 +++---
.../mm/{hugepage-shm.c => hugetlb-shm.c} | 64 +--
.../selftests/mm/hugetlb-soft-offline.c | 45 +--
.../{hugepage-vmemmap.c => hugetlb-vmemmap.c} | 46 +--
tools/testing/selftests/mm/hugetlb_dio.c | 15 +-
.../selftests/mm/hugetlb_fault_after_madv.c | 7 +-
.../selftests/mm/hugetlb_madv_vs_map.c | 22 +-
tools/testing/selftests/mm/khugepaged.c | 380 ++++++++----------
tools/testing/selftests/mm/ksm_tests.c | 182 ++++-----
tools/testing/selftests/mm/map_hugetlb.c | 88 ----
tools/testing/selftests/mm/migration.c | 152 ++++---
tools/testing/selftests/mm/pagemap_ioctl.c | 13 +-
tools/testing/selftests/mm/pkey-helpers.h | 6 +-
.../testing/selftests/mm/prctl_thp_disable.c | 2 +-
tools/testing/selftests/mm/protection_keys.c | 130 +++---
tools/testing/selftests/mm/run_vmtests.sh | 177 ++------
tools/testing/selftests/mm/soft-dirty.c | 2 +-
.../selftests/mm/split_huge_page_test.c | 2 +-
tools/testing/selftests/mm/thuge-gen.c | 80 +---
tools/testing/selftests/mm/transhuge-stress.c | 2 +-
tools/testing/selftests/mm/uffd-common.h | 1 +
tools/testing/selftests/mm/uffd-stress.c | 44 +-
tools/testing/selftests/mm/uffd-unit-tests.c | 110 +++--
tools/testing/selftests/mm/uffd-wp-mremap.c | 12 +-
.../selftests/mm/va_high_addr_switch.c | 42 +-
.../selftests/mm/va_high_addr_switch.sh | 41 +-
tools/testing/selftests/mm/vm_util.c | 133 +++---
tools/testing/selftests/mm/vm_util.h | 15 +-
43 files changed, 1424 insertions(+), 1520 deletions(-)
delete mode 100644 tools/testing/selftests/mm/hugepage-mmap.c
rename tools/testing/selftests/mm/{thp_settings.c => hugepage_settings.c} (60%)
rename tools/testing/selftests/mm/{thp_settings.h => hugepage_settings.h} (55%)
create mode 100644 tools/testing/selftests/mm/hugetlb-mmap.c
rename tools/testing/selftests/mm/{hugepage-mremap.c => hugetlb-mremap.c} (94%)
rename tools/testing/selftests/mm/{hugepage-shm.c => hugetlb-shm.c} (57%)
rename tools/testing/selftests/mm/{hugepage-vmemmap.c => hugetlb-vmemmap.c} (76%)
delete mode 100644 tools/testing/selftests/mm/map_hugetlb.cbase-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731 --
2.53.0
Mike Rapoport
maintainerPATCH v3 1/54 selftests/mm: hugetlb-read-hwpoison: add SIGBUS handler
From: "Mike Rapoport (Microsoft)" <[email protected]>
Access of poisoned memory causes SIGBUS, which terminates the hugetlb-read-hwpoison test prematurely.
Add a dummy SIGBUS handler to allow the test to continue regardless of SIGBUS.
--- tools/testing/selftests/mm/hugetlb-read-hwpoison.c | 7 +++++++ 1 file changed, 7 insertions(+)
tools/testing/selftests/mm/hugetlb-read-hwpoison.c +7 -0
@@ -10,6 +10,7 @@#include <sys/statfs.h> #include <errno.h> #include <stdbool.h> +#include <signal.h> #include "kselftest.h"
@@ -261,6 +262,11 @@ static int create_hugetlbfs_file(struct statfs *file_stat)return -1 ; } +static void sigbus_handler (int sig) +{ +printf (PREFIX "received SIGBUS\n" ); +} +int main (void ) {int fd;
@@ -273,6 +279,7 @@ int main(void) };size_t i; +signal (SIGBUS , sigbus_handler);for (i =0 ; i <ARRAY_SIZE (wr_chunk_sizes); ++i) {printf ("Write/read chunk size=0x%lx\n" , wr_chunk_sizes[i]);
Signature · 1 line
2.53.0
Re: [PATCH v3 01/54] selftests/mm: hugetlb-read-hwpoison: add SIGBUS handler
Quoted 10 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Access of poisoned memory causes SIGBUS, which terminates the hugetlb-read-hwpoison test prematurely.
Add a dummy SIGBUS handler to allow the test to continue regardless of SIGBUS.
Tested-by Sarthak Sharma <[email protected]>Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Signature · 2 lines
Regards, Li Wang
Mike Rapoport
maintainerPATCH v3 2/54 selftests/mm: migration: don't assume hupe page is TWOMEG
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
migration tests presume that both THP and HugeTLB huge pages are 2MB.
Add dynamic detection of huge page size with read_pmd_pagesize() for THP and with default_huge_page_size() for HugeTLB.
--- tools/testing/selftests/mm/migration.c | 44 +++++++++++++++++++------- 1 file changed, 32 insertions(+), 12 deletions(-)
tools/testing/selftests/mm/migration.c +32 -12
@@ -184,22 +184,27 @@ TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME)*/ TEST_F_TIMEOUT(migration, private_anon_thp, 2*RUNTIME) { +uint64_t pmdsize;uint64_t *ptr;int i;if (!thp_is_enabled ())SKIP (return,"Transparent Hugepages not available" ); + pmdsize =read_pmd_pagesize (); +if (!pmdsize) +SKIP (return,"Reading PMD pagesize failed" ); +if (self->nthreads <2 || self->n1 <0 || self->n2 <0 )SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,2 *TWOMEG ,PROT_READ |PROT_WRITE , + ptr =mmap (NULL ,2 * pmdsize,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED ); - ptr = (uint64_t *)ALIGN ((uintptr_t ) ptr,TWOMEG ); -ASSERT_EQ (madvise (ptr,TWOMEG ,MADV_HUGEPAGE ),0 ); -memset (ptr,0xde ,TWOMEG ); + ptr = (uint64_t *)ALIGN ((uintptr_t ) ptr, pmdsize); +ASSERT_EQ (madvise (ptr, pmdsize,MADV_HUGEPAGE ),0 ); +memset (ptr,0xde , pmdsize);for (i =0 ; i < self->nthreads -1 ; i++)if (pthread_create (&self->threads[i],NULL , access_mem, ptr))perror ("Couldn't create thread" );
@@ -215,6 +220,7 @@ TEST_F_TIMEOUT(migration, private_anon_thp, 2*RUNTIME) TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME) { +unsigned long pmdsize;pid_t pid;uint64_t *ptr;int i;
@@ -222,17 +228,21 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME)if (!thp_is_enabled ())SKIP (return,"Transparent Hugepages not available" ); + pmdsize =read_pmd_pagesize (); +if (!pmdsize) +SKIP (return,"Reading PMD pagesize failed" ); +if (self->nthreads <2 || self->n1 <0 || self->n2 <0 )SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,2 *TWOMEG ,PROT_READ |PROT_WRITE , + ptr =mmap (NULL ,2 * pmdsize,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED ); - ptr = (uint64_t *)ALIGN ((uintptr_t ) ptr,TWOMEG ); -ASSERT_EQ (madvise (ptr,TWOMEG ,MADV_HUGEPAGE ),0 ); + ptr = (uint64_t *)ALIGN ((uintptr_t ) ptr, pmdsize); +ASSERT_EQ (madvise (ptr, pmdsize,MADV_HUGEPAGE ),0 ); -memset (ptr,0xde ,TWOMEG ); +memset (ptr,0xde , pmdsize);for (i =0 ; i < self->nthreads -1 ; i++) { pid =fork ();if (!pid) {
@@ -256,17 +266,22 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME)*/ TEST_F_TIMEOUT(migration, private_anon_htlb, 2*RUNTIME) { +unsigned long hugepage_size;uint64_t *ptr;int i;if (self->nthreads <2 || self->n1 <0 || self->n2 <0 )SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,TWOMEG ,PROT_READ |PROT_WRITE , + hugepage_size =default_huge_page_size (); +if (!hugepage_size) +SKIP (return,"Reading HugeTLB pagesize failed\n" ); + + ptr =mmap (NULL , hugepage_size,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED ); -memset (ptr,0xde ,TWOMEG ); +memset (ptr,0xde , hugepage_size);for (i =0 ; i < self->nthreads -1 ; i++)if (pthread_create (&self->threads[i],NULL , access_mem, ptr))perror ("Couldn't create thread" );
@@ -281,6 +296,7 @@ TEST_F_TIMEOUT(migration, private_anon_htlb, 2*RUNTIME)*/ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME) { +unsigned long hugepage_size;pid_t pid;uint64_t *ptr;int i;
@@ -288,11 +304,15 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME)if (self->nthreads <2 || self->n1 <0 || self->n2 <0 )SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,TWOMEG ,PROT_READ |PROT_WRITE , + hugepage_size =default_huge_page_size (); +if (!hugepage_size) +SKIP (return,"Reading HugeTLB pagesize failed\n" ); + + ptr =mmap (NULL , hugepage_size,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_ANONYMOUS |MAP_HUGETLB ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED ); -memset (ptr,0xde ,TWOMEG ); +memset (ptr,0xde , hugepage_size);for (i =0 ; i < self->nthreads -1 ; i++) { pid =fork ();if (!pid) {
Signature · 1 line
2.53.0
Re: [PATCH v3 02/54] selftests/mm: migration: don't assume hupe page is TWOMEG
Quoted 50 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
migration tests presume that both THP and HugeTLB huge pages are 2MB.
Add dynamic detection of huge page size with
read_pmd_pagesize()for THP and withdefault_huge_page_size()for HugeTLB.Reviewed-by Donet Tom <[email protected]>Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/migration.c | 44 +++++++++++++++++++------- 1 file changed, 32 insertions(+), 12 deletions(-)tools/testing/selftests/mm/migration.c +9
-4@@ -184,22 +184,27 @@ TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME)*/ TEST_F_TIMEOUT(migration, private_anon_thp, 2*RUNTIME) { +uint64_t pmdsize;uint64_t *ptr;int i;if (!thp_is_enabled ())SKIP (return,"Transparent Hugepages not available" ); + pmdsize =read_pmd_pagesize (); +if (!pmdsize) +SKIP (return,"Reading PMD pagesize failed" ); +if (self->nthreads <2 || self->n1 <0 || self->n2 <0 )SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,2 *TWOMEG ,PROT_READ |PROT_WRITE , + ptr =mmap (NULL ,2 * pmdsize,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED ); - ptr = (uint64_t *)ALIGN ((uintptr_t ) ptr,TWOMEG ); -ASSERT_EQ (madvise (ptr,TWOMEG ,MADV_HUGEPAGE ),0 ); -memset (ptr,0xde ,TWOMEG ); + ptr = (uint64_t *)ALIGN ((uintptr_t ) ptr, pmdsize); +ASSERT_EQ (madvise (ptr, pmdsize,MADV_HUGEPAGE ),0 ); +memset (ptr,0xde , pmdsize);for (i =0 ; i < self->nthreads -1 ; i++)if (pthread_create (&self->threads[i],NULL , access_mem, ptr))perror ("Couldn't create thread" );
tools/testing/selftests/mm/migration.c TEST_F_TIMEOUT(migration, private_anon_thp, 2*RUNTIME) · line 223
Quoted 4 lines
tools/testing/selftests/mm/migration.c +1
-0@@ -215,6 +220,7 @@ TEST_F_TIMEOUT(migration, private_anon_thp, 2*RUNTIME) TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME) { +unsigned long pmdsize;
I don't remember if this is the one I caught before, but would be good to match read_pmd_pagesize() uint64_t type.
As this is minor:
Quoted 80 lines
tools/testing/selftests/mm/migration.c +22
-8pid_t pid;uint64_t *ptr;int i;@@ -222,17 +228,21 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME)if (!thp_is_enabled ())SKIP (return,"Transparent Hugepages not available" ); + pmdsize =read_pmd_pagesize (); +if (!pmdsize) +SKIP (return,"Reading PMD pagesize failed" ); +if (self->nthreads <2 || self->n1 <0 || self->n2 <0 )SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,2 *TWOMEG ,PROT_READ |PROT_WRITE , + ptr =mmap (NULL ,2 * pmdsize,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED ); - ptr = (uint64_t *)ALIGN ((uintptr_t ) ptr,TWOMEG ); -ASSERT_EQ (madvise (ptr,TWOMEG ,MADV_HUGEPAGE ),0 ); + ptr = (uint64_t *)ALIGN ((uintptr_t ) ptr, pmdsize); +ASSERT_EQ (madvise (ptr, pmdsize,MADV_HUGEPAGE ),0 ); -memset (ptr,0xde ,TWOMEG ); +memset (ptr,0xde , pmdsize);for (i =0 ; i < self->nthreads -1 ; i++) { pid =fork ();if (!pid) {@@ -256,17 +266,22 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME)*/ TEST_F_TIMEOUT(migration, private_anon_htlb, 2*RUNTIME) { +unsigned long hugepage_size;uint64_t *ptr;int i;if (self->nthreads <2 || self->n1 <0 || self->n2 <0 )SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,TWOMEG ,PROT_READ |PROT_WRITE , + hugepage_size =default_huge_page_size (); +if (!hugepage_size) +SKIP (return,"Reading HugeTLB pagesize failed\n" ); + + ptr =mmap (NULL , hugepage_size,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED ); -memset (ptr,0xde ,TWOMEG ); +memset (ptr,0xde , hugepage_size);for (i =0 ; i < self->nthreads -1 ; i++)if (pthread_create (&self->threads[i],NULL , access_mem, ptr))perror ("Couldn't create thread" );@@ -281,6 +296,7 @@ TEST_F_TIMEOUT(migration, private_anon_htlb, 2*RUNTIME)*/ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME) { +unsigned long hugepage_size;pid_t pid;uint64_t *ptr;int i;@@ -288,11 +304,15 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME)if (self->nthreads <2 || self->n1 <0 || self->n2 <0 )SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,TWOMEG ,PROT_READ |PROT_WRITE , + hugepage_size =default_huge_page_size (); +if (!hugepage_size) +SKIP (return,"Reading HugeTLB pagesize failed\n" ); + + ptr =mmap (NULL , hugepage_size,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_ANONYMOUS |MAP_HUGETLB ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED ); -memset (ptr,0xde ,TWOMEG ); +memset (ptr,0xde , hugepage_size);for (i =0 ; i < self->nthreads -1 ; i++) { pid =fork ();if (!pid) {
Mike Rapoport
maintainerPATCH v3 3/54 selftests/mm: migration: make nthreads represent number of working threads
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Fixture setup sets self->nthreads to number of available CPUs minus 1 and then each test creates 'self->nthreads - 1' threads or processes, so essentially nthreads counts the worker tasks and the main task.
Make nthreads represent the number of spawned tasks to simplify thread/process creation and teardown.
While on it, make the fixture setup skip the tests if there are not enough CPUs or NUMA nodes instead of checking this in each test.
--- tools/testing/selftests/mm/migration.c | 47 +++++++++----------------- 1 file changed, 16 insertions(+), 31 deletions(-)
tools/testing/selftests/mm/migration.c +16 -31
@@ -38,7 +38,7 @@ FIXTURE_SETUP(migration)if (numa_available () <0 )SKIP (return,"NUMA not available" ); - self->nthreads =numa_num_task_cpus () -1 ; + self->nthreads =numa_num_task_cpus () -2 ; self->n1 =-1 ; self->n2 =-1 ;
@@ -52,6 +52,9 @@ FIXTURE_SETUP(migration) } } +if (self->nthreads <1 || self->n1 <0 || self->n2 <0 ) +SKIP (return,"Not enough threads or NUMA nodes available" ); + self->threads =malloc (self->nthreads *sizeof (*self->threads));ASSERT_NE (self->threads,NULL ); self->pids =malloc (self->nthreads *sizeof (*self->pids));
@@ -127,20 +130,17 @@ TEST_F_TIMEOUT(migration, private_anon, 2*RUNTIME)uint64_t *ptr;int i; -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,TWOMEG ,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );memset (ptr,0xde ,TWOMEG ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)if (pthread_create (&self->threads[i],NULL , access_mem, ptr))perror ("Couldn't create thread" );ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (pthread_cancel (self->threads[i]),0 ); }
@@ -153,15 +153,12 @@ TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME)uint64_t *ptr;int i; -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,TWOMEG ,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );memset (ptr,0xde ,TWOMEG ); -for (i =0 ; i < self->nthreads -1 ; i++) { +for (i =0 ; i < self->nthreads; i++) { pid =fork ();if (!pid) {prctl (PR_SET_PDEATHSIG ,SIGHUP );
@@ -175,7 +172,7 @@ TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME) }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); }
@@ -195,9 +192,6 @@ TEST_F_TIMEOUT(migration, private_anon_thp, 2*RUNTIME)if (!pmdsize)SKIP (return,"Reading PMD pagesize failed" ); -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,2 * pmdsize,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );
@@ -205,12 +199,12 @@ TEST_F_TIMEOUT(migration, private_anon_thp, 2*RUNTIME) ptr = (uint64_t *)ALIGN ((uintptr_t ) ptr, pmdsize);ASSERT_EQ (madvise (ptr, pmdsize,MADV_HUGEPAGE ),0 );memset (ptr,0xde , pmdsize); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)if (pthread_create (&self->threads[i],NULL , access_mem, ptr))perror ("Couldn't create thread" );ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (pthread_cancel (self->threads[i]),0 ); }
@@ -232,9 +226,6 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME)if (!pmdsize)SKIP (return,"Reading PMD pagesize failed" ); -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,2 * pmdsize,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );
@@ -243,7 +234,7 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME)ASSERT_EQ (madvise (ptr, pmdsize,MADV_HUGEPAGE ),0 );memset (ptr,0xde , pmdsize); -for (i =0 ; i < self->nthreads -1 ; i++) { +for (i =0 ; i < self->nthreads; i++) { pid =fork ();if (!pid) {prctl (PR_SET_PDEATHSIG ,SIGHUP );
@@ -257,7 +248,7 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME) }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); }
@@ -270,9 +261,6 @@ TEST_F_TIMEOUT(migration, private_anon_htlb, 2*RUNTIME)uint64_t *ptr;int i; -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - hugepage_size =default_huge_page_size ();if (!hugepage_size)SKIP (return,"Reading HugeTLB pagesize failed\n" );
@@ -282,12 +270,12 @@ TEST_F_TIMEOUT(migration, private_anon_htlb, 2*RUNTIME)ASSERT_NE (ptr,MAP_FAILED );memset (ptr,0xde , hugepage_size); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)if (pthread_create (&self->threads[i],NULL , access_mem, ptr))perror ("Couldn't create thread" );ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (pthread_cancel (self->threads[i]),0 ); }
@@ -301,9 +289,6 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME)uint64_t *ptr;int i; -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - hugepage_size =default_huge_page_size ();if (!hugepage_size)SKIP (return,"Reading HugeTLB pagesize failed\n" );
@@ -313,7 +298,7 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME)ASSERT_NE (ptr,MAP_FAILED );memset (ptr,0xde , hugepage_size); -for (i =0 ; i < self->nthreads -1 ; i++) { +for (i =0 ; i < self->nthreads; i++) { pid =fork ();if (!pid) {prctl (PR_SET_PDEATHSIG ,SIGHUP );
@@ -327,7 +312,7 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME) }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); }
Signature · 1 line
2.53.0
Re: [PATCH v3 03/54] selftests/mm: migration: make nthreads represent number of working threads
Quoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Fixture setup sets
self->nthreadsto number of available CPUs minus 1 and then each test creates 'self->nthreads- 1' threads or processes, so essentially nthreads counts the worker tasks and the main task.
Quoted 7 lines
Make nthreads represent the number of spawned tasks to simplify thread/process creation and teardown.
While on it, make the fixture setup skip the tests if there are not enough CPUs or NUMA nodes instead of checking this in each test.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Quoted 182 lines
--- tools/testing/selftests/mm/migration.c | 47 +++++++++----------------- 1 file changed, 16 insertions(+), 31 deletions(-)tools/testing/selftests/mm/migration.c +16
-31@@ -38,7 +38,7 @@ FIXTURE_SETUP(migration)if (numa_available () <0 )SKIP (return,"NUMA not available" ); - self->nthreads =numa_num_task_cpus () -1 ; + self->nthreads =numa_num_task_cpus () -2 ; self->n1 =-1 ; self->n2 =-1 ;@@ -52,6 +52,9 @@ FIXTURE_SETUP(migration) } } +if (self->nthreads <1 || self->n1 <0 || self->n2 <0 ) +SKIP (return,"Not enough threads or NUMA nodes available" ); + self->threads =malloc (self->nthreads *sizeof (*self->threads));ASSERT_NE (self->threads,NULL ); self->pids =malloc (self->nthreads *sizeof (*self->pids));@@ -127,20 +130,17 @@ TEST_F_TIMEOUT(migration, private_anon, 2*RUNTIME)uint64_t *ptr;int i; -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,TWOMEG ,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );memset (ptr,0xde ,TWOMEG ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)if (pthread_create (&self->threads[i],NULL , access_mem, ptr))perror ("Couldn't create thread" );ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (pthread_cancel (self->threads[i]),0 ); }@@ -153,15 +153,12 @@ TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME)uint64_t *ptr;int i; -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,TWOMEG ,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );memset (ptr,0xde ,TWOMEG ); -for (i =0 ; i < self->nthreads -1 ; i++) { +for (i =0 ; i < self->nthreads; i++) { pid =fork ();if (!pid) {prctl (PR_SET_PDEATHSIG ,SIGHUP );@@ -175,7 +172,7 @@ TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME) }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); }@@ -195,9 +192,6 @@ TEST_F_TIMEOUT(migration, private_anon_thp, 2*RUNTIME)if (!pmdsize)SKIP (return,"Reading PMD pagesize failed" ); -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,2 * pmdsize,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );@@ -205,12 +199,12 @@ TEST_F_TIMEOUT(migration, private_anon_thp, 2*RUNTIME) ptr = (uint64_t *)ALIGN ((uintptr_t ) ptr, pmdsize);ASSERT_EQ (madvise (ptr, pmdsize,MADV_HUGEPAGE ),0 );memset (ptr,0xde , pmdsize); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)if (pthread_create (&self->threads[i],NULL , access_mem, ptr))perror ("Couldn't create thread" );ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (pthread_cancel (self->threads[i]),0 ); }@@ -232,9 +226,6 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME)if (!pmdsize)SKIP (return,"Reading PMD pagesize failed" ); -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - ptr =mmap (NULL ,2 * pmdsize,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_ANONYMOUS ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );@@ -243,7 +234,7 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME)ASSERT_EQ (madvise (ptr, pmdsize,MADV_HUGEPAGE ),0 );memset (ptr,0xde , pmdsize); -for (i =0 ; i < self->nthreads -1 ; i++) { +for (i =0 ; i < self->nthreads; i++) { pid =fork ();if (!pid) {prctl (PR_SET_PDEATHSIG ,SIGHUP );@@ -257,7 +248,7 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME) }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); }@@ -270,9 +261,6 @@ TEST_F_TIMEOUT(migration, private_anon_htlb, 2*RUNTIME)uint64_t *ptr;int i; -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - hugepage_size =default_huge_page_size ();if (!hugepage_size)SKIP (return,"Reading HugeTLB pagesize failed\n" );@@ -282,12 +270,12 @@ TEST_F_TIMEOUT(migration, private_anon_htlb, 2*RUNTIME)ASSERT_NE (ptr,MAP_FAILED );memset (ptr,0xde , hugepage_size); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)if (pthread_create (&self->threads[i],NULL , access_mem, ptr))perror ("Couldn't create thread" );ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (pthread_cancel (self->threads[i]),0 ); }@@ -301,9 +289,6 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME)uint64_t *ptr;int i; -if (self->nthreads <2 || self->n1 <0 || self->n2 <0 ) -SKIP (return,"Not enough threads or NUMA nodes available" ); - hugepage_size =default_huge_page_size ();if (!hugepage_size)SKIP (return,"Reading HugeTLB pagesize failed\n" );@@ -313,7 +298,7 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME)ASSERT_NE (ptr,MAP_FAILED );memset (ptr,0xde , hugepage_size); -for (i =0 ; i < self->nthreads -1 ; i++) { +for (i =0 ; i < self->nthreads; i++) { pid =fork ();if (!pid) {prctl (PR_SET_PDEATHSIG ,SIGHUP );@@ -327,7 +312,7 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME) }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads -1 ; i++) +for (i =0 ; i < self->nthreads; i++)ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); }
Mike Rapoport
maintainerPATCH v3 4/54 selftests/mm: migration: properly cleanup fork()ed processes
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Several migration test use fork() to create worker processes. These processes are later killed, but nothing collects their exit status and they remain as zombies in the system.
Add a helper function that kills the worker processes, waitpid()s for them and verifies the exit status.
Replace the loops that call kill() for each process with a call to that helper.
The addition of waitpid() calls also makes sure the migrating process never exits before the processes accessing the memory, so it's possible to drop the call to prctl(PR_SET_PDEATHSIG, SIGHUP).
--- tools/testing/selftests/mm/migration.c | 54 +++++++++++++------------- 1 file changed, 27 insertions(+), 27 deletions(-)
tools/testing/selftests/mm/migration.c +27 -27
@@ -67,6 +67,24 @@ FIXTURE_TEARDOWN(migration)free (self->pids); } +static bool kill_children (FIXTURE_DATA (migration) * self) +{ +int i, status; +pid_t pid; + +for (i =0 ; i < self->nthreads; i++) { + pid = self->pids[i]; +if (kill (pid,SIGTERM )) +return false; +if (pid !=waitpid (pid, &status,0 )) +return false; +if (!WIFSIGNALED (status) ||WTERMSIG (status) !=SIGTERM ) +return false; + } + +return true; +} +int migrate (uint64_t *ptr,int n1,int n2) {int ret, tmp;
@@ -160,20 +178,14 @@ TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME)memset (ptr,0xde ,TWOMEG );for (i =0 ; i < self->nthreads; i++) { pid =fork (); -if (!pid) { -prctl (PR_SET_PDEATHSIG ,SIGHUP ); -/* Parent may have died before prctl so check now. */ -if (getppid () ==1 ) -kill (getpid (),SIGHUP ); +if (!pid)access_mem (ptr); - }else { +else self->pids[i] = pid; - } }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads; i++) -ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); +ASSERT_EQ (kill_children (self), true); }/*
@@ -236,20 +248,14 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME)memset (ptr,0xde , pmdsize);for (i =0 ; i < self->nthreads; i++) { pid =fork (); -if (!pid) { -prctl (PR_SET_PDEATHSIG ,SIGHUP ); -/* Parent may have died before prctl so check now. */ -if (getppid () ==1 ) -kill (getpid (),SIGHUP ); +if (!pid)access_mem (ptr); - }else { +else self->pids[i] = pid; - } }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads; i++) -ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); +ASSERT_EQ (kill_children (self), true); }/*
@@ -300,20 +306,14 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME)memset (ptr,0xde , hugepage_size);for (i =0 ; i < self->nthreads; i++) { pid =fork (); -if (!pid) { -prctl (PR_SET_PDEATHSIG ,SIGHUP ); -/* Parent may have died before prctl so check now. */ -if (getppid () ==1 ) -kill (getpid (),SIGHUP ); +if (!pid)access_mem (ptr); - }else { +else self->pids[i] = pid; - } }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads; i++) -ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); +ASSERT_EQ (kill_children (self), true); }TEST_HARNESS_MAIN
Signature · 1 line
2.53.0
Re: [PATCH v3 04/54] selftests/mm: migration: properly cleanup fork()ed processes
Quoted 34 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Several migration test use
fork()to create worker processes. These processes are later killed, but nothing collects their exit status and they remain as zombies in the system.Add a helper function that kills the worker processes, waitpid()s for them and verifies the exit status.
Replace the loops that call
kill()for each process with a call to that helper.The addition of
waitpid()calls also makes sure the migrating process never exits before the processes accessing the memory, so it's possible to drop the call toprctl(PR_SET_PDEATHSIG, SIGHUP).Reported-by Luiz Capitulino <[email protected]>Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/migration.c | 54 +++++++++++++------------- 1 file changed, 27 insertions(+), 27 deletions(-)tools/testing/selftests/mm/migration.c +4
-0@@ -67,6 +67,24 @@ FIXTURE_TEARDOWN(migration)free (self->pids); } +static bool kill_children (FIXTURE_DATA (migration) * self) +{ +int i, status; +pid_t pid;
tools/testing/selftests/mm/migration.c FIXTURE_TEARDOWN(migration) · line 83
Quoted 10 lines
tools/testing/selftests/mm/migration.c +10
-0+ +for (i =0 ; i < self->nthreads; i++) { + pid = self->pids[i]; +if (kill (pid,SIGTERM )) +return false; +if (pid !=waitpid (pid, &status,0 )) +return false; +if (!WIFSIGNALED (status) ||WTERMSIG (status) !=SIGTERM ) +return false; + }
Why do we stop at the first error? Wouldn't it be better to kill & collect all of them and then return false if at least one fails?
Quoted 8 lines
tools/testing/selftests/mm/migration.c +4
-0+ +return true; +} +int migrate (uint64_t *ptr,int n1,int n2) {int ret, tmp;
tools/testing/selftests/mm/migration.c TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME) · line 167
Quoted 8 lines
tools/testing/selftests/mm/migration.c +0
-5@@ -160,20 +178,14 @@ TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME)memset (ptr,0xde ,TWOMEG );for (i =0 ; i < self->nthreads; i++) { pid =fork (); -if (!pid) { -prctl (PR_SET_PDEATHSIG ,SIGHUP ); -/* Parent may have died before prctl so check now. */ -if (getppid () ==1 ) -kill (getpid (),SIGHUP );
This goes beyond this series, but I wonder if we should have this check in access_mem() in case the parent dies in migrate().
Quoted 63 lines
tools/testing/selftests/mm/migration.c +9
-22+if (!pid)access_mem (ptr); - }else { +else self->pids[i] = pid; - } }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads; i++) -ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); +ASSERT_EQ (kill_children (self), true); }/* @@ -236,20 +248,14 @@ TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME)memset (ptr,0xde , pmdsize);for (i =0 ; i < self->nthreads; i++) { pid =fork (); -if (!pid) { -prctl (PR_SET_PDEATHSIG ,SIGHUP ); -/* Parent may have died before prctl so check now. */ -if (getppid () ==1 ) -kill (getpid (),SIGHUP ); +if (!pid)access_mem (ptr); - }else { +else self->pids[i] = pid; - } }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads; i++) -ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); +ASSERT_EQ (kill_children (self), true); }/* @@ -300,20 +306,14 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME)memset (ptr,0xde , hugepage_size);for (i =0 ; i < self->nthreads; i++) { pid =fork (); -if (!pid) { -prctl (PR_SET_PDEATHSIG ,SIGHUP ); -/* Parent may have died before prctl so check now. */ -if (getppid () ==1 ) -kill (getpid (),SIGHUP ); +if (!pid)access_mem (ptr); - }else { +else self->pids[i] = pid; - } }ASSERT_EQ (migrate (ptr, self->n1, self->n2),0 ); -for (i =0 ; i < self->nthreads; i++) -ASSERT_EQ (kill (self->pids[i],SIGTERM ),0 ); +ASSERT_EQ (kill_children (self), true); }TEST_HARNESS_MAIN
Mike Rapoport
maintainerRe: [PATCH v3 04/54] selftests/mm: migration: properly cleanup fork()ed processes
Quoted 39 lines · Luiz Capitulino
Quoted 38 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Several migration test use
fork()to create worker processes. These processes are later killed, but nothing collects their exit status and they remain as zombies in the system.Add a helper function that kills the worker processes, waitpid()s for them and verifies the exit status.
Replace the loops that call
kill()for each process with a call to that helper.The addition of
waitpid()calls also makes sure the migrating process never exits before the processes accessing the memory, so it's possible to drop the call toprctl(PR_SET_PDEATHSIG, SIGHUP).Reported-by Luiz Capitulino <[email protected]>Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/migration.c | 54 +++++++++++++------------- 1 file changed, 27 insertions(+), 27 deletions(-)tools/testing/selftests/mm/migration.c +9
-0@@ -67,6 +67,24 @@ FIXTURE_TEARDOWN(migration)free (self->pids); } +static bool kill_children (FIXTURE_DATA (migration) * self) +{ +int i, status; +pid_t pid; + +for (i =0 ; i < self->nthreads; i++) { + pid = self->pids[i]; +if (kill (pid,SIGTERM )) +return false;
Quoted 8 lines
tools/testing/selftests/mm/migration.c FIXTURE_TEARDOWN(migration) · line 83
Quoted 5 lines
tools/testing/selftests/mm/migration.c +5
-0+if (pid !=waitpid (pid, &status,0 )) +return false; +if (!WIFSIGNALED (status) ||WTERMSIG (status) !=SIGTERM ) +return false; + }Why do we stop at the first error? Wouldn't it be better to kill & collect all of them and then return false if at least one fails?
Yep, makes sense.
Quoted 11 lines
Quoted 11 lines
tools/testing/selftests/mm/migration.c +4
-0+ +return true; +} +int migrate (uint64_t *ptr,int n1,int n2) {int ret, tmp;@@ -160,20 +178,14 @@ TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME)memset (ptr,0xde ,TWOMEG );for (i =0 ; i < self->nthreads; i++) { pid =fork ();
Quoted 8 lines
tools/testing/selftests/mm/migration.c TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME) · line 167
Quoted 5 lines
tools/testing/selftests/mm/migration.c +0
-5-if (!pid) { -prctl (PR_SET_PDEATHSIG ,SIGHUP ); -/* Parent may have died before prctl so check now. */ -if (getppid () ==1 ) -kill (getpid (),SIGHUP );This goes beyond this series, but I wonder if we should have this check in
access_mem()in case the parent dies inmigrate().
With added kill() + waitpid() this should not happen.
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v3 04/54] selftests/mm: migration: properly cleanup fork()ed processes
Quoted 71 lines · Mike Rapoport
Quoted 39 lines · Luiz Capitulino
Quoted 38 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Several migration test use
fork()to create worker processes. These processes are later killed, but nothing collects their exit status and they remain as zombies in the system.Add a helper function that kills the worker processes, waitpid()s for them and verifies the exit status.
Replace the loops that call
kill()for each process with a call to that helper.The addition of
waitpid()calls also makes sure the migrating process never exits before the processes accessing the memory, so it's possible to drop the call toprctl(PR_SET_PDEATHSIG, SIGHUP).Reported-by Luiz Capitulino <[email protected]>Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/migration.c | 54 +++++++++++++------------- 1 file changed, 27 insertions(+), 27 deletions(-)tools/testing/selftests/mm/migration.c +9
-0@@ -67,6 +67,24 @@ FIXTURE_TEARDOWN(migration)free (self->pids); } +static bool kill_children (FIXTURE_DATA (migration) * self) +{ +int i, status; +pid_t pid; + +for (i =0 ; i < self->nthreads; i++) { + pid = self->pids[i]; +if (kill (pid,SIGTERM )) +return false;Quoted 8 lines
tools/testing/selftests/mm/migration.c FIXTURE_TEARDOWN(migration) · line 83
Quoted 5 lines
tools/testing/selftests/mm/migration.c +5
-0+if (pid !=waitpid (pid, &status,0 )) +return false; +if (!WIFSIGNALED (status) ||WTERMSIG (status) !=SIGTERM ) +return false; + }Why do we stop at the first error? Wouldn't it be better to kill & collect all of them and then return false if at least one fails?
Yep, makes sense.
Quoted 11 lines
Quoted 11 lines
tools/testing/selftests/mm/migration.c +4
-0+ +return true; +} +int migrate (uint64_t *ptr,int n1,int n2) {int ret, tmp;@@ -160,20 +178,14 @@ TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME)memset (ptr,0xde ,TWOMEG );for (i =0 ; i < self->nthreads; i++) { pid =fork ();Quoted 8 lines
tools/testing/selftests/mm/migration.c TEST_F_TIMEOUT(migration, shared_anon, 2*RUNTIME) · line 167
Quoted 5 lines
tools/testing/selftests/mm/migration.c +0
-5-if (!pid) { -prctl (PR_SET_PDEATHSIG ,SIGHUP ); -/* Parent may have died before prctl so check now. */ -if (getppid () ==1 ) -kill (getpid (),SIGHUP );This goes beyond this series, but I wonder if we should have this check in
access_mem()in case the parent dies inmigrate().
Quoted 1 line
With added
kill()+waitpid()this should not happen.
I was thinking in the case of a crash, but looking at it again it seems to be impossible so it's fine.
Mike Rapoport
maintainerPATCH v3 5/54 selftests/mm: run_vmtests.sh: don't gate THP and KSM tests on HAVE_HUGEPAGES
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
HAVE_HUGEPAGES indicates availability of free HugeTLB pages. It should not be used to gate KSM test that merges transparent huge pages or split_huge_page_test.
Remove check for HAVE_HUGEPAGES when running these tests.
--- tools/testing/selftests/mm/run_vmtests.sh | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-)
tools/testing/selftests/mm/run_vmtests.sh +9 -13
@@ -436,9 +436,7 @@ CATEGORY="memfd_secret" run_test ./memfd_secret fi # KSM KSM_MERGE_TIME_HUGE_PAGES test with size of 100 -if [ "${HAVE_HUGEPAGES}" = "1" ]; then - CATEGORY="ksm" run_test ./ksm_tests -H -s 100 -fi +CATEGORY="ksm" run_test ./ksm_tests -H -s 100 # KSM KSM_MERGE_TIME test with size of 100 CATEGORY="ksm" run_test ./ksm_tests -P -s 100 # KSM MADV_MERGEABLE test with 10 identical pages
@@ -493,16 +491,14 @@ CATEGORY="thp" run_test ./transhuge-stress -d 20 # Try to create XFS if not provided if [ -z "${SPLIT_HUGE_PAGE_TEST_XFS_PATH}" ]; then - if [ "${HAVE_HUGEPAGES}" = "1" ]; then - if test_selected "thp"; then - if grep xfs /proc/filesystems &>/dev/null; then - XFS_IMG=$(mktemp /tmp/xfs_img_XXXXXX) - SPLIT_HUGE_PAGE_TEST_XFS_PATH=$(mktemp -d /tmp/xfs_dir_XXXXXX) - truncate -s 314572800 ${XFS_IMG} - mkfs.xfs -q ${XFS_IMG} - mount -o loop ${XFS_IMG} ${SPLIT_HUGE_PAGE_TEST_XFS_PATH} - MOUNTED_XFS=1 - fi + if test_selected "thp"; then + if grep xfs /proc/filesystems &>/dev/null; then + XFS_IMG=$(mktemp /tmp/xfs_img_XXXXXX) + SPLIT_HUGE_PAGE_TEST_XFS_PATH=$(mktemp -d /tmp/xfs_dir_XXXXXX) + truncate -s 314572800 ${XFS_IMG} + mkfs.xfs -q ${XFS_IMG} + mount -o loop ${XFS_IMG} ${SPLIT_HUGE_PAGE_TEST_XFS_PATH} + MOUNTED_XFS=1 fi fi fi
Signature · 1 line
2.53.0
Re: [PATCH v3 05/54] selftests/mm: run_vmtests.sh: don't gate THP and KSM tests on HAVE_HUGEPAGES
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
HAVE_HUGEPAGESindicates availability of free HugeTLB pages. It should not be used to gate KSM test that merges transparent huge pages orsplit_huge_page_test.Remove check for
HAVE_HUGEPAGESwhen running these tests.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Signature · 2 lines
Regards, Li Wang
Mike Rapoport
maintainerPATCH v3 6/54 selftests/mm: merge map_hugetlb into hugepage-mmap
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Both tests create a hugettlb mapping, fill it with data and verify the data, the only difference is that one uses file-backed memory and another one uses anonymous memory.
Merge both tests into a single file.
--- tools/testing/selftests/mm/Makefile | 1 - tools/testing/selftests/mm/hugepage-mmap.c | 111 ++++++++++++++++----- tools/testing/selftests/mm/map_hugetlb.c | 88 ---------------- tools/testing/selftests/mm/run_vmtests.sh | 1 - 4 files changed, 86 insertions(+), 115 deletions(-) delete mode 100644 tools/testing/selftests/mm/map_hugetlb.c
tools/testing/selftests/mm/Makefile +0 -1
@@ -70,7 +70,6 @@ TEST_GEN_FILES += hugepage-vmemmap TEST_GEN_FILES += khugepaged TEST_GEN_FILES += madv_populate TEST_GEN_FILES += map_fixed_noreplace -TEST_GEN_FILES += map_hugetlb TEST_GEN_FILES += map_populate ifneq (,$(filter $(ARCH),arm64 riscv riscv64 x86 x86_64 loongarch32 loongarch64)) TEST_GEN_FILES += memfd_secret
tools/testing/selftests/mm/hugepage-mmap.c +86 -25
146 lines not on this page · raw message
tools/testing/selftests/mm/map_hugetlb.c deleted +0 -88
88 lines not on this page · raw message
tools/testing/selftests/mm/run_vmtests.sh +0 -1
@@ -292,7 +292,6 @@ CATEGORY="hugetlb" run_test ./hugepage-shm echo "$shmmax" > /proc/sys/kernel/shmmax echo "$shmall" > /proc/sys/kernel/shmall -CATEGORY="hugetlb" run_test ./map_hugetlb CATEGORY="hugetlb" run_test ./hugepage-mremap CATEGORY="hugetlb" run_test ./hugepage-vmemmap CATEGORY="hugetlb" run_test ./hugetlb-madvise
Signature · 1 line
2.53.0
Re: [PATCH v3 06/54] selftests/mm: merge map_hugetlb into hugepage-mmap
Hi Mike
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Both tests create a hugettlb mapping, fill it with data and verify the data, the only difference is that one uses file-backed memory and another one uses anonymous memory.
Merge both tests into a single file.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
[]# ./hugetlb-mmap 1024 21
TAP version 13
# [INFO] detected hugetlb page size: 2048 KiB # [INFO] detected hugetlb page size: 1048576 KiB # 2048 kB hugepages
1..2
# Mapping 1024 Mbytes
# Returned address is 0x7fff5fa00000
# First hex is 0
# First hex is 3020100
ok 1 hugetlb anonymous mmap
# Returned address is 0x7fff5fa00000
# First hex is 0
# First hex is 3020100
ok 2 hugetlb file mmap
# Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0
[]# ./hugetlb-mmap 1024 30
TAP version 13
# [INFO] detected hugetlb page size: 2048 KiB # [INFO] detected hugetlb page size: 1048576 KiB # 1048576 kB hugepages
1..2
# Mapping 1024 Mbytes
# Returned address is 0x7fff40000000
# First hex is 0
# First hex is 3020100
ok 1 hugetlb anonymous mmap
# Returned address is 0x7fff40000000
# First hex is 0
# First hex is 3020100
ok 2 hugetlb file mmap
# Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0 []#
looks good to me.
Quoted 278 lines
--- tools/testing/selftests/mm/Makefile | 1 - tools/testing/selftests/mm/hugepage-mmap.c | 111 ++++++++++++++++----- tools/testing/selftests/mm/map_hugetlb.c | 88 ---------------- tools/testing/selftests/mm/run_vmtests.sh | 1 - 4 files changed, 86 insertions(+), 115 deletions(-)delete mode 100644
tools/testing/selftests/mm/map_hugetlb.ctools/testing/selftests/mm/Makefile +0
-1@@ -70,7 +70,6 @@ TEST_GEN_FILES += hugepage-vmemmap TEST_GEN_FILES += khugepaged TEST_GEN_FILES += madv_populate TEST_GEN_FILES += map_fixed_noreplace -TEST_GEN_FILES += map_hugetlb TEST_GEN_FILES += map_populate ifneq (,$(filter $(ARCH),arm64 riscv riscv64 x86 x86_64 loongarch32 loongarch64)) TEST_GEN_FILES += memfd_secrettools/testing/selftests/mm/hugepage-mmap.c +86
-25@@ -15,6 +15,8 @@#include <unistd.h> #include <sys/mman.h> #include <fcntl.h> +#include <linux/memfd.h> +#include "vm_util.h" #include "kselftest.h" #define LENGTH (256UL*1024*1024)@@ -25,54 +27,113 @@ static void check_bytes(char *addr)ksft_print_msg ("First hex is %x\n" , *((unsigned int *)addr)); } -static void write_bytes (char *addr) +static void write_bytes (char *addr,size_t length) {unsigned long i; -for (i =0 ; i <LENGTH ; i++) +for (i =0 ; i < length; i++) *(addr + i) = (char )i; } -static int read_bytes (char *addr) +static bool verify_bytes (char *addr,size_t length) {unsigned long i;check_bytes (addr); -for (i =0 ; i <LENGTH ; i++) +for (i =0 ; i < length; i++)if (*(addr + i) != (char )i) { -ksft_print_msg ("Error: Mismatch at %lu\n" , i); -return 1 ; +ksft_print_msg ("Error: Mismatch at %lu(%p)\n" , i, addr); +return false; } -return 0 ; + +return true; } -int main (void ) +static bool test_mmap (size_t length,int mmap_flags,int fd, +const char *test_name) {void *addr; -int fd, ret; -ksft_print_header (); -ksft_set_plan (1 ); + addr =mmap (NULL , length,PROTECTION , mmap_flags, fd,0 ); +if (addr ==MAP_FAILED ) +ksft_exit_fail_perror ("mmap" ); + +ksft_print_msg ("Returned address is %p\n" , addr); +check_bytes (addr); +write_bytes (addr, length); +if (!verify_bytes (addr, length)) +ksft_exit_fail_msg ("%s\n" , test_name); + +/* munmap() length of MAP_HUGETLB memory must be hugepage aligned */ +if (munmap (addr, length)) +ksft_exit_fail_perror ("munmap" ); + +return true; +} + +static void test_anon_mmap (size_t length,int shift) +{ +const char *test_name ="hugetlb anonymous mmap" ; +int mmap_flags =MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB ; +bool passed; + +if (shift) + mmap_flags |= (shift &MAP_HUGE_MASK ) <<MAP_HUGE_SHIFT ; + + passed =test_mmap (length, mmap_flags,-1 , test_name); +ksft_test_result (passed,"%s\n" , test_name); +} + +static void test_file_mmap (size_t length,int shift) +{ +const char *test_name ="hugetlb file mmap" ; +int mfd_flags =MFD_HUGETLB ; +bool passed; +int fd; + +if (shift) + mfd_flags |= (shift &MFD_HUGE_MASK ) <<MFD_HUGE_SHIFT ; - fd =memfd_create ("hugepage-mmap" ,MFD_HUGETLB ); + fd =memfd_create ("hugetlb-mmap" , mfd_flags);if (fd <0 ) -ksft_exit_fail_msg ("memfd_create() failed: %s\n" ,strerror (errno)); +ksft_exit_fail_perror ("memfd_create" ); + + passed =test_mmap (length,MAP_SHARED , fd, test_name); + +close (fd); +ksft_test_result (passed,"%s\n" , test_name); +} + +int main (int argc,char **argv) +{ +size_t hugepage_size; +size_t length =LENGTH ; +int shift =0 ; + +ksft_print_header (); +ksft_set_plan (2 ); + +if (argc >1 ) + length =atol (argv[1 ]) <<20 ; +if (argc >2 ) + shift =atoi (argv[2 ]); - addr =mmap (NULL ,LENGTH ,PROTECTION ,MAP_SHARED , fd,0 ); -if (addr ==MAP_FAILED ) { -close (fd); -ksft_exit_fail_msg ("mmap(): %s\n" ,strerror (errno)); +if (shift) { + hugepage_size = (1 << shift); +ksft_print_msg ("%u kB hugepages\n" ,1 << (shift -10 )); + }else { + hugepage_size =default_huge_page_size (); +ksft_print_msg ("Default size hugepages (%lu Kb)\n" , hugepage_size >>10 ); } -ksft_print_msg ("Returned address is %p\n" , addr); -check_bytes (addr); -write_bytes (addr); - ret =read_bytes (addr); +/* munmap with fail if the length is not page aligned */ +if (hugepage_size > length) + length = hugepage_size; -munmap (addr,LENGTH ); -close (fd); +ksft_print_msg ("Mapping %lu Mbytes\n" , (unsigned long )length >>20 ); -ksft_test_result (!ret,"Read same data\n" ); +test_anon_mmap (length, shift); +test_file_mmap (length, shift); -ksft_exit (!ret); +ksft_finished (); }tools/testing/selftests/mm/map_hugetlb.c deleted +0
-88@@ -1,88 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* -* Example of using hugepage memory in a user application using the mmap -* system call with MAP_HUGETLB flag. Before running this program make -* sure the administrator has allocated enough default sized huge pages -* to cover the 256 MB allocation. -*/ -#include <stdlib.h> -#include <stdio.h> -#include <unistd.h> -#include <sys/mman.h> -#include <fcntl.h> -#include "vm_util.h" -#include "kselftest.h" - -#define LENGTH (256UL*1024*1024) -#define PROTECTION (PROT_READ | PROT_WRITE) - -static void check_bytes (char *addr) -{ -ksft_print_msg ("First hex is %x\n" , *((unsigned int *)addr)); -} - -static void write_bytes (char *addr,size_t length) -{ -unsigned long i; - -for (i =0 ; i < length; i++) - *(addr + i) = (char )i; -} - -static void read_bytes (char *addr,size_t length) -{ -unsigned long i; - -check_bytes (addr); -for (i =0 ; i < length; i++) -if (*(addr + i) != (char )i) -ksft_exit_fail_msg ("Mismatch at %lu\n" , i); - -ksft_test_result_pass ("Read correct data\n" ); -} - -int main (int argc,char **argv) -{ -void *addr; -size_t hugepage_size; -size_t length =LENGTH ; -int flags =MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB ; -int shift =0 ; - - hugepage_size =default_huge_page_size (); -/* munmap with fail if the length is not page aligned */ -if (hugepage_size > length) - length = hugepage_size; - -ksft_print_header (); -ksft_set_plan (1 ); - -if (argc >1 ) - length =atol (argv[1 ]) <<20 ; -if (argc >2 ) { - shift =atoi (argv[2 ]); -if (shift) - flags |= (shift &MAP_HUGE_MASK ) <<MAP_HUGE_SHIFT ; - } - -if (shift) -ksft_print_msg ("%u kB hugepages\n" ,1 << (shift -10 )); -else -ksft_print_msg ("Default size hugepages\n" ); -ksft_print_msg ("Mapping %lu Mbytes\n" , (unsigned long )length >>20 ); - - addr =mmap (NULL , length,PROTECTION , flags,-1 ,0 ); -if (addr ==MAP_FAILED ) -ksft_exit_fail_msg ("mmap: %s\n" ,strerror (errno)); - -ksft_print_msg ("Returned address is %p\n" , addr); -check_bytes (addr); -write_bytes (addr, length); -read_bytes (addr, length); - -/* munmap() length of MAP_HUGETLB memory must be hugepage aligned */ -if (munmap (addr, length)) -ksft_exit_fail_msg ("munmap: %s\n" ,strerror (errno)); - -ksft_finished (); -}tools/testing/selftests/mm/run_vmtests.sh +0
-1@@ -292,7 +292,6 @@ CATEGORY="hugetlb" run_test ./hugepage-shm echo "$shmmax" > /proc/sys/kernel/shmmax echo "$shmall" > /proc/sys/kernel/shmall -CATEGORY="hugetlb" run_test ./map_hugetlb CATEGORY="hugetlb" run_test ./hugepage-mremap CATEGORY="hugetlb" run_test ./hugepage-vmemmap CATEGORY="hugetlb" run_test ./hugetlb-madvise
Re: [PATCH v3 06/54] selftests/mm: merge map_hugetlb into hugepage-mmap
In reply to Mike RapoportQuoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Both tests create a hugettlb mapping, fill it with data and verify the data, the only difference is that one uses file-backed memory and another one uses anonymous memory.
Merge both tests into a single file.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Quoted 278 lines
--- tools/testing/selftests/mm/Makefile | 1 - tools/testing/selftests/mm/hugepage-mmap.c | 111 ++++++++++++++++----- tools/testing/selftests/mm/map_hugetlb.c | 88 ---------------- tools/testing/selftests/mm/run_vmtests.sh | 1 - 4 files changed, 86 insertions(+), 115 deletions(-)delete mode 100644
tools/testing/selftests/mm/map_hugetlb.ctools/testing/selftests/mm/Makefile +0
-1@@ -70,7 +70,6 @@ TEST_GEN_FILES += hugepage-vmemmap TEST_GEN_FILES += khugepaged TEST_GEN_FILES += madv_populate TEST_GEN_FILES += map_fixed_noreplace -TEST_GEN_FILES += map_hugetlb TEST_GEN_FILES += map_populate ifneq (,$(filter $(ARCH),arm64 riscv riscv64 x86 x86_64 loongarch32 loongarch64)) TEST_GEN_FILES += memfd_secrettools/testing/selftests/mm/hugepage-mmap.c +86
-25@@ -15,6 +15,8 @@#include <unistd.h> #include <sys/mman.h> #include <fcntl.h> +#include <linux/memfd.h> +#include "vm_util.h" #include "kselftest.h" #define LENGTH (256UL*1024*1024)@@ -25,54 +27,113 @@ static void check_bytes(char *addr)ksft_print_msg ("First hex is %x\n" , *((unsigned int *)addr)); } -static void write_bytes (char *addr) +static void write_bytes (char *addr,size_t length) {unsigned long i; -for (i =0 ; i <LENGTH ; i++) +for (i =0 ; i < length; i++) *(addr + i) = (char )i; } -static int read_bytes (char *addr) +static bool verify_bytes (char *addr,size_t length) {unsigned long i;check_bytes (addr); -for (i =0 ; i <LENGTH ; i++) +for (i =0 ; i < length; i++)if (*(addr + i) != (char )i) { -ksft_print_msg ("Error: Mismatch at %lu\n" , i); -return 1 ; +ksft_print_msg ("Error: Mismatch at %lu(%p)\n" , i, addr); +return false; } -return 0 ; + +return true; } -int main (void ) +static bool test_mmap (size_t length,int mmap_flags,int fd, +const char *test_name) {void *addr; -int fd, ret; -ksft_print_header (); -ksft_set_plan (1 ); + addr =mmap (NULL , length,PROTECTION , mmap_flags, fd,0 ); +if (addr ==MAP_FAILED ) +ksft_exit_fail_perror ("mmap" ); + +ksft_print_msg ("Returned address is %p\n" , addr); +check_bytes (addr); +write_bytes (addr, length); +if (!verify_bytes (addr, length)) +ksft_exit_fail_msg ("%s\n" , test_name); + +/* munmap() length of MAP_HUGETLB memory must be hugepage aligned */ +if (munmap (addr, length)) +ksft_exit_fail_perror ("munmap" ); + +return true; +} + +static void test_anon_mmap (size_t length,int shift) +{ +const char *test_name ="hugetlb anonymous mmap" ; +int mmap_flags =MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB ; +bool passed; + +if (shift) + mmap_flags |= (shift &MAP_HUGE_MASK ) <<MAP_HUGE_SHIFT ; + + passed =test_mmap (length, mmap_flags,-1 , test_name); +ksft_test_result (passed,"%s\n" , test_name); +} + +static void test_file_mmap (size_t length,int shift) +{ +const char *test_name ="hugetlb file mmap" ; +int mfd_flags =MFD_HUGETLB ; +bool passed; +int fd; + +if (shift) + mfd_flags |= (shift &MFD_HUGE_MASK ) <<MFD_HUGE_SHIFT ; - fd =memfd_create ("hugepage-mmap" ,MFD_HUGETLB ); + fd =memfd_create ("hugetlb-mmap" , mfd_flags);if (fd <0 ) -ksft_exit_fail_msg ("memfd_create() failed: %s\n" ,strerror (errno)); +ksft_exit_fail_perror ("memfd_create" ); + + passed =test_mmap (length,MAP_SHARED , fd, test_name); + +close (fd); +ksft_test_result (passed,"%s\n" , test_name); +} + +int main (int argc,char **argv) +{ +size_t hugepage_size; +size_t length =LENGTH ; +int shift =0 ; + +ksft_print_header (); +ksft_set_plan (2 ); + +if (argc >1 ) + length =atol (argv[1 ]) <<20 ; +if (argc >2 ) + shift =atoi (argv[2 ]); - addr =mmap (NULL ,LENGTH ,PROTECTION ,MAP_SHARED , fd,0 ); -if (addr ==MAP_FAILED ) { -close (fd); -ksft_exit_fail_msg ("mmap(): %s\n" ,strerror (errno)); +if (shift) { + hugepage_size = (1 << shift); +ksft_print_msg ("%u kB hugepages\n" ,1 << (shift -10 )); + }else { + hugepage_size =default_huge_page_size (); +ksft_print_msg ("Default size hugepages (%lu Kb)\n" , hugepage_size >>10 ); } -ksft_print_msg ("Returned address is %p\n" , addr); -check_bytes (addr); -write_bytes (addr); - ret =read_bytes (addr); +/* munmap with fail if the length is not page aligned */ +if (hugepage_size > length) + length = hugepage_size; -munmap (addr,LENGTH ); -close (fd); +ksft_print_msg ("Mapping %lu Mbytes\n" , (unsigned long )length >>20 ); -ksft_test_result (!ret,"Read same data\n" ); +test_anon_mmap (length, shift); +test_file_mmap (length, shift); -ksft_exit (!ret); +ksft_finished (); }tools/testing/selftests/mm/map_hugetlb.c deleted +0
-88@@ -1,88 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* -* Example of using hugepage memory in a user application using the mmap -* system call with MAP_HUGETLB flag. Before running this program make -* sure the administrator has allocated enough default sized huge pages -* to cover the 256 MB allocation. -*/ -#include <stdlib.h> -#include <stdio.h> -#include <unistd.h> -#include <sys/mman.h> -#include <fcntl.h> -#include "vm_util.h" -#include "kselftest.h" - -#define LENGTH (256UL*1024*1024) -#define PROTECTION (PROT_READ | PROT_WRITE) - -static void check_bytes (char *addr) -{ -ksft_print_msg ("First hex is %x\n" , *((unsigned int *)addr)); -} - -static void write_bytes (char *addr,size_t length) -{ -unsigned long i; - -for (i =0 ; i < length; i++) - *(addr + i) = (char )i; -} - -static void read_bytes (char *addr,size_t length) -{ -unsigned long i; - -check_bytes (addr); -for (i =0 ; i < length; i++) -if (*(addr + i) != (char )i) -ksft_exit_fail_msg ("Mismatch at %lu\n" , i); - -ksft_test_result_pass ("Read correct data\n" ); -} - -int main (int argc,char **argv) -{ -void *addr; -size_t hugepage_size; -size_t length =LENGTH ; -int flags =MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB ; -int shift =0 ; - - hugepage_size =default_huge_page_size (); -/* munmap with fail if the length is not page aligned */ -if (hugepage_size > length) - length = hugepage_size; - -ksft_print_header (); -ksft_set_plan (1 ); - -if (argc >1 ) - length =atol (argv[1 ]) <<20 ; -if (argc >2 ) { - shift =atoi (argv[2 ]); -if (shift) - flags |= (shift &MAP_HUGE_MASK ) <<MAP_HUGE_SHIFT ; - } - -if (shift) -ksft_print_msg ("%u kB hugepages\n" ,1 << (shift -10 )); -else -ksft_print_msg ("Default size hugepages\n" ); -ksft_print_msg ("Mapping %lu Mbytes\n" , (unsigned long )length >>20 ); - - addr =mmap (NULL , length,PROTECTION , flags,-1 ,0 ); -if (addr ==MAP_FAILED ) -ksft_exit_fail_msg ("mmap: %s\n" ,strerror (errno)); - -ksft_print_msg ("Returned address is %p\n" , addr); -check_bytes (addr); -write_bytes (addr, length); -read_bytes (addr, length); - -/* munmap() length of MAP_HUGETLB memory must be hugepage aligned */ -if (munmap (addr, length)) -ksft_exit_fail_msg ("munmap: %s\n" ,strerror (errno)); - -ksft_finished (); -}tools/testing/selftests/mm/run_vmtests.sh +0
-1@@ -292,7 +292,6 @@ CATEGORY="hugetlb" run_test ./hugepage-shm echo "$shmmax" > /proc/sys/kernel/shmmax echo "$shmall" > /proc/sys/kernel/shmall -CATEGORY="hugetlb" run_test ./map_hugetlb CATEGORY="hugetlb" run_test ./hugepage-mremap CATEGORY="hugetlb" run_test ./hugepage-vmemmap CATEGORY="hugetlb" run_test ./hugetlb-madvise
Mike Rapoport
maintainerPATCH v3 7/54 selftests/mm: rename hugepage-* tests to hugetlb-*
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hugepage could mean both THP and HugeTLB these days.
Rename hugepage-* tests for HugeTLB to hugetlb-* to avoid confusion.
---
tools/testing/selftests/mm/.gitignore | 4 ++++
tools/testing/selftests/mm/Makefile | 8 ++++----
tools/testing/selftests/mm/hugetlb-madvise.c | 2 +-
.../selftests/mm/{hugepage-mmap.c => hugetlb-mmap.c} | 2 +-
.../selftests/mm/{hugepage-mremap.c => hugetlb-mremap.c} | 2 +-
.../selftests/mm/{hugepage-shm.c => hugetlb-shm.c} | 2 +-
.../mm/{hugepage-vmemmap.c => hugetlb-vmemmap.c} | 0
tools/testing/selftests/mm/run_vmtests.sh | 8 ++++----
8 files changed, 16 insertions(+), 12 deletions(-)
rename tools/testing/selftests/mm/{hugepage-mmap.c => hugetlb-mmap.c} (99%)
rename tools/testing/selftests/mm/{hugepage-mremap.c => hugetlb-mremap.c} (99%)
rename tools/testing/selftests/mm/{hugepage-shm.c => hugetlb-shm.c} (99%)
rename tools/testing/selftests/mm/{hugepage-vmemmap.c => hugetlb-vmemmap.c} (100%)tools/testing/selftests/mm/.gitignore +4 -0
@@ -4,6 +4,10 @@ hugepage-mmap hugepage-mremap hugepage-shm hugepage-vmemmap +hugetlb-mmap +hugetlb-mremap +hugetlb-shm +hugetlb-vmemmap hugetlb-madvise hugetlb-read-hwpoison hugetlb-soft-offline
tools/testing/selftests/mm/Makefile +4 -4
@@ -63,10 +63,10 @@ TEST_GEN_FILES += hmm-tests TEST_GEN_FILES += hugetlb-madvise TEST_GEN_FILES += hugetlb-read-hwpoison TEST_GEN_FILES += hugetlb-soft-offline -TEST_GEN_FILES += hugepage-mmap -TEST_GEN_FILES += hugepage-mremap -TEST_GEN_FILES += hugepage-shm -TEST_GEN_FILES += hugepage-vmemmap +TEST_GEN_FILES += hugetlb-mremap +TEST_GEN_FILES += hugetlb-shm +TEST_GEN_FILES += hugetlb-vmemmap +TEST_GEN_FILES += hugetlb-mmap TEST_GEN_FILES += khugepaged TEST_GEN_FILES += madv_populate TEST_GEN_FILES += map_fixed_noreplace
tools/testing/selftests/mm/hugetlb-madvise.c +1 -1
@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 /* -* hugepage-madvise: +* hugetlb-madvise: * * Basic functional testing of madvise MADV_DONTNEED and MADV_REMOVE * on hugetlb mappings.
tools/testing/selftests/mm/hugetlb-mmap.c renamed from tools/testing/selftests/mm/hugepage-mmap.c +1 -1
@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 /* -* hugepage-mmap: +* hugetlb-mmap: * * Example of using huge page memory in a user application using the mmap * system call. Before running this application, make sure that the
tools/testing/selftests/mm/hugetlb-mremap.c renamed from tools/testing/selftests/mm/hugepage-mremap.c +1 -1
@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 /* -* hugepage-mremap: +* hugetlb-mremap: * * Example of remapping huge page memory in a user application using the * mremap system call. The path to a file in a hugetlbfs filesystem must
tools/testing/selftests/mm/hugetlb-shm.c renamed from tools/testing/selftests/mm/hugepage-shm.c +1 -1
@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 /* -* hugepage-shm: +* hugetlb-shm: * * Example of using huge page memory in a user application using Sys V shared * memory system calls. In this example the app is requesting 256MB of
tools/testing/selftests/mm/hugetlb-vmemmap.c renamed from tools/testing/selftests/mm/hugepage-vmemmap.c +0 -0
tools/testing/selftests/mm/run_vmtests.sh +4 -4
@@ -282,18 +282,18 @@ run_test() { echo "TAP version 13" | tap_output -CATEGORY="hugetlb" run_test ./hugepage-mmap +CATEGORY="hugetlb" run_test ./hugetlb-mmap shmmax=$(cat /proc/sys/kernel/shmmax) shmall=$(cat /proc/sys/kernel/shmall) echo 268435456 > /proc/sys/kernel/shmmax echo 4194304 > /proc/sys/kernel/shmall -CATEGORY="hugetlb" run_test ./hugepage-shm +CATEGORY="hugetlb" run_test ./hugetlb-shm echo "$shmmax" > /proc/sys/kernel/shmmax echo "$shmall" > /proc/sys/kernel/shmall -CATEGORY="hugetlb" run_test ./hugepage-mremap -CATEGORY="hugetlb" run_test ./hugepage-vmemmap +CATEGORY="hugetlb" run_test ./hugetlb-mremap +CATEGORY="hugetlb" run_test ./hugetlb-vmemmap CATEGORY="hugetlb" run_test ./hugetlb-madvise CATEGORY="hugetlb" run_test ./hugetlb_dio
Signature · 1 line
2.53.0
Re: [PATCH v3 07/54] selftests/mm: rename hugepage-* tests to hugetlb-*
tools/testing/selftests/mm/.gitignore hugepage-mmap · line 7
Quoted 7 lines · Mike Rapoport
tools/testing/selftests/mm/.gitignore +0
-0@@ -4,6 +4,10 @@ hugepage-mmap hugepage-mremap hugepage-shm hugepage-vmemmap
Maybe we'd better remove those old files from .gitignore as well. Otherwise, looks good.
Signature · 2 lines
Regards, Li Wang
Mike Rapoport
maintainerRe: [PATCH v3 07/54] selftests/mm: rename hugepage-* tests to hugetlb-*
Quoted 6 lines · Li Wang
Quoted 3 lines · Mike Rapoport
tools/testing/selftests/mm/.gitignore +0
-0@@ -4,6 +4,10 @@ hugepage-mmap
Quoted 6 lines
tools/testing/selftests/mm/.gitignore hugepage-mmap · line 6
Quoted 3 lines
tools/testing/selftests/mm/.gitignore +0
-0hugepage-mremap hugepage-shm hugepage-vmemmapMaybe we'd better remove those old files from .gitignore as well. Otherwise, looks good.
A while ago Linus objected to removing old executables from .gitignore :)
Quoted 1 line
Reviewed-by Li Wang <[email protected]>
Thanks!
Quoted 4 lines
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
Sincerely yours, Mike.
Mike Rapoport
maintainerPATCH v3 8/54 selftests/mm: hugetlb-shm: use kselftest framework
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Convert hugetlb-shm test to use kselftest framework for reporting and tracking successful and failing runs.
--- tools/testing/selftests/mm/hugetlb-shm.c | 45 ++++++++++++------------ 1 file changed, 22 insertions(+), 23 deletions(-)
tools/testing/selftests/mm/hugetlb-shm.c +22 -23
@@ -28,9 +28,9 @@#include <sys/shm.h> #include <sys/mman.h> -#define LENGTH (256UL*1024*1024) +#include "vm_util.h" -#define dprintf (x) printf(x) +#define LENGTH (256UL*1024*1024)int main (void ) {
@@ -38,44 +38,43 @@ int main(void)unsigned long i;char *shmaddr; +ksft_print_header (); +ksft_set_plan (1 ); + shmid =shmget (2 ,LENGTH ,SHM_HUGETLB |IPC_CREAT |SHM_R |SHM_W ); -if (shmid <0 ) { -perror ("shmget" ); -exit (1 ); - } -printf ("shmid: 0x%x\n" , shmid); +if (shmid <0 ) +ksft_exit_fail_perror ("shmget" ); + +ksft_print_msg ("shmid: 0x%x\n" , shmid); shmaddr =shmat (shmid,NULL ,0 );if (shmaddr == (char *)-1 ) { -perror ("Shared memory attach failure" ); +ksft_perror ("Shared memory attach failure" );shmctl (shmid,IPC_RMID ,NULL ); -exit (2 ); +ksft_exit_fail (); } -printf ("shmaddr: %p\n" , shmaddr); +ksft_print_msg ("shmaddr: %p\n" , shmaddr); -dprintf ("Starting the writes:\n" ); -for (i =0 ; i <LENGTH ; i++) { +ksft_print_msg ("Starting the writes:\n" ); +for (i =0 ; i <LENGTH ; i++) shmaddr[i] = (char )(i); -if (!(i % (1024 *1024 ))) -dprintf ("." ); - } -dprintf ("\n" ); -dprintf ("Starting the Check..." ); +ksft_print_msg ("Starting the Check..." );for (i =0 ; i <LENGTH ; i++)if (shmaddr[i] != (char )i) { -printf ("\nIndex %lu mismatched\n" , i); -exit (3 ); +shmctl (shmid,IPC_RMID ,NULL ); +ksft_exit_fail_msg ("Data mismatch at index %lu\n" , i); } -dprintf ("Done.\n" ); +ksft_print_msg ("Done.\n" );if (shmdt ((const void *)shmaddr) !=0 ) { -perror ("Detach failure" ); +ksft_perror ("Detach failure" );shmctl (shmid,IPC_RMID ,NULL ); -exit (4 ); +ksft_exit_fail (); }shmctl (shmid,IPC_RMID ,NULL ); -return 0 ; +ksft_test_result_pass ("hugepage using SysV shmget/shmat\n" ); +ksft_finished (); }
Signature · 1 line
2.53.0
Mike Rapoport
maintainerPATCH v3 9/54 selftests/mm: hugetlb-vmemmap: use kselftest framework
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Convert hugetlb-vmemmap test to use kselftest framework for reporting and tracking successful and failing runs.
--- tools/testing/selftests/mm/hugetlb-vmemmap.c | 42 +++++++++----------- 1 file changed, 18 insertions(+), 24 deletions(-)
tools/testing/selftests/mm/hugetlb-vmemmap.c +18 -24
@@ -63,7 +63,7 @@ static int check_page_flags(unsigned long pfn)read (fd, &pageflags,sizeof (pageflags));if ((pageflags &HEAD_PAGE_FLAGS ) !=HEAD_PAGE_FLAGS ) {close (fd); -printf ("Head page flags (%lx) is invalid\n" , pageflags); +ksft_print_msg ("Head page flags (%lx) is invalid\n" , pageflags);return -1 ; }
@@ -77,7 +77,7 @@ static int check_page_flags(unsigned long pfn)if ((pageflags &TAIL_PAGE_FLAGS ) !=TAIL_PAGE_FLAGS || (pageflags &HEAD_PAGE_FLAGS ) ==HEAD_PAGE_FLAGS ) {close (fd); -printf ("Tail page flags (%lx) is invalid\n" , pageflags); +ksft_print_msg ("Tail page flags (%lx) is invalid\n" , pageflags);return -1 ; } }
@@ -91,44 +91,38 @@ int main(int argc, char **argv) {void *addr;unsigned long pfn; +int ret; + +ksft_print_header (); +ksft_set_plan (1 ); pagesize =psize (); maplength =default_huge_page_size (); -if (!maplength) { -printf ("Unable to determine huge page size\n" ); -exit (1 ); - } +if (!maplength) +ksft_exit_skip ("Unable to determine huge page size\n" ); addr =mmap (NULL , maplength,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB ,-1 ,0 ); -if (addr ==MAP_FAILED ) { -perror ("mmap" ); -exit (1 ); - } +if (addr ==MAP_FAILED ) +ksft_exit_fail_perror ("mmap" );/* Trigger allocation of HugeTLB page. */ write_bytes (addr, maplength); pfn =virt_to_pfn (addr);if (pfn ==-1UL ) { +ksft_perror ("virt_to_pfn" );munmap (addr, maplength); -perror ("virt_to_pfn" ); -exit (1 ); +ksft_exit_fail (); } -printf ("Returned address is %p whose pfn is %lx\n" , addr, pfn); +ksft_print_msg ("Returned address is %p whose pfn is %lx\n" , addr, pfn); -if (check_page_flags (pfn) <0 ) { -munmap (addr, maplength); -perror ("check_page_flags" ); -exit (1 ); - } + ret =check_page_flags (pfn); -/* munmap() length of MAP_HUGETLB memory must be hugepage aligned */ -if (munmap (addr, maplength)) { -perror ("munmap" ); -exit (1 ); - } +if (munmap (addr, maplength)) +ksft_exit_fail_perror ("munmap" ); -return 0 ; +ksft_test_result (!ret,"HugeTLB vmemmap page flags\n" ); +ksft_finished (); }
Signature · 1 line
2.53.0
Mike Rapoport
maintainerPATCH v3 10/54 selftests/mm: hugetlb-madvise: use kselftest framework
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Convert hugetlb-madvise test to use kselftest framework for reporting and tracking successful and failing runs.
--- tools/testing/selftests/mm/hugetlb-madvise.c | 204 ++++++++----------- 1 file changed, 82 insertions(+), 122 deletions(-)
tools/testing/selftests/mm/hugetlb-madvise.c +82 -122
@@ -26,12 +26,11 @@#define validate_free_pages (exp_free) \ do { \ - int fhp = get_free_hugepages(); \ - if (fhp != (exp_free)) { \ - printf("Unexpected number of free huge " \ - "pages line %d\n", __LINE__); \ - exit(1); \ - } \ + unsigned long fhp = get_free_hugepages(); \ + if (fhp != (exp_free)) \ + ksft_exit_fail_msg("Unexpected number of free " \ + "huge pages %ld, expected %ld line %d\n", \ + fhp, (exp_free), __LINE__); \ } while (0)unsigned long huge_page_size;
@@ -57,28 +56,24 @@ int main(int argc, char **argv)int fd;int ret; +ksft_print_header (); +ksft_set_plan (1 ); + huge_page_size =default_huge_page_size (); -if (!huge_page_size) { -printf ("Unable to determine huge page size, exiting!\n" ); -exit (1 ); - } +if (!huge_page_size) +ksft_exit_skip ("Unable to determine huge page size\n" ); + base_page_size =sysconf (_SC_PAGE_SIZE); -if (!huge_page_size) { -printf ("Unable to determine base page size, exiting!\n" ); -exit (1 ); - } +if (!base_page_size) +ksft_exit_fail_msg ("Unable to determine base page size\n" ); free_hugepages =get_free_hugepages (); -if (free_hugepages <MIN_FREE_PAGES ) { -printf ("Not enough free huge pages to test, exiting!\n" ); -exit (KSFT_SKIP ); - } +if (free_hugepages <MIN_FREE_PAGES ) +ksft_exit_skip ("Not enough free huge pages (have %lu, need %d)\n" , free_hugepages,MIN_FREE_PAGES ); fd =memfd_create (argv[0 ],MFD_HUGETLB ); -if (fd <0 ) { -perror ("memfd_create() failed" ); -exit (1 ); - } +if (fd <0 ) +ksft_exit_fail_perror ("memfd_create" );/* * Test validity of MADV_DONTNEED addr and length arguments. mmap
@@ -90,16 +85,13 @@ int main(int argc, char **argv)PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB , -1, 0); -if (addr ==MAP_FAILED ) { -perror ("mmap" ); -exit (1 ); - } +if (addr ==MAP_FAILED ) +ksft_exit_fail_perror ("mmap" ); +if (munmap (addr, huge_page_size) ||munmap (addr + (NR_HUGE_PAGES +1 ) * huge_page_size, - huge_page_size)) { -perror ("munmap" ); -exit (1 ); - } + huge_page_size)) +ksft_exit_fail_perror ("munmap" ); addr = addr + huge_page_size;write_fault_pages (addr,NR_HUGE_PAGES );
@@ -108,20 +100,14 @@ int main(int argc, char **argv)/* addr before mapping should fail */ ret =madvise (addr - base_page_size,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED ); -if (!ret) { -printf ("Unexpected success of madvise call with invalid addr line %d\n" , - __LINE__); -exit (1 ); - } +if (!ret) +ksft_exit_fail_msg ("madvise with invalid addr unexpectedly succeeded line %d\n" , __LINE__);/* addr + length after mapping should fail */ ret =madvise (addr, (NR_HUGE_PAGES * huge_page_size) + base_page_size,MADV_DONTNEED ); -if (!ret) { -printf ("Unexpected success of madvise call with invalid length line %d\n" , - __LINE__); -exit (1 ); - } +if (!ret) +ksft_exit_fail_msg ("madvise with invalid length unexpectedly succeeded line %d\n" , __LINE__); (void )munmap (addr,NR_HUGE_PAGES * huge_page_size);
@@ -132,10 +118,9 @@ int main(int argc, char **argv)PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB , -1, 0); -if (addr ==MAP_FAILED ) { -perror ("mmap" ); -exit (1 ); - } +if (addr ==MAP_FAILED ) +ksft_exit_fail_perror ("mmap" ); +write_fault_pages (addr,NR_HUGE_PAGES );validate_free_pages (free_hugepages -NR_HUGE_PAGES );
@@ -143,19 +128,14 @@ int main(int argc, char **argv) ret =madvise (addr + base_page_size,NR_HUGE_PAGES * huge_page_size - base_page_size,MADV_DONTNEED ); -if (!ret) { -printf ("Unexpected success of madvise call with unaligned start address %d\n" , - __LINE__); -exit (1 ); - } +if (!ret) +ksft_exit_fail_msg ("madvise with unaligned start unexpectedly succeeded line %d\n" , __LINE__);/* addr + length should be aligned down to huge page size */ if (madvise (addr, ((NR_HUGE_PAGES -1 ) * huge_page_size) + base_page_size, -MADV_DONTNEED )) { -perror ("madvise" ); -exit (1 ); - } +MADV_DONTNEED )) +ksft_exit_fail_perror ("madvise" );/* should free all but last page in mapping */ validate_free_pages (free_hugepages -1 );
@@ -170,17 +150,14 @@ int main(int argc, char **argv)PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB , -1, 0); -if (addr ==MAP_FAILED ) { -perror ("mmap" ); -exit (1 ); - } +if (addr ==MAP_FAILED ) +ksft_exit_fail_perror ("mmap" ); +write_fault_pages (addr,NR_HUGE_PAGES );validate_free_pages (free_hugepages -NR_HUGE_PAGES ); -if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) { -perror ("madvise" ); -exit (1 ); - } +if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) +ksft_exit_fail_perror ("madvise" );/* should free all pages in mapping */ validate_free_pages (free_hugepages);
@@ -190,29 +167,25 @@ int main(int argc, char **argv)/* * Test MADV_DONTNEED on private mapping of hugetlb file */ -if (fallocate (fd,0 ,0 ,NR_HUGE_PAGES * huge_page_size)) { -perror ("fallocate" ); -exit (1 ); - } +if (fallocate (fd,0 ,0 ,NR_HUGE_PAGES * huge_page_size)) +ksft_exit_fail_perror ("fallocate" ); +validate_free_pages (free_hugepages -NR_HUGE_PAGES ); addr =mmap (NULL ,NR_HUGE_PAGES * huge_page_size,PROT_READ |PROT_WRITE ,MAP_PRIVATE , fd,0 ); -if (addr ==MAP_FAILED ) { -perror ("mmap" ); -exit (1 ); - } +if (addr ==MAP_FAILED ) +ksft_exit_fail_perror ("mmap" );/* read should not consume any pages */ read_fault_pages (addr,NR_HUGE_PAGES );validate_free_pages (free_hugepages -NR_HUGE_PAGES );/* madvise should not free any pages */ -if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) { -perror ("madvise" ); -exit (1 ); - } +if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) +ksft_exit_fail_perror ("madvise" ); +validate_free_pages (free_hugepages -NR_HUGE_PAGES );/* writes should allocate private pages */
@@ -220,10 +193,9 @@ int main(int argc, char **argv)validate_free_pages (free_hugepages - (2 *NR_HUGE_PAGES ));/* madvise should free private pages */ -if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) { -perror ("madvise" ); -exit (1 ); - } +if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) +ksft_exit_fail_perror ("madvise" ); +validate_free_pages (free_hugepages -NR_HUGE_PAGES );/* writes should allocate private pages */
@@ -238,10 +210,9 @@ int main(int argc, char **argv)* implementation. */ if (fallocate (fd,FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE , -0 ,NR_HUGE_PAGES * huge_page_size)) { -perror ("fallocate" ); -exit (1 ); - } +0 ,NR_HUGE_PAGES * huge_page_size)) +ksft_exit_fail_perror ("fallocate" ); +validate_free_pages (free_hugepages); (void )munmap (addr,NR_HUGE_PAGES * huge_page_size);
@@ -249,29 +220,25 @@ int main(int argc, char **argv)/* * Test MADV_DONTNEED on shared mapping of hugetlb file */ -if (fallocate (fd,0 ,0 ,NR_HUGE_PAGES * huge_page_size)) { -perror ("fallocate" ); -exit (1 ); - } +if (fallocate (fd,0 ,0 ,NR_HUGE_PAGES * huge_page_size)) +ksft_exit_fail_perror ("fallocate" ); +validate_free_pages (free_hugepages -NR_HUGE_PAGES ); addr =mmap (NULL ,NR_HUGE_PAGES * huge_page_size,PROT_READ |PROT_WRITE ,MAP_SHARED , fd,0 ); -if (addr ==MAP_FAILED ) { -perror ("mmap" ); -exit (1 ); - } +if (addr ==MAP_FAILED ) +ksft_exit_fail_perror ("mmap" );/* write should not consume any pages */ write_fault_pages (addr,NR_HUGE_PAGES );validate_free_pages (free_hugepages -NR_HUGE_PAGES );/* madvise should not free any pages */ -if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) { -perror ("madvise" ); -exit (1 ); - } +if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) +ksft_exit_fail_perror ("madvise" ); +validate_free_pages (free_hugepages -NR_HUGE_PAGES );/*
@@ -279,29 +246,25 @@ int main(int argc, char **argv)* * madvise is same as hole punch and should free all pages. */ -if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_REMOVE )) { -perror ("madvise" ); -exit (1 ); - } +if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_REMOVE )) +ksft_exit_fail_perror ("madvise" ); +validate_free_pages (free_hugepages); (void )munmap (addr,NR_HUGE_PAGES * huge_page_size);/* * Test MADV_REMOVE on shared and private mapping of hugetlb file */ -if (fallocate (fd,0 ,0 ,NR_HUGE_PAGES * huge_page_size)) { -perror ("fallocate" ); -exit (1 ); - } +if (fallocate (fd,0 ,0 ,NR_HUGE_PAGES * huge_page_size)) +ksft_exit_fail_perror ("fallocate" ); +validate_free_pages (free_hugepages -NR_HUGE_PAGES ); addr =mmap (NULL ,NR_HUGE_PAGES * huge_page_size,PROT_READ |PROT_WRITE ,MAP_SHARED , fd,0 ); -if (addr ==MAP_FAILED ) { -perror ("mmap" ); -exit (1 ); - } +if (addr ==MAP_FAILED ) +ksft_exit_fail_perror ("mmap" );/* shared write should not consume any additional pages */ write_fault_pages (addr,NR_HUGE_PAGES );
@@ -310,10 +273,8 @@ int main(int argc, char **argv) addr2 =mmap (NULL ,NR_HUGE_PAGES * huge_page_size,PROT_READ |PROT_WRITE ,MAP_PRIVATE , fd,0 ); -if (addr2 ==MAP_FAILED ) { -perror ("mmap" ); -exit (1 ); - } +if (addr2 ==MAP_FAILED ) +ksft_exit_fail_perror ("mmap" );/* private read should not consume any pages */ read_fault_pages (addr2,NR_HUGE_PAGES );
@@ -324,17 +285,15 @@ int main(int argc, char **argv)validate_free_pages (free_hugepages - (2 *NR_HUGE_PAGES ));/* madvise of shared mapping should not free any pages */ -if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) { -perror ("madvise" ); -exit (1 ); - } +if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) +ksft_exit_fail_perror ("madvise" ); +validate_free_pages (free_hugepages - (2 *NR_HUGE_PAGES ));/* madvise of private mapping should free private pages */ -if (madvise (addr2,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) { -perror ("madvise" ); -exit (1 ); - } +if (madvise (addr2,NR_HUGE_PAGES * huge_page_size,MADV_DONTNEED )) +ksft_exit_fail_perror ("madvise" ); +validate_free_pages (free_hugepages -NR_HUGE_PAGES );/* private write should consume additional pages again */
@@ -346,15 +305,16 @@ int main(int argc, char **argv)* not correct. private pages should not be freed, but this is * expected. See comment associated with FALLOC_FL_PUNCH_HOLE call. */ -if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_REMOVE )) { -perror ("madvise" ); -exit (1 ); - } +if (madvise (addr,NR_HUGE_PAGES * huge_page_size,MADV_REMOVE )) +ksft_exit_fail_perror ("madvise" ); +validate_free_pages (free_hugepages); (void )munmap (addr,NR_HUGE_PAGES * huge_page_size); (void )munmap (addr2,NR_HUGE_PAGES * huge_page_size);close (fd); -return 0 ; + +ksft_test_result_pass ("MADV_DONTNEED and MADV_REMOVE on hugetlb\n" ); +ksft_finished (); }
Signature · 1 line
2.53.0
Mike Rapoport
maintainerPATCH v3 11/54 selftests/mm: hugetlb_madv_vs_map: use kselftest framework
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Convert hugetlb_madv_vs_map test to use kselftest framework for reporting and tracking successful and failing runs.
--- .../testing/selftests/mm/hugetlb_madv_vs_map.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
tools/testing/selftests/mm/hugetlb_madv_vs_map.c +9 -9
@@ -25,7 +25,6 @@#include <unistd.h> #include "vm_util.h" -#include "kselftest.h" #define INLOOP_ITER 100
@@ -86,12 +85,14 @@ int main(void)*/ int max =10 ; +ksft_print_header (); +ksft_set_plan (1 ); + free_hugepages =get_free_hugepages (); -if (free_hugepages !=1 ) { +if (free_hugepages !=1 )ksft_exit_skip ("This test needs one and only one page to execute. Got %lu\n" , free_hugepages); - } mmap_size =default_huge_page_size ();
@@ -100,10 +101,8 @@ int main(void)MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB , -1, 0); -if ((unsigned long )huge_ptr ==-1 ) { -ksft_test_result_fail ("Failed to allocate huge page\n" ); -return KSFT_FAIL ; - } +if ((unsigned long )huge_ptr ==-1 ) +ksft_exit_fail_msg ("Failed to allocate huge page\n" );pthread_create (&thread1,NULL , madv,NULL );pthread_create (&thread2,NULL , touch,NULL );
@@ -115,12 +114,13 @@ int main(void)if (ret) {ksft_test_result_fail ("Unexpected huge page allocation\n" ); -return KSFT_FAIL ; +ksft_finished (); }/* Unmap and restart */ munmap (huge_ptr, mmap_size); } -return KSFT_PASS ; +ksft_test_result_pass ("No unexpected huge page allocations\n" ); +ksft_finished (); }
Signature · 1 line
2.53.0
Mike Rapoport
maintainerPATCH v3 12/54 selftests/mm: hugetlb-read-hwpoison: use kselftest framework
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Convert hugetlb-read-hwpoison test to use kselftest framework for reporting and tracking successful and failing runs.
--- .../selftests/mm/hugetlb-read-hwpoison.c | 118 +++++++++--------- 1 file changed, 57 insertions(+), 61 deletions(-)
tools/testing/selftests/mm/hugetlb-read-hwpoison.c +57 -61
@@ -13,9 +13,7 @@#include <signal.h> #include "kselftest.h" - -#define PREFIX " ... " -#define ERROR_PREFIX " !!! " +#include "vm_util.h" #define MAX_WRITE_READ_CHUNK_SIZE (getpagesize() * 16)#define MAX (a, b) (((a) > (b)) ? (a) : (b))
@@ -26,17 +24,22 @@ enum test_status { TEST_SKIPPED = 2, }; -static char *status_to_str (enum test_status status) +static void report_status (enum test_status status,const char *test_name, +size_t chunk_size) {switch (status) {case TEST_PASSED : -return "TEST_PASSED" ; +ksft_test_result_pass ("%s chunk_size=0x%lx\n" , + test_name, chunk_size); +break ;case TEST_FAILED : -return "TEST_FAILED" ; +ksft_test_result_fail ("%s chunk_size=0x%lx\n" , + test_name, chunk_size); +break ;case TEST_SKIPPED : -return "TEST_SKIPPED" ; -default : -return "TEST_???" ; +ksft_test_result_skip ("%s chunk_size=0x%lx\n" , + test_name, chunk_size); +break ; } }
@@ -59,8 +62,8 @@ static bool verify_chunk(char *buf, size_t len, char val)for (i =0 ; i < len; ++i) {if (buf[i] != val) { - printf(PREFIX ERROR_PREFIX "check fail: buf[%lu] = %u != %u\n", - i, buf[i], val); +ksft_print_msg ("check fail: buf[%lu] = %u != %u\n" , + i, buf[i], val);return false; } }
@@ -76,21 +79,21 @@ static bool seek_read_hugepage_filemap(int fd, size_t len, size_t wr_chunk_size,ssize_t total_ret_count =0 ;char val = offset / wr_chunk_size + offset % wr_chunk_size; - printf(PREFIX PREFIX "init val=%u with offset=0x%lx\n", val, offset); - printf(PREFIX PREFIX "expect to read 0x%lx bytes of data in total\n", - expected); +ksft_print_msg ("init val=%u with offset=0x%lx\n" , val, offset); +ksft_print_msg ("expect to read 0x%lx bytes of data in total\n" , + expected);if (lseek (fd, offset,SEEK_SET ) <0 ) { - perror(PREFIX ERROR_PREFIX "seek failed"); +ksft_perror ("seek failed" );return false; }while (offset + total_ret_count < len) { ret_count =read (fd, buf, wr_chunk_size);if (ret_count ==0 ) { - printf(PREFIX PREFIX "read reach end of the file\n"); +ksft_print_msg ("read reach end of the file\n" );break ; }else if (ret_count <0 ) { - perror(PREFIX ERROR_PREFIX "read failed"); +ksft_perror ("read failed" );break ; } ++val;
@@ -99,8 +102,8 @@ static bool seek_read_hugepage_filemap(int fd, size_t len, size_t wr_chunk_size, total_ret_count += ret_count; } - printf(PREFIX PREFIX "actually read 0x%lx bytes of data in total\n", - total_ret_count); +ksft_print_msg ("actually read 0x%lx bytes of data in total\n" , + total_ret_count);return total_ret_count == expected; }
@@ -113,15 +116,15 @@ static bool read_hugepage_filemap(int fd, size_t len,ssize_t total_ret_count =0 ;char val =0 ; - printf(PREFIX PREFIX "expect to read 0x%lx bytes of data in total\n", - expected); +ksft_print_msg ("expect to read 0x%lx bytes of data in total\n" , + expected);while (total_ret_count < len) { ret_count =read (fd, buf, wr_chunk_size);if (ret_count ==0 ) { - printf(PREFIX PREFIX "read reach end of the file\n"); +ksft_print_msg ("read reach end of the file\n" );break ; }else if (ret_count <0 ) { - perror(PREFIX ERROR_PREFIX "read failed"); +ksft_perror ("read failed" );break ; } ++val;
@@ -130,8 +133,8 @@ static bool read_hugepage_filemap(int fd, size_t len, total_ret_count += ret_count; } - printf(PREFIX PREFIX "actually read 0x%lx bytes of data in total\n", - total_ret_count); +ksft_print_msg ("actually read 0x%lx bytes of data in total\n" , + total_ret_count);return total_ret_count == expected; }
@@ -143,14 +146,14 @@ test_hugetlb_read(int fd, size_t len, size_t wr_chunk_size)char *filemap =NULL ;if (ftruncate (fd, len) <0 ) { - perror(PREFIX ERROR_PREFIX "ftruncate failed"); +ksft_perror ("ftruncate failed" );return status; } filemap =mmap (NULL , len,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_POPULATE , fd,0 );if (filemap ==MAP_FAILED ) { - perror(PREFIX ERROR_PREFIX "mmap for primary mapping failed"); +ksft_perror ("mmap for primary mapping failed" ); goto done; }
@@ -163,7 +166,7 @@ test_hugetlb_read(int fd, size_t len, size_t wr_chunk_size)munmap (filemap, len); done:if (ftruncate (fd,0 ) <0 ) { - perror(PREFIX ERROR_PREFIX "ftruncate back to 0 failed"); +ksft_perror ("ftruncate back to 0 failed" ); status =TEST_FAILED ; }
@@ -180,14 +183,14 @@ test_hugetlb_read_hwpoison(int fd, size_t len, size_t wr_chunk_size,const unsigned long pagesize =getpagesize ();if (ftruncate (fd, len) <0 ) { - perror(PREFIX ERROR_PREFIX "ftruncate failed"); +ksft_perror ("ftruncate failed" );return status; } filemap =mmap (NULL , len,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_POPULATE , fd,0 );if (filemap ==MAP_FAILED ) { - perror(PREFIX ERROR_PREFIX "mmap for primary mapping failed"); +ksft_perror ("mmap for primary mapping failed" ); goto done; }
@@ -202,7 +205,7 @@ test_hugetlb_read_hwpoison(int fd, size_t len, size_t wr_chunk_size,*/ hwp_addr = filemap + len /2 + pagesize;if (madvise (hwp_addr, pagesize,MADV_HWPOISON ) <0 ) { - perror(PREFIX ERROR_PREFIX "MADV_HWPOISON failed"); +ksft_perror ("MADV_HWPOISON failed" ); goto unmap; }
@@ -229,7 +232,7 @@ test_hugetlb_read_hwpoison(int fd, size_t len, size_t wr_chunk_size,munmap (filemap, len); done:if (ftruncate (fd,0 ) <0 ) { - perror(PREFIX ERROR_PREFIX "ftruncate back to 0 failed"); +ksft_perror ("ftruncate back to 0 failed" ); status =TEST_FAILED ; }
@@ -242,17 +245,17 @@ static int create_hugetlbfs_file(struct statfs *file_stat) fd =memfd_create ("hugetlb_tmp" ,MFD_HUGETLB );if (fd <0 ) { - perror(PREFIX ERROR_PREFIX "could not open hugetlbfs file"); +ksft_perror ("could not open hugetlbfs file" );return -1 ; }memset (file_stat,0 ,sizeof (*file_stat));if (fstatfs (fd, file_stat)) { - perror(PREFIX ERROR_PREFIX "fstatfs failed"); +ksft_perror ("fstatfs failed" ); goto close; }if (file_stat->f_type !=HUGETLBFS_MAGIC ) { - printf(PREFIX ERROR_PREFIX "not hugetlbfs file\n"); +ksft_print_msg ("not hugetlbfs file\n" ); goto close; }
@@ -264,7 +267,7 @@ static int create_hugetlbfs_file(struct statfs *file_stat)static void sigbus_handler (int sig) { -printf (PREFIX "received SIGBUS\n" ); +ksft_print_msg ("received SIGBUS\n" ); }int main (void )
@@ -279,51 +282,44 @@ int main(void) };size_t i; +ksft_print_header (); +ksft_set_plan (ARRAY_SIZE (wr_chunk_sizes) *3 ); +signal (SIGBUS , sigbus_handler);for (i =0 ; i <ARRAY_SIZE (wr_chunk_sizes); ++i) { -printf ("Write/read chunk size=0x%lx\n" , - wr_chunk_sizes[i]); +ksft_print_msg ("Write/read chunk size=0x%lx\n" , + wr_chunk_sizes[i]); fd =create_hugetlbfs_file (&file_stat);if (fd <0 ) - goto create_failure; -printf (PREFIX "HugeTLB read regression test...\n" ); +ksft_exit_fail_msg ("Failed to create hugetlbfs file\n" ); + status =test_hugetlb_read (fd, file_stat.f_bsize, wr_chunk_sizes[i]); -printf (PREFIX "HugeTLB read regression test...%s\n" , -status_to_str (status));close (fd); -if (status ==TEST_FAILED ) -return -1 ; +report_status (status,"HugeTLB read regression" , + wr_chunk_sizes[i]); fd =create_hugetlbfs_file (&file_stat);if (fd <0 ) - goto create_failure; -printf (PREFIX "HugeTLB read HWPOISON test...\n" ); +ksft_exit_fail_msg ("Failed to create hugetlbfs file\n" ); + status =test_hugetlb_read_hwpoison (fd, file_stat.f_bsize, wr_chunk_sizes[i], false); -printf (PREFIX "HugeTLB read HWPOISON test...%s\n" , -status_to_str (status));close (fd); -if (status ==TEST_FAILED ) -return -1 ; +report_status (status,"HugeTLB read HWPOISON" , + wr_chunk_sizes[i]); fd =create_hugetlbfs_file (&file_stat);if (fd <0 ) - goto create_failure; -printf (PREFIX "HugeTLB seek then read HWPOISON test...\n" ); +ksft_exit_fail_msg ("Failed to create hugetlbfs file\n" ); + status =test_hugetlb_read_hwpoison (fd, file_stat.f_bsize, wr_chunk_sizes[i], true); -printf (PREFIX "HugeTLB seek then read HWPOISON test...%s\n" , -status_to_str (status));close (fd); -if (status ==TEST_FAILED ) -return -1 ; +report_status (status,"HugeTLB seek then read HWPOISON" , + wr_chunk_sizes[i]); } -return 0 ; - -create_failure: -printf (ERROR_PREFIX "Abort test: failed to create hugetlbfs file\n" ); -return -1 ; +ksft_finished (); }
Signature · 1 line
2.53.0
Re: [PATCH v3 12/54] selftests/mm: hugetlb-read-hwpoison: use kselftest framework
Quoted 8 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Convert hugetlb-read-hwpoison test to use kselftest framework for reporting and tracking successful and failing runs.
Reviewed-by Donet Tom <[email protected]>Reviewed-by Mark Brown <[email protected]>Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Signature · 2 lines
Regards, Li Wang
Mike Rapoport
maintainerPATCH v3 13/54 selftests/mm: khugepaged: group tests in an array
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Currently khugepaged decides if a test can run using TEST() macro that checks what mem_ops and collapse_context are set by the command line arguments.
For better compatibility with ksefltest framework, add an array of 'struct test_case's and redefine TEST() macro to conditionally add enabled tests to that array.
Then execute the enabled test by looping the test_case's array.
--- tools/testing/selftests/mm/khugepaged.c | 43 +++++++++++++++++++++---- 1 file changed, 36 insertions(+), 7 deletions(-)
tools/testing/selftests/mm/khugepaged.c +36 -7
@@ -1165,6 +1165,34 @@ static void parse_test_type(int argc, char **argv)get_finfo (argv[1 ]); } +typedef void (*test_fn )(struct collapse_context *c,struct mem_ops *ops); + +struct test_case { +struct collapse_context *ctx; +struct mem_ops *ops; +const char *desc; +test_fn fn; +}; + +#define MAX_TEST_CASES 64 +static struct test_case test_cases[MAX_TEST_CASES ]; +static int nr_test_cases; + +#define TEST (t, c, o) do { \ + if (c && o) { \ + if (nr_test_cases >= MAX_TEST_CASES) { \ + printf("MAX_ADD_CASES is too small\n"); \ + exit(EXIT_FAILURE); \ + } \ + test_cases[nr_test_cases++] = (struct test_case){ \ + .ctx = c, \ + .ops = o, \ + .desc = #t, \ + .fn = t, \ + }; \ + } \ + } while (0) +int main (int argc,char **argv) {int hpage_pmd_order;
@@ -1220,13 +1248,6 @@ int main(int argc, char **argv)alloc_at_fault (); -#define TEST (t, c, o) do { \ - if (c && o) { \ - printf("\nRun test: " #t " (%s:%s)\n", c->name, o->name); \ - t(c, o); \ - } \ - } while (0) -TEST (collapse_full, khugepaged_context, anon_ops);TEST (collapse_full, khugepaged_context, file_ops);TEST (collapse_full, khugepaged_context, shmem_ops);
@@ -1286,5 +1307,13 @@ int main(int argc, char **argv)TEST (madvise_retracted_page_tables, madvise_context, file_ops);TEST (madvise_retracted_page_tables, madvise_context, shmem_ops); +for (int i =0 ; i < nr_test_cases; i++) { +struct test_case *t = &test_cases[i]; + + exit_status =KSFT_PASS ; +printf ("\nRun test: %s: (%s:%s)\n" , t->desc, t->ctx->name, t->ops->name); + t->fn (t->ctx, t->ops); + } +restore_settings (0 ); }
Signature · 1 line
2.53.0
Mike Rapoport
maintainerPATCH v3 14/54 selftests/mm: khugepaged: use ksefltest framework
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Convert khugepaged tests to use kselftest framework for reporting and tracking successful and failing runs.
The conversion is mostly about replacing printf()/perror() + exit() pairs with their ksft_ counterparts.
The nice colored success and failure indications are left intact.
Replace the progress report in collapse_compound_extreme() with a single ksft_print_msg() to avoid headache with formatting and make the test output more concise.
--- tools/testing/selftests/mm/khugepaged.c | 311 ++++++++++-------------- 1 file changed, 128 insertions(+), 183 deletions(-)
tools/testing/selftests/mm/khugepaged.c +128 -183
@@ -83,12 +83,13 @@ static void success(const char *msg)static void fail (const char *msg) {printf (" \e[31m%s\e[0m\n" , msg); - exit_status++; + exit_status =KSFT_FAIL ; }static void skip (const char *msg) {printf (" \e[33m%s\e[0m\n" , msg); + exit_status =KSFT_SKIP ; }static void restore_settings_atexit (void )
@@ -96,22 +97,23 @@ static void restore_settings_atexit(void)if (skip_settings_restore)return ; -printf ("Restore THP and khugepaged settings..." ); +ksft_print_msg ("Restore THP and khugepaged settings..." );thp_restore_settings ();success ("OK" ); skip_settings_restore = true; +ksft_finished (); }static void restore_settings (int sig) {/* exit() will invoke the restore_settings_atexit handler. */ -exit (sig ?EXIT_FAILURE : exit_status); +exit (sig ?KSFT_FAIL : exit_status); }static void save_settings (void ) { -printf ("Save THP and khugepaged settings..." ); +ksft_print_msg ("Save THP and khugepaged settings..." );if (file_ops && finfo.type ==VMA_FILE )thp_set_read_ahead_path (finfo.dev_queue_read_ahead_path);thp_save_settings ();
@@ -135,19 +137,13 @@ static void get_finfo(const char *dir) finfo.dir = dir;stat (finfo.dir, &path_stat); -if (!S_ISDIR (path_stat.st_mode)) { -printf ("%s: Not a directory (%s)\n" , __func__, finfo.dir); -exit (EXIT_FAILURE ); - } +if (!S_ISDIR (path_stat.st_mode)) +ksft_exit_fail_msg ("%s: Not a directory (%s)\n" , __func__, finfo.dir);if (snprintf (finfo.path,sizeof (finfo.path),"%s/" TEST_FILE , - finfo.dir) >=sizeof (finfo.path)) { -printf ("%s: Pathname is too long\n" , __func__); -exit (EXIT_FAILURE ); - } -if (statfs (finfo.dir, &fs)) { -perror ("statfs()" ); -exit (EXIT_FAILURE ); - } + finfo.dir) >=sizeof (finfo.path)) +ksft_exit_fail_msg ("%s: Pathname is too long\n" , __func__); +if (statfs (finfo.dir, &fs)) +ksft_exit_fail_perror ("statfs()" ); finfo.type = fs.f_type ==TMPFS_MAGIC ?VMA_SHMEM :VMA_FILE ;if (finfo.type ==VMA_SHMEM )return ;
@@ -155,40 +151,30 @@ static void get_finfo(const char *dir)/* Find owning device's queue/read_ahead_kb control */ if (snprintf (path,sizeof (path),"/sys/dev/block/%d:%d/uevent" ,major (path_stat.st_dev),minor (path_stat.st_dev)) - >=sizeof (path)) { -printf ("%s: Pathname is too long\n" , __func__); -exit (EXIT_FAILURE ); - } -if (read_file (path, buf,sizeof (buf)) <0 ) { -perror ("read_file(read_num)" ); -exit (EXIT_FAILURE ); - } + >=sizeof (path)) +ksft_exit_fail_msg ("%s: Pathname is too long\n" , __func__); +if (read_file (path, buf,sizeof (buf)) <0 ) +ksft_exit_fail_perror ("read_file(read_num)" );if (strstr (buf,"DEVTYPE=disk" )) {/* Found it */ if (snprintf (finfo.dev_queue_read_ahead_path,sizeof (finfo.dev_queue_read_ahead_path),"/sys/dev/block/%d:%d/queue/read_ahead_kb" ,major (path_stat.st_dev),minor (path_stat.st_dev)) - >=sizeof (finfo.dev_queue_read_ahead_path)) { -printf ("%s: Pathname is too long\n" , __func__); -exit (EXIT_FAILURE ); - } + >=sizeof (finfo.dev_queue_read_ahead_path)) +ksft_exit_fail_msg ("%s: Pathname is too long\n" , __func__);return ; } -if (!strstr (buf,"DEVTYPE=partition" )) { -printf ("%s: Unknown device type: %s\n" , __func__, path); -exit (EXIT_FAILURE ); - } +if (!strstr (buf,"DEVTYPE=partition" )) +ksft_exit_fail_msg ("%s: Unknown device type: %s\n" , __func__, path);/* * Partition of block device - need to find actual device. * Using naming convention that devnameN is partition of * device devname. */ str =strstr (buf,"DEVNAME=" ); -if (!str) { -printf ("%s: Could not read: %s" , __func__, path); -exit (EXIT_FAILURE ); - } +if (!str) +ksft_exit_fail_msg ("%s: Could not read: %s" , __func__, path); str +=8 ; end = str;while (*end) {
@@ -197,16 +183,13 @@ static void get_finfo(const char *dir)if (snprintf (finfo.dev_queue_read_ahead_path,sizeof (finfo.dev_queue_read_ahead_path),"/sys/block/%s/queue/read_ahead_kb" , - str) >=sizeof (finfo.dev_queue_read_ahead_path)) { -printf ("%s: Pathname is too long\n" , __func__); -exit (EXIT_FAILURE ); - } + str) >=sizeof (finfo.dev_queue_read_ahead_path)) +ksft_exit_fail_msg ("%s: Pathname is too long\n" , __func__);return ; } ++end; } -printf ("%s: Could not read: %s\n" , __func__, path); -exit (EXIT_FAILURE ); +ksft_exit_fail_msg ("%s: Could not read: %s\n" , __func__, path); }static bool check_swap (void *addr,unsigned long size)
@@ -219,26 +202,19 @@ static bool check_swap(void *addr, unsigned long size) ret =snprintf (addr_pattern,MAX_LINE_LENGTH ,"%08lx-" , (unsigned long ) addr); -if (ret >=MAX_LINE_LENGTH ) { -printf ("%s: Pattern is too long\n" , __func__); -exit (EXIT_FAILURE ); - } - +if (ret >=MAX_LINE_LENGTH ) +ksft_exit_fail_msg ("%s: Pattern is too long\n" , __func__); fp =fopen (PID_SMAPS ,"r" ); -if (!fp) { -printf ("%s: Failed to open file %s\n" , __func__,PID_SMAPS ); -exit (EXIT_FAILURE ); - } +if (!fp) +ksft_exit_fail_msg ("%s: Failed to open file %s\n" , __func__,PID_SMAPS );if (!check_for_pattern (fp, addr_pattern, buffer,sizeof (buffer))) goto err_out; ret =snprintf (addr_pattern,MAX_LINE_LENGTH ,"Swap:%19ld kB" , size >>10 ); -if (ret >=MAX_LINE_LENGTH ) { -printf ("%s: Pattern is too long\n" , __func__); -exit (EXIT_FAILURE ); - } +if (ret >=MAX_LINE_LENGTH ) +ksft_exit_fail_msg ("%s: Pattern is too long\n" , __func__);/* * Fetch the Swap: in the same block and check whether it got * the expected number of hugeepages next.
@@ -261,10 +237,8 @@ static void *alloc_mapping(int nr) p =mmap (BASE_ADDR , nr * hpage_pmd_size,PROT_READ |PROT_WRITE ,MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 ); -if (p !=BASE_ADDR ) { -printf ("Failed to allocate VMA at %p\n" ,BASE_ADDR ); -exit (EXIT_FAILURE ); - } +if (p !=BASE_ADDR ) +ksft_exit_fail_msg ("Failed to allocate VMA at %p\n" ,BASE_ADDR );return p; }
@@ -314,19 +288,13 @@ static void *alloc_hpage(struct mem_ops *ops)* khugepaged on low-load system (like a test machine), which * would cause MADV_COLLAPSE to fail with EAGAIN. */ -printf ("Allocate huge page..." ); -if (madvise_collapse_retry (p, hpage_pmd_size)) { -perror ("madvise(MADV_COLLAPSE)" ); -exit (EXIT_FAILURE ); - } -if (!ops->check_huge (p,1 )) { -perror ("madvise(MADV_COLLAPSE)" ); -exit (EXIT_FAILURE ); - } -if (madvise (p, hpage_pmd_size,MADV_HUGEPAGE )) { -perror ("madvise(MADV_HUGEPAGE)" ); -exit (EXIT_FAILURE ); - } +ksft_print_msg ("Allocate huge page..." ); +if (madvise_collapse_retry (p, hpage_pmd_size)) +ksft_exit_fail_perror ("madvise(MADV_COLLAPSE)" ); +if (!ops->check_huge (p,1 )) +ksft_exit_fail_perror ("madvise(MADV_COLLAPSE)" ); +if (madvise (p, hpage_pmd_size,MADV_HUGEPAGE )) +ksft_exit_fail_perror ("madvise(MADV_HUGEPAGE)" );success ("OK" );return p; }
@@ -336,11 +304,9 @@ static void validate_memory(int *p, unsigned long start, unsigned long end)int i;for (i = start / page_size; i < end / page_size; i++) { -if (p[i * page_size /sizeof (*p)] != i +0xdead0000 ) { -printf ("Page %d is corrupted: %#x\n" , - i, p[i * page_size /sizeof (*p)]); -exit (EXIT_FAILURE ); - } +if (p[i * page_size /sizeof (*p)] != i +0xdead0000 ) +ksft_exit_fail_msg ("Page %d is corrupted: %#x\n" , + i, p[i * page_size /sizeof (*p)]); } }
@@ -371,14 +337,12 @@ static void *file_setup_area(int nr_hpages)unsigned long size;unlink (finfo.path);/* Cleanup from previous failed tests */ -printf ("Creating %s for collapse%s..." , finfo.path, - finfo.type ==VMA_SHMEM ?" (tmpfs)" :"" ); +ksft_print_msg ("Creating %s for collapse%s..." , finfo.path, + finfo.type ==VMA_SHMEM ?" (tmpfs)" :"" ); fd =open (finfo.path,O_DSYNC |O_CREAT |O_RDWR |O_TRUNC |O_EXCL ,777 ); -if (fd <0 ) { -perror ("open()" ); -exit (EXIT_FAILURE ); - } +if (fd <0 ) +ksft_exit_fail_perror ("open()" ); size = nr_hpages * hpage_pmd_size; p =alloc_mapping (nr_hpages);
@@ -388,18 +352,14 @@ static void *file_setup_area(int nr_hpages)munmap (p, size);success ("OK" ); -printf ("Opening %s read only for collapse..." , finfo.path); +ksft_print_msg ("Opening %s read only for collapse..." , finfo.path); finfo.fd =open (finfo.path,O_RDONLY ,777 ); -if (finfo.fd <0 ) { -perror ("open()" ); -exit (EXIT_FAILURE ); - } +if (finfo.fd <0 ) +ksft_exit_fail_perror ("open()" ); p =mmap (BASE_ADDR , size,PROT_READ ,MAP_PRIVATE , finfo.fd,0 ); -if (p ==MAP_FAILED || p !=BASE_ADDR ) { -perror ("mmap()" ); -exit (EXIT_FAILURE ); - } +if (p ==MAP_FAILED || p !=BASE_ADDR ) +ksft_exit_fail_perror ("mmap()" );/* Drop page cache */ write_file ("/proc/sys/vm/drop_caches" ,"3" ,2 );
@@ -416,10 +376,8 @@ static void file_cleanup_area(void *p, unsigned long size)static void file_fault (void *p,unsigned long start,unsigned long end) { -if (madvise (((char *)p) + start, end - start,MADV_POPULATE_READ )) { -perror ("madvise(MADV_POPULATE_READ" ); -exit (EXIT_FAILURE ); - } +if (madvise (((char *)p) + start, end - start,MADV_POPULATE_READ )) +ksft_exit_fail_perror ("madvise(MADV_POPULATE_READ" ); }static bool file_check_huge (void *addr,int nr_hpages)
@@ -441,20 +399,14 @@ static void *shmem_setup_area(int nr_hpages)unsigned long size = nr_hpages * hpage_pmd_size; finfo.fd =memfd_create ("khugepaged-selftest-collapse-shmem" ,0 ); -if (finfo.fd <0 ) { -perror ("memfd_create()" ); -exit (EXIT_FAILURE ); - } -if (ftruncate (finfo.fd, size)) { -perror ("ftruncate()" ); -exit (EXIT_FAILURE ); - } +if (finfo.fd <0 ) +ksft_exit_fail_perror ("memfd_create()" ); +if (ftruncate (finfo.fd, size)) +ksft_exit_fail_perror ("ftruncate()" ); p =mmap (BASE_ADDR , size,PROT_READ |PROT_WRITE ,MAP_SHARED , finfo.fd,0 ); -if (p !=BASE_ADDR ) { -perror ("mmap()" ); -exit (EXIT_FAILURE ); - } +if (p !=BASE_ADDR ) +ksft_exit_fail_perror ("mmap()" );return p; }
@@ -499,7 +451,7 @@ static void __madvise_collapse(const char *msg, char *p, int nr_hpages,int ret;struct thp_settings settings = *thp_current_settings (); -printf ("%s..." , msg); +ksft_print_msg ("%s..." , msg);/* * Prevent khugepaged interference and tests that MADV_COLLAPSE
@@ -526,10 +478,8 @@ static void madvise_collapse(const char *msg, char *p, int nr_hpages, struct mem_ops *ops, bool expect) {/* Sanity check */ -if (!ops->check_huge (p,0 )) { -printf ("Unexpected huge page\n" ); -exit (EXIT_FAILURE ); - } +if (!ops->check_huge (p,0 )) +ksft_exit_fail_msg ("Unexpected huge page\n" );__madvise_collapse (msg, p, nr_hpages, ops, expect); }
@@ -541,17 +491,15 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages,int timeout =6 ;/* 3 seconds */ /* Sanity check */ -if (!ops->check_huge (p,0 )) { -printf ("Unexpected huge page\n" ); -exit (EXIT_FAILURE ); - } +if (!ops->check_huge (p,0 )) +ksft_exit_fail_msg ("Unexpected huge page\n" );madvise (p, nr_hpages * hpage_pmd_size,MADV_HUGEPAGE );/* Wait until the second full_scan completed */ full_scans =thp_read_num ("khugepaged/full_scans" ) +2 ; -printf ("%s..." , msg); +ksft_print_msg ("%s..." , msg);while (timeout--) {if (ops->check_huge (p, nr_hpages))break ;
@@ -621,7 +569,7 @@ static void alloc_at_fault(void) p =alloc_mapping (1 ); *p =1 ; -printf ("Allocate huge page on fault..." ); +ksft_print_msg ("Allocate huge page on fault..." );if (check_huge_anon (p,1 , hpage_pmd_size))success ("OK" );else
@@ -630,12 +578,14 @@ static void alloc_at_fault(void)thp_pop_settings ();madvise (p, page_size,MADV_DONTNEED ); -printf ("Split huge PMD on MADV_DONTNEED..." ); +ksft_print_msg ("Split huge PMD on MADV_DONTNEED..." );if (check_huge_anon (p,0 , hpage_pmd_size))success ("OK" );else fail ("Fail" );munmap (p, hpage_pmd_size); + +ksft_test_result_report (exit_status,"allocate on fault and split\n" ); }static void collapse_full (struct collapse_context *c,struct mem_ops *ops)
@@ -650,6 +600,8 @@ static void collapse_full(struct collapse_context *c, struct mem_ops *ops) ops, true);validate_memory (p,0 , size); ops->cleanup_area (p, size); + +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void collapse_empty (struct collapse_context *c,struct mem_ops *ops)
@@ -659,6 +611,7 @@ static void collapse_empty(struct collapse_context *c, struct mem_ops *ops) p = ops->setup_area (1 ); c->collapse ("Do not collapse empty PTE table" , p,1 , ops, false); ops->cleanup_area (p, hpage_pmd_size); +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void collapse_single_pte_entry (struct collapse_context *c,struct mem_ops *ops)
@@ -670,6 +623,7 @@ static void collapse_single_pte_entry(struct collapse_context *c, struct mem_ops c->collapse ("Collapse PTE table with single PTE entry present" , p,1 , ops, true); ops->cleanup_area (p, hpage_pmd_size); +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void collapse_max_ptes_none (struct collapse_context *c,struct mem_ops *ops)
@@ -706,6 +660,7 @@ static void collapse_max_ptes_none(struct collapse_context *c, struct mem_ops *o skip: ops->cleanup_area (p, hpage_pmd_size);thp_pop_settings (); +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void collapse_swapin_single_pte (struct collapse_context *c,struct mem_ops *ops)
@@ -715,11 +670,9 @@ static void collapse_swapin_single_pte(struct collapse_context *c, struct mem_op p = ops->setup_area (1 ); ops->fault (p,0 , hpage_pmd_size); -printf ("Swapout one page..." ); -if (madvise (p, page_size,MADV_PAGEOUT )) { -perror ("madvise(MADV_PAGEOUT)" ); -exit (EXIT_FAILURE ); - } +ksft_print_msg ("Swapout one page..." ); +if (madvise (p, page_size,MADV_PAGEOUT )) +ksft_exit_fail_perror ("madvise(MADV_PAGEOUT)" );if (check_swap (p, page_size)) {success ("OK" ); }else {
@@ -732,6 +685,7 @@ static void collapse_swapin_single_pte(struct collapse_context *c, struct mem_opvalidate_memory (p,0 , hpage_pmd_size); out: ops->cleanup_area (p, hpage_pmd_size); +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void collapse_max_ptes_swap (struct collapse_context *c,struct mem_ops *ops)
@@ -742,11 +696,9 @@ static void collapse_max_ptes_swap(struct collapse_context *c, struct mem_ops *o p = ops->setup_area (1 ); ops->fault (p,0 , hpage_pmd_size); -printf ("Swapout %d of %d pages..." , max_ptes_swap +1 , hpage_pmd_nr); -if (madvise (p, (max_ptes_swap +1 ) * page_size,MADV_PAGEOUT )) { -perror ("madvise(MADV_PAGEOUT)" ); -exit (EXIT_FAILURE ); - } +ksft_print_msg ("Swapout %d of %d pages..." , max_ptes_swap +1 , hpage_pmd_nr); +if (madvise (p, (max_ptes_swap +1 ) * page_size,MADV_PAGEOUT )) +ksft_exit_fail_perror ("madvise(MADV_PAGEOUT)" );if (check_swap (p, (max_ptes_swap +1 ) * page_size)) {success ("OK" ); }else {
@@ -760,12 +712,10 @@ static void collapse_max_ptes_swap(struct collapse_context *c, struct mem_ops *oif (c->enforce_pte_scan_limits) { ops->fault (p,0 , hpage_pmd_size); -printf ("Swapout %d of %d pages..." , max_ptes_swap, +ksft_print_msg ("Swapout %d of %d pages..." , max_ptes_swap, hpage_pmd_nr); -if (madvise (p, max_ptes_swap * page_size,MADV_PAGEOUT )) { -perror ("madvise(MADV_PAGEOUT)" ); -exit (EXIT_FAILURE ); - } +if (madvise (p, max_ptes_swap * page_size,MADV_PAGEOUT )) +ksft_exit_fail_perror ("madvise(MADV_PAGEOUT)" );if (check_swap (p, max_ptes_swap * page_size)) {success ("OK" ); }else {
@@ -779,6 +729,7 @@ static void collapse_max_ptes_swap(struct collapse_context *c, struct mem_ops *o } out: ops->cleanup_area (p, hpage_pmd_size); +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void collapse_single_pte_entry_compound (struct collapse_context *c,struct mem_ops *ops)
@@ -795,7 +746,7 @@ static void collapse_single_pte_entry_compound(struct collapse_context *c, struc }madvise (p, hpage_pmd_size,MADV_NOHUGEPAGE ); -printf ("Split huge page leaving single PTE mapping compound page..." ); +ksft_print_msg ("Split huge page leaving single PTE mapping compound page..." );madvise (p + page_size, hpage_pmd_size - page_size,MADV_DONTNEED );if (ops->check_huge (p,0 ))success ("OK" );
@@ -807,6 +758,7 @@ static void collapse_single_pte_entry_compound(struct collapse_context *c, strucvalidate_memory (p,0 , page_size); skip: ops->cleanup_area (p, hpage_pmd_size); +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void collapse_full_of_compound (struct collapse_context *c,struct mem_ops *ops)
@@ -814,7 +766,7 @@ static void collapse_full_of_compound(struct collapse_context *c, struct mem_opsvoid *p; p =alloc_hpage (ops); -printf ("Split huge page leaving single PTE page table full of compound pages..." ); +ksft_print_msg ("Split huge page leaving single PTE page table full of compound pages..." );madvise (p, page_size,MADV_NOHUGEPAGE );madvise (p, hpage_pmd_size,MADV_NOHUGEPAGE );if (ops->check_huge (p,0 ))
@@ -826,6 +778,7 @@ static void collapse_full_of_compound(struct collapse_context *c, struct mem_ops true);validate_memory (p,0 , hpage_pmd_size); ops->cleanup_area (p, hpage_pmd_size); +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void collapse_compound_extreme (struct collapse_context *c,struct mem_ops *ops)
@@ -834,16 +787,12 @@ static void collapse_compound_extreme(struct collapse_context *c, struct mem_opsint i; p = ops->setup_area (1 ); +ksft_print_msg ("Construct PTE page table full of different PTE-mapped compound pagesd\n" );for (i =0 ; i < hpage_pmd_nr; i++) { -printf ("\rConstruct PTE page table full of different PTE-mapped compound pages %3d/%d..." , - i +1 , hpage_pmd_nr); -madvise (BASE_ADDR , hpage_pmd_size,MADV_HUGEPAGE ); ops->fault (BASE_ADDR ,0 , hpage_pmd_size); -if (!ops->check_huge (BASE_ADDR ,1 )) { -printf ("Failed to allocate huge page\n" ); -exit (EXIT_FAILURE ); - } +if (!ops->check_huge (BASE_ADDR ,1 )) +ksft_exit_fail_msg ("Failed to allocate huge page\n" );madvise (BASE_ADDR , hpage_pmd_size,MADV_NOHUGEPAGE ); p = mremap(BASE_ADDR - i * page_size,
@@ -851,20 +800,16 @@ static void collapse_compound_extreme(struct collapse_context *c, struct mem_ops (i +1 ) * page_size,MREMAP_MAYMOVE |MREMAP_FIXED , BASE_ADDR + 2 * hpage_pmd_size); -if (p ==MAP_FAILED ) { -perror ("mremap+unmap" ); -exit (EXIT_FAILURE ); - } +if (p ==MAP_FAILED ) +ksft_exit_fail_perror ("mremap+unmap" ); p =mremap (BASE_ADDR +2 * hpage_pmd_size, (i +1 ) * page_size, (i +1 ) * page_size + hpage_pmd_size,MREMAP_MAYMOVE |MREMAP_FIXED ,BASE_ADDR - (i +1 ) * page_size); -if (p ==MAP_FAILED ) { -perror ("mremap+alloc" ); -exit (EXIT_FAILURE ); - } +if (p ==MAP_FAILED ) +ksft_exit_fail_perror ("mremap+alloc" ); } ops->cleanup_area (BASE_ADDR , hpage_pmd_size);
@@ -879,6 +824,7 @@ static void collapse_compound_extreme(struct collapse_context *c, struct mem_opsvalidate_memory (p,0 , hpage_pmd_size); ops->cleanup_area (p, hpage_pmd_size); +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void collapse_fork (struct collapse_context *c,struct mem_ops *ops)
@@ -888,18 +834,17 @@ static void collapse_fork(struct collapse_context *c, struct mem_ops *ops) p = ops->setup_area (1 ); -printf ("Allocate small page..." ); +ksft_print_msg ("Allocate small page..." ); ops->fault (p,0 , page_size);if (ops->check_huge (p,0 ))success ("OK" );else fail ("Fail" ); -printf ("Share small page over fork()..." ); +ksft_print_msg ("Share small page over fork()..." );if (!fork ()) {/* Do not touch settings on child exit */ skip_settings_restore = true; - exit_status =0 ;if (ops->check_huge (p,0 ))success ("OK" );
@@ -916,15 +861,16 @@ static void collapse_fork(struct collapse_context *c, struct mem_ops *ops) }wait (&wstatus); - exit_status +=WEXITSTATUS (wstatus); + exit_status =WEXITSTATUS (wstatus); -printf ("Check if parent still has small page..." ); +ksft_print_msg ("Check if parent still has small page..." );if (ops->check_huge (p,0 ))success ("OK" );else fail ("Fail" );validate_memory (p,0 , page_size); ops->cleanup_area (p, hpage_pmd_size); +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void collapse_fork_compound (struct collapse_context *c,struct mem_ops *ops)
@@ -933,18 +879,17 @@ static void collapse_fork_compound(struct collapse_context *c, struct mem_ops *ovoid *p; p =alloc_hpage (ops); -printf ("Share huge page over fork()..." ); +ksft_print_msg ("Share huge page over fork()..." );if (!fork ()) {/* Do not touch settings on child exit */ skip_settings_restore = true; - exit_status =0 ;if (ops->check_huge (p,1 ))success ("OK" );else fail ("Fail" ); -printf ("Split huge page PMD in child process..." ); +ksft_print_msg ("Split huge page PMD in child process..." );madvise (p, page_size,MADV_NOHUGEPAGE );madvise (p, hpage_pmd_size,MADV_NOHUGEPAGE );if (ops->check_huge (p,0 ))
@@ -965,15 +910,16 @@ static void collapse_fork_compound(struct collapse_context *c, struct mem_ops *o }wait (&wstatus); - exit_status +=WEXITSTATUS (wstatus); + exit_status =WEXITSTATUS (wstatus); -printf ("Check if parent still has huge page..." ); +ksft_print_msg ("Check if parent still has huge page..." );if (ops->check_huge (p,1 ))success ("OK" );else fail ("Fail" );validate_memory (p,0 , hpage_pmd_size); ops->cleanup_area (p, hpage_pmd_size); +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void collapse_max_ptes_shared (struct collapse_context *c,struct mem_ops *ops)
@@ -983,18 +929,17 @@ static void collapse_max_ptes_shared(struct collapse_context *c, struct mem_opsvoid *p; p =alloc_hpage (ops); -printf ("Share huge page over fork()..." ); +ksft_print_msg ("Share huge page over fork()..." );if (!fork ()) {/* Do not touch settings on child exit */ skip_settings_restore = true; - exit_status =0 ;if (ops->check_huge (p,1 ))success ("OK" );else fail ("Fail" ); -printf ("Trigger CoW on page %d of %d..." , +ksft_print_msg ("Trigger CoW on page %d of %d..." , hpage_pmd_nr - max_ptes_shared -1 , hpage_pmd_nr); ops->fault (p,0 , (hpage_pmd_nr - max_ptes_shared -1 ) * page_size);if (ops->check_huge (p,0 ))
@@ -1006,7 +951,7 @@ static void collapse_max_ptes_shared(struct collapse_context *c, struct mem_ops 1, ops, !c->enforce_pte_scan_limits);if (c->enforce_pte_scan_limits) { -printf ("Trigger CoW on page %d of %d..." , +ksft_print_msg ("Trigger CoW on page %d of %d..." , hpage_pmd_nr - max_ptes_shared, hpage_pmd_nr); ops->fault (p,0 , (hpage_pmd_nr - max_ptes_shared) * page_size);
@@ -1025,15 +970,16 @@ static void collapse_max_ptes_shared(struct collapse_context *c, struct mem_ops }wait (&wstatus); - exit_status +=WEXITSTATUS (wstatus); + exit_status =WEXITSTATUS (wstatus); -printf ("Check if parent still has huge page..." ); +ksft_print_msg ("Check if parent still has huge page..." );if (ops->check_huge (p,1 ))success ("OK" );else fail ("Fail" );validate_memory (p,0 , hpage_pmd_size); ops->cleanup_area (p, hpage_pmd_size); +ksft_test_result_report (exit_status,"%s\n" , __func__); } static void madvise_collapse_existing_thps(struct collapse_context *c,
@@ -1050,6 +996,7 @@ static void madvise_collapse_existing_thps(struct collapse_context *c,__madvise_collapse ("Re-collapse PMD-mapped hugepage" , p,1 , ops, true);validate_memory (p,0 , hpage_pmd_size); ops->cleanup_area (p, hpage_pmd_size); +ksft_test_result_report (exit_status,"%s\n" , __func__); }/*
@@ -1077,6 +1024,7 @@ static void madvise_retracted_page_tables(struct collapse_context *c, true);validate_memory (p,0 , size); ops->cleanup_area (p, size); +ksft_test_result_report (exit_status,"%s\n" , __func__); }static void usage (void )
@@ -1180,10 +1128,8 @@ static int nr_test_cases;#define TEST (t, c, o) do { \ if (c && o) { \ - if (nr_test_cases >= MAX_TEST_CASES) { \ - printf("MAX_ADD_CASES is too small\n"); \ - exit(EXIT_FAILURE); \ - } \ + if (nr_test_cases >= MAX_TEST_CASES) \ + ksft_exit_fail_msg("MAX_ADD_CASES is too small\n"); \ test_cases[nr_test_cases++] = (struct test_case){ \ .ctx = c, \ .ops = o, \
@@ -1216,10 +1162,10 @@ int main(int argc, char **argv) .read_ahead_kb = 0, }; -if (!thp_is_enabled ()) { -printf ("Transparent Hugepages not available\n" ); -return KSFT_SKIP ; - } +ksft_print_header (); + +if (!thp_is_enabled ()) +ksft_exit_skip ("Transparent Hugepages not available\n" );parse_test_type (argc, argv);
@@ -1227,10 +1173,8 @@ int main(int argc, char **argv) page_size =getpagesize (); hpage_pmd_size =read_pmd_pagesize (); -if (!hpage_pmd_size) { -printf ("Reading PMD pagesize failed" ); -exit (EXIT_FAILURE ); - } +if (!hpage_pmd_size) +ksft_exit_fail_msg ("Reading PMD pagesize failed\n" ); hpage_pmd_nr = hpage_pmd_size / page_size; hpage_pmd_order =__builtin_ctz (hpage_pmd_nr);
@@ -1246,8 +1190,6 @@ int main(int argc, char **argv)save_settings ();thp_push_settings (&default_settings); -alloc_at_fault (); -TEST (collapse_full, khugepaged_context, anon_ops);TEST (collapse_full, khugepaged_context, file_ops);TEST (collapse_full, khugepaged_context, shmem_ops);
@@ -1307,11 +1249,14 @@ int main(int argc, char **argv)TEST (madvise_retracted_page_tables, madvise_context, file_ops);TEST (madvise_retracted_page_tables, madvise_context, shmem_ops); +ksft_set_plan (nr_test_cases +1 ); + +alloc_at_fault ();for (int i =0 ; i < nr_test_cases; i++) {struct test_case *t = &test_cases[i]; exit_status =KSFT_PASS ; -printf ("\nRun test: %s: (%s:%s)\n" , t->desc, t->ctx->name, t->ops->name); +ksft_print_msg ("\nRun test: %s: (%s:%s)\n" , t->desc, t->ctx->name, t->ops->name); t->fn (t->ctx, t->ops); }
Signature · 1 line
2.53.0
Mike Rapoport
maintainerPATCH v3 15/54 selftests/mm: ksm_tests: use kselftest framework
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Convert ksm_tests to use kselftest framework for reporting and tracking successful and failing runs.
Reviewed by: Donet Tom <[email protected]>
--- tools/testing/selftests/mm/ksm_tests.c | 180 +++++++++++-------------- 1 file changed, 81 insertions(+), 99 deletions(-)
tools/testing/selftests/mm/ksm_tests.c +81 -99
@@ -175,12 +175,12 @@ static void *allocate_memory(void *ptr, int prot, int mapping, char data, size_void *map_ptr =mmap (ptr, map_size,PROT_WRITE , mapping,-1 ,0 );if (!map_ptr) { -perror ("mmap" ); +ksft_perror ("mmap" );return NULL ; }memset (map_ptr, data, map_size);if (mprotect (map_ptr, map_size, prot)) { -perror ("mprotect" ); +ksft_perror ("mprotect" );munmap (map_ptr, map_size);return NULL ; }
@@ -201,11 +201,11 @@ static int ksm_do_scan(int scan_count, struct timespec start_time, int timeout)if (ksm_read_sysfs (KSM_FP ("full_scans" ), &cur_scan))return 1 ;if (clock_gettime (CLOCK_MONOTONIC_RAW , &cur_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" );return 1 ; }if ((cur_time.tv_sec - start_time.tv_sec) > timeout) { -printf ("Scan time limit exceeded\n" ); +ksft_print_msg ("Scan time limit exceeded\n" );return 1 ; } }
@@ -218,12 +218,12 @@ static int ksm_merge_pages(int merge_type, void *addr, size_t size, {if (merge_type ==KSM_MERGE_MADVISE ) {if (madvise (addr, size,MADV_MERGEABLE )) { -perror ("madvise" ); +ksft_perror ("madvise" );return 1 ; } }else if (merge_type ==KSM_MERGE_PRCTL ) {if (prctl (PR_SET_MEMORY_MERGE ,1 ,0 ,0 ,0 )) { -perror ("prctl" ); +ksft_perror ("prctl" );return 1 ; } }
@@ -242,7 +242,7 @@ static int ksm_unmerge_pages(void *addr, size_t size, struct timespec start_time, int timeout) { if (madvise(addr, size, MADV_UNMERGEABLE)) { -perror ("madvise" ); +ksft_perror ("madvise" );return 1 ; }return 0 ;
@@ -324,7 +324,7 @@ static int check_ksm_merge(int merge_type, int mapping, int prot,struct timespec start_time;if (clock_gettime (CLOCK_MONOTONIC_RAW , &start_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" );return KSFT_FAIL ; }
@@ -338,7 +338,6 @@ static int check_ksm_merge(int merge_type, int mapping, int prot,/* verify that the right number of pages are merged */ if (assert_ksm_pages_count (page_count)) { -printf ("OK\n" );munmap (map_ptr, page_size * page_count);if (merge_type ==KSM_MERGE_PRCTL )prctl (PR_SET_MEMORY_MERGE ,0 ,0 ,0 ,0 );
@@ -346,7 +345,6 @@ static int check_ksm_merge(int merge_type, int mapping, int prot, } err_out: -printf ("Not OK\n" );munmap (map_ptr, page_size * page_count);return KSFT_FAIL ; }
@@ -358,7 +356,7 @@ static int check_ksm_unmerge(int merge_type, int mapping, int prot, int timeout,int page_count =2 ;if (clock_gettime (CLOCK_MONOTONIC_RAW , &start_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" );return KSFT_FAIL ; }
@@ -380,13 +378,11 @@ static int check_ksm_unmerge(int merge_type, int mapping, int prot, int timeout,/* check that unmerging was successful and 0 pages are currently merged */ if (assert_ksm_pages_count (0 )) { -printf ("OK\n" );munmap (map_ptr, page_size * page_count);return KSFT_PASS ; } err_out: -printf ("Not OK\n" );munmap (map_ptr, page_size * page_count);return KSFT_FAIL ; }
@@ -398,7 +394,7 @@ static int check_ksm_zero_page_merge(int merge_type, int mapping, int prot, longstruct timespec start_time;if (clock_gettime (CLOCK_MONOTONIC_RAW , &start_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" );return KSFT_FAIL ; }
@@ -425,12 +421,10 @@ static int check_ksm_zero_page_merge(int merge_type, int mapping, int prot, longelse if (!use_zero_pages && !assert_ksm_pages_count (page_count)) goto err_out; -printf ("OK\n" );munmap (map_ptr, page_size * page_count);return KSFT_PASS ; err_out: -printf ("Not OK\n" );munmap (map_ptr, page_size * page_count);return KSFT_FAIL ; }
@@ -465,16 +459,16 @@ static int check_ksm_numa_merge(int merge_type, int mapping, int prot, int timeoint first_node;if (clock_gettime (CLOCK_MONOTONIC_RAW , &start_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" );return KSFT_FAIL ; }if (numa_available () <0 ) { -perror ("NUMA support not enabled" ); +ksft_perror ("NUMA support not enabled" );return KSFT_SKIP ; }if (numa_num_configured_nodes () <=1 ) { -printf ("At least 2 NUMA nodes must be available\n" ); +ksft_print_msg ("At least 2 NUMA nodes must be available\n" );return KSFT_SKIP ; }if (ksm_write_sysfs (KSM_FP ("merge_across_nodes" ), merge_across_nodes))
@@ -485,7 +479,7 @@ static int check_ksm_numa_merge(int merge_type, int mapping, int prot, int timeo numa1_map_ptr =numa_alloc_onnode (page_size, first_node); numa2_map_ptr =numa_alloc_onnode (page_size,get_next_mem_node (first_node));if (!numa1_map_ptr || !numa2_map_ptr) { -perror ("numa_alloc_onnode" ); +ksft_perror ("numa_alloc_onnode" );return KSFT_FAIL ; }
@@ -510,13 +504,11 @@ static int check_ksm_numa_merge(int merge_type, int mapping, int prot, int timeonuma_free (numa1_map_ptr, page_size);numa_free (numa2_map_ptr, page_size); -printf ("OK\n" );return KSFT_PASS ; err_out:numa_free (numa1_map_ptr, page_size);numa_free (numa2_map_ptr, page_size); -printf ("Not OK\n" );return KSFT_FAIL ; }
@@ -529,7 +521,7 @@ static int ksm_merge_hugepages_time(int merge_type, int mapping, int prot,int pagemap_fd, n_normal_pages, n_huge_pages;if (!thp_is_enabled ()) { -printf ("Transparent Hugepages not available\n" ); +ksft_print_msg ("Transparent Hugepages not available\n" );return KSFT_SKIP ; }
@@ -559,36 +551,35 @@ static int ksm_merge_hugepages_time(int merge_type, int mapping, int prot,else n_huge_pages++; } -printf ("Number of normal pages: %d\n" , n_normal_pages); -printf ("Number of huge pages: %d\n" , n_huge_pages); +ksft_print_msg ("Number of normal pages: %d\n" , n_normal_pages); +ksft_print_msg ("Number of huge pages: %d\n" , n_huge_pages);memset (map_ptr,'*' , len);if (clock_gettime (CLOCK_MONOTONIC_RAW , &start_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" ); goto err_out; }if (ksm_merge_pages (merge_type, map_ptr, map_size, start_time, timeout)) goto err_out;if (clock_gettime (CLOCK_MONOTONIC_RAW , &end_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" ); goto err_out; } scan_time_ns = (end_time.tv_sec - start_time.tv_sec) *NSEC_PER_SEC + (end_time.tv_nsec - start_time.tv_nsec); -printf ("Total size: %lu MiB\n" , map_size /MB ); -printf ("Total time: %ld.%09ld s\n" , scan_time_ns /NSEC_PER_SEC , +ksft_print_msg ("Total size: %lu MiB\n" , map_size /MB ); +ksft_print_msg ("Total time: %ld.%09ld s\n" , scan_time_ns /NSEC_PER_SEC , scan_time_ns %NSEC_PER_SEC ); -printf ("Average speed: %.3f MiB/s\n" , (map_size /MB ) / +ksft_print_msg ("Average speed: %.3f MiB/s\n" , (map_size /MB ) / ((double )scan_time_ns /NSEC_PER_SEC ));munmap (map_ptr_orig, len +HPAGE_SIZE );return KSFT_PASS ; err_out: -printf ("Not OK\n" );munmap (map_ptr_orig, len +HPAGE_SIZE );return KSFT_FAIL ; }
@@ -606,30 +597,29 @@ static int ksm_merge_time(int merge_type, int mapping, int prot, int timeout, sireturn KSFT_FAIL ;if (clock_gettime (CLOCK_MONOTONIC_RAW , &start_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" ); goto err_out; }if (ksm_merge_pages (merge_type, map_ptr, map_size, start_time, timeout)) goto err_out;if (clock_gettime (CLOCK_MONOTONIC_RAW , &end_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" ); goto err_out; } scan_time_ns = (end_time.tv_sec - start_time.tv_sec) *NSEC_PER_SEC + (end_time.tv_nsec - start_time.tv_nsec); -printf ("Total size: %lu MiB\n" , map_size /MB ); -printf ("Total time: %ld.%09ld s\n" , scan_time_ns /NSEC_PER_SEC , +ksft_print_msg ("Total size: %lu MiB\n" , map_size /MB ); +ksft_print_msg ("Total time: %ld.%09ld s\n" , scan_time_ns /NSEC_PER_SEC , scan_time_ns %NSEC_PER_SEC ); -printf ("Average speed: %.3f MiB/s\n" , (map_size /MB ) / +ksft_print_msg ("Average speed: %.3f MiB/s\n" , (map_size /MB ) / ((double )scan_time_ns /NSEC_PER_SEC ));munmap (map_ptr, map_size);return KSFT_PASS ; err_out: -printf ("Not OK\n" );munmap (map_ptr, map_size);return KSFT_FAIL ; }
@@ -646,37 +636,36 @@ static int ksm_unmerge_time(int merge_type, int mapping, int prot, int timeout,if (!map_ptr)return KSFT_FAIL ;if (clock_gettime (CLOCK_MONOTONIC_RAW , &start_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" ); goto err_out; }if (ksm_merge_pages (merge_type, map_ptr, map_size, start_time, timeout)) goto err_out;if (clock_gettime (CLOCK_MONOTONIC_RAW , &start_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" ); goto err_out; }if (ksm_unmerge_pages (map_ptr, map_size, start_time, timeout)) goto err_out;if (clock_gettime (CLOCK_MONOTONIC_RAW , &end_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" ); goto err_out; } scan_time_ns = (end_time.tv_sec - start_time.tv_sec) *NSEC_PER_SEC + (end_time.tv_nsec - start_time.tv_nsec); -printf ("Total size: %lu MiB\n" , map_size /MB ); -printf ("Total time: %ld.%09ld s\n" , scan_time_ns /NSEC_PER_SEC , +ksft_print_msg ("Total size: %lu MiB\n" , map_size /MB ); +ksft_print_msg ("Total time: %ld.%09ld s\n" , scan_time_ns /NSEC_PER_SEC , scan_time_ns %NSEC_PER_SEC ); -printf ("Average speed: %.3f MiB/s\n" , (map_size /MB ) / +ksft_print_msg ("Average speed: %.3f MiB/s\n" , (map_size /MB ) / ((double )scan_time_ns /NSEC_PER_SEC ));munmap (map_ptr, map_size);return KSFT_PASS ; err_out: -printf ("Not OK\n" );munmap (map_ptr, map_size);return KSFT_FAIL ; }
@@ -695,24 +684,24 @@ static int ksm_cow_time(int merge_type, int mapping, int prot, int timeout, sizereturn KSFT_FAIL ;if (clock_gettime (CLOCK_MONOTONIC_RAW , &start_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" );return KSFT_FAIL ; }for (size_t i =0 ; i < page_count -1 ; i = i +2 )memset (map_ptr + page_size * i,'-' ,1 );if (clock_gettime (CLOCK_MONOTONIC_RAW , &end_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" );return KSFT_FAIL ; } cow_time_ns = (end_time.tv_sec - start_time.tv_sec) *NSEC_PER_SEC + (end_time.tv_nsec - start_time.tv_nsec); -printf ("Total size: %lu MiB\n\n" , (page_size * page_count) /MB ); -printf ("Not merged pages:\n" ); -printf ("Total time: %ld.%09ld s\n" , cow_time_ns /NSEC_PER_SEC , +ksft_print_msg ("Total size: %lu MiB\n\n" , (page_size * page_count) /MB ); +ksft_print_msg ("Not merged pages:\n" ); +ksft_print_msg ("Total time: %ld.%09ld s\n" , cow_time_ns /NSEC_PER_SEC , cow_time_ns %NSEC_PER_SEC ); -printf ("Average speed: %.3f MiB/s\n\n" , ((page_size * (page_count /2 )) /MB ) / +ksft_print_msg ("Average speed: %.3f MiB/s\n\n" , ((page_size * (page_count /2 )) /MB ) / ((double )cow_time_ns /NSEC_PER_SEC ));/* Create 2000 pairs of duplicate pages */
@@ -724,30 +713,29 @@ static int ksm_cow_time(int merge_type, int mapping, int prot, int timeout, size goto err_out;if (clock_gettime (CLOCK_MONOTONIC_RAW , &start_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" ); goto err_out; }for (size_t i =0 ; i < page_count -1 ; i = i +2 )memset (map_ptr + page_size * i,'-' ,1 );if (clock_gettime (CLOCK_MONOTONIC_RAW , &end_time)) { -perror ("clock_gettime" ); +ksft_perror ("clock_gettime" ); goto err_out; } cow_time_ns = (end_time.tv_sec - start_time.tv_sec) *NSEC_PER_SEC + (end_time.tv_nsec - start_time.tv_nsec); -printf ("Merged pages:\n" ); -printf ("Total time: %ld.%09ld s\n" , cow_time_ns /NSEC_PER_SEC , +ksft_print_msg ("Merged pages:\n" ); +ksft_print_msg ("Total time: %ld.%09ld s\n" , cow_time_ns /NSEC_PER_SEC , cow_time_ns %NSEC_PER_SEC ); -printf ("Average speed: %.3f MiB/s\n" , ((page_size * (page_count /2 )) /MB ) / +ksft_print_msg ("Average speed: %.3f MiB/s\n" , ((page_size * (page_count /2 )) /MB ) / ((double )cow_time_ns /NSEC_PER_SEC ));munmap (map_ptr, page_size * page_count);return KSFT_PASS ; err_out: -printf ("Not OK\n" );munmap (map_ptr, page_size * page_count);return KSFT_FAIL ; }
@@ -765,6 +753,10 @@ int main(int argc, char *argv[])bool use_zero_pages =KSM_USE_ZERO_PAGES_DEFAULT ;bool merge_across_nodes =KSM_MERGE_ACROSS_NODES_DEFAULT ;long size_MB =0 ; +const char *test_descr ="KSM merging" ; + +ksft_print_header (); +ksft_set_plan (1 );while ((opt =getopt (argc, argv,"dha:p:l:z:m:s:t:MUZNPCHD" )) !=-1 ) {switch (opt) {
@@ -773,17 +765,13 @@ int main(int argc, char *argv[])break ;case 'p' : page_count =atol (optarg); -if (page_count <=0 ) { -printf ("The number of pages must be greater than 0\n" ); -return KSFT_FAIL ; - } +if (page_count <=0 ) +ksft_exit_fail_msg ("The number of pages must be greater than 0\n" );break ;case 'l' : ksm_scan_limit_sec =atoi (optarg); -if (ksm_scan_limit_sec <=0 ) { -printf ("Timeout value must be greater than 0\n" ); -return KSFT_FAIL ; - } +if (ksm_scan_limit_sec <=0 ) +ksft_exit_fail_msg ("Timeout value must be greater than 0\n" );break ;case 'h' :print_help ();
@@ -805,19 +793,15 @@ int main(int argc, char *argv[])break ;case 's' : size_MB =atoi (optarg); -if (size_MB <=0 ) { -printf ("Size must be greater than 0\n" ); -return KSFT_FAIL ; - } +if (size_MB <=0 ) +ksft_exit_fail_msg ("Size must be greater than 0\n" );break ;case 't' : {int tmp =atoi (optarg); -if (tmp <0 || tmp >KSM_MERGE_LAST ) { -printf ("Invalid merge type\n" ); -return KSFT_FAIL ; - } +if (tmp <0 || tmp >KSM_MERGE_LAST ) +ksft_exit_fail_msg ("Invalid merge type\n" ); merge_type = tmp; }break ;
@@ -845,82 +829,80 @@ int main(int argc, char *argv[]) test_name =KSM_COW_TIME ;break ;default : -return KSFT_FAIL ; +ksft_exit_fail_msg ("Unknown option\n" ); } }if (prot ==0 ) prot =str_to_prot (KSM_PROT_STR_DEFAULT ); -if (access (KSM_SYSFS_PATH ,F_OK )) { -printf ("Config KSM not enabled\n" ); -return KSFT_SKIP ; - } +if (access (KSM_SYSFS_PATH ,F_OK )) +ksft_exit_skip ("Config KSM not enabled\n" ); -if (ksm_save_def (&ksm_sysfs_old)) { -printf ("Cannot save default tunables\n" ); -return KSFT_FAIL ; - } +if (ksm_save_def (&ksm_sysfs_old)) +ksft_exit_fail_msg ("Cannot save default tunables\n" );if (ksm_write_sysfs (KSM_FP ("run" ),2 ) ||ksm_write_sysfs (KSM_FP ("sleep_millisecs" ),0 ) ||numa_available () ?0 :ksm_write_sysfs (KSM_FP ("merge_across_nodes" ),1 ) ||ksm_write_sysfs (KSM_FP ("pages_to_scan" ), page_count)) -return KSFT_FAIL ; +ksft_exit_fail_msg ("Cannot set up KSM tunables\n" );switch (test_name) {case CHECK_KSM_MERGE : + test_descr ="KSM merging" ; ret =check_ksm_merge (merge_type,MAP_PRIVATE |MAP_ANONYMOUS , prot, page_count, ksm_scan_limit_sec, page_size);break ;case CHECK_KSM_UNMERGE : + test_descr ="KSM unmerging" ; ret =check_ksm_unmerge (merge_type,MAP_PRIVATE |MAP_ANONYMOUS , prot, ksm_scan_limit_sec, page_size);break ;case CHECK_KSM_ZERO_PAGE_MERGE : + test_descr ="KSM zero page merging" ; ret =check_ksm_zero_page_merge (merge_type,MAP_PRIVATE |MAP_ANONYMOUS , prot, page_count, ksm_scan_limit_sec, use_zero_pages, page_size);break ;case CHECK_KSM_NUMA_MERGE : + test_descr ="KSM NUMA merging" ; ret =check_ksm_numa_merge (merge_type,MAP_PRIVATE |MAP_ANONYMOUS , prot, ksm_scan_limit_sec, merge_across_nodes, page_size);break ;case KSM_MERGE_TIME : -if (size_MB ==0 ) { -printf ("Option '-s' is required.\n" ); -return KSFT_FAIL ; - } +if (size_MB ==0 ) +ksft_exit_fail_msg ("Option '-s' is required\n" ); + test_descr ="KSM merge time" ; ret =ksm_merge_time (merge_type,MAP_PRIVATE |MAP_ANONYMOUS , prot, ksm_scan_limit_sec, size_MB);break ;case KSM_MERGE_TIME_HUGE_PAGES : -if (size_MB ==0 ) { -printf ("Option '-s' is required.\n" ); -return KSFT_FAIL ; - } +if (size_MB ==0 ) +ksft_exit_fail_msg ("Option '-s' is required\n" ); + test_descr ="KSM merge time with huge pages" ; ret =ksm_merge_hugepages_time (merge_type,MAP_PRIVATE |MAP_ANONYMOUS , prot, ksm_scan_limit_sec, size_MB);break ;case KSM_UNMERGE_TIME : -if (size_MB ==0 ) { -printf ("Option '-s' is required.\n" ); -return KSFT_FAIL ; - } +if (size_MB ==0 ) +ksft_exit_fail_msg ("Option '-s' is required\n" ); + test_descr ="KSM unmerge time" ; ret =ksm_unmerge_time (merge_type,MAP_PRIVATE |MAP_ANONYMOUS , prot, ksm_scan_limit_sec, size_MB);break ;case KSM_COW_TIME : + test_descr ="KSM COW time" ; ret =ksm_cow_time (merge_type,MAP_PRIVATE |MAP_ANONYMOUS , prot, ksm_scan_limit_sec, page_size);break ; } -if (ksm_restore (&ksm_sysfs_old)) { -printf ("Cannot restore default tunables\n" ); -return KSFT_FAIL ; - } +if (ksm_restore (&ksm_sysfs_old)) +ksft_print_msg ("Cannot restore default tunables\n" ); + +ksft_test_result_report (ret,"%s\n" , test_descr); -return ret; +ksft_finished (); }
Signature · 1 line
2.53.0
Re: [PATCH v3 15/54] selftests/mm: ksm_tests: use kselftest framework
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Convert
ksm_teststo use kselftest framework for reporting and tracking successful and failing runs.Reviewed by: Donet Tom <[email protected]>
Reviewed-by Mark Brown <[email protected]>Signed-off-by Mike Rapoport (Microsoft) <[email protected]>> ---
I've run the test before and after applying the patch. The patch correctly formats the output according to KTAP format with no change in results.
Mike Rapoport
maintainerPATCH v3 16/54 selftests/mm: protection_keys: use descriptive test names in TAP output
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Replace the numeric test index in TAP output with the actual test function name.
Use a structure containing function pointer and its name rather than only the function pointer in the pkey_tests array.
Reviewed by: Donet Tom <[email protected]>
--- tools/testing/selftests/mm/protection_keys.c | 55 +++++++++++--------- 1 file changed, 31 insertions(+), 24 deletions(-)
tools/testing/selftests/mm/protection_keys.c +31 -24
@@ -1692,29 +1692,36 @@ static void test_mprotect_pkey_on_unsupported_cpu(int *ptr, u16 pkey)pkey_assert (sret <0 ); } -static void (*pkey_tests[])(int *ptr,u16 pkey) = { - test_read_of_write_disabled_region, - test_read_of_access_disabled_region, - test_read_of_access_disabled_region_with_page_already_mapped, - test_write_of_write_disabled_region, - test_write_of_write_disabled_region_with_page_already_mapped, - test_write_of_access_disabled_region, - test_write_of_access_disabled_region_with_page_already_mapped, - test_kernel_write_of_access_disabled_region, - test_kernel_write_of_write_disabled_region, - test_kernel_gup_of_access_disabled_region, - test_kernel_gup_write_to_write_disabled_region, - test_executing_on_unreadable_memory, - test_implicit_mprotect_exec_only_memory, - test_mprotect_with_pkey_0, - test_ptrace_of_child, - test_pkey_init_state, - test_pkey_syscalls_on_non_allocated_pkey, - test_pkey_syscalls_bad_args, - test_pkey_alloc_exhaust, - test_pkey_alloc_free_attach_pkey0, +struct pkey_test { +void (*func)(int *ptr,u16 pkey); +const char *name; +}; + +#define PKEY_TEST (fn) { fn, #fn } + +static struct pkey_test pkey_tests[] = { +PKEY_TEST (test_read_of_write_disabled_region), +PKEY_TEST (test_read_of_access_disabled_region), +PKEY_TEST (test_read_of_access_disabled_region_with_page_already_mapped), +PKEY_TEST (test_write_of_write_disabled_region), +PKEY_TEST (test_write_of_write_disabled_region_with_page_already_mapped), +PKEY_TEST (test_write_of_access_disabled_region), +PKEY_TEST (test_write_of_access_disabled_region_with_page_already_mapped), +PKEY_TEST (test_kernel_write_of_access_disabled_region), +PKEY_TEST (test_kernel_write_of_write_disabled_region), +PKEY_TEST (test_kernel_gup_of_access_disabled_region), +PKEY_TEST (test_kernel_gup_write_to_write_disabled_region), +PKEY_TEST (test_executing_on_unreadable_memory), +PKEY_TEST (test_implicit_mprotect_exec_only_memory), +PKEY_TEST (test_mprotect_with_pkey_0), +PKEY_TEST (test_ptrace_of_child), +PKEY_TEST (test_pkey_init_state), +PKEY_TEST (test_pkey_syscalls_on_non_allocated_pkey), +PKEY_TEST (test_pkey_syscalls_bad_args), +PKEY_TEST (test_pkey_alloc_exhaust), +PKEY_TEST (test_pkey_alloc_free_attach_pkey0), #if defined(__i386__) || defined(__x86_64__) || defined(__aarch64__) - test_ptrace_modifies_pkru, +PKEY_TEST (test_ptrace_modifies_pkru), #endif };
@@ -1735,7 +1742,7 @@ static void run_tests_once(void)dprintf1 ("test %d starting with pkey: %d\n" , test_nr, pkey); ptr =malloc_pkey (PAGE_SIZE , prot, pkey);dprintf1 ("test %d starting...\n" , test_nr); - pkey_tests[test_nr](ptr, pkey); + pkey_tests[test_nr].func (ptr, pkey);dprintf1 ("freeing test memory: %p\n" , ptr);free_pkey_malloc (ptr);sys_pkey_free (pkey);
@@ -1746,7 +1753,7 @@ static void run_tests_once(void)tracing_off ();close_test_fds (); -printf ("test %2d PASSED (iteration %d)\n" , test_nr, iteration_nr); +printf ("test %s PASSED (iteration %d)\n" , pkey_tests[test_nr].name, iteration_nr);dprintf1 ("======================\n\n" ); } iteration_nr++;
Signature · 1 line
2.53.0
Mike Rapoport
maintainerPATCH v3 17/54 selftests/mm: protection_keys: use kselftest framework
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Convert protection_keys test to use kselftest framework for reporting and tracking successful and failing runs.
Adjust dprintf0() printouts to use "#" in the beginning of the line for TAP compatibility.
--- tools/testing/selftests/mm/pkey-helpers.h | 6 +++-- tools/testing/selftests/mm/protection_keys.c | 26 +++++++++++--------- 2 files changed, 19 insertions(+), 13 deletions(-)
tools/testing/selftests/mm/pkey-helpers.h +4 -2
@@ -71,10 +71,12 @@ static inline void sigsafe_printf(const char *format, ...)extern void abort_hooks (void );#define pkey_assert (condition) do { \ if (!(condition)) { \ - dprintf0("assert() at %s::%d test_nr: %d iteration: %d\n", \ + dprintf0("# assert() at %s::%d test_nr: %d iteration: %d\n", \ __FILE__, __LINE__, \ test_nr, iteration_nr); \ - dprintf0("errno at assert: %d", errno); \ + dprintf0("# errno at assert: %d\n", errno); \ + ksft_test_result_fail("test %d (iteration %d)\n", \ + test_nr, iteration_nr); \ abort_hooks(); \ exit(__LINE__); \ } \
tools/testing/selftests/mm/protection_keys.c +15 -11
@@ -136,6 +136,7 @@ static void tracing_off(void)void abort_hooks (void ) { +fflush (stdout);fprintf (stderr,"running %s()...\n" , __func__);tracing_off ();#ifdef SLEEP_ON_ABORT
@@ -370,8 +371,8 @@ static void signal_handler(int signum, siginfo_t *si, void *vucontext)if ((si->si_code ==SEGV_MAPERR ) || (si->si_code ==SEGV_ACCERR ) || (si->si_code ==SEGV_BNDERR )) { -printf ("non-PK si_code, exiting...\n" ); -exit (4 ); +dprintf0 ("# non-PK si_code: %d, exiting...\n" , si->si_code); +exit (1 ); } si_pkey_ptr =siginfo_get_pkey_ptr (si);
@@ -719,7 +720,7 @@ static void setup_hugetlbfs(void)long hpagesz_mb;if (geteuid () !=0 ) { -fprintf (stderr,"WARNING: not run as root, can not do hugetlb test\n" ); +ksft_print_msg ("WARNING: not run as root, can not do hugetlb test\n" );return ; }
@@ -855,7 +856,7 @@ void expected_pkey_fault(int pkey)#define do_not_expect_pkey_fault (msg) do { \ if (last_pkey_faults != pkey_faults) \ - dprintf0("unexpected PKey fault: %s\n", msg); \ + dprintf0("# unexpected PKey fault: %s\n", msg); \ pkey_assert(last_pkey_faults == pkey_faults); \ } while (0)
@@ -1753,7 +1754,7 @@ static void run_tests_once(void)tracing_off ();close_test_fds (); -printf ("test %s PASSED (iteration %d)\n" , pkey_tests[test_nr].name, iteration_nr); +ksft_test_result_pass ("test %s (iteration %d)\n" , pkey_tests[test_nr].name, iteration_nr);dprintf1 ("======================\n\n" ); } iteration_nr++;
@@ -1773,27 +1774,30 @@ int main(void)setup_handlers (); -printf ("has pkeys: %d\n" , pkeys_supported); +ksft_print_header ();if (!pkeys_supported) {int size =PAGE_SIZE ;int *ptr; -printf ("running PKEY tests for unsupported CPU/OS\n" ); +ksft_set_plan (1 ); +ksft_print_msg ("running PKEY tests for unsupported CPU/OS\n" ); ptr =mmap (NULL , size,PROT_NONE ,MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 );assert (ptr != (void *)-1 );test_mprotect_pkey_on_unsupported_cpu (ptr,1 ); -exit (0 ); +ksft_test_result_pass ("pkey on unsupported CPU/OS\n" ); +ksft_finished (); } +ksft_set_plan (ARRAY_SIZE (pkey_tests) * nr_iterations); +pkey_setup_shadow (); -printf ("startup pkey_reg: %016llx\n" ,read_pkey_reg ()); +ksft_print_msg ("startup pkey_reg: %016llx\n" ,read_pkey_reg ());setup_hugetlbfs ();while (nr_iterations-- >0 )run_tests_once (); -printf ("done (all tests OK)\n" ); -return 0 ; +ksft_finished (); }
Signature · 1 line
2.53.0
Re: [PATCH v3 17/54] selftests/mm: protection_keys: use kselftest framework
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Convert
protection_keystest to use kselftest framework for reporting and tracking successful and failing runs.Adjust
dprintf0()printouts to use "#" in the beginning of the line for TAP compatibility.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I have run the test before and after applying the patch. It correctly converts the test output into KTAP format.
Quoted 111 lines
--- tools/testing/selftests/mm/pkey-helpers.h | 6 +++-- tools/testing/selftests/mm/protection_keys.c | 26 +++++++++++--------- 2 files changed, 19 insertions(+), 13 deletions(-)tools/testing/selftests/mm/pkey-helpers.h +4
-2@@ -71,10 +71,12 @@ static inline void sigsafe_printf(const char *format, ...)extern void abort_hooks (void );#define pkey_assert (condition) do { \ if (!(condition)) { \ - dprintf0("assert() at %s::%d test_nr: %d iteration: %d\n", \ + dprintf0("# assert() at %s::%d test_nr: %d iteration: %d\n", \ __FILE__, __LINE__, \ test_nr, iteration_nr); \ - dprintf0("errno at assert: %d", errno); \ + dprintf0("# errno at assert: %d\n", errno); \ + ksft_test_result_fail("test %d (iteration %d)\n", \ + test_nr, iteration_nr); \ abort_hooks(); \ exit(__LINE__); \ } \tools/testing/selftests/mm/protection_keys.c +15
-11@@ -136,6 +136,7 @@ static void tracing_off(void)void abort_hooks (void ) { +fflush (stdout);fprintf (stderr,"running %s()...\n" , __func__);tracing_off ();#ifdef SLEEP_ON_ABORT @@ -370,8 +371,8 @@ static void signal_handler(int signum, siginfo_t *si, void *vucontext)if ((si->si_code ==SEGV_MAPERR ) || (si->si_code ==SEGV_ACCERR ) || (si->si_code ==SEGV_BNDERR )) { -printf ("non-PK si_code, exiting...\n" ); -exit (4 ); +dprintf0 ("# non-PK si_code: %d, exiting...\n" , si->si_code); +exit (1 ); } si_pkey_ptr =siginfo_get_pkey_ptr (si);@@ -719,7 +720,7 @@ static void setup_hugetlbfs(void)long hpagesz_mb;if (geteuid () !=0 ) { -fprintf (stderr,"WARNING: not run as root, can not do hugetlb test\n" ); +ksft_print_msg ("WARNING: not run as root, can not do hugetlb test\n" );return ; }@@ -855,7 +856,7 @@ void expected_pkey_fault(int pkey)#define do_not_expect_pkey_fault (msg) do { \ if (last_pkey_faults != pkey_faults) \ - dprintf0("unexpected PKey fault: %s\n", msg); \ + dprintf0("# unexpected PKey fault: %s\n", msg); \ pkey_assert(last_pkey_faults == pkey_faults); \ } while (0)@@ -1753,7 +1754,7 @@ static void run_tests_once(void)tracing_off ();close_test_fds (); -printf ("test %s PASSED (iteration %d)\n" , pkey_tests[test_nr].name, iteration_nr); +ksft_test_result_pass ("test %s (iteration %d)\n" , pkey_tests[test_nr].name, iteration_nr);dprintf1 ("======================\n\n" ); } iteration_nr++;@@ -1773,27 +1774,30 @@ int main(void)setup_handlers (); -printf ("has pkeys: %d\n" , pkeys_supported); +ksft_print_header ();if (!pkeys_supported) {int size =PAGE_SIZE ;int *ptr; -printf ("running PKEY tests for unsupported CPU/OS\n" ); +ksft_set_plan (1 ); +ksft_print_msg ("running PKEY tests for unsupported CPU/OS\n" ); ptr =mmap (NULL , size,PROT_NONE ,MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 );assert (ptr != (void *)-1 );test_mprotect_pkey_on_unsupported_cpu (ptr,1 ); -exit (0 ); +ksft_test_result_pass ("pkey on unsupported CPU/OS\n" ); +ksft_finished (); } +ksft_set_plan (ARRAY_SIZE (pkey_tests) * nr_iterations); +pkey_setup_shadow (); -printf ("startup pkey_reg: %016llx\n" ,read_pkey_reg ()); +ksft_print_msg ("startup pkey_reg: %016llx\n" ,read_pkey_reg ());setup_hugetlbfs ();while (nr_iterations-- >0 )run_tests_once (); -printf ("done (all tests OK)\n" ); -return 0 ; +ksft_finished (); }
Mike Rapoport
maintainerPATCH v3 18/54 selftests/mm: uffd-stress: use kselftest framework
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Convert uffd-stress test to use kselftest framework for reporting and tracking successful and failing runs.
--- tools/testing/selftests/mm/uffd-stress.c | 41 +++++++++++------------- 1 file changed, 19 insertions(+), 22 deletions(-)
tools/testing/selftests/mm/uffd-stress.c +19 -22
@@ -286,19 +286,12 @@ static int userfaultfd_stress(uffd_global_test_opts_t *gopts)pthread_attr_setstacksize (&attr,16 *1024 *1024 );while (bounces--) { -printf ("bounces: %d, mode:" , bounces); -if (bounces &BOUNCE_RANDOM ) -printf (" rnd" ); -if (bounces &BOUNCE_RACINGFAULTS ) -printf (" racing" ); -if (bounces &BOUNCE_VERIFY ) -printf (" ver" ); -if (bounces &BOUNCE_POLL ) -printf (" poll" ); -else -printf (" read" ); -printf (", " ); -fflush (stdout); +ksft_print_msg ("bounces: %d, mode:%s%s%s%s, " , + bounces, + bounces &BOUNCE_RANDOM ?" rnd" :"" , + bounces &BOUNCE_RACINGFAULTS ?" racing" :"" , + bounces &BOUNCE_VERIFY ?" ver" :"" , + bounces &BOUNCE_POLL ?" poll" :" read" );if (bounces &BOUNCE_POLL )fcntl (gopts->uffd,F_SETFL , gopts->uffd_flags |O_NONBLOCK );
@@ -456,6 +449,9 @@ int main(int argc, char **argv)unsigned long nr_cpus;size_t bytes; +ksft_print_header (); +ksft_set_plan (1 ); + gopts = (uffd_global_test_opts_t *)malloc (sizeof (uffd_global_test_opts_t));if (argc <4 )
@@ -484,10 +480,8 @@ int main(int argc, char **argv)* for racy extra reservation of hugepages. */ if (gopts->test_type ==TEST_HUGETLB && -get_free_hugepages () <2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 ) { -printf ("skip: Skipping userfaultfd... not enough hugepages\n" ); -return KSFT_SKIP ; - } +get_free_hugepages () <2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 ) +ksft_exit_skip ("Skipping userfaultfd... not enough hugepages\n" ); gopts->nr_pages_per_cpu = bytes / gopts->page_size / gopts->nr_parallel;if (!gopts->nr_pages_per_cpu) {
@@ -503,9 +497,12 @@ int main(int argc, char **argv) } gopts->nr_pages = gopts->nr_pages_per_cpu * gopts->nr_parallel; -printf ("nr_pages: %lu, nr_pages_per_cpu: %lu\n" , - gopts->nr_pages, gopts->nr_pages_per_cpu); -return userfaultfd_stress (gopts); +ksft_print_msg ("nr_pages: %lu, nr_pages_per_cpu: %lu\n" , + gopts->nr_pages, gopts->nr_pages_per_cpu); + +ksft_test_result (!userfaultfd_stress (gopts), +"uffd-stress %s\n" , argv[1 ]); +ksft_finished (); }#else /* __NR_userfaultfd */
@@ -514,8 +511,8 @@ int main(int argc, char **argv)int main (void ) { -printf ("skip: Skipping userfaultfd test (missing __NR_userfaultfd)\n" ); -return KSFT_SKIP ; +ksft_print_header (); +ksft_exit_skip ("missing __NR_userfaultfd definition\n" ); }#endif /* __NR_userfaultfd */
Signature · 1 line
2.53.0
Re: [PATCH v3 18/54] selftests/mm: uffd-stress: use kselftest framework
Quoted 7 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Convert uffd-stress test to use kselftest framework for reporting and tracking successful and failing runs.
Reviewed-by Mark Brown <[email protected]>Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I've run the test before and after applying the patch. The patch correctly formats the output according to KTAP format with no change in results.
Quoted 84 lines
--- tools/testing/selftests/mm/uffd-stress.c | 41 +++++++++++------------- 1 file changed, 19 insertions(+), 22 deletions(-)tools/testing/selftests/mm/uffd-stress.c +19
-22@@ -286,19 +286,12 @@ static int userfaultfd_stress(uffd_global_test_opts_t *gopts)pthread_attr_setstacksize (&attr,16 *1024 *1024 );while (bounces--) { -printf ("bounces: %d, mode:" , bounces); -if (bounces &BOUNCE_RANDOM ) -printf (" rnd" ); -if (bounces &BOUNCE_RACINGFAULTS ) -printf (" racing" ); -if (bounces &BOUNCE_VERIFY ) -printf (" ver" ); -if (bounces &BOUNCE_POLL ) -printf (" poll" ); -else -printf (" read" ); -printf (", " ); -fflush (stdout); +ksft_print_msg ("bounces: %d, mode:%s%s%s%s, " , + bounces, + bounces &BOUNCE_RANDOM ?" rnd" :"" , + bounces &BOUNCE_RACINGFAULTS ?" racing" :"" , + bounces &BOUNCE_VERIFY ?" ver" :"" , + bounces &BOUNCE_POLL ?" poll" :" read" );if (bounces &BOUNCE_POLL )fcntl (gopts->uffd,F_SETFL , gopts->uffd_flags |O_NONBLOCK );@@ -456,6 +449,9 @@ int main(int argc, char **argv)unsigned long nr_cpus;size_t bytes; +ksft_print_header (); +ksft_set_plan (1 ); + gopts = (uffd_global_test_opts_t *)malloc (sizeof (uffd_global_test_opts_t));if (argc <4 )@@ -484,10 +480,8 @@ int main(int argc, char **argv)* for racy extra reservation of hugepages. */ if (gopts->test_type ==TEST_HUGETLB && -get_free_hugepages () <2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 ) { -printf ("skip: Skipping userfaultfd... not enough hugepages\n" ); -return KSFT_SKIP ; - } +get_free_hugepages () <2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 ) +ksft_exit_skip ("Skipping userfaultfd... not enough hugepages\n" ); gopts->nr_pages_per_cpu = bytes / gopts->page_size / gopts->nr_parallel;if (!gopts->nr_pages_per_cpu) {@@ -503,9 +497,12 @@ int main(int argc, char **argv) } gopts->nr_pages = gopts->nr_pages_per_cpu * gopts->nr_parallel; -printf ("nr_pages: %lu, nr_pages_per_cpu: %lu\n" , - gopts->nr_pages, gopts->nr_pages_per_cpu); -return userfaultfd_stress (gopts); +ksft_print_msg ("nr_pages: %lu, nr_pages_per_cpu: %lu\n" , + gopts->nr_pages, gopts->nr_pages_per_cpu); + +ksft_test_result (!userfaultfd_stress (gopts), +"uffd-stress %s\n" , argv[1 ]); +ksft_finished (); }#else /* __NR_userfaultfd */ @@ -514,8 +511,8 @@ int main(int argc, char **argv)int main (void ) { -printf ("skip: Skipping userfaultfd test (missing __NR_userfaultfd)\n" ); -return KSFT_SKIP ; +ksft_print_header (); +ksft_exit_skip ("missing __NR_userfaultfd definition\n" ); }#endif /* __NR_userfaultfd */
Mike Rapoport
maintainerPATCH v3 19/54 selftests/mm: uffd-unit-tests: use kselftest framework
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Convert uffd-unit-tests to use kselftest framework for reporting and tracking successful and failing runs.
Reviewed by: Donet Tom <[email protected]>
--- tools/testing/selftests/mm/uffd-unit-tests.c | 79 +++++++++++--------- 1 file changed, 44 insertions(+), 35 deletions(-)
tools/testing/selftests/mm/uffd-unit-tests.c +44 -35
@@ -86,39 +86,24 @@ typedef struct {uffd_test_case_ops_t *test_case_ops; } uffd_test_case_t; -static void uffd_test_report (void ) -{ -printf ("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n" , -ksft_get_pass_cnt (), -ksft_get_xskip_cnt (), -ksft_get_fail_cnt (), -ksft_test_num ()); -} +static char current_test[256 ];static void uffd_test_pass (void ) { -printf ("done\n" ); -ksft_inc_pass_cnt (); +ksft_test_result_pass ("%s\n" , current_test); }#define uffd_test_start (...) do { \ - printf("Testing "); \ - printf(__VA_ARGS__); \ - printf("... "); \ - fflush(stdout); \ + snprintf(current_test, sizeof(current_test), __VA_ARGS__); \ } while (0)#define uffd_test_fail (...) do { \ - printf("failed [reason: "); \ - printf(__VA_ARGS__); \ - printf("]\n"); \ - ksft_inc_fail_cnt(); \ + ksft_test_result_fail("%s\n", current_test); \ } while (0)static void uffd_test_skip (const char *message) { -printf ("skipped [reason: %s]\n" , message); -ksft_inc_xskip_cnt (); +ksft_test_result_skip ("%s (%s)\n" , current_test, message); }/*
@@ -1701,6 +1686,26 @@ static void usage(const char *prog)exit (KSFT_FAIL ); } +static int uffd_count_tests (int n_tests,int n_mems,const char *test_filter) +{ +uffd_test_case_t *test; +int i, j, count =0 ; + +if (!test_filter) + count +=2 ;/* test_uffd_api(false) + test_uffd_api(true) */ + +for (i =0 ; i < n_tests; i++) { + test = &uffd_tests[i]; +if (test_filter && !strstr (test->name, test_filter)) +continue ; +for (j =0 ; j < n_mems; j++) +if (test->mem_targets & mem_types[j].mem_flag) + count++; + } + +return count; +} +int main (int argc,char *argv[]) {int n_tests =sizeof (uffd_tests) /sizeof (uffd_test_case_t);
@@ -1730,24 +1735,31 @@ int main(int argc, char *argv[]) } } -if (!test_filter && !list_only) { +if (list_only) { +for (i =0 ; i < n_tests; i++) { + test = &uffd_tests[i]; +if (test_filter && !strstr (test->name, test_filter)) +continue ; +printf ("%s\n" , test->name); + } +return KSFT_PASS ; + } + +ksft_print_header (); +ksft_set_plan (uffd_count_tests (n_tests, n_mems, test_filter)); + +if (!test_filter) { has_uffd =test_uffd_api (false); has_uffd |=test_uffd_api (true); -if (!has_uffd) { -printf ("Userfaultfd not supported or unprivileged, skip all tests\n" ); -exit (KSFT_SKIP ); - } +if (!has_uffd) +ksft_exit_skip ("Userfaultfd not supported or unprivileged\n" ); }for (i =0 ; i < n_tests; i++) { test = &uffd_tests[i];if (test_filter && !strstr (test->name, test_filter))continue ; -if (list_only) { -printf ("%s\n" , test->name); -continue ; - }for (j =0 ; j < n_mems; j++) { mem_type = &mem_types[j];
@@ -1794,10 +1806,7 @@ int main(int argc, char *argv[]) } } -if (!list_only) -uffd_test_report (); - -return ksft_get_fail_cnt () ?KSFT_FAIL :KSFT_PASS ; +ksft_finished (); }#else /* __NR_userfaultfd */
@@ -1806,8 +1815,8 @@ int main(int argc, char *argv[])int main (void ) { -printf ("Skipping %s (missing __NR_userfaultfd)\n" , __file__); -return KSFT_SKIP ; +ksft_print_header (); +ksft_exit_skip ("missing __NR_userfaultfd definition\n" ); }#endif /* __NR_userfaultfd */
Signature · 1 line
2.53.0
Re: [PATCH v3 19/54] selftests/mm: uffd-unit-tests: use kselftest framework
Hi Mike!
Quoted 107 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Convert uffd-unit-tests to use kselftest framework for reporting and tracking successful and failing runs.
Reviewed by: Donet Tom <[email protected]>
Reviewed-by Mark Brown <[email protected]>Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/uffd-unit-tests.c | 79 +++++++++++--------- 1 file changed, 44 insertions(+), 35 deletions(-)tools/testing/selftests/mm/uffd-unit-tests.c +38
-21@@ -86,39 +86,24 @@ typedef struct {uffd_test_case_ops_t *test_case_ops; } uffd_test_case_t; -static void uffd_test_report (void ) -{ -printf ("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n" , -ksft_get_pass_cnt (), -ksft_get_xskip_cnt (), -ksft_get_fail_cnt (), -ksft_test_num ()); -} +static char current_test[256 ];static void uffd_test_pass (void ) { -printf ("done\n" ); -ksft_inc_pass_cnt (); +ksft_test_result_pass ("%s\n" , current_test); }#define uffd_test_start (...) do { \ - printf("Testing "); \ - printf(__VA_ARGS__); \ - printf("... "); \ - fflush(stdout); \ + snprintf(current_test, sizeof(current_test), __VA_ARGS__); \ } while (0)#define uffd_test_fail (...) do { \ - printf("failed [reason: "); \ - printf(__VA_ARGS__); \ - printf("]\n"); \ - ksft_inc_fail_cnt(); \ + ksft_test_result_fail("%s\n", current_test); \ } while (0)static void uffd_test_skip (const char *message) { -printf ("skipped [reason: %s]\n" , message); -ksft_inc_xskip_cnt (); +ksft_test_result_skip ("%s (%s)\n" , current_test, message); }/* @@ -1701,6 +1686,26 @@ static void usage(const char *prog)exit (KSFT_FAIL ); } +static int uffd_count_tests (int n_tests,int n_mems,const char *test_filter) +{ +uffd_test_case_t *test; +int i, j, count =0 ; + +if (!test_filter) + count +=2 ;/* test_uffd_api(false) + test_uffd_api(true) */ + +for (i =0 ; i < n_tests; i++) { + test = &uffd_tests[i]; +if (test_filter && !strstr (test->name, test_filter)) +continue ; +for (j =0 ; j < n_mems; j++) +if (test->mem_targets & mem_types[j].mem_flag) + count++; + } + +return count; +} +int main (int argc,char *argv[]) {int n_tests =sizeof (uffd_tests) /sizeof (uffd_test_case_t);@@ -1730,24 +1735,31 @@ int main(int argc, char *argv[]) } } -if (!test_filter && !list_only) { +if (list_only) { +for (i =0 ; i < n_tests; i++) { + test = &uffd_tests[i]; +if (test_filter && !strstr (test->name, test_filter)) +continue ; +printf ("%s\n" , test->name); + } +return KSFT_PASS ; + } + +ksft_print_header (); +ksft_set_plan (uffd_count_tests (n_tests, n_mems, test_filter)); +
tools/testing/selftests/mm/uffd-unit-tests.c int main(int argc, char *argv[]) · line 1756
Quoted 10 lines
tools/testing/selftests/mm/uffd-unit-tests.c +3
-4+if (!test_filter) { has_uffd =test_uffd_api (false); has_uffd |=test_uffd_api (true); -if (!has_uffd) { -printf ("Userfaultfd not supported or unprivileged, skip all tests\n" ); -exit (KSFT_SKIP ); - } +if (!has_uffd) +ksft_exit_skip ("Userfaultfd not supported or unprivileged\n" );
If has_uffd is 0, two test outputs from test_uffd_api() are being printed and the third output is from the ksft_exit_skip(). Since we have declared the ksft plan before this, we are getting the following diagnostic line:
# Planned tests != run tests (67 != 3)
If we remove this if (!has_uffd) check, still all the tests are individually being skipped with a "feature missing" message and the planned tests are being equal to the run tests. So do we really need this check or can this be skipped?
Quoted 36 lines
tools/testing/selftests/mm/uffd-unit-tests.c +3
-10}for (i =0 ; i < n_tests; i++) { test = &uffd_tests[i];if (test_filter && !strstr (test->name, test_filter))continue ; -if (list_only) { -printf ("%s\n" , test->name); -continue ; - }for (j =0 ; j < n_mems; j++) { mem_type = &mem_types[j];@@ -1794,10 +1806,7 @@ int main(int argc, char *argv[]) } } -if (!list_only) -uffd_test_report (); - -return ksft_get_fail_cnt () ?KSFT_FAIL :KSFT_PASS ; +ksft_finished (); }#else /* __NR_userfaultfd */ @@ -1806,8 +1815,8 @@ int main(int argc, char *argv[])int main (void ) { -printf ("Skipping %s (missing __NR_userfaultfd)\n" , __file__); -return KSFT_SKIP ; +ksft_print_header (); +ksft_exit_skip ("missing __NR_userfaultfd definition\n" ); }#endif /* __NR_userfaultfd */
Mike Rapoport
maintainerRe: [PATCH v3 19/54] selftests/mm: uffd-unit-tests: use kselftest framework
Hi Sarthak,
Quoted 21 lines · Sarthak Sharma
Quoted 4 lines · Mike Rapoport
tools/testing/selftests/mm/uffd-unit-tests.c +4
-0+ +ksft_print_header (); +ksft_set_plan (uffd_count_tests (n_tests, n_mems, test_filter)); +tools/testing/selftests/mm/uffd-unit-tests.c int main(int argc, char *argv[]) · line 1756
Quoted 10 lines
tools/testing/selftests/mm/uffd-unit-tests.c +3
-4+if (!test_filter) { has_uffd =test_uffd_api (false); has_uffd |=test_uffd_api (true); -if (!has_uffd) { -printf ("Userfaultfd not supported or unprivileged, skip all tests\n" ); -exit (KSFT_SKIP ); - } +if (!has_uffd) +ksft_exit_skip ("Userfaultfd not supported or unprivileged\n" );If
has_uffdis 0, two test outputs fromtest_uffd_api()are being printed and the third output is from theksft_exit_skip(). Since we have declared the ksft plan before this, we are getting the following diagnostic line:
Quoted 6 lines
# Planned tests != run tests (67 != 3)
If we remove this if (!has_uffd) check, still all the tests are individually being skipped with a "feature missing" message and the planned tests are being equal to the run tests. So do we really need this check or can this be skipped?
Apparently not :)
Signature · 2 lines
Sincerely yours, Mike.
Mike Rapoport
maintainerPATCH v3 20/54 selftests/mm: va_high_addr_switch: use kselftest framework
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Convert va_high_addr_switch test to use kselftest framework for reporting and tracking successful and failing runs.
--- .../selftests/mm/va_high_addr_switch.c | 41 +++++++++---------- 1 file changed, 20 insertions(+), 21 deletions(-)
tools/testing/selftests/mm/va_high_addr_switch.c +20 -21
@@ -257,40 +257,35 @@ void testcases_init(void) switch_hint = addr_switch_hint; } -static int run_test (struct testcase *test,int count) +static void run_test (struct testcase *test,int count) {void *p; -int i, ret =KSFT_PASS ; +int i;for (i =0 ; i < count; i++) {struct testcase *t = test + i; p =mmap (t->addr, t->size,PROT_READ |PROT_WRITE , t->flags,-1 ,0 ); - -printf ("%s: %p - " , t->msg, p); -if (p ==MAP_FAILED ) { -printf ("FAILED\n" ); - ret =KSFT_FAIL ; +ksft_perror ("MAP_FAILED" ); +ksft_test_result_fail ("%s\n" , t->msg);continue ; }if (t->low_addr_required && p >= (void *)(switch_hint)) { -printf ("FAILED\n" ); - ret =KSFT_FAIL ; +ksft_print_msg ("%p not below switch hint\n" , p); +ksft_test_result_fail ("%s\n" , t->msg); }else {/* * Do a dereference of the address returned so that we catch * bugs in page fault handling */ memset (p,0 , t->size); -printf ("OK\n" ); +ksft_test_result_pass ("%s\n" , t->msg); }if (!t->keep_mapped)munmap (p, t->size); } - -return ret; }#ifdef __aarch64__
@@ -322,19 +317,23 @@ static int supported_arch(void)int main (int argc,char **argv) { -int ret, hugetlb_ret =KSFT_PASS ; +bool run_hugetlb = false; + +ksft_print_header ();if (!supported_arch ()) -return KSFT_SKIP ; +ksft_exit_skip ("Architecture not supported\n" ); + +if (argc ==2 && !strcmp (argv[1 ],"--run-hugetlb" )) + run_hugetlb = true;testcases_init (); - ret =run_test (testcases, sz_testcases); -if (argc ==2 && !strcmp (argv[1 ],"--run-hugetlb" )) - hugetlb_ret =run_test (hugetlb_testcases, sz_hugetlb_testcases); +ksft_set_plan (sz_testcases + (run_hugetlb ? sz_hugetlb_testcases :0 )); + +run_test (testcases, sz_testcases); +if (run_hugetlb) +run_test (hugetlb_testcases, sz_hugetlb_testcases); -if (ret ==KSFT_PASS && hugetlb_ret ==KSFT_PASS ) -return KSFT_PASS ; -else -return KSFT_FAIL ; +ksft_finished (); }
Signature · 1 line
2.53.0
Re: [PATCH v3 20/54] selftests/mm: va_high_addr_switch: use kselftest framework
Quoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Convert
va_high_addr_switchtest to use kselftest framework for reporting and tracking successful and failing runs.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Quoted 89 lines
--- .../selftests/mm/va_high_addr_switch.c | 41 +++++++++---------- 1 file changed, 20 insertions(+), 21 deletions(-)tools/testing/selftests/mm/va_high_addr_switch.c +20
-21@@ -257,40 +257,35 @@ void testcases_init(void) switch_hint = addr_switch_hint; } -static int run_test (struct testcase *test,int count) +static void run_test (struct testcase *test,int count) {void *p; -int i, ret =KSFT_PASS ; +int i;for (i =0 ; i < count; i++) {struct testcase *t = test + i; p =mmap (t->addr, t->size,PROT_READ |PROT_WRITE , t->flags,-1 ,0 ); - -printf ("%s: %p - " , t->msg, p); -if (p ==MAP_FAILED ) { -printf ("FAILED\n" ); - ret =KSFT_FAIL ; +ksft_perror ("MAP_FAILED" ); +ksft_test_result_fail ("%s\n" , t->msg);continue ; }if (t->low_addr_required && p >= (void *)(switch_hint)) { -printf ("FAILED\n" ); - ret =KSFT_FAIL ; +ksft_print_msg ("%p not below switch hint\n" , p); +ksft_test_result_fail ("%s\n" , t->msg); }else {/* * Do a dereference of the address returned so that we catch * bugs in page fault handling */ memset (p,0 , t->size); -printf ("OK\n" ); +ksft_test_result_pass ("%s\n" , t->msg); }if (!t->keep_mapped)munmap (p, t->size); } - -return ret; }#ifdef __aarch64__@@ -322,19 +317,23 @@ static int supported_arch(void)int main (int argc,char **argv) { -int ret, hugetlb_ret =KSFT_PASS ; +bool run_hugetlb = false; + +ksft_print_header ();if (!supported_arch ()) -return KSFT_SKIP ; +ksft_exit_skip ("Architecture not supported\n" ); + +if (argc ==2 && !strcmp (argv[1 ],"--run-hugetlb" )) + run_hugetlb = true;testcases_init (); - ret =run_test (testcases, sz_testcases); -if (argc ==2 && !strcmp (argv[1 ],"--run-hugetlb" )) - hugetlb_ret =run_test (hugetlb_testcases, sz_hugetlb_testcases); +ksft_set_plan (sz_testcases + (run_hugetlb ? sz_hugetlb_testcases :0 )); + +run_test (testcases, sz_testcases); +if (run_hugetlb) +run_test (hugetlb_testcases, sz_hugetlb_testcases); -if (ret ==KSFT_PASS && hugetlb_ret ==KSFT_PASS ) -return KSFT_PASS ; -else -return KSFT_FAIL ; +ksft_finished (); }
Re: [PATCH v3 20/54] selftests/mm: va_high_addr_switch: use kselftest framework
In reply to Mike RapoportQuoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Convert
va_high_addr_switchtest to use kselftest framework for reporting and tracking successful and failing runs.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I have run this test before and after applying the patch. It correctly converts the test output into KTAP format with no change in results.
Quoted 89 lines
--- .../selftests/mm/va_high_addr_switch.c | 41 +++++++++---------- 1 file changed, 20 insertions(+), 21 deletions(-)tools/testing/selftests/mm/va_high_addr_switch.c +20
-21@@ -257,40 +257,35 @@ void testcases_init(void) switch_hint = addr_switch_hint; } -static int run_test (struct testcase *test,int count) +static void run_test (struct testcase *test,int count) {void *p; -int i, ret =KSFT_PASS ; +int i;for (i =0 ; i < count; i++) {struct testcase *t = test + i; p =mmap (t->addr, t->size,PROT_READ |PROT_WRITE , t->flags,-1 ,0 ); - -printf ("%s: %p - " , t->msg, p); -if (p ==MAP_FAILED ) { -printf ("FAILED\n" ); - ret =KSFT_FAIL ; +ksft_perror ("MAP_FAILED" ); +ksft_test_result_fail ("%s\n" , t->msg);continue ; }if (t->low_addr_required && p >= (void *)(switch_hint)) { -printf ("FAILED\n" ); - ret =KSFT_FAIL ; +ksft_print_msg ("%p not below switch hint\n" , p); +ksft_test_result_fail ("%s\n" , t->msg); }else {/* * Do a dereference of the address returned so that we catch * bugs in page fault handling */ memset (p,0 , t->size); -printf ("OK\n" ); +ksft_test_result_pass ("%s\n" , t->msg); }if (!t->keep_mapped)munmap (p, t->size); } - -return ret; }#ifdef __aarch64__@@ -322,19 +317,23 @@ static int supported_arch(void)int main (int argc,char **argv) { -int ret, hugetlb_ret =KSFT_PASS ; +bool run_hugetlb = false; + +ksft_print_header ();if (!supported_arch ()) -return KSFT_SKIP ; +ksft_exit_skip ("Architecture not supported\n" ); + +if (argc ==2 && !strcmp (argv[1 ],"--run-hugetlb" )) + run_hugetlb = true;testcases_init (); - ret =run_test (testcases, sz_testcases); -if (argc ==2 && !strcmp (argv[1 ],"--run-hugetlb" )) - hugetlb_ret =run_test (hugetlb_testcases, sz_hugetlb_testcases); +ksft_set_plan (sz_testcases + (run_hugetlb ? sz_hugetlb_testcases :0 )); + +run_test (testcases, sz_testcases); +if (run_hugetlb) +run_test (hugetlb_testcases, sz_hugetlb_testcases); -if (ret ==KSFT_PASS && hugetlb_ret ==KSFT_PASS ) -return KSFT_PASS ; -else -return KSFT_FAIL ; +ksft_finished (); }
Mike Rapoport
maintainerPATCH v3 21/54 selftests/mm: add atexit() and signal handlers to thp_settings
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
khugepaged registers atexit() and signal handlers that ensure that THP settings are restored regardless of how the test exited.
Make these handlers available for all users of thp_settings.
The call to thp_save_settings() installs thp_restore_settings as the atexit() callback and makes sure that signals that kill a process would still call exit() and atexit() callback.
Update child process in tests using thp_settings to use _exit() instead of exit() to avoid altering THP settings in the middle of a test.
--- tools/testing/selftests/mm/cow.c | 21 +++++------ tools/testing/selftests/mm/khugepaged.c | 40 +++------------------ tools/testing/selftests/mm/thp_settings.c | 28 ++++++++++++++- tools/testing/selftests/mm/uffd-wp-mremap.c | 4 --- 4 files changed, 39 insertions(+), 54 deletions(-)
tools/testing/selftests/mm/cow.c +8 -13
@@ -202,7 +202,7 @@ static void do_test_cow_in_parent(char *mem, size_t size, bool do_mprotect,log_test_result (KSFT_FAIL ); goto close_comm_pipes; }else if (!ret) { -exit (fn (mem, size, &comm_pipes)); +_exit (fn (mem, size, &comm_pipes)); }while (read (comm_pipes.child_ready[0 ], &buf,1 ) !=1 )
@@ -333,7 +333,7 @@ static void do_test_vmsplice_in_parent(char *mem, size_t size, ;/* Modify page content in the child. */ memset (mem,0xff , size); -exit (0 ); +_exit (0 ); }if (!before_fork) {
@@ -480,7 +480,7 @@ static void do_test_iouring(char *mem, size_t size, bool use_fork)write (comm_pipes.child_ready[1 ],"0" ,1 );while (read (comm_pipes.parent_ready[0 ], &buf,1 ) !=1 ) ; -exit (0 ); +_exit (0 ); }while (read (comm_pipes.child_ready[0 ], &buf,1 ) !=1 )
@@ -645,7 +645,7 @@ static void do_test_ro_pin(char *mem, size_t size, enum ro_pin_test test,write (comm_pipes.child_ready[1 ],"0" ,1 );while (read (comm_pipes.parent_ready[0 ], &buf,1 ) !=1 ) ; -exit (0 ); +_exit (0 ); }/* Wait until our child is ready. */
@@ -956,7 +956,7 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)log_test_result (KSFT_FAIL ); goto munmap; }else if (!ret) { -exit (0 ); +_exit (0 ); }wait (&ret);/* Allow for sharing all pages again. */
@@ -1347,13 +1347,13 @@ static void do_test_anon_thp_collapse(char *mem, size_t size,switch (test) {case ANON_THP_COLLAPSE_UNSHARED :case ANON_THP_COLLAPSE_FULLY_SHARED : -exit (child_memcmp_fn (mem, size, &comm_pipes)); +_exit (child_memcmp_fn (mem, size, &comm_pipes));break ;case ANON_THP_COLLAPSE_LOWER_SHARED : -exit (child_memcmp_fn (mem, size /2 , &comm_pipes)); +_exit (child_memcmp_fn (mem, size /2 , &comm_pipes));break ;case ANON_THP_COLLAPSE_UPPER_SHARED : -exit (child_memcmp_fn (mem + size /2 , size /2 , +_exit (child_memcmp_fn (mem + size /2 , size /2 , &comm_pipes));break ;default :
@@ -1911,10 +1911,5 @@ int main(int argc, char **argv)run_anon_thp_test_cases ();run_non_anon_test_cases (); -if (pmdsize) { -/* Only if THP is supported. */ -thp_restore_settings (); - } -ksft_finished (); }
tools/testing/selftests/mm/khugepaged.c +4 -36
@@ -72,7 +72,6 @@ struct file_info { };static struct file_info finfo; -static bool skip_settings_restore;static int exit_status;static void success (const char *msg)
@@ -92,25 +91,6 @@ static void skip(const char *msg) exit_status =KSFT_SKIP ; } -static void restore_settings_atexit (void ) -{ -if (skip_settings_restore) -return ; - -ksft_print_msg ("Restore THP and khugepaged settings..." ); -thp_restore_settings (); -success ("OK" ); - - skip_settings_restore = true; -ksft_finished (); -} - -static void restore_settings (int sig) -{ -/* exit() will invoke the restore_settings_atexit handler. */ -exit (sig ?KSFT_FAIL : exit_status); -} -static void save_settings (void ) {ksft_print_msg ("Save THP and khugepaged settings..." );
@@ -119,12 +99,6 @@ static void save_settings(void)thp_save_settings ();success ("OK" ); - -atexit (restore_settings_atexit); -signal (SIGTERM , restore_settings); -signal (SIGINT , restore_settings); -signal (SIGHUP , restore_settings); -signal (SIGQUIT , restore_settings); }static void get_finfo (const char *dir)
@@ -844,8 +818,6 @@ static void collapse_fork(struct collapse_context *c, struct mem_ops *ops)ksft_print_msg ("Share small page over fork()..." );if (!fork ()) {/* Do not touch settings on child exit */ - skip_settings_restore = true; -if (ops->check_huge (p,0 ))success ("OK" );else
@@ -857,7 +829,7 @@ static void collapse_fork(struct collapse_context *c, struct mem_ops *ops)validate_memory (p,0 , page_size); ops->cleanup_area (p, hpage_pmd_size); -exit (exit_status); +_exit (exit_status); }wait (&wstatus);
@@ -882,8 +854,6 @@ static void collapse_fork_compound(struct collapse_context *c, struct mem_ops *oksft_print_msg ("Share huge page over fork()..." );if (!fork ()) {/* Do not touch settings on child exit */ - skip_settings_restore = true; -if (ops->check_huge (p,1 ))success ("OK" );else
@@ -906,7 +876,7 @@ static void collapse_fork_compound(struct collapse_context *c, struct mem_ops *ovalidate_memory (p,0 , hpage_pmd_size); ops->cleanup_area (p, hpage_pmd_size); -exit (exit_status); +_exit (exit_status); }wait (&wstatus);
@@ -932,8 +902,6 @@ static void collapse_max_ptes_shared(struct collapse_context *c, struct mem_opsksft_print_msg ("Share huge page over fork()..." );if (!fork ()) {/* Do not touch settings on child exit */ - skip_settings_restore = true; -if (ops->check_huge (p,1 ))success ("OK" );else
@@ -966,7 +934,7 @@ static void collapse_max_ptes_shared(struct collapse_context *c, struct mem_opsvalidate_memory (p,0 , hpage_pmd_size); ops->cleanup_area (p, hpage_pmd_size); -exit (exit_status); +_exit (exit_status); }wait (&wstatus);
@@ -1260,5 +1228,5 @@ int main(int argc, char **argv) t->fn (t->ctx, t->ops); } -restore_settings (0 ); +ksft_finished (); }
tools/testing/selftests/mm/thp_settings.c +27 -1
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 #include <fcntl.h> #include <limits.h> +#include <signal.h> #include <stdio.h> #include <stdlib.h> #include <string.h>
@@ -15,6 +16,7 @@ static struct thp_settings settings_stack[MAX_SETTINGS_DEPTH];static int settings_index;static struct thp_settings saved_settings;static char dev_queue_read_ahead_path[PATH_MAX ]; +static bool thp_settings_saved;static const char *const thp_enabled_strings[] = {"never" ,
@@ -298,12 +300,36 @@ void thp_pop_settings(void)void thp_restore_settings (void ) { -thp_write_settings (&saved_settings); +if (thp_settings_saved) +thp_write_settings (&saved_settings); +} + +static void thp_restore_settings_atexit (void ) +{ +thp_restore_settings (); +} + +static void thp_restore_settings_sighandler (int sig) +{ +/* exit() will invoke the thp_restore_settings_atexit handler. */ +exit (KSFT_FAIL ); }void thp_save_settings (void ) {thp_read_settings (&saved_settings); + +/* +* setup exit hooks to make sure THP settings are restored on graceful +* and error exits and signals +*/ +atexit (thp_restore_settings_atexit); +signal (SIGTERM , thp_restore_settings_sighandler); +signal (SIGINT , thp_restore_settings_sighandler); +signal (SIGHUP , thp_restore_settings_sighandler); +signal (SIGQUIT , thp_restore_settings_sighandler); + + thp_settings_saved = true; }void thp_set_read_ahead_path (char *path)
tools/testing/selftests/mm/uffd-wp-mremap.c +0 -4
@@ -368,10 +368,6 @@ int main(int argc, char **argv) tc->swapout, tc->hugetlb); } -/* If THP is supported, restore original THP settings. */ -if (nr_thpsizes) -thp_restore_settings (); - i =ksft_get_fail_cnt ();if (i) ksft_exit_fail_msg("%d out of %d tests failed\n",
Signature · 1 line
2.53.0
Re: [PATCH v3 21/54] selftests/mm: add atexit() and signal handlers to thp_settings
Quoted 11 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
khugepaged registers
atexit()and signal handlers that ensure that THP settings are restored regardless of how the test exited.Make these handlers available for all users of
thp_settings.The call to
thp_save_settings()installsthp_restore_settingsas theatexit()callback and makes sure that signals that kill a process would still callexit()andatexit()callback.
Quoted 5 lines
Update child process in tests using
thp_settingsto use_exit()instead ofexit()to avoid altering THP settings in the middle of a test.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
I ran the modified mm selftests after applying this patch and the THP settings are being correctly restored to their original values both on normal exit and when the test is being interrupted with Ctrl + C.
Mike Rapoport
maintainerPATCH v3 22/54 selftests/mm: rename thp_settings.[ch] to hugepage_settings.[ch]
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
... for upcoming addition of HugeTLB helpers.
---
tools/testing/selftests/mm/Makefile | 4 ++--
tools/testing/selftests/mm/cow.c | 2 +-
tools/testing/selftests/mm/folio_split_race_test.c | 2 +-
tools/testing/selftests/mm/guard-regions.c | 2 +-
.../selftests/mm/{thp_settings.c => hugepage_settings.c} | 2 +-
.../selftests/mm/{thp_settings.h => hugepage_settings.h} | 0
tools/testing/selftests/mm/khugepaged.c | 2 +-
tools/testing/selftests/mm/ksm_tests.c | 2 +-
tools/testing/selftests/mm/migration.c | 2 +-
tools/testing/selftests/mm/prctl_thp_disable.c | 2 +-
tools/testing/selftests/mm/soft-dirty.c | 2 +-
tools/testing/selftests/mm/split_huge_page_test.c | 2 +-
tools/testing/selftests/mm/transhuge-stress.c | 2 +-
tools/testing/selftests/mm/uffd-wp-mremap.c | 2 +-
14 files changed, 14 insertions(+), 14 deletions(-)
rename tools/testing/selftests/mm/{thp_settings.c => hugepage_settings.c} (99%)
rename tools/testing/selftests/mm/{thp_settings.h => hugepage_settings.h} (100%)tools/testing/selftests/mm/Makefile +2 -2
@@ -186,8 +186,8 @@ TEST_FILES += write_hugetlb_memory.sh include ../lib.mk -$(TEST_GEN_PROGS): vm_util.c thp_settings.c -$(TEST_GEN_FILES): vm_util.c thp_settings.c +$(TEST_GEN_PROGS): vm_util.c hugepage_settings.c +$(TEST_GEN_FILES): vm_util.c hugepage_settings.c $(OUTPUT)/uffd-stress: uffd-common.c $(OUTPUT)/uffd-unit-tests: uffd-common.c
tools/testing/selftests/mm/cow.c +1 -1
@@ -29,7 +29,7 @@#include "../../../../mm/gup_test.h" #include "kselftest.h" #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" static size_t pagesize;static int pagemap_fd;
tools/testing/selftests/mm/folio_split_race_test.c +1 -1
@@ -25,7 +25,7 @@#include <unistd.h> #include "vm_util.h" #include "kselftest.h" -#include "thp_settings.h" +#include "hugepage_settings.h" uint64_t page_size;uint64_t pmd_pagesize;
tools/testing/selftests/mm/guard-regions.c +1 -1
@@ -21,7 +21,7 @@#include <sys/uio.h> #include <unistd.h> #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #include "../pidfd/pidfd.h"
tools/testing/selftests/mm/hugepage_settings.c renamed from tools/testing/selftests/mm/thp_settings.c +1 -1
@@ -8,7 +8,7 @@#include <unistd.h> #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #define THP_SYSFS "/sys/kernel/mm/transparent_hugepage/"#define MAX_SETTINGS_DEPTH 4
tools/testing/selftests/mm/hugepage_settings.h renamed from tools/testing/selftests/mm/thp_settings.h +0 -0
tools/testing/selftests/mm/khugepaged.c +1 -1
@@ -22,7 +22,7 @@#include "linux/magic.h" #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #define BASE_ADDR ((void *)(1UL << 30))static unsigned long hpage_pmd_size;
tools/testing/selftests/mm/ksm_tests.c +1 -1
@@ -15,7 +15,7 @@#include "kselftest.h" #include <include/vdso/time64.h> #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #define KSM_SYSFS_PATH "/sys/kernel/mm/ksm/"#define KSM_FP (s) (KSM_SYSFS_PATH s)
tools/testing/selftests/mm/migration.c +1 -1
@@ -5,7 +5,7 @@*/ #include "kselftest_harness.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #include <strings.h> #include <pthread.h>
tools/testing/selftests/mm/prctl_thp_disable.c +1 -1
@@ -14,7 +14,7 @@#include <sys/wait.h> #include "kselftest_harness.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #include "vm_util.h" #ifndef PR_THP_DISABLE_EXCEPT_ADVISED
tools/testing/selftests/mm/soft-dirty.c +1 -1
@@ -9,7 +9,7 @@#include "kselftest.h" #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #define PAGEMAP_FILE_PATH "/proc/self/pagemap"#define TEST_ITERATIONS 10000
tools/testing/selftests/mm/split_huge_page_test.c +1 -1
@@ -21,7 +21,7 @@#include <time.h> #include "vm_util.h" #include "kselftest.h" -#include "thp_settings.h" +#include "hugepage_settings.h" uint64_t pagesize;unsigned int pageshift;
tools/testing/selftests/mm/transhuge-stress.c +1 -1
@@ -17,7 +17,7 @@#include <sys/mman.h> #include "vm_util.h" #include "kselftest.h" -#include "thp_settings.h" +#include "hugepage_settings.h" int backing_fd =-1 ;int mmap_flags =MAP_ANONYMOUS |MAP_NORESERVE |MAP_PRIVATE ;
tools/testing/selftests/mm/uffd-wp-mremap.c +1 -1
@@ -8,7 +8,7 @@#include <linux/mman.h> #include <sys/mman.h> #include "kselftest.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #include "uffd-common.h" static int pagemap_fd;
Signature · 1 line
2.53.0
Re: [PATCH v3 22/54] selftests/mm: rename thp_settings.[ch] to hugepage_settings.[ch]
Quoted 5 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
... for upcoming addition of HugeTLB helpers.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Looks good to me.
Quoted 197 lines
--- tools/testing/selftests/mm/Makefile | 4 ++-- tools/testing/selftests/mm/cow.c | 2 +- tools/testing/selftests/mm/folio_split_race_test.c | 2 +- tools/testing/selftests/mm/guard-regions.c | 2 +- .../selftests/mm/{thp_settings.c => hugepage_settings.c} | 2 +- .../selftests/mm/{thp_settings.h => hugepage_settings.h} | 0 tools/testing/selftests/mm/khugepaged.c | 2 +- tools/testing/selftests/mm/ksm_tests.c | 2 +- tools/testing/selftests/mm/migration.c | 2 +- tools/testing/selftests/mm/prctl_thp_disable.c | 2 +- tools/testing/selftests/mm/soft-dirty.c | 2 +- tools/testing/selftests/mm/split_huge_page_test.c | 2 +- tools/testing/selftests/mm/transhuge-stress.c | 2 +- tools/testing/selftests/mm/uffd-wp-mremap.c | 2 +- 14 files changed, 14 insertions(+), 14 deletions(-) rename tools/testing/selftests/mm/{thp_settings.c => hugepage_settings.c} (99%) rename tools/testing/selftests/mm/{thp_settings.h => hugepage_settings.h} (100%)tools/testing/selftests/mm/Makefile +2
-2@@ -186,8 +186,8 @@ TEST_FILES += write_hugetlb_memory.sh include ../lib.mk -$(TEST_GEN_PROGS): vm_util.c thp_settings.c -$(TEST_GEN_FILES): vm_util.c thp_settings.c +$(TEST_GEN_PROGS): vm_util.c hugepage_settings.c +$(TEST_GEN_FILES): vm_util.c hugepage_settings.c $(OUTPUT)/uffd-stress: uffd-common.c $(OUTPUT)/uffd-unit-tests: uffd-common.ctools/testing/selftests/mm/cow.c +1
-1@@ -29,7 +29,7 @@#include "../../../../mm/gup_test.h" #include "kselftest.h" #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" static size_t pagesize;static int pagemap_fd;tools/testing/selftests/mm/folio_split_race_test.c +1
-1@@ -25,7 +25,7 @@#include <unistd.h> #include "vm_util.h" #include "kselftest.h" -#include "thp_settings.h" +#include "hugepage_settings.h" uint64_t page_size;uint64_t pmd_pagesize;tools/testing/selftests/mm/guard-regions.c +1
-1@@ -21,7 +21,7 @@#include <sys/uio.h> #include <unistd.h> #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #include "../pidfd/pidfd.h" tools/testing/selftests/mm/hugepage_settings.c renamed from tools/testing/selftests/mm/thp_settings.c +1
-1@@ -8,7 +8,7 @@#include <unistd.h> #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #define THP_SYSFS "/sys/kernel/mm/transparent_hugepage/"#define MAX_SETTINGS_DEPTH 4tools/testing/selftests/mm/hugepage_settings.h renamed from tools/testing/selftests/mm/thp_settings.h +0
-0tools/testing/selftests/mm/khugepaged.c +1
-1@@ -22,7 +22,7 @@#include "linux/magic.h" #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #define BASE_ADDR ((void *)(1UL << 30))static unsigned long hpage_pmd_size;tools/testing/selftests/mm/ksm_tests.c +1
-1@@ -15,7 +15,7 @@#include "kselftest.h" #include <include/vdso/time64.h> #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #define KSM_SYSFS_PATH "/sys/kernel/mm/ksm/"#define KSM_FP (s) (KSM_SYSFS_PATH s)tools/testing/selftests/mm/migration.c +1
-1@@ -5,7 +5,7 @@*/ #include "kselftest_harness.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #include <strings.h> #include <pthread.h> tools/testing/selftests/mm/prctl_thp_disable.c +1
-1@@ -14,7 +14,7 @@#include <sys/wait.h> #include "kselftest_harness.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #include "vm_util.h" #ifndef PR_THP_DISABLE_EXCEPT_ADVISED tools/testing/selftests/mm/soft-dirty.c +1
-1@@ -9,7 +9,7 @@#include "kselftest.h" #include "vm_util.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #define PAGEMAP_FILE_PATH "/proc/self/pagemap"#define TEST_ITERATIONS 10000tools/testing/selftests/mm/split_huge_page_test.c +1
-1@@ -21,7 +21,7 @@#include <time.h> #include "vm_util.h" #include "kselftest.h" -#include "thp_settings.h" +#include "hugepage_settings.h" uint64_t pagesize;unsigned int pageshift;tools/testing/selftests/mm/transhuge-stress.c +1
-1@@ -17,7 +17,7 @@#include <sys/mman.h> #include "vm_util.h" #include "kselftest.h" -#include "thp_settings.h" +#include "hugepage_settings.h" int backing_fd =-1 ;int mmap_flags =MAP_ANONYMOUS |MAP_NORESERVE |MAP_PRIVATE ;tools/testing/selftests/mm/uffd-wp-mremap.c +1
-1@@ -8,7 +8,7 @@#include <linux/mman.h> #include <sys/mman.h> #include "kselftest.h" -#include "thp_settings.h" +#include "hugepage_settings.h" #include "uffd-common.h" static int pagemap_fd;
Mike Rapoport
maintainerPATCH v3 23/54 selftests/mm: move HugeTLB helpers to hugepage_settings
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Move library functions that abstract HugeTLB /proc and /sysfs access from vm_util to hugepage_settings.
This will help creating common helpers that save and restore HugeTLB and THP settings.
--- tools/testing/selftests/mm/gup_longterm.c | 1 + tools/testing/selftests/mm/hmm-tests.c | 2 +- .../testing/selftests/mm/hugepage_settings.c | 68 +++++++++++++++++++ .../testing/selftests/mm/hugepage_settings.h | 8 +++ tools/testing/selftests/mm/hugetlb-madvise.c | 1 + tools/testing/selftests/mm/hugetlb-mmap.c | 1 + tools/testing/selftests/mm/hugetlb-vmemmap.c | 1 + tools/testing/selftests/mm/hugetlb_dio.c | 1 + .../selftests/mm/hugetlb_fault_after_madv.c | 1 + .../selftests/mm/hugetlb_madv_vs_map.c | 1 + tools/testing/selftests/mm/thuge-gen.c | 1 + tools/testing/selftests/mm/uffd-common.h | 1 + .../selftests/mm/va_high_addr_switch.c | 1 + tools/testing/selftests/mm/vm_util.c | 66 ------------------ tools/testing/selftests/mm/vm_util.h | 3 - 15 files changed, 87 insertions(+), 70 deletions(-)
tools/testing/selftests/mm/gup_longterm.c +1 -0
@@ -29,6 +29,7 @@#include "../../../../mm/gup_test.h" #include "kselftest.h" #include "vm_util.h" +#include "hugepage_settings.h" static size_t pagesize;static int nr_hugetlbsizes;
tools/testing/selftests/mm/hmm-tests.c +1 -1
@@ -11,6 +11,7 @@*/ #include "kselftest_harness.h" +#include "hugepage_settings.h" #include <errno.h> #include <fcntl.h>
@@ -27,7 +28,6 @@#include <sys/ioctl.h> #include <sys/time.h> -/* * This is a private UAPI to the kernel test module so it isn't exported * in the usual include/uapi/... directory.
tools/testing/selftests/mm/hugepage_settings.c +68 -0
78 lines not on this page · raw message
tools/testing/selftests/mm/hugepage_settings.h +8 -0
@@ -6,6 +6,8 @@#include <stddef.h> #include <stdint.h> +/* Transparent Huge Pages (THP) */ +enum thp_enabled {THP_NEVER ,THP_ALWAYS ,
@@ -86,4 +88,10 @@ unsigned long thp_shmem_supported_orders(void);bool thp_available (void );bool thp_is_enabled (void ); +/* HugeTLB */ + +int detect_hugetlb_page_sizes (size_t sizes[],int max); +unsigned long default_huge_page_size (void ); +unsigned long get_free_hugepages (void ); +#endif /* __THP_SETTINGS_H__ */
tools/testing/selftests/mm/hugetlb-madvise.c +1 -0
@@ -20,6 +20,7 @@#include <fcntl.h> #include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" #define MIN_FREE_PAGES 20#define NR_HUGE_PAGES 10/* common number of pages to map/allocate */
tools/testing/selftests/mm/hugetlb-mmap.c +1 -0
@@ -18,6 +18,7 @@#include <linux/memfd.h> #include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" #define LENGTH (256UL*1024*1024)#define PROTECTION (PROT_READ | PROT_WRITE)
tools/testing/selftests/mm/hugetlb-vmemmap.c +1 -0
@@ -11,6 +11,7 @@#include <sys/mman.h> #include <fcntl.h> #include "vm_util.h" +#include "hugepage_settings.h" #define PAGE_COMPOUND_HEAD (1UL << 15)#define PAGE_COMPOUND_TAIL (1UL << 16)
tools/testing/selftests/mm/hugetlb_dio.c +1 -0
@@ -20,6 +20,7 @@#include <sys/syscall.h> #include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" #ifndef STATX_DIOALIGN #define STATX_DIOALIGN 0x00002000U
tools/testing/selftests/mm/hugetlb_fault_after_madv.c +1 -0
@@ -10,6 +10,7 @@#include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" #define INLOOP_ITER 100
tools/testing/selftests/mm/hugetlb_madv_vs_map.c +1 -0
@@ -25,6 +25,7 @@#include <unistd.h> #include "vm_util.h" +#include "hugepage_settings.h" #define INLOOP_ITER 100
tools/testing/selftests/mm/thuge-gen.c +1 -0
@@ -28,6 +28,7 @@#include <string.h> #include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" #if !defined(MAP_HUGETLB )#define MAP_HUGETLB 0x40000
tools/testing/selftests/mm/uffd-common.h +1 -0
@@ -37,6 +37,7 @@#include "kselftest.h" #include "vm_util.h" +#include "hugepage_settings.h" #define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY)
tools/testing/selftests/mm/va_high_addr_switch.c +1 -0
@@ -11,6 +11,7 @@#include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" /* * The hint addr value is used to allocate addresses
tools/testing/selftests/mm/vm_util.c +0 -66
78 lines not on this page · raw message
tools/testing/selftests/mm/vm_util.h +0 -3
@@ -94,8 +94,6 @@ bool check_huge_anon(void *addr, int nr_hpages, uint64_t hpage_size);bool check_huge_file (void *addr,int nr_hpages,uint64_t hpage_size);bool check_huge_shmem (void *addr,int nr_hpages,uint64_t hpage_size);int64_t allocate_transhuge (void *ptr,int pagemap_fd); -unsigned long default_huge_page_size (void ); -int detect_hugetlb_page_sizes (size_t sizes[],int max);int pageflags_get (unsigned long pfn,int kpageflags_fd,uint64_t *flags); int uffd_register(int uffd, void *addr, uint64_t len,
@@ -103,7 +101,6 @@ int uffd_register(int uffd, void *addr, uint64_t len,int uffd_unregister (int uffd,void *addr,uint64_t len);int uffd_register_with_ioctls (int uffd,void *addr,uint64_t len,bool miss,bool wp,bool minor,uint64_t *ioctls); -unsigned long get_free_hugepages (void );bool check_vmflag_io (void *addr);bool check_vmflag_pfnmap (void *addr);bool check_vmflag_guard (void *addr);
Signature · 1 line
2.53.0
Re: [PATCH v3 23/54] selftests/mm: move HugeTLB helpers to hugepage_settings
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Move library functions that abstract HugeTLB /proc and /sysfs access from
vm_utiltohugepage_settings.This will help creating common helpers that save and restore HugeTLB and THP settings.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Looks good to me.
Quoted 371 lines
--- tools/testing/selftests/mm/gup_longterm.c | 1 + tools/testing/selftests/mm/hmm-tests.c | 2 +- .../testing/selftests/mm/hugepage_settings.c | 68 +++++++++++++++++++ .../testing/selftests/mm/hugepage_settings.h | 8 +++ tools/testing/selftests/mm/hugetlb-madvise.c | 1 + tools/testing/selftests/mm/hugetlb-mmap.c | 1 + tools/testing/selftests/mm/hugetlb-vmemmap.c | 1 + tools/testing/selftests/mm/hugetlb_dio.c | 1 + .../selftests/mm/hugetlb_fault_after_madv.c | 1 + .../selftests/mm/hugetlb_madv_vs_map.c | 1 + tools/testing/selftests/mm/thuge-gen.c | 1 + tools/testing/selftests/mm/uffd-common.h | 1 + .../selftests/mm/va_high_addr_switch.c | 1 + tools/testing/selftests/mm/vm_util.c | 66 ------------------ tools/testing/selftests/mm/vm_util.h | 3 - 15 files changed, 87 insertions(+), 70 deletions(-)tools/testing/selftests/mm/gup_longterm.c +1
-0@@ -29,6 +29,7 @@#include "../../../../mm/gup_test.h" #include "kselftest.h" #include "vm_util.h" +#include "hugepage_settings.h" static size_t pagesize;static int nr_hugetlbsizes;tools/testing/selftests/mm/hmm-tests.c +1
-1@@ -11,6 +11,7 @@*/ #include "kselftest_harness.h" +#include "hugepage_settings.h" #include <errno.h> #include <fcntl.h> @@ -27,7 +28,6 @@#include <sys/ioctl.h> #include <sys/time.h> -/* * This is a private UAPI to the kernel test module so it isn't exported * in the usual include/uapi/... directory. tools/testing/selftests/mm/hugepage_settings.c +68
-0@@ -1,7 +1,9 @@// SPDX-License-Identifier: GPL-2.0 +#include <dirent.h> #include <fcntl.h> #include <limits.h> #include <signal.h> +#include <stddef.h> #include <stdio.h> #include <stdlib.h> #include <string.h> @@ -396,3 +398,69 @@ bool thp_is_enabled(void)/* THP is considered enabled if it's either "always" or "madvise" */ return mode ==1 || mode ==3 ; } + +int detect_hugetlb_page_sizes (size_t sizes[],int max) +{ +DIR *dir =opendir ("/sys/kernel/mm/hugepages/" ); +int count =0 ; + +if (!dir) +return 0 ; + +while (count < max) { +struct dirent *entry =readdir (dir); +size_t kb; + +if (!entry) +break ; +if (entry->d_type !=DT_DIR ) +continue ; +if (sscanf (entry->d_name,"hugepages-%zukB" , &kb) !=1 ) +continue ; + sizes[count++] = kb *1024 ; +ksft_print_msg ("[INFO] detected hugetlb page size: %zu KiB\n" , + kb); + } +closedir (dir); +return count; +} + +unsigned long default_huge_page_size (void ) +{ +unsigned long hps =0 ; +char *line =NULL ; +size_t linelen =0 ; +FILE *f =fopen ("/proc/meminfo" ,"r" ); + +if (!f) +return 0 ; +while (getline (&line, &linelen, f) >0 ) { +if (sscanf (line,"Hugepagesize: %lu kB" , &hps) ==1 ) { + hps <<=10 ; +break ; + } + } + +free (line); +fclose (f); +return hps; +} + +unsigned long get_free_hugepages (void ) +{ +unsigned long fhp =0 ; +char *line =NULL ; +size_t linelen =0 ; +FILE *f =fopen ("/proc/meminfo" ,"r" ); + +if (!f) +return fhp; +while (getline (&line, &linelen, f) >0 ) { +if (sscanf (line,"HugePages_Free: %lu" , &fhp) ==1 ) +break ; + } + +free (line); +fclose (f); +return fhp; +}tools/testing/selftests/mm/hugepage_settings.h +8
-0@@ -6,6 +6,8 @@#include <stddef.h> #include <stdint.h> +/* Transparent Huge Pages (THP) */ +enum thp_enabled {THP_NEVER ,THP_ALWAYS ,@@ -86,4 +88,10 @@ unsigned long thp_shmem_supported_orders(void);bool thp_available (void );bool thp_is_enabled (void ); +/* HugeTLB */ + +int detect_hugetlb_page_sizes (size_t sizes[],int max); +unsigned long default_huge_page_size (void ); +unsigned long get_free_hugepages (void ); +#endif /* __THP_SETTINGS_H__ */ tools/testing/selftests/mm/hugetlb-madvise.c +1
-0@@ -20,6 +20,7 @@#include <fcntl.h> #include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" #define MIN_FREE_PAGES 20#define NR_HUGE_PAGES 10/* common number of pages to map/allocate */ tools/testing/selftests/mm/hugetlb-mmap.c +1
-0@@ -18,6 +18,7 @@#include <linux/memfd.h> #include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" #define LENGTH (256UL*1024*1024)#define PROTECTION (PROT_READ | PROT_WRITE)tools/testing/selftests/mm/hugetlb-vmemmap.c +1
-0@@ -11,6 +11,7 @@#include <sys/mman.h> #include <fcntl.h> #include "vm_util.h" +#include "hugepage_settings.h" #define PAGE_COMPOUND_HEAD (1UL << 15)#define PAGE_COMPOUND_TAIL (1UL << 16)tools/testing/selftests/mm/hugetlb_dio.c +1
-0@@ -20,6 +20,7 @@#include <sys/syscall.h> #include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" #ifndef STATX_DIOALIGN #define STATX_DIOALIGN 0x00002000Utools/testing/selftests/mm/hugetlb_fault_after_madv.c +1
-0@@ -10,6 +10,7 @@#include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" #define INLOOP_ITER 100tools/testing/selftests/mm/hugetlb_madv_vs_map.c +1
-0@@ -25,6 +25,7 @@#include <unistd.h> #include "vm_util.h" +#include "hugepage_settings.h" #define INLOOP_ITER 100tools/testing/selftests/mm/thuge-gen.c +1
-0@@ -28,6 +28,7 @@#include <string.h> #include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" #if !defined(MAP_HUGETLB )#define MAP_HUGETLB 0x40000tools/testing/selftests/mm/uffd-common.h +1
-0@@ -37,6 +37,7 @@#include "kselftest.h" #include "vm_util.h" +#include "hugepage_settings.h" #define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY)tools/testing/selftests/mm/va_high_addr_switch.c +1
-0@@ -11,6 +11,7 @@#include "vm_util.h" #include "kselftest.h" +#include "hugepage_settings.h" /* * The hint addr value is used to allocate addresses tools/testing/selftests/mm/vm_util.c +0
-66@@ -291,53 +291,6 @@ int64_t allocate_transhuge(void *ptr, int pagemap_fd)return -1 ; } -unsigned long default_huge_page_size (void ) -{ -unsigned long hps =0 ; -char *line =NULL ; -size_t linelen =0 ; -FILE *f =fopen ("/proc/meminfo" ,"r" ); - -if (!f) -return 0 ; -while (getline (&line, &linelen, f) >0 ) { -if (sscanf (line,"Hugepagesize: %lu kB" , &hps) ==1 ) { - hps <<=10 ; -break ; - } - } - -free (line); -fclose (f); -return hps; -} - -int detect_hugetlb_page_sizes (size_t sizes[],int max) -{ -DIR *dir =opendir ("/sys/kernel/mm/hugepages/" ); -int count =0 ; - -if (!dir) -return 0 ; - -while (count < max) { -struct dirent *entry =readdir (dir); -size_t kb; - -if (!entry) -break ; -if (entry->d_type !=DT_DIR ) -continue ; -if (sscanf (entry->d_name,"hugepages-%zukB" , &kb) !=1 ) -continue ; - sizes[count++] = kb *1024 ; -ksft_print_msg ("[INFO] detected hugetlb page size: %zu KiB\n" , - kb); - } -closedir (dir); -return count; -} -int pageflags_get (unsigned long pfn,int kpageflags_fd,uint64_t *flags) {size_t count;@@ -396,25 +349,6 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)return ret; } -unsigned long get_free_hugepages (void ) -{ -unsigned long fhp =0 ; -char *line =NULL ; -size_t linelen =0 ; -FILE *f =fopen ("/proc/meminfo" ,"r" ); - -if (!f) -return fhp; -while (getline (&line, &linelen, f) >0 ) { -if (sscanf (line,"HugePages_Free: %lu" , &fhp) ==1 ) -break ; - } - -free (line); -fclose (f); -return fhp; -} -static bool check_vmflag (void *addr,const char *flag) {char buffer[MAX_LINE_LENGTH ];tools/testing/selftests/mm/vm_util.h +0
-3@@ -94,8 +94,6 @@ bool check_huge_anon(void *addr, int nr_hpages, uint64_t hpage_size);bool check_huge_file (void *addr,int nr_hpages,uint64_t hpage_size);bool check_huge_shmem (void *addr,int nr_hpages,uint64_t hpage_size);int64_t allocate_transhuge (void *ptr,int pagemap_fd); -unsigned long default_huge_page_size (void ); -int detect_hugetlb_page_sizes (size_t sizes[],int max);int pageflags_get (unsigned long pfn,int kpageflags_fd,uint64_t *flags); int uffd_register(int uffd, void *addr, uint64_t len,@@ -103,7 +101,6 @@ int uffd_register(int uffd, void *addr, uint64_t len,int uffd_unregister (int uffd,void *addr,uint64_t len);int uffd_register_with_ioctls (int uffd,void *addr,uint64_t len,bool miss,bool wp,bool minor,uint64_t *ioctls); -unsigned long get_free_hugepages (void );bool check_vmflag_io (void *addr);bool check_vmflag_pfnmap (void *addr);bool check_vmflag_guard (void *addr);
Mike Rapoport
maintainerPATCH v3 24/54 selftests/mm: hugepage_settings: use unsigned long in detect_hugetlb_page_size
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
... instead of size_t to avoid type mismatch in 32 and 64 bit builds.
--- tools/testing/selftests/mm/hugepage_settings.c | 2 +- tools/testing/selftests/mm/hugepage_settings.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/mm/hugepage_settings.c +1 -1
@@ -399,7 +399,7 @@ bool thp_is_enabled(void)return mode ==1 || mode ==3 ; } -int detect_hugetlb_page_sizes (size_t sizes[],int max) +int detect_hugetlb_page_sizes (unsigned long sizes[],int max) {DIR *dir =opendir ("/sys/kernel/mm/hugepages/" );int count =0 ;
tools/testing/selftests/mm/hugepage_settings.h +1 -1
@@ -90,7 +90,7 @@ bool thp_is_enabled(void);/* HugeTLB */ -int detect_hugetlb_page_sizes (size_t sizes[],int max); +int detect_hugetlb_page_sizes (unsigned long sizes[],int max);unsigned long default_huge_page_size (void );unsigned long get_free_hugepages (void );
Signature · 1 line
2.53.0
Re: [PATCH v3 24/54] selftests/mm: hugepage_settings: use unsigned long in detect_hugetlb_page_size
Quoted 5 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
... instead of
size_tto avoid type mismatch in 32 and 64 bit builds.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Looks good to me.
Quoted 31 lines
--- tools/testing/selftests/mm/hugepage_settings.c | 2 +- tools/testing/selftests/mm/hugepage_settings.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)tools/testing/selftests/mm/hugepage_settings.c +1
-1@@ -399,7 +399,7 @@ bool thp_is_enabled(void)return mode ==1 || mode ==3 ; } -int detect_hugetlb_page_sizes (size_t sizes[],int max) +int detect_hugetlb_page_sizes (unsigned long sizes[],int max) {DIR *dir =opendir ("/sys/kernel/mm/hugepages/" );int count =0 ;tools/testing/selftests/mm/hugepage_settings.h +1
-1@@ -90,7 +90,7 @@ bool thp_is_enabled(void);/* HugeTLB */ -int detect_hugetlb_page_sizes (size_t sizes[],int max); +int detect_hugetlb_page_sizes (unsigned long sizes[],int max);unsigned long default_huge_page_size (void );unsigned long get_free_hugepages (void );
Mike Rapoport
maintainerPATCH v3 25/54 selftests/mm: hugepage_settings: add APIs to get and set nr_hugepages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Add APIs that allow reading and writing of
/sys/kernel/mm/hugepages/hugepages-NkB/nr_hugepages
to detect and change the amount of HugeTLB pages of different sizes.
--- .../testing/selftests/mm/hugepage_settings.c | 25 +++++++++++++++++++ .../testing/selftests/mm/hugepage_settings.h | 23 +++++++++++++++++ 2 files changed, 48 insertions(+)
tools/testing/selftests/mm/hugepage_settings.c +25 -0
@@ -464,3 +464,28 @@ unsigned long get_free_hugepages(void)fclose (f);return fhp; } + +static void hugetlb_sysfs_path (char *buf,size_t buflen, +unsigned long size,const char *attr) +{ +snprintf (buf, buflen,"/sys/kernel/mm/hugepages/hugepages-%lukB/%s" , + size /1024 , attr); +} + +unsigned long hugetlb_nr_pages (unsigned long size) +{ +char path[PATH_MAX ]; + +hugetlb_sysfs_path (path,sizeof (path), size,"nr_hugepages" ); + +return read_num (path); +} + +void hugetlb_set_nr_pages (unsigned long size,unsigned long nr) +{ +char path[PATH_MAX ]; + +hugetlb_sysfs_path (path,sizeof (path), size,"nr_hugepages" ); + +write_num (path, nr); +}
tools/testing/selftests/mm/hugepage_settings.h +23 -0
@@ -94,4 +94,27 @@ int detect_hugetlb_page_sizes(unsigned long sizes[], int max);unsigned long default_huge_page_size (void );unsigned long get_free_hugepages (void ); +unsigned long hugetlb_nr_pages (unsigned long size); +void hugetlb_set_nr_pages (unsigned long size,unsigned long nr); + +static inline unsigned long hugetlb_nr_default_pages (void ) +{ +unsigned long size =default_huge_page_size (); + +if (!size) +return 0 ; + +return hugetlb_nr_pages (size); +} + +static inline void hugetlb_set_nr_default_pages (unsigned long nr) +{ +unsigned long size =default_huge_page_size (); + +if (!size) +return ; + +hugetlb_set_nr_pages (size, nr); +} +#endif /* __THP_SETTINGS_H__ */
Signature · 1 line
2.53.0
Re: [PATCH v3 25/54] selftests/mm: hugepage_settings: add APIs to get and set nr_hugepages
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Add APIs that allow reading and writing of
/sys/kernel/mm/hugepages/hugepages-NkB/nr_hugepagesto detect and change the amount of HugeTLB pages of different sizes.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Quoted 70 lines
--- .../testing/selftests/mm/hugepage_settings.c | 25 +++++++++++++++++++ .../testing/selftests/mm/hugepage_settings.h | 23 +++++++++++++++++ 2 files changed, 48 insertions(+)tools/testing/selftests/mm/hugepage_settings.c +25
-0@@ -464,3 +464,28 @@ unsigned long get_free_hugepages(void)fclose (f);return fhp; } + +static void hugetlb_sysfs_path (char *buf,size_t buflen, +unsigned long size,const char *attr) +{ +snprintf (buf, buflen,"/sys/kernel/mm/hugepages/hugepages-%lukB/%s" , + size /1024 , attr); +} + +unsigned long hugetlb_nr_pages (unsigned long size) +{ +char path[PATH_MAX ]; + +hugetlb_sysfs_path (path,sizeof (path), size,"nr_hugepages" ); + +return read_num (path); +} + +void hugetlb_set_nr_pages (unsigned long size,unsigned long nr) +{ +char path[PATH_MAX ]; + +hugetlb_sysfs_path (path,sizeof (path), size,"nr_hugepages" ); + +write_num (path, nr); +}tools/testing/selftests/mm/hugepage_settings.h +23
-0@@ -94,4 +94,27 @@ int detect_hugetlb_page_sizes(unsigned long sizes[], int max);unsigned long default_huge_page_size (void );unsigned long get_free_hugepages (void ); +unsigned long hugetlb_nr_pages (unsigned long size); +void hugetlb_set_nr_pages (unsigned long size,unsigned long nr); + +static inline unsigned long hugetlb_nr_default_pages (void ) +{ +unsigned long size =default_huge_page_size (); + +if (!size) +return 0 ; + +return hugetlb_nr_pages (size); +} + +static inline void hugetlb_set_nr_default_pages (unsigned long nr) +{ +unsigned long size =default_huge_page_size (); + +if (!size) +return ; + +hugetlb_set_nr_pages (size, nr); +} +#endif /* __THP_SETTINGS_H__ */
Re: [PATCH v3 25/54] selftests/mm: hugepage_settings: add APIs to get and set nr_hugepages
In reply to Mike RapoportQuoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Add APIs that allow reading and writing of
/sys/kernel/mm/hugepages/hugepages-NkB/nr_hugepagesto detect and change the amount of HugeTLB pages of different sizes.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
This patch looks good to me.
Quoted 70 lines
--- .../testing/selftests/mm/hugepage_settings.c | 25 +++++++++++++++++++ .../testing/selftests/mm/hugepage_settings.h | 23 +++++++++++++++++ 2 files changed, 48 insertions(+)tools/testing/selftests/mm/hugepage_settings.c +25
-0@@ -464,3 +464,28 @@ unsigned long get_free_hugepages(void)fclose (f);return fhp; } + +static void hugetlb_sysfs_path (char *buf,size_t buflen, +unsigned long size,const char *attr) +{ +snprintf (buf, buflen,"/sys/kernel/mm/hugepages/hugepages-%lukB/%s" , + size /1024 , attr); +} + +unsigned long hugetlb_nr_pages (unsigned long size) +{ +char path[PATH_MAX ]; + +hugetlb_sysfs_path (path,sizeof (path), size,"nr_hugepages" ); + +return read_num (path); +} + +void hugetlb_set_nr_pages (unsigned long size,unsigned long nr) +{ +char path[PATH_MAX ]; + +hugetlb_sysfs_path (path,sizeof (path), size,"nr_hugepages" ); + +write_num (path, nr); +}tools/testing/selftests/mm/hugepage_settings.h +23
-0@@ -94,4 +94,27 @@ int detect_hugetlb_page_sizes(unsigned long sizes[], int max);unsigned long default_huge_page_size (void );unsigned long get_free_hugepages (void ); +unsigned long hugetlb_nr_pages (unsigned long size); +void hugetlb_set_nr_pages (unsigned long size,unsigned long nr); + +static inline unsigned long hugetlb_nr_default_pages (void ) +{ +unsigned long size =default_huge_page_size (); + +if (!size) +return 0 ; + +return hugetlb_nr_pages (size); +} + +static inline void hugetlb_set_nr_default_pages (unsigned long nr) +{ +unsigned long size =default_huge_page_size (); + +if (!size) +return ; + +hugetlb_set_nr_pages (size, nr); +} +#endif /* __THP_SETTINGS_H__ */
Mike Rapoport
maintainerPATCH v3 26/54 selftests/mm: hugepage_settings: rename get_free_hugepages()
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
... to hugetlb_free_default_hugepages() for consistency with hugetlb_nr_default_pages().
--- .../testing/selftests/mm/hugepage_settings.c | 28 ++++++------------- .../testing/selftests/mm/hugepage_settings.h | 12 +++++++- tools/testing/selftests/mm/hugetlb-madvise.c | 4 +-- tools/testing/selftests/mm/hugetlb_dio.c | 6 ++-- .../selftests/mm/hugetlb_fault_after_madv.c | 2 +- .../selftests/mm/hugetlb_madv_vs_map.c | 2 +- tools/testing/selftests/mm/uffd-stress.c | 2 +- 7 files changed, 28 insertions(+), 28 deletions(-)
tools/testing/selftests/mm/hugepage_settings.c +9 -19
@@ -446,25 +446,6 @@ unsigned long default_huge_page_size(void)return hps; } -unsigned long get_free_hugepages (void ) -{ -unsigned long fhp =0 ; -char *line =NULL ; -size_t linelen =0 ; -FILE *f =fopen ("/proc/meminfo" ,"r" ); - -if (!f) -return fhp; -while (getline (&line, &linelen, f) >0 ) { -if (sscanf (line,"HugePages_Free: %lu" , &fhp) ==1 ) -break ; - } - -free (line); -fclose (f); -return fhp; -} -static void hugetlb_sysfs_path (char *buf,size_t buflen,unsigned long size,const char *attr) {
@@ -489,3 +470,12 @@ void hugetlb_set_nr_pages(unsigned long size, unsigned long nr)write_num (path, nr); } + +unsigned long hugetlb_free_pages (unsigned long size) +{ +char path[PATH_MAX ]; + +hugetlb_sysfs_path (path,sizeof (path), size,"free_hugepages" ); + +return read_num (path); +}
tools/testing/selftests/mm/hugepage_settings.h +11 -1
@@ -92,10 +92,10 @@ bool thp_is_enabled(void);int detect_hugetlb_page_sizes (unsigned long sizes[],int max);unsigned long default_huge_page_size (void ); -unsigned long get_free_hugepages (void );unsigned long hugetlb_nr_pages (unsigned long size);void hugetlb_set_nr_pages (unsigned long size,unsigned long nr); +unsigned long hugetlb_free_pages (unsigned long size);static inline unsigned long hugetlb_nr_default_pages (void ) {
@@ -117,4 +117,14 @@ static inline void hugetlb_set_nr_default_pages(unsigned long nr) hugetlb_set_nr_pages(size, nr); } +static inline unsigned long hugetlb_free_default_pages (void ) +{ +unsigned long size =default_huge_page_size (); + +if (!size) +return 0 ; + +return hugetlb_free_pages (size); +} +#endif /* __THP_SETTINGS_H__ */
tools/testing/selftests/mm/hugetlb-madvise.c +2 -2
@@ -27,7 +27,7 @@#define validate_free_pages (exp_free) \ do { \ - unsigned long fhp = get_free_hugepages(); \ + unsigned long fhp = hugetlb_free_default_pages(); \ if (fhp != (exp_free)) \ ksft_exit_fail_msg("Unexpected number of free " \ "huge pages %ld, expected %ld line %d\n", \
@@ -68,7 +68,7 @@ int main(int argc, char **argv)if (!base_page_size)ksft_exit_fail_msg ("Unable to determine base page size\n" ); - free_hugepages =get_free_hugepages (); + free_hugepages =hugetlb_free_default_pages ();if (free_hugepages <MIN_FREE_PAGES )ksft_exit_skip ("Not enough free huge pages (have %lu, need %d)\n" , free_hugepages,MIN_FREE_PAGES );
tools/testing/selftests/mm/hugetlb_dio.c +3 -3
@@ -93,7 +93,7 @@ static void run_dio_using_hugetlb(int fd, unsigned int start_off,ksft_exit_fail_perror ("lseek failed\n" );/* Get the free huge pages before allocation */ - free_hpage_b =get_free_hugepages (); + free_hpage_b =hugetlb_free_default_pages ();if (free_hpage_b ==0 ) {close (fd);ksft_exit_skip ("No free hugepage, exiting!\n" );
@@ -121,7 +121,7 @@ static void run_dio_using_hugetlb(int fd, unsigned int start_off,munmap (orig_buffer, h_pagesize);/* Get the free huge pages after unmap*/ - free_hpage_a =get_free_hugepages (); + free_hpage_a =hugetlb_free_default_pages ();ksft_print_msg ("No. Free pages before allocation : %d\n" , free_hpage_b);ksft_print_msg ("No. Free pages after munmap : %d\n" , free_hpage_a);
@@ -142,7 +142,7 @@ int main(void)ksft_print_header ();/* Check if huge pages are free */ -if (!get_free_hugepages ()) +if (!hugetlb_free_default_pages ())ksft_exit_skip ("No free hugepage, exiting\n" ); fd =open ("/tmp" ,O_TMPFILE |O_RDWR |O_DIRECT ,0664 );
tools/testing/selftests/mm/hugetlb_fault_after_madv.c +1 -1
@@ -78,7 +78,7 @@ int main(void)ksft_print_msg ("[INFO] detected default hugetlb page size: %zu KiB\n" , huge_page_size /1024 ); - free_hugepages =get_free_hugepages (); + free_hugepages =hugetlb_free_default_pages ();if (free_hugepages !=1 ) {ksft_exit_skip ("This test needs one and only one page to execute. Got %lu\n" , free_hugepages);
tools/testing/selftests/mm/hugetlb_madv_vs_map.c +1 -1
@@ -89,7 +89,7 @@ int main(void)ksft_print_header ();ksft_set_plan (1 ); - free_hugepages =get_free_hugepages (); + free_hugepages =hugetlb_free_default_pages ();if (free_hugepages !=1 ) ksft_exit_skip("This test needs one and only one page to execute. Got %lu\n",
tools/testing/selftests/mm/uffd-stress.c +1 -1
@@ -480,7 +480,7 @@ int main(int argc, char **argv)* for racy extra reservation of hugepages. */ if (gopts->test_type ==TEST_HUGETLB && -get_free_hugepages () <2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 ) +hugetlb_free_default_pages () <2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 )ksft_exit_skip ("Skipping userfaultfd... not enough hugepages\n" ); gopts->nr_pages_per_cpu = bytes / gopts->page_size / gopts->nr_parallel;
Signature · 1 line
2.53.0
Re: [PATCH v3 26/54] selftests/mm: hugepage_settings: rename get_free_hugepages()
Quoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
... to
hugetlb_free_default_hugepages()for consistency withhugetlb_nr_default_pages().Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Small nit, the commit message says hugetlb_free_default_hugepages() but the actual function added is hugetlb_free_default_pages().
This change looks good to me. All modified tests produce the same output as before the patch.
Quoted 176 lines
--- .../testing/selftests/mm/hugepage_settings.c | 28 ++++++------------- .../testing/selftests/mm/hugepage_settings.h | 12 +++++++- tools/testing/selftests/mm/hugetlb-madvise.c | 4 +-- tools/testing/selftests/mm/hugetlb_dio.c | 6 ++-- .../selftests/mm/hugetlb_fault_after_madv.c | 2 +- .../selftests/mm/hugetlb_madv_vs_map.c | 2 +- tools/testing/selftests/mm/uffd-stress.c | 2 +- 7 files changed, 28 insertions(+), 28 deletions(-)tools/testing/selftests/mm/hugepage_settings.c +9
-19@@ -446,25 +446,6 @@ unsigned long default_huge_page_size(void)return hps; } -unsigned long get_free_hugepages (void ) -{ -unsigned long fhp =0 ; -char *line =NULL ; -size_t linelen =0 ; -FILE *f =fopen ("/proc/meminfo" ,"r" ); - -if (!f) -return fhp; -while (getline (&line, &linelen, f) >0 ) { -if (sscanf (line,"HugePages_Free: %lu" , &fhp) ==1 ) -break ; - } - -free (line); -fclose (f); -return fhp; -} -static void hugetlb_sysfs_path (char *buf,size_t buflen,unsigned long size,const char *attr) {@@ -489,3 +470,12 @@ void hugetlb_set_nr_pages(unsigned long size, unsigned long nr)write_num (path, nr); } + +unsigned long hugetlb_free_pages (unsigned long size) +{ +char path[PATH_MAX ]; + +hugetlb_sysfs_path (path,sizeof (path), size,"free_hugepages" ); + +return read_num (path); +}tools/testing/selftests/mm/hugepage_settings.h +11
-1@@ -92,10 +92,10 @@ bool thp_is_enabled(void);int detect_hugetlb_page_sizes (unsigned long sizes[],int max);unsigned long default_huge_page_size (void ); -unsigned long get_free_hugepages (void );unsigned long hugetlb_nr_pages (unsigned long size);void hugetlb_set_nr_pages (unsigned long size,unsigned long nr); +unsigned long hugetlb_free_pages (unsigned long size);static inline unsigned long hugetlb_nr_default_pages (void ) {@@ -117,4 +117,14 @@ static inline void hugetlb_set_nr_default_pages(unsigned long nr) hugetlb_set_nr_pages(size, nr); } +static inline unsigned long hugetlb_free_default_pages (void ) +{ +unsigned long size =default_huge_page_size (); + +if (!size) +return 0 ; + +return hugetlb_free_pages (size); +} +#endif /* __THP_SETTINGS_H__ */ tools/testing/selftests/mm/hugetlb-madvise.c +2
-2@@ -27,7 +27,7 @@#define validate_free_pages (exp_free) \ do { \ - unsigned long fhp = get_free_hugepages(); \ + unsigned long fhp = hugetlb_free_default_pages(); \ if (fhp != (exp_free)) \ ksft_exit_fail_msg("Unexpected number of free " \ "huge pages %ld, expected %ld line %d\n", \@@ -68,7 +68,7 @@ int main(int argc, char **argv)if (!base_page_size)ksft_exit_fail_msg ("Unable to determine base page size\n" ); - free_hugepages =get_free_hugepages (); + free_hugepages =hugetlb_free_default_pages ();if (free_hugepages <MIN_FREE_PAGES )ksft_exit_skip ("Not enough free huge pages (have %lu, need %d)\n" , free_hugepages,MIN_FREE_PAGES );tools/testing/selftests/mm/hugetlb_dio.c +3
-3@@ -93,7 +93,7 @@ static void run_dio_using_hugetlb(int fd, unsigned int start_off,ksft_exit_fail_perror ("lseek failed\n" );/* Get the free huge pages before allocation */ - free_hpage_b =get_free_hugepages (); + free_hpage_b =hugetlb_free_default_pages ();if (free_hpage_b ==0 ) {close (fd);ksft_exit_skip ("No free hugepage, exiting!\n" );@@ -121,7 +121,7 @@ static void run_dio_using_hugetlb(int fd, unsigned int start_off,munmap (orig_buffer, h_pagesize);/* Get the free huge pages after unmap*/ - free_hpage_a =get_free_hugepages (); + free_hpage_a =hugetlb_free_default_pages ();ksft_print_msg ("No. Free pages before allocation : %d\n" , free_hpage_b);ksft_print_msg ("No. Free pages after munmap : %d\n" , free_hpage_a);@@ -142,7 +142,7 @@ int main(void)ksft_print_header ();/* Check if huge pages are free */ -if (!get_free_hugepages ()) +if (!hugetlb_free_default_pages ())ksft_exit_skip ("No free hugepage, exiting\n" ); fd =open ("/tmp" ,O_TMPFILE |O_RDWR |O_DIRECT ,0664 );tools/testing/selftests/mm/hugetlb_fault_after_madv.c +1
-1@@ -78,7 +78,7 @@ int main(void)ksft_print_msg ("[INFO] detected default hugetlb page size: %zu KiB\n" , huge_page_size /1024 ); - free_hugepages =get_free_hugepages (); + free_hugepages =hugetlb_free_default_pages ();if (free_hugepages !=1 ) {ksft_exit_skip ("This test needs one and only one page to execute. Got %lu\n" , free_hugepages);tools/testing/selftests/mm/hugetlb_madv_vs_map.c +1
-1@@ -89,7 +89,7 @@ int main(void)ksft_print_header ();ksft_set_plan (1 ); - free_hugepages =get_free_hugepages (); + free_hugepages =hugetlb_free_default_pages ();if (free_hugepages !=1 ) ksft_exit_skip("This test needs one and only one page to execute. Got %lu\n",tools/testing/selftests/mm/uffd-stress.c +1
-1@@ -480,7 +480,7 @@ int main(int argc, char **argv)* for racy extra reservation of hugepages. */ if (gopts->test_type ==TEST_HUGETLB && -get_free_hugepages () <2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 ) +hugetlb_free_default_pages () <2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 )ksft_exit_skip ("Skipping userfaultfd... not enough hugepages\n" ); gopts->nr_pages_per_cpu = bytes / gopts->page_size / gopts->nr_parallel;
Mike Rapoport
maintainerPATCH v3 27/54 selftests/mm: hugepage_settings: add APIs for HugeTLB setup and teardown
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
A lot of tests require free HugeTLB pages. Some need just a few default huge pages, some need a certain amount of memory available as HugeTLB, and some just skip lots of tests if huge pages of all supported sizes are not available.
This all resulted in a huge mess in run_vmtests.sh that sets up some huge pages, adjusts them later and restores some of the settings if the stars align.
Add APIs that allow saving the state of HugeTLB and setting up the desired amount of HugeTLB pages.
Saving the state also registers atexit() callback and signal handler that will ensure restoration of HugeTLB state.
Since many tests use both HugeTLB and THP, the atexit() callbacks and signal handler are restoring both.
For kselftest_harness tests that run fixture setups and test in child processes add a constructor that will save and restore settings in the main process.
--- .../testing/selftests/mm/hugepage_settings.c | 178 +++++++++++++++--- .../testing/selftests/mm/hugepage_settings.h | 30 ++- 2 files changed, 183 insertions(+), 25 deletions(-)
tools/testing/selftests/mm/hugepage_settings.c +154 -24
@@ -306,31 +306,12 @@ void thp_restore_settings(void)thp_write_settings (&saved_settings); } -static void thp_restore_settings_atexit (void ) +static void __thp_save_settings (void ) { -thp_restore_settings (); -} - -static void thp_restore_settings_sighandler (int sig) -{ -/* exit() will invoke the thp_restore_settings_atexit handler. */ -exit (KSFT_FAIL ); -} +if (!thp_available ()) +return ; -void thp_save_settings (void ) -{thp_read_settings (&saved_settings); - -/* -* setup exit hooks to make sure THP settings are restored on graceful -* and error exits and signals -*/ -atexit (thp_restore_settings_atexit); -signal (SIGTERM , thp_restore_settings_sighandler); -signal (SIGINT , thp_restore_settings_sighandler); -signal (SIGHUP , thp_restore_settings_sighandler); -signal (SIGQUIT , thp_restore_settings_sighandler); - thp_settings_saved = true; }
@@ -399,11 +380,31 @@ bool thp_is_enabled(void)return mode ==1 || mode ==3 ; } +#define HUGETLB_MAX_NR_PAGESIZES 10 +struct hugetlb_settings { +unsigned long free_hugepages[HUGETLB_MAX_NR_PAGESIZES ]; +unsigned long nr_hugepages[HUGETLB_MAX_NR_PAGESIZES ]; +unsigned long sizes[HUGETLB_MAX_NR_PAGESIZES ]; +unsigned long default_size; +int nr_sizes; +}; + +static struct hugetlb_settings hugetlb_saved_settings; +bool hugetlb_settings_saved; +int detect_hugetlb_page_sizes (unsigned long sizes[],int max) { -DIR *dir =opendir ("/sys/kernel/mm/hugepages/" ); +static struct hugetlb_settings *settings = &hugetlb_saved_settings; +DIR *dir;int count =0 ; +if (settings->nr_sizes) { +for (count =0 ; count < settings->nr_sizes; count++) + sizes[count] = settings->sizes[count]; +return settings->nr_sizes; + } + + dir =opendir ("/sys/kernel/mm/hugepages/" );if (!dir)return 0 ;
@@ -427,11 +428,16 @@ int detect_hugetlb_page_sizes(unsigned long sizes[], int max)unsigned long default_huge_page_size (void ) { +static struct hugetlb_settings *settings = &hugetlb_saved_settings;unsigned long hps =0 ;char *line =NULL ;size_t linelen =0 ; -FILE *f =fopen ("/proc/meminfo" ,"r" ); +FILE *f; +if (settings->default_size) +return settings->default_size; + + f =fopen ("/proc/meminfo" ,"r" );if (!f)return 0 ;while (getline (&line, &linelen, f) >0 ) {
@@ -479,3 +485,127 @@ unsigned long hugetlb_free_pages(unsigned long size)return read_num (path); } + +bool hugetlb_setup_default (unsigned long nr) +{ +unsigned long size; + +hugetlb_save_settings (); + + size =default_huge_page_size (); +hugetlb_set_nr_pages (size, nr); + +return hugetlb_free_pages (size) == nr; +} + +unsigned long hugetlb_setup (unsigned long nr,unsigned long sizes[], +int max) +{ +unsigned long enabled[10 ]; +int nr_sizes =0 ; +int nr_enabled; + +hugetlb_save_settings (); + + nr_enabled =detect_hugetlb_page_sizes (enabled,ARRAY_SIZE (enabled)); +if (!nr_enabled) +return 0 ; + +if (nr_enabled > max) { +ksft_print_msg ("detected %d huge page sizes, will only test %d\n" , nr_enabled, max); + nr_enabled = max; + } + +/* If HugeTLB is supported, request 2 HugeTLB pages of every size. */ +for (int i =0 ; i < nr_enabled; i++) { +hugetlb_set_nr_pages (enabled[i], nr); +if (hugetlb_free_pages (enabled[i]) < nr) +continue ; + + sizes[nr_sizes++] = enabled[i]; + } + +return nr_sizes; +} + +static void __hugetlb_save_settings (void ) +{ +struct hugetlb_settings *settings = &hugetlb_saved_settings; +int nr_sizes; + + settings->default_size =default_huge_page_size (); +if (!settings->default_size) +return ; + + nr_sizes =detect_hugetlb_page_sizes (settings->sizes, +HUGETLB_MAX_NR_PAGESIZES ); +if (!nr_sizes) { + settings->default_size =0 ; +return ; + } + +for (int i =0 ; i < nr_sizes; i++) { +unsigned long sz = settings->sizes[i]; + +if (!sz) +continue ; + + settings->free_hugepages[i] =hugetlb_free_pages (sz); + settings->nr_hugepages[i] =hugetlb_nr_pages (sz); + } + + settings->nr_sizes = nr_sizes; + hugetlb_settings_saved = true; +} + +void hugetlb_restore_settings (void ) +{ +struct hugetlb_settings *settings = &hugetlb_saved_settings; + +if (!hugetlb_settings_saved || !settings->default_size) +return ; + +for (int i =0 ; i <HUGETLB_MAX_NR_PAGESIZES ; i++) { +unsigned long sz = settings->sizes[i]; + +if (!sz) +continue ; + +hugetlb_set_nr_pages (sz, settings->nr_hugepages[i]); + } +} + +static void hugepage_restore_settings_atexit (void ) +{ +if (thp_settings_saved) +thp_restore_settings (); +if (hugetlb_settings_saved) +hugetlb_restore_settings (); +} + +static void hugepage_restore_settings_sighandler (int sig) +{ +/* exit() will invoke the hugetlb_restore_settings_atexit handler. */ +exit (KSFT_FAIL ); +} + +void hugepage_save_settings (bool thp,bool hugetlb) +{ +if (!thp && !hugetlb) +return ; + +if (thp) +__thp_save_settings (); +if (hugetlb) +__hugetlb_save_settings (); + +/* +* setup exit hooks to make sure THP settings are restored on graceful +* and error exits and signals +*/ +atexit (hugepage_restore_settings_atexit); +signal (SIGTERM , hugepage_restore_settings_sighandler); +signal (SIGINT , hugepage_restore_settings_sighandler); +signal (SIGHUP , hugepage_restore_settings_sighandler); +signal (SIGQUIT , hugepage_restore_settings_sighandler); +}
tools/testing/selftests/mm/hugepage_settings.h +29 -1
@@ -6,6 +6,8 @@#include <stddef.h> #include <stdint.h> +void hugepage_save_settings (bool thp,bool hugetlb); +/* Transparent Huge Pages (THP) */ enum thp_enabled {
@@ -79,7 +81,11 @@ struct thp_settings *thp_current_settings(void);void thp_push_settings (struct thp_settings *settings);void thp_pop_settings (void );void thp_restore_settings (void ); -void thp_save_settings (void ); + +static inline void thp_save_settings (void ) +{ +hugepage_save_settings (/* thp = */ true,/* hugetlb = */ false); +}void thp_set_read_ahead_path (char *path);unsigned long thp_supported_orders (void );
@@ -97,6 +103,13 @@ unsigned long hugetlb_nr_pages(unsigned long size);void hugetlb_set_nr_pages (unsigned long size,unsigned long nr);unsigned long hugetlb_free_pages (unsigned long size); +static inline void hugetlb_save_settings (void ) +{ +hugepage_save_settings (/* thp = */ false,/* hugetlb = */ true); +} + +void hugetlb_restore_settings (void ); +static inline unsigned long hugetlb_nr_default_pages (void ) {unsigned long size =default_huge_page_size ();
@@ -127,4 +140,19 @@ static inline unsigned long hugetlb_free_default_pages(void) return hugetlb_free_pages(size); } +static inline bool hugetlb_available (void ) +{ +return default_huge_page_size () !=0 ; +} + +bool hugetlb_setup_default (unsigned long nr); +unsigned long hugetlb_setup (unsigned long nr,unsigned long sizes[], +int max); + +#define HUGETLB_SETUP_DEFAULT_PAGES (nr_pages) \ +static void __attribute__((constructor)) __hugetlb_setup_default(void) \ +{ \ + hugetlb_setup_default((nr_pages)); \ +} +#endif /* __THP_SETTINGS_H__ */
Signature · 1 line
2.53.0
Re: [PATCH v3 27/54] selftests/mm: hugepage_settings: add APIs for HugeTLB setup and teardown
Quoted 70 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
A lot of tests require free HugeTLB pages. Some need just a few default huge pages, some need a certain amount of memory available as HugeTLB, and some just skip lots of tests if huge pages of all supported sizes are not available.
This all resulted in a huge mess in
run_vmtests.shthat sets up some huge pages, adjusts them later and restores some of the settings if the stars align.Add APIs that allow saving the state of HugeTLB and setting up the desired amount of HugeTLB pages.
Saving the state also registers
atexit()callback and signal handler that will ensure restoration of HugeTLB state.Since many tests use both HugeTLB and THP, the
atexit()callbacks and signal handler are restoring both.For
kselftest_harnesstests that run fixture setups and test in child processes add a constructor that will save and restore settings in the main process.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- .../testing/selftests/mm/hugepage_settings.c | 178 +++++++++++++++--- .../testing/selftests/mm/hugepage_settings.h | 30 ++- 2 files changed, 183 insertions(+), 25 deletions(-)tools/testing/selftests/mm/hugepage_settings.c +3
-22@@ -306,31 +306,12 @@ void thp_restore_settings(void)thp_write_settings (&saved_settings); } -static void thp_restore_settings_atexit (void ) +static void __thp_save_settings (void ) { -thp_restore_settings (); -} - -static void thp_restore_settings_sighandler (int sig) -{ -/* exit() will invoke the thp_restore_settings_atexit handler. */ -exit (KSFT_FAIL ); -} +if (!thp_available ()) +return ; -void thp_save_settings (void ) -{thp_read_settings (&saved_settings); - -/* -* setup exit hooks to make sure THP settings are restored on graceful -* and error exits and signals -*/ -atexit (thp_restore_settings_atexit); -signal (SIGTERM , thp_restore_settings_sighandler); -signal (SIGINT , thp_restore_settings_sighandler); -signal (SIGHUP , thp_restore_settings_sighandler); -signal (SIGQUIT , thp_restore_settings_sighandler); - thp_settings_saved = true; }
tools/testing/selftests/mm/hugepage_settings.c bool thp_is_enabled(void) · line 385
Quoted 6 lines
tools/testing/selftests/mm/hugepage_settings.c +3
-0@@ -399,11 +380,31 @@ bool thp_is_enabled(void)return mode ==1 || mode ==3 ; } +#define HUGETLB_MAX_NR_PAGESIZES 10 +struct hugetlb_settings { +unsigned long free_hugepages[HUGETLB_MAX_NR_PAGESIZES ];
It looks like we don't actually use this in the API or maybe I'm missing something?
Otherwise, the API looks great to me. But if you allow me to bikeshed :)
I think I'd do:
struct hugetlb_state {
unsigned long nr_hugepages;
unsigned long size;
};struct hugetlb_settings {
struct hugetlb_state hstate[HUGETLB_MAX_PAGESIZES];
unsigned long default_size;
int nr_sizes;
};Maybe for the future...
Quoted 232 lines
tools/testing/selftests/mm/hugepage_settings.c +148
-2+unsigned long nr_hugepages[HUGETLB_MAX_NR_PAGESIZES ]; +unsigned long sizes[HUGETLB_MAX_NR_PAGESIZES ]; +unsigned long default_size; +int nr_sizes; +}; + +static struct hugetlb_settings hugetlb_saved_settings; +bool hugetlb_settings_saved; +int detect_hugetlb_page_sizes (unsigned long sizes[],int max) { -DIR *dir =opendir ("/sys/kernel/mm/hugepages/" ); +static struct hugetlb_settings *settings = &hugetlb_saved_settings; +DIR *dir;int count =0 ; +if (settings->nr_sizes) { +for (count =0 ; count < settings->nr_sizes; count++) + sizes[count] = settings->sizes[count]; +return settings->nr_sizes; + } + + dir =opendir ("/sys/kernel/mm/hugepages/" );if (!dir)return 0 ;@@ -427,11 +428,16 @@ int detect_hugetlb_page_sizes(unsigned long sizes[], int max)unsigned long default_huge_page_size (void ) { +static struct hugetlb_settings *settings = &hugetlb_saved_settings;unsigned long hps =0 ;char *line =NULL ;size_t linelen =0 ; -FILE *f =fopen ("/proc/meminfo" ,"r" ); +FILE *f; +if (settings->default_size) +return settings->default_size; + + f =fopen ("/proc/meminfo" ,"r" );if (!f)return 0 ;while (getline (&line, &linelen, f) >0 ) {@@ -479,3 +485,127 @@ unsigned long hugetlb_free_pages(unsigned long size)return read_num (path); } + +bool hugetlb_setup_default (unsigned long nr) +{ +unsigned long size; + +hugetlb_save_settings (); + + size =default_huge_page_size (); +hugetlb_set_nr_pages (size, nr); + +return hugetlb_free_pages (size) == nr; +} + +unsigned long hugetlb_setup (unsigned long nr,unsigned long sizes[], +int max) +{ +unsigned long enabled[10 ]; +int nr_sizes =0 ; +int nr_enabled; + +hugetlb_save_settings (); + + nr_enabled =detect_hugetlb_page_sizes (enabled,ARRAY_SIZE (enabled)); +if (!nr_enabled) +return 0 ; + +if (nr_enabled > max) { +ksft_print_msg ("detected %d huge page sizes, will only test %d\n" , nr_enabled, max); + nr_enabled = max; + } + +/* If HugeTLB is supported, request 2 HugeTLB pages of every size. */ +for (int i =0 ; i < nr_enabled; i++) { +hugetlb_set_nr_pages (enabled[i], nr); +if (hugetlb_free_pages (enabled[i]) < nr) +continue ; + + sizes[nr_sizes++] = enabled[i]; + } + +return nr_sizes; +} + +static void __hugetlb_save_settings (void ) +{ +struct hugetlb_settings *settings = &hugetlb_saved_settings; +int nr_sizes; + + settings->default_size =default_huge_page_size (); +if (!settings->default_size) +return ; + + nr_sizes =detect_hugetlb_page_sizes (settings->sizes, +HUGETLB_MAX_NR_PAGESIZES ); +if (!nr_sizes) { + settings->default_size =0 ; +return ; + } + +for (int i =0 ; i < nr_sizes; i++) { +unsigned long sz = settings->sizes[i]; + +if (!sz) +continue ; + + settings->free_hugepages[i] =hugetlb_free_pages (sz); + settings->nr_hugepages[i] =hugetlb_nr_pages (sz); + } + + settings->nr_sizes = nr_sizes; + hugetlb_settings_saved = true; +} + +void hugetlb_restore_settings (void ) +{ +struct hugetlb_settings *settings = &hugetlb_saved_settings; + +if (!hugetlb_settings_saved || !settings->default_size) +return ; + +for (int i =0 ; i <HUGETLB_MAX_NR_PAGESIZES ; i++) { +unsigned long sz = settings->sizes[i]; + +if (!sz) +continue ; + +hugetlb_set_nr_pages (sz, settings->nr_hugepages[i]); + } +} + +static void hugepage_restore_settings_atexit (void ) +{ +if (thp_settings_saved) +thp_restore_settings (); +if (hugetlb_settings_saved) +hugetlb_restore_settings (); +} + +static void hugepage_restore_settings_sighandler (int sig) +{ +/* exit() will invoke the hugetlb_restore_settings_atexit handler. */ +exit (KSFT_FAIL ); +} + +void hugepage_save_settings (bool thp,bool hugetlb) +{ +if (!thp && !hugetlb) +return ; + +if (thp) +__thp_save_settings (); +if (hugetlb) +__hugetlb_save_settings (); + +/* +* setup exit hooks to make sure THP settings are restored on graceful +* and error exits and signals +*/ +atexit (hugepage_restore_settings_atexit); +signal (SIGTERM , hugepage_restore_settings_sighandler); +signal (SIGINT , hugepage_restore_settings_sighandler); +signal (SIGHUP , hugepage_restore_settings_sighandler); +signal (SIGQUIT , hugepage_restore_settings_sighandler); +}tools/testing/selftests/mm/hugepage_settings.h +29
-1@@ -6,6 +6,8 @@#include <stddef.h> #include <stdint.h> +void hugepage_save_settings (bool thp,bool hugetlb); +/* Transparent Huge Pages (THP) */ enum thp_enabled {@@ -79,7 +81,11 @@ struct thp_settings *thp_current_settings(void);void thp_push_settings (struct thp_settings *settings);void thp_pop_settings (void );void thp_restore_settings (void ); -void thp_save_settings (void ); + +static inline void thp_save_settings (void ) +{ +hugepage_save_settings (/* thp = */ true,/* hugetlb = */ false); +}void thp_set_read_ahead_path (char *path);unsigned long thp_supported_orders (void );@@ -97,6 +103,13 @@ unsigned long hugetlb_nr_pages(unsigned long size);void hugetlb_set_nr_pages (unsigned long size,unsigned long nr);unsigned long hugetlb_free_pages (unsigned long size); +static inline void hugetlb_save_settings (void ) +{ +hugepage_save_settings (/* thp = */ false,/* hugetlb = */ true); +} + +void hugetlb_restore_settings (void ); +static inline unsigned long hugetlb_nr_default_pages (void ) {unsigned long size =default_huge_page_size ();@@ -127,4 +140,19 @@ static inline unsigned long hugetlb_free_default_pages(void) return hugetlb_free_pages(size); } +static inline bool hugetlb_available (void ) +{ +return default_huge_page_size () !=0 ; +} + +bool hugetlb_setup_default (unsigned long nr); +unsigned long hugetlb_setup (unsigned long nr,unsigned long sizes[], +int max); + +#define HUGETLB_SETUP_DEFAULT_PAGES (nr_pages) \ +static void __attribute__((constructor)) __hugetlb_setup_default(void) \ +{ \ + hugetlb_setup_default((nr_pages)); \ +} +#endif /* __THP_SETTINGS_H__ */
Mike Rapoport
maintainerRe: [PATCH v3 27/54] selftests/mm: hugepage_settings: add APIs for HugeTLB setup and teardown
Quoted 68 lines · Luiz Capitulino
Quoted 67 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
A lot of tests require free HugeTLB pages. Some need just a few default huge pages, some need a certain amount of memory available as HugeTLB, and some just skip lots of tests if huge pages of all supported sizes are not available.
This all resulted in a huge mess in
run_vmtests.shthat sets up some huge pages, adjusts them later and restores some of the settings if the stars align.Add APIs that allow saving the state of HugeTLB and setting up the desired amount of HugeTLB pages.
Saving the state also registers
atexit()callback and signal handler that will ensure restoration of HugeTLB state.Since many tests use both HugeTLB and THP, the
atexit()callbacks and signal handler are restoring both.For
kselftest_harnesstests that run fixture setups and test in child processes add a constructor that will save and restore settings in the main process.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- .../testing/selftests/mm/hugepage_settings.c | 178 +++++++++++++++--- .../testing/selftests/mm/hugepage_settings.h | 30 ++- 2 files changed, 183 insertions(+), 25 deletions(-)tools/testing/selftests/mm/hugepage_settings.c +3
-22@@ -306,31 +306,12 @@ void thp_restore_settings(void)thp_write_settings (&saved_settings); } -static void thp_restore_settings_atexit (void ) +static void __thp_save_settings (void ) { -thp_restore_settings (); -} - -static void thp_restore_settings_sighandler (int sig) -{ -/* exit() will invoke the thp_restore_settings_atexit handler. */ -exit (KSFT_FAIL ); -} +if (!thp_available ()) +return ; -void thp_save_settings (void ) -{thp_read_settings (&saved_settings); - -/* -* setup exit hooks to make sure THP settings are restored on graceful -* and error exits and signals -*/ -atexit (thp_restore_settings_atexit); -signal (SIGTERM , thp_restore_settings_sighandler); -signal (SIGINT , thp_restore_settings_sighandler); -signal (SIGHUP , thp_restore_settings_sighandler); -signal (SIGQUIT , thp_restore_settings_sighandler); - thp_settings_saved = true; }@@ -399,11 +380,31 @@ bool thp_is_enabled(void)
Quoted 8 lines
tools/testing/selftests/mm/hugepage_settings.c bool thp_is_enabled(void) · line 385
Quoted 5 lines
tools/testing/selftests/mm/hugepage_settings.c +3
-0return mode ==1 || mode ==3 ; } +#define HUGETLB_MAX_NR_PAGESIZES 10 +struct hugetlb_settings { +unsigned long free_hugepages[HUGETLB_MAX_NR_PAGESIZES ];It looks like we don't actually use this in the API or maybe I'm missing something?
Yes, it's used only in later patches.
I felt it would be too churny to add users in this patch.
Quoted 9 lines
Otherwise, the API looks great to me. But if you allow me to bikeshed :)
I think I'd do:
struct hugetlb_state { unsigned long nr_hugepages; unsigned long size; };
Quoted 5 lines
struct hugetlb_settings {
struct hugetlb_state hstate[HUGETLB_MAX_PAGESIZES];
unsigned long default_size;
int nr_sizes;
};It might be slightly nicer abstraction but ...
Quoted 11 lines
tools/testing/selftests/mm/hugepage_settings.c unsigned long hugetlb_free_pages(unsigned long size) · line 541
Quoted 11 lines
tools/testing/selftests/mm/hugepage_settings.c +11
-0+static void __hugetlb_save_settings (void ) +{ +struct hugetlb_settings *settings = &hugetlb_saved_settings; +int nr_sizes; + + settings->default_size =default_huge_page_size (); +if (!settings->default_size) +return ; + + nr_sizes =detect_hugetlb_page_sizes (settings->sizes, +HUGETLB_MAX_NR_PAGESIZES );
... this immediately becomes more complex.
Quoted 5 lines
Quoted 5 lines
tools/testing/selftests/mm/hugepage_settings.c +5
-0+if (!nr_sizes) { + settings->default_size =0 ; +return ; + } +
Quoted 8 lines
tools/testing/selftests/mm/hugepage_settings.c unsigned long hugetlb_free_pages(unsigned long size) · line 554
Quoted 8 lines
tools/testing/selftests/mm/hugepage_settings.c +8
-0+for (int i =0 ; i < nr_sizes; i++) { +unsigned long sz = settings->sizes[i]; + +if (!sz) +continue ; + + settings->free_hugepages[i] =hugetlb_free_pages (sz); + settings->nr_hugepages[i] =hugetlb_nr_pages (sz);
And this as well.
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v3 27/54] selftests/mm: hugepage_settings: add APIs for HugeTLB setup and teardown
Quoted 78 lines · Mike Rapoport
Quoted 68 lines · Luiz Capitulino
Quoted 67 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
A lot of tests require free HugeTLB pages. Some need just a few default huge pages, some need a certain amount of memory available as HugeTLB, and some just skip lots of tests if huge pages of all supported sizes are not available.
This all resulted in a huge mess in
run_vmtests.shthat sets up some huge pages, adjusts them later and restores some of the settings if the stars align.Add APIs that allow saving the state of HugeTLB and setting up the desired amount of HugeTLB pages.
Saving the state also registers
atexit()callback and signal handler that will ensure restoration of HugeTLB state.Since many tests use both HugeTLB and THP, the
atexit()callbacks and signal handler are restoring both.For
kselftest_harnesstests that run fixture setups and test in child processes add a constructor that will save and restore settings in the main process.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- .../testing/selftests/mm/hugepage_settings.c | 178 +++++++++++++++--- .../testing/selftests/mm/hugepage_settings.h | 30 ++- 2 files changed, 183 insertions(+), 25 deletions(-)tools/testing/selftests/mm/hugepage_settings.c +3
-22@@ -306,31 +306,12 @@ void thp_restore_settings(void)thp_write_settings (&saved_settings); } -static void thp_restore_settings_atexit (void ) +static void __thp_save_settings (void ) { -thp_restore_settings (); -} - -static void thp_restore_settings_sighandler (int sig) -{ -/* exit() will invoke the thp_restore_settings_atexit handler. */ -exit (KSFT_FAIL ); -} +if (!thp_available ()) +return ; -void thp_save_settings (void ) -{thp_read_settings (&saved_settings); - -/* -* setup exit hooks to make sure THP settings are restored on graceful -* and error exits and signals -*/ -atexit (thp_restore_settings_atexit); -signal (SIGTERM , thp_restore_settings_sighandler); -signal (SIGINT , thp_restore_settings_sighandler); -signal (SIGHUP , thp_restore_settings_sighandler); -signal (SIGQUIT , thp_restore_settings_sighandler); - thp_settings_saved = true; }@@ -399,11 +380,31 @@ bool thp_is_enabled(void)Quoted 8 lines
tools/testing/selftests/mm/hugepage_settings.c bool thp_is_enabled(void) · line 384
Quoted 5 lines
tools/testing/selftests/mm/hugepage_settings.c +3
-0return mode ==1 || mode ==3 ; } +#define HUGETLB_MAX_NR_PAGESIZES 10 +struct hugetlb_settings { +unsigned long free_hugepages[HUGETLB_MAX_NR_PAGESIZES ];It looks like we don't actually use this in the API or maybe I'm missing something?
Quoted 1 line
Yes, it's used only in later patches.
I don't see it, the patch below builds fine (again, sorry in advance if I'm missing something obvious):
tools/testing/selftests/mm/hugepage_settings.c +0 -2
@@ -341,7 +341,6 @@ bool thp_is_enabled(void)#define HUGETLB_MAX_NR_PAGESIZES 10struct hugetlb_settings { -unsigned long free_hugepages[HUGETLB_MAX_NR_PAGESIZES ];unsigned long nr_hugepages[HUGETLB_MAX_NR_PAGESIZES ];unsigned long sizes[HUGETLB_MAX_NR_PAGESIZES ];unsigned long default_size;
@@ -509,7 +508,6 @@ static void __hugetlb_save_settings(void)if (!sz)continue ; - settings->free_hugepages[i] =hugetlb_free_pages (sz); settings->nr_hugepages[i] =hugetlb_nr_pages (sz); }
Quoted 33 lines
I felt it would be too churny to add users in this patch.
Quoted 9 lines
Otherwise, the API looks great to me. But if you allow me to bikeshed :)
I think I'd do:
struct hugetlb_state { unsigned long nr_hugepages; unsigned long size; };Quoted 5 lines
struct hugetlb_settings { struct hugetlb_state hstate[HUGETLB_MAX_PAGESIZES]; unsigned long default_size; int nr_sizes; };It might be slightly nicer abstraction but ...
Quoted 11 lines
tools/testing/selftests/mm/hugepage_settings.c bool thp_is_enabled(void) · line 395
Quoted 11 lines
tools/testing/selftests/mm/hugepage_settings.c +11
-0+static void __hugetlb_save_settings (void ) +{ +struct hugetlb_settings *settings = &hugetlb_saved_settings; +int nr_sizes; + + settings->default_size =default_huge_page_size (); +if (!settings->default_size) +return ; + + nr_sizes =detect_hugetlb_page_sizes (settings->sizes, +HUGETLB_MAX_NR_PAGESIZES );... this immediately becomes more complex.
Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
tools/testing/selftests/mm/hugepage_settings.c +5
-0+if (!nr_sizes) { + settings->default_size =0 ; +return ; + } +Quoted 8 lines
tools/testing/selftests/mm/hugepage_settings.c bool thp_is_enabled(void) · line 408
Quoted 8 lines
tools/testing/selftests/mm/hugepage_settings.c +8
-0+for (int i =0 ; i < nr_sizes; i++) { +unsigned long sz = settings->sizes[i]; + +if (!sz) +continue ; + + settings->free_hugepages[i] =hugetlb_free_pages (sz); + settings->nr_hugepages[i] =hugetlb_nr_pages (sz);And this as well.
We'd need a little more code to handle those cases, may or may not be worth it. But enough bikesheding from my side :) the series is great as is.
Mike Rapoport
maintainerRe: [PATCH v3 27/54] selftests/mm: hugepage_settings: add APIs for HugeTLB setup and teardown
Quoted 10 lines · Luiz Capitulino
Quoted 9 lines · Mike Rapoport
Quoted 8 lines
Review
Quoted 5 lines
return mode == 1 || mode == 3; }(quoted diff) +3
-0+#define HUGETLB_MAX_NR_PAGESIZES 10 +struct hugetlb_settings { + unsigned long free_hugepages[HUGETLB_MAX_NR_PAGESIZES];It looks like we don't actually use this in the API or maybe I'm missing something?
Quoted 4 lines
Quoted 1 line
Yes, it's used only in later patches.
I don't see it, the patch below builds fine (again, sorry in advance if I'm missing something obvious):
You are right, I misunderstood you comment and I thought you referred to entire API being unused in this patch.
free_hugepages is not used and it should not be a part of the saved state.
Quoted 70 lines
tools/testing/selftests/mm/hugepage_settings.c +0
-2@@ -341,7 +341,6 @@ bool thp_is_enabled(void)#define HUGETLB_MAX_NR_PAGESIZES 10struct hugetlb_settings { -unsigned long free_hugepages[HUGETLB_MAX_NR_PAGESIZES ];unsigned long nr_hugepages[HUGETLB_MAX_NR_PAGESIZES ];unsigned long sizes[HUGETLB_MAX_NR_PAGESIZES ];unsigned long default_size;@@ -509,7 +508,6 @@ static void __hugetlb_save_settings(void)if (!sz)continue ; - settings->free_hugepages[i] =hugetlb_free_pages (sz); settings->nr_hugepages[i] =hugetlb_nr_pages (sz); }Quoted 31 lines
I felt it would be too churny to add users in this patch.
Quoted 9 lines
Otherwise, the API looks great to me. But if you allow me to bikeshed :)
I think I'd do:
struct hugetlb_state { unsigned long nr_hugepages; unsigned long size; };Quoted 5 lines
struct hugetlb_settings { struct hugetlb_state hstate[HUGETLB_MAX_PAGESIZES]; unsigned long default_size; int nr_sizes; };It might be slightly nicer abstraction but ...
Quoted 11 lines
Review
Quoted 11 lines
(quoted diff) +11
-0+static void __hugetlb_save_settings(void) +{ + struct hugetlb_settings *settings = &hugetlb_saved_settings; + int nr_sizes; + + settings->default_size = default_huge_page_size(); + if (!settings->default_size) + return; + + nr_sizes = detect_hugetlb_page_sizes(settings->sizes, + HUGETLB_MAX_NR_PAGESIZES);... this immediately becomes more complex.
Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
(quoted diff) +5
-0+ if (!nr_sizes) { + settings->default_size = 0; + return; + } +Quoted 8 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ for (int i = 0; i < nr_sizes; i++) { + unsigned long sz = settings->sizes[i]; + + if (!sz) + continue; + + settings->free_hugepages[i] = hugetlb_free_pages(sz); + settings->nr_hugepages[i] = hugetlb_nr_pages(sz);And this as well.
We'd need a little more code to handle those cases, may or may not be worth it. But enough bikesheding from my side :) the series is great as is.
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v3 27/54] selftests/mm: hugepage_settings: add APIs for HugeTLB setup and teardown
Quoted 13 lines · Mike Rapoport
Quoted 12 lines · Luiz Capitulino
Quoted 10 lines · Mike Rapoport
Quoted 8 lines
Review
Quoted 5 lines
return mode == 1 || mode == 3; }(quoted diff) +3
-0+#define HUGETLB_MAX_NR_PAGESIZES 10 +struct hugetlb_settings { + unsigned long free_hugepages[HUGETLB_MAX_NR_PAGESIZES];It looks like we don't actually use this in the API or maybe I'm missing something?
Yes, it's used only in later patches.
Quoted 7 lines
Quoted 2 lines
I don't see it, the patch below builds fine (again, sorry in advance if I'm missing something obvious):
You are right, I misunderstood you comment and I thought you referred to entire API being unused in this patch.
free_hugepagesis not used and it should not be a part of the saved state.
Thanks, I'll try to be more specific next time.
Quoted 72 lines
Quoted 71 lines
tools/testing/selftests/mm/hugepage_settings.c +0
-2@@ -341,7 +341,6 @@ bool thp_is_enabled(void)#define HUGETLB_MAX_NR_PAGESIZES 10struct hugetlb_settings { -unsigned long free_hugepages[HUGETLB_MAX_NR_PAGESIZES ];unsigned long nr_hugepages[HUGETLB_MAX_NR_PAGESIZES ];unsigned long sizes[HUGETLB_MAX_NR_PAGESIZES ];unsigned long default_size;@@ -509,7 +508,6 @@ static void __hugetlb_save_settings(void)if (!sz)continue ; - settings->free_hugepages[i] =hugetlb_free_pages (sz); settings->nr_hugepages[i] =hugetlb_nr_pages (sz); }Quoted 31 lines
I felt it would be too churny to add users in this patch.
Quoted 9 lines
Otherwise, the API looks great to me. But if you allow me to bikeshed :)
I think I'd do:
struct hugetlb_state { unsigned long nr_hugepages; unsigned long size; };Quoted 5 lines
struct hugetlb_settings { struct hugetlb_state hstate[HUGETLB_MAX_PAGESIZES]; unsigned long default_size; int nr_sizes; };It might be slightly nicer abstraction but ...
Quoted 11 lines
Review
Quoted 11 lines
(quoted diff) +11
-0+static void __hugetlb_save_settings(void) +{ + struct hugetlb_settings *settings = &hugetlb_saved_settings; + int nr_sizes; + + settings->default_size = default_huge_page_size(); + if (!settings->default_size) + return; + + nr_sizes = detect_hugetlb_page_sizes(settings->sizes, + HUGETLB_MAX_NR_PAGESIZES);... this immediately becomes more complex.
Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
(quoted diff) +5
-0+ if (!nr_sizes) { + settings->default_size = 0; + return; + } +Quoted 8 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ for (int i = 0; i < nr_sizes; i++) { + unsigned long sz = settings->sizes[i]; + + if (!sz) + continue; + + settings->free_hugepages[i] = hugetlb_free_pages(sz); + settings->nr_hugepages[i] = hugetlb_nr_pages(sz);And this as well.
We'd need a little more code to handle those cases, may or may not be worth it. But enough bikesheding from my side :) the series is great as is.
Re: [PATCH v3 27/54] selftests/mm: hugepage_settings: add APIs for HugeTLB setup and teardown
In reply to Mike RapoportQuoted 20 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
A lot of tests require free HugeTLB pages. Some need just a few default huge pages, some need a certain amount of memory available as HugeTLB, and some just skip lots of tests if huge pages of all supported sizes are not available.
This all resulted in a huge mess in
run_vmtests.shthat sets up some huge pages, adjusts them later and restores some of the settings if the stars align.Add APIs that allow saving the state of HugeTLB and setting up the desired amount of HugeTLB pages.
Saving the state also registers
atexit()callback and signal handler that will ensure restoration of HugeTLB state.Since many tests use both HugeTLB and THP, the
atexit()callbacks and signal handler are restoring both.
Quoted 6 lines
For
kselftest_harnesstests that run fixture setups and test in child processes add a constructor that will save and restore settings in the main process.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Looks good to me.
Mike Rapoport
maintainerPATCH v3 28/54 selftests/mm: move read_file(), read_num() and write_num() to vm_util
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
These are useful helpers for writing and reading sysfs and proc files.
Make them available to the tests that don't use thp_settings.
While on it make write_num() use "%lu" instead of "%ld" to match 'unsigned long num' argument type.
--- .../testing/selftests/mm/hugepage_settings.c | 41 ------------------- .../testing/selftests/mm/hugepage_settings.h | 4 -- tools/testing/selftests/mm/vm_util.c | 39 ++++++++++++++++++ tools/testing/selftests/mm/vm_util.h | 3 ++ 4 files changed, 42 insertions(+), 45 deletions(-)
tools/testing/selftests/mm/hugepage_settings.c +0 -41
@@ -48,47 +48,6 @@ static const char * const shmem_enabled_strings[] = { NULL }; -int read_file (const char *path,char *buf,size_t buflen) -{ -int fd; -ssize_t numread; - - fd =open (path,O_RDONLY ); -if (fd ==-1 ) -return 0 ; - - numread =read (fd, buf, buflen -1 ); -if (numread <1 ) { -close (fd); -return 0 ; - } - - buf[numread] ='\0' ; -close (fd); - -return (unsigned int ) numread; -} - -unsigned long read_num (const char *path) -{ -char buf[21 ]; - -if (read_file (path, buf,sizeof (buf)) <0 ) { -perror ("read_file()" ); -exit (EXIT_FAILURE ); - } - -return strtoul (buf,NULL ,10 ); -} - -void write_num (const char *path,unsigned long num) -{ -char buf[21 ]; - -sprintf (buf,"%ld" , num); -write_file (path, buf,strlen (buf) +1 ); -} -int thp_read_string (const char *name,const char *const strings[]) {char path[PATH_MAX ];
tools/testing/selftests/mm/hugepage_settings.h +0 -4
@@ -66,10 +66,6 @@ struct thp_settings {struct shmem_hugepages_settings shmem_hugepages[NR_ORDERS ]; }; -int read_file (const char *path,char *buf,size_t buflen); -unsigned long read_num (const char *path); -void write_num (const char *path,unsigned long num); -int thp_read_string (const char *name,const char *const strings[]);void thp_write_string (const char *name,const char *val);unsigned long thp_read_num (const char *name);
tools/testing/selftests/mm/vm_util.c +39 -0
@@ -699,6 +699,27 @@ int unpoison_memory(unsigned long pfn)return ret >0 ?0 : -errno; } +int read_file (const char *path,char *buf,size_t buflen) +{ +int fd; +ssize_t numread; + + fd =open (path,O_RDONLY ); +if (fd ==-1 ) +return 0 ; + + numread =read (fd, buf, buflen -1 ); +if (numread <1 ) { +close (fd); +return 0 ; + } + + buf[numread] ='\0' ; +close (fd); + +return (unsigned int ) numread; +} +void write_file (const char *path,const char *buf,size_t buflen) {int fd, saved_errno;
@@ -722,3 +743,21 @@ void write_file(const char *path, const char *buf, size_t buflen)ksft_exit_fail_msg ("%s write(%.*s) is truncated, expected %zu bytes, got %zd bytes\n" , path, (int )(buflen -1 ), buf, buflen -1 , numwritten); } + +unsigned long read_num (const char *path) +{ +char buf[21 ]; + +if (read_file (path, buf,sizeof (buf)) <0 ) +ksft_exit_fail_perror ("read_file()" ); + +return strtoul (buf,NULL ,10 ); +} + +void write_num (const char *path,unsigned long num) +{ +char buf[21 ]; + +sprintf (buf,"%lu" , num); +write_file (path, buf,strlen (buf) +1 ); +}
tools/testing/selftests/mm/vm_util.h +3 -0
@@ -165,3 +165,6 @@ int unpoison_memory(unsigned long pfn);#define PAGEMAP_PFN (ent) ((ent) & ((1ull << 55) - 1))void write_file (const char *path,const char *buf,size_t buflen); +int read_file (const char *path,char *buf,size_t buflen); +unsigned long read_num (const char *path); +void write_num (const char *path,unsigned long num);
Signature · 1 line
2.53.0
Re: [PATCH v3 28/54] selftests/mm: move read_file(), read_num() and write_num() to vm_util
Quoted 11 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
These are useful helpers for writing and reading sysfs and proc files.
Make them available to the tests that don't use
thp_settings.While on it make
write_num()use "%lu" instead of "%ld" to match 'unsigned long num' argument type.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
Looks good to me.
Mike Rapoport
maintainerPATCH v3 29/54 selftests/mm: vm_util: add helpers to set and restore shm limits
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-shm and thuge-gen tests require that limits defined by /proc/sys/kernel/{shmmax,shmall} should be higher than certain values.
Add helpers that allow setting these limits and restoring their settings on a test exit.
They will be used later in hugetlb-shm and thuge-gen.
--- tools/testing/selftests/mm/vm_util.c | 28 ++++++++++++++++++++++++++++ tools/testing/selftests/mm/vm_util.h | 9 +++++++++ 2 files changed, 37 insertions(+)
tools/testing/selftests/mm/vm_util.c +28 -0
@@ -761,3 +761,31 @@ void write_num(const char *path, unsigned long num)sprintf (buf,"%lu" , num);write_file (path, buf,strlen (buf) +1 ); } + +static unsigned long shmall, shmmax; + +void __shm_limits_restore (void ) +{ +if (shmmax) +write_num ("/proc/sys/kernel/shmmax" , shmmax); +if (shmall) +write_num ("/proc/sys/kernel/shmall" , shmall); +} + +void shm_limits_prepare (unsigned long length) +{ +unsigned long nr = length /psize (); +unsigned long val; + + val =read_num ("/proc/sys/kernel/shmmax" ); +if (val < length) { +write_num ("/proc/sys/kernel/shmmax" , length); + shmmax = val; + } + + val =read_num ("/proc/sys/kernel/shmall" ); +if (val < nr) { +write_num ("/proc/sys/kernel/shmall" , nr); + shmall = val; + } +}
tools/testing/selftests/mm/vm_util.h +9 -0
@@ -168,3 +168,12 @@ void write_file(const char *path, const char *buf, size_t buflen);int read_file (const char *path,char *buf,size_t buflen);unsigned long read_num (const char *path);void write_num (const char *path,unsigned long num); + +void shm_limits_prepare (unsigned long length); +void __shm_limits_restore (void ); + +#define SHM_LIMITS_RESTORE () \ +static void __attribute__((destructor)) shm_limits_restore(void) \ +{ \ + __shm_limits_restore(); \ +}
Signature · 1 line
2.53.0
Re: [PATCH v3 29/54] selftests/mm: vm_util: add helpers to set and restore shm limits
Quoted 11 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-shm and thuge-gen tests require that limits defined by
/proc/sys/kernel/{shmmax,shmall}should be higher than certain values.Add helpers that allow setting these limits and restoring their settings on a test exit.
They will be used later in hugetlb-shm and thuge-gen.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Looks good to me.
Quoted 58 lines
--- tools/testing/selftests/mm/vm_util.c | 28 ++++++++++++++++++++++++++++ tools/testing/selftests/mm/vm_util.h | 9 +++++++++ 2 files changed, 37 insertions(+)tools/testing/selftests/mm/vm_util.c +28
-0@@ -761,3 +761,31 @@ void write_num(const char *path, unsigned long num)sprintf (buf,"%lu" , num);write_file (path, buf,strlen (buf) +1 ); } + +static unsigned long shmall, shmmax; + +void __shm_limits_restore (void ) +{ +if (shmmax) +write_num ("/proc/sys/kernel/shmmax" , shmmax); +if (shmall) +write_num ("/proc/sys/kernel/shmall" , shmall); +} + +void shm_limits_prepare (unsigned long length) +{ +unsigned long nr = length /psize (); +unsigned long val; + + val =read_num ("/proc/sys/kernel/shmmax" ); +if (val < length) { +write_num ("/proc/sys/kernel/shmmax" , length); + shmmax = val; + } + + val =read_num ("/proc/sys/kernel/shmall" ); +if (val < nr) { +write_num ("/proc/sys/kernel/shmall" , nr); + shmall = val; + } +}tools/testing/selftests/mm/vm_util.h +9
-0@@ -168,3 +168,12 @@ void write_file(const char *path, const char *buf, size_t buflen);int read_file (const char *path,char *buf,size_t buflen);unsigned long read_num (const char *path);void write_num (const char *path,unsigned long num); + +void shm_limits_prepare (unsigned long length); +void __shm_limits_restore (void ); + +#define SHM_LIMITS_RESTORE () \ +static void __attribute__((destructor)) shm_limits_restore(void) \ +{ \ + __shm_limits_restore(); \ +}
Mike Rapoport
maintainerPATCH v3 30/54 selftests/mm: compaction_test: use HugeTLB helpers ...
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
... instead of open coded access of HugeTLB parameters via /proc.
--- tools/testing/selftests/mm/compaction_test.c | 113 +++---------------- 1 file changed, 17 insertions(+), 96 deletions(-)
tools/testing/selftests/mm/compaction_test.c +17 -96
@@ -17,6 +17,7 @@#include <string.h> #include "kselftest.h" +#include "hugepage_settings.h" #define MAP_SIZE_MB 100#define MAP_SIZE (MAP_SIZE_MB * 1024 * 1024)
@@ -82,75 +83,33 @@ int prereq(void)return -1 ; } -int check_compaction (unsigned long mem_free,unsigned long hugepage_size, -unsigned long initial_nr_hugepages) +int check_compaction (unsigned long mem_free,unsigned long hugepage_size) { -unsigned long nr_hugepages_ul; -int fd, ret =-1 ; +unsigned long nr_hugepages;int compaction_index =0 ; -char nr_hugepages[20 ] = {0 }; -char init_nr_hugepages[24 ] = {0 }; -char target_nr_hugepages[24 ] = {0 }; -int slen; - -snprintf (init_nr_hugepages,sizeof (init_nr_hugepages), -"%lu" , initial_nr_hugepages); +int ret =-1 ;/* We want to test with 80% of available memory. Else, OOM killer comes in to play */ mem_free = mem_free *0.8 ; - fd =open ("/proc/sys/vm/nr_hugepages" ,O_RDWR |O_NONBLOCK ); -if (fd <0 ) { -ksft_print_msg ("Failed to open /proc/sys/vm/nr_hugepages: %s\n" , -strerror (errno)); - ret =-1 ; - goto out; - } -/* * Request huge pages for about half of the free memory. The Kernel * will allocate as much as it can, and we expect it will get at least 1/3 */ - nr_hugepages_ul = mem_free / hugepage_size /2 ; -snprintf (target_nr_hugepages,sizeof (target_nr_hugepages), -"%lu" , nr_hugepages_ul); - - slen =strlen (target_nr_hugepages); -if (write (fd, target_nr_hugepages, slen) != slen) { -ksft_print_msg ("Failed to write %lu to /proc/sys/vm/nr_hugepages: %s\n" , - nr_hugepages_ul,strerror (errno)); - goto close_fd; - } - -lseek (fd,0 ,SEEK_SET ); - -if (read (fd, nr_hugepages,sizeof (nr_hugepages)) <=0 ) { -ksft_print_msg ("Failed to re-read from /proc/sys/vm/nr_hugepages: %s\n" , -strerror (errno)); - goto close_fd; - } + nr_hugepages = mem_free / hugepage_size /2 ; +hugetlb_set_nr_default_pages (nr_hugepages);/* We should have been able to request at least 1/3 rd of the memory in huge pages */ - nr_hugepages_ul =strtoul (nr_hugepages,NULL ,10 ); -if (!nr_hugepages_ul) { + nr_hugepages =hugetlb_nr_default_pages (); +if (!nr_hugepages) {ksft_print_msg ("ERROR: No memory is available as huge pages\n" ); goto close_fd; } - compaction_index = mem_free/(nr_hugepages_ul * hugepage_size); - -lseek (fd,0 ,SEEK_SET ); - -if (write (fd, init_nr_hugepages,strlen (init_nr_hugepages)) - !=strlen (init_nr_hugepages)) { -ksft_print_msg ("Failed to write value to /proc/sys/vm/nr_hugepages: %s\n" , -strerror (errno)); - goto close_fd; - } + compaction_index = mem_free/(nr_hugepages * hugepage_size); -ksft_print_msg ("Number of huge pages allocated = %lu\n" , - nr_hugepages_ul); +ksft_print_msg ("Number of huge pages allocated = %lu\n" , nr_hugepages);if (compaction_index >3 ) { ksft_print_msg("ERROR: Less than 1/%d of memory is available\n"
@@ -161,48 +120,10 @@ int check_compaction(unsigned long mem_free, unsigned long hugepage_size, ret =0 ; close_fd: -close (fd); - out:ksft_test_result (ret ==0 ,"check_compaction\n" );return ret; } -int set_zero_hugepages (unsigned long *initial_nr_hugepages) -{ -int fd, ret =-1 ; -char nr_hugepages[20 ] = {0 }; - - fd =open ("/proc/sys/vm/nr_hugepages" ,O_RDWR |O_NONBLOCK ); -if (fd <0 ) { -ksft_print_msg ("Failed to open /proc/sys/vm/nr_hugepages: %s\n" , -strerror (errno)); - goto out; - } -if (read (fd, nr_hugepages,sizeof (nr_hugepages)) <=0 ) { -ksft_print_msg ("Failed to read from /proc/sys/vm/nr_hugepages: %s\n" , -strerror (errno)); - goto close_fd; - } - -lseek (fd,0 ,SEEK_SET ); - -/* Start with the initial condition of 0 huge pages */ -if (write (fd,"0" ,sizeof (char )) !=sizeof (char )) { -ksft_print_msg ("Failed to write 0 to /proc/sys/vm/nr_hugepages: %s\n" , -strerror (errno)); - goto close_fd; - } - - *initial_nr_hugepages =strtoul (nr_hugepages,NULL ,10 ); - ret =0 ; - - close_fd: -close (fd); - - out: -return ret; -} -int main (int argc,char **argv) {struct rlimit lim;
@@ -212,18 +133,19 @@ int main(int argc, char **argv)unsigned long mem_free =0 ;unsigned long hugepage_size =0 ;long mem_fragmentable_MB =0 ; -unsigned long initial_nr_hugepages;ksft_print_header ();if (prereq () ||geteuid ())ksft_exit_skip ("Prerequisites unsatisfied\n" ); -ksft_set_plan (1 ); -/* Start the test without hugepages reducing mem_free */ -if (set_zero_hugepages (&initial_nr_hugepages)) -ksft_exit_fail (); +hugetlb_save_settings (); +hugetlb_set_nr_default_pages (0 ); +if (hugetlb_nr_default_pages ()) +ksft_exit_skip ("Could not reset nr_hugepages\n" ); + +ksft_set_plan (1 ); lim.rlim_cur =RLIM_INFINITY ; lim.rlim_max =RLIM_INFINITY ;
@@ -268,8 +190,7 @@ int main(int argc, char **argv) entry = entry->next; } -if (check_compaction (mem_free, hugepage_size, - initial_nr_hugepages) ==0 ) +if (check_compaction (mem_free, hugepage_size) ==0 )ksft_exit_pass ();ksft_exit_fail ();
Signature · 1 line
2.53.0
Re: [PATCH v3 30/54] selftests/mm: compaction_test: use HugeTLB helpers ...
Hi Mike!
Quoted 17 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
... instead of open coded access of HugeTLB parameters via /proc.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/compaction_test.c | 113 +++---------------- 1 file changed, 17 insertions(+), 96 deletions(-)tools/testing/selftests/mm/compaction_test.c +0
-0@@ -17,6 +17,7 @@#include <string.h>
tools/testing/selftests/mm/compaction_test.c int check_compaction(unsigned long mem_free, unsigned long hugepage_size, · line 122
Quoted 3 lines
tools/testing/selftests/mm/compaction_test.c +0
-0@@ -161,48 +120,10 @@ int check_compaction(unsigned long mem_free, unsigned long hugepage_size, ret =0 ; close_fd:
A small nit, should we rename close_fd: to out: now? After this change the label no longer closes an fd and is the common exit path, so close_fd feels a bit misleading.
Quoted 42 lines
tools/testing/selftests/mm/compaction_test.c +0
-38-close (fd); - out:ksft_test_result (ret ==0 ,"check_compaction\n" );return ret; } -int set_zero_hugepages (unsigned long *initial_nr_hugepages) -{ -int fd, ret =-1 ; -char nr_hugepages[20 ] = {0 }; - - fd =open ("/proc/sys/vm/nr_hugepages" ,O_RDWR |O_NONBLOCK ); -if (fd <0 ) { -ksft_print_msg ("Failed to open /proc/sys/vm/nr_hugepages: %s\n" , -strerror (errno)); - goto out; - } -if (read (fd, nr_hugepages,sizeof (nr_hugepages)) <=0 ) { -ksft_print_msg ("Failed to read from /proc/sys/vm/nr_hugepages: %s\n" , -strerror (errno)); - goto close_fd; - } - -lseek (fd,0 ,SEEK_SET ); - -/* Start with the initial condition of 0 huge pages */ -if (write (fd,"0" ,sizeof (char )) !=sizeof (char )) { -ksft_print_msg ("Failed to write 0 to /proc/sys/vm/nr_hugepages: %s\n" , -strerror (errno)); - goto close_fd; - } - - *initial_nr_hugepages =strtoul (nr_hugepages,NULL ,10 ); - ret =0 ; - - close_fd: -close (fd); - - out: -return ret; -} -
Mike Rapoport
maintainerRe: [PATCH v3 30/54] selftests/mm: compaction_test: use HugeTLB helpers ...
Quoted 19 lines · Sarthak Sharma
Hi Mike!
Quoted 16 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
... instead of open coded access of HugeTLB parameters via /proc.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/compaction_test.c | 113 +++---------------- 1 file changed, 17 insertions(+), 96 deletions(-)tools/testing/selftests/mm/compaction_test.c +0
-0@@ -17,6 +17,7 @@#include <string.h>
Quoted 8 lines
tools/testing/selftests/mm/compaction_test.c int check_compaction(unsigned long mem_free, unsigned long hugepage_size, · line 122
Quoted 4 lines
tools/testing/selftests/mm/compaction_test.c +0
-0@@ -161,48 +120,10 @@ int check_compaction(unsigned long mem_free, unsigned long hugepage_size, ret =0 ; close_fd:A small nit, should we rename
close_fd: to out: now? After this change the label no longer closes an fd and is the common exit path, soclose_fdfeels a bit misleading.
Thanks, will fix.
Quoted 6 lines
Quoted 6 lines
tools/testing/selftests/mm/compaction_test.c +0
-2-close (fd); - out:ksft_test_result (ret ==0 ,"check_compaction\n" );return ret; }
Signature · 2 lines
Sincerely yours, Mike.
Mike Rapoport
maintainerPATCH v3 31/54 selftests/mm: cow: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
cow tests skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/cow.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tools/testing/selftests/mm/cow.c +3 -3
@@ -1881,21 +1881,21 @@ int main(int argc, char **argv)ksft_print_header (); +hugepage_save_settings (true, true); + pagesize =getpagesize (); pmdsize =read_pmd_pagesize ();if (pmdsize) {/* Only if THP is supported. */ thp_read_settings (&default_settings); default_settings.hugepages[sz2ord (pmdsize, pagesize)].enabled =THP_INHERIT ; -thp_save_settings ();thp_push_settings (&default_settings);ksft_print_msg ("[INFO] detected PMD size: %zu KiB\n" , pmdsize /1024 ); nr_thpsizes =detect_thp_sizes (thpsizes,ARRAY_SIZE (thpsizes)); } - nr_hugetlbsizes =detect_hugetlb_page_sizes (hugetlbsizes, -ARRAY_SIZE (hugetlbsizes)); + nr_hugetlbsizes =hugetlb_setup (2 , hugetlbsizes,ARRAY_SIZE (hugetlbsizes)); has_huge_zeropage =detect_huge_zeropage (); ksft_set_plan(ARRAY_SIZE(anon_test_cases) * tests_per_anon_test_case() +
Signature · 1 line
2.53.0
Re: [PATCH v3 31/54] selftests/mm: cow: add setup of HugeTLB pages
Hi Mike!
Quoted 18 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
cow tests skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/cow.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)tools/testing/selftests/mm/cow.c +0
-0
tools/testing/selftests/mm/cow.c int main(int argc, char **argv) · line 1884
Quoted 4 lines
tools/testing/selftests/mm/cow.c +1
-0@@ -1881,21 +1881,21 @@ int main(int argc, char **argv)ksft_print_header (); +hugepage_save_settings (true, true);
Since hugetlb_setup below already calls hugepage_save_settings() internally, perhaps we can change this to hugepage_save_settings(true, false) or simply keep thp_save_settings()? Otherwise we're saving the HugeTLB state twice here and it seems redundant.
Quoted 20 lines
tools/testing/selftests/mm/cow.c +2
-3+ pagesize =getpagesize (); pmdsize =read_pmd_pagesize ();if (pmdsize) {/* Only if THP is supported. */ thp_read_settings (&default_settings); default_settings.hugepages[sz2ord (pmdsize, pagesize)].enabled =THP_INHERIT ; -thp_save_settings ();thp_push_settings (&default_settings);ksft_print_msg ("[INFO] detected PMD size: %zu KiB\n" , pmdsize /1024 ); nr_thpsizes =detect_thp_sizes (thpsizes,ARRAY_SIZE (thpsizes)); } - nr_hugetlbsizes =detect_hugetlb_page_sizes (hugetlbsizes, -ARRAY_SIZE (hugetlbsizes)); + nr_hugetlbsizes =hugetlb_setup (2 , hugetlbsizes,ARRAY_SIZE (hugetlbsizes)); has_huge_zeropage =detect_huge_zeropage (); ksft_set_plan(ARRAY_SIZE(anon_test_cases) * tests_per_anon_test_case() +
Mike Rapoport
maintainerPATCH v3 32/54 selftests/mm: gup_longterm: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
gup_longterm tests skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/gup_longterm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/mm/gup_longterm.c +1 -1
@@ -510,7 +510,7 @@ int main(int argc, char **argv)int i; pagesize =getpagesize (); - nr_hugetlbsizes =detect_hugetlb_page_sizes (hugetlbsizes, + nr_hugetlbsizes =hugetlb_setup (2 , hugetlbsizes,ARRAY_SIZE (hugetlbsizes));ksft_print_header ();
Signature · 1 line
2.53.0
Re: [PATCH v3 32/54] selftests/mm: gup_longterm: add setup of HugeTLB pages
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
gup_longtermtests skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I have run the test before and after applying the patch. Before the patch, many tests were skipped with the message "need more free huge pages". After the patch, these tests passed successfully.
Quoted 17 lines
--- tools/testing/selftests/mm/gup_longterm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/testing/selftests/mm/gup_longterm.c +1
-1@@ -510,7 +510,7 @@ int main(int argc, char **argv)int i; pagesize =getpagesize (); - nr_hugetlbsizes =detect_hugetlb_page_sizes (hugetlbsizes, + nr_hugetlbsizes =hugetlb_setup (2 , hugetlbsizes,ARRAY_SIZE (hugetlbsizes));ksft_print_header ();
Mike Rapoport
maintainerPATCH v3 33/54 selftests/mm: gup_test: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
gup_test fails to run HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/gup_test.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
tools/testing/selftests/mm/gup_test.c +12 -0
@@ -14,6 +14,7 @@#include <mm/gup_test.h> #include "kselftest.h" #include "vm_util.h" +#include "hugepage_settings.h" #define MB (1UL << 20)
@@ -94,6 +95,7 @@ int main(int argc, char **argv)int filed, i, opt, nr_pages =1 , thp =-1 , write =1 , nthreads =1 , ret;int flags =MAP_PRIVATE ;char *file ="/dev/zero" ; +bool hugetlb = false;pthread_t *tid;char *p;
@@ -168,6 +170,7 @@ int main(int argc, char **argv)break ;case 'H' : flags |= (MAP_HUGETLB |MAP_ANONYMOUS ); + hugetlb = true;break ;default :ksft_exit_fail_msg ("Wrong argument\n" );
@@ -199,6 +202,15 @@ int main(int argc, char **argv) }ksft_print_header (); + +if (hugetlb) { +unsigned long hp_size =default_huge_page_size (); + + size = (size + hp_size -1 ) & ~(hp_size -1 ); +if (!hugetlb_setup_default (size / hp_size)) +ksft_exit_skip ("Not enough huge pages\n" ); + } +ksft_set_plan (nthreads); filed =open (file,O_RDWR |O_CREAT ,0664 );
Signature · 1 line
2.53.0
Re: [PATCH v3 33/54] selftests/mm: gup_test: add setup of HugeTLB pages
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
gup_testfails to run HugeTLB tests if there are no free huge pages prepared by a wrapper script.Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I ran the test with the -H flag before and after applying the patch. Before the patch, the test bailed out with "mmap: Cannot allocate memory". After the patch, it automatically detects and allocates hugepages and the test passes successfully.
Quoted 48 lines
--- tools/testing/selftests/mm/gup_test.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)tools/testing/selftests/mm/gup_test.c +12
-0@@ -14,6 +14,7 @@#include <mm/gup_test.h> #include "kselftest.h" #include "vm_util.h" +#include "hugepage_settings.h" #define MB (1UL << 20)@@ -94,6 +95,7 @@ int main(int argc, char **argv)int filed, i, opt, nr_pages =1 , thp =-1 , write =1 , nthreads =1 , ret;int flags =MAP_PRIVATE ;char *file ="/dev/zero" ; +bool hugetlb = false;pthread_t *tid;char *p;@@ -168,6 +170,7 @@ int main(int argc, char **argv)break ;case 'H' : flags |= (MAP_HUGETLB |MAP_ANONYMOUS ); + hugetlb = true;break ;default :ksft_exit_fail_msg ("Wrong argument\n" );@@ -199,6 +202,15 @@ int main(int argc, char **argv) }ksft_print_header (); + +if (hugetlb) { +unsigned long hp_size =default_huge_page_size (); + + size = (size + hp_size -1 ) & ~(hp_size -1 ); +if (!hugetlb_setup_default (size / hp_size)) +ksft_exit_skip ("Not enough huge pages\n" ); + } +ksft_set_plan (nthreads); filed =open (file,O_RDWR |O_CREAT ,0664 );
Mike Rapoport
maintainerPATCH v3 34/54 selftests/mm: hmm-tests: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hmm-tests skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Since kselftest_harness runs fixture setup and the tests in child processes, use HUGETLB_SETUP_DEFAULT_PAGES() that defines a constructor that runs in the main process and add verification that there are enough free huge pages to the tests that use them.
Replace exit() calls with _exit() to avoid restoring HugeTLB settings in the middle of test and use SIGLILL to kill a child process in hmm_cow_in_device test to avoid interference with signal handlers in hugepage_restore_settings().
--- tools/testing/selftests/mm/hmm-tests.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)
tools/testing/selftests/mm/hmm-tests.c +13 -9
@@ -69,6 +69,9 @@ enum {#ifndef FOLL_LONGTERM #define FOLL_LONGTERM 0x100/* mapping lifetime is indefinite */ #endif + +HUGETLB_SETUP_DEFAULT_PAGES(1) +FIXTURE (hmm) {int fd;
@@ -632,7 +635,7 @@ TEST_F(hmm, anon_write_child) }close (child_fd); -exit (0 ); +_exit (0 ); } } }
@@ -712,7 +715,7 @@ TEST_F(hmm, anon_write_child_shared)ASSERT_EQ (ptr[i], -i);close (child_fd); -exit (0 ); +_exit (0 ); }/*
@@ -784,8 +787,9 @@ TEST_F(hmm, anon_write_hugetlbfs)int *ptr;int ret; -if (!default_hsize) -SKIP (return,"Huge page size could not be determined" ); +if (!hugetlb_free_default_pages ()) +SKIP (return,"Not enough huge pages" ); + default_hsize =default_huge_page_size (); size =ALIGN (TWOMEG , default_hsize); npages = size >> self->page_shift;
@@ -1564,9 +1568,9 @@ TEST_F(hmm, compound)unsigned long i;/* Skip test if we can't allocate a hugetlbfs page. */ - -if (!default_hsize) -SKIP (return,"Huge page size could not be determined" ); +if (!hugetlb_free_default_pages ()) +SKIP (return,"Not enough huge pages" ); + default_hsize =default_huge_page_size (); size =ALIGN (TWOMEG , default_hsize); npages = size >> self->page_shift;
@@ -2025,10 +2029,10 @@ TEST_F(hmm, hmm_cow_in_device) ptr[i] = i;/* Terminate child and wait */ -EXPECT_EQ (0 ,kill (pid,SIGTERM )); +EXPECT_EQ (0 ,kill (pid,SIGKILL ));EXPECT_EQ (pid,waitpid (pid, &status,0 ));EXPECT_NE (0 ,WIFSIGNALED (status)); -EXPECT_EQ (SIGTERM ,WTERMSIG (status)); +EXPECT_EQ (SIGKILL ,WTERMSIG (status));/* Take snapshot to CPU pagetables */ ret =hmm_dmirror_cmd (self->fd,HMM_DMIRROR_SNAPSHOT , buffer, npages);
Signature · 1 line
2.53.0
Re: [PATCH v3 34/54] selftests/mm: hmm-tests: add setup of HugeTLB pages
Hi Mike!
Quoted 56 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hmm-tests skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Since
kselftest_harnessruns fixture setup and the tests in child processes, useHUGETLB_SETUP_DEFAULT_PAGES()that defines a constructor that runs in the main process and add verification that there are enough free huge pages to the tests that use them.Replace
exit()calls with_exit()to avoid restoring HugeTLB settings in the middle of test and use SIGLILL to kill a child process inhmm_cow_in_devicetest to avoid interference with signal handlers inhugepage_restore_settings().Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/hmm-tests.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)tools/testing/selftests/mm/hmm-tests.c +5
-2@@ -69,6 +69,9 @@ enum {#ifndef FOLL_LONGTERM #define FOLL_LONGTERM 0x100/* mapping lifetime is indefinite */ #endif + +HUGETLB_SETUP_DEFAULT_PAGES(1) +FIXTURE (hmm) {int fd;@@ -632,7 +635,7 @@ TEST_F(hmm, anon_write_child) }close (child_fd); -exit (0 ); +_exit (0 ); } } }@@ -712,7 +715,7 @@ TEST_F(hmm, anon_write_child_shared)ASSERT_EQ (ptr[i], -i);close (child_fd); -exit (0 ); +_exit (0 ); }/*
tools/testing/selftests/mm/hmm-tests.c TEST_F(hmm, anon_write_hugetlbfs) · line 792
Quoted 8 lines
tools/testing/selftests/mm/hmm-tests.c +3
-2@@ -784,8 +787,9 @@ TEST_F(hmm, anon_write_hugetlbfs)int *ptr;int ret; -if (!default_hsize) -SKIP (return,"Huge page size could not be determined" ); +if (!hugetlb_free_default_pages ()) +SKIP (return,"Not enough huge pages" ); + default_hsize =default_huge_page_size ();
default_hsize has already been initialized at declaration above, so this reassignment seems redundant. Shall we drop this?
tools/testing/selftests/mm/hmm-tests.c TEST_F(hmm, compound) · line 1573
Quoted 13 lines
tools/testing/selftests/mm/hmm-tests.c +3
-3size =ALIGN (TWOMEG , default_hsize); npages = size >> self->page_shift;@@ -1564,9 +1568,9 @@ TEST_F(hmm, compound)unsigned long i;/* Skip test if we can't allocate a hugetlbfs page. */ - -if (!default_hsize) -SKIP (return,"Huge page size could not be determined" ); +if (!hugetlb_free_default_pages ()) +SKIP (return,"Not enough huge pages" ); + default_hsize =default_huge_page_size ();
Same thing here as well.
Quoted 16 lines
tools/testing/selftests/mm/hmm-tests.c +2
-2size =ALIGN (TWOMEG , default_hsize); npages = size >> self->page_shift;@@ -2025,10 +2029,10 @@ TEST_F(hmm, hmm_cow_in_device) ptr[i] = i;/* Terminate child and wait */ -EXPECT_EQ (0 ,kill (pid,SIGTERM )); +EXPECT_EQ (0 ,kill (pid,SIGKILL ));EXPECT_EQ (pid,waitpid (pid, &status,0 ));EXPECT_NE (0 ,WIFSIGNALED (status)); -EXPECT_EQ (SIGTERM ,WTERMSIG (status)); +EXPECT_EQ (SIGKILL ,WTERMSIG (status));/* Take snapshot to CPU pagetables */ ret =hmm_dmirror_cmd (self->fd,HMM_DMIRROR_SNAPSHOT , buffer, npages);
Mike Rapoport
maintainerPATCH v3 35/54 selftests/mm: hugepage_dio: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hugepage_dio test fails if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/hugetlb_dio.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-)
tools/testing/selftests/mm/hugetlb_dio.c +2 -8
@@ -85,8 +85,6 @@ static void run_dio_using_hugetlb(int fd, unsigned int start_off,/* Get the default huge page size */ h_pagesize =default_huge_page_size (); -if (!h_pagesize) -ksft_exit_fail_msg ("Unable to determine huge page size\n" );/* Reset file position since fd is shared across tests */ if (lseek (fd,0 ,SEEK_SET ) <0 )
@@ -94,10 +92,6 @@ static void run_dio_using_hugetlb(int fd, unsigned int start_off,/* Get the free huge pages before allocation */ free_hpage_b =hugetlb_free_default_pages (); -if (free_hpage_b ==0 ) { -close (fd); -ksft_exit_skip ("No free hugepage, exiting!\n" ); - }/* Allocate a hugetlb page */ orig_buffer =mmap (NULL , h_pagesize, mmap_prot, mmap_flags,-1 ,0 );
@@ -141,8 +135,8 @@ int main(void)ksft_print_header (); -/* Check if huge pages are free */ -if (!hugetlb_free_default_pages ()) +/* request a huge page */ +if (!hugetlb_setup_default (1 ))ksft_exit_skip ("No free hugepage, exiting\n" ); fd =open ("/tmp" ,O_TMPFILE |O_RDWR |O_DIRECT ,0664 );
Signature · 1 line
2.53.0
Re: [PATCH v3 35/54] selftests/mm: hugepage_dio: add setup of HugeTLB pages
Hi Mike!
Quoted 38 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugepage_diotest fails if there are no free huge pages prepared by a wrapper script.Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/hugetlb_dio.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-)tools/testing/selftests/mm/hugetlb_dio.c +0
-6@@ -85,8 +85,6 @@ static void run_dio_using_hugetlb(int fd, unsigned int start_off,/* Get the default huge page size */ h_pagesize =default_huge_page_size (); -if (!h_pagesize) -ksft_exit_fail_msg ("Unable to determine huge page size\n" );/* Reset file position since fd is shared across tests */ if (lseek (fd,0 ,SEEK_SET ) <0 )@@ -94,10 +92,6 @@ static void run_dio_using_hugetlb(int fd, unsigned int start_off,/* Get the free huge pages before allocation */ free_hpage_b =hugetlb_free_default_pages (); -if (free_hpage_b ==0 ) { -close (fd); -ksft_exit_skip ("No free hugepage, exiting!\n" ); - }/* Allocate a hugetlb page */ orig_buffer =mmap (NULL , h_pagesize, mmap_prot, mmap_flags,-1 ,0 );
tools/testing/selftests/mm/hugetlb_dio.c int main(void) · line 140
Quoted 8 lines
tools/testing/selftests/mm/hugetlb_dio.c +2
-2@@ -141,8 +135,8 @@ int main(void)ksft_print_header (); -/* Check if huge pages are free */ -if (!hugetlb_free_default_pages ()) +/* request a huge page */ +if (!hugetlb_setup_default (1 ))ksft_exit_skip ("No free hugepage, exiting\n" );
Before applying the patch, I get the following output:
TAP version 13
1..4
# No. Free pages before allocation : 112
# No. Free pages after munmap : 112
ok 1 free huge pages from 0-12288
# No. Free pages before allocation : 112
# No. Free pages after munmap : 112
ok 2 free huge pages from 0-10240
# No. Free pages before allocation : 112
# No. Free pages after munmap : 112
ok 3 free huge pages from 2048-12288
# No. Free pages before allocation : 112
# No. Free pages after munmap : 112
ok 4 free huge pages from 2048-14336
# Totals: pass:4 fail:0 xfail:0 xpass:0 skip:0 error:0
After applying the patch, I get this:
TAP version 13
# [INFO] detected hugetlb page size: 2048 KiB
# [INFO] detected hugetlb page size: 32768 KiB
# [INFO] detected hugetlb page size: 64 KiB
# [INFO] detected hugetlb page size: 1048576 KiB
1..0 # SKIP No free hugepage, exiting
even when I have 112 free and 128 total hugepages of size 2MB (default for my system).
Could it be because hugetlb_setup_default(1) here is trying to set the default hugetlb pool size to exactly 1, and then checking whether there is exactly 1 free hugepage, instead of simply checking whether at least 1 free hugepage is available?
Quoted 2 lines
tools/testing/selftests/mm/hugetlb_dio.c +0
-0fd =open ("/tmp" ,O_TMPFILE |O_RDWR |O_DIRECT ,0664 );
Mike Rapoport
maintainerRe: [PATCH v3 35/54] selftests/mm: hugepage_dio: add setup of HugeTLB pages
Hi Sarthak,
Quoted 38 lines · Sarthak Sharma
tools/testing/selftests/mm/hugetlb_dio.c int main(void) · line 140
Quoted 9 lines · Mike Rapoport
tools/testing/selftests/mm/hugetlb_dio.c +2
-2@@ -141,8 +135,8 @@ int main(void)ksft_print_header (); -/* Check if huge pages are free */ -if (!hugetlb_free_default_pages ()) +/* request a huge page */ +if (!hugetlb_setup_default (1 ))ksft_exit_skip ("No free hugepage, exiting\n" );Before applying the patch, I get the following output:
TAP version 13
1..4
# No. Free pages before allocation : 112
# No. Free pages after munmap : 112
ok 1 free huge pages from 0-12288
# No. Free pages before allocation : 112
# No. Free pages after munmap : 112
ok 2 free huge pages from 0-10240
# No. Free pages before allocation : 112
# No. Free pages after munmap : 112
ok 3 free huge pages from 2048-12288
# No. Free pages before allocation : 112
# No. Free pages after munmap : 112
ok 4 free huge pages from 2048-14336
# Totals: pass:4 fail:0 xfail:0 xpass:0 skip:0 error:0After applying the patch, I get this:
TAP version 13
# [INFO] detected hugetlb page size: 2048 KiB
# [INFO] detected hugetlb page size: 32768 KiB
# [INFO] detected hugetlb page size: 64 KiB
# [INFO] detected hugetlb page size: 1048576 KiB
1..0 # SKIP No free hugepage, exiting
Quoted 7 lines
even when I have 112 free and 128 total hugepages of size 2MB (default for my system).
Could it be because
hugetlb_setup_default(1)here is trying to set the default hugetlb pool size to exactly 1, and then checking whether there is exactly 1 free hugepage, instead of simply checking whether at least 1 free hugepage is available?
I'm going to update hugetlb_setup_default() to check if there are enough free huge pages and add more if necessary and I will add hugetlb_setup_default_exact() for tests that require exact number of huge pages.
Quoted 3 lines
Quoted 2 lines
tools/testing/selftests/mm/hugetlb_dio.c +0
-0fd =open ("/tmp" ,O_TMPFILE |O_RDWR |O_DIRECT ,0664 );
Signature · 2 lines
Sincerely yours, Mike.
Mike Rapoport
maintainerPATCH v3 36/54 selftests/mm: hugetlb_fault_after_madv: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb_fault_after_madv test skips testing if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/hugetlb_fault_after_madv.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
tools/testing/selftests/mm/hugetlb_fault_after_madv.c +2 -4
@@ -54,7 +54,6 @@ void *madv(void *unused)int main (void ) { -unsigned long free_hugepages;pthread_t thread1, thread2;/* * On kernel 6.4, we are able to reproduce the problem with ~1000
@@ -78,10 +77,9 @@ int main(void)ksft_print_msg ("[INFO] detected default hugetlb page size: %zu KiB\n" , huge_page_size /1024 ); - free_hugepages =hugetlb_free_default_pages (); -if (free_hugepages !=1 ) { +if (!hugetlb_setup_default (1 )) {ksft_exit_skip ("This test needs one and only one page to execute. Got %lu\n" , - free_hugepages); +hugetlb_free_default_pages ()); }while (max--) {
Signature · 1 line
2.53.0
Re: [PATCH v3 36/54] selftests/mm: hugetlb_fault_after_madv: add setup of HugeTLB pages
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb_fault_after_madvtest skips testing if there are no free huge pages prepared by a wrapper script.Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I have run the test before and after applying the patch. Before the patch, the test skipped if there was not exactly 1 free hugepage available. After the patch, it correctly allocates one hugepage and the test passes successfully, restoring the original hugepage settings on exit.
Quoted 29 lines
--- tools/testing/selftests/mm/hugetlb_fault_after_madv.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)tools/testing/selftests/mm/hugetlb_fault_after_madv.c +2
-4@@ -54,7 +54,6 @@ void *madv(void *unused)int main (void ) { -unsigned long free_hugepages;pthread_t thread1, thread2;/* * On kernel 6.4, we are able to reproduce the problem with ~1000 @@ -78,10 +77,9 @@ int main(void)ksft_print_msg ("[INFO] detected default hugetlb page size: %zu KiB\n" , huge_page_size /1024 ); - free_hugepages =hugetlb_free_default_pages (); -if (free_hugepages !=1 ) { +if (!hugetlb_setup_default (1 )) {ksft_exit_skip ("This test needs one and only one page to execute. Got %lu\n" , - free_hugepages); +hugetlb_free_default_pages ()); }while (max--) {
Mike Rapoport
maintainerPATCH v3 37/54 selftests/mm: hugetlb-madvise: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-madvise test skips testing if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/hugetlb-madvise.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/mm/hugetlb-madvise.c +2 -2
@@ -68,9 +68,9 @@ int main(int argc, char **argv)if (!base_page_size)ksft_exit_fail_msg ("Unable to determine base page size\n" ); +if (!hugetlb_setup_default (MIN_FREE_PAGES )) +ksft_exit_skip ("Not enough free huge pages (have %lu, need %d)\n" ,hugetlb_free_default_pages (),MIN_FREE_PAGES ); free_hugepages =hugetlb_free_default_pages (); -if (free_hugepages <MIN_FREE_PAGES ) -ksft_exit_skip ("Not enough free huge pages (have %lu, need %d)\n" , free_hugepages,MIN_FREE_PAGES ); fd =memfd_create (argv[0 ],MFD_HUGETLB );if (fd <0 )
Signature · 1 line
2.53.0
Re: [PATCH v3 37/54] selftests/mm: hugetlb-madvise: add setup of HugeTLB pages
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-madvise test skips testing if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I have run this test before and after applying the patch. Before the patch, the test skipped if the required hugepages were not allocated. After the patch, it correctly allocates the required hugepages and passes successfully, restoring the original hugepage settings at exit.
Quoted 20 lines
--- tools/testing/selftests/mm/hugetlb-madvise.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/testing/selftests/mm/hugetlb-madvise.c +2
-2@@ -68,9 +68,9 @@ int main(int argc, char **argv)if (!base_page_size)ksft_exit_fail_msg ("Unable to determine base page size\n" ); +if (!hugetlb_setup_default (MIN_FREE_PAGES )) +ksft_exit_skip ("Not enough free huge pages (have %lu, need %d)\n" ,hugetlb_free_default_pages (),MIN_FREE_PAGES ); free_hugepages =hugetlb_free_default_pages (); -if (free_hugepages <MIN_FREE_PAGES ) -ksft_exit_skip ("Not enough free huge pages (have %lu, need %d)\n" , free_hugepages,MIN_FREE_PAGES ); fd =memfd_create (argv[0 ],MFD_HUGETLB );if (fd <0 )
Mike Rapoport
maintainerPATCH v3 38/54 selftests/mm: hugetlb_madv_vs_map: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb_madv_vs_map test skips testing if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/hugetlb_madv_vs_map.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
tools/testing/selftests/mm/hugetlb_madv_vs_map.c +2 -5
@@ -77,7 +77,6 @@ void *map_extra(void *unused)int main (void ) {pthread_t thread1, thread2, thread3; -unsigned long free_hugepages;void *ret;/*
@@ -89,11 +88,9 @@ int main(void)ksft_print_header ();ksft_set_plan (1 ); - free_hugepages =hugetlb_free_default_pages (); - -if (free_hugepages !=1 ) +if (!hugetlb_setup_default (1 ))ksft_exit_skip ("This test needs one and only one page to execute. Got %lu\n" , - free_hugepages); +hugetlb_free_default_pages ()); mmap_size =default_huge_page_size ();
Signature · 1 line
2.53.0
Re: [PATCH v3 38/54] selftests/mm: hugetlb_madv_vs_map: add setup of HugeTLB pages
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb_madv_vs_maptest skips testing if there are no free huge pages prepared by a wrapper script.Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I have run the test before and after applying the patch. Before the patch, the test skipped if there was not exactly 1 free hugepage available. After the patch, it correctly allocates one hugepage and the test passes successfully, restoring the original hugepage settings on exit.
Quoted 30 lines
--- tools/testing/selftests/mm/hugetlb_madv_vs_map.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)tools/testing/selftests/mm/hugetlb_madv_vs_map.c +2
-5@@ -77,7 +77,6 @@ void *map_extra(void *unused)int main (void ) {pthread_t thread1, thread2, thread3; -unsigned long free_hugepages;void *ret;/* @@ -89,11 +88,9 @@ int main(void)ksft_print_header ();ksft_set_plan (1 ); - free_hugepages =hugetlb_free_default_pages (); - -if (free_hugepages !=1 ) +if (!hugetlb_setup_default (1 ))ksft_exit_skip ("This test needs one and only one page to execute. Got %lu\n" , - free_hugepages); +hugetlb_free_default_pages ()); mmap_size =default_huge_page_size ();
Mike Rapoport
maintainerPATCH v3 39/54 selftests/mm: hugetlb-mmap: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-mmap test fails if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/hugetlb-mmap.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
tools/testing/selftests/mm/hugetlb-mmap.c +9 -4
@@ -109,16 +109,16 @@ int main(int argc, char **argv) {size_t hugepage_size;size_t length =LENGTH ; -int shift =0 ; +int shift =0 , nr;ksft_print_header (); -ksft_set_plan (2 );if (argc >1 ) length =atol (argv[1 ]) <<20 ;if (argc >2 ) shift =atoi (argv[2 ]); +hugetlb_save_settings ();if (shift) { hugepage_size = (1 << shift);ksft_print_msg ("%u kB hugepages\n" ,1 << (shift -10 ));
@@ -128,9 +128,14 @@ int main(int argc, char **argv) }/* munmap with fail if the length is not page aligned */ -if (hugepage_size > length) - length = hugepage_size; + length = (length + hugepage_size -1 ) & ~(hugepage_size -1 ); + nr = length / hugepage_size; + +hugetlb_set_nr_pages (hugepage_size, nr); +if (hugetlb_free_pages (hugepage_size) < nr) +ksft_exit_skip ("Not enough %lu Kb pages\n" , hugepage_size >>10 ); +ksft_set_plan (2 );ksft_print_msg ("Mapping %lu Mbytes\n" , (unsigned long )length >>20 );test_anon_mmap (length, shift);
Signature · 1 line
2.53.0
Re: [PATCH v3 39/54] selftests/mm: hugetlb-mmap: add setup of HugeTLB pages
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-mmap test fails if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Quoted 44 lines
--- tools/testing/selftests/mm/hugetlb-mmap.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)tools/testing/selftests/mm/hugetlb-mmap.c +9
-4@@ -109,16 +109,16 @@ int main(int argc, char **argv) {size_t hugepage_size;size_t length =LENGTH ; -int shift =0 ; +int shift =0 , nr;ksft_print_header (); -ksft_set_plan (2 );if (argc >1 ) length =atol (argv[1 ]) <<20 ;if (argc >2 ) shift =atoi (argv[2 ]); +hugetlb_save_settings ();if (shift) { hugepage_size = (1 << shift);ksft_print_msg ("%u kB hugepages\n" ,1 << (shift -10 ));@@ -128,9 +128,14 @@ int main(int argc, char **argv) }/* munmap with fail if the length is not page aligned */ -if (hugepage_size > length) - length = hugepage_size; + length = (length + hugepage_size -1 ) & ~(hugepage_size -1 ); + nr = length / hugepage_size; + +hugetlb_set_nr_pages (hugepage_size, nr); +if (hugetlb_free_pages (hugepage_size) < nr) +ksft_exit_skip ("Not enough %lu Kb pages\n" , hugepage_size >>10 ); +ksft_set_plan (2 );ksft_print_msg ("Mapping %lu Mbytes\n" , (unsigned long )length >>20 );test_anon_mmap (length, shift);
Re: [PATCH v3 39/54] selftests/mm: hugetlb-mmap: add setup of HugeTLB pages
In reply to Mike RapoportQuoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-mmap test fails if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I have run the test before and after applying the patch. Before the patch, the test bailed out with mmap error if hugepages were not preallocated. After the patch, the test correctly allocates the required hugepages and successfully passes.
Quoted 44 lines
--- tools/testing/selftests/mm/hugetlb-mmap.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)tools/testing/selftests/mm/hugetlb-mmap.c +9
-4@@ -109,16 +109,16 @@ int main(int argc, char **argv) {size_t hugepage_size;size_t length =LENGTH ; -int shift =0 ; +int shift =0 , nr;ksft_print_header (); -ksft_set_plan (2 );if (argc >1 ) length =atol (argv[1 ]) <<20 ;if (argc >2 ) shift =atoi (argv[2 ]); +hugetlb_save_settings ();if (shift) { hugepage_size = (1 << shift);ksft_print_msg ("%u kB hugepages\n" ,1 << (shift -10 ));@@ -128,9 +128,14 @@ int main(int argc, char **argv) }/* munmap with fail if the length is not page aligned */ -if (hugepage_size > length) - length = hugepage_size; + length = (length + hugepage_size -1 ) & ~(hugepage_size -1 ); + nr = length / hugepage_size; + +hugetlb_set_nr_pages (hugepage_size, nr); +if (hugetlb_free_pages (hugepage_size) < nr) +ksft_exit_skip ("Not enough %lu Kb pages\n" , hugepage_size >>10 ); +ksft_set_plan (2 );ksft_print_msg ("Mapping %lu Mbytes\n" , (unsigned long )length >>20 );test_anon_mmap (length, shift);
Mike Rapoport
maintainerPATCH v3 40/54 selftests/mm: hugetlb-mremap: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-mremap test fails if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/hugetlb-mremap.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
tools/testing/selftests/mm/hugetlb-mremap.c +10 -1
@@ -26,6 +26,7 @@#include <stdbool.h> #include "kselftest.h" #include "vm_util.h" +#include "hugepage_settings.h" #define DEFAULT_LENGTH_MB 10UL#define MB_TO_BYTES (x) (x * 1024 * 1024)
@@ -108,8 +109,9 @@ static void register_region_with_uffd(char *addr, size_t len)int main (int argc,char *argv[]) { +unsigned long hugepage_size; +int ret =0 , fd, nr;size_t length =0 ; -int ret =0 , fd;ksft_print_header ();ksft_set_plan (1 );
@@ -125,7 +127,14 @@ int main(int argc, char *argv[])else length =DEFAULT_LENGTH_MB ; + hugepage_size =default_huge_page_size (); length =MB_TO_BYTES (length); + length = (length + hugepage_size) & ~(hugepage_size -1 ); + nr = length / hugepage_size; + +if (!hugetlb_setup_default (nr)) +ksft_exit_skip ("Not enough huge pages\n" ); + fd =memfd_create (argv[0 ],MFD_HUGETLB );if (fd <0 )ksft_exit_fail_msg ("Open failed: %s\n" ,strerror (errno));
Signature · 1 line
2.53.0
Re: [PATCH v3 40/54] selftests/mm: hugetlb-mremap: add setup of HugeTLB pages
Hi Mike!
Quoted 37 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-mremap test fails if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/hugetlb-mremap.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)tools/testing/selftests/mm/hugetlb-mremap.c +3
-1@@ -26,6 +26,7 @@#include <stdbool.h> #include "kselftest.h" #include "vm_util.h" +#include "hugepage_settings.h" #define DEFAULT_LENGTH_MB 10UL#define MB_TO_BYTES (x) (x * 1024 * 1024)@@ -108,8 +109,9 @@ static void register_region_with_uffd(char *addr, size_t len)int main (int argc,char *argv[]) { +unsigned long hugepage_size; +int ret =0 , fd, nr;size_t length =0 ; -int ret =0 , fd;ksft_print_header ();ksft_set_plan (1 );
tools/testing/selftests/mm/hugetlb-mremap.c int main(int argc, char *argv[]) · line 132
Quoted 6 lines
tools/testing/selftests/mm/hugetlb-mremap.c +2
-0@@ -125,7 +127,14 @@ int main(int argc, char *argv[])else length =DEFAULT_LENGTH_MB ; + hugepage_size =default_huge_page_size (); length =MB_TO_BYTES (length); + length = (length + hugepage_size) & ~(hugepage_size -1 );
I might be missing something, but if we're trying to align length to the hugepage size, should this be:
length = (length + hugepage_size - 1) & ~(hugepage_size - 1);
like in hugetlb-mmap.c? Because the current code will add an extra hugepage if the length is already aligned.
Quoted 8 lines
tools/testing/selftests/mm/hugetlb-mremap.c +5
-0+ nr = length / hugepage_size; + +if (!hugetlb_setup_default (nr)) +ksft_exit_skip ("Not enough huge pages\n" ); + fd =memfd_create (argv[0 ],MFD_HUGETLB );if (fd <0 )ksft_exit_fail_msg ("Open failed: %s\n" ,strerror (errno));
Mike Rapoport
maintainerPATCH v3 41/54 selftests/mm: hugetlb-shm: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-shm test fails if there are no free huge pages prepared by a wrapper script and shm liimts in proc are too low.
Add setup of HugeTLB pages and shm limits to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/hugetlb-shm.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
tools/testing/selftests/mm/hugetlb-shm.c +19 -0
@@ -29,9 +29,24 @@#include <sys/mman.h> #include "vm_util.h" +#include "hugepage_settings.h" #define LENGTH (256UL*1024*1024) +static void prepare (void ) +{ +unsigned long length, hugepage_size, nr; + + hugepage_size =default_huge_page_size (); + length = (LENGTH + hugepage_size) & ~(hugepage_size -1 ); + nr = length / hugepage_size; + +if (!hugetlb_setup_default (nr)) +ksft_exit_skip ("Not enough free huge pages\n" ); + +shm_limits_prepare (length); +} +int main (void ) {int shmid;
@@ -41,6 +56,8 @@ int main(void)ksft_print_header ();ksft_set_plan (1 ); +prepare (); + shmid =shmget (2 ,LENGTH ,SHM_HUGETLB |IPC_CREAT |SHM_R |SHM_W );if (shmid <0 )ksft_exit_fail_perror ("shmget" );
@@ -78,3 +95,5 @@ int main(void)ksft_test_result_pass ("hugepage using SysV shmget/shmat\n" );ksft_finished (); } + +SHM_LIMITS_RESTORE ()
Signature · 1 line
2.53.0
Re: [PATCH v3 41/54] selftests/mm: hugetlb-shm: add setup of HugeTLB pages
Hi Mike!
Quoted 21 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-shm test fails if there are no free huge pages prepared by a wrapper script and shm liimts in proc are too low.
Add setup of HugeTLB pages and shm limits to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/hugetlb-shm.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)tools/testing/selftests/mm/hugetlb-shm.c +0
-0@@ -29,9 +29,24 @@#include <sys/mman.h> #include "vm_util.h"
tools/testing/selftests/mm/hugetlb-shm.c line 41
Quoted 10 lines
tools/testing/selftests/mm/hugetlb-shm.c +7
-0+#include "hugepage_settings.h" #define LENGTH (256UL*1024*1024) +static void prepare (void ) +{ +unsigned long length, hugepage_size, nr; + + hugepage_size =default_huge_page_size (); + length = (LENGTH + hugepage_size) & ~(hugepage_size -1 );
Same issue here as in hugetlb-mremap.c.
length = (LENGTH + hugepage_size) & ~(hugepage_size - 1)
will add an extra hugepage if LENGTH is already aligned. Should this be (LENGTH + hugepage_size - 1) & ~(hugepage_size - 1) instead?
Quoted 26 lines
tools/testing/selftests/mm/hugetlb-shm.c +12
-0+ nr = length / hugepage_size; + +if (!hugetlb_setup_default (nr)) +ksft_exit_skip ("Not enough free huge pages\n" ); + +shm_limits_prepare (length); +} +int main (void ) {int shmid;@@ -41,6 +56,8 @@ int main(void)ksft_print_header ();ksft_set_plan (1 ); +prepare (); + shmid =shmget (2 ,LENGTH ,SHM_HUGETLB |IPC_CREAT |SHM_R |SHM_W );if (shmid <0 )ksft_exit_fail_perror ("shmget" );@@ -78,3 +95,5 @@ int main(void)ksft_test_result_pass ("hugepage using SysV shmget/shmat\n" );ksft_finished (); } + +SHM_LIMITS_RESTORE ()
Mike Rapoport
maintainerPATCH v3 42/54 selftests/mm: hugetlb-soft-online: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-soft-online test uses open coded access to /proc to determine availability of huge pages and fails if there are no enough free huget pages..
Replace open coded access to /proc with hugepage helpers and add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- .../selftests/mm/hugetlb-soft-offline.c | 45 ++++--------------- 1 file changed, 8 insertions(+), 37 deletions(-)
tools/testing/selftests/mm/hugetlb-soft-offline.c +8 -37
@@ -6,9 +6,7 @@* - if enable_soft_offline = 1, a hugepage should be dissolved and * nr_hugepages/free_hugepages should be reduced by 1. * -* Before running, make sure more than 2 hugepages of default_hugepagesz -* are allocated. For example, if /proc/meminfo/Hugepagesize is 2048kB: -* echo 8 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages +* The test allocates 8 default hugepages */ #define _GNU_SOURCE
@@ -25,6 +23,7 @@#include <sys/types.h> #include "kselftest.h" +#include "hugepage_settings.h" #ifndef MADV_SOFT_OFFLINE #define MADV_SOFT_OFFLINE 101
@@ -100,32 +99,6 @@ static int set_enable_soft_offline(int value)return 0 ; } -static int read_nr_hugepages (unsigned long hugepage_size, -unsigned long *nr_hugepages) -{ -char buffer[256 ] = {0 }; -char cmd[256 ] = {0 }; - -sprintf (cmd,"cat /sys/kernel/mm/hugepages/hugepages-%ldkB/nr_hugepages" , - hugepage_size); -FILE *cmdfile =popen (cmd,"r" ); - -if (cmdfile ==NULL ) { -ksft_perror (EPREFIX "failed to popen nr_hugepages" ); -return -1 ; - } - -if (!fgets (buffer,sizeof (buffer), cmdfile)) { -ksft_perror (EPREFIX "failed to read nr_hugepages" ); -pclose (cmdfile); -return -1 ; - } - - *nr_hugepages =atoll (buffer); -pclose (cmdfile); -return 0 ; -} -static int create_hugetlbfs_file (struct statfs *file_stat) {int fd;
@@ -177,20 +150,14 @@ static void test_soft_offline_common(int enable_soft_offline)ksft_exit_fail_msg ("Failed to set enable_soft_offline\n" ); } -if (read_nr_hugepages (hugepagesize_kb, &nr_hugepages_before) !=0 ) { -close (fd); -ksft_exit_fail_msg ("Failed to read nr_hugepages\n" ); - } + nr_hugepages_before =hugetlb_nr_default_pages ();ksft_print_msg ("Before MADV_SOFT_OFFLINE nr_hugepages=%ld\n" , nr_hugepages_before); ret =do_soft_offline (fd,2 * file_stat.f_bsize, expect_errno); -if (read_nr_hugepages (hugepagesize_kb, &nr_hugepages_after) !=0 ) { -close (fd); -ksft_exit_fail_msg ("Failed to read nr_hugepages\n" ); - } + nr_hugepages_after =hugetlb_nr_default_pages ();ksft_print_msg ("After MADV_SOFT_OFFLINE nr_hugepages=%ld\n" , nr_hugepages_after);
@@ -219,6 +186,10 @@ static void test_soft_offline_common(int enable_soft_offline)int main (int argc,char **argv) {ksft_print_header (); + +if (!hugetlb_setup_default (8 )) +ksft_exit_skip ("not enough hugetlb pages\n" ); +ksft_set_plan (2 );test_soft_offline_common (1 );
Signature · 1 line
2.53.0
Re: [PATCH v3 42/54] selftests/mm: hugetlb-soft-online: add setup of HugeTLB pages
Hi Mike!
Quoted 11 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-soft-online test uses open coded access to /proc to determine availability of huge pages and fails if there are no enough free huget pages..
Replace open coded access to /proc with hugepage helpers and add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
The test is hugetlb-soft-offline, so let's change the commit title and commit message to say "offine" instead of "online".
I have run the test before and after applying the patch. Before the patch, it failed with an mmap error when hugepages were not preallocated. After applying the patch, the test correctly allocates the required hugepages and the test passes.
Quoted 94 lines
--- .../selftests/mm/hugetlb-soft-offline.c | 45 ++++--------------- 1 file changed, 8 insertions(+), 37 deletions(-)tools/testing/selftests/mm/hugetlb-soft-offline.c +8
-37@@ -6,9 +6,7 @@* - if enable_soft_offline = 1, a hugepage should be dissolved and * nr_hugepages/free_hugepages should be reduced by 1. * -* Before running, make sure more than 2 hugepages of default_hugepagesz -* are allocated. For example, if /proc/meminfo/Hugepagesize is 2048kB: -* echo 8 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages +* The test allocates 8 default hugepages */ #define _GNU_SOURCE@@ -25,6 +23,7 @@#include <sys/types.h> #include "kselftest.h" +#include "hugepage_settings.h" #ifndef MADV_SOFT_OFFLINE #define MADV_SOFT_OFFLINE 101@@ -100,32 +99,6 @@ static int set_enable_soft_offline(int value)return 0 ; } -static int read_nr_hugepages (unsigned long hugepage_size, -unsigned long *nr_hugepages) -{ -char buffer[256 ] = {0 }; -char cmd[256 ] = {0 }; - -sprintf (cmd,"cat /sys/kernel/mm/hugepages/hugepages-%ldkB/nr_hugepages" , - hugepage_size); -FILE *cmdfile =popen (cmd,"r" ); - -if (cmdfile ==NULL ) { -ksft_perror (EPREFIX "failed to popen nr_hugepages" ); -return -1 ; - } - -if (!fgets (buffer,sizeof (buffer), cmdfile)) { -ksft_perror (EPREFIX "failed to read nr_hugepages" ); -pclose (cmdfile); -return -1 ; - } - - *nr_hugepages =atoll (buffer); -pclose (cmdfile); -return 0 ; -} -static int create_hugetlbfs_file (struct statfs *file_stat) {int fd;@@ -177,20 +150,14 @@ static void test_soft_offline_common(int enable_soft_offline)ksft_exit_fail_msg ("Failed to set enable_soft_offline\n" ); } -if (read_nr_hugepages (hugepagesize_kb, &nr_hugepages_before) !=0 ) { -close (fd); -ksft_exit_fail_msg ("Failed to read nr_hugepages\n" ); - } + nr_hugepages_before =hugetlb_nr_default_pages ();ksft_print_msg ("Before MADV_SOFT_OFFLINE nr_hugepages=%ld\n" , nr_hugepages_before); ret =do_soft_offline (fd,2 * file_stat.f_bsize, expect_errno); -if (read_nr_hugepages (hugepagesize_kb, &nr_hugepages_after) !=0 ) { -close (fd); -ksft_exit_fail_msg ("Failed to read nr_hugepages\n" ); - } + nr_hugepages_after =hugetlb_nr_default_pages ();ksft_print_msg ("After MADV_SOFT_OFFLINE nr_hugepages=%ld\n" , nr_hugepages_after);@@ -219,6 +186,10 @@ static void test_soft_offline_common(int enable_soft_offline)int main (int argc,char **argv) {ksft_print_header (); + +if (!hugetlb_setup_default (8 )) +ksft_exit_skip ("not enough hugetlb pages\n" ); +ksft_set_plan (2 );test_soft_offline_common (1 );
Mike Rapoport
maintainerRe: [PATCH v3 42/54] selftests/mm: hugetlb-soft-online: add setup of HugeTLB pages
Hi Sarthak,
Quoted 9 lines · Sarthak Sharma
Hi Mike!
Quoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-soft-online test uses open coded access to /proc to determine availability of huge pages and fails if there are no enough free huget pages..
Quoted 8 lines
Quoted 5 lines
Replace open coded access to /proc with hugepage helpers and add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>The test is hugetlb-soft-offline, so let's change the commit title and commit message to say "offine" instead of "online".
Good catch :)
Quoted 6 lines
I have run the test before and after applying the patch. Before the patch, it failed with an mmap error when hugepages were not preallocated. After applying the patch, the test correctly allocates the required hugepages and the test passes.
Tested-by Sarthak Sharma <[email protected]>
Signature · 2 lines
Sincerely yours, Mike.
Mike Rapoport
maintainerPATCH v3 43/54 selftests/mm: hugetlb-vmemmap: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-vmemmap test fails if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/hugetlb-vmemmap.c | 3 +++ 1 file changed, 3 insertions(+)
tools/testing/selftests/mm/hugetlb-vmemmap.c +3 -0
@@ -97,6 +97,9 @@ int main(int argc, char **argv)ksft_print_header ();ksft_set_plan (1 ); +if (!hugetlb_setup_default (1 )) +ksft_exit_skip ("Not enough free huge pages\n" ); + pagesize =psize (); maplength =default_huge_page_size ();if (!maplength)
Signature · 1 line
2.53.0
Re: [PATCH v3 43/54] selftests/mm: hugetlb-vmemmap: add setup of HugeTLB pages
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
hugetlb-vmemmap test fails if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I have run the test before and after applying the patch. Before the patch, the test bailed out with mmap error if there was not exactly 1 free hugepage. After the patch, the test correctly allocates the hugepage and successfully passes, restoring the hugepage settings at exit.
Quoted 18 lines
--- tools/testing/selftests/mm/hugetlb-vmemmap.c | 3 +++ 1 file changed, 3 insertions(+)tools/testing/selftests/mm/hugetlb-vmemmap.c +3
-0@@ -97,6 +97,9 @@ int main(int argc, char **argv)ksft_print_header ();ksft_set_plan (1 ); +if (!hugetlb_setup_default (1 )) +ksft_exit_skip ("Not enough free huge pages\n" ); + pagesize =psize (); maplength =default_huge_page_size ();if (!maplength)
Mike Rapoport
maintainerPATCH v3 44/54 selftests/mm: migration: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
migration skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Since kselftest_harness runs fixture setup and the tests in child processes, use HUGETLB_SETUP_DEFAULT_PAGES() that defines a constructor that runs in the main process and add verification that there are enough free huge pages to the tests that use them.
Reset signal handlers to defaults in FIXTURE_SETUP() so that sending SIGTERM and SIGHUP during the tests won't cause restoration of HugeTLB settings.
--- tools/testing/selftests/mm/migration.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
tools/testing/selftests/mm/migration.c +19 -0
@@ -23,6 +23,8 @@#define MAX_RETRIES 100#define ALIGN (x, a) (((x) + (a - 1)) & (~((a) - 1))) +HUGETLB_SETUP_DEFAULT_PAGES(1) +FIXTURE (migration) {pthread_t *threads;
@@ -32,10 +34,21 @@ FIXTURE(migration)int n2; }; +static void reset_signals (void ) +{ +struct sigaction sa = { .sa_handler =SIG_DFL }; + +sigemptyset (&sa.sa_mask); +sigaction (SIGTERM , &sa,NULL ); +sigaction (SIGHUP , &sa,NULL ); +} +FIXTURE_SETUP (migration) {int n; +reset_signals (); +if (numa_available () <0 )SKIP (return,"NUMA not available" ); self->nthreads =numa_num_task_cpus () -2 ;
@@ -271,6 +284,9 @@ TEST_F_TIMEOUT(migration, private_anon_htlb, 2*RUNTIME)if (!hugepage_size)SKIP (return,"Reading HugeTLB pagesize failed\n" ); +if (hugetlb_free_default_pages () <1 ) +SKIP (return,"Not enough huge pages\n" ); + ptr =mmap (NULL , hugepage_size,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );
@@ -299,6 +315,9 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME)if (!hugepage_size)SKIP (return,"Reading HugeTLB pagesize failed\n" ); +if (hugetlb_free_default_pages () <1 ) +SKIP (return,"Not enough huge pages\n" ); + ptr =mmap (NULL , hugepage_size,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_ANONYMOUS |MAP_HUGETLB ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );
Signature · 1 line
2.53.0
Re: [PATCH v3 44/54] selftests/mm: migration: add setup of HugeTLB pages
Quoted 13 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
migration skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Since
kselftest_harnessruns fixture setup and the tests in child processes, useHUGETLB_SETUP_DEFAULT_PAGES()that defines a constructor that runs in the main process and add verification that there are enough free huge pages to the tests that use them.
Quoted 5 lines
Reset signal handlers to defaults in
FIXTURE_SETUP()so that sending SIGTERM and SIGHUP during the tests won't cause restoration of HugeTLB settings.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I have run the test before and after applying the patch. Before the patch, the hugetlb subtests failed with MAP_FAILED because hugepages were not preallocated. After the patch, those subtests correctly allocate the required hugepages and pass successfully, restoring the original settings at exit.
Quoted 59 lines
--- tools/testing/selftests/mm/migration.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)tools/testing/selftests/mm/migration.c +19
-0@@ -23,6 +23,8 @@#define MAX_RETRIES 100#define ALIGN (x, a) (((x) + (a - 1)) & (~((a) - 1))) +HUGETLB_SETUP_DEFAULT_PAGES(1) +FIXTURE (migration) {pthread_t *threads;@@ -32,10 +34,21 @@ FIXTURE(migration)int n2; }; +static void reset_signals (void ) +{ +struct sigaction sa = { .sa_handler =SIG_DFL }; + +sigemptyset (&sa.sa_mask); +sigaction (SIGTERM , &sa,NULL ); +sigaction (SIGHUP , &sa,NULL ); +} +FIXTURE_SETUP (migration) {int n; +reset_signals (); +if (numa_available () <0 )SKIP (return,"NUMA not available" ); self->nthreads =numa_num_task_cpus () -2 ;@@ -271,6 +284,9 @@ TEST_F_TIMEOUT(migration, private_anon_htlb, 2*RUNTIME)if (!hugepage_size)SKIP (return,"Reading HugeTLB pagesize failed\n" ); +if (hugetlb_free_default_pages () <1 ) +SKIP (return,"Not enough huge pages\n" ); + ptr =mmap (NULL , hugepage_size,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );@@ -299,6 +315,9 @@ TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME)if (!hugepage_size)SKIP (return,"Reading HugeTLB pagesize failed\n" ); +if (hugetlb_free_default_pages () <1 ) +SKIP (return,"Not enough huge pages\n" ); + ptr =mmap (NULL , hugepage_size,PROT_READ |PROT_WRITE ,MAP_SHARED |MAP_ANONYMOUS |MAP_HUGETLB ,-1 ,0 );ASSERT_NE (ptr,MAP_FAILED );
Mike Rapoport
maintainerPATCH v3 45/54 selftests/mm: pagemap_ioctl: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
pagemap-ioctl skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/pagemap_ioctl.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
tools/testing/selftests/mm/pagemap_ioctl.c +9 -4
@@ -7,8 +7,6 @@#include <sys/mman.h> #include <errno.h> #include <malloc.h> -#include "vm_util.h" -#include "kselftest.h" #include <linux/types.h> #include <linux/memfd.h> #include <linux/userfaultfd.h>
@@ -23,6 +21,10 @@#include <sys/ipc.h> #include <sys/shm.h> +#include "vm_util.h" +#include "kselftest.h" +#include "hugepage_settings.h" +#define PAGEMAP_BITS_ALL (PAGE_IS_WPALLOWED | PAGE_IS_WRITTEN | \ PAGE_IS_FILE | PAGE_IS_PRESENT | \ PAGE_IS_SWAPPED | PAGE_IS_PFNZERO | \
@@ -1554,6 +1556,9 @@ int main(int __attribute__((unused)) argc, char *argv[])if (init_uffd ())ksft_exit_skip ("Failed to initialize userfaultfd\n" ); +if (!hugetlb_setup_default (4 )) +ksft_exit_skip ("Not enough huge pages\n" ); +ksft_set_plan (117 ); page_size =getpagesize ();
@@ -1605,7 +1610,7 @@ int main(int __attribute__((unused)) argc, char *argv[]) }/* 5. SHM Hugetlb page testing */ - mem_size =2 *1024 *1024 ; + mem_size =default_huge_page_size (); mem =gethugetlb_mem (mem_size, &shmid);if (mem) {wp_init (mem, mem_size);
@@ -1633,7 +1638,7 @@ int main(int __attribute__((unused)) argc, char *argv[]) }/* 7. File Hugetlb testing */ - mem_size =2 *1024 *1024 ; + mem_size =default_huge_page_size (); fd =memfd_create ("uffd-test" ,MFD_HUGETLB |MFD_NOEXEC_SEAL );if (fd <0 )ksft_exit_fail_msg ("uffd-test creation failed %d %s\n" , errno,strerror (errno));
Signature · 1 line
2.53.0
Re: [PATCH v3 45/54] selftests/mm: pagemap_ioctl: add setup of HugeTLB pages
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
pagemap-ioctl skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I have run the test before and after applying the patch. Before the patch, 18 HugeTLB subtests were skipped due to no preallocated hugepages. After the patch, those subtests correctly allocate the required hugepages and successfully pass.
Quoted 56 lines
--- tools/testing/selftests/mm/pagemap_ioctl.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)tools/testing/selftests/mm/pagemap_ioctl.c +9
-4@@ -7,8 +7,6 @@#include <sys/mman.h> #include <errno.h> #include <malloc.h> -#include "vm_util.h" -#include "kselftest.h" #include <linux/types.h> #include <linux/memfd.h> #include <linux/userfaultfd.h> @@ -23,6 +21,10 @@#include <sys/ipc.h> #include <sys/shm.h> +#include "vm_util.h" +#include "kselftest.h" +#include "hugepage_settings.h" +#define PAGEMAP_BITS_ALL (PAGE_IS_WPALLOWED | PAGE_IS_WRITTEN | \ PAGE_IS_FILE | PAGE_IS_PRESENT | \ PAGE_IS_SWAPPED | PAGE_IS_PFNZERO | \@@ -1554,6 +1556,9 @@ int main(int __attribute__((unused)) argc, char *argv[])if (init_uffd ())ksft_exit_skip ("Failed to initialize userfaultfd\n" ); +if (!hugetlb_setup_default (4 )) +ksft_exit_skip ("Not enough huge pages\n" ); +ksft_set_plan (117 ); page_size =getpagesize ();@@ -1605,7 +1610,7 @@ int main(int __attribute__((unused)) argc, char *argv[]) }/* 5. SHM Hugetlb page testing */ - mem_size =2 *1024 *1024 ; + mem_size =default_huge_page_size (); mem =gethugetlb_mem (mem_size, &shmid);if (mem) {wp_init (mem, mem_size);@@ -1633,7 +1638,7 @@ int main(int __attribute__((unused)) argc, char *argv[]) }/* 7. File Hugetlb testing */ - mem_size =2 *1024 *1024 ; + mem_size =default_huge_page_size (); fd =memfd_create ("uffd-test" ,MFD_HUGETLB |MFD_NOEXEC_SEAL );if (fd <0 )ksft_exit_fail_msg ("uffd-test creation failed %d %s\n" , errno,strerror (errno));
Mike Rapoport
maintainerPATCH v3 46/54 selftests/mm: protection_keys: use library code for HugeTLB setup
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
protection_keys open codes setup of HugeTLB pages.
Replace it with the library functions from hugepage_setup.
Replace exit() calls with _exit() to avoid restoring HugeTLB settings in the middle of test.
--- tools/testing/selftests/mm/protection_keys.c | 51 ++++++-------------- 1 file changed, 15 insertions(+), 36 deletions(-)
tools/testing/selftests/mm/protection_keys.c +15 -36
@@ -47,6 +47,7 @@#include <setjmp.h> #include "pkey-helpers.h" +#include "hugepage_settings.h" int iteration_nr =1 ;int test_nr;
@@ -61,6 +62,7 @@ noinline int read_ptr(int *ptr)return *ptr; } +#if CONTROL_TRACING >0 static void cat_into_file (char *str,char *file) {int fd =open (file,O_RDWR );
@@ -86,7 +88,6 @@ static void cat_into_file(char *str, char *file)close (fd); } -#if CONTROL_TRACING >0 static int warned_tracing;static int tracing_root_ok (void ) {
@@ -709,50 +710,28 @@ static void *malloc_pkey_anon_huge(long size, int prot, u16 pkey) }static int hugetlb_setup_ok; -#define SYSFS_FMT_NR_HUGE_PAGES "/sys/kernel/mm/hugepages/hugepages-%ldkB/nr_hugepages"#define GET_NR_HUGE_PAGES 10static void setup_hugetlbfs (void ) { -int err; -int fd; -char buf[256 ]; -long hpagesz_kb; -long hpagesz_mb; +long hpagesz_mb =HPAGE_SIZE /1024 /1024 ; +unsigned long free_pages;if (geteuid () !=0 ) {ksft_print_msg ("WARNING: not run as root, can not do hugetlb test\n" );return ; } -cat_into_file (__stringify (GET_NR_HUGE_PAGES ),"/proc/sys/vm/nr_hugepages" ); -/* -* Now go make sure that we got the pages and that they +* Make sure that we got the pages and that they * are PMD-level pages. Someone might have made PUD-level * pages the default. */ - hpagesz_kb =HPAGE_SIZE /1024 ; - hpagesz_mb = hpagesz_kb /1024 ; -sprintf (buf,SYSFS_FMT_NR_HUGE_PAGES , hpagesz_kb); - fd =open (buf,O_RDONLY ); -if (fd <0 ) { -fprintf (stderr,"opening sysfs %ldM hugetlb config: %s\n" , - hpagesz_mb,strerror (errno)); -return ; - } - -/* -1 to guarantee leaving the trailing \0 */ - err =read (fd, buf,sizeof (buf)-1 ); -close (fd); -if (err <=0 ) { -fprintf (stderr,"reading sysfs %ldM hugetlb config: %s\n" , - hpagesz_mb,strerror (errno)); -return ; - } - -if (atoi (buf) !=GET_NR_HUGE_PAGES ) { -fprintf (stderr,"could not confirm %ldM pages, got: '%s' expected %d\n" , - hpagesz_mb, buf,GET_NR_HUGE_PAGES ); +hugetlb_save_settings (); +hugetlb_set_nr_pages (HPAGE_SIZE ,GET_NR_HUGE_PAGES ); + free_pages =hugetlb_free_pages (HPAGE_SIZE ); +if (free_pages <GET_NR_HUGE_PAGES ) { +ksft_print_msg ("could not confirm %ldM pages, got: '%ld' expected %d\n" , + hpagesz_mb, free_pages,GET_NR_HUGE_PAGES );return ; }
@@ -1129,7 +1108,7 @@ static void become_child(void)/* in the child */ return ; } -exit (0 ); +_exit (0 ); }/* Assumes that all pkeys other than 'pkey' are unallocated */
@@ -1508,18 +1487,18 @@ static void test_ptrace_modifies_pkru(int *ptr, u16 pkey)* checking */ if (__read_pkey_reg () != new_pkru) -exit (1 ); +_exit (1 );/* Stop and allow the tracer to clear XSTATE_BV for PKRU */ raise (SIGSTOP );if (__read_pkey_reg () !=0 ) -exit (1 ); +_exit (1 );/* Stop and allow the tracer to examine PKRU */ raise (SIGSTOP ); -exit (0 ); +_exit (0 ); }pkey_assert (child ==waitpid (child, &status,0 ));
Signature · 1 line
2.53.0
Mike Rapoport
maintainerPATCH v3 47/54 selftests/mm: thuge-gen: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
thuge-gen skips tests if there are no free huge pages prepared by a wrapper script and shm liimts in proc are too low.
Replace custom detection of huge pages with the library functions and add setup of HugeTLB pages and shm limits to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/thuge-gen.c | 79 ++++---------------------- 1 file changed, 11 insertions(+), 68 deletions(-)
tools/testing/selftests/mm/thuge-gen.c +11 -68
@@ -38,15 +38,6 @@#ifndef SHM_HUGE_SHIFT #define SHM_HUGE_SHIFT 26#endif -#ifndef SHM_HUGE_MASK -#define SHM_HUGE_MASK 0x3f -#endif -#ifndef SHM_HUGE_2MB -#define SHM_HUGE_2MB (21 << SHM_HUGE_SHIFT) -#endif -#ifndef SHM_HUGE_1GB -#define SHM_HUGE_1GB (30 << SHM_HUGE_SHIFT) -#endif #define NUM_PAGESIZES 5#define NUM_PAGES 4
@@ -64,32 +55,10 @@ int ilog2(unsigned long v)void show (unsigned long ps) { -char buf[100 ]; -if (ps ==getpagesize ())return ; -ksft_print_msg ("%luMB: " , ps >>20 ); - -fflush (stdout); -snprintf (buf,sizeof buf, -"cat /sys/kernel/mm/hugepages/hugepages-%lukB/free_hugepages" , - ps >>10 ); -system (buf); -} - -unsigned long read_free (unsigned long ps) -{ -unsigned long val =0 ; -char buf[100 ]; - -snprintf (buf,sizeof (buf), -"/sys/kernel/mm/hugepages/hugepages-%lukB/free_hugepages" , - ps >>10 ); -if (read_sysfs (buf, &val) && ps !=getpagesize ()) -ksft_print_msg ("missing %s\n" , buf); - -return val; +ksft_print_msg ("%luMB: %ld\n" , ps >>20 ,hugetlb_free_pages (ps)); }void test_mmap (unsigned long size,unsigned flags)
@@ -97,14 +66,14 @@ void test_mmap(unsigned long size, unsigned flags)char *map;unsigned long before, after; - before =read_free (size); + before =hugetlb_free_pages (size); map =mmap (NULL , size*NUM_PAGES ,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB |flags,-1 ,0 );if (map ==MAP_FAILED )ksft_exit_fail_msg ("mmap: %s\n" ,strerror (errno));memset (map,0xff , size*NUM_PAGES ); - after =read_free (size); + after =hugetlb_free_pages (size);show (size); ksft_test_result(size == getpagesize() || (before - after) == NUM_PAGES,
@@ -121,7 +90,7 @@ void test_shmget(unsigned long size, unsigned flags)struct shm_info i;char *map; - before =read_free (size); + before =hugetlb_free_pages (size); id =shmget (IPC_PRIVATE , size *NUM_PAGES ,IPC_CREAT |0600 |flags);if (id <0 ) {if (errno ==EPERM ) {
@@ -142,7 +111,7 @@ void test_shmget(unsigned long size, unsigned flags)shmctl (id,IPC_RMID ,NULL );memset (map,0xff , size*NUM_PAGES ); - after =read_free (size); + after =hugetlb_free_pages (size);show (size); ksft_test_result(size == getpagesize() || (before - after) == NUM_PAGES,
@@ -154,43 +123,15 @@ void test_shmget(unsigned long size, unsigned flags)void find_pagesizes (void ) {unsigned long largest =getpagesize (); -unsigned long shmmax_val =0 ;int i; -glob_t g; -glob ("/sys/kernel/mm/hugepages/hugepages-*kB" ,0 ,NULL , &g); -assert (g.gl_pathc <=NUM_PAGESIZES ); -for (i =0 ; (i < g.gl_pathc) && (num_page_sizes <NUM_PAGESIZES ); i++) { -sscanf (g.gl_pathv[i],"/sys/kernel/mm/hugepages/hugepages-%lukB" , - &page_sizes[num_page_sizes]); - page_sizes[num_page_sizes] <<=10 ; -ksft_print_msg ("Found %luMB\n" , page_sizes[i] >>20 ); + num_page_sizes =hugetlb_setup (NUM_PAGES , page_sizes,ARRAY_SIZE (page_sizes)); -if (page_sizes[num_page_sizes] > largest) +for (i =0 ; i < num_page_sizes; i++) +if (page_sizes[i] > largest) largest = page_sizes[i]; -if (read_free (page_sizes[num_page_sizes]) >=NUM_PAGES ) - num_page_sizes++; -else -ksft_print_msg ("SKIP for size %lu MB as not enough huge pages, need %u\n" , - page_sizes[num_page_sizes] >>20 ,NUM_PAGES ); - } -globfree (&g); - -read_sysfs ("/proc/sys/kernel/shmmax" , &shmmax_val); -if (shmmax_val <NUM_PAGES * largest) { -ksft_print_msg ("WARNING: shmmax is too small to run this test.\n" ); -ksft_print_msg ("Please run the following command to increase shmmax:\n" ); -ksft_print_msg ("echo %lu > /proc/sys/kernel/shmmax\n" , largest *NUM_PAGES ); -ksft_exit_skip ("Test skipped due to insufficient shmmax value.\n" ); - } - -#if defined(__x86_64__) -if (largest !=1U <<30 ) { -ksft_exit_skip ("No GB pages available on x86-64\n" -"Please boot with hugepagesz=1G hugepages=%d\n" ,NUM_PAGES ); - } -#endif +shm_limits_prepare (NUM_PAGES * largest); }int main (void )
@@ -233,3 +174,5 @@ int main(void)ksft_finished (); } + +SHM_LIMITS_RESTORE ()
Signature · 1 line
2.53.0
Re: [PATCH v3 47/54] selftests/mm: thuge-gen: add setup of HugeTLB pages
Quoted 11 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
thuge-gen skips tests if there are no free huge pages prepared by a wrapper script and shm liimts in proc are too low.
Replace custom detection of huge pages with the library functions and add setup of HugeTLB pages and shm limits to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
I have run the test before and after applying the patch. Before the patch, all test sizes were skipped because hugepages were not preallocated. After the patch, the test correctly allocates the required hugepages and all tests pass, with the original settings restored at exit.
Mike Rapoport
maintainerPATCH v3 48/54 selftests/mm: uffd-stress: use hugetlb_save and alloc huge pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
uffd-stress skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/uffd-stress.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
tools/testing/selftests/mm/uffd-stress.c +6 -3
@@ -479,9 +479,12 @@ int main(int argc, char **argv)* Ensure nr_parallel - 1 hugepages on top of that to account * for racy extra reservation of hugepages. */ -if (gopts->test_type ==TEST_HUGETLB && -hugetlb_free_default_pages () <2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 ) -ksft_exit_skip ("Skipping userfaultfd... not enough hugepages\n" ); +if (gopts->test_type ==TEST_HUGETLB ) { +unsigned long nr =2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 ; + +if (!hugetlb_setup_default (nr)) +ksft_exit_skip ("Skipping userfaultfd... not enough hugepages\n" ); + } gopts->nr_pages_per_cpu = bytes / gopts->page_size / gopts->nr_parallel;if (!gopts->nr_pages_per_cpu) {
Signature · 1 line
2.53.0
Re: [PATCH v3 48/54] selftests/mm: uffd-stress: use hugetlb_save and alloc huge pages
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
uffd-stress skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
I ran the test using ./uffd-stress hugetlb 16 20
Before the patch, the test skipped with "not enough hugepages". After the patch, the test correctly allocates the required hugepages and passes successfully, restoring the original settings at exit.
Quoted 24 lines
--- tools/testing/selftests/mm/uffd-stress.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)tools/testing/selftests/mm/uffd-stress.c +6
-3@@ -479,9 +479,12 @@ int main(int argc, char **argv)* Ensure nr_parallel - 1 hugepages on top of that to account * for racy extra reservation of hugepages. */ -if (gopts->test_type ==TEST_HUGETLB && -hugetlb_free_default_pages () <2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 ) -ksft_exit_skip ("Skipping userfaultfd... not enough hugepages\n" ); +if (gopts->test_type ==TEST_HUGETLB ) { +unsigned long nr =2 * (bytes / gopts->page_size) + gopts->nr_parallel -1 ; + +if (!hugetlb_setup_default (nr)) +ksft_exit_skip ("Skipping userfaultfd... not enough hugepages\n" ); + } gopts->nr_pages_per_cpu = bytes / gopts->page_size / gopts->nr_parallel;if (!gopts->nr_pages_per_cpu) {
Mike Rapoport
maintainerPATCH v3 49/54 selftests/mm: uffd-unit-tests: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
uffd-unit-tests skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Replace exit() calls with _exit() to avoid restoring HugeTLB settings in the middle of test.
--- tools/testing/selftests/mm/uffd-unit-tests.c | 31 +++++++++++++++++--- 1 file changed, 27 insertions(+), 4 deletions(-)
tools/testing/selftests/mm/uffd-unit-tests.c +27 -4
@@ -305,7 +305,7 @@ static int pagemap_test_fork(uffd_global_test_opts_t *gopts, bool with_event, boif (test_pin)unpin_pages (&args);/* Succeed */ -exit (0 ); +_exit (0 ); }waitpid (child, &result,0 );
@@ -773,7 +773,7 @@ static void uffd_sigbus_test_common(uffd_global_test_opts_t *gopts, bool wp)err ("fork" );if (!pid) -exit (faulting_process (gopts,2 , wp)); +_exit (faulting_process (gopts,2 , wp));waitpid (pid, &err,0 );if (err)
@@ -827,7 +827,7 @@ static void uffd_events_test_common(uffd_global_test_opts_t *gopts, bool wp)err ("fork" );if (!pid) -exit (faulting_process (gopts,0 , wp)); +_exit (faulting_process (gopts,0 , wp));waitpid (pid, &err,0 );if (err)
@@ -1706,11 +1706,32 @@ static int uffd_count_tests(int n_tests, int n_mems, const char *test_filter)return count; } +static unsigned long uffd_setup_hugetlb (void ) +{ +unsigned long nr_hugepages, hp_size; + +hugetlb_save_settings (); + hp_size =default_huge_page_size (); + +if (!hp_size) +return 0 ; + +/* need twice UFFD_TEST_MEM_SIZE, one for src area and one for dst */ + nr_hugepages =2 *MAX (UFFD_TEST_MEM_SIZE , hp_size *2 ) / hp_size; +hugetlb_set_nr_default_pages (nr_hugepages); + +if (hugetlb_free_default_pages () < nr_hugepages) +return 0 ; + +return hp_size; +} +int main (int argc,char *argv[]) {int n_tests =sizeof (uffd_tests) /sizeof (uffd_test_case_t);int n_mems =sizeof (mem_types) /sizeof (mem_type_t);const char *test_filter =NULL ; +unsigned long hugepage_size;bool list_only = false;uffd_test_case_t *test;mem_type_t *mem_type;
@@ -1745,6 +1766,8 @@ int main(int argc, char *argv[])return KSFT_PASS ; } + hugepage_size =uffd_setup_hugetlb (); +ksft_print_header ();ksft_set_plan (uffd_count_tests (n_tests, n_mems, test_filter));
@@ -1771,7 +1794,7 @@ int main(int argc, char *argv[]) uffd_test_case_ops = test->test_case_ops;if (mem_type->mem_flag & (MEM_HUGETLB_PRIVATE |MEM_HUGETLB )) { - gopts.page_size =default_huge_page_size (); + gopts.page_size = hugepage_size;if (gopts.page_size ==0 ) {uffd_test_skip ("huge page size is 0, feature missing?" );continue ;
Signature · 1 line
2.53.0
Re: [PATCH v3 49/54] selftests/mm: uffd-unit-tests: add setup of HugeTLB pages
Hi Mike!
Quoted 90 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
uffd-unit-tests skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Replace
exit()calls with_exit()to avoid restoring HugeTLB settings in the middle of test.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/uffd-unit-tests.c | 31 +++++++++++++++++--- 1 file changed, 27 insertions(+), 4 deletions(-)tools/testing/selftests/mm/uffd-unit-tests.c +26
-3@@ -305,7 +305,7 @@ static int pagemap_test_fork(uffd_global_test_opts_t *gopts, bool with_event, boif (test_pin)unpin_pages (&args);/* Succeed */ -exit (0 ); +_exit (0 ); }waitpid (child, &result,0 );@@ -773,7 +773,7 @@ static void uffd_sigbus_test_common(uffd_global_test_opts_t *gopts, bool wp)err ("fork" );if (!pid) -exit (faulting_process (gopts,2 , wp)); +_exit (faulting_process (gopts,2 , wp));waitpid (pid, &err,0 );if (err)@@ -827,7 +827,7 @@ static void uffd_events_test_common(uffd_global_test_opts_t *gopts, bool wp)err ("fork" );if (!pid) -exit (faulting_process (gopts,0 , wp)); +_exit (faulting_process (gopts,0 , wp));waitpid (pid, &err,0 );if (err)@@ -1706,11 +1706,32 @@ static int uffd_count_tests(int n_tests, int n_mems, const char *test_filter)return count; } +static unsigned long uffd_setup_hugetlb (void ) +{ +unsigned long nr_hugepages, hp_size; + +hugetlb_save_settings (); + hp_size =default_huge_page_size (); + +if (!hp_size) +return 0 ; + +/* need twice UFFD_TEST_MEM_SIZE, one for src area and one for dst */ + nr_hugepages =2 *MAX (UFFD_TEST_MEM_SIZE , hp_size *2 ) / hp_size; +hugetlb_set_nr_default_pages (nr_hugepages); + +if (hugetlb_free_default_pages () < nr_hugepages) +return 0 ; + +return hp_size; +} +int main (int argc,char *argv[]) {int n_tests =sizeof (uffd_tests) /sizeof (uffd_test_case_t);int n_mems =sizeof (mem_types) /sizeof (mem_type_t);const char *test_filter =NULL ; +unsigned long hugepage_size;bool list_only = false;uffd_test_case_t *test;mem_type_t *mem_type;@@ -1745,6 +1766,8 @@ int main(int argc, char *argv[])return KSFT_PASS ; } + hugepage_size =uffd_setup_hugetlb (); +ksft_print_header ();ksft_set_plan (uffd_count_tests (n_tests, n_mems, test_filter));
tools/testing/selftests/mm/uffd-unit-tests.c int main(int argc, char *argv[]) · line 1799
Quoted 7 lines
tools/testing/selftests/mm/uffd-unit-tests.c +1
-1@@ -1771,7 +1794,7 @@ int main(int argc, char *argv[]) uffd_test_case_ops = test->test_case_ops;if (mem_type->mem_flag & (MEM_HUGETLB_PRIVATE |MEM_HUGETLB )) { - gopts.page_size =default_huge_page_size (); + gopts.page_size = hugepage_size;if (gopts.page_size ==0 ) {uffd_test_skip ("huge page size is 0, feature missing?" );
A small nit, after this patch, uffd_setup_hugetlb() can return 0 in 2 cases:
a) default_huge_page_size() == 0
b) hugetlb_free_default_pages() < nr_hugepages
The skip message only covers the first case and would be misleading for the second case. Should we differentiate both these using different skip messages?
Quoted 1 line
tools/testing/selftests/mm/uffd-unit-tests.c +0
-0continue ;
Mike Rapoport
maintainerRe: [PATCH v3 49/54] selftests/mm: uffd-unit-tests: add setup of HugeTLB pages
Hi Sarthak,
Quoted 12 lines · Sarthak Sharma
tools/testing/selftests/mm/uffd-unit-tests.c int main(int argc, char *argv[]) · line 1799
Quoted 8 lines · Mike Rapoport
tools/testing/selftests/mm/uffd-unit-tests.c +1
-1@@ -1771,7 +1794,7 @@ int main(int argc, char *argv[]) uffd_test_case_ops = test->test_case_ops;if (mem_type->mem_flag & (MEM_HUGETLB_PRIVATE |MEM_HUGETLB )) { - gopts.page_size =default_huge_page_size (); + gopts.page_size = hugepage_size;if (gopts.page_size ==0 ) {uffd_test_skip ("huge page size is 0, feature missing?" );A small nit, after this patch,
uffd_setup_hugetlb()can return 0 in 2 cases:
Quoted 6 lines
a)
default_huge_page_size()== 0
b)hugetlb_free_default_pages()<nr_hugepagesThe skip message only covers the first case and would be misleading for the second case. Should we differentiate both these using different skip messages?
I'm more inclined to have a single "Not enough huge pages" message to keep the things simple.
Quoted 2 lines
Quoted 1 line
tools/testing/selftests/mm/uffd-unit-tests.c +0
-0continue ;
Signature · 2 lines
Sincerely yours, Mike.
Mike Rapoport
maintainerPATCH v3 50/54 selftests/mm: uffd-wp-mremap: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
uffd-wp-remap skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/uffd-wp-mremap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tools/testing/selftests/mm/uffd-wp-mremap.c +3 -3
@@ -336,14 +336,14 @@ int main(int argc, char **argv)struct thp_settings settings;int i, j, plan =0 ; +hugepage_save_settings (true, true); + pagesize =getpagesize (); nr_thpsizes =detect_thp_sizes (thpsizes,ARRAY_SIZE (thpsizes)); - nr_hugetlbsizes =detect_hugetlb_page_sizes (hugetlbsizes, -ARRAY_SIZE (hugetlbsizes)); + nr_hugetlbsizes =hugetlb_setup (1 , hugetlbsizes,ARRAY_SIZE (hugetlbsizes));/* If THP is supported, save THP settings and initially disable THP. */ if (nr_thpsizes) { -thp_save_settings ();thp_read_settings (&settings);for (i =0 ; i <NR_ORDERS ; i++) { settings.hugepages[i].enabled =THP_NEVER ;
Signature · 1 line
2.53.0
Re: [PATCH v3 50/54] selftests/mm: uffd-wp-mremap: add setup of HugeTLB pages
Hi Mike!
Quoted 18 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
uffd-wp-remap skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- tools/testing/selftests/mm/uffd-wp-mremap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)tools/testing/selftests/mm/uffd-wp-mremap.c +0
-0
tools/testing/selftests/mm/uffd-wp-mremap.c int main(int argc, char **argv) · line 339
Quoted 4 lines
tools/testing/selftests/mm/uffd-wp-mremap.c +1
-0@@ -336,14 +336,14 @@ int main(int argc, char **argv)struct thp_settings settings;int i, j, plan =0 ; +hugepage_save_settings (true, true);
Since hugetlb_setup below already calls hugepage_save_settings() internally, perhaps we can change this to hugepage_save_settings(true, false) or simply keep thp_save_settings()? Otherwise we're saving the HugeTLB state twice here and it seems redundant.
Quoted 13 lines
tools/testing/selftests/mm/uffd-wp-mremap.c +2
-3+ pagesize =getpagesize (); nr_thpsizes =detect_thp_sizes (thpsizes,ARRAY_SIZE (thpsizes)); - nr_hugetlbsizes =detect_hugetlb_page_sizes (hugetlbsizes, -ARRAY_SIZE (hugetlbsizes)); + nr_hugetlbsizes =hugetlb_setup (1 , hugetlbsizes,ARRAY_SIZE (hugetlbsizes));/* If THP is supported, save THP settings and initially disable THP. */ if (nr_thpsizes) { -thp_save_settings ();thp_read_settings (&settings);for (i =0 ; i <NR_ORDERS ; i++) { settings.hugepages[i].enabled =THP_NEVER ;
Mike Rapoport
maintainerPATCH v3 51/54 selftests/mm: va_high_addr_switch: add setup of HugeTLB pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
va_high_addr_switch skips HugeTLB tests if there are no free huge pages prepared by a wrapper script.
Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
--- tools/testing/selftests/mm/va_high_addr_switch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/mm/va_high_addr_switch.c +1 -1
@@ -325,7 +325,7 @@ int main(int argc, char **argv)if (!supported_arch ())ksft_exit_skip ("Architecture not supported\n" ); -if (argc ==2 && !strcmp (argv[1 ],"--run-hugetlb" )) +if (hugetlb_setup_default (6 )) run_hugetlb = true;testcases_init ();
Signature · 1 line
2.53.0
Re: [PATCH v3 51/54] selftests/mm: va_high_addr_switch: add setup of HugeTLB pages
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
va_high_addr_switchskips HugeTLB tests if there are no free huge pages prepared by a wrapper script.Add setup of HugeTLB pages to the test and make sure that the original settings are restored on the test exit.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Quoted 17 lines
--- tools/testing/selftests/mm/va_high_addr_switch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/testing/selftests/mm/va_high_addr_switch.c +1
-1@@ -325,7 +325,7 @@ int main(int argc, char **argv)if (!supported_arch ())ksft_exit_skip ("Architecture not supported\n" ); -if (argc ==2 && !strcmp (argv[1 ],"--run-hugetlb" )) +if (hugetlb_setup_default (6 )) run_hugetlb = true;testcases_init ();
Mike Rapoport
maintainerPATCH v3 52/54 selftests/mm: va_high_addr_switch.sh: drop huge pages setup
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Since va_high_addr_switch takes care of setting up huge pages, there is no need to set them up in the va_high_addr_switch.sh wrapper script.
--- .../selftests/mm/va_high_addr_switch.sh | 41 +------------------ 1 file changed, 1 insertion(+), 40 deletions(-)
tools/testing/selftests/mm/va_high_addr_switch.sh +1 -40
@@ -9,7 +9,6 @@ # Kselftest framework requirement - SKIP code is 4. ksft_skip=4 -orig_nr_hugepages=0 skip() {
@@ -77,43 +76,5 @@ check_test_requirements() esac } -save_nr_hugepages() -{ - orig_nr_hugepages=$(cat /proc/sys/vm/nr_hugepages) -} - -restore_nr_hugepages() -{ - echo "$orig_nr_hugepages" > /proc/sys/vm/nr_hugepages -} - -setup_nr_hugepages() -{ - local needpgs=$1 - while read -r name size unit; do - if [ "$name" = "HugePages_Free:" ]; then - freepgs="$size" - break - fi - done < /proc/meminfo - if [ "$freepgs" -ge "$needpgs" ]; then - return - fi - local hpgs=$((orig_nr_hugepages + needpgs)) - echo $hpgs > /proc/sys/vm/nr_hugepages - - local nr_hugepgs=$(cat /proc/sys/vm/nr_hugepages) - if [ "$nr_hugepgs" != "$hpgs" ]; then - restore_nr_hugepages - skip "$0: no enough hugepages for testing" - fi -} - check_test_requirements -save_nr_hugepages -# The HugeTLB tests require 6 pages -setup_nr_hugepages 6 -./va_high_addr_switch --run-hugetlb -retcode=$? -restore_nr_hugepages -exit $retcode +./va_high_addr_switch
Signature · 1 line
2.53.0
Re: [PATCH v3 52/54] selftests/mm: va_high_addr_switch.sh: drop huge pages setup
Quoted 6 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Since
va_high_addr_switchtakes care of setting up huge pages, there is no need to set them up in theva_high_addr_switch.shwrapper script.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
Quoted 61 lines
--- .../selftests/mm/va_high_addr_switch.sh | 41 +------------------ 1 file changed, 1 insertion(+), 40 deletions(-)tools/testing/selftests/mm/va_high_addr_switch.sh +1
-40@@ -9,7 +9,6 @@ # Kselftest framework requirement - SKIP code is 4. ksft_skip=4 -orig_nr_hugepages=0 skip() {@@ -77,43 +76,5 @@ check_test_requirements() esac } -save_nr_hugepages() -{ - orig_nr_hugepages=$(cat /proc/sys/vm/nr_hugepages) -} - -restore_nr_hugepages() -{ - echo "$orig_nr_hugepages" > /proc/sys/vm/nr_hugepages -} - -setup_nr_hugepages() -{ - local needpgs=$1 - while read -r name size unit; do - if [ "$name" = "HugePages_Free:" ]; then - freepgs="$size" - break - fi - done < /proc/meminfo - if [ "$freepgs" -ge "$needpgs" ]; then - return - fi - local hpgs=$((orig_nr_hugepages + needpgs)) - echo $hpgs > /proc/sys/vm/nr_hugepages - - local nr_hugepgs=$(cat /proc/sys/vm/nr_hugepages) - if [ "$nr_hugepgs" != "$hpgs" ]; then - restore_nr_hugepages - skip "$0: no enough hugepages for testing" - fi -} - check_test_requirements -save_nr_hugepages -# The HugeTLB tests require 6 pages -setup_nr_hugepages 6 -./va_high_addr_switch --run-hugetlb -retcode=$? -restore_nr_hugepages -exit $retcode +./va_high_addr_switch
Mike Rapoport
maintainerPATCH v3 53/54 selftests/mm: run_vmtests.sh: free memory if available memory is low
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Currently when running THP and HugeTLB tests, if HAVE_HUGEPAGES is set run_test() drops caches, compacts memory and runs the test. But if HAVE_HUGEPAGES is not set it skips the tests entirely, even if THP tests have nothing to do with HAVE_HUGEPAGES.
Replace the check if HAVE_HUGEPAGES is set with a check of how much memory is available. If there is less than 256 MB of available memory, drop caches and run compaction and then continue to run a test regardless of HAVE_HUGEPAGES value.
--- tools/testing/selftests/mm/run_vmtests.sh | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
tools/testing/selftests/mm/run_vmtests.sh +9 -9
@@ -99,6 +99,9 @@ EOF exit 0 } +mem_available_kb=$(awk '/MemAvailable/ {print $2}' /proc/meminfo) +mem_available_Mb=$((mem_available_kb / 1024)) + RUN_ALL=false RUN_DESTRUCTIVE=false TAP_PREFIX="# "
@@ -239,15 +242,12 @@ run_test() { # On memory constrainted systems some tests can fail to allocate hugepages. # perform some cleanup before the test for a higher success rate. if [ ${CATEGORY} == "thp" -o ${CATEGORY} == "hugetlb" ]; then - if [ "${HAVE_HUGEPAGES}" = "1" ]; then - echo 3 > /proc/sys/vm/drop_caches - sleep 2 - echo 1 > /proc/sys/vm/compact_memory - sleep 2 - else - echo "hugepages not supported" | tap_prefix - skip=1 - fi + if (( $mem_available_Mb < 256 )); then + echo 3 > /proc/sys/vm/drop_caches + sleep 2 + echo 1 > /proc/sys/vm/compact_memory + sleep 2 + fi fi local test=$(pretty_name "$*")
Signature · 1 line
2.53.0
Re: [PATCH v3 53/54] selftests/mm: run_vmtests.sh: free memory if available memory is low
Quoted 7 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Currently when running THP and HugeTLB tests, if
HAVE_HUGEPAGESis setrun_test()drops caches, compacts memory and runs the test. But ifHAVE_HUGEPAGESis not set it skips the tests entirely, even if THP tests have nothing to do withHAVE_HUGEPAGES.
Quoted 6 lines
Replace the check if
HAVE_HUGEPAGESis set with a check of how much memory is available. If there is less than 256 MB of available memory, drop caches and run compaction and then continue to run a test regardless ofHAVE_HUGEPAGESvalue.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
This patch looks good to me.
Quoted 40 lines
--- tools/testing/selftests/mm/run_vmtests.sh | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)tools/testing/selftests/mm/run_vmtests.sh +9
-9@@ -99,6 +99,9 @@ EOF exit 0 } +mem_available_kb=$(awk '/MemAvailable/ {print $2}' /proc/meminfo) +mem_available_Mb=$((mem_available_kb / 1024)) + RUN_ALL=false RUN_DESTRUCTIVE=false TAP_PREFIX="# "@@ -239,15 +242,12 @@ run_test() { # On memory constrainted systems some tests can fail to allocate hugepages. # perform some cleanup before the test for a higher success rate. if [ ${CATEGORY} == "thp" -o ${CATEGORY} == "hugetlb" ]; then - if [ "${HAVE_HUGEPAGES}" = "1" ]; then - echo 3 > /proc/sys/vm/drop_caches - sleep 2 - echo 1 > /proc/sys/vm/compact_memory - sleep 2 - else - echo "hugepages not supported" | tap_prefix - skip=1 - fi + if (( $mem_available_Mb < 256 )); then + echo 3 > /proc/sys/vm/drop_caches + sleep 2 + echo 1 > /proc/sys/vm/compact_memory + sleep 2 + fi fi local test=$(pretty_name "$*")
Mike Rapoport
maintainerPATCH v3 54/54 selftests/mm: run_vmtests.sh: drop detection and setup of HugeTLB
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
All the tests that use HugeTLB can detect and setup HugeTLB pages on their own.
Drop detection and setup of HugeTLB from run_vmtests.sh.
--- tools/testing/selftests/mm/run_vmtests.sh | 128 ++-------------------- 1 file changed, 9 insertions(+), 119 deletions(-)
tools/testing/selftests/mm/run_vmtests.sh +9 -119
@@ -157,60 +157,6 @@ run_gup_matrix() { done } -# get huge pagesize and freepages from /proc/meminfo -while read -r name size unit; do - if [ "$name" = "HugePages_Free:" ]; then - freepgs="$size" - fi - if [ "$name" = "Hugepagesize:" ]; then - hpgsize_KB="$size" - fi -done < /proc/meminfo - -# Simple hugetlbfs tests have a hardcoded minimum requirement of -# huge pages totaling 256MB (262144KB) in size. The userfaultfd -# hugetlb test requires a minimum of 2 * nr_cpus huge pages. Take -# both of these requirements into account and attempt to increase -# number of huge pages available. -nr_cpus=$(nproc) -uffd_min_KB=$((hpgsize_KB * nr_cpus * 2)) -hugetlb_min_KB=$((256 * 1024)) -if [[ $uffd_min_KB -gt $hugetlb_min_KB ]]; then - needmem_KB=$uffd_min_KB -else - needmem_KB=$hugetlb_min_KB -fi - -# set proper nr_hugepages -if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then - orig_nr_hugepgs=$(cat /proc/sys/vm/nr_hugepages) - needpgs=$((needmem_KB / hpgsize_KB)) - tries=2 - while [ "$tries" -gt 0 ] && [ "$freepgs" -lt "$needpgs" ]; do - lackpgs=$((needpgs - freepgs)) - echo 3 > /proc/sys/vm/drop_caches - if ! echo $((lackpgs + orig_nr_hugepgs)) > /proc/sys/vm/nr_hugepages; then - echo "Please run this test as root" - exit $ksft_skip - fi - while read -r name size unit; do - if [ "$name" = "HugePages_Free:" ]; then - freepgs=$size - fi - done < /proc/meminfo - tries=$((tries - 1)) - done - nr_hugepgs=$(cat /proc/sys/vm/nr_hugepages) - if [ "$freepgs" -lt "$needpgs" ]; then - printf "Not enough huge pages available (%d < %d)\n" \ - "$freepgs" "$needpgs" - fi - HAVE_HUGEPAGES=1 -else - echo "no hugetlbfs support in kernel?" - HAVE_HUGEPAGES=0 -fi - # filter 64bit architectures ARCH64STR="arm64 mips64 parisc64 ppc64 ppc64le riscv64 s390x sparc64 x86_64" if [ -z "$ARCH" ]; then
@@ -283,29 +229,13 @@ run_test() { echo "TAP version 13" | tap_output CATEGORY="hugetlb" run_test ./hugetlb-mmap - -shmmax=$(cat /proc/sys/kernel/shmmax) -shmall=$(cat /proc/sys/kernel/shmall) -echo 268435456 > /proc/sys/kernel/shmmax -echo 4194304 > /proc/sys/kernel/shmall CATEGORY="hugetlb" run_test ./hugetlb-shm -echo "$shmmax" > /proc/sys/kernel/shmmax -echo "$shmall" > /proc/sys/kernel/shmall - CATEGORY="hugetlb" run_test ./hugetlb-mremap CATEGORY="hugetlb" run_test ./hugetlb-vmemmap CATEGORY="hugetlb" run_test ./hugetlb-madvise CATEGORY="hugetlb" run_test ./hugetlb_dio - -if [ "${HAVE_HUGEPAGES}" = "1" ]; then - nr_hugepages_tmp=$(cat /proc/sys/vm/nr_hugepages) - # For this test, we need one and just one huge page - echo 1 > /proc/sys/vm/nr_hugepages - CATEGORY="hugetlb" run_test ./hugetlb_fault_after_madv - CATEGORY="hugetlb" run_test ./hugetlb_madv_vs_map - # Restore the previous number of huge pages, since further tests rely on it - echo "$nr_hugepages_tmp" > /proc/sys/vm/nr_hugepages -fi +CATEGORY="hugetlb" run_test ./hugetlb_fault_after_madv +CATEGORY="hugetlb" run_test ./hugetlb_madv_vs_map if test_selected "hugetlb"; then echo "NOTE: These hugetlb tests provide minimal coverage. Use" | tap_prefix
@@ -332,44 +262,12 @@ CATEGORY="gup_test" run_test ./gup_longterm CATEGORY="userfaultfd" run_test ./uffd-unit-tests uffd_stress_bin=./uffd-stress CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon 20 16 -# Hugetlb tests require source and destination huge pages. Pass in almost half -# the size of the free pages we have, which is used for *each*. An adjustment -# of (nr_parallel - 1) is done (see nr_parallel in uffd-stress.c) to have some -# extra hugepages - this is done to prevent the test from failing by racily -# reserving more hugepages than strictly required. -# uffd-stress expects a region expressed in MiB, so we adjust -# half_ufd_size_MB accordingly. adjustment=$(( (31 < (nr_cpus - 1)) ? 31 : (nr_cpus - 1) )) -half_ufd_size_MB=$((((freepgs - adjustment) * hpgsize_KB) / 1024 / 2)) -CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb "$half_ufd_size_MB" 32 -CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb-private "$half_ufd_size_MB" 32 +CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb 128 32 +CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb-private 128 32 CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem 20 16 CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem-private 20 16 -# uffd-wp-mremap requires at least one page of each size. -have_all_size_hugepgs=true -declare -A nr_size_hugepgs -for f in /sys/kernel/mm/hugepages/**/nr_hugepages; do - old=$(cat $f) - nr_size_hugepgs["$f"]="$old" - if [ "$old" == 0 ]; then - echo 1 > "$f" - fi - if [ $(cat "$f") == 0 ]; then - have_all_size_hugepgs=false - break - fi -done -if $have_all_size_hugepgs; then - CATEGORY="userfaultfd" run_test ./uffd-wp-mremap -else - echo "# SKIP ./uffd-wp-mremap" -fi - -#cleanup -for f in "${!nr_size_hugepgs[@]}"; do - echo "${nr_size_hugepgs["$f"]}" > "$f" -done -echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages +CATEGORY="userfaultfd" run_test ./uffd-wp-mremap CATEGORY="compaction" run_test ./compaction_test
@@ -393,14 +291,11 @@ CATEGORY="mremap" run_test ./mremap_test CATEGORY="hugetlb" run_test ./thuge-gen CATEGORY="hugetlb" run_test ./charge_reserved_hugetlb.sh -cgroup-v2 CATEGORY="hugetlb" run_test ./hugetlb_reparenting_test.sh -cgroup-v2 + if $RUN_DESTRUCTIVE; then -nr_hugepages_tmp=$(cat /proc/sys/vm/nr_hugepages) -enable_soft_offline=$(cat /proc/sys/vm/enable_soft_offline) -echo 8 > /proc/sys/vm/nr_hugepages -CATEGORY="hugetlb" run_test ./hugetlb-soft-offline -echo "$nr_hugepages_tmp" > /proc/sys/vm/nr_hugepages -echo "$enable_soft_offline" > /proc/sys/vm/enable_soft_offline -CATEGORY="hugetlb" run_test ./hugetlb-read-hwpoison + CATEGORY="hugetlb" run_test ./hugetlb-soft-offline + echo "$enable_soft_offline" > /proc/sys/vm/enable_soft_offline + CATEGORY="hugetlb" run_test ./hugetlb-read-hwpoison fi if [ $VADDR64 -ne 0 ]; then
@@ -464,7 +359,6 @@ if [ -x ./protection_keys_64 ] then CATEGORY="pkey" run_test ./protection_keys_64 fi -echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages if [ -x ./soft-dirty ] then
@@ -545,10 +439,6 @@ if [ -n "${LOADED_MOD}" ]; then modprobe -r hwpoison_inject > /dev/null 2>&1 fi -if [ "${HAVE_HUGEPAGES}" = 1 ]; then - echo "$orig_nr_hugepgs" > /proc/sys/vm/nr_hugepages -fi - echo "SUMMARY: PASS=${count_pass} SKIP=${count_skip} FAIL=${count_fail}" | tap_prefix echo "1..${count_total}" | tap_output
Signature · 1 line
2.53.0
Re: [PATCH v3 54/54] selftests/mm: run_vmtests.sh: drop detection and setup of HugeTLB
Quoted 9 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
All the tests that use HugeTLB can detect and setup HugeTLB pages on their own.
Drop detection and setup of HugeTLB from
run_vmtests.sh.Signed-off-by Mike Rapoport (Microsoft) <[email protected]>---
This patch looks good to me.
Quoted 30 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Hi,
There's a lot of dancing around HugeTLB settings in
run_vmtests.sh. Some test need just a few default huge pages, some require at least 256 MB, and some just skip lots of tests if huge pages of all supported sizes are not available.The goal of this set is to make tests deal with HugeTLB setup and teardown.
There are already convenient helpers that allow easy reading and writing of /proc and /sysfs, so adding a few APIs that will detect and update HugeTLB settings shouldn't be a big deal. But these nice helpers use kselftest framework, and many of HugeTLB (and even THP) test don't, so as a result this patchset also includes a lot of churn for conversion of those tests to kselftest framework (patches 7-19).
The series break out:
patches 1-5: small fixes patch 6: merge of hugetlb mmap tests patch 7: renaming of hugepage-* to hugetlb-* patches 8-20: mechanical conversion to kselftest framework patches 21-27: extension of thp_settings to hugepage_settings to also includeHugeTLB helpers
patches 28-29: add helpers for setting up SHM limits in hugetlb-shm and
thuge-gen tests
patches 30-52: integrate the new APIs in all the tests that use HugeTLB patches 53-54: drop HugeTLB setup fromrun_vmtests.sh
Quoted 8 lines
v3 changes:
* add patches that fix cleanup of fork()ed processes in migration test
* dropMADV_REMOVEchanges to khugepage test
* properly setup memfd for non-default hugepages in hugetlb-mmap test
* add forgottenhugetlb_save_settings()tohugetlb_setup()
* fixes for ksft reporting
* rebase on v7.1-rc1
* add more tags, thanks everyone!
I reran my testing that previously reveled some of the bugs I reported for v2, they pass now. I ran them on an x86-64 guest with 88 vCPUs and also on a small s390 guest. The suite as a whole works as before (I also checked the HugeTLB state gets restored for each test).
I'll be reviewing some of the individual patches, but for the suite as whole:
Quoted 121 lines
v2: https://lore.kernel.org/all/[email protected]
* fix hugetlb mmap tests to actually test file mappings
* increase maximal tests number in khugepaged
* add tags, thanks everyone!v1: https://lore.kernel.org/all/[email protected]
Mike Rapoport (Microsoft) (54):
selftests/mm: hugetlb-read-hwpoison: add SIGBUS handler selftests/mm: migration: don't assume hupe page is TWOMEG selftests/mm: migration: make nthreads represent number of working threads selftests/mm: migration: properly cleanup fork()ed processes selftests/mm:run_vmtests.sh: don't gate THP and KSM tests onHAVE_HUGEPAGESselftests/mm: mergemap_hugetlbinto hugepage-mmap
selftests/mm: rename hugepage-* tests to hugetlb-*
selftests/mm: hugetlb-shm: use kselftest framework
selftests/mm: hugetlb-vmemmap: use kselftest framework selftests/mm: hugetlb-madvise: use kselftest framework selftests/mm:hugetlb_madv_vs_map: use kselftest framework selftests/mm: hugetlb-read-hwpoison: use kselftest framework selftests/mm: khugepaged: group tests in an array
selftests/mm: khugepaged: use ksefltest framework
selftests/mm:ksm_tests: use kselftest framework
selftests/mm:protection_keys: use descriptive test names in TAP output selftests/mm:protection_keys: use kselftest framework selftests/mm: uffd-stress: use kselftest framework
selftests/mm: uffd-unit-tests: use kselftest framework selftests/mm:va_high_addr_switch: use kselftest framework selftests/mm: addatexit()and signal handlers tothp_settingsselftests/mm: rename thp_settings.[ch] to hugepage_settings.[ch] selftests/mm: move HugeTLB helpers tohugepage_settingsselftests/mm:hugepage_settings: use unsigned long indetect_hugetlb_page_sizeselftests/mm:hugepage_settings: add APIs to get and setnr_hugepagesselftests/mm:hugepage_settings: renameget_free_hugepages()selftests/mm:hugepage_settings: add APIs for HugeTLB setup and teardown selftests/mm: moveread_file(),read_num()andwrite_num()tovm_utilselftests/mm:vm_util: add helpers to set and restore shm limits selftests/mm:compaction_test: use HugeTLB helpers ... selftests/mm: cow: add setup of HugeTLB pages
selftests/mm:gup_longterm: add setup of HugeTLB pages selftests/mm:gup_test: add setup of HugeTLB pages
selftests/mm: hmm-tests: add setup of HugeTLB pages
selftests/mm:hugepage_dio: add setup of HugeTLB pages selftests/mm:hugetlb_fault_after_madv: add setup of HugeTLB pages selftests/mm: hugetlb-madvise: add setup of HugeTLB pages selftests/mm:hugetlb_madv_vs_map: add setup of HugeTLB pages selftests/mm: hugetlb-mmap: add setup of HugeTLB pages selftests/mm: hugetlb-mremap: add setup of HugeTLB pages selftests/mm: hugetlb-shm: add setup of HugeTLB pages selftests/mm: hugetlb-soft-online: add setup of HugeTLB pages selftests/mm: hugetlb-vmemmap: add setup of HugeTLB pages selftests/mm: migration: add setup of HugeTLB pages
selftests/mm:pagemap_ioctl: add setup of HugeTLB pages selftests/mm:protection_keys: use library code for HugeTLB setup selftests/mm: thuge-gen: add setup of HugeTLB pages
selftests/mm: uffd-stress: usehugetlb_saveand alloc huge pages selftests/mm: uffd-unit-tests: add setup of HugeTLB pages selftests/mm: uffd-wp-mremap: add setup of HugeTLB pages selftests/mm:va_high_addr_switch: add setup of HugeTLB pages selftests/mm:va_high_addr_switch.sh: drop huge pages setup selftests/mm:run_vmtests.sh: free memory if available memory is low selftests/mm:run_vmtests.sh: drop detection and setup of HugeTLBtools/testing/selftests/mm/.gitignore | 4 + tools/testing/selftests/mm/Makefile | 13 +- tools/testing/selftests/mm/compaction_test.c | 113 +----- tools/testing/selftests/mm/cow.c | 29 +- .../selftests/mm/folio_split_race_test.c | 2 +- tools/testing/selftests/mm/guard-regions.c | 2 +- tools/testing/selftests/mm/gup_longterm.c | 3 +- tools/testing/selftests/mm/gup_test.c | 12 + tools/testing/selftests/mm/hmm-tests.c | 24 +- tools/testing/selftests/mm/hugepage-mmap.c | 78 ---- .../{thp_settings.c => hugepage_settings.c} | 286 +++++++++++-- .../{thp_settings.h => hugepage_settings.h} | 75 +++- tools/testing/selftests/mm/hugetlb-madvise.c | 209 ++++------ tools/testing/selftests/mm/hugetlb-mmap.c | 145 +++++++ .../{hugepage-mremap.c => hugetlb-mremap.c} | 13 +- .../selftests/mm/hugetlb-read-hwpoison.c | 123 +++--- .../mm/{hugepage-shm.c => hugetlb-shm.c} | 64 +-- .../selftests/mm/hugetlb-soft-offline.c | 45 +-- .../{hugepage-vmemmap.c => hugetlb-vmemmap.c} | 46 +-- tools/testing/selftests/mm/hugetlb_dio.c | 15 +- .../selftests/mm/hugetlb_fault_after_madv.c | 7 +- .../selftests/mm/hugetlb_madv_vs_map.c | 22 +- tools/testing/selftests/mm/khugepaged.c | 380 ++++++++---------- tools/testing/selftests/mm/ksm_tests.c | 182 ++++----- tools/testing/selftests/mm/map_hugetlb.c | 88 ---- tools/testing/selftests/mm/migration.c | 152 ++++--- tools/testing/selftests/mm/pagemap_ioctl.c | 13 +- tools/testing/selftests/mm/pkey-helpers.h | 6 +- .../testing/selftests/mm/prctl_thp_disable.c | 2 +- tools/testing/selftests/mm/protection_keys.c | 130 +++--- tools/testing/selftests/mm/run_vmtests.sh | 177 ++------ tools/testing/selftests/mm/soft-dirty.c | 2 +- .../selftests/mm/split_huge_page_test.c | 2 +- tools/testing/selftests/mm/thuge-gen.c | 80 +--- tools/testing/selftests/mm/transhuge-stress.c | 2 +- tools/testing/selftests/mm/uffd-common.h | 1 + tools/testing/selftests/mm/uffd-stress.c | 44 +- tools/testing/selftests/mm/uffd-unit-tests.c | 110 +++-- tools/testing/selftests/mm/uffd-wp-mremap.c | 12 +- .../selftests/mm/va_high_addr_switch.c | 42 +- .../selftests/mm/va_high_addr_switch.sh | 41 +- tools/testing/selftests/mm/vm_util.c | 133 +++--- tools/testing/selftests/mm/vm_util.h | 15 +- 43 files changed, 1424 insertions(+), 1520 deletions(-)delete mode 100644
tools/testing/selftests/mm/hugepage-mmap.c
renametools/testing/selftests/mm/{thp_settings.c=>hugepage_settings.c} (60%) renametools/testing/selftests/mm/{thp_settings.h=>hugepage_settings.h} (55%) create mode 100644tools/testing/selftests/mm/hugetlb-mmap.c
renametools/testing/selftests/mm/{hugepage-mremap.c=>hugetlb-mremap.c} (94%) renametools/testing/selftests/mm/{hugepage-shm.c=>hugetlb-shm.c} (57%) renametools/testing/selftests/mm/{hugepage-vmemmap.c=>hugetlb-vmemmap.c} (76%) delete mode 100644tools/testing/selftests/mm/map_hugetlb.cbase-commit:
254f49634ee16a731174d2ae34bc50bd5f45e731--
2.53.0