selftests/mm: khugepaged: handle partial write in file_setup_area()
Subsystems (4)
PATCH 1/1 selftests/mm: khugepaged: handle partial write in file_setup_area()
file_setup_area() writes initialized test data into the backing file used for collapse testing, but it ignores the return value from write().
If write() fails or completes only partially, the test continues with incomplete file contents, which can lead to incorrect test results and make failures harder to diagnose.
Handle partial writes by retrying until all data is written and abort the test if write() fails.
--- tools/testing/selftests/mm/khugepaged.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)
tools/testing/selftests/mm/khugepaged.c +16 -2
@@ -369,7 +369,9 @@ static void *file_setup_area(int nr_hpages)int fd;void *p;unsigned long size; - +size_t remaining; +ssize_t ret; +char *buf;unlink (finfo.path);/* Cleanup from previous failed tests */ printf ("Creating %s for collapse%s..." , finfo.path, finfo.type ==VMA_SHMEM ?" (tmpfs)" :"" );
@@ -383,7 +385,19 @@ static void *file_setup_area(int nr_hpages) size = nr_hpages * hpage_pmd_size; p =alloc_mapping (nr_hpages);fill_memory (p,0 , size); -write (fd, p, size); + remaining = size; + buf = p; + +while (remaining >0 ) { + ret =write (fd, buf, remaining); +if (ret <=0 ) { +close (fd); +munmap (p, size); +ksft_exit_fail_msg ("write() failed while preparing test file\n" ); + } + buf += ret; + remaining -= ret; + }close (fd);munmap (p, size);success ("OK" );
Signature · 1 line
2.54.0
David Hildenbrand (Arm)
maintainerQuoted 39 lines · Vineet Agarwal
file_setup_area()writes initialized test data into the backing file used for collapse testing, but it ignores the return value fromwrite().If
write()fails or completes only partially, the test continues with incomplete file contents, which can lead to incorrect test results and make failures harder to diagnose.Handle partial writes by retrying until all data is written and abort the test if
write()fails.Signed-off-by Vineet Agarwal <[email protected]>--- tools/testing/selftests/mm/khugepaged.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)tools/testing/selftests/mm/khugepaged.c +6
-2@@ -369,7 +369,9 @@ static void *file_setup_area(int nr_hpages)int fd;void *p;unsigned long size; - +size_t remaining; +ssize_t ret; +char *buf;unlink (finfo.path);/* Cleanup from previous failed tests */ printf ("Creating %s for collapse%s..." , finfo.path, finfo.type ==VMA_SHMEM ?" (tmpfs)" :"" );@@ -383,7 +385,19 @@ static void *file_setup_area(int nr_hpages) size = nr_hpages * hpage_pmd_size; p =alloc_mapping (nr_hpages);fill_memory (p,0 , size); -write (fd, p, size); + remaining = size; + buf = p; +
tools/testing/selftests/mm/khugepaged.c static void *file_setup_area(int nr_hpages) · line 400
Quoted 10 lines
tools/testing/selftests/mm/khugepaged.c +10
-0+while (remaining >0 ) { + ret =write (fd, buf, remaining); +if (ret <=0 ) { +close (fd); +munmap (p, size); +ksft_exit_fail_msg ("write() failed while preparing test file\n" ); + } + buf += ret; + remaining -= ret; + }
Why are we even using write() instead of just mapping the fd and writing to that area?
(quoted diff) +0 -0
b/tools/testing/selftests/mm/khugepaged.c
index 3fe7ef04ac62..a77f753ccf38 100644
tools/testing/selftests/mm/khugepaged.c +3 -2
@@ -381,9 +381,10 @@ static void *file_setup_area(int nr_hpages) } size = nr_hpages * hpage_pmd_size; - p =alloc_mapping (nr_hpages); + p =mmap (BASE_ADDR , size,PROT_READ |PROT_WRITE ,MAP_SHARED , fd,0 ); +if (p !=BASE_ADDR ) +ksft_exit_fail_msg ("mmap() failed while preparing test file\n" );fill_memory (p,0 , size); -write (fd, p, size);close (fd);munmap (p, size);success ("OK" );
Signature · 3 lines
Cheers, David
Thanks for pointing that out.
I’ll take a closer look at whether mapping the file descriptor directly and initializing the mapped area would be a cleaner approach here instead of relying on write().
If mmap-based initialization works for this test setup, that would indeed avoid the partial write handling entirely. I’ll investigate and send a v2 accordingly.
Thanks,
Vineet
Quoted 84 lines · David Hildenbrand (Arm)
Quoted 11 lines · Vineet Agarwal
file_setup_area()writes initialized test data into the backing file used for collapse testing, but it ignores the return value fromwrite().If
write()fails or completes only partially, the test continues with incomplete file contents, which can lead to incorrect test results and make failures harder to diagnose.Handle partial writes by retrying until all data is written and abort the test if
write()fails.Review
Quoted 6 lines
Signed-off-by Vineet Agarwal <[email protected]>--- tools/testing/selftests/mm/khugepaged.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/tools/testing/selftests/mm/khugepaged.cQuoted 18 lines
index
3fe7ef04ac62..c13f06758750100644tools/testing/selftests/mm/khugepaged.c +3
-1@@ -369,7 +369,9 @@ static void *file_setup_area(int nr_hpages)int fd;void *p;unsigned long size; - +size_t remaining; +ssize_t ret; +char *buf;unlink (finfo.path);/* Cleanup from previous failed tests */ printf ("Creating %s for collapse%s..." , finfo.path, finfo.type ==VMA_SHMEM ?" (tmpfs)" :"" );@@ -383,7 +385,19 @@ static void *file_setup_area(int nr_hpages) size = nr_hpages * hpage_pmd_size; p =alloc_mapping (nr_hpages);fill_memory (p,0 , size);tools/testing/selftests/mm/khugepaged.c static void *file_setup_area(int nr_hpages) · line 396
Quoted 10 lines
tools/testing/selftests/mm/khugepaged.c +9
-1-write (fd, p, size); + remaining = size; + buf = p; + +while (remaining >0 ) { + ret =write (fd, buf, remaining); +if (ret <=0 ) { +close (fd); +munmap (p, size); + ksft_exit_fail_msg("write() failed while preparingtest file\n");
tools/testing/selftests/mm/khugepaged.c static void *file_setup_area(int nr_hpages) · line 400
Quoted 4 lines
tools/testing/selftests/mm/khugepaged.c +4
-0+ } + buf += ret; + remaining -= ret; + }Why are we even using
write()instead of just mapping the fd and writing to that
area?tools/testing/selftests/mm/khugepaged.c +3
-2@@ -381,9 +381,10 @@ static void *file_setup_area(int nr_hpages) } size = nr_hpages * hpage_pmd_size; - p =alloc_mapping (nr_hpages); + p =mmap (BASE_ADDR , size,PROT_READ |PROT_WRITE ,MAP_SHARED , fd,0 ); +if (p !=BASE_ADDR ) +ksft_exit_fail_msg ("mmap() failed while preparing test file\n");fill_memory (p,0 , size); -write (fd, p, size);close (fd);munmap(p, size); success("OK");--
Cheers,David
PATCH v2 1/1 selftests/mm: khugepaged: initialize file contents via mmap
In reply to David Hildenbrand (Arm)file_setup_area() currently allocates anonymous memory, fills it, and writes it into the backing file used for collapse testing.
Instead of copying data through write(), map the file directly with MAP_SHARED after resizing it with ftruncate(), initialize the mapped area in place, and sync it with msync().
This simplifies the setup path and avoids the need for explicit partial write handling.
--- tools/testing/selftests/mm/khugepaged.c | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-)
tools/testing/selftests/mm/khugepaged.c +21 -4
@@ -369,7 +369,6 @@ static void *file_setup_area(int nr_hpages)int fd;void *p;unsigned long size; -unlink (finfo.path);/* Cleanup from previous failed tests */ printf ("Creating %s for collapse%s..." , finfo.path, finfo.type ==VMA_SHMEM ?" (tmpfs)" :"" );
@@ -381,11 +380,29 @@ static void *file_setup_area(int nr_hpages) } size = nr_hpages * hpage_pmd_size; - p =alloc_mapping (nr_hpages); +if (ftruncate (fd, size)) { +perror ("ftruncate()" ); +close (fd); +exit (EXIT_FAILURE ); + } + p =mmap (BASE_ADDR , size,PROT_READ |PROT_WRITE , +MAP_SHARED , fd,0 ); +if (p ==MAP_FAILED || p !=BASE_ADDR ) { +perror ("mmap()" ); +close (fd); +exit (EXIT_FAILURE ); + }fill_memory (p,0 , size); -write (fd, p, size); -close (fd); + +if (msync (p, size,MS_SYNC )) { +perror ("msync()" ); +munmap (p, size); +close (fd); +exit (EXIT_FAILURE ); + } +munmap (p, size); +close (fd);success ("OK" );printf ("Opening %s read only for collapse..." , finfo.path);
Signature · 1 line
2.54.0
David Hildenbrand (Arm)
maintainerRe: [PATCH v2] selftests/mm: khugepaged: initialize file contents via mmap
Quoted 28 lines · Vineet Agarwal
file_setup_area()currently allocates anonymous memory, fills it, and writes it into the backing file used for collapse testing.Instead of copying data through
write(), map the file directly withMAP_SHAREDafter resizing it withftruncate(), initialize the mapped area in place, and sync it withmsync().This simplifies the setup path and avoids the need for explicit partial write handling.
Signed-off-by Vineet Agarwal <[email protected]>--- tools/testing/selftests/mm/khugepaged.c | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-)tools/testing/selftests/mm/khugepaged.c +0
-1@@ -369,7 +369,6 @@ static void *file_setup_area(int nr_hpages)int fd;void *p;unsigned long size; -unlink (finfo.path);/* Cleanup from previous failed tests */ printf ("Creating %s for collapse%s..." , finfo.path, finfo.type ==VMA_SHMEM ?" (tmpfs)" :"" );
tools/testing/selftests/mm/khugepaged.c static void *file_setup_area(int nr_hpages) · line 387
Quoted 9 lines
tools/testing/selftests/mm/khugepaged.c +5
-1@@ -381,11 +380,29 @@ static void *file_setup_area(int nr_hpages) } size = nr_hpages * hpage_pmd_size; - p =alloc_mapping (nr_hpages); +if (ftruncate (fd, size)) { +perror ("ftruncate()" ); +close (fd); +exit (EXIT_FAILURE ); + }
Why is that required? We write all pages.
Quoted 6 lines
tools/testing/selftests/mm/khugepaged.c +6
-0+ p =mmap (BASE_ADDR , size,PROT_READ |PROT_WRITE , +MAP_SHARED , fd,0 ); +if (p ==MAP_FAILED || p !=BASE_ADDR ) { +perror ("mmap()" ); +close (fd); +exit (EXIT_FAILURE );
tools/testing/selftests/mm/khugepaged.c static void *file_setup_area(int nr_hpages) · line 401
Quoted 10 lines
tools/testing/selftests/mm/khugepaged.c +7
-2+ }fill_memory (p,0 , size); -write (fd, p, size); -close (fd); + +if (msync (p, size,MS_SYNC )) { +perror ("msync()" ); +munmap (p, size); +close (fd); +exit (EXIT_FAILURE );
Why is that required?
Signature · 3 lines
Cheers, David
Re: [PATCH v2] selftests/mm: khugepaged: initialize file contents via mmap
ftruncate() is needed before mmap(), since the file is created with O_TRUNC and initially has size 0. Without resizing it first, mapping and accessing the full range can go beyond EOF.
You are right about msync() though — since all pages are written through the shared mapping and the test later drops page cache explicitly, an explicit msync() is unnecessary here.
I'll remove msync() and send a v3.
Thanks,
Vineet
Quoted 66 lines · David Hildenbrand (Arm)
Quoted 10 lines · Vineet Agarwal
file_setup_area()currently allocates anonymous memory, fills it, and writes it into the backing file used for collapse testing.Instead of copying data through
write(), map the file directly withMAP_SHAREDafter resizing it withftruncate(), initialize the mapped area in place, and sync it withmsync().This simplifies the setup path and avoids the need for explicit partial write handling.
Review
Quoted 6 lines
Signed-off-by Vineet Agarwal <[email protected]>--- tools/testing/selftests/mm/khugepaged.c | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-)(quoted diff) +0
-0
b/tools/testing/selftests/mm/khugepaged.cQuoted 12 lines
index
3fe7ef04ac62..f6e171e1da4f100644tools/testing/selftests/mm/khugepaged.c +0
-1@@ -369,7 +369,6 @@ static void *file_setup_area(int nr_hpages)int fd;void *p;unsigned long size; -unlink (finfo.path);/* Cleanup from previous failed tests */ printf ("Creating %s for collapse%s..." , finfo.path, finfo.type ==VMA_SHMEM ?" (tmpfs)" :"" );tools/testing/selftests/mm/khugepaged.c static void *file_setup_area(int nr_hpages) · line 387
Quoted 9 lines
tools/testing/selftests/mm/khugepaged.c +5
-1@@ -381,11 +380,29 @@ static void *file_setup_area(int nr_hpages) } size = nr_hpages * hpage_pmd_size; - p =alloc_mapping (nr_hpages); +if (ftruncate (fd, size)) { +perror ("ftruncate()" ); +close (fd); +exit (EXIT_FAILURE ); + }Why is that required? We write all pages.
Quoted 6 lines
tools/testing/selftests/mm/khugepaged.c +6
-0+ p =mmap (BASE_ADDR , size,PROT_READ |PROT_WRITE , +MAP_SHARED , fd,0 ); +if (p ==MAP_FAILED || p !=BASE_ADDR ) { +perror ("mmap()" ); +close (fd); +exit (EXIT_FAILURE );tools/testing/selftests/mm/khugepaged.c static void *file_setup_area(int nr_hpages) · line 401
Quoted 10 lines
tools/testing/selftests/mm/khugepaged.c +7
-2+ }fill_memory (p,0 , size); -write (fd, p, size); -close (fd); + +if (msync (p, size,MS_SYNC )) { +perror ("msync()" ); +munmap (p, size); +close (fd); +exit (EXIT_FAILURE );Why is that required?
--
Cheers,David
David Hildenbrand (Arm)
maintainerRe: [PATCH v2] selftests/mm: khugepaged: initialize file contents via mmap
Quoted 3 lines · Vineet Agarwal
ftruncate()is needed beforemmap(), since the file is created withO_TRUNCand initially has size 0. Without resizing it first, mapping and accessing the full range can go beyond EOF.
Ah, that makes sense, worth spelling it out. Indeed, we otherwise get
$ sudo ./khugepaged madvise:file . Save THP and khugepaged settings... OK Allocate huge page on fault... OK Split huge PMD on MADV_DONTNEED... OK
Run test: collapse_full (madvise:file) Creating ./collapse_test_file for collapse...Bus error
Signature · 3 lines
Cheers, David
PATCH v3 1/1 selftests/mm: khugepaged: initialize file contents via mmap
In reply to David Hildenbrand (Arm)file_setup_area() currently allocates anonymous memory, fills it, and writes it into the backing file used for collapse testing.
Instead of copying data through write(), resize the file with ftruncate(), map it directly with MAP_SHARED, and initialize the mapped area in place.
This simplifies the setup path and avoids the need for explicit partial write handling.
--- tools/testing/selftests/mm/khugepaged.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-)
tools/testing/selftests/mm/khugepaged.c +13 -4
@@ -369,7 +369,6 @@ static void *file_setup_area(int nr_hpages)int fd;void *p;unsigned long size; -unlink (finfo.path);/* Cleanup from previous failed tests */ printf ("Creating %s for collapse%s..." , finfo.path, finfo.type ==VMA_SHMEM ?" (tmpfs)" :"" );
@@ -381,11 +380,21 @@ static void *file_setup_area(int nr_hpages) } size = nr_hpages * hpage_pmd_size; - p =alloc_mapping (nr_hpages); +if (ftruncate (fd, size)) { +perror ("ftruncate()" ); +close (fd); +exit (EXIT_FAILURE ); + } + p =mmap (BASE_ADDR , size,PROT_READ |PROT_WRITE , +MAP_SHARED , fd,0 ); +if (p ==MAP_FAILED || p !=BASE_ADDR ) { +perror ("mmap()" ); +close (fd); +exit (EXIT_FAILURE ); + }fill_memory (p,0 , size); -write (fd, p, size); -close (fd);munmap (p, size); +close (fd);success ("OK" );printf ("Opening %s read only for collapse..." , finfo.path);
Signature · 1 line
2.54.0
David Hildenbrand (Arm)
maintainerRe: [PATCH v3] selftests/mm: khugepaged: initialize file contents via mmap
Quoted 9 lines · Vineet Agarwal
file_setup_area()currently allocates anonymous memory, fills it, and writes it into the backing file used for collapse testing.Instead of copying data through
write(), resize the file withftruncate(), map it directly withMAP_SHARED, and initialize the mapped area in place.This simplifies the setup path and avoids the need for explicit partial write handling.
Please
a) Wait a bit longer before you resend.
b) Don't send as reply to earlier revisions.
Quoted 11 lines
Signed-off-by Vineet Agarwal <[email protected]>--- tools/testing/selftests/mm/khugepaged.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-)tools/testing/selftests/mm/khugepaged.c +0
-0
tools/testing/selftests/mm/khugepaged.c static void *file_setup_area(int nr_hpages) · line 372
Quoted 4 lines
tools/testing/selftests/mm/khugepaged.c +0
-1@@ -369,7 +369,6 @@ static void *file_setup_area(int nr_hpages)int fd;void *p;unsigned long size; -
Unrelated change.
Quoted 15 lines
tools/testing/selftests/mm/khugepaged.c +7
-1unlink (finfo.path);/* Cleanup from previous failed tests */ printf ("Creating %s for collapse%s..." , finfo.path, finfo.type ==VMA_SHMEM ?" (tmpfs)" :"" );@@ -381,11 +380,21 @@ static void *file_setup_area(int nr_hpages) } size = nr_hpages * hpage_pmd_size; - p =alloc_mapping (nr_hpages); +if (ftruncate (fd, size)) { +perror ("ftruncate()" ); +close (fd); +exit (EXIT_FAILURE ); + } + p =mmap (BASE_ADDR , size,PROT_READ |PROT_WRITE , +MAP_SHARED , fd,0 );
tools/testing/selftests/mm/khugepaged.c static void *file_setup_area(int nr_hpages) · line 397
Quoted 10 lines
tools/testing/selftests/mm/khugepaged.c +6
-2+if (p ==MAP_FAILED || p !=BASE_ADDR ) { +perror ("mmap()" ); +close (fd); +exit (EXIT_FAILURE ); + }fill_memory (p,0 , size); -write (fd, p, size); -close (fd);munmap (p, size); +close (fd);
That change is not strictly required. We can munmap() after close().
Apart from that LGTM.
Signature · 3 lines
Cheers, David