Migrating the libhugetlbfs/testcases/stack_grow_into_huge.c test
PATCH v2 1/1 Migrating the libhugetlbfs/testcases/stack_grow_into_huge.c test
Test Description:On PowerPC, the address space is divided into segments. These segments can contain either huge pages or normal pages, but not both.
All segments are initially set up to map normal pages. When a huge page mapping is created within a set of empty segments, they are "enabled" for huge pages at that time. Once enabled for huge pages, they can not be used again for normal pages for the remaining lifetime of the process.
If the segment immediately preceeding the segment containing the stack is converted to huge pages and the stack is made to grow into the this preceeding segment, some kernels may attempt to map normal pages into the huge page-only segment -- resulting in bugs.
---
v2:
Below are the requested changes in version 2:
1. Fixed "make check warnings"
2. Removed explicit calls to exit(1)
3.[To get the information about the child state], adapted the "tst_strstatus" implementation.
--- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/hugemmap34.c | 130 ++++++++++++++++++ 3 files changed, 132 insertions(+) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c
runtest/hugetlb +1 -0
@@ -35,6 +35,7 @@ hugemmap29 hugemmap29 hugemmap30 hugemmap30 hugemmap31 hugemmap31 hugemmap32 hugemmap32 +hugemmap34 hugemmap34 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -m
testcases/kernel/mem/.gitignore +1 -0
@@ -34,6 +34,7 @@ /hugetlb/hugemmap/hugemmap30 /hugetlb/hugemmap/hugemmap31 /hugetlb/hugemmap/hugemmap32 +/hugetlb/hugemmap/hugemmap34 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +130 -0
130 lines not on this page · raw message
Signature · 5 lines
2.39.3 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Hello,
Quoted 70 lines · Samir Mulani
Test Description:On PowerPC, the address space is divided into segments. These segments can contain either huge pages or normal pages, but not both.
All segments are initially set up to map normal pages. When a huge page mapping is created within a set of empty segments, they are "enabled" for huge pages at that time. Once enabled for huge pages, they can not be used again for normal pages for the remaining lifetime of the process.If the segment immediately preceeding the segment containing the stack is converted to huge pages and the stack is made to grow into the this preceeding segment, some kernels may attempt to map normal pages into the huge page-only segment -- resulting in bugs.
Signed-off-by Samir Mulani <[email protected]>---
v2:
Below are the requested changes in version 2:
1. Fixed "make check warnings"
2. Removed explicit calls to exit(1)
3.[To get the information about the child state], adapted the "tst_strstatus" implementation.--- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/hugemmap34.c | 130 ++++++++++++++++++ 3 files changed, 132 insertions(+) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.cruntest/hugetlb +1
-0@@ -35,6 +35,7 @@ hugemmap29 hugemmap29 hugemmap30 hugemmap30 hugemmap31 hugemmap31 hugemmap32 hugemmap32 +hugemmap34 hugemmap34 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -mtestcases/kernel/mem/.gitignore +1
-0@@ -34,6 +34,7 @@ /hugetlb/hugemmap/hugemmap30 /hugetlb/hugemmap/hugemmap31 /hugetlb/hugemmap/hugemmap32 +/hugetlb/hugemmap/hugemmap34 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +12
-0@@ -0,0 +1,130 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2005-2006 IBM Corporation +* Author: David Gibson & Adam Litke +*/ + +/*\ +* [Description] +* +* Test Name: stack_grow_into_huge +* +* On PowerPC, the address space is divided into segments. These
Quoted 1 line
segments can
So does this test work on other architectures?
There don't seem to be any ifdefs to prevent it running on them. We may need to limit it to PowerPC and call TST_TEST_TCONF otherwise.
Quoted 53 lines
+ * contain either huge pages or normal pages, but not both. All segments are
+ * initially set up to map normal pages. When a huge page mapping is created
+ * within a set of empty segments, they are "enabled" for huge pages at that
+ * time. Once enabled for huge pages, they can not be used again for normal
+ * pages for the remaining lifetime of the process.
+ *
+ * If the segment immediately preceeding the segment containing the stack is
+ * converted to huge pages and the stack is made to grow into the this
+ * preceeding segment, some kernels may attempt to map normal pages into the
+ * huge page-only segment -- resulting in bugs.
+ */testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +42
-0+ +#include "hugetlb.h" + +#ifdef __LP64__ +#define STACK_ALLOCATION_SIZE (256*1024*1024) +#else +#define STACK_ALLOCATION_SIZE (16*1024*1024) +#endif +#define PALIGN (p, a) ((void *)LTP_ALIGN((unsigned long)(p), (a))) +#define MNTPOINT "hugetlbfs/" +static int fd =-1 ; +static unsigned long long hpage_size; +static int page_size; + + +void do_child (void *stop_address) +{ +struct rlimit r; +volatile int *x; + +/* corefile from this process is not interesting and limiting +* its size can save a lot of time. '1' is a special value, +* that will also abort dumping via pipe, which by default +* sets limit to RLIM_INFINITY. +*/ + r.rlim_cur =1 ; + r.rlim_max =1 ; +SAFE_SETRLIMIT (RLIMIT_CORE , &r); + +do { + x =alloca (STACK_ALLOCATION_SIZE ); + *x =1 ; + }while ((void *)x >= stop_address); +} + +static void run_test (void ) +{ +int pid, status; +void *stack_address, *mmap_address, *heap_address; + + stack_address =alloca (0 ); + heap_address =sbrk (0 );
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c line 75
Quoted 10 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +10
-0+ +/* +* paranoia: start mapping two hugepages below the start of the stack, +* in case the alignment would cause us to map over something if we +* only used a gap of one hugepage. +*/ + mmap_address =PALIGN (stack_address -2 * hpage_size, hpage_size); + do { + SAFE_MMAP(mmap_address, hpage_size, PROT_READ|PROT_WRITE, + MAP_FIXED|MAP_SHARED, fd, 0);
I think this will fail if CGroup limits are in effect as well. It could also hit other limits, but probably not on systems where hugepages are important.
In any case I think if mmap fails with ENOMEM (at least), then it should exit with TCONF.
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c line 80
Quoted 5 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +5
-0+ mmap_address -= hpage_size; +/* +* if we get all the way down to the heap, stop trying +*/ + }while (mmap_address <= heap_address);
Won't this clobber the heap and any other mappings the LTP libs or compiler instrumentations have (if any)?
Could we use MAP_FIXED_NOREPLACE instead?
Looking at the description we only need a single mapping to set the segment to a hugepage segment. It's not clear why such a large mapping is needed?
Quoted 38 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +38
-0+ pid =SAFE_FORK (); +if (pid ==0 ) +do_child (mmap_address); + +SAFE_WAITPID (pid, &status,0 ); +if (WIFSIGNALED (status) &&WTERMSIG (status) ==SIGSEGV ) +tst_res (TPASS ,"Child killed by %s as expected" ,tst_strsig (SIGSEGV )); +else +tst_res (TFAIL ,"Child: %s" ,tst_strstatus (status)); +} + +void setup (void ) +{ +struct rlimit r; + + page_size =getpagesize (); + hpage_size =tst_get_hugepage_size (); +/* +* Setting the stack size to unlimited. +*/ + r.rlim_cur =RLIM_INFINITY ; + r.rlim_max =RLIM_INFINITY ; +SAFE_SETRLIMIT (RLIMIT_STACK , &r); +SAFE_GETRLIMIT (RLIMIT_STACK , &r); +if (r.rlim_cur !=RLIM_INFINITY ) +tst_brk (TCONF ,"Stack rlimit must be 'unlimited'" ); + fd =tst_creat_unlinked (MNTPOINT ,0 ); +} + +void cleanup (void ) +{ +if (fd >0 ) +SAFE_CLOSE (fd); +} + +static struct tst_test test = { + .tags = (struct tst_tag []) { + {"linux-git" ,"0d59a01bc461" },
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c line 128
Quoted 10 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +10
-0+ {} + }, + .needs_root = 1, + .mntpoint = MNTPOINT, + .needs_hugetlbfs = 1, + .needs_tmpdir = 1, + .setup = setup, + .cleanup = cleanup, + .test_all = run_test, + .hugepages = {2, TST_NEEDS},
It looks like the test could map a lot more than 2 pages.
Quoted 4 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +2
-0+ .forks_child = 1, +};Signature · 1 line
2.39.3
Signature · 5 lines
Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp
Quoted 161 lines · Richard Palethorpe
Hello,
Quoted 70 lines · Samir Mulani
Test Description:On PowerPC, the address space is divided into segments. These segments can contain either huge pages or normal pages, but not both.
All segments are initially set up to map normal pages. When a huge page mapping is created within a set of empty segments, they are "enabled" for huge pages at that time. Once enabled for huge pages, they can not be used again for normal pages for the remaining lifetime of the process.If the segment immediately preceeding the segment containing the stack is converted to huge pages and the stack is made to grow into the this preceeding segment, some kernels may attempt to map normal pages into the huge page-only segment -- resulting in bugs.
Signed-off-by Samir Mulani <[email protected]>---
v2:
Below are the requested changes in version 2:
1. Fixed "make check warnings"
2. Removed explicit calls to exit(1)
3.[To get the information about the child state], adapted the "tst_strstatus" implementation.--- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/hugemmap34.c | 130 ++++++++++++++++++ 3 files changed, 132 insertions(+) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.cruntest/hugetlb +1
-0@@ -35,6 +35,7 @@ hugemmap29 hugemmap29 hugemmap30 hugemmap30 hugemmap31 hugemmap31 hugemmap32 hugemmap32 +hugemmap34 hugemmap34 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -mtestcases/kernel/mem/.gitignore +1
-0@@ -34,6 +34,7 @@ /hugetlb/hugemmap/hugemmap30 /hugetlb/hugemmap/hugemmap31 /hugetlb/hugemmap/hugemmap32 +/hugetlb/hugemmap/hugemmap34 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +12
-0@@ -0,0 +1,130 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2005-2006 IBM Corporation +* Author: David Gibson & Adam Litke +*/ + +/*\ +* [Description] +* +* Test Name: stack_grow_into_huge +* +* On PowerPC, the address space is divided into segments. These Quoted 1 line
segments can
So does this test work on other architectures?
There don't seem to be any ifdefs to prevent it running on them. We may need to limit it to PowerPC and call
TST_TEST_TCONFotherwise.Quoted 53 lines
+ * contain either huge pages or normal pages, but not both. All segments are
+ * initially set up to map normal pages. When a huge page mapping is created
+ * within a set of empty segments, they are "enabled" for huge pages at that
+ * time. Once enabled for huge pages, they can not be used again for normal
+ * pages for the remaining lifetime of the process.
+ *
+ * If the segment immediately preceeding the segment containing the stack is
+ * converted to huge pages and the stack is made to grow into the this
+ * preceeding segment, some kernels may attempt to map normal pages into the
+ * huge page-only segment -- resulting in bugs.
+ */testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +42
-0+ +#include "hugetlb.h" + +#ifdef __LP64__ +#define STACK_ALLOCATION_SIZE (256*1024*1024) +#else +#define STACK_ALLOCATION_SIZE (16*1024*1024) +#endif +#define PALIGN (p, a) ((void *)LTP_ALIGN((unsigned long)(p), (a))) +#define MNTPOINT "hugetlbfs/" +static int fd =-1 ; +static unsigned long long hpage_size; +static int page_size; + + +void do_child (void *stop_address) +{ +struct rlimit r; +volatile int *x; + +/* corefile from this process is not interesting and limiting +* its size can save a lot of time. '1' is a special value, +* that will also abort dumping via pipe, which by default +* sets limit to RLIM_INFINITY. +*/ + r.rlim_cur =1 ; + r.rlim_max =1 ; +SAFE_SETRLIMIT (RLIMIT_CORE , &r); + +do { + x =alloca (STACK_ALLOCATION_SIZE ); + *x =1 ; + }while ((void *)x >= stop_address); +} + +static void run_test (void ) +{ +int pid, status; +void *stack_address, *mmap_address, *heap_address; + + stack_address =alloca (0 ); + heap_address =sbrk (0 );testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c line 75
Quoted 10 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +10
-0+ +/* +* paranoia: start mapping two hugepages below the start of the stack, +* in case the alignment would cause us to map over something if we +* only used a gap of one hugepage. +*/ + mmap_address =PALIGN (stack_address -2 * hpage_size, hpage_size); + do { + SAFE_MMAP(mmap_address, hpage_size, PROT_READ|PROT_WRITE, + MAP_FIXED|MAP_SHARED, fd, 0);I think this will fail if CGroup limits are in effect as well. It could also hit other limits, but probably not on systems where hugepages are important.
In any case I think if mmap fails with ENOMEM (at least), then it should exit with TCONF.
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c line 80
Quoted 5 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +5
-0+ mmap_address -= hpage_size; +/* +* if we get all the way down to the heap, stop trying +*/ + }while (mmap_address <= heap_address);Won't this clobber the heap and any other mappings the LTP libs or compiler instrumentations have (if any)?
Quoted 5 lines
Could we use
MAP_FIXED_NOREPLACEinstead?Looking at the description we only need a single mapping to set the segment to a hugepage segment. It's not clear why such a large mapping is needed?
I reread the V1 patch and it seems like the original intent is to keep calling mmap *until it succeeds once* then exit the loop.
So you can't use SAFE_MMAP in this case.
Signature · 5 lines
Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp