Introduce new field .hptype for reserving gigantic page
v3 cover Introduce new field .hptype for reserving gigantic page
In the beginning, I was just thinking of using specified .hpsize by users to reserve gigantic or default huge pages. But after thinking over and discussing with Richard, we both think that it might be easy to get typos and limit the test case to run on more potential platforms.
Here change the field to .hptype with two enum TST_HUGE and TST_GIGANITC, as the name suggests, if people want to reserve variant page size in their case, this field could be defined.
But considering there are many different sizes and it could be configured by linux users, we don't assume what the size of the gigantic page is, it all depends on the system set, for instance, an aarch64 has four types:
hugepages-1048576kB, hugepages-2048kB,
hugepages-32768kB, hugepages-64kBLTP library will pick up the largest one as gigantic page. For a system without configuring the 1GB or similar large size, it also gives a chance to test on default hpage size. Compared with just TCONF on no-gigantic page, this is adding test coverage.
(To Richard: the reason why I didn't go to use the smallest and second
smallest size is because, platform like aarch64 typically configure many medium sizes for user, but the default huge is 2MB and gigantic is 1GB, that doesn't fit to most popular scenarios. But to x86_64, situations becomes common and easy, they only has two: 2MB, 1GB, this method could handle the case smoothly.)
Note: Patchset test passed on the RHEL8.9 and RHEL9.3 (x86_64/aarch64).
V2 --> V3
* introduce new field .hptype
* add enum tst_hp_type { TST_HUGE, TST_GIGANTIC }
* drop the is_hugetlb_gigantic function
* just choose the largest huge pagesize as gigantic
Li Wang (3):
lib: add support for kinds of hpsize reservation
hugemmap32: improvement test
hugemmap34: Test to detect bug with migrating gigantic pages
doc/C-Test-API.asciidoc | 42 +++++- include/tst_hugepage.h | 11 ++ lib/tst_hugepage.c | 63 ++++++++- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/hugemmap32.c | 59 +------- .../kernel/mem/hugetlb/hugemmap/hugemmap34.c | 129 ++++++++++++++++++ 7 files changed, 240 insertions(+), 66 deletions(-) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c
Signature · 5 lines
2.40.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 1/3 lib: add support for kinds of hpsize reservation
Typically when we make use of huge page via LTP library, .hugepages choose the default hugepage size, but this can not satisfy all scenarios.
So this patch introduces applying a specified types of hugepage for user.
There is nothing that needs to change for the existing test cases which already using .hugepages, it only needs to fill one more field in the structure of .hugepages if a different type (GIGANTIC or HUGE) is required.
e.g.
static struct tst_test test = {
.needs_root = 1,
...
.hugepages = {2, TST_NEEDS, TST_GIGANTIC},
};--- doc/C-Test-API.asciidoc | 42 +++++++++++++++++++++++++-- include/tst_hugepage.h | 11 +++++++ lib/tst_hugepage.c | 63 ++++++++++++++++++++++++++++++++++++----- 3 files changed, 107 insertions(+), 9 deletions(-)
doc/C-Test-API.asciidoc +40 -2
@@ -2034,9 +2034,13 @@ For full documentation see the comments in 'include/tst_fuzzy_sync.h'. ~~~~~~~~~~~~~~~~~~~~~~~~ Many of the LTP tests need to use hugepage in their testing, this allows the -test can reserve hugepages from system via '.hugepages = {xx, TST_REQUEST}'. +test can reserve specify size of hugepages from system via: + '.hugepages = {xx, TST_REQUEST, TST_HUGE}' or, + '.hugepages = {xx, TST_NEEDS, TST_GIGANTIC}'. -We achieved two policies for reserving hugepages: +xx: This is used to set how many pages we wanted. + +Two policies for reserving hugepage: TST_REQUEST: It will try the best to reserve available huge pages and return the number
@@ -2049,6 +2053,17 @@ TST_NEEDS: use these specified numbers correctly. Otherwise, test exits with TCONF if the attempt to reserve hugepages fails or reserves less than requested. +Two types of the reserved hugepage (optional field): + +TST_HUGE: + It means target for reserve the default hugepage size (e.g. 2MB on x86_64). + And, if nothing fills in this field LTP also chooses the default hugepage + size to reserve. i.e. + '.hugepages = {xx, TST_REQUEST}' == '.hugepages = {xx, TST_REQUEST, TST_HUGE}' + +TST_GIGANTIC: + It means target for reserve the largest hugepage size (e.g. 1GB on x86_64) + With success test stores the reserved hugepage number in 'tst_hugepages'. For system without hugetlb supporting, variable 'tst_hugepages' will be set to 0. If the hugepage number needs to be set to 0 on supported hugetlb system, please
@@ -2103,6 +2118,29 @@ struct tst_test test = { }; ------------------------------------------------------------------------------- +or, + +[source,c] +------------------------------------------------------------------------------- +#include "tst_test.h" + +static void run(void) +{ + ... +} + +struct tst_test test = { + .test_all = run, + /* + * Specify gigantic page sizes reserved automatically in the library + * $ echo 2 > /sys/kernel/mm//hugepages/hugepages-1048576kB/nr_hugepages + * Do check if 2 hpages are reserved correctly in there. + */ + .hugepages = {2, TST_NEEDS, TST_GIGANTIC}, + ... +}; +------------------------------------------------------------------------------- + 1.35 Checking for required commands ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/tst_hugepage.h +11 -0
@@ -24,9 +24,15 @@ enum tst_hp_policy { TST_NEEDS, }; +enum tst_hp_type { +TST_HUGE , +TST_GIGANTIC , +}; +struct tst_hugepage {const unsigned long number;enum tst_hp_policy policy; +enum tst_hp_type hptype; };/*
@@ -34,6 +40,11 @@ struct tst_hugepage {*/ size_t tst_get_hugepage_size (void ); +/* +* Get the largest hugepage (gigantic) size. Returns 0 if hugepages are not supported. +*/ +size_t tst_get_gigantic_size (void ); +/* * Try the best to request a specified number of huge pages from system, * it will store the reserved hpage number in tst_hugepages.
lib/tst_hugepage.c +56 -7
112 lines not on this page · raw message
Signature · 5 lines
2.40.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/3] lib: add support for kinds of hpsize reservation
Hi Li,
Quoted 9 lines
Typically when we make use of huge page via LTP library, .hugepages choose the default hugepage size, but this can not satisfy all scenarios.
So this patch introduces applying a specified types of hugepage for user.
There is nothing that needs to change for the existing test cases which already using .hugepages, it only needs to fill one more field in the structure of .hugepages if a different type (GIGANTIC or HUGE) is required.
Quoted 7 lines
e.g.
static struct tst_test test = { .needs_root = 1, ... .hugepages = {2, TST_NEEDS, TST_GIGANTIC}, };
+1
Algorithm looks good to me.
It would be nice to add a new test or to modify at least one of the library tests to use some of the new functionality.
Below only very minor formatting notes.
Quoted 12 lines
Signed-off-by Li Wang <[email protected]>--- doc/C-Test-API.asciidoc | 42 +++++++++++++++++++++++++-- include/tst_hugepage.h | 11 +++++++ lib/tst_hugepage.c | 63 ++++++++++++++++++++++++++++++++++++----- 3 files changed, 107 insertions(+), 9 deletions(-)doc/C-Test-API.asciidoc +0
-0
doc/C-Test-API.asciidoc For full documentation see the comments in 'include/tst_fuzzy_sync.h'. · line 2039
Quoted 7 lines
doc/C-Test-API.asciidoc +3
-1@@ -2034,9 +2034,13 @@ For full documentation see the comments in 'include/tst_fuzzy_sync.h'. ~~~~~~~~~~~~~~~~~~~~~~~~ Many of the LTP tests need to use hugepage in their testing, this allows the -test can reserve hugepages from system via '.hugepages = {xx, TST_REQUEST}'. +test can reserve specify size of hugepages from system via: + '.hugepages = {xx, TST_REQUEST, TST_HUGE}' or, + '.hugepages = {xx, TST_NEEDS, TST_GIGANTIC}'.
very nit: this formats in wiki as inline. If you meant to have the previous following showing also as a list, it would have to be:
test can reserve specify size of hugepages from system via:
* '.hugepages = {xx, TST_REQUEST, TST_HUGE}' or,
* '.hugepages = {xx, TST_NEEDS, TST_GIGANTIC}'.
very nit: maybe N instead of XX?
Quoted 11 lines
doc/C-Test-API.asciidoc +3
-1-We achieved two policies for reserving hugepages: +xx: This is used to set how many pages we wanted. + +Two policies for reserving hugepage: TST_REQUEST: It will try the best to reserve available huge pages and return the number@@ -2049,6 +2053,17 @@ TST_NEEDS: use these specified numbers correctly. Otherwise, test exits with TCONF if the attempt to reserve hugepages fails or reserves less than requested.
doc/C-Test-API.asciidoc TST_NEEDS: · line 2065
Quoted 10 lines
doc/C-Test-API.asciidoc +10
-0+Two types of the reserved hugepage (optional field): + +TST_HUGE: + It means target for reserve the default hugepage size (e.g. 2MB on x86_64). + And, if nothing fills in this field LTP also chooses the default hugepage + size to reserve. i.e. + '.hugepages = {xx, TST_REQUEST}' == '.hugepages = {xx, TST_REQUEST, TST_HUGE}' + +TST_GIGANTIC: + It means target for reserve the largest hugepage size (e.g. 1GB on x86_64)
very nit: missing '.' at the end.
Quoted 1 line
doc/C-Test-API.asciidoc +1
-0+
...
Quoted 7 lines
doc/C-Test-API.asciidoc +7
-0+or, + +[source,c] +------------------------------------------------------------------------------- +#include "tst_test.h" + +static void run(void)
doc/C-Test-API.asciidoc struct tst_test test = { · line 2137
Quoted 10 lines
doc/C-Test-API.asciidoc +10
-0+{ + ... +} + +struct tst_test test = { + .test_all = run, + /* + * Specify gigantic page sizes reserved automatically in the library + * $ echo 2 > /sys/kernel/mm//hugepages/hugepages-1048576kB/nr_hugepages + * Do check if 2 hpages are reserved correctly in there.
Maybe my bad English, but this looked to me at first as imperative, e.g.:
"Hey, test author, do check the value!" Maybe something like "Library checks whether 2 hpages is actually set.".
Quoted 37 lines
doc/C-Test-API.asciidoc +6
-0+ */ + .hugepages = {2, TST_NEEDS, TST_GIGANTIC}, + ... +}; +------------------------------------------------------------------------------- + 1.35 Checking for required commands ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~include/tst_hugepage.h +10
-0@@ -24,9 +24,15 @@ enum tst_hp_policy { TST_NEEDS, }; +enum tst_hp_type { +TST_HUGE , +TST_GIGANTIC , +}; +struct tst_hugepage {const unsigned long number;enum tst_hp_policy policy; +enum tst_hp_type hptype; };/* @@ -34,6 +40,11 @@ struct tst_hugepage {*/ size_t tst_get_hugepage_size (void ); +/* +* Get the largest hugepage (gigantic) size. Returns 0 if hugepages are not supported. +*/ +size_t tst_get_gigantic_size (void );
..
Quoted 13 lines
+++
b/lib/tst_hugepage.cinclude/tst_hugepage.h +1
-0@@ -5,6 +5,7 @@#define TST_NO_DEFAULT_MAIN +#include <stdio.h> #include "tst_test.h" #include "tst_hugepage.h" @@ -20,11 +21,35 @@ size_t tst_get_hugepage_size(void)return SAFE_READ_MEMINFO ("Hugepagesize:" ) *1024 ; }
include/tst_hugepage.h size_t tst_get_hugepage_size(void) · line 33
Quoted 10 lines
include/tst_hugepage.h +10
-0+size_t tst_get_gigantic_size (void ) +{ +DIR *dir; +struct dirent *ent; +unsigned long max, g_pgsz; + + max =tst_get_hugepage_size () /1024 ; + +/* +* Scanning the largest hugepage sisze, for example aarch64 configuration:
nit: s/sisze/size/
include/tst_hugepage.h size_t tst_get_hugepage_size(void) · line 45
Quoted 12 lines
include/tst_hugepage.h +12
-0+* hugepages-1048576kB hugepages-32768kB hugepages-2048kB hugepages-64kB +*/ + dir =SAFE_OPENDIR (PATH_HUGEPAGES ); +while ((ent =SAFE_READDIR (dir)) !=NULL ) { +if (sscanf (ent->d_name,"hugepages-%lukB" , &g_pgsz) + && (g_pgsz > max)) + max = g_pgsz; + } + +SAFE_CLOSEDIR (dir); +return max *1024 ; +}
...
The rest LGTM.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 2/3 hugemmap32: improvement test
In reply to Li Wang--- .../kernel/mem/hugetlb/hugemmap/hugemmap32.c | 59 +------------------ 1 file changed, 2 insertions(+), 57 deletions(-)
testcases/kernel/mem/hugetlb/hugemmap/hugemmap32.c +2 -57
@@ -18,67 +18,13 @@#define _GNU_SOURCE#include <dirent.h> -#include <stdio.h> #include "hugetlb.h" -#define PATH_HUGEPAGE "/sys/kernel/mm/hugepages" -#define GIGANTIC_MIN_ORDER 10 - -static int org_g_hpages; -static char g_hpage_path[4096 ]; -static void run_test (void ) { -if (FILE_PRINTF (g_hpage_path,"%d" ,1 )) -tst_brk (TCONF ,"Can't update the gigantic hugepages." ); -SAFE_FILE_PRINTF (g_hpage_path,"%d" ,0 ); - -if (tst_taint_check ()) -tst_res (TFAIL ,"Freeing Gigantic pages resulted in Bad Page State bug." ); -else -tst_res (TPASS ,"Successfully freed the gigantic hugepages" ); -} - -static void setup (void ) -{ -DIR *dir; -struct dirent *ent; -unsigned long hpage_size; - -if (access (PATH_HUGEPAGE ,F_OK )) -tst_brk (TCONF ,"hugetlbfs is not supported" ); - - dir =SAFE_OPENDIR (PATH_HUGEPAGE ); -while ((ent =SAFE_READDIR (dir))) { -if ((sscanf (ent->d_name,"hugepages-%lukB" , &hpage_size) ==1 ) && -is_hugetlb_gigantic (hpage_size *1024 )) { -sprintf (g_hpage_path,"%s/%s/%s" ,PATH_HUGEPAGE , - ent->d_name,"nr_hugepages" ); -break ; - } - } -if (!g_hpage_path[0 ]) -tst_brk (TCONF ,"Gigantic hugepages not supported" ); - -SAFE_CLOSEDIR (dir); - -SAFE_FILE_PRINTF ("/proc/sys/vm/drop_caches" ,"3" ); -SAFE_FILE_PRINTF ("/proc/sys/vm/compact_memory" ,"1" ); - -if (tst_available_mem () < (long long )hpage_size) { - g_hpage_path[0 ] ='\0' ; -tst_brk (TCONF ,"No enough memory for gigantic hugepage reservation" ); - } - -SAFE_FILE_LINES_SCANF (g_hpage_path,"%d" , &org_g_hpages); -} - -static void cleanup (void ) -{ -if (g_hpage_path[0 ]) -SAFE_FILE_PRINTF (g_hpage_path,"%d" , org_g_hpages); +tst_res (TPASS ,"If reserved & freed the gigantic page completely, then regard as pass" ); }static struct tst_test test = {
@@ -88,8 +34,7 @@ static struct tst_test test = { {} }, .needs_root = 1, - .setup = setup, - .cleanup = cleanup, + .hugepages = {1, TST_NEEDS, TST_GIGANTIC}, .test_all = run_test, .taint_check = TST_TAINT_B, };
Signature · 5 lines
2.40.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 2/3] hugemmap32: improvement test
Hi Li,
FYI previous version on Tumbleweed in our CI reported:
hugemmap32.c:41: TPASS: Successfully freed the gigantic hugepages
Now, after check, I get:
tst_hugepage.c:119: TCONF: nr_hugepages = 0, but expect 1. Not enough hugepages for testing.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 3/3 hugemmap34: Test to detect bug with migrating gigantic pages
In reply to Li Wang--- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/hugemmap34.c | 129 ++++++++++++++++++ 3 files changed, 131 insertions(+) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c
runtest/hugetlb +1 -0
@@ -35,6 +35,7 @@ hugemmap29 hugemmap29 hugemmap30 hugemmap30 hugemmap31 hugemmap31 hugemmap32 hugemmap32 +hugemmap34 hugemmap34 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -m
testcases/kernel/mem/.gitignore +1 -0
@@ -34,6 +34,7 @@ /hugetlb/hugemmap/hugemmap30 /hugetlb/hugemmap/hugemmap31 /hugetlb/hugemmap/hugemmap32 +/hugetlb/hugemmap/hugemmap34 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03
testcases/kernel/mem/hugetlb/hugemmap/hugemmap34.c new file +129 -0
129 lines not on this page · raw message
Signature · 5 lines
2.40.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 3/3] hugemmap34: Test to detect bug with migrating gigantic pages
Hi Li,
FYI hugemmap34 fails on x86_64 and s390x various SLES kernels (tested: x86_64 from 4.4 to 6.4, s390x from 4.12. to 6.4):
tst_test.c:1627: TINFO: Timeout per run is 0h 01m 30s hugemmap34.c:85: TBROK: mmap() failed: EINVAL (22)
Removing MAP_HUGE_1GB shows TBROK due ENOMEM.
I suppose low memory resource for this VM.
=> Maybe we need .min_mem_avail = 1024?
+ define the value and reuse also here (so that we don't forget to update on other place if later changed):
hugetlbsize = 1 * 1024 * 1024 * 1024u;
(also remove '1 *')
Although you do #include <linux/mman.h>, for some reason I get some failures on aarch64 and s390x on old gcc 4.8.5:
hugemmap34.c:82:39: error: ‘MAP_HUGE_1GB’ undeclared (first use in this function)
MAP_PRIVATE|MAP_ANON|MAP_HUGETLB|MAP_HUGE_1GB,
^But works on the same toolchain with gcc 7.5.0, also x86_64 and ppc64le works fine on them). It looks to me MAP_HUGE_1GB will need to be backported to lapi. It works on x86_64 and ppc64le.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 3/3] hugemmap34: Test to detect bug with migrating gigantic pages
Quoted 4 lines
Hi Li,
FYI hugemmap34 fails on
x86_64and s390x various SLES kernels (tested:x86_64from 4.4 to 6.4, s390x from 4.12. to 6.4):
Also Tumbleweed with 6.8.0-rc4 fails
hugemmap34.c:85: TBROK: mmap() failed: EINVAL (22)
(no code modification - with MAP_HUGE_1GB).
But in CI I reproduced also tst_hugepage.c:119: TCONF: nr_hugepages = 0, but expect 2. Not enough hugepages for testing.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp