Hugetlb:Migrating the libhugetlbfs tests
v3 cover Hugetlb:Migrating the libhugetlbfs tests
Hi,
Libhugetlbfs is not being maintained actively, and some distro is dropping support for it. There are some tests that are good for testing hugetlb functionality in kernel.
As per previous dicussion in RFC[1], Here, this patch series consists of hugetlb tests taken from libhugetlbfs modified to work in ltp environment.
Based on suggestion[2], I am reposting the patches again. Also, I added suggested changes at [3].
ref:
1:https://lore.kernel.org/all/[email protected]/ 2:https://lore.kernel.org/all/[email protected]/ 3:https://lore.kernel.org/all/[email protected]/
v1 -> v2
1. In (brk near huge) test [PATCH 1/3] removed unused library function test_addr_huge() and read_maps().
v2 -> v3
1. Added a new patch commit for two new flags "needs_hugetlbfs" & "needs_unlinked_hugetlb_file" in tst_test and modified tests to use these flags.
2. Added taint check in test [PATCH 1/3].
3. Removed redundant Hopt and nr_opt option.
4. Corrected pre-processor ARCH based conditional flags in test [PATCH 1/3]
Tarun Sahu (4):
Hugetlb: Add new tst_test options for hugeltb test support
Hugetlb: Migrating libhugetlbfs brk_near_huge
Hugetlb: Migrating libhugetlbfs chunk-overcommit
Hugetlb: Migrating libhugetlbfs corrupt-by-cow-opt
include/tst_test.h | 20 ++- lib/tst_test.c | 51 ++++++- runtest/hugetlb | 3 + testcases/kernel/mem/.gitignore | 3 + .../kernel/mem/hugetlb/hugemmap/Makefile | 5 + .../kernel/mem/hugetlb/hugemmap/hugemmap07.c | 129 ++++++++++++++++ .../kernel/mem/hugetlb/hugemmap/hugemmap08.c | 144 ++++++++++++++++++ .../kernel/mem/hugetlb/hugemmap/hugemmap09.c | 71 +++++++++ testcases/kernel/mem/hugetlb/lib/hugetlb.h | 3 + 9 files changed, 425 insertions(+), 4 deletions(-) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap07.c create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap09.c
Signature · 5 lines
2.31.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 1/4 Hugetlb: Add new tst_test options for hugeltb test support
Most of libhugetlbfs test require mounted hugetlbfs and random opened unlinked file for follow-up test actions.
Here, this patch adds two new field in tst_test struct(include/tst_test.h) which user can set if the test requires mounted hugetlbfs and other one for if test requires opened unlinked file.
--- include/tst_test.h | 20 +++++++++++++++++- lib/tst_test.c | 51 +++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 67 insertions(+), 4 deletions(-)
include/tst_test.h +19 -1
@@ -131,7 +131,7 @@ struct tst_tag { };extern unsigned int tst_variant; - +extern int tst_hugetlb_fd;#define TST_NO_HUGEPAGES ((unsigned long)-1)#define TST_UNLIMITED_RUNTIME (-1)
@@ -176,6 +176,18 @@ struct tst_test { int all_filesystems:1; int skip_in_lockdown:1; int skip_in_compat:1; +/* +* If set, the test function will create a hugetlbfs mount point +* at /tmp/xxxxxx, where xxxxxx is a random string. +*/ + int needs_hugetlbfs:1; +/* +* If set, the test function will create and open a random file +* under mounted hugetlbfs. To use this option, needs_hugetlbfs must +* be set. The file descriptior will be set in tst_hugetlb_fd. +* The close(tst_hugetlb_fd) will be called on test exit(cleanup). +*/ + int needs_unlinked_hugetlb_file:1;/* * The skip_filesystems is a NULL terminated list of filesystems the
@@ -357,6 +369,12 @@ unsigned int tst_remaining_runtime(void);*/ void tst_set_max_runtime (int max_runtime); +/* +* Create and open a random file inside the given dir path. +* It unlinks the file after opening and return file descriptor. +*/ +int tst_create_unlinked_file (const char *path); +/* * Returns path to the test temporary directory in a newly allocated buffer. */
lib/tst_test.c +48 -3
81 lines not on this page · raw message
Signature · 5 lines
2.31.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/4] Hugetlb: Add new tst_test options for hugeltb test support
Hi Tarun,
This version is much better, comments are inline below.
Quoted 40 lines · Tarun Sahu
Most of libhugetlbfs test require mounted hugetlbfs and random opened unlinked file for follow-up test actions.
Here, this patch adds two new field in
tst_teststruct(include/tst_test.h)which user can set if the test requires mounted hugetlbfs and other one for if test requires opened unlinked file.Signed-off-by Tarun Sahu <[email protected]>--- include/tst_test.h | 20 +++++++++++++++++- lib/tst_test.c | 51 +++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 67 insertions(+), 4 deletions(-)include/tst_test.h +9
-1@@ -131,7 +131,7 @@ struct tst_tag { };extern unsigned int tst_variant; - +extern int tst_hugetlb_fd;#define TST_NO_HUGEPAGES ((unsigned long)-1)#define TST_UNLIMITED_RUNTIME (-1)@@ -176,6 +176,18 @@ struct tst_test { int all_filesystems:1; int skip_in_lockdown:1; int skip_in_compat:1; +/* +* If set, the test function will create a hugetlbfs mount point +* at /tmp/xxxxxx, where xxxxxx is a random string. +*/ + int needs_hugetlbfs:1; +/* +* If set, the test function will create and open a random file +* under mounted hugetlbfs. To use this option, needs_hugetlbfs must
include/tst_test.h struct tst_test { · line 190
Quoted 4 lines
include/tst_test.h +4
-0+* be set. The file descriptior will be set in tst_hugetlb_fd. +* The close(tst_hugetlb_fd) will be called on test exit(cleanup). +*/ + int needs_unlinked_hugetlb_file:1;
Why not consider encapsulating these two new fields in 'struct tst_hugepage' ?
Then the tst_test in the case can simply initialize to:
....
static struct tst_test test = {
.needs_root = 1,
.taint_check = TST_TAINT_D | TST_TAINT_W,
.setup = setup,
.test_all = run_test,
.hugepages = {1, TST_NEEDS, 1, 1},
};
Quoted 5 lines
/*
* Theskip_filesystemsis a NULL terminated list of filesystems
the
include/tst_test.h unsigned int tst_remaining_runtime(void); · line 377
Quoted 9 lines
include/tst_test.h +5
-0@@ -357,6 +369,12 @@ unsigned int tst_remaining_runtime(void);*/ void tst_set_max_runtime (int max_runtime); +/* +* Create and open a random file inside the given dir path. +* It unlinks the file after opening and return file descriptor. +*/ +int tst_create_unlinked_file (const char *path);
what about renaming this function to tst_'get|create'_unlinked_file_fd? I guess the 'fd' part should be emphasized here.
Quoted 76 lines
include/tst_test.h +1
-0+/* * Returns path to the test temporary directory in a newly allocated buffer.
*/lib/tst_test.c +26
-1@@ -925,7 +925,8 @@ static int needs_tmpdir(void) tst_test->needs_device || tst_test->mntpoint || tst_test->resource_files || - tst_test->needs_checkpoints; + tst_test->needs_checkpoints || + tst_test->needs_hugetlbfs; }static void copy_resources (void )@@ -1021,6 +1022,30 @@ static void prepare_and_mount_dev_fs(const char *mntpoint) } } +static void prepare_and_mount_hugetlb_fs (void ) +{ + tst_test->mntpoint =tst_get_tmpdir (); +SAFE_MOUNT ("none" , tst_test->mntpoint,"hugetlbfs" ,0 ,NULL ); + mntpoint_mounted =1 ; +} + +int tst_create_unlinked_file (const char *path) +{ +char template[PATH_MAX ]; +int fd; + +snprintf (template,PATH_MAX ,"%s/ltp_%.3sXXXXXX" , + path,TCID ); + + fd =mkstemp (template); +if (fd <0 ) +tst_brk (TBROK |TERRNO , +"%s: mkstemp(%s) failed" , __func__, template); + +SAFE_UNLINK (template); +return fd; +} +static const char *limit_tmpfs_mount_size (const char *mnt_data,char *buf,size_t buf_size,const char *fs_type){
lib/tst_test.c +8
-0@@ -1094,6 +1119,8 @@ static void do_cgroup_requires(void)tst_cg_init (); } +int tst_hugetlb_fd =-1 ; +static void do_setup (int argc,char *argv[]) {if (!tst_test)@@ -1217,6 +1244,17 @@ static void do_setup(int argc, char *argv[]) } } +if (tst_test->needs_hugetlbfs) +prepare_and_mount_hugetlb_fs (); + +if (tst_test->needs_unlinked_hugetlb_file) { +if (!(tst_test->needs_hugetlbfs)) { + tst_brk(TBROK, "Option needs_unlinked_hugetlb_file"
lib/tst_test.c +3
-0+ "requires option needs_hugetlbfs"); + } + tst_hugetlb_fd =
tst_create_unlinked_file(tst_test->mntpoint);
lib/tst_test.c
Quoted 2 lines
lib/tst_test.c +2
-0+ } +
Seems we have to add a confliction check[1] to avoid multiple mounting at 'tst_test->mntpoint'. Or maybe go another method to move all the hugetlbfs
operations into tst_hugetlb.c to isolate details from the tst_test library, but
this will require more changes for all preexisting hugetlb tests.
[1] something like this:
@@ -1224,9 +1224,9 @@ static void do_setup(int argc, char *argv[])
}
if (!!tst_test->needs_rofs + !!tst_test->needs_devfs +- !!tst_test->needs_device > 1) {
+ !!tst_test->needs_device + !!tst_test->needs_hugetlbfs > 1) {
tst_brk(TBROK,
- "Two or more of needs_{rofs, devfs, device} are
set");
+ "Two or more of needs_{rofs, devfs, device,
hugetlb} are set");
}
if (tst_test->needs_device && !mntpoint_mounted) {Quoted 30 lines
tdev.dev = tst_acquire_device_(NULL,
tst_test->dev_min_size);lib/tst_test.c +9
-3@@ -1299,8 +1337,15 @@ static void do_cleanup(void)if (ovl_mounted)SAFE_UMOUNT (OVL_MNT ); -if (mntpoint_mounted) -tst_umount (tst_test->mntpoint); +if (tst_hugetlb_fd >=0 ) +SAFE_CLOSE (tst_hugetlb_fd); + +if (mntpoint_mounted) { +if (tst_test->needs_hugetlbfs) +SAFE_UMOUNT (tst_test->mntpoint); +else +tst_umount (tst_test->mntpoint); + }if (tst_test->needs_device && tdev.dev)tst_release_device (tdev.dev); --2.31.1
--
Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/4] Hugetlb: Add new tst_test options for hugeltb test support
Hello,
Thanks for reviewing patch Li,
Quoted 56 lines · Li Wang
Hi Tarun,
This version is much better, comments are inline below.
Quoted 40 lines · Tarun Sahu
Most of libhugetlbfs test require mounted hugetlbfs and random opened unlinked file for follow-up test actions.
Here, this patch adds two new field in
tst_teststruct(include/tst_test.h)which user can set if the test requires mounted hugetlbfs and other one for if test requires opened unlinked file.Signed-off-by Tarun Sahu <[email protected]>--- include/tst_test.h | 20 +++++++++++++++++- lib/tst_test.c | 51 +++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 67 insertions(+), 4 deletions(-)include/tst_test.h +9
-1@@ -131,7 +131,7 @@ struct tst_tag { };extern unsigned int tst_variant; - +extern int tst_hugetlb_fd;#define TST_NO_HUGEPAGES ((unsigned long)-1)#define TST_UNLIMITED_RUNTIME (-1)@@ -176,6 +176,18 @@ struct tst_test { int all_filesystems:1; int skip_in_lockdown:1; int skip_in_compat:1; +/* +* If set, the test function will create a hugetlbfs mount point +* at /tmp/xxxxxx, where xxxxxx is a random string. +*/ + int needs_hugetlbfs:1; +/* +* If set, the test function will create and open a random file +* under mounted hugetlbfs. To use this option, needs_hugetlbfs must
include/tst_test.h struct tst_test { · line 190
Quoted 4 lines
include/tst_test.h +4
-0+* be set. The file descriptior will be set in tst_hugetlb_fd. +* The close(tst_hugetlb_fd) will be called on test exit(cleanup). +*/ + int needs_unlinked_hugetlb_file:1;Why not consider encapsulating these two new fields in 'struct
tst_hugepage' ?Then the
tst_testin the case can simply initialize to:
Quoted 9 lines
....
static structtst_testtest = {
.needs_root = 1,
.taint_check =TST_TAINT_D|TST_TAINT_W,
.setup = setup,
.test_all =run_test,
.hugepages = {1,TST_NEEDS, 1, 1},
};
Ok, I will move these fields in tst_hugepages struct.
Quoted 10 lines
Quoted 10 lines
/*
* Theskip_filesystemsis a NULL terminated list of filesystems
theinclude/tst_test.h +0
-0@@ -357,6 +369,12 @@ unsigned int tst_remaining_runtime(void);*/ void tst_set_max_runtime (int max_runtime);
Quoted 10 lines
include/tst_test.h unsigned int tst_remaining_runtime(void);
Quoted 6 lines
include/tst_test.h +5
-0+/* +* Create and open a random file inside the given dir path. +* It unlinks the file after opening and return file descriptor. +*/ +int tst_create_unlinked_file (const char *path);what about renaming this function to tst_'get|create'_unlinked_file_fd? I guess the 'fd' part should be emphasized here.
OK. Will update it.
Quoted 94 lines
Quoted 78 lines
include/tst_test.h +1
-0+/* * Returns path to the test temporary directory in a newly allocated buffer.
*/lib/tst_test.c +26
-1@@ -925,7 +925,8 @@ static int needs_tmpdir(void) tst_test->needs_device || tst_test->mntpoint || tst_test->resource_files || - tst_test->needs_checkpoints; + tst_test->needs_checkpoints || + tst_test->needs_hugetlbfs; }static void copy_resources (void )@@ -1021,6 +1022,30 @@ static void prepare_and_mount_dev_fs(const char *mntpoint) } } +static void prepare_and_mount_hugetlb_fs (void ) +{ + tst_test->mntpoint =tst_get_tmpdir (); +SAFE_MOUNT ("none" , tst_test->mntpoint,"hugetlbfs" ,0 ,NULL ); + mntpoint_mounted =1 ; +} + +int tst_create_unlinked_file (const char *path) +{ +char template[PATH_MAX ]; +int fd; + +snprintf (template,PATH_MAX ,"%s/ltp_%.3sXXXXXX" , + path,TCID ); + + fd =mkstemp (template); +if (fd <0 ) +tst_brk (TBROK |TERRNO , +"%s: mkstemp(%s) failed" , __func__, template); + +SAFE_UNLINK (template); +return fd; +} +static const char *limit_tmpfs_mount_size (const char *mnt_data,char *buf,size_t buf_size,const char *fs_type){
lib/tst_test.c +8
-0@@ -1094,6 +1119,8 @@ static void do_cgroup_requires(void)tst_cg_init (); } +int tst_hugetlb_fd =-1 ; +static void do_setup (int argc,char *argv[]) {if (!tst_test)@@ -1217,6 +1244,17 @@ static void do_setup(int argc, char *argv[]) } } +if (tst_test->needs_hugetlbfs) +prepare_and_mount_hugetlb_fs (); + +if (tst_test->needs_unlinked_hugetlb_file) { +if (!(tst_test->needs_hugetlbfs)) { + tst_brk(TBROK, "Option needs_unlinked_hugetlb_file"
lib/tst_test.c +3
-0+ "requires option needs_hugetlbfs"); + } + tst_hugetlb_fd =
tst_create_unlinked_file(tst_test->mntpoint);lib/tst_test.c
Quoted 2 lines
lib/tst_test.c +2
-0+ } +Seems we have to add a confliction check[1] to avoid multiple mounting at '
tst_test->mntpoint'. Or maybe go another method to move all the hugetlbfs
operations intotst_hugetlb.cto isolate details from thetst_testlibrary, but
this will require more changes for all preexisting hugetlb tests.[1] something like this:
(quoted diff) +0
-0@@ -1224,9 +1224,9 @@ static void do_setup(int argc, char *argv[]) }
Review static void do_setup(int argc, char *argv[]) · line 1232
Quoted 10 lines
(quoted diff) +2
-2if (!!tst_test->needs_rofs + !!tst_test->needs_devfs + - !!tst_test->needs_device > 1) { + !!tst_test->needs_device + !!tst_test->needs_hugetlbfs > 1) { tst_brk(TBROK, - "Two or more of needs_{rofs, devfs, device} are set"); + "Two or more of needs_{rofs, devfs, device, hugetlb} are set"); }
OK. Will update it.
Quoted 40 lines
if (tst_test->needs_device && !mntpoint_mounted) {Quoted 30 lines
tdev.dev = tst_acquire_device_(NULL,
tst_test->dev_min_size);lib/tst_test.c +9
-3@@ -1299,8 +1337,15 @@ static void do_cleanup(void)if (ovl_mounted)SAFE_UMOUNT (OVL_MNT ); -if (mntpoint_mounted) -tst_umount (tst_test->mntpoint); +if (tst_hugetlb_fd >=0 ) +SAFE_CLOSE (tst_hugetlb_fd); + +if (mntpoint_mounted) { +if (tst_test->needs_hugetlbfs) +SAFE_UMOUNT (tst_test->mntpoint); +else +tst_umount (tst_test->mntpoint); + }if (tst_test->needs_device && tdev.dev)tst_release_device (tdev.dev); --2.31.1
--
Mailing list info: https://lists.linux.it/listinfo/ltpSignature · 6 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/4] Hugetlb: Add new tst_test options for hugeltb test support
In reply to Li WangHi!
Quoted 4 lines
Quoted 4 lines
include/tst_test.h +0
-0@@ -357,6 +369,12 @@ unsigned int tst_remaining_runtime(void);*/ void tst_set_max_runtime (int max_runtime);
Quoted 9 lines
include/tst_test.h unsigned int tst_remaining_runtime(void);
Quoted 6 lines
include/tst_test.h +5
-0+/* +* Create and open a random file inside the given dir path. +* It unlinks the file after opening and return file descriptor. +*/ +int tst_create_unlinked_file (const char *path);what about renaming this function to tst_'get|create'_unlinked_file_fd? I guess the 'fd' part should be emphasized here.
It has create in name and in UNIX creat() creates file and returns a file descriptor so I think it's fine. Maybe we can be even shorted with something as tst_creat_unlinked(const char *path).
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/4] Hugetlb: Add new tst_test options for hugeltb test support
In reply to Li WangHi!
Quoted 5 lines
Why not consider encapsulating these two new fields in 'struct
tst_hugepage' ?Then the
tst_testin the case can simply initialize to:
Quoted 8 lines
....
static structtst_testtest = {
.needs_root = 1,
.taint_check =TST_TAINT_D|TST_TAINT_W,
.setup = setup,
.test_all =run_test,
.hugepages = {1,TST_NEEDS, 1, 1},
};
I do not like that we have magic constants in the .hugepages that are not self describing. I would treat the hugetltbfs just as we treat devfs, that would be:
#define MNTPOINT "hugetlbfs/"
#define HUGEFILE MNTPOINT "hugefile"
static int huge_fd;
static void setup(void)
{
huge_fd = tst_creat_unlinked(HUGEFILE);
...
}static void cleanup(void)
{
if (huge_fd > 0)
SAFE_CLOSE(huge_fd);
}static struct tst_test test = {
...
.mntpoint = MNTPOINT,
.needs_hugetlbfs = 1,
.setup = setup,
.cleanup = cleanup,
...
}
What do you think?
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/4] Hugetlb: Add new tst_test options for hugeltb test support
Hi Cyril,
Please find my comments inline.
Quoted 46 lines · Cyril Hrubis
Hi!
Quoted 5 lines
Why not consider encapsulating these two new fields in 'struct
tst_hugepage' ?Then the
tst_testin the case can simply initialize to:Quoted 8 lines
....
static structtst_testtest = {
.needs_root = 1,
.taint_check =TST_TAINT_D|TST_TAINT_W,
.setup = setup,
.test_all =run_test,
.hugepages = {1,TST_NEEDS, 1, 1},
};I do not like that we have magic constants in the .hugepages that are not self describing. I would treat the hugetltbfs just as we treat devfs, that would be:
#define MNTPOINT "hugetlbfs/"
#define HUGEFILE MNTPOINT "hugefile"static int
huge_fd;static void setup(void)
{ huge_fd = tst_creat_unlinked(HUGEFILE); ... }static void cleanup(void)
{ if (huge_fd > 0) SAFE_CLOSE(huge_fd); }static struct
tst_testtest = {
...
.mntpoint = MNTPOINT,
.needs_hugetlbfs = 1,
.setup = setup,
.cleanup = cleanup,
...
}
Quoted 2 lines
What do you think?
My original idea behind putting it in tst_test struct instead of tst_hugepages was based on below two reasoning-
1. tst_hugepages seems to have only hugepages related info. It would be better to rename it to tst_hugetlb and rename <hugepages> field in tst_test struct to <hugetlb>. If we rename it which will require changes in already existing tests. and Moreover, there were similar field like needs_tmpdir in tst_test so I put it(needs_unlinked_hugetlb_file) in tst_test.
2. There was already related fields in tst_test for mounting fs, like needs_devfs, needs_rofs, So keeping all the needs_ABCfs at same structure.
Quoted 6 lines
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/4] Hugetlb: Add new tst_test options for hugeltb test support
In reply to Cyril HrubisQuoted 46 lines · Cyril Hrubis
Hi!
Quoted 5 lines
Why not consider encapsulating these two new fields in 'struct
tst_hugepage' ?Then the
tst_testin the case can simply initialize to:Quoted 8 lines
....
static structtst_testtest = {
.needs_root = 1,
.taint_check =TST_TAINT_D|TST_TAINT_W,
.setup = setup,
.test_all =run_test,
.hugepages = {1,TST_NEEDS, 1, 1},
};I do not like that we have magic constants in the .hugepages that are not self describing. I would treat the hugetltbfs just as we treat devfs, that would be:
#define MNTPOINT "hugetlbfs/"
#define HUGEFILE MNTPOINT "hugefile"static int
huge_fd;static void setup(void)
{ huge_fd = tst_creat_unlinked(HUGEFILE); ... }static void cleanup(void)
{ if (huge_fd > 0) SAFE_CLOSE(huge_fd); }static struct
tst_testtest = {
...
.mntpoint = MNTPOINT,
.needs_hugetlbfs = 1,
.setup = setup,
.cleanup = cleanup,
...
}
Quoted 2 lines
What do you think?
+1 Looks good, this treats it as an FS and is separated from the hugepage usage.
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/4] Hugetlb: Add new tst_test options for hugeltb test support
In reply to Tarun SahuHi!
Quoted 96 lines
extern unsigned int
tst_variant;include/tst_test.h +19
-1- +extern int tst_hugetlb_fd;#define TST_NO_HUGEPAGES ((unsigned long)-1)#define TST_UNLIMITED_RUNTIME (-1)@@ -176,6 +176,18 @@ struct tst_test { int all_filesystems:1; int skip_in_lockdown:1; int skip_in_compat:1; +/* +* If set, the test function will create a hugetlbfs mount point +* at /tmp/xxxxxx, where xxxxxx is a random string. +*/ + int needs_hugetlbfs:1; +/* +* If set, the test function will create and open a random file +* under mounted hugetlbfs. To use this option, needs_hugetlbfs must +* be set. The file descriptior will be set in tst_hugetlb_fd. +* The close(tst_hugetlb_fd) will be called on test exit(cleanup). +*/ + int needs_unlinked_hugetlb_file:1;/* * The skip_filesystems is a NULL terminated list of filesystems the @@ -357,6 +369,12 @@ unsigned int tst_remaining_runtime(void);*/ void tst_set_max_runtime (int max_runtime); +/* +* Create and open a random file inside the given dir path. +* It unlinks the file after opening and return file descriptor. +*/ +int tst_create_unlinked_file (const char *path); +/* * Returns path to the test temporary directory in a newly allocated buffer. */ lib/tst_test.c +28
-1@@ -925,7 +925,8 @@ static int needs_tmpdir(void) tst_test->needs_device || tst_test->mntpoint || tst_test->resource_files || - tst_test->needs_checkpoints; + tst_test->needs_checkpoints || + tst_test->needs_hugetlbfs; }static void copy_resources (void )@@ -1021,6 +1022,30 @@ static void prepare_and_mount_dev_fs(const char *mntpoint) } } +static void prepare_and_mount_hugetlb_fs (void ) +{ + tst_test->mntpoint =tst_get_tmpdir (); +SAFE_MOUNT ("none" , tst_test->mntpoint,"hugetlbfs" ,0 ,NULL ); + mntpoint_mounted =1 ; +} + +int tst_create_unlinked_file (const char *path) +{ +char template[PATH_MAX ]; +int fd; + +snprintf (template,PATH_MAX ,"%s/ltp_%.3sXXXXXX" , + path,TCID ); + + fd =mkstemp (template); +if (fd <0 ) +tst_brk (TBROK |TERRNO , +"%s: mkstemp(%s) failed" , __func__, template); + +SAFE_UNLINK (template); +return fd; +} +static const char *limit_tmpfs_mount_size (const char *mnt_data,char *buf,size_t buf_size,const char *fs_type) {@@ -1094,6 +1119,8 @@ static void do_cgroup_requires(void)tst_cg_init (); } +int tst_hugetlb_fd =-1 ; +static void do_setup (int argc,char *argv[]) {if (!tst_test)@@ -1217,6 +1244,17 @@ static void do_setup(int argc, char *argv[]) } }
lib/tst_test.c static void do_setup(int argc, char *argv[]) · line 1256
Quoted 10 lines
lib/tst_test.c +10
-0+if (tst_test->needs_hugetlbfs) +prepare_and_mount_hugetlb_fs (); + +if (tst_test->needs_unlinked_hugetlb_file) { +if (!(tst_test->needs_hugetlbfs)) { +tst_brk (TBROK ,"Option needs_unlinked_hugetlb_file " +"requires option needs_hugetlbfs" ); + } + tst_hugetlb_fd =tst_create_unlinked_file (tst_test->mntpoint); + }
The function tst_create_unlinked_file() looks useful, but I do not think that adding the needs_unlinked_hugetlb_file flag simplifies things that much. Also this will not scale well when we would need two filedescripors like that. Maybe we it would be cleaner to add only the mount/umount functionality to the test library and call the tst_create_unlinked_file() in the test setup in the testcases.
Quoted 8 lines
lib/tst_test.c +0
-1if (tst_test->needs_device && !mntpoint_mounted) { tdev.dev =tst_acquire_device_ (NULL , tst_test->dev_min_size);@@ -1299,8 +1337,15 @@ static void do_cleanup(void)if (ovl_mounted)SAFE_UMOUNT (OVL_MNT ); -if (mntpoint_mounted)
lib/tst_test.c static void do_cleanup(void) · line 1348
Quoted 10 lines
lib/tst_test.c +9
-1-tst_umount (tst_test->mntpoint); +if (tst_hugetlb_fd >=0 ) +SAFE_CLOSE (tst_hugetlb_fd); + +if (mntpoint_mounted) { +if (tst_test->needs_hugetlbfs) +SAFE_UMOUNT (tst_test->mntpoint); +else +tst_umount (tst_test->mntpoint); + }
Is there a good reason for this, why can't we call tst_umount() for hugetlbfs?
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/4] Hugetlb: Add new tst_test options for hugeltb test support
Quoted 106 lines · Cyril Hrubis
Hi!
Quoted 105 lines
extern unsigned int
tst_variant;include/tst_test.h +19
-1- +extern int tst_hugetlb_fd;#define TST_NO_HUGEPAGES ((unsigned long)-1)#define TST_UNLIMITED_RUNTIME (-1)@@ -176,6 +176,18 @@ struct tst_test { int all_filesystems:1; int skip_in_lockdown:1; int skip_in_compat:1; +/* +* If set, the test function will create a hugetlbfs mount point +* at /tmp/xxxxxx, where xxxxxx is a random string. +*/ + int needs_hugetlbfs:1; +/* +* If set, the test function will create and open a random file +* under mounted hugetlbfs. To use this option, needs_hugetlbfs must +* be set. The file descriptior will be set in tst_hugetlb_fd. +* The close(tst_hugetlb_fd) will be called on test exit(cleanup). +*/ + int needs_unlinked_hugetlb_file:1;/* * The skip_filesystems is a NULL terminated list of filesystems the @@ -357,6 +369,12 @@ unsigned int tst_remaining_runtime(void);*/ void tst_set_max_runtime (int max_runtime); +/* +* Create and open a random file inside the given dir path. +* It unlinks the file after opening and return file descriptor. +*/ +int tst_create_unlinked_file (const char *path); +/* * Returns path to the test temporary directory in a newly allocated buffer. */ lib/tst_test.c +37
-1@@ -925,7 +925,8 @@ static int needs_tmpdir(void) tst_test->needs_device || tst_test->mntpoint || tst_test->resource_files || - tst_test->needs_checkpoints; + tst_test->needs_checkpoints || + tst_test->needs_hugetlbfs; }static void copy_resources (void )@@ -1021,6 +1022,30 @@ static void prepare_and_mount_dev_fs(const char *mntpoint) } } +static void prepare_and_mount_hugetlb_fs (void ) +{ + tst_test->mntpoint =tst_get_tmpdir (); +SAFE_MOUNT ("none" , tst_test->mntpoint,"hugetlbfs" ,0 ,NULL ); + mntpoint_mounted =1 ; +} + +int tst_create_unlinked_file (const char *path) +{ +char template[PATH_MAX ]; +int fd; + +snprintf (template,PATH_MAX ,"%s/ltp_%.3sXXXXXX" , + path,TCID ); + + fd =mkstemp (template); +if (fd <0 ) +tst_brk (TBROK |TERRNO , +"%s: mkstemp(%s) failed" , __func__, template); + +SAFE_UNLINK (template); +return fd; +} +static const char *limit_tmpfs_mount_size (const char *mnt_data,char *buf,size_t buf_size,const char *fs_type) {@@ -1094,6 +1119,8 @@ static void do_cgroup_requires(void)tst_cg_init (); } +int tst_hugetlb_fd =-1 ; +static void do_setup (int argc,char *argv[]) {if (!tst_test)@@ -1217,6 +1244,17 @@ static void do_setup(int argc, char *argv[]) } } +if (tst_test->needs_hugetlbfs) +prepare_and_mount_hugetlb_fs (); + +if (tst_test->needs_unlinked_hugetlb_file) { +if (!(tst_test->needs_hugetlbfs)) { +tst_brk (TBROK ,"Option needs_unlinked_hugetlb_file " +"requires option needs_hugetlbfs" ); + } + tst_hugetlb_fd =tst_create_unlinked_file (tst_test->mntpoint);
Quoted 9 lines
lib/tst_test.c static void do_setup(int argc, char *argv[]) · line 1256
Quoted 1 line
lib/tst_test.c +1
-0+ }The function
tst_create_unlinked_file()looks useful, but I do not think that adding theneeds_unlinked_hugetlb_fileflag simplifies things that much. Also this will not scale well when we would need two filedescripors like that. Maybe we it would be cleaner to add only the mount/umount functionality to the test library and call thetst_create_unlinked_file()in the test setup in the testcases.
yes I agree, There is a test that requires two such unlinked file, there is one that requires one hugetlb and one normal such unlinked file.
Quoted 7 lines
Quoted 7 lines
lib/tst_test.c +0
-0if (tst_test->needs_device && !mntpoint_mounted) { tdev.dev =tst_acquire_device_ (NULL , tst_test->dev_min_size);@@ -1299,8 +1337,15 @@ static void do_cleanup(void)if (ovl_mounted)SAFE_UMOUNT (OVL_MNT );
Quoted 15 lines
lib/tst_test.c static void do_cleanup(void) · line 1348
Quoted 11 lines
lib/tst_test.c +9
-2-if (mntpoint_mounted) -tst_umount (tst_test->mntpoint); +if (tst_hugetlb_fd >=0 ) +SAFE_CLOSE (tst_hugetlb_fd); + +if (mntpoint_mounted) { +if (tst_test->needs_hugetlbfs) +SAFE_UMOUNT (tst_test->mntpoint); +else +tst_umount (tst_test->mntpoint); + }Is there a good reason for this, why can't we call
tst_umount()for hugetlbfs?
My reason behind this is,tst_umount doesnt break the test, if umount fails. One of the cause of failure can be that test left the mounted fs busy, which is not expected.
Quoted 6 lines
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/4] Hugetlb: Add new tst_test options for hugeltb test support
Hi!
Quoted 8 lines
Quoted 3 lines
Is there a good reason for this, why can't we call
tst_umount()for hugetlbfs?My reason behind this is,
tst_umountdoesnt break the test, if umount fails. One of the cause of failure can be that test left the mounted fs busy, which is not expected.
This does not matter, because the do_cleanup() is called only once at the end of the test. The test exists once the function does finish anyways.
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 2/4 Hugetlb: Migrating libhugetlbfs brk_near_huge
In reply to Tarun SahuMigrating the libhugetlbfs/testcases/brk_near_huge.c test
Test Description:
Certain kernels have a bug where brk() does not perform the same checks that a MAP_FIXED mmap() will, allowing brk() to create a normal page VMA in a hugepage only address region. This can lead to oopses or other badness.
--- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/Makefile | 5 + .../kernel/mem/hugetlb/hugemmap/hugemmap07.c | 129 ++++++++++++++++++ testcases/kernel/mem/hugetlb/lib/hugetlb.h | 3 + 5 files changed, 139 insertions(+) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap07.c
runtest/hugetlb +1 -0
@@ -3,6 +3,7 @@ hugemmap02 hugemmap02 hugemmap04 hugemmap04 hugemmap05 hugemmap05 hugemmap06 hugemmap06 +hugemmap07 hugemmap07 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -m
testcases/kernel/mem/.gitignore +1 -0
@@ -4,6 +4,7 @@ /hugetlb/hugemmap/hugemmap04 /hugetlb/hugemmap/hugemmap05 /hugetlb/hugemmap/hugemmap06 +/hugetlb/hugemmap/hugemmap07 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03
testcases/kernel/mem/hugetlb/hugemmap/Makefile +5 -0
@@ -9,3 +9,8 @@ include $(abs_srcdir)/../Makefile.inc include $(top_srcdir)/include/mk/generic_leaf_target.mk hugemmap06: CFLAGS+=-pthread + +hugemmap07: +ifeq ($(ARCH),ppc) + CPPFLAGS += -DPPC_NO_SEGMENTS +endif
testcases/kernel/mem/hugetlb/hugemmap/hugemmap07.c new file +129 -0
129 lines not on this page · raw message
testcases/kernel/mem/hugetlb/lib/hugetlb.h +3 -0
@@ -20,6 +20,9 @@#include "old_tmpdir.h" #include "mem.h" +#define ALIGN (x, a) (((x) + (a) - 1) & ~((a) - 1)) +#define PALIGN (p, a) ((void *)ALIGN((unsigned long)(p), (a))) +#define SHM_RD 0400#define SHM_WR 0200#define SHM_RW (SHM_RD|SHM_WR)
Signature · 5 lines
2.31.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 3/4 Hugetlb: Migrating libhugetlbfs chunk-overcommit
In reply to Tarun SahuMigrating the libhugetlbfs/testcases/chunk-overcommit.c test
Test Description: Some kernel versions after hugepage demand allocation was added used a dubious heuristic to check if there was enough hugepage space available for a given mapping. The number of not-already-instantiated pages in the mapping was compared against the total hugepage free pool. It was very easy to confuse this heuristic into overcommitting by allocating hugepage memory in chunks, each less than the total available pool size but together more than available. This would generally lead to OOM SIGKILLs of one process or another when it tried to instantiate pages beyond the available pool.
--- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/hugemmap08.c | 144 ++++++++++++++++++ 3 files changed, 146 insertions(+) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c
runtest/hugetlb +1 -0
@@ -4,6 +4,7 @@ hugemmap04 hugemmap04 hugemmap05 hugemmap05 hugemmap06 hugemmap06 hugemmap07 hugemmap07 +hugemmap08 hugemmap08 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -m
testcases/kernel/mem/.gitignore +1 -0
@@ -5,6 +5,7 @@ /hugetlb/hugemmap/hugemmap05 /hugetlb/hugemmap/hugemmap06 /hugetlb/hugemmap/hugemmap07 +/hugetlb/hugemmap/hugemmap08 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03
testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c new file +144 -0
144 lines not on this page · raw message
Signature · 5 lines
2.31.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 3/4] Hugetlb: Migrating libhugetlbfs chunk-overcommit
Quoted 154 lines · Tarun Sahu
Migrating the
libhugetlbfs/testcases/chunk-overcommit.ctestTest Description: Some kernel versions after hugepage demand allocation was added used a dubious heuristic to check if there was enough hugepage space available for a given mapping. The number of not-already-instantiated pages in the mapping was compared against the total hugepage free pool. It was very easy to confuse this heuristic into overcommitting by allocating hugepage memory in chunks, each less than the total available pool size but together more than available. This would generally lead to OOM SIGKILLs of one process or another when it tried to instantiate pages beyond the available pool.
Signed-off-by Tarun Sahu <[email protected]>--- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/hugemmap08.c | 144 ++++++++++++++++++ 3 files changed, 146 insertions(+) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.cruntest/hugetlb +1
-0@@ -4,6 +4,7 @@ hugemmap04 hugemmap04 hugemmap05 hugemmap05 hugemmap06 hugemmap06 hugemmap07 hugemmap07 +hugemmap08 hugemmap08 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -mtestcases/kernel/mem/.gitignore +1
-0@@ -5,6 +5,7 @@ /hugetlb/hugemmap/hugemmap05 /hugetlb/hugemmap/hugemmap06 /hugetlb/hugemmap/hugemmap07 +/hugetlb/hugemmap/hugemmap08 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c new file +58
-0@@ -0,0 +1,144 @@ +// SPDX-License-Identifier: LGPL-2.1-or-later +/* +* Copyright (C) 2005-2006 David Gibson & Adam Litke, IBM Corporation. +* Author: David Gibson & Adam Litke +*/ + +/*\ +* [Description] +* +* Chunk Overcommit: +* Some kernel versions after hugepage demand allocation was added used a +* dubious heuristic to check if there was enough hugepage space available +* for a given mapping. The number of not-already-instantiated pages in +* the mapping was compared against the total hugepage free pool. It was +* very easy to confuse this heuristic into overcommitting by allocating +* hugepage memory in chunks, each less than the total available pool size +* but together more than available. This would generally lead to OOM +* SIGKILLs of one process or another when it tried to instantiate pages +* beyond the available pool. +* +* HISTORY +* +*/ + +#define _GNU_SOURCE +#include <stdio.h> +#include <stdlib.h> +#include <sys/mount.h> +#include <limits.h> +#include <sys/param.h> +#include <sys/types.h> +#include <sys/wait.h> +#include <signal.h> + +#include "hugetlb.h" + +#define PROC_OVERCOMMIT "/proc/sys/vm/nr_overcommit_hugepages" +#define WITH_OVERCOMMIT 0 +#define WITHOUT_OVERCOMMIT 1 + +static long hpage_size; + +static void test_chunk_overcommit (void ) +{ +unsigned long totpages, chunk1, chunk2; +void *p, *q; +pid_t child; +int status; + + totpages =SAFE_READ_MEMINFO ("HugePages_Free:" ); + + chunk1 = (totpages /2 ) +1 ; + chunk2 = totpages - chunk1 +1 ; + +tst_res (TINFO ,"Free: %ld hugepages available: " +"chunk1=%ld chunk2=%ld" , totpages, chunk1, chunk2); + + p = SAFE_MMAP(NULL, chunk1*hpage_size, PROT_READ|PROT_WRITE,
MAP_SHARED,testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c new file +10
-0+ tst_hugetlb_fd, 0); + + q =mmap (NULL , chunk2*hpage_size,PROT_READ |PROT_WRITE ,MAP_SHARED , + tst_hugetlb_fd, chunk1*hpage_size); +if (q ==MAP_FAILED ) { +if (errno !=ENOMEM ) { +tst_res (TFAIL |TERRNO ,"mmap() chunk2" ); + goto cleanup1; + }else { + tst_res(TPASS, "Successful without overcommitpages");
testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c new file +31
-0+ goto cleanup1; + } + } + +tst_res (TINFO ,"Looks like we've overcommitted, testing..." ); +/* Looks like we're overcommited, but we need to confirm that +* this is bad. We touch it all in a child process because an +* overcommit will generally lead to a SIGKILL which we can't +* handle, of course. +*/ + child =SAFE_FORK (); + +if (child ==0 ) { +memset (p,0 , chunk1*hpage_size); +memset (q,0 , chunk2*hpage_size); +exit (0 ); + } + +SAFE_WAITPID (child, &status,0 ); + +if (WIFSIGNALED (status)) { +tst_res (TFAIL ,"Killed by signal '%s' due to overcommit" , +tst_strsig (WTERMSIG (status))); + goto cleanup2; + } + +tst_res (TPASS ,"Successful with overcommit pages" ); + +cleanup2: +SAFE_MUNMAP (q, chunk2*hpage_size); +
testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c line 109
Quoted 10 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c new file +10
-0+cleanup1: +SAFE_MUNMAP (p, chunk1*hpage_size); +SAFE_FTRUNCATE (tst_hugetlb_fd,0 ); +} + +static void run_test (unsigned int test_type) +{ +unsigned long saved_oc_hugepages; + +SAFE_FILE_SCANF (PROC_OVERCOMMIT ,"%ld" , &saved_oc_hugepages);
There is unnecessary to read PROC_OVERCOMMIT value again,
we already save/restore it in struct tst_path_val[], so here we can set it directly to what we expected no matter if the original is 0 or 2.
static void run_test(unsigned int test_type)
{
switch (test_type) {
case WITHOUT_OVERCOMMIT:
tst_res(TINFO, "Without overcommit testing...");
SAFE_FILE_PRINTF(PROC_OVERCOMMIT, "%d", 0);
break;
case WITH_OVERCOMMIT:
tst_res(TINFO, "With overcommit testing...");
SAFE_FILE_PRINTF(PROC_OVERCOMMIT, "%d", 2);
break;
}
test_chunk_overcommit();
}Quoted 43 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c new file +35
-1+switch (test_type) { +case WITHOUT_OVERCOMMIT : +tst_res (TINFO ,"Without overcommit testing..." ); +if (saved_oc_hugepages >0 ) +SAFE_FILE_PRINTF (PROC_OVERCOMMIT ,"%d" ,0 ); +break ; +case WITH_OVERCOMMIT : +tst_res (TINFO ,"With overcommit testing..." ); +if (saved_oc_hugepages ==0 ) +SAFE_FILE_PRINTF (PROC_OVERCOMMIT ,"%d" ,2 ); +break ; + } +test_chunk_overcommit (); +} + +static void setup (void ) +{ + hpage_size =SAFE_READ_MEMINFO ("Hugepagesize:" )*1024 ; +} + +static struct tst_test test = { + .needs_root =1 , + .needs_hugetlbfs =1 , + .needs_unlinked_hugetlb_file =1 , + .forks_child =1 , + .save_restore = (const struct tst_path_val []) { + {PROC_OVERCOMMIT ,NULL }, + {} + }, + .tcnt =2 , + .setup = setup, + .test = run_test, + .hugepages = {3 ,TST_NEEDS }, +}; + --2.31.1
--
Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 3/4] Hugetlb: Migrating libhugetlbfs chunk-overcommit
Quoted 172 lines · Li Wang
Quoted 154 lines · Tarun Sahu
Migrating the
libhugetlbfs/testcases/chunk-overcommit.ctestTest Description: Some kernel versions after hugepage demand allocation was added used a dubious heuristic to check if there was enough hugepage space available for a given mapping. The number of not-already-instantiated pages in the mapping was compared against the total hugepage free pool. It was very easy to confuse this heuristic into overcommitting by allocating hugepage memory in chunks, each less than the total available pool size but together more than available. This would generally lead to OOM SIGKILLs of one process or another when it tried to instantiate pages beyond the available pool.
Signed-off-by Tarun Sahu <[email protected]>--- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/hugemmap08.c | 144 ++++++++++++++++++ 3 files changed, 146 insertions(+) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.cruntest/hugetlb +1
-0@@ -4,6 +4,7 @@ hugemmap04 hugemmap04 hugemmap05 hugemmap05 hugemmap06 hugemmap06 hugemmap07 hugemmap07 +hugemmap08 hugemmap08 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -mtestcases/kernel/mem/.gitignore +1
-0@@ -5,6 +5,7 @@ /hugetlb/hugemmap/hugemmap05 /hugetlb/hugemmap/hugemmap06 /hugetlb/hugemmap/hugemmap07 +/hugetlb/hugemmap/hugemmap08 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c new file +58
-0@@ -0,0 +1,144 @@ +// SPDX-License-Identifier: LGPL-2.1-or-later +/* +* Copyright (C) 2005-2006 David Gibson & Adam Litke, IBM Corporation. +* Author: David Gibson & Adam Litke +*/ + +/*\ +* [Description] +* +* Chunk Overcommit: +* Some kernel versions after hugepage demand allocation was added used a +* dubious heuristic to check if there was enough hugepage space available +* for a given mapping. The number of not-already-instantiated pages in +* the mapping was compared against the total hugepage free pool. It was +* very easy to confuse this heuristic into overcommitting by allocating +* hugepage memory in chunks, each less than the total available pool size +* but together more than available. This would generally lead to OOM +* SIGKILLs of one process or another when it tried to instantiate pages +* beyond the available pool. +* +* HISTORY +* +*/ + +#define _GNU_SOURCE +#include <stdio.h> +#include <stdlib.h> +#include <sys/mount.h> +#include <limits.h> +#include <sys/param.h> +#include <sys/types.h> +#include <sys/wait.h> +#include <signal.h> + +#include "hugetlb.h" + +#define PROC_OVERCOMMIT "/proc/sys/vm/nr_overcommit_hugepages" +#define WITH_OVERCOMMIT 0 +#define WITHOUT_OVERCOMMIT 1 + +static long hpage_size; + +static void test_chunk_overcommit (void ) +{ +unsigned long totpages, chunk1, chunk2; +void *p, *q; +pid_t child; +int status; + + totpages =SAFE_READ_MEMINFO ("HugePages_Free:" ); + + chunk1 = (totpages /2 ) +1 ; + chunk2 = totpages - chunk1 +1 ; + +tst_res (TINFO ,"Free: %ld hugepages available: " +"chunk1=%ld chunk2=%ld" , totpages, chunk1, chunk2); + + p = SAFE_MMAP(NULL, chunk1*hpage_size, PROT_READ|PROT_WRITE,
MAP_SHARED,testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c new file +10
-0+ tst_hugetlb_fd, 0); + + q =mmap (NULL , chunk2*hpage_size,PROT_READ |PROT_WRITE ,MAP_SHARED , + tst_hugetlb_fd, chunk1*hpage_size); +if (q ==MAP_FAILED ) { +if (errno !=ENOMEM ) { +tst_res (TFAIL |TERRNO ,"mmap() chunk2" ); + goto cleanup1; + }else { + tst_res(TPASS, "Successful without overcommitpages");
testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c new file +31
-0+ goto cleanup1; + } + } + +tst_res (TINFO ,"Looks like we've overcommitted, testing..." ); +/* Looks like we're overcommited, but we need to confirm that +* this is bad. We touch it all in a child process because an +* overcommit will generally lead to a SIGKILL which we can't +* handle, of course. +*/ + child =SAFE_FORK (); + +if (child ==0 ) { +memset (p,0 , chunk1*hpage_size); +memset (q,0 , chunk2*hpage_size); +exit (0 ); + } + +SAFE_WAITPID (child, &status,0 ); + +if (WIFSIGNALED (status)) { +tst_res (TFAIL ,"Killed by signal '%s' due to overcommit" , +tst_strsig (WTERMSIG (status))); + goto cleanup2; + } + +tst_res (TPASS ,"Successful with overcommit pages" ); + +cleanup2: +SAFE_MUNMAP (q, chunk2*hpage_size); +testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c line 109
Quoted 10 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c new file +10
-0+cleanup1: +SAFE_MUNMAP (p, chunk1*hpage_size); +SAFE_FTRUNCATE (tst_hugetlb_fd,0 ); +} + +static void run_test (unsigned int test_type) +{ +unsigned long saved_oc_hugepages; + +SAFE_FILE_SCANF (PROC_OVERCOMMIT ,"%ld" , &saved_oc_hugepages);There is unnecessary to read
PROC_OVERCOMMITvalue again,
we already save/restore it in struct tst_path_val[], so here we can set it directly to what we expected no matter if the original is 0 or 2.static void
run_test(unsigned int test_type)
Quoted 14 lines
{
switch (test_type) {
case WITHOUT_OVERCOMMIT:
tst_res(TINFO, "Without overcommit testing...");
SAFE_FILE_PRINTF(PROC_OVERCOMMIT, "%d", 0);
break;
case WITH_OVERCOMMIT:
tst_res(TINFO, "With overcommit testing...");
SAFE_FILE_PRINTF(PROC_OVERCOMMIT, "%d", 2);
break;
}
test_chunk_overcommit();
}Yeah, Missed it. Will update it.
Quoted 53 lines
Quoted 45 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap08.c new file +35
-1+switch (test_type) { +case WITHOUT_OVERCOMMIT : +tst_res (TINFO ,"Without overcommit testing..." ); +if (saved_oc_hugepages >0 ) +SAFE_FILE_PRINTF (PROC_OVERCOMMIT ,"%d" ,0 ); +break ; +case WITH_OVERCOMMIT : +tst_res (TINFO ,"With overcommit testing..." ); +if (saved_oc_hugepages ==0 ) +SAFE_FILE_PRINTF (PROC_OVERCOMMIT ,"%d" ,2 ); +break ; + } +test_chunk_overcommit (); +} + +static void setup (void ) +{ + hpage_size =SAFE_READ_MEMINFO ("Hugepagesize:" )*1024 ; +} + +static struct tst_test test = { + .needs_root =1 , + .needs_hugetlbfs =1 , + .needs_unlinked_hugetlb_file =1 , + .forks_child =1 , + .save_restore = (const struct tst_path_val []) { + {PROC_OVERCOMMIT ,NULL }, + {} + }, + .tcnt =2 , + .setup = setup, + .test = run_test, + .hugepages = {3 ,TST_NEEDS }, +}; + --2.31.1
--
Mailing list info: https://lists.linux.it/listinfo/ltpSignature · 6 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 4/4 Hugetlb: Migrating libhugetlbfs corrupt-by-cow-opt
In reply to Tarun SahuMigrating the libhugetlbfs/testcases/corrupt-by-cow-opt.c test
Test Description: Test sanity of cow optimization on page cache. If a page in page cache has only 1 ref count, it is mapped for a private mapping directly and is overwritten freely, so next time we access the page, we can see corrupt data.
--- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/hugemmap09.c | 71 +++++++++++++++++++ 3 files changed, 73 insertions(+) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap09.c
runtest/hugetlb +1 -0
@@ -5,6 +5,7 @@ hugemmap05 hugemmap05 hugemmap06 hugemmap06 hugemmap07 hugemmap07 hugemmap08 hugemmap08 +hugemmap09 hugemmap09 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -m
testcases/kernel/mem/.gitignore +1 -0
@@ -6,6 +6,7 @@ /hugetlb/hugemmap/hugemmap06 /hugetlb/hugemmap/hugemmap07 /hugetlb/hugemmap/hugemmap08 +/hugetlb/hugemmap/hugemmap09 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03
testcases/kernel/mem/hugetlb/hugemmap/hugemmap09.c new file +71 -0
71 lines not on this page · raw message
Signature · 5 lines
2.31.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp