Adding new function read_maps required for hugemmap40.
PATCH 1/3 Adding new function read_maps required for hugemmap40.
Function to read and parse the '/proc/self/maps' file to debug memory-related issues.
--- testcases/kernel/mem/hugetlb/lib/hugetlb.c | 42 ++++++++++++++++++++++ testcases/kernel/mem/hugetlb/lib/hugetlb.h | 1 + 2 files changed, 43 insertions(+)
testcases/kernel/mem/hugetlb/lib/hugetlb.c +42 -0
@@ -141,3 +141,45 @@ void update_shm_size(size_t * shm_size) *shm_size = shmmax; } } + +#define MAPS_BUF_SZ 4096 +int read_maps (unsigned long addr,char *buf) +{ +FILE *f; +char line[MAPS_BUF_SZ ]; +char *tmp; + + f =fopen ("/proc/self/maps" ,"r" ); +if (!f) { +tst_res (TFAIL ,"Failed to open /proc/self/maps: %s\n" ,strerror (errno)); +return -1 ; + } + +while (1 ) { +unsigned long start, end, off, ino; +int ret; + + tmp =fgets (line,MAPS_BUF_SZ , f); +if (!tmp) +break ; + + buf[0 ] ='\0' ; + ret =sscanf (line,"%lx-%lx %*s %lx %*s %ld %255s" , + &start, &end, &off, &ino, + buf); +if ((ret <4 ) || (ret >5 )) { +tst_res (TFAIL ,"Couldn't parse /proc/self/maps line: %s\n" , + line); +fclose (f); +return -1 ; + } + +if ((start <= addr) && (addr < end)) { +fclose (f); +return 1 ; + } + } + +fclose (f); +return 0 ; +}
testcases/kernel/mem/hugetlb/lib/hugetlb.h +1 -0
@@ -57,6 +57,7 @@ int getipckey(void);int getuserid (char *user);void rm_shm (int shm_id);int do_readback (void *p,size_t size,char *desc); +int read_maps (unsigned long addr,char *buf);void update_shm_size (size_t *shm_size);
Signature · 5 lines
2.43.5 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 2/3 Adding magic definition required for hugemmap40.
Defining HUGETLBFS_MAGIC.
--- include/tst_fs.h | 1 + 1 file changed, 1 insertion(+)
include/tst_fs.h +1 -0
@@ -33,6 +33,7 @@#define TST_FUSE_MAGIC 0x65735546#define TST_VFAT_MAGIC 0x4d44/* AKA MSDOS */ #define TST_EXFAT_MAGIC 0x2011BAB0UL +#define HUGETLBFS_MAGIC 0x958458f6/* fs/bcachefs/bcachefs_format.h */ #define TST_BCACHE_MAGIC 0xca451a4e
Signature · 5 lines
2.43.5 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 2/3] Adding magic definition required for hugemmap40.
Hi Pavithra,
...
include/tst_fs.h line 36
Quoted 6 lines
+++
b/include/tst_fs.hinclude/tst_fs.h +1
-0@@ -33,6 +33,7 @@#define TST_FUSE_MAGIC 0x65735546#define TST_VFAT_MAGIC 0x4d44/* AKA MSDOS */ #define TST_EXFAT_MAGIC 0x2011BAB0UL +#define HUGETLBFS_MAGIC 0x958458f6
Could you please use "TST_" prefix for that definition (i.e. TST_HUGETLBFS_MAGIC) as it is used for other LTP specific definitions? That way we avoid constant redefinition in case of including <linux/magic.h>.
With that you may add:
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 3/3 Migrating the libhugetlbfs/testcases/straddle_4GB.c v3
In reply to PavithraThis testcase attempts to map a memory range that straddles the 4GB boundary using mmap() with and without the MAP_FIXED flag.
Changes in v3:
- Added magic definition to include/tst_fs.h as separate patch.
- Moved CFLAGS to make file.
- Added read_maps definition to separate patch.
- Removed \n from tst_res
--- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/Makefile | 1 + .../kernel/mem/hugetlb/hugemmap/hugemmap40.c | 147 ++++++++++++++++++ 4 files changed, 150 insertions(+) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c
runtest/hugetlb +1 -0
@@ -36,6 +36,7 @@ hugemmap30 hugemmap30 hugemmap31 hugemmap31 hugemmap32 hugemmap32 hugemmap34 hugemmap34 +hugemmap40 hugemmap40 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -m
testcases/kernel/mem/.gitignore +1 -0
@@ -36,6 +36,7 @@ /hugetlb/hugemmap/hugemmap31 /hugetlb/hugemmap/hugemmap32 /hugetlb/hugemmap/hugemmap34 +/hugetlb/hugemmap/hugemmap40 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03
testcases/kernel/mem/hugetlb/hugemmap/Makefile +1 -0
@@ -12,3 +12,4 @@ CFLAGS_no_stack_prot := $(filter-out -fstack-clash-protection, $(CFLAGS)) hugemmap06: CFLAGS+=-pthread hugemmap34: CFLAGS=$(CFLAGS_no_stack_prot) +hugemmap40: CFLAGS += -D_LARGEFILE64_SOURCE
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +147 -0
147 lines not on this page · raw message
Signature · 5 lines
2.43.5 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 3/3] [PATCH] [PATCH] Migrating the libhugetlbfs/testcases/straddle_4GB.c v3
Hi Pavithra, Geetika,
I very briefly looked into the original source file (I'd personally put that link into the commit message).
Quoted 3 lines
This testcase attempts to map a memory range that straddles the 4GB boundary using
mmap()with and without theMAP_FIXEDflag.Changes in v3:
FYI I tried to search in the mailing history [2] [3], but I found only v1, sent by Geetika. Is there any other discussion than I found?
[1] https://github.com/libhugetlbfs/libhugetlbfs/blob/master/tests/straddle_4GB.c
[2] https://lore.kernel.org/ltp/?q=straddle_4GB.c
[3] https://lore.kernel.org/ltp/?q=hugemmap40
Quoted 11 lines
- Added magic definition to
include/tst_fs.has separate patch.
- Moved CFLAGS to make file.
- Addedread_mapsdefinition to separate patch.
- Removed \n fromtst_restestcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +0
-0
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 8
Quoted 8 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +8
-0@@ -0,0 +1,147 @@ +// SPDX-License-Identifier: LGPL-2.1-or-later +/* +* Copyright (C) 2005-2006 David Gibson & Adam Litke, IBM Corporation. +* Copyright (C) 2006 Hugh Dickins <[email protected]> +*/ + +/*\ +* [Description]
nit: we don't use "[Description]" any more, because /*\ is enough for parsing comments.
Please remove the line.
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 11
Quoted 3 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +3
-0+* +* Test tries to allocate hugepages to cover a memory range that straddles the +* 4GB boundary, using mmap(2) with and without MAP_FIXED.
nit: Could you please use :man2:mmap (this converts to a man page link [4] in our docs, see e.g. stack_clash [5]).
[4] https://man7.org/linux/man-pages/man2/mmap.2.html
[5] https://linux-test-project.readthedocs.io/en/latest/users/test_catalog.html#stack-clash
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 15
Quoted 4 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +4
-0+*/ + +#define MAPS_BUF_SZ 4096 +#define FOURGB (1UL << 32)
FYI we also have TST_GB in include/tst_fs.h.
Quoted 11 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +11
-0+#define MNTPOINT "hugetlbfs/" + +#include "hugetlb.h" + +static unsigned long hpage_size; +static int fd =-1 ; +static unsigned long straddle_addr; + +static int test_addr_huge (void *p) +{ +char name[256 ];
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 36
Quoted 10 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +10
-0+char *dirend; +int ret; +struct statfs64 sb; + + ret =read_maps ((unsigned long )p, name); +if (ret <0 ) +return ret; + +if (ret ==0 ) { +tst_res (TINFO ,"Couldn't find address %p in /proc/self/maps" , p);
Having this as info message which is later followed by tst_brk(TFAIL) is is not optimal. I suspect test is unnecessary complicated, but I'm not sure now how to improve it.
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 45
Quoted 9 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +9
-0+return -1 ; + } + +/* looks like a filename? */ +if (name[0 ] !='/' ) +return 0 ; + +/* Truncate the filename portion */ +
nit: empty line above, please remove it.
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 52
Quoted 7 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +7
-0+ dirend =strrchr (name,'/' ); +if (dirend && dirend > name) + *dirend ='\0' ; + + ret =statfs64 (name, &sb); +if (ret) +return -1 ;
I guess we really need statfs64() not just statfs(), right?
In case of the failure test should quit with tst_brk(TBROK | TERRNO, "...").
Quoted 7 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +7
-0+ +return (sb.f_type ==HUGETLBFS_MAGIC ); +} + +static void run_test (void ) +{ +void *p;
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 69
Quoted 10 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +10
-0+ +/* We first try to get the mapping without MAP_FIXED */ +tst_res (TINFO ,"Mapping without MAP_FIXED at %lx..." , straddle_addr); + p =mmap ((void *)straddle_addr,2 *hpage_size,PROT_READ |PROT_WRITE , +MAP_SHARED , fd,0 ); +if (p == (void *)straddle_addr) { +/* These tests irrelevant if we didn't get the straddle address*/ +if (test_addr_huge (p) !=1 ) { +tst_brk (TFAIL ,"1st Mapped address is not hugepage" ); + goto windup;
FYI tst_brk() quits testing (test executes only cleanup function. You need to use tst_res(TFAIL, ...).
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 80
Quoted 11 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +11
-0+ } +if (test_addr_huge (p + hpage_size) !=1 ) { +tst_brk (TFAIL ,"2nd Mapped address is not hugepage" ); + goto windup; + } +memset (p,0 , hpage_size); +memset (p + hpage_size,0 , hpage_size); +tst_res (TPASS ,"Mapping without MAP_FIXED at %lx... completed" , straddle_addr); + }else { +tst_res (TINFO ,"Got %p instead, never mind, let's move to mapping with MAP_FIXED" , p); +munmap (p,2 *hpage_size);
We have SAFE_MUNMAP().
Quoted 5 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +5
-0+ } + +tst_res (TINFO ,"Mapping with MAP_FIXED at %lx..." , straddle_addr); + p =mmap ((void *)straddle_addr,2 *hpage_size,PROT_READ |PROT_WRITE , +MAP_SHARED |MAP_FIXED , fd,0 );
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 95
Quoted 10 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +10
-0+ +if (p ==MAP_FAILED ) { +/* this area crosses last low slice and first high slice */ +unsigned long below_start =FOURGB -256L *1024 *1024 ; +unsigned long above_end =1024L *1024 *1024 *1024 ; + +if (tst_mapping_in_range (below_start, above_end) ==1 ) { +tst_res (TINFO ,"region (4G-256M)-1T is not free" ); +tst_res (TINFO ,"mmap() failed: %s" ,strerror (errno)); +tst_res (TWARN ,"Pass Inconclusive!" );
I don't understand "Pass Inconclusive!". Could it be just described what happen?
We have TERRNO which we use instead strerror(). Instead of all 3 tst_res() mesagess above I would use something like:
tst_res(TFAIL | TERRNO, "region (4G-256M)-1T is not free");
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 98
Quoted 3 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +3
-0+ goto windup; + }else +tst_res (TFAIL ,"mmap() FIXED failed: %s" ,strerror (errno));
I also don't understand "FIXED". I guess it was supposed to be "mmap() with MAP_FIXED failed". Does it even make sense to continue on mmap() failure? The original test does, but I'm not sure if that's ok.
very nit: if you use { } on if, they should be also on else. } else {
Quoted 1 line
tst_res(TFAIL | TERRNO, "mmap() failed: %s", strerror(errno));
}
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c
Quoted 2 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +2
-0+ } +
The code below uses wrong indent
Quoted 7 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +7
-0+if (p != (void *)straddle_addr) { +tst_res (TINFO ,"got %p instead" , p); +tst_res (TFAIL ,"Wrong address with MAP_FIXED" ); + goto windup; + } + +if (test_addr_huge (p) !=1 ) {
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 117
Quoted 10 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +10
-0+tst_brk (TFAIL ,"1st Mapped address is not hugepage" ); + goto windup; + } + +if (test_addr_huge (p + hpage_size) !=1 ) { +tst_brk (TFAIL ,"2nd Mapped address is not hugepage" ); + goto windup; + } +memset (p,0 , hpage_size); +memset (p + hpage_size,0 , hpage_size);
Why is this memset() for? If needed for testing, shouldn't it be at the beginning?
Quoted 4 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +4
-0+tst_res (TPASS ,"Mapping with MAP_FIXED at %lx... completed" , straddle_addr); + +windup: +SAFE_MUNMAP (p,2 *hpage_size);
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c line 130
Quoted 10 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +9
-0+} + +static void setup (void ) +{ + straddle_addr =FOURGB - hpage_size; + hpage_size =tst_get_hugepage_size (); + fd =tst_creat_unlinked (MNTPOINT ,0 ,0600 ); +if (hpage_size >FOURGB ) +tst_brk (TCONF ,"Huge page size is too large!" );
I suppose the original check is really needed.
The original test does this check for this:
static inline long check_hugepagesize()
{
long __hpage_size = gethugepagesize();
if (__hpage_size < 0) {
if (errno == ENOSYS)
CONFIG("No hugepage kernel support\n");
else if (errno == EOVERFLOW)
CONFIG("Hugepage size too large");
else
CONFIG("Hugepage size (%s)", strerror(errno));
}
return __hpage_size;
}Kind regards,
Petr
[6] https://github.com/libhugetlbfs/libhugetlbfs/blob/master/tests/hugetests.h#L126-L138
Quoted 17 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap40.c new file +17
-0+} + +static void cleanup (void ) +{ +if (fd >=0 ) +SAFE_CLOSE (fd); +} + +static struct tst_test test = { + .needs_root =1 , + .mntpoint =MNTPOINT , + .needs_hugetlbfs =1 , + .hugepages = {2 ,TST_NEEDS }, + .setup = setup, + .cleanup = cleanup, + .test_all = run_test, +};
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Pavithra, Geetika,
Quoted 1 line
Function to read and parse the '
/proc/self/maps' file to debug memory-related issues.
First of all, do you plan to use read_maps() in other hugemmap tests? Otherwise it does not make sense to add it to hugetlb.c library.
Quoted 11 lines
Signed-off-by Pavithra <[email protected]>--- testcases/kernel/mem/hugetlb/lib/hugetlb.c | 42 ++++++++++++++++++++++ testcases/kernel/mem/hugetlb/lib/hugetlb.h | 1 + 2 files changed, 43 insertions(+)testcases/kernel/mem/hugetlb/lib/hugetlb.c +0
-0
testcases/kernel/mem/hugetlb/lib/hugetlb.c void update_shm_size(size_t * shm_size) · line 145
Quoted 5 lines
testcases/kernel/mem/hugetlb/lib/hugetlb.c +2
-0@@ -141,3 +141,45 @@ void update_shm_size(size_t * shm_size) *shm_size = shmmax; } } + +#define MAPS_BUF_SZ 4096
NOTE: we usually use BUFSIZ (from <stdio.h>) for these purposes.
nit: we usually have definitions at the top. (I know this file has #define RANDOM_CONSTANT 0x1234ABCD, but this file is outdated, not following LTP standards.)
testcases/kernel/mem/hugetlb/lib/hugetlb.c void update_shm_size(size_t * shm_size) · line 153
Quoted 9 lines
testcases/kernel/mem/hugetlb/lib/hugetlb.c +8
-0+int read_maps (unsigned long addr,char *buf) +{ +FILE *f; +char line[MAPS_BUF_SZ ]; +char *tmp; + + f =fopen ("/proc/self/maps" ,"r" ); +if (!f) {
nit: we have SAFE_FOPEN(), this check is not needed. See my commit today:
https://github.com/linux-test-project/ltp/commit/f3803d4bfabe4da10d9fc1824df5b10c249dbed6
and please rebase your next version.
Quoted 1 line
+
tst_res(TFAIL, "Failed to open /proc/self/maps: %s\n", strerror(errno));
Even we did not have not have safe function, failure like this we consider hard and use tst_brk(TBROK | TERRNO) to quit whole testing.
testcases/kernel/mem/hugetlb/lib/hugetlb.c void update_shm_size(size_t * shm_size) · line 164
Quoted 10 lines
testcases/kernel/mem/hugetlb/lib/hugetlb.c +10
-0+return -1 ; + } + +while (1 ) { +unsigned long start, end, off, ino; +int ret; + + tmp =fgets (line,MAPS_BUF_SZ , f); +if (!tmp) +break ;
Using
while (fgets(line, BUFSIZ, f) != NULL) {
is much simpler than having char *tmp just for checking.
testcases/kernel/mem/hugetlb/lib/hugetlb.c
Quoted 2 lines
testcases/kernel/mem/hugetlb/lib/hugetlb.c +2
-0+ + buf[0 ] ='\0' ;
Why this? That's IMHO unnecessary.
testcases/kernel/mem/hugetlb/lib/hugetlb.c void update_shm_size(size_t * shm_size) · line 174
Quoted 8 lines
testcases/kernel/mem/hugetlb/lib/hugetlb.c +8
-0+ ret =sscanf (line,"%lx-%lx %*s %lx %*s %ld %255s" , + &start, &end, &off, &ino, + buf); +if ((ret <4 ) || (ret >5 )) { +tst_res (TFAIL ,"Couldn't parse /proc/self/maps line: %s\n" , + line); +fclose (f); +return -1 ;
FYI we can also have FILE_LINES_SCANF() and SAFE_FILE_LINES_SCANF() macros, which wraps file_lines_scanf() from lib/safe_file_ops.c which uses vsscanf(). I believe it could be used for the task you want to achieve.
Quoted 11 lines
testcases/kernel/mem/hugetlb/lib/hugetlb.c +11
-0+ } + +if ((start <= addr) && (addr < end)) { +fclose (f); +return 1 ; + } + } + +fclose (f); +return 0 ; +}
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp